.\" Automatically generated by Pandoc 2.1.3
.\"
.TH "PMREORDER" "1" "2018-10-12" "PMDK - pmreorder version 1.5" "PMDK Programmer's Manual"
.hy
.\" Copyright 2014-2018, Intel Corporation
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\"
.\"     * Redistributions of source code must retain the above copyright
.\"       notice, this list of conditions and the following disclaimer.
.\"
.\"     * Redistributions in binary form must reproduce the above copyright
.\"       notice, this list of conditions and the following disclaimer in
.\"       the documentation and/or other materials provided with the
.\"       distribution.
.\"
.\"     * Neither the name of the copyright holder nor the names of its
.\"       contributors may be used to endorse or promote products derived
.\"       from this software without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.SH NAME
.PP
\f[B]pmreorder\f[] \- performs a persistent consistency check using a
store reordering mechanism
.SH SYNOPSIS
.IP
.nf
\f[C]
$\ python\ pmreorder\ <options>
\f[]
.fi
.SH DESCRIPTION
.PP
The pmreorder tool is a collection of python scripts designed to parse
and replay operations logged by pmemcheck \- a persistent memory
checking tool.
.PP
Pmreorder performs the store reordering between persistent memory
barriers \- a sequence of flush\-fence operations.
It uses a consistency checking routine provided in the command line
options to check whether files are in a consistent state.
.PP
Considering that logging, replaying and reordering of operations are
very time consuming, it is recommended to use as few stores as possible
in test workloads.
.SH OPTIONS
.PP
\f[C]\-h,\ \-\-help\f[]
.PP
Prints synopsis and list of options.
.PP
\f[C]\-l\ <store_log>,\ \-\-logfile\ <store_log>\f[]
.PP
The pmemcheck log file to process.
.PP
\f[C]\-c\ <prog|lib>,\ \-\-checker\ <prog|lib>\f[]
.PP
Consistency checker type.
.PP
\f[C]\-p\ <path>,\ \-\-path\ <path>\f[]
.PP
Path to the consistency checker.
Checker function has to return 0 for consistent cases and 1 otherwise.
.PP
\f[C]\-n\ <name>,\ \-\-name\ <name>\f[]
.PP
The symbol name of the consistency checking function in the library.
Valid only if the checker type is \f[C]lib\f[].
.PP
\f[C]\-o\ <pmreorder_output>,\ \-\-output\ <pmreorder_output>\f[]
.PP
Set the logger output file.
.PP
\f[C]\-e\ <debug|info|warning|error|critical>,\f[]
\f[C]\-\-output\-level\ <debug|info|warning|error|critical>\f[]
.PP
Set the output log level.
.PP
\f[C]\-r\ \ <NoReorderNoCheck|\f[] \f[C]NoReorderDoCheck|\f[]
\f[C]ReorderFull|\f[] \f[C]ReorderPartial|\f[]
\f[C]ReorderAccumulative|\f[] \f[C]ReorderReverseAccumulative>,\f[]
\f[C]\-\-default\-engine\ \ <NoReorderNoCheck|\f[]
\f[C]NoReorderDoCheck|\f[] \f[C]ReorderFull|\f[]
\f[C]ReorderPartial|\f[] \f[C]ReorderAccumulative|\f[]
\f[C]ReorderReverseAccumulative>\f[]
.PP
Set the initial reorder engine.
Default value is \f[C]NoReorderNoCheck\f[].
.PP
\f[C]\-x\ <cli_macros|config_file>,\ \-\-extended\-macros\ <cli_macros|config_file>\f[]
.PP
Assign an engine types to the defined marker.
.SH ENGINES
.PP
By default, the \f[B]NoReorderNoCheck\f[] engine is used, which means
that for each set of stores, the tool will pass\-through all sequences
of stores not reordered and will not run consistency checker on them.
.PP
To enable different types of the reorder engine and begin proper
reordering tests, a number of other engines exist:
.IP \[bu] 2
\f[B]NoReorderDoCheck\f[] \- pass\-through of unchanged operations.
Checks correctness of the stores as they were logged.
Useful for operations that do not require fail safety.
.IP
.nf
\f[C]
Example:
\ \ \ \ \ \ \ \ input:\ (a,\ b,\ c)
\ \ \ \ \ \ \ \ output:\ (a,\ b,\ c)
\f[]
.fi
.IP \[bu] 2
\f[B]ReorderAccumulative\f[] \- checks correctness on a growing subset
of the original sequence.
.IP
.nf
\f[C]
Example:
\ \ \ \ \ \ \ \ input:\ (a,\ b,\ c)
\ \ \ \ \ \ \ \ output:
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ()
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c)
\f[]
.fi
.IP \[bu] 2
\f[B]ReorderReverseAccumulative\f[] \- checks correctness on a reverted
growing subset of the original sequence.
.IP
.nf
\f[C]
Example:
\ \ \ \ \ \ \ \ input:\ (a,\ b,\ c)
\ \ \ \ \ \ \ \ output:
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ()
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b,\ a)
\f[]
.fi
.IP \[bu] 2
\f[B]ReorderPartial\f[] \- checks consistency on 3 randomly selected
sequences from a set of 1000 combinations of the original log, without
repetitions.
.IP
.nf
\f[C]
\ Example:
\ \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c)
\ \ \ \ \ \ \ \ \ output:
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c)
\f[]
.fi
.IP \[bu] 2
\f[B]ReorderFull\f[] \- for each set of stores generates and checks
consistency of all possible store permutations.
This might prove to be very computationally expensive for most
workloads.
It can be useful for critical sections of code with limited number of
stores.
.IP
.nf
\f[C]
\ Example:
\ \ \ \ \ \ \ \ input:\ (a,\ b,\ c)
\ \ \ \ \ \ \ \ output:
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ()
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ a)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ a)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ c,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ a,\ c)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c,\ a)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ a,\ b)
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b,\ a)
\f[]
.fi
.PP
When the engine is passed with an \f[C]\-r\f[] option, it will be used
for each logged set of stores.
Additionally, the \f[C]\-x\f[] parameter can be used to switch engines
separately for any marked code sections.
For more details about \f[C]\-x\f[] extended macros functionality see
section INSTRUMENTATION below.
.SH INSTRUMENTATION
.PP
The core of \f[B]pmreorder\f[] is based on user\-provided named markers.
Sections of code can be `marked' depending on their importance, and the
degree of reordering can be customized by the use of various provided
engines.
.PP
For this purpose, Valgrind's pmemcheck tool exposes a generic marker
macro:
.IP \[bu] 2
\f[B]VALGRIND_EMIT_LOG(value)\f[]
.PP
It emits log to \f[I]store_log\f[] during pmemcheck processing.
\f[I]value\f[] is a user\-defined marker name.
For more details about pmemcheck execution see PMEMCHECK STORE LOG
section below.
.PP
Example:
.IP
.nf
\f[C]
main.c
\&.
\&.
\&.
VALGRIND_EMIT_LOG("PMREORDER_MEMSET_PERSIST.BEGIN");

pmem_memset_persist(...);

VALGRIND_EMIT_LOG("PMREORDER_MEMSET_PERSIST.END");
\&.
\&.
\&.
\f[]
.fi
.PP
There are a few rules for macros creation:
.IP \[bu] 2
Valid macro can have any name, but begin and end section have to match
\- they are case sensitive.
.IP \[bu] 2
Macro must have \f[C]\&.BEGIN\f[] or \f[C]\&.END\f[] suffix.
.IP \[bu] 2
Macros can't be crossed.
.PP
Defined markers can be assigned engines types and configured through the
\f[B]pmreorder\f[] tool using the \f[C]\-x\f[] parameter.
.PP
There are two ways to set macro options:
.IP \[bu] 2
Using command line interface in format:
.IP
.nf
\f[C]

+\ Using\ configuration\ file\ in\ .json\ format:
\f[]
.fi
.PP
{ \[lq]PMREORDER_MARKER_NAME1\[rq]=\[lq]ReorderName1\[rq],
\[lq]PMREORDER_MARKER_NAME2\[rq]=\[lq]ReorderName2\[rq] }
.IP
.nf
\f[C]

For\ more\ details\ about\ available
engines\ types,\ see\ ENGINES\ section\ above.

**libpmemobj**(7)\ also\ provides\ set\ of\ macros\ that\ allows\ change
reordering\ engine\ on\ library\ or\ function\ level:

`<library_name|api_function_name>`

Example\ of\ configuration\ on\ function\ level:
\f[]
.fi
.PP
{ \[lq]pmemobj_open\[rq]=\[lq]NoReorderNoCheck\[rq],
\[lq]pmemobj_memcpy_persist\[rq]=\[lq]ReorderPartial\[rq] }
.IP
.nf
\f[C]

Example\ of\ configuration\ on\ library\ level
(affecting\ all\ library\ functions):
\f[]
.fi
.PP
{ \[lq]libpmemobj\[rq]=\[lq]NoReorderNoCheck\[rq] }
.IP
.nf
\f[C]

List\ of\ marked\ **libpmemobj**(7)\ API\ functions:
\f[]
.fi
.PP
pmemobj_alloc pmemobj_cancel pmemobj_check pmemobj_close pmemobj_create
pmemobj_ctl_exec pmemobj_ctl_set pmemobj_free pmemobj_list_insert
pmemobj_list_insert_new pmemobj_list_move pmemobj_list_remove
pmemobj_memcpy pmemobj_memmove pmemobj_memset pmemobj_memcpy_persist
pmemobj_memset_persist pmemobj_open pmemobj_publish pmemobj_realloc
pmemobj_reserve pmemobj_root pmemobj_root_construct pmemobj_strdup
pmemobj_tx_abort pmemobj_tx_add_range pmemobj_tx_add_range_direct
pmemobj_tx_alloc pmemobj_tx_commit pmemobj_tx_free pmemobj_tx_publish
pmemobj_tx_realloc pmemobj_tx_strdup pmemobj_tx_wcsdup
pmemobj_tx_xadd_range pmemobj_tx_xadd_range_direct pmemobj_tx_xalloc
pmemobj_tx_zalloc pmemobj_tx_zrealloc pmemobj_wcsdup pmemobj_xalloc
pmemobj_xreserve pmemobj_zalloc pmemobj_zrealloc
.IP
.nf
\f[C]


#\ PMEMCHECK\ STORE\ LOG\ #

To\ generate\ *store_log*\ for\ **pmreorder**\ run\ pmemcheck
with\ additional\ parameters:
\f[]
.fi
.PP
valgrind
.PD 0
.P
.PD
\[en]tool=pmemcheck
.PD 0
.P
.PD
\-q
.PD 0
.P
.PD
\[en]log\-stores=yes
.PD 0
.P
.PD
\[en]print\-summary=no
.PD 0
.P
.PD
\[en]log\-file=store_log.log
.PD 0
.P
.PD
\[en]log\-stores\-stacktraces=yes
.PD 0
.P
.PD
\[en]log\-stores\-stacktraces\-depth=2
.PD 0
.P
.PD
\[en]expect\-fence\-after\-clflush=yes
.PD 0
.P
.PD
test_binary writer_parameter
.IP
.nf
\f[C]

For\ further\ details\ of\ pmemcheck\ parameters\ see\ [pmemcheck\ documentation](https://github.com/pmem/valgrind/blob/pmem\-3.13/pmemcheck/docs/pmc\-manual.xml)


#\ ENVIRONMENT\ #

By\ default\ all\ logging\ from\ PMDK\ libraries\ is\ disabled.
To\ enable\ API\ macros\ logging\ set\ environment\ variable:

+\ **PMREORDER_EMIT_LOG**=1


#\ EXAMPLE\ #
\f[]
.fi
.PP
python pmreorder.py
.PD 0
.P
.PD
\-l store_log.log
.PD 0
.P
.PD
\-r NoReorderDoCheck
.PD 0
.P
.PD
\-o pmreorder_out.log
.PD 0
.P
.PD
\-c prog
.PD 0
.P
.PD
\-x PMREORDER_MARKER_NAME=ReorderPartial
.PD 0
.P
.PD
\-p checker_binary checker_parameter ```
.PP
Checker binary will be used to run consistency checks on
\[lq]store_log.log\[rq], output of pmemcheck tool.
Any inconsistent stores found during \f[B]pmreorder\f[] analysis will be
logged to \f[C]pmreorder_out.log\f[].
.SH SEE ALSO
.PP
\f[B]<http://pmem.io>\f[]