.TH OPENSM 8 "Sept 15, 2014" "OpenIB" "OpenIB Management"

.SH NAME

opensm \- InfiniBand subnet manager and administration (SM/SA)

.SH SYNOPSIS

.B opensm

[\-\-version]]

[\-F | \-\-config <file_name>]

[\-c(reate-config) <file_name>]

[\-g(uid) <GUID in hex>]

[\-l(mc) <LMC>]

[\-p(riority) <PRIORITY>]

[\-\-subnet_prefix <PREFIX in hex>]

[\-\-smkey <SM_Key>]

[\-\-sm_sl <SL number>]

[\-r(eassign_lids)]

[\-R <engine name(s)> | \-\-routing_engine <engine name(s)>]

[\-\-do_mesh_analysis]

[\-\-lash_start_vl <vl number>]

[\-\-nue_max_num_vls <vl number>]

[\-A | \-\-ucast_cache]

[\-z | \-\-connect_roots]

[\-M <file name> | \-\-lid_matrix_file <file name>]

[\-U <file name> | \-\-lfts_file <file name>]

[\-S | \-\-sadb_file <file name>]

[\-a | \-\-root_guid_file <path to file>]

[\-u | \-\-cn_guid_file <path to file>]

[\-G | \-\-io_guid_file <path to file>]

[\-\-port\-shifting]

[\-\-scatter\-ports <random seed>]

[\-H | \-\-max_reverse_hops <max reverse hops allowed>]

[\-X | \-\-guid_routing_order_file <path to file>]

[\-m | \-\-ids_guid_file <path to file>]

[\-o(nce)]

[\-s(weep) <interval>]

[\-t(imeout) <milliseconds>]

[\-\-retries <number>]

[\-\-maxsmps <number>]

[\-\-console [off | local | socket | loopback]]

[\-\-console-port <port>]

[\-i | \-\-ignore_guids <equalize-ignore-guids-file>]

[\-w | \-\-hop_weights_file <path to file>]

[\-O | \-\-port_search_ordering_file <path to file>]

[\-O | \-\-dimn_ports_file <path to file>] (DEPRECATED)

[\-\-dump_files_dir <directory-name>]

[\-f <log file path> | \-\-log_file <log file path> ]

[\-L | \-\-log_limit <size in MB>] [\-e(rase_log_file)]

[\-P(config) <partition config file> ]

[\-N | \-\-no_part_enforce] (DEPRECATED)

[\-Z | \-\-part_enforce [both | in | out | off]]

[\-W | \-\-allow_both_pkeys]

[\-Q | \-\-qos [\-Y | \-\-qos_policy_file <file name>]]

[\-\-congestion\-control]

[\-\-cckey <key>]

[\-y | \-\-stay_on_fatal]

[\-B | \-\-daemon]

[\-J | \-\-pidfile <file_name>]

[\-I | \-\-inactive]

[\-\-perfmgr]

[\-\-perfmgr_sweep_time_s <seconds>]

[\-\-prefix_routes_file <path>]

[\-\-consolidate_ipv6_snm_req]

[\-\-log_prefix <prefix text>]

[\-\-torus_config <path to file>]

[\-v(erbose)] [\-V] [\-D <flags>] [\-d(ebug) <number>]

[\-h(elp)] [\-?]

.SH DESCRIPTION

.PP

opensm is an InfiniBand compliant Subnet Manager and Administration,

and runs on top of OpenIB.

opensm provides an implementation of an InfiniBand Subnet Manager and

Administration. Such a software entity is required to run for in order

to initialize the InfiniBand hardware (at least one per each

InfiniBand subnet).

opensm also now contains an experimental version of a performance

manager as well.

opensm defaults were designed to meet the common case usage on clusters with up to a few hundred nodes. Thus, in this default mode, opensm will scan the IB

fabric, initialize it, and sweep occasionally for changes.

opensm attaches to a specific IB port on the local machine and configures only

the fabric connected to it. (If the local machine has other IB ports,

opensm will ignore the fabrics connected to those other ports). If no port is

specified, it will select the first "best" available port.

opensm can present the available ports and prompt for a port number to

attach to.

By default, the run is logged to two files: /var/log/messages and /var/log/opensm.log.

The first file will register only general major events, whereas the second

will include details of reported errors. All errors reported in this second

file should be treated as indicators of IB fabric health issues.

(Note that when a fatal and non-recoverable error occurs, opensm will exit.)

Both log files should include the message "SUBNET UP" if opensm was able to

setup the subnet correctly.

.SH OPTIONS

.PP

.TP

\fB\-\-version\fR

Prints OpenSM version and exits.

.TP

\fB\-F\fR, \fB\-\-config\fR <config file>

The name of the OpenSM config file. When not specified

\fB\% @OPENSM_CONFIG_DIR@/@OPENSM_CONFIG_FILE@\fP will be used (if exists).

.TP

\fB\-c\fR, \fB\-\-create-config\fR <file name>

OpenSM will dump its configuration to the specified file and exit.

This is a way to generate OpenSM configuration file template.

.TP

\fB\-g\fR, \fB\-\-guid\fR <GUID in hex>

This option specifies the local port GUID value

with which OpenSM should bind.  OpenSM may be

bound to 1 port at a time.

If GUID given is 0, OpenSM displays a list

of possible port GUIDs and waits for user input.

Without -g, OpenSM tries to use the default port.

.TP

\fB\-l\fR, \fB\-\-lmc\fR <LMC value>

This option specifies the subnet's LMC value.

The number of LIDs assigned to each port is 2^LMC.

The LMC value must be in the range 0-7.

LMC values > 0 allow multiple paths between ports.

LMC values > 0 should only be used if the subnet

topology actually provides multiple paths between

ports, i.e. multiple interconnects between switches.

Without -l, OpenSM defaults to LMC = 0, which allows

one path between any two ports.

.TP

\fB\-p\fR, \fB\-\-priority\fR <Priority value>

This option specifies the SM\'s PRIORITY.

This will effect the handover cases, where master

is chosen by priority and GUID.  Range goes from 0

(default and lowest priority) to 15 (highest).

.TP

\fB\-\-subnet_prefix\fR <PREFIX in hex>

This option specifies the subnet prefix to use in

on the fabric.  The default prefix is

0xfe80000000000000.

.TP

\fB\-\-smkey\fR <SM_Key value>

This option specifies the SM\'s SM_Key (64 bits).

This will effect SM authentication.

Note that OpenSM version 3.2.1 and below used the default value '1'

in a host byte order, it is fixed now but you may need this option to

interoperate with old OpenSM running on a little endian machine.

.TP

\fB\-\-sm_sl\fR <SL number>

This option sets the SL to use for communication with the SM/SA.

Defaults to 0.

.TP

\fB\-r\fR, \fB\-\-reassign_lids\fR

This option causes OpenSM to reassign LIDs to all

end nodes. Specifying -r on a running subnet

may disrupt subnet traffic.

Without -r, OpenSM attempts to preserve existing

LID assignments resolving multiple use of same LID.

.TP

\fB\-R\fR, \fB\-\-routing_engine\fR <Routing engine names>

This option chooses routing engine(s) to use instead of Min Hop

algorithm (default).  Multiple routing engines can be specified

separated by commas so that specific ordering of routing algorithms

will be tried if earlier routing engines fail.  If all configured

routing engines fail, OpenSM will always attempt to route with Min Hop

unless 'no_fallback' is included in the list of routing engines.

Supported engines: minhop, updn, dnup, file, ftree, lash, dor, torus-2QoS,

nue, dfsssp, sssp.

.TP

\fB\-\-do_mesh_analysis\fR

This option enables additional analysis for the lash routing engine to

precondition switch port assignments in regular cartesian meshes which

may reduce the number of SLs required to give a deadlock free routing.

.TP

\fB\-\-lash_start_vl\fR <vl number>

This option sets the starting VL to use for the lash routing algorithm.

Defaults to 0.

.TP

\fB\-\-nue_max_num_vls\fR <vl number>

This option sets the maximum number of VLs to use for the Nue routing engine.

Every number greater or equal to 0 is allowed, and the default is 1 to enforce

deadlock-freedom even if QoS is not enabled. If set to 0, then Nue routing will

automatically determine and choose maximum supported by the fabric. And if set

to any integer >= 1, then Nue uses min(max_supported,nue_max_num_vls).

Rule of thumb is: higher nue_max_num_vls results in better path balancing.

.TP

\fB\-A\fR, \fB\-\-ucast_cache\fR

This option enables unicast routing cache and prevents routing

recalculation (which is a heavy task in a large cluster) when

there was no topology change detected during the heavy sweep, or

when the topology change does not require new routing calculation,

e.g. when one or more CAs/RTRs/leaf switches going down, or one or

more of these nodes coming back after being down.

A very common case that is handled by the unicast routing cache

is host reboot, which otherwise would cause two full routing

recalculations: one when the host goes down, and the other when

the host comes back online.

.TP

\fB\-z\fR, \fB\-\-connect_roots\fR

This option enforces routing engines (up/down and

fat-tree) to make connectivity between root switches and in

this way to be fully IBA compliant. In many cases this can

violate "pure" deadlock free algorithm, so use it carefully.

.TP

\fB\-M\fR, \fB\-\-lid_matrix_file\fR <file name>

This option specifies the name of the lid matrix dump file

from where switch lid matrices (min hops tables) will be

loaded.

.TP

\fB\-U\fR, \fB\-\-lfts_file\fR <file name>

This option specifies the name of the LFTs file

from where switch forwarding tables will be loaded when using "file" routing

engine.

.TP

\fB\-S\fR, \fB\-\-sadb_file\fR <file name>

This option specifies the name of the SA DB dump file

from where SA database will be loaded.

.TP

\fB\-a\fR, \fB\-\-root_guid_file\fR <file name>

Set the root nodes for the Up/Down or Fat-Tree routing

algorithm to the guids provided in the given file (one to a line).

.TP

\fB\-u\fR, \fB\-\-cn_guid_file\fR <file name>

Set the compute nodes for the Fat-Tree or DFSSSP/SSSP routing algorithms

to the port GUIDs provided in the given file (one to a line).

.TP

\fB\-G\fR, \fB\-\-io_guid_file\fR <file name>

Set the I/O nodes for the Fat-Tree or DFSSSP/SSSP routing algorithms

to the port GUIDs provided in the given file (one to a line).

.br

In the case of Fat-Tree routing:

.br

I/O nodes are non-CN nodes allowed to use up to max_reverse_hops switches

the wrong way around to improve connectivity.

.br

In the case of (DF)SSSP routing:

.br

Providing guids of compute and/or I/O nodes will ensure that paths towards

those nodes are as much separated as possible within their node category,

i.e., I/O traffic will not share the same link if multiple links are available.

.TP

\fB\-\-port\-shifting\fR

This option enables a feature called \fBport shifting\fR.  In some

fabrics, particularly cluster environments, routes commonly align and

congest with other routes due to algorithmically unchanging traffic

patterns.  This routing option will "shift" routing around in an

attempt to alleviate this problem.

.TP

\fB\-\-scatter\-ports\fR <random seed>

This option is used to randomize port selection in routing rather than

using a round-robin algorithm (which is the default). Value supplied

with option is used as a random seed.  If value is 0,

which is the default, the scatter ports option is disabled.

.TP

\fB\-H\fR, \fB\-\-max_reverse_hops\fR <max reverse hops allowed>

Set the maximum number of reverse hops an I/O node is allowed

to make. A reverse hop is the use of a switch the wrong way around.

.TP

\fB\-m\fR, \fB\-\-ids_guid_file\fR <file name>

Name of the map file with set of the IDs which will be used

by Up/Down routing algorithm instead of node GUIDs

(format: <guid> <id> per line).

.TP

\fB\-X\fR, \fB\-\-guid_routing_order_file\fR <file name>

Set the order port guids will be routed for the MinHop

and Up/Down routing algorithms to the guids provided in the

given file (one to a line).

.TP

\fB\-o\fR, \fB\-\-once\fR

This option causes OpenSM to configure the subnet

once, then exit.  Ports remain in the ACTIVE state.

.TP

\fB\-s\fR, \fB\-\-sweep\fR <interval value>

This option specifies the number of seconds between

subnet sweeps.  Specifying -s 0 disables sweeping.

Without -s, OpenSM defaults to a sweep interval of

10 seconds.

.TP

\fB\-t\fR, \fB\-\-timeout\fR <value>

This option specifies the time in milliseconds

used for transaction timeouts.

Timeout values should be > 0.

Without -t, OpenSM defaults to a timeout value of

200 milliseconds.

.TP

\fB\-\-retries\fR <number>

This option specifies the number of retries used

for transactions.

Without --retries, OpenSM defaults to 3 retries

for transactions.

.TP

\fB\-\-maxsmps\fR <number>

This option specifies the number of VL15 SMP MADs

allowed on the wire at any one time.

Specifying \-\-maxsmps 0 allows unlimited outstanding

SMPs.

Without \-\-maxsmps, OpenSM defaults to a maximum of

4 outstanding SMPs.

.TP

\fB\-\-console [off | local | loopback | socket]\fR

This option brings up the OpenSM console (default off).  Note, loopback and

socket open a socket which can be connected to WITHOUT CREDENTIALS.  Loopback

is safer if access to your SM host is controlled.  tcp_wrappers

(hosts.[allow|deny]) is used with loopback and socket.  loopback and socket

will only be available if OpenSM was built with --enable-console-loopback

(default yes) and --enable-console-socket (default no) respectively.

.TP

\fB\-\-console-port\fR <port>

Specify an alternate telnet port for the socket console (default 10000).

Note that this option only appears if OpenSM was built with

--enable-console-socket.

.TP

\fB\-i\fR, \fB\-\-ignore_guids\fR <equalize-ignore-guids-file>

This option provides the means to define a set of ports

(by node guid and port number) that will be ignored by the link load

equalization algorithm.

.TP

\fB\-w\fR, \fB\-\-hop_weights_file\fR <path to file>

This option provides weighting factors per port representing a hop cost in

computing the lid matrix.  The file consists of lines containing a switch port

GUID (specified as a 64 bit hex number, with leading 0x), output port number,

and weighting factor.  Any port not listed in the file defaults to a weighting

factor of 1.  Lines starting with # are comments.  Weights affect only the

output route from the port, so many useful configurations will require weights

to be specified in pairs.

.TP

\fB\-O\fR, \fB\-\-port_search_ordering_file\fR <path to file>

This option tweaks the routing. It suitable for two cases:

1. While using DOR routing algorithm.

This option provides a mapping between hypercube dimensions and ports

on a per switch basis for the DOR routing engine.  The file consists

of lines containing a switch node GUID (specified as a 64 bit hex

number, with leading 0x) followed by a list of non-zero port numbers,

separated by spaces, one switch per line.  The order for the port

numbers is in one to one correspondence to the dimensions.  Ports not

listed on a line are assigned to the remaining dimensions, in port

order.  Anything after a # is a comment.

2. While using general routing algorithm.

This option provides the order of the ports that would be chosen for routing,

from each switch rather than searching for an appropriate port from port 1 to N.

The file consists of lines containing a switch node GUID (specified as a 64 bit

hex number, with leading 0x) followed by a list of non-zero port numbers,

separated by spaces, one switch per line.  In case of DOR, the order for the

port numbers is in one to one correspondence to the dimensions.  Ports not

listed on a line are assigned to the remaining dimensions, in port

order.  Anything after a # is a comment.

.TP

\fB\-O\fR, \fB\-\-dimn_ports_file\fR <path to file> \fB(DEPRECATED)\fR

This is a deprecated flag. Please use \fB\-\-port_search_ordering_file\fR instead.

This option provides a mapping between hypercube dimensions and ports

on a per switch basis for the DOR routing engine.  The file consists

of lines containing a switch node GUID (specified as a 64 bit hex

number, with leading 0x) followed by a list of non-zero port numbers,

separated by spaces, one switch per line.  The order for the port

numbers is in one to one correspondence to the dimensions.  Ports not

listed on a line are assigned to the remaining dimensions, in port

order.  Anything after a # is a comment.

.TP

\fB\-x\fR, \fB\-\-honor_guid2lid\fR

This option forces OpenSM to honor the guid2lid file,

when it comes out of Standby state, if such file exists

under OSM_CACHE_DIR, and is valid.

By default, this is FALSE.

.TP

\fB\-\-dump_files_dir <directory name>

This option will set the directory to hold the file dumps.

.TP

\fB\-f\fR, \fB\-\-log_file\fR <file name>

This option defines the log to be the given file.

By default, the log goes to /var/log/opensm.log.

For the log to go to standard output use -f stdout.

.TP

\fB\-L\fR, \fB\-\-log_limit\fR <size in MB>

This option defines maximal log file size in MB. When

specified the log file will be truncated upon reaching

this limit.

.TP

\fB\-e\fR, \fB\-\-erase_log_file\fR

This option will cause deletion of the log file

(if it previously exists). By default, the log file

is accumulative.

.TP

\fB\-P\fR, \fB\-\-Pconfig\fR <partition config file>

This option defines the optional partition configuration file.

The default name is \fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP.

.TP

\fB\-\-prefix_routes_file\fR <file name>

Prefix routes control how the SA responds to path record queries for

off-subnet DGIDs.  By default, the SA fails such queries. The

.B PREFIX ROUTES

section below describes the format of the configuration file.

The default path is \fB\%@OPENSM_CONFIG_DIR@/prefix\-routes.conf\fP.

.TP

\fB\-Q\fR, \fB\-\-qos\fR

This option enables QoS setup. It is disabled by default.

.TP

\fB\-Y\fR, \fB\-\-qos_policy_file\fR <file name>

This option defines the optional QoS policy file. The default

name is \fB\%@OPENSM_CONFIG_DIR@/@QOS_POLICY_FILE@\fP. See

QoS_management_in_OpenSM.txt in opensm doc for more information on

configuring QoS policy via this file.

.TP

\fB\-\-congestion_control\fR

(EXPERIMENTAL) This option enables congestion control configuration.

It is disabled by default.  See config file for congestion control

configuration options.

\fB\-\-cc_key\fR <key>

(EXPERIMENTAL) This option configures the CCkey to use when configuring

congestion control.  Note that this option does not configure a new

CCkey into switches and CAs.  Defaults to 0.

.TP

\fB\-N\fR, \fB\-\-no_part_enforce\fR \fB(DEPRECATED)\fR

This is a deprecated flag. Please use \fB\-\-part_enforce\fR instead.

This option disables partition enforcement on switch external ports.

.TP

\fB\-Z\fR, \fB\-\-part_enforce\fR [both | in | out | off]

This option indicates the partition enforcement type (for switches).

Enforcement type can be inbound only (in), outbound only (out),

both or disabled (off). Default is both.

.TP

\fB\-W\fR, \fB\-\-allow_both_pkeys\fR

This option indicates whether both full and limited membership on the

same partition can be configured in the PKeyTable. Default is not

to allow both pkeys.

.TP

\fB\-y\fR, \fB\-\-stay_on_fatal\fR

This option will cause SM not to exit on fatal initialization

issues: if SM discovers duplicated guids or a 12x link with

lane reversal badly configured.

By default, the SM will exit on these errors.

.TP

\fB\-B\fR, \fB\-\-daemon\fR

Run in daemon mode - OpenSM will run in the background.

.TP

\fB\-J\fR, \fB\-\-pidfile <file_name>\fR

Makes the SM write its own PID to the specified file when started in daemon

mode.

.TP

\fB\-I\fR, \fB\-\-inactive\fR

Start SM in inactive rather than init SM state.  This

option can be used in conjunction with the perfmgr so as to

run a standalone performance manager without SM/SA.  However,

this is NOT currently implemented in the performance manager.

.TP

\fB\-\-perfmgr\fR

Enable the perfmgr.  Only takes effect if --enable-perfmgr was specified at

configure time.  See performance-manager-HOWTO.txt in opensm doc for

more information on running perfmgr.

.TP

\fB\-\-perfmgr_sweep_time_s\fR <seconds>

Specify the sweep time for the performance manager in seconds

(default is 180 seconds).  Only takes

effect if --enable-perfmgr was specified at configure time.

.TP

.BI --consolidate_ipv6_snm_req

Use shared MLID for IPv6 Solicited Node Multicast groups per MGID scope

and P_Key.

.TP

\fB\-\-log_prefix\fR <prefix text>

This option specifies the prefix to the syslog messages from OpenSM.

A suitable prefix can be used to identify the IB subnet in syslog messages

when two or more instances of OpenSM run in a single node to manage multiple

fabrics. For example, in a dual-fabric (or dual-rail) IB cluster, the prefix

for the first fabric could be "mpi" and the other fabric could be "storage".

.TP

\fB\-\-torus_config\fR <path to torus\-2QoS config file>

This option defines the file name for the extra configuration

information needed for the torus-2QoS routing engine.   The default

name is \fB\%@OPENSM_CONFIG_DIR@/@TORUS2QOS_CONF_FILE@\fP

.TP

\fB\-v\fR, \fB\-\-verbose\fR

This option increases the log verbosity level.

The -v option may be specified multiple times

to further increase the verbosity level.

See the -D option for more information about

log verbosity.

.TP

\fB\-V\fR

This option sets the maximum verbosity level and

forces log flushing.

The -V option is equivalent to \'-D 0xFF -d 2\'.

See the -D option for more information about

log verbosity.

.TP

\fB\-D\fR <value>

This option sets the log verbosity level.

A flags field must follow the -D option.

A bit set/clear in the flags enables/disables a

specific log level as follows:

 BIT    LOG LEVEL ENABLED

 ----   -----------------

 0x01 - ERROR (error messages)

 0x02 - INFO (basic messages, low volume)

 0x04 - VERBOSE (interesting stuff, moderate volume)

 0x08 - DEBUG (diagnostic, high volume)

 0x10 - FUNCS (function entry/exit, very high volume)

 0x20 - FRAMES (dumps all SMP and GMP frames)

 0x40 - ROUTING (dump FDB routing information)

 0x80 - SYS (syslog at LOG_INFO level in addition to OpenSM logging)

Without -D, OpenSM defaults to ERROR + INFO (0x3).

Specifying -D 0 disables all messages.

Specifying -D 0xFF enables all messages (see -V).

High verbosity levels may require increasing

the transaction timeout with the -t option.

.TP

\fB\-d\fR, \fB\-\-debug\fR <value>

This option specifies a debug option.

These options are not normally needed.

The number following -d selects the debug

option to enable as follows:

 OPT   Description

 ---    -----------------

 -d0  - Ignore other SM nodes

 -d1  - Force single threaded dispatching

 -d2  - Force log flushing after each log message

 -d3  - Disable multicast support

.TP

\fB\-h\fR, \fB\-\-help\fR

Display this usage info then exit.

.TP

\fB\-?\fR

Display this usage info then exit.

.SH ENVIRONMENT VARIABLES

.PP

The following environment variables control opensm behavior:

OSM_TMP_DIR - controls the directory in which the temporary files generated by

opensm are created. These files are: opensm-subnet.lst, opensm.fdbs, and

opensm.mcfdbs. By default, this directory is /var/log. Note that

\-\-dump_files_dir command line option or dump_file_dir option in

option/config file takes precedence over this environment variable.

OSM_CACHE_DIR - opensm stores certain data to the disk such that subsequent

runs are consistent. The default directory used is /var/cache/opensm.

The following files are included in it:

 guid2lid  - stores the LID range assigned to each GUID

 guid2mkey - stores the MKey previously assigned to each GUID

 neighbors - stores a map of the GUIDs at either end of each link

             in the fabric

.SH NOTES

.PP

When opensm receives a HUP signal, it starts a new heavy sweep as if a trap was received or a topology change was found.

.PP

Also, SIGUSR1 can be used to trigger a reopen of /var/log/opensm.log for

logrotate purposes.

.SH PARTITION CONFIGURATION

.PP

The default name of OpenSM partitions configuration file is

\fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP. The default may be changed

by using the --Pconfig (-P) option with OpenSM.

The default partition will be created by OpenSM unconditionally even

when partition configuration file does not exist or cannot be accessed.

The default partition has P_Key value 0x7fff. OpenSM\'s port will always

have full membership in default partition. All other end ports will have

full membership if the partition configuration file is not found or cannot

be accessed, or limited membership if the file exists and can be accessed

but there is no rule for the Default partition.

Effectively, this amounts to the same as if one of the following rules

below appear in the partition configuration file.

In the case of no rule for the Default partition:

Default=0x7fff : ALL=limited, SELF=full ;

In the case of no partition configuration file or file cannot be accessed:

Default=0x7fff : ALL=full ;

File Format

Comments:

Line content followed after \'#\' character is comment and ignored by

parser.

General file format:

<Partition Definition>:[<newline>]<Partition Properties>;

     Partition Definition:

.nf

       [PartitionName][=PKey][,indx0][,ipoib_bc_flags][,defmember=full|limited]

.fi

        PartitionName  - string, will be used with logging. When

                         omitted, empty string will be used.

        PKey           - P_Key value for this partition. Only low 15

                         bits will be used. When omitted will be

                         autogenerated.

        indx0          - indicates that this pkey should be inserted in

                         block 0 index 0.

        ipoib_bc_flags - used to indicate/specify IPoIB capability of

                         this partition.

        defmember=full|limited|both - specifies default membership for

                         port guid list. Default is limited.

     ipoib_bc_flags:

        ipoib_flag|[mgroup_flag]*

        ipoib_flag:

            ipoib  - indicates that this partition may be used for

                     IPoIB, as a result the IPoIB broadcast group will

                     be created with the mgroup_flag flags given,

                     if any.

     Partition Properties:

       [<Port list>|<MCast Group>]* | <Port list>

     Port list:

        <Port Specifier>[,<Port Specifier>]

     Port Specifier:

        <PortGUID>[=[full|limited|both]]

        PortGUID         - GUID of partition member EndPort.

                           Hexadecimal numbers should start from

                           0x, decimal numbers are accepted too.

        full, limited,   - indicates full and/or limited membership for

        both               this port.  When omitted (or unrecognized)

                           limited membership is assumed.  Both

                           indicates both full and limited membership

                           for this port.

     MCast Group:

        mgid=gid[,mgroup_flag]*<newline>

                         - gid specified is verified to be a Multicast

                           address.  IP groups are verified to match

                           the rate and mtu of the broadcast group.

                           The P_Key bits of the mgid for IP groups are

                           verified to either match the P_Key specified

                           in by "Partition Definition" or if they are

                           0x0000 the P_Key will be copied into those

                           bits.

     mgroup_flag:

        rate=<val>  - specifies rate for this MC group

                      (default is 3 (10GBps))

        mtu=<val>   - specifies MTU for this MC group

                      (default is 4 (2048))

        sl=<val>    - specifies SL for this MC group

                      (default is 0)

        scope=<val> - specifies scope for this MC group

                      (default is 2 (link local)).  Multiple scope

                      settings are permitted for a partition.

                      NOTE: This overwrites the scope nibble of the

                            specified mgid.  Furthermore specifying

                            multiple scope settings will result in

                            multiple MC groups being created.

        Q_Key=<val>     - specifies the Q_Key for this MC group

                          (default: 0x0b1b for IP groups, 0 for other

                           groups)

                          WARNING: changing this for the broadcast

                                   group may break IPoIB on client

                                   nodes!! 

        TClass=<val>    - specifies tclass for this MC group

                          (default is 0)

        FlowLabel=<val> - specifies FlowLabel for this MC group

                          (default is 0)

	NOTE: All mgroup_flag flags MUST be separated by comma (,).

Note that values for rate, mtu, and scope, for both partitions and multicast

groups, should be specified as defined in the IBTA specification (for example,

mtu=4 for 2048).

There are several useful keywords for PortGUID definition:

 - 'ALL' means all end ports in this subnet.

 - 'ALL_CAS' means all Channel Adapter end ports in this subnet.

 - 'ALL_SWITCHES' means all Switch end ports in this subnet.

 - 'ALL_ROUTERS' means all Router end ports in this subnet.

 - 'SELF' means subnet manager's port.

Empty list means no ports in this partition.

Notes:

White space is permitted between delimiters ('=', ',',':',';').

PartitionName does not need to be unique, PKey does need to be unique.

If PKey is repeated then those partition configurations will be merged

and first PartitionName will be used (see also next note).

It is possible to split partition configuration in more than one

definition, but then PKey should be explicitly specified (otherwise

different PKey values will be generated for those definitions).

Examples:

 Default=0x7fff : ALL, SELF=full ;

 Default=0x7fff : ALL, ALL_SWITCHES=full, SELF=full ;

 NewPartition , ipoib : 0x123456=full, 0x3456789034=limi, 0x2134af2306 ;

 YetAnotherOne = 0x300 : SELF=full ;

 YetAnotherOne = 0x300 : ALL=limited ;

 ShareIO = 0x80 , defmember=full : 0x123451, 0x123452;

 # 0x123453, 0x123454 will be limited

 ShareIO = 0x80 : 0x123453, 0x123454, 0x123455=full;

 # 0x123456, 0x123457 will be limited

 ShareIO = 0x80 : defmember=limited : 0x123456, 0x123457, 0x123458=full;

 ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;

 ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;

 # multicast groups added to default

 Default=0x7fff,ipoib:

        mgid=ff12:401b::0707,sl=1 # random IPv4 group

        mgid=ff12:601b::16    # MLDv2-capable routers

        mgid=ff12:401b::16    # IGMP

        mgid=ff12:601b::2     # All routers

        mgid=ff12::1,sl=1,Q_Key=0xDEADBEEF,rate=3,mtu=2 # random group

        ALL=full;

Note:

The following rule is equivalent to how OpenSM used to run prior to the

partition manager:

 Default=0x7fff,ipoib:ALL=full;

.SH QOS CONFIGURATION

.PP

There are a set of QoS related low-level configuration parameters.

All these parameter names are prefixed by "qos_" string. Here is a full

list of these parameters:

 qos_max_vls    - The maximum number of VLs that will be on the subnet

 qos_high_limit - The limit of High Priority component of VL

                  Arbitration table (IBA 7.6.9)

 qos_vlarb_low  - Low priority VL Arbitration table (IBA 7.6.9)

                  template

 qos_vlarb_high - High priority VL Arbitration table (IBA 7.6.9)

                  template

                  Both VL arbitration templates are pairs of

                  VL and weight

 qos_sl2vl      - SL2VL Mapping table (IBA 7.6.6) template. It is

                  a list of VLs corresponding to SLs 0-15 (Note

                  that VL15 used here means drop this SL)

Typical default values (hard-coded in OpenSM initialization) are:

 qos_max_vls 15

 qos_high_limit 0

.nf

 qos_vlarb_low 0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4

 qos_vlarb_high 0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0

.fi

 qos_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7

The syntax is compatible with rest of OpenSM configuration options and

values may be stored in OpenSM config file (cached options file).

In addition to the above, we may define separate QoS configuration

parameters sets for various target types. As targets, we currently support

CAs, routers, switch external ports, and switch's enhanced port 0. The

names of such specialized parameters are prefixed by "qos_<type>_"

string. Here is a full list of the currently supported sets:

 qos_ca_  - QoS configuration parameters set for CAs.

 qos_rtr_ - parameters set for routers.

 qos_sw0_ - parameters set for switches' port 0.

 qos_swe_ - parameters set for switches' external ports.

Examples:

 qos_sw0_max_vls=2

 qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0,

 qos_swe_high_limit=0

.SH PREFIX ROUTES

.PP

Prefix routes control how the SA responds to path record queries for

off-subnet DGIDs.  By default, the SA fails such queries.

Note that IBA does not specify how the SA should obtain off-subnet path

record information.

The prefix routes configuration is meant as a stop-gap until the

specification is completed.

.PP

Each line in the configuration file is a 64-bit prefix followed by a

64-bit GUID, separated by white space.

The GUID specifies the router port on the local subnet that will

handle the prefix.

Blank lines are ignored, as is anything between a \fB#\fP character

and the end of the line.

The prefix and GUID are both in hex, the leading 0x is optional.

Either, or both, can be wild-carded by specifying an

asterisk instead of an explicit prefix or GUID.

.PP

When responding to a path record query for an off-subnet DGID,

opensm searches for the first prefix match in the configuration file.

Therefore, the order of the lines in the configuration file is important:

a wild-carded prefix at the beginning of the configuration file renders

all subsequent lines useless.

If there is no match, then opensm fails the query.

It is legal to repeat prefixes in the configuration file,

opensm will return the path to the first available matching router.

A configuration file with a single line where both prefix and GUID

are wild-carded means that a path record query specifying any

off-subnet DGID should return a path to the first available router.

This configuration yields the same behavior formerly achieved by

compiling opensm with -DROUTER_EXP which has been obsoleted.

.SH MKEY CONFIGURATION

.PP

OpenSM supports configuring a single management key (MKey) for use across

the subnet.

The following configuration options are available:

 m_key                  - the 64-bit MKey to be used on the subnet

                          (IBA 14.2.4)

 m_key_protection_level - the numeric value of the MKey ProtectBits

                          (IBA 14.2.4.1)

 m_key_lease_period     - the number of seconds a CA will wait for a

                          response from the SM before resetting the

                          protection level to 0 (IBA 14.2.4.2).

OpenSM will configure all ports with the MKey specified by m_key, defaulting

to a value of 0. A m_key value of 0 disables MKey protection on the subnet.

Switches and HCAs with a non-zero MKey will not accept requests to change

their configuration unless the request includes the proper MKey.

MKey Protection Levels

MKey protection levels modify how switches and CAs respond to SMPs lacking

a valid MKey.

OpenSM will configure each port's ProtectBits to support the level defined by

the m_key_protection_level parameter.  If no parameter is specified, OpenSM

defaults to operating at protection level 0.

There are currently 4 protection levels defined by the IBA:

 0 - Queries return valid data, including MKey.  Configuration changes

     are not allowed unless the request contains a valid MKey.

 1 - Like level 0, but the MKey is set to 0 (0x00000000) in queries,

     unless the request contains a valid MKey.

 2 - Neither queries nor configuration changes are allowed, unless the

     request contains a valid MKey.

 3 - Identical to 2.  Maintained for backwards compatibility.

MKey Lease Period

InfiniBand supports a MKey lease timeout, which is intended to allow

administrators or a new SM to recover/reset lost MKeys on a fabric.

If MKeys are enabled on the subnet and a switch or CA receives a request that

requires a valid MKey but does not contain one, it warns the SM by sending a trap

(Bad M_Key, Trap 256).  If the MKey lease period is non-zero, it also starts a

countdown timer for the time specified by the lease period.

If a SM (or other agent) responds with the correct MKey, the timer is stopped

and reset.  Should the timer reach zero, the switch or CA will reset its MKey

protection level to 0, exposing the MKey and allowing recovery.

OpenSM will initialize all ports to use a mkey lease period of the number of

seconds specified in the config file.  If no mkey_lease_period is specified,

a default of 0 will be used.

OpenSM normally quickly responds to all Bad_M_Key traps, resetting the lease

timers.  Additionally, OpenSM's subnet sweeps will also cancel

any running timers.  For maximum protection against accidentally-exposed MKeys,

the MKey lease time should be a few multiples of the subnet sweep time.

If OpenSM detects at startup that your sweep interval is greater than your

MKey lease period, it will reset the lease period to be greater than the