Copyright (c) 2017, Intel Corporation

All rights reserved.

Redistribution and use in source and binary forms, with or without modification,

 are permitted provided that the following conditions are met:

- Redistributions of source code must retain the above copyright notice,

  this list of conditions and the following disclaimer.

- Redistributions in binary form must reproduce the above copyright notice,

  this list of conditions and the following disclaimer in the documentation

  and/or other materials provided with the distribution.

- Neither the name of Intel Corporation nor the names of its contributors may

  be used to endorse or promote products derived from this software without

  specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"

AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE

ARE DISCLAIMED. IN NO EVENT SHALL INTEL, THE COPYRIGHT OWNER OR CONTRIBUTORS

BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR

CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF

SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN

CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)

ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE

POSSIBILITY OF SUCH DAMAGE.

EXPORT LAWS: THIS LICENSE ADDS NO RESTRICTIONS TO THE EXPORT LAWS OF YOUR

JURISDICTION. It is licensee's responsibility to comply with any export

regulations applicable in licensee's jurisdiction. Under CURRENT (May 2000)

U.S. export regulations this software is eligible for export from the U.S.

and can be downloaded by or otherwise exported or reexported worldwide EXCEPT

to U.S. embargoed destinations which include Cuba, Iraq, Libya, North Korea,

Iran, Syria, Sudan, Afghanistan and any other country to which the U.S. has

embargoed goods and services.

This package contains SHMEM applications and packages for testing and

evaluation.

MPICH_PREFIX ENVIRONMENT VARIABLE:

----------------------------------

The SHMEM samples require an MPI implementation to run. Sandia

OpenSHMEM requires a Hydra* based MPI to run. The select_mpi script

will look for MVAPICH2 directories matching either

mvapich2*hfi (first) or mvapich* (second).

If desired the MPICH_PREFIX may be exported to control the selection.

Alternatively, a .prefix file can be created in this directory with the

preferred selection.

SHMEM_PREFIX ENVIRONMENT VARIABLE:

----------------------------------

By default, Make will build these programs with Sandia OpenSHMEM installed

by RPM sandia-openshmem_gcc_hfi-devel. SHMEM_PREFIX is also used by the

run_* scripts to define where bin/oshrun is found.

You can override this by setting SHMEM_PREFIX in your environment. For

example:

export SHMEM_PREFIX=/my/copy_of_shmem

The run_* sample runner scripts set SHMEM_PREFIX from a file

.shmem_prefix in the current working directory if that

SHMEM_PREFIX is not already set.

USING THESE TOOLS WITH MPI-SELECTOR:

------------------------------------

SHMEM applications are started using the mpirun from an MPI

implementation such mvapich2.

In OFED (and Intel IFS), the normal method of selecting an MPI is to

use the mpi-selector commands (type "man mpi-selector-menu" and "man

mpi-selector" for details on those commands).

To use mpi-selector with these scripts, add this line to your .bashrc

file:

source /usr/src/opa/mpi_apps/get_selected_mpi.sh

COMPILING THE TEST PROGRAMS:

----------------------------

The top level Makefile will build a simple SHMEM hello_worl application,

using the default (described above) or the value of SHMEM_PREFIX (also

described above).

RUNNING THE TEST PROGRAMS:

--------------------------

A "run_*" script is provided for the applications/benchmark.  These

scripts assume the existence of a local "shmem_hosts" file [alternate name can be

exported in MPI_HOSTS if desired] (in mpirun mode)

and that mpd is already running (if you're using mvapich2 in mpd mode).

For example, to run the barrier test, simply type:

# ./run_barrier 2

The scripts log the output of the SHMEM programs to the

/usr/src/opa/shmem_apps/logs directory.