README for auothbw library
Ruchira Sasanka (ruchira DOT sasanka AT intel DOT com)
2015 April 8

PURPOSE
-------
The autohbw library is used to automatically allocate high-bandwidth (HBW)
memory without any modifications to source code of your application.
This library intercepts the standard heap allocations (e.g., malloc, calloc)
in your program so that it can serve those allocations out of HBW memory.

There are two libraries installed at the same location as memkind library:
  1) libautohbw.so -- dynamic library
  2) libautohbw.a  -- static library (provided for static linking)

USAGE
-----
To use the dynamic library, insert LD_PRELOAD=libautohbw.so and libmekind.so
before your commandline. For instance,

   LD_PRELOAD=libautohbw.so /bin/ls

executes Unix ls utility with autohbw library. You can execute the
above command on the prompt or put it in a shell script. An example of
the latter approach is given in ./autohbw_test.sh. Make sure that
LD_LIBRARY_PATH environment variable contains the path to libautohbw.so
and libmekind.so before you exeute the above command.

To use the statically linked library, insert -lautohbw -lmemkind
on your link line. You will have to provide the library path with -L to
point to the above libraries. If you are linking your program statically
(e.g., with -static or -fast flags in ifort/icc), you must use the static
version of autohbw library (libautohbw.a).

It is possible to temporarily disable/enable automatic HBM allocations by
calling disableAutoHBW() and enableAutoHBW() in your source code. To call
these routines, please include autohbw_api.h header file and link with
-lautohbw.


Usage with MPI programs
-----------------------
To use with MPI programs, you need to create a shell script to do the
LD_PRELOAD. E.g.,
   mpirun -n 2 ./autohbw_test.sh

ENVIRONMENT VARIABLES
---------------------
The following environment variables control the behavior of the autohbw
library:

  AUTO_HBW_SIZE=x[:y]
  Indicates that any allocation larger than x and smaller than y should be
  allocated in HBW memory. x and y can be followed by a K, M, or G to
  indicate the size in Kilo/Memga/Giga bytes (e.g., 4K, 3M, 2G).
  Examples:
    AUTO_HBW_SIZE=4K     # all allocation larger than 4K allocated in HBW
    AUTO_HBW_SIZE=1M:5M  # allocations between 1M and 5M allocated in HBW


  AUTO_HBW_LOG=level
  Sets the value of logging (printing) level. If level is:
   -1 = no messages are printed
    0 = no log messages for allocations are printed but INFO messages
        are printed
    1 = a log message is printed for each allocation (Default)
    2 = a log message is printed for each allocation with a backtrace
        Redirect this output and use autohbw_get_src_lines.pl to find
        source lines for each allocation. Your application must be compiled
        with -g to see source lines.
  Notes:
    1. Logging adds extra overhead. Therefore, performance critical runs,
       logging level should be 0
    2. The amount of free memory printed with log messages is only
       approximate -- e.g. pages that are not touched yet are excluded
  Examples:
    AUTO_HBW_LOG=1


  AUTO_HBW_MEM_TYPE=memory_type
  Sets the type of memory type that should be automatically allocated. By
  default, this type is MEMKIND_HBW_PREFERRED, if MCDRAM is found in your
  system; otherwise, the default is MEMKIND_DEFAULT. The names of memory
  types are definedin memkind(3) man page. If you are requesting any huge
  TLB pages, pleasemake sure that the requested type is currently enabled
  in your OS.
  Examples:
    AUTO_HBW_MEM_TYPE=MEMKIND_HBW_PREFERRED   (default, if MCDRAM present)
    AUTO_HBW_MEM_TYPE=MEMKIND_DEFAULT         (default, if MCDRAM absent)
    AUTO_HBW_MEM_TYPE=MEMKIND_HBW_HUGETLB
    AUTO_HBW_MEM_TYPE=MEMKIND_HUGETLB