Blame jemalloc/INSTALL.md

Packit Service 724aca
Building and installing a packaged release of jemalloc can be as simple as
Packit Service 724aca
typing the following while in the root directory of the source tree:
Packit Service 724aca
Packit Service 724aca
    ./configure
Packit Service 724aca
    make
Packit Service 724aca
    make install
Packit Service 724aca
Packit Service 724aca
If building from unpackaged developer sources, the simplest command sequence
Packit Service 724aca
that might work is:
Packit Service 724aca
Packit Service 724aca
    ./autogen.sh
Packit Service 724aca
    make dist
Packit Service 724aca
    make
Packit Service 724aca
    make install
Packit Service 724aca
Packit Service 724aca
Note that documentation is not built by the default target because doing so
Packit Service 724aca
would create a dependency on xsltproc in packaged releases, hence the
Packit Service 724aca
requirement to either run 'make dist' or avoid installing docs via the various
Packit Service 724aca
install_* targets documented below.
Packit Service 724aca
Packit Service 724aca
Packit Service 724aca
## Advanced configuration
Packit Service 724aca
Packit Service 724aca
The 'configure' script supports numerous options that allow control of which
Packit Service 724aca
functionality is enabled, where jemalloc is installed, etc.  Optionally, pass
Packit Service 724aca
any of the following arguments (not a definitive list) to 'configure':
Packit Service 724aca
Packit Service 724aca
* `--help`
Packit Service 724aca
Packit Service 724aca
    Print a definitive list of options.
Packit Service 724aca
Packit Service 724aca
* `--prefix=<install-root-dir>`
Packit Service 724aca
Packit Service 724aca
    Set the base directory in which to install.  For example:
Packit Service 724aca
Packit Service 724aca
        ./configure --prefix=/usr/local
Packit Service 724aca
Packit Service 724aca
    will cause files to be installed into /usr/local/include, /usr/local/lib,
Packit Service 724aca
    and /usr/local/man.
Packit Service 724aca
Packit Service 724aca
* `--with-version=(<major>.<minor>.<bugfix>-<nrev>-g<gid>|VERSION)`
Packit Service 724aca
Packit Service 724aca
    The VERSION file is mandatory for successful configuration, and the
Packit Service 724aca
    following steps are taken to assure its presence:
Packit Service 724aca
    1) If --with-version=<major>.<minor>.<bugfix>-<nrev>-g<gid> is specified,
Packit Service 724aca
       generate VERSION using the specified value.
Packit Service 724aca
    2) If --with-version is not specified in either form and the source
Packit Service 724aca
       directory is inside a git repository, try to generate VERSION via 'git
Packit Service 724aca
       describe' invocations that pattern-match release tags.
Packit Service 724aca
    3) If VERSION is missing, generate it with a bogus version:
Packit Service 724aca
       0.0.0-0-g0000000000000000000000000000000000000000
Packit Service 724aca
Packit Service 724aca
    Note that --with-version=VERSION bypasses (1) and (2), which simplifies
Packit Service 724aca
    VERSION configuration when embedding a jemalloc release into another
Packit Service 724aca
    project's git repository.
Packit Service 724aca
Packit Service 724aca
* `--with-rpath=<colon-separated-rpath>`
Packit Service 724aca
Packit Service 724aca
    Embed one or more library paths, so that libjemalloc can find the libraries
Packit Service 724aca
    it is linked to.  This works only on ELF-based systems.
Packit Service 724aca
Packit Service 724aca
* `--with-mangling=<map>`
Packit Service 724aca
Packit Service 724aca
    Mangle public symbols specified in <map> which is a comma-separated list of
Packit Service 724aca
    name:mangled pairs.
Packit Service 724aca
Packit Service 724aca
    For example, to use ld's --wrap option as an alternative method for
Packit Service 724aca
    overriding libc's malloc implementation, specify something like:
Packit Service 724aca
Packit Service 724aca
      --with-mangling=malloc:__wrap_malloc,free:__wrap_free[...]
Packit Service 724aca
Packit Service 724aca
    Note that mangling happens prior to application of the prefix specified by
Packit Service 724aca
    --with-jemalloc-prefix, and mangled symbols are then ignored when applying
Packit Service 724aca
    the prefix.
Packit Service 724aca
Packit Service 724aca
* `--with-jemalloc-prefix=<prefix>`
Packit Service 724aca
Packit Service 724aca
    Prefix all public APIs with <prefix>.  For example, if <prefix> is
Packit Service 724aca
    "prefix_", API changes like the following occur:
Packit Service 724aca
Packit Service 724aca
      malloc()         --> prefix_malloc()
Packit Service 724aca
      malloc_conf      --> prefix_malloc_conf
Packit Service 724aca
      /etc/malloc.conf --> /etc/prefix_malloc.conf
Packit Service 724aca
      MALLOC_CONF      --> PREFIX_MALLOC_CONF
Packit Service 724aca
Packit Service 724aca
    This makes it possible to use jemalloc at the same time as the system
Packit Service 724aca
    allocator, or even to use multiple copies of jemalloc simultaneously.
Packit Service 724aca
Packit Service 724aca
    By default, the prefix is "", except on OS X, where it is "je_".  On OS X,
Packit Service 724aca
    jemalloc overlays the default malloc zone, but makes no attempt to actually
Packit Service 724aca
    replace the "malloc", "calloc", etc. symbols.
Packit Service 724aca
Packit Service 724aca
* `--without-export`
Packit Service 724aca
Packit Service 724aca
    Don't export public APIs.  This can be useful when building jemalloc as a
Packit Service 724aca
    static library, or to avoid exporting public APIs when using the zone
Packit Service 724aca
    allocator on OSX.
Packit Service 724aca
Packit Service 724aca
* `--with-private-namespace=<prefix>`
Packit Service 724aca
Packit Service 724aca
    Prefix all library-private APIs with <prefix>je_.  For shared libraries,
Packit Service 724aca
    symbol visibility mechanisms prevent these symbols from being exported, but
Packit Service 724aca
    for static libraries, naming collisions are a real possibility.  By
Packit Service 724aca
    default, <prefix> is empty, which results in a symbol prefix of je_ .
Packit Service 724aca
Packit Service 724aca
* `--with-install-suffix=<suffix>`
Packit Service 724aca
Packit Service 724aca
    Append <suffix> to the base name of all installed files, such that multiple
Packit Service 724aca
    versions of jemalloc can coexist in the same installation directory.  For
Packit Service 724aca
    example, libjemalloc.so.0 becomes libjemalloc<suffix>.so.0.
Packit Service 724aca
Packit Service 724aca
* `--with-malloc-conf=<malloc_conf>`
Packit Service 724aca
Packit Service 724aca
    Embed `<malloc_conf>` as a run-time options string that is processed prior to
Packit Service 724aca
    the malloc_conf global variable, the /etc/malloc.conf symlink, and the
Packit Service 724aca
    MALLOC_CONF environment variable.  For example, to change the default decay
Packit Service 724aca
    time to 30 seconds:
Packit Service 724aca
Packit Service 724aca
      --with-malloc-conf=decay_ms:30000
Packit Service 724aca
Packit Service 724aca
* `--enable-debug`
Packit Service 724aca
Packit Service 724aca
    Enable assertions and validation code.  This incurs a substantial
Packit Service 724aca
    performance hit, but is very useful during application development.
Packit Service 724aca
Packit Service 724aca
* `--disable-stats`
Packit Service 724aca
Packit Service 724aca
    Disable statistics gathering functionality.  See the "opt.stats_print"
Packit Service 724aca
    option documentation for usage details.
Packit Service 724aca
Packit Service 724aca
* `--enable-prof`
Packit Service 724aca
Packit Service 724aca
    Enable heap profiling and leak detection functionality.  See the "opt.prof"
Packit Service 724aca
    option documentation for usage details.  When enabled, there are several
Packit Service 724aca
    approaches to backtracing, and the configure script chooses the first one
Packit Service 724aca
    in the following list that appears to function correctly:
Packit Service 724aca
Packit Service 724aca
    + libunwind      (requires --enable-prof-libunwind)
Packit Service 724aca
    + libgcc         (unless --disable-prof-libgcc)
Packit Service 724aca
    + gcc intrinsics (unless --disable-prof-gcc)
Packit Service 724aca
Packit Service 724aca
* `--enable-prof-libunwind`
Packit Service 724aca
Packit Service 724aca
    Use the libunwind library (http://www.nongnu.org/libunwind/) for stack
Packit Service 724aca
    backtracing.
Packit Service 724aca
Packit Service 724aca
* `--disable-prof-libgcc`
Packit Service 724aca
Packit Service 724aca
    Disable the use of libgcc's backtracing functionality.
Packit Service 724aca
Packit Service 724aca
* `--disable-prof-gcc`
Packit Service 724aca
Packit Service 724aca
    Disable the use of gcc intrinsics for backtracing.
Packit Service 724aca
Packit Service 724aca
* `--with-static-libunwind=<libunwind.a>`
Packit Service 724aca
Packit Service 724aca
    Statically link against the specified libunwind.a rather than dynamically
Packit Service 724aca
    linking with -lunwind.
Packit Service 724aca
Packit Service 724aca
* `--disable-fill`
Packit Service 724aca
Packit Service 724aca
    Disable support for junk/zero filling of memory.  See the "opt.junk" and
Packit Service 724aca
    "opt.zero" option documentation for usage details.
Packit Service 724aca
Packit Service 724aca
* `--disable-zone-allocator`
Packit Service 724aca
Packit Service 724aca
    Disable zone allocator for Darwin.  This means jemalloc won't be hooked as
Packit Service 724aca
    the default allocator on OSX/iOS.
Packit Service 724aca
Packit Service 724aca
* `--enable-utrace`
Packit Service 724aca
Packit Service 724aca
    Enable utrace(2)-based allocation tracing.  This feature is not broadly
Packit Service 724aca
    portable (FreeBSD has it, but Linux and OS X do not).
Packit Service 724aca
Packit Service 724aca
* `--enable-xmalloc`
Packit Service 724aca
Packit Service 724aca
    Enable support for optional immediate termination due to out-of-memory
Packit Service 724aca
    errors, as is commonly implemented by "xmalloc" wrapper function for malloc.
Packit Service 724aca
    See the "opt.xmalloc" option documentation for usage details.
Packit Service 724aca
Packit Service 724aca
* `--enable-lazy-lock`
Packit Service 724aca
Packit Service 724aca
    Enable code that wraps pthread_create() to detect when an application
Packit Service 724aca
    switches from single-threaded to multi-threaded mode, so that it can avoid
Packit Service 724aca
    mutex locking/unlocking operations while in single-threaded mode.  In
Packit Service 724aca
    practice, this feature usually has little impact on performance unless
Packit Service 724aca
    thread-specific caching is disabled.
Packit Service 724aca
Packit Service 724aca
* `--disable-cache-oblivious`
Packit Service 724aca
Packit Service 724aca
    Disable cache-oblivious large allocation alignment for large allocation
Packit Service 724aca
    requests with no alignment constraints.  If this feature is disabled, all
Packit Service 724aca
    large allocations are page-aligned as an implementation artifact, which can
Packit Service 724aca
    severely harm CPU cache utilization.  However, the cache-oblivious layout
Packit Service 724aca
    comes at the cost of one extra page per large allocation, which in the
Packit Service 724aca
    most extreme case increases physical memory usage for the 16 KiB size class
Packit Service 724aca
    to 20 KiB.
Packit Service 724aca
Packit Service 724aca
* `--disable-syscall`
Packit Service 724aca
Packit Service 724aca
    Disable use of syscall(2) rather than {open,read,write,close}(2).  This is
Packit Service 724aca
    intended as a workaround for systems that place security limitations on
Packit Service 724aca
    syscall(2).
Packit Service 724aca
Packit Service 724aca
* `--disable-cxx`
Packit Service 724aca
Packit Service 724aca
    Disable C++ integration.  This will cause new and delete operator
Packit Service 724aca
    implementations to be omitted.
Packit Service 724aca
Packit Service 724aca
* `--with-xslroot=<path>`
Packit Service 724aca
Packit Service 724aca
    Specify where to find DocBook XSL stylesheets when building the
Packit Service 724aca
    documentation.
Packit Service 724aca
Packit Service 724aca
* `--with-lg-page=<lg-page>`
Packit Service 724aca
Packit Service 724aca
    Specify the base 2 log of the allocator page size, which must in turn be at
Packit Service 724aca
    least as large as the system page size.  By default the configure script
Packit Service 724aca
    determines the host's page size and sets the allocator page size equal to
Packit Service 724aca
    the system page size, so this option need not be specified unless the
Packit Service 724aca
    system page size may change between configuration and execution, e.g. when
Packit Service 724aca
    cross compiling.
Packit Service 724aca
Packit Service 724aca
* `--with-lg-hugepage=<lg-hugepage>`
Packit Service 724aca
Packit Service 724aca
    Specify the base 2 log of the system huge page size.  This option is useful
Packit Service 724aca
    when cross compiling, or when overriding the default for systems that do
Packit Service 724aca
    not explicitly support huge pages.
Packit Service 724aca
Packit Service 724aca
* `--with-lg-quantum=<lg-quantum>`
Packit Service 724aca
Packit Service 724aca
    Specify the base 2 log of the minimum allocation alignment.  jemalloc needs
Packit Service 724aca
    to know the minimum alignment that meets the following C standard
Packit Service 724aca
    requirement (quoted from the April 12, 2011 draft of the C11 standard):
Packit Service 724aca
Packit Service 724aca
    >  The pointer returned if the allocation succeeds is suitably aligned so
Packit Service 724aca
      that it may be assigned to a pointer to any type of object with a
Packit Service 724aca
      fundamental alignment requirement and then used to access such an object
Packit Service 724aca
      or an array of such objects in the space allocated [...]
Packit Service 724aca
Packit Service 724aca
    This setting is architecture-specific, and although jemalloc includes known
Packit Service 724aca
    safe values for the most commonly used modern architectures, there is a
Packit Service 724aca
    wrinkle related to GNU libc (glibc) that may impact your choice of
Packit Service 724aca
    <lg-quantum>.  On most modern architectures, this mandates 16-byte
Packit Service 724aca
    alignment (<lg-quantum>=4), but the glibc developers chose not to meet this
Packit Service 724aca
    requirement for performance reasons.  An old discussion can be found at
Packit Service 724aca
    <https://sourceware.org/bugzilla/show_bug.cgi?id=206> .  Unlike glibc,
Packit Service 724aca
    jemalloc does follow the C standard by default (caveat: jemalloc
Packit Service 724aca
    technically cheats for size classes smaller than the quantum), but the fact
Packit Service 724aca
    that Linux systems already work around this allocator noncompliance means
Packit Service 724aca
    that it is generally safe in practice to let jemalloc's minimum alignment
Packit Service 724aca
    follow glibc's lead.  If you specify `--with-lg-quantum=3` during
Packit Service 724aca
    configuration, jemalloc will provide additional size classes that are not
Packit Service 724aca
    16-byte-aligned (24, 40, and 56).
Packit Service 724aca
Packit Service 724aca
* `--with-lg-vaddr=<lg-vaddr>`
Packit Service 724aca
Packit Service 724aca
    Specify the number of significant virtual address bits.  By default, the
Packit Service 724aca
    configure script attempts to detect virtual address size on those platforms
Packit Service 724aca
    where it knows how, and picks a default otherwise.  This option may be
Packit Service 724aca
    useful when cross-compiling.
Packit Service 724aca
Packit Service 724aca
* `--disable-initial-exec-tls`
Packit Service 724aca
Packit Service 724aca
    Disable the initial-exec TLS model for jemalloc's internal thread-local
Packit Service 724aca
    storage (on those platforms that support explicit settings).  This can allow
Packit Service 724aca
    jemalloc to be dynamically loaded after program startup (e.g. using dlopen).
Packit Service 724aca
    Note that in this case, there will be two malloc implementations operating
Packit Service 724aca
    in the same process, which will almost certainly result in confusing runtime
Packit Service 724aca
    crashes if pointers leak from one implementation to the other.
Packit Service 724aca
Packit Service 724aca
* `--disable-libdl`
Packit Service 724aca
Packit Service 724aca
    Disable the usage of libdl, namely dlsym(3) which is required by the lazy
Packit Service 724aca
    lock option.  This can allow building static binaries.
Packit Service 724aca
Packit Service 724aca
The following environment variables (not a definitive list) impact configure's
Packit Service 724aca
behavior:
Packit Service 724aca
Packit Service 724aca
* `CFLAGS="?"`
Packit Service 724aca
* `CXXFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags to the C/C++ compiler.  Any flags set by the configure
Packit Service 724aca
    script are prepended, which means explicitly set flags generally take
Packit Service 724aca
    precedence.  Take care when specifying flags such as -Werror, because
Packit Service 724aca
    configure tests may be affected in undesirable ways.
Packit Service 724aca
Packit Service 724aca
* `EXTRA_CFLAGS="?"`
Packit Service 724aca
* `EXTRA_CXXFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Append these flags to CFLAGS/CXXFLAGS, without passing them to the
Packit Service 724aca
    compiler(s) during configuration.  This makes it possible to add flags such
Packit Service 724aca
    as -Werror, while allowing the configure script to determine what other
Packit Service 724aca
    flags are appropriate for the specified configuration.
Packit Service 724aca
Packit Service 724aca
* `CPPFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags to the C preprocessor.  Note that CFLAGS is not passed to
Packit Service 724aca
    'cpp' when 'configure' is looking for include files, so you must use
Packit Service 724aca
    CPPFLAGS instead if you need to help 'configure' find header files.
Packit Service 724aca
Packit Service 724aca
* `LD_LIBRARY_PATH="?"`
Packit Service 724aca
Packit Service 724aca
    'ld' uses this colon-separated list to find libraries.
Packit Service 724aca
Packit Service 724aca
* `LDFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags when linking.
Packit Service 724aca
Packit Service 724aca
* `PATH="?"`
Packit Service 724aca
Packit Service 724aca
    'configure' uses this to find programs.
Packit Service 724aca
Packit Service 724aca
In some cases it may be necessary to work around configuration results that do
Packit Service 724aca
not match reality.  For example, Linux 4.5 added support for the MADV_FREE flag
Packit Service 724aca
to madvise(2), which can cause problems if building on a host with MADV_FREE
Packit Service 724aca
support and deploying to a target without.  To work around this, use a cache
Packit Service 724aca
file to override the relevant configuration variable defined in configure.ac,
Packit Service 724aca
e.g.:
Packit Service 724aca
Packit Service 724aca
    echo "je_cv_madv_free=no" > config.cache && ./configure -C
Packit Service 724aca
Packit Service 724aca
Packit Service 724aca
## Advanced compilation
Packit Service 724aca
Packit Service 724aca
To build only parts of jemalloc, use the following targets:
Packit Service 724aca
Packit Service 724aca
    build_lib_shared
Packit Service 724aca
    build_lib_static
Packit Service 724aca
    build_lib
Packit Service 724aca
    build_doc_html
Packit Service 724aca
    build_doc_man
Packit Service 724aca
    build_doc
Packit Service 724aca
Packit Service 724aca
To install only parts of jemalloc, use the following targets:
Packit Service 724aca
Packit Service 724aca
    install_bin
Packit Service 724aca
    install_include
Packit Service 724aca
    install_lib_shared
Packit Service 724aca
    install_lib_static
Packit Service 724aca
    install_lib_pc
Packit Service 724aca
    install_lib
Packit Service 724aca
    install_doc_html
Packit Service 724aca
    install_doc_man
Packit Service 724aca
    install_doc
Packit Service 724aca
Packit Service 724aca
To clean up build results to varying degrees, use the following make targets:
Packit Service 724aca
Packit Service 724aca
    clean
Packit Service 724aca
    distclean
Packit Service 724aca
    relclean
Packit Service 724aca
Packit Service 724aca
Packit Service 724aca
## Advanced installation
Packit Service 724aca
Packit Service 724aca
Optionally, define make variables when invoking make, including (not
Packit Service 724aca
exclusively):
Packit Service 724aca
Packit Service 724aca
* `INCLUDEDIR="?"`
Packit Service 724aca
Packit Service 724aca
    Use this as the installation prefix for header files.
Packit Service 724aca
Packit Service 724aca
* `LIBDIR="?"`
Packit Service 724aca
Packit Service 724aca
    Use this as the installation prefix for libraries.
Packit Service 724aca
Packit Service 724aca
* `MANDIR="?"`
Packit Service 724aca
Packit Service 724aca
    Use this as the installation prefix for man pages.
Packit Service 724aca
Packit Service 724aca
* `DESTDIR="?"`
Packit Service 724aca
Packit Service 724aca
    Prepend DESTDIR to INCLUDEDIR, LIBDIR, DATADIR, and MANDIR.  This is useful
Packit Service 724aca
    when installing to a different path than was specified via --prefix.
Packit Service 724aca
Packit Service 724aca
* `CC="?"`
Packit Service 724aca
Packit Service 724aca
    Use this to invoke the C compiler.
Packit Service 724aca
Packit Service 724aca
* `CFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags to the compiler.
Packit Service 724aca
Packit Service 724aca
* `CPPFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags to the C preprocessor.
Packit Service 724aca
Packit Service 724aca
* `LDFLAGS="?"`
Packit Service 724aca
Packit Service 724aca
    Pass these flags when linking.
Packit Service 724aca
Packit Service 724aca
* `PATH="?"`
Packit Service 724aca
Packit Service 724aca
    Use this to search for programs used during configuration and building.
Packit Service 724aca
Packit Service 724aca
Packit Service 724aca
## Development
Packit Service 724aca
Packit Service 724aca
If you intend to make non-trivial changes to jemalloc, use the 'autogen.sh'
Packit Service 724aca
script rather than 'configure'.  This re-generates 'configure', enables
Packit Service 724aca
configuration dependency rules, and enables re-generation of automatically
Packit Service 724aca
generated source files.
Packit Service 724aca
Packit Service 724aca
The build system supports using an object directory separate from the source
Packit Service 724aca
tree.  For example, you can create an 'obj' directory, and from within that
Packit Service 724aca
directory, issue configuration and build commands:
Packit Service 724aca
Packit Service 724aca
    autoconf
Packit Service 724aca
    mkdir obj
Packit Service 724aca
    cd obj
Packit Service 724aca
    ../configure --enable-autogen
Packit Service 724aca
    make
Packit Service 724aca
Packit Service 724aca
Packit Service 724aca
## Documentation
Packit Service 724aca
Packit Service 724aca
The manual page is generated in both html and roff formats.  Any web browser
Packit Service 724aca
can be used to view the html manual.  The roff manual page can be formatted
Packit Service 724aca
prior to installation via the following command:
Packit Service 724aca
Packit Service 724aca
    nroff -man -t doc/jemalloc.3