Blame jemalloc/INSTALL.md

Packit 345191
Building and installing a packaged release of jemalloc can be as simple as
Packit 345191
typing the following while in the root directory of the source tree:
Packit 345191
Packit 345191
    ./configure
Packit 345191
    make
Packit 345191
    make install
Packit 345191
Packit 345191
If building from unpackaged developer sources, the simplest command sequence
Packit 345191
that might work is:
Packit 345191
Packit 345191
    ./autogen.sh
Packit 345191
    make dist
Packit 345191
    make
Packit 345191
    make install
Packit 345191
Packit 345191
Note that documentation is not built by the default target because doing so
Packit 345191
would create a dependency on xsltproc in packaged releases, hence the
Packit 345191
requirement to either run 'make dist' or avoid installing docs via the various
Packit 345191
install_* targets documented below.
Packit 345191
Packit 345191
Packit 345191
## Advanced configuration
Packit 345191
Packit 345191
The 'configure' script supports numerous options that allow control of which
Packit 345191
functionality is enabled, where jemalloc is installed, etc.  Optionally, pass
Packit 345191
any of the following arguments (not a definitive list) to 'configure':
Packit 345191
Packit 345191
* `--help`
Packit 345191
Packit 345191
    Print a definitive list of options.
Packit 345191
Packit 345191
* `--prefix=<install-root-dir>`
Packit 345191
Packit 345191
    Set the base directory in which to install.  For example:
Packit 345191
Packit 345191
        ./configure --prefix=/usr/local
Packit 345191
Packit 345191
    will cause files to be installed into /usr/local/include, /usr/local/lib,
Packit 345191
    and /usr/local/man.
Packit 345191
Packit 345191
* `--with-version=(<major>.<minor>.<bugfix>-<nrev>-g<gid>|VERSION)`
Packit 345191
Packit 345191
    The VERSION file is mandatory for successful configuration, and the
Packit 345191
    following steps are taken to assure its presence:
Packit 345191
    1) If --with-version=<major>.<minor>.<bugfix>-<nrev>-g<gid> is specified,
Packit 345191
       generate VERSION using the specified value.
Packit 345191
    2) If --with-version is not specified in either form and the source
Packit 345191
       directory is inside a git repository, try to generate VERSION via 'git
Packit 345191
       describe' invocations that pattern-match release tags.
Packit 345191
    3) If VERSION is missing, generate it with a bogus version:
Packit 345191
       0.0.0-0-g0000000000000000000000000000000000000000
Packit 345191
Packit 345191
    Note that --with-version=VERSION bypasses (1) and (2), which simplifies
Packit 345191
    VERSION configuration when embedding a jemalloc release into another
Packit 345191
    project's git repository.
Packit 345191
Packit 345191
* `--with-rpath=<colon-separated-rpath>`
Packit 345191
Packit 345191
    Embed one or more library paths, so that libjemalloc can find the libraries
Packit 345191
    it is linked to.  This works only on ELF-based systems.
Packit 345191
Packit 345191
* `--with-mangling=<map>`
Packit 345191
Packit 345191
    Mangle public symbols specified in <map> which is a comma-separated list of
Packit 345191
    name:mangled pairs.
Packit 345191
Packit 345191
    For example, to use ld's --wrap option as an alternative method for
Packit 345191
    overriding libc's malloc implementation, specify something like:
Packit 345191
Packit 345191
      --with-mangling=malloc:__wrap_malloc,free:__wrap_free[...]
Packit 345191
Packit 345191
    Note that mangling happens prior to application of the prefix specified by
Packit 345191
    --with-jemalloc-prefix, and mangled symbols are then ignored when applying
Packit 345191
    the prefix.
Packit 345191
Packit 345191
* `--with-jemalloc-prefix=<prefix>`
Packit 345191
Packit 345191
    Prefix all public APIs with <prefix>.  For example, if <prefix> is
Packit 345191
    "prefix_", API changes like the following occur:
Packit 345191
Packit 345191
      malloc()         --> prefix_malloc()
Packit 345191
      malloc_conf      --> prefix_malloc_conf
Packit 345191
      /etc/malloc.conf --> /etc/prefix_malloc.conf
Packit 345191
      MALLOC_CONF      --> PREFIX_MALLOC_CONF
Packit 345191
Packit 345191
    This makes it possible to use jemalloc at the same time as the system
Packit 345191
    allocator, or even to use multiple copies of jemalloc simultaneously.
Packit 345191
Packit 345191
    By default, the prefix is "", except on OS X, where it is "je_".  On OS X,
Packit 345191
    jemalloc overlays the default malloc zone, but makes no attempt to actually
Packit 345191
    replace the "malloc", "calloc", etc. symbols.
Packit 345191
Packit 345191
* `--without-export`
Packit 345191
Packit 345191
    Don't export public APIs.  This can be useful when building jemalloc as a
Packit 345191
    static library, or to avoid exporting public APIs when using the zone
Packit 345191
    allocator on OSX.
Packit 345191
Packit 345191
* `--with-private-namespace=<prefix>`
Packit 345191
Packit 345191
    Prefix all library-private APIs with <prefix>je_.  For shared libraries,
Packit 345191
    symbol visibility mechanisms prevent these symbols from being exported, but
Packit 345191
    for static libraries, naming collisions are a real possibility.  By
Packit 345191
    default, <prefix> is empty, which results in a symbol prefix of je_ .
Packit 345191
Packit 345191
* `--with-install-suffix=<suffix>`
Packit 345191
Packit 345191
    Append <suffix> to the base name of all installed files, such that multiple
Packit 345191
    versions of jemalloc can coexist in the same installation directory.  For
Packit 345191
    example, libjemalloc.so.0 becomes libjemalloc<suffix>.so.0.
Packit 345191
Packit 345191
* `--with-malloc-conf=<malloc_conf>`
Packit 345191
Packit 345191
    Embed `<malloc_conf>` as a run-time options string that is processed prior to
Packit 345191
    the malloc_conf global variable, the /etc/malloc.conf symlink, and the
Packit 345191
    MALLOC_CONF environment variable.  For example, to change the default decay
Packit 345191
    time to 30 seconds:
Packit 345191
Packit 345191
      --with-malloc-conf=decay_ms:30000
Packit 345191
Packit 345191
* `--enable-debug`
Packit 345191
Packit 345191
    Enable assertions and validation code.  This incurs a substantial
Packit 345191
    performance hit, but is very useful during application development.
Packit 345191
Packit 345191
* `--disable-stats`
Packit 345191
Packit 345191
    Disable statistics gathering functionality.  See the "opt.stats_print"
Packit 345191
    option documentation for usage details.
Packit 345191
Packit 345191
* `--enable-prof`
Packit 345191
Packit 345191
    Enable heap profiling and leak detection functionality.  See the "opt.prof"
Packit 345191
    option documentation for usage details.  When enabled, there are several
Packit 345191
    approaches to backtracing, and the configure script chooses the first one
Packit 345191
    in the following list that appears to function correctly:
Packit 345191
Packit 345191
    + libunwind      (requires --enable-prof-libunwind)
Packit 345191
    + libgcc         (unless --disable-prof-libgcc)
Packit 345191
    + gcc intrinsics (unless --disable-prof-gcc)
Packit 345191
Packit 345191
* `--enable-prof-libunwind`
Packit 345191
Packit 345191
    Use the libunwind library (http://www.nongnu.org/libunwind/) for stack
Packit 345191
    backtracing.
Packit 345191
Packit 345191
* `--disable-prof-libgcc`
Packit 345191
Packit 345191
    Disable the use of libgcc's backtracing functionality.
Packit 345191
Packit 345191
* `--disable-prof-gcc`
Packit 345191
Packit 345191
    Disable the use of gcc intrinsics for backtracing.
Packit 345191
Packit 345191
* `--with-static-libunwind=<libunwind.a>`
Packit 345191
Packit 345191
    Statically link against the specified libunwind.a rather than dynamically
Packit 345191
    linking with -lunwind.
Packit 345191
Packit 345191
* `--disable-fill`
Packit 345191
Packit 345191
    Disable support for junk/zero filling of memory.  See the "opt.junk" and
Packit 345191
    "opt.zero" option documentation for usage details.
Packit 345191
Packit 345191
* `--disable-zone-allocator`
Packit 345191
Packit 345191
    Disable zone allocator for Darwin.  This means jemalloc won't be hooked as
Packit 345191
    the default allocator on OSX/iOS.
Packit 345191
Packit 345191
* `--enable-utrace`
Packit 345191
Packit 345191
    Enable utrace(2)-based allocation tracing.  This feature is not broadly
Packit 345191
    portable (FreeBSD has it, but Linux and OS X do not).
Packit 345191
Packit 345191
* `--enable-xmalloc`
Packit 345191
Packit 345191
    Enable support for optional immediate termination due to out-of-memory
Packit 345191
    errors, as is commonly implemented by "xmalloc" wrapper function for malloc.
Packit 345191
    See the "opt.xmalloc" option documentation for usage details.
Packit 345191
Packit 345191
* `--enable-lazy-lock`
Packit 345191
Packit 345191
    Enable code that wraps pthread_create() to detect when an application
Packit 345191
    switches from single-threaded to multi-threaded mode, so that it can avoid
Packit 345191
    mutex locking/unlocking operations while in single-threaded mode.  In
Packit 345191
    practice, this feature usually has little impact on performance unless
Packit 345191
    thread-specific caching is disabled.
Packit 345191
Packit 345191
* `--disable-cache-oblivious`
Packit 345191
Packit 345191
    Disable cache-oblivious large allocation alignment for large allocation
Packit 345191
    requests with no alignment constraints.  If this feature is disabled, all
Packit 345191
    large allocations are page-aligned as an implementation artifact, which can
Packit 345191
    severely harm CPU cache utilization.  However, the cache-oblivious layout
Packit 345191
    comes at the cost of one extra page per large allocation, which in the
Packit 345191
    most extreme case increases physical memory usage for the 16 KiB size class
Packit 345191
    to 20 KiB.
Packit 345191
Packit 345191
* `--disable-syscall`
Packit 345191
Packit 345191
    Disable use of syscall(2) rather than {open,read,write,close}(2).  This is
Packit 345191
    intended as a workaround for systems that place security limitations on
Packit 345191
    syscall(2).
Packit 345191
Packit 345191
* `--disable-cxx`
Packit 345191
Packit 345191
    Disable C++ integration.  This will cause new and delete operator
Packit 345191
    implementations to be omitted.
Packit 345191
Packit 345191
* `--with-xslroot=<path>`
Packit 345191
Packit 345191
    Specify where to find DocBook XSL stylesheets when building the
Packit 345191
    documentation.
Packit 345191
Packit 345191
* `--with-lg-page=<lg-page>`
Packit 345191
Packit 345191
    Specify the base 2 log of the allocator page size, which must in turn be at
Packit 345191
    least as large as the system page size.  By default the configure script
Packit 345191
    determines the host's page size and sets the allocator page size equal to
Packit 345191
    the system page size, so this option need not be specified unless the
Packit 345191
    system page size may change between configuration and execution, e.g. when
Packit 345191
    cross compiling.
Packit 345191
Packit 345191
* `--with-lg-hugepage=<lg-hugepage>`
Packit 345191
Packit 345191
    Specify the base 2 log of the system huge page size.  This option is useful
Packit 345191
    when cross compiling, or when overriding the default for systems that do
Packit 345191
    not explicitly support huge pages.
Packit 345191
Packit 345191
* `--with-lg-quantum=<lg-quantum>`
Packit 345191
Packit 345191
    Specify the base 2 log of the minimum allocation alignment.  jemalloc needs
Packit 345191
    to know the minimum alignment that meets the following C standard
Packit 345191
    requirement (quoted from the April 12, 2011 draft of the C11 standard):
Packit 345191
Packit 345191
    >  The pointer returned if the allocation succeeds is suitably aligned so
Packit 345191
      that it may be assigned to a pointer to any type of object with a
Packit 345191
      fundamental alignment requirement and then used to access such an object
Packit 345191
      or an array of such objects in the space allocated [...]
Packit 345191
Packit 345191
    This setting is architecture-specific, and although jemalloc includes known
Packit 345191
    safe values for the most commonly used modern architectures, there is a
Packit 345191
    wrinkle related to GNU libc (glibc) that may impact your choice of
Packit 345191
    <lg-quantum>.  On most modern architectures, this mandates 16-byte
Packit 345191
    alignment (<lg-quantum>=4), but the glibc developers chose not to meet this
Packit 345191
    requirement for performance reasons.  An old discussion can be found at
Packit 345191
    <https://sourceware.org/bugzilla/show_bug.cgi?id=206> .  Unlike glibc,
Packit 345191
    jemalloc does follow the C standard by default (caveat: jemalloc
Packit 345191
    technically cheats for size classes smaller than the quantum), but the fact
Packit 345191
    that Linux systems already work around this allocator noncompliance means
Packit 345191
    that it is generally safe in practice to let jemalloc's minimum alignment
Packit 345191
    follow glibc's lead.  If you specify `--with-lg-quantum=3` during
Packit 345191
    configuration, jemalloc will provide additional size classes that are not
Packit 345191
    16-byte-aligned (24, 40, and 56).
Packit 345191
Packit 345191
* `--with-lg-vaddr=<lg-vaddr>`
Packit 345191
Packit 345191
    Specify the number of significant virtual address bits.  By default, the
Packit 345191
    configure script attempts to detect virtual address size on those platforms
Packit 345191
    where it knows how, and picks a default otherwise.  This option may be
Packit 345191
    useful when cross-compiling.
Packit 345191
Packit 345191
* `--disable-initial-exec-tls`
Packit 345191
Packit 345191
    Disable the initial-exec TLS model for jemalloc's internal thread-local
Packit 345191
    storage (on those platforms that support explicit settings).  This can allow
Packit 345191
    jemalloc to be dynamically loaded after program startup (e.g. using dlopen).
Packit 345191
    Note that in this case, there will be two malloc implementations operating
Packit 345191
    in the same process, which will almost certainly result in confusing runtime
Packit 345191
    crashes if pointers leak from one implementation to the other.
Packit 345191
Packit 345191
* `--disable-libdl`
Packit 345191
Packit 345191
    Disable the usage of libdl, namely dlsym(3) which is required by the lazy
Packit 345191
    lock option.  This can allow building static binaries.
Packit 345191
Packit 345191
The following environment variables (not a definitive list) impact configure's
Packit 345191
behavior:
Packit 345191
Packit 345191
* `CFLAGS="?"`
Packit 345191
* `CXXFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags to the C/C++ compiler.  Any flags set by the configure
Packit 345191
    script are prepended, which means explicitly set flags generally take
Packit 345191
    precedence.  Take care when specifying flags such as -Werror, because
Packit 345191
    configure tests may be affected in undesirable ways.
Packit 345191
Packit 345191
* `EXTRA_CFLAGS="?"`
Packit 345191
* `EXTRA_CXXFLAGS="?"`
Packit 345191
Packit 345191
    Append these flags to CFLAGS/CXXFLAGS, without passing them to the
Packit 345191
    compiler(s) during configuration.  This makes it possible to add flags such
Packit 345191
    as -Werror, while allowing the configure script to determine what other
Packit 345191
    flags are appropriate for the specified configuration.
Packit 345191
Packit 345191
* `CPPFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags to the C preprocessor.  Note that CFLAGS is not passed to
Packit 345191
    'cpp' when 'configure' is looking for include files, so you must use
Packit 345191
    CPPFLAGS instead if you need to help 'configure' find header files.
Packit 345191
Packit 345191
* `LD_LIBRARY_PATH="?"`
Packit 345191
Packit 345191
    'ld' uses this colon-separated list to find libraries.
Packit 345191
Packit 345191
* `LDFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags when linking.
Packit 345191
Packit 345191
* `PATH="?"`
Packit 345191
Packit 345191
    'configure' uses this to find programs.
Packit 345191
Packit 345191
In some cases it may be necessary to work around configuration results that do
Packit 345191
not match reality.  For example, Linux 4.5 added support for the MADV_FREE flag
Packit 345191
to madvise(2), which can cause problems if building on a host with MADV_FREE
Packit 345191
support and deploying to a target without.  To work around this, use a cache
Packit 345191
file to override the relevant configuration variable defined in configure.ac,
Packit 345191
e.g.:
Packit 345191
Packit 345191
    echo "je_cv_madv_free=no" > config.cache && ./configure -C
Packit 345191
Packit 345191
Packit 345191
## Advanced compilation
Packit 345191
Packit 345191
To build only parts of jemalloc, use the following targets:
Packit 345191
Packit 345191
    build_lib_shared
Packit 345191
    build_lib_static
Packit 345191
    build_lib
Packit 345191
    build_doc_html
Packit 345191
    build_doc_man
Packit 345191
    build_doc
Packit 345191
Packit 345191
To install only parts of jemalloc, use the following targets:
Packit 345191
Packit 345191
    install_bin
Packit 345191
    install_include
Packit 345191
    install_lib_shared
Packit 345191
    install_lib_static
Packit 345191
    install_lib_pc
Packit 345191
    install_lib
Packit 345191
    install_doc_html
Packit 345191
    install_doc_man
Packit 345191
    install_doc
Packit 345191
Packit 345191
To clean up build results to varying degrees, use the following make targets:
Packit 345191
Packit 345191
    clean
Packit 345191
    distclean
Packit 345191
    relclean
Packit 345191
Packit 345191
Packit 345191
## Advanced installation
Packit 345191
Packit 345191
Optionally, define make variables when invoking make, including (not
Packit 345191
exclusively):
Packit 345191
Packit 345191
* `INCLUDEDIR="?"`
Packit 345191
Packit 345191
    Use this as the installation prefix for header files.
Packit 345191
Packit 345191
* `LIBDIR="?"`
Packit 345191
Packit 345191
    Use this as the installation prefix for libraries.
Packit 345191
Packit 345191
* `MANDIR="?"`
Packit 345191
Packit 345191
    Use this as the installation prefix for man pages.
Packit 345191
Packit 345191
* `DESTDIR="?"`
Packit 345191
Packit 345191
    Prepend DESTDIR to INCLUDEDIR, LIBDIR, DATADIR, and MANDIR.  This is useful
Packit 345191
    when installing to a different path than was specified via --prefix.
Packit 345191
Packit 345191
* `CC="?"`
Packit 345191
Packit 345191
    Use this to invoke the C compiler.
Packit 345191
Packit 345191
* `CFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags to the compiler.
Packit 345191
Packit 345191
* `CPPFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags to the C preprocessor.
Packit 345191
Packit 345191
* `LDFLAGS="?"`
Packit 345191
Packit 345191
    Pass these flags when linking.
Packit 345191
Packit 345191
* `PATH="?"`
Packit 345191
Packit 345191
    Use this to search for programs used during configuration and building.
Packit 345191
Packit 345191
Packit 345191
## Development
Packit 345191
Packit 345191
If you intend to make non-trivial changes to jemalloc, use the 'autogen.sh'
Packit 345191
script rather than 'configure'.  This re-generates 'configure', enables
Packit 345191
configuration dependency rules, and enables re-generation of automatically
Packit 345191
generated source files.
Packit 345191
Packit 345191
The build system supports using an object directory separate from the source
Packit 345191
tree.  For example, you can create an 'obj' directory, and from within that
Packit 345191
directory, issue configuration and build commands:
Packit 345191
Packit 345191
    autoconf
Packit 345191
    mkdir obj
Packit 345191
    cd obj
Packit 345191
    ../configure --enable-autogen
Packit 345191
    make
Packit 345191
Packit 345191
Packit 345191
## Documentation
Packit 345191
Packit 345191
The manual page is generated in both html and roff formats.  Any web browser
Packit 345191
can be used to view the html manual.  The roff manual page can be formatted
Packit 345191
prior to installation via the following command:
Packit 345191
Packit 345191
    nroff -man -t doc/jemalloc.3