Blame NON-AUTOTOOLS-BUILD

Packit 504f36
Building PCRE2 without using autotools
Packit 504f36
--------------------------------------
Packit 504f36
Packit 504f36
This document contains the following sections:
Packit 504f36
Packit 504f36
  General
Packit 504f36
  Generic instructions for the PCRE2 C library
Packit 504f36
  Stack size in Windows environments
Packit 504f36
  Linking programs in Windows environments
Packit 504f36
  Calling conventions in Windows environments
Packit 504f36
  Comments about Win32 builds
Packit 504f36
  Building PCRE2 on Windows with CMake
Packit 504f36
  Building PCRE2 on Windows with Visual Studio
Packit 504f36
  Testing with RunTest.bat
Packit 504f36
  Building PCRE2 on native z/OS and z/VM
Packit 504f36
Packit 504f36
Packit 504f36
GENERAL
Packit 504f36
Packit 504f36
The basic PCRE2 library consists entirely of code written in Standard C, and so
Packit 504f36
should compile successfully on any system that has a Standard C compiler and
Packit 504f36
library.
Packit 504f36
Packit 504f36
The PCRE2 distribution includes a "configure" file for use by the
Packit 504f36
configure/make (autotools) build system, as found in many Unix-like
Packit 504f36
environments. The README file contains information about the options for
Packit 504f36
"configure".
Packit 504f36
Packit 504f36
There is also support for CMake, which some users prefer, especially in Windows
Packit 504f36
environments, though it can also be run in Unix-like environments. See the
Packit 504f36
section entitled "Building PCRE2 on Windows with CMake" below.
Packit 504f36
Packit 504f36
Versions of src/config.h and src/pcre2.h are distributed in the PCRE2 tarballs
Packit 504f36
under the names src/config.h.generic and src/pcre2.h.generic. These are
Packit 504f36
provided for those who build PCRE2 without using "configure" or CMake. If you
Packit 504f36
use "configure" or CMake, the .generic versions are not used.
Packit 504f36
Packit 504f36
Packit 504f36
GENERIC INSTRUCTIONS FOR THE PCRE2 C LIBRARY
Packit 504f36
Packit 504f36
The following are generic instructions for building the PCRE2 C library "by
Packit 504f36
hand". If you are going to use CMake, this section does not apply to you; you
Packit 504f36
can skip ahead to the CMake section.
Packit 504f36
Packit 504f36
 (1) Copy or rename the file src/config.h.generic as src/config.h, and edit the
Packit 504f36
     macro settings that it contains to whatever is appropriate for your
Packit 504f36
     environment. In particular, you can alter the definition of the NEWLINE
Packit 504f36
     macro to specify what character(s) you want to be interpreted as line
Packit 504f36
     terminators by default.
Packit 504f36
Packit 504f36
     When you compile any of the PCRE2 modules, you must specify
Packit 504f36
     -DHAVE_CONFIG_H to your compiler so that src/config.h is included in the
Packit 504f36
     sources.
Packit 504f36
Packit 504f36
     An alternative approach is not to edit src/config.h, but to use -D on the
Packit 504f36
     compiler command line to make any changes that you need to the
Packit 504f36
     configuration options. In this case -DHAVE_CONFIG_H must not be set.
Packit 504f36
Packit 504f36
     NOTE: There have been occasions when the way in which certain parameters
Packit 504f36
     in src/config.h are used has changed between releases. (In the
Packit 504f36
     configure/make world, this is handled automatically.) When upgrading to a
Packit 504f36
     new release, you are strongly advised to review src/config.h.generic
Packit 504f36
     before re-using what you had previously.
Packit 504f36
Packit 504f36
 (2) Copy or rename the file src/pcre2.h.generic as src/pcre2.h.
Packit 504f36
Packit 504f36
 (3) EITHER:
Packit 504f36
       Copy or rename file src/pcre2_chartables.c.dist as
Packit 504f36
       src/pcre2_chartables.c.
Packit 504f36
Packit 504f36
     OR:
Packit 504f36
       Compile src/dftables.c as a stand-alone program (using -DHAVE_CONFIG_H
Packit 504f36
       if you have set up src/config.h), and then run it with the single
Packit 504f36
       argument "src/pcre2_chartables.c". This generates a set of standard
Packit 504f36
       character tables and writes them to that file. The tables are generated
Packit 504f36
       using the default C locale for your system. If you want to use a locale
Packit 504f36
       that is specified by LC_xxx environment variables, add the -L option to
Packit 504f36
       the dftables command. You must use this method if you are building on a
Packit 504f36
       system that uses EBCDIC code.
Packit 504f36
Packit 504f36
     The tables in src/pcre2_chartables.c are defaults. The caller of PCRE2 can
Packit 504f36
     specify alternative tables at run time.
Packit 504f36
Packit 504f36
 (4) For an 8-bit library, compile the following source files from the src
Packit 504f36
     directory, setting -DPCRE2_CODE_UNIT_WIDTH=8 as a compiler option. Also
Packit 504f36
     set -DHAVE_CONFIG_H if you have set up src/config.h with your
Packit 504f36
     configuration, or else use other -D settings to change the configuration
Packit 504f36
     as required.
Packit 504f36
Packit 504f36
       pcre2_auto_possess.c
Packit 504f36
       pcre2_chartables.c
Packit 504f36
       pcre2_compile.c
Packit 504f36
       pcre2_config.c
Packit 504f36
       pcre2_context.c
Packit 504f36
       pcre2_convert.c
Packit 504f36
       pcre2_dfa_match.c
Packit 504f36
       pcre2_error.c
Packit 504f36
       pcre2_extuni.c
Packit 504f36
       pcre2_find_bracket.c
Packit 504f36
       pcre2_jit_compile.c
Packit 504f36
       pcre2_maketables.c
Packit 504f36
       pcre2_match.c
Packit 504f36
       pcre2_match_data.c
Packit 504f36
       pcre2_newline.c
Packit 504f36
       pcre2_ord2utf.c
Packit 504f36
       pcre2_pattern_info.c
Packit 504f36
       pcre2_serialize.c
Packit 504f36
       pcre2_string_utils.c
Packit 504f36
       pcre2_study.c
Packit 504f36
       pcre2_substitute.c
Packit 504f36
       pcre2_substring.c
Packit 504f36
       pcre2_tables.c
Packit 504f36
       pcre2_ucd.c
Packit 504f36
       pcre2_valid_utf.c
Packit 504f36
       pcre2_xclass.c
Packit 504f36
Packit 504f36
     Make sure that you include -I. in the compiler command (or equivalent for
Packit 504f36
     an unusual compiler) so that all included PCRE2 header files are first
Packit 504f36
     sought in the src directory under the current directory. Otherwise you run
Packit 504f36
     the risk of picking up a previously-installed file from somewhere else.
Packit 504f36
Packit 504f36
     Note that you must compile pcre2_jit_compile.c, even if you have not
Packit 504f36
     defined SUPPORT_JIT in src/config.h, because when JIT support is not
Packit 504f36
     configured, dummy functions are compiled. When JIT support IS configured,
Packit 504f36
     pcre2_jit_compile.c #includes other files from the sljit subdirectory,
Packit 504f36
     all of whose names begin with "sljit". It also #includes
Packit 504f36
     src/pcre2_jit_match.c and src/pcre2_jit_misc.c, so you should not compile
Packit 504f36
     these yourself.
Packit 504f36
Packit 504f36
     Note also that the pcre2_fuzzsupport.c file contains special code that is
Packit 504f36
     useful to those who want to run fuzzing tests on the PCRE2 library. Unless
Packit 504f36
     you are doing that, you can ignore it.
Packit 504f36
Packit 504f36
 (5) Now link all the compiled code into an object library in whichever form
Packit 504f36
     your system keeps such libraries. This is the basic PCRE2 C 8-bit library.
Packit 504f36
     If your system has static and shared libraries, you may have to do this
Packit 504f36
     once for each type.
Packit 504f36
Packit 504f36
 (6) If you want to build a 16-bit library or 32-bit library (as well as, or
Packit 504f36
     instead of the 8-bit library) just supply 16 or 32 as the value of
Packit 504f36
     -DPCRE2_CODE_UNIT_WIDTH when you are compiling.
Packit 504f36
Packit 504f36
 (7) If you want to build the POSIX wrapper functions (which apply only to the
Packit 504f36
     8-bit library), ensure that you have the src/pcre2posix.h file and then
Packit 504f36
     compile src/pcre2posix.c. Link the result (on its own) as the pcre2posix
Packit 504f36
     library.
Packit 504f36
Packit 504f36
 (8) The pcre2test program can be linked with any combination of the 8-bit,
Packit 504f36
     16-bit and 32-bit libraries (depending on what you selected in
Packit 504f36
     src/config.h). Compile src/pcre2test.c; don't forget -DHAVE_CONFIG_H if
Packit 504f36
     necessary, but do NOT define PCRE2_CODE_UNIT_WIDTH. Then link with the
Packit 504f36
     appropriate library/ies. If you compiled an 8-bit library, pcre2test also
Packit 504f36
     needs the pcre2posix wrapper library.
Packit 504f36
Packit 504f36
 (9) Run pcre2test on the testinput files in the testdata directory, and check
Packit 504f36
     that the output matches the corresponding testoutput files. There are
Packit 504f36
     comments about what each test does in the section entitled "Testing PCRE2"
Packit 504f36
     in the README file. If you compiled more than one of the 8-bit, 16-bit and
Packit 504f36
     32-bit libraries, you need to run pcre2test with the -16 option to do
Packit 504f36
     16-bit tests and with the -32 option to do 32-bit tests.
Packit 504f36
Packit 504f36
     Some tests are relevant only when certain build-time options are selected.
Packit 504f36
     For example, test 4 is for Unicode support, and will not run if you have
Packit 504f36
     built PCRE2 without it. See the comments at the start of each testinput
Packit 504f36
     file. If you have a suitable Unix-like shell, the RunTest script will run
Packit 504f36
     the appropriate tests for you. The command "RunTest list" will output a
Packit 504f36
     list of all the tests.
Packit 504f36
Packit 504f36
     Note that the supplied files are in Unix format, with just LF characters
Packit 504f36
     as line terminators. You may need to edit them to change this if your
Packit 504f36
     system uses a different convention.
Packit 504f36
Packit 504f36
(10) If you have built PCRE2 with SUPPORT_JIT, the JIT features can be tested
Packit 504f36
     by running pcre2test with the -jit option. This is done automatically by
Packit 504f36
     the RunTest script. You might also like to build and run the freestanding
Packit 504f36
     JIT test program, src/pcre2_jit_test.c.
Packit 504f36
Packit 504f36
(11) If you want to use the pcre2grep command, compile and link
Packit 504f36
     src/pcre2grep.c; it uses only the basic 8-bit PCRE2 library (it does not
Packit 504f36
     need the pcre2posix library). If you have built the PCRE2 library with JIT
Packit 504f36
     support by defining SUPPORT_JIT in src/config.h, you can also define
Packit 504f36
     SUPPORT_PCRE2GREP_JIT, which causes pcre2grep to make use of JIT (unless
Packit 504f36
     it is run with --no-jit). If you define SUPPORT_PCRE2GREP_JIT without
Packit 504f36
     defining SUPPORT_JIT, pcre2grep does not try to make use of JIT.
Packit 504f36
Packit 504f36
Packit 504f36
STACK SIZE IN WINDOWS ENVIRONMENTS
Packit 504f36
Packit 504f36
Prior to release 10.30 the default system stack size of 1MiB in some Windows
Packit 504f36
environments caused issues with some tests. This should no longer be the case
Packit 504f36
for 10.30 and later releases.
Packit 504f36
Packit 504f36
Packit 504f36
LINKING PROGRAMS IN WINDOWS ENVIRONMENTS
Packit 504f36
Packit 504f36
If you want to statically link a program against a PCRE2 library in the form of
Packit 504f36
a non-dll .a file, you must define PCRE2_STATIC before including src/pcre2.h.
Packit 504f36
Packit 504f36
Packit 504f36
CALLING CONVENTIONS IN WINDOWS ENVIRONMENTS
Packit 504f36
Packit 504f36
It is possible to compile programs to use different calling conventions using
Packit 504f36
MSVC. Search the web for "calling conventions" for more information. To make it
Packit 504f36
easier to change the calling convention for the exported functions in the
Packit 504f36
PCRE2 library, the macro PCRE2_CALL_CONVENTION is present in all the external
Packit 504f36
definitions. It can be set externally when compiling (e.g. in CFLAGS). If it is
Packit 504f36
not set, it defaults to empty; the default calling convention is then used
Packit 504f36
(which is what is wanted most of the time).
Packit 504f36
Packit 504f36
Packit 504f36
COMMENTS ABOUT WIN32 BUILDS (see also "BUILDING PCRE2 ON WINDOWS WITH CMAKE")
Packit 504f36
Packit 504f36
There are two ways of building PCRE2 using the "configure, make, make install"
Packit 504f36
paradigm on Windows systems: using MinGW or using Cygwin. These are not at all
Packit 504f36
the same thing; they are completely different from each other. There is also
Packit 504f36
support for building using CMake, which some users find a more straightforward
Packit 504f36
way of building PCRE2 under Windows.
Packit 504f36
Packit 504f36
The MinGW home page (http://www.mingw.org/) says this:
Packit 504f36
Packit 504f36
  MinGW: A collection of freely available and freely distributable Windows
Packit 504f36
  specific header files and import libraries combined with GNU toolsets that
Packit 504f36
  allow one to produce native Windows programs that do not rely on any
Packit 504f36
  3rd-party C runtime DLLs.
Packit 504f36
Packit 504f36
The Cygwin home page (http://www.cygwin.com/) says this:
Packit 504f36
Packit 504f36
  Cygwin is a Linux-like environment for Windows. It consists of two parts:
Packit 504f36
Packit 504f36
  . A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing
Packit 504f36
    substantial Linux API functionality
Packit 504f36
Packit 504f36
  . A collection of tools which provide Linux look and feel.
Packit 504f36
Packit 504f36
On both MinGW and Cygwin, PCRE2 should build correctly using:
Packit 504f36
Packit 504f36
  ./configure && make && make install
Packit 504f36
Packit 504f36
This should create two libraries called libpcre2-8 and libpcre2-posix. These
Packit 504f36
are independent libraries: when you link with libpcre2-posix you must also link
Packit 504f36
with libpcre2-8, which contains the basic functions.
Packit 504f36
Packit 504f36
Using Cygwin's compiler generates libraries and executables that depend on
Packit 504f36
cygwin1.dll. If a library that is generated this way is distributed,
Packit 504f36
cygwin1.dll has to be distributed as well. Since cygwin1.dll is under the GPL
Packit 504f36
licence, this forces not only PCRE2 to be under the GPL, but also the entire
Packit 504f36
application. A distributor who wants to keep their own code proprietary must
Packit 504f36
purchase an appropriate Cygwin licence.
Packit 504f36
Packit 504f36
MinGW has no such restrictions. The MinGW compiler generates a library or
Packit 504f36
executable that can run standalone on Windows without any third party dll or
Packit 504f36
licensing issues.
Packit 504f36
Packit 504f36
But there is more complication:
Packit 504f36
Packit 504f36
If a Cygwin user uses the -mno-cygwin Cygwin gcc flag, what that really does is
Packit 504f36
to tell Cygwin's gcc to use the MinGW gcc. Cygwin's gcc is only acting as a
Packit 504f36
front end to MinGW's gcc (if you install Cygwin's gcc, you get both Cygwin's
Packit 504f36
gcc and MinGW's gcc). So, a user can:
Packit 504f36
Packit 504f36
. Build native binaries by using MinGW or by getting Cygwin and using
Packit 504f36
  -mno-cygwin.
Packit 504f36
Packit 504f36
. Build binaries that depend on cygwin1.dll by using Cygwin with the normal
Packit 504f36
  compiler flags.
Packit 504f36
Packit 504f36
The test files that are supplied with PCRE2 are in UNIX format, with LF
Packit 504f36
characters as line terminators. Unless your PCRE2 library uses a default
Packit 504f36
newline option that includes LF as a valid newline, it may be necessary to
Packit 504f36
change the line terminators in the test files to get some of the tests to work.
Packit 504f36
Packit 504f36
Packit 504f36
BUILDING PCRE2 ON WINDOWS WITH CMAKE
Packit 504f36
Packit 504f36
CMake is an alternative configuration facility that can be used instead of
Packit 504f36
"configure". CMake creates project files (make files, solution files, etc.)
Packit 504f36
tailored to numerous development environments, including Visual Studio,
Packit 504f36
Borland, Msys, MinGW, NMake, and Unix. If possible, use short paths with no
Packit 504f36
spaces in the names for your CMake installation and your PCRE2 source and build
Packit 504f36
directories.
Packit 504f36
Packit 504f36
The following instructions were contributed by a PCRE1 user, but they should
Packit 504f36
also work for PCRE2. If they are not followed exactly, errors may occur. In the
Packit 504f36
event that errors do occur, it is recommended that you delete the CMake cache
Packit 504f36
before attempting to repeat the CMake build process. In the CMake GUI, the
Packit 504f36
cache can be deleted by selecting "File > Delete Cache".
Packit 504f36
Packit 504f36
1.  Install the latest CMake version available from http://www.cmake.org/, and
Packit 504f36
    ensure that cmake\bin is on your path.
Packit 504f36
Packit 504f36
2.  Unzip (retaining folder structure) the PCRE2 source tree into a source
Packit 504f36
    directory such as C:\pcre2. You should ensure your local date and time
Packit 504f36
    is not earlier than the file dates in your source dir if the release is
Packit 504f36
    very new.
Packit 504f36
Packit 504f36
3.  Create a new, empty build directory, preferably a subdirectory of the
Packit 504f36
    source dir. For example, C:\pcre2\pcre2-xx\build.
Packit 504f36
Packit 504f36
4.  Run cmake-gui from the Shell envirornment of your build tool, for example,
Packit 504f36
    Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++. Do not try
Packit 504f36
    to start Cmake from the Windows Start menu, as this can lead to errors.
Packit 504f36
Packit 504f36
5.  Enter C:\pcre2\pcre2-xx and C:\pcre2\pcre2-xx\build for the source and
Packit 504f36
    build directories, respectively.
Packit 504f36
Packit 504f36
6.  Hit the "Configure" button.
Packit 504f36
Packit 504f36
7.  Select the particular IDE / build tool that you are using (Visual
Packit 504f36
    Studio, MSYS makefiles, MinGW makefiles, etc.)
Packit 504f36
Packit 504f36
8.  The GUI will then list several configuration options. This is where
Packit 504f36
    you can disable Unicode support or select other PCRE2 optional features.
Packit 504f36
Packit 504f36
9.  Hit "Configure" again. The adjacent "Generate" button should now be
Packit 504f36
    active.
Packit 504f36
Packit 504f36
10. Hit "Generate".
Packit 504f36
Packit 504f36
11. The build directory should now contain a usable build system, be it a
Packit 504f36
    solution file for Visual Studio, makefiles for MinGW, etc. Exit from
Packit 504f36
    cmake-gui and use the generated build system with your compiler or IDE.
Packit 504f36
    E.g., for MinGW you can run "make", or for Visual Studio, open the PCRE2
Packit 504f36
    solution, select the desired configuration (Debug, or Release, etc.) and
Packit 504f36
    build the ALL_BUILD project.
Packit 504f36
Packit 504f36
12. If during configuration with cmake-gui you've elected to build the test
Packit 504f36
    programs, you can execute them by building the test project. E.g., for
Packit 504f36
    MinGW: "make test"; for Visual Studio build the RUN_TESTS project. The
Packit 504f36
    most recent build configuration is targeted by the tests. A summary of
Packit 504f36
    test results is presented. Complete test output is subsequently
Packit 504f36
    available for review in Testing\Temporary under your build dir.
Packit 504f36
Packit 504f36
Packit 504f36
BUILDING PCRE2 ON WINDOWS WITH VISUAL STUDIO
Packit 504f36
Packit 504f36
The code currently cannot be compiled without a stdint.h header, which is
Packit 504f36
available only in relatively recent versions of Visual Studio. However, this
Packit 504f36
portable and permissively-licensed implementation of the header worked without
Packit 504f36
issue:
Packit 504f36
Packit 504f36
  http://www.azillionmonkeys.com/qed/pstdint.h
Packit 504f36
Packit 504f36
Just rename it and drop it into the top level of the build tree.
Packit 504f36
Packit 504f36
Packit 504f36
TESTING WITH RUNTEST.BAT
Packit 504f36
Packit 504f36
If configured with CMake, building the test project ("make test" or building
Packit 504f36
ALL_TESTS in Visual Studio) creates (and runs) pcre2_test.bat (and depending
Packit 504f36
on your configuration options, possibly other test programs) in the build
Packit 504f36
directory. The pcre2_test.bat script runs RunTest.bat with correct source and
Packit 504f36
exe paths.
Packit 504f36
Packit 504f36
For manual testing with RunTest.bat, provided the build dir is a subdirectory
Packit 504f36
of the source directory: Open command shell window. Chdir to the location
Packit 504f36
of your pcre2test.exe and pcre2grep.exe programs. Call RunTest.bat with
Packit 504f36
"..\RunTest.Bat" or "..\..\RunTest.bat" as appropriate.
Packit 504f36
Packit 504f36
To run only a particular test with RunTest.Bat provide a test number argument.
Packit 504f36
Packit 504f36
Otherwise:
Packit 504f36
Packit 504f36
1. Copy RunTest.bat into the directory where pcre2test.exe and pcre2grep.exe
Packit 504f36
   have been created.
Packit 504f36
Packit 504f36
2. Edit RunTest.bat to indentify the full or relative location of
Packit 504f36
   the pcre2 source (wherein which the testdata folder resides), e.g.:
Packit 504f36
Packit 504f36
   set srcdir=C:\pcre2\pcre2-10.00
Packit 504f36
Packit 504f36
3. In a Windows command environment, chdir to the location of your bat and
Packit 504f36
   exe programs.
Packit 504f36
Packit 504f36
4. Run RunTest.bat. Test outputs will automatically be compared to expected
Packit 504f36
   results, and discrepancies will be identified in the console output.
Packit 504f36
Packit 504f36
To independently test the just-in-time compiler, run pcre2_jit_test.exe.
Packit 504f36
Packit 504f36
Packit 504f36
BUILDING PCRE2 ON NATIVE Z/OS AND Z/VM
Packit 504f36
Packit 504f36
z/OS and z/VM are operating systems for mainframe computers, produced by IBM.
Packit 504f36
The character code used is EBCDIC, not ASCII or Unicode. In z/OS, UNIX APIs and
Packit 504f36
applications can be supported through UNIX System Services, and in such an
Packit 504f36
environment it should be possible to build PCRE2 in the same way as in other
Packit 504f36
systems, with the EBCDIC related configuration settings, but it is not known if
Packit 504f36
anybody has tried this.
Packit 504f36
Packit 504f36
In native z/OS (without UNIX System Services) and in z/VM, special ports are
Packit 504f36
required. For details, please see file 939 on this web site:
Packit 504f36
Packit 504f36
  http://www.cbttape.org
Packit 504f36
Packit 504f36
Everything in that location, source and executable, is in EBCDIC and native
Packit 504f36
z/OS file formats. The port provides an API for LE languages such as COBOL and
Packit 504f36
for the z/OS and z/VM versions of the Rexx languages.
Packit 504f36
Packit 504f36
===========================
Packit 504f36
Last Updated: 19 April 2018
Packit 504f36
===========================