# Building SWI-Prolog using cmake

As of version 7.7.20,  SWI-Prolog   ships  with `cmake` `CMakeLists.txt`
configuration files that cover the  entire   project.  Builds  have been
tested on Ubuntu 16.04,  18.04,  18.10,   Fedora  28,  MacOSX  and cross
compilation for Win32 as well  as  Win64   using  Ubuntu  18.04  as host
system.

The build has  been  tested  with   the  "Unix  Makefiles"  and  "Ninja"
generators.  We  use  [Ninja](https://ninja-build.org/)   as  it  builds
faster, avoids warning  from  being   cluttered  and  better facilitates
debugging dependency issues. It can be   selected  using `cmake -G Ninja
..`,  after which the usual `make` _target_   can be replaced by `ninja`
_target_. The examples below all  use  Ninja.   Drop  `-G  Ninja` to use
classical Unix make.

## Getting cmake

Building SWI-Prolog requires cmake version 3.5  or later (*). Many Linux
systems ship with a cmake package. On  MacOS we use the Macport version.
If the shipped cmake version is too old   you may wish to download cmake
from https://cmake.org/download/ On  Linux   systems,  installing  e.g.,
cmake 3.12 (please pick the latest stable version) is as simple as:

    wget https://cmake.org/files/v3.12/cmake-3.12.0-Linux-x86_64.sh
    sudo sh cmake-3.12.0-Linux-x86_64.sh --prefix=/usr/local --exclude-subdir

(*) Tested with 3.5 (Ubuntu Xenial), 3.10 (Ubuntu Bionic), 3.14 (MacOS).


## Native build

### Getting the source

The   source   may   be    downloaded    as     a    tar    ball    from
http://www.swi-prolog.org or downloaded using git.  The git sequence is:

    git clone https://github.com/SWI-Prolog/swipl-devel.git
    cd swipl-devel
    git submodule update --init

If not all modules are needed,  one can  clone/update particular ones as
follows:

    git submodule update --init packages/jpl packages/clib packages/sgml


### Building from source

The typical sequence to build SWI-Prolog  and install in `/usr/local` is
as follows:

    cd swipl-devel
    mkdir build
    cd build
    cmake -G Ninja ..
    ninja
    ctest -j 8
    ninja install

### Upgrading

In most cases the following should  update   an  installed system to the
latest version:

    git pull
    git submodule update --init
    cd build
    cmake ..
    ninja
    ctest -j 8
    ninja install

If this fails, one of these measures may be appropriate:

  1. run `ninja clean` before `ninja`
  2. remove the entire `build` directory and re-create it as above.
     Note that the build process makes no modifications outside the
     `build` directory.

## Build types

The default build type is `RelWithDebInfo`. Alternatives may be selected
using e.g.,

    cmake -DCMAKE_BUILD_TYPE=Debug -G Ninja ..
    cmake -DCMAKE_BUILD_TYPE=Release -G Ninja ..

## Install location

To install in a particular   location, use `-DCMAKE_INSTALL_PREFIX`. For
example, this will build SWI to be installed in  `/usr/local/swipl-git`:

    cmake -DCMAKE_INSTALL_PREFIX=/usr/local/swipl-git -G Ninja ..

After    `sudo    ninja    install`,     `swipl`      will     be     in
`/usr/local/swipl-git/bin/swipl`.


## Customizing SWI-Prolog

By default the system configures all   features. Several cmake _options_
allow for restricting the system.

  | Option                        | Description                           |
  | ----------------------------- | ------------------------------------- |
  | `-DMULTI_THREADED=OFF`        | Drop support for Prolog threads       |
  | `-DUSE_SIGNALS=OFF`           | Drop signal support                   |
  | `-DUSE_GMP=OFF`               | Drop bignum and rational numbers      |
  | `-DUSE_TCMALLOC=OFF`          | Do not link against `-ltcmalloc`      |
  | `-DSWIPL_SHARED_LIB=OFF`      | Build Prolog kernel as static lib     |
  | `-DSWIPL_INSTALL_IN_LIB=ON`   | Install libswipl.so in `<prefix>/lib` |
  | `-DSWIPL_INSTALL_IN_SHARE=ON` | Install docs in `<prefix>/share`      |
  | `-DSWIPL_M32=ON`              | Make 32-bit version on 64-bit Linux   |
  | `-DSWIPL_PACKAGES=OFF`        | Only build the core system            |
  | `-DSWIPL_PACKAGES_BASIC=OFF`  | Drop all basic packages               |
  | `-DSWIPL_PACKAGES_ODBC=OFF`   | Drop ODBC and CQL packages            |
  | `-DSWIPL_PACKAGES_JAVA=OFF`   | Drop JPL Java interface               |
  | `-DSWIPL_PACKAGES_X=OFF`      | Drop graphics (xpce)                  |
  | `-DBUILD_TESTING=OFF`         | Do not setup for ctest unit tests     |
  | `-DINSTALL_TESTS=ON`          | Add tests to installed system         |
  | `-DINSTALL_DOCUMENTATION=OFF` | Drop generating the HTML docs         |

Note that packages for  which  the   prerequisites  cannot  be found are
dropped automatically, as are packages  for   which  the sources are not
installed.

## Embedding SWI-Prolog in Java, C, C++, etc.

If SWI-Prolog is to be embedded in another executable it must be able to
find its home directory and the main   application  must be able to find
the SWI-Prolog shared library `libswipl.so`   (extension  depends on the
platform).  The following environment variables are commonly used:

- `SWI_HOME_DIR` should point at SWI-Prolog's main directory, e.g.
  ``${CMAKE_INSTALL_PREFIX}/lib/swipl``
- The shared object search path should include the directory where
  `libswipl.{so,dll,...}` resides.  The variable depends on the
  platform.  Some popular names:

  - `LD_LIBRARY_PATH` (ELF based systems such as Linux)
  - `DYLD_LIBRARY_PATH` (MacOS)
  - `PATH` (Windows)

If you build SWI-Prolog  you  must   __remove  these  variables from the
environment when building__. Failure  to  do   so  may  cause  the build
process to use parts  of  an   incompatible  installed  system.  Running
`cmake` warns if  such  an  environment   variable  is  found,  but  the
environment must be cleaned when running `ninja` or `make`.

## Profile Guided Optimization

When using Ninja and GCC, the system  may be built using _Profile Guided
Optimization_ (PGO). This  first  builds   the  system  instrumented  to
collect profile information, then runs  a   benchmark  suite and finally
recompiles it using the benchmark suite  output to help (notably) branch
prediction.  The  performance  improvement   is    about   10   to  20%.
Unfortunately the process is a little hard   and the entire system needs
to be recompiled on any change made to it. To build the PGO version, run
the commands below. The `../scripts/pgo-compile.sh`   performs the three
steps above using  the  core  system.   The  final  `ninja`  builds  the
packages and documentation.

    cmake -DCMAKE_BUILD_TYPE=Release -G Ninja ..
    ../scripts/pgo-compile.sh
    ninja

To stop using PGO builds in the current build directory run the commands
below. The `pgo-compile.sh --off`  restores   the  normal  build  flags,
deletes the PGO data and cleans the core system files.

    ../scripts/pgo-compile.sh --off
    ninja


## Cross build

Cross building for Windows using (Ubuntu)   linux. Use `README.mingw` to
download  and  build  the  dependencies.  Next,  set  these  environment
variables:

  - `MINGW64_ROOT` must point at the prefix where the dependencies
    are for 64-bit Windows.
  - `MINGW32_ROOT` must point at the prefix where the dependencies
    are for 32-bit Windows.

The cmake toolchain  config  files  (see   below)  search  for  Java  in
`$HOME/.wine`. Please check these locations.

### 64 bit Windows from Linux

    mkdir win64
    cd win64
    cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win64.cmake -G Ninja ..
    ninja

### 32 bit Windows from Linux

    mkdir win32
    cd win32
    cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win32.cmake -G Ninja ..

### WASM (Emscripten)

__Note__: due to a  bug  in   the  current  Emscripten  directory access
functions we need the _native friend_   mechanism  to create the library
index. The flags below   include `-DSWIPL_NATIVE_FRIEND=build`, assuming
you built a  native  executable  in   the  directory  `build`  below the
sources. Adjust as necessary.

    [Assumes native Prolog in `build`.  See note above]

    mkdir build.wasm
    cd build.wasm
    source ~/emsdk/emsdk_env.sh
    cmake -DCMAKE_TOOLCHAIN_FILE=$EMSCRIPTEN/cmake/Modules/Platform/Emscripten.cmake \
          -DCMAKE_BUILD_TYPE=Release \
          -DZLIB_LIBRARY=$HOME/zlib-1.2.11/libz.a \
          -DZLIB_INCLUDE_DIR=$HOME/zlib-1.2.11 \
          -DMULTI_THREADED=OFF \
          -DUSE_SIGNALS=OFF \
          -DUSE_GMP=OFF \
          -DBUILD_SWIPL_LD=OFF \
          -DSWIPL_PACKAGES=OFF \
          -DINSTALL_DOCUMENTATION=OFF \
          -DSWIPL_NATIVE_FRIEND=build \
          -G Ninja ..

### Building a 32-bit version on 64-bit Debian based Linux

Building the 32-bit version on  a  64   bit  platform  can be useful for
testing and creating  32-bit  .qlf  files   or  saved  states.  A fairly
complete system is created using the configuration command below.

    cmake -DSWIPL_M32=ON \
          -DSWIPL_PACKAGES_JAVA=OFF -DSWIPL_PACKAGES_QT=OFF \
          -G Ninja ..

### Cross-building for targets without an emulator

In the above scenarios we have an  emulator (Wine, Node.js) that can run
the compiled Prolog system  so  we  can   do  the  Prolog  steps  of the
installation such as building  the  boot   file,  building  .qlf  files,
library indexes and the documentation. On some  systems we do not have a
suitable emulator. Experimental support is  provided using the following
steps:

  - Build a native Prolog system in a directory, say `native`.  This
    version must have the same _word-size_ (32 or 64-bits) as the
    cross-compiled target.  One the core Prolog system (no packages)
    is required and the system only needs to be build, i.e., the
    _install_ step is allowed but not needed.  See above.

  - Specify `-DSWIPL_NATIVE_FRIEND=native` for the cross-compilation.
    This will cause the above system to be used for the cross
    compilation steps.

## Development

When building SWI-Prolog using cmake a  complete installation is created
in the cmake _build_ directory. If possible,   the files from the source
tree that do not need modification are   created  as _symbolic links_ to
the real sources. This  implies  that  `src/swipl`   can  be  used  as a
complete development environment and library   and system predicates can
be edited using edit/1 and friends.

The script `scripts/swi-activate` may be used   to  create symlinks from
$HOME/bin to the version in the  current   working  directory. It may be
used to activate the system  in  the   build  directory  or  where it is
installed. It is called from the build directory as one of:

    ../scripts/swipl-activate
    ../scripts/swipl-activate --installed

Developers    may    wish    to    set    the    environment    variable
`SWIPL_INSTALL_PREFIX`,   which   is   used   as     the   default   for
`CMAKE_INSTALL_PREFIX`. Moreover, if this variable includes `@builddir@`
this  string  is  replaced  with  the  basename  of  the  current  build
directory. This aims at the following scenario:

  1. Set e.g. `export SWIPL_INSTALL_PREFIX=$HOME/cmake/@builddir@`
  2. Use multiple build directories  for   debug,  different  targets or
     different configurations.  Typically these are called
     `build.<config>`, for example `build.single-threaded`.
  3. Configure without specifying a `CMAKE_INSTALL_PREFIX`
  4. Build, test and install.  Optionally use `swipl-activate` to use
     this version as default.

When  developing  on  the  core  system  one  often  does  not  want  to
re-generate documentation and possible package dependencies. This can be
achieved using the target `core`, which   builds `swipl`, `libswipl` and
`boot.prc`:

    ninja core


### Testing

Tests are registered for use with `ctest`.  To run all tests, simply run
this in the build directory.  Tests  can   be  run  concurrently (`-j 8`
below).

    % ctest -j 8

Note that there seem to be  few   tests.  This is misleading. Each ctest
test loads a Prolog file that  may  run   hundreds  of  tests. If a test
fails,  run  the  command  below  to    get  details.  Tests  are  named
_package_:_name_, so `ctest -V -R clib:` runs   the tests for the `clib`
package.

    % ctest -V -R name

Note that all tests can be executed   interactively  by loading the test
file and calling the entry point  as   illustrated.  The  entry point is
always the base  name  of  the   file  (without  directory  and  without
extension).

    % src/swipl ../src/Tests/core/test_arith.pl
    ?- test_arith.
    % PL-Unit: div ... done
    ...

### Trapping memory issues using AddressSanitizer

[AddressSanitizer](https://en.wikipedia.org/wiki/AddressSanitizer) is an
extension to Clang and GCC to  instrument executables for finding common
memory    management    issues.    It    traps     similar    bugs    as
[Valgrind](http://valgrind.org/), but if a suspected   bug does not show
up using one tool it might  be  worthwhile   to  try  the  other. A nice
property of Valgrind is that it can   be used directly on the executable
without recompilation. The downside is that   Valgrind makes the program
run about 20 times slower. The slowdown   by AddressSanitizer is about a
factor two. To compile for using with AddressSanitizer, do e.g.,

    % mkdir build.sanitize
    % cd build.sanitize
    % cmake -DCMAKE_BUILD_TYPE=Sanitize -G Ninja ..
    % ninja

See also `cmake/BuildType.cmake` and `PL_halt()` in `src/pl-fli.c`.

You can run the tests normally using   `ctest`. Note that the `swipl:GC`
test requires more stack than the   default when using AddressSanitizer.
To fix this run (bash) `ulimit  -s   20000`  before running `ctest`. The
test  `jpl:prolog_in_java`  will  fail  because  Java is not loaded with
AddressSanitizer preloaded.   All other tests should pass (about 4 times
slower than normal).


## Packaging

### Windows

The windows installer is created from the cross-compiled version using a
Linux native port of the NSIS  installer generator. Ensure `makensis` is
installed  (`apt-get install nsis`)  and  run the commands below  in the
build directory to create the installer:

    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win64.cmake -G Ninja ..
    ninja
    cpack

And, for the 32-bit version:

    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../cmake/cross/linux_win32.cmake -G Ninja ..
    ninja
    cpack


### Debian based Linux systems (.deb or .rpm)

The following commands create  `swipl-<version>-<nr>.<cpu>.deb/rpm` file
with  SWI-Prolog  to  be  installed  in  /usr.  The  process  creates  a
monolithic installer for a particular configuration of SWI-Prolog.  This
is  *not*  what is typically used to create packages for  distributions.
Distro package maintainers are referred to  _Modular packages for Linux_
below.  The procedure here is intended to create  custom Debian packages
for in-house deployment.

    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -G Ninja ..
    ninja
    cpack -G DEB

for generating an RPM, use  `cpack  -G   RPM`.  The  cmake configure run
selects a default packager depending on  the availability of the package
installer `apt` (assuming DEB) or `dnf` (assuming RPM).


#### Modular packages for Linux

Most Linux distributions  with  to   install  SWI-Prolog  using multiple
packages, notably to reduce dependencies. For  example, the xpce package
os normally provided  by  a  package   `swi-prolog-x`  and  the  core of
SWI-Prolog as `swi-prolog-nox`. This  allows installing `swi-prolog-nox`
on headless servers without installing X11.

Modular installation can be based on cmake _COMPONENTS_. The files for a
particular component can be installed using, for  example  (note this is
a one-line command):

    DESTDIR=$(pwd)/<component> \
        cmake -DCMAKE_INSTALL_COMPONENT=<component> \
              -P cmake_install.cmake

The defined components are:

  | Component            | Description                          |
  | -------------------- | ------------------------------------ |
  | Core_system          | Compiler and core libraries          |
  | Core_packages        | Packages with few dependencies       |
  | Archive_interface    | Libarchive binding                   |
  | Commandline_editors  | Readline and libedit interfaces      |
  | ODBC_interface       | ODBC binding                         |
  | BerkeleyDB_interface | BDB interface                        |
  | Perl_regex           | PCRE library binding                 |
  | YAML_support         | Libyaml binding                      |
  | Java_interface       | Java interface (JPL)                 |
  | OpenSSL_interface    | Binding to OpenSSL/LibreSSL          |
  | TIPC_networking      | Linux TIPC network support           |
  | Qt_console           | Qt windowed interface                |
  | Graphics_subsystem   | The xpce graphics system (needs X11) |
  | Documentation        | System HTML documentation            |
  | Examples             | Example files                        |

See the `debian` subdirectory for the complete   set  of rules we use to
generate the Ubuntu PPA releases.

### Create a MacOSX Bundle

    cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_MACOS_BUNDLE=ON -G Ninja ..
    ninja
    cpack

## Issues

- Provide a FindSWIPL.cmake?
- Problem compiling SWI when another SWI is installed already and you
  have environment variables set to facilitate e.g., embedding in Java.
  The variable names and possibly conflicting values depend on the OS.
  See [issue](https://github.com/SWI-Prolog/swipl-devel/issues/435)
