Building from Source with Meson

Building from Source with Meson#

This chapter describes how to build HPCToolkit from source using Meson. This approach is recommended for HPCToolkit developers.

Quickstart#

Install:

Then run:

$ meson setup builddir/
$ cd builddir/
$ meson compile  # -OR- ninja
$ meson test     # -OR- ninja test

Do not use meson install or ninja install. Instead, working versions of the tools themselves are available using meson devenv. One can use meson devenv to either run an individual command, or create a subshell environment in which HPCToolkit’s commands are available, as shown below:

$ meson devenv hpcrun --version  # -OR-
$ meson devenv
[builddir] $ hpcrun --version

Configuration#

Configuration arguments can be passed to the initial meson setup or later meson configure invocations with invocations as follows meson setup [arguments] builddir/ or meson configure [arguments]. Arguments that control the build of HPCToolkit are listed below:

  • -Dtests=(disabled|auto|enabled): Enable unit tests with meson test. (disabled only disables tests that require additional dependencies.)

  • -Dmanpages=(disabled|auto|enabled): Generate and build man pages. Requires Docutils.

  • -Dmanual=(disabled|auto|enabled): Generate and build user manual. Requires Sphinx.

  • -Dhpcprof_mpi=(disabled|auto|enabled): Build hpcprof-mpi in addition to hpcprof. Requires MPI.

  • -Dpython=(disabled|auto|enabled): Enable an (experimental) Python unwinder. Requires Python headers.

  • -Dpapi=(disabled|auto|enabled): Enable PAPI metrics. Requires PAPI.

  • -Dcuda=(disabled|auto|enabled): Enable CUDA metrics (-e gpu=nvidia). Requires CUDA.

  • -Dlevel0=(disabled|auto|enabled): Enable Level Zero metrics (-e gpu=level0). Requires Level Zero.

  • -Dgtpin=(disabled|auto|enabled): Also enable Level Zero instrumentation metrics (-e gpu=level0,inst). Requires Level Zero, IGC/IGA and GTPin.

  • -Dopencl=(disabled|auto|enabled): Enable OpenCL metrics (-e gpu=opencl). Requires OpenCL headers.

  • -Drocm=(disabled|auto|enabled): Enable ROCm metrics (-e gpu=amd). Requires ROCm.

  • -Dvalgrind_annotations=(false|true): Inject annotations for debugging with Valgrind.

Note that many of the features above require additional optional dependencies when enabled.

Installing Dependencies without Root#

Meson Wraps#

If core dependencies needed to build and run HPCToolkit can be found on your system, Meson will use those. To assist you in manually providing dependencies to Meson, see the discussion about how Meson looks for dependencies later on this page.

If Meson cannot find HPCToolkit’s dependencies on your system, meson (setup|configure) will use Meson wraps to build HPCToolkit’s core dependencies. No root privileges are needed to use Meson wraps.

If you let Meson wraps provide HPCToolkit’s dependencies, then building HPCToolkit requires little more than Meson and a C/C++ compiler; any missing dependencies will be downloaded and built as subprojects. Below, we list Meson wraps that HPCToolkit will download and build if they are needed by your build configuration and they are not found on your system:

  Subprojects
    bzip2         : YES
    dyninst       : YES 347 warnings
    elfutils      : YES 2 warnings
    libiberty     : YES (from dyninst)
    liblzma       : YES (from elfutils)
    libpfm        : YES
    libunwind     : YES
    opencl-headers: YES
    tbb           : YES
    xed           : YES
    xerces-c      : YES
    xxhash        : YES
    yaml-cpp      : YES
    zstd          : YES (from elfutils)

Meson wraps are downloaded and built only as needed. In some cases, a wrap will not be considered unless you enable a particular feature of HPCToolkit. For instance, opencl-headers will not be downloaded unless your meson (setup|configure) command includes the configuration option -Dopencl=enabled. If the internet is not available on the system running meson setup, you may also need to run meson subprojects download first on a system with internet to download the required sources.

If wraps are not desired, they can be disabled with meson (setup|configure) --wrap-mode=nofallback. With this setting, all dependencies must be installed or otherwise loaded into the build environment, except for force-fallback dependencies meson (setup|configure) --force-fallback-for=dep1,dep2,....

See the official Meson documentation about wraps for more configuration options and implementation details.

Dev Containers (BETA)#

Another way to manage dependencies without root permission is to simply build and run HPCToolkit in container.

Multiple Dev Container configurations are provided for common distros and vendor software:

  • ubuntu20.04, rhel8, leap15, fedora39: CPU-only configurations for common distros

  • cuda*: Ubuntu 20.04 with NVIDIA CUDA toolkit installed

  • rocm*: Ubuntu 20.04 with AMD ROCm installed

  • intel: Ubuntu 20.04 with Intel OneAPI Base Toolkit

  • bare: Bare minimum dependencies to build HPCToolkit

Installing Dependencies as Root#

Debian/Ubuntu and Derivatives#

For Debian Sid/Ubuntu 23.10:

sudo apt-get install git build-essential ccache ninja-build meson cmake pkg-config python3-venv libboost-all-dev libbz2-dev libtbb-dev libelf-dev libdw-dev libunwind-dev libxerces-c-dev libiberty-dev libyaml-cpp-dev libpfm4-dev libxxhash-dev zlib1g-dev
pipx install 'meson>=1.6.0'

For older versions:

sudo apt-get install git build-essential ccache ninja-build cmake pkg-config pipx python3 python3-pip python3-venv libboost-all-dev libbz2-dev libtbb-dev libelf-dev libdw-dev libunwind-dev libxerces-c-dev libiberty-dev libyaml-cpp-dev libpfm4-dev libxxhash-dev zlib1g-dev
pipx install 'meson>=1.6.0'

Note that some dependencies may be built from wraps by default, either because they aren’t packaged in Debian/Ubuntu (e.g. Dyninst) or because the version is too old. Additional packages can be installed for optional features:

Package

Feature option

Notes

mpi-default-dev

-Dhpcprof_mpi=enabled

libpapi-dev

-Dpapi=enabled

nvidia-cuda-toolkit

-Dcuda=enabled

level-zero-dev

-Dlevel0=enabled

libigdfcl-dev

-Dlevel0=enabled

libigc-dev

-Dgtpin=enabled

opencl-headers

-Dopencl=enabled

Optional, available as wrap

rocm-hip-libraries

-Drocm=enabled

Fedora/RHEL and Derivatives#

For RHEL 8:

sudo dnf install git gcc gcc-c++ ccache ninja-build cmake pkg-config python39 python39-pip boost-devel bzip2-devel tbb-devel elfutils-devel xerces-c-devel binutils-devel yaml-cpp-devel libpfm-devel xxhash-devel zlib-devel
python3.9 -m pip install pipx
pipx install 'meson>=1.6.0'

For RHEL 9:

sudo dnf install git gcc gcc-c++ ccache ninja-build cmake pkg-config python3 pipx boost-devel bzip2-devel tbb-devel elfutils-devel xerces-c-devel binutils-devel yaml-cpp-devel libpfm-devel libunwind-devel xxhash-devel zlib-devel
pipx install 'meson>=1.6.0'

For Fedora:

sudo dnf install git gcc gcc-c++ ccache ninja-build cmake pkg-config python3 meson pipx boost-devel bzip2-devel tbb-devel elfutils-devel xerces-c-devel binutils-devel yaml-cpp-devel libpfm-devel libunwind-devel xxhash-devel zlib-devel
pipx install 'meson>=1.6.0'  # Optional but recommended

Note that some dependencies may be built from wraps by default, either because they aren’t packaged in Fedora/RHEL (e.g. Xed) or because the version is too old. Additional packages can be installed for optional features:

Package

Feature option

Notes

openmpi-devel or mpich-devel or mvapich2-devel

-Dhpcprof_mpi=enabled

papi-devel

-Dpapi=enabled

cuda-toolkit

-Dcuda=enabled

level-zero-devel

-Dlevel0=enabled

intel-igc-opencl-devel

-Dlevel0=enabled

opencl-headers

-Dopencl=enabled

Optional, available as wrap

rocm

-Drocm=enabled

SUSE Leap/SLES 15 and Derivatives#

sudo zypper install git gcc12 gcc12-c++ ccache ninja-build cmake pkg-config python311 python311-pip libboost_atomic-devel libboost_chrono-devel libboost_date_time-devel libboost_filesystem-devel libboost_graph-devel libboost_system-devel libboost_thread-devel libboost_timer-devel libbz2-devel tbb-devel libxerces-c-devel binutils-devel yaml-cpp-devel libpfm-devel xxhash-devel zlib-devel
python3.11 -m pip install pipx
pipx install 'meson>=1.6.0'

Note that some dependencies may be built from wraps by default, either because they aren’t packaged in SUSE Leap (e.g. Xed) or because the version is too old. Additional packages can be installed for optional features:

Package

Feature option

Notes

openmpi-devel or mpich-devel

-Dhpcprof_mpi=enabled

papi-devel

-Dpapi=enabled

cuda-toolkit

-Dcuda=enabled

level-zero-devel

-Dlevel0=enabled

intel-igc-opencl-devel

-Dlevel0=enabled

opencl-headers

-Dopencl=enabled

Optional, available as wrap

rocm

-Drocm=enabled

Custom Dependencies#

It is also possible to build and run HPCToolkit using custom versions of dependencies, e.g. one you have modified to fix a bug. However, to expose such dependencies to Meson, you will need some basic understanding of how Meson finds dependencies.

This section is intended as a “peek behind the curtain” of how Meson understands dependencies, and an abridged version of [Meson’s dependency() docs] for developers.

Meson fetches all dependencies from the “build environment” aka the “machine” (technically two, the build_machine and the host_machine, which are only different when cross-compiling). Whenever Meson configures (i.e. during meson setup or automatically during meson compile/install/test/etc.), queries the “machine” as defined by various environment variables:

  • Finding programs (find_program()) is done by searching the PATH,

  • The C/C++ compilers (CC/CXX), which are also affected by CFLAGS, CXXFLAGS, LDFLAGS, INCLUDE_PATH, LIBRARY_PATH, CRAY_*, …

  • Dependencies are found via pkg-config, which is affected by PKG_CONFIG_PATH (and other PKG_CONFIG_* variables),

  • Dependencies are found via CMake, which is affected by CMAKE_PREFIX_PATH (and other CMAKE_*, *_ROOT and *_ROOT_DIR variables),

  • Dependencies with Meson built-in functionality have their own quirks:

    • Boost is found under BOOST_ROOT,

    • The CUDA Toolkit is found under CUDA_PATH.

These variables are frequently altered by commands designed to load software into a shell environment, for example module un/load. Meson assumes the “machine” does not change between meson invocations, however if it does by running the above commands it is possible Meson’s cache will not match reality. In these cases reconfiguration or later compilation may fail due to the inconsistent state between dependencies. In some cases this can be fixed by clearing Meson’s cache (meson configure --clearcache), Meson will efficiently avoid rebuilding files if the compile command did not actually change. If build errors persist it may be necessary to wipe the build directory completely (meson setup --wipe), it is recommended to use ccache to accelerate the subsequent rebuild.

If you want to use a custom version of a dependency library, install it first and then adjust one or more of the variables mentioned above or corresponding Meson options:

  • If the dependency installs a pkg-config file (usually lib/pkgconfig/*.pc), append the directory containing these files to PKG_CONFIG_PATH or -Dpkg_config_path.

  • In all other cases, append the install prefix (i.e. the directory with include/ and lib/ or lib64/) to CMAKE_PREFIX_PATH or -Dcmake_prefix_path.

  • For select dependencies (i.e. if the dependency doesn’t install pkg-config or CMake files and there is no matching share/cmake-*/Modules/Find*.cmake script in the CMake installation), you may also opt to configure the compiler instead, by:

    • Extending -Dc_args with appropriate -I include flags and -Dc_link_args with -L link flags, or

    • Extending CPATH with include/ directories and LIBRARY_PATH with lib/ or lib64/ directories.

Some of these variables can be persisted as Meson built-in options (e.g. -Dpkg_config_path) or more generally [native files][meson native file]. For example, a native file to use GCC 11 with extra flags and a custom dependency search flags would look like:

# …/my-env.ini
[binaries]
c = ['ccache', 'gcc-11']
cpp = ['ccache', 'g++-11']

[built-in options]
c_args = ['-fstack-protector']
cpp_args = ['-fhardened']
cmake_prefix_path = ['/…/path/to/custom/install/', '/…/path/to/other/install/']
pkg_config_path = ['/…/path/to/custom/install/lib/pkgconfig', '/…/path/to/other/install/lib/pkgconfig']

And then can be used in a build directory by passing additional arguments to meson setup:

$ meson setup --native-file …/my-elf-dyn.ini builddir/

See [doc/developers/meson.ini] for a more complete template listing the settings available in a native file.

Meson Documentation References#

Contents