Four EasyBuild 5 surfaces the earlier chapters skipped

Questions

  • What does an EasyBuild install lock prevent, and how is it not a Resolvo lock?

  • When does --bwrap move bytes, and when does the namespace fail?

  • What does a Dataset easyconfig install that a software easyconfig does not?

What you will be able to do

  1. Name the directory EasyBuild uses for install locks, and the flag that ignores them.

  2. Say what EBPYTHONPREFIXES is for, and what a 5.3 entry point changes about a Python module.

  3. Install a dataset as a module and depend on it from a software easyconfig, naming the two path variables that keep data off the software prefix.

Before this chapter

Four EasyBuild 5 surfaces appear beside the machinery in chapters 13–19 and lack a page. They are not part of eb-stack. They explain why a campaign finding of class install or runtime can represent the framework rather than the recipe.

Install locks are not solver locks

Two eb processes installing the same easyconfig into the same prefix will corrupt the tree. EasyBuild writes a lock file for the duration of an installation. --locks-dir sets the directory; the default is a .locks subdirectory of the software installation prefix, and the documentation says it should be on a shared filesystem. --ignore-locks is the escape that two operators will reach for and that a site should not leave on.

That lock is a mutex. It names no versions and solves no SAT instance. eb-stack ports a recipe and bumps one’s locks/default.lock.json is a Resolvo assignment. Mixing the words in a status report is how a “lock wait” gets escalated as a solver bug.

--wait-on-lock-* controls whether a second process waits or errors. The filetools API is create_lock / check_lock / remove_lock. A stale lock after a killed eb is an operator problem on the shared filesystem, not a recipe problem.

–bwrap installs to a temporary namespace

--bwrap (EasyBuild 5.3, improved in 5.4.0 for read‑only filesystems and --subdir-{modules,software}) builds and installs within a bubblewrap namespace, then transfers the result to the final prefix. --bwrap-installpath is the temporary location. The goal is a prefix that other users must not see while it is partially written, together with a CernVM-FS or similar read‑only view that cannot be written in place. The overlayfs work in 5.4.0 enables EESSI‑style writable‑over‑read‑only installations to use this approach. RPATH wrappers still fail when /tmp is mounted with noexec (EasyBuild #4910). A --bwrap detection that appears as a missing linker is frequently caused by that mount option. What a build is allowed to see’s isolation policy provides the basis; this flag represents one implementation.

EBPYTHONPREFIXES and entry points

A Python installation in EasyBuild can see packages in another prefix via EBPYTHONPREFIXES. This allows a user or site prefix on top of EESSI (EESSI, somebody else’s stack, A site stack next to EESSI, not inside it) to contribute .so and .py files without rebuilding CPython. It also enables a module that only wraps a site wheel to become loadable. EasyBuild 5.3 added entry-point handling so a console script installed into an extension prefix appears on PATH when the extension’s module is loaded. A campaign that verifies python -c 'import pkg' and never runs the console script will not encounter a missing entry point. binary_verified in A campaign finding is a typed failure, not a log must name the command the user will type.

Dataset is a module whose payload is data

Datasets were added as modules in EasyBuild 5.1.0. The origin is Sam Moors’ EUM’24 talk. The easyconfig uses data_sources instead of, or in addition to, sources.

easyblock = 'Dataset'

name = 'RFdiffusion-models'
version = '1.1.0'
toolchain = SYSTEM

source_urls = ['https://example.com/data/']
data_sources = ['example-data-%(version)s.tar.gz']
checksums = ['d3adb33f...']

The generic Dataset easyblock derives from Binary. No compile step is performed. install_step extracts into the data prefix. The default for extract_sources is true. cleanup_data_sources deletes the fetched archives after install. object_storage_ignore_dirs (5.4.0) lists directories to keep out of object storage; . means the whole installdir. Two path variables keep data off the software prefix:.

export EASYBUILD_SOURCEPATH_DATA=/sources/data
export EASYBUILD_INSTALLPATH_DATA=/data
eb RFdiffusion-models-1.1.0.eb
module load RFdiffusion-models/1.1.0

If installpath-data is unset, installpath combined with subdir-data (default data) is used. That module may appear in a software easyconfig’s dependencies list. A GROMACS that requires a weight file depends on the dataset module in the same way it depends on FFTW. The robot resolves the dependency. The payload consists of bytes, not a binary.

EB-Five-1 — Which lock, which prefix

Four incidents. Name the EasyBuild 5 surface, and the first check.

  1. Two eb processes on a shared prefix, same easyconfig, one dies with a lock error, the other is still in install_step.

  2. --bwrap on a node where /tmp is noexec; the log talks about RPATH wrappers.

  3. A user prefix installed with EESSI-extend; import numpy works from the EESSI CPython; the f2py console script is not on PATH.

  4. A two-gigabyte weight tarball landed under .../software/linux/x86_64/amd/zen3/software/ next to GROMACS.

Solution

  1. Install locks. Check --locks-dir (default .locks under the software prefix) and whether the surviving process holds the mutex. Not a Resolvo lock. Do not pass --ignore-locks to “fix” it.

  2. --bwrap plus #4910. Check the /tmp mount options. The wrapper binaries need exec.

  3. Entry points / EBPYTHONPREFIXES. import finding the package does not put a console script on PATH. Verify the command the user types, not only the import.

  4. A Dataset easyconfig was not used, or EASYBUILD_INSTALLPATH_DATA was unset and the default subdir-data was not what the site expected. Weights are data. They get a data prefix and a module, not a software prefix.

What to remember

  • An install lock is a mutex on a shared prefix. A Resolvo lock is an assignment of versions. They share a word and nothing else.

  • --bwrap installs in a namespace and then moves. /tmp noexec breaks the RPATH wrappers.

  • EBPYTHONPREFIXES makes a second Python prefix visible. Entry points are a 5.3 addition; an import test does not cover them.

  • A Dataset module is data. EASYBUILD_INSTALLPATH_DATA keeps it off the software prefix. Software depends on it like any other module.