A toolchain is a hierarchy

Questions

  • What does foss-2025a name, if it is not a label?

  • Which toolchains may a foss package depend on?

  • Is system in the default hierarchy walk?

What you will be able to do

  1. Say what a toolchain names, and why it is not a label.

  2. Given a package’s toolchain, say which other toolchains its dependencies may have been built with.

  3. Recognise a _verify_toolchain failure as belonging to the toolchain module rather than to your easyconfig.

The line appears to be a sticker. It is not.

name = 'code-server'
version = '4.130.0'
toolchain = SYSTEM

SYSTEM is the simplest value that line accepts, and it is rarely used on a real cluster. Instead, other values appear:

toolchain = {'name': 'foss', 'version': '2025a'}

Read as a name, foss-2025a looks like a label for “the 2025a open-source compiler stack”. It is the top of a structure. That structure is which versions of which libraries a package on this toolchain is allowed to see. Geimer et al. [2014] is why the module tree has the same shape: a Core compiler extends $MODULEPATH into a Compiler layer, an MPI extends it again, and the user types WRF/3.5 rather than WRF/3.5-GCC-4.8.2-OpenMPI-1.7.3. Alvarez et al. [2016] add the layer under the compilers, GCCcore, so CMake is not rebuilt once per toolchain. Ask the sample. What that line names is not one toolchain. It is a list.

The list has an order, and the order does work

The function get_toolchain_hierarchy is defined in easybuild/framework/easyconfig/easyconfig.py and its docstring describes its output.

Determine list of subtoolchains for specified parent toolchain. Result
starts with the most minimal subtoolchains first, ends with specified
toolchain.

Within the foss family, the members are GCCcore, GCC, gompi, gfbf and foss, each representing a distinct toolchain and each provided as a module. The package declares the last one and may depend on artifacts built with any of them. The ordering is significant and does not follow a simple linear sequence. GCCcore precedes GCC, which precedes both gompi and gfbf, and both precede foss. The two middle entries occupy the same hierarchical level: gompi equals GCC plus MPI, gfbf equals GCC plus numerical libraries, and neither is built atop the other. a displayed list must place one of them first, though the choice carries no semantic weight. The sample demonstrates this behavior. A foss package may depend on a GCCcore build. A GCCcore package may not depend on a foss build. Courtesy does not influence this relationship. The hierarchy of a GCCcore package does not contain foss, therefore a foss dependency is never considered. The system then reports the inability to locate the required item on disk.

The shape is a graph

The foss line forms a chain. It a poor example for understanding the code’s behavior. EasyBuild’s own docstring selects a more complex example and illustrates it:

   goolfc
   /     \
gompic    golfc(*)
     \    /   \      (*) optional toolchains, not compulsory
     gcccuda golf(*)
         \   /
          GCC
         /  |
 GCCcore(*) |
         \  |
        (system)

Two toolchains reside at the same hierarchical level and share a common ancestor, therefore the walk cannot be described as “follow SUBTOOLCHAIN upward until it runs out”.

A comment in the implementation notes that the search is breadth‑first, required to consider the potential for multiple subtoolchains.

The asterisks indicate that GCCcore is optional.

A toolchain class includes an OPTIONAL attribute, and an optional level is counted only when present.

the hierarchy of foss-2025a is not a property of the literal string foss-2025a; it reflects the installed configuration.

EB-Toolchain-1 — What may this package depend on

Ask the sample above for foss-2025a and read the members back.

Now suppose you are writing an easyconfig whose toolchain is GCCcore-14.2.0.

  1. Which of the members it printed could your dependencies have been built with?

  2. Which could they not?

  3. A colleague says “but the foss module is right there, I can see it in module avail”. What has gone wrong with that reasoning?

Solution

Your package is on GCCcore, so its hierarchy is system then GCCcore-14.2.0, and nothing above. A dependency built with GCC, gompi, gfbf or foss is not a candidate.

The direction is the thing people invert. A foss package may depend on a GCCcore build. A GCCcore package may not depend on a foss build in the GCCcore one.

Your colleague is reading module avail, which lists what is installed. The robot is asking a different question: what is installed and in this package’s hierarchy. The module is there. It is not a candidate. The error says it cannot be found questions.

The system toolchain is not always in it

The same docstring then qualifies the bottom of the list.

The system toolchain is considered the most minimal subtoolchain only
if the ``add_system_to_minimal_toolchains`` build option is enabled.

That option only changes the walk when a dependency did not name a toolchain. ('CUDA', '12.9.1', '', SYSTEM) still finds the SYSTEM module, flag or no flag. What the option does not do, off by default, is treat a bare ('zlib', '1.3.1') as “also try SYSTEM”. A foss package that omitted the fourth field looks for zlib/1.3.1-foss-… or a foss subtoolchain, not zlib/1.3.1.

What a toolchain module has to contain

A toolchain occupies a graph position. It also is a module, and the module must conform to the class. EasyBuild performs checks:

List of toolchain dependency modules and toolchain definition do not match
(found [...] vs expected {...})

That comes from _verify_toolchain in easybuild/tools/toolchain/toolchain.py, and it fires in the prepare step, seconds in, before anything is unpacked. Three properties of that check, together, make it unfixable from your easyconfig:

  • It reads the direct dependencies of the toolchain module, and only those.

The call is dependencies_for(mod_name,..., depth=0). A library that arrives transitively does not count.

  • It matches on module name, so versions and versionsuffixes are invisible to it.

  • Elements are dropped from the expected set only when they are optional and absent, and the base is_required returns True for everything.

So a subclass has to opt an element out; nothing opts out by accident. The failure looks like your package’s fault and is not. A framework release that renames or splits a toolchain element breaks every package on that generation at once. No easyconfig parameter reaches the check. EasyBuild 5.2 did exactly this. In toolchains built on the NVIDIA HPC SDK (NVHPC), the compiler element became a module named nvidia-compilers. Toolchain modules written before the split still expected the old name. The tell is that the two lists in the error differ by one element. Read the lists, not the package. The difference names the element, and the fix is a toolchain module that carries it. In practice that means moving to a generation whose toolchain modules already do.

The same name can mean two toolchains

get_toolchain_hierarchy contains this, in full:

Stopgap solution until deprecated compiler-only NVHPC toolchain is
removed. Both the new and old NVHPC toolchains share the same name, we
need to selectively filter one out based on the version of the
toolchain.

Two distinct toolchain classes are represented by a single string. The framework resolves the ambiguity using the version number, and a TODO: delete with EasyBuild 6.0 note is placed beside it. If foss-2025a functioned as a label, the code would be unnecessary.