A portable test leaves the machine in the config

a portable test plus a mixin; the site stays in the config

A portable test leaves the machine in the config. The mixin is what changes per site, not the class.

test plus config equals any site

A portable test leaves the machine in the config.

the class plus mixin stay portable; the site config is the machine

valid_systems = [‘snellius’] is a trap. The machine belongs in the configuration.

Questions

  • Why is valid_systems = ['snellius'] a trap?

  • Where does a portable test read the core count from?

  • What does the EESSI mixin replace in the class body?

What you will be able to do

  1. Point at the three lines that make a ReFrame test specific to one cluster, and say where each line belongs instead.

  2. Read an EESSI mixin test and say what device_type and compute_unit cause the hooks to set.

  3. Skip a test when the node cannot run it, rather than hardcoding a size that node will never have.

Before this page

valid_systems = ['snellius']
self.num_tasks_per_node = min(self.num_tasks, 128)
module_name = parameter([
 'mpi4py/3.1.4-gompi-2023a',
 'mpi4py/3.1.5-gompi-2023b',
])

The first binds the test to a single system name. The second provides the system’s core count as a constant. The third supplies two module strings present in that site’s tree. A 64-core node running EESSI’s mpi4py/3.1.5-gompi-2023b still fails the first filter and oversubscribes when the name is patched manually. EESSI’s writing-tests page presents a three-step tutorial. Write the Slurm scripts for one machine, write the ReFrame test that encodes them, then remove the machine. The third step is a mixin. The test class retains the workload. The configuration retains the node.

What the mixin changes

import reframe as rfm
import reframe.utility.sanity as sn
from eessi.testsuite.eessi_mixin import EESSI_Mixin
from eessi.testsuite.constants import DEVICE_TYPES, COMPUTE_UNITS, SCALES
from eessi.testsuite.utils import find_modules
from reframe.core.builtins import sanity_function

@rfm.simple_test
class EESSI_MPI4PY(rfm.RunOnlyRegressionTest, EESSI_Mixin):
    device_type = DEVICE_TYPES.CPU
    compute_unit = COMPUTE_UNITS.CPU
    module_name = parameter(find_modules('mpi4py'))
    scale = parameter(SCALES.keys())
    n_iterations = variable(int, value=1000)
    n_warmup = variable(int, value=100)
    executable = 'python3'
    executable_opts = [
        'mpi4py_reduce.py',
        '--n_iter', f'{n_iterations}',
        '--n_warmup', f'{n_warmup}',
    ]
    time_limit = '5m00s'
    readonly_files = ['mpi4py_reduce.py']

    def required_mem_per_node(self):
        return self.num_tasks_per_node * 100 + 250

    @sanity_function
    def validate(self):
        return sn.assert_found(r'Sum of all ranks', self.stdout)

EESSI_Mixin provides hooks that read the partition. compute_unit = COMPUTE_UNITS.CPU is one task per core. A full node is a SCALES value, not that assignment. The hook assigns num_tasks from processor.num_cpus in the configuration. A node with 64 cores receives 64 tasks. A node with 128 cores receives 128 tasks. The test file lacks both numbers. find_modules('mpi4py') queries the modules tool for names available on this system, then the suite loads each case. The parameter list is no longer two strings from last year’s Snellius tree. A site with a single mpi4py module yields one case. A site with four mpi4py modules yields four cases. valid_systems is no longer a hostname. The mixin requests the configuration features declared: CPU, GPU and a minimum memory. A partition that does not advertise those features skips the test. That skip is a result. It is the correct result. required_mem_per_node follows the same concept as a residual in Inspect writes a work queue, not a recipe. The test defines a requirement. The framework compares the requirement to the partition. A mismatch results in a skip rather than a job that OOM‑kills the node.

The configuration is now the only machine file

processor.num_cpus and features values are provided by the configuration file and read by these hooks. A site that omits them produces tests that cannot compute a task count. Those tests skip or fall back to a conservative option. A site that misrepresents the configuration causes tests to follow the same misrepresentation. The portable suite reflects the accuracy of that file. CSCS’s webinar shows the same expansion: STREAM as a local binary, followed by the same class on any partition described by the configuration. ExCALIBUR-tests applies this in item 3 of its outline, “Configuring ReFrame to run tests on HPC systems”, before creating a second test. The order conveys the lesson. The machine is entered into the configuration first.

EB-ReFrame-45-1 — What to move

A GROMACS test contains these four assignments. For each, say whether it stays in the test, moves to the configuration, or becomes a hook that reads the configuration.

  1. executable = 'gmx'

  2. valid_systems = ['snellius:gpu']

  3. num_tasks = 128

  4. modules = ['GROMACS/2024.4-foss-2023b']

Solution

  1. Stays. The workload names the binary.

  2. Stays as a feature request. The configuration marks the GPU partition with a gpu feature. The test requests +gpu. Binding the cluster name makes every other site skip the test.

  3. Becomes a hook. compute_unit of one node, or of one GPU, reads num_cpus or the device count from the partition. 128 is Snellius.

  4. Becomes parameter(find_modules('GROMACS')), or a single find_modules result if the suite is checking one installation. The version string is a fact about one tree.

What to remember

  • A cluster name, a core count and a module string in the test are three facts about one machine.

  • The mixin reads the partition. “One full node” is a request; the configuration supplies the number.

  • find_modules builds the parameter list from modules available on the system, then the suite loads each case.

  • A missing feature or too little memory is a skip, not a failed job.