Extensions, and what an entry in exts_list inherits¶
An exts_list entry inherits the parent easyconfig. Templates rebind inside the entry.¶
An extension inherits the parent. Templates rebind inside the entry.¶
An exts_list entry does not pick a toolchain. It inherits the parent’s and installs into that module.¶
What you will be able to do
Read any
exts_listentry, in all three of its forms, and say which options it will actually get.Say which parameters an extension inherits from its parent easyconfig and which are reset before it runs.
Read an
exts_filterand say what it decides, twice.
Before this page
One easyconfig, many installs, for what
exts_listis for.How a build executes is one of the eighteen steps and behaves like one.
Languages that bring their own package manager if the extensions in question are Python packages, which they usually are.
An extension is software installed inside another installation rather than as a separate module.
The mechanism consists of the easyconfig parameter exts_list and the step extensions, the tenth of the eighteen steps.
Those two objects live in the easyblock. No single page lists every key.
Three forms, and the one that gets nothing¶
An entry may be a plain string, a (name, version) tuple, or a (name, version, {options}) tuple.
Within the framework, the definition is in easybuild/framework/easyblock.py.
if len(ext) == 1:
exts_sources.append({'name': ext_name})
else:
ext_version = resolve_template(ext[1], self.cfg.template_values)
ext_options = copy.deepcopy(self.cfg.get_ref('exts_default_options'))
if len(ext) == 3:
if isinstance(ext_options, dict):
ext_options.update(ext[2])
The branch is read.
The exts_default_options is fetched only in the else.
An entry with a name and no version returns ({'name':...}).
The entry gets no options at all, defaults included.
A bare string takes the same path.
The two entries are not treated alike in this easyconfig.
alpha receives the default source_urls.
beta lacks it.
The plan receives no warning.
eb -x displays the difference prior to building.
The merge operation is a shallow dict.update.
When a collision occurs, the per‑extension dictionary prevails.
A conflicting list is substituted instead of extended.
This causes a per‑extension source_urls to overwrite the default rather than augment it.
Two kinds of option, and only one of them is checked¶
The third element’s options are split into two groups with different behavior. Identifying the groups clarifies most surprises. Read directly by the extensions machinery. These are not easyconfig parameters; they are keys the framework searches for in the options dict.
All other items that qualify as an easyconfig parameter. Any additional key is copied into the extension’s own configuration. The loop performing this also processes the remaining items.
for key, value in self.options.items():
if key in self.cfg:
self.cfg[key] = value
self.log.debug("Customising known easyconfig parameter '%s' ...")
else:
self.log.debug("Skipping unknown custom easyconfig parameter '%s' ...")
Unknown keys are discarded at debug level.
Normal runs produce no warnings, errors, or mentions.
A misspelled per-extension option in a bundle of two hundred extensions remains invisible unless specifically searched for.
This invisibility is the single most useful aspect of this page.
Alternative parameter names are honoured on input.
The install_opts name works where installopts is intended, and real easyconfigs in the upstream repository use it.
What an extension does not inherit¶
Configuring an extension initially inherits from its parent, which is convenient, but it is wrong for parameters that describe the parent’s sources. Eight parameters are reset to their defaults first, in easybuild/framework/extension.py.
restore_options = (
'checksums',
'data_sources',
'patches',
'postinstallcmds',
'sanity_check_commands',
'sanity_check_paths',
'skipsteps',
'sources',
)
exts_filter decides the same question twice¶
exts_filter is a pair (command, input).
Each half is a template resolved per extension with %(ext_name)s, %(ext_version)s and %(src)s.
Anything that is not a two‑element list or tuple is refused.
The error message is exts_filter should be a list or tuple of ("command","input").
The framework runs exts_filter for two different purposes.
Before installing, to skip what is already there.
EasyBlock.skip_extensions builds one command per extension.
It runs them sequentially or in parallel to decide which entries can be left alone.
This makes a resumed bundle cheap instead of a re‑run.
After installing each extension, as its sanity check.
Extension.sanity_check_step builds the same commands.
It runs them with fail_on_error=False.
The exit code is the measurement. A failure is collected with the command, its input and its output rather than aborting the run.
Two results follow from it being a single mechanism.
An extension whose modulename is False produces no commands.
No check runs before and no check runs after.
This provides an escape hatch for an extension whose importable name does not exist.
get_modulenames can return more than one name.
An extension providing three importable modules gets three commands.
Each command must pass.
For Python, the conventional filter is an import.
exts_filter = ("python -c 'import %(ext_name)s'", '')
EB-Exts-1 — Four entries, four outcomes
An easyconfig sets
exts_default_options = {
'source_urls': ['https://example.org/dl'],
'preinstallopts': 'echo DEFAULT &&',
}
and then lists four entries:
exts_list = [
('alpha', '1.2'),
('beta', '2.0', {'preinstallopts': 'echo MINE &&'}),
('gamma', '3.1', {'source_urls': ['https://other.example/dl']}),
'delta',
]
Identify the source_urls and the preinstallopts associated with each entry.
Adding {'instalopts': '--verbose'} causes the verbose option to be used during installation and it appears in the build log.
The parent easyconfig defines sanity_check_paths for its binary and skipsteps = ['test']; beta inherits the skipsteps setting.
Change exts_filter = ("python -c 'import %(ext_name)s'", '') to reference the actual importable name gamma_core.
Setting modulename to False disables the import check and allows the build to continue without verifying the module.
Solution
alphagets both defaults.betagets the defaultsource_urlsand its ownpreinstallopts.gammagets its ownsource_urls, which replaces the default rather than adding to it, and the defaultpreinstallopts.deltagets neither default. It never reaches the branch that reads them.Nothing, visibly.
instaloptsis not a known parameter, so it is dropped with adebugline reading “Skipping unknown custom easyconfig parameter”. You would see it by running with--debugand searching the log, which is the only place it appears.Neither. Both are on the reset list, along with
checksums,sources,patches,postinstallcmds,sanity_check_commandsanddata_sources.Set
modulenameto'gamma_core'ingamma’s options, so the filter command imports the name that exists. Setting it toFalseinstead produces no filter commands at all for that extension, which means it is neither skipped when already installed nor checked after installation: you have switched the check off rather than corrected it, which is Testing, and the four things it can mean’s argument in miniature.
What to remember
exts_default_optionsreaches only entries that carry a version; a bare-string entry gets no options at all.The merge is a shallow update, so a per-extension list replaces the default list instead of extending it.
An unknown per-extension option is dropped at
debuglevel, with no warning: a typo in a bundle of two hundred extensions is silent.Eight parameters plus
start_dirare reset before an extension runs.exts_filteris one mechanism doing two jobs, skipping what is already installed and checking what was just installed, andmodulename: Falseswitches off both.