A checksum is a claim about bytes¶
What you will be able to do
Read a
checksumsblock, including the case where one entry is a dictionary, and say why it is one.Say what a verified checksum establishes and what it leaves open.
Say which parameters break loudly on a version bump and which break quietly.
The easyconfig in An easyconfig is Python allocates more lines to describing source locations than to any other aspect. The following excerpt presents that section in isolation.
Three parameters exist, each representing a different type of statement. This chapter discusses the third, which people trust most beyond its literal meaning.
The list is positional, and then one entry is a dictionary¶
The variable sources is a list.
The variable checksums is a list.
Elements of checksums correspond to elements of sources by position, with the first checksum belonging to the first source.
The outer list in this example contains exactly one element.
That element is a dictionary rather than a string.
The dictionary maps filenames to checksums and contains two entries for the single source.
Both facts are required to interpret the structure.
The dictionary exists.org][An easyconfig is Python]].
the single line can resolve to different filenames on x86-64 and on aarch64.
One source line therefore yields two possible files, and the dictionary indicates which checksum belongs to each file.
The keys of the dictionary are full resolved filenames, not architecture identifiers such as amd64 or arm64.
The dictionary does not contain architecture information.
It stores names, and the templating system determines which name to look up.
Adding support for a third architecture does not cause the dictionary to raise an error for the missing entry.
The lookup simply fails to find a key for that architecture on the machine that provides it.
A per-architecture dictionary remains silent about architectures that are not listed.
What a checksum can and cannot report¶
A checksum verifies the bytes. That is its sole purpose. It cannot indicate that the archive contains the expected version. If upstream re‑tags a release and the archive changes, the checksum fails. Information. Suppose upstream publishes a different archive also labeled 4.130.0 and a checksum is computed for that archive. The checksum then passes indefinitely and reports nothing. It cannot indicate that the archive matches the layout expected by your easyconfig. This mirrors the failure described in An easyconfig is Python, and the two claims merit side‑by‑side comparison.
The checksum verified against the archive.
The archive did not contain the directory the build expected.
Both are true at once.
Verification passed at full strength over exactly the right bytes.
The build then failed on start_dir.
EB-Checksum-1 — One architecture, one missing key
A site adds riscv64 support to the code-server easyconfig. The build passes on every x86-64 and arm64 worker and fails on the riscv64 one, before anything is compiled.
The sources line is unchanged. The checksums block still carries the
two entries from the chapter’s sample, keyed amd64 and arm64.
What is the failure, precisely?
Why did no x86-64 or arm64 build catch it?
What is the smallest change that fixes it, and what would you need in order to make that change?
Solution
The dictionary has no key for the filename that architecture produces.
%(mapped_arch)s resolved to riscv64, which the block does not
mention, so the lookup found nothing to verify against. (aarch64 would
not have failed: that easyblock maps it to arm64, and that key is
already there.)
No x86-64 or arm64 build could catch it template resolves to a key that is present. The dictionary is keyed by resolved filename rather than by architecture, and it says nothing at all about the filenames it omits. A per-architecture gap is invisible on every architecture except the one with the gap.
The fix is one more entry in the dictionary. To make it you need the actual checksum of the actual archive for that architecture, which means fetching it, which is the point: you cannot derive it from the others, and copying a value from anywhere else would verify the wrong file.
Which is why a source change is the dangerous kind¶
An easyconfig is Python put this as a rule about parameters that describe the shape of the source. Here it is as a rule about editing.
Change the version and the checksum must change. Forgetting is not available: the build stops on a mismatch and returns the value it computed. That loudness belongs to the parameter, not to anyone’s diligence.
Change what kind of thing is being downloaded and nothing is loud.
The QMCPACK easyconfig in An easyconfig is Python had been building from a pinned development commit and moved to a release tarball.
That change rewrites sources, and the checksum with it, so both got attention.
Everything else about the source block also changed, and none of it announced itself:
a git checkout gets archived under the repository’s name. The sources were one level down.
a release tarball unpacks to
name-version/, sources at the top.start_dirdescribed the first arrangement, and nobody touched it.
A checksum protects against the wrong bytes. Nothing in the format protects against the right bytes in a different shape.
Borrowing a checksum from another packaging system¶
Another way a verified checksum can be the wrong verified checksum.
A single project version may be represented by multiple legitimate archives.
GitHub creates a tarball from a tag.
The project also publishes a release archive, often including vendored dependencies or a generated configure not present in the tag.
Both correspond to that version.
They differ in bytes, yet neither is wrong.
Thus a checksum copied from a conda-forge recipe or a Spack package.py can be perfectly correct for the wrong archive, i.e., not the one referenced by your source_urls.
The checksum value is correct and the version is correct.
the build fails a verification that is functioning exactly as intended.
The following rule is narrow and worth preserving.
A borrowed checksum requires the URL it was computed over; the version alone is insufficient.
Two ecosystems agreeing on a version number does not mean they agree on an artifact.
Loud and quiet¶
The reason a version bump is safer than a source change is that the two break differently.
Change |
How it is found |
|---|---|
The version |
The checksum mismatches and the build stops. Out the value it computed. |
The archive’s layout |
|
The kind of source, tag to release |
Both of the above, and only the first announces itself. |
A checksum borrowed from another ecosystem |
Verification fails against an artifact that really is that version. |
Everything a checksum protects is in the first row.
Nothing in the other three is a checksum’s job, which is why An easyconfig is Python’s failure got past a verification working perfectly.
Next: exts_list, where one easyconfig installs hundreds of packages and every row above applies to each of them.