AIBoMGen makes that possible by restricting the training environment. Users provide inputs and parameters rather than arbitrary code or shell access, allowing the platform
to remain an independent observer of a constrained workflow. This limits generality and leaves trust in the platform, workers, signing keys, containers, and cloud control
plane, but demonstrates a broader principle: trustworthy provenance may require constraining execution enough that evidence generation cannot simply be bypassed.
For a related approach to authenticating ML transformation history with hardware-backed attestation and transparency logs, see my notes on Atlas.
The paper shows how Thompson-class attacks aren’t restricted to compilers by
tampering with strip in the NixOS bootstrap seed. The malicious strip
repurposes PT_NOTE as PT_LOAD, and the payload propagates into later
generations of strip rebuilt from clean source. All but one binary in
stdenv ends up infected, without breaking the build or functional tests.
I walked away thinking about diverse artifact structure verification:
population-level artifact checks run from independent trust roots, with
independently built parsers and tooling. Could we come up with a stable ELF
morphology of a distribution and use that to detect a coordinated structural
shift across thousands of unrelated binaries?
For a related approach to detecting compiler subversion, see my notes on
Rosencrantz’s Diverse Double-Compiling (DDC),
including the challenges of extending verification across a real build graph.
The core argument is that coding agents have crossed the threshold where mandatory human code review is no longer economically justified. Human review consumes an
estimated 10–15% of developer time, and scaling code generation while keeping humans as the approval gate simply moves the bottleneck downstream.
Readers might find the implication for SCM architecture most interesting. GitHub/GitLab-like platforms may need first-class agent identities, cryptographically attributable
actions, specialized permissions, structured findings and confidence, and machine-readable review artifacts—not agents impersonating humans through comment threads. Agent
review also seems better described as auditable and replayable than deterministic.
This suggests code review may evolve from a human synchronization gate into continuous agentic assurance, with human inspection becoming a risk-based escalation path in
some scenarios.
The key idea in FG-Trac is the distinction between explainability and operational traceability. Influence methods such as TracInCP can retrospectively estimate how
training samples affect predictions, but they do not establish when, where, and under which model state those samples actually participated. FG-Trac instruments the ML
pipeline itself, recording sample lifecycle events and binding checkpoint-based influence scores to those events.
These records are committed through Merkle trees, with detailed encrypted logs kept off-chain and only constant-sized roots anchored on-chain. This lets a user trace a
pseudonymous sample through training, inspect its positive or negative influence, and cryptographically verify that the resulting history has not been modified.
The important limitation is that the commitment proves integrity of the recorded history, not completeness or truthfulness of its creation; a malicious operator could
bypass instrumentation before commitment. Hardware-backed attestation or TEEs could strengthen that boundary.
For a related approach combining hardware-backed attestation and transparency logs to authenticate ML transformation history, see my notes on
Atlas.
Kettle turns build provenance from an assertion into hardware-rooted evidence. It runs builds inside a measured confidential VM,
records source, resolved dependencies, toolchain, environment, and output digests as SLSA/in-toto provenance, then commits the provenance hash into the TEE attestation
report. Verification becomes an attestation check plus digest comparisons rather than trusting CI infrastructure or reproducing the build.
The interesting part is the composition: Kettle reproducibly builds its own CVM image, providing a way to derive the expected launch
measurement; uses a Merkle commitment over build inputs; and can optionally attest the CVM before confidentially delivering source. Reproducibility answers whether another
build produces the same bytes; attestation proves that one measured environment actually observed specific inputs and produced specific bytes.
The broader lesson is that attestation can move build infrastructure outside the trust boundary—but verifier policy still determines which measured builders deserve trust.
For a related approach to ML provenance, see my notes on Atlas, which combines attestation and transparency logs to authenticate a model’s transformation history.
The interesting idea here is shifting software provenance from reproducing or trusting a build to cryptographically verifying the computation itself. Reproducible
builds effectively say “repeat my computation”; TEE-based systems attest that trusted hardware performed it. COSMICTURTLE instead runs compilation inside the RISC Zero
zkVM and produces the binary plus a succinct proof binding the claimed source, compiler, and output.
The prototype runs unmodified ChibiCC and successfully zk-compiles 252 C programs, including OpenSSL and libsodium source files. Adversarial tests reject compiler
substitution, source or binary tampering, and replay. This is still far from proving an industrial GCC/LLVM build graph, but it suggests that software provenance can
be dealt with as a verifiable-computation problem rather than primarily a reproducibility or hardware-attestation problem.
aflock applies familiar ideas from SPIFFE/SPIRE, in-toto, capability systems, and policy engines to agent
execution. Rather than trusting an agent to report its own behavior, aflock moves authorization and evidence generation into an external MCP server. Agent identity is
derived from introspectable properties; short-lived JWTs convey authority; and the server signs attestations with keys the agent never sees.
Policies can constrain individual actions and cumulative properties of an execution trajectory. Session attestations are committed through a Merkle tree to provide
ordering and completeness evidence, while Rego evaluates cross-attestation constraints. Hierarchical sub-agent delegation uses in-toto-style sublayouts with mandatory
attenuation, so delegated authority cannot exceed the parent’s constraints.
The broader systems principle is compelling: securing autonomous agents may require treating execution trajectories much like software supply chains—externally
constrained, cryptographically recorded, and recursively verifiable—rather than relying on the agent itself to enforce or describe its boundaries.
I maintain Awesome Software Supply Chain Security, a popular and
comprehensive reference of tools, trends, and challenges in the software supply chain security.
I’ve built Linux distributions of various sizes, from a consumer-focused one
that has millions of end users
to custom-built Linux desktop distributions for the enterprise. From there, I transitioned to
researching how distribution package management, release cadence, and
security are relevant to the open source software supply chain problem.
Source review and reproducible builds alone cannot establish that a compiler binary faithfully implements its source. A trusting-trust attack can survive compiler upgrades
without appearing anywhere in the source tree.
Diverse Double-Compiling (DDC) addresses this by rebuilding the compiler source through an independently trusted compiler, then using that result to rebuild the compiler
again and comparing the resulting executable with the compiler under test. Assuming deterministic builds and sufficient toolchain diversity, equality provides evidence
that the executable corresponds to the reviewed source; divergence exposes either build nondeterminism, a defect, or possible subversion.
The thesis demonstrates this concretely with TCC: a compromised compiler reinfects its successor while also modifying cryptocurrency transaction code, and DDC detects the
subversion. More interestingly, attempts to extend the technique to GCC expose the practical problem: real build graphs quickly expand the trusted surface to assemblers,
linkers, loaders, OSes, firmware, and hardware.
ML provenance is not just software provenance with models added. The interesting problem is that datasets, code, configurations, weights, and execution environments are
coupled through transformations, while models can subsequently be fine-tuned or otherwise adapted. Atlas treats lineage as an authenticated chain of transformations rather
than simply an inventory of artifacts.
The individual mechanisms presented in the paper include Intel TDX, remote attestation, cryptographic measurements, C2PA manifests, and Merkle-tree transparency logs. What
I found interesting is their composition. An attestation client monitors PyTorch/Kubeflow execution, measures inputs and outputs, records runtime/configuration metadata,
and signs a transformation attestation containing hashes of precursor attestations. The transparency service makes these records tamper-evident; verification reconstructs
and validates the lineage.
More broadly, Atlas suggests that ML supply-chain integrity may require authenticating the transformation history, not merely signing the resulting model.