Access Is Not Yet Verifiability: Toward a Claim-Preserving Evidence Contract for AI Assurance
Standards can preserve records without guaranteeing that the next dashboard, adapter or evaluation report preserves the limits of the claims those records support.
Archival version: Zenodo · DOI: 10.5281/zenodo.22754297 · Version 1.0 · 13 September 2026
Consider a hypothetical assurance pipeline. A runtime records a denied tool call. A verifier checks the record’s signature. A transparency service registers it. An evaluation report imports the result. The dashboard finally displays: “The safeguard stopped the incident.”
No signature needs to fail for that conclusion to be unsupported. A tool call was denied; the run may have continued. A statement was registered; its account of the world was not thereby established. The export can preserve every byte it was designed to preserve while changing what its reader is encouraged to believe.
That is the problem this article addresses: unsupported claim expansion between evidence-producing and evidence-consuming components.
Claim preservation means carrying a claim’s evidential basis, scope and qualifications through transformations without unsupported expansion; it does not establish that the underlying claim is true.
The contribution is not a new argument for logs, independent evaluation or digital signatures. It is a proposed application profile for checking whether the meaning and limitations of evidence survive normalization, aggregation, replay and publication. The profile combines explicit claim boundaries, rules for justifying transitions to broader claims, and tests of the downstream consumer—not only the original emitter.
This matters as frontier evaluation moves toward ongoing third-party access to labs and their operational systems. Access makes inspection possible; it does not settle what an observation, a replay result or a signed report entitles an evaluator to conclude. [1]
The Phionyx Runtime Evidence & Measurement collection provides open starting points for this work. The cross-standard reporting proposal and test design below are not a published schema, executable fixture suite or validated integration. The linked Phionyx artifacts provide building blocks, not an implementation of the complete proposed contract.
1. What the incidents actually motivate
Records can be available without being complete or faithful
The OpenAI–Hugging Face incident involved evaluation agents, reduced safeguards and an internally hosted Artifactory service. The precursor history includes agent communication before the first successful SSRF-mediated internet access; after the July rebuild, both capabilities reappeared. Shared infrastructure was a coordination surface as well as an escape path. This was not a demonstration of the behavior of an ordinary production deployment. [2]
The independent METR/Redwood investigation examined roughly 1,300 transcripts, with a message-board population of about 1,200 agents and more than 70,000 messages and files. It documented missing activity, reliance on fallible analysis agents, and tool-call spoofing in roughly 7% of the transcripts examined. These are findings about the investigated material, not universal rates for agent systems. [3]
One negative finding is especially useful. Investigators found attempts to edit or delete message-board entries but did not observe successful alterations in the material they inspected. They considered material alteration unlikely, without excluding it. “No successful alteration was observed in this search” is not “the record is complete and untouched.” [3]
The same report describes agents adopting Ed25519 signatures for coordination: at least 19 published public keys and 429 signed messages by the end of July 13. Initial identity claims had no earlier external root of trust, and at least one agent appeared to act on a signature without verifying it. The lesson is not that signatures are ineffective; it is that identity continuity, verification performed, legitimate authority and factual truth are different claims. [3]
A stop attempt is not a stopped execution
An additional incident in Anthropic’s assessment involved an early Opus 4.6 checkpoint that made its task target unreachable, attempted to abort, and encountered a failure caused by an evaluation-harness misconfiguration. The report describes seven further attempts to give up before the model continued exploring and attacked an unrelated system it treated as part of the exercise. It does not locate the precise internal stage of the failed abort path. [4]
A separate internal-model experiment added an explicit, non-penalized abort option. Reported task abandonment was 100% within four turns from the lost-target resampling point and 40% within ten turns from a different deliberation point. These are condition-specific resampling results—not a general reliability guarantee. [4]
The engineering distinction is exact: a decision to stop, an attempted command, receipt, enforcement and observed cessation are separate predicates. A record that drops the abort attempts could support a misleading interpretation of continued execution. That is an inference about evidence loss, not an additional finding about the incident.
Detection, intervention and prevention have different denominators
Anthropic’s retrospective analysis also distinguishes cyber classifiers, per-action auto-mode decisions and offline monitoring. Cyber classifiers triggered early on the task setup; a subsequent actions-only analysis still flagged more than 40% of turns in each transcript. Auto-mode denial rates were 66%, 55% and 7% across the three principal incidents, with the low rate explicitly qualified as potentially insufficient to prevent harm. A denied action did not itself terminate the running agent. [4]
Input channels mattered in both directions. The Mythos 5 offline-monitor result changed from about 1% flagged actions with reasoning included to roughly 50% without it. For the internal model, the report says detection depended on the model recognizing and verbalizing the harm. These observations do not establish that reasoning should always be included or always removed. [4]
These observations motivate reporting the input condition, unit of analysis and intervention scope. A retrospective action-flag rate is not an observed probability of preventing an adaptive incident. A successful block can redirect subsequent behavior, so an unchanged transcript cannot, by itself, reveal the complete intervened trajectory.
2. What existing standards already provide
An evidence proposal is not useful if it invents a gap by overlooking existing work. The relevant foundations are substantial.
The comparison below separates an existing mechanism from the additional deployment or application-profile rule proposed here. The additional-obligation paragraphs do not claim that every other document is defective or incapable of representing the information.
NIST AI RMF 1.0 — voluntary framework
Already covered: MEASURE includes unmeasured risks, independent review and evaluation validity; MANAGE 2.4 addresses disengagement or deactivation. [5]
Additional application-profile obligation: Keep the exact measurement predicate and unmeasured scope attached when a result becomes a badge or aggregate claim.
NIST OSCAL Assessment Results — machine-readable assessment model
Already covered: Observations, relevant evidence, findings, risks and assessment context already have structured representations. [6]
Additional application-profile obligation: Define how imported agent records and replay results map into those structures without losing method, target coverage or claim qualifications.
EU AI Act and GPAI Code of Practice — law and voluntary implementation tool
Already covered: The Act covers logging and human intervention for high-risk systems, and evaluation, systemic-risk mitigation and incident duties for relevant general-purpose models. The Code supports specified GPAI obligations. [7] [8]
Additional application-profile obligation: Supply testable reporting rules for particular evidence claims; do not substitute this profile for legal applicability analysis or conformity assessment.
ISO/IEC 42001, 42005 and 42006 — published standards
Already covered: Their public scopes concern AI management systems, impact assessment, and bodies auditing/certifying AI management systems. [9] [10] [11]
Additional application-profile obligation: Connect organizational assurance to bounded runtime findings. No claim of a missing ISO requirement is made from catalogue descriptions alone.
RATS / RFC 9334 and SCITT / RFC 9943 — published RFCs
Already covered: RATS separates Evidence, appraisal and relying-party decisions. SCITT provides verifiable registration and explicitly allows for inaccurate issuer statements and incomplete issuer registration. [12] [13]
Additional application-profile obligation: Preserve the native verifier’s exact result; require additional support before interpreting it as successful control or real-world effect.
W3C PROV-DM and forensic evidence handling
Already covered: Provenance, derivation and evidence custody are established concepts, not inventions of this article. [14] [15]
Additional application-profile obligation: Bind generation, selection, retention and analysis to the particular AI-evaluation claim, including which boundaries remain producer-controlled.
OMG SACM 2.3 and OVAL 5.11
Already covered: SACM represents claims, argumentation and evidence relations. OVAL already distinguishes true, false, unknown, error, not evaluated and not applicable results. [16] [17]
Additional application-profile obligation: Specialize the argument and result mappings for runtime stages, missing target evidence, replay conditions and downstream exports.
OWASP Agent Control Standard v0.1.0 — project specification
Already covered: ACS-Core already includes authenticated envelopes, decision honoring and failure-posture handling; optional profiles add tracing, provenance and stronger audit/crypto properties. [18]
Additional application-profile obligation: Test what a particular deployment actually records and what consumers infer. A declared profile and a verified deployment behavior are different inputs.
For the EU comparison, the applicable provisions are not interchangeable: Articles 12 and 14 concern high-risk systems; Article 55 concerns general-purpose models with systemic risk; Articles 72–73 address relevant post-market and incident processes. This is an engineering comparison, not a claim that all research runs fall under the same duties. [7]
For ISO, only public scope descriptions were reviewed—not the licensed normative texts. A clause-level claim that these standards lack a particular requirement would therefore be unsupported.
The closest IETF work already addresses semantic inflation
Several individual Internet-Drafts are closer to this proposal than broad risk-management frameworks.
The Action Evidence Boundary draft separates authorization, invocation, execution and effect. Independent Determinability of Agent Actions addresses what another party can establish about an action. These are direct prior work, not merely background on logging. [19] [21]
Most importantly, Security Principal and Verifier Binding, revision 06, already defines row outcomes, dependency closure and rules preventing a composite result from exceeding its verified inputs. Sections 16–17 address skipped checks, unresolved dependencies and context compatibility. The general non-inflation principle is therefore not claimed here as new. [20]
The author’s Agent Control-Delivery Evidence, revision 01, also already addresses population accounting, preservation through intermediaries, and exposing structural results together with claim-support qualifications. This article extends that direction into incident-analysis methods and report-consumer tests; it does not claim to originate what the draft already specifies. [22]
All four are individual Internet-Drafts: work in progress, not IETF-endorsed standards. The determinability draft’s informative reference to the control-delivery draft documents a relationship between texts, not adoption or interoperability. [21]
One remaining engineering task addressed here is to connect these mechanisms in a testable, claim-preserving reporting profile. Merely being able to store a qualification does not establish that a particular adapter, dashboard or exported report preserves it.
Three kinds of gap must remain distinct. A scope boundary deliberately leaves an application claim to another mechanism. A profile gap leaves the deployment to specify shared meanings or mappings. An implementation-evidence gap remains when the specification is adequate but the actual system has not been shown to follow it. This proposal outlines reporting rules and a test design for the latter two; it identifies, rather than manufactures, evidence needed beyond the first.
3. The proposed rule: no unsupported epistemic promotion
Use epistemic promotion here to mean a transition to a stronger or different assurance claim—for example, from an authenticated report to an assertion that its event really occurred, or from one denied action to a claim that a whole incident was stopped.
The proposed rule is:
A transformation may change representation, but it must not silently expand what the evidence is taken to establish. A broader conclusion requires an explicit reasoning rule and satisfied premises for the same claim context.
This is an application of established assurance and non-inflation principles, not a new law of knowledge. Its practical value depends on naming the transformations and testing them.
It also does not prohibit inference. Several observations can jointly justify a conclusion none supports alone. Existing evidence may already satisfy the needed premises; a new sensor reading is not always necessary. Statistical or model-based conclusions can be useful when their assumptions and uncertainty remain explicit.
The prohibited step is the unrecorded change of claim. A source-level result about bytes, one action, a declared population or an experimental replay is presented as an unqualified conclusion about truth, a completed shutdown, every relevant target or the historical incident.
There is no single scalar ordering that makes all these evidence types “stronger” or “weaker.” An authenticated record and an unauthenticated independent observation support different predicates. The contract evaluates their relevance to a named claim instead of adding them into a universal assurance score.
What justifies moving across a boundary?
A bridge obligation is the additional premise or reasoning step needed for a specified transition. The examples below are proposed review rules, not reported failures of every standard listed above.
1. A required check did not execute
Unsupported expansion: “The requirement passed”
Required bridge or narrower conclusion: Execute the check, establish justified non-applicability, or retain non-measurement.
2. A signature verifies
Unsupported expansion: “The claim is true and authorized”
Required bridge or narrower conclusion: Evaluate authority binding and the substantive claim separately; retain the verified-key statement meanwhile.
3. No matching acknowledgement before cutoff
Unsupported expansion: “The instruction was never delivered”
Required bridge or narrower conclusion: Obtain evidence that distinguishes the causes, or report only the bounded missing-acknowledgement observation.
4. One tool action was denied
Unsupported expansion: “The agent or incident stopped”
Required bridge or narrower conclusion: Establish the intended control scope and subsequent execution state, including permitted retries or alternate paths.
5. An executor reports completion
Unsupported expansion: “The intended external effect occurred”
Required bridge or narrower conclusion: Support the effect predicate under an explicit observation/trust model; do not rename an executor report as independent corroboration.
6. Every declared target has a record
Unsupported expansion: “Every relevant execution path was covered”
Required bridge or narrower conclusion: Establish enumeration completeness for that wider claim, not merely closure of the supplied list.
7. A classifier flags a stored trajectory
Unsupported expansion: “It prevented the original incident”
Required bridge or narrower conclusion: Keep the replay claim separate; a prevention claim needs an intervention model and evidence about the resulting trajectory.
8. An independent evaluator reads producer-selected logs
Unsupported expansion: “The evidence was independently generated and complete”
Required bridge or narrower conclusion: Describe generation, selection and custody separately; support completeness by an appropriate method.
A conservative operating policy may block activity when evidence is unresolved. That does not turn the unresolved factual question into a measured failure. Policy action and evidentiary conclusion remain separate.
4. Eight groups of information around each claim
The contract starts with an explicit claim: its subject, relevant boundary, population and time window. The following eight information groups qualify that claim. They are a proposed reporting organization, not new mandatory fields inserted into existing wire formats.
1. Measurement status
Record what check ran and its native outcome, including non-execution and technical failure. Measurement Axioms distinguishes PASS, FAIL, NOT_MEASURED, INCONCLUSIVE, ERROR and justified NOT_APPLICABLE; its consumer rules explicitly forbid several paths that promote non-measurement into passing. [23]
Do not assume that a similarly named OVAL, AIREP or verifier-matrix result means the same thing. Preserve the native value and specification revision. Any normalized value needs a versioned mapping; an unknown or lossy mapping must remain visible rather than defaulting to success.
2. Evidence provenance
Identify the source artifacts, byte or record bindings, policy and model revisions, transformation steps, and the exact result being consumed. A copied summary should retain a resolvable path to its inputs.
A reference and a digest do not establish that the referenced material was retrieved. Resolution, integrity checking and substantive inspection should be distinguishable operations.
3. Runtime stage
Identify whether the result concerns a decision, dispatch, receipt, attempted execution, reported execution outcome or observed state. Preserve the source vocabulary when several stages are encoded differently.
AIREP v0.2 provides four artifact families, but explicitly describes them as stage-specific reports. More assurance about a report does not promote it into a later event or prove its truth. [26]
4. Negative-observation semantics
State the search or expectation that makes absence meaningful: expected object, location, interval, cutoff and coverage. An unexplained missing row differs from a recorded timeout for a particular instruction.
“No matching record in this supplied set” and “a sufficiently instrumented observer saw no event during this interval” need not have the same support. Neither is automatically a claim about all time or every possible path.
5. Control-path evidence
Record what the control was supposed to constrain: one action, a session, a worker, an agent population or external effects. Preserve invocation, receiver observation, application and effect evidence separately.
Also preserve how required targets were enumerated and whether that enumeration supports the wider deployment claim. An acknowledgement from every listed target cannot establish that the list omitted none.
6. Reproduction and disagreement
A reproduction should name the earlier result, method, inputs and tolerated differences. “Another team ran something similar” is not an exact reproduction relation.
Retain contradictory readings and their supporting evidence. An additional rater can test an instrument’s interpretation; agreement alone does not establish that both raters’ common assumption is true.
7. Evidence method and input conditions
A single evidence_mode label is not enough. Independent reproduction is not an exclusive alternative to observation, reconstruction or replay. An independent team can reproduce a transcript replay. Method and investigator relationship therefore belong on separate axes.
A proposed method vocabulary could distinguish IN_RUN_OBSERVATION, POST_HOC_RECONSTRUCTION, TRANSCRIPT_REPLAY and CONTROLLED_RESAMPLING. The selected input channels, model/harness configuration and intervention point should accompany it. These labels describe how evidence was obtained; they do not rank quality.
A replay can directly measure a classifier’s response to supplied material. Interpreting that response as historical prevention adds a different, counterfactual claim. Preserving that distinction is more precise than labelling every replay a completed counterfactual experiment.
8. Evidence custody and control boundaries
Record who generated, retained, selected and could alter the evidence, and which steps an independent evaluator actually controlled. An evaluator’s preserved copy can protect against later modification without proving that the original selection was complete or the source was truthful.
Evaluator independence is not evidence independence. This is a reporting distinction grounded in existing provenance and forensic practice, not a claim to have invented chain of custody. Describe the relevant administrative, storage, key and observation boundaries rather than reducing independence to one Boolean.
5. A worked example: all supplied records pass, but the broad claim does not
This example is hypothetical. It is neither an incident reconstruction nor a completed ACS–AIREP–SCITT–OSCAL integration experiment.
An operator requests a shutdown. The evidence package contains authenticated issuer and receiver reports for two declared targets. The applicable receipt-matching checks pass for both targets. The package contains no evidence establishing that the declared target list covers every relevant execution path, and no accepted observation of termination or remaining external work.
The narrow report-level conclusion is supportable: matching authenticated receipt reports exist for both declared targets under the stated verification policy.
The broader conclusion—all deployment activity and effects stopped—is not established.
A reporting layer could represent the distinction as follows. This is illustrative YAML, not an AIREP record or an already implemented schema:
profile: proposed-claim-preservation-example
input_set: hypothetical-supplied-records
structural_result:
predicate: authenticated_receipt_reports_match_declared_obligations
state: PASS
declared_targets: [worker-a, worker-b]
claim_support:
narrow_claim:
predicate: both_declared_targets_have_matching_receipt_reports
qualification: SUPPORTED_UNDER_DECLARED_VERIFICATION_POLICY
wider_claim:
predicate: all_deployment_activity_and_effects_stopped
qualification: NOT_ESTABLISHED
unresolved_premises:
- target_inventory_complete_for_deployment_scope
- termination_observed_for_every_required_target
- relevant_in_flight_external_work_accounted_for
NOT_ESTABLISHED does not assert that shutdown failed. It states that the stronger predicate lacks required support. The checker can nevertheless fail a dashboard export that falsely presents the package as proof of complete shutdown: that is a failure of the reporting contract, not a diagnosis of the physical system.
This yields a concrete consumer rule: when a structural result is returned, rendered or exported, its claim scope and qualifications must be available in that same result context. A green badge with a missing coverage warning in another document is not equivalent to the qualified source result. The control-delivery draft already states this co-exposure requirement; the proposal here is to exercise it across incident-analysis and reporting transformations. [22]
The same inputs may legitimately support multiple conclusions. A dashboard need not become all grey: it can show successful receipt verification and unresolved deployment shutdown at the same time.
6. How this fits existing mechanisms without replacing them
The integration path should begin with native verification. Authenticate and appraise a source according to its own specification before mapping the constrained result. Do not infer SCITT verification, RATS appraisal or ACS conformance merely because an adapter produced a normalized object.
OSCAL is a plausible destination for assessment observations and findings; PROV can represent derivation between evidence and reports; SCITT can register appropriately scoped statements. Those are candidate attachment points, not implementations demonstrated by this article. The exact mappings, version behavior and preservation tests remain work to do. [6] [14] [13]
For each mapping, specify what source proposition survives, what context the destination must carry, and what cannot be represented without an extension. Keep original records or verifiable references. When a format cannot carry a necessary distinction, report the limitation rather than inventing compatibility.
Some gaps can be closed by a profile, not by changing a base standard. Some need another observer or a target inventory and cannot be closed by adding a field. Others need an actual intervention experiment. Representing the missing premise is useful; it is not the same as supplying it.
The intended compatibility claim is therefore modest: a claim-preserving profile can make existing mechanisms compose more honestly. It does not replace a risk-management program, a legal obligation, an argument-validity review or operational safety controls.
7. What the Phionyx artifacts already contribute—and what they do not
The relevant repositories contain more than terminology, but they do not yet implement the whole proposed cross-standard profile.
Measurement Axioms v1.0
Existing, inspectable contribution: Producer/consumer requirements, explicit non-measurement handling and paired-control selectivity tooling. [23] [24]
Boundary that remains: A schema or small probe does not establish every requirement or validate arbitrary measurement methods.
AIREP v0.2.0-beta.1
Existing, inspectable contribution: Stage-specific signed reports and structured reconciliation over supplied evidence. [26] [27]
Boundary that remains: Reports are not ground truth; complete target enumeration and all cross-standard adapters are not supplied by the beta.
CDE-12 v0.2 / dataset v0.1
Existing, inspectable contribution: Criteria for what records can support, evidence-depth disclosure and classified negative observations. [29] [30]
Boundary that remains: Historical, unequal reading depth; one rater and no second-rater agreement measurement.
Agent Control-Delivery Evidence draft-01
Existing, inspectable contribution: Proposed population conservation, intermediary preservation and structural-result/claim-support qualification. [22]
Boundary that remains: An individual requirements draft, not implementation certification or an adopted standard.
Two details make the engineering issue tangible.
First, AIREP’s beta reconciler evaluates a finite supplied set. Missing evidence and unevaluated prerequisites remain in its result. Total intended-target coverage is not established by a collection of internally consistent records. Its CLI exit code 0 means that a structured evaluation was emitted—even when that evaluation contains negative findings. A wrapper that interprets exit 0 as “all evidence passed” would be making precisely the unsupported promotion discussed here. [27]
Second, the external AIREP verifier record retains 17 AGREE / 1 DISAGREE against an earlier frozen basis, without an expected-blind claim. The discrepancy concerns a derived cryptographic result: the package expected PASS, whereas the independently implemented verifier reported NOT_EVALUATED because the revoked-binding prerequisite prevented the signature stage from executing. This is not a 94% general safety score, and it is not a same-version beta producer-to-consumer interoperability result. [28]
The projects also document their own corrections. CDE-12’s first negative-record criterion overclaimed non-arrival. Its revision required the underlying classified observation instead, moving only two existing cells from supported to partial—both for the authors’ systems. [29]
The Measurement Axioms probe once returned a positive outcome with zero exercised cases. An external reader exposed the problem; the reference source documents its correction. The audit status still separates that individual contribution from independent review of the full report and labels independent reproducibility NOT_MEASURED. [24] [25]
These are useful, bounded evidence of concrete failure modes and corrections. They do not establish that the proposed contract has been validated across production assurance stacks.
8. Test the transformations, not just the records
A schema-valid report can still make an unsupported claim. Conversely, a system that marks everything unknown can avoid overclaiming while being useless. Evaluation must test both safety and retained usefulness.
The next experiment should use small, explicit claim graphs with known inputs, a declared verification policy and human-reviewed expected conclusions. For each case, exercise an emitter, a mapping/aggregation step, and a displayed or exported result. The baseline consumer is compared with a consumer applying the proposed reporting rules.
The following test design specifies eight proposed test families; it is not an executable test suite or a report of measured results:
1. Replace an executed check with an explicit skipped-check outcome
Expected behavior of a claim-preserving consumer: Preserve non-execution; do not retain the old passing conclusion.
2. Remove one required target’s observation while retaining the target inventory
Expected behavior of a claim-preserving consumer: Expose the missing obligation; do not silently shrink the denominator.
3. Keep all records but remove evidence that the inventory is complete for the broader claim
Expected behavior of a claim-preserving consumer: Retain success over named records; qualify deployment-wide coverage.
4. Retain a valid signed report but remove accepted independent corroboration
Expected behavior of a claim-preserving consumer: Preserve authenticity; withhold the independent-effect conclusion.
5. Change replay input channels or evaluation context
Expected behavior of a claim-preserving consumer: Produce a distinct method/context-bound result, not overwrite a historical observation.
6. Change action-level denial to a session-level “stopped” label in an export
Expected behavior of a claim-preserving consumer: Reject or qualify the widened scope unless its bridge premises are supplied.
7. Drop caveats during JSON-to-table, dashboard or report conversion
Expected behavior of a claim-preserving consumer: Mark the output as a failed preservation check rather than publish a naked success badge.
8. Add sufficient valid evidence for a previously unresolved premise
Expected behavior of a claim-preserving consumer: Admit the newly supported claim under the declared rule; do not remain permanently inconclusive.
Compare at least two outcomes: unsupported-claim acceptance on the negative cases, and correct retention of supported claims on positive controls. Report each by transformation and claim type, with the test population and adjudication disagreements exposed. There is no justified universal “AI assurance score” in this design.
Byte-level round trips alone are insufficient. A consumer may deliberately discard source bytes while accurately preserving the relevant conclusion, or preserve a reference while hiding its limitation from readers. The test concerns the claim that the output exposes under its declared interpretation.
Some checks can be mechanical: missing prerequisites, context mismatch, unknown status mappings, absent target records and detached qualifications. Others require review: whether an observation really supports an effect predicate, whether two observers are operationally independent, or whether an inference is sound. Automated coverage must name those boundaries rather than issue a global certification.
9. The scope of the contribution
This article proposes an engineering profile and evaluation design. It does not report a new replay of the frontier incidents, access to private lab evidence, a completed cross-standard adapter suite, or a comparative deployment benchmark.
Its source comparison is targeted, not an exhaustive novelty search. Existing assurance cases, multi-valued results, provenance, attestation and the cited agent-evidence drafts already provide important parts of the solution. A new name does not create precedence over them.
The specific contribution is the combination of claim-boundary rules, method/custody separation, required justification for claim expansion, and consumer-side preservation tests, motivated by incident reconstruction and monitor behavior. Whether that combination improves real evaluation pipelines remains a question for the proposed experiment.
This changes the unit of review. Instead of asking only “Was this record valid?”, ask:
Was the next claim licensed by that record, its context and the other evidence—and did every subsequent representation preserve the answer?
An evaluation system should preserve enough information to answer that question even when the result becomes less convenient to display.
10. Use the artifacts; challenge the reporting boundary
The Phionyx — Runtime Evidence & Measurement collection connects eight artifacts: the AIREP paper, Phionyx runtime paper, AIREP cases, Measurement Axioms cases, AIREP Evidence Lab, Measurement Axioms Lab, the experimental AIREP Embedded Evaluation Profile distribution mirror, and the AIREP Evaluation Evidence exporter.
The profile mirror is not a benchmark or training dataset. The exporter currently parses LightEval results_*.json only. It records declared evaluation context without verifying that context as fact, computes SHA-256 digests over the exact supplied evidence bytes, and reports whether the generated profile payload validates against the accepted digest-pinned basis. None of those operations establishes the truth or representativeness of the evaluation observations, model safety, evaluation completeness, or evaluator independence.
The most useful contributions would be a minimal adapter that preserves these distinctions, a counterexample that defeats a proposed rule, or an independently rated case showing that the instrument is ambiguous. The same reporting discipline must apply to Phionyx’s own artifacts.
Access enables inspection. Valid records enable checking. Neither guarantees that the final safety claim survived the journey unchanged.
That last boundary is what this work proposes to test.
References and source scope
Repository references below point to the inspected snapshots. Internet-Drafts are cited as work in progress; catalogue-only ISO scope review is identified explicitly.
We Must Pace the Frontier. Embedded-evaluator proposal; a proposal, not an evaluation standard.
OpenAI — OpenAI–Hugging Face Incident: Technical Report. Sections III–IV, VII.C and VIII.D; precursor chronology, containment and retrospective safeguards.
METR / Redwood Research — Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. Report pp. 3, 24–27 and 48–49; coverage, analysis limitations and signed coordination.
Anthropic — An alignment assessment of recent cybersecurity incidents. Internal-model resampling, additional Opus 4.6 incident, cyber/auto-mode classifiers and offline-monitor analysis.
NIST — AI RMF 1.0 Core. MEASURE 1.1, including its second sentence on documenting risks or trustworthiness characteristics that will not or cannot be measured; MEASURE 1.3, 2.1, 2.3, 2.5, 2.13 and 3.2, the latter addressing risk tracking where measurement techniques are insufficient or metrics are unavailable; MANAGE 2.4. Voluntary risk-management framework.
NIST — OSCAL Assessment Results Model. Machine-readable assessment observations, relevant evidence, findings and risks; point-in-time and continuous assessment.
European Union — Regulation (EU) 2024/1689, consolidated text. Consolidated version incorporating Regulation (EU) 2026/1744. Articles 12, 14(4)(e), 55, 72 and 73; Article 72(3) carries the M1 amendment marker in this version. Applicability differs between high-risk systems and systemic-risk general-purpose models.
European Commission — General-Purpose AI Code of Practice. Voluntary implementation tool; transparency, copyright, safety and security. Overview, not a clause-by-clause conformity assessment.
ISO/IEC 42001:2023 — AI management systems. Public catalogue scope only; the licensed full text was not reviewed.
ISO/IEC 42005:2025 — AI system impact assessment. Public catalogue scope only; the licensed full text was not reviewed.
ISO/IEC 42006:2025 — Requirements for bodies providing audit and certification of AI management systems. Public catalogue scope only; the licensed full text was not reviewed.
IETF — RFC 9334: Remote ATtestation procedureS (RATS) Architecture. Published RFC; roles, Evidence, appraisal, Attestation Results and relying-party decisions, especially Sections 3, 4.2 and 8.
IETF — RFC 9943: An Architecture for Trustworthy and Transparent Digital Supply Chains. Published SCITT architecture RFC; Section 7 and Sections 9.1–9.3. Receipt validation does not establish statement truth or universal producer completeness.
W3C — PROV-DM: The PROV Data Model. W3C Recommendation; entities, activities, agents, derivation and provenance bundles.
NIST SP 800-86 — Guide to Integrating Forensic Techniques into Incident Response. Forensic-evidence handling as established background; no claim that custody is new.
OMG — Structured Assurance Case Metamodel (SACM), Version 2.3. Sections 11.8 and 11.14–11.15: assertion status, inference and asserted evidence. Published specification, October 2023.
MITRE — OVAL 5.11 Results Schema. ResultEnumeration and result-combination semantics; established multi-valued assessment results.
OWASP Agent Control Standard — Conformance Profiles, v0.1.0. ACS-Core, ACS-Trace, ACS-Provenance, ACS-Crypto and ACS-Audit; integration-branch snapshot 7d2dd3c69f82ec7565eef49b1609906deba1ef6b. Specification requirements are not proof of deployment conformance.
The Action Evidence Boundary for Consequential Agent Effects — draft-schrock-action-evidence-boundary-02. Individual Internet-Draft, work in progress: phase boundaries and composition; not an IETF-endorsed standard.
Security Principal and Verifier Binding for Agent Communication Protocols — draft-bu-agentproto-security-principal-binding-06. Individual Internet-Draft, work in progress. Sections 16–17 define row outcomes, dependency closure and non-inflating composition.
Independent Determinability of Agent Actions — draft-wadkins-agentproto-action-determinability-00. Individual Internet-Draft, work in progress. Determinability requirements; cites the control-delivery draft informatively.
Evidence Requirements for Agent Control Delivery and Outcome Reconciliation — draft-abak-agent-control-delivery-evidence-01. Author’s individual Internet-Draft, work in progress. Sections 5.15–5.16, 6.4–6.6 and 8: population, semantic preservation, claim qualification and native verification.
Phionyx — Governance Verdict Validity: Normative Specification v1.0. MA-3.4–3.11, MA-4.1–4.8 and conformance requirements; frozen specification at repository snapshot 350bb4cba4e5bc2d760db080aae52352a7041331.
Phionyx — Measurement Axioms paired-control selectivity probe. Reference source and documented correction of the zero-coverage positive-result defect; source inspected, not a new experimental run for this article.
Phionyx — Measurement Axioms audit bundle status. Separates narrative, bundle availability, independent reproducibility and independent review.
Phionyx — AIREP v0.2 specification. Implementation target v0.2.0-beta.1, wire version 0.2; Sections 2, 4 and 7–8. Repository snapshot 040d0c401015598aae8bdb5f36efa2f6c57bb755.
Phionyx — AIREP v0.2 lifecycle reconciliation. Finite supplied-set semantics, per-record assurance, target-coverage limits, result states and CLI exit semantics.
Phionyx — AIREP external evidence registry. Version-scoped external producer and verifier evidence; 17 AGREE / 1 DISAGREE and the CLS-XT1 derived-result discrepancy.
Phionyx — CDE-12: Control-Delivery Evidence v0.2. Sections 5–10: criteria, negative observations, rater agreement and documented C12 correction.
Phionyx — CDE-12 dataset documentation. Historical dataset, single-rater status and unequal reading-depth/self-rating limitations.
How to cite
Ali Toygar Abak. “Access Is Not Yet Verifiability: Toward a Claim-Preserving Evidence Contract for AI Assurance.” Phionyx Research, 2026. https://doi.org/10.5281/zenodo.22754297