When an AI starts citing itself

I tested whether explicit claim dependencies help an AI handle its own interpretation returning as evidence. In this small pilot, competent notes worked just as well.

An AI writes that a release has passed its security review. An operations update repeats it. A project summary repeats it again. On its next task, the AI finds three records saying the release is ready. But the two later records came from its own earlier interpretation.

Those records tell us that the interpretation has circulated. They do not provide three independent reasons to believe the review happened. A system can remember every sentence and still lose that distinction.

I’m building Sho, an AI system intended to help people run a company. One question within that work is whether maintaining explicit relationships between claims improves what an assistant does when information changes. This pilot tests a narrow version: can an inherited interpretation survive correction by returning through other company records?

One episode, from interpretation to handoff

The records below are fictional. The initial AI interpretation was deliberately supplied, rather than produced by an earlier model call. This tests how a system handles an inherited claim; it does not measure how often an AI creates a mistaken interpretation in the first place.

The later calls received retained memory and the new record, without the earlier raw documents. The correction therefore had to survive in what the assistant carried forward. An unrelated Friday handoff deadline stayed valid throughout; revising the review status should not erase that separate fact.

All three systems made the same decisions in this episode. The complete notes record, structured-beliefs record, and maintained-dependencies record include their outputs and retained state. This example was selected by case order, before the full comparison finished.

Proposition: Aster release has passed the required independent security review. Evidence status: authoritatively rejected; 0 currently usable independent direct supporting verifications. R3 (Security review owner, authoritative determination, 2026-01-07 09:00Z) declares the proposition FALSE for the same scope and cutoff, says the requirement is not satisfied, and supersedes R0, R1 and R2. Earlier positive interpretations are unusable as current verification.
Source history: R0 (Earlier AI assistant, 2026-01-05 09:00Z) was the supplied AI interpretation without underlying verification. R1 (Operations analyst, 2026-01-06 10:00Z) and R2 (Project coordinator, 2026-01-06 11:00Z) derived from R0 and explicitly performed no fresh verification. R4 (Weekly digest generator, digest, 2026-01-08 09:00Z) copied the positive proposition from R0 and explicitly conducted no new verification. R4 is a derivative republication of superseded evidence, not independent support or a correction of R3. Its later publication does not make R0 current. Prior insufficient evidence/hold became authoritative rejection/do_not_proceed with R3 and remains so after R4.
Action eligibility depends on the proposition's evidence status: stating the proposition as an established fact in the weekly handoff is ineligible; decision do_not_proceed. Previously requested marking of the release ready on the internal checklist also remains ineligible.
Policy: proceed requires at least two independent current direct supporting verifications or an explicit current authoritative positive determination from the owner. Insufficient evidence requires hold; current authoritative rejection requires do_not_proceed. An authoritative determination counts as one verification toward the position it establishes; R3 provides no supporting verification. Never count the supplied AI interpretation or derivative repetitions as direct verification. Authoritative corrections or confirmations supersede named older records. Reassess the proposition and dependent action eligibility when relevant evidence changes.
Separate handoff_deadline: Friday at 16:00 UTC remains supported by R5 (Scheduling owner, authoritative determination, 2026-01-05 08:00Z), directly verified against the current team schedule. R5 is independent of the proposition and R0/R1/R2/R4 and is not superseded by R3. The deadline contributes nothing to the proposition support count. Revise this separate conclusion only if relevant deadline evidence changes.

What changed between the systems

The baseline could keep competent notes, including source IDs, uncertainty, correction history and dependencies. It was not restricted to a vague summary. The second system stored explicit claim records. The third used the same records plus a small deterministic helper: when a source was superseded, it marked the affected claims and their dependents for review. The helper did not decide whether a claim was true.

Each system received the same records, provenance and decision instructions, with the same model and resource limits. Maintaining memory used the same three-call allowance. Notes and structured state had the same 5,000-byte ceiling. The comparison changes the representation and, in the third system, its maintenance; it cannot isolate an abstract effect of “having beliefs.”

Two scenario families were used for development. Four were held out: release review, support coverage, sensor calibration and backup deletion. Each crossed a true or false initial claim with copied or independently obtained support. Three repetitions across three systems produced 144 held-out episodes, using 432 model calls.

The held-out run was a tie on every behavioral measure. Each system completed 48 valid episodes. All chose the expected decisions, distinguished copied from independent support, retained revisions on the later task and preserved the separate deadline. None abstained unnecessarily. The dependency helper was active: it marked a dependent claim at the revision in all 48 of its episodes.

Counts are errors / eligible valid episodes. Revision includes corrections and confirmations; the last column includes only initially false claims. Full scoring separates decision errors from support-count errors and reports invalid runs separately.

Notes also used less retained state and fewer output tokens in this run. Average state size was 2,124 bytes for notes, 2,634 for structured beliefs and 2,420 with maintained dependencies. That reflects these particular implementations and their fields; it does not establish a general efficiency ranking.

A reference the code could not follow

Development exposed a more concrete problem than a persuasive-looking answer. A structured state contained a source like R1: Operations analyst; …. The helper expected the exact key R1. A person could recognize the reference, but the helper did not match it and missed the dependent claims.

The model still made the correct decisions in that episode. Checking the final answer alone would have missed the broken maintenance mechanism. The representation looked useful without being fully usable by the code that was supposed to maintain it.

Before any held-out run, both structured systems received an enforced source-ID format, unique claim keys and validation that dependencies resolve. The notes baseline received the same instruction to preserve IDs verbatim. The fixtures and scoring stayed fixed. The original development batch and frozen source are included in the download, alongside the correction and subsequent runs.

This matters if other software will act on the state. A model’s ability to read a source description cannot substitute for a reference that resolves. Decision scores and maintenance checks answer different questions; both are needed to know whether the proposed mechanism is doing anything.

What this test can tell us

The sources make the answer unusually legible. Copies announce where they came from. Independent checks announce their separate verification. A named owner explicitly confirms or rejects the proposition. Every family uses the same short sequence and a fixed decision rule. Four families with repeated variants do not stand in for months of work inside a company.

“Correct” here means following that rule using the available evidence. For example, two ostensibly independent checks initially justify proceeding, even in a case where the later audit will reveal they were mistaken. The model does not receive that future audit early. This is an evaluation of evidence handling, rather than access to hidden ground truth.

Reported support counts also do not expose the model’s internal reasoning. They show whether its answer distinguishes copies from independent support. The pilot tests no live company data, customer actions or Sho component.

Tracking origins is established work: the W3C PROV family describes derivation and attribution, and memory systems such as Hindsight and Zep document ways to maintain information as evidence changes. A new format needs to earn its place through behavior, maintenance cost or inspectability.

What changes for Sho

This pilot gives me no behavioral evidence for requiring a separate belief structure. Given explicit origins and sufficient memory, the model preserved the relevant distinctions in prose. Competent notes should remain the baseline for this part of the design.

The next question comes earlier: how does the system establish where a claim came from when a company update no longer announces that it copied an AI note? Adding a dependency field cannot recover a relationship nobody recorded.

I would next test records with recoverable publication history but without the convenient source labels used here, giving every system the same evidence. The task would be to establish the shared origin before deciding whether two records corroborate each other. That test has not been run.

For now, there is a working comparison, a strong baseline and a specific maintenance defect that a correct final answer failed to expose. Those are concrete constraints on what to build next.

Protocol and materials

The pilot uses gpt-6-astra at medium reasoning through Codex 0.153.1, with tools disabled. The serving model name is not an immutable model snapshot, so a rerun may produce different outputs. All model calls, retained states, deterministic scores and recorded usage are available below. There are no live model calls on this page.

• Download the complete pilot — Python source, fixtures, protocol, frozen hashes, all outputs and scoring.
• Read the protocol · Results JSON · Runner source

python3 verify-company-evidence.py \
  --experiment /path/to/extracted \
  --package /path/to/company-evidence-pilot.zip

cd ~
python3 /path/to/extracted/run.py prepare --output /path/to/new-results
python3 /path/to/extracted/run.py development --output /path/to/new-results
python3 /path/to/extracted/run.py heldout --output /path/to/new-results