dsh-compaction-audit

dsh-compaction-audit

Compaction quality / hallucination detector. When a conversation span is compacted (a single compaction/summary event with a summary ContentBlock[] shadowing a range of original events), this plugin checks on two axes that key information survived into the summary:

• FIDELITY (hard): every "verifiable atomic assertion" the summary writes (UUID / absolute path / file path / number-with-unit / user correction) must be found verbatim in the original leaf events; if not, it is flagged unverified (a candidate — not proof of fabrication).
• COVERAGE (weighted): key atomic facts present in the original leaf events but not asserted in the summary are flagged lost.
• COMPLIANCE (layer A, deterministic, zero-LLM, optional): a literal word-list check (keep/drop/scope) for "did the compaction drop a term the user explicitly required, or resurrect a corrected-away term?" See below.

Compliance axis (layer A, deterministic, zero-LLM)

A narrow, literal (closed-set) guard over a configurable vocabulary:

• keep terms present in the source leaves but absent from the summary -> missing (a term the user explicitly required was dropped by compaction).
• drop terms present in the summary -> superseded (a corrected-away / forbidden term is present).
• scope terms present in the source but absent from the summary -> scopeLost.
• coexists_review: a keep term and a drop term both present in the summary.

Honest boundary: this is a literal (substring) check only. It detects "your pinned term's literal form is gone / a forbidden literal term appeared". It does not detect semantic paraphrase (e.g. "打分" rewritten as "评估"). All signals are candidates for human review; the plugin is read-only, never blocks, and never modifies compaction output.

Config: checkCompliance (default true), complianceVocab (JSON string {keep:[],drop:[],scope:[]}; empty/broken -> defaults), complianceFpVersion.

Design invariants (red/blue-team consensus)

1. Deterministic CLOSED SET: only regex-definable classes are checked; never judges prose.
2. "断言即升级": once an atomic fact is written in the summary it is forced through a verbatim comparison.
3. Ground truth is ONLY the non-summary leaf events reachable via the provenance ledger (session.events[seq], resolved at the moment the summary lands — verified readable by dsh-compaction-probe).
4. Read-only signal producer: writes an audit.signals record; never injects LLM-synthesized content; any re-injection is opt-in and default OFF.
5. "Never block" is a mechanism: everything is try/catch'd, handlers are isolated, failures are swallowed.

Semantics & limits (read before use)

• unverified is a CANDIDATE, not "fabricated". It can be a real fabrication OR an artifact of ledger incompleteness (ledgerIncomplete), a non-closed-set expression, or a fact outside the visible ledger.
• This is a closed-set hard-fact lower-bound signal, NOT a general narrative hallucination detector. A summary that invents a plausible-but-wrong story with no hard fact is not caught.
• Thresholds (criticalNumberCount, PATH_MARKERS, maxFactsPerCategory) are defaults pending calibration against an N=50 labeled set before use as a CI gate.

See README.zh.md for the full Chinese walkthrough (config table, sample audit.signals record, and note that lib/index.js is the loaded artifact and src/index.ts the maintainable TS mirror).