- Главная
- Плагины
- Процессы и автоматизация
- dsh-jev-preset
dsh-jev-preset
yum-wu/dsh-jev-preset
JEV agent preset bundle with prompt-guided task routing, cross-verification and quantitative checklists.
Установка
dsh plugin --profile web add github:yum-wu/dsh-jev-presetREADME
dsh-jev-preset
English | 简体中文
An agent preset bundle for DeepSeek Harness (DSH).
It does one thing: before the model commits to a numeric or risk-control answer, it runs real code once and checks the number.
What it fixes
Single-session models fail on quantitative work in three ways:
- Wrong but fluent. Bad numbers get wrapped in smooth reasoning that survives human review.
- Autoregressive self-contamination. Asking one session to "analyze from three perspectives" anchors later tokens on earlier ones — not real multi-path.
- No physical ground truth. A formula that never ran in an interpreter should not ship.
- "Start with 20 cases" is a runnable module, not a promise —
small_sample.pyemits a pre-registration block (case_ids+sha256+ coverage) that any third party can rebuild from(suite, n, seed)alone. - Admission rules must stop new offenders, not just list a one-off fix — T3 is now a ratchet: the legacy list is frozen, new offenders fail, and fixing a listed file also fails (the list only tightens).
Install
A standard DSH bundle: an npm package carrying one cordis.patch.yml layer, mounted into the profile's loader tree.
dsh plugin --profile <your-profile> add dsh-jev-preset
Verify the layer landed without booting:
dsh --profile <your-profile> --dump-config
You should see a # == dsh-jev-preset layer. Then boot — JEV Cross-Verification appears in the preset list.
Remove the same way: dsh plugin --profile <your-profile> remove dsh-jev-preset.
This bundle declares no
dependencies/peerDependenciesand hardcodes no DSH version. The declaration row shipsinject = ["agentPresets"], so the loader activates it once the registry is ready, and sub-plugin paths resolve from the host@deepseek-ai/dsh-agent-presetbaseUrl. Tested working on 0.2.0-rc.2.
Runtime half: auto reasoning effort (merged in 2026-10-13)
The persona half is prompt-only. Its 3-way sampling and structured re-derivation both assume
"pick the thinking effort by task complexity" — too low an effort and a wrong assertion goes unnoticed.
That logic used to live in a separate npm package (dsh-auto-reasoning), referenced by this bundle's
second declaration row. Since 2026-10-13 the code is in this package (lib/), declared by a
self-mounting row (id: plugin-jev-runtime, name: 'dsh-jev-preset').
| Code | lib/ — index.js (runtime plugin), auto-reasoning.js (scoring), jev-client.js (SystemOne channel), client.js (the composer pill) |
| Row | second row of the same insert group in cordis.patch.yml |
| Does | ① projects the reasoningEffort: auto sentinel onto the model's own ladder; ② counts how many paths this turn actually launched (emission ledger) |
Why merge: duplicate ids are silently overridden, not rejected. The old row was
id: plugin-auto-reasoning / name: 'dsh-auto-reasoning', and dsh-auto-reasoning is itself a bundle
whose patch inserts the same id. Measured 2026-10-13 (preflight script compose-check.mjs driving the
runtime's own loadProfileDirectory + composeEntries): the composed tree carried 2 such rows.
⚠ The 2026-10-05 comment in this repo claimed "duplicate loader entry ids stop the profile from booting". That is false on 0.2.0-rc.2:
EntryGroup.update()in@deepseek-ai/cordis-plugin-loader/src/config/group.ts:48-65builds a map viaObject.fromEntries(config.map(o => [o.id, o])), so only one entry per id is created (this.tree.store[id] ??= new Entry(...)reuses the same instance) — last one wins. A repo-wide grep forduplicate loader entry idfinds 0 hits on 0.2.0-rc.2.The real damage is silent override: bundle layers apply in
dsh.profile.bundlesorder anddsh-auto-reasoningcame after this package, so its row won and every config value written here was discarded.
Upgrading from a pre-2026-10-13 install: remove dsh-auto-reasoning from your profile's
dsh.profile.bundles and from dependencies — otherwise its own bundle layer mounts a second copy.
Removing only bundles is not enough: reconcileProfilePlugins re-adds any dependency that declares
dsh.bundle. node validate.mjs rejects the old row shape outright.
Emission ledger (2026-10-13)
Persona §三 3.1 requires "record how many paths this turn actually launched (including re-dispatches)",
and that number used to be purely self-reported — in the C3F 30-task run, 11 tasks (36.7%) dispatched
4–5 paths while still declaring 3/3. The runtime now counts the same fact: a ledger line is appended to
subagent / subagent_fork / workflow results, and exposed as emissions on
GET /api/auto-reasoning.effort (per session, reset by the next human message). Config switch
emissionLedger (default true). The grader does not check the number in the route marker, so this
is the only post-hoc audit entry point.
Gating: what happens when
The model routes on its own; no user action needed.
| Task | Handling |
|---|---|
| Everyday / low risk (lookup, Q&A, small single-file edits) | Fast-Pass, single shot |
| Numeric, risk, position sizing, liquidation price, indicator math | Single pass + real code re-check; ship only if assertions pass |
| High risk but no single numeric answer (security boundaries, concurrency, design tradeoffs) | 3-way isolated sampling |
| Any of the above, but assertion/test fails | Do not ship. Fix the assertion first; escalate to 3-way only if it can't be fixed |
When the gate can't decide (added 2026-10-13, after magpie's intent-classifier failure matrix):
The gate is a helper, not a gate — when it can't tell which path applies it never blocks delivery: take the most conservative path, and say in the answer why the gate couldn't decide. Silent degradation is a debugging disaster.
- No feature matches at all → Fast-Pass, single shot.
- High risk matched but multi-path launch is unavailable → degrade to single path, mark
[JEV: 单路未验证], and say it was capability, not a single-path verdict. - Assertion library unavailable → locate it first; if that fails, do not claim an assertion ran; use Fast-Pass or 3-way instead.
- Assertion ran but
exit 2(bad call, it never ran) → fix the command and recompute; no numeric conclusion exists at that point. - Assertion
exit 3(input insufficient) → supply the data and rerun; recomputing is meaningless. - Same instrument fails twice in a row this turn → it rests for the turn; switch to another path and say which.
This matrix adds no new route markers — it only fixes which existing marker to use when the gate can't decide.
3-way is the fallback, not the default. See the numbers below.
Measured results
All figures come from reproducible experiments under benchmarks/accuracy/.
| Method | Effect | Cost |
|---|---|---|
| Prompt structuring | +89pp (100% on 5/5 models) | ×1 |
| Execution assertion | 24/30 → 30/30, McNemar p=0.0312 | ×2.0 |
| Switch to a heterogeneous model | +89pp (some models) | ×5.4 |
| 3-way isolated sampling | Gain 0 (p=1.0) | ×10.6 |
The assertion pass is the only statistically significant positive result in this repo. Stacking 3-way sampling on top of an already-asserted answer adds nothing (30/30 vs 30/30, p=1.0) at 15.3× the cost, with a higher run-failure rate (2/12 vs 0/12).
Why 3-way can't fix reading errors: on candy trap problems, all 5 wrong answers equal the "blind" value exactly — failures sit in the information-extraction layer (missed constraints). Voting reduces variance, not bias. Calculation problems are the opposite: errors are scattered arithmetic slips, which assertions do catch.
Retracted: the 30-problem batch
runs-c30-c3-sb.jsonl(3.3% / p=1.0 / blind rate 33%) had 18/30 rows affected by a measurement bug — the runner scored before subagent settlement. The 9-problem batch is unaffected and still valid. Audit:benchmarks/accuracy/MEASUREMENT-BUG-2026-09-30.md.
Relation to published work
| Dimension | Source | Here |
|---|---|---|
| Sampling converts to performance only with a verifier | Large Language Monkeys (UC Berkeley/CMU) | assertions ×2.0 significant; 3-way ×10.6 gain 0 |
| Mindset effect causes missed constraints | MisguidedAttention; Anthropic changed its system prompt for the same failure in 2024-10 | §3.0 constraint diagnosis, +89pp |
| 85.5% sycophantic conformity in unisolated agents | arXiv:2605.00914, arXiv:2503.13657 | no answer-forward anchoring; maxDepth:1 |
| Heterogeneous models share latent entanglement; agreement ≠ independence | arXiv:2604.07650 | execution result outranks 3-way consensus |
Assertion library
When the gate decides "this can be an assertion", reuse these instead of writing throwaway scripts — reuse removes the bugs and float drift of hand-written code.
Python: packages/assertions/python/jev_assertions/
tick.py— tick truncation, rounding direction, grid alignment, inverse contract lotsmargin.py— tiered maintenance margin, quick deduction, linear/inverse liquidation priceslippage.py— orderbook VWAP, Almgren-Chriss impact, AMM slippagecalendar.py— DST transitions, monotonic clock regressionsplit.py— ex-rights price, forward-adjusted returns, reverse split factor
PowerShell: packages/assertions/pwsh/JevAssertions.psm1 (PS 5.1 / 7, UTF-8 BOM)
⚠ The PS surface and the Python surface are NOT equivalent (fixed 2026-10-01):
the Python side has 18 assertions; the PS side has only 3
(Assert-JevTickFloor / Assert-JevTieredMargin / Assert-JevSlippageBudget) —
the other 15 assertion kinds do not exist on the PS side.
Prefer the Python CLI below whenever an assertion can express the check;
use the PS module only when the environment is PowerShell-only and the
check is one of those 3 kinds.
Call all 18 assertions from the CLI.
⚠ The path below is relative to the plugin root and only resolves when the
current directory is plugins/dsh-jev-preset/. Copy-pasting it from the repository
root (the default session cwd) yields [Errno 2]. Locate first:
$JEV = @(Resolve-Path "$env:USERPROFILE\.dsh\profiles\*\node_modules\dsh-jev-preset\packages\assertions\python\jev_assertions\cli.py" -ErrorAction SilentlyContinue | Sort-Object Path)[0].Path
if (-not (Test-Path $JEV)) { $JEV = @(Get-ChildItem . -Recurse -Depth 5 -Directory -Filter jev_assertions)[0].FullName + '\cli.py' }
if (-not (Test-Path $JEV)) { throw "assertion library not found ($JEV)" }
python $JEV --list
python $JEV --func tick_floor --args '{"raw_price": 67432.178, "tick_size": 0.01, "expected": "67432.17"}'
When
$JEVis empty you must throw first: runningpython $JEV --func ...degrades topython --func ..., which reportsunknown option --func— easily misread as "the assertion actually ran".
Exit-code contract (three values):
| exit | status | meaning | action |
|---|---|---|---|
0 | pass | assertion passed | release the conclusion |
1 | fail | assertion ran, the value is wrong | do not release; recompute or escalate to 3-way |
2 | error | bad call, nothing ran (missing --func / malformed JSON / parameter mismatch / unknown assertion / internal exception) | fix the command first, then recompute |
3 | insufficient_data | input cannot decide (e.g. not enough order-book depth) | supply more data, then rerun — recomputing is pointless |
exit 2 matters most: no numeric conclusion exists at that point. Treating it as "the assertion rejected my value" fabricates a verification that never happened. exit 3 is the mirror image: the value is not wrong, the input is inadequate — recomputing wastes effort. The contract is locked cell-by-cell by
tests/test_cli_exit_codes.py(including the invariantstatus == "error" ⟺ exit == 2).
The PowerShell
Assert-Jev*module has only two values (failure = 1; missing parameter / unknown function also = 1, the PowerShell convention) — no exit 2/3. As noted above, it also exposes only 3 functions, far fewer than the 18 on the Python side. On that surface, read the exception text to tell "wrong value" from "bad call"; the exit code alone cannot.
Defenses that exist (and ones that don't)
Refreshed 2026-10-01: 9 entries added to "present", 5 to "absent". Based on Round 1–38 ledgers and two independent external red-team recomputes, not on self-report.
Present and verified:
-
Recursion block
maxDepth: 1— subagents cannot spawn further; measured 0 grandchild sessions, engine throwsSubagentDepthError. -
Isolated service realms — compaction / delegation live in private realms; checked statically by
validate.mjs. -
Per-guardrail coverage in the assertion library — 5 of 28
asserts (budget penetration / min-notional / depth / non-positive adjusted price / tz fallback branch) could be deleted with all tests still green. Now covered and mutation-verified (npm run test:mutation). -
Four/five mutually exclusive exit codes —
0not-rising /1rising or persistently high /2malformed ledger /3insufficient sample /4rollback suspected. In JSON,_okis three-state (true/false/null);nullmeans "verdict unusable", not "you may continue". -
All 18 assertions reachable via CLI — each is called with independently recomputed correct values and must return
status=pass. The count in--listnever implied callability. -
Input-domain validation — 9 of 11 divisions in the assertion library had user-controllable zero denominators, and they were reported as "command error" (exit 2) instead of "data error" (exit 1) — opposite remediation actions. Fixed.
-
Cross-implementation consistency — the three PS functions used to return different results than Python (double floor error in
TickFloor, wrong speed-calculation deduction inTieredMargin). Fixed and locked by theC3cases. -
Mutation-testing infrastructure —
tests/mutate_guardrails.pymutates a TEMP copy (never the repo in place) and treats "restore succeeded" as a hard failure. -
G5 evasion-audit chain — pre-registered ledger + Wilson 95% + sample-size gate + "worst rate in window" criterion + git HEAD baseline anchor (
tools/evasion_audit.py). -
Default-regression coverage — every Python suite declared in
scripts.test, plus both PS7 and PS5.1, runs insidenpm test; includes a repo-wide encoding-damage (U+FFFD) scan and a repo-wide compile-time-warning (SyntaxWarning) scan. (This line used to carry a hard count, corrected every few rounds: 20 → 22 → 23 → 30 → 31 → 32 → 33 — and it had already drifted again by 2026-10-07. The number was therefore removed, perdocs/structural-boundaries.mdB7: a hard-coded value in prose rots because no judgement reads it. The authoritative count lives intests/pre_registered.py'sSUITE_FILES, andtools/g_check.pycross-checks it againstscripts.testitself via_declared_suites()/_expected_suites().) -
persona / README vs code consistency — "claimed vs actual" is test-backed: assertion counts, the exact function-name list, and the capability-asymmetry wording, across both READMEs.
-
Gate judge distinguishes a declaration from a mention (Round 48) — persona
cordis.patch.yml:38promises "the first line must carry the route marker", butextract_routetook the first match anywhere. So a sentence in the body like我不会用 [JEV: 断言通过] 这条路was judged correct on 14/17 gate questions (measured by red team93f0497b— the largest gaming surface in this family). Addedextract_declared_route(first-line-start only; leading blanks, blank lines and paired**emphasis allowed) and switched the gate branch to it. Over 901 historical records, output changes = 0.extract_route's positional semantics are deliberately left alone — R4 measured that changing them flips 28 records (25% of real outputs put the marker off the first line), and it also feeds cost / arm classification, which would disturb the already-settled 3-path gain conclusion.D13intests/test_gate_no_route_evidence.pypins this boundary. -
A complete "empty declaration" rule (Round 48) — zero-width / invisible non-whitespace characters such as
[JEV: \u200b]slipped past Round 47's.strip() or None('\u200b'.isspace() is False). Now_wellformed: the route name must contain at least one letter or digit (isalnum; CJK counts). No whitelist, so "a newly added multi-path route is auto-accepted" still holds; all 26 distinct historical route values pass → 0 rejected. -
npm testis environment-independent (Round 48) — every earlier "all green" claim depended on manually settingPYTHONIOENCODING=utf-8. In the default environment (gbk on this box) four assertions intest_evasion_audit.pyfail on mojibake and the&&chain breaks there, so every later suite (including the new tests of that round) never ran. Both subprocess calls now pass-X utf8. Acceptance rule: the number ofRanlines must equal the number of suites (the only executable guard against a silently broken chain). -
Declaration extraction's leading-character rule matches
str.strip()(Round 49) — the old regex used^[ \t]*(ASCII only) while the blank check usedline.strip(), which accepts Unicode whitespace (\u3000ideographic space,\u00a0NBSP); a BOM (\ufeff) is not even stripped bystrip(). The mismatch caused false negatives: a perfectly valid first-line declaration was judged "no declaration". 4 leading characters killed it (measured by red team0354a970). Now^\s*plus stripping leading invisible characters, and strip before the blank check (the reverse order misfires). -
Tests no longer use the system under test as the oracle (Round 49) —
D12usedgrade_case(...)["route"]to decide which cases to assert (if is_three_path(got) is not want: continue). Consequence: the more broken the criterion, the greener the test. The measured escape hatch is a class flip (a three-path name classified as single-path): old D12 rc=0 all green, new D12 rc=1 red. Expectations now come from the fixture; the extractor's return value only decides pass/fail. -
extract_route's contract is pinned value by value (Round 49) —D13only asserted "those 10 mentions are still extracted", which cannot detect an internal filter change: I added_wellformedto it and D13 stayed green — false assurance, and it made my written claim "the shared extractor is untouched" false. Now pinned value by value (10 positional + 10 well-formedness cases). -
R4 guard: a criterion must first prove it measured something (Round 50) — the new encoding criterion contains a line that looks redundant:
any(b > 127 for b in raw)(the output must actually contain non-ASCII bytes). The reason is that gbk and UTF-8 agree exactly on pure ASCII — without it, the day a tool emits English only, the criterion would go green while measuring nothing. Such an idle criterion is worse than a missing one: it looks like it guards something. -
S1's single authoritative source is now guarded (Round 51) —
docs/appendix-status.mdis the only authority for the stop condition "every appendix A/B/C item has an explicit status", and it had no criterion at all. This round found three layers: ① the file says "未处理is not a legal status" but nothing enforced it; ② the legend declared 4 legal statuses while the table used 7 — the "partial" meaning had three spellings (部分已修/部分修/半修); ③ the "S1 verdict" section was stale (claiming Round 45 while the table had reached Round 50) and gave a contradictory reason for C5. Guarded bytests/test_appendix_status_table.pyA1–A7: item completeness / every row has a status cell / closed status vocabulary / banned values absent / falsifiability / ascending item numbers within a letter / verdict section not stale. The legend is now a closed set, not prose. -
A criterion must be falsifiable, not merely "currently compliant" (Round 51) — A5 does not just assert "nothing violates the rule now"; it manufactures a violation in a mirror and requires the criterion to go red. Without it, A1–A4 passing only means the file happens to be compliant right now, not that anything guards the rule. The same move was done wrong in Round 50's T2 (deleting a line produced a
SyntaxError, mistaking a syntax crash for behavioural falsification); this round deliberately avoids that trap. -
A criterion must not depend on the host environment (Round 54) —
tests/test_small_sample.pyasserted "the selected 20 cases are the same three times", but the selection key usedhash(), whose value forstris randomized per process (PYTHONHASHSEED). In the default environment the test was red; under any of four fixed seeds (0/1/7/12345) it went green — i.e. the criterion was decided by an env var nobody controlled, and "green" meant only "this box happens to be unrandomized right now". Fixed by making the environment an explicit parameter (_env(hashseed=...)), driving S1 with three deliberately different seeds and requiring the child process to self-reportpythonhashseedas["0","1","2"](end-to-end delivery evidence, not an assumption), plus S8 (falsification under a fixed seed) and S9b (child self-reports its env). The pre-registration record now also carriespythonhashseed, so a third-party recompute can tell whether a differingsha256is a real content change. -
G1–G5 no longer go through a shell (Round 54) —
python ... | Select-String | Select-Object -First 1followed by reading$LASTEXITCODEreturned 0 when the true code was 1:-First Ntruncates the pipeline, so the upstream native command's exit code is never recorded and$LASTEXITCODEkeeps the previous command's stale value (does not reproduce on small output; on large output it also comes back empty). The five gates are now consolidated intools/g_check.py, which runs every step throughsubprocess.run(capture_output=True)— no shell, no PS redirection — and resolvesnpmviashutil.which(on Windows it isnpm.cmd). G4 counts--listitems withlen(json.loads(out)["assertions"])(18), not by counting non-empty lines (which reported 115). -
A hard-coded "current value" in docs is a false fact unless something guards it (Round 58) —
docs/FINAL-CONCLUSIONS.mdstatedA5 self-catch rate 2/9(the real value was long since 11/37) and20 Python suites(real value 28) in the present tense, with nothing watching them; and theg_checkoutput pasted verbatim indocs/self-optimize-rounds.md§9 went stale once (9/35 → 10/36) and again immediately after being corrected in place. The fix is not a new number: §9 now carries a structural skeleton (placeholders) guarded bytests/test_rounds_doc_snapshot.py— seven classes of stale-prone numbers (T2), a non-empty region (T1b, which blocks the "empty region makes the criterion vacuously green" trap), and T3 instantiating every pattern in a mirror (deleting any one pattern's regex now goes red; testing only one pattern did not — measured by red team). Four spots indocs/FINAL-CONCLUSIONS.mdbecame round-scoped records or lost their hard-coded numbers, and two tables gained an explicit measurement-basis note. Still open: the remaining "not yet stale but will be" counts in both READMEs anddocs/FINAL-CONCLUSIONS.md(the README itself already says that number has no test behind it). -
The "silent failure" class now has a guard (Round 59) — Python does not error on
{'a': 1, 'a': 2}; the later key silently overwrites the earlier one. Inbenchmarks/accuracy/jevbench/grading.py,compare()listedn11/n10/n01/n00/agreement/n_gate_excludedtwice — both times with identical values, so no test ever went red. The real hazard is the next edit: change only the first copy and the change is silently lost while every test stays green. The duplicates are gone (7 scenarios × both versions compared key-by-key, all values identical) and an AST guard now exists (tests/test_no_duplicate_dict_keys.py, 7 cases, scanning all 123 Python files in the repo). Coverage is stated honestly: only statically evaluable keys (constants compared by runtime identity —reprcannot tell{1:…, 1.0:…}apart), plus tuples, unary minus, constant expressions, placeholder-free f-strings, and the inner literal of a**unpack. Not covered: variable keys, thedict()call family (a TypeError under CPython, not a silent overwrite), and data-layer JSON — all three pinned by the D6 characterisation test, so fixing one turns it red. -
OKdoes not mean "verified" — the skip channel now has a guard (Round 60) —unittestreturns exit 0 even when tests are skipped, printingOK (skipped=N); but G1/G3 intools/g_check.pyusedok = (rc == 0), so "3 tests were skipped" and "everything was verified" were indistinguishable (and theRan N linesin the detail was only a suite count, yet read like "everything passed"). Fixed:g_checkgained_skip_total(), and G1/G3 now requirerc == 0 and skipped == 0; a newtests/test_no_silent_skips.py(7 cases) requires every skip channel undertests/to be declared with a reason, with the allowlist itself checked for rot. Measured: all 5 skip channels are currently dormant (tzdata present, junctions creatable, PS5.1 installed) — a latent defect, not an observed false green. Mutation 5/5 caught: with one declared skip injected,npm teststill returnsexit=0whileg_checkreports!! G1 ... skipped 1and exits 1. Round 61 closed out the red team's 7 findings: the detector now only recognises qualified forms (self.skipTest/@unittest.skip*, eliminating 4/6 false positives); the allowlist key changed from "file + 4-line text window" to "file + enclosing method + the skip site's own source segment" (closing the retargeting bypass of deleting a declared skip, adding a new one elsewhere, and stuffing the old fragment into its window); S1 now compares multisets (guards against copy-pasted identical skips); and S6 changed from an AST check to a behavioural assertion (monkeypatching_runwith synthetic output) plus S6b proving its own discriminating power (the old AST check stayed green against the_skip_total("")mutant). Cases 7 → 10, mutations 13/13 caught. Still open and registered: G1/G3 do not validate output shape (wrapping/filtering/no-op runs still fool them — appendix C12). Separately, this round empirically refuted "line endings must be LF" as a fix tar... (line truncated to 2000 chars) -
"Green" does not mean "something actually ran" — G1/G3 now validate output shape (Round 62) — the Round 61 red team measured that under 5 output transforms (
WRAPprefixing every line /FILTERswallowing the summary line /EMPTYoutput / allRan 0 tests/ a no-op script) both G1 and G3 stayed green, because the criterion only readrcandskippedand never checked that the output contained anything at all. This is the "idling criterion" family: the criterion looks like it guards something while feeding its guarded object zero input. Fixed:tools/g_check.pygained_unittest_shape()(suite count / total tests / verdict-line count), andg1now requiresrc == 0 and skipped == 0 and expected >= 1 and suites == expected == declared and tests_total >= suites and verdicts == suites, whereexpectedis self-anchored to thetests/filesystem anddeclaredis self-anchored to thepackage.jsontestscript (counting only segments that start with a python interpreter and end with a test file that really exists) — no hard-coded numbers, since a hard-coded number goes stale and a stale number has nothing guarding it. Both regexes are anchored to whole lines and ANSI escapes are stripped before parsing. Newtests/test_g_check_actually_ran.py(10 cases, including behavioural assertions and mutation-based self-proof of discriminating power). Two red-team rounds: the first judged it 【not sound】(6 new bypasses + 2 classes of real false positives), driving 5 hardening changes; the re-dispatch judged it 【partially complete】(all 5 hardening changes verified effective, plus a new "path embedded in a string" forgery and an ANSI false positive), driving 2 more. Mutations 12/12 caught; product-side A/B: after really replacing thepackage.jsontestscript withecho okand running the realnpm test, **the old criterion returnedok = True(fooled) while the new one returnedok = False(bloc... (line truncated to 2000 chars) -
"No error" does not mean "no problem" — compile-time silent defects now have a guard (Round 63) — the whole repo produced 2
SyntaxWarnings at compile time (unescaped\d/\^inside the docstrings oftests/test_no_unsupported_claims.py:71andtests/test_pass_k.py:2), and they had sat there for 29 rounds — because CPython only warns, execution is unaffected, so nothing went red and no reported number changed; yet what they pollute is exactly the docstrings that carry conclusions: a reader cannot tell from the text whether\dis "the regex\d" or "a swallowed backslash". The fix is not escaping them one by one — CPython reports only the first invalid escape per string, so fixing what the error names never finishes; the correct fix is making the whole docstring raw (r"""). The guard was attached to the existing repo-wide static-scan familytests/test_no_encoding_damage.py(E1 = U+FFFD) as E3 (walk +compile()+ captureSyntaxWarning/SyntaxError), E3b (self-proof of discriminating power, plus a falsifiable anti-idling guard) and E3c (meta-criterion: case count and named cases) — no new suite. Both red-team rounds judged it 【partially complete】: the first reported that E3 itself was an idling criterion (once the walk count drops to zero,bad == []is vacuously true — the very pit C12 had just fixed, stepped into again with a different criterion), plus.pyw/.pyi/uppercase-extension escapes and junction escape; after hardening, the re-dispatch confirmed all 4 holes were closed and reported.pyiwith zero coverage, E3c bypassed by renaming (counting only thetest_prefix means renamingtest_E3binstead of deleting it keeps the count intact and the meta-criterion fully green — "guards against deletion" is not "guards against disappearance"), and_insidecomparing a raw string prefix, letting sibling directories in. All three were fixed. **Mutations 9/9 caugh... (line truncated to 2000 chars) -
"Fixing one instance" is not "fixing a defect class" — meta-criteria moved from a count to a named list (Round 64) — Round 63 found in
tests/test_no_encoding_damage.py::E3cthat a meta-criterion counting thetest_prefix guards against deletion but not against renaming: renamingtest_E3bto another name starting withtest_leaves the count unchanged, keeps the meta-criterion fully green, and the protected case has already disappeared. Round 63 fixed that one file and never ran the sweep. Round 64 did: 6 files in the repo carry a meta-criterion, and 5 had the same disease — exactly the failure mode this repo has recorded before, "scoping a fix by directory (one file) instead of by defect class". The fix: all 6 files replaceEXPECTED_CASES = Nwith aCASE_NAMES = (...)named list plusassertEqual(sorted(loaded), sorted(CASE_NAMES))— now bidirectional (delete / rename / add all go red), and it retires a number that goes stale (it went stale immediately:_real_output(n_suites=31)intest_g_check_actually_ran.pyturned three cases red once the new suite was wired in; it now self-anchors to_expected_suites()). Newtests/test_meta_criteria_bind_to_names.py— a cross-file behavioural criterion: for every file carrying a meta-criterion it applies a source-level mutation (rename / add), reloads it, and actually calls that meta-criterion, which must raiseAssertionError;M3guards against idling andM4requires mutants to be semantically intact (a crash is not a catch). Mutations 11/11 caught. The red team judged this 【partial】 and all three findings were fixed: (1)M2("an added case must be caught") was a vacuous criterion — it went red for the named-list version and for the old count-only version (adding a case necessarily changes the count), so it had zero discriminating power; it was deleted rather than kept to pad the count. (2)M3used a lower bound `MIN_META_FILES = ... (line truncated to 2000 chars) -
A "count lower bound" proves nothing about coverage — anti-idling guards now bind to a named directory list (Round 65) — this repo has a class of guard whose only job is "prove I really scanned something", without which the criterion it protects is vacuously true on empty input. It used to be expressed as a number:
assertGreaterEqual(len(scanned), 50). Measured across three instances (real / bound / margin): 83 / 50 / 40%, 126 / 100 / 21%, 33 / 25 / 24%. The hole is not that the bound is too low — it is that a bound cannot prove coverage at all: the structural self-check intests/test_no_encoding_damage.pylooked at only two directory names,testsandtools, while there are really four — deletingpackages/(7 sources) ortools/(3 sources) outright still left the total ≥ 50, so it went green; deletingbenchmarks/(39 sources, 47%) dropped the total to 44, just barely stopped by the bound — pure luck, and a slightly looser bound would have missed it. The fix: bound → a named directory list (SOURCE_DIRS) that must be fully covered, and the falsifiability sample changed from "make up a fake directory" to "remove each top-level directory that actually appears in the scan, one at a time". Coverage must come from identity, not from numeric margin. Mutations 6/6 caught, and one of them caught me: the per-directory loop intests/test_no_duplicate_dict_keys.pyiterated overSOURCE_DIRSitself — the anchor was the guarded object — so when the list shrank the loop shrank with it andSOURCE_DIRS = ("tests",)still went fully green; iterating over the directories that actually appear in the scan closed it. ⚠ Same round, same defect, two different spellings in two files — only mutation separated them; reading the code did not. A second finding fell out: that same per-directory check showedtests/test_no_duplicate_dict_keys.pywas treating a temporary directory in ... (line truncated to 2000 chars) -
"Fixing one function" is not "fixing one defect class" — a junction escape was fixed in only one of three scanners (Round 66) — on Windows a junction is not a symlink:
os.path.islink(junction)isFalse, and bothos.walk(followlinks=False)andpathlib.Path.rglobfollow it, pulling files from outside the repo into the scan. ⚠ "cycles hang it" is wrong: the red team measured that on this machine (Win + CPython 3.12.7) an in-repo junction cycle makes bothwalkandrglobterminate finitely (the Windows kernel caps reparse resolution per path at 32 →ERROR_CANT_RESOLVE_FILENAME, which both silently swallow, yielding just 64–66 duplicate paths) — I wrote "hang" in three places in the first draft, an unverified comment, caught the same round. Pruning's justification is "escaping the root", one reason, and it suffices. -
R73–R76: "Every mutant whose value coincides on the two ledgers is invisible." Four rounds of closing the loop, and each round the red team proved one layer was still missing. R73 generalised discriminating power from a single mutant class to a pre-registered matrix (6 classes × 5 numbers = 30 cells; the set must be exactly equal) — and the red team immediately produced two serious counterexamples: aggregation (
sum→len({round})) and widening (n<1 or n>9999) both left the suiteRan 24 OKwhile the value was genuinely wrong. R74 added a third ledger with a deliberately different shape (two same-valued unparseable rows in the same round, around=100000row, several same-round rows) → the four aggregation mutants are now caught 4/4; ⚠ my first version put the two unparseable rows in different rounds and stayed fully green — becausebuild_reportsums per round, so an aggregation mutant only shows itself when one round contains duplicate values. R75 replaced the human-face assertionassertIn("… 4 条")(the red team proved it was both too strict and too loose: a same-valued literal passed, deleting one space went falsely red) with a numeric regex pinned across all three ledgers. R76 fixed the self-referential loop the red team found:backfilled_evaded's expected value was read from production's own JSON, so it was an identity; the red team's N1b (min(1,sum)per-round capping) left the whole suite green while the true value 99 was reported as 23 — it is now independently recomputed + pre-registered as a constant, andre.searchbecamere.findallasserting exactly one hit (adding one same-worded output line used to be enough to bypass it). ⚠ Not fixed: F-4 rule-same-source (needs a second independent rule set), the real ledger's unguarded numbers, and cross-run drift (needs an external anchor). ⚠ Bookkeeping gap: the red team found the ledger stopped at round... (line truncated to 2000 chars) -
R73–R79. Four rounds of closing the "every mutant whose value coincides on the two ledgers is invisible" gap (R73 pre-registered 6-class x 5-number = 30-cell matrix; R74 a third, deliberately differently-shaped ledger; R75 numeric-regex pinning of the human face across all three ledgers; R76 killing the self-referential
backfilled_evadedexpectation and making the human-face uniqueness check exact). R78-R79 then fixed the mutation-verdict framework itself: my R77 "4/4" was fake because the anchor was the first physical line of a multi-line f-string, so the mutants were SyntaxErrors recorded as detections. The crash/equivalent/detection trichotomy is now a mechanism (tools/mutation_harness.py, in-repo, with a verdict self-test). The red team immediately caught two bugs in that framework itself (aTracebackpre-check that misfired ontests/, and immunity to the test count); both fixed, though a neutralised test case remains invisible. Unicode categoryCf/Mnstripping replaced character enumeration, but semantically identical forms still bypass it. Still unfixed: F-4 rule-same-source, the real ledger's unguarded numbers, cross-run drift (needs an external anchor). "Memory cannot plug it; only mechanism can" - sixth and seventh instances. -
R72: "Patching one cell is not plugging the hole" — non-zero is not discriminating power. The R71 red team said "on the real ledger
settle_unparseable ≡ 0, and a constant-zero quantity has no discriminating power". In R72 I added a synthetic ledger where all four numbers are non-zero, and the red team showed that is still not enough. One line of editing silently disarms it:backfill_unlabeled's only source of discriminating power in the whole suite is the synthetic table's single rowf(upheld + same round + unlabelled), while all 23 unlabelled rows on the real ledger are "evaded" rows → the real ledger has zero power on that axis; flippingffrom成立to规避(all four numbers unchanged at 2/4/2/1, self-check passes) plus production mutant M3 →exit=0 Ran 24 OK, while the production code is still broken (run it on the original table: unlab reports 1, true value 2). "All non-zero" guards against a number going to zero, not against a number losing its discriminating power. ⚠ Worse: that "all four non-zero" self-check reads the helper'swant, and the helper is rule-same-source with production (C23's F-4) → when production and helper are narrowed together the self-check fails with them; it is not an independent second anchor. ⚠ Also: non-zero ≠ the domain was sampled — the synthetic table covers only 6/16 of the four-dimensional space,settle_unparseable's 8 combinations are covered once, and then < 1channel is never walked (mutant M5 leaves the synthetic table unmoved but is caught bytest_A11d, so it is not a suite hole). Fix: the expected values became a pre-registered constantSYNTHETIC_A11_EXPECTED(R9, not recomputed from the helper) plus a new discriminating-power structural check_synthetic_discrimination()— "has discriminating power" is now executable: for each number it requires full-total value ≠ evaded-rows-only value, and equality ... (line truncated to 2000 chars) -
R71: "Objective" was derived from self-reported fields, and a constant-zero quantity has no discriminating power. In R70 I added
A11b(count of unparseable settle rounds) and claimed it made the zeroing channel "no longer silent". The red team disproved that immediately, on two independent grounds. (a) The zeroing channels went from one to two, and the new one is quieter:int()parsing is not validity — with结算轮set to0/-1/999999/3_0(=30) the objective count still collapses to zero whilesettle_unparseablestays 0 and the exit code is unchanged;—is now reported,0is not. I traded one silent channel for two. (b) Those two numbers are not "objective" at all: they derive from状态(evaded/pending) and结算轮, both written by me — flipping one genuinely violating row's状态to "pending" made all three counts collapse to zero with the exit code unchanged. R69's F-C only said "conflated properties"; F-2 shows it is far stronger: I can zero the entire signal. ⚠ Same-source problem:test_A11c's_independent_a11is implementation-independent but rule-same-source (same ledger, same rules) → under a rule-level bypass both sides collapse together, structurally undiscoverable; and on the real ledger that quantity is≡ 0, so a constant-zero quantity cannot give a "shrink it" mutant any discriminating power (the red team's M-e mutant left the whole suite green while a true value of 1 was reported as 0). Fixed: the domain now covers "unparseable or invalid (< 1)"; the human-readable wording now says "a zero here does not mean the signal above is complete:状态and结算轮are both self-reported fields";test_A11dcovers0/-1. Not fixed: anchoring状态/结算轮externally (no in-repo answer), parameterisingtest_A11cwith synthetic ledgers, and the same-source rule problem (needs a second independent rule set). ⚠⚠ **The red team also caught the same mistake I made two... (line truncated to 2000 chars) -
R70: A window is not a guarantee — a local check was presented as a global property. What the R69 red team rated critical was "the version the red team audited ≠ the final version". In R70 I added A12 (compare ledger digests at two points within a run; drift ⇒ exit 2), and the red team showed it does not fix that problem: A12 covers only the window inside a single run, between
parse()andbuild_report(). It produced three counterexamples with unmutated production code — ① the ledger is changed to "all held" duringparse()and restored before it returns → both independent reads see the original, whileparseconsumed the modified content (evaded_total=0, true value 104), yet the reported sha equals the on-disk sha → a recomputer replaying that sha gets 104, not 0, exactly the R69 form; ② changed after the second read → JSON reports the old digest while the log records the new one; ③ changed between two runs (R69's actual scenario) → no reaction at all. ① is fixed:parse()now digests the very bytes it read, andmain()uses that asbeforeagainst the on-disk digest taken just before the verdict — the root cause was thatbefore,afterandparsewere three reads and two digests, with nobody minding the middle one. ②③ are logged, not fixed: spanning runs needs an external anchor (git blob / remote); two points inside a run cannot do it in principle. The wording is narrowed to "the ledger did not change within this run's window, which is not the same as the verdict being reproducible", and is now printed on the human-readable face (previouslyshadid not appear at all in its 49 lines). ⚠ Two mistakes of mine this round, both caught by the red team: (a) I claimed the mutants were "5/5 caught"; the red team's M5 was a miss — movingbeforeafterparse()leftRan 19 testsfully green, exit 0. "I ran mutants" is not "I ran the mutant that mattered"; (b) my `test_A12... (line truncated to 2000 chars) -
R69: Self-reported metrics — I write the label, the script treats it as authoritative. R68's back-fill count (A10) claimed to make "written after the fact" visible; the red team's F6 immediately pointed out that I write the two characters
回填, so simply not labelling makes A10 look good, and the script had zero objective cross-check on it — the self-attestation source is the audited party's own free text. The fix adds an objective anchor: iron rule 3 says "an item proposed this round may be settled no earlier than the next round", so结算轮 == 提出轮is an objectively computable process violation that does not depend on my remarks at all. New fieldsbackfilled_struct_total(objective) andbackfill_unlabeled(unlabelled yet objectively violating), measured on the real ledger: 81 self-reported / 94 objective / 23 unlabelled. ⚠ I deliberately did not call it "23 unlabelled back-fills" —结算轮==提出轮proves a breach of rule 3, which is not the same as back-filling (pre-registering at the start of a round, being caught by the red team during it, and settling at its end lands in the same cell). Treating the two as one thing would be yet another unverified equivalence claim (the third after R67's "one level too wide → no-op" and R68's "2.8pp → 86pp"). So the three are reported separately, never merged — one is "I admit it", the other is "the ledger gave it away"; merging hides "should have labelled but didn't". ⚠ R13 check: I can shrink the gap myself (by adding labels) — that is the correct direction, but the gap trends to zero while the objective count does not move, so A11 must report the objective count too. Mutants 5/5 caught (including "merge self-reported and objective into one number"). -
R68: "Making the rule match the implementation" can be the self-beautifying direction. The ledger's iron rule 1 says "anything written after the fact must be marked
回填and is not counted in the evasion rate", whiletools/evasion_audit.pynever reads the备注column — an inconsistency that had sat for 22 rounds. Round 45 logged it as a "binary choice, pending human ruling" but missed one thing: one of the two options is the direction R13 forbids — implementing "not counted" means I can paste a回填label and pull an evasion out of the metric, putting the metric back in my hands. Measured magnitude (⚠ after red-team correction F4 — my first version said "79 of 92 evasions are back-filled, 100% → ~14%, 86pp", which mixed the scored-round subtotal with an all-rows figure and, when excluding, subtracted from the numerator but not the denominator): across the whole ledger, 114 rows / 108 settled / 107 evasions / 81 back-filled — counting them gives 99.1%, excluding them 96.3%, a 2.8pp gap, and the exit code does not change (1 either way). So the R13 argument holds but it is structural, not magnitude-based: I write the label, so implementing exclusion means pulling an evasion out of both numerator and denominator — the metric back in my hands; the gap is only 2.8pp because 75% of the ledger's rows are back-filled, not because the mechanism is harmless — and it is not an escape hatch (the verdict is unchanged). The fix takes a third path: count them anyway (the uglier side) + report the counts (backfilled_total/backfilled_evadedinto the JSON and the G5 header line — the very thing rule 1 wanted to prevent, "back-fill passing as pre-registration", had never even shown a number) + a guard pinning the direction (test_A10asserts back-filled rows must still count; mutant P-20, a real exclusion implementation, goes red immediately). ⚠ A harder fact fell ou... (line truncated to 2000 chars) -
R67: "I fixed the class" had nothing guarding it. Round 66 extracted the shared primitive and fixed three
.pyscanners, but missed the.mdscanner (tests/test_markdown_tables.py) and an audit script (tools/_wilson_doc_scan.py) — found only by an AST census; reading code point-by-point cannot see them. This is the fourth recurrence of "scoping the fix by file instead of by defect class" (R48, R64, R66, R67). Two-layer fix: (1) add the two missing prunes; (2) a new meta-criteriontests/test_fs_walkers_are_pruned.py— thefile::functionset of everyos.walk/rglob/glob(recursive=True)in the repo must exactly equal the hand-writtenWALKERSlist. "Remember to check" becomes "forget and go red." ⚠ The census script's first version countedast.walkand non-recursiveglob.globas recursive walkers — 7 of its 9 hits were false — hence W4, which guards against that false positive. -
R67 (two red-team rounds, a real bug, and one causal claim I overstated): round one ruled 【partially complete】: my "does it prune" check was a substring match
"prune_escaped" in c— one line ofprune_escaped_fake(...)bypasses it (the "whitelist by text fragment" lesson from R50, repeated), with a closed-loop demo where the criterion stayed green while a real junction test pulled an out-of-repo file into the scan. That bypass shape also made me spot thattools/_wilson_doc_scan.pycompared against too wide a root (it walksbasebut pruned againstROOT). Replaced by four structural checks (exact attribute name / inside the walk loop body / 2nd arg is dirnames / 3rd arg isrealpathof this loop's own root), plus alias-import resolution and a hard error on scope-key collisions. The dispatched round two then caught B6: my check was existential — "one qualifying loop and it's green" — so a second, unpruned walk loop in the same function was masked. Now universally quantified (every loop must prune). ⚠ It also falsified a causal claim of mine: I had written in five places that the wide-root bug made "the prune a total no-op → escapes happen"; the old logic still pruned junctions pointing out of the repo — the real consequence was only "one level too wide, so in-repo junctions aren't pruned", no escape. A bug's consequence is an assertion that must be measured too. Re-running both red-team scripts: all 8 old bypasses closed or now hard errors, B6 turns red; 9/9 mutants caught. Round 63 found this inscan_syntax_warningsand added a local_insidein that one function only; the Round 65 red team handed over the reproduction —iter_text_filesin the same file (which had no check at all) anditer_py_filesintests/test_no_duplicate_dict_keys.py(a barerglob) followed it too, measurably pulling intests\jlink\c.py. ⚠ This is the third recurrence of this repo's "**scopin... (line truncated to 2000 chars)
Absent (do not assume otherwise):
- Standalone circuit breaker — does not exist. Failure recovery comes from the host
@deepseek-ai/dsh-llm-retry. The one intests/test-circuit-breaker.mjsis a reference implementation only. - Role mutex — does not exist. 3-way role uniqueness rests on persona discipline, not code, so the model can still dispatch two Path 3s.
- Jev Rerank / decision-model gating — design reference, not a deployed capability. No implementation code exists in the plugin; execution depends entirely on the agent reading persona and choosing to comply. The
/systemoneendpoint is not provided by this plugin either. Marked as such in persona. - Full PS-side coverage — the PS surface has 3 functions; Python has 18. The other 15 assertion kinds do not exist on the PS side. Declared honestly, but not implemented.
- Correctness of
inverse_liq_price— the formula is empirically refuted (a 20x change in entry price leaves the liquidation price bit-identical; the physically derived value differs by 1e15 and in the opposite direction). It was not changed, because the correct formula needs a business decision; the corresponding test is deliberately left red. - G5 chain measuring the real evasion rate — it only measures "how much I admit to faking." The ledger is maintained solely by the audited party; the git HEAD anchor prevents deleting evidence, not rewriting records.
Похожие плагины
webnovel-writer
lingfengqaq/webnovel-writer
loopx (dsh-loopx-plugin)
loopx-project/loopx
ouroboros (dsh-plugin)
q00/ouroboros
loopx (dsh-loopx-plugin)
huangruiteng/loopx