DSH Supervisor

DSH Supervisor is a lifecycle supervision and acceptance-control plugin for DeepSeek Harness (DSH). It records deterministic facts continuously, asks isolated audit subagents for semantic review only at high-value checkpoints, and applies bounded intervention policies before a task can be accepted.

It complements, rather than replaces, specialist subagents: audit agents provide independent depth, while the Supervisor owns state, evidence, deduplication, intervention, recovery, and the final acceptance gate.

DSH Supervisor is an early 0.1.x release. Use it as an additional control layer, not as a substitute for sandboxing, backups, code review, or deployment controls.

中文文档 · Architecture · Security policy · Changelog

What it does

Creates an append-only, versioned acceptance contract from user requirements and supplies this authority policy to audit components: latest user instruction → latest acceptance contract → repository rules → execution plan → audit advice.
Records file, tool, validation, agent, and checkpoint facts without calling a model for every event.
Runs deterministic guards for failures, risky external operations, and likely credential exposure.
Selects up to three isolated audit roles at semantic checkpoints: requirements, architecture, testing, security, UI, or documentation/data.
Produces one deduplicated correction package per checkpoint and routes it as an emergency stop, next-step steer, queued correction, or report-only finding.
Tracks findings with stable SUP-<session>-<fingerprint> IDs and the open, fixed, rejected, waived, and unresolved lifecycle.
Performs a blind final audit from the original request, latest contract, current contract/revision-bound final evidence, and current validation evidence—not from the main agent's explanation or hidden reasoning.
Persists a redacted JSONL ledger and a human-readable Markdown report, and exposes a polling, read-only Web panel.

New ideas are report-only by default. They do not expand task scope unless the latest user instruction adopts them or they directly satisfy the existing acceptance contract.

Requirements

Node.js 22.19+ or 24+
pnpm available on PATH for the DSH plugin installer
@deepseek-ai/dsh@0.1.0-rc.7 or a compatible 0.1.x build
A DeepSeek-compatible audit credential supplied through the DSH credential service, DEEPSEEK_API_KEY, or an explicitly configured key file

Install

Install the host plugin and its Web client into the Web profile:

dsh plugin --profile web add dsh-supervisor@0.1.0

The package bundle patch inserts the dsh-supervisor service. Restart the affected DSH host/Web process after installation if it is already running.

Configure credentials

Prefer the DSH credential service. Environment variables are the next choice:

# POSIX shells
export DEEPSEEK_API_KEY="your-api-key"

# PowerShell
$env:DEEPSEEK_API_KEY = "your-api-key"

For a local, uncommitted key file, point the bundle configuration at a path through an environment variable:

$env:DSH_SUPERVISOR_API_KEY_FILE = "C:\private\deepseek-api-key.txt"

The file should contain only the key (surrounding whitespace is ignored). Never commit the key, the file, or a literal machine-specific path. Credential resolution order is: DSH credential service → configured environment variable → configured key file.

The default patch is equivalent to:

- insert:
    - id: dsh-supervisor
      name: dsh-supervisor
      config:
        enabled: true
        model: deepseek-v4-pro
        reasoningEffort: high
        apiKeyEnv: DEEPSEEK_API_KEY
        apiKeyFile: !!js process.env.DSH_SUPERVISOR_API_KEY_FILE
        includeRecentReasoning: false
        maxAuditRoles: 3
        maxAuditRetries: 3
        auditDeadlineMs: 300000
        allowAuditDegradedForLocal: true

Important defaults:

Setting	Default	Meaning
`includeRecentReasoning`	`false`	Raw execution-agent reasoning is not collected. Explicit diagnostic opt-in stores it as redacted, untrusted checkpoint evidence; it remains outside final blind-audit input and the Markdown report.
`maxAuditRoles`	`3`	Maximum audit roles selected at one checkpoint.
`maxAuditRetries`	`3`	Infrastructure retries after the first audit attempt.
`auditDeadlineMs`	`300000`	One monotonic five-minute window including attempts, queueing, and backoff.
`auditAttemptTimeoutMs`	`90000`	Per-attempt ceiling, also capped by the remaining audit window.
`auditBackoffMs`	`5000, 15000, 30000`	Retry delays for audit infrastructure errors.
`allowAuditDegradedForLocal`	`true`	Allows a local-only task to close with an explicit degraded warning after deterministic validation passes.
`sameClassHardLimit` / `totalHardLimit`	`2` / `5`	Circuit breakers for model-driven hard intervention; deterministic safety guards remain active.

SupervisorConfig supports per-task and per-role model routing, call budgets, and Token budgets. The dispatcher checks budget headroom before a role batch or retry, reserves concurrent output allowances, and forwards each available output ceiling to supported providers. Call budgets are enforced per recorded backend call. Token budgets are a concurrency-safe output reservation plus a soft total-usage ceiling: input and separately reported reasoning Tokens are known only after a call, so that call can take the reported total above the configured limit; later calls are then blocked. Role-specific provider/model routes apply to DSH subagents; the direct fallback uses the top-level DeepSeek model configuration.

validationScriptNames controls validation-command recognition and package.json script discovery. The Host discovers scripts when a root run opens and refreshes them immediately before completion or completion recovery. Every configured script that is then present is required; no script name, including check, is assumed to aggregate the others. Repository-script evidence must use the exact foreground form <package-manager> run <script> without forwarded arguments; the standard no-argument npm test, pnpm test, or yarn test shorthand is the only exception. Completion-grade evidence must run at the root session workspace through the trusted global definition of DSH's official bash or pwsh tool, with the Agent-scoped lookup resolving to that same definition, and must return the canonical foreground result shape.

Direct test, typecheck, lint, and pack runners use a conservative positive command/argument grammar. They are classified as full or partial: an unrequested partial run cannot close a task or satisfy a discovered repository-script gate. A safe, recognized validation command explicitly named in the acceptance contract retains its complete argument list and is independently mandatory, but it does not waive discovered repository gates. A trusted root-workspace shell command with a validation identity that returns a canonical non-zero or infrastructure result is recorded as failed even when its command is outside the positive pass grammar, so it cannot leave an older pass authoritative. Configure custom script names explicitly. Keep credential values outside configuration committed to source control.

Before execution, the Host persists pending state for recognized validations and mutation-capable intervals: all shell executions, known writers, and tools not proven to be one of the Host-captured read-only definitions. A read-only name is exempt only when both global and Agent-scoped tool lookups resolve to the same trusted definition; a scoped shadow is treated as mutation-capable. Shell, unknown, and untrusted-shadow intervals receive bounded before/after fingerprints over file bytes, modification/change times, permission bits, directory metadata, and in-workspace symlink targets. Fingerprinting includes ignored project content such as lib/ and .env, excludes dependency trees, Git metadata, Supervisor state/report directories, and common generated artifact/cache directories, and separately fingerprints recorded in-workspace deliverables. Pending validation or mutation blocks completion. An overlap, observed event, unavailable fingerprint, or endpoint difference makes the affected validation stale; unchanged sequential required validations can share one revision. On restart, an interrupted validation becomes an infrastructure error and an interrupted mutation advances the revision.

On POSIX, the Host adds per-directory filesystem-event observation to detect transient changes during bounded tool intervals and invalidates observed out-of-band changes. Windows Hosts do not enable that observation because of instability seen in the Node.js 24 filesystem-watch backend, so they use endpoint content/metadata fingerprints around observed intervals. Neither mode is a complete filesystem or process-security boundary: operators must control out-of-band writers and rerun every required gate after any change the Host may not have observed. The preguard rejects generic background/detach flags, common PowerShell/Bash detached-launch forms (including a standalone Bash & inside quoted nested commands), and terminal_send/write_stdin; it is not a general process supervisor. Any non-running state rejects further workspace mutation until a latest user instruction reopens the task.

During the matching validation only, standard build outputs under lib/, dist/, build/, or out/, TypeScript build-info files, and a root .tgz produced by npm pack are treated as bounded generated outputs so they do not invalidate their own successful build/pack result. The same bounded paths are omitted from the validation source fingerprint stored with results and compared at completion or restart; changes outside those allowances still stale the validation. A recorded path alone is not artifact proof: an artifact requirement needs readable in-workspace final-file evidence captured at the current revision and associated with the active contract item. A quoted artifact path containing directories must exactly match its complete normalized workspace-relative path, not merely its basename. A screenshot-named tool returning text alone is not visual evidence; its result must contain an actual image attachment block (including one backed by a verified image file). Screenshot evidence is additionally bound to the full bounded workspace fingerprint: a later output-only build may preserve the validation source revision while still invalidating the old screenshot, so the rendered state must be recaptured. Final file, diff, and screenshot evidence carries the workspace revision and contract version/source-message binding, and a new context receives a new evidence identity even when its bytes are unchanged. Both the pre-external-operation audit and final blind audit also compare the same bounded workspace fingerprint across the entire audit window; a change or fingerprint failure invalidates that audit decision and requires a fresh stable-window audit.

Runtime model

DSH Supervisor distinguishes facts from interpretation:

The host creates or versions an AcceptanceContract when user requirements change.
Normal events enter a redacted, content-addressed evidence ledger. Repository files, diffs, tool results, Web content, and subagent output are always marked as untrusted evidence.
Deterministic policy handles tool failures, likely secrets, and high-risk external operations.
At a checkpoint, the coordinator selects at most three read-only audit roles. It prefers DSH ctx.subagents with independent context and can use a direct DeepSeek audit provider as fallback.
Findings are merged by stable fingerprint and evidence. The Supervisor sends at most one correction package for the checkpoint.
Before completion, a blind audit and deterministic validation determine the terminal status.

Audit subagents receive bounded evidence envelopes and no write-capable tools. Model output must match a strict schema; it cannot grant itself permissions or authorize an external operation.

Every checkpoint records the acceptance-contract version and source user-message sequence that it reviewed. A newer root-user message advances an in-memory authority epoch immediately; a completion or pre-operation result is discarded if that epoch, the contract binding, pending execution state, or the bounded workspace state changes while the audit is running.

An increase in completed Todo items creates a milestone checkpoint. When different execution Agents write different content to the same recognized file path, the Host records an agent-conflict checkpoint. These are deterministic, bounded signals rather than general semantic change detection.

Statuses and completion gates

Status	Meaning
`accepted`	Final blind audit passed, mandatory contract items passed, and deterministic validation passed.
`unaccepted`	A normal local task still has an explicit audit failure after at most two repair-and-revalidation cycles.
`completed_with_audit_degraded`	A normal local task passed every required and recorded recognized deterministic validation, but the final AI blind audit could not complete because of an API, queue, model, or audit-infrastructure failure. The report explicitly states that the final AI audit did not pass.
`awaiting_audit`	A high-risk task has no valid passing audit. External actions and completion claims remain blocked until a later user instruction explicitly resumes the run or a provider-recovery event opens one new audit window and that audit succeeds.
`running`	Supervision is active and no terminal decision has been recorded.

Publishing, deployment, deletion, external submission, security, and secret-related work is monotonic high-risk within a run. It cannot downgrade itself to a local task. A target-matching active must-not contract item vetoes the external operation before positive category authorization or any existing grant is considered. Otherwise, the downstream DSH policy must first return allow; the Supervisor then requires current contract authorization, no pending validation/mutation, a stable workspace, and a fresh pre-operation audit. It rechecks authority, contract version/source sequence, pending state, revision, and fingerprint before synchronously preparing the exact execution. An ask or denied downstream decision cannot later execute the mutation without a new pre-execute pass. Grants are persisted as audit traces and consumed for the approved dispatch, not replayable capabilities.

An audit service error is not an audit finding. Explicit audit failures follow the repair flow; infrastructure failures follow the bounded retry/degraded-or-awaiting flow.

A latest user instruction can close a finding as waived or rejected only when it cites the full SUP-… ID and explicitly states a waiver/accepted risk or false-positive/counter-evidence decision. Blocker and safety-related findings cannot be waived; they may be rejected only through the explicit counter-evidence form. A recurrence reopens the stable ID.

Intervention policy

Emergency stop is reserved for likely credential exposure, unauthorized destructive work, or unsafe external actions. A pre-execution guard is preferred; cancellation is used only when safe cancellation can limit ongoing harm.
Next-step steer waits for the current atomic tool boundary, then corrects a blocking deviation.
Queued correction is delivered after the current turn for non-emergency quality, test, UX, or documentation issues.
Report only records ideas, weak claims, or interventions demoted by circuit breakers.

After an emergency interruption, work should not resume until workspace, lock, active-task, and external-operation state is understood. An untrustworthy non-idempotent external result is persisted as an unresolved outcome and keeps the matching operation fail-closed. It is never retried automatically; the operator must verify remote state and send an explicit latest user instruction to resume.

Persistence and read-only UI

For each root session, the default workspace-relative paths are:

.dsh-supervisor/runs/<root-session-id>.jsonl
.dsh-supervisor/reports/<root-session-id>.md

The ledger records snapshots and state transitions so a host restart can recover finding IDs, fail-closed state, audit attempts, workspace revisions, pending validation/mutation attempts, unresolved external outcomes, and already-issued correction packages. Checkpoints remain bound to the contract version and source message they reviewed. Terminal accepted, unaccepted, and degraded decisions persist the full bounded workspace fingerprint used at closure; startup compares it again, including separately recorded deliverables such as a root package tarball, and reopens the run if it differs. An audit interrupted by restart is recorded once as cancelled; a pending validation becomes an infrastructure-error record; a pending mutation invalidates earlier validation; every unconsumed external grant is invalidated; and a high-risk run returns to its persisted fail-closed phase instead of silently opening a new window. Direct execution-Agent mutations that target the configured ledger or report directories are denied.

The ledger is serialized and redacted, but it is not cryptographically protected from another process running as the same workspace user. An indirectly invoked shell script can still tamper with these files outside the direct path guard. Protect them with OS/workspace access controls, do not commit them, and treat unexplained ledger changes as a reason to stop and investigate. These files may still contain sensitive project context after redaction.

The Host service exposes unary Typert methods:

getSnapshot(rootSessionId?)
getReport(rootSessionId)
listRuns()

The Web client polls these read-only methods and presents the panel from a sidebar action and shell overlay. It does not claim push delivery and exposes no mutation RPC.

Metrics

The report uses fixed definitions and prints N/A when a denominator is zero:

False-positive rate: rejected / (fixed + rejected)
Effective intervention rate: interventions whose target set contains at least one finding that is fixed at report time / executed emergency, steer, and queued interventions
Fix success rate: fixed / (fixed + unresolved) for confirmed findings that required repair
Validation pass rate: passed validations / completed passed and failed validations; infrastructure errors are reported separately
Additional tokens: reported supervisor/audit input, output, and reasoning tokens, excluding the main agent; DSH subagent backends currently contribute zero where they do not expose usage
Added critical-path wall time: elapsed wall time of blocking pre-external-operation and pre-completion audit cycles
Audit compute time: the sum of locally measured durations for all recorded audit backend calls, including failed calls and roles that ran in parallel; this is cumulative compute, not critical-path delay

Security and privacy boundaries

Raw reasoning is off by default and excluded from the blind final audit and Markdown report. If explicitly enabled for diagnostics, redacted reasoning from execution Agents may be persisted, shown in the panel, and sent as untrusted evidence to non-blind checkpoint audits.
Evidence is truncated, hashed, source-labelled, and redacted before ledger storage, model transfer, panel display, and report generation.
Evidence text is data, not instruction. Strings such as “ignore previous rules” cannot change authority, tool permissions, or the acceptance contract.
Redaction is defense in depth, not a proof that arbitrary secrets can never appear. Use least-privilege credentials and avoid placing secrets in task text, files, tool arguments, or logs.
Audit requests send selected task evidence to the configured model/provider. Review your provider's data policy before enabling the plugin for sensitive repositories.
High-risk gates are an additional policy layer; they do not replace OS permissions, DSH sandboxing, repository protections, or deployment approvals.
Shell, egress, and detach classification uses conservative static argument parsing; it is not OS-level network or process containment and cannot observe effects outside the DSH tool pipeline.
Windows uses bounded endpoint fingerprints rather than a continuous filesystem watcher. POSIX event observation is also only an additional freshness signal, not a complete filesystem monitor.

See SECURITY.md for reporting and operational guidance.

Development

pnpm install --frozen-lockfile
pnpm typecheck
pnpm lint
pnpm test
pnpm build
pnpm exec publint
pnpm pack

CI runs the validation suite on Node.js 22 and 24. Release checks also inspect the packed tarball and scan tracked/package files for common credential patterns. Tests cannot prove the absence of every secret, so review the pack contents before publication.

Current 0.1.0 limitations are documented in the architecture notes. In particular, contract-conflict resolution remains intentionally narrow, and automatic required-validation discovery is limited to configured package.json scripts.

License

MIT

dsh-supervisor

Установка

README