dsh-auto-classifier

dsh-auto-classifier — autonomous ("auto") mode for DeepSeek Harness

English | 中文

A Claude-Code-auto-mode-like permission classifier for DeepSeek Harness. Adds a fourth permission preset auto (Autonomous) beside read-only / workspace-write / danger-full-access: tool calls are classified automatically — dangerous operations are blocked before they run, safe ones flow, and sandbox escalations are decided by the classifier without a human watching the approval prompts.

How it works

Host-plane plugin. Two { prepend: true } listeners decide BEFORE the browser answerer, active only in sessions whose permission preset is auto; every other session keeps the stock interactive behavior (handlers call next()):

Hook	Role
tools/pre-execute (the tool pre-execution waterfall)	Every tool call sees its name + full arguments: dangerous commands (system-directory deletion, formatting, registry writes, git reset --hard / force push, credential access, …) are denied BEFORE anything executes
approval/request (the approval waterfall)	Sandbox escalations (sandbox_permissions) are auto-allowed / auto-rejected by the classifier — no browser prompt in auto sessions
git snapshot	Before an allowed high-risk escalation the workspace is checkpointed (git add -A && git commit, throttled); auto_snapshot tool snapshots manually anytime
systemPrompt section	Injects autonomous-mode discipline: risk tiers, git rescue, no infinite retry loops, email-and-stop when a human decision is needed
web control page (v0.1.5, bilingual since v0.1.7, model config since v0.1.10, explicit credentials since v0.1.14)	Settings → Plugins → Auto Classifier · 自动分类器: live config toggles (LLM judge, write-content scan, strict default, judge stages, default decision) as phone-style switches, judge-model config (provider / base URL / API key / model fields — the credentials you type are saved and the judge calls that provider directly over HTTPS (OpenAI-compatible or Anthropic), with a Test · 测试 button that pings the exact credentials typed and a Save · 保存 that persists them; a mode badge shows Direct · 直连 vs Service · 服务), session stats, and the recently-denied list — labels are bilingual (EN · 中文). The browser half ships via exports["./client"] + dsh.client; static bundle clients have no host.call, so the page fetches same-origin routes the host half registers on webServer (deferred with ctx.inject(['webServer']) since v0.1.6 — the service mounts after webStartup): GET /dsh-auto-classifier/status, GET /dsh-auto-classifier/denials, POST /dsh-auto-classifier/set (whitelisted live-config keys only), GET/POST /dsh-auto-classifier/model (dedicated judge-model endpoint; provider/baseURL/apiKey/model persisted, the API key is never returned — only a mask), POST /dsh-auto-classifier/model/test (live ping with the credentials typed in the form, 20s cap). Toggles apply immediately to the running classifier and persist across restarts (config.json)

Rule engine (Claude Code style, tool-scoped)

• Rule syntax: Tool(pattern) — pwsh(^git\s+commit\b), write,edit(C:\Windows); comma-separated tool names allowed; a bare regex without parens applies to every tool. Patterns are case-insensitive regexes.
• Field projection (no false positives): command tools (pwsh/bash/run_code) scan only command/code (cleaned first: env-var prefixes FOO=bar and > / >> redirections stripped); path tools (write/edit/read) scan only file_path; other tools are not scanned — text in file content or code that merely mentions a dangerous word is not a false positive. Approval requests scan the reason sentence.
• Order: deny rules (highest priority) → in-workspace structural allow (write/edit/read only) → allow rules → LLM judge (optional) → pwsh strict default → defaultDecision.
• In-workspace file operations are never judged (Claude Code Tier-1, since v0.1.3): write/edit/read whose resolved target lies inside the session workspace (agent.session.header.cwd) are allowed deterministically — edits inside the project are reviewable via version control, and the file sandbox already confines the write. Deny rules still run first, so protected targets (.dsh configs, credentials, system dirs, .git/config|hooks metadata) stay blocked.
• Default deny table (inspired by Claude Code soft_deny): destructive file ops, disk/boot/system servicing, shutdown/reboot, account/service tampering, registry writes, git push / force-push / reset --hard, external code execution (curl|bash, iwr|iex), privilege escalation (sudo/runas/gsudo), network listeners, system-level package installs, credential files, protected-path writes, DSH/shell config tampering.
• Default allow table: routine git (no push), package managers, language runtimes, gh/dsh/winget/scoop/choco, PowerShell cmdlets.
• LLM judge (llmJudge: true): when no rule matches, the model decides semantically. It mirrors Claude Code's production classifier (a readable, un-escaped copy of that prompt lives at reference/claude-automode-classifier-reference.md):
  • Two-stage (judgeStages: "both", default): a fast single-token filter (cheap, errs toward blocking — Claude: FPR 8.5%) is followed by a chain-of-thought re-review only when the filter flags the call (FPR → 0.4%). "fast" / "thinking" run a single stage.
  • Reasoning-blind input: the judge sees the session's recent user messages (authorization evidence) + recent tool calls (name + projected target only, results and payload content stripped) + workspace facts (workspace root, resolved path, in-workspace, git repo). Assistant text and tool outputs are stripped.
  • HARD / SOFT tiers (since v0.1.4): HARD (external code exec, irreversible destruction, persistence, security weakening, privilege escalation, network listeners, secret exfiltration) never clears; SOFT (writes outside workspace, package mgmt outside project, git push, shared-state deletion, large-scale changes) clears only on named+specifics user intent — the user's own message names the action AND the dangerous parameter; questions are not consent; a post-block reaffirmation is strengthened consent; explicit boundaries persist until lifted. Out-of-scope items (poor code, wrong answers, looping, wasted tokens, transient retries) are never blocked.
  • suspectedBypass (since v0.1.4): a target denied earlier in the session that is re-attempted through a different tool, or whose path is embedded in a command, is flagged for the judge (bypassWindow recent denials participate). It is a judge signal, not an automatic denial — the judge weighs it against the user-intent evidence.
  • scanWriteContent (since v0.1.4, default false): when on, write/edit payloads that carry a deny-pattern hit are judged NOW, at write time (Claude's WRITTEN FILE EXECUTION), instead of the in-workspace structural allow.
  • Failure degrades to defaultDecision. Applied on the pre-execute path only — escalations use rules + default, since the underlying command was already screened.
• Denial message (denyMessage): classifier denials return a fixed Kafka-style ruling — "Before the classifier stands a doorkeeper. Today the doorkeeper says: not yet…" — stating that this is a classifier judgement (not a sandbox denial, so sandbox_permissions is the wrong channel), to seek another entrance, and that a different tool is re-classified anew. Customize or empty it as you like.
• Denial log: every classifier denial is appended to ~/.dsh/auto-classifier/denials.jsonl (rotated past 1 MB) — a "Recently denied" review trail that survives restarts; auto_status prints the path.
• Denial caps: 3 consecutive / 20 total (same as Claude Code denialTracking) — past a cap the classifier hard-stops and hints the agent to notify the user. DSH has no built-in mail: the hint points to the dsh-notify-skill email plugin (also on the awesome-dsh-plugin list) or any notification channel you configure.
• Every decision is logged via ctx.logger; auto_status shows the last 20 and the cap counters.

Install (web profile)

# 1. Pack and add to the profile's dependencies + bundles (physical tarball — do NOT use link:)
#    cd dsh-auto-classifier && npm pack --cache <workspace-path>   # workspace-write sandbox blocks npm's default cache dir
#    package.json dependencies:  "dsh-auto-classifier": "file:C:/.../dsh-auto-classifier-0.1.0.tgz"
#    package.json dsh.profile.bundles: append "dsh-auto-classifier"
cd ~/.dsh/profiles/web
pnpm add "dsh-auto-classifier@file:C:/.../dsh-auto-classifier-0.1.0.tgz" --force

# 2. Validate the composed config (no server start)
dsh --profile web --dump-config   # auto-classifier row + 4 presets

# 3. Restart dsh web, then switch the session preset to auto (or /permission auto)

The plugin's cordis.patch.yml injects its rows as a bundle patch — never manually insert the same row ids in the profile/home layers (duplicate loader entry kills web startup). After source changes: npm pack → pnpm add ... --force (refreshes lockfile integrity).

Updating

The plugin's cordis.patch.yml ships inside the package, so the injected rows update with it — no manual profile-layer edits. Prefer the official dsh plugin form (it forwards to pnpm and reconciles dsh.profile.bundles against the installed state, so a package that gains its dsh.bundle declaration in a newer version is activated automatically, and re-adding an already-listed bundle never duplicates the entry):

dsh plugin --profile web update dsh-auto-classifier              # respects the ^0.1.x range
# or pin explicitly (add --registry=https://registry.npmjs.org/ while npmmirror lags):
dsh plugin --profile web add "dsh-auto-classifier@0.1.3" --registry=https://registry.npmjs.org/

dsh --profile web --dump-config                                 # verify: exit 0 + auto-classifier row

Then restart dsh web. auto_status reports the live config — 0.1.3+ shows stages: both in the llmJudge line. A plain pnpm update inside the profile dir also bumps the version but skips the bundle reconciliation (fine for an in-place bump). If the previous install used a local file:...tgz, remove it first (dsh plugin --profile web remove dsh-auto-classifier) before adding from the registry, to avoid a duplicate injection.

Configuration (auto-classifier row in cordis.patch.yml)

Key	Default	Meaning
presetName	auto	The permission preset under which the classifier is authoritative
defaultDecision	allow	Decision when no rule matches and no judge (deny = fail closed)
llmJudge	false	Enable the model semantic judge (one call per unmatched operation)
judgeProvider / judgeBaseURL / judgeApiKey / judgeModel	empty	Explicit judge credentials (since v0.1.14), set in the web control page. When a non-empty API key + model + base URL are present, the judge calls that provider directly over HTTPS (OpenAI-compatible /chat/completions, or Anthropic /messages when the provider/base URL is Anthropic) with zero dependency on the harness llm registry or its config-file credentials. Empty judgeApiKey falls back to llmProvider/llmModel below.
llmProvider / llmModel	deepseek-official / deepseek-v4-flash	Judge model via the harness llm service (fallback when no direct credentials are set)
judgeStages	both	both = fast filter + thinking re-review on flag; fast / thinking = single stage
denyMessage	Kafka doorkeeper copy	Fixed text appended to classifier denials (explains the sandbox_permissions channel does not apply); empty to disable
hardDenyMessage	hard-boundary note	Extra sentence appended when a deny RULE (HARD) blocked the call — suggests running the step outside auto mode
judgeMaxUserMessages / judgeUserMessageChars	3 / 400	How many recent user messages (truncated) the judge sees as intent evidence
judgeToolHistory	6	How many recent tool calls (name + target, stripped) the judge sees for session context
judgeFastMaxTokens / judgeThinkingTimeoutMs	8 / 30000	Stage-1 output cap and stage-2 abort timeout
scanWriteContent / writeContentScanChars	false / 2000	When on, judge write/edit payloads that carry a deny-pattern hit (content truncated to this many chars)
bypassWindow	5	How many recent denials participate in suspectedBypass matching
pwshStrict	true	Unmatched command-tool calls go to the judge / strict default (more conservative than path tools, mirroring Claude Code's default for PowerShell)
denyPatterns / allowPatterns	built-in	Tool(pattern) regex arrays, override or extend
denialLimitConsecutive / denialLimitTotal	3 / 20	Denial caps, hard-stop hint past them
gitSnapshotOnAllow	true	Snapshot the git worktree before allowing an escalation
gitSnapshotIntervalMs	30000	Per-session snapshot interval floor

License

MIT