dsh-rule-evolve (plugin)

dsh-rule-evolve

Verification-driven self-evolution loop for DeepSeek Harness.

Renamed from dsh-evolve (old URL redirects). Reason: another community project (william-jin-cmu/dsh-evolve) already uses that name for in-session plugin hot-mounting. Different focus: that project evolves which plugins are mounted; this project evolves which rules are trusted, with verification gates. The CLI command stays dsh-evolve for compatibility (dsh-rule-evolve is also available).

Agents keep repeating the same expensive lessons: a crash fixed last week gets re-debugged today, a Windows quirk gets rediscovered, an install trap gets re-hit. dsh-evolve turns that loop into a traceable pipeline:

experience (markdown)  →  learn  →  rules (AGENTS.md)  →  verify (real checks)

Every rule carries its source and its last verification result — evolution is auditable, and nothing is ever "learned" without being checked.

Quick start

node scripts/dsh-evolve.mjs learn --from docs/troubleshooting.md --out experience.jsonl
node scripts/dsh-evolve.mjs rules --experience experience.jsonl --out AGENTS.md
node scripts/dsh-evolve.mjs verify --experience experience.jsonl --dir ./my-plugin

verify runs the real check pipeline (dsh-plugin-doctor check <dir> by default, override with --cmd) and stamps every entry verified: true/false.

Self-improvement loop (v0.2.0)

Full evolution with reflection, profile installation and an evolution log:

# 1. reflect on a completed task/retrospective
node scripts/dsh-evolve.mjs reflect --task "make my plugin pass its own checks" --result retro.md --out experience.jsonl

# 2. evolve: verify rules against the real repo, install them into a dsh profile,
#    and append one round to EVOLUTION.md
node scripts/dsh-evolve.mjs evolve --experience experience.jsonl --dir ./my-plugin --profile web --log EVOLUTION.md

The rules land in <DSH_HOME>/profiles/web/AGENTS.md (previous file backed up), so the next session actually starts with the lessons. Every round is logged:

## Round 1 — 2026-08-15T…
- New rules: 5
- Verified: yes ✅
- Sources: examples/doctor-experience.md
- Command: dsh-evolve evolve --experience … --dir … --profile web

Dogfood: our doctor repo now passes its own check with 5 verified rules installed into a profile (see examples/demo/EVOLUTION.md).

Learning from failure logs (v0.3.0)

extract turns raw failure logs into conditional rules automatically — no manual retrospective needed:

node scripts/dsh-evolve.mjs extract --task "publish to npm" --from install.log \
  --out experience.jsonl --hint "configure the credential and retry"

Every ERROR/failed/ERR_ line becomes a rule: When "<error>" occurs, <hint>. Noise lines (progress/warnings/success) are skipped; rules are deduplicated and tagged.

audit keeps the rule library healthy:

node scripts/dsh-evolve.mjs audit --experience experience.jsonl

It reports totals, verified/unverified counts, tag and source distribution, and near-duplicate rule pairs (Jaccard ≥ 0.7) so overlapping lessons can be merged. Dogfood: extracted 6 rules from our real npm ENEEDAUTH failure log.

Tool learning (v0.5.0)

tool-verify is the "agent wrote a tool → is it ready?" gate:

node scripts/dsh-evolve.mjs tool-verify --dir ./my-plugin --out readiness.md \
  --experience experience.jsonl

It runs the real readiness pipeline (dsh-plugin-doctor check by default), renders a readiness report (checks table, pass/warn/fail, overall READY/ISSUES), and learns: every failing/warning check becomes an experience entry (stamped with the verification result). A tool that ships with failures therefore teaches the next tool how not to ship.

Dogfood: our doctor repo reports 8/8 PASS → READY ✅; before v1.10.1 this same gate caught its own lint bug.

Rule lifecycle (v0.6.0)

Rules get a health score and the library self-maintains — no manual cleanup:

node scripts/dsh-evolve.mjs score --experience experience.jsonl        # score + lifecycle table
node scripts/dsh-evolve.mjs prune --experience experience.jsonl --min-score 3 [--dry-run]
node scripts/dsh-evolve.mjs merge-duplicates --experience experience.jsonl [--threshold 0.7] [--dry-run]

• score: verified count, usage count, recency → active / stale
• prune: soft-retires low-score rules (retired: true, reason kept — auditable, never deleted)
• merge-duplicates: near-duplicate pairs merge into the higher-scoring rule (mergedInto kept)

The library tells you what to keep, what to merge, and what to retire — that is the "memory that forgets usefully" half of long-term agent learning.

Cross-session reinforcement (v0.7.0)

Rules now carry usage and verification history:

node scripts/dsh-evolve.mjs touch --experience experience.jsonl --id EXP-001

Every successful verify/evolve/tool-verify increments verifiedCount; every touch (or the plugin's evolve_touch tool) increments usageCount. The health score already rewards both — so rules that are repeatedly verified and repeatedly used survive pruning, while stale rules fade. That is cross-session reinforcement: the library remembers what works.

Local recall (v0.8.0)

recall is a zero-dependency BM25-lite retrieval over the rule library — the retrieval half of #1881's Layer 2 without needing an embedding model:

node scripts/dsh-evolve.mjs recall --experience experience.jsonl --query "windows bash" --limit 5
# [3.28] EXP-003 ✅ — Never rely on /bin/bash as a default shellPath on Windows…

Verified and frequently used rules rank higher. Works offline, no model, no network — and it is the seam where a real embedding provider can plug in later.

Agent Growth Dashboard (v0.10.0)

Turn the rule library into something you can see and share:

node scripts/dsh-evolve.mjs dash --experience experience.jsonl --evolution EVOLUTION.md --out agent-growth.html

One self-contained HTML page (no dependencies, opens offline): rule growth curve, library health, what the agent learned, most-used rules, evolution timeline, and a shareable Agent Growth Report card. Bilingual.

Live example: examples/marathon/agent-growth.html — the actual growth diary from the self-evolution marathon.

One command for the whole loop:

node scripts/dsh-evolve.mjs loop --from docs/troubleshooting.md --out experience.jsonl --dir ./my-plugin

Evolution badge (v0.11.0)

One command turns the rule library into a README-ready SVG badge:

dsh-evolve badge --experience experience.jsonl --evolution EVOLUTION.md --out badge.svg

Tiers by verified-rule count: starting (0) · learning (1-4) · building (5-9) · growing (10-19) · legend (20+). Add --json for the machine-readable summary plus the SVG payload.

How rules get into the agent

The generated block is plain AGENTS.md — drop it into your repo root or dsh profile rules, and every future session sees it:

## Self-evolution rules

> Auto-generated by dsh-evolve. Rules are traceable to their source and to the last verification run.

- [EXP-001] Never run recursive directory listings through node_modules; prefer Glob/Grep. _(source: docs/troubleshooting.md · ✅ verified)_
- [EXP-002] allowBuilds entries are required before git installs can run prepare scripts. _(source: docs/troubleshooting.md · ○ unverified)_

Design

• Zero runtime dependencies, Node ≥ 18.
• Experience entries are JSONL: { id, rule, source, tags, addedAt, verified, lastVerifiedAt }.
• Dedup by rule hash; tags are inferred heuristics (install/crash/windows/test/memory/security/performance).
• The verify step is pluggable: point --cmd at any command that exits 0 on success.

Why this fits dsh

Official discussion #1881 proposes triple-layer persistent memory; its Layer 1 is Git-tracked Markdown rules. dsh-evolve is the production and validation half of that layer — where rules come from, and how they stay honest. The ecosystem already has plenty of memory engines; it has no verification-driven rule evolution loop.

License

MIT

中文说明

面向 DeepSeek Harness 的、带验证的智能体自我进化循环。

项目已从 dsh-evolve 改名 dsh-rule-evolve（旧地址会跳转）。CLI 命令仍保留 dsh-evolve 以兼容。

智能体会反复踩同一个坑：上周修好的崩溃这周又 debug 一遍，Windows 的怪癖被重新发现，安装陷阱被再次触发。这个项目把这个循环变成一条可追踪的流水线：

经验（markdown）→ learn → 规则（AGENTS.md）→ verify（真实检查）

每条规则都带着来源和最近一次验证结果——进化全程可审计，没有经过检查的东西永远不会被「学到」。

快速上手

node scripts/dsh-evolve.mjs learn --from docs/troubleshooting.md --out experience.jsonl
node scripts/dsh-evolve.mjs rules --experience experience.jsonl --out AGENTS.md
node scripts/dsh-evolve.mjs verify --experience experience.jsonl --dir ./my-plugin

verify 会跑真实的检查流水线（默认是 dsh-plugin-doctor check <dir>，可用 --cmd 覆盖），并把每条经验盖上 verified: true/false。

核心命令一览

命令	作用
learn	从 markdown 经验文档提取规则
rules	把规则库渲染成 AGENTS.md
verify	用真实检查验证每条规则
reflect	对完成的任务/复盘做反思
evolve	验证规则 + 装进 dsh profile + 追加进化日志
extract	从失败日志自动提取「当 X 出错时做 Y」的条件规则
audit	规则库健康报告（去重、来源分布、近重复）
tool-verify	「agent 写了个工具 → 它能用吗？」就绪门禁
score / prune / merge-duplicates	规则打分、软淘汰、合并近重复
touch	记录规则被使用（跨会话强化）
recall	零依赖 BM25 检索规则库
dash	生成可分享的 Agent 成长报告 HTML（双语）
badge	把规则库生成 README 用 SVG 徽章
loop	一条命令跑完整循环

规则怎么进入 agent

生成的是纯文本 AGENTS.md，放到仓库根目录或 dsh profile 规则里，以后每个会话都能看到：

## Self-evolution rules
- [EXP-001] Never run recursive directory listings through node_modules; prefer Glob/Grep. _(source: docs/troubleshooting.md · ✅ verified)_

设计要点

• 零运行时依赖，Node ≥ 18。
• 经验条目是 JSONL：{ id, rule, source, tags, addedAt, verified, lastVerifiedAt }。
• 按规则哈希去重；标签是启发式推断（install/crash/windows/test/memory/security/performance）。
• 验证步骤可插拔：用 --cmd 指向任意「成功即退出 0」的命令。

为什么契合 dsh

官方讨论 #1881 提出了三层持久记忆；其中第一层是 Git 托管的 Markdown 规则。dsh-rule-evolve 就是这一层的生产和验证半边——规则从哪来，以及它们如何保持诚实。生态里已经有很多记忆引擎，但还没有一个带验证驱动的规则进化循环。

许可

MIT