dsh-rs

dsh-rs — in-process search for the DeepSeek Harness

English | 中文

The grep and glob tools of deepseek-ai/deepseek-harness spawn a packaged ripgrep binary per call and parse its --json stdout. This runs ripgrep's own library crates inside the harness process instead, shipped to Node as the napi-rs addon @saidouahdachi/dsh-native and wired into the fork behind a fallback.

Measured on Apple M2 Pro, macOS 26.6.2, Node v25.9.0, --release, 2026-09-05.

The reason this exists

grep "const" under packages/ is an ordinary thing for an agent to ask. ripgrep answers it with 21.1 MB of --json, over the tool's 20 MB raw-output cap, so the spawn path fails and the model gets an error telling it to narrow the search. In-process there is no transport to overflow:

grep "const" in packages/ — 21 MB of rg --json, over the tool's 20 MB cap:
  spawned rg : FAILS  SEARCH_RAW_OUTPUT_OVERFLOW  (68 ms)
  dsh-rs     : ok     Found 250 of 75108 matches  (818 ms)

That is a capability difference, not a latency one, and it is the argument for the addon. The latency table below is the smaller half of the case.

Measured in the harness

bench/harness-bench.mjs runs one workload through the real built harness twice — once per backend, one process each — issuing tool calls through ctx.tools.execute:

Tool call (median of 7)	spawned rg	dsh-rs	speedup
grep, whole packages/ tree	57.7 ms	52.2 ms	1.11x
grep, packages/session	11.0 ms	6.2 ms	1.77x
glob *.ts, whole packages/ tree	69.4 ms	64.6 ms	1.07x
glob *.ts, packages/session	6.1 ms	2.8 ms	2.20x

The fixed spawn cost is what disappears, so narrow searches — the shape of a typical agent call — win most, and a broad walk barely moves. Under concurrency the gap narrows rather than widens (1.38x with one search in flight, 1.17x with eight): threads and processes both queue.

bench/search-bench.mjs measures the same thing one level lower (spawn + --json parse, against the packaged @vscode/ripgrep) and agrees: 1.5x on a whole-tree grep, 1.6x narrow, 1.1–1.2x on glob.

Honest cost: this is a 2–5 ms saving per tool call inside turns that spend seconds in the model. Ship it for the failure mode, not the milliseconds.

Scope

Search is the one hot path in the harness where a native port changes what an agent can do. The other candidates were assessed and left in TypeScript; the reasoning is in docs/ANALYSIS.md so the question does not get asked twice: the session read is JSON.parse-bound, the durable append is fsync-bound, and per-turn token pricing pays more crossing the FFI boundary than V8 spends on the arithmetic.

Installed as a plugin, without a fork

plugin/ is @saidouahdachi/dsh-tool-fs-search-native: a Cordis plugin that registers grep and glob against this addon and ships a patch layer disabling the in-box entry, so a profile picks up the in-process backend through dsh plugin add instead of a forked harness. Names, schemas, guidance, render text, search cards, and spill recovery are the in-box package's own exports, so the swap is invisible to a session; with no usable addon the plugin registers the in-box spawn-backed tools itself.

Integrated into the harness

The fork's master adds @saidouahdachi/dsh-native as a dependency of packages/fs/tool-fs-search and selects it at runtime, with the ripgrep spawn as the fallback: DSH_NATIVE=0, a platform with no prebuilt binary, or an unloadable addon keeps the spawn behaviour exactly. execute calls grepSearchAsync / globSearchAsync uncapped, so the retention and spill layers keep owning the caps and previews, and the work runs on libuv's thread pool — upstream spawns a process, so the tool call must not hold the event loop for the length of a tree walk.

Suite	Result
packages/fs/tool-fs-search	163 pass — the real-filesystem integration suite runs twice, once per backend
full harness suite (vitest run)	18 233 pass, 0 fail — on the fork at upstream 0.1.3-alpha.1
npm run typecheck, oxlint, 45 static gates	clean

tools.spec.ts scripts a fake subprocess for every case, so it pins DSH_NATIVE=0 and owns the spawn transport; integration.spec.ts proves the two backends are indistinguishable to a model against the real filesystem.

Known contract difference. Upstream's cooperative tool timeout terminates the rg process tree. A running in-process search has no process to terminate, so exec.signal is honoured before the search starts and observed when it returns. The 818 ms broad search above is the case to watch.

What parity means here

ripgrep matches every glob — the include filter, the glob pattern, the VCS exclusions — against the path it prints: the path argument joined with the walked suffix. A search running in-process cannot change the process cwd, so the walk resolves an absolute root and reconstructs that printed form for glob matching and display. Globs go through the walker's own ignore::overrides, rooted at the workdir exactly as ripgrep roots them at its process cwd, which is what keeps ripgrep's precedence rule — an explicit positive --glob outranks the hidden-file filter — intact.

bench/search-parity.mjs pins this against the packaged ripgrep binary (resolved from the harness checkout, not whatever rg is on PATH), on identical argv:

• match sets and totals over a fixture tree (gitignore, hidden, binary-with-NUL, invalid-UTF-8, node_modules) and over the upstream source tree;
• the printed-path form for every target shape — undefined, ., a subdirectory, a single file;
• an include glob with a separator, anchored at the workdir (include: 'src/*.ts' under path: 'src');
• one contiguous block per file, in path order, so the 250-match head cap is the same page every run (ripgrep's own parallel file order is not);
• a target at or inside a VCS directory listing nothing, as --glob=!**/.git/** makes ripgrep list nothing;
• a pattern containing a literal newline rejected, as ripgrep rejects it without --multiline;
• --sort=modified order, compared as the mtime sequence — ties break by path here and by walk order in ripgrep, and the two agree over the 4 020 TypeScript files under packages/.

Packaging and release

The addon publishes as a family: @saidouahdachi/dsh-native carries the loader and declares one prebuilt package per target as optional dependencies, each gated by os/cpu, so npm installs only the binary that matches the host. Five targets are covered — darwin arm64/x64, linux x64/arm64 (glibc), and win32 x64. A host outside that set (musl, 32-bit, anything else) resolves no prebuild and searches by spawning ripgrep, which is what the consumer already does when the addon fails to load.

scripts/platforms.mjs is the single matrix every consumer reads. Add a target there, run node scripts/gen-platform-manifests.mjs to write the platform manifests and the entry's optionalDependencies, and the release workflow picks it up; --check fails instead of writing, which is what CI runs.

.github/workflows/release.yml builds each target on its own runner, stages the cdylib with scripts/stage-platform.mjs, and packs it — the platform package's prepack guard rejects a missing, empty, or wrong-OS binary, so a Mach-O file can never ship inside a linux package. Publishing is an explicit workflow input and refuses to run outside a v* tag; the platform packages publish before the entry, so its optional dependencies already resolve when it lands. Rehearsal and publication consume the same tarballs.

The plugin package is not built there. It compiles against the harness packages, whose matching versions are not on npm yet, so it is published by hand until they are.

Tests

./scripts/build.sh                                  # build addon + smoke test
cargo test -p dsh-core                              # 15 Rust unit tests
cd bench && DSH_UPSTREAM=… node search-parity.mjs   # 13 searcher ≡ real rg checks
cd bench && DSH_UPSTREAM=… node search-bench.mjs    # spawn vs in-process
cd bench && DSH_UPSTREAM=… node harness-bench.mjs   # in-harness, both backends

Usage

const dsh = require('./npm')
const hits = await dsh.grepSearchAsync('SearchError', 'packages', '*.ts', cwd)
const files = await dsh.globSearchAsync('*.ts', 'src', cwd)

Four exports: grepSearch / globSearch run on the calling thread, grepSearchAsync / globSearchAsync on libuv's thread pool. The harness uses the async pair.

License

MIT, like upstream.