dsh-web-search-ext
fno2010/dsh-web-search-ext
Multi-backend web_search provider for the DeepSeek Harness web seam (ctx.web): Exa (REST with key, anonymous hosted MCP without) and Firecrawl (v2 search API) today, extensible to more backends (SearXNG, ...), with automatic failover when one backend rate
安装
dsh plugin --profile web add github:fno2010/dsh-web-search-extREADME
dsh-web-search-ext
English | 中文
Multi-backend web_search provider for DeepSeek Harness (DSH). Works with no API keys at all; add keys to unlock higher limits. Registered into the web capability seam (ctx.web) under one stable provider id (web-search-ext).
Why
The built-in web_search tool is backend-pluggable; the in-box default provider (deepseek-official) requires a DeepSeek API key. This plugin is a key-free-capable alternative: it works out of the box via Exa's anonymous MCP endpoint, and fails over automatically when one backend saturates.
Features
- Two backends today: Exa (REST with key, anonymous hosted MCP without) and Firecrawl (v2 search API, keyed or keyless)
- Automatic failover: on any backend failure (429, 401/402/403, 5xx, network, malformed body) the search falls through to the next backend in preference order
- Per-backend 429 cooldown (default 60 s): a saturated backend is skipped on subsequent searches; when all backends fail, the error lists every failure including cooldown state
- Optional keys with per-backend precedence: settings literal → credentials service → launch environment variable
- No install-time scripts: plain ESM JavaScript, no build step, no
postinstall/prepare - Extensible: adding a backend is one search function + one plan entry + config fields — see CONTRIBUTING
Backends
| Backend | With key | Without key |
|---|---|---|
| Exa | REST POST https://api.exa.ai/search (higher limits, highlight snippets) | Anonymous hosted MCP POST https://mcp.exa.ai/mcp (JSON-RPC 2.0, documented public fallback, rate-limited → HTTP 429) |
| Firecrawl | POST https://api.firecrawl.dev/v2/search (Bearer) | Keyless requests when firecrawlKeyless: true (unofficial; may be rate-limited or removed) |
Install
dsh plugin --profile web add @fno2010/dsh-web-search-ext
# or from a local checkout:
dsh plugin --profile web add ./path/to/dsh-web-search-ext
Installing a plugin requires restarting the running dsh web process (the profile bundle list is resolved at boot). Config changes afterwards are hot — no restart.
The bundle patch selects this provider for the web_search tool by setting web.searchProvider: web-search-ext. The official deepseek-official provider stays registered but unused; an explicit selection also prevents WEB_PROVIDER_AMBIGUOUS.
Configuration
Settings namespace web-search-ext in ~/.dsh/settings.yaml (hot-reloaded):
| Field | Default | Description |
|---|---|---|
preferred | exa | Backend to try first: exa | firecrawl |
numResults | 8 | Default result count when the tool doesn't cap it |
maxSnippetChars | 500 | Snippet length bound |
rateLimitCooldownSec | 60 | Skip a 429'd backend this long; 0 disables |
firecrawlKeyless | true | Allow keyless Firecrawl requests |
exaApiKey / firecrawlApiKey | — | Literal API key per backend |
exaApiKeyEnv / firecrawlApiKeyEnv | EXA_API_KEY / FIRECRAWL_API_KEY | Env var names for key resolution |
exaApiUrl / exaMcpUrl / firecrawlBaseUrl | https://api.exa.ai/search / https://mcp.exa.ai/mcp / https://api.firecrawl.dev/v2 | Endpoint overrides |
web-search-ext:
preferred: exa
numResults: 8
# rateLimitCooldownSec: 60 # all other values are defaults
Or select this provider without the bundle patch: DSH_WEB_SEARCH_PROVIDER=web-search-ext.
Keys (optional but recommended)
Any of these, in order of precedence per backend:
- Literal key in the settings section (
exaApiKey/firecrawlApiKey) - Credentials service: the
EXA_API_KEY/FIRECRAWL_API_KEYentries in~/.dsh/.credentials.yaml(the Web "Models" page manages LLM provider credentials only and has no field for these — add them by editing the file; a settings-UI entry point is tracked as a feature request) - Launch environment variable of the same name
No keys at all still works: Exa uses its anonymous MCP endpoint and Firecrawl is tried keyless.
How failover works
Each search builds an ordered plan (preferred backend first) from the backends that are available under the current key situation. The first backend whose request fails is reported as the failure only if every later backend also fails — a 429 additionally starts that backend's cooldown so it is skipped on subsequent searches until the window expires.
Uninstall
dsh plugin --profile web remove @fno2010/dsh-web-search-ext # then restart dsh web
Security notes
- The only outbound requests are to the configured Exa and Firecrawl endpoints; nothing else is contacted.
- API keys travel only in the
authorizationheader of their own backend's requests — never in bodies, never to the other backend, never in error messages. - No install-time scripts: plain ESM JavaScript, no build step, no
postinstall/prepare. - Snippets are bounded (
maxSnippetChars) and Firecrawl's page-markdown descriptions are stripped of image links before entering model context.
Development
- Tests:
npm test— 10 mocked failover/mapping scenarios plus live keyless smoke calls (smoke is skipped in CI). - Adding a backend, branch/PR conventions, and the release process: CONTRIBUTING.md.