dsh-vision-worker

@try-works/dsh-vision-worker

A DeepSeek Harness plugin that gives text-only models real vision.

When a caller cannot see images, dsh-vision-worker routes the image to Cloudflare Workers AI @cf/moonshotai/kimi-k2.6 (vision-capable) and returns a deterministic, versioned envelope — so the text-only caller can act on structured facts instead of guessing from prose.

The problem

• Text-only models reject image content, so they cannot read a website screenshot, a photo, a chart, or a scanned table.
• Handing them raw vision prose forces re-parsing and loses structure.

The fix

One call returns a righthand.vision.v1 envelope:

{
  "schema": "righthand.vision.v1",
  "model": "@cf/moonshotai/kimi-k2.6",
  "status": "ok",
  "summary": "...",
  "answer": "...",
  "content": {
    "text": [{"value": "...", "confidence": 0.97, "bbox": [x,y,w,h]}],
    "measurements": [{"name": "...", "value": 12, "unit": "%"}],
    "tables": [{"name": "...", "columns": [...], "rows": [[...]]}],
    "code": [{"language": "...", "text": "..."}],
    "entities": [{"type": "...", "value": "..."}]
  },
  "actions": ["verify the Q3 total against the source table"],
  "confidence": 0.9,
  "warnings": [],
  "images": []
}

Install

dsh plugin --profile <profile> add @try-works/dsh-vision-worker

Or apply the overlay directly:

dsh --profile <profile> --patch ./cordis.patch.yml

Configure

Set one of these transports (the plugin tries workerUrl first, then accountId + apiTokenRef):

# cordis.patch.yml
- insert:
    - id: dsh-vision-worker
      name: '@try-works/dsh-vision-worker'
      config:
        # Deployed worker (see cloud/):
        workerUrl: 'https://vision-worker.<you>.workers.dev'
        # …or Workers AI REST directly:
        # accountId: '<your Cloudflare account id>'
        # apiTokenRef: 'CLOUDFLARE_API_TOKEN'
        # model: '@cf/moonshotai/kimi-k2.6'
        # locale: 'en'

Store the token with the harness credential provider (never inline):

rh_credential_set ref=CLOUDFLARE_API_TOKEN value=<token>

Tools

Tool	Purpose
vw_analyze	Analyze image(s) with a prompt → envelope + callerText
vw_ask	Follow-up question about the same image(s)
vw_status	Report active transport/model without exposing the token
vw_locale	Get/set UI language (en or zh-CN)

Tool names are stable identifiers; descriptions and rendered status text localize live via the locale setting.

Deploy the worker

cd cloud
npx wrangler deploy

cloud/index.js is a zero-build Worker exposing POST /analyze (plus GET /health). It takes { prompt, images, system? } and returns the same righthand.vision.v1 envelope.

Test

pnpm install
pnpm test          # 20 tests: core, schema, transport, tools, fixtures
pnpm typecheck
node --experimental-strip-types test.ts   # inside cloud/ — worker handler test

Routing architecture

The observable Workers AI routing protocol (name→effective-model aliasing, task-family dispatch, result-envelope shapes, gating) is documented in docs/workers-ai-routing-protocol.md, derived from live probes of every cataloged model.

Layout

• src/core/vision.ts — capability routing, multimodal input, normalization, multi-step (transport injected; zero deps).
• src/core/schema.ts — the envelope format, prompt template, tolerant parser, and callerText renderer.
• src/transport.ts — Workers AI REST client + deployed-worker client.
• src/vision-tools.ts — the DSH-native tool surface (ctx.tools/ ctx.settings/ctx.credentials).
• src/i18n.ts — EN + zh-CN string tables.
• src/fixture.ts — deterministic PNG fixtures (real bytes, no deps).
• cloud/ — deployable Worker.