dsh-browser-vision

English  |  简体中文

dsh-browser-vision — a browser tool for DeepSeek that can see the page

A fully contained browser-use tool driven by DeepSeek through DeepSeek's OpenAI-compatible API. It is designed as a cheap and fast alternative to running browser-use with a general-purpose model, and to be callable from any agent or harness (Codex, Claude Code, OpenCode, custom evals, CI, ...) via a plain CLI with a machine-readable JSON mode.

Since DeepSeek shipped deepseek-v4-flash-vision-exp, the agent can look at the page, not just read its DOM. That model is priced identically to the text Flash model, and a screenshot costs at most 384 input tokens, so vision is no longer a premium feature to be switched off for cost. It is on by default, and the eval suite says that is also the cheapest setting.

What makes it cheap and fast (all overridable via env):

• DeepSeek Flash (deepseek-v4-flash) / Flash Vision (deepseek-v4-flash-vision-exp) — far cheaper and faster than GPT-class models, and the same price as each other.
• Vision by default (DSBROWSER_VISION_MODE=on) — a screenshot every step, because a blind agent burns whole step budgets on tasks it cannot see, and steps cost more than pictures.
• Flash mode, no judge, no planning — fewer extra LLM calls per task.
• One shared browser — a single Chrome instance is reused across all tasks in a process; the eval suite runs all cases in one browser instead of one Chrome per case.

Vision

DSBROWSER_VISION_MODE (or --vision) has three settings:

Mode	Model	Behaviour	Use it when
on (default)	deepseek-v4-flash-vision-exp	A screenshot rides along with every step	General use, and everything visual
auto	deepseek-v4-flash-vision-exp	DOM-driven; browser-use registers a screenshot tool the agent calls when it decides it needs one	Long text-heavy runs where most steps genuinely need no picture
off	deepseek-v4-flash	DOM only, no screenshots ever	Pure text and form work, or when no vision quota is available

Vision works the same headless and headed — screenshots come off the same CDP call either way. dsbrowser "task" --no-headless --vision on is a supported way to watch the agent look at a page, and the eval suite takes --no-headless for exactly that.

What vision unlocks, which the DOM cannot answer at all:

• text drawn on a <canvas> (voucher codes, editors, map labels)
• text baked into an image (scanned invoices, screenshots-in-pages, banners)
• charts and any answer that lives in a shape rather than a string
• "which of these is highlighted / greyed out / overlapping" layout questions

Measured

Six cases, three attempts each, per mode, on the offline fixture suite (dsbrowser-eval --vision all --repeat 3), off-peak pricing:

mode	passed	DOM cases	vision cases	median latency	screenshots	cost (USD)	cost (CNY)
off	7/18	7/9	0/9	33.7s	0	$0.1021	¥0.6998
auto	14/18	5/9	9/9	8.3s	10	$0.0310	¥0.2132
on	18/18	9/9	9/9	6.6s	111	$0.0318	¥0.2180

The surprise is the cost column. on sent 111 screenshots and still cost a third of what off cost sending none. A blind agent does not fail fast, it flails: it spends its whole 20-step budget re-reading a DOM that does not contain the answer, and twenty text steps cost far more than one 384-token picture. Vision is not a premium you pay for accuracy here — it is what makes the run short enough to be cheap.

auto is the weak middle. It solved every vision case, but it did worse on the plain DOM cases than either extreme, and it saves nothing over on.

One machine, one afternoon, n=3. Re-run it yourself before trusting it: dsbrowser-eval --vision all --repeat 5.

Cost of looking

DeepSeek resizes every image so its pixel count is roughly that of an 800×800 image, which caps an image at 384 input tokens. At the off-peak cache-miss rate of $0.22 (¥1.5) per 1M input tokens, a screenshot costs about $0.00008 / ¥0.00058. A whole 20-step run with a screenshot at every step adds under a cent.

Both currencies come from DeepSeek's own price list, the USD one from the English page and the CNY one from the Chinese page. They are not conversions of each other, so they will not match any spot exchange rate. Peak rates are double, in the windows Beijing 09:00-12:00 and 14:00-18:00 (01:00-04:00 and 06:00-10:00 UTC); the reported cost accounts for which window the run fell in.

Two consequences worth knowing:

• Never use detail: low. It crops to 512×512 first, and on-page text stops being legible — measured on the fixture, the model invented a confirmation code rather than reading it. Every image this tool sends is pinned to detail: high.
• Downscale the capture, not the detail. DSBROWSER_SCREENSHOT_SIZE=1024x768 shrinks the upload without changing what DeepSeek tokenizes.

Use it from DeepSeek Harness

One line

Paste this to your agent and it will do the whole setup:

Install the dsh-browser-vision plugin into my DeepSeek Harness profile. Run `dsh plugin --profile web add "github:tristan-mcinnis/dsh-browser-vision"` to add the plugin, then `uv tool install "git+https://github.com/tristan-mcinnis/dsh-browser-vision"` to install the Python engine (pipx or pip --user also work). Check that DEEPSEEK_API_KEY is set in my shell environment and tell me if it is not — do not ask me to paste the key into a config file. Restart the profile, then call the browser_vision_status tool and show me what it reports. If it says the engine is unavailable, follow the hint it gives.

Or run it yourself:

dsh plugin --profile web add "github:tristan-mcinnis/dsh-browser-vision" && uv tool install "git+https://github.com/tristan-mcinnis/dsh-browser-vision"

Restart the profile afterwards (the web profile has HMR disabled). Put DEEPSEEK_API_KEY in the environment — the plugin never stores or forwards a key, it only inherits the environment the engine runs in. Then browser_vision_status tells you whether the engine is reachable.

Tools

Tool	What it does
browser_vision_task	Run a web task in plain language and get the answer. Navigates, clicks, types, fills forms, and reads what is only visible as pixels.
browser_vision_extract	Extract fields from a page as JSON validated against a schema you pass in, so nothing has to be re-parsed downstream.
browser_vision_status	Report whether the engine is installed, so an install problem is distinguishable from a task failure.

Plugin options (in cordis.patch.yml, or Settings → Plugins):

Option	Default	Meaning
command	dsbrowser	Path to the engine executable
vision	on	Default vision mode for tool calls
maxSteps	20	Default step cap
headless	true	Run Chrome headless
timeoutMs	300000	Hard timeout for one task

Requirements

• Python 3.11 or 3.12
• A Chrome/Chromium binary on the system (browser-use ≥0.13 drives Chrome over CDP — Playwright is not used, so playwright install is neither needed nor available)

Install

With uv (recommended):

cd ~/Documents/code/deepseek-browser-use
uv venv --python 3.12 .venv
VIRTUAL_ENV=.venv uv pip install -e '.[dev]'
cp .env.example .env   # then add your DEEPSEEK_API_KEY

With pip:

python3.12 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

This installs the dsbrowser package plus two console scripts: dsbrowser (alias deepseek-browser) and dsbrowser-eval.

Secrets: plain env or 1Password CLI

The DeepSeek key can come from the environment/.env (DEEPSEEK_API_KEY) or from 1Password through the op CLI. Point DEEPSEEK_API_KEY_OP at a secret reference and the tool resolves it with op read --no-newline:

# .env
DEEPSEEK_API_KEY_OP="op://Personal/DeepSeek/api key"

op:// references also work directly as DEEPSEEK_API_KEY. The op CLI must be installed and authenticated (op signin, or export OP_SERVICE_ACCOUNT_TOKEN for headless/CI use). No keychain involvement in the tool itself — and Chromium is explicitly steered away from the macOS keychain too (see below).

CLI

dsbrowser "Open example.com and return its page title"
dsbrowser "Read the voucher code drawn on the canvas at example.com/ticket" --vision on
dsbrowser "task" --vision off              # DOM only, cheapest floor
dsbrowser "task" --json                    # machine-readable output
dsbrowser "task" --max-steps 40            # override step cap
dsbrowser "task" --no-headless --vision on # watch it, and let it look
dsbrowser "task" --model deepseek-v4-pro
python -m dsbrowser "task"                 # module form

Structured output

Reading a scanned invoice with --schema, which is the whole tool in one call — see the page, extract the fields, return validated JSON:

{
  "invoiceNumber": "5512-B",
  "totalDue": "$128.40",
  "lineItems": [
    {"label": "Standing desk", "amount": "$96.00"},
    {"label": "Cable tray", "amount": "$18.40"},
    {"label": "Delivery", "amount": "$14.00"}
  ]
}

Nothing above is in the DOM: the invoice is a PNG. One LLM call, 10.9s, $0.0013 (¥0.0089).

Pass a JSON Schema and the answer comes back as a validated object instead of prose, so a calling agent does not need a second model call to parse it:

dsbrowser "Open https://example.com/pricing and list every plan" --json --schema '{
  "type": "object",
  "properties": {
    "plans": {"type": "array", "items": {
      "type": "object",
      "properties": {"name": {"type": "string"}, "price": {"type": "string"}},
      "required": ["name", "price"]
    }}
  },
  "required": ["plans"]
}'

--schema also takes a path to a .json file. Objects, arrays, primitives, enums, unions and local $refs are supported; the top level must be an object, so wrap a bare list in a property. Optional properties come back as null rather than forcing the agent to invent a value it could not find.

For prose rather than fields, --markdown asks for the page content as clean Markdown with the navigation, ads and cookie banners left out.

JSON contract (for agents and harnesses)

With --json, stdout carries exactly one JSON object and stderr stays clean:

{
  "tool": "dsbrowser",
  "version": "0.2.0",
  "success": true,
  "result": "PLUM-4482",
  "steps": 3,
  "duration_s": 14.2,
  "error": null,
  "model": "deepseek-v4-flash-vision-exp",
  "vision_mode": "auto",
  "usage": {
    "llm_calls": 4,
    "images_sent": 1,
    "prompt_tokens": 18422,
    "completion_tokens": 611,
    "cached_tokens": 12800,
    "cost_usd": 0.00119,
    "cost_cny": 0.00812
  }
}

usage.images_sent is the number of screenshots the model actually looked at — in auto mode that is how you tell whether the agent needed to see the page. cost_usd and cost_cny use DeepSeek's published rates in each currency, and account for the peak/off-peak window and cache hits. They are estimates, not a bill.

Exit codes: 0 task completed · 1 task failed (agent/network/model error, error is set) · 2 configuration error (e.g. missing DEEPSEEK_API_KEY, or vision on with a model that has no image input).

Calling it from other agents

Any harness that can run a shell command can use it:

result=$(dsbrowser "Summarize the pricing page at https://example.com/pricing" --json)
# then parse $result with jq: .result, .steps, .usage.cost_usd, .error

# Python harness
import json, subprocess
out = subprocess.run(["dsbrowser", task, "--json"], capture_output=True, text=True, check=False)
payload = json.loads(out.stdout)
print(payload["result"], payload["usage"]["cost_usd"])

Or use it as a library:

import asyncio
from dataclasses import replace
from dsbrowser import Config, DeepSeekBrowserAgent

async def main():
    config = replace(Config.from_env(), vision_mode="on", headless=False)
    async with DeepSeekBrowserAgent(config=config) as agent:
        outcome = await agent.run_task("Read the code on the canvas at example.com/ticket")
        print(outcome.result, outcome.images_sent, outcome.cost_usd)

asyncio.run(main())

Configuration

Variable	Default	Meaning
DEEPSEEK_API_KEY	—	required (or DEEPSEEK_API_KEY_OP)
DEEPSEEK_API_KEY_OP	—	1Password CLI secret reference (op://Vault/Item/field) resolved via op read
DEEPSEEK_BASE_URL	https://api.deepseek.com	OpenAI-compatible endpoint
DEEPSEEK_MODEL	deepseek-v4-flash	model used when vision is off
DEEPSEEK_VISION_MODEL	deepseek-v4-flash-vision-exp	model used when vision is auto or on
DSBROWSER_VISION_MODE	auto	off · auto · on (legacy DSBROWSER_USE_VISION=true/false still parses)
DSBROWSER_SCREENSHOT_SIZE	—	WIDTHxHEIGHT to downscale screenshots before upload
HEADLESS	true	headless browser (vision works either way)
MAX_STEPS	20	max agent steps per task
DSBROWSER_CHROME_ARGS	--password-store=basic,--use-mock-keychain	extra Chrome flags; defaults keep Chromium off the macOS keychain
DSBROWSER_FLASH_MODE	true	browser-use flash-mode prompt optimization
DSBROWSER_USE_THINKING	false	agent step-by-step thinking (slower, sometimes better)
DSBROWSER_USE_JUDGE	false	extra judge pass to validate completion (slower, costs more)
DSBROWSER_ENABLE_PLANNING	false	planner sub-agent for long tasks (slower)
DSBROWSER_DISABLE_THINKING	true	disable DeepSeek's native reasoning (thinking: disabled); ~10x fewer tokens and much lower latency

Evaluation

The offline suite serves a local deterministic website. Three cases are answerable from the DOM; three are answerable only by looking, which is what makes the suite able to tell the vision modes apart — with --vision off those three are expected to fail.

Case	Capability	Needs vision	Success signal
search_and_extract	type, click, extract	no	exact name, price, and path
navigate_and_submit_form	navigate, fill, select, check, submit	no	deterministic confirmation code
negative_search	handle empty state	no	zero results reported
canvas_code	read text drawn on a <canvas>	yes	the voucher code PLUM-4482
image_invoice	read text baked into an image	yes	invoice number and total
chart_reading	interpret a chart rendered as an image	yes	tallest month and its printed value

# Offline contract tests — no API key, no browser
pytest -q

# Live eval — uses DeepSeek tokens
dsbrowser-eval                      # current DSBROWSER_VISION_MODE
dsbrowser-eval --vision all         # sweep off/auto/on over identical tasks
dsbrowser-eval --vision-only        # just the three see-the-page cases
dsbrowser-eval --vision on --no-headless   # same suite, headed browser
dsbrowser-eval --repeat 5 --json    # five attempts per case
dsbrowser-eval --case canvas_code
python -m dsbrowser.evals

Each run writes raw outputs, errors, timings, per-case token usage and cost, and a per-mode summary to eval-results/run-<timestamp>.json. The summary line reports pass rate split into DOM cases and vision cases, median latency, screenshots sent, and total cost, so the modes can be compared on identical work. Run the suite at least five times before comparing configurations. The local fixture measures agent reliability without public-site drift — keep public-site tasks (like a Hacker News one) as separate realism tests, since network state and page changes make them unsuitable as a regression gate.

Notes

• browser-use disables vision for DeepSeek and this tool undoes it. agent/service.py still contains if 'deepseek' in self.llm.model.lower(): use_vision = False, written before a DeepSeek vision model existed. DeepSeekBrowserAgent restores the setting on the constructed Agent; without that repair the agent silently never takes a screenshot, whatever you configure.
• Images are legal in user messages only. DeepSeek answers 400 "Image in system message is unsupported" otherwise, and a single such message kills a whole run. Every request is sanitized before it goes out: images found in system or assistant messages are replaced with a text placeholder.
• Vision needs the vision model. Sending an image to deepseek-v4-flash returns 400 "This model does not support image", so the configuration is rejected up front with exit code 2 rather than failing mid-run.
• Structured output via response_format: json_object. DeepSeek's function-calling backend unwraps nested single-key objects, so browser-use's AgentOutput action union ({action_name: params}) came back as bare param dicts that failed pydantic validation. Routing the schema through JSON-object mode (with the schema embedded in the system prompt) makes DeepSeek return the exact expected shape — verified to keep working when the message also carries a screenshot.
• Usage accounting is ours. Upstream ChatDeepSeek returns usage=None on every path, so this project's DeepSeekBrowserLLM reads DeepSeek's prompt_cache_hit_tokens / prompt_cache_miss_tokens itself and aggregates per run. That is what makes cost_usd and the eval's cost column possible.
• Browser reuse with keep_alive=True. Agent.run() kills the browser session after every task unless the profile opts out; without this, a shared browser dies after the first task ("CDP client not initialized").
• deepseek-v4-flash reasons by default, so DSBROWSER_DISABLE_THINKING=true sends thinking: {"type": "disabled"} on every request via an httpx event hook — measured at ~10x fewer tokens on a trivial call (98 → 10).
• browser-use ≥0.13 no longer uses Playwright — it drives a system Chrome/Chromium over CDP. Make sure Chrome is installed.
• Chromium is launched with --password-store=basic and --use-mock-keychain by default, so headless runs never read or prompt against the (often stale) macOS keychain. Override with DSBROWSER_CHROME_ARGS if you need real password storage in the profile.