dsh-doyourweb-remote

dsh-doyourweb-remote

Reference and control your existing, logged-in Chrome and Edge tabs from DeepSeek Harness, with compact page context and batched browser actions.

Version 0.1.6 improves desktop snapshots, input diagnostics and connection UI, and adds app_type for text plus key chords. It supports draggable element annotations collected in the chat draft and @ references to running Windows apps. App control discovers matching MCPs first. Installed local MCPs connect directly; installation requests approval. With no matching MCP, use computer use. Windows are shown when possible while background execution remains available. See annotations and app control.

English | 简体中文

Local Demo · Quick Start · Usage

The local pairing page connects your browser extension to the DSH plugin.

⚠️ This fork shares the plugin id browser-tabs with upstream SheepofSheep/dsh-doyourweb, so do not install both in the same DSH profile — the second one replaces the first one's config.

This fork (DeveloperYe/dsh-doyourweb-remote) adds remote browsers and hardens the local management surface. Another machine's Chrome/Edge can drive the tabs it already has; see docs/remote-browser.md.

01 · Introduction

Browser tasks often depend on pages you have already opened and accounts you have already signed into. dsh-doyourweb-remote connects those tabs to DeepSeek Harness instead of launching a separate automation browser or exporting cookies.

The project has two parts: a DSH plugin and a companion Manifest V3 extension. The plugin adds browser references to the native @ and / menus; the extension supplies tab metadata and browser control. Once paired, the extension reconnects in the background without keeping its popup open.

The current version is the version in package.json; the package and the browser extension share it, and this guide is checked against 0.8.x. See the changelog.

02 · Preview

1. Use /browser to open the connection page and locate the extension folder.
2. Load the extension in Chrome or Edge, then approve its matching four-digit pairing code.
3. Select an existing page with @ and describe the task in DSH.

The screenshot above shows the pairing interface, not a recorded automation run.

Local Demo

There is no public hosted demo. After the DSH plugin starts, its default local connection page is http://127.0.0.1:43187/setup. Browser control runs through this local bridge and the installed extension.

03 · Core Features

Browser budgets count JavaScript characters, including JSON escaping and metadata. App tool limit values bound body text; status, steps and image-path metadata are additional. MCP results omit duplicate JSON, and CLI stdout/stderr share one body budget. Character counts are not model tokens, and output budgets do not limit DOM traversal time.

0.6.0 Addition: browser_close

• Only its own tabs: the host tracks the references browser_open returned per session; anything else is refused, so a page you opened can never be closed this way.
• Omit tab to close every tab the conversation opened; pass one to close just that tab.
• Idempotent: a tab that is already gone (including one you closed yourself) reports closed, not an error.
• Closing revokes the grant, so later references fail loudly instead of acting on a stale page.

0.5.1 Addition: browser_open

When the page you need is not open yet, the model can call browser_open instead of asking you to open a tab and cite it:

• http(s) only; file://, chrome:// and javascript: are refused so no privileged page ever reaches the bridge;
• the tab is created through the extension (chrome.tabs.create), and the call returns only after the extension publishes it, so the reference is readable immediately;
• the grant covers that new tab only and ends with the session; every other tab still needs @;
• with several browsers connected, pass tab from an authorized tab to choose which browser opens it.

0.5.0 Additions (upstream 0.1.7-0.1.9 merged)

• Upload and drag: browser_upload plus upload, drag and drop in browser_act; see docs/browser-workflow.md.
• Network and diagnostics: browser_network_requests for request metadata, plus stage-based diagnostics that show whether a call stalled in the extension, the debugger or the return path; rationale in docs/architecture-optimization.md.
• Interface: a native browser sidebar, collapsed context chips, and a stop button for long actions.
• Reading controls: browser_read accepts scope: "main"; observe: "handle" returns only a handle and cursor (cheaper, but read before reusing refs).

WebMCP-First Control (Experimental)

Version 0.1.1 checks for page-provided WebMCP tools before collecting a DOM snapshot. It prefers tools that cover the task, falls back to DOM when needed, and reuses unchanged catalogs and validators. Successful calls return a fresh token for the next call without relisting schemas. WebMCP requires a compatible browser API and site-provided tools; ordinary pages keep using DOM automation. See the experiment guide for requirements, tools and local browser tests.

04 · Architecture

flowchart LR
    subgraph DSH["DeepSeek Harness"]
        UI["Native @ / menus"]
        Host["Host plugin"]
        Grants["Conversation grants and context budget"]
        Engine["Playwright engine"]
        Bridge["Loopback WebSocket / CDP relay"]
    end
    subgraph Browser["Existing Chrome / Edge"]
        Extension["Manifest V3 extension"]
        Tabs["Logged-in HTTP/HTTPS tabs"]
    end
    UI <-->|"Strict Remote RPC"| Host
    Host --> Grants --> Engine
    Engine <-->|"CDP"| Bridge
    Bridge <-->|"Origin check and pairing token"| Extension
    Extension <-->|"Tab inventory and debugger"| Tabs
    Engine -->|"Snapshots, receipts, PNG paths"| Host
    Host --> Apps["App discovery and human approval"]
    Apps --> Structured["Official / community MCP and CLI"]
    Apps --> Desktop["Cua Driver desktop fallback"]

Only tabs referenced by the user in the current conversation are available to the model's browser tools. Page content is treated as untrusted data, not instructions or permission.

05 · Quick Start

Requirements

• Node.js 22+ and npm.
• DeepSeek Harness; plugin API compatibility has been checked against 0.2.0-rc.2.
• Chromium-based Chrome or Edge 125+, with developer mode available for loading an unpacked extension.

Install the DSH Plugin

For npm installation, open Plugins > Add Plugin in DSH, select the official npm registry, and enter:

dsh-doyourweb-remote

This requires a published package. If the registry returns 404, use the local installation below. Mirrors may take time to sync a newly published version.

Build and Install Locally

git clone https://github.com/DeveloperYe/dsh-doyourweb-remote.git
cd dsh-doyourweb
npm ci
npm run build

In DSH, open Plugins > Add Plugin and enter the absolute path to this built project directory. The bridge starts when the plugin activates. No separate development server is required.

Connect the Browser

1. Enter /browser in DSH and open the local connection page.
2. Open edge://extensions or chrome://extensions, enable developer mode, and load the project's extension/ folder as an unpacked extension.
3. Compare the extension's four-digit badge code with the pending request on the connection page, then approve it.
4. Return to DSH, select a tab with @, and send your request.

The extension popup and pairing page do not need to remain open. Installing the DSH plugin does not install the browser extension automatically.

After upgrading the DSH plugin, fully quit DSH from its tray/background process and relaunch it so the new host module is loaded.

Remote Browser (the browser lives on another machine)

To drive a browser on a different machine (or this one over the public internet) you install nothing on that machine beyond the extension:

1. Add one setting on the host, replacing <your-host> and <external-port>:

   - id: browser-tabs
     config:
       bind: 0.0.0.0
       remote: true
       tls: true                                   # the plugin terminates TLS itself
       publicUrl: "https://<your-host>:<external-port>"
   

   The port the extension dials, the Host allowlist, the wss scheme and the certificate SANs are all derived from publicUrl; you do not fill them in separately.

2. Open /browser in DSH and download the matching extension zip (it already contains the pairing token and the CA certificate).

3. On the controlled machine, unpack it, open edge://extensions, enable developer mode and load the unpacked folder.

4. Double-click the bundled ca.crt and install it into Trusted Root Certification Authorities — browsers cannot install a root CA themselves, so this is one manual click.

5. Back in DSH, pick a tab with @.

NAT/tunnel setups, certificate rotation and the full option list: docs/remote-browser.md.

Configuration

Defaults are declared in cordis.patch.yml:

The data directory is <DSH_HOME>/browser-tabs/, or ~/.dsh/browser-tabs/ when DSH_HOME is unset. An explicit plugin home setting takes precedence. Set environment variables on the DSH process; the plugin does not require its own model API key.

Usage

After referencing a page, you can ask:

Read the result list on this page and collect its titles and links. Do not submit any forms.

When multiple tabs are authorized, pass the selected tab's reference as tab.

Annotations and App Control

After installing or upgrading, fully exit DSH from the system tray, restart it, and reload the browser extension. For local development, build this working tree and install its absolute directory path. Do not enable the npm copy and local copy on the same port.

• Annotations: choose “批注当前页面” in the extension popup, or ask DSH to enable annotations on a referenced tab. Pick an element, write a comment, then press Enter or click “加入聊天草稿”. It appends to the target chat draft with an optional image attachment. Drag the toolbar as needed, collect multiple annotations, then press Enter in chat to send them together. Shift+Enter inserts a newline. The plugin keeps no annotation files or library. Controlled pages also show a top-right annotation button.
• Apps: select a running application with @Unity or another program name. DSH discovers connectors with app_discover, requests connection/installation through app_connect, then invokes MCP tools or CLI arguments with app_call. A connector found outside discovery can be proposed with its source documentation for human review.
• Management: /apps opens the local application, connector and approval manager. It need not remain open. Files, folders and conversations in @ each show three previews with expand/collapse controls; individual folders retain native navigation.
• Fallback: app_desktop uses upstream Cua Driver when structured tools cannot cover the task. Default reads return bounded controls without screenshots. Use screenshot: true only for visual grounding; app_mouse then supports up to 8 known-target clicks, scrolls or drags without intermediate captures. Missing windows and repeated inspections return stop: true. It does not bypass OS permissions.

See installation policy, Unity requirements and limitations.

{
  "mode": "dom",
  "depth": 8,
  "limit": 2400
}

{
  "snapshot": "<latest snapshot ID>",
  "observe": "summary",
  "steps": [
    {
      "op": "extract",
      "selector": "a.result-title",
      "fields": ["text", "href"],
      "maxRows": 10
    }
  ]
}

06 · Structure and Technical Details

src/
  host.js            Plugin lifecycle, tools and conversation context
  client.js          Native mention and slash-menu sources
  client-entry.js    React UI and icons
  bridge.js          Pairing, extension connection and CDP relay
  engine.js          Snapshots, actions, extraction and screenshots
  annotations.js     Isolated element picker and acknowledged chat drafts
  apps/              App inventory, discovery, approval, installation and execution
  manage.js          Local application and connector management UI
  webmcp.js          Site-tool discovery, routing, validation and calls
  grants.js          Conversation-scoped tab authorization
  protocol.js        Tab references and Remote RPC descriptors
  output.js          Serialized output budgeting
  vendor/            Adapted Playwright browser model
extension/           Manifest V3 extension and popup
scripts/             Build, integration and compatibility checks
test/                Unit tests and fixtures
docs/setup-page.png  Pairing-page screenshot
cordis.patch.yml     DSH plugin registration and defaults
lib/                 Generated build output, not tracked in Git

Snapshots and actions. Full snapshot text is cached locally for pagination. Element references require a current snapshot; reads, actions and navigation can invalidate previous references. Calls are serialized per tab. Cancellation reports completed or uncertain steps rather than promising rollback.

Connection lifetime. Idle connections detach the debugger without closing the tab. A later read reconnects. Restarting the browser changes tab-reference sessions, so select the page again with @.

Data handling. Pairing credentials and authorization records live in the local data directory; screenshots are stored under captures/. Referenced page excerpts enter DSH's model context and may be sent to your configured model provider. Keep pairing files and screenshots private. Purchases, destructive actions and sensitive submissions require explicit user authorization; the plugin does not provide a separate transaction-confirmation dialog.

npm run build
npm test
npx playwright-core install chromium
npm run test:browser
npm run verify:dsh -- --asar "<path to DeepSeek Harness app.asar>"

07 · Roadmap / FAQ / License

Roadmap

Version 0.1.3 includes page annotations, app references and MCP-first app control. Further work focuses on real-application compatibility and connector configuration. No release date is promised. Report bugs and discuss changes in Issues.

FAQ

Do I have to open the extension every time? No. Pair once; the extension reconnects in the background. Reload it after updating its files.

Why are some pages missing? Only regular HTTP/HTTPS and authorized local file:// tabs are exposed. Browser-internal and incognito pages are excluded. Check that the extension is enabled and paired if no tabs appear.

Why does a referenced page dump a wall of text into the chat? The injected page snapshot is not being folded into a chip; folding relies on client UI services that are not wired in every DSH version. Set snapshotChars to 0 to switch the injection off: the @ reference still authorises the tab and the model reads it on demand with browser_read (reload the plugin or restart DSH afterwards). What if port 43187 is already in use? Avoid activating two plugin copies on the same port. Stop the duplicate bridge or change the plugin's port setting and reconnect the extension to that port.

What are the current limits? The companion extension is loaded unpacked, not installed from a browser store. Firefox and Safari are not supported. Host API compatibility has been checked against DSH 0.2.0-rc.2; other versions and native UI workflows need their own verification.

Contributing and License

Issues and pull requests are welcome. Include reproduction steps and relevant DSH/browser versions; remove tokens, private page contents and account data from logs. For code changes, run the checks relevant to the affected behavior.

Project code is licensed under MIT. Playwright-derived code and Lucide artwork retain their respective licenses; see NOTICE, Apache-2.0 and ISC.