dsh-omni-workstation

dsh-omni-workstation

English | 中文

An omni-modal workstation plugin for DeepSeek Harness (dsh). It gives the AI eyes, a brush, a camera and a voice: image analysis backed by an ordered multi-card VLM failover chain, a 6-tool local vision toolkit, image generation (incl. ComfyUI workflows), multi-card async video generation with an AI tool builder, and TTS / voice cloning across 3 cloud + 4 local providers — all configured from one auto-saving settings page (English / 中文).

Feature Overview

Module	Tool	Highlights
VLM	analyze_image	Ordered API card list, single-request failover, per-card timeout, JPEG→PNG fallback, 28 built-in providers, mirror models, dynamic multimodal adaptation
Vision Toolkit	zoom_image · sample_colors · image_diff · ocr_image · detect_elements · show_image	4 tools are pure-local (zero tokens); shared image resolution + card chain; artifact paths only
Image Gen	generate_image	OpenAI / DashScope / ComfyUI protocols, multi-workflow management with role mapping, reference-image support, auto verify reminder
Video	generate_video (+ per-card names)	Multi-card (limit 10), 7 protocols, /build-video-tool AI builder with custom-adapter runtime
Voice	speak · clone_voice	MiMo / MiniMax / Doubao + IndexTTS / GPT-SoVITS / VoxCPM / TTS-WebUI; zero-registration inline & persisted cloning

Every module has its own switch — turning one off unregisters its tools completely (0 token cost) while keeping your configuration.

Why a plugin instead of a Skill or a fixed script

Approach	Typical pain	What this plugin does
Long Skill text (official-API recipes)	A big instruction dump every turn — expensive tokens	Config lives only in the settings page / omni-vision.json; tool schemas inject only when a module is on
Fixed scripts (hand-written API calls)	Locked in a project folder; you must restate path and usage each time	Tools register into the harness — the AI finds and reuses them automatically
Changing config / switching models	Edit scripts or re-paste the Skill body	Change a field in Settings; it takes effect immediately

In short: less context, ready to use, config without code.

Custom tools (video)

Today you can AI-build a custom video tool: type /build-video-tool in chat. The plugin injects a build guide (card limit, existing tools, hard constraints); the AI collects the platform details and writes a new card plus a callable tool — no hand-written script, no re-pasting API docs.

[!TIP] Card limit defaults to 10; the command errors out when the cap is hit. Custom tools run on the custom-adapter runtime — see the video docs.

Settings Panel

Settings → Omni Workstation — four tabs (VLM / Image Gen / Video / Voice) plus a global settings tab. Every edit auto-saves and takes effect immediately; no Save button.

Requirements

• dsh CLI (DeepSeek Harness) with a web profile installed
• pnpm on PATH (or use npx --yes pnpm@<version>)

Install

The plugin is a bundle: it carries its own cordis.patch.yml and self-activates — one command, no manual patch editing.

# From a local directory
dsh plugin --profile web add ./dsh-omni-workstation

# From GitHub
dsh plugin --profile web add github:huashenglian/dsh-omni-workstation

# From a packed tarball (pnpm pack / npm pack)
dsh plugin --profile web add ./dsh-omni-workstation-0.1.0.tgz

dsh plugin add installs the dependency and appends the bundle to dsh.profile.bundles automatically.

[!NOTE] Do not add a manual - insert: - id: omni-workstation row to the profile cordis.patch.yml — the bundle already inserts it. A second insert throws duplicate loader entry id: omni-workstation at boot.

Manual alternative: put the package under $DSH_HOME/profiles/web/plugins/dsh-omni-workstation/, add "dsh-omni-workstation": "file:./plugins/dsh-omni-workstation" to the profile package.json dependencies and "dsh-omni-workstation" to the dsh.profile.bundles array, run pnpm install, then restart dsh web.

Quick Start

1. Restart dsh web and open Settings → Omni Workstation.
2. On the VLM tab, click Add Model (or edit the default card): pick a provider, paste your API key, fetch and pick a model.
3. Send the AI an image (or a local path) and ask about it — the analyze_image tool is now live.

All configuration lives in a single JSON file, omni-vision.json, stored inside the plugin installation directory (git-ignored; contains real API keys — never commit it). The settings page reads and writes this file; you can also edit it directly while dsh web is stopped:

{
  "retryCount": 3,
  "vlmEnabled": true,
  "apis": [
    {
      "id": "c_yyy",
      "name": "VLM API",
      "provider": "custom",
      "protocol": "openai-completions",
      "endpoint": "https://api.example.com/v1",
      "apiKey": "sk-...",
      "model": "gpt-4o",
      "timeoutMs": 120000
    }
  ]
}

How It Works

The package is dual-face:

• Host half (lib/index.js) — a cordis plugin: registers tools on the global tools registry and /omni/* web routes; loads and persists omni-vision.json; provider-gated tool registration re-syncs on config changes.
• Client half (lib/client.js) — the browser module (loaded via the dsh.client entry): registers the Settings → Omni Workstation section and its locale namespace (settings.omni-workstation).

Documentation

• docs/features/vlm.md — VLM analysis, card failover, mirror models, multimodal adaptation (中文)
• docs/features/vision-toolkit.md — the 6 vision tools (中文)
• docs/features/imggen.md — image generation & ComfyUI workflows (中文)
• docs/features/video.md — multi-card video generation & /build-video-tool (中文)
• docs/features/voice.md — TTS & voice cloning (中文)
• docs/reference/api.md — /omni/* API contract
• docs/changelog/changelog.md — changelog

Uninstall

dsh plugin --profile web remove dsh-omni-workstation

This removes the dependency and the bundle entry. Your omni-vision.json config file is left untouched.

License

MIT