Vision, Voice & Multimodal

Speaks agent replies in the DeepSeek Harness web UI through a provider fallback chain (OpenAI, ElevenLabs, Google, Azure, Groq, Deepgram, OpenRouter, Edge, Piper, eSpeak), so a failing or rate-limited provider falls through to the next instead of going silent.

Native image attachments for text-only DeepSeek in the Web GUI: pasted or dropped images appear as thumbnails in the session, and before dispatch the host reads them with the free Zhipu GLM-4V-Flash vision API (glm-4v-flash fallback chain) and substitutes the description, so DeepSeek answers about the image while the original is kept in history.

World-model research toolkit: inspect frames, name 3D / pixel / latent routes, score pred vs GT, and RSI skills / wm.yaml.

Analyzes conversation images through mode-specific prompts (caption, UI, document, grounding, topology, etc.) and injects structured evidence with coordinate primitives (boxes, points, refs) as text, with session-level caching for reuse across replay and compaction.

智能输入插件 for DeepSeek Harness — 语音输入 + 提示词优化

Routes images to a vision model of your choice - auto-rewrite, explicit tools, or hybrid - so a text-only chat model does not fail a turn that contains a picture.

Local-first, full-duplex voice for DeepSeek Harness, orchestrated by Muxiva

Agent-initiated voice calls: `offer_call` rings the human (接听/拒接/稍后再说); accepted calls synthesize and play locally via CrispASR + Qwen3-TTS (9 speakers, 2 Chinese dialects), rejected calls return the decision to the agent.

Reads assistant replies aloud via Fish Audio API only (bring your own key): per-message read-aloud, auto-read toggle, and a settings page for model, voice reference_id, encrypted API key, and proxy.

Native vision capability extension, using either Zhipu (free) or Qwen-VL (local).

Vision provider route that transcribes attached images to text through a configurable model (15+ OpenAI-compatible and Anthropic vendors) while DeepSeek keeps answering.

Vision bridge for text-only DeepSeek routes that analyzes attached and local images through configurable OpenAI Responses, Chat Completions, or Anthropic Messages endpoints while leaving image-capable routes native.

Vision toolkit for text-only DeepSeek: model-invokable `vision` tool, wrapper adapters for deepseek/opencode-go (v4 flash/pro), Antigravity IDE quota (default, flash/pro) / any OpenAI-compatible VLM / Gemini / local Ollama channels, evidence memory with compaction rehydration, content-hash cache, and a bilingual client panel.

Model-facing `vision` tool for DeepSeek Harness: describe and OCR image files by calling the free Zhipu GLM vision API directly (glm-4v-flash fallback chain), no external CLI required.

Registers a `deepseek-vision` provider route: the Web GUI accepts pasted images and transcribes them to text via the free Zhipu GLM vision API before delegating to the DeepSeek adapter.

Composer-attached images are transcribed to text by an OpenAI-compatible vision model before reaching text-only DeepSeek models.

Live2D session view with realtime voice: continuous voice input, streaming TTS, subtitle translation, multi-model catalog, standalone entry / Live2D 实时语音会话视图：连续语音输入、流式 TTS、字幕翻译、多模型目录、独立入口

Per-reply read-aloud with a multi-provider settings page — MiniMax or any OpenAI-compatible /audio/speech endpoint, with voice, emotion, speed and model selection plus an auto-read toggle.

Local-first voice conversations for DSH, with local speech recognition and synthesis and optional external providers.

DeepSeek Harness plugin: auto-route image-bearing requests to deepseek-v4-flash-vision-exp, then fall back to the original model.

Live speech-to-text dictation into the dsh web composer via OpenRouter STT

MiniMax 语音识别（asr-1.0）+ 语音合成（speech-2.8-hd）的 DeepSeek Harness 全局插件：转写工具、任务结束后用喇叭念一句的播报、实时免手对话、303 个音色可选

Text-to-image for DeepSeek Harness: a `draw_image` model tool, an input-bar 生图 button with a prompt popup (async generation, 4-grid results, download / keep / delete), an /imgdraw image route, and persisted history. Backends: DashScope wan2.7-image (free

Voice conversation mode for the DSH Web GUI: tap the mic to start a full-screen call, talk hands-free, and hear each reply read aloud as it streams. Bilingual (Chinese/English) UI, light and dark themes, adjustable speech rate, and switchable voices.