Vision, Voice & Multimodal

Hosted free vision sidecar for DeepSeek Harness with durable session evidence

Bridges session images to configurable vision providers and returns text-only analysis to eligible DeepSeek Harness model routes.

Omni-modal workstation for DSH: analyze_image over an ordered multi-card VLM failover chain, a six-tool vision toolkit (zoom, colour sampling, pixel diff, OCR, element detection, inline display) sharing the same image resolver, generate_image over OpenAI/DashScope/ComfyUI protocols, multi-card async generate_video with a /build-video-tool builder, and speak/clone_voice TTS over seven providers, all driven by one auto-saving settings page.

One-handed, keyboard-free input for the composer: press and hold the mouse on the input box, speak, release — the text lands in the draft, and sliding up cancels without leaving a half sentence behind. No mic button to aim at and no shortcut to remember; the hand never has to leave the input area, which is the point when the other hand is busy. Recognition runs fully offline (SenseVoice via sherpa-onnx, no API key, audio never leaves the machine).

Read assistant replies aloud in DeepSeek Harness: text-to-speech providers with streaming playback.

Voice assistant for dsh web: say the wake phrase (e.g. "小鲸") to activate hands-free dictation — what you say is transcribed and typed into the chat box automatically. Supports spoken edit commands (send, clear, new line, stop reading) and reads assistant replies aloud in Chinese. Speech recognition runs locally in-browser via sherpa-onnx WASM, so it works offline without an API key.

AI audio generation for the DeepSeek Harness web GUI — multi-vendor TTS, music, sound effects and voice design with a sidebar panel, model comparison, resource library and Agent tools.

Continuous voice conversations for DSH with hands-free listening, push-to-talk, speech recognition, TTS replies, and background Agent delegation.

Real-time local TTS for DeepSeek Harness: voice presets, auto-read, engine setup assistant, a read-aloud button, and a settings panel for the GSV-TTS-Lite engine.

Listen to Ximalaya podcasts and audiobooks inside the DSH web UI: search albums, browse paginated tracks, and play through a now-playing bar (prev/play/next, seek, volume, rate). After QR login the "Mine" page lists your liked tracks (click to play), subscribed albums with latest-episode hints, and followed anchors with their public albums; the host relays audio streams with Range support and registers an `ximalaya_play` tool so the agent can search and play on request.

Let your AI agent see and operate a real Android phone: phone_look (vision + UI-tree fusion), tap/swipe/type, screenshot — over adb, for any MCP client.

Image understanding for any DSH model: vision, OCR, grounding, and crop tools with domain presets for histopathology, cell biology, anatomy, clinical images and scientific figures.

Local image, voice, music and SFX generation plus transcription, with pinned identity: characters, animals, objects and actor voices are defined once and reused on every later call, degenerate output (a near-flat image, silent audio) is rejected instead of returned as success, and a generated line can be read back as text so a clone that swallowed its ending becomes visible. The engines unload when idle, and image generation and transcription can each be pointed at an OpenAI-shaped API instead of the local Vulkan backend.

DeepSeek Harness plugin that turns UI screenshots into implementation-grade web specs using OCR, deterministic geometry, scene graphs, assets, and render comparison.

Auditable vision and cross-platform Computer Use runtime for DeepSeek Harness with source-preserving evidence.

Voice input for the Web UI: a mic button that drives local VocoType offline speech recognition and auto-inserts recognized text into the composer (auto-deploy, dedupe, continuous dictation).

Speech-to-text voice input for the web UI: a mic button in the composer transcribes speech into the draft via the browser Web Speech API (zero-config) or an OpenAI-compatible Whisper API (OpenAI / Groq), with a selectable model and language in Settings.

Renders pasted/uploaded images inline in the DSH Web chat and gives text-only models vision: the model-invokable visual_review tool calls any OpenAI-compatible multimodal API first, falling back to a local Qwen3-VL worker.

Agent-callable vision tool that describes local images via any OpenAI-compatible vision endpoint you configure, with an optional multi-model cross-check and no built-in keys.

Advertise image paste on text-only DeepSeek routes, describe attachments with a vision sidecar, and leave native vision models untouched.

Full vision-capability bundle for DeepSeek Harness: a vision_understand tool (OpenAI-compatible vision APIs, free Zhipu GLM-4V-Flash by default) plus paste/drag-and-drop/button entry points for image recognition.

Gives dsh the ability to generate images and videos through the grok2api API.

GPT Image 2 `image_gen` with Codex subscription OAuth by default or explicit API-key mode: developing card, up to three live API partials, durable attachment replay/lightbox/download, text-only model output, and bounded credential-safe requests.