dsh-video-narrator

Video Narrator for DeepSeek Harness

English | 中文

A DSH skill that turns a video, video URL, article URL, PDF, or plain text into a narrated video: cleaned transcript, segmented narration script, title and 10 tags, per-scene image prompts, generated images, TTS audio, and a final MP4 assembled with ffmpeg.

What it does

vcs auto runs the whole pipeline in one local process; the agent only starts it and handles failures.

1. Ingests local video/audio, YouTube/Bilibili URLs, article URLs, PDFs, or text.
2. Transcribes video in 240s chunks with 30s overlap using Qwen3-ASR (MLX), then merges overlaps with a Python algorithm.
3. Cleans the source into Markdown, then writes a timed narration script.
4. Generates a title, description, and exactly 10 tags for 04_metadata.json.
5. Writes orientation-aware image prompts and generates one image per scene through ModelScope.
6. Generates one TTS clip per scene with Qwen3-TTS (MLX), fixed seed and a required reference voice.
7. Composes scene clips with ffmpeg, concatenates them, names the result from metadata.title, and removes intermediates.

Requirements

• macOS on Apple Silicon for the local MLX stages (ASR and TTS).
• ffmpeg and ffprobe on PATH.
• yt-dlp for video URLs.
• Python 3.12/3.13 with pypdf and trafilatura for ingestion.
• Python 3.12/3.13 with mlx-audio and mlx-qwen3-asr for ASR/TTS.
• A ModelScope API token for image generation. Domestic and international endpoints are both supported.

Install

Install from the DSH plugin market, or use Plugins → Add plugin with this repository. The bundle registers the video-narrator skill; no separate skill copy is needed.

Configure

Copy the environment template and fill only the keys you use:

cp .env.example .env

For multiple text or image accounts, copy providers.example.json to providers.json and edit the ordered profiles:

cp providers.example.json providers.json
python3 scripts/vcs.py providers --file providers.json

providers.json is gitignored. Keys are read from environment variables named by api_key_env. The text array is tried in order by vcs text; the image array is tried in order per scene. ModelScope domestic (api-inference.modelscope.cn) and international (api-inference.modelscope.ai) endpoints can be mixed.

Control UI

After installation, a Video entry appears in the left sidebar. It opens a persistent main-panel workbench: the left side lists past projects from runs/ and projects/ (title, status, orientation, duration, image count), and the right side configures the next video (source, orientation, duration, segment length, voice, style). Start creates a new session with a structured request. The request carries [VCS_PARAMS_CONFIRMED], so the skill starts without asking the same parameters again.

Use

Ask the agent for a video from a URL, PDF, article, or text. Before doing anything, the skill opens the parameter confirmation dialog and asks for orientation, duration (fixed, range, or unlimited), segment length, resolution, voice, and style. After confirmation it runs one vcs auto command; the agent does not write or orchestrate the intermediate files.

The CLI is also usable directly:

# ingest
python3 scripts/vcs.py ingest --input "https://www.youtube.com/watch?v=..." --orientation portrait --duration 5-8

# transcribe video input
HF_HOME="$HOME/.cache/vcs/hf" python3 scripts/vcs.py asr --project <project> --language English

# Agent writes 02_transcript.md / 03_script.json / 04_metadata.json / 05_image_prompts.json, then:
python3 scripts/vcs.py validate --project <project>

# media
python3 scripts/vcs.py images --project <project>
HF_HOME="$HOME/.cache/vcs/hf" python3 scripts/vcs.py tts --project <project>
python3 scripts/vcs.py compose --project <project>

Output

projects/
└── YouTube_11122026_<metadata title>/
    ├── 01_transcript.raw.txt
    ├── 02_transcript.md
    ├── 03_script.json
    ├── 04_metadata.json
    ├── 05_image_prompts.json
    ├── 06_images/scene_001.jpg
    ├── 07_<metadata title>.mp4
    └── source/<original>.pdf   # PDF input only

Audio, scene clips, logs, and temporary files are removed after compose by default; pass --keep-intermediates to keep them. 04_metadata.json records the source type, platform or domain, original URL, syndicated URL, or copied PDF path.

Notes and limits

• v1 does not generate subtitles.
• Scene images are JPEG (-q:v 2) to keep projects small; legacy PNG projects remain readable.
• Resolution presets are 1080p, 720p, or custom. Scene images use the same output resolution by default (Z-Image accepts 512×512–2048×2048 total pixels; 1920×1080 is tested); larger frames are scaled down for generation and upscaled at compose.
• Image generation needs a ModelScope token; it is not bundled.
• Two reference samples are bundled: assets/voice_female.mp3 and assets/voice_male.mp3. TTS always requires a reference voice; pass --voice-ref and --voice-ref-text to use a different authorized voice. The samples are not MIT-licensed; see assets/VOICE-LICENSE.md.
• Paywall circumvention is not implemented. For restricted articles, use an authorized copy, a free syndicated copy, or saved HTML via vcs ingest --html.
• Local model sizes: ASR about 2.3 GB and TTS about 4.5 GB on first use; image generation runs remotely.
• The compose step requires the same codec/fps/audio parameters across all scene clips; the included script enforces this.

License

MIT. See LICENSE.