Vai al contenuto principale
A

dsh-bilibili_video-study_buddy (plugin)

aaarookie/dsh-bilibili_video-study_buddy/plugin

Bilibili course study buddy for DeepSeek Harness. Four tools: study_frame grabs a timestamped frame at 1080p with region cropping and a disk cache; study_scan samples a time range into one grid image for locating; study_subs extracts AI subtitles and scores three signals to detect contaminated subtitle tracks; study_asr transcribes a short window with surrounding context. Read-only; all data stays local.

Installazione

dsh plugin --profile web add github:aaarookie/dsh-bilibili_video-study_buddy

Questo plugin si trova nella sottodirectory plugin del repository.

README

dsh-bilibili_video-study_buddy

npm license

Let the model look at the right frame at the right timestamp, then turn a lecture into checkable notes.

A DeepSeek Harness plugin, plus the methodology and real note artifacts behind it.

Repo: https://github.com/aaarookie/dsh-bilibili_video-study_buddy | Plugin: plugin/ (npm: dsh-bilibili_video-study_buddy) | 中文说明: README.zh.md


What this is

PathContents
plugin/The DSH bundle. Four tools: study_frame / study_scan / study_subs / study_asr
docs/Methodology: ecosystem survey, plugin comparison, workflow, environment, verification log
examples/Real artifacts: Karpathy GPT lecture notes (md + docx), ASR transcripts, frame grids
scripts/Standalone PowerShell tools (optional, Windows)

Three counter-intuitive findings

1. Do not feed the whole video — feed the right frames

A 30-minute lecture carries its signal in roughly 20 frames. Grab one frame at a timestamp, crop away the browser chrome, and let the model read it. That is one to two orders of magnitude cheaper than "video understanding".

2. A grid is for locating, not reading

ApproachSizeEst. tokens
6 full frames at 1080p—~5,850
6-cell grid (400 px/cell)1224×450~585 (−90%)

At 400 px per cell you cannot read code. The grid only answers "which stretch is worth a close look".

3. Bilibili AI subtitles lie — verify before trusting

On a lecture about GPT, the subtitle endpoint returned:

[0.0s] 核能一种很硬核的能源        (nuclear energy is a hardcore energy source)
[2.4s] 比如说一克浓缩铀             (say, one gram of enriched uranium)

and on another attempt, English song lyrics. Both had nothing to do with the video. study_subs therefore scores three independent signals — coverage, title-term hits, speech rate — and refuses to emit the body when the verdict is FAIL.

Install

dsh plugin --profile web add dsh-bilibili_video-study_buddy

Or in DSH: Settings → Plugins → Add plugin and enter the package name. Restart DSH once.

Requirements: Node ≥ 20, ffmpeg (required); a Bilibili session (optional, unlocks 1080p and AI subtitles); faster-whisper (only for study_asr).

Usage

You: explain the RL Training part at P8 of BV1mqrTBvEaf

The model will, on its own:

  1. study_subs to probe subtitles → verdict: contaminated (this video really is)
  2. study_scan -from 14:00 -to 18:00 → a grid that locates the slide at 16:20
  3. study_frame -at 16:20 -crop "0,0,0,190" → one high-res frame
  4. study_asr -at 16:20 → the lecturer's own words with surrounding context
  5. read the frame against the transcript → explain, then write timestamped notes → Word

Measured numbers

ItemResult
Cropped 1080p frame1234×800, ~975 tokens (−45% vs full frame)
6-cell grid1224×450, ~585 tokens (−93% vs reading each)
90-second ASR window8.8 s (faster-whisper small, CPU)
Full 42-minute transcript229 s
Frame cache hit< 1 s (first fetch 13 s)
Understanding one 30-minute lecture≈ ¥0.03 (DeepSeek off-peak)

Cost model: examples/karpathy-gpt/p8-cache/cost-analysis.md (Chinese).

Limitations (all from real failures)

  • A timestamp belongs to one part, not to the whole video. Multi-part courses need the right page.
  • Anonymous caps at 640×480; 1080p requires a Bilibili session.
  • ASR systematically mishears code terms (free(p) → "4AP", 栈上 → 站上). Code always comes from the frame.
  • The grid cannot be used to read code — locating only.
  • Read-only: never likes, comments, or coins.

Docs

English: docs/en/ | 中文: docs/zh/

#TopicEN中文
01Survey of 102 Bilibili-related DSH pluginsenzh
02Nine candidates compared, three technical routesenzh
03The five-step study loop and prompt templatesenzh
04Install, login, and error lookupenzh
05Every measurement, including what was not testedenzh
06The timestamp → frame → notes pipelineenzh
07What was installed and how to undo itenzh
08Plugin install and integration recordenzh

License

MIT

Plugin correlati