플러그인

로컬 AI 동반자용 AstrBot/DSH 네이티브 GPU 조정 도구입니다. LLM, IndexTTS, ComfyUI가 GPU를 번갈아 사용하며 프록시는 없습니다.

로컬 Qwen3.8 계열(llama.cpp 또는 NInfer)용: 요청별 thinking 예산, 압축 백엔드, 설정 탭.

stdio MCP 서버 앞에 세 라우터 도구(list_capabilities, describe_tool, call_tool)를 두어 모든 요청에 스키마를 싣지 않고 필요할 때 로드합니다. 각 도구 결과는 결정적 단계형 규칙으로 가공하고 원본을 먼저 보관하며 retrieve_spill로 다시 읽습니다. 도구 그룹의 토큰 비용 지정 및 에이전트별 내장 그룹 숨김도 지원합니다. 표준 라이브러리 기반 Python 라우터를 연결하므로 Python 3.9 이상이 필요합니다.

llama.cpp Docker 패널 `llamapad`용 DeepSeek Harness(DSH) plugin입니다.

One-step local model onboarding plus a light harness for DSH. On load it probes loopback and private-network inference endpoints (llama.cpp, Ollama, vLLM, LM Studio), reads each model's context window and output budget from the backend, and registers the provider through settings, so a local model becomes selectable without hand-writing config. It then trims only that provider: masks the tool list through tools.restrict, drops named system-prompt sections and memory contexts, swaps the injected instruction block for a short one while preserving its first 80 and last 96 characters, and truncates tool-schema descriptions without touching structure or required fields. Cloud providers are returned as the same assembly object, untouched. The injected block is re-read from disk on every assembly, so editing the file takes effect on the next request with no reload. The panel in the plugin detail page is a live operations view: it shows whether the takeover is active and by which hook, how many characters the last assembly was trimmed from and to, and which providers are online (probed concurrently, with a real one-shot completion reporting latency and reply text per provider). If a local backend is not running, the panel offers a one-click launch: it probes the port first (never starting a duplicate), then starts the matching server (ollama serve, or a custom command you supply) detached and polls until the endpoint answers, clamping the wait to 3-600 seconds, reaping the child it just spawned if it never answers, and naming the paths it searched when it cannot find the binary. A stop button kills backends this plugin started (SIGTERM, then SIGKILL) and refuses to touch a server somebody else launched. When a model has just been downloaded and no backend exists yet, the panel can also scan the machine for .gguf files with hard caps (300 models, 20000 directory entries, Downloads only one level deep), start llama-server on the chosen one (skipping the .mmproj. vision projection and reusing a server that already answers), register the provider under the model id the backend reports, and run one real completion to prove it works. Before spawning anything it does a fit check that counts only the part of the file the GPU cannot hold: hostBytes = fileSize - VRAM, compared against max(total RAM - 2GB, free RAM). A file far larger than RAM is refused outright; a heavy host-side remainder is refused unless you confirm; the same remainder under mmap is only a warning, because the weights land in reclaimable page cache. A load-mode selector (auto / mmap / none / mlock / mmap+mlock) maps to llama.cpp --load-mode, and the default is whatever the engine picks. From 0.4.2 everything above is derived from the platform instead of assumed: home and data directories come from os.homedir() and the OS conventions (LOCALAPPDATA on Windows, ~/Library/Application Support on macOS, XDG_DATA_HOME elsewhere); backend binaries are looked up in the official install locations of each OS and can be pinned with OLLAMA_BIN, LLAMA_SERVER_BIN, VLLM_BIN or LMSTUDIO_BIN; model discovery walks DSH_LOCAL_MODELS_DIR, ~/models, ~/llama.cpp/models, ~/.cache/llama.cpp and the LM Studio data directory, with the WorkBuddy layout demoted to one candidate among them. VRAM is asked from nvidia-smi, then rocm-smi, then sysctl hw.memsize on Apple Silicon (flagged unified, so the same bytes are not charged twice), and any flip to a unified-memory budget is handled explicitly. Before building a command line the plugin runs llama-server --help once and keeps only the flags that build actually has, so older engines get --n-gpu-layers 99 and no KV quantization while newer ones get auto plus q8_0; anything it cannot probe is passed through rather than dropped. LM Studio is started with lms server start --port N rather than being lumped in with the engines that need a model argument, and vLLM gets its own --model example instead of llama.cpp syntax. Windows, macOS and Linux are all exercised in CI, including a windows-latest and a macos-latest job, because a cross-platform claim that has only ever run on one OS is not a claim. Two of those suites exist specifically to catch this machine's blind spots: one replays the whole suite with process.platform rewritten to each OS, and one deletes every advantage this development box has — no nvidia-smi, no rocm-smi, no sysctl, an empty PATH, a home directory with no models — and requires graceful degradation rather than a crash. That last one caught a real class of bug: several assertions hardcoded 'lms.exe', which cannot pass where the executable has no suffix. Measured on the reference machine: a 16.9GB 27.2B MoE with 14.5GB of VRAM holds a 285-691MB working set and sustains 76 tok/s after a 1000-2000 token warm-up, so file size alone is not a fit signal. From 0.4.3 the scanned models are shown as a table rather than a dropdown (one row per gguf, with its size and any fit warning), so you can see what is on disk before choosing. From 0.4.4 the delete button on the provider table removes the provider entry, not the weights file on disk. That distinction matters: what people actually want to clear out is the dead entry - one pointing at a port that never answered, one registered twice for the same port, one whose model file was deleted long ago - not a 15.8GB gguf. Deleting a config is undone by importing again; deleting a gguf is not. The removal goes through settings.mutate with an unset op on \['providers', id], because update merges layers: writing null there leaves a null provider key that the loader reads as a broken config rather than an absent entry, and only mutate/unset is implemented as Reflect.deleteProperty. The result is read back afterwards, because unset on a key that comes from a lower layer (the profile cordis.patch.yml or a home patch) does not delete it - it restores the inherited value, so the call succeeds while the entry stays. When the entry is still there the panel says so and points at the file to edit instead of reporting success, and a DSH whose settings service predates mutate() is told plainly rather than silently ignored. Deleting takes two clicks and clearing the confirmation is as easy as clicking another row; on success the online and baseURL caches are cleared and the tool surface is recomputed, since the provider list changed. The gguf itself is never touched by this plugin - the scanned-model table is display and selection only.VRAM reporting is driver-dependent. From 0.4.5 model discovery also reads Ollama's own storage: its blobs are named sha256-… with no extension, so extension matching could never list them. Weights are now accepted by extension OR by GGUF magic header (only dot-free files of at least 8MB, capped at 600 header reads per scan; obviously-not-weights extensions like .json/.md/.safetensors/.part are never opened). Ollama models are resolved from their manifests: the model layer is picked by mediaType application/vnd.ollama.image.model (never adapter/mmproj layers), the human-readable name is restored from the manifest path (…/qwen3/8b becomes qwen3:8b, …/latest stays bare), tags sharing one blob merge into a single row, and the metadata kv supplies name, quantization and context length. Adopting an Ollama model does not start a duplicate llama-server over the same weights: it confirms ollama serve answers on 11434 (launching it if not) and registers the provider under the tag the endpoint itself reports — the blob's sha256 name would never match anything, since 11434 lists qwen3:14b, not sha256-a8cc…. Two real-machine-only traps fixed on the way: existsSync('<file>/') returns true on Windows, so manifest-vs-directory branching now goes by parseable content instead of trailing-slash semantics; and the quantization label table had been copied from a stale ggml enum generation, labelling ggml_type 15 as Q5_K_S while ollama show reports Q4_K_M. Same-blob dedup is by (dev,ino), keeping whichever entry has a name. Verified on the reference machine: the scan goes from 1 row to 4 — qwen3:14b (8.64G) and qwen3:8b (4.87G) now appear with Q4_K_M badges — in 4ms; npm run verify exits 0 (launcher 57, portable 35, panel-host green including the linux/darwin simulated-platform runs); the new panel-host suite builds a real miniature Ollama store in tmpdir and points OLLAMA_MODELS at it, so results no longer depend on whether the test machine happens to have Ollama installed. From 0.4.6 the panel shows each model its real name instead of its file name, because some download tooling stores every weight under one generic file name, leaving the real name only inside the GGUF header. The scan now reads each file GGUF KV section - not a fixed-size head buffer, because tokenizer.ggml.tokens (hundreds of thousands of strings on a large MoE model) and the chat template push general.file_type past any window; and GGUF string arrays are interleaved (u64 length plus payload per element), so treating them as a length table walks the cursor off the rails and produces astronomically large reads. Reading and skipping are accounted separately: key names and types are always read, string values over 256KB and string arrays are only walked past without copying, non-string arrays over 64 elements are skipped outright, and a cursor past the file size stops the read, because an out-of-range position means the cursor is wrong rather than that a limit should be raised. The displayed name goes Ollama tag, then general.basename plus size_label, then general.name, then the file name, with the full string kept for the hover text since it usually carries internal build numbers; mmproj files are now identified by general.type as well as by name. A short read (truncated file) no longer feeds half a value into the parser. On the reference machine, reading one file header parses 44 keys in about 16ms, and a whole scan takes about 31ms. From 0.4.7 the smoke test no longer fails the first time and passes the second. The old code gave both request attempts one shared AbortController and one shared 30-second timer, so the total budget covered both rounds: a cold load of a large model burned it in round one, the abort left the signal permanently aborted, and the second round (the Ollama-style reasoning_effort: none retry) could not even be sent - which is exactly why the first click returned This operation was aborted and the second succeeded. Now each attempt gets its own controller and timer, a timeout retries once automatically (retryOnTimeout, on by default) because a first-request timeout almost always means the weights are still being read from disk, and non-timeout errors (refused connection, HTTP error) are not retried at all. Timeouts report themselves: the message says the wait exceeded smokeTimeout, whether a retry already happened, and points at the new smokeTimeout setting (seconds, default 60, clamped to 5-600) - port responding is not the same as tokens coming out, so it is a separate knob from launchTimeout. Unrelated: an example naming one specific model was dropped from the panel copy, since not every machine has that model downloaded, and the explanation now describes the mechanism instead.

자신의 GPU Hugging Face Space 모델을 모델 선택기에 추가하고 메시지를 보내면 Space를 시작하며 진행 상황을 표시합니다. 유휴 상태이거나 시간 제한에 도달하면 일시 중지합니다. Hugging Face 계정, OpenAI 호환 API를 제공하는 GPU Space, DSH vault에 HF_TOKEN으로 저장한 Hugging Face 토큰, 유료 GPU 하드웨어 요금제가 필요합니다. GPU 사용료는 Hugging Face에서 청구합니다.

작은 context의 로컬 모델을 위한 host-side 장기 작업 감지입니다. 긴 출력은 parts skeleton(응답마다 part 하나, 이후 조립), 다단계 작업은 PLAN.md ledger와 notes/(먼저 계획, 응답마다 한 단계)로 나눕니다. 휴리스틱 host-side 판정으로 모델의 자기 한계 판단에 의존하지 않습니다.

DeepSeek Harness(dsh) 플러그인입니다. Web 오른쪽 사이드바에서 로컬 LLM 엔드포인트 상태를 실시간 표시합니다. llama.cpp 슬롯(busy/TTFT/응답 멈춤, `/metrics`)과 Ollama MODELS(로드됨/비어 있음)를 보여줍니다. 응답 형태로 엔진을 식별하며 `dsh.bundle`로 설치할 수 있습니다.

DSH 플러그인: 로컬 llama.cpp/llama-server 게이트웨이로 보내는 모든 chat-completions 요청에 모델별 샘플링 매개변수(top_p, top_k, min_p, repeat_penalty, presence_penalty, frequency_penalty)를 주입합니다.

DSH Web 뷰에서 로컬 LLM 추론 서버를 관리하는 패널입니다. 상태 검사 및 중지 기능의 서비스 레지스트리, llama.cpp/SGLang/vLLM용 이름 지정 파라미터 프로필(KV/VRAM 저장 전 검증), GPU 감지, tok/s 벤치마크를 제공합니다.