dsh-comfyui-workbench

dsh-comfyui-workbench

A local ComfyUI workbench for DeepSeek Harness (DSH).

It combines media generation, sprite animation, item-studio assets, Hugging Face model downloads, and advanced tools that expose the rest of ComfyUI:

Tool	What it does
generate_image	Text-to-image, image-to-image, inpainting and background removal
generate_audio	Music/audio generation with ACE-Step 1.5
generate_video	Short video generation with Wan 2.1 text-to-video
generate_sprite_animation	One-call W2/W3 turn → video → sprite card pipeline from a base PNG
item_studio	Catalogue-driven item generation, BiRefNet cutout, 64/tall sprites and 32 px icons
comfyui_inspect	Browse every installed model, node, schema, queue entry and history item
comfyui_workflow	Validate and run arbitrary API-format workflows, upload inputs and control jobs
comfyui_api	Reach any other ComfyUI HTTP endpoint
huggingface_models	Search Hugging Face, list model files and download models for ComfyUI

Everything runs against a ComfyUI server you control. There is no commercial API and no cloud account. The only runtime package is DSH's small @deepseek-ai/schemastery schema validator; everything else uses Node's built-in fetch, FormData and filesystem APIs.

Published on npm as dsh-comfyui-workbench. Current release: 0.8.0.

---

Table of Contents

• Why This Exists
• Capabilities
• Requirements
• Installation
• Verification
• Quick Start
• Tool Reference
  • generate_image
  • generate_audio
  • generate_video
  • generate_sprite_animation
  • item_studio
  • comfyui_inspect
  • comfyui_workflow
  • comfyui_api
  • huggingface_models
• Model Detection and Missing Models
• Configuration
• Outputs, Metadata and Files
• Custom Workflows via generate_image
• Security and Limits
• Troubleshooting
• Architecture
• Development and Testing
• Release and npm
• Migrating from dsh-comfyui-image
• Documentation
• License

---

Why This Exists

DSH has no media generation of its own, while an Agent frequently needs to draw a picture, edit an image, generate a track, make a short clip, discover what a local ComfyUI can run, or execute a workflow no high-level tool covers.

This plugin gives the Agent all of those paths in one bundle:

• Simple paths with sensible defaults: generate_image, generate_audio, generate_video.
• Escape hatches for the full ComfyUI surface: comfyui_inspect, comfyui_workflow, comfyui_api, plus huggingface_models for getting models in the first place.
• Actionable failures: when a required model or node is missing, the error says what was detected and what is missing instead of returning an opaque ComfyUI node failure.
• Local-first operation: generation requests go to the configured ComfyUI baseUrl; generated files land in the current project root by default. Hugging Face is contacted only when you explicitly use huggingface_models.

Capabilities

Generation

Capability	Tool	Requires
Text-to-image	generate_image mode=text2img	Any installed image checkpoint
Image-to-image	generate_image mode=img2img	Image checkpoint + input image
Inpainting	generate_image mode=inpaint	Image checkpoint + input image + mask
Background removal	generate_image mode=remove_background	Image + one supported background-removal node pack
Music/audio	generate_audio	ACE-Step 1.5 model set
Text-to-video	generate_video	Wan 2.1 T2V model set
Model search/download	huggingface_models	Network access; optional HF token
Item art	item_studio	SDXL + pixel LoRA + BiRefNet_lite-matting

Discovery and execution

Capability	Tool
List every installed model, grouped by ComfyUI model folder	comfyui_inspect scope=models
Search all available node classes	comfyui_inspect scope=nodes
Read a node's complete input/output schema	comfyui_inspect scope=node
Read server stats, embeddings, queue and history	comfyui_inspect scope=system, embeddings, queue, history, overview
Validate an API-format workflow locally	comfyui_workflow action=validate
Run a workflow and download every output	comfyui_workflow action=run
Collect a submitted prompt later	comfyui_workflow action=wait
Upload images, masks, audio, video or other inputs	comfyui_workflow action=upload
Inspect/clear/delete queue entries, interrupt, free memory	comfyui_workflow action=queue, queue_clear, queue_delete, interrupt, free
Call any other ComfyUI endpoint	comfyui_api

Cross-cutting behavior

• Model detection before submission for audio and video.
• Metadata sidecar next to every generated file, including the complete submitted workflow.
• Never overwrites an existing file: a numeric suffix is added.
• Four-layer configuration with immediate reload; config-file edits need no DSH restart.
• All settings in the DSH plugin page — every config key is editable in the web UI, with blank fields inheriting from file/env/default.
• Node catalogue caching with refresh=true after installing custom nodes.
• Local validation of arbitrary workflows: unknown nodes/inputs, missing required inputs, dangling links, enum/range violations.
• Binary outputs for images, GIFs, audio, video and arbitrary /view responses.
• Bounded responses so a large model list or raw API response cannot flood the Agent context.

Requirements

Item	Requirement
DSH	A version with the tools service; the plugin declares dsh.engines.dsh >= 0.2.0-rc.1
Node.js	>= 20 (built-in fetch / FormData / Blob)
ComfyUI	A reachable ComfyUI service, default http://127.0.0.1:8188
Image generation	At least one checkpoint in models/checkpoints
Audio generation	ACE-Step 1.5 AIO checkpoint or the split model layout
Video generation	Wan 2.1 T2V diffusion model, UMT5 safetensors encoder and Wan VAE
Model downloads	Optional Hugging Face token; a local ComfyUI/models path for direct installs
Sprite animation	Wan 2.2 i2v Q4 GGUFs, UMT5, Wan 2.1 VAE, GGUF/VRAM helper nodes and ComfyUI-RMBG (BiRefNet_lite); walk_up prefers Qwen-Image-Edit
Item Studio Flux.2	flux-2-klein-9b-fp8, qwen_3_8b_fp8mixed, full_encoder_small_decoder
Item Studio guided	SDXL, pixel LoRA, ControlNet Union, IPAdapter Plus, ComfyUI-RMBG
Background removal	One supported background-removal node pack (optional)

Image generation is the baseline. Audio and video are optional and are enabled only when their model sets are present. If they are missing, the tools return a precise support report; they do not silently substitute a different model family.

Background removal is the only image mode that needs extra nodes. Supported packs: ComfyUI_essentials, ComfyUI-BRIA_AI-RMBG, ComfyUI-LayerStyle and WAS Node Suite. The rembg Python dependency must have an ONNX runtime, for example pip install "rembg[cpu]".

Core ComfyUI's newer RemoveBackground node is not wired into this path because it outputs a mask rather than the finished image; use one of the supported packs above.

Installation

1. Start ComfyUI

Install ComfyUI following its official instructions, start it, and confirm:

• http://127.0.0.1:8188/system_stats returns JSON.
• http://127.0.0.1:8188/ opens the ComfyUI UI.

Put at least one image checkpoint in:

ComfyUI/models/checkpoints/

For Windows portable builds and Intel GPU notes, see docs/INSTALL.md.

2. Install the plugin from npm (recommended)

dsh plugin --profile <your-profile> add dsh-comfyui-workbench

Pin the current release explicitly if you prefer:

dsh plugin --profile <your-profile> add dsh-comfyui-workbench@0.8.0

You can also use DSH → Settings → Plugins and enter:

dsh-comfyui-workbench

Restart DSH after installing.

3. Alternative: install from GitHub

dsh plugin --profile <your-profile> add \
  github:TheRealChickenlegs/dsh-comfyui-workbench

The repository contains compiled lib/ output, so this path does not require TypeScript or a build step.

4. Alternative: link a local checkout

git clone https://github.com/TheRealChickenlegs/dsh-comfyui-workbench.git
cd dsh-comfyui-workbench
npm install
npm run build
npm test

Then in DSH → Settings → Plugins, enter:

link:/absolute/path/to/dsh-comfyui-workbench

Or add it manually to the profile's package.json:

{
  "dependencies": {
    "dsh-comfyui-workbench": "link:/absolute/path/to/dsh-comfyui-workbench"
  },
  "dsh": {
    "profile": {
      "bundles": ["dsh-comfyui-workbench"]
    }
  }
}

Then install profile dependencies and restart DSH.

Upgrade note

The package was renamed from dsh-comfyui-image to dsh-comfyui-workbench in 0.2.0. Remove the old bundle before installing the new one. Configuration and environment-variable fallbacks exist through 0.8.x. See docs/MIGRATION.md.

Verification

After installing and restarting:

1. The plugin list shows dsh-comfyui-workbench / ComfyUI Workbench, enabled.
2. The Agent can see nine tools:
   • generate_image
   • generate_audio
   • generate_video
   • generate_sprite_animation
   • item_studio
   • comfyui_inspect
   • comfyui_workflow
   • comfyui_api
   • huggingface_models
3. The startup log shows the effective ComfyUI address and configured checkpoint.
4. A basic image request works:

Generate a picture of a corgi astronaut on the moon, digital art style

For a local health check without going through DSH:

cd <plugin-directory>
npm run doctor

doctor prints the effective configuration, ComfyUI version/device, available checkpoints/samplers and a per-image-mode verdict. Add --json for machine-readable output or --base-url http://host:port to test another address.

Quick Start

Image

Generate a picture of a corgi astronaut on the moon, digital art style

Equivalent call:

{
  "prompt": "a corgi astronaut on the moon, digital art, highly detailed",
  "negative_prompt": "blurry, low quality, watermark",
  "width": 1024,
  "height": 1024,
  "seed": 42
}

The result includes an absolute path on its own line, a metadata_path, the seed, model, sampler and ComfyUI prompt_id.

Audio

Generate a warm neo-soul track at 92 BPM, 20 seconds, with a short chorus

{
  "prompt": "warm neo-soul, live drums, brushed hi-hats, 92 BPM, intimate studio",
  "lyrics": "[Verse]\nLate night glow on your skin\n[Chorus]\nStay right there",
  "duration_seconds": 20,
  "bpm": 92,
  "keyscale": "E minor",
  "seed": 42
}

Omit lyrics for instrumental music. The tool detects the ACE-Step 1.5 layout before submitting.

Video

Generate a 2-second video of a fox running through a snowy forest, tracking camera

{
  "prompt": "a fox running through a snowy forest, tracking camera, cinematic light",
  "width": 832,
  "height": 480,
  "length_frames": 33,
  "fps": 16,
  "seed": 7
}

generate_video supports the tested Wan 2.1 text-to-video graph. Other video families can be run through comfyui_workflow with their own API workflow.

Model download

Search Hugging Face for Wan 2.2 ComfyUI models, then download the high-noise GGUF into diffusion_models.

{
  "action": "search",
  "query": "Wan 2.2 ComfyUI"
}

Set the token in Plugins → ComfyUI Workbench → Configuration, in ~/.dsh-comfyui-workbench/config.json, or via HF_TOKEN. Set comfyuiModelsDir to your local ComfyUI/models path for direct installation; otherwise downloads go to a local staging directory. See docs/HUGGINGFACE.md.

Item Studio

Generate a base candidate for the long sword in the Item Studio style.

{ "action": "generate", "item_id": "long_sword", "variant": "base" }

See docs/ITEM_STUDIO.md for catalogue, batch, cutout and review details.

Sprite animation card

Animate this base sprite into a full ten-row card.

{
  "sprite_path": "godot/assets/player_sprites/player_draconid_bard_female.png"
}

Use animation: "walk_left" (or any single row) for a quick one-row test. See docs/SPRITE_ANIMATION.md.

Discovery

Use comfyui_inspect scope=models to show the installed checkpoints, loras and vaes.
Use comfyui_inspect scope=nodes query=audio to find audio-related nodes.

Custom workflow

Inspect the KSampler schema, then validate and run my API-format workflow.

See Tool Reference and docs/ADVANCED.md.

Tool Reference

generate_image

Four curated image modes.

Mode	Inputs	What it does
text2img	prompt	Text-to-image. Only this mode honors width / height.
img2img	prompt + input_image	Redraws an existing image. strength defaults to 0.6.
inpaint	prompt + input_image + mask_image	Repaints the white/bright mask area. strength defaults to 1.0.
remove_background	input_image	Removes the background. Requires one supported node pack.

Mode inference

When mode is omitted:

1. input_image + mask_image → inpaint
2. only input_image → img2img
3. neither → text2img

Conflicts are errors, not guesses. Passing a mask but no source image, passing input_image with text2img, or asking for inpaint without a mask all fail with an explanation.

Parameters

Parameter	Default	Notes
prompt	required for generation modes	Ignored by remove_background
negative_prompt	config default	Quality-focused default
input_image	required for edit modes	Absolute path, ~, or path relative to the Agent cwd
mask_image	required for inpaint	White/bright = region to repaint; alpha is not read
width / height	checkpoint-derived	64–8192, aligned to 8; only effective for text2img
seed	random	0–4294967295; reuse to reproduce
output_path	current project root	Directory or exact file path; never overwrites
strength	0.6 img2img / 1.0 inpaint	0–1; ignored by text2img/remove_background
steps	20	1–500
cfg_scale	7	0–100
model	auto-pick	Checkpoint file name; auto-pick prefers names containing xl
sampler_name	euler	Unknown explicit value is an error; stale config default falls back with a note
scheduler	normal	Same behavior as sampler
style_reference / character_reference	—	Reserved: verified and recorded in metadata, but do not affect generation yet
extra_options.workflow_path	—	Takes over generation with a custom API-format workflow

Output fields include text, path, files, mode, seed, width, height, steps, cfg_scale, strength, model, sampler, scheduler, prompt_id, comfy_url, duration_ms, success, mode_inferred, metadata_path and notes.

For dimension rules, ignored parameters and full error behavior, see docs/PARAMS.md.

Custom workflow placeholders

When extra_options.workflow_path is used, these placeholders are substituted:

Placeholder	Value
{{prompt}}	Positive prompt
{{negative_prompt}}	Negative prompt
{{width}} / {{height}}	Requested dimensions
{{seed}}	Seed
{{steps}}	Steps
{{cfg_scale}}	CFG
{{strength}}	Strength
{{checkpoint}}	Checkpoint actually used
{{input_image}}	Uploaded ComfyUI input filename
{{mask_image}}	Uploaded ComfyUI mask filename
{{mode}}	Actual mode

A whole-value placeholder preserves its type (a number stays a number); placeholders embedded in a larger string become strings; unknown placeholders are left as-is so typos are visible. End the workflow with SaveImage / PreviewImage (or another output node).

---

generate_audio

Generates music/audio with ACE-Step 1.5.

Supported layouts

Recommended: AIO checkpoint

ComfyUI/models/checkpoints/ace_step_1.5_turbo_aio.safetensors

AIO defaults: 8 steps, CFG 1.

Split layout

ComfyUI/models/diffusion_models/acestep_v1.5_xl_base_bf16.safetensors
ComfyUI/models/text_encoders/qwen_0.6b_ace15.safetensors
ComfyUI/models/text_encoders/qwen_4b_ace15.safetensors
ComfyUI/models/vae/ace_1.5_vae.safetensors

The split path also needs a DualCLIPLoader with the ace type. Split defaults: 50 steps, CFG 6.

Required nodes: TextEncodeAceStepAudio1.5, EmptyAceStep1.5LatentAudio, ModelSamplingAuraFlow, ConditioningZeroOut, VAEDecodeAudio, KSampler, SaveAudioMP3 (or SaveAudio), plus the relevant loaders.

Parameters

Parameter	Default	Notes
prompt	required	Style/tags description
lyrics	empty	Section tags such as [Verse] / [Chorus]; omit for instrumental
duration_seconds	30	1–600; used by the text encoder and latent node
bpm	120	10–300
timesignature	"4"	2, 3, 4, 6
language	en	Lyrics language code
keyscale	C major	For example E minor
seed	random	Reuse to reproduce
steps / cfg_scale	family default	8/1 AIO, 50/6 split
sampler_name / scheduler	euler / simple	Same values as ComfyUI
model	auto	Specific ACE-Step checkpoint/diffusion file
output_path	current project root	Directory or exact file path
timeout_ms	config timeoutMs	Max 1 hour
write_metadata	true	Write a .json sidecar

The result includes path, files, mode, seed, model, duration_seconds, prompt_id, comfy_url, metadata_path, notes, success and the detected family.

If no supported ACE-Step 1.5 setup is found, the tool reports detected audio models and the missing model/node pieces. It does not submit a broken graph. See docs/MEDIA.md.

---

generate_video

Generates a short video with Wan 2.1 text-to-video.

Supported layout

ComfyUI/models/diffusion_models/wan2.1_t2v_1.3B_fp16.safetensors
ComfyUI/models/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors
ComfyUI/models/vae/wan_2.1_vae.safetensors

Required nodes: UNETLoader, CLIPLoader with the wan type, VAELoader, CLIPTextEncode, EmptyHunyuanLatentVideo, ModelSamplingSD3, KSampler, VAEDecode, CreateVideo and SaveVideo.

Parameters

Parameter	Default	Notes
prompt	required	Video description
negative_prompt	quality-focused default	What to avoid
width / height	832 / 480	Aligned to 16
length_frames	33	Aligned to 1+4k; at 16 fps, 33 ≈ 2 s
fps	16	Output frame rate
seed	random	Reuse for a similar clip
steps / cfg_scale	30 / 6	Sampling quality
sampler_name / scheduler	uni_pc / simple	Same values as ComfyUI
model	auto	Specific Wan T2V diffusion file
output_path	current project root	Directory or exact file path
timeout_ms	config timeoutMs	Max 1 hour
write_metadata	true	Write a .json sidecar

The result includes path, files, mode, seed, model, width, height, fps, prompt_id, comfy_url, metadata_path, notes, success and the detected family.

Current video limitation

Only the tested Wan 2.1 text-to-video graph is enabled in generate_video.

Wan 2.2 I2V/VACE, HunyuanVideo, LTX-Video, SVD, Mochi and other families are intentionally not wired into the high-level tool because their graphs differ significantly (dual experts, image conditioning, VACE control, etc.). Use comfyui_workflow with a matching API workflow for those. The detection error still reports the video files it found so you can see what is available.

See docs/MEDIA.md for complete setup and troubleshooting.

---

generate_sprite_animation

One call from a base PNG to an animated sprite card.

Input	Default	Output
sprite_path	required	Base/front sprite
animation	all	One of idle, walk_down, walk_left, walk_right, walk_up, attack_down, attack_left, attack_right, attack_up, death
cell_size	64	Final card cell size (32, 64, 128)

The tool generates left/right turn clips and uses their last frames as keyframes. walk_down uses 25 W2 frames from the front master; walk_up uses a Qwen-Image-Edit back-view keyframe, 25 W2 frames, W3/BiRefNet and a per-frame scale lock. Other rows use W2/W3 with feet/centre alignment, box-downsampling and an eight-column card. The full card is 512×640; a single row is 512×64.

This is intentionally slower than generate_video: the full card is 13 W2 clips, a Qwen back-view edit and 10 W3 passes. Use animation: "idle", "walk_down" or "walk_left" to test before committing to all rows.

See docs/SPRITE_ANIMATION.md for models, prompts, the turn trick, alignment and the cache workaround.

item_studio

The saved Item Studio pack as one tool.

Action	What it does
list	Browse the bundled 108-item catalogue
generate	One item/variant via local Flux.2, guided SDXL or simple SDXL → sprite + icon
cutout	Re-cut an existing raw render
batch	Generate selected items/variants and write a candidate manifest

The catalogue supplies the orientation-aware prompt, family, tier, size, key colour and four variant suffixes. auto prefers local Flux.2 Klein at 256×256 (chunkier and faster than 1024), then the guided SDXL graph, then the simple pack. Flux.2 output uses border-key cutout + palette; guided/simple use BiRefNet. The tool writes raw/cutout/sprite/icon candidates and never auto-adopts.

See docs/ITEM_STUDIO.md for the full workflow, background policy and review gates.

comfyui_inspect

Read-only discovery. The default scope is overview.

Scope	Returns
overview	Model/node counts, system summary, queue state and recent history
system	ComfyUI system stats and device information
models	Every installed model file grouped by model folder
nodes	Search all node classes by name/category/description
node	Full input/output schema for one or more node classes
embeddings	Textual inversion embeddings
queue	Current queue state
history	Completed jobs, statuses, node types and output names

Options:

Option	Use
query	Filter node/model/history names
category	Restrict nodes to a category substring
folder	Restrict models to a folder such as checkpoints, loras, vae
node	Node class name, or a comma-separated list, for scope=node
limit / offset	Pagination for nodes/history
max_enum_values	Enum values shown per node input; 0 means no limit
refresh	Bypass the 60-second /object_info cache after installing nodes

Examples:

{ "scope": "models", "folder": "checkpoints" }
{ "scope": "nodes", "query": "controlnet", "limit": 25 }
{ "scope": "node", "node": "KSampler,VAEDecode,SaveImage" }
{ "scope": "history", "limit": 20 }
{ "scope": "nodes", "refresh": true }

A misspelled node name returns suggestions instead of failing silently.

---

comfyui_workflow

Validate, run, wait on, upload for, and control arbitrary ComfyUI workflows in API format (node id → { class_type, inputs }).

Action	Purpose
validate	Local structural validation
run	Validate and submit a workflow; optionally wait and download outputs
wait	Collect an existing prompt_id
upload	Upload an input file to ComfyUI's input directory
queue	Inspect the current queue
queue_clear	Clear pending queue entries
queue_delete	Delete specific prompt ids
interrupt	Interrupt the currently running job
free	Unload models and/or release cached memory

Common options:

Option	Use
workflow	Inline API-format workflow object
workflow_path	Path to an API-format workflow JSON; mutually exclusive with workflow
wait	For run: wait and save outputs (default true)
validate	Validate before submitting (default true)
output_path	Output directory, or exact file path for a single output
timeout_ms	Wait timeout; max 1 hour
poll_interval_ms	History polling interval
max_outputs	Maximum files to download (default 20, max 100)
save_outputs / write_metadata	Skip downloads or the sidecar
prompt_id	For wait
file_path / kind / subfolder / overwrite	For upload
queue_ids	For queue_delete
unload_models / free_memory	For free
include_history	Include the full history entry in the result data
refresh	Bypass the node-catalogue cache during validation

Validation checks:

• unknown node classes, with “did you mean” suggestions;
• unknown input names;
• missing required inputs;
• links to missing nodes or invalid output slots;
• enum and numeric range violations;
• missing output nodes (warning).

Example:

{
  "action": "run",
  "workflow_path": "D:/workflows/outpaint.json",
  "output_path": "D:/images/runs/outpaint-1",
  "timeout_ms": 600000,
  "max_outputs": 20
}

With wait: false, run submits and returns the prompt_id immediately; collect it later with action: wait. A metadata sidecar records the workflow, prompt id, status, outputs and history. Images, GIFs, audio, video and other binary outputs are downloaded through /view; text outputs are returned in the result.

---

comfyui_api

Raw HTTP access to any other endpoint on the configured ComfyUI server.

Parameter	Notes
method	GET, POST, PUT, PATCH, DELETE, HEAD
path	Must start with /; absolute URLs are rejected
query	Query parameters; arrays append multiple values
body / body_text	JSON or plain-text body; mutually exclusive
headers	Extra headers; unsafe/hop-by-hop headers are rejected
timeout_ms	Request timeout
response_type	auto, json, text or binary
save_path	Binary destination (file or directory)
max_bytes	Text/JSON render cap, default 100 KB, max 50 MB

Examples:

{ "method": "GET", "path": "/models" }
{ "method": "GET", "path": "/object_info/KSampler" }
{ "method": "POST", "path": "/free", "body": { "unload_models": true, "free_memory": true } }
{ "method": "GET", "path": "/settings" }

Guardrails:

• only the configured ComfyUI host is reachable;
• .. path segments, WebSocket paths and multipart upload endpoints are rejected;
• Host, Content-Length, Connection, Transfer-Encoding and Upgrade headers are ignored;
• redirects are not followed automatically, so a 3xx cannot silently redirect to another host;
• binary responses are saved in full; textual/JSON responses are capped by max_bytes.

Use the structured tools first when they cover the task.

huggingface_models

Search Hugging Face, list repository files, and download model files for ComfyUI.

Action	What it does
search	Find repositories by name/tag
files	List file paths, sizes and LFS SHA-256 values
download	Download one file atomically, with optional dry_run

Download destination:

1. output_dir when explicitly supplied;
2. otherwise <comfyuiModelsDir>/<folder>/ when comfyuiModelsDir is configured;
3. otherwise <modelDownloadDir>/<folder>/, defaulting to ~/.dsh-comfyui-workbench/downloads/<folder>/.

For a local ComfyUI, set comfyuiModelsDir to the ComfyUI/models directory. A remote ComfyUI cannot be written through its HTTP API, so configure a staging directory and copy the file to the server, or run the tool on the ComfyUI host.

{ "action": "search", "query": "Wan 2.2 ComfyUI" }
{ "action": "files", "repo_id": "Comfy-Org/Wan_2.2_ComfyUI_Repackaged" }
{
  "action": "download",
  "repo_id": "Comfy-Org/Wan_2.2_ComfyUI_Repackaged",
  "filename": "split_files/diffusion_models/wan2.2_i2v_high_noise_14B_Q4_K_M.gguf",
  "folder": "diffusion_models"
}

A token is optional for public repositories and required for gated/private repositories. Configure huggingFaceToken, use HF_TOKEN, or enter it in the plugin page. Accept the model license on huggingface.co first. The token is never returned in tool output or logged, and it is not forwarded across a cross-origin CDN redirect.

See docs/HUGGINGFACE.md for the complete download guide.

Model Detection and Missing Models

generate_audio and generate_video call comfyui_inspect internally before building a workflow. Detection checks:

1. the model enums exposed by ComfyUI's loader nodes;
2. the required node classes in /object_info;
3. the required text-encoder/VAE combinations.

If anything is missing, the tool returns a report like:

Audio generation is not available on this ComfyUI.

Detected audio-related models:
  - checkpoints/...

Missing:
  - No ACE-Step 1.5 audio model found...

Supported setups:
  - ACE-Step 1.5 AIO ...
  - ACE-Step 1.5 split ...

Video reports the same way, including cases where Wan 2.2 I2V/VACE GGUF files are present but the high-level Wan 2.1 T2V safetensors setup is not. This is intentional: it tells you what to install or that comfyui_workflow is the right path for that model family.

For complete supported layouts and examples, see docs/MEDIA.md.

Configuration

Plugin page (recommended)

Open Plugins → ComfyUI Workbench → Configuration in DSH. The page exposes all configuration keys and writes the active profile's plugin config, so it has the same highest-priority role as a profile config: block.

• Every field is optional.
• Leave a field blank to inherit from the config file, environment variable, or built-in default.
• Only fields you change are written; blank fields keep falling through to the lower layers.
• Saving takes effect on the next tool call, without a DSH restart.
• If a deployment stores settings read-only, the page shows that state and disables saving.

The server address is the first field; its placeholder is http://127.0.0.1:8188.

Priority

Four sources, highest priority first:

Plugin config in the profile's cordis.patch.yml
  > config file ~/.dsh-comfyui-workbench/config.json
    > environment variables (COMFYUI_URL, etc.)
      > built-in defaults

Merging is per key. Config-file edits and plugin-page saves take effect on the next tool call; no DSH restart is needed. Manual edits to the profile YAML require a DSH restart.

Config file

Default path:

~/.dsh-comfyui-workbench/config.json

Windows:

C:\Users\<you>\.dsh-comfyui-workbench\config.json

Example:

{
  "baseUrl": "http://127.0.0.1:8188",
  "checkpoint": "sd_xl_base_1.0.safetensors",
  "outputDir": "D:/images",
  "steps": 25,
  "cfgScale": 7,
  "timeoutMs": 600000,
  "saveToComfyUI": false
}

All keys

Key	Default	Environment	Description
baseUrl	http://127.0.0.1:8188	COMFYUI_URL / COMFYUI_BASE_URL	ComfyUI address; trailing / removed. Do not use 0.0.0.0 as a request target
outputDir	current project root	DSH_COMFYUI_WORKBENCH_OUTPUT	Default output directory; tool output_path wins; ~ expanded
requestTimeoutMs	30000	—	Per-request timeout, 1000–600000 ms
timeoutMs	300000	COMFYUI_TIMEOUT_MS	Total wait timeout, 1000–3600000 ms
pollIntervalMs	700	—	History poll interval, 100–10000 ms
checkpoint	auto-pick	COMFYUI_CHECKPOINT	Image checkpoint; auto-pick prefers names containing xl
negativePrompt	built-in quality prompt	—	Default negative prompt
steps	20	COMFYUI_STEPS	Sampling steps, 1–500
cfgScale	7	COMFYUI_CFG_SCALE	CFG, 0–100
samplerName	euler	COMFYUI_SAMPLER	Fallback with note if unavailable
scheduler	normal	COMFYUI_SCHEDULER	Fallback with note if unavailable
width / height	1024 / 1024	—	Default image size, 64–8192; only text2img
strength	0.6	—	Default img2img strength, 0–1
inpaintGrowMask	6	—	Mask growth in pixels, 0–256
saveToComfyUI	true	—	true = SaveImage copy in ComfyUI output; false = PreviewImage only
writeMetadata	true	—	Write the .json sidecar
rembgNode	auto	COMFYUI_REMBG_NODE	Background-removal plan: essentials, bria, layerstyle, was, or a node class
huggingFaceToken	none	HF_TOKEN / HUGGING_FACE_TOKEN	Optional Hugging Face read token for gated/private repositories
huggingFaceEndpoint	https://huggingface.co	HF_ENDPOINT	Hub API endpoint (compatible mirror supported)
comfyuiModelsDir	unset	DSH_COMFYUI_WORKBENCH_MODELS_DIR	Local path to ComfyUI/models; enables direct installation
modelDownloadDir	~/.dsh-comfyui-workbench/downloads	DSH_COMFYUI_WORKBENCH_DOWNLOAD_DIR	Staging root when comfyuiModelsDir is unset
maxDownloadBytes	53687091200 (50 GiB)	—	Per-file download safety cap

Plugin-only environment variables

Variable	Description
DSH_COMFYUI_WORKBENCH_CONFIG	Override the config-file path
DSH_COMFYUI_WORKBENCH_OUTPUT	Default output directory
DSH_COMFYUI_WORKBENCH_DEBUG	1 / true / yes enables debug logging
HF_TOKEN / HUGGING_FACE_TOKEN	Hugging Face read token
HF_ENDPOINT	Optional Hugging Face-compatible endpoint
DSH_COMFYUI_WORKBENCH_MODELS_DIR	Local ComfyUI models directory
DSH_COMFYUI_WORKBENCH_DOWNLOAD_DIR	Download staging directory

Legacy DSH_COMFYUI_IMAGE_* variables and ~/.dsh-comfyui-image/config.json are accepted as fallbacks throughout 0.8.x. They will be removed in 0.9.0.

Profile config

Edit ~/.dsh/profiles/<profile>/cordis.patch.yml:

- id: dsh-comfyui-workbench
  name: dsh-comfyui-workbench
  config:
    baseUrl: 'http://127.0.0.1:8188'
    checkpoint: 'sd_xl_base_1.0.safetensors'
    outputDir: 'D:/images'
    timeoutMs: 900000

Restart DSH after editing the profile. To disable temporarily, use the plugin manager or add disabled: true.

For the full config reference, see docs/CONFIG.md.

Outputs, Metadata and Files

Local outputs

Media	Default filename
Image	comfyui-<timestamp>-<random>.png
Audio	comfyui-wf-<timestamp>-<id>.<ext> (normally .mp3)
Video	comfyui-wf-<timestamp>-<id>.mp4

• Default directory: the Agent's current project directory.
• output_path can be a directory or an exact file path.
• Existing files are never overwritten: a -1, -2, ... suffix is added.
• Every generated file gets a .json metadata sidecar unless writeMetadata / write_metadata is false.

Metadata sidecar

Image sidecar:

<image>.png.json

Workflow/audio/video sidecar:

comfyui-workflow-<timestamp>-<prompt>.json

It records the plugin version, kind, prompt id, status, media type/family where applicable, output files, the complete workflow and the ComfyUI history entry.

ComfyUI-side files

• Images: ComfyUI/output/dsh-comfyui-workbench/
• Audio: ComfyUI/output/audio/dsh-comfyui-workbench/
• Video: ComfyUI/output/video/dsh-comfyui-workbench/

Set saveToComfyUI: false to avoid image copies; audio/video workflows always end in their respective save node.

Custom Workflows via generate_image

Anything the four image modes do not cover can be handled by extra_options.workflow_path:

{
  "mode": "img2img",
  "prompt": "upscale this, keep details",
  "input_image": "D:/pics/in.png",
  "extra_options": { "workflow_path": "D:/workflows/upscale.json" }
}

The workflow must be exported from ComfyUI in API format. Use the placeholders listed in generate_image and end with an output node.

For workflows that do not fit the image tool at all, use comfyui_workflow directly; it supports arbitrary node graphs, uploads and job control.

Security and Limits

• The plugin contacts only the configured ComfyUI baseUrl. It does not call a cloud service or telemetry endpoint.
• Local input files are uploaded only to the local ComfyUI /upload/image endpoint.
• comfyui_workflow can execute anything the installed nodes can execute. Treat it as privileged local code execution.
• comfyui_api can change ComfyUI state (/free, /queue, /settings, custom-node manager routes). Use it deliberately.
• Wait timeouts are capped at one hour; output downloads at 100 files; raw API text/JSON responses default to 100 KB.
• Local workflow validation is best-effort; ComfyUI remains the final authority.
• The plugin does not install models or custom nodes for you, except for files explicitly downloaded with huggingface_models. Model licenses and Hugging Face access are your responsibility.
• The Hugging Face token is used only in Authorization headers to the configured endpoint; it is never returned in tool output or debug logs. Prefer HF_TOKEN over storing it in config when possible.

Troubleshooting

Symptom	Fix
Plugin not visible in DSH	Check the bundle is enabled, then restart DSH; verify the npm package is installed under the profile
Cannot reach ComfyUI	Start ComfyUI, check baseUrl, remember that 127.0.0.1 inside a container is not the host
No checkpoints	Add a .safetensors checkpoint to ComfyUI/models/checkpoints/ and retry
generate_audio says unavailable	Install the ACE-Step 1.5 AIO checkpoint or the full split layout; check the reported missing nodes
generate_video says unavailable	Install wan2.1_t2v_*, a UMT5 safetensors encoder and wan_2.1_vae.safetensors; Wan 2.2 GGUF files require comfyui_workflow
Background removal unavailable	Install one supported node pack and pip install "rembg[cpu]"
comfyui_workflow: unknown node	Install/load the custom node, then search with comfyui_inspect and pass refresh: true
Workflow finished but no files	End the graph in SaveImage, PreviewImage, SaveAudio, SaveVideo or another output node
comfyui_workflow timed out	Raise timeout_ms, check the queue, or lower the workload; a timeout leaves the job running
Config change ignored	Check the profile config: and environment variables, which have higher priority; validate the JSON
width ignored	Only text2img honors width / height; edit modes take their size from the input image
npm install fails	Verify npm view dsh-comfyui-workbench version; pin @0.8.0 if a newer tag is unavailable
Generated media won't open	Check the ComfyUI console and the metadata sidecar for the submitted graph
Hugging Face download says 401/403	Check HF_TOKEN/huggingFaceToken read access and accept the model license on huggingface.co
Downloaded model not visible in ComfyUI	Set comfyuiModelsDir to the local ComfyUI/models path; for a remote server, copy the staged file into models/<folder>/ and restart/refresh ComfyUI

For the detailed FAQ, see docs/FAQ.md. For audio/video-specific troubleshooting, see docs/MEDIA.md. For advanced tool errors, see docs/ADVANCED.md.

Architecture

src/
  index.ts              Plugin entry: registers all nine tools
  tool.ts               generate_image contract, schema and result formatting
  media-tools.ts        generate_audio / generate_video contracts, detection and results
  huggingface.ts        Hugging Face API, redirect-safe download and atomic writes
  huggingface-tool.ts   huggingface_models search/files/download contract
  sprite-animation-tool.ts generate_sprite_animation W2/W3 pipeline
  item-studio-tool.ts   item_studio catalogue, generator selection, cutout, sprite/icon pipeline
  item-studio-flux2.ts  local Flux.2 Klein text-to-image graph
  item-studio-guided.ts ControlNet Union + IPAdapter Plus guided SDXL graph
  sprite-qwen.ts        Qwen-Image-Edit back-view + BiRefNet cutout graphs
  png.ts                dependency-free PNG decode/encode, crop/resize/outline
  runner.ts             Image execution engine
  params.ts             Image mode inference, defaults and conflict validation
  config.ts             Four-layer configuration + volatile unwrapping
  files.ts              Paths, filenames, MIME handling
  result.ts             Image result text/structure
  errors.ts             Errors with actionable hints
  log.ts                Debug logging
  comfy-tools.ts        comfyui_inspect / comfyui_workflow / comfyui_api
  media/
    capabilities.ts     Audio/video model and node detection
    workflows.ts        ACE-Step 1.5 and Wan 2.1 workflow builders
    runner.ts           Media submission, polling and collection
  comfy/
    client.ts           HTTP client for ComfyUI
    nodes.ts            Checkpoint/sampler/rembg probing
    workflows.ts        Image workflow builders + placeholder substitution
    introspect.ts       Model catalogue, node search, queue/history
    workflow-runner.ts  Arbitrary workflow validation, execution and output collection
  config-schema.ts     Volatile Config schema for the plugin-page editor
  client.js            Browser-side plugin-page configuration form
lib/                    Compiled output (committed so users need no build step)
test/                   164 tests + a mock ComfyUI
scripts/build-client.mjs Copy the browser bundle into lib/
scripts/doctor.mjs      Local self-check
docs/                   Install, config, usage, media, advanced, params, FAQ, roadmap

Design constraints:

• The plugin does not import host service packages; it obtains the tool registry through the host context. @deepseek-ai/schemastery is a normal runtime dependency used only for the editable config schema.
• The browser half is plain JS served by DSH's client module loader; npm run build copies it into lib/client.js.
• Tool parameters are raw JSON Schema using the DSH-supported subset. Unsupported keywords such as minimum or format are avoided; runtime validation provides the detailed errors.
• The execution layer is separate from the public contracts so additional ComfyUI families can be added without breaking existing tool calls.

Development and Testing

npm install
npm run typecheck
npm run build
npm test
npm run doctor

• lib/ is committed; after changing src/, run npm run build and commit the generated files.
• CI runs typecheck, build and tests, and checks that lib/ matches src/; the build also copies src/client.js to lib/client.js.
• The test suite uses a real mock ComfyUI with PNG encoding, uploads, polling, audio/video capability checks and error paths; no real ComfyUI is required.
• Current suite: 164 tests.

Release and npm

The package is published as dsh-comfyui-workbench on the official npm registry. The release process is automated by GitHub Actions:

• pushing a v* tag creates a GitHub Release;
• the Publish workflow runs tests and publishes to npm with provenance.

See docs/RELEASING.md for the tag, token, staged-publishing and manual fallback procedures.

Migrating from dsh-comfyui-image

Version 0.2.0 renamed the package/plugin id, config directory, environment variables and output prefixes. Tool names and call signatures are unchanged.

• Install dsh-comfyui-workbench instead of the old package.
• Move ~/.dsh-comfyui-image/config.json to ~/.dsh-comfyui-workbench/config.json, or rely on the 0.8.x fallback.
• Rename DSH_COMFYUI_IMAGE_* environment variables to DSH_COMFYUI_WORKBENCH_*.
• Expect new ComfyUI-side output prefixes, but existing files are untouched.

Full steps: docs/MIGRATION.md.

Documentation

Document	Contents
docs/INSTALL.md	ComfyUI + plugin installation, Windows/Intel notes, background-removal nodes
docs/CONFIG.md	Complete configuration, environment variables and priority rules
docs/USAGE.md	Image, audio, video and advanced usage examples
docs/MEDIA.md	Audio/video model layouts, parameters and detection troubleshooting
docs/HUGGINGFACE.md	Hugging Face token setup, model search/files/download and local-vs-remote ComfyUI paths
docs/SPRITE_ANIMATION.md	One-call W2/W3 sprite cards: turns, prompts, models, alignment and the 8×10 packer
docs/ITEM_STUDIO.md	Catalogue, Item Studio generation/cutout, sprites/icons, batch and review gates
docs/ADVANCED.md	Inspection, arbitrary workflows, uploads, job control and raw API
docs/PARAMS.md	generate_image parameters, ranges and output structure
docs/MIGRATION.md	Upgrade from dsh-comfyui-image
docs/FAQ.md	FAQ and troubleshooting
docs/ROADMAP.md	Future plans and limitations
docs/RELEASING.md	GitHub Release, npm publishing and provenance
CHANGELOG.md	Version history

License

MIT.