dsh-ui-layout

dsh-ui-layout

A DeepSeek Harness plugin that stops an AI agent from drawing UI blind.

Agents usually start painting before they know the coordinate system, the unit, the breakpoints or the stacking rules of the page they are building. This plugin forces a contract -> measure -> audit loop and gates UI-file writes on the contract:

1. ui_contract — declare the layout facts before writing UI code (coordinate system, unit, viewports, breakpoints, grid, spacing/type scales, layer stack, named regions, minimum interactive target size).
2. ui_measure — render a local HTML file or a localhost page in headless Chrome at the declared viewports and record real geometry: bounding boxes, computed styles, scroll overflow, stacking/source order and hit-test samples for interactive controls. Godot .tscn scenes and declared layout JSON are measured without a browser (rects resolved from anchors/offsets or canvas coordinates).
3. ui_audit — run deterministic rules over the measured data and report violations with node selectors, evidence and fix hints. Fix the source, re-run until the verdict is pass.
4. ui_diff — compare two snapshots and report appeared / disappeared / moved / resized nodes beyond a pixel threshold, so a layout change can be proved local instead of eyeballed.
5. ui_matrix — sweep every declared breakpoint in one pass, run the same rules per breakpoint, and write a rule × breakpoint matrix, one PNG per breakpoint, annotated violation PNGs (red boxes + numbered legend) and a self-contained HTML acceptance page. A page that passes at one width can overflow, overlap or lose controls at another.
6. A Web GUI panel ("UI 布局" in the left sidebar) shows the contract, latest verdict, violation cards, the breakpoint matrix with clickable annotated screenshots, and the latest layout tree — no file digging needed.
7. Engine sources — the same audit loop works for game UI: Godot .tscn scenes (Control anchors/offsets) and declared layout JSON (canvas pixels) are measured without a browser, with an SVG/HTML scene preview instead of screenshots.

A soft gate also blocks writes of UI files (html, css, jsx, tsx, vue, svelte, astro, …) inside the workspace until a contract exists. The plugin injects a "UI layout protocol" section into the system prompt so the agent declares a contract first, derives sizes from it, audits, and diffs after changes.

A stale-audit reminder closes the loop after the audit: when a measured page changes after its last ui_audit / ui_matrix, the next runtime context says the audit is stale and asks for a re-run, so an old PASS report cannot be cited as evidence for a page that has since moved.

Install

From the plugin market (npm)

The published package is dsh-ui-layout and the market index is served from jsDelivr:

https://cdn.jsdelivr.net/npm/dsh-ui-layout@1.0.0/market-index.json

Add that URL in Discover → Index sources, then install dsh-ui-layout@1.0.0 from the market list (the market calls plugin_manager with the npm spec). Equivalently:

plugin_manager action=install_bundle target="dsh-ui-layout@1.0.0"

From this checkout (development)

The bundle lives at plugins/dsh-ui-layout in this workspace and can be installed as a profile bundle:

plugin_manager action=install_bundle target="<absolute path to plugins/dsh-ui-layout>"

If a linked checkout is already installed, apply code changes by toggling it in the running host:

plugin_manager action=set_bundle target="dsh-ui-layout" enabled=false
plugin_manager action=set_bundle target="dsh-ui-layout" enabled=true

A tarball built with npm pack installs the same way (install_bundle with the .tgz path); that is the closest local check to a real market install.

Publish

npm run validate:market   # parse market-index.json with the DSH-shipped validator
npm run publish:market    # stamp publishTimestamp, npm publish --access public, verify the tarball

publish:market prints the jsDelivr index URL to add as an index source and the plugin_manager install target. publish:market -- --otp=123456 passes a one-time password; -- --dry-run shows the tarball contents without uploading. The index must be published at the same version as the package (https://cdn.jsdelivr.net/npm/dsh-ui-layout@<version>/market-index.json).

application: "applied" means the running composition recomposed. application: "failed" carries a diagnostic string with the real activation error. Replacing an installed npm package needs an app restart; a linked local checkout reloads through the bundle toggle.

Requirements: Google Chrome (or Edge / Chromium) on the machine, and puppeteer-core resolvable from the DSH profile. The plugin finds Chrome via CHROME_PATH / PUPPETEER_EXECUTABLE_PATH, then the usual Windows, macOS and Linux locations. puppeteer-core is resolved from the DSH profile directory, DSH_HOME, the process cwd, or the plugin directory.

Quick start

// 1. Declare the contract (missing fields fall back to safe defaults).
ui_contract action="set" contract={
  "platform": "web",
  "viewports": [{"id":"mobile","width":390,"height":844},{"id":"desktop","width":1440,"height":900}],
  "regions": [{"id":"header","selector":"header"},{"id":"main","selector":"#main"}],
  "interactive": {"minSize": 24, "recommendedSize": 44},
  "safeArea": {"insets": {"top": 0, "right": 0, "bottom": 34, "left": 0}}
}

// 2. Render and measure the page.
ui_measure source={"type":"file","path":"src/index.html"}

// 3. Audit the snapshot (or pass a source to measure+audit in one step).
ui_audit snapshotId="e73f4cd3cc8b"
ui_audit source={"type":"url","url":"http://127.0.0.1:5173/"}

// 4. Prove a change stayed local: compare the before/after snapshots.
ui_diff before={"snapshotId":"08974af9e9b5"} after={"snapshotId":"e74e99cc31a3"} thresholdPx=4

// 5. Sweep every declared breakpoint and get a readable matrix + screenshots + annotated violation images.
ui_matrix source={"type":"file","path":"src/index.html"} annotate=true

ui_audit returns verdict: pass|fail, counts, and per-violation rule, severity, viewportId, message, selector and fixHint in the stored report. Exit criterion for an agent: pass, or every remaining violation explicitly accepted by the user.

ui_matrix returns the per-breakpoint verdicts, a responsive note list (which rule fails at which width), the screenshot paths, the annotated paths, and reportPath / htmlPath. Open matrix.html in a browser to see the rule × breakpoint table, the screenshots side by side, a separate annotated section and every violation with its selector. Annotated PNGs draw each violation's real rect on the screenshot (error red #e5484d, warning amber #f5a524), number the boxes and append a legend strip (number → rule → selector, max 10 rows); passing breakpoints get no annotated image. Set annotate=false to skip the extra rendering pass.

ui_diff returns verdict: changed|unchanged, summary counts and per-node changes (appeared, disappeared, moved, resized, with old/new rects). Its before/after sides accept either a snapshotId or a source (measured and snapshotted on the fly).

Web GUI panel

The bundle ships a client half, so once the plugin is enabled the DSH Web UI gains a left-sidebar entry "UI 布局" / "UI layout guard" (order 3) that opens a central panel instead of a settings page. Four tabs:

• 概览 / Overview — the declared contract (platform, unit, viewports, breakpoints, spacing/type scale, layers, regions, hash) and the latest audit or matrix evidence with its verdict and counts.
• 违规 / Violations — the latest report's violations as cards: severity chip, rule id, message, node selectors and fix hint. Warnings are kept visible even when the verdict is pass.
• 断点 / Breakpoints — the rule × breakpoint table, responsive notes, the annotated PNG per failing breakpoint (click to open the full-size file) and a link to the full matrix.html.
• 布局树 / Layout tree — the latest snapshot's node tree for a chosen viewport: tag, selector, rect, indentation, and an interactive chip.

Data comes from the read-only host endpoints /ui-layout-api/state, /report, /snapshot and /file, registered by the plugin when webServer is present. They answer GET/HEAD only, return JSON errors as { ok:false, error }, and file refuses any path outside <workspace>/.dsh-ui-layout (path-traversal guard). The panel only reads the workspace store; it never triggers a measurement by itself — run ui_audit / ui_matrix first, then refresh the panel.

Client locales: Chinese and English dictionaries are registered through the client locale service; styling uses the GUI theme tokens (--dsw-alias-bg-layer-*, --dsw-alias-border-l1), so it follows the current theme.

Engine sources (Godot .tscn, declared layout JSON)

Game and canvas UI have no DOM, so ui_measure / ui_audit / ui_matrix accept two browser-less sources:

// Godot 4 scene: Control anchors/offsets are resolved per viewport. Scripts/themes are ignored.
ui_audit  source={"type":"godot","path":"ui/Shop.tscn"}
ui_matrix source={"type":"godot","path":"ui/Shop.tscn"}

// Declared layout: x/y/width/height are canvas pixels relative to the parent's top-left corner.
ui_audit  source={"type":"layout","path":"ui/shop.layout.json"}
ui_audit  source={"type":"layout","layout":{"nodes":[
  {"name":"Root","width":390,"height":844},
  {"name":"Buy","parent":"Root","type":"Button","x":0,"y":0,"width":120,"height":44,"interactive":true}
]}}

How the engine view works:

• Rects: Godot anchor_* + offset_* (or the layout JSON numbers) → absolute rect per measured viewport. Declared layouts are not rescaled per viewport, so measuring a narrow window shows real overflow/collision.
• Hit testing is synthesized: paint order is depth-first, siblings by z_index then declaration order; a control is "occluded" only when an unrelated topmost node covers its samples.
• Rules: overlap, occlusion, overflow, target.min-size, spacing.drift, contract.mismatch run; zorder.inversion, clip.text, interaction.dead and safearea.obstruction are skipped (they need real stacking contexts / scroll metrics / href-a11y / fixed chrome).
• Container children (VBoxContainer, PanelContainer, …) are laid out by the engine at runtime, so they are marked approx, excluded from geometric rules and listed in the snapshot warnings — no false positives.
• Region selectors for engine scenes accept node paths (Panel/BuyButton), the final path segment (BuyButton) or an engine type (Button).
• Artifacts instead of screenshots: each report writes preview-<viewport>.svg plus a self-contained preview.html (green = interactive, red/amber = violating node, dashed = approximated rect).

Soft gate

Writes of UI files into the workspace are denied until .dsh-ui-layout/contract.json exists:

Error: UI_LAYOUT_CONTRACT_REQUIRED: this workspace has no UI layout contract, so index.html was not written.
Call ui_contract action="set" first ...

• Gated tools: write, edit, multi_edit, create_file, apply_patch, patch_file, replace_in_file.
• Gated extensions: .html, .htm, .xhtml, .css, .scss, .sass, .less, .jsx, .tsx, .vue, .svelte, .astro, .cshtml, .razor.
• Not gated: non-UI files, paths outside the workspace, read-only tools, shell commands and the contract file itself. The gate is a nudge, not a security boundary — bash/pwsh writes bypass it.
• To pass: declare the contract. To remove the gate: disable the bundle.

Stale-audit reminder

ui_audit and ui_matrix record, per measured file, when the last measurement ran (<workspace>/.dsh-ui-layout/state.json). A registered system-prompt context provider (ui-layout-stale-audit, order 250) reads those records on every prompt assembly and emits, for up to 4 known workspaces:

UI layout audit is stale (dsh-ui-layout):
- E:\work\DSH: .dsh-tmp\page.html changed less than a minute after the last audit
  (334951a51858, pass). Re-run ui_audit (or ui_matrix) before declaring UI work done;
  the previous evidence describes the old page.

Re-running the audit clears the note. URL sources are not tracked (the plugin cannot tell when a dev-server page changed), and the reminder is advisory — it never blocks a tool call. Workspaces are discovered through a small registry at %DSH_HOME%/ui-layout/roots.json, written the first time any layout tool resolves that workspace.

Contract defaults

Field	Default
platform	web
coordinateSystem	{unit:"css-px", origin:"top-left", yDirection:"down"}
viewports	mobile 390x844, desktop 1440x900
breakpoints	sm 0, md 768, lg 1024, xl 1280
grid	12 columns, 16 gutter, 24 margin, max width 1200
spacingScale	0, 4, 8, 12, 16, 24, 32, 48, 64
typeScale	12, 14, 16, 20, 24, 32, 40
layers	base 0, sticky 100, dropdown 500, modal 1000, toast 2000
interactive	min 24, recommended 44 (WCAG 2.5.8 AA / 2.5.5 AAA)
safeArea	{insets:{top:0,right:0,bottom:0,left:0}} (declare device insets such as a 34px bottom strip)

interactive.minSize is the error threshold; recommendedSize is advisory (reported as info).

Rules

Rule	Severity	Meaning
overlap.unintended	error	Sibling boxes overlap by more than the allowed ratio; usually a missing grid/flex track or a negative offset.
occlusion.hit-test	error	An interactive element's hit-test samples are covered by another element, so users cannot click it (stacking context / overlay / pointer-events).
zorder.inversion	error	An element declares a high z-index yet is painted below a lower-z sibling: its nearest stacking-context ancestor caps it, so adding more z-index to the child cannot help.
overflow.viewport	error	A node extends past the viewport or its scroll container, producing horizontal scroll or cut content.
clip.text	error	Text is clipped by its own box (scroll size exceeds client size) without an intentional ellipsis/clamp.
target.min-size	error	An interactive target is smaller than the contract minimum in either dimension.
interaction.dead	error	An interactive control is pointer-events: none, so the click it advertises can never reach it.
interaction.dead	warning	A link has no usable destination (href missing, empty, # or javascript:), or a role-based control cannot take keyboard focus.
spacing.drift	warning	A gap between adjacent siblings is 1-3px off the contract spacing scale (hand-tuned values), so the layout stops following the design system.
safearea.obstruction	error	Bar-shaped fixed/sticky chrome (bottom bar, header) covers content because no safe-area padding was reserved.
safearea.obstruction	warning	A fixed bar crosses the contract's declared safeArea.insets (e.g. the Home-indicator strip).
contract.mismatch	error	A declared region is missing from the rendered page (plus contract validation errors).

Occlusion is deliberately a hit-test problem, not a geometry problem: elementsFromPoint at five samples (center plus four inset corners) resolves which element actually receives the pointer.

zorder.inversion escalates the same idea to non-interactive nodes that claim a numeric z-index of 100 or more: three point samples (elementsFromPoint) find which element really paints on top, and when that element's z-index is lower than the claim, the report names both the covering element and the nearest stacking-context ancestor (the trap) with its z-index. Message example:

Element declares z-index 9999 yet is painted below z-index 5 (#main > div:nth-child(2)):
its nearest stacking-context ancestor #main > div:nth-child(1) sits at z-index 1,
so the high z-index cannot escape that context.

spacing.drift compares the measured gap between adjacent siblings (stacked or side-by-side, same parent, in-flow) with the nearest step of contract.spacingScale; a drift larger than 0.5px and up to 3px is a warning, because 1-3px is the usual residue of hand-tuned margins. It does not fail the audit — the report still says PASS — but nextActions suggests snapping the gap to the scale. Matrix output shows these per breakpoint as amber 3w cells and a Warning violations section. Message example:

Sibling spacing is 13px, 1px off the contract spacing scale (nearest 12px) between stacked siblings.

safearea.obstruction covers the "content scrolls underneath fixed chrome" bug class: a bottom bar over the last list row, a header over the first paragraph. It only considers bar-shaped fixed/sticky elements (width ≥ 50% of the viewport and height ≤ 40%, or the transposed shape), because full-screen modals and backdrops intentionally cover content — those stay under occlusion.hit-test. A bar covering at least 35% of a content node's box is an error; a bar crossing a declared safeArea.insets side is a warning. Message example:

Fixed fixed element html > body > div:nth-child(2) covers 92% of div "Retention policy was updated."
(358x48px) at mobile: content scrolls underneath it because no safe-area padding was reserved.

The fix is to reserve padding (or a spacer element) equal to the bar height on the scrolling content, and to keep overlay chrome inside contract.safeArea.insets.

interaction.dead reports controls that look actionable but cannot fire. Only statically provable cases are reported: pointer-events: none on an interactive node (error — the pointer can never reach it), an anchor whose href is missing/empty/#/javascript: (warning), and a role=button|link|checkbox|… element that is not natively focusable and has no non-negative tabindex (warning). Native <button> elements, real hrefs and disabled controls stay clean. Message examples:

Interactive div "Archive" (#main > section:nth-child(2) > div > div:nth-child(2)) has pointer-events: none,
so it can never receive the click it appears to offer.
Link #main > section:nth-child(2) > div > a:nth-child(3) "Preferences" has no usable destination (href=null),
so clicking it does nothing.

Storage

Per workspace, under .dsh-ui-layout/:

.dsh-ui-layout/
  contract.json          # the declared layout contract
  state.json             # last audit/matrix per measured file (drives the stale-audit reminder)
  snapshots/<id>.json    # measured layout trees (per viewport)
  reports/<id>.json      # audit and diff reports, plus latest.json / latest-matrix.json
  reports/<id>/          # matrix artifacts: matrix.json, matrix.html, shots/<bp>.png, shots/annotated-<bp>.png

The global workspace registry lives outside the workspace, at %DSH_HOME%/ui-layout/roots.json.

Node schema in a snapshot: id, parentId, depth, siblingIndex, tag, role, selector, text, rect, scroll, style (display/position/zIndex/opacity/overflow/pointerEvents/transform/fontSize), stacking (createsContext/contextRootId/paintOrder), interactive, visible, hitTest (samples/occluded/topmost).

Tests

The repo test suite runs the real pipeline against local fixtures without the host, using the same Chrome resolution as the plugin:

node plugins/dsh-ui-layout/tests/run.mjs

It covers: contract defaults/validation/hash, three clean pages passing at both viewports, each rule firing on its dedicated fixture (including zorder.inversion on a stacking-context trap, spacing.drift on both axes, safearea.obstruction for a fixed bottom bar plus the reserved-padding and declared-inset cases, and no false positives on clean pages), missing-contract detection, report storage, the ui_contract -> ui_measure -> ui_audit tool pipeline, the write gate (blocked without a contract, allowed after, non-UI and outside-workspace paths unaffected), ui_diff (same snapshot unchanged, a 12px shift detected as moved, report saved) and ui_matrix (every contract breakpoint swept, a responsive-broken fixture failing only at narrow widths, viewport screenshots, annotated violation PNGs only for failing breakpoints, warning-only rules reported as warns at ... without failing the verdict, and the HTML/JSON artifacts written), the stale-audit state machine (audit recorded per file, fresh audit silent, modified page triggers the reminder, a newer audit clears it, workspace root registry), the Web GUI panel data layer (four exact routes under /ui-layout-api/, report-relative artifacts mapped into the store, path traversal rejected, matrix view paths rewritten, non-GET rejected with 405), and the engine importers (Godot control tree and resolved rects, synthesized hit tests flagging only the lower-painted control, browser-only rules skipped, overflow only at narrow breakpoints, declared layout JSON overlap/min-size detection, the clean layout passing, inline layout objects, and the SVG/HTML preview artifacts) — 93 checks, no host required.

Known limitations (P1)

• Measures rendered DOM for HTML files and localhost URLs; framework dev servers work through their URL, but there is no component-level import or screenshot baseline yet.
• Engine sources cover Godot 4 .tscn text scenes and declared layout JSON. Godot rects use Control anchors/offsets only (theme custom_minimum_size is not applied), container-managed children are approximated and excluded from geometric rules, and no engine scaling factor is applied. Unity YAML and Unreal are not supported yet.
• Hit-test sampling is 5 points per element, which is accurate for rectangular controls but can miss small irregular overlays.
• The soft gate covers file-write tools only; shell writes bypass it.
• The Web GUI panel is a read-only viewer over the workspace store: it lists workspaces seen by the layout tools (URL-only sources never appear), caps the violation list at 127 and the tree at 600 nodes, and needs a page refresh after a plugin reload to pick up client-side changes.
• Breakpoint heights come from the contract viewport with the same width, otherwise 900px; the sweep covers widths only, not short-height/landscape cases.
• zorder.inversion only inspects non-interactive nodes that declare a numeric z-index of 100 or more, at three sample points; it reports the covering element and the trapping ancestor, not the full paint list. Elements with z-index: auto that lose because of paint order between contexts still surface as occlusion.hit-test when they are interactive.
• spacing.drift only compares adjacent siblings under the same parent, skips absolute/fixed/inline nodes and gaps below 6px, and only reports 1-3px drift; cross-section spacing and larger hand-picked values are out of scope for now.
• safearea.obstruction recognises bar-shaped fixed/sticky elements only, skips ancestor/descendant pairs and uses a fixed 35% coverage threshold; safeArea.insets come from the contract, not from real env(safe-area-inset-*) device values (headless Chrome has no device safe area).
• interaction.dead is static-signal based: page JS is not executed, so an href="#" link with a JS click handler is still flagged as a warning (it does not fail the audit), keyboard reachability is a tabindex / native-tag heuristic, and aria-disabled is not consulted yet.
• The stale-audit reminder only tracks file sources (a localhost URL cannot be checked for changes), uses a 250 ms mtime tolerance, and needs the workspace to have been seen by at least one layout tool before it can appear in a later session.

Roadmap

• P1 complete: contract, measure, audit (10 rule entries / 9 unique rules), soft gate, diff, breakpoint matrix with annotated violation PNGs, stale-audit reminder.
• P2 delivered: Web GUI panel with contract/evidence overview, violation cards, breakpoint matrix with annotated previews and the layout tree; engine importers for Godot .tscn scenes and declared layout JSON with SVG/HTML scene previews. Remaining P2: Unity YAML importer, packaging and publishing.