Skip to main content
M

phone-use-dsh (dsh)

mig-everything/phone-use-dsh/packages/dsh

Agent tools for connected HarmonyOS, Android and iPhone devices, with native controls, on-demand screenshots and OCR, and a multi-device sidebar.

Install

dsh plugin --profile web add github:mig-everything/phone-use-dsh

This plugin lives in the packages/dsh subdirectory of the repository.

README

Phone Use

M8ven Score

Operate connected HarmonyOS, Android and iPhone devices through a DeepSeek Harness plugin, a local Codex/ChatGPT Work plugin, standard MCP tools, or a TypeScript SDK. Each integration uses the same device core; supported desktop hosts can also display the multi-device panel for viewing and manual input.

中文安装与使用:DSH 指南 · Codex/Work 与设备准备 · MCP/SDK 接入 · 发行方案 · 验证范围

IntegrationInstallation available todayInterface
DSHPrebuilt phone-use-dsh.tgz from GitHub ReleasesNative conversation tools and sidebar
Codex / ChatGPT Work local pluginPortable plugin archive, then scripts/install-work.mjsTools and skill; conversation panel in a compatible desktop host
Other MCP clientsMCP/SDK archive, then register the installed CLIStandard stdio or local Streamable HTTP; no sidebar required
Node.js / TypeScript applicationsInstall the same MCP/SDK archive in your projectSDK exports for devices, tools and preview integration

Version 0.9.5 has separate DSH, Codex/Work and MCP/SDK Releases, each with SHA-256 checksums. Only DSH uses the repository's Latest entry. See distribution and upgrades.

Install in DSH

Download the prebuilt bundle from GitHub Releases. In DSH, open Plugins → Add plugin, enter the downloaded .tgz file path, and enable Phone Use. No npm account or source compilation is required.

The official CLI can also install the public Release asset:

dsh plugin --profile desktop add --save-exact --ignore-scripts https://github.com/mig-everything/phone-use-dsh/releases/latest/download/phone-use-dsh.tgz

Refresh the plugin list and confirm the component is running. Open Phone Use ▾ in a conversation header, then add a device. Ask the agent to inspect or operate that device; the sidebar is optional for agent tools.

The package is not published to npm. @mig-everything/phone-use-dsh is the bundle's package identity; install the Release URL or downloaded file, not that name alone. Releases include a versioned archive, the stable phone-use-dsh.tgz filename, and SHA-256 checksums. Each release tag keeps its own assets.

Install the Codex / ChatGPT Work local plugin

Download the portable plugin. Node.js 22+, npm and network access for production dependencies are required; the archive is already compiled:

tar -xzf phone-use-0.9.5.tar.gz
cd phone-use-0.9.5
node scripts/install-work.mjs

The installer prepares a managed plugin copy with production dependencies and prints its marketplace-specific installation command. For the default marketplace:

codex plugin add phone-use@phone-use-local
node scripts/check-work.mjs

Use the command printed by the installer if your personal marketplace has another name. To select a device configuration, pass --config /absolute/path/phone-use.json to the installer. Then reconnect a local desktop conversation and select Phone Use; open the conversation panel once and switch devices within it. Terminal clients use the tools without a graphical sidebar.

The loader check verifies the installed MCP version, tool inventory, conversation entrypoint and panel HTML. It does not test desktop rendering or physical phone input. See setup and updates for configuration, duplicate MCP registrations and host reloads.

Connect another MCP client or use the SDK

Install the MCP/SDK archive in your Node.js project:

npm install --ignore-scripts https://github.com/mig-everything/phone-use-dsh/releases/download/core-v0.9.5/phone-use-0.9.5.tgz

Clients with an MCP subprocess configuration can start the installed entry:

node /absolute/path/to/project/node_modules/phone-use/scripts/start.mjs stdio --config /absolute/path/phone-use.json

The same CLI supports authenticated local Streamable HTTP through serve. MCP tools work without a panel; screenshots require the client to forward image content to a model that accepts it. MCP Apps rendering depends on the host, and each agent framework still needs its own integration check. See MCP setup and integration contracts.

For a Node.js application, import from phone-use, phone-use/mcp or phone-use/preview after the same installation. This root package contains both the MCP CLI and SDK; the DSH archive is a different package. Neither package name is published to npm. To build from source, use npm ci and npm run package:core.

Separate agent conversations need separate trusted scopes. Ordinary MCP clients share one scope within each server process unless they provide the supported conversation metadata; use one process per conversation or a host adapter that supplies its own trusted scope. Reuse the same device state directory so separate hosts respect device ownership.

Device requirements

Configure the development tools on the computer running Phone Use and authorize the device's debugging connection. See setup and the example configuration.

PlatformDevice connectionPreview and input
Native HarmonyOSHDC and system UiTestBundled ARM64/x86_64 UiTest bridge; CLI/screenshot fallback
AndroidADBBundled scrcpy server for H.264 and touch; optional emulator gRPC
iPhoneA running, configured WDA endpointWDA screenshots/MJPEG and actions
iOS SimulatormacOS, Xcode, runtime and WDAOptional idb video and bundled native input bridge

The bundles include the shared runtime, browser UI, native helpers and local OCR models. Installation downloads production Node dependencies. They do not include platform SDKs, emulator images, WDA runners or signing credentials. DSH Desktop provides its own Node runtime; the standalone MCP and local plugin installer require Node.js 22+.

Pass PHONE_USE_CONFIG=/absolute/path/phone-use.json in the host environment, or configure the component in the profile's cordis.patch.yml:

- id: phone-use-dsh
  config:
    configPath: /absolute/path/phone-use.json

To try the interface without a device, use demo: true instead of configPath. Demo devices are synthetic; they do not verify hardware compatibility.

What it does

  • Discovers devices and keeps device ownership scoped to the conversation.
  • Reads compact native control indexes before requesting screenshots. Local Chinese/English OCR is available on demand, with up to three workers.
  • Supports guarded taps, swipes, focused text input, keys, app launch and supported emulator posture controls.
  • Maps a shared target to each device separately; batch actions retain per-device results and observation checks.
  • Reuses one sidebar for switching or displaying multiple devices. Authenticated DSH Fetch transports preview and manual input without manually configuring local certificates.
  • Rejects stale observations and changed display geometry. Uncertain actions are not automatically replayed.

Preview frames stay out of model context. Screenshot observations are supplied to models that declare image input; native controls and OCR remain available to text models.

Multi-device Phone Use interface with synthetic demo devices

The image runs the real panel in a browser protocol-test host with synthetic demo devices. It is not a physical-phone or DSH desktop screenshot. Dark layout · Narrow layout.

Validation and limitations

The release checks cover the shared core, three platform protocols, MCP without UI, DSH conversation isolation, browser input, local OCR, native artifacts, and an installation into the official DSH runtime with independently installed production dependencies. See verification for the exact results and Actions for CI.

Device checks have used HarmonyOS 6/6.1, Android and iOS simulators. Physical devices and HarmonyOS 5/7 are not comprehensively validated. iOS native simulator input is limited to verified portrait geometry; other cases use WDA. Android text input currently supports ASCII. Fold/rotation changes invalidate old coordinates; physically folding a phone remains a user action. There is no fixed FPS guarantee or built-in cross-platform business-comparison workflow.

Build and extend

git clone https://github.com/mig-everything/phone-use-dsh.git
cd phone-use-dsh
npm ci
npm test
npm run check:native
npm run package:dsh

Install the resulting releases/mig-everything-phone-use-dsh-0.9.5.tgz. The packages/dsh directory is a build input, not a complete installable source package. The Release bundle already includes its compiled shared core and client entry.

DirectoryResponsibility
src/core, src/adapters, src/transportShared device sessions, observation validation and platform channels
src/hosts/dsh, packages/dsh, ui/dsh-client.jsDSH tools, skills, conversation identity and sidebar transport
src/mcp.ts, src/tools.tsMCP and host-independent tool definitions
native, assets/ocrNative helpers, model assets and upstream notices

The SDK and MCP operate without DSH. Build this repository, then use the integration guide, MCP setup, or npm run demo. Changes belong in the shared source; the DSH bundle embeds its build output rather than a separately maintained device implementation.

npm run package:plugin builds the portable Codex/Work archive; npm run package:core builds the MCP/SDK root package. Both produce SHA-256 checksums. npm run check:distributions installs and exercises these archives with independent production dependencies and demo devices. These commands create local files; they do not publish releases or update installed plugins. See release layout and upgrade impact before uploading artifacts.

Credits and license

MIT. The tool + screen + skill design was inspired by iPhone-use; this service is an independent implementation. Bundled upstream components retain their licenses: scrcpy, OpenHarmony, HMBot bridge, Meta idb, and OCR models. Device frame artwork is read from locally installed SDKs, not redistributed here.

Related plugins