dsh-plugin-android-vscreen

dsh-plugin-android-vscreen

Let an agent work on your Android phone without taking over your screen. 让 AI 在安卓手机的虚拟显示器上干活 —— 手机你自己的屏幕照常用，互不打扰。

A DeepSeek Harness plugin. Tested with DSH on Android; the adb transport also works with DSH on a computer.

Looking for the whole thing ready to install on a phone (no PC, no Termux)? That is the Android port — same author, APK + in-app installer.

---

Why

Most "agent drives my phone" plugins tap the main screen: your phone is hijacked while the agent works — you can't use it, and it can't be used quietly in the background.

Android can create a virtual display: a second, invisible screen. The agent launches an app onto that display, looks at it and taps it there, while the physical screen keeps doing whatever you were doing. Every action reports a "main-screen proof" (mainBefore → mainAfter) so you can see the physical screen was not touched.

⚠️ A virtual display isolates the picture, not the effects / 只隔离画面，不隔离副作用

An app running on a virtual display still acts on the whole device. Tap Airplane mode, Wi-Fi or Do not disturb inside a virtual display and that switch really flips — it is the same system setting. Installing, sending messages, changing settings: all real.

So "your screen stays untouched" is not the same as "your device stays untouched".

Rule for agents using this plugin: take a frame with android_vscreen_see first, and derive tap coordinates from what you actually see. Never tap a guessed fraction of the screen.

Tools

Tool	What it does
android_vscreen_doctor	Can I drive this device, and over which transport. Run it first.
android_vscreen_status	Is a virtual display running, and how big is it.
android_vscreen_start	Create the virtual display and optionally launch an app onto it.
android_vscreen_see	Grab a frame of the virtual display as a PNG.
android_vscreen_tap	Tap on the virtual display.
android_vscreen_swipe	Swipe on the virtual display.
android_vscreen_close	Destroy the virtual display.

How it reaches the phone

Two transports, picked automatically:

Transport	When it's used
adb	DSH runs on your computer (USB or wireless debugging)
local bridge (127.0.0.1:3081)	DSH runs on the phone itself (e.g. the Android build)

Both end up as a shell on the device — the same privilege level adb shell has. That privilege is required: an app cannot put another app onto a virtual display with its own identity, so the bundled core runs as shell via app_process. It uses only public Android APIs (DisplayManager.createVirtualDisplay, ImageReader) plus documented reflection constants — no MediaProjection, no screen-capture permission dialog.

Install

dsh plugin forwards to pnpm, so several specs work. The release tarball needs neither git nor a clone:

# 1. release tarball (28 kB, no git involved)
dsh plugin --profile web add https://github.com/guzhou079-arch/dsh-plugin-android-vscreen/releases/download/v0.1.0/dsh-plugin-android-vscreen-0.1.0.tgz

# 2. straight from the repository
dsh plugin --profile web add github:guzhou079-arch/dsh-plugin-android-vscreen

# 3. from a clone you already have
git clone https://github.com/guzhou079-arch/dsh-plugin-android-vscreen
dsh plugin --profile web add link:./dsh-plugin-android-vscreen

The tarball is dsh-plugin-android-vscreen-0.1.0.tgz, 28227 bytes, SHA-256 915369c3b42a3834636ee22e1dd0e8858fff48358c28ccb0590f9371988953f4.

Where github.com will not open (mainland China), prefix the URL with a mirror — https://ghproxy.net/, https://ghfast.top/, https://gh-proxy.com/ — or use spec 3 against a clone fetched through one.

Then ask the agent to run android_vscreen_doctor: it reports which transport is available and probes the device.

Requirements

• An Android device reachable by adb (USB or adb connect), or DSH running on the phone with its privileged shell bridge available.
• Node.js 20+ (whatever your DSH runs on).

Development

The device side is a small Java program in assets/vscreen-core.jar. To rebuild it from source, see the Android port repository (android-app/src/com/deepseek/harness/vscreen/). Two smoke tests run the whole chain without going through a model:

node test-m1.mjs            # create → launch → see → tap → see → main-screen proof → close
node vs-safe.mjs start      # same thing, but aimed at a harmless target

License

LGPL-3.0-only — see LICENSE and THIRD_PARTY_NOTICES.md. The bundled assets/vscreen-core.jar descends from Operit (LGPL-3.0), which is why this plugin is LGPL rather than MIT.