# ChatGPT-Computer-MCP — Implementation Plan

**Target:** v0.2.0
**Authoritative design:** [`SPEC.md`](./SPEC.md)
**Protocol:** MCP 2026-07-28

## Phase 0 — Project foundation

- create a completely separate `chatgpt-computer-mcp` repository;
- TypeScript, ESM, Node.js 22+;
- pin pnpm 11;
- add strict TypeScript config, build/test/gate scripts;
- add MIT license, `.gitignore`, example config, CI gate.

Exit: clean install, typecheck, test and build commands exist.

## Phase 1 — Host-neutral desktop model

Implement typed models for:

- display metadata;
- virtual desktop bounds;
- desktop/display coordinate points and regions;
- observations;
- the nine explicit action variants;
- structured backend errors.

Implement coordinate conversion/bounds validation independently from X11 command execution.

Exit: coordinate behavior is covered by unit tests.

## Phase 2 — Backend abstraction

Define `ComputerBackend` with only:

- `info()`;
- `observe(request)`;
- `act(action)`.

Add backend selection/detection. `auto` chooses native Wayland for a Wayland session and X11 otherwise. Explicit `x11` remains available for intentional X11/XWayland use; explicit `wayland` requires the portal/PipeWire/libei backend.

Exit: MCP layer imports no X11 helper commands.

## Phase 3 — X11 implementation

Implement:

- `xrandr --listmonitors` topology parsing;
- FFmpeg x11grab PNG capture for desktop/display/region targets;
- pointer move;
- click/double-click;
- vertical/horizontal scroll;
- drag;
- mouseDown/mouseUp;
- key chords;
- literal text input;
- action/text/scroll limits;
- bounded subprocess execution with `shell: false`.

Exit: command construction and topology parsing are tested with a fake runner; a real `computer.info`/observe smoke check can run on an X11 workstation without mutating the desktop.

## Phase 4 — MCP tool layer

Register exactly:

- `computer.info`;
- `computer.observe`;
- `computer.act`.

Use SDK v2 `serveStdio` and reject legacy protocol openings. Return screenshots as MCP image content plus compact structured metadata.

Exit: in-memory MCP tests prove tool discovery and call behavior; a protocol test pins MCP `2026-07-28` over stdio-compatible server construction.

## Phase 5 — Installation and operations

Add:

- `install.sh`;
- `scripts/status.sh`;
- `scripts/doctor.sh`;
- `scripts/uninstall.sh`;
- Secure MCP Tunnel guide and optional installer setup path;
- ChatGPT Developer Mode/plugin instructions using name `ChatGPT-Computer-MCP`;
- local config/secrets protections.

Exit: scripts are syntax-checked and documentation contains both automated and manual stdio/tunnel flows.

## Phase 6 — CI and release verification

Run in order:

1. `pnpm install`;
2. `pnpm typecheck`;
3. `pnpm test`;
4. `pnpm build`;
5. `pnpm gate`;
6. tracked-file secret/local-artifact scan;
7. initialize/commit `main`;
8. create `alexcodeplace/chatgpt-computer-mcp` with local `gh` CLI as private;
9. push `main`;
10. verify owner, visibility, default branch, version, license, clean working tree, remote commit, and CI/gate status.

## Phase 7 — native Wayland backend (v0.2)

Implemented as a separate backend using:

- xdg-desktop-portal `RemoteDesktop` + `ScreenCast`;
- authorized PipeWire monitor frames;
- `RemoteDesktop.ConnectToEIS` + libei pointer/keyboard input;
- `mapping_id` synchronization between streams and libei absolute regions;
- logical input coordinates with explicit PipeWire pixel dimensions/scale;
- cursor metadata composition;
- UTF-8 text capability with XKB fallback;
- one persistent portal session with clean revocation/shutdown behavior;
- native helper build, CI dependencies, installer integration, doctor checks, and backend tests.

Exit: native helper compiles with warnings-as-errors, TypeScript tests cover Wayland selection/coordinate/action behavior, and the full repository gate passes. A compositor-level Wayland smoke test remains an operational validation because the development workstation can be running X11 while building the release.