diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index b234fad2..83804596 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -25,7 +25,7 @@ Thanks for the pull request! CONTRIBUTING.md says what a change needs. Write in ## Checklist · 檢查清單 -- [ ] `npm run typecheck`, `npm test` and `npm run build` pass · 都通過 +- [ ] `npm run format:check`, `npm run typecheck`, `npm test` and `npm run build` pass · 都通過 - [ ] `npm run test:e2e` passes, if I changed the UI, the HTTP API, the orchestrator or an adapter · 修改了介面、HTTP API、協調器或轉接器時,端對端測試通過 - [ ] `npm run desktop:typecheck` passes and I tried `npm run desktop`, if I changed `desktop/` · 修改了 `desktop/` 時,型別檢查通過,也用 `npm run desktop` 試過 - [ ] User-facing text and docs come in both English and 繁體中文, or I said which one I need help with · 給使用者看的文字與文件都有中英文,或我已說明需要協助的語言 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6c5c23ef..940d6be3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -65,6 +65,9 @@ jobs: - name: Install run: npm ci + - name: Check the formatting + run: npm run format:check + - name: Typecheck run: npm run typecheck diff --git a/.prettierignore b/.prettierignore new file mode 100644 index 00000000..ba21bef5 --- /dev/null +++ b/.prettierignore @@ -0,0 +1,17 @@ +# What Prettier leaves alone (it also skips what .gitignore lists, and node_modules). +# The desktop app, a package of its own, has its own list: desktop/.prettierignore. + +# Docs: written by hand, their tables and line breaks as they are. +*.md + +# Written by npm, or by a program of ours: never by hand. +package-lock.json +src/web/public/manifest.webmanifest +src/web/public/icons/ + +# Recorded from the real CLIs: replayed byte for byte. +test/fixtures/ + +# The landing page's HTML: its whitespace and JSON-LD as written (site/DESIGN-GUIDELINES.md), and +# the search engines' verification files, which must stay byte for byte. +site/**/*.html diff --git a/.prettierrc.json b/.prettierrc.json new file mode 100644 index 00000000..7b779e88 --- /dev/null +++ b/.prettierrc.json @@ -0,0 +1,6 @@ +{ + "$schema": "https://json.schemastore.org/prettierrc", + "printWidth": 140, + "singleQuote": true, + "embeddedLanguageFormatting": "off" +} diff --git a/AGENTS.md b/AGENTS.md index ae5dc53a..5f8e46d8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,6 +24,7 @@ Read these before a larger change; they are kept up to date and go deeper than t | `npm install` | Install dependencies | | `npm run dev` | API server on :8789 (`tsx watch`) + Vite on :5173 (proxies `/api/`), off `npm start`'s and the desktop app's 8787 (`src/server/ports.ts`); not beside an `npm start` of the same checkout, which holds its `data/` | | `npm run typecheck` | `tsc --noEmit` over `src`, `test`, `scripts`, `e2e` and the config files | +| `npm run format` | Format the code with Prettier (`.prettierrc.json`): TypeScript, JavaScript, JSON, CSS and YAML, the desktop app's included; `.prettierignore` lists what it leaves alone (the docs, recorded fixtures, generated files, the landing page's HTML). `npm run format:check` checks without changing anything, as CI does; `npm --prefix desktop run format` formats `desktop/` alone, with the same settings | | `npm test` | All Vitest tests (offline) | | `npx vitest test/store.test.ts` | One test file | | `npm run test:e2e` | End-to-end tests (Playwright): the built UI in Chromium against the real server, with fake CLIs (`e2e/`); builds the UI first when it is out of date, also a development build of it (`dist/web-dev`, where React's StrictMode runs each effect twice) | @@ -33,6 +34,7 @@ Read these before a larger change; they are kept up to date and go deeper than t | `npm run build` | Build the UI into `dist/web` | | `npm start` | One process: API + built UI on :8787; prints a one-time sign-in link | | `npm run login` | Print another one-time sign-in link (`scripts/login.ts`) | +| `npx tsx scripts/mcp-headers-check.ts ` | Whether a CLI, as it is signed in here, keeps an HTTP MCP server's headers in memory only: gives it a stand-in server with a marker header in a session (ACP, or Codex's thread config), then lists the files it wrote that hold it. Run it before adding a CLI to `GATEWAY_OVER_HTTP` (`src/server/gateway.ts`), and after a CLI's update | | `npx tsx scripts/sandbox-check.ts` | Run Harness Relay's own sandbox for real on this computer (macOS, or Linux with bubblewrap, socat and ripgrep) and check what it allows; `--only ` runs the checks whose name contains it, `--keep` keeps its scratch folder. Run it after changing `relaySandbox.ts`, `relaySandboxHost.ts`, `sandboxProcs.ts` or an adapter's sandbox needs | | `npm run site:preview` | Serve the landing page (`site/`) on 127.0.0.1:4180, as GitHub Pages will | | `npm run icons` | Redraw the home-screen icons and the web app manifest in `src/web/public/` (`scripts/app-icons.ts`) | @@ -47,11 +49,11 @@ Read these before a larger change; they are kept up to date and go deeper than t | `npm run stats` | For maintainers: the releases' downloads by system, new installs apart from the app's updates (the copies' repository), GitHub's counts and about how many downloads they are (the new installs halved, two requests GitHub counts for each; an update counts once), in all and release by release (release candidates left out), and how many codespaces the README's and website's links made, in all and on each of the last seven days in Taiwan (`scripts/stats.ts`, with your own `gh` and `wrangler` sign-ins; `-- --json` for JSON) | | `npm run demo` | Record the README and landing-page demo again with the real CLIs, then update the videos, stills, chapter rails and the video's structured data (`scripts/demo/`; see the developer guide, "Recording the demo") | -Before you commit, run `npm run typecheck`, `npm test` and `npm run build` (and `npm run desktop:typecheck` when you changed `desktop/`, and `npm run test:e2e` when you changed the UI, the HTTP API, the orchestrator or an adapter; and `npm run test:e2e:live` on each system you can when you changed an adapter, the sandboxes or the model backends and have a model server). CI ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)) runs the same three on Node 22 and 24 (and on Windows with Node 24) for every pull request and every push to `main`, and also typechecks the desktop app, builds it and runs its bundled server as the app does (`desktop/scripts/smoke.ts`), and packages it for Linux and Windows and checks it (`desktop/scripts/check-package.ts`). [`.github/workflows/e2e.yml`](.github/workflows/e2e.yml) runs the end-to-end tests on Linux, macOS and Windows for the same events. On a pull request, the tests on Windows, the packaged app and the end-to-end tests run only when it touches a file they depend on (`scripts/ci-scope.ts`, whose `IGNORES` list for each what it does not depend on); a push to `main` runs them all: when one of them comes to read, import or ship a file on its list, take the file off the list in the same commit. Each workflow's last check (`CI passed`, `End-to-end passed`) stands for all its jobs. Releases are made only by the Release workflow ([`.github/workflows/release.yml`](.github/workflows/release.yml)), run by hand: see the developer guide, "Releasing". Never make a release or a version tag any other way. +Before you commit, run `npm run format`, `npm run typecheck`, `npm test` and `npm run build` (and `npm run desktop:typecheck` when you changed `desktop/`, and `npm run test:e2e` when you changed the UI, the HTTP API, the orchestrator or an adapter; and `npm run test:e2e:live` on each system you can when you changed an adapter, the sandboxes or the model backends and have a model server). CI ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)) checks the formatting (`npm run format:check`) and runs the same three on Node 22 and 24 (and on Windows with Node 24) for every pull request and every push to `main`, and also typechecks the desktop app, builds it and runs its bundled server as the app does (`desktop/scripts/smoke.ts`), and packages it for Linux and Windows and checks it (`desktop/scripts/check-package.ts`). [`.github/workflows/e2e.yml`](.github/workflows/e2e.yml) runs the end-to-end tests on Linux, macOS and Windows for the same events. On a pull request, the tests on Windows, the packaged app and the end-to-end tests run only when it touches a file they depend on (`scripts/ci-scope.ts`, whose `IGNORES` list for each what it does not depend on); a push to `main` runs them all: when one of them comes to read, import or ship a file on its list, take the file off the list in the same commit. Each workflow's last check (`CI passed`, `End-to-end passed`) stands for all its jobs. Releases are made only by the Release workflow ([`.github/workflows/release.yml`](.github/workflows/release.yml)), run by hand: see the developer guide, "Releasing". Never make a release or a version tag any other way. ## Where things are -- `src/shared/`: code shared by server and client. `types.ts` (all shared types, `PROVIDERS`), `mentions.ts` (the `@mention` rule, and `unknownMentions`), `links.ts` (where a bare URL ends, also before Chinese text written right after it), `ask.ts` (`[ASK_USER]`), `media.ts`, `schedule.ts` (a schedule's next times), `notify.ts` (when the user is told that agents finished or need them, and what the notification says), `search.ts`, `family.ts` (families of branched conversations: each one's parent and top, for the sidebar's tree and for moving a family between groups as one), `review.ts` (lead review's shared parts: the words a review quotes found as the chat shows them, where a review starts reading, what a review says for search), `share.ts` (a shared page's placeholder links to the files that go along, in their own gist). +- `src/shared/`: code shared by server and client. `types.ts` (all shared types, `PROVIDERS`), `mentions.ts` (the `@mention` rule, and `unknownMentions`), `links.ts` (where a bare URL ends, also before Chinese text written right after it), `ask.ts` (`[ASK_USER]`), `media.ts`, `schedule.ts` (a schedule's next times), `tracking.ts` (a tracked goal: a reply's `` block, its notes, the waits before a retry), `notify.ts` (when the user is told that agents finished or need them, and what the notification says), `search.ts`, `family.ts` (families of branched conversations: each one's parent and top, for the sidebar's tree and for moving a family between groups as one), `review.ts` (lead review's shared parts: the words a review quotes found as the chat shows them, where a review starts reading, what a review says for search), `share.ts` (a shared page's placeholder links to the files that go along, in their own gist). - `src/server/app.ts`: routes, security middleware, SSE (`createApp()`, testable with `app.request()`). - `src/server/orchestrator.ts`: team runs: speakers by mode, prompt delivery, streaming into messages, auto team, the message queue, pausing for the user's answer, retry and resend, and the lead agent's review once the team has answered. - `src/server/prompt.ts` and `planner.ts`: agent briefings and turn prompts; the auto-team lead agent's prompt and reply parsing; `roleGuides.ts`: how each model family takes a role best (from its makers' own prompting guides), which the lead's prompts give it, only for the families its CLIs can run, where it writes roles. `reviewer.ts`: lead review, the lead agent reviewing the team's replies once it has answered (what it reads, its prompt, its answer checked; `src/shared/review.ts` finds the words it quotes, for the server and the chat, which marks them, `src/web/review.ts`). @@ -76,6 +78,7 @@ Before you commit, run `npm run typecheck`, `npm test` and `npm run build` (and - `src/server/cliInstall.ts`: installing a CLI from its card in Settings, each into a folder of its own (`clis/` in the data folder, `cliFolder()` in `protect.ts`): with npm (`npm install -g --prefix`; for Claude Code, Codex and Copilot without npm on the PATH, Node.js's own build put in the folder first, its checksum checked, laid out as npm's global folder is beside its Node and kept there: `NODE_VERSION`, `nodeBuildFor()`, `folderNpm()`), its maker's installer script (downloaded with curl, run with that folder as its home: bash, or on Windows its PowerShell installer or Git for Windows' bash) or uv (put in the folder first when missing) (`CLI_RECIPES`); what an installer left outside the folder that leads into it (links) is taken out, and on Windows the user's PATH is put back as it was before each script (one at a time, `keepingUserPath()`, `recover()` at startup), in the background, checked to run (`--version`), then the CLI's executable path set to it; removing deletes only a folder with its note (`harness-relay-install.json`) and sets the path back. The card's part is `src/web/components/CliInstall.tsx`, its pure logic `src/web/clis.ts`. - `src/server/sandboxSetup.ts`: what the agents' OS sandboxes need that this computer lacks, and installing it from Settings → Sandboxes and from the dialog New chat opens (once per browser, `src/web/sandboxSetup.ts`): on Linux the packages (bubblewrap, socat, ripgrep; slirp4netns, iproute2, iptables, util-linux for Copilot's) with apt, dnf, pacman or zypper, and on Ubuntu 23.10 or later the AppArmor profile for `/usr/bin/bwrap`, in one run of a fixed script with administrator rights (root, `sudo -n`, or pkexec's prompt in a desktop session; else the command to run by hand); in the desktop app on macOS and Linux, Node.js's own build for Harness Relay's own sandbox's host (`sandboxNodeFolder()` in `relaySandbox.ts`, `clis/sandbox-node`); nothing on Windows. - `src/server/tools.ts`: the tool library: MCP servers (their secret values in the OS keyring, by server id), skills and memory, and what each agent gets from it. +- `src/server/gateway.ts`, `gatewayShim.ts`, `outbox.ts`: Harness Relay's own tools (browser, mail, calendar), for every CLI (Grok, Muse Code, Goose, Cline, OpenHands and Pi in full access only, `GATEWAY_FULL_ACCESS_ONLY`) through one MCP server, `relay`: OpenCode, Codex and Copilot connect to its loopback HTTP endpoint with the turn's token in a header (`GATEWAY_OVER_HTTP`); the program the other CLIs start (`gatewayShim.ts`, a copy of it written for each turn beside its token in `data/tmp`) passes each call over a local socket with its turn's token; a call's tier (`read` runs, `act` is asked about below full access, `send` waits in the outbox until the user's Send); the outbox (`outbox` table). `src/server/browser/`: the browser tools, a headless Chromium of Harness Relay's own per conversation, driven over its debugging pipe (no port), every connection through a proxy of its own (`proxy.ts`, loopback only, a password per browser) that connects only to addresses it checked, opening only http and https and asking about each new host below full access. `src/server/connectors/`: the mail and calendar tools (IMAP, SMTP, MIME and CalDAV of Harness Relay's own, no library), for the accounts in Settings, passwords in the credential store. - `src/server/native.ts`, `grokTrust.ts`: what each CLI loads by itself (asked without a prompt; Cursor's read from the files it reads, as are Antigravity's MCP servers and instruction files and Copilot's hooks and custom agents, and Muse Code's MCP servers and hooks from its settings); recording Grok's trust in a project root. - `src/server/usage.ts`: each account's plan usage, asked of its CLI without a prompt in that account's home (Claude Code's `get_usage` control request, Codex's `account/read` and `account/rateLimits/read`, Grok's `_x.ai/billing` over ACP, `muse usage --json`, Copilot's `account.getQuota` on its headless server `copilot --headless --stdio` (`Content-Length` framing), Antigravity's `/usage` and `/credits` in print mode; Cursor's plan only, from `about`), kept a while by `ProviderService.usage()` and shown in each account's box in Settings (`components/UsagePanel.tsx`, its pure logic in `src/web/usage.ts`); none, without asking, for an account on a key its provider bills by use (`BILLED_KEY_ENV`). - `src/server/files.ts`: browsing a chat's working directory and agents' worktrees (inside the folder only, never `.git`), and what a folder's ZIP download holds; `zip.ts` writes that ZIP file as it is sent. @@ -99,6 +102,7 @@ Before you commit, run `npm run typecheck`, `npm test` and `npm run build` (and ### Code - Strict TypeScript, ES modules. Shared types live only in `src/shared/types.ts`. +- Prettier formats the code (`npm run format`, before you commit; CI checks it). Leave `.prettierrc.json` as it is: a change there reformats the whole repository, so it is a commit of its own, with nothing else in it. Markdown is not formatted: keep the docs' tables and line breaks as they are. - Spawn child processes only through `src/server/process.ts`, so they are tracked and killed on shutdown; a quick probe the server waits for (`lsof`, `ps`, `chflags`) goes through its `runSync()`, which kills it after its timeout. The relay sandbox's host (`relaySandboxHost.ts`) is a program of its own and starts the agent itself. Start programs directly, never through a shell; a shell script (`runShellScript()`, or `sh -c` given to `spawnDetached()`) is fixed text, and the values it needs (paths, names, settings) go in its environment, never in its text or its arguments. A program that gives what it runs an environment of its own (`sudo`, `pkexec`) runs `env NAME=value … /bin/sh -c `, the values Harness Relay's own (`sandboxSetup.ts`). On Windows a CLI found as a `.cmd` (npm's shims, Cursor's and Muse Code's launchers) is started as the program it runs (`src/server/winLaunch.ts`), never through `cmd.exe`. - Keep CLI output parsers pure (lines in, `AgentEvent`s out) so they can be tested with recorded output. - Tests run in a Node environment without a DOM. Put client logic you want to test in plain modules (like `src/web/chat.ts` and `src/web/copy.ts`), not inside components. @@ -112,6 +116,9 @@ The server can start agents with write access, so keep its defences intact (see - Never send secrets to the client (`SettingsView` returns masked keys only). API keys belong in the OS credential store (`secrets.ts`), never in the database as they are: without a credential store they go to the database encrypted, under the key in `secrets.key` (`openSecretStore()`), through the same `SecretStore`. - The server binds to `127.0.0.1`. Agent replies cannot make the browser load remote content (images are links; local media goes through the media endpoint's path checks). Keep it that way. - An open agent's process starts only through `startOpenProcess()` (`src/server/agents/open.ts`): below full access it runs in the relay sandbox, which fails closed. Never run one around it. Model keys go to the CLI in environment variables (only the run's own backend's), never into the config files its adapter writes; everything a run writes goes in the conversation member's own state folder (`home(root, state)`, `/private/`); the files that decide what the CLI does when it starts are its `configPaths`, read-only in the sandbox. A sandbox opening (`localPorts`, `connectPorts`, `allowPty`, `allowUnixSocketsInTmp`, `allowTlsTrustd`, `quietlyDeniedHosts`) is asked for by one adapter for its own CLI only, and documented in `docs/open-agents.md` and `docs/sandboxing.md`. +- Harness Relay's own tools (`src/server/gateway.ts`) reach the server only over the gateway's socket, with a token made for one turn and kept in a file under `data/tmp`, or over its loopback HTTP endpoint (`GATEWAY_OVER_HTTP`), with the token only in the CLI's memory (given over its stdin, in its session's MCP servers), never through the HTTP API, an environment value or a command line. A CLI joins `GATEWAY_OVER_HTTP` only once `scripts/mcp-headers-check.ts` shows that it writes an MCP server's headers to no file. Their credentials stay in the server. A call that reaches other people (tier `send`) never runs in the turn: it goes to the outbox (`src/server/outbox.ts`), and only the user's Send (`POST /api/conversations/:id/outbox/:itemId`) sends it, once. What a tool returns from elsewhere (a page, a mail) is given to agents as data, never instructions. +- The browser tools' Chromium (`src/server/browser/session.ts`) connects only through its own proxy (`proxy.ts`), which checks the address each connection goes to: never start it without `--proxy-server` and `--proxy-bypass-list=<-loopback>`, nor let anything it loads reach the network another way. Its page scripts run in an isolated world, and its environment is `launchEnv()`'s (`browserEnv()`'s, and on Linux its config and cache folders in its profile, where its crash reports go), never the server's, nor may it write in the user's own browsers' folders. +- A schedule's webhook (`POST /api/hooks/:id`) is the one route that takes its own token instead of a browser's sign-in; keep its token hashed (`schedules.hook_hash`, compared in constant time) and its body an `` block of data. ### Database diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 97b9bf34..c46f7964 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -55,6 +55,7 @@ Fork the repository (or create a branch if you have write access), branch off `m ### Code - Strict TypeScript with ES modules. Shared types live only in `src/shared/types.ts`. +- [Prettier](https://prettier.io) formats the code: run `npm run format` before you commit (your editor's Prettier extension uses the same settings, `.prettierrc.json`). It leaves the docs alone (`.prettierignore`). Don't change its settings in a pull request about something else: that reformats the whole repository. - Start child processes only through `src/server/process.ts`, so they are tracked and stopped on shutdown; a quick probe the server waits for (`lsof`, `ps`, `chflags`) goes through its `runSync()`, which kills it after its timeout. The relay sandbox's host (`relaySandboxHost.ts`) is a program of its own and starts the agent itself. Start programs directly, never through a shell; a shell script (`runShellScript()`, or `sh -c` given to `spawnDetached()`) is fixed text, and the values it needs (paths, names, settings) go in its environment, never in its text or its arguments. A program that gives what it runs an environment of its own (`sudo`, `pkexec`) runs `env NAME=value … /bin/sh -c `, the values Harness Relay's own (`sandboxSetup.ts`). On Windows a CLI found as a `.cmd` (npm's shims, Cursor's and Muse Code's launchers) is started as the program it runs (`src/server/winLaunch.ts`), never through `cmd.exe`. - Keep CLI output parsers pure (lines in, `AgentEvent`s out) so they can be tested with recorded output. - Tests run in Node without a DOM. Put client logic you want to test in plain modules (like `src/web/chat.ts`), not inside components. @@ -67,6 +68,8 @@ Fork the repository (or create a branch if you have write access), branch off `m - Never send secrets to the client. API keys belong in the OS credential store (`src/server/secrets.ts`), never in the database as they are: without a credential store they go to the database encrypted (`openSecretStore()`), through the same `SecretStore`. - The server binds to `127.0.0.1`, and agent replies must not make the browser load remote content (images are shown as links; local media goes through the media endpoint's path checks). - An open-source agent's process starts only through `startOpenProcess()` (`src/server/agents/open.ts`), which puts it in Harness Relay's own sandbox below full access and fails closed. Model keys reach the CLI in environment variables (only the run's own backend's), never in the config files its adapter writes, and everything a run writes goes in the conversation member's own state folder (`home(root, state)`). A sandbox opening is asked for by one adapter for its own CLI only, and documented in [docs/open-agents.md](docs/open-agents.md) and [docs/sandboxing.md](docs/sandboxing.md). +- Harness Relay's own tools (`src/server/gateway.ts`) reach the server only over the gateway's socket, with a token for one turn kept in a file under `data/tmp`, or over its loopback HTTP endpoint, with the token only in the CLI's memory (a CLI gets it so only once `scripts/mcp-headers-check.ts` shows it writes the token to no file), never through the HTTP API. Their credentials stay in the server, and a call that reaches other people never runs in the turn: it waits in the outbox until the user's Send. A schedule's webhook is the one route that takes its own token (kept hashed) instead of a browser's sign-in. +- The browser tools' Chromium connects only through its own proxy (`src/server/browser/proxy.ts`), which checks the address each connection goes to: never start it without that proxy, nor let anything it loads reach the network another way. Its page scripts run in an isolated world, and it gets only the environment `launchEnv()` gives, never the server's; it writes nothing in the user's own browsers' folders (on Linux its config and cache folders are in its profile). ### Database @@ -109,6 +112,7 @@ Tests also have to pass on Windows, where CI runs them too: build paths with `jo Run the same checks as CI: ```bash +npm run format:check # npm run format fixes what it finds npm run typecheck npm test npm run build @@ -120,7 +124,7 @@ If you changed the desktop app, also run `npm run desktop:typecheck`, and try it Releases are made by maintainers with the Release workflow only (**Actions → Release**, from `main`, today's date as the version): never tag a version by hand. See [Releasing](docs/developer-guide.md#releasing) in the developer guide. -[CI](.github/workflows/ci.yml) runs the typecheck, the tests and the build on Node 22 and 24 for every pull request and every push to `main`, and on Windows with Node 24. It also typechecks and builds the desktop app, runs its bundled server as the app does, and packages it for Linux to check it as a release is checked. [Another workflow](.github/workflows/e2e.yml) runs the end-to-end tests on Linux, macOS and Windows. On a pull request, the tests on Windows, the packaged app and the end-to-end tests run only when it touches a file they depend on ([`scripts/ci-scope.ts`](scripts/ci-scope.ts)): the run's summary says which ran and why. A push to `main` runs them all. If you make one of them read, import or ship a file its `IGNORES` list names, take the file off the list. Each workflow's last check (**CI passed**, **End-to-end passed**) stands for all its jobs. +[CI](.github/workflows/ci.yml) checks the formatting, and runs the typecheck, the tests and the build on Node 22 and 24 for every pull request and every push to `main`, and on Windows with Node 24. It also typechecks and builds the desktop app, runs its bundled server as the app does, and packages it for Linux to check it as a release is checked. [Another workflow](.github/workflows/e2e.yml) runs the end-to-end tests on Linux, macOS and Windows. On a pull request, the tests on Windows, the packaged app and the end-to-end tests run only when it touches a file they depend on ([`scripts/ci-scope.ts`](scripts/ci-scope.ts)): the run's summary says which ran and why. A push to `main` runs them all. If you make one of them read, import or ship a file its `IGNORES` list names, take the file off the list. Each workflow's last check (**CI passed**, **End-to-end passed**) stands for all its jobs. If your change affects what agents see or do, also try it in the app with at least one real CLI, and say which ones you used. diff --git a/CONTRIBUTING.zh-TW.md b/CONTRIBUTING.zh-TW.md index c6f76f08..18ae5f09 100644 --- a/CONTRIBUTING.zh-TW.md +++ b/CONTRIBUTING.zh-TW.md @@ -55,6 +55,7 @@ npm run dev # API 在 :8789(檔案變更時重啟)+ 介面在 http:// ### 程式碼 - 使用 strict TypeScript 與 ES modules。共用型別只放在 `src/shared/types.ts`。 +- 程式碼由 [Prettier](https://prettier.io) 排版:commit 前執行 `npm run format`(編輯器的 Prettier 擴充套件也用同一份設定 `.prettierrc.json`)。它不動文件(`.prettierignore`)。不要在處理其他事情的 pull request 裡改它的設定:那會重新排版整個 repository。 - 子程序一律透過 `src/server/process.ts` 啟動,這樣才會被追蹤並在關閉時結束;伺服器等它結束的簡短檢查(`lsof`、`ps`、`chflags`)用它的 `runSync()`,逾時就結束它。Harness Relay 沙箱的 host(`relaySandboxHost.ts`)是獨立的程式,由它自己啟動 agent。程式要直接啟動,不經過 shell;shell 腳本(`runShellScript()`,或交給 `spawnDetached()` 的 `sh -c`)是固定的文字,它需要的值(路徑、名稱、設定)放在它的環境變數裡,絕不放進腳本文字或參數。會替要執行的程式另設環境的程式(`sudo`、`pkexec`)則執行 `env 名稱=值 … /bin/sh -c <固定文字>`,值都是 Harness Relay 自己的(`sandboxSetup.ts`)。在 Windows 上,找到的 CLI 若是 `.cmd`(npm 的包裝檔、Cursor 與 Muse Code 的啟動檔),會改為啟動它執行的程式(`src/server/winLaunch.ts`),絕不經過 `cmd.exe`。 - CLI 輸出的 parser 保持純函式(輸入一行行文字,輸出 `AgentEvent`),方便用錄下來的輸出測試。 - 測試在沒有 DOM 的 Node 環境執行。想測試的前端邏輯請放在一般模組(例如 `src/web/chat.ts`),不要寫在元件裡。 @@ -67,6 +68,8 @@ npm run dev # API 在 :8789(檔案變更時重啟)+ 介面在 http:// - 絕不把機密傳給前端。API key 放在作業系統的金鑰儲存區(`src/server/secrets.ts`),絕不以原文放進資料庫:沒有金鑰儲存區時,透過同一個 `SecretStore` 加密後存進資料庫(`openSecretStore()`)。 - 伺服器只綁定 `127.0.0.1`,而且 agent 的回覆不能讓瀏覽器載入遠端內容(圖片以連結顯示;本機媒體要經過媒體端點的路徑檢查)。 - 開源 agent 的程序只能透過 `startOpenProcess()`(`src/server/agents/open.ts`)啟動:完全存取以外,它會把程序放進 Harness Relay 自己的沙箱,沙箱無法啟動就不執行。模型的金鑰以環境變數交給 CLI(只有這次執行的模型來源那一把),絕不寫進轉接器寫的設定檔;一次執行寫入的一切都放在對話成員自己的狀態資料夾(`home(root, state)`)。沙箱的開口只由一個轉接器為它自己的 CLI 申請,並寫在 [docs/open-agents.zh-TW.md](docs/open-agents.zh-TW.md) 與 [docs/sandboxing.zh-TW.md](docs/sandboxing.zh-TW.md)。 +- Harness Relay 自己的工具(`src/server/gateway.ts`)只能透過 gateway 的 socket 連到伺服器,用只給一回合、存在 `data/tmp` 底下檔案中的權杖;或透過它的本機 HTTP 端點,權杖只留在 CLI 的記憶體中(要等 `scripts/mcp-headers-check.ts` 確認某個 CLI 不會把權杖寫進任何檔案,它才能這樣拿到),絕不經過 HTTP API。它們的憑證留在伺服器中,會送到別人那裡的呼叫絕不在那一回合執行:它會在待送出匣等到使用者按送出。排程的 webhook 是唯一用自己的權杖(只存雜湊)代替瀏覽器登入的路由。 +- 瀏覽器工具的 Chromium 只透過它自己的 proxy(`src/server/browser/proxy.ts`)連線,proxy 會檢查每個連線要去的位址:絕不能不帶那個 proxy 啟動它,也不能讓它載入的任何東西用別的方式連上網路。它在網頁中執行的腳本在 isolated world 中執行,它只拿到 `launchEnv()` 給的環境變數,絕不是伺服器的;它也不在使用者自己瀏覽器的資料夾裡寫任何東西(在 Linux 上,它的設定與快取資料夾都在它的設定檔裡)。 ### 資料庫 @@ -109,6 +112,7 @@ Harness Relay 是雙語的:每一段給使用者看的文字都有繁體中文 請執行和 CI 一樣的檢查: ```bash +npm run format:check # npm run format 會修正它找到的地方 npm run typecheck npm test npm run build @@ -120,7 +124,7 @@ npm run build 發佈只由維護者透過發佈 workflow 進行(**Actions → Release**,從 `main`,以今天的日期為版本):絕不要手動建立版本 tag。詳見開發者說明的[發佈](docs/developer-guide.zh-TW.md#發佈)。 -[CI](.github/workflows/ci.yml) 會在每個 pull request 與每次推送到 `main` 時,以 Node 22 與 24 執行型別檢查、測試與建置,也在 Windows 上以 Node 24 執行。它也會型別檢查並建置桌面 app、像 app 一樣執行打包好的伺服器,並為 Linux 打包它,像檢查發佈一樣檢查。[另一個 workflow](.github/workflows/e2e.yml) 會在 Linux、macOS 與 Windows 上執行端對端測試。在 pull request 上,Windows 上的測試、打包好的 app 與端對端測試,只在它碰到它們依賴的檔案時才會跑([`scripts/ci-scope.ts`](scripts/ci-scope.ts)),執行摘要會說明哪些跑了、為什麼。推送到 `main` 時全部都會跑。如果你讓其中一個開始讀取、import 或打包它的 `IGNORES` 清單上的檔案,請把那個檔案從清單拿掉。每個 workflow 最後的檢查(**CI passed**、**End-to-end passed**)代表它所有的 job。 +[CI](.github/workflows/ci.yml) 會在每個 pull request 與每次推送到 `main` 時檢查排版,並以 Node 22 與 24 執行型別檢查、測試與建置,也在 Windows 上以 Node 24 執行。它也會型別檢查並建置桌面 app、像 app 一樣執行打包好的伺服器,並為 Linux 打包它,像檢查發佈一樣檢查。[另一個 workflow](.github/workflows/e2e.yml) 會在 Linux、macOS 與 Windows 上執行端對端測試。在 pull request 上,Windows 上的測試、打包好的 app 與端對端測試,只在它碰到它們依賴的檔案時才會跑([`scripts/ci-scope.ts`](scripts/ci-scope.ts)),執行摘要會說明哪些跑了、為什麼。推送到 `main` 時全部都會跑。如果你讓其中一個開始讀取、import 或打包它的 `IGNORES` 清單上的檔案,請把那個檔案從清單拿掉。每個 workflow 最後的檢查(**CI passed**、**End-to-end passed**)代表它所有的 job。 如果你的修改會影響 agent 看到或做的事,也請在 app 中用至少一個真正的 CLI 試過,並說明用了哪些。 diff --git a/README.md b/README.md index d21708a0..c5bf8bff 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,8 @@ For each question, a lead agent picks the team from your templates (or adapts on - **Safe editing with git**: each working directory (up to eight per chat) becomes a git repository, each reply shows the files it changed with a diff, and in parallel mode every agent works in its own worktree. ([Reviewing and applying changes](docs/user-guide.md#reviewing-and-applying-changes)) - **You stay in charge**: a permission per chat (read-only, edit, full access), approvals in the chat for anything beyond it, agents that ask you rather than guess, and messages you can send while they reply. ([Approvals](docs/user-guide.md#approvals), [chatting](docs/user-guide.md#4-chat)) - **Everything live**: replies, reasoning, plans and tool activity stream in; Mermaid diagrams, images and the files agents make show in the chat; copy, edit, retry or branch from any reply. ([Copying, editing and retrying](docs/user-guide.md#copying-editing-and-retrying), [branching](docs/user-guide.md#branching-a-conversation)) -- **Schedules**: send the team a message at set times, such as a review of yesterday's commits every weekday at 9:00. ([Schedules](docs/user-guide.md#schedules)) +- **Harness Relay's own tools**: a browser of its own, mail and a calendar for the agents of every CLI (Grok's, Muse Code's, and the open-source agents' but OpenCode's, in full access only), through one MCP server; what reaches other people waits in an outbox until you send it. ([Harness Relay's tools](docs/user-guide.md#harness-relays-tools-browser-mail-and-calendar)) +- **Automations**: send the team a message when you run one, at set times (a review of yesterday's commits every weekday at 9:00), or when a program on this computer calls its webhook; an automation can track a goal across days until an agent says it is reached, and tries a failed run again. ([Automations](docs/user-guide.md#automations)) - **Files, voice and sharing**: attach files and screenshots, browse the working directory, dictate with voice input that stays on your computer, and share a chat as Markdown or a link. ([Browsing files](docs/user-guide.md#browsing-files), [voice input](docs/user-guide.md#voice-input), [sharing](docs/user-guide.md#sharing-a-conversation)) - **Organised**: groups, pinned chats at the top, search, each chat's status in the sidebar, notifications and sound, as push notifications on a phone, and a bell that keeps every notification, the same in every browser. ([Groups](docs/user-guide.md#groups), [pinned chats](docs/user-guide.md#pinned-conversations), [search](docs/user-guide.md#search), [notifications](docs/user-guide.md#notifications-and-sound), [notification inbox](docs/user-guide.md#notification-inbox)) - **Set up once for every CLI**: a CLI not found installs from its card in one click (and removes again), a tool library of MCP servers, skills and memory, several accounts per CLI, each account's plan usage, what each CLI loads by itself, and the agents' sessions in [AgentsView](https://www.agentsview.io). ([Installing a CLI](docs/user-guide.md#installing-a-cli-from-settings), [tool library](docs/user-guide.md#tool-library-mcp-servers-skills-and-memory), [accounts](docs/user-guide.md#more-than-one-account), [plan usage](docs/user-guide.md#plan-usage), [AgentsView](docs/user-guide.md#agent-sessions-the-clis-own-folder-and-agentsview)) @@ -83,7 +84,7 @@ Agents can change your files and run commands, so Harness Relay keeps them withi | `npm start` | Serve the built UI and the API from one process on http://127.0.0.1:8787 | | `npm run login` | Print another one-time sign-in link, for when the one `npm start` printed is used or old | | `npm run dev` | For development: the API server (port 8789, auto-restart) and Vite (port 5173) | -| `npm run typecheck`, `npm test`, `npm run build` | What CI runs for every pull request | +| `npm run format:check`, `npm run typecheck`, `npm test`, `npm run build` | What CI runs for every pull request (`npm run format` formats the code with Prettier) | The other commands (end-to-end tests, the desktop app, the website, the demo) are in the [developer guide](docs/developer-guide.md#running), with what [CI](docs/developer-guide.md#continuous-integration) runs. diff --git a/README.zh-TW.md b/README.zh-TW.md index f6090828..5b28bb2d 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -24,7 +24,8 @@ - **用 git 安全地改檔**:每個工作目錄(一個對話最多八個)都會成為 git repository,每則回覆都列出它改了哪些檔案並附上 diff,平行模式中每個 agent 在自己的 worktree 工作。([檢視與套用修改](docs/user-guide.zh-TW.md#檢視與套用修改)) - **由你掌控**:每個對話有自己的權限(唯讀、可編輯、完全存取),超出範圍的動作在對話中請你核准;agent 不確定時先問你而不亂猜;團隊回覆時你也能插話。([核准](docs/user-guide.zh-TW.md#核准)、[對話](docs/user-guide.zh-TW.md#4-對話)) - **一切即時呈現**:回覆、推理、計畫與工具動作即時串流;Mermaid 圖表、圖片與 agent 做出的檔案直接顯示在對話中;任何一則回覆都能複製、編輯、重試或分支。([複製、編輯與重試](docs/user-guide.zh-TW.md#複製編輯與重試)、[分支對話](docs/user-guide.zh-TW.md#分支對話)) -- **排程**:在設定的時間把訊息傳給團隊,例如每個平日 9:00 審查昨天的 commit。([排程](docs/user-guide.zh-TW.md#排程)) +- **Harness Relay 自己的工具**:給每個 CLI 的 agent(Grok、Muse Code 與 OpenCode 以外的開源 agent 只在完全存取時)一個它自己的瀏覽器、郵件與行事曆,透過同一個 MCP server;會送到別人那裡的東西在待送出匣等你送出。([Harness Relay 的工具](docs/user-guide.zh-TW.md#harness-relay-的工具瀏覽器郵件與行事曆)) +- **自動任務**:在你執行它、到了設定的時間(例如每個平日 9:00 審查昨天的 commit),或這台電腦上的程式呼叫它的 webhook 時,把訊息傳給團隊;自動任務可以跨天追蹤一個目標,直到 agent 回報已達成,失敗的執行也會重試。([自動任務](docs/user-guide.zh-TW.md#自動任務)) - **檔案、語音與分享**:附加檔案與截圖、瀏覽工作目錄、用只在你電腦上執行的語音輸入說話,並把對話分享成 Markdown 或連結。([瀏覽檔案](docs/user-guide.zh-TW.md#瀏覽檔案)、[語音輸入](docs/user-guide.zh-TW.md#語音輸入)、[分享對話](docs/user-guide.zh-TW.md#分享對話)) - **井然有序**:群組、釘選在最上面的對話、搜尋、側邊欄上每個對話的狀態、通知與提示音(手機上是推播通知),以及收著每一則通知的鈴鐺(每個瀏覽器都一樣)。([群組](docs/user-guide.zh-TW.md#群組)、[釘選](docs/user-guide.zh-TW.md#釘選的對話)、[搜尋](docs/user-guide.zh-TW.md#搜尋)、[通知與提示音](docs/user-guide.zh-TW.md#通知與提示音)、[通知收件匣](docs/user-guide.zh-TW.md#通知收件匣)) - **設定一次,每個 CLI 都適用**:找不到的 CLI 在它的卡片上一鍵安裝(也能再移除),MCP 伺服器、skill 與記憶的工具庫,每個 CLI 多個帳號、每個帳號的方案用量、各 CLI 自己載入的內容,以及在 [AgentsView](https://www.agentsview.io) 中檢視 agent 的 session。([安裝 CLI](docs/user-guide.zh-TW.md#從設定頁安裝-cli)、[工具庫](docs/user-guide.zh-TW.md#工具庫mcp-伺服器skill-與記憶)、[多個帳號](docs/user-guide.zh-TW.md#多個帳號)、[方案用量](docs/user-guide.zh-TW.md#方案用量)、[AgentsView](docs/user-guide.zh-TW.md#agent-的-session各-cli-自己的資料夾與-agentsview)) @@ -83,7 +84,7 @@ agent 會修改你的檔案、執行指令,所以 Harness Relay 讓它們只 | `npm start` | 由同一個程序在 提供建置好的介面與 API | | `npm run login` | 印出另一個一次性的登入連結,用在 `npm start` 印出的那個已經用過或太舊時 | | `npm run dev` | 開發用:API 伺服器(port 8789,自動重啟)與 Vite(port 5173) | -| `npm run typecheck`、`npm test`、`npm run build` | CI 對每個 pull request 執行的檢查 | +| `npm run format:check`、`npm run typecheck`、`npm test`、`npm run build` | CI 對每個 pull request 執行的檢查(`npm run format` 用 Prettier 排版程式碼) | 其他指令(端對端測試、桌面 app、網站、示範影片)都在[開發者說明](docs/developer-guide.zh-TW.md#執行),也說明了 [CI](docs/developer-guide.zh-TW.md#持續整合ci) 會執行什麼。 diff --git a/desktop/.prettierignore b/desktop/.prettierignore new file mode 100644 index 00000000..17bc4e46 --- /dev/null +++ b/desktop/.prettierignore @@ -0,0 +1,6 @@ +# What Prettier leaves alone when it runs in desktop/ (npm --prefix desktop run format): the +# repository's own list (../.prettierignore) is read only from the repository's root. +*.md +package-lock.json +build/ +release/ diff --git a/desktop/README.md b/desktop/README.md index 5ff10661..46cf5b7f 100644 --- a/desktop/README.md +++ b/desktop/README.md @@ -24,6 +24,7 @@ Run them from the repository's root. | `npm run desktop` | Build the UI and the app (`desktop/build/`) and run it without installing it | | `npm run desktop:dist` | The same, then make the installer for this system in `desktop/release/` (`.dmg`, `.exe`, `.AppImage`; not signed for distribution); `-- --output ` puts it elsewhere | | `npm run desktop:typecheck` | Typecheck the app (`desktop/tsconfig.json`) | +| `npm --prefix desktop run format` | Format the app's code with Prettier, with the repository's settings (`.prettierrc.json`; `desktop/.prettierignore`: what it leaves alone); `format:check` checks it. The root's `npm run format` formats it too | | `npx tsx desktop/scripts/smoke.ts` | Run the bundled server the way the app does, with plain Node standing in for Electron, and check what the app needs of it, a question answered by the e2e fake CLIs included | | `npx tsx desktop/scripts/check-package.ts` | Check the packaged app in `desktop/release/` as a release is checked: its fuses, its resources, no `github_pat_` token in the server's files nor as text in the app's, the repository and the token (encoded) that `RELAY_UPDATES_REPO` and `RELAY_UPDATES_TOKEN` name in its own environment (give it the build's), and its own `--smoke-test`, in which the e2e fake CLIs answer a question (on Linux, under `xvfb-run`) | diff --git a/desktop/README.zh-TW.md b/desktop/README.zh-TW.md index 84b305bc..b5c2ef2a 100644 --- a/desktop/README.zh-TW.md +++ b/desktop/README.zh-TW.md @@ -24,6 +24,7 @@ npm install --prefix desktop # Electron、electron-builder 與 esbuild,一 | `npm run desktop` | 建置 UI 與 app(`desktop/build/`),不安裝直接執行 | | `npm run desktop:dist` | 同上,再於 `desktop/release/` 產生這個系統的安裝檔(`.dmg`、`.exe`、`.AppImage`;沒有為發佈簽署);加上 `-- --output <資料夾>` 可放到別處 | | `npm run desktop:typecheck` | 檢查 app 的型別(`desktop/tsconfig.json`) | +| `npm --prefix desktop run format` | 用 Prettier 排版 app 的程式碼,設定和 repository 相同(`.prettierrc.json`;`desktop/.prettierignore`:它不動的檔案);`format:check` 只檢查。根目錄的 `npm run format` 也會排版它 | | `npx tsx desktop/scripts/smoke.ts` | 用 app 的方式執行打包好的伺服器(以一般的 Node 代替 Electron),檢查 app 需要它做到的事,包括由 e2e 的假 CLI 回答一個問題 | | `npx tsx desktop/scripts/check-package.ts` | 照發佈時的方式檢查 `desktop/release/` 裡打包好的 app:它的 fuse、資源、伺服器的檔案裡沒有 `github_pat_` token、app 的檔案裡也沒有 token 的文字、它自己的環境變數 `RELAY_UPDATES_REPO` 與 `RELAY_UPDATES_TOKEN` 指定的 repository 與(編碼後的)token(請給它建置時的值),以及它自己的 `--smoke-test`(其中 e2e 的假 CLI 會回答一個問題;Linux 上要在 `xvfb-run` 下執行) | diff --git a/desktop/package-lock.json b/desktop/package-lock.json index 2779d03a..123de206 100644 --- a/desktop/package-lock.json +++ b/desktop/package-lock.json @@ -10,7 +10,8 @@ "@types/node": "^26.6.6", "electron": "^45.2.3", "electron-builder": "^26.18.0", - "esbuild": "^0.28.2" + "esbuild": "^0.28.2", + "prettier": "3.10.1" } }, "node_modules/@electron-internal/extract-zip": { @@ -3285,6 +3286,22 @@ "node": "^12.20.0 || >=14" } }, + "node_modules/prettier": { + "version": "3.10.1", + "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.10.1.tgz", + "integrity": "sha512-70vp3ZUYoPlFbF9uiTdXHUUCGEIp7d4+6p+8DYbkm2jP7x/3i/paXVF6dopSI3zd/CEitrZN1rc4i85xWmmXlg==", + "dev": true, + "license": "MIT", + "bin": { + "prettier": "bin/prettier.cjs" + }, + "engines": { + "node": ">=14" + }, + "funding": { + "url": "https://github.com/prettier/prettier?sponsor=1" + } + }, "node_modules/proc-log": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/proc-log/-/proc-log-6.1.0.tgz", diff --git a/desktop/package.json b/desktop/package.json index a7cf5a1a..2a6fc876 100644 --- a/desktop/package.json +++ b/desktop/package.json @@ -8,13 +8,16 @@ "build": "tsx scripts/build.ts", "start": "electron build/app", "dist": "tsx scripts/dist.ts", - "typecheck": "tsc -p ." + "typecheck": "tsc -p .", + "format": "prettier --write .", + "format:check": "prettier --check ." }, "devDependencies": { "@types/node": "^26.6.6", "electron": "^45.2.3", "electron-builder": "^26.18.0", - "esbuild": "^0.28.2" + "esbuild": "^0.28.2", + "prettier": "3.10.1" }, "overrides": { "minimatch@>=7.0.0 <11.0.1": "^11.0.6" diff --git a/desktop/scripts/build.ts b/desktop/scripts/build.ts index d0dc8218..2cedb588 100644 --- a/desktop/scripts/build.ts +++ b/desktop/scripts/build.ts @@ -52,7 +52,10 @@ const rootPackage = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf8')) /** The commit being built, when this is a git checkout. */ function commit(): string | undefined { try { - return execFileSync('git', ['rev-parse', '--short', 'HEAD'], { cwd: ROOT, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim() || undefined; + return ( + execFileSync('git', ['rev-parse', '--short', 'HEAD'], { cwd: ROOT, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim() || + undefined + ); } catch { return undefined; } @@ -141,7 +144,11 @@ await build({ external: ['electron'], // import.meta.url: scripts/release.ts (version rules) checks it to run as a script, never true in the app. // Where it downloads its updates from (main.ts), the token included, encoded (updateSource.ts). - define: { __RELAY_VERSION__: JSON.stringify(version.shown), __RELAY_UPDATE_SOURCE__: JSON.stringify(encodeUpdateSource(updates.value)), 'import.meta.url': 'undefined' }, + define: { + __RELAY_VERSION__: JSON.stringify(version.shown), + __RELAY_UPDATE_SOURCE__: JSON.stringify(encodeUpdateSource(updates.value)), + 'import.meta.url': 'undefined', + }, legalComments: 'none', logLevel: 'warning', }); @@ -156,4 +163,6 @@ for (const icon of DESKTOP_ICONS) { } const from = `updates downloaded from ${updates.value.repo}${updates.value.token ? ', with a token' : ''}`; -console.log(`Desktop app ${version.shown} (${version.app}) built in ${OUT} (keyring: ${keyring.filter((n) => n !== 'keyring').join(', ')}; ${from})`); +console.log( + `Desktop app ${version.shown} (${version.app}) built in ${OUT} (keyring: ${keyring.filter((n) => n !== 'keyring').join(', ')}; ${from})`, +); diff --git a/desktop/scripts/check-package.ts b/desktop/scripts/check-package.ts index 985338ac..53fcfb96 100644 --- a/desktop/scripts/check-package.ts +++ b/desktop/scripts/check-package.ts @@ -118,16 +118,27 @@ async function main(): Promise { check(wire[Number(option)] === state, `fuse ${name}: ${state === ON ? 'on' : 'off'}`); } const server = join(app.resources, 'server'); - check(existsSync(join(server, 'server.mjs')) && existsSync(join(server, 'web', 'index.html')), 'the server and the web app are in its resources'); + check( + existsSync(join(server, 'server.mjs')) && existsSync(join(server, 'web', 'index.html')), + 'the server and the web app are in its resources', + ); const napi = join(server, 'node_modules', '@napi-rs'); const keyring = existsSync(napi) ? readdirSync(napi) : []; - check(keyring.includes('keyring') && keyring.some((n) => n.startsWith('keyring-')), `the keyring module is in its resources (${keyring.join(', ') || 'none'})`); + check( + keyring.includes('keyring') && keyring.some((n) => n.startsWith('keyring-')), + `the keyring module is in its resources (${keyring.join(', ') || 'none'})`, + ); if (app.universal) { - check(keyring.includes('keyring-darwin-arm64') && keyring.includes('keyring-darwin-x64'), 'the keyring module has its native part for Apple Silicon and Intel'); + check( + keyring.includes('keyring-darwin-arm64') && keyring.includes('keyring-darwin-x64'), + 'the keyring module has its native part for Apple Silicon and Intel', + ); } const runtime = join(server, 'node_modules', '@anthropic-ai', 'sandbox-runtime'); check( - existsSync(join(server, 'relaySandboxHost.mjs')) && existsSync(join(server, 'piGate.js')) && existsSync(join(runtime, 'package.json')), + existsSync(join(server, 'relaySandboxHost.mjs')) && + existsSync(join(server, 'piGate.js')) && + existsSync(join(runtime, 'package.json')), "the relay sandbox's host, Pi's gate and the sandbox runtime are in its resources", ); if (process.platform === 'linux') { @@ -136,11 +147,19 @@ async function main(): Promise { } const shipped = join(app.resources, 'LICENSE'); check(existsSync(shipped) && readFileSync(shipped).equals(license), "the repository's LICENSE is in its resources"); - check(fromArchive(join(app.resources, 'app.asar'), 'LICENSE')?.equals(license), "the repository's LICENSE is in its archive, beside its package.json"); + check( + fromArchive(join(app.resources, 'app.asar'), 'LICENSE')?.equals(license), + "the repository's LICENSE is in its archive, beside its package.json", + ); // The token for the app's updates, if it was built with one, is its main process's alone (desktop/scripts/build.ts). - const bundles = ['server.mjs', 'relaySandboxHost.mjs', 'piGate.js'].map((name) => join(server, name)).filter((path) => existsSync(path)); + const bundles = ['server.mjs', 'relaySandboxHost.mjs', 'piGate.js'] + .map((name) => join(server, name)) + .filter((path) => existsSync(path)); const token = /github_pat_[A-Za-z0-9_]{20,}/; - check(bundles.length === 3 && bundles.every((path) => !token.test(readFileSync(path, 'latin1'))), 'no file the server runs holds a GitHub token'); + check( + bundles.length === 3 && bundles.every((path) => !token.test(readFileSync(path, 'latin1'))), + 'no file the server runs holds a GitHub token', + ); const main = fromArchive(join(app.resources, 'app.asar'), 'main.cjs')?.toString('latin1'); check(main !== undefined && !token.test(main), "the app's main process holds no GitHub token as text (only encoded)"); // What it was built to download its updates with, as this environment names it. @@ -149,7 +168,7 @@ async function main(): Promise { const { repo, encodedToken } = encodeUpdateSource(source.value); // As the build's define puts it (`repo: "…"`): the default repository's name is in the app's code anyway. check(main?.includes(`repo: ${JSON.stringify(repo)}`), `the app downloads its updates from ${repo}`); - if (encodedToken) check(main?.includes(JSON.stringify(encodedToken)), "the app holds the read token it was built with, encoded"); + if (encodedToken) check(main?.includes(JSON.stringify(encodedToken)), 'the app holds the read token it was built with, encoded'); } else { check(false, source.error); } diff --git a/desktop/scripts/dist.ts b/desktop/scripts/dist.ts index bb1bd593..83322306 100644 --- a/desktop/scripts/dist.ts +++ b/desktop/scripts/dist.ts @@ -32,7 +32,9 @@ async function runningApps(): Promise { return r.stdout .split('\n') .map((line) => line.trim()) - .filter((line) => /Harness Relay\.app\/Contents\/MacOS\/Harness Relay(\s|$)|\/harness-relay(\s|$)/.test(line) && !/--type=/.test(line)); + .filter( + (line) => /Harness Relay\.app\/Contents\/MacOS\/Harness Relay(\s|$)|\/harness-relay(\s|$)/.test(line) && !/--type=/.test(line), + ); } catch { return []; } diff --git a/desktop/scripts/packaged.ts b/desktop/scripts/packaged.ts index afd2f3e9..f19ce75b 100644 --- a/desktop/scripts/packaged.ts +++ b/desktop/scripts/packaged.ts @@ -28,13 +28,22 @@ export function packagedApps(dir: string): PackagedApp[] { const at = join(dir, entry.name); const mac = join(at, 'Harness Relay.app'); if (entry.name.startsWith('mac') && existsSync(mac)) { - apps.push({ fuses: mac, exe: join(mac, 'Contents', 'MacOS', 'Harness Relay'), resources: join(mac, 'Contents', 'Resources'), universal: entry.name === 'mac-universal' }); + apps.push({ + fuses: mac, + exe: join(mac, 'Contents', 'MacOS', 'Harness Relay'), + resources: join(mac, 'Contents', 'Resources'), + universal: entry.name === 'mac-universal', + }); } else if (entry.name.startsWith('win') && existsSync(join(at, 'Harness Relay.exe'))) { - apps.push({ fuses: join(at, 'Harness Relay.exe'), exe: join(at, 'Harness Relay.exe'), resources: join(at, 'resources'), universal: false }); + apps.push({ + fuses: join(at, 'Harness Relay.exe'), + exe: join(at, 'Harness Relay.exe'), + resources: join(at, 'resources'), + universal: false, + }); } else if (entry.name.startsWith('linux') && existsSync(join(at, 'harness-relay'))) { apps.push({ fuses: join(at, 'harness-relay'), exe: join(at, 'harness-relay'), resources: join(at, 'resources'), universal: false }); } } return apps; } - diff --git a/desktop/scripts/smoke.ts b/desktop/scripts/smoke.ts index 98cf88ff..e4b486ff 100644 --- a/desktop/scripts/smoke.ts +++ b/desktop/scripts/smoke.ts @@ -60,7 +60,16 @@ export async function smoke(o: SmokeOptions): Promise { const fakes = o.fakeClis ? withFakeClis(base, o.fakeClis, root) : undefined; const child: ChildProcess = fork(o.entry, [], { execArgv: [...(o.execArgv ?? []), '--import', pathToFileURL(join(DESKTOP, 'scripts/parent-port.mjs')).href], - env: { ...(fakes?.env ?? base), PORT: '0', HOST: '127.0.0.1', RELAY_DATA_DIR: data, RELAY_KEYRING: 'off', RELAY_WEB_DIR: o.webDir, RELAY_ALLOWED_HOSTS: '', ...o.env }, + env: { + ...(fakes?.env ?? base), + PORT: '0', + HOST: '127.0.0.1', + RELAY_DATA_DIR: data, + RELAY_KEYRING: 'off', + RELAY_WEB_DIR: o.webDir, + RELAY_ALLOWED_HOSTS: '', + ...o.env, + }, stdio: ['ignore', 'pipe', 'pipe', 'ipc'], }); child.stdout!.on('data', (b: Buffer) => (output += b.toString())); diff --git a/desktop/src/env.ts b/desktop/src/env.ts index 4eb0a95e..c001d331 100644 --- a/desktop/src/env.ts +++ b/desktop/src/env.ts @@ -169,11 +169,16 @@ export function mergeWindowsEnv(base: NodeJS.ProcessEnv, current: WindowsEnv): N export async function windowsEnv(env: NodeJS.ProcessEnv, timeoutMs = 15_000): Promise { const root = env.SystemRoot ?? env.SYSTEMROOT ?? 'C:\\Windows'; try { - const r = await capture(`${root}\\System32\\WindowsPowerShell\\v1.0\\powershell.exe`, ['-NoProfile', '-NonInteractive', '-Command', WINDOWS_ENV_SCRIPT], env, { - timeoutMs, - maxCapture: 4 * 1024 * 1024, - keepStart: true, - }); + const r = await capture( + `${root}\\System32\\WindowsPowerShell\\v1.0\\powershell.exe`, + ['-NoProfile', '-NonInteractive', '-Command', WINDOWS_ENV_SCRIPT], + env, + { + timeoutMs, + maxCapture: 4 * 1024 * 1024, + keepStart: true, + }, + ); return r.code === 0 ? parseWindowsEnv(r.stdout) : undefined; } catch { return undefined; @@ -216,9 +221,7 @@ export const DESKTOP_KEYRING_SERVICE = 'harness-relay-desktop'; export function serverEnv(o: ServerEnvOptions): NodeJS.ProcessEnv { const env: NodeJS.ProcessEnv = {}; for (const [k, v] of Object.entries(o.base)) if (v !== undefined && !isElectronVar(k)) env[k] = v; - const hosts = [...(o.base.RELAY_ALLOWED_HOSTS ?? '').split(','), ...o.allowedHosts] - .map((h) => h.trim().toLowerCase()) - .filter(Boolean); + const hosts = [...(o.base.RELAY_ALLOWED_HOSTS ?? '').split(','), ...o.allowedHosts].map((h) => h.trim().toLowerCase()).filter(Boolean); const ownData = !o.base.RELAY_DATA_DIR?.trim(); env.RELAY_DATA_DIR = o.base.RELAY_DATA_DIR?.trim() || o.dataDir; if (ownData && !o.base.RELAY_KEYRING_SERVICE?.trim()) env.RELAY_KEYRING_SERVICE = DESKTOP_KEYRING_SERVICE; @@ -235,5 +238,8 @@ export function serverEnv(o: ServerEnvOptions): NodeJS.ProcessEnv { /** The names in RELAY_ALLOWED_HOSTS. */ export function allowedHostsOf(env: NodeJS.ProcessEnv): string[] { - return (env.RELAY_ALLOWED_HOSTS ?? '').split(',').map((h) => h.trim().toLowerCase()).filter(Boolean); + return (env.RELAY_ALLOWED_HOSTS ?? '') + .split(',') + .map((h) => h.trim().toLowerCase()) + .filter(Boolean); } diff --git a/desktop/src/i18n.ts b/desktop/src/i18n.ts index 54c67e19..895aacc9 100644 --- a/desktop/src/i18n.ts +++ b/desktop/src/i18n.ts @@ -93,8 +93,7 @@ const zhTW = { mClose: '關閉視窗', tailscaleMissing: '找不到 Tailscale', - tailscaleMissingDetail: - '要從手機使用,請在這台電腦和手機都安裝 Tailscale,並登入同一個 tailnet。在 Mac 上,裝好 Tailscale app 即可。', + tailscaleMissingDetail: '要從手機使用,請在這台電腦和手機都安裝 Tailscale,並登入同一個 tailnet。在 Mac 上,裝好 Tailscale app 即可。', tailscaleNotRunning: 'Tailscale 還沒有連線', tailscaleNotRunningDetail: '請打開 Tailscale 並登入,再選一次「分享到手機」。', tailscaleApproval: '要先為你的 tailnet 開啟 Serve 和 HTTPS', @@ -123,10 +122,12 @@ const zhTW = { updateWaiting: '更新已就緒,等回答結束後切換', updateSwitchNow: '立即切換到新版…', updateSwitchConfirm: '要立即切換到新版嗎?', - updateSwitchDetail: (replying: number | undefined) => (replying ? `有 ${replying} 個對話正在回覆,切換時會被中斷。` : '正在回覆的對話會被中斷。'), + updateSwitchDetail: (replying: number | undefined) => + replying ? `有 ${replying} 個對話正在回覆,切換時會被中斷。` : '正在回覆的對話會被中斷。', updateSwitch: '立即切換', updateConfirm: (version: string) => `要把 Harness Relay 更新到 ${version} 嗎?`, - updateConfirmDetail: 'Harness Relay 會下載新版並檢查它,然後結束、換成新版再打開。有對話正在回覆時,會等它們都回答完再切換;也可以從選單立即切換。', + updateConfirmDetail: + 'Harness Relay 會下載新版並檢查它,然後結束、換成新版再打開。有對話正在回覆時,會等它們都回答完再切換;也可以從選單立即切換。', updateNow: '立即更新', whatChanged: '查看變更內容', updateManual: '這個 Harness Relay 無法自己更新', @@ -135,7 +136,8 @@ const zhTW = { 'not-packaged': '它是從原始碼執行的(npm run desktop)。請到發布頁下載新版。', 'no-installer': '新版沒有這個系統的安裝檔。請到發布頁看看。', 'no-checksum': 'GitHub 沒有提供這個安裝檔的檢查碼,無法確認下載的內容。請到發布頁手動下載。', - 'read-only': 'Harness Relay 不在可以替換的位置(例如直接從 .dmg 執行,或所在的資料夾不能寫入)。請到發布頁下載新版;在 Mac 上,請把它拖到「應用程式」資料夾。', + 'read-only': + 'Harness Relay 不在可以替換的位置(例如直接從 .dmg 執行,或所在的資料夾不能寫入)。請到發布頁下載新版;在 Mac 上,請把它拖到「應用程式」資料夾。', 'not-appimage': '在 Linux 上只有 AppImage 能自己更新。請到發布頁下載新版。', })[why], updateFailed: '無法更新 Harness Relay', @@ -220,13 +222,13 @@ const en: DesktopMessages = { tailscaleNotRunningDetail: 'Open Tailscale and sign in, then choose Share with your phone again.', tailscaleApproval: 'Serve and HTTPS have to be turned on for your tailnet first', tailscaleApprovalDetail: "Tailscale's page is open in your browser. Allow it there, then choose Share with your phone again.", - tailscaleAccessDenied: "Tailscale does not let this user change its settings", + tailscaleAccessDenied: 'Tailscale does not let this user change its settings', tailscaleAccessDeniedDetail: (output) => `Run this once in a terminal:\n\nsudo tailscale set --operator=$USER\n\nThen choose Share with your phone again.\n\nTailscale said:\n${output}`, tailscaleFailed: 'Could not share with your tailnet', tailscaleFailedDetail: (output) => `Tailscale said:\n${output}`, tailscaleStatusUnknown: - "Could not read what this computer already shares with the tailnet (tailscale serve status did not answer), so nothing was shared: it could take over another service or Funnel. Check that Tailscale runs, then choose Share with your phone again.", + 'Could not read what this computer already shares with the tailnet (tailscale serve status did not answer), so nothing was shared: it could take over another service or Funnel. Check that Tailscale runs, then choose Share with your phone again.', tailscalePortsTaken: "HTTPS ports 443, 8443 and 10000 of this computer's tailnet address already share something else. Share Harness Relay by hand, as in Use it from your phone, or first stop one of them with tailscale serve --https= off.", shared: 'Shared with your tailnet', @@ -260,7 +262,8 @@ const en: DesktopMessages = { ({ 'not-packaged': 'It runs from the source code (npm run desktop). Download the new version from the releases page.', 'no-installer': 'The new version has no installer for this system. See the releases page.', - 'no-checksum': 'GitHub gives no checksum for its installer, so the download could not be checked. Download it from the releases page.', + 'no-checksum': + 'GitHub gives no checksum for its installer, so the download could not be checked. Download it from the releases page.', 'read-only': 'Harness Relay cannot be replaced where it is (it runs from the .dmg, say, or from a folder it may not write in). Download the new version from the releases page; on a Mac, drag it into Applications.', 'not-appimage': 'On Linux, only the AppImage updates itself. Download the new version from the releases page.', @@ -275,7 +278,8 @@ const en: DesktopMessages = { replace: 'The app could not be replaced by the new version: this version stays as it is.', start: 'The new version could not be started. Harness Relay runs on as this version.', })[failure], - updateOpenedInstaller: "The new version's installer is open: drag Harness Relay into Applications, replacing this one (macOS may ask you to allow App Management).", + updateOpenedInstaller: + "The new version's installer is open: drag Harness Relay into Applications, replacing this one (macOS may ask you to allow App Management).", }; export const DESKTOP_MESSAGES: Record = { 'zh-TW': zhTW, en }; diff --git a/desktop/src/lifecycle.ts b/desktop/src/lifecycle.ts index 47b6f2c0..29a25752 100644 --- a/desktop/src/lifecycle.ts +++ b/desktop/src/lifecycle.ts @@ -52,7 +52,12 @@ export function linkAction(url: string, serverUrl: string): 'in-app' | 'external * only the microphone (voice input, no camera), writing to the clipboard (the copy buttons), full * screen (a video's own button) and notifications (Settings → Notifications: agents finished or need you). */ -export function permissionAllowed(permission: string, requestingUrl: string | undefined, serverUrl: string, mediaTypes: readonly string[] = []): boolean { +export function permissionAllowed( + permission: string, + requestingUrl: string | undefined, + serverUrl: string, + mediaTypes: readonly string[] = [], +): boolean { if (!requestingUrl || linkAction(requestingUrl, serverUrl) !== 'in-app') return false; if (permission === 'media') return mediaTypes.length > 0 && mediaTypes.every((type) => type === 'audio'); return permission === 'clipboard-sanitized-write' || permission === 'fullscreen' || permission === 'notifications'; @@ -62,13 +67,20 @@ export function permissionAllowed(permission: string, requestingUrl: string | un * Electron's permission check (asked without a request, such as before listing the microphones): a * check for media that names no kind counts as one for the microphone, the only kind ever allowed. */ -export function permissionChecked(permission: string, origin: string | undefined, serverUrl: string | undefined, mediaType?: string): boolean { +export function permissionChecked( + permission: string, + origin: string | undefined, + serverUrl: string | undefined, + mediaType?: string, +): boolean { return !!serverUrl && permissionAllowed(permission, origin, serverUrl, mediaType ? [mediaType] : ['audio']); } /** What a window's page may open: the parts of Electron's WebContents that `guardContents` uses. */ export interface GuardedContents { - setWindowOpenHandler(handler: (details: { url: string }) => { action: 'allow'; overrideBrowserWindowOptions?: object } | { action: 'deny' }): void; + setWindowOpenHandler( + handler: (details: { url: string }) => { action: 'allow'; overrideBrowserWindowOptions?: object } | { action: 'deny' }, + ): void; // eslint-disable-next-line @typescript-eslint/no-explicit-any on(event: string, listener: (...args: any[]) => void): unknown; } diff --git a/desktop/src/main.ts b/desktop/src/main.ts index 1fdd582b..fdebf0cd 100644 --- a/desktop/src/main.ts +++ b/desktop/src/main.ts @@ -246,7 +246,10 @@ function smokeTest(report: string | undefined, clis: string | undefined): void { await win.loadURL(chatAddress(url, await supervisor.signInLink())); let signedIn = false; for (let i = 0; i < 150 && !signedIn; i++) { - signedIn = (await win.webContents.executeJavaScript('!document.querySelector("[data-signin]") && (document.getElementById("root")?.childElementCount ?? 0) > 0')) === true; + signedIn = + (await win.webContents.executeJavaScript( + '!document.querySelector("[data-signin]") && (document.getElementById("root")?.childElementCount ?? 0) > 0', + )) === true; if (!signedIn) await new Promise((r) => setTimeout(r, 100)); } check(signedIn, 'signs a sandboxed window in with a link and shows the chat'); @@ -665,7 +668,12 @@ function run(): void { return; } if (result.kind === 'failed') { - await dialog.showMessageBox({ type: 'error', message: t.tailscaleFailed, detail: t.tailscaleFailedDetail(result.output.slice(-800)), buttons: [t.ok] }); + await dialog.showMessageBox({ + type: 'error', + message: t.tailscaleFailed, + detail: t.tailscaleFailedDetail(result.output.slice(-800)), + buttons: [t.ok], + }); return; } url = (await net.tailscale.servedUrl(prefs.port)) ?? result.url ?? `https://${net.dnsName}${https === 443 ? '' : `:${https}`}`; @@ -853,7 +861,14 @@ function run(): void { /** Offers the release's page instead, when this copy cannot update itself (`why`), or the update failed. */ async function offerReleasePage(release: UpdateRelease, message: string, detail: string): Promise { - const { response } = await dialog.showMessageBox({ type: 'info', message, detail, buttons: [t.openDownloadPage, t.cancel], defaultId: 0, cancelId: 1 }); + const { response } = await dialog.showMessageBox({ + type: 'info', + message, + detail, + buttons: [t.openDownloadPage, t.cancel], + defaultId: 0, + cancelId: 1, + }); if (response === 0) void shell.openExternal(release.page); } @@ -873,7 +888,13 @@ function run(): void { if (!release || updating) return; const system = updateSystem(process.platform, process.arch); const asset = system ? release.assets[system] : undefined; - const plan = installPlan({ platform: process.platform, packaged: app.isPackaged, execPath: process.execPath, appImage: process.env.APPIMAGE, asset }); + const plan = installPlan({ + platform: process.platform, + packaged: app.isPackaged, + execPath: process.execPath, + appImage: process.env.APPIMAGE, + asset, + }); if (plan.kind === 'manual' || !asset) { await offerReleasePage(release, t.updateManual, t.updateManualDetail(plan.kind === 'manual' ? plan.why : 'no-installer')); return; @@ -922,7 +943,8 @@ function run(): void { setUpdating({ kind: 'installing' }); if (!(await waitForConversations())) return; placeAppImage(downloaded, target, plan.appImage); - if (process.platform === 'linux' && isAutostartOn(autostartFile(process.env, homedir()))) writeAutostart([target, ...relaunchArgs()]); + if (process.platform === 'linux' && isAutostartOn(autostartFile(process.env, homedir()))) + writeAutostart([target, ...relaunchArgs()]); await finishUpdate(relaunchCommand('linux', process.pid, target, relaunchEnv(process.env), relaunchArgs())); return; } @@ -982,7 +1004,12 @@ function run(): void { // --- Open at login ------------------------------------------------------------------------ function appCommand(): string[] { - const command = launchCommand({ execPath: process.execPath, appImage: process.env.APPIMAGE, packaged: app.isPackaged, appPath: app.getAppPath() }); + const command = launchCommand({ + execPath: process.execPath, + appImage: process.env.APPIMAGE, + packaged: app.isPackaged, + appPath: app.getAppPath(), + }); // An AppImage that only starts without Chromium's sandbox (Ubuntu 24.04) must start so at login too. return process.argv.includes('--no-sandbox') ? [...command, '--no-sandbox'] : command; } @@ -1092,7 +1119,12 @@ function run(): void { { type: 'separator' }, { id: 'version', label: `${t.version} ${__RELAY_VERSION__}`, enabled: false }, { id: 'update-status', label: updating ? updatingLabel(updating) : '', enabled: false, visible: !!updating }, - { id: 'update-offer', label: update ? t.updateTo(update.version) : '', visible: !updating && !!update, click: () => void installUpdate() }, + { + id: 'update-offer', + label: update ? t.updateTo(update.version) : '', + visible: !updating && !!update, + click: () => void installUpdate(), + }, { id: 'update-switch', label: t.updateSwitchNow, visible: updating?.kind === 'waiting', click: () => void confirmSwitchNow() }, { id: 'quit', label: t.quit, click: () => app.quit() }, ]; @@ -1277,7 +1309,11 @@ function run(): void { refreshAutostart(); createTray(); setKeepAwake(prefs.keepAwake); - const atLogin = startedAtLogin(process.argv, process.platform, process.platform === 'darwin' && app.getLoginItemSettings().wasOpenedAtLogin); + const atLogin = startedAtLogin( + process.argv, + process.platform, + process.platform === 'darwin' && app.getLoginItemSettings().wasOpenedAtLogin, + ); // Before the start, whose dialogs (a port in use, say) wait for the user. scheduleUpdateChecks(); await startServer({ openWindow: !atLogin }); diff --git a/desktop/src/selftest.ts b/desktop/src/selftest.ts index c7de17a1..bcd03015 100644 --- a/desktop/src/selftest.ts +++ b/desktop/src/selftest.ts @@ -86,7 +86,8 @@ export function withFakeClis(env: NodeJS.ProcessEnv, fakeDir: string, root: stri CODEX_HOME: join(home, '.codex'), GROK_HOME: join(home, '.grok'), }); - for (const dir of [out.XDG_CONFIG_HOME!, out.CLAUDE_CONFIG_DIR!, out.CODEX_HOME!, out.GROK_HOME!, cwd]) mkdirSync(dir, { recursive: true }); + for (const dir of [out.XDG_CONFIG_HOME!, out.CLAUDE_CONFIG_DIR!, out.CODEX_HOME!, out.GROK_HOME!, cwd]) + mkdirSync(dir, { recursive: true }); writeFileSync(join(cwd, 'README.md'), '# Smoke test\n\nThe folder the desktop app check asks its agents a question in.\n'); return { env: out, cwd }; } @@ -114,7 +115,8 @@ export async function checkTurn(base: string, auth: SignedIn, check: Check, o: { } return { ok: r.ok, status: r.status, json, text }; }; - const said = (what: string, r: { ok: boolean; status: number; text: string }) => (r.ok ? what : `${what} (${r.status} ${r.text.slice(0, 500)})`); + const said = (what: string, r: { ok: boolean; status: number; text: string }) => + r.ok ? what : `${what} (${r.status} ${r.text.slice(0, 500)})`; const names = TURN_CLIS.join(', ').replace(/, (?=[^,]*$)/, ' and '); const created = await request('/api/conversations', { @@ -150,6 +152,11 @@ export async function checkTurn(base: string, auth: SignedIn, check: Check, o: { const text = (reply?.parts ?? []).map((p) => (p.type === 'text' ? p.text : '')).join(''); const ok = reply?.status === 'done' && text.includes(question); const what = `${provider} answers it`; - check(ok, ok ? what : `${what} (${reply ? `${reply.status}${reply.error ? `: ${reply.error}` : ''}: ${JSON.stringify(text.slice(0, 300))}` : 'no reply'})`); + check( + ok, + ok + ? what + : `${what} (${reply ? `${reply.status}${reply.error ? `: ${reply.error}` : ''}: ${JSON.stringify(text.slice(0, 300))}` : 'no reply'})`, + ); } } diff --git a/desktop/src/supervisor.ts b/desktop/src/supervisor.ts index d8b969eb..d77e57d0 100644 --- a/desktop/src/supervisor.ts +++ b/desktop/src/supervisor.ts @@ -25,10 +25,7 @@ export type StartError = | { kind: 'timeout'; log: string }; export type ServerState = - | { kind: 'stopped'; error?: StartError } - | { kind: 'starting' } - | { kind: 'running'; url: string; port: number } - | { kind: 'stopping' }; + { kind: 'stopped'; error?: StartError } | { kind: 'starting' } | { kind: 'running'; url: string; port: number } | { kind: 'stopping' }; export interface SupervisorOptions { fork(env: NodeJS.ProcessEnv): ServerChild; diff --git a/desktop/src/update.ts b/desktop/src/update.ts index 059a8302..3dfc5752 100644 --- a/desktop/src/update.ts +++ b/desktop/src/update.ts @@ -30,12 +30,31 @@ * Electron: main.ts passes in what it needs (net.fetch, the paths). */ import { createHash } from 'node:crypto'; -import { accessSync, chmodSync, constants, createReadStream, existsSync, mkdtempSync, readdirSync, renameSync, rmdirSync, rmSync } from 'node:fs'; +import { + accessSync, + chmodSync, + constants, + createReadStream, + existsSync, + mkdtempSync, + readdirSync, + renameSync, + rmdirSync, + rmSync, +} from 'node:fs'; import { open, type FileHandle } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { basename, dirname, join } from 'node:path'; import { compareSemver, parseVersion, semverOf } from '../../scripts/release'; -import { INSTALLERS, isInstallerUrl, mirrorCopyUrl, RELEASES_PAGE, REPO, updateCopyName, updateCopyUrl } from '../../site/assets/downloads.js'; +import { + INSTALLERS, + isInstallerUrl, + mirrorCopyUrl, + RELEASES_PAGE, + REPO, + updateCopyName, + updateCopyUrl, +} from '../../site/assets/downloads.js'; import { capture } from '../../src/server/process'; import type { DesktopUpdate, UpdateProgress } from '../../src/shared/types'; import type { UpdateSource } from './updateSource'; @@ -219,7 +238,12 @@ export function parseRelease(json: unknown, source: UpdateSource): UpdateRelease const installer = a.browser_download_url as string; const copy = updateCopyUrl(r.tag_name, a.name, source.repo); const sha256 = typeof a.digest === 'string' ? /^sha256:([0-9a-f]{64})$/i.exec(a.digest)?.[1]?.toLowerCase() : undefined; - assets[system] = { name: a.name, ...(copy ? { url: copy, fallback: installer } : { url: installer }), size: typeof a.size === 'number' ? a.size : undefined, sha256 }; + assets[system] = { + name: a.name, + ...(copy ? { url: copy, fallback: installer } : { url: installer }), + size: typeof a.size === 'number' ? a.size : undefined, + sha256, + }; } } const page = typeof r.html_url === 'string' && r.html_url.startsWith(`${RELEASES_PAGE}/tag/`) ? r.html_url : RELEASES_PAGE; @@ -296,7 +320,10 @@ export type Fetch = ( * Asks GitHub for the newest release, release candidates included; undefined when there is none yet. * Each installer is fetched from its copy in the copies' repository of `source` (never asked here). */ -export async function checkLatest(fetch: Fetch, o: { source: UpdateSource; candidates?: boolean; signal?: AbortSignal }): Promise { +export async function checkLatest( + fetch: Fetch, + o: { source: UpdateSource; candidates?: boolean; signal?: AbortSignal }, +): Promise { const { source, candidates = true, signal } = o; // A release candidate (or a build of your own) looks through the newest releases, candidates too; a // release asks for the newest release alone. @@ -375,10 +402,7 @@ export type ManualReason = | 'not-appimage'; export type InstallPlan = - | { kind: 'mac'; app: string } - | { kind: 'win' } - | { kind: 'linux'; appImage: string } - | { kind: 'manual'; why: ManualReason }; + { kind: 'mac'; app: string } | { kind: 'win' } | { kind: 'linux'; appImage: string } | { kind: 'manual'; why: ManualReason }; /** The .app an executable is in (…/Harness Relay.app/Contents/MacOS/Harness Relay). */ export function macAppOf(execPath: string): string | undefined { @@ -491,7 +515,15 @@ export async function download(o: DownloadOptions): Promise { // itself, once, without the token, checked against the same SHA-256. Not when the bytes came from // the installer already (the copy could not be had): it would only be fetched again. const { fallback } = o.asset; - if (!(err instanceof UpdateError) || err.failure !== 'checksum' || !o.asset.sha256 || !fallback || fallback === o.asset.url || went.toFallback) throw err; + if ( + !(err instanceof UpdateError) || + err.failure !== 'checksum' || + !o.asset.sha256 || + !fallback || + fallback === o.asset.url || + went.toFallback + ) + throw err; await downloadOnce({ ...o, asset: { ...o.asset, url: fallback, fallback: undefined }, headers: undefined }, went); } } @@ -565,7 +597,13 @@ export function mirrorAsset(mirror: string | undefined, tag: string, asset: Upda * both, as for a download from GitHub (docs/developer-guide.md, "Releasing", "Download counts"). Never * fails: GitHub not reached within `answerMs`, the update goes uncounted. */ -export async function countUpdate(o: { fetch: Fetch; locate: Locate; asset: UpdateAsset; headers?: Record; answerMs?: number }): Promise { +export async function countUpdate(o: { + fetch: Fetch; + locate: Locate; + asset: UpdateAsset; + headers?: Record; + answerMs?: number; +}): Promise { const ms = o.answerMs ?? CHECK_TIMEOUT_MS; const signal = AbortSignal.timeout(ms); try { @@ -597,7 +635,9 @@ export async function countUpdate(o: { fetch: Fetch; locate: Locate; asset: Upda * say, stalled, its file not the installer) and the download is made from GitHub as without a mirror, * counted there by itself. What main.ts calls. */ -export async function downloadUpdate(o: Omit & { source: UpdateSource; tag: string; lookupSignal?: AbortSignal; mirror?: string }): Promise { +export async function downloadUpdate( + o: Omit & { source: UpdateSource; tag: string; lookupSignal?: AbortSignal; mirror?: string }, +): Promise { const { source, tag, lookupSignal, mirror, ...rest } = o; const from = await copyToDownload(o.fetch, source, tag, o.asset, lookupSignal); const { locate } = o; @@ -640,15 +680,52 @@ async function downloadOnce(o: DownloadOptions, went: { toFallback: boolean }): const write = writer(file); if (o.connections && o.connections.length > 1 && o.locate && size >= 2 * minPart) { try { - await inParts({ connections: o.connections, locate: o.locate, asset: o.asset, headers: o.headers, size, minPart, stall, stallMs, answerMs, write, onProgress: o.onProgress, fellBack }); + await inParts({ + connections: o.connections, + locate: o.locate, + asset: o.asset, + headers: o.headers, + size, + minPart, + stall, + stallMs, + answerMs, + write, + onProgress: o.onProgress, + fellBack, + }); } catch (err) { if (!(err instanceof NoRanges)) throw err; // A server (or a proxy) that sends the whole file for a part of it: all of it in one request, from there. await file.truncate(0); - await whole({ fetch: o.fetch, locate: o.locate, url: err.address, headers: err.headers, fallback: o.asset.fallback, size, stall, answerMs, write, onProgress: o.onProgress, fellBack }); + await whole({ + fetch: o.fetch, + locate: o.locate, + url: err.address, + headers: err.headers, + fallback: o.asset.fallback, + size, + stall, + answerMs, + write, + onProgress: o.onProgress, + fellBack, + }); } } else { - await whole({ fetch: o.fetch, locate: o.locate, url: o.asset.url, headers: o.headers, fallback: o.asset.fallback, size, stall, answerMs, write, onProgress: o.onProgress, fellBack }); + await whole({ + fetch: o.fetch, + locate: o.locate, + url: o.asset.url, + headers: o.headers, + fallback: o.asset.fallback, + size, + stall, + answerMs, + write, + onProgress: o.onProgress, + fellBack, + }); } } catch (err) { failed = true; @@ -714,7 +791,11 @@ class Stall { * Gives `body`'s chunks to `take`, one at a time, until it ends or `take` has had enough (false: the rest * is cancelled). Stops with `signal`'s reason once it aborts, also with a body that would go on. */ -async function readBody(body: ReadableStream, signal: AbortSignal, take: (chunk: Uint8Array) => Promise): Promise { +async function readBody( + body: ReadableStream, + signal: AbortSignal, + take: (chunk: Uint8Array) => Promise, +): Promise { const reader = body.getReader(); const cancel = () => void reader.cancel(signal.reason).catch(() => undefined); signal.addEventListener('abort', cancel, { once: true }); @@ -758,7 +839,13 @@ function unavailable(status: number, authorized: boolean): boolean { * `answerMs`, counts as one that is `unavailable` (status 0), so that the installer is fetched instead, * unless the download itself was given up on (`signal`). */ -async function locateWithToken(locate: Locate, url: string, signal: AbortSignal, headers: Record, answerMs: number): Promise<{ status: number; location?: string }> { +async function locateWithToken( + locate: Locate, + url: string, + signal: AbortSignal, + headers: Record, + answerMs: number, +): Promise<{ status: number; location?: string }> { try { return await answeredWithin(answerMs, signal, (s) => locate(url, s, headers)); } catch (err) { @@ -808,7 +895,8 @@ async function whole(o: { let target: { url: string; headers?: Record } | undefined = { url: o.url }; if (o.headers) { const found = o.locate ? await locateWithToken(o.locate, o.url, signal, o.headers, o.answerMs) : undefined; - if (found && unavailable(found.status, true) && !o.fallback) throw new UpdateError('download', found.status ? `HTTP ${found.status}` : 'no answer'); + if (found && unavailable(found.status, true) && !o.fallback) + throw new UpdateError('download', found.status ? `HTTP ${found.status}` : 'no answer'); if (!found || unavailable(found.status, true)) target = undefined; else { const at = addressFrom(found, o.url); @@ -819,7 +907,9 @@ async function whole(o: { if (target) { try { const { url, headers } = target; - res = headers ? await answeredWithin(o.answerMs, signal, (s) => o.fetch(url, { headers, signal: s, redirect: 'error' })) : await o.fetch(url, { signal }); + res = headers + ? await answeredWithin(o.answerMs, signal, (s) => o.fetch(url, { headers, signal: s, redirect: 'error' })) + : await o.fetch(url, { signal }); } catch (err) { // No answer where the token went (GitHub's API answering with the file itself), also none within // `answerMs`: the installer, as below. @@ -904,7 +994,9 @@ async function inParts(o: { const { size, stall } = o; let source = o.asset.url; let headers = o.headers; - let found = headers ? await locateWithToken(o.locate, source, stall.signal, headers, o.answerMs) : await o.locate(source, stall.signal, headers); + let found = headers + ? await locateWithToken(o.locate, source, stall.signal, headers, o.answerMs) + : await o.locate(source, stall.signal, headers); // Its copy for updates cannot be had (a release published before the copies, its repository gone, the // token expired): the installer itself, without the token. if (unavailable(found.status, !!headers) && o.asset.fallback) { @@ -959,7 +1051,10 @@ async function inParts(o: { }; const count = Math.min(o.connections.length, Math.floor(size / o.minPart)); - const waiting: Part[] = Array.from({ length: count }, (_, i) => ({ pos: Math.floor((size * i) / count), end: Math.floor((size * (i + 1)) / count) })); + const waiting: Part[] = Array.from({ length: count }, (_, i) => ({ + pos: Math.floor((size * i) / count), + end: Math.floor((size * (i + 1)) / count), + })); const active = new Set(); let received = 0; let failures = 0; @@ -983,7 +1078,12 @@ async function inParts(o: { const watch = new Stall(o.stallMs); const signal = AbortSignal.any([stop, watch.signal]); try { - const ask = (s: AbortSignal) => fetch(at, { headers: { ...sent, Range: `bytes=${from}-${part.end - 1}` }, signal: s, ...(sent ? { redirect: 'error' as const } : {}) }); + const ask = (s: AbortSignal) => + fetch(at, { + headers: { ...sent, Range: `bytes=${from}-${part.end - 1}` }, + signal: s, + ...(sent ? { redirect: 'error' as const } : {}), + }); let res: Response; try { res = sent ? await answeredWithin(o.answerMs, signal, ask) : await ask(signal); @@ -1077,7 +1177,10 @@ export const IDLE_CHECK_MS = 2000; * the way to a crash; and unknown for good once the app is quitting (or the system shutting down), * so that quitting never switches to the new version. */ -export function busyWithoutAsking(server: 'stopped' | 'starting' | 'running' | 'stopping', o: { starting: boolean; quitting: boolean }): 0 | 'ask' | undefined { +export function busyWithoutAsking( + server: 'stopped' | 'starting' | 'running' | 'stopping', + o: { starting: boolean; quitting: boolean }, +): 0 | 'ask' | undefined { if (o.quitting || o.starting) return undefined; if (server === 'running') return 'ask'; return server === 'stopped' ? 0 : undefined; @@ -1103,7 +1206,10 @@ export async function whenIdle(o: { if (busy === 0) return 'idle'; o.onWaiting?.(busy); let timer: ReturnType | undefined; - const waited = await Promise.race([new Promise<'again'>((r) => (timer = setTimeout(() => r('again'), o.everyMs ?? IDLE_CHECK_MS))), now]); + const waited = await Promise.race([ + new Promise<'again'>((r) => (timer = setTimeout(() => r('again'), o.everyMs ?? IDLE_CHECK_MS))), + now, + ]); clearTimeout(timer); if (waited === 'now') return 'now'; } @@ -1121,11 +1227,22 @@ export function macStaging(app: string): { staged: string; old: string } { * renames), and checks it: its signature holds, and it is this app (`bundleId`) at the release's * version (`semver`). Returns where it is. */ -export async function stageMacApp(o: { dmg: string; app: string; bundleId: string; semver: string; env: NodeJS.ProcessEnv }): Promise { +export async function stageMacApp(o: { + dmg: string; + app: string; + bundleId: string; + semver: string; + env: NodeJS.ProcessEnv; +}): Promise { const { staged } = macStaging(o.app); rmSync(staged, { recursive: true, force: true }); const mount = mkdtempSync(join(tmpdir(), 'harness-relay-update-')); - const attach = await capture('/usr/bin/hdiutil', ['attach', '-nobrowse', '-readonly', '-noautoopen', '-mountpoint', mount, o.dmg], o.env, { timeoutMs: 120_000 }); + const attach = await capture( + '/usr/bin/hdiutil', + ['attach', '-nobrowse', '-readonly', '-noautoopen', '-mountpoint', mount, o.dmg], + o.env, + { timeoutMs: 120_000 }, + ); if (attach.code !== 0) { try { rmdirSync(mount); @@ -1152,7 +1269,8 @@ export async function stageMacApp(o: { dmg: string; app: string; bundleId: strin const signature = await capture('/usr/bin/codesign', ['--verify', '--deep', '--strict', staged], o.env, { timeoutMs: 120_000 }); if (signature.code !== 0) throw new UpdateError('verify', signature.stderr.trim() || 'codesign --verify failed'); const plist = join(staged, 'Contents', 'Info.plist'); - const read = async (key: string) => (await capture('/usr/bin/plutil', ['-extract', key, 'raw', '-o', '-', plist], o.env, { timeoutMs: 10_000 })).stdout.trim(); + const read = async (key: string) => + (await capture('/usr/bin/plutil', ['-extract', key, 'raw', '-o', '-', plist], o.env, { timeoutMs: 10_000 })).stdout.trim(); const id = await read('CFBundleIdentifier'); if (id !== o.bundleId) throw new UpdateError('verify', `bundle id ${id}, expected ${o.bundleId}`); const version = await read('CFBundleShortVersionString'); @@ -1258,7 +1376,13 @@ export const RELAUNCH_TARGET = 'HARNESS_RELAY_RELAUNCH_TARGET'; * it exits. The script is fixed text, and the pid and the path reach it in its environment, never in * its arguments. */ -export function relaunchCommand(platform: NodeJS.Platform, pid: number, target: string, env: NodeJS.ProcessEnv, args: readonly string[] = []): StartCommand { +export function relaunchCommand( + platform: NodeJS.Platform, + pid: number, + target: string, + env: NodeJS.ProcessEnv, + args: readonly string[] = [], +): StartCommand { const wait = `pid=$${RELAUNCH_PID}; target=$${RELAUNCH_TARGET}; unset ${RELAUNCH_PID} ${RELAUNCH_TARGET}; ` + 'i=0; while kill -0 "$pid" 2>/dev/null && [ "$i" -lt 600 ]; do sleep 0.1; i=$((i+1)); done;'; diff --git a/desktop/src/updateSource.ts b/desktop/src/updateSource.ts index 7260485a..0a6e25bf 100644 --- a/desktop/src/updateSource.ts +++ b/desktop/src/updateSource.ts @@ -37,21 +37,30 @@ const TOKEN = /^github_pat_[A-Za-z0-9_]{20,244}$/; * The update source an environment names (RELAY_UPDATES_REPO, RELAY_UPDATES_TOKEN, each trimmed; an * empty one is not set), or why it cannot be used. The token never appears in an error. */ -export function updateSourceFrom(env: Record): { ok: true; value: UpdateSource } | { ok: false; error: string } { +export function updateSourceFrom( + env: Record, +): { ok: true; value: UpdateSource } | { ok: false; error: string } { const repo = env.RELAY_UPDATES_REPO?.trim() || DEFAULT_UPDATE_SOURCE.repo; const token = env.RELAY_UPDATES_TOKEN?.trim() || undefined; const parts = REPO.exec(repo); if (!parts || /^\.+$/.test(parts[2]!) || parts[2]!.endsWith('.git')) { - return { ok: false, error: `RELAY_UPDATES_REPO: "${repo}" is not a GitHub repository as owner/name (Agentic-System-Lab/harness-relay-updates, say)` }; + return { + ok: false, + error: `RELAY_UPDATES_REPO: "${repo}" is not a GitHub repository as owner/name (Agentic-System-Lab/harness-relay-updates, say)`, + }; } // GitHub's names are the same in any letter case. if (repo.toLowerCase() === MAIN_REPO.toLowerCase()) { - return { ok: false, error: `RELAY_UPDATES_REPO: the copies go in a repository of their own, not in ${MAIN_REPO}, whose releases hold the installers alone` }; + return { + ok: false, + error: `RELAY_UPDATES_REPO: the copies go in a repository of their own, not in ${MAIN_REPO}, whose releases hold the installers alone`, + }; } if (token !== undefined && !TOKEN.test(token)) { return { ok: false, - error: 'RELAY_UPDATES_TOKEN: not a fine-grained personal access token (github_pat_…): any other kind cannot be kept to one repository, and it goes into every copy of the app', + error: + 'RELAY_UPDATES_TOKEN: not a fine-grained personal access token (github_pat_…): any other kind cannot be kept to one repository, and it goes into every copy of the app', }; } return { ok: true, value: token ? { repo, token } : { repo } }; @@ -87,7 +96,9 @@ export function decodeUpdateSource(built: BuiltUpdateSource): UpdateSource { * this run's (`sameUpdateSource`). The repository in any letter case, as GitHub takes it. */ export function updateSourceFingerprint(source: UpdateSource): string { - return createHash('sha256').update(JSON.stringify([source.repo.toLowerCase(), source.token ?? ''])).digest('hex'); + return createHash('sha256') + .update(JSON.stringify([source.repo.toLowerCase(), source.token ?? ''])) + .digest('hex'); } /** @@ -106,7 +117,10 @@ export function sameUpdateSource(built: string[], source: UpdateSource, installe } /** What fetching `copiesReadable` needs: the global fetch, or a stand-in for it. */ -export type CopiesFetch = (url: string, init: { headers: Record; redirect: 'manual'; signal: AbortSignal }) => Promise; +export type CopiesFetch = ( + url: string, + init: { headers: Record; redirect: 'manual'; signal: AbortSignal }, +) => Promise; /** * Why an app built with `source` could not download the copies (`copies`: each file's name and size) @@ -143,7 +157,12 @@ export async function copiesReadable( const { repo } = o.source; const at = `the release ${o.tag} in ${repo}`; // A 404 asked once more: the release just made may not be found at once. - const asked = await askGitHub(fetch, `https://api.github.com/repos/${repo}/releases/tags/${encodeURIComponent(o.tag)}`, o.source.token ?? o.token, { ...o, notFoundAgain: true }); + const asked = await askGitHub( + fetch, + `https://api.github.com/repos/${repo}/releases/tags/${encodeURIComponent(o.tag)}`, + o.source.token ?? o.token, + { ...o, notFoundAgain: true }, + ); if ('limited' in asked) { return { warning: @@ -152,7 +171,9 @@ export async function copiesReadable( }; } if ('failure' in asked) { - return { error: `GitHub could not be asked for ${at} (${asked.failure}), ${COPIES_TRIES} times: run the failed job again (Re-run failed jobs); nothing needs building again` }; + return { + error: `GitHub could not be asked for ${at} (${asked.failure}), ${COPIES_TRIES} times: run the failed job again (Re-run failed jobs); nothing needs building again`, + }; } const { res } = asked; if (res.status >= 300 && res.status < 400) { @@ -163,7 +184,8 @@ export async function copiesReadable( }; } if (!res.ok) { - const again = ', then run the whole workflow again (Re-run all jobs, not only the failed ones), so that the installers are built with it'; + const again = + ', then run the whole workflow again (Re-run all jobs, not only the failed ones), so that the installers are built with it'; const slow = res.status === 404 ? ' If the release was only slow to appear, Re-run failed jobs is enough.' : ''; return { error: o.source.token @@ -172,8 +194,12 @@ export async function copiesReadable( : `${at[0]!.toUpperCase()}${at.slice(1)} cannot be read without a token of its own (GitHub answered ${res.status}): for a private repository, set the UPDATES_READ_TOKEN secret${again}.${slow}`, }; } - const warning = o.source.token ? expiryWarning(res.headers.get('github-authentication-token-expiration'), o.now ?? new Date()) : undefined; - const release = (await res.json().catch(() => null)) as { assets?: { id?: unknown; name?: unknown; state?: unknown; size?: unknown }[] } | null; + const warning = o.source.token + ? expiryWarning(res.headers.get('github-authentication-token-expiration'), o.now ?? new Date()) + : undefined; + const release = (await res.json().catch(() => null)) as { + assets?: { id?: unknown; name?: unknown; state?: unknown; size?: unknown }[]; + } | null; const listed = new Map((Array.isArray(release?.assets) ? release.assets : []).map((a) => [a?.name, a])); const missing = o.copies.filter(({ name, size }) => { const a = listed.get(name); @@ -208,8 +234,11 @@ export async function repositoryReadable( ): Promise<{ error?: string; warning?: string }> { const { repo } = o.source; const asked = await askGitHub(fetch, `https://api.github.com/repos/${repo}/releases?per_page=1`, o.source.token ?? o.token, o); - const later = 'the copies\' check after the builds will ask again'; - if ('limited' in asked) return { warning: `GitHub's limit on requests (answered ${asked.limited}) kept the copies' repository ${repo} from being checked now: ${later}` }; + const later = "the copies' check after the builds will ask again"; + if ('limited' in asked) + return { + warning: `GitHub's limit on requests (answered ${asked.limited}) kept the copies' repository ${repo} from being checked now: ${later}`, + }; if ('failure' in asked) return { warning: `GitHub could not be asked for the copies' repository ${repo} (${asked.failure}): ${later}` }; const { res } = asked; if (res.status >= 300 && res.status < 400) { @@ -250,7 +279,11 @@ async function askGitHub( let res: Response | undefined; try { res = await fetch(url, { - headers: { Accept: 'application/vnd.github+json', 'X-GitHub-Api-Version': '2022-11-28', ...(token ? { Authorization: `Bearer ${token}` } : {}) }, + headers: { + Accept: 'application/vnd.github+json', + 'X-GitHub-Api-Version': '2022-11-28', + ...(token ? { Authorization: `Bearer ${token}` } : {}), + }, // Not followed, so that the token goes nowhere else: a repository renamed or moved answers 301. redirect: 'manual', signal: AbortSignal.timeout(o.timeoutMs ?? COPIES_REQUEST_TIMEOUT_MS), @@ -303,8 +336,15 @@ export const COPIES_LIMIT_WAIT_MS = 5 * 60_000; async function limitedFor(res: Response, now: number): Promise { const retryAfter = Number(res.headers.get('retry-after')); const reset = Number(res.headers.get('x-ratelimit-reset')); - let limited = res.status === 429 || (res.status === 403 && (res.headers.get('x-ratelimit-remaining') === '0' || res.headers.has('retry-after'))); - if (!limited && res.status === 403) limited = /secondary rate limit/i.test(await res.clone().text().catch(() => '')); + let limited = + res.status === 429 || (res.status === 403 && (res.headers.get('x-ratelimit-remaining') === '0' || res.headers.has('retry-after'))); + if (!limited && res.status === 403) + limited = /secondary rate limit/i.test( + await res + .clone() + .text() + .catch(() => ''), + ); if (!limited) return undefined; if (Number.isFinite(retryAfter) && retryAfter > 0) return retryAfter * 1000; if (Number.isFinite(reset) && reset > 0) return Math.max(0, reset * 1000 - now) + 1000; diff --git a/docs/architecture.md b/docs/architecture.md index 25a3e8d0..77cc5a0c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -201,7 +201,7 @@ A run that designs the team writes nothing (the lead is read-only and nobody els ## Lead review -With `settings.leadReview` (any conversation, with or without auto team), the lead agent reads what the team said once it has answered, and tells the user what to look at closely and what to do next (`reviewReplies()` in the orchestrator, `src/server/reviewer.ts`). Every new chat the New chat and New schedule pages make starts with it on: their box is ticked each time (`autoTeam()` in `src/web/components/NewChat.tsx` sets `leadReview: true`; the pages do not remember the last choice), while a conversation created through the API without it has it off. The new chat a schedule opens each time (`newChat`) copies the schedule's conversation's setting, as it copies its other settings, and a branch starts with the setting its conversation has when the branch is made, not the one it had at the branched message. The user switches it with **Have the lead agent review the team's replies** on the New chat page, the New schedule page, in the team settings, or in the box on the latest team design card; `PUT /api/conversations/:id/lead-review` (`{ on }`, `Orchestrator.setLeadReview()`) switches it at any time, also while the team replies, since it is read once the team has answered. So while the team replies and no team design waits for confirmation, the team settings keep Save disabled for any other change but save a draft that changes only this box through that route (`savesLeadReviewAlone()` in `src/web/chat.ts`), whatever problems the rest of the draft shows (a missing folder, say), since only the switch is sent, and their busy note says so (with a design waiting, Save is the usual `PATCH`, which confirms the design); a chat without auto team, which has no design card, can switch it for the question being answered too. Switched off while the lead agent reviews, it stops that review (see **When.**). +With `settings.leadReview` (any conversation, with or without auto team), the lead agent reads what the team said once it has answered, and tells the user what to look at closely and what to do next (`reviewReplies()` in the orchestrator, `src/server/reviewer.ts`). Every new chat the New chat and New automation pages make starts with it on: their box is ticked each time (`autoTeam()` in `src/web/components/NewChat.tsx` sets `leadReview: true`; the pages do not remember the last choice), while a conversation created through the API without it has it off. The new chat a schedule opens each time (`newChat`) copies the schedule's conversation's setting, as it copies its other settings, and a branch starts with the setting its conversation has when the branch is made, not the one it had at the branched message. The user switches it with **Have the lead agent review the team's replies** on the New chat page, the New automation page, in the team settings, or in the box on the latest team design card; `PUT /api/conversations/:id/lead-review` (`{ on }`, `Orchestrator.setLeadReview()`) switches it at any time, also while the team replies, since it is read once the team has answered. So while the team replies and no team design waits for confirmation, the team settings keep Save disabled for any other change but save a draft that changes only this box through that route (`savesLeadReviewAlone()` in `src/web/chat.ts`), whatever problems the rest of the draft shows (a missing folder, say), since only the switch is sent, and their busy note says so (with a design waiting, Save is the usual `PATCH`, which confirms the design); a chat without auto team, which has no design card, can switch it for the question being answered too. Switched off while the lead agent reviews, it stops that review (see **When.**). **Only the user switches it.** The lead agent has no say in it: the planner prompt says nothing of it, and a design never changes it (a reply that names it changes nothing). A switch holds for the question being answered (if the team has not answered yet) and every later one in the chat, until the user switches it again; it is not remembered for new chats, which start with it on. @@ -217,11 +217,11 @@ With `settings.leadReview` (any conversation, with or without auto team), the le ## Schedules -A conversation can have one **schedule** (`conversation.schedule`, a `ConversationSchedule`, in the `schedules` table): a message the team is sent at set times, in the same conversation or (`newChat`) in a new one each time, as if the user had sent it. It is the first step of the [triggers proposal](proposals/triggers.md), with one schedule per conversation, and the option of a new conversation each time from its third. +A conversation can have one **schedule** (`conversation.schedule`, a `ConversationSchedule`, in the `schedules` table), which the UI and the user guide call an **automation** (自動任務; **New automation**, a chat's **Automation**, the **Automation** tag): only the words shown changed, not the API, the code or the data. It is a message the team is sent when something starts it (**Run once now**, set times, its webhook), in the same conversation or (`newChat`) in a new one each time, as if the user had sent it. It is the first step of the [triggers proposal](proposals/triggers.md), with one schedule per conversation, and the option of a new conversation each time from its third. -- **When.** `repeat` is once (a moment), every day at `HH:MM`, on chosen days of the week at `HH:MM`, or every 1–168 hours from when its times were last set (saving a new message or other choices keeps the cadence). Times of day are wall-clock times in the schedule's time zone (the browser's when it was saved; the server's when none is given). `nextRun()` (`src/shared/schedule.ts`) works out the next time: a time the clocks skip moves forward by the jump, one they repeat is the first. The browser uses the same function to show the next three times before saving, and shows a schedule's times in its own time zone (with a button to switch to the device's). +- **When.** `repeat` is once (a moment), every day at `HH:MM`, on chosen days of the week at `HH:MM`, every 1–168 hours from when its times were last set (saving a new message or other choices keeps the cadence), or none (`manual`: no set times; `nextRun()` gives none, so only **Run once now** and the webhook send it, and the UI saves it on, with no **On** and no missed-times choice; a new schedule's form starts with it). Times of day are wall-clock times in the schedule's time zone (the browser's when it was saved; the server's when none is given). `nextRun()` (`src/shared/schedule.ts`) works out the next time: a time the clocks skip moves forward by the jump, one they repeat is the first. The browser uses the same function to show the next three times before saving, and shows a schedule's times in its own time zone (with a button to switch to the device's). - **Saving** (`PUT /api/conversations/:id/schedule`, checked by `validateSchedule()`) replaces the schedule, works out `nextRunAt` (none while it is off), records the UI language for the notes it writes, and keeps how it went last time. A schedule that is on needs a team: not while the lead agent still has to design the first one or the user to confirm one (409). Scheduled messages go to the current team: in an [auto-team](#auto-team) conversation the lead agent designs no team for them, since nobody may be there to confirm it. One that comes due while the lead agent reviews the team's replies waits for the review, and the current team answers it then (see [Lead review](#lead-review)). -- **New schedule** (the sidebar's **New schedule**, `POST /api/schedules`) creates a conversation with auto team and its schedule in one step — the schedule on, the title from the message's first line unless one is given — leaves a note, sorts it into the user's groups by the message, and has the lead agent design the team for the schedule's message at once, while the user is there to confirm it (`Orchestrator.designForSchedule()`; `POST …/schedule/design` asks again, e.g. after a design was stopped). That design run's question is the message itself (`Run.designFor`), and nothing is posted; the planner prompt says the team gets it on a schedule (here, or in a new conversation each time) with nobody watching. Its plan is marked `forSchedule`: once it is confirmed — on the card, by setting the team by hand, or at once with `autoConfirm` — the team answers nothing now (`startConfirmed()` only starts what the user wrote while the design waited); it gets the message when the schedule sends it. A time that comes due while that first team is designed or waits for confirmation is `busy`: tried again for up to 30 minutes, then skipped (no confirmed team). The plan card and the schedule drawer word this design for the schedule (`planWording()` in `src/web/copy.ts`). A name the scheduled message @mentions that is not on the team reaches nobody (the members it does @mention answer, or with none, the whole team): the schedule form warns as the message is typed (and, with no team yet, says the lead agent names the members after the mentions), the team editor warns when a member it @mentions is renamed or removed, and in a chat with a schedule the card of any team design warns before it is confirmed, or when it was used at once (`mentionWarning()` and `planChecksMentions()` in `src/web/copy.ts`, on `unknownMentions()`; a name followed by `/` or `(` is code, such as `@types/node` or `@Component(…)`). +- **New automation** (the sidebar's **New automation**, `POST /api/schedules`) creates a conversation with auto team and its schedule in one step — the schedule on, the title from the message's first line unless one is given — leaves a note, sorts it into the user's groups by the message, and has the lead agent design the team for the schedule's message at once, while the user is there to confirm it (`Orchestrator.designForSchedule()`; `POST …/schedule/design` asks again, e.g. after a design was stopped). That design run's question is the message itself (`Run.designFor`), and nothing is posted; the planner prompt says the team gets it on a schedule (here, or in a new conversation each time) with nobody watching. Its plan is marked `forSchedule`: once it is confirmed — on the card, by setting the team by hand, or at once with `autoConfirm` — the team answers nothing now (`startConfirmed()` only starts what the user wrote while the design waited); it gets the message when the schedule sends it. A time that comes due while that first team is designed or waits for confirmation is `busy`: tried again for up to 30 minutes, then skipped (no confirmed team). The plan card and the schedule drawer word this design for the schedule (`planWording()` in `src/web/copy.ts`). A name the scheduled message @mentions that is not on the team reaches nobody (the members it does @mention answer, or with none, the whole team): the schedule form warns as the message is typed (and, with no team yet, says the lead agent names the members after the mentions), the team editor warns when a member it @mentions is renamed or removed, and in a chat with a schedule the card of any team design warns before it is confirmed, or when it was used at once (`mentionWarning()` and `planChecksMentions()` in `src/web/copy.ts`, on `unknownMentions()`; a name followed by `/` or `(` is code, such as `@types/node` or `@Component(…)`). - **Sending.** `Scheduler` (`src/server/scheduler.ts`, started once the server listens and stopped first on shutdown) looks every 15 seconds — an unref'd interval compared with the wall clock, so a computer that slept sends on waking — for schedules that are on and due (`Store.dueSchedules()`), the most overdue first, one at a time, inside `withLang(schedule.lang)`. `Orchestrator.sendScheduled()` then: - holds the message back (`skipped`, with the reason) while the team cannot take it: no confirmed team; an agent's question waiting for the user's answer, with no run carrying the answer being set up (the message would be taken for the answer); an earlier scheduled message still in the queue of a team that is replying (they do not pile up); a full queue; - answers `busy` while the lead agent designs a team that will wait for confirmation (a design run under way, no `autoConfirm`): queued then, the message would go into that design's question and wait with it, so it is tried again later (`scheduleDesigning`), and once the design waits for confirmation, its times are skipped until the team is confirmed (no confirmed team, above); @@ -242,6 +242,9 @@ A conversation can have one **schedule** (`conversation.schedule`, a `Conversati - **Missed times.** A time more than 10 minutes past (`MISSED_AFTER_MS`: Harness Relay was not running, or the computer slept) is skipped with a note (`missed: 'skip'`, the default) or sent late (`'run'`) — once, however many times were missed. - **Run now** (`POST /api/conversations/:id/schedule/run`) sends the saved message the same way, without changing the schedule's times or what it recorded. - The `conversation` event carries the schedule, so an open chat updates its header badge; the conversation list carries `schedule` (on, next time, last result) for the sidebar's clock, which turns amber when the last time was not sent. +- **A tracked goal** (`goal`, at most 2,000 characters; `tracking`, a `ScheduleTracking`; schema version 32). Every turn of a run it works on gets `trackingSection()` at the end of the briefing's chat rules (`src/server/prompt.ts`): the goal and its status in a `` block, the latest 12 of the notes (`TRACKING_LIMITS.shown`) in ``, each on one line with the blocks' tags in it written so they end or start nothing (`dataBody()`), and called agents' reports, data, never instructions, and the rule to end a reply to a scheduled message with `…`. Which runs: those a tracking schedule starts, here or in a chat it opened (`Run.tracking`, `trackingSource()`), so a new chat each time knows what earlier runs found, and those that carry one on (`carriedTracking()`): the schedule's message that waited in the queue (its own time's run, `ChatMessage.tracked`), the user's answer to a question one asked (`RunPause.tracked`), a Retry of one of its replies and Continue after them (each reply keeps its run, `messages.tracked`, schema 34). A run the user's own message starts is none: a reply to "how is CI?" reports nothing. A retry the scheduler planned is not made once a report came after the time that failed (the user's Retry of it, say). When a turn ends, `takeProgress()` takes the block off the end of the reply (before or after an `[ASK_USER]` marker, also inside a code fence; `splitProgress()` in `src/shared/tracking.ts`) and keeps it as a `progress` part, which the chat draws as a card (and hides while the reply streams, `withoutProgressTail()`); `recordProgress()` adds a note (`at`, `status`, `text` cut to 1,000 characters, `agent`, and `chatId` when written in another chat) and keeps the latest 30. `done` switches the schedule off and clears a retry, with a note in both chats; `blocked` keeps it on, with a note, and marks the conversation as needing the user (`ConversationSummary.blocked`: its latest agent reply has a `blocked` progress part, so a later reply that says otherwise clears it), an alert of its own (`blocked`) and the sidebar's **Goal held up**. A block in a conversation without a goal stays in the reply's text. Saving the schedule (`PUT`) keeps the notes while the goal stays the same (a reached goal is `working` again once the schedule is on), starts afresh for a new goal, and drops them with the goal. +- **Retries.** A tracking schedule's run is followed from `sendScheduled()` (`TrackedRun`: the time it came due, which try, and whether it is retried) to its end. While it runs, `tracking.inFlight` says so in the database. At its end (`trackedRunEnded()`, from the run's clean-up): when it is still the schedule's run in flight (switching the schedule off or giving it another goal takes it out of flight, and so does the user's **Stop**), every agent reply in it failed, and it was not stopped, `retryTracked()` sets `tracking.retry` (`{ due, attempt, at }`) and `nextRunAt` to 5, 30 or 120 minutes later (`TRACKING_LIMITS.retryMinutes`), unless the schedule's own next time comes first; after the third, a note says it waits for its next time. A one-time schedule is switched on again for its retry, but not one the user switched off meanwhile; one whose goal is reached is not retried. Only the schedule's own times are retried (`sendScheduled(…, { retries: true })`, from the scheduler): **Run now** and a webhook's call are not, so a one-time schedule keeps its own time. The scheduler sends a retry as the time it came due for (`sendScheduled(…, retry.due, { attempt })`), and `record()` drops the retry it used. A run that could not start (`failed`) is retried the same way. A server that stops (quit, restart, the desktop app's update) leaves its runs in flight (`trackedRunEnded()` does nothing while the orchestrator closes), and so does a crash: at startup, `Scheduler.recover()` treats a run still `inFlight` as cut off by the stop and retries it at once (`trackingRestartNote`). A scheduled message that waited in the queue of a team that was replying (`queued`) is not followed: it is not retried if all its replies fail. The orchestrator's `clock` is the one the notes and retries go by. +- **Webhook** (step 2 of the [proposal](proposals/triggers.md), for the conversation's schedule). `POST /api/conversations/:id/schedule/hook` makes a token (32 random bytes, base64url), returns it once with the address (`http://127.0.0.1:/api/hooks/`), and keeps only its SHA-256 (`schedules.hook_hash`, which no conversation object carries; `schedules.hook` keeps when it was made and last called, `ScheduleHook`). A new token replaces the old; `DELETE` removes it, and so does deleting the schedule. New automation's **Create a webhook too** makes it in the same request (`POST /api/schedules` with `hook: true`, its answer's `newHook`, by the same code); the page hands the token to the automation's chat in its memory only (`handOverHook()` in `src/web/schedule.ts`, never the address or the browser's storage), which opens the drawer to show it once. `POST /api/hooks/:id` with `Authorization: Bearer ` and a JSON body is the one route a signed-in browser's cookie and key are not asked of: the token is compared, as a hash in constant time, and a missing or wrong one, or no schedule, is a 404 like an unknown conversation. The Host, Origin and JSON checks still apply. Its body (at most 20 KB, `413` above; pretty-printed and cut to 20 KB) goes with the schedule's message, sent at once as **Run now** would (also while the schedule's times are off), as `ChatMessage.event` (`messages.event`), which the agents get after the message's text as `…` and the briefing calls data, never instructions. One call per 10 seconds per webhook (`429`); a call to a schedule whose tracked goal is reached and that is off is refused (`409`); one the conversation cannot take now answers `409` with the reason, like **Run now**. A webhook's run is never retried, since the retry would not have its body. ## Sidebar groups and search @@ -257,7 +260,7 @@ Search (`GET /api/search?q=`) splits the query on whitespace (at most 8 words) a ## Notifications -Besides the sidebar, a browser can be told when a conversation's team finishes its reply (`done`, or `error`) or needs the user (`approval`, `confirm`, `answer`), with a system notification and a sound (**Settings → Notifications and sound**, kept per browser in `localStorage`, `relay:alerts`), and the server keeps every such alert in the notification inbox behind the sidebar's bell, the same for every browser. What counts is one rule, `AlertTracker` in `src/shared/notify.ts`, which the page and the server both apply to the conversation lists one after another: something the conversation needs that it did not before (not what it needed when a run began, back once that run is over with no reply of its own: the run that takes the user's answer could not be set up, say, and the question still waits; the tracker remembers it with the run's start, `runFrom.need`); else, when a run ended with an agent reply the conversation did not have when the run began, how that reply ended (a stopped one is no news). The tracker remembers the latest reply when it sees a run begin (`lastReplyId` in the summary; for a run already going at the first look, its latest reply unless that one is still being written), so a run that ended without a reply of its own (the lead agent designing a team, a run stopped before anyone answered) tells of nothing, and a run that began and ended between two looks counts by the new reply it left. A run counts as running from the moment it is set up (`isRunning()`: `starting` too, before its workspace is prepared), so a run that follows another at once (a message sent during the lead agent's review or with "stop and send this instead", what waited for a run of its own, the team the lead agent just designed) leaves no look at which the conversation seems done: one alert, once the last of them is over. The first list is where it starts: nothing before is news, and a branch seen for the first time is none either, also when it waits for the answer to the question it was branched at. `alertDetail()` names the agent and what it said (its reply's start, the command an approval waits on, the question asked), as one short line. +Besides the sidebar, a browser can be told when a conversation's team finishes its reply (`done`, or `error`) or needs the user (`approval`, which an item waiting in the outbox is too, `confirm`, `answer`, `blocked`), with a system notification and a sound (**Settings → Notifications and sound**, kept per browser in `localStorage`, `relay:alerts`), and the server keeps every such alert in the notification inbox behind the sidebar's bell, the same for every browser. What counts is one rule, `AlertTracker` in `src/shared/notify.ts`, which the page and the server both apply to the conversation lists one after another: something the conversation needs that it did not before (not what it needed when a run began, back once that run is over with no reply of its own: the run that takes the user's answer could not be set up, say, and the question still waits; the tracker remembers it with the run's start, `runFrom.need`); else, when a run ended with an agent reply the conversation did not have when the run began, how that reply ended (a stopped one is no news). The tracker remembers the latest reply when it sees a run begin (`lastReplyId` in the summary; for a run already going at the first look, its latest reply unless that one is still being written), so a run that ended without a reply of its own (the lead agent designing a team, a run stopped before anyone answered) tells of nothing, and a run that began and ended between two looks counts by the new reply it left. A run counts as running from the moment it is set up (`isRunning()`: `starting` too, before its workspace is prepared), so a run that follows another at once (a message sent during the lead agent's review or with "stop and send this instead", what waited for a run of its own, the team the lead agent just designed) leaves no look at which the conversation seems done: one alert, once the last of them is over. The first list is where it starts: nothing before is news, and a branch seen for the first time is none either, also when it waits for the answer to the question it was branched at. `alertDetail()` names the agent and what it said (its reply's start, the command an approval waits on, the question asked), as one short line. - **The page** (`src/web/alerts.ts`, `PageAlerts`, fed the list the sidebar polls every 3 seconds; list answers older than the one shown are dropped) does nothing for the conversation it shows with its window focused; in front, it plays the sound (Web Audio, two notes for a reply, three for a need; the browser plays it only once the user did something on the page since it loaded) and shows a notification when its window is not focused; in the background, it shows the notification and plays the sound only when its browser gets no push notifications, 3.5 seconds later. One tab of a browser handles each alert (`claimAlert()`, in `localStorage`, by the conversation, the kind and the reply): a tab in front claims every alert, also one it says nothing of, so a tab in the background, which waits, does not repeat it. Its notifications go through the service worker where there is one (Chrome on Android shows no other kind), else `new Notification()` (the desktop app's window, whose permission handler allows notifications for the server's pages). - **Push** (`src/server/push.ts`, `src/server/alerts.ts`) reaches a phone, or a browser with no page open. Turning notifications on registers the service worker (`src/web/public/sw.js`, written by hand and copied into the build) and subscribes with the browser's push service under this server's VAPID key (`GET /api/push`), right after the browser's question for permission, which is asked in the tap that turned them on (Safari wants one for it); the page gives the server its subscription and choices (`PUT /api/push/subscription`, again at each load, change, return to the front and every 10 minutes, so a subscription the server dropped comes back). Subscriptions (`push_subscriptions`: the push service's address, the browser's P-256 key and auth secret, its choices and language) belong to a signed-in browser's session and are removed with it (`ON DELETE CASCADE`), and none is sent to once its session is past its 400 days; another signed-in browser cannot take one over (409, `code: "push-taken"`: a browser signed in anew while its old session lives on subscribes anew), and a browser keeps its latest five. While some browser has subscribed, `AlertWatcher` checks every 1.5 seconds, reading the conversations only when a conversation event came (`Hub.subscribeAll()`) or agents are at work (`Orchestrator.busy()`), and sends each subscriber the alerts it wants: the payload (title, body, tag, the chat's URL, whether silent) encrypted for that browser (RFC 8291, `aes128gcm`), signed with the server's key (RFC 8292, ES256), with `Urgency` (high for a need), `TTL` (a day for a need, six hours for a reply) and a `Topic` per conversation, all with `node:crypto`; sends do not hold up the next check. A subscription its push service says is gone (404, 410) or no longer takes the key (401, 403) is removed, and refused when the page gives it again (`code: "push-gone"`), so the page subscribes anew. The service worker shows every push it gets (Safari takes away a subscription whose pushes show nothing), opens the chat on a click, and subscribes again with the same key when the push service replaces a subscription (`pushsubscriptionchange`; the page gives the server the new one when it is next opened). @@ -429,6 +432,18 @@ MCP servers, skills and memory that Harness Relay gives its agents, the same way The open agents get the library too: MCP servers in ACP's `mcpServers` or in the config their adapter writes (Pi through its MCP extension), skills and memory in the briefing, as for every agent; each agent's section in [Open agents](open-agents.md) says how. +### Harness Relay's own tools (the gateway) + +Tools Harness Relay serves itself, for every CLI the same way (`src/server/gateway.ts`, `ToolGateway`). They come in groups (`RELAY_TOOL_GROUPS`: `browser`, `mail`, `calendar`), stored with the library (`relayTools`: the CLIs whose agents get each, none at first) and given per agent like a library item (`relay:` in `AgentConfig.tools`); `forAgent()` returns the agent's `relayGroups`. + +- **The MCP server.** At startup the server listens on a socket (`gateway/relay.sock` in its data folder, in a folder only the owner may enter; a named pipe on Windows). A turn whose agent has a group gets a grant (`grant()`): a 32-byte token of its own and a copy of a small program, the shim (`GATEWAY_SHIM_SOURCE` in `src/server/gatewayShim.ts`: plain JavaScript with Node's built-ins), each in a file of the grant's own (`gateway-.token`, `gateway-.mjs`) in Harness Relay's own temporary folder (`data/tmp`, which no sandbox may read or write on macOS and Linux: a command can neither read the token nor change the program a CLI starts outside its sandbox; on Windows, which gives no sandbox protected paths, each token serves one connection instead: the server deletes its file once the CLI's shim has shown it, and refuses a second connection with it, `claimOnce`; the CLIs start their MCP servers as their session starts, before any command of the agent's runs), and an MCP server spec `relay` (stdio: the Node.js that runs the server, or in the desktop app the one Harness Relay's own sandbox host uses, `gatewayNode()`; the shim's path; `RELAY_GATEWAY_SOCKET` and `RELAY_GATEWAY_TOKEN_FILE` in its environment), added to the agent's MCP servers (`RunOptions.mcpServers`), so every adapter passes it on as it passes library servers. The shim answers MCP's `initialize`, `ping`, `tools/list` and `tools/call` on stdio, newline-delimited, and passes the last two over the socket after the token; the token ends with the turn (`revoke()`, both files deleted; what a stop of the server left is removed at the next start with the runs' other files, `RUN_TMP_PREFIXES`), and a call with a token the server does not know is answered with an error. A line that is no request (JSON `null`, an array) is answered as an invalid request, and a call over 1 MB is refused by the shim (`GATEWAY_MAX_CALL`), so neither ends the shim or its socket; a connection that shows no token within 10 seconds, or sends more than 512 bytes before it, is ended; one whose token is no turn's (or, on Windows, is served already) gets its first request answered with why and is ended, or ended after those 10 seconds if it asks nothing; at most 256 connections of turns going on are open at once, and 256 others apart from them, so these cannot keep the shims out; stopping the gateway ends them all. Grok, Muse Code, Goose, Cline, OpenHands and Pi get it in full access only (`GATEWAY_FULL_ACCESS_ONLY`): below it the sandbox that holds their commands (Grok's and Muse Code's own, Harness Relay's around the open agents) holds the MCP servers they start too, so it could open the program and its token to the one only by opening them to the other, and it does not open the socket either. +- **Over loopback HTTP** (`GATEWAY_OVER_HTTP`: OpenCode, Codex and Copilot, [#88](https://github.com/Agentic-System-Lab/harness-relay/issues/88)). The gateway also listens on `127.0.0.1`, at a port the system picks (`httpPort()`), and answers MCP's Streamable HTTP at `/mcp` with JSON (`initialize`, `ping`, `tools/list`, `tools/call`, and batches of them; a notification gets 202). For these CLIs a grant (`grant(…, { http: true })`) is an HTTP MCP server: its URL, and the turn's token in an `Authorization: Bearer` header, which reaches the CLI only over its stdin (OpenCode's and Copilot's ACP `session/new` and `session/load`, Codex's thread config), so only the CLI's memory holds it: no program, no file and no Node.js. A request is refused unless it is a `POST` of `application/json` (at most 4 MiB) whose `Host` is `127.0.0.1:` or `localhost:`, with no `Origin` (a page in the user's own browser can reach 127.0.0.1) and a token granted for HTTP (a socket's token there, or an HTTP one on the socket, is unknown: 404), and a refusal ends its connection; the tiers and revocation are the socket's. A request must come at once (its headers within the token's 10 seconds, all of it within 30; its answer may wait for the user as long as it takes), and connections that have shown no turn's token count apart, at most 256, the oldest giving way to a new one, so they cannot keep the CLIs' own out. An act the user allows after the CLI stopped waiting for it (its request closed: its own time limit) is not done, since the agent was told it failed and may try again; Codex gets a turn's time for the gateway's calls (`tool_timeout_sec`), not its 5 minutes. Harness Relay's own browser refuses the port, as it refuses the server's. A CLI that turns out to take no HTTP server gets the stdio program instead, made only then (`McpServerSpec.fallback`), where it could run (never below full access in a sandbox that holds its MCP servers). Below full access OpenCode runs in Harness Relay's own sandbox, which opens it that port alone (`connectPorts`). Checked with the real CLIs, signed in (OpenCode 1.19.5, Codex 0.169, Copilot 1.0.112, `scripts/mcp-headers-check.ts`): every request carried the header, and no file they wrote (sessions, rollouts, databases, logs) held it, after a new session, a turn and a resume (Copilot's resume refused in the same process, which keeps the session loaded); a command in Harness Relay's own sandbox cannot read its agent's memory, on Linux or macOS (`scripts/sandbox-check.ts`). Copilot has no other way: over ACP it takes only HTTP and SSE MCP servers ("Rejecting non-http/sse MCP server" in its log), so the stdio program never reached it; below full access its MCP client reaches the port from outside the sandbox that holds its commands. The others stay as they are until that check passes for them: Grok passes it, signed in, but below full access its sandbox holds its MCP servers, and whether its HTTP client would reach the port from there is not checked (full access only); Muse Code's sandbox holds them too, an HTTP server's connection included (full access only); Cursor passes it too but keeps the stdio program, which works for it; Antigravity fails it, since it writes an HTTP server's headers to its conversation database (`~/.gemini/antigravity-cli/conversations/.db`), so it keeps the program, which is given the token file's path, never the token; Muse Code's is inconclusive (it opened no connection without a prompt); Goose keeps the MCP servers a session was given, to start them again when it is loaded, in its session database, which its commands can read; OpenHands writes them to `mcp.json` when it resumes; Cline and Pi take none Harness Relay could give them so; Claude Code reads its MCP servers from a file. +- **Tiers** (`GatewayTier`, `gatewayDecision()`). `read` runs. `act` runs in full access; below it, it is asked in the chat through the turn's own `requestApproval` (kind `other`, tool `relay:`, `gatewayApprovalTool()`, which keys it: "always allow" holds for that tool in the conversation, and for no other server's tool of the same name; a run that refuses approvals refuses it unless it is always allowed; allowing it takes no folder lock, `noLock`). The CLI's own question about one of the gateway's tools (a request of kind `other` whose tool is exactly `mcp__relay__`, Codex's `relay:`, OpenCode's `relay_`, Antigravity's `mcp_relay_`, `relay__`, `relay/` or `relay.`: `gatewayToolNamed()`; never a command, path or address that holds such a name; since OpenCode's one underscore could join another server's name and its tool into the same words, no library server may be named `relay` or `relay_…`, and a run that has the gateway is given none named so, `reservedMcpName()`) is answered yes by the turn, with no card and no folder lock, also in a run that refuses approvals or a conversation that refuses every request (`cliApproval()` in the orchestrator): the gateway then decides by the tier as the call reaches it. The CLI's own card would show only the call's arguments, and a scheduled run would refuse even a read. `send` never runs in the turn, in any permission: it goes to the outbox. +- **The outbox** (`src/server/outbox.ts`, the `outbox` table, schema version 33). A `send` call adds an item (the tool, its arguments, a title and a preview of what it sends, `OutboxPreview`) and its card to the reply (an `outbox` part), and returns at once, telling the agent it waits for the user; the same tool with the same arguments while one waits in the conversation is that one. `POST /api/conversations/:id/outbox/:itemId` with `{ action: 'send' | 'discard' }` answers it: Send runs the tool's `send()` with the arguments as they were given, once (the item is `sending` meanwhile, so a second answer is a 409), and keeps the result; the part and a note in the chat say what happened. A conversation with an item waiting counts as waiting for the user (`isWaiting()`), so the sidebar and notifications treat it as an approval. At startup an item still `sending` is marked `failed` (whether it went out is not known) and never sent again, and its card says so (`reconcileOutbox()`). A reply taken back (**Retry**, **Resend**, an edited message: `truncateFrom()`) discards what it had waiting (`discardFor()`), so no item waits without a card and the reply that comes again asks anew; a branch copies no waiting card, whose item stays with the conversation it came from. Agents read the outbox with `outbox_status` (`outboxTool()`, given with any group). +- **The browser** (`src/server/browser/`). One Chromium per conversation (`BrowserService`), started through `process.ts` (`runProcess` with two pipes, fds 3 and 4) with `--remote-debugging-pipe`, so no debugging port is opened, headless, with a profile in `browser/` in the data folder (removed with the conversation), downloads denied, file choosers intercepted, dialogs dismissed and page preloading off (`network_prediction_options` in the profile's preferences: no speculation-rules prefetch or prerender, which the browser makes outside the page; a prerendered page is closed all the same); its environment only what it needs to start (`browserEnv()`: home, the temporary folder, the language, the time zone, Windows' system folders), so no key, token or proxy setting of the server's reaches it, and on Linux its config and cache folders in its profile (`launchEnv()`: Chromium there keeps its crash reports under `~/.config/` whatever its profile, the user's own browser's folder); Chromium's own sandbox is left on, but as root on Linux, where Chromium cannot start with it. Which browser: the one set in Settings → Tool library (`browser` in the tool library, checked to be a file when saved), else `RELAY_BROWSER`'s, else the system's Chrome, Chromium or Edge, kept up to date by itself or the system's packages, else Playwright's newest Chromium (`findBrowser()`). The CDP client (`cdp.ts`) sends NUL-ended JSON and matches answers by id. Every request pauses (`Fetch.enable` for every resource type), in the tab and in what it starts in processes of its own (another site's frames, workers: attached before they start, `Target.setAutoAttach` with `waitForDebuggerOnStart`, gated, then let go), and a host name's addresses are looked up first (`dns.lookup`, kept a minute): `navigationVerdict()` refuses anything but http and https, Harness Relay's own ports (and the proxy's) on a loopback or own address and link-local, multicast, unspecified and metadata addresses, also behind a name that leads to one. A main-frame page on a host the session has not opened is asked about through the call's `ask` (kind `fetch`, keyed by host and port; allowed at once in full access; refused when nobody can be asked). An address on this computer or the local network (loopback, own, private, shared and unique-local ones: `leadsNearby()`) is reached only on a host the session opened while it led there (`nearby`): any other request there (`requestVerdict()`: a frame, an image, a script's fetch, a WebSocket) is refused, so a page cannot reach the user's local services by itself, and a host allowed while it led to the internet that now leads here is `moved`, refused even in the tab (DNS rebinding). A name whose addresses lead both to the internet and here is `mixed`, never opened, asked about or not (a connection falls back from one address to the next, so allowing it as one would let its pages reach the other); the question about a page here says so (`openNearbyReason`). A request to the internet goes unless it is refused so. **Every connection goes through the browser's own proxy** (`proxy.ts`, `BrowserProxy`: `--proxy-server` on 127.0.0.1, `--proxy-bypass-list=<-loopback>` so this computer's addresses too, `--force-webrtc-ip-handling-policy=disable_non_proxied_udp` so WebRTC keeps to it): http requests passed on, CONNECT tunnels (https, wss, ws) and upgrades, each checked by `requestVerdict()` and connected to an address from the same lookup the gate used, never one Chromium looked up, since it looks up nothing behind a proxy. The proxy listens on loopback only and serves only the browsers Harness Relay started, each with a password of its own (proxy Basic authentication, given through CDP's `Fetch.authRequired`; a site's own challenge is cancelled); anything else gets a 407. A window the page opens (a link with `target="_blank"`, `window.open()`) is attached by the browser-wide auto-attach, held before it starts where the browser holds it (Chromium; Google Chrome sometimes lets a script's window make its first request first, which goes through the proxy like any connection and so reaches nothing a page could not), every request of its refused, and closed: the browser keeps to one tab, and the agent is told, to open the page with `browser_open` if it needs it. A refused main-frame page leaves Chromium's error page, so the browser goes back to the page it was on, and the agent is told why (the user declined, it is never opened, or nobody could be asked: `refusalText()`). The browser's own parts (its UI pages, its built-in extensions) are left alone. Calls on a conversation's browser run one at a time, in the order they came, since its agents share its one tab; an address in the page open now (`#anchor`) loads nothing and is not waited for. A snapshot (`SNAPSHOT_SCRIPT`) runs in an isolated world of its own (`Page.createIsolatedWorld`), which the page's script can neither see nor change the functions of: the visible text with each link, button and field numbered (a password field's value never read, nor taken for its label), at most 20,000 characters, checked and cut again in the server. The elements stay in that world (`__relayRefs`), never marked in the page, and numbers go on from one snapshot to the next, so an old one names nothing: a click or typing goes to the element the latest snapshot found (an element gone, or a page that changed since, is an error). A click is a real mouse event at the element's middle, refused when another element lies over it there (`elementFromPoint()`), which would get it instead; typing goes through `Input.insertText` (and an Enter key). A page a click or an Enter opens is waited for from the moment the main frame starts loading (`afterInput()`), however late its server answers and while the user is asked about its host, until it is there and has loaded (at most 15 s more), or until the frame stops loading without one (a download, an answer with no page, a refused page): a slow server's page is never read half loaded. The question about an `act` call names the element as the page does, from the latest snapshot (`refInfo()`, `refWords()`: a link with its address, a button, a field with its name), never the agent's words, and a typing question shows the whole text (up to 2,000 characters, with how many in all). Which hosts a call may open again without asking (`grantsFor()`): in full access, all the browser opened; below it, those the user allowed (or an "always allow" did), not those full access opened without asking; in a scheduled run, only those the call itself opened, through its question, which lets only what is always allowed through; a screenshot is a PNG in the conversation's attachments folder, which the media endpoint serves, also given to the agent as an image. The browser closes after 10 minutes with no call going on or waiting for the user's answer (a call holds it, `hold()`, also while the user is asked about a host, and so does the gateway's question about a click or a key, `GatewayTool.asking`), when the conversation is deleted, and on shutdown. +- **Mail and calendar** (`src/server/connectors/`). The accounts (`ConnectorStore`, `app_settings` key `connectors`, without passwords; each password in the credential store as `mail-` or `calendar-`, masked in `GET /api/connectors`) are checked by `validateMailAccount()` and `validateCalendarAccount()`: TLS or STARTTLS, plain only to a loopback host; https, http only to a loopback host, no credentials in the address; never one of Harness Relay's own ports on this computer. The protocols are Harness Relay's own, without a library: `imap.ts` (LOGIN, LIST, EXAMINE, UID SEARCH, UID FETCH and APPEND, literals both ways, STARTTLS), `smtp.ts` (EHLO, STARTTLS, AUTH PLAIN or LOGIN, the envelope, DATA with dot-stuffing); both refuse an answer to STARTTLS that has more after it (`STARTTLS_EXTRA`), which came before TLS, from anyone on the way. `mime.ts` (a plain-text UTF-8 message with RFC 2047 headers folded so no line is long, never a Bcc header; reading one: its text/plain part, else its HTML as text, in its charset and transfer encoding, attachments by name), `caldav.ts` (`REPORT calendar-query` for a time range, `PUT` with `If-None-Match: *`, discovery through `current-user-principal` and `calendar-home-set` (whatever namespace prefix and attributes the answer's elements carry: iCloud writes ``), leaving out a calendar whose `supported-calendar-component-set` has no `VEVENT` (iCloud's Reminders lists), staying on the same provider: `sameProvider()`, the same host, or one under the domain the given host is in unless that looks like a country's public one (`co.uk`), an address only as itself, never from https to http; redirects not followed; events written in UTC from a wall-clock time in a time zone, their text with every line break, a bare CR too, written `\n`, and lines folded at 75 octets between characters; an all-day event's end shown as its last day, as `calendar_create` takes it). A password left empty when an account is saved again is kept, but only for the same servers (`sameMailServers()`, `keptPasswordFits()`): a mail account whose IMAP or SMTP host changed, or whose security became none, a calendar whose address is not its provider's, and discovery at another provider's address need it typed again. A connection serves one call. What a server sends is limited: an IMAP answer to 16 MB with its literals and untagged responses (a literal announced larger is refused), an SMTP reply to 256 KB, a calendar's answer to 16 MB; reading a mail's HTML takes time in proportion to it, whatever it holds. An address may name no other: its name holds no `<`, `>`, `"`, `@`, backslash, control or formatting character (`validAddress()`), so the card shows exactly who a mail goes to, and the envelope takes that address. An outbox item goes out from the account its card showed, found by its id alone (`accountId`; gone, nothing is sent). Passwords are shown as there or not, never any of their characters. The tools (`connectors/tools.ts`): `mail_search`, `mail_read`, `calendar_events` (read), `mail_draft`, `calendar_create` (act), `mail_send` and `calendar_invite` (send, through the outbox: sent by `send()` on the user's Send). What they return is marked as data from the user's mailbox or calendar. `POST /api/connectors/:kind/:id/test` signs in (IMAP and SMTP, or a calendar query) without sending anything; `POST /api/connectors/calendar/discover` lists an account's calendars. +- What a tool returns is text (and images) the agent is told is data from a page, a mail or a calendar, never instructions, in a block nothing in it can end (`dataBody()` in `gateway.ts`: the block's tags, closing or opening, written with a backslash after their `<`, `<\/…`; `dataLine()`: a row's or header's line breaks as spaces; `dataAttr()`); the credentials a tool uses stay in the server. + ## What each CLI loads by itself Each account of each CLI in Settings can list what the CLI loads by itself in that account's home for the default working directory — its own skills, MCP servers and their tools, built-in tools, hooks, subagents, plugins, instruction and memory files — separate from the tool library (`src/server/native.ts`, `POST /api/providers/:provider/native[?account=]`). Every account has a home of its own (see [Accounts](#accounts)), so each one loads its own and is asked on its own. The CLI is asked the way an IDE would ask it, without a prompt, so no model is called. Cursor has no such question over ACP, so its lists come from the files it is known to read (`probeCursor`): no CLI is started and nothing is spent, and every Cursor account lists the same, since they are read from the home folder, which its config folder does not move. Antigravity is asked with commands it answers in print mode without a model, a conversation or quota (`agy -p /skills`, `/hooks` and `/agents` with `--output-format json`, and `agy plugin list --json`), in the account's home (given as `HOME`); signed out they fail with one note. Its MCP servers and instruction files come from the files it reads. Copilot is asked with its own list commands, which open no session (`copilot mcp list --json`, `skill list --json`, `instruction list --json` and `plugin list --json`, in the account's home and the working directory); its hooks and custom agents come from the files it reads. Muse Code lists its skills with `muse skills list --json`, which starts no session, in a config folder of the probe's own made as a run's is (`prepareMuseRun()`, read-only, with what the account leaves out), and its MCP servers and hooks come from its `settings.json`: @@ -562,15 +577,15 @@ When the stream drops, the browser keeps retrying (and shows "reconnecting") unl ## Persistence -Everything except API keys (in the OS credential store, or in the database only encrypted, with their key in `data/secrets.key`) and attached files lives in one SQLite database, `data/relay.db` (WAL mode, foreign keys on, `secure_delete` on so removed rows are overwritten, file mode `0600`). Attached files are stored as `data/attachments///` (mode `0600`) and referenced by `attachment` parts in `message_parts.data`; they are removed with their conversation, or when a resend or retry drops the messages they belonged to. +Everything except API keys (in the OS credential store, or in the database only encrypted, with their key in `data/secrets.key`) and attached files lives in one SQLite database, `data/relay.db` (WAL mode, foreign keys on, `secure_delete` on so removed rows are overwritten, file mode `0600`). Attached files are stored as `data/attachments///` (mode `0600`) and referenced by `attachment` parts in `message_parts.data`; they are removed with their conversation, or when a resend or retry drops the messages they belonged to. The browser tools' screenshots are files of the same folder (`browser-