diff --git a/.agents/skills/codexhost-release/SKILL.md b/.agents/skills/codexhost-release/SKILL.md index 53cc20e17..347156c04 100644 --- a/.agents/skills/codexhost-release/SKILL.md +++ b/.agents/skills/codexhost-release/SKILL.md @@ -17,6 +17,7 @@ description: 发布 codexhost 正式版、预览版,编写或确认 Release No - `.github/workflows/release-packages.yml` 与 `.github/workflows/ci.yml` - `packages/repository-automation/src/release.mjs` - `docs/operations/repository-maintenance.md` +- 本仓库若是 fork `karlorz/codex-host`:先读 [fork versioning](../../../docs/operations/fork-versioning.md)(`0.13.3-0` / `0.13.3-1`…,控制台版本同源),再读本 Skill 的上游渠道表 - 涉及更新提示时:`packages/update-manager/src/github-release.ts` 源码和工作流是执行行为的依据。发现指南与当前实现不一致时,说明差异;不要假装已有不支持的发布能力。 @@ -38,6 +39,8 @@ description: 发布 codexhost 正式版、预览版,编写或确认 Release No 用户意图不清楚时,先问清两个渠道的目标。最终执行前明确报告版本、npm dist-tag、GitHub 状态与是否提示更新。 +在 `karlorz/codex-host` 上,上表的 BytePioneer npm `@codexhost/cli` / 正式 GitHub latest **不适用**。Fork 标签跟随当前 upstream 版本加计数器(upstream 为 `0.13.3` 时用 `v0.13.3-0`、`v0.13.3-1`,不要写成 `0.13.3-cpa.1`)。不向 `upstream` 推送。Fork **必须**把同一版本发布为 `@karlorz/codexhost-cli@latest`(及对应平台包),并把 `karlorz/codex-host` 上的 GitHub Release 标成 latest,这样 Settings 的 `npm install -g @karlorz/codexhost-cli@latest` 和 in-app Update 才指向 fork。禁止发布 `@codexhost/*`。`Release packages` 已启用,Trusted Publisher 指向 `karlorz/codex-host`;数字计数(如 `0.13.3-0`、`0.13.3-1`)由工作流发成 npm `latest` + GitHub latest。控制台「codexhost 版本」必须与该准备版本一致。`gh pr create` / `gh release` 必须带 `-R karlorz/codex-host`。规则全文:[fork versioning](../../../docs/operations/fork-versioning.md)。 + ## 2. 检查现场与验证 1. 检查工作区、当前分支、远端与已有标签;快进同步 `main`,不覆盖无关修改。 diff --git a/.agents/skills/codexhost-release/references/release-guide.md b/.agents/skills/codexhost-release/references/release-guide.md index b7f3533da..9ff9fa7bd 100644 --- a/.agents/skills/codexhost-release/references/release-guide.md +++ b/.agents/skills/codexhost-release/references/release-guide.md @@ -20,7 +20,7 @@ | 稳定 SemVer,默认发布 | `latest` | 正式 Release、latest | | 稳定 SemVer,手动发布且 `github_prerelease=true` | `latest` | Prerelease、非 latest | -静默预览发行用于“小修复让用户自行下载,但不推送更新提示”:npm 正常发布到 `latest`,GitHub 标记为 Prerelease。当前更新检查读取 GitHub 正式 latest Release,不读取 npm dist-tag;预览发行仍公开可下载。SemVer 预发布则在 npm 使用 `next`,用户需显式安装 `@codexhost/cli@next`。 +静默预览发行用于“小修复让用户自行下载,但不推送更新提示”:npm 正常发布到 `latest`,GitHub 标记为 Prerelease。当前更新检查读取 GitHub 正式 latest Release,不读取 npm dist-tag;预览发行仍公开可下载。SemVer 预发布则在 npm 使用 `next`,用户需显式安装 `@karlorz/codexhost-cli@next`。 ## 发布步骤 @@ -104,7 +104,7 @@ Composer 增加 Harness 命令按钮和模型搜索。Grok 支持自动或手动 ### npm ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` @@ -178,7 +178,7 @@ Tag 推送后,GitHub Actions 会自动: - 校验 Tag、npm 版本和 Cargo Workspace 版本一致。 - 构建 Windows x64/arm64 EXE。 - 构建 macOS x64/arm64 DMG。 -- 发布 macOS、Windows、Linux 的 x64/arm64 共六个平台 npm 包和 `@codexhost/cli` 元包。 +- 发布 macOS、Windows、Linux 的 x64/arm64 共六个平台 npm 包和 `@karlorz/codexhost-cli` 元包。 - 根据版本及手动发布输入创建公开 GitHub Release Prerelease 或正式 Release;`test` 渠道不创建 GitHub Release。 - GitHub Release 只上传四个安装包,不上传 npm tarball。 diff --git a/.agents/skills/codexhost-update-impact-audit/SKILL.md b/.agents/skills/codexhost-update-impact-audit/SKILL.md index 8ac006b90..b3f661c0c 100644 --- a/.agents/skills/codexhost-update-impact-audit/SKILL.md +++ b/.agents/skills/codexhost-update-impact-audit/SKILL.md @@ -89,7 +89,7 @@ Read the current call sites and follow each surface independently from discovery - sidebar decoration; - settings entry; - Fork; -- Codex usage gate (`renderer-codex-usage-gate.ts`, driven by `renderComposerAgentControl`'s external-submission readiness); +- Codex usage gate (`renderer-codex-usage-gate.ts`, driven by `renderComposerAgentControl`'s usage-gate allowance for Codex or a ready external Adapter); - Host create and subsequent-Turn routing. Prefer unique semantic candidates plus ownership checks. Record fail-closed behavior for absent or ambiguous candidates. Keep source relocation separate from an actual anchor or ownership change. @@ -126,9 +126,11 @@ Use `tools/renderer-binding/run.mjs` only when a clean controlled lifecycle, rel The runner does not accept the `.app` directory. A controlled flow may verify Agent switching, stale-prewarm clearing, new Thread creation, title behavior, or Fork. State clearly when user interaction or creation boundaries were not exercised. +For a requested live-voice fix on an already running Desktop, use the narrow live voice-hook refresh in [cpa-live-voice-desktop-hook.md](../../../docs/operations/cpa-live-voice-desktop-hook.md#live-voice-hook-debug-on-a-running-desktop) to validate the current document without a full Desktop restart. Keep the read-only probe first and skip mutation during an active or unknown call. + ### Codex usage gate probe -This surface lets an external Harness submit while the ChatGPT-signed-in Codex subscription is out of usage by projecting `false` for one Composer's two Desktop usage-gate subscriptions. It depends on private React/state-library internals, so check it on every update. `npm run audit:codex-desktop` reports its structure as the `codex-usage-gate` surface (see `tools/codex-desktop-contract-audit/README.md`); `codex-usage-account-gate-dormant` means the audit ran without a ChatGPT sign-in whose Account gate reads usage. The audit does not verify behavior. +This surface lets a Composer submit while the ChatGPT-signed-in Codex subscription is out of usage by projecting `false` for one Composer's two Desktop usage-gate subscriptions. It depends on private React/state-library internals, so check it on every update. `npm run audit:codex-desktop` reports its structure as the `codex-usage-gate` surface (see `tools/codex-desktop-contract-audit/README.md`); `codex-usage-account-gate-dormant` means the audit ran without a ChatGPT sign-in whose Account gate reads usage. The audit does not verify behavior. Bundle evidence (evidence only, not contracts): the Composer owner combines `submitDisabled` from an Account rate-limit selector (`authMethod !== "chatgpt"` early return, then `rate_limit.allowed === false`, gated to host `local`) and a reserve `hardBlocked` selector. Confirm both still exist and still feed `submitDisabled`, and that no new Account-wide usage blocker was added beside them. If Desktop now exempts external models/Threads or stops blocking in the Renderer, report that the module can be removed. @@ -141,7 +143,7 @@ Manual read-only live probe, when the audit surface is not `no-impact` or its co With codexhost running and an external Agent selected, bound gates show an instance `getSnapshot` of `() => false` for both hooks. The Agent control's hover title shows the "cannot separate from Codex usage" message when binding failed. -Behavioral check requires a ChatGPT account that is out of Codex usage; API-key sign-in never triggers the gate. With non-empty text and an external Agent, Send is enabled in a new draft and an existing Thread; after switching to Codex it returns to disabled. Do not submit a message unless the user asks. Remove any temporary text without touching text the user typed. Report `unverified` when no exhausted account is available. +Behavioral check requires a ChatGPT account that is out of Codex usage; API-key sign-in never triggers the gate. With non-empty text, Send and Enter are enabled in a new draft, an existing Thread, and a Codex Thread. Do not submit a message unless the user asks. Remove any temporary text without touching text the user typed. Report `unverified` when no exhausted account is available. Completion criterion: no surface receives a live verdict from bundle inspection alone. @@ -161,7 +163,7 @@ For a requested fix: 1. Add a regression test that models the observed DOM, Fiber, API-shape, visual relationship, or routing boundary. 2. Run it red for the observed reason. 3. Make one minimal production change in the owning module. -4. Run it green and repeat the relevant live probe. +4. Run it green and repeat the relevant live probe. For a requested live-voice fix, use the same narrow refresh linked under Controlled probe. A test that proves only that a control exists or a mock was called does not prove visual alignment, ownership, or routing. @@ -205,5 +207,5 @@ Do not bury the answer in the evidence. “Frontend changed” and “codexhost - Declaring Host routing, title creation, or Fork healthy without exercising that boundary. - Updating multiple controls before isolating the failed ownership contract. - Reformatting, resetting, staging, or overwriting unrelated work while investigating. -- Declaring the Codex usage gate healthy from a Codex-selected, empty, API-key-signed-in, or not-exhausted Composer; only the external Agent path with exhausted ChatGPT usage exercises it. +- Declaring the Codex usage gate healthy from an empty, API-key-signed-in, or not-exhausted Composer; exhausted ChatGPT usage on a new draft, existing Thread, or Codex Thread exercises it. - Checking only the Send button's DOM `disabled`; Desktop also guards Enter and the submit function with the same `submitDisabled`. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4bdb7d64f..950fc555f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,6 +1,7 @@ name: CI on: + workflow_dispatch: pull_request: paths-ignore: - "docs/**" @@ -92,6 +93,19 @@ jobs: --exclude 'packages/renderer-extension/test/**' --exclude 'tools/gate-claude-code/run.test.mjs' + # Asset pairs and export contracts differ even when About versions match. + - name: Test Windows native voice profiles + if: runner.os == 'Windows' + run: > + npx vitest run --config tests/vitest.config.js + packages/renderer-extension/test/renderer-voice-assets.test.ts + packages/renderer-extension/test/renderer-client-owned-voice.test.ts + packages/renderer-extension/test/renderer-native-chat-voice.test.ts + packages/renderer-extension/test/renderer-voice-settings.test.ts + packages/renderer-extension/test/renderer-voice-start-capability.test.ts + packages/renderer-extension/test/renderer-external-voice-launch.test.ts + packages/renderer-extension/test/renderer-voice-capability.test.ts + # Rust formatting is already covered by the Linux x64 formatting step. - name: Check Rust run: > diff --git a/.github/workflows/compat-daily.yml b/.github/workflows/compat-daily.yml new file mode 100644 index 000000000..b15e919cc --- /dev/null +++ b/.github/workflows/compat-daily.yml @@ -0,0 +1,180 @@ +name: Compat Daily + +# Compare macOS ChatGPT/Codex Desktop Sparkle latest to Renderer voice pins. +# Opens one GitHub issue per surface when a pin is behind. Does not download the .app. + +on: + schedule: + - cron: "17 7 * * *" + workflow_dispatch: + push: + branches: [main] + paths: + - tools/compat-daily/** + - .github/workflows/compat-daily.yml + - packages/renderer-extension/src/renderer-client-owned-voice.ts + - crates/platform/src/asar_metadata.rs + - crates/platform/src/installation.rs + - packages/renderer-extension/src/renderer-voice-assets.ts + - packages/renderer-extension/src/renderer-native-chat-voice-assets.ts + - packages/renderer-extension/src/renderer-native-chat-voice.ts + - packages/renderer-extension/src/renderer-voice-start-capability.ts + pull_request: + paths: + - tools/compat-daily/** + - .github/workflows/compat-daily.yml + - packages/renderer-extension/src/renderer-client-owned-voice.ts + - crates/platform/src/asar_metadata.rs + - crates/platform/src/installation.rs + - packages/renderer-extension/src/renderer-voice-assets.ts + - packages/renderer-extension/src/renderer-native-chat-voice-assets.ts + - packages/renderer-extension/src/renderer-native-chat-voice.ts + - packages/renderer-extension/src/renderer-voice-start-capability.ts + +permissions: + contents: read + +jobs: + pins: + name: macOS voice release pins + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + issues: write + steps: + - uses: actions/checkout@v4 + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version-file: .node-version + + - name: Unit-test pin parser + run: node --test tools/compat-daily/run.test.mjs + + - name: Compare Sparkle latest to voice pins + id: compare + run: | + set -o pipefail + node tools/compat-daily/run.mjs | tee compat-daily.json + + - name: Open or update issues on pin drift + if: failure() && steps.compare.outcome == 'failure' && github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + const report = JSON.parse(fs.readFileSync('compat-daily.json', 'utf8')); + if (report.error) return; + const findings = Array.isArray(report.findings) ? report.findings : []; + if (findings.length === 0) return; + const owner = context.repo.owner; + const repo = context.repo.repo; + const runUrl = `${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`; + for (const name of ['compat-broken', 'bug']) { + try { + await github.rest.issues.getLabel({ owner, repo, name }); + } catch (error) { + if (error.status !== 404) throw error; + await github.rest.issues.createLabel({ + owner, + repo, + name, + color: name === 'bug' ? 'd73a4a' : 'e4e669', + description: name === 'bug' ? 'Something is broken' : 'compat-daily Desktop pin drift', + }); + } + } + + const { data: open } = await github.rest.issues.listForRepo({ + owner, + repo, + state: 'open', + labels: 'compat-broken', + per_page: 100, + }); + for (const finding of findings) { + const dup = open.find((issue) => issue.title === finding.title); + const body = [ + finding.detail, + '', + `- macOS Desktop: \`${report.latest?.version ?? 'unknown'}\` build \`${report.latest?.build ?? 'unknown'}\``, + `- Event: \`${context.eventName}\``, + `- Checked commit: \`${context.sha}\` (\`${context.ref}\`)`, + `- Run: ${runUrl}`, + '', + 'Auto-opened by `.github/workflows/compat-daily.yml` on this repository.', + ].join('\n'); + if (dup) { + await github.rest.issues.createComment({ + owner, + repo, + issue_number: dup.number, + body: `macOS pin drift re-failed ${new Date().toISOString().slice(0, 10)} — ${runUrl}\n\nChecked commit: \`${context.sha}\` (\`${context.ref}\`)`, + }); + } else { + await github.rest.issues.create({ + owner, + repo, + title: finding.title, + labels: ['compat-broken', 'bug'], + body, + }); + } + } + + - name: Comment on matching voice pins + if: always() && (steps.compare.outcome == 'success' || steps.compare.outcome == 'failure') && github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + const report = JSON.parse(fs.readFileSync('compat-daily.json', 'utf8')); + if (report.error || !report.latest || !report.pins) return; + const { recoveredPinIssues } = await import(`${process.env.GITHUB_WORKSPACE}/tools/compat-daily/run.mjs`); + const owner = context.repo.owner; + const repo = context.repo.repo; + const issues = await github.paginate(github.rest.issues.listForRepo, { + owner, repo, state: 'open', labels: 'compat-broken', per_page: 100, + }); + for (const issue of recoveredPinIssues(report, issues)) { + const marker = ``; + const comments = await github.paginate(github.rest.issues.listComments, { + owner, repo, issue_number: issue.number, per_page: 100, + }); + if (comments.some(comment => comment.user?.type === 'Bot' && comment.body?.includes(marker))) continue; + await github.rest.issues.createComment({ + owner, repo, issue_number: issue.number, + body: `${marker}\nThe macOS ${issue.surface} pin is now ${issue.version}, matching this issue's tracked Desktop target or the current macOS Sparkle release. Current macOS Sparkle: ${report.latest.version}. Any newer version drift is reported separately. This check does not verify microphone capture, connected audio or conversation lifecycle; the issue remains open for acceptance review.\n\nChecked commit: \`${context.sha}\` (\`${context.ref}\`)\nRun: ${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`, + }); + } + + windows-msix: + name: Windows MSIX metadata and voice contracts + runs-on: windows-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version-file: .node-version + cache: npm + - uses: dtolnay/rust-toolchain@master + with: + toolchain: 1.97.1 + - uses: Swatinem/rust-cache@v2 + with: + save-if: ${{ github.ref == 'refs/heads/main' }} + - run: npm ci --prefer-offline + - name: Test bounded ASAR metadata + run: cargo test --locked -p codexhost-platform asar_metadata + - name: Test MSIX identity and product detection + run: cargo test --locked -p codexhost-platform installation::windows_tests + - name: Build TypeScript + run: npm run build:typescript + - name: Test platform profile selection and bootstrap + run: > + npx vitest run --config tests/vitest.config.js + packages/renderer-extension/test/renderer-voice-assets.test.ts + packages/desktop-control/test/cpa-live-voice-hook.test.ts diff --git a/Cargo.lock b/Cargo.lock index 08ee22825..163c79457 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -84,7 +84,7 @@ dependencies = [ [[package]] name = "codexhost-gate-a-native" -version = "0.14.0" +version = "0.14.0-3" dependencies = [ "codexhost-platform", "codexhost-shim", @@ -92,7 +92,7 @@ dependencies = [ [[package]] name = "codexhost-launcher" -version = "0.14.0" +version = "0.14.0-3" dependencies = [ "codexhost-platform", "fs2", @@ -106,7 +106,7 @@ dependencies = [ [[package]] name = "codexhost-platform" -version = "0.14.0" +version = "0.14.0-3" dependencies = [ "libproc", "nix", @@ -121,7 +121,7 @@ dependencies = [ [[package]] name = "codexhost-shim" -version = "0.14.0" +version = "0.14.0-3" dependencies = [ "codexhost-platform", "fs2", @@ -131,7 +131,7 @@ dependencies = [ [[package]] name = "codexhost-updater" -version = "0.14.0" +version = "0.14.0-3" dependencies = [ "codexhost-platform", "serde", diff --git a/Cargo.toml b/Cargo.toml index a73213f01..2a80bba00 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -15,7 +15,7 @@ default-members = [ resolver = "3" [workspace.package] -version = "0.14.0" +version = "0.14.0-3" edition = "2024" rust-version = "1.97.1" publish = false diff --git a/README.md b/README.md index f30ebc485..b679a75c5 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ But **Codex** isn't the only great **Agent Harness** — **Claude Code** and **P


-

Download · Cross-Agent Collaboration · Remote · 微信交流群 · 简体中文 · 한국어

+

Download · Cross-Agent Collaboration · Remote · 微信交流群 · 简体中文 · 한국어


@@ -68,13 +68,13 @@ https://github.com/user-attachments/assets/c48192d7-23ff-4f6e-b61a-6345a655bb76 **Option 1: npm** (macOS / Windows / Linux) ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` **Option 2: Installer** (macOS / Windows) -Grab the installer for your platform from [Releases](https://github.com/BytePioneer-AI/codex-host/releases). +Grab the installer for your platform from [Releases](https://github.com/karlorz/codex-host/releases). > Linux is supported on x64 and ARM64. See the [Linux guide](docs/platforms/linux/linux.md). @@ -176,7 +176,7 @@ Harnesses integrate with Codex Desktop's native UI according to their capabiliti | Streaming responses | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Tool status | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Edit Diff | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Questions / cancellation | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Questions / cancellation | ✅ | ✅ | ✅ | ✅ | ◐ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Model / Thinking selection | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Tool approvals | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | Permission modes | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | @@ -186,6 +186,21 @@ Harnesses integrate with Codex Desktop's native UI according to their capabiliti | Context compaction | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | | Slash commands | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | | Edit previous message | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Live voice chat | ? | ? | ✅ | ? | ◐ | ? | ? | ? | ? | ? | ? | ? | ? | ? | +| Multi-select questions | — | ◐ | ◐ | ◐ | — | ◐ | — | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | +| Subagents | ✅ | ✅ | ✅ | — | ✅ | — | ✅ | ✅ | ✅ | ◐ | — | — | — | — | +| Computer Use / in-app browser | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | +| Generated images | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | + +Legend: ✅ supported integration · ◐ partial, degraded, or saved intent only · — not exposed through the current plugin · ? unverified. + +- **Live voice chat** uses this fork's version-audited Desktop hook and a configured gateway. The Host `thread/realtime/*` path and new-chat Harness inheritance are generic. Claude Code has a documented attended Desktop call; Grok is ◐ because verification covers a real gateway call and an isolated Host trial with the public plugin, rather than installed Desktop button acceptance. Other Harnesses are unverified (?). See [CPA live-voice Desktop hook](docs/operations/cpa-live-voice-desktop-hook.md). +- **Multi-select questions** do not have a multi-select Desktop card. The shared projector omits `multiple` while the Host contract retains it. In the reported delegated Claude Code test, `AskUserQuestion` with `multiSelect: true` and Alpha/Beta/Gamma rendered as single-choice and returned one string (`"Alpha"`), with no Host error. Claude Code, OpenCode, DSH, CodeBuddy/WorkBuddy, Cursor, Hermes, Qoder, Kimi Code and Copilot elicitation forms are ◐ because their adapters preserve native multi-select intent but cannot project it faithfully; only Claude Code was live-tested. OMP is ◐ for its documented native composition of separate single-choice/text interactions, not a multi-select card. Pi hardcodes single-choice; Antigravity explicitly rejects multi-select. See [Antigravity question interaction postmortem](docs/harnesses/antigravity/antigravity-question-interaction-postmortem.md) and [OMP interactions](docs/harnesses/omp/omp-interactions.md). +- **Grok build and questions**: this fork uses a patched **grokgod Grok build**; the patch-dependent behavior below is not a stock-Grok guarantee. Patch `0011-ask-question-timeout-action` adds configurable timeout selection and activity-based timeout reset to the existing native `ask_user_question`; it does not connect a Desktop card. In the reported grokgod ACP test, the call failed with `Failed to reach the client for user question: "Method not found": _x.ai/ask_user_question`. The Host ACP client handles only `_x.ai/mcp/elicit` extension requests, so no question card appears. Questions / cancellation stays ◐: turn cancellation and bounded MCP consent approvals have separate implemented paths. See [Harness capability boundaries](docs/harnesses/capability-boundaries.md). +- **Grok MCP approval — Always allow** requires grokgod patch `0025-mcp-elicitation-persist` to forward the native persist offer and return the selected scope as MCP `_meta.persist`. Desktop offers it only when the request includes `always`; unpatched Grok has Allow once / Deny only. Allow this conversation is not exposed because native session lifetime is not qualified. See [Grok MCP elicitation](docs/harnesses/grok/mcp-elicitation.md). +- **Subagents**: ✅ native status plus a read-only child transcript · ◐ native status only. Pi requires the supported `pi-subagents` extension protocol; OMP requires RPC subagent subscription support. These cells describe plugin projection, not whether an upstream Harness can run subagents internally. +- **Computer Use / in-app browser** uses the user-installed Grok `cua_repl` MCP. This fork's Computer Use integration requires the patched grokgod Grok build with `0026-cua-turn-end-release`, including turn-boundary cleanup and the Mac Sky native release bridge, plus the installed ChatGPT Computer Use runtime. Stock-Grok parity is not established. IAB is saved intent only: the Host keeps its browser binding disabled pending Desktop authorization and a session-owned browser route. See [Grok Computer Use settings](docs/harnesses/grok/computer-use-settings.md). +- **Generated images** describes Grok Imagine projection to Desktop `imageGeneration`, not general image-file creation. Grok is the only current adapter emitting this Item; the other cells mean no dedicated gallery projection. Projection has test coverage, but live gallery display remains unverified. See [Grok generated images](docs/harnesses/grok/generated-images.md). @@ -226,7 +241,7 @@ Before you start, add the remote machine in Codex Desktop under **Settings → C 1. Install and start codexhost on the remote machine: ```bash - npm install -g @codexhost/cli + npm install -g @karlorz/codexhost-cli codexhost remote install codexhost remote start codexhost remote status @@ -291,7 +306,7 @@ Please read the [contributing guide](CONTRIBUTING.md) before opening an issue or Requirements: the official Codex Desktop, Node.js 22.19+ or 24, and Rust. ```bash -git clone https://github.com/BytePioneer-AI/codex-host +git clone https://github.com/karlorz/codex-host cd codex-host npm ci npm start @@ -314,6 +329,8 @@ Most of the work is implementing the plugin's Manifest, factory, Adapter, and Se Tip: point your coding Agent at the in-repo [codexhost-add-harness Skill](.agents/skills/codexhost-add-harness/SKILL.md). It covers plugin structure, the shared Adapter interface, capability implementation, and testing requirements. +Desktop Grok Threads receive CodexHost session rules through Grok's official ACP `session/new` `_meta.rules` field. Host Runtime retires leftover Host-owned `~/.grok/rules/codexhost-desktop.md` copies and leaves user-edited files in place. New Harness plugins do not add a home-rule copy. Details: [operations/cpa-live-voice-desktop-hook.md](docs/operations/cpa-live-voice-desktop-hook.md). + ## Acknowledgements - Thanks to the [LINUX DO](https://linux.do/) community for their ongoing support. @@ -323,8 +340,8 @@ Tip: point your coding Agent at the in-repo [codexhost-add-harness Skill](.agent Thanks to everyone who has contributed to CodexHost. - - CodexHost contributors + + CodexHost contributors ## Star History diff --git a/crates/launcher/src/desktop_attachment.rs b/crates/launcher/src/desktop_attachment.rs index e5e84dca3..272bd76d9 100644 --- a/crates/launcher/src/desktop_attachment.rs +++ b/crates/launcher/src/desktop_attachment.rs @@ -5,11 +5,14 @@ use std::ffi::OsString; use std::io::{self, BufRead, BufReader, Write}; use std::net::{TcpListener, TcpStream}; use std::path::{Path, PathBuf}; +#[cfg(target_os = "macos")] +use std::process::Command; use std::thread; use std::time::{Duration, Instant}; use codexhost_platform::{ DesktopInstallation, descendant_executable_exists, desktop_root_process_ids_for_installation, + process_snapshot, terminate_process_instance, }; #[cfg(target_os = "windows")] use codexhost_platform::{process_executable_path, process_exists, terminate_process_by_id}; @@ -23,6 +26,7 @@ use crate::runtime_instance::{ const ATTACHMENT_RETRY_INITIAL_DELAY: Duration = Duration::from_millis(100); const ATTACHMENT_RETRY_MAX_DELAY: Duration = Duration::from_secs(1); +const ORPHANED_CONTROLLER_STOP_GRACE: Duration = Duration::from_secs(2); #[derive(Debug)] pub(super) struct RuntimeControl { @@ -74,8 +78,7 @@ pub(super) fn acquire_launcher_ownership( return Ok(LauncherOwnership::Attached); } if desktop_root_process_ids_for_installation(installation)?.is_empty() { - stop_stale_launcher(descriptor)?; - let _ = remove_matching_descriptor(&descriptor_path, descriptor)?; + recover_stale_controller(descriptor, &descriptor_path)?; } } if let Some(guard) = try_acquire_launcher_guard(&guard_path)? { @@ -231,6 +234,178 @@ pub(super) fn stop_stale_launcher(_descriptor: &RuntimeDescriptor) -> Result<(), Ok(()) } +/// Stop a Desktop Controller that outlived its Launcher and Desktop. +/// +/// A failed or interrupted `npm start` can kill Desktop while leaving the +/// Controller listening on the recorded attachment port. The next launch then +/// owns the Launcher lock with no Desktop. Only a listener whose command line +/// still carries this descriptor's attachment port and nonce is ours to stop. +pub(super) fn recover_stale_controller( + descriptor: &RuntimeDescriptor, + descriptor_path: &Path, +) -> Result<(), Box> { + stop_orphaned_controller(descriptor)?; + stop_stale_launcher(descriptor)?; + let _ = remove_matching_descriptor(descriptor_path, descriptor)?; + Ok(()) +} + +pub(super) fn stop_orphaned_controller( + descriptor: &RuntimeDescriptor, +) -> Result<(), Box> { + let matches = matching_controller_process_ids(descriptor)?; + if matches.is_empty() { + return Ok(()); + } + for process_id in &matches { + terminate_controller_process(*process_id, false)?; + } + if wait_for_controller_exit(descriptor, ORPHANED_CONTROLLER_STOP_GRACE)? { + return Ok(()); + } + for process_id in matching_controller_process_ids(descriptor)? { + terminate_controller_process(process_id, true)?; + } + if wait_for_controller_exit(descriptor, ORPHANED_CONTROLLER_STOP_GRACE)? { + return Ok(()); + } + Err(format!( + "leftover Desktop Controller on port {} did not exit before timeout", + descriptor.control_port + ) + .into()) +} + +fn command_line_matches_controller(command: &str, descriptor: &RuntimeDescriptor) -> bool { + let port = descriptor.control_port.to_string(); + let mut saw_port = false; + let mut saw_nonce = false; + let mut arguments = command.split_whitespace(); + while let Some(argument) = arguments.next() { + match argument { + "--attachment-port" => { + saw_port |= arguments.next() == Some(port.as_str()); + } + "--attachment-nonce" => { + saw_nonce |= arguments.next() == Some(descriptor.nonce.as_str()); + } + _ => {} + } + } + saw_port && saw_nonce +} + +pub(super) fn matching_controller_process_ids( + descriptor: &RuntimeDescriptor, +) -> Result, Box> { + let mut matches = Vec::new(); + for (process_id, command) in process_command_lines()? { + if process_id == std::process::id() { + continue; + } + if command_line_matches_controller(&command, descriptor) { + matches.push(process_id); + } + } + Ok(matches) +} + +fn terminate_controller_process(process_id: u32, force: bool) -> Result<(), Box> { + if process_id == std::process::id() { + return Ok(()); + } + match process_snapshot(process_id) { + Ok(snapshot) => { + terminate_process_instance(&snapshot, force)?; + Ok(()) + } + Err(_) => Ok(()), + } +} + +pub(super) fn wait_for_controller_exit( + descriptor: &RuntimeDescriptor, + timeout: Duration, +) -> Result> { + wait_for_known_controller_exit(descriptor, &[], timeout) +} + +pub(super) fn wait_for_known_controller_exit( + descriptor: &RuntimeDescriptor, + known_process_ids: &[u32], + timeout: Duration, +) -> Result> { + let started = Instant::now(); + loop { + // A Controller can close its socket before its asynchronous cleanup finishes. + // The Launcher cannot finish shutdown while that process remains alive. + let known_alive = known_process_ids + .iter() + .copied() + .any(|process_id| process_snapshot(process_id).is_ok()); + if matching_controller_process_ids(descriptor)?.is_empty() + && !known_alive + && !endpoint_ready(descriptor.control_port, Duration::from_millis(50)) + { + return Ok(true); + } + if started.elapsed() >= timeout { + return Ok(false); + } + thread::sleep(Duration::from_millis(20)); + } +} + +#[cfg(target_os = "macos")] +fn process_command_lines() -> Result, Box> { + let output = Command::new("/bin/ps") + .args(["-axww", "-o", "pid=,command="]) + .output()?; + if !output.status.success() { + return Err("cannot list process command lines".into()); + } + Ok(String::from_utf8_lossy(&output.stdout) + .lines() + .filter_map(|line| { + let line = line.trim(); + let (pid, command) = line.split_once(char::is_whitespace)?; + Some((pid.parse().ok()?, command.trim().to_owned())) + }) + .collect()) +} + +#[cfg(target_os = "linux")] +fn process_command_lines() -> Result, Box> { + let mut commands = Vec::new(); + for entry in std::fs::read_dir("/proc")?.filter_map(Result::ok) { + let Some(process_id) = entry + .file_name() + .to_str() + .and_then(|name| name.parse::().ok()) + else { + continue; + }; + let Ok(bytes) = std::fs::read(entry.path().join("cmdline")) else { + continue; + }; + let command = bytes + .split(|byte| *byte == 0) + .filter(|argument| !argument.is_empty()) + .map(|argument| String::from_utf8_lossy(argument).into_owned()) + .collect::>() + .join(" "); + if !command.is_empty() { + commands.push((process_id, command)); + } + } + Ok(commands) +} + +#[cfg(target_os = "windows")] +fn process_command_lines() -> Result, Box> { + Ok(Vec::new()) +} + /// A recorded launcher PID may since have been reused by an unrelated process. /// Only an inspected executable matching this launcher proves ownership; a PID /// the current user cannot inspect (for example a reused SYSTEM service PID) @@ -327,4 +502,130 @@ mod tests { ); server.join().expect("attachment server"); } + + #[test] + fn controller_command_matches_exact_attachment_port_and_nonce() { + let descriptor = descriptor(60218); + assert!(command_line_matches_controller( + "node release-main.js --attachment-port 60218 --attachment-nonce 0123456789abcdef0123456789abcdef", + &descriptor + )); + assert!( + !command_line_matches_controller( + "node release-main.js --attachment-port 602181 --attachment-nonce 0123456789abcdef0123456789abcdef", + &descriptor + ), + "port 60218 must not match 602181" + ); + assert!(!command_line_matches_controller( + "node release-main.js --attachment-port 60218 --attachment-nonce ffffffffffffffffffffffffffffffff", + &descriptor + )); + assert!(!command_line_matches_controller( + "node console-server.mjs serve", + &descriptor + )); + } + + #[cfg(unix)] + fn spawn_matching_listener(port: u16, nonce: &str, close_only: bool) -> std::process::Child { + let signal_handler = if close_only { + "signal.signal(signal.SIGTERM, lambda *_: server.close())\n" + } else { + "" + }; + let script = format!( + "import socket, time, signal\n\ + server = socket.socket()\n\ + server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\n\ + server.bind(('127.0.0.1', {port}))\n\ + server.listen(1)\n\ + {signal_handler}\ + while True:\n\ + \ttime.sleep(0.2)\n" + ); + let mut command = std::process::Command::new("python3"); + command + .args([ + "-c", + &script, + "--attachment-port", + &port.to_string(), + "--attachment-nonce", + nonce, + ]) + .stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()); + #[cfg(unix)] + { + use std::os::unix::process::CommandExt; + command.process_group(0); + } + command.spawn().expect("start leftover Controller fixture") + } + + #[cfg(unix)] + fn wait_until_listening(port: u16) { + let started = Instant::now(); + while !endpoint_ready(port, Duration::from_millis(50)) { + assert!( + started.elapsed() < Duration::from_secs(2), + "leftover Controller fixture did not listen" + ); + thread::sleep(Duration::from_millis(20)); + } + } + + #[cfg(unix)] + #[test] + fn stop_orphaned_controller_terminates_a_matching_listener() { + let listener = TcpListener::bind(("127.0.0.1", 0)).expect("reserve port"); + let port = listener.local_addr().expect("reserved address").port(); + drop(listener); + let descriptor = descriptor(port); + let mut child = spawn_matching_listener(port, &descriptor.nonce, false); + wait_until_listening(port); + + let result = stop_orphaned_controller(&descriptor); + let _ = child.kill(); + let _ = child.wait(); + result.expect("stop leftover Controller"); + assert!( + !endpoint_ready(port, Duration::from_millis(50)), + "matching leftover Controller kept its attachment port" + ); + } + + #[cfg(unix)] + #[test] + fn stop_orphaned_controller_escalates_after_socket_closes_but_process_stays_alive() { + let listener = TcpListener::bind(("127.0.0.1", 0)).expect("reserve port"); + let port = listener.local_addr().unwrap().port(); + drop(listener); + let descriptor = descriptor(port); + let mut child = spawn_matching_listener(port, &descriptor.nonce, true); + wait_until_listening(port); + let result = stop_orphaned_controller(&descriptor); + let exited = child.try_wait().unwrap().is_some(); + let _ = child.kill(); + let _ = child.wait(); + result.expect("stop Controller whose socket closed during cleanup"); + assert!(exited, "Controller process survived shutdown"); + } + + #[cfg(unix)] + #[test] + fn stop_orphaned_controller_leaves_an_unrelated_listener() { + let listener = TcpListener::bind(("127.0.0.1", 0)).expect("unrelated listener"); + let port = listener.local_addr().expect("unrelated address").port(); + let descriptor = descriptor(port); + + stop_orphaned_controller(&descriptor).expect("ignore unrelated listener"); + assert!( + endpoint_ready(port, Duration::from_millis(50)), + "unrelated listener on the recorded port was terminated" + ); + drop(listener); + } } diff --git a/crates/launcher/src/main.rs b/crates/launcher/src/main.rs index 64cfa35cb..2c993c4c7 100644 --- a/crates/launcher/src/main.rs +++ b/crates/launcher/src/main.rs @@ -7,6 +7,7 @@ mod desktop_attachment; mod desktop_path_overrides; mod installation_layout; mod native_harness_broker; +mod restart; mod runtime_instance; #[cfg(target_os = "linux")] mod secure_storage; @@ -52,13 +53,14 @@ use codexhost_platform::{ use compatibility::{MAX_CONTROLLER_READINESS_LINE_BYTES, parse_controller_readiness_line}; use desktop_attachment::{ LauncherOwnership, RuntimeControl, acquire_launcher_ownership, allocate_runtime_control, - endpoint_ready, publish_runtime_descriptor, stop_stale_launcher, wait_for_host_chain, + endpoint_ready, publish_runtime_descriptor, recover_stale_controller, wait_for_host_chain, }; use installation_layout::InstalledResources; use native_harness_broker::run_native_harness_broker_cli; +#[cfg(target_os = "windows")] +use runtime_instance::remove_matching_descriptor; use runtime_instance::{ StartupObservation, StartupState, classify_startup, default_descriptor_path, read_descriptor, - remove_matching_descriptor, }; use startup_record::StartupOutcome; #[cfg(target_os = "macos")] @@ -129,7 +131,7 @@ impl Error for UnmanagedDesktopConflict {} fn usage() { eprintln!( - "usage:\n codexhost\n codexhost inspect [--json] [--custom-install ]\n codexhost console\n codexhost update\n codexhost launch [--shim ] [--node ] [--host-runtime ] [--desktop-controller ] [--renderer ] [--pi ] [--custom-install ]\n codexhost broker install|status|stop|uninstall\n codexhost delegate --help\n codexhost harness inspect ...\n codexhost delegate start ...\n codexhost thread send|cancel|read|wait|list ..." + "usage:\n codexhost\n codexhost inspect [--json] [--custom-install ]\n codexhost console\n codexhost update\n codexhost restart\n codexhost launch [--shim ] [--node ] [--host-runtime ] [--desktop-controller ] [--renderer ] [--pi ] [--custom-install ]\n codexhost broker install|status|stop|uninstall\n codexhost delegate --help\n codexhost harness inspect ...\n codexhost delegate start ...\n codexhost thread send|cancel|read|wait|list ..." ); } @@ -496,11 +498,15 @@ impl LaunchOptions { } fn desktop_controller_command( + desktop_version: &str, options: &ResolvedLaunchOptions, control: &RuntimeControl, environment: &[(OsString, OsString)], ) -> Command { let mut command = Command::new(&options.node); + if std::env::var("CODEXHOST_DESKTOP_VERSION").map_or(true, |value| value.trim().is_empty()) { + command.env("CODEXHOST_DESKTOP_VERSION", desktop_version); + } command // Keep proxy support without surfacing its experimental API notice. .arg("--disable-warning=UNDICI-EHPA") @@ -582,16 +588,15 @@ fn wait_for_controller_ready( } fn start_desktop_controller( + installation: &DesktopInstallation, options: &ResolvedLaunchOptions, control: &RuntimeControl, environment: &[(OsString, OsString)], ) -> Result> { startup_trace("spawning Desktop Controller"); - let mut controller = spawn_supervised(&mut desktop_controller_command( - options, - control, - environment, - ))?; + let mut command = + desktop_controller_command(&installation.version, options, control, environment); + let mut controller = spawn_supervised(&mut command)?; startup_trace("waiting for Desktop Controller readiness"); match wait_for_controller_ready(&mut controller, Duration::from_secs(120)) { Ok(()) => { @@ -748,7 +753,7 @@ fn supervise_desktop( Duration::from_secs(30), )?; startup_trace("Codex Desktop launched"); - let mut controller = start_desktop_controller(options, control, environment)?; + let mut controller = start_desktop_controller(installation, options, control, environment)?; let desktop_pid = desktop.root_snapshot().id; startup_trace("waiting for Host chain"); if !wait_for_host_chain(desktop_pid, options, Duration::from_secs(30))? { @@ -782,7 +787,12 @@ fn supervise_desktop( } } if let Some(status) = controller.try_wait()? { - let _ = desktop.shutdown(Duration::from_secs(2)); + desktop.shutdown(Duration::from_secs(2))?; + desktop.cleanup_escaped(Duration::from_secs(2))?; + desktop.disarm_cleanup(); + if status.success() { + return Ok(()); + } return Err( format!("Desktop Controller exited while Desktop was running: {status}").into(), ); @@ -832,7 +842,8 @@ fn supervise_desktop( startup_trace("Codex Desktop launched"); let desktop_pid = desktop.id(); wait_for_launched_desktop_ownership(installation, &mut desktop, Duration::from_secs(5))?; - let mut controller = match start_desktop_controller(options, control, environment) { + let mut controller = match start_desktop_controller(installation, options, control, environment) + { Ok(started) => started, Err(error) => { let _ = desktop.kill(); @@ -882,6 +893,9 @@ fn supervise_desktop( } let _ = desktop.kill(); let _ = desktop.wait(); + if status.success() { + return Ok(()); + } return Err( format!("Desktop Controller exited while Desktop was running: {status}").into(), ); @@ -1093,8 +1107,8 @@ fn launch( match state { StartupState::RecoverStale => { if let Some(descriptor) = &descriptor { - stop_stale_launcher(descriptor)?; - let _ = remove_matching_descriptor(&descriptor_path, descriptor)?; + startup_trace("recovering leftover Desktop Controller"); + recover_stale_controller(descriptor, &descriptor_path)?; } else if descriptor_present { std::fs::remove_file(&descriptor_path)?; } @@ -1130,14 +1144,7 @@ fn launch( continue; } } - StartupState::CleanLaunch => { - if descriptor_present && control_endpoint_ready { - return Err( - "codexhost control endpoint is still active without a live Desktop; retry after it exits" - .into(), - ); - } - } + StartupState::CleanLaunch => {} } let control = allocate_runtime_control()?; @@ -1230,19 +1237,13 @@ fn launch( }) { StartupState::RecoverStale => { if let Some(descriptor) = &descriptor { - stop_stale_launcher(descriptor)?; - let _ = remove_matching_descriptor(&descriptor_path, descriptor)?; + startup_trace("recovering leftover Desktop Controller"); + recover_stale_controller(descriptor, &descriptor_path)?; } else if descriptor_present { return Err("codexhost runtime descriptor is invalid; remove it after checking its ownership".into()); } } StartupState::Attach => unreachable!("no Desktop roots were observed"), - StartupState::CleanLaunch if descriptor_present && control_endpoint_ready => { - return Err( - "codexhost control endpoint is still active without a live Desktop; retry after it exits" - .into(), - ); - } StartupState::CleanLaunch => {} } let control = allocate_runtime_control()?; @@ -1305,7 +1306,7 @@ fn run_console_command(action: &str) -> Result<(), Box> { fn is_launch_command(arguments: &[String]) -> bool { matches!( arguments.first().map(String::as_str), - None | Some(START_MENU_ARGUMENT) | Some("launch") + None | Some(START_MENU_ARGUMENT) | Some("launch") | Some("restart") ) } @@ -1342,6 +1343,13 @@ fn run(arguments: &[String]) -> Result<(), Box> { Some("update") => Err("update accepts no arguments".into()), Some("console") => Err("console accepts no arguments".into()), Some("launch") => launch(parse_launch_options(&arguments[1..])?, false), + Some("restart") => { + let options = parse_launch_options(&arguments[1..])?; + let resolved = parse_launch_options(&arguments[1..])?.resolve()?; + let _ = discover_desktop(resolved.custom_install_root.as_deref())?; + restart::stop_managed_runtime()?; + launch(options, false) + } Some("open-loopback-url") if arguments.len() == 1 => { let url = read_bounded_loopback_url(std::io::stdin().lock())?; validate_loopback_root_url(&url)?; @@ -1372,11 +1380,16 @@ fn main() -> ExitCode { let launching = is_launch_command(&arguments); if launching { startup_record::begin(); - console::set_presentation(if arguments.first().map(String::as_str) == Some("launch") { - console::Presentation::Print - } else { - console::Presentation::Browser - }); + console::set_presentation( + if matches!( + arguments.first().map(String::as_str), + Some("launch" | "restart") + ) { + console::Presentation::Print + } else { + console::Presentation::Browser + }, + ); } match run(&arguments) { Ok(()) => ExitCode::SUCCESS, @@ -1678,7 +1691,17 @@ mod tests { #[test] fn production_controller_uses_private_node_and_loopback_renderer_cdp() { let options = resolved_options(); - let command = desktop_controller_command(&options, &runtime_control(), &[]); + let command = + desktop_controller_command("26.1021.31345", &options, &runtime_control(), &[]); + if std::env::var("CODEXHOST_DESKTOP_VERSION").map_or(true, |value| value.trim().is_empty()) + { + assert!( + command + .get_envs() + .any(|(key, value)| key == "CODEXHOST_DESKTOP_VERSION" + && value == Some(std::ffi::OsStr::new("26.1021.31345"))) + ); + } assert_eq!(command.get_program(), "/opt/node"); assert_eq!( command.get_args().collect::>(), @@ -1949,6 +1972,7 @@ mod tests { fn controller_receives_only_the_managed_network_environment() { let options = resolved_options(); let command = desktop_controller_command( + "26.1021.31345", &options, &runtime_control(), &[ @@ -2047,7 +2071,8 @@ mod tests { desktop_controller: PathBuf::from(r"\\?\C:\Program Files\codexhost\controller.mjs"), ..resolved_options() }; - let command = desktop_controller_command(&options, &runtime_control(), &[]); + let command = + desktop_controller_command("26.1021.31345", &options, &runtime_control(), &[]); assert_eq!( command.get_args().nth(1), diff --git a/crates/launcher/src/restart.rs b/crates/launcher/src/restart.rs new file mode 100644 index 000000000..6593aeff6 --- /dev/null +++ b/crates/launcher/src/restart.rs @@ -0,0 +1,217 @@ +//! Restart only the runtime authenticated by this installation's descriptor. +use std::error::Error; +use std::io::{BufRead, BufReader, Read, Write}; +use std::net::TcpStream; +use std::thread; +use std::time::{Duration, Instant}; + +use codexhost_platform::process_executable_path; + +use crate::runtime_instance::{ + RuntimeDescriptor, default_descriptor_path, default_guard_path, read_descriptor, + try_acquire_launcher_guard, +}; + +fn request_stop(descriptor: &RuntimeDescriptor) -> Result<(), Box> { + let address = format!("127.0.0.1:{}", descriptor.control_port).parse()?; + let mut stream = match TcpStream::connect_timeout(&address, Duration::from_secs(2)) { + Ok(stream) => stream, + Err(error) if error.kind() == std::io::ErrorKind::ConnectionRefused => { + return crate::desktop_attachment::stop_orphaned_controller(descriptor); + } + Err(error) => return Err(error.into()), + }; + let observed = crate::desktop_attachment::matching_controller_process_ids(descriptor)?; + stream.set_read_timeout(Some(Duration::from_secs(5)))?; + stream.set_write_timeout(Some(Duration::from_secs(2)))?; + writeln!(stream, "STOP {}", descriptor.nonce)?; + let mut response = String::new(); + BufReader::new(stream).take(128).read_line(&mut response)?; + if response.trim_end() == "rejected" { + // Older controllers have no STOP command. Only the process whose command + // line matches this descriptor's port and nonce is eligible for shutdown. + return crate::desktop_attachment::stop_orphaned_controller(descriptor); + } + if response.trim_end() != "stopping" { + return Err("Controller cannot restart this runtime; fully quit and reopen codexhost once to load the new controller".into()); + } + if crate::desktop_attachment::wait_for_known_controller_exit( + descriptor, + &observed, + Duration::from_secs(2), + )? { + return Ok(()); + } + crate::desktop_attachment::stop_orphaned_controller(descriptor) +} + +pub(super) fn stop_managed_runtime() -> Result<(), Box> { + stop_managed_runtime_at( + &default_descriptor_path()?, + &default_guard_path()?, + &std::env::current_exe()?.canonicalize()?, + Duration::from_secs(45), + ) +} + +fn stop_managed_runtime_at( + descriptor_path: &std::path::Path, + guard_path: &std::path::Path, + expected: &std::path::Path, + timeout: Duration, +) -> Result<(), Box> { + let descriptor = read_descriptor(descriptor_path)?.ok_or("No managed Desktop is running")?; + if process_executable_path(descriptor.launcher_pid)?.canonicalize()? != expected { + return Err("The running Desktop belongs to another codexhost installation".into()); + } + request_stop(&descriptor)?; + let started = Instant::now(); + while started.elapsed() < timeout { + if let Some(_guard) = try_acquire_launcher_guard(guard_path)? + && read_descriptor(descriptor_path)?.is_none() + { + return Ok(()); + } + thread::sleep(Duration::from_millis(100)); + } + Err("Managed Desktop did not stop before the restart timeout".into()) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::net::TcpListener; + + #[test] + fn stop_authenticates_and_requires_explicit_acknowledgment() { + for (reply, accepted) in [ + ("stopping\n", true), + ("ready\n", false), + ("failed\n", false), + ] { + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + let descriptor = RuntimeDescriptor::new( + 1, + listener.local_addr().unwrap().port(), + "0123456789abcdef0123456789abcdef".into(), + ) + .unwrap(); + let server = thread::spawn(move || { + let (mut socket, _) = listener.accept().unwrap(); + let mut line = String::new(); + BufReader::new(socket.try_clone().unwrap()) + .read_line(&mut line) + .unwrap(); + assert_eq!(line, "STOP 0123456789abcdef0123456789abcdef\n"); + socket.write_all(reply.as_bytes()).unwrap(); + }); + assert_eq!(request_stop(&descriptor).is_ok(), accepted); + server.join().unwrap(); + } + } + + #[cfg(unix)] + #[test] + fn acknowledged_stop_escalates_when_controller_closes_socket_but_remains_alive() { + use std::process::{Command, Stdio}; + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + let port = listener.local_addr().unwrap().port(); + drop(listener); + let descriptor = + RuntimeDescriptor::new(1, port, crate::runtime_instance::random_nonce().unwrap()) + .unwrap(); + let mut command = Command::new("python3"); + command + .args([ + "-c", + "import socket,signal,time,sys\nsignal.signal(signal.SIGTERM,signal.SIG_IGN)\ns=socket.socket()\ns.bind(('127.0.0.1',int(sys.argv[2])))\ns.listen(1)\nprint('ready',flush=True)\nc,_=s.accept()\nline=c.makefile('rb').readline()\nassert line == ('STOP '+sys.argv[4]+'\\n').encode()\nc.sendall(b'stopping\\n')\nc.close()\ns.close()\nwhile True: time.sleep(0.1)", + "--attachment-port", + &port.to_string(), + "--attachment-nonce", + &descriptor.nonce, + ]) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()); + { + use std::os::unix::process::CommandExt; + command.process_group(0); + } + let mut child = command.spawn().unwrap(); + let mut ready = String::new(); + BufReader::new(child.stdout.take().unwrap()) + .read_line(&mut ready) + .unwrap(); + assert_eq!(ready.trim(), "ready"); + let visible = Instant::now(); + while crate::desktop_attachment::matching_controller_process_ids(&descriptor) + .unwrap() + .is_empty() + { + assert!( + visible.elapsed() < Duration::from_secs(2), + "acknowledging Controller fixture was not visible" + ); + thread::sleep(Duration::from_millis(20)); + } + let result = request_stop(&descriptor); + let exited = child.try_wait().unwrap().is_some(); + let _ = child.kill(); + let _ = child.wait(); + result.expect("finish acknowledged Controller stop"); + assert!(exited, "acknowledging Controller survived shutdown"); + } + + #[test] + fn restart_waits_for_owned_lifecycle_and_refuses_another_installation() { + let root = std::env::temp_dir().join(format!( + "codexhost-restart-{}", + crate::runtime_instance::random_nonce().unwrap() + )); + std::fs::create_dir(&root).unwrap(); + let descriptor_path = root.join("runtime.json"); + let guard_path = root.join("owner.lock"); + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + let descriptor = RuntimeDescriptor::new( + std::process::id(), + listener.local_addr().unwrap().port(), + "0123456789abcdef0123456789abcdef".into(), + ) + .unwrap(); + crate::runtime_instance::write_descriptor(&descriptor_path, &descriptor).unwrap(); + let guard = try_acquire_launcher_guard(&guard_path).unwrap().unwrap(); + let executable = std::env::current_exe().unwrap().canonicalize().unwrap(); + assert_eq!( + stop_managed_runtime_at( + &descriptor_path, + &guard_path, + &root.join("other"), + Duration::from_secs(1) + ) + .unwrap_err() + .to_string(), + "The running Desktop belongs to another codexhost installation" + ); + let removed = descriptor_path.clone(); + let server = thread::spawn(move || { + let (mut socket, _) = listener.accept().unwrap(); + let mut line = String::new(); + BufReader::new(socket.try_clone().unwrap()) + .read_line(&mut line) + .unwrap(); + socket.write_all(b"stopping\n").unwrap(); + thread::sleep(Duration::from_millis(100)); + std::fs::remove_file(removed).unwrap(); + drop(guard); + }); + stop_managed_runtime_at( + &descriptor_path, + &guard_path, + &executable, + Duration::from_secs(2), + ) + .unwrap(); + server.join().unwrap(); + std::fs::remove_dir_all(root).unwrap(); + } +} diff --git a/crates/launcher/src/runtime_instance.rs b/crates/launcher/src/runtime_instance.rs index 42debcfd7..c88a17995 100644 --- a/crates/launcher/src/runtime_instance.rs +++ b/crates/launcher/src/runtime_instance.rs @@ -42,7 +42,7 @@ pub fn classify_startup(observation: StartupObservation) -> StartupState { if observation.desktop_running { return StartupState::Attach; } - if observation.descriptor_present && !observation.control_endpoint_ready { + if observation.descriptor_present { return StartupState::RecoverStale; } StartupState::CleanLaunch @@ -362,6 +362,14 @@ mod tests { }), StartupState::RecoverStale ); + assert_eq!( + classify_startup(StartupObservation { + desktop_running: false, + descriptor_present: true, + control_endpoint_ready: true, + }), + StartupState::RecoverStale + ); assert_eq!( classify_startup(StartupObservation { desktop_running: false, diff --git a/crates/launcher/tests/cli.rs b/crates/launcher/tests/cli.rs index 82fa69638..9bc4797c7 100644 --- a/crates/launcher/tests/cli.rs +++ b/crates/launcher/tests/cli.rs @@ -32,8 +32,12 @@ fn production_launcher_uses_the_three_state_running_desktop_flow() { assert!(source.contains("StartupState::CleanLaunch")); assert!(source.contains("StartupState::Attach")); assert!(source.contains("acquire_launcher_ownership")); + assert!(source.contains("recover_stale_controller")); assert!(source.contains("completely quit it before starting codexhost")); assert!(!source.contains("attach_unmanaged_desktop")); + assert!(!source.contains( + "codexhost control endpoint is still active without a live Desktop; retry after it exits" + )); } #[test] diff --git a/crates/platform/Cargo.toml b/crates/platform/Cargo.toml index 7068e0500..8520d4036 100644 --- a/crates/platform/Cargo.toml +++ b/crates/platform/Cargo.toml @@ -6,6 +6,7 @@ rust-version.workspace = true publish.workspace = true [dependencies] +serde_json = "1.0.145" serde = { version = "1.0.228", features = ["derive"] } sha2 = "0.10.9" @@ -17,7 +18,6 @@ nix = { version = "0.31.3", features = ["feature", "fs", "process", "signal", "u [target.'cfg(target_os = "linux")'.dependencies] rustix = { version = "1.1.4", features = ["process"] } -serde_json = "1.0.145" [target.'cfg(target_os = "macos")'.dependencies] libproc = "0.14.11" diff --git a/crates/platform/src/asar_metadata.rs b/crates/platform/src/asar_metadata.rs new file mode 100644 index 000000000..23d377704 --- /dev/null +++ b/crates/platform/src/asar_metadata.rs @@ -0,0 +1,137 @@ +//! Bounded reads of Desktop product metadata; MSIX deployment identity lives elsewhere. +#[cfg(target_os = "windows")] +use std::fs::File; +use std::io::{Read, Seek, SeekFrom}; +#[cfg(target_os = "windows")] +use std::path::Path; + +use serde_json::Value; + +const MAX_HEADER_BYTES: u32 = 32 * 1024 * 1024; +const MAX_PACKAGE_BYTES: u64 = 64 * 1024; + +#[cfg(target_os = "windows")] +pub(crate) fn desktop_product_version(path: &Path) -> Option { + read_product_version(&mut File::open(path).ok()?) +} + +fn read_product_version(reader: &mut (impl Read + Seek)) -> Option { + let length = reader.seek(SeekFrom::End(0)).ok()?; + reader.seek(SeekFrom::Start(0)).ok()?; + let mut prefix = [0_u8; 16]; + reader.read_exact(&mut prefix).ok()?; + let word = |offset| { + u32::from_le_bytes( + prefix[offset..offset + 4] + .try_into() + .expect("four-byte prefix word"), + ) + }; + let header_size = word(4); + let json_size = word(12); + if word(0) != 4 + || !(8..=MAX_HEADER_BYTES).contains(&header_size) + || word(8) != header_size - 4 + || json_size > header_size - 8 + { + return None; + } + let data_start = 8 + u64::from(header_size); + if data_start > length { + return None; + } + let mut header = vec![0; json_size as usize]; + reader.read_exact(&mut header).ok()?; + let header: Value = serde_json::from_slice(&header).ok()?; + let entry = header.get("files")?.get("package.json")?; + if entry.get("link").is_some() || entry.get("unpacked").is_some_and(|v| v != false) { + return None; + } + let size = entry.get("size")?.as_u64()?; + if size == 0 || size > MAX_PACKAGE_BYTES { + return None; + } + let offset = entry.get("offset")?.as_str()?.parse::().ok()?; + let start = data_start.checked_add(offset)?; + if start.checked_add(size)? > length { + return None; + } + reader.seek(SeekFrom::Start(start)).ok()?; + let mut package = vec![0; size as usize]; + reader.read_exact(&mut package).ok()?; + let package: Value = serde_json::from_slice(&package).ok()?; + let version = package.get("version")?.as_str()?; + let components: Vec<_> = version.split('.').collect(); + (version.len() <= 64 + && components.len() == 3 + && components + .iter() + .all(|part| !part.is_empty() && part.bytes().all(|b| b.is_ascii_digit()))) + .then(|| version.to_owned()) +} + +#[cfg(test)] +pub(crate) mod tests { + use super::*; + use serde_json::json; + use std::io::Cursor; + + pub(crate) fn archive(entry: Value, package: &[u8]) -> Vec { + let header = serde_json::to_vec(&json!({"files":{"package.json":entry}})).unwrap(); + let header_size = (8 + header.len()).next_multiple_of(4) as u32; + let mut bytes = Vec::new(); + for word in [4, header_size, header_size - 4, header.len() as u32] { + bytes.extend_from_slice(&word.to_le_bytes()); + } + bytes.extend_from_slice(&header); + bytes.resize(8 + header_size as usize, 0); + bytes.extend_from_slice(package); + bytes + } + + #[test] + fn reads_product_version_independently_of_msix_deployment_version() { + let package = br#"{"version":"26.1021.31345"}"#; + let bytes = archive(json!({"size":package.len(),"offset":"0"}), package); + assert_eq!( + read_product_version(&mut Cursor::new(bytes)).as_deref(), + Some("26.1021.31345") + ); + } + + #[test] + fn rejects_oversized_truncated_unpacked_linked_and_out_of_bounds_entries() { + let package = br#"{"version":"26.1021.31345"}"#; + for entry in [ + json!({"size":MAX_PACKAGE_BYTES+1,"offset":"0"}), + json!({"size":package.len(),"offset":"18446744073709551615"}), + json!({"size":package.len()+1,"offset":"0"}), + json!({"size":package.len(),"offset":"0","unpacked":true}), + json!({"size":package.len(),"offset":"0","link":"elsewhere"}), + ] { + assert!(read_product_version(&mut Cursor::new(archive(entry, package))).is_none()); + } + let mut bytes = archive(json!({"size":package.len(),"offset":"0"}), package); + bytes[4..8].copy_from_slice(&(MAX_HEADER_BYTES + 1).to_le_bytes()); + assert!(read_product_version(&mut Cursor::new(bytes)).is_none()); + assert!(read_product_version(&mut Cursor::new(b"not an asar")).is_none()); + } + + #[test] + fn rejects_invalid_product_metadata_and_four_part_package_versions() { + for package in [ + b"{}".as_slice(), + b"invalid", + br#"{"version":"26.1021.3705.0"}"#, + br#"{"version":"26..31345"}"#, + ] { + assert!( + read_product_version(&mut Cursor::new(archive( + json!({"size":package.len(),"offset":"0"}), + package + ))) + .is_none() + ); + } + } +} diff --git a/crates/platform/src/desktop_launch.rs b/crates/platform/src/desktop_launch.rs index f204d136f..1ac2c1100 100644 --- a/crates/platform/src/desktop_launch.rs +++ b/crates/platform/src/desktop_launch.rs @@ -31,8 +31,7 @@ pub type DesktopProcess = super::windows_desktop::WindowsDesktopProcess; #[cfg(not(target_os = "windows"))] pub type DesktopProcess = Child; -const CODEXHOST_RELEASES_LATEST_URL: &str = - "https://github.com/BytePioneer-AI/codex-host/releases/latest"; +const CODEXHOST_RELEASES_LATEST_URL: &str = "https://github.com/karlorz/codex-host/releases/latest"; const REMOTE_SSH_MANAGED_ENV: &str = "CODEXHOST_REMOTE_SSH_MANAGED"; const REMOTE_PROFILE_ONLY_ENVIRONMENT: [&str; 3] = [ "CODEX_INSTALL_DIR", diff --git a/crates/platform/src/installation.rs b/crates/platform/src/installation.rs index 5a8d2ecb4..5eb689c06 100644 --- a/crates/platform/src/installation.rs +++ b/crates/platform/src/installation.rs @@ -303,7 +303,8 @@ fn windows_installation( appx_activation: Some(details.appx_activation), }, build: details.version.clone(), - version: details.version, + version: crate::asar_metadata::desktop_product_version(&asar_path) + .unwrap_or(details.version), asar_integrity: sha256_file(&asar_path)?, install_root, desktop_launcher: desktop_executable.clone(), @@ -444,8 +445,9 @@ pub fn discover_codex_desktop() -> Result { /// /// A portable/unpacked installation has no registered AppX package, so the /// version is not available from the Windows PackageManager. The official MSIX -/// embeds the marketing version in the `` attribute of -/// its manifest; fall back to a placeholder when it cannot be parsed. +/// embeds its deployment version in the `` attribute of +/// its manifest; fall back to a placeholder when it cannot be parsed. Product +/// metadata comes from app.asar. #[cfg(target_os = "windows")] fn portable_package_version(package_root: &Path) -> Option { let content = std::fs::read_to_string(package_root.join("AppxManifest.xml")).ok()?; @@ -509,7 +511,9 @@ pub fn discover_codex_desktop_from_root( } let executable_codex_cli = packaged_codex_cli.clone(); - let version = portable_package_version(&package_root).unwrap_or_else(|| "0.0.0.0".to_owned()); + let build = portable_package_version(&package_root).unwrap_or_else(|| "0.0.0.0".to_owned()); + let version = + crate::asar_metadata::desktop_product_version(&asar_path).unwrap_or_else(|| build.clone()); Ok(DesktopInstallation { identity: DesktopIdentity::WindowsPackage { @@ -517,7 +521,7 @@ pub fn discover_codex_desktop_from_root( package_family_name: format!("{WINDOWS_CODEX_PACKAGE_NAME}_portable"), appx_activation: None, }, - build: version.clone(), + build, version, asar_integrity: sha256_file(&asar_path)?, install_root: package_root, @@ -670,6 +674,53 @@ mod windows_tests { (root, install_root) } + #[test] + fn msix_product_metadata_does_not_replace_deployment_identity() { + let (root, install_root) = portable_fixture( + "codexhost-windows-msix-metadata", + Some(br#""#), + ); + let package = br#"{"version":"26.1021.31345"}"#; + fs::write( + install_root.join("app/resources/app.asar"), + crate::asar_metadata::tests::archive( + serde_json::json!({"size":package.len(),"offset":"0"}), + package, + ), + ) + .expect("write metadata fixture"); + let portable = super::discover_codex_desktop_from_root(&install_root).unwrap(); + assert_eq!(portable.version, "26.1021.31345"); + assert_eq!(portable.build, "26.1021.3705.0"); + let cache_root = root.join("LocalAppData"); + let cli = cache_root.join("OpenAI/Codex/bin/test/codex.exe"); + fs::create_dir_all(cli.parent().unwrap()).unwrap(); + fs::write(cli, b"cached cli").unwrap(); + let activation = WindowsAppxActivationIdentity { + package_full_name: "OpenAI.Codex_26.1021.3705.0_x64_family".into(), + app_user_model_id: "OpenAI.Codex_family!App".into(), + }; + let registered = windows_installation( + WindowsPackageDetails { + package_name: "OpenAI.Codex".into(), + package_family_name: "OpenAI.Codex_family".into(), + appx_activation: activation.clone(), + version: "26.1021.3705.0".into(), + install_root, + }, + &cache_root, + ) + .unwrap(); + assert_eq!(registered.version, "26.1021.31345"); + assert_eq!(registered.build, "26.1021.3705.0"); + assert!( + matches!(registered.identity, DesktopIdentity::WindowsPackage { + appx_activation: Some(identity), .. + } if identity == activation) + ); + fs::remove_dir_all(root).unwrap(); + } + #[test] fn custom_root_discovers_a_portable_installation_without_a_registered_package() { let (root, install_root) = portable_fixture( diff --git a/crates/platform/src/lib.rs b/crates/platform/src/lib.rs index 12ab780d2..a7b32954a 100644 --- a/crates/platform/src/lib.rs +++ b/crates/platform/src/lib.rs @@ -5,6 +5,8 @@ use std::fmt::{self, Display, Formatter}; use std::io; use std::path::{Path, PathBuf}; +#[cfg(any(target_os = "windows", test))] +mod asar_metadata; #[cfg(target_os = "windows")] #[allow(unsafe_code)] mod background; diff --git a/crates/updater/src/install.rs b/crates/updater/src/install.rs index baec17330..daca67a44 100644 --- a/crates/updater/src/install.rs +++ b/crates/updater/src/install.rs @@ -23,7 +23,7 @@ use crate::request::{ #[cfg(target_os = "macos")] use crate::status::unix_seconds; -const NPM_PACKAGE_NAME: &str = "@codexhost/cli"; +pub(crate) const NPM_PACKAGE_NAME: &str = "@karlorz/codexhost-cli"; #[cfg(any(target_os = "windows", target_os = "macos"))] const DISTRIBUTION_FILE: &str = "codexhost-distribution.json"; diff --git a/crates/updater/src/remote.rs b/crates/updater/src/remote.rs index 61f76f4f6..1e3cc1bd0 100644 --- a/crates/updater/src/remote.rs +++ b/crates/updater/src/remote.rs @@ -201,7 +201,7 @@ fn apply_locked(request: &Request) -> Result<(), Box> { "npm uses a different global installation; update this remote manually".into(), ); } - let package = format!("@codexhost/cli@{}", request.version); + let package = format!("{}@{}", crate::install::NPM_PACKAGE_NAME, request.version); // Unpublished local builds fail here, before touching the installation or service. run(Command::new(&request.node).arg(&request.npm).args([ "view", @@ -231,16 +231,16 @@ fn apply_locked(request: &Request) -> Result<(), Box> { .and_then(|name| name.to_str()) .ok_or("Invalid platform package path")?; if ![ - "cli-darwin-arm64", - "cli-darwin-x64", - "cli-linux-arm64", - "cli-linux-x64", + "codexhost-cli-darwin-arm64", + "codexhost-cli-darwin-x64", + "codexhost-cli-linux-arm64", + "codexhost-cli-linux-x64", ] .contains(&name) { return Err("Unknown remote platform package".into()); } - command.arg(format!("@codexhost/{name}@{}", request.version)); + command.arg(format!("@karlorz/{name}@{}", request.version)); } run(&mut command).map_err(|error| format!("npm installation failed; check remote network access and write permissions: {error}"))?; } @@ -285,7 +285,7 @@ mod tests { std::process::id(), COUNTER.fetch_add(1, Ordering::Relaxed) )); - let package_root = root.join("node_modules/@codexhost/cli-darwin-arm64"); + let package_root = root.join("node_modules/@karlorz/codexhost-cli-darwin-arm64"); fs::create_dir_all(package_root.join("app")).unwrap(); fs::write( package_root.join("app/codexhost-distribution.json"), diff --git a/crates/updater/src/ssh.rs b/crates/updater/src/ssh.rs index b8e3b4acb..675e0679c 100644 --- a/crates/updater/src/ssh.rs +++ b/crates/updater/src/ssh.rs @@ -8,6 +8,8 @@ use std::{ time::{Duration, Instant}, }; +use crate::install::NPM_PACKAGE_NAME; + #[derive(Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] struct Request { @@ -66,7 +68,8 @@ node -e 'const [major, minor] = process.versions.node.split(".").map(Number); pr "install" => { let version = release_version(request)?; Ok(format!( - "{node}set -e; {lock}[ ! -f \"$HOME/.codexhost/remote/manifest.json\" ] || exit 43; case $(uname -s) in Darwin|Linux) ;; *) exit 44;; esac; command -v npm >/dev/null 2>&1 || exit 56; command -v codex >/dev/null 2>&1 || exit 57; npm install -g @codexhost/cli@{version} >/dev/null 2>&1 || exit 46; codexhost remote install >/dev/null 2>&1 || exit 47; codexhost remote start >/dev/null 2>&1 || {started} || exit 48; printf '\\n__CODEXHOST_SSH_STATE__=installed\\n'" + "{node}set -e; {lock}[ ! -f \"$HOME/.codexhost/remote/manifest.json\" ] || exit 43; case $(uname -s) in Darwin|Linux) ;; *) exit 44;; esac; command -v npm >/dev/null 2>&1 || exit 56; command -v codex >/dev/null 2>&1 || exit 57; npm install -g {package}@{version} >/dev/null 2>&1 || exit 46; codexhost remote install >/dev/null 2>&1 || exit 47; codexhost remote start >/dev/null 2>&1 || {started} || exit 48; printf '\\n__CODEXHOST_SSH_STATE__=installed\\n'", + package = NPM_PACKAGE_NAME, )) } // For services too old to update themselves. A newer CLI refuses to stop an installation @@ -75,7 +78,8 @@ node -e 'const [major, minor] = process.versions.node.split(".").map(Number); pr "update" => { let version = release_version(request)?; Ok(format!( - "{node}set -e; {lock}[ -f \"$HOME/.codexhost/remote/manifest.json\" ] || exit 52; command -v npm >/dev/null 2>&1 || exit 56; npm view @codexhost/cli@{version} version --fetch-retries=0 --fetch-timeout=30000 >/dev/null 2>&1 || exit 53; if command -v codexhost >/dev/null; then codexhost remote stop >/dev/null 2>&1 || true; fi; npm install -g @codexhost/cli@{version} >/dev/null 2>&1 || {{ codexhost remote start >/dev/null 2>&1 || true; exit 46; }}; command -v codexhost >/dev/null || exit 54; codexhost remote install >/dev/null 2>&1 || exit 47; codexhost remote stop >/dev/null 2>&1 || {stock} || exit 50; codexhost remote start >/dev/null 2>&1 || {started} || exit 48; printf '\\n__CODEXHOST_SSH_STATE__=installed\\n'" + "{node}set -e; {lock}[ -f \"$HOME/.codexhost/remote/manifest.json\" ] || exit 52; command -v npm >/dev/null 2>&1 || exit 56; npm view {package}@{version} version --fetch-retries=0 --fetch-timeout=30000 >/dev/null 2>&1 || exit 53; if command -v codexhost >/dev/null; then codexhost remote stop >/dev/null 2>&1 || true; fi; npm install -g {package}@{version} >/dev/null 2>&1 || {{ codexhost remote start >/dev/null 2>&1 || true; exit 46; }}; command -v codexhost >/dev/null || exit 54; codexhost remote install >/dev/null 2>&1 || exit 47; codexhost remote stop >/dev/null 2>&1 || {stock} || exit 50; codexhost remote start >/dev/null 2>&1 || {started} || exit 48; printf '\\n__CODEXHOST_SSH_STATE__=installed\\n'", + package = NPM_PACKAGE_NAME, )) } // A damaged installation is exactly what the CLI refuses to stop, so the first stop is @@ -85,18 +89,24 @@ node -e 'const [major, minor] = process.versions.node.split(".").map(Number); pr )), "uninstall" => { let package_check = if request.uninstall_package { - r#"command -v npm >/dev/null 2>&1 || exit 56; + format!( + r#"command -v npm >/dev/null 2>&1 || exit 56; NPM_ROOT=$(npm root --global 2>/dev/null) || exit 59; CLI=$(command -v codexhost) || exit 58; -node -e 'const fs = require("node:fs"), path = require("node:path"); try { const expected = path.join(process.argv[1], "@codexhost/cli/bin/codexhost.js"); process.exit(fs.realpathSync(expected) === fs.realpathSync(process.argv[2]) ? 0 : 1); } catch { process.exit(1); }' "$NPM_ROOT" "$CLI" >/dev/null 2>&1 || exit 59; -"# +node -e 'const fs = require("node:fs"), path = require("node:path"); try {{ const expected = path.join(process.argv[1], "{package}/bin/codexhost.js"); process.exit(fs.realpathSync(expected) === fs.realpathSync(process.argv[2]) ? 0 : 1); }} catch {{ process.exit(1); }}' "$NPM_ROOT" "$CLI" >/dev/null 2>&1 || exit 59; +"#, + package = NPM_PACKAGE_NAME, + ) } else { - "" + String::new() }; let package_remove = if request.uninstall_package { - "npm uninstall --global @codexhost/cli >/dev/null 2>&1 || exit 60; [ ! -e \"$NPM_ROOT/@codexhost/cli/package.json\" ] || exit 60;" + format!( + "npm uninstall --global {package} >/dev/null 2>&1 || exit 60; [ ! -e \"$NPM_ROOT/{package}/package.json\" ] || exit 60;", + package = NPM_PACKAGE_NAME, + ) } else { - "" + String::new() }; // Keep the npm package until its CLI has stopped the service and removed the // managed entrypoint/profile (and the macOS broker). No user data is deleted. diff --git a/crates/updater/src/ssh/tests.rs b/crates/updater/src/ssh/tests.rs index 118342dc8..d9e83d9a4 100644 --- a/crates/updater/src/ssh/tests.rs +++ b/crates/updater/src/ssh/tests.rs @@ -165,7 +165,7 @@ node(){ case "$2" in *realpathSync*) return "${MATCH_CODE:-0}";; *) return 0;; e npm(){ case "$1" in root) printf '%s\n' "$HOME/npm";; - uninstall) printf '%s\n' 'npm uninstall' >> "$HOME/calls"; [ "${FAIL:-}" != npm ] || return 1; rm "$HOME/npm/@codexhost/cli/package.json";; + uninstall) printf '%s\n' 'npm uninstall' >> "$HOME/calls"; [ "${FAIL:-}" != npm ] || return 1; rm "$HOME/npm/@karlorz/codexhost-cli/package.json";; *) return 99;; esac } @@ -224,7 +224,7 @@ codexhost(){ ] { let home = root.join(name); let remote = home.join(".codexhost/remote"); - let package = home.join("npm/@codexhost/cli"); + let package = home.join("npm/@karlorz/codexhost-cli"); std::fs::create_dir_all(remote.join("data")).unwrap(); std::fs::create_dir_all(&package).unwrap(); std::fs::write(remote.join("manifest.json"), "{}").unwrap(); @@ -370,7 +370,7 @@ fn install_refuses_to_replace_existing_remote_services() { r.version = Some("1.2.3".into()); let value = script(&r).unwrap(); assert!(value.contains("exit 43")); - assert!(value.contains("@codexhost/cli@1.2.3")); + assert!(value.contains("@karlorz/codexhost-cli@1.2.3")); assert!(!value.contains("remote stop")); } #[cfg(unix)] @@ -399,7 +399,7 @@ fn executes_initial_setup_and_refuses_an_existing_installation() { let calls = std::fs::read_to_string(root.join("calls")).unwrap(); assert_eq!( calls, - "install -g @codexhost/cli@1.2.3\nremote install\nremote start\n" + "install -g @karlorz/codexhost-cli@1.2.3\nremote install\nremote start\n" ); std::fs::create_dir_all(root.join(".codexhost/remote")).unwrap(); std::fs::write(root.join(".codexhost/remote/manifest.json"), "{}").unwrap(); @@ -421,7 +421,7 @@ fn executes_initial_setup_and_refuses_an_existing_installation() { .unwrap(); assert!(updated.status.success()); let calls = format!( - "{calls}view @codexhost/cli@1.2.4 version --fetch-retries=0 --fetch-timeout=30000\nremote stop\ninstall -g @codexhost/cli@1.2.4\nremote install\nremote stop\nremote start\n" + "{calls}view @karlorz/codexhost-cli@1.2.4 version --fetch-retries=0 --fetch-timeout=30000\nremote stop\ninstall -g @karlorz/codexhost-cli@1.2.4\nremote install\nremote stop\nremote start\n" ); assert_eq!(std::fs::read_to_string(root.join("calls")).unwrap(), calls); assert!(!root.join(".codexhost/ssh-setup.lock").exists()); diff --git a/docs/NOTICE.md b/docs/NOTICE.md index 7c1a77097..42035611e 100644 --- a/docs/NOTICE.md +++ b/docs/NOTICE.md @@ -12,4 +12,4 @@ type: warning - 用户需要采取的临时措施。 - 修复进度或预计恢复时间。 -[查看项目发布记录](https://github.com/BytePioneer-AI/codex-host/releases) +[查看项目发布记录](https://github.com/karlorz/codex-host/releases) diff --git a/docs/PRIVACY.md b/docs/PRIVACY.md index a76443f50..3bb1cdd21 100644 --- a/docs/PRIVACY.md +++ b/docs/PRIVACY.md @@ -32,4 +32,4 @@ After stopping CodexHost, you can delete its applicable local data. Native Harne ## Contact -For non-sensitive privacy questions, open an issue at [BytePioneer-AI/codex-host](https://github.com/BytePioneer-AI/codex-host/issues). Do not include credentials, private code, or unredacted logs in public reports. +For non-sensitive privacy questions, open an issue at [karlorz/codex-host](https://github.com/karlorz/codex-host/issues). Do not include credentials, private code, or unredacted logs in public reports. diff --git a/docs/architecture/external-thread-steering.md b/docs/architecture/external-thread-steering.md index 1f2ddde23..0e4d002c3 100644 --- a/docs/architecture/external-thread-steering.md +++ b/docs/architecture/external-thread-steering.md @@ -77,9 +77,11 @@ Host 和 Renderer 接入必须配套发布;仅升级 Host、让旧 Renderer ## 跟进消息队列 -Codex Desktop **26.903.61454 / build 8378** 在功能开关及 app-server 版本满足条件时使用服务端队列。运行中追加消息会先调用 `thread/queue/list`,再调用 `thread/queue/add`;仅让 Host 返回空列表不能恢复入队。外部 Thread 不属于官方 app-server,Host 对这些请求的拒绝仍需保留,不能透传或伪造服务端队列。 +Codex Desktop **26.903.61454 / build 8378** 起,在功能开关及 app-server 版本满足条件时使用服务端队列。运行中追加消息会先调用 `thread/queue/list`,再调用 `thread/queue/add`。Desktop **26.1016** 及之后的 Manager 带有 `getStreamRole` 时,`renderer-external-queue.ts` 不再把外部 Thread 从 `serverQueue.isEnabled` 里排除,因此跟进消息走 Host 的 `thread/queue/*`。Host 按官方列表信封返回 `{ data, nextCursor }`。队列里的文本项写成 Desktop `type: "text"`(`input_text` 入队时也转成该类型),并带有 `text_elements` 数组(缺省 `[]`),供 Desktop **26.1021.31345 / build 21828** 读取 `text_elements.length`。空的 `turn/start` 会消费 Host 队列中匹配或队首的跟进文本;队列为空时仍返回 `-32602`。官方 Codex Thread 仍使用原生 app-server 队列。 -`renderer-external-queue.ts` 在当前 Manager 的 `getTurnCoordinator().serverQueue.isEnabled(threadId)` 上排除外部 Thread,让 Desktop 继续使用保留的本地队列: +Avatar overlay 是独立的 Desktop webcontents,在语音、IAB 或 Computer Use 之后可以成为该 Thread 的 stream owner。主窗口的 Renderer Binding 不会注入 overlay;`cpa-live-voice-hook.js` 在 overlay 里发现 RequestManager 后安装 steer 包装,并把 External Thread 的 stream role 视为 owner,使 follower 窗口可以把跟进消息写入 Host `thread/queue/*`。Overlay 作为 owner 通过 `thread/queue/start` 执行队列。缺少 type:text 的 `turn/start` 会从 Host 队列或本地队列条目恢复文本。 + +没有 `getStreamRole` 的 Desktop 上,`renderer-external-queue.ts` 仍在当前 Manager 的 `getTurnCoordinator().serverQueue.isEnabled(threadId)` 上排除外部 Thread,让 Desktop 继续使用保留的本地队列: - 只读取同一个 Manager 的 `getConversation(threadId)`,核对 Thread ID 及 Host 投影的保留标记 `modelProvider: "codexhost"`。不根据当前 Agent 选择或 Model 名称猜测,不跨 Host 缓存身份,也不增加 RPC。 - 普通入队、编辑、重排、删除/恢复、暂停和终态后的自动执行仍由 Desktop 原有队列负责,不新增 Host 队列或 Harness 命令。 @@ -112,6 +114,10 @@ Codex Desktop **26.924.22138 / build 11645** 的本地队列在 `turnCompleted` - `packages/renderer-extension/test/renderer-external-steering.test.ts`:正常 start 占位、唯一输入、无旧轮 steer Item、去重、队列暂停恢复、owner/follower、失败和卸载。 - `packages/renderer-extension/test/renderer-external-steering-rpc.test.ts`:类方法的 RPC 可访问性、模型/权限列表读取、官方与外部 steer、实例隔离、卸载和重复安装,以及卸载期间待定请求的清理。 - `packages/renderer-extension/test/renderer-external-queue.test.ts`:经生产 Adapter 安装路径验证外部入队、已有暂停消息、队列清空后再次入队、同连接官方队列、Manager 隔离、元数据更新、旧版后端和卸载恢复。 +- `packages/renderer-extension/test/renderer-overlay-external-followups.test.ts`:avatar overlay 发现 RequestManager 后把 External Thread 视为 stream owner。 +- `packages/renderer-extension/test/renderer-external-steering.test.ts`:缺少 type:text 的 queued `turn/start` 从本地队列恢复文本。 +- `packages/host-runtime/test/app-server-host.follow-up-queue.test.ts`:Host `thread/queue/add` 与 `thread/queue/start`,空 `turn/start` 消费队列,以及中断恢复后空队列仍返回 `-32602`。 +- `packages/host-runtime/test/external-thread-follow-up-queue.test.ts`:入队项补齐 `text_elements`,并把 `input_text` 规范为 `type: "text"`。 - `packages/renderer-extension/test/versioned-renderer-adapter.test.ts`:现有版本化绑定与清理回归。 - `packages/host-runtime/test/app-server-host.manual-compaction.test.ts`:命令菜单与输入 `/compact` 两条路径在压缩 `item/started` 前恰好发送一次通知,普通 Turn 与非 `/compact` 命令 Turn 的自动压缩均不发送。 - `packages/renderer-extension/test/renderer-manual-compaction.test.ts`:经窗口 `mcp-notification` 消息的外部 Thread 登记,follower、未加载/官方 Thread、其他 Host、其他 frame 与非法消息跳过,缺少 Desktop 绑定、卸载,以及按 Host 连接(含 SSH)安装。 diff --git a/docs/architecture/harness-executable-discovery.md b/docs/architecture/harness-executable-discovery.md index eb74b2771..b3ae1c2bb 100644 --- a/docs/architecture/harness-executable-discovery.md +++ b/docs/architecture/harness-executable-discovery.md @@ -28,7 +28,7 @@ packages/harness-discovery/ } ``` -该包参与 TypeScript 构建,但不会作为独立 npm 包发布。正式 npm 发布物仍是 `@codexhost/cli` 及对应的平台包;发现代码随 Host Runtime 和 Adapter 的发布产物一起交付。 +该包参与 TypeScript 构建,但不会作为独立 npm 包发布。正式 npm 发布物仍是 `@karlorz/codexhost-cli` 及对应的平台包;发现代码随 Host Runtime 和 Adapter 的发布产物一起交付。 ## 公共能力 diff --git a/docs/architecture/harness-plugin-architecture.md b/docs/architecture/harness-plugin-architecture.md index 85f7c1fef..8c245418c 100644 --- a/docs/architecture/harness-plugin-architecture.md +++ b/docs/architecture/harness-plugin-architecture.md @@ -107,6 +107,7 @@ Codex Desktop 协议、官方 app-server、Renderer 兼容绑定仍是 codexhost - [`run-host-runtime.ts`](../../packages/host-runtime/src/run-host-runtime.ts) 显式预取 Claude/Antigravity 目录。 - [`harness-broker`](../../packages/harness-broker/src/protocol.ts) 的描述、客户端与服务端实际限定 Claude Code,不是已经通用的插件进程协议。 - [`delegation-skill.ts`](../../packages/host-runtime/src/delegation-skill.ts) 包含 `.claude` 安装位置。 +- [`codexhost-desktop-rule.ts`](../../packages/host-runtime/src/codexhost-desktop-rule.ts) 只回收 Host 曾经写入的 `~/.grok/rules/codexhost-desktop.md`;Grok 会话规则走 Adapter 的 ACP `_meta.rules`。 - 基线中的 [`remote-host-install.ts`](../../packages/host-runtime/src/remote-host-install.ts) 曾固化 `claudeCommand`;该安装与启动特例现已移除,Harness 可执行文件发现仍由 Adapter 负责。 - 发布脚本维护各原生 SDK 与许可证。 diff --git a/docs/architecture/harness-plugin-runtime.md b/docs/architecture/harness-plugin-runtime.md index 74dffdbe9..f57b5d939 100644 --- a/docs/architecture/harness-plugin-runtime.md +++ b/docs/architecture/harness-plugin-runtime.md @@ -141,6 +141,10 @@ Codex 的会话用量解析已从公共协议层迁入 `codex-usage`,账户配 **修改需要重启 codexhost。** 已创建的 Adapter 与 Session 不热替换;`restartRequired` 比较当前持久化值与此 Host 构造时的值。仅刷新连接状态不会应用新路径。设置页填写应用安装目录,例如 `D:\program\WorkBuddy`,不要求用户定位 `.exe` 或脚本。WorkBuddy Adapter 定位 `WorkBuddy.exe` / `WorkBuddy AI.exe` / `WorkBuddyAI.exe` 及同目录内置脚本。目录布局不完整时检查失败,不借用其他安装的文件,也不回退到 PATH 或默认安装。底层保留原有文件入口覆盖兼容能力。注册表自动发现不在本功能范围内。 +## Grok Computer Use 设置 + +本地 Host 的 Grok 连接详情和未运行时的控制台连接页提供 Computer Use 开关。`codexhost/harness/grok-computer-use/get·set` 读写 `~/.codexhost/grok-computer-use.json`,(`enabled` 与 `browser` 两个独立开关)。Computer Use 需要 `cua_node` 与 Sky;In-app browser 需要 `cua_node` 与 `@oai/browser-desktop`,不依赖 Sky。在 Desktop 为外部 Harness 提供授权与会话绑定之前,`browser` 只保存意向,不写入 `browser` 表面。写入先改 Grok config,再写 preference;失败时返回 `applyFailed`。ChatGPT 设置中的 Computer use / Browser 仍只作用于官方 Codex。新的 Grok 线程读取该开关;已打开的线程不热加载 MCP。 + ## Harness CLI 版本与更新 设置 → 连接,在已安装 Harness 的右侧详情展示「Harness CLI 版本」、当前版本、最新版本与更新按钮。进入连接页或切换 Host 时,自动检查当前 Host 列表中已安装且不在安装、检测中的 Harness,无须先选择详情;诊断完成后符合条件的 Harness 也会自动检查。有新版本时,列表操作区显示灰色「有更新」小字,悬停展示当前与最新版本;支持自动更新时在小字右侧提供更新按钮,点击后才调用该 Host 的原生更新能力,更新期间禁用重复操作。仅支持手动更新的安装保留「有更新」小字及详情中的原安装方式提示。不提供手动检查更新按钮;后台诊断刷新和切换列表不会丢弃该页按 Host/Harness 保存的检查结果或重复启动更新。重新打开连接页后再次自动检查。所选 Harness 的详情卡片标题旁提供官网外链图标,列表行及详情正文不重复提供通用官网或安装说明链接。未安装且有自动安装入口时显示下载图标;点击显式发起 CLI 安装,点击行本身查看手动安装命令、复制按钮、接入提示和「重新检测」,正文不提供额外的通用安装指南跳转入口。WorkBuddy 在详情正文提供一个「下载」按钮,跳转官网首页,并提示下载安装桌面应用,不提供分系统的安装指南链接。未支持版本管理能力的插件、旧 Host 和 Broker 路径显示不可用,不回退更新本机的另一份安装。 diff --git a/docs/harnesses/antigravity/antigravity-subagents.md b/docs/harnesses/antigravity/antigravity-subagents.md index 17de3095a..afe42ab83 100644 --- a/docs/harnesses/antigravity/antigravity-subagents.md +++ b/docs/harnesses/antigravity/antigravity-subagents.md @@ -20,7 +20,7 @@ Validated with agy 1.1.27 on Windows: The Antigravity Adapter emits the existing `subagentDelegation`, `subagent.state.changed`, and `subagent.transcript.changed` contracts. The common Host creates a read-only child Thread and the common projector emits -`collabAgentToolCall`. No Antigravity-specific Renderer or Host RPC is added. +`subAgentActivity` for Desktop’s native read-only Subagents panel. No Antigravity-specific Renderer or Host RPC is added. The creation Item completes when the native spawn call completes. Its child remains running until the separate native status confirms otherwise. During an diff --git a/docs/harnesses/capability-boundaries.md b/docs/harnesses/capability-boundaries.md index 33ecb7e9c..52a658f60 100644 --- a/docs/harnesses/capability-boundaries.md +++ b/docs/harnesses/capability-boundaries.md @@ -9,6 +9,10 @@ - **CodeBuddy**:从 ACP 动态目录读取斜杠命令,按原生压缩事件及持久化结果确认上下文压缩;通过原生附加系统提示让会话发现 Host 委派 CLI。Fork 和修订通过无模型的原生复制、原生 `/fork` 与仅作用于新副本的 rollback 完成,校验完整历史前缀与源会话不变;派生配置随原生引用恢复。成功派生后通过同一临时 ACP 进程的原生 HTTP 接口删除中间副本;早期失败或强杀仍可能残留,详见专属文档。详见 [CodeBuddy 接入](codebuddy/codebuddy-harness-integration.md)。 - **Cursor**:参数化模型目录提供 Thinking 组合;ACP 原生命令目录提供命令;原生 shell 通过 Host 安装的 `codexhost-delegation` skill 调用 CLI 向外委派。详见 [Cursor 接入](cursor/cursor-cli-experimental.md)。 - **Hermes**:会话统一使用官方 Gateway,不以版本数值差异阻断,接入提问、Thinking、命令、压缩、原生 Diff,以及未压缩历史的独立 Fork/修订;通过进程内注册的私有临时 Skill 发现 Host CLI 并保留父任务环境,不向共享技能目录写入临时 Skill。不再保留 ACP 实现或 SDK 依赖;历史发现和导入使用原生 SessionDB 只读查询,导入的原生身份通过 Gateway 恢复。详见 [Hermes 能力与边界](hermes/hermes-capabilities.md)。 +- **Grok**:本 fork 使用补丁版 **grokgod Grok build**,补丁清单见本机 `~/.grokgod/src/docs/patch-inventory.md`,不能把依赖补丁的行为视为 stock Grok 能力。 + - MCP 同意表单通过 `_x.ai/mcp/elicit` 投影为 Desktop 审批(Allow once / Deny)。Always allow 依赖 grokgod 补丁 `0025-mcp-elicitation-persist`:原生 MCP persist 选项经 ACP 转发,接受后的范围作为 MCP `_meta.persist` 回传;Host 仅在请求包含 `always` 时显示该选项。未打补丁的 Grok 不提供该持久化往返,只有 Allow once / Deny。Allow this conversation 在原生 session 生命周期核实前不提供。仅支持无字段或已核验的空 object schema;URL elicitation、任意输入表单和原生 TUI elicitation 卡片均不提供。工具 `requestPermission` 仍走独立路径。详见 [Grok MCP elicitation](grok/mcp-elicitation.md)。 + - 原生 `ask_user_question` 已有于 Grok;grokgod 补丁 `0011-ask-question-timeout-action` 增加可配置的超时选择和基于活动的超时重置,并非 Desktop 提问接入。Host ACP 客户端的扩展请求处理器只实现 `_x.ai/mcp/elicit`,对 `_x.ai/ask_user_question` 返回 `Method not found`。用户报告的 grokgod ACP 实测错误为 `Failed to reach the client for user question: "Method not found": _x.ai/ask_user_question`,因此 Desktop 不出现提问卡片,工具调用收到错误。补丁不会补齐缺失的客户端方法;接入仍需在 Grok 插件内映射原生请求/应答,不能复用审批或 MCP elicitation 的形状。回合取消有独立的已实现路径。 + - Computer Use 使用用户安装的 `cua_repl` MCP,依赖 grokgod 补丁 `0026-cua-turn-end-release` 提供原生回合边界清理及 Mac Sky 原生释放桥,并需要已安装的 ChatGPT Computer Use runtime;未证明 stock Grok 具备同等行为。CodexHost 连接页分别保存 Computer Use 与 In-app browser(IAB)意向;当前 Host 固定关闭 IAB 绑定,等待 Desktop 的外部 Harness 授权及会话所属的浏览器路由。ChatGPT 设置中的 Computer use / Browser 仍只作用于官方 Codex。详见 [Grok Computer Use setting](grok/computer-use-settings.md)。 以上实现归各 Harness 插件所有,不在公共 Adapter、Host 或 Renderer 中添加 Harness 专用分支。跨插件参考实现不等于共用原生协议;各自保留配置确认、取消、持久化和权限语义。 diff --git a/docs/harnesses/grok/computer-use-release.md b/docs/harnesses/grok/computer-use-release.md new file mode 100644 index 000000000..f441d87e4 --- /dev/null +++ b/docs/harnesses/grok/computer-use-release.md @@ -0,0 +1,43 @@ +# Grok Computer Use release + +Grok uses its user-installed `cua_repl` MCP. Computer Use release requires grokgod patch `0026-cua-turn-end-release` on Grok Build `2bdd1d6a` (1.0.45). An unpatched Grok CLI keeps the MCP connection alive without notifying it when a turn ends. CodexHost does not install `computer-use@openai-bundled` into Grok. + +## Turn lifecycle + +The native Grok Session actor awaits cleanup on successful and failed completion, cancellation (including rewind), and Session close. Completion cleanup runs after claiming the exact turn finalization lease and before replying to the ACP prompt. Cancellation cleanup runs after aborting the matching task. Stale completions cannot release a successor turn. + +Cleanup calls the hidden `cua_repl.turn_ended` tool through the existing MCP connection: + +```json +{ + "hook_event_name": "Stop", + "session_id": "", + "turn_id": "" +} +``` + +Cancellation uses `Interrupt`; completed Subagent turns use `SubagentStop`. Model tool calls carry the same identities in MCP `_meta["x-codex-turn-metadata"]`, together with their call ID. The cleanup tool is excluded from the model's tool catalog. + +## macOS Sky bridge + +The installed `cua_node` runtime advertises `turn_ended`, but its Mac Sky service does not release native control on that notification alone. Resetting the REPL or closing the MCP is also insufficient. Grokgod adds a narrow trusted-service wrapper when the configured server is named `cua_repl`, enables the `computer` surface, and uses `@oai/sky/service`. + +The wrapper delegates ordinary Sky requests and error classification to the installed service. After `turn_ended`, the host calls the wrapper's release RPC over the same MCP. It sends Sky's `ComputerUseIPCCodexTurnEndedRequest` through the installed trusted native-pipe transport, with `{ threadID: session_id, turnID: turn_id }`. Sky then rejects further Computer Use in that turn; a later turn can use it again. + +The wrapper is materialized at `/computer-use/sky-release-service.mjs`. The child MCP's trusted-service environment selects it and adds the exact wrapper file to trusted code paths; user configuration files are unchanged. Other configured trusted services are preserved. The wrapper uses the installed Sky package and IPC version `CodexComputerUseIPC-5`; incompatible or custom Sky services fail closed. + +## Failure and limits + +- The entire cleanup has a three-second deadline. A missing `turn_ended` prevents Computer Use tool admission. A tool error, protocol error, or timeout disables that MCP connection; restart the Session after fixing the runtime. +- Successful cleanup preserves the MCP process and REPL bindings. App approvals and Always allow keep their existing native elicitation behavior; see [MCP elicitation](mcp-elicitation.md). +- Release follows native turn boundaries. There is no additional idle timer during a running turn. Sky's default 120-second IPC timeout bounds individual requests and is separate from turn cleanup. +- Abrupt process termination, OS failure, and independently running sessions are outside the awaited cleanup path. Sky owns its overlay and input enforcement; CodexHost does not draw a replacement overlay. +- The Mac bridge depends on an installed private Sky transport contract. Windows and Linux use the MCP notification path and require their own live acceptance checks. + +## Verification + +Focused native MCP tests cover notification payloads, connection reuse, hidden-tool admission, turn metadata, Mac bridge ordering, missing tools, protocol errors, and timeout. Live Mac contract probes use Clock and verify that Sky rejects a read in an ended turn with “Computer Use is unavailable because the current turn ended.” JavaScript bindings remain available after release. + +The full patched-Grok Clock-through-ACP check and visible overlay disappearance remain pending: Sky reported that the user explicitly stopped Computer Use during live verification. The direct native contract check proves that ended-turn control is rejected, but does not by itself prove the full ACP path or overlay disappearance. + +Session-specific build and ACP acceptance evidence is recorded in the wiki work item `projects/codex/work/2026-10-21-grok-computer-use-release/`. diff --git a/docs/harnesses/grok/computer-use-settings.md b/docs/harnesses/grok/computer-use-settings.md new file mode 100644 index 000000000..0638e60d1 --- /dev/null +++ b/docs/harnesses/grok/computer-use-settings.md @@ -0,0 +1,79 @@ +# Grok Computer Use and in-app browser settings + +CodexHost enables Grok's ChatGPT-backed `cua_repl` surfaces from two independent switches. Nobody edits `~/.grok/config.toml` by hand. + +## Surfaces + +- **Codex Desktop → Settings → Connections → Grok** and **codexhost console → Connections** show one card with two switches: **Computer Use** and **In-app browser**. The Harness list and Harness CLI version block stay as they are. +- Both switches write `~/.codexhost/grok-computer-use.json` (`{ "enabled": boolean, "browser": boolean }`; a file with only `enabled` reads as `browser: false`) and upsert `[mcp_servers.cua_repl]` in `~/.grok/config.toml`. +- RPC `codexhost/harness/grok-computer-use/set` and `POST /api/harnesses/grok-computer-use` accept `{ enabled }`, `{ browser }`, or both. +- CodexHost does not inject `computer-use@openai-bundled`. The stanza uses ChatGPT's `cua_node`, `cua-repl` and the same trusted-service names as the official plugin. + +## Switch matrix + +| Computer Use | In-app browser | `CUA_REPL_ENABLED_SURFACES` | Trusted services | `BROWSER_USE_AVAILABLE_BACKENDS` | +| --- | --- | --- | --- | --- | +| off | off | stanza `enabled = false` | empty JSON object (inert) | unset | +| on | off | `computer` | `sky` | unset | +| on | on, binding unsupported | `computer` | `sky` | unset | +| off | on, binding unsupported | stanza `enabled = false` | empty JSON object (inert) | unset | + +Computer-only users keep the computer-only stanza. The in-app browser switch stores intent in `~/.codexhost/grok-computer-use.json` even while ChatGPT Desktop refuses Grok IAB. Until Desktop authorizes an external Harness, CodexHost does not add the `browser` surface. + +Availability is three separate facts: saved intent (`enabled` / `browser`), installed files (`available` / `browserFilesPresent`), and usable binding (`browserAvailable`). File presence is not IAB success. + +The current implementation sets `GROK_BROWSER_BINDING_SUPPORTED = false`. Both `browserAvailable` and the generated Grok configuration use that policy flag. The console's authorization explanation describes the known unsupported integration; reading settings does not perform a live Desktop authorization check. Restarting Desktop or creating a new Grok thread does not change this policy. + +Computer Use needs the shared `cua_node` runtime, the packaged Codex CLI, and `Codex Computer Use.app`. The browser switch needs that same runtime plus `@oai/browser-desktop`; it does not need Sky. + +Saves and session applies acquire filesystem locks for both the preference directory and Grok config directory, so separate Host and console processes re-read intent inside the same critical section. Grok config is written first, then the preference file. A failed publication attempts to restore the previous config and records `applyFailed` without accepting the new intent. New Grok threads stop before transport startup if applying settings fails. Locks time out after five seconds without stealing another writer’s lock; an abandoned `.grok-computer-use.lock` or `.codexhost-cua.lock` directory requires removal after confirming that its writer has stopped. + +## IAB, Computer Use and Chrome extension + +| Path | What it drives | Owner | +| --- | --- | --- | +| In-app browser (IAB) | ChatGPT Desktop's built-in browser pane through `cua.getBrowser({ id: "iab" })` / `createBrowserTab` | Grok `cua_repl` `browser` surface, opt-in here. Official Codex uses the same backend through ChatGPT **Settings → Browser** | +| Computer Use | Mac apps through Sky (`Codex Computer Use.app`) | Grok `cua_repl` `computer` surface, opt-in here. ChatGPT **Settings → Computer use** governs official Codex | +| Chrome extension | A user's Chrome profile | Official Codex only. Not wired to Grok and out of scope here | + +The official Browser Use helpers are never intercepted (see [macOS native tools](../../platforms/macos/macos-native-tools.md)). Grok gets its own `cua_repl` process. + +## Default and availability + +- Users who never opted in stay off. An enabled legacy `cua_repl` stanza without a surface list reads as Computer Use on, browser off. An explicit surface list preserves separate Computer Use and browser intent; a browser-only stanza never opts into Computer Use. +- Computer Use needs ChatGPT.app `cua_node`, `cua-repl`, the packaged Codex CLI, and `Codex Computer Use.app`. +- The browser switch also needs `cua_node/lib/node_modules/@oai/browser-desktop`. Without it the switch cannot be turned on. +- New Grok threads read the switches. An open thread does not reload MCP. + +## Fail-closed binding + +ChatGPT Desktop serves IAB through per-session unix sockets and checks each caller: + +1. The MCP call must carry `x-codex-turn-metadata.session_id` equal to a Desktop session with a captured browser route. +2. The native pipe verifies the code-signing identity of the connecting process and its parent chain (OpenAI `com.openai.codex*` identities). + +When either check fails, `cua.getBrowser({ id: "iab" })` reports `Browser is not available: iab` and Grok does not open a tab. CodexHost does not synthesize session ids, disable peer authorization, or reuse another backend as IAB evidence. + +## Verified on Desktop 26.1016.51333 + +A real Grok ACP session invoked a read-only diagnostic MCP tool and its own browser-only `cua_repl` server. The diagnostic received no `x-codex-turn-metadata`: session and turn metadata were absent, an effective session could not be compared with the native Grok session, and Desktop route ownership was unverified. The subsequent Grok IAB selection was unavailable. Existing Desktop logs also contain the native peer rejection reason `missing-code-signing-identity`; that log observation is separate from metadata captured by the diagnostic tool and does not prove route registration. + +In an official Codex thread, the official Browser Use tool successfully bound IAB, opened a blank test tab, read its state and closed it. The Grok probe’s before/after snapshot found `~/.codex/config.toml`, `auth.json` and `AGENTS.md` byte-identical. No official Browser Use helper was intercepted or reconfigured. Computer Use remains an independent surface. + +`browserAvailable` stays false, including when `CODEXHOST_GROK_IAB_BINDING` is inherited. There is no environment override that qualifies the blocked binding. Sanitized evidence is retained under ignored `.codexhost/update-impact/26.1016.51333/`. + +## Qualification on Desktop 26.1016.62342 + +On macOS, Grok `1.0.45` (`2bdd1d6a6369`) sends `x-codex-turn-metadata` on model tool calls when the MCP server is named `cua_repl`. A real ACP session using a read-only, session-local diagnostic confirmed nonempty session, turn and call identities. The session identity matched the native Grok session, and the awaited `turn_ended` call matched the same session and turn. A diagnostic registered under a different server name does not establish the metadata behavior of `cua_repl`. + +A separate real ACP session supplied the installed browser-only `cua_repl` through its session MCP configuration, without changing global browser preferences or removing native authorization. Browser APIs were exposed, but selecting IAB returned `Browser is not available: iab`. During that probe window Desktop logged six native socket rejections with reason `missing-code-signing-identity`. The installed `cua_node` Node and `node_repl` executables have OpenAI signatures; the installed Grok executable is unsigned. The rejection reason does not identify the exact failed ancestry check. + +This establishes a native authorization blocker after the Host configuration gate is removed for a bounded diagnostic. It does not establish an authorized browser route for the Grok session. Desktop's current bundle retains separate peer authorization and session-route ownership checks; metadata forwarding alone cannot satisfy either check. A supported external caller authorization and session-registration contract is still required before enabling normal Grok browser sessions. + +Sanitized metadata and browser-probe summaries are retained under ignored `.codexhost/update-impact/26.1016.62342/`. They contain presence/equality booleans and rejection reasons rather than raw session identities or transcripts. Actual Grok navigation, turn cleanup of browser resources, and isolation between two browser-enabled Grok sessions remain unverified because binding failed. + +## Limits + +- The in-app browser switch is intent until Desktop supports external authorization and a session-owned browser route. CodexHost does not synthesize session ids, disable peer authorization, or treat another backend as IAB evidence. +- The Grok session id differs from the CodexHost thread id, so even an authorized peer needs a Desktop session route for the id Grok sends. +- Chrome extension and Excel stay out of scope. diff --git a/docs/harnesses/grok/generated-images.md b/docs/harnesses/grok/generated-images.md new file mode 100644 index 000000000..38f5b6f9d --- /dev/null +++ b/docs/harnesses/grok/generated-images.md @@ -0,0 +1,11 @@ +# Grok generated images in Codex Desktop + +Grok Imagine (`image_gen` / `image_edit`) writes a file under the native session `images/` directory and reports ACP `rawOutput.type` `ImageGen` or `ImageEdit` with an absolute `path`. + +The Grok Adapter emits a Host `generatedImage` Item with that `savedPath`. Protocol Core projects it to Desktop `imageGeneration` (`item/started` then `item/completed` with `completedAtMs`). Desktop renders the official generated-image gallery. + +This is separate from local-media Markdown rewrite (`![](/abs/file)` / backticks), which remains for videos and for agent text that already cites a file. + +Do not project Imagine files as `fileChange`. Do not send image base64 through the 64 KB tool-output budget. + +Live JPEG gallery display still needs a Desktop session check after this projection. diff --git a/docs/harnesses/grok/mcp-elicitation.md b/docs/harnesses/grok/mcp-elicitation.md new file mode 100644 index 000000000..6ea221262 --- /dev/null +++ b/docs/harnesses/grok/mcp-elicitation.md @@ -0,0 +1,27 @@ +# Grok MCP elicitation + +CodexHost Grok sessions project bounded MCP consent forms as Desktop approvals. This is how a user-installed Computer Use MCP (`cua_repl`) can ask to use Clock or another app. + +## Wire + +Grok sends ACP extension `x.ai/mcp/elicit`. The JSON-RPC method on the wire is `_x.ai/mcp/elicit`. The Grok Adapter handles it through `Client.extMethod`. Initialization keeps `clientCapabilities: {}`; this Grok revision does not gate elicitation on a capability flag. + +Supported request: `sessionId`, `toolCallId`, `serverName`, `message`, `mode: "form"`, and either no `requestedSchema` or a reviewed empty object schema. URL mode and non-empty/unreviewed schemas cancel without granting access. + +Native replies are `{ outcome: "accept", content: {} }`, `{ outcome: "decline" }`, or `{ outcome: "cancel" }`. + +## Desktop + +An in-turn form becomes a Host approval with Allow once and Deny. Host Runtime labels the Desktop request with the Harness name (`Grok`); the Adapter title and description keep the MCP server name and consent text. + +Allow once maps to native accept. Desktop deny or UI dismiss maps to native decline. Turn cancellation, Session close, process fault, and native `interaction_resolved` (`tool_call_id`) map to native cancel and close the Desktop card once. + +When the patched Grok offer includes `persist: ["always"]` (or `session` plus `always`), Desktop adds **Always allow**. That maps to `{ outcome: "accept", content: {}, persist: "always" }`. Allow once still omits persist. **Allow this conversation** is not shown: native `session` lifetime is not proven to match a Grok conversation. + +Unpatched Grok (no persist field) keeps Allow once / Deny only. Auto and Always approve permission modes do not auto-accept elicitation. + +## Limits + +- User-installed `cua_repl` remains Grok configuration. CodexHost does not inject `computer-use@openai-bundled`. +- Arbitrary data-entry forms, URL elicitation, and native TUI elicitation-card parity are unsupported. +- Tool `requestPermission` stays on its own nested ACP response path. diff --git a/docs/harnesses/grok/subagent-status-and-model.md b/docs/harnesses/grok/subagent-status-and-model.md index 1f75413ae..eb5655106 100644 --- a/docs/harnesses/grok/subagent-status-and-model.md +++ b/docs/harnesses/grok/subagent-status-and-model.md @@ -2,7 +2,7 @@ ## Scope -The Grok Adapter maps native spawn/task tools to the existing Host `subagentDelegation` contract. Desktop receives the existing `collabAgentToolCall` projection and remains responsible for its Subagent list, collapse behavior, navigation, and visual presentation. +The Grok Adapter maps native spawn/task tools to the existing Host `subagentDelegation` contract. Desktop receives native `subAgentActivity` Items and remains responsible for its Subagent list, collapse behavior, navigation, and visual presentation. This integration does not inject a replacement list, custom status subtitles, avatars, or a history panel. It does not inspect React Fiber, add child `thread/read` requests, or scan and rewrite official Codex protocol frames. Native Codex Subagent behavior is unchanged. @@ -13,15 +13,23 @@ This integration does not inject a replacement list, custom status subtitles, av - Child transcripts are read-only snapshots of the child Native Session; they are not separate writable Host Sessions. - Parent Native history is projected back into Subagent Items when the Thread is reopened. Presentation uses the existing Desktop history path rather than a Renderer-maintained copy of the list. +## Read-only native presentation + +Desktop uses `collabAgentToolCall` with `tool: "spawnAgent"` to mark a child as interactive. `canAcceptDirectInput: false` alone does not select its read-only panel in the inspected Desktop build. External native children therefore project as `subAgentActivity`, which selects Desktop's existing Subagents panel and read-only conversation. Native Codex protocol traffic and independent writable external Threads do not pass through this projection. + +Each receiver gets a stable activity identity, including multi-child operations. Spawn operations retain a `started` activity for native descendant discovery; send operations use `interacted`. Observed terminal states add a separate terminal activity, preserving the spawn identity during live updates and history replay. The native activity vocabulary represents successful completion as `completed` and failed or interrupted execution as `interrupted`. Per-child Thread status remains driven by native lifecycle observations. Prompt and result text remain in Harness history and child transcripts rather than being encoded into an activity's display name. + ## Model and reasoning effort The public `HostSubagentState` has optional `model` and `reasoningEffort` fields. Adapters must omit unknown values rather than infer them from parent Session settings. Grok supplies the raw child Model ID when explicitly present in spawn arguments or a native spawned event. A reported child Model supersedes the spawn argument. An explicit argument describes the requested child configuration; it is not independent verification of the inference backend. Grok does not currently supply independently observed child reasoning effort, so that field is omitted. -The protocol projector passes these values in separate native `model` and `reasoningEffort` fields without formatting an ID into a display label. Since a native collaboration Item has only one configuration slot, a multi-child Item supplies it only when all children have the same configuration. Missing or heterogeneous configurations are not guessed. Adapters that omit both optional fields retain the previous null projection. +The Host persists each child's reported `model` and `reasoningEffort` as child metadata and exposes them on child Thread reads and resumes. The native Subagents header reads these Thread fields. A multi-child operation keeps each child's own configuration. Missing observations preserve previously reported child values; a child with no observations exposes null values, including legacy mappings that copied a parent transport selection. New child mappings use an unconfigured Harness route and never copy parent model, permission, or thinking selections. The metadata lives outside the strict core record so existing core record readers remain compatible. + +Claude Code's normal `Agent` and legacy `Task` tools preserve an explicit child `model` argument through lifecycle updates and completion. Raw aliases such as `haiku` and `sonnet` describe the requested configuration. A tracked child's native assistant response Model supersedes that argument, including when the child inherited its Model. Synthetic response Models are ignored. Omitted, blank, or `inherit` arguments remain unknown until a child reports its Model; the Adapter does not copy the parent's Model. `SendMessage` does not change child configuration. Persisted live observations survive reopening, but cold Claude history projected as ordinary Tool Items does not backfill Model metadata for legacy children. -Whether and where those fields are visible depends on the installed Desktop version; this integration does not promise a custom `status · Model · effort` subtitle. +Desktop formats the raw Model ID and supported reasoning effort in its native header; the Host does not invent display names or resolve a local alias into an inference backend. No custom `status · Model · effort` subtitle is injected. ## Summary panel discovery and restored identity @@ -37,8 +45,7 @@ the child transcript has never been opened. After Host restart, an unobserved child remains `notLoaded` until a real status observation is available. Live projection and parent history hydration reuse the same child Host Thread -identity. Historical collaboration Items carry their parent Host Thread ID and -resolved child receiver IDs, rather than raw CLI-native IDs. This also recovers +identity. Historical activity Items carry resolved child Host Thread IDs rather than raw CLI-native IDs. This also recovers children from history recorded before the Host had materialized them. Existing child mappings retain their IDs; concurrent hydration and live events cannot create duplicate children for one parent Native Session and native child ID. @@ -69,4 +76,4 @@ New Grok Sessions receive an explicitly requested startup Model through the nati ## Validation scope -Focused automated tests cover tool/event mapping, lifecycle projection, child transcript reads, history replay, explicit versus unknown child Model metadata, startup arguments, unchanged user input, native protocol projection for missing and heterogeneous configurations, rollback child/descendant identity rebinding, in-flight read retirement, reused native IDs, fork isolation, and indexed/legacy metadata lookup costs. These checks do not replace live Grok/Desktop validation; the removed custom UI screenshot is not evidence for this native-only version. +Focused automated tests cover tool/event mapping, lifecycle projection, child transcript reads, history replay, explicit versus unknown child Model metadata, startup arguments, unchanged user input, per-child configuration persistence, legacy unknown configuration, parent isolation, native read-only activity projection for multiple receivers, rollback child/descendant identity rebinding, in-flight read retirement, reused native IDs, fork isolation, and indexed/legacy metadata lookup costs. These checks do not replace live Grok/Desktop validation. Live Reveal and header appearance still require a manual check in the running Desktop; no GUI acceptance is claimed from protocol tests alone. diff --git a/docs/index.md b/docs/index.md index f533ae8de..b5f71f845 100644 --- a/docs/index.md +++ b/docs/index.md @@ -30,7 +30,7 @@ ## Harness 专项 -[能力表边界](harnesses/capability-boundaries.md)记录各 Harness 的接入条件、Kimi 动态 Thinking、Cursor 的单向委派,以及 Pi 权限模式、Antigravity 审批与压缩等剩余缺口。 +[能力表边界](harnesses/capability-boundaries.md)记录各 Harness 的接入条件、Kimi 动态 Thinking、Cursor 的单向委派,Pi 权限模式、Antigravity 审批与压缩,以及 Grok MCP 同意表单等剩余缺口。 ### Antigravity @@ -92,6 +92,10 @@ | 文档 | 内容与阅读时机 | | --- | --- | | [`harnesses/grok/subagent-status-and-model.md`](harnesses/grok/subagent-status-and-model.md) | Grok Subagent 状态、Model、Transcript 与 Desktop 投影;修改 Grok Subagent 时阅读。 | +| [`harnesses/grok/generated-images.md`](harnesses/grok/generated-images.md) | Grok Imagine files project to Desktop `imageGeneration`; keep Markdown rewrite for videos and cited paths. | +| [`harnesses/grok/mcp-elicitation.md`](harnesses/grok/mcp-elicitation.md) | Grok MCP consent forms project to Desktop approval, including offered Always allow; bounded empty forms only. | +| [`harnesses/grok/computer-use-release.md`](harnesses/grok/computer-use-release.md) | Grok native turn-end Computer Use cleanup, Mac Sky bridge, failure behavior, and runtime limits. | +| [`harnesses/grok/computer-use-settings.md`](harnesses/grok/computer-use-settings.md) | Grok `cua_repl` Computer Use and in-app browser (IAB) switches, surface matrix, IAB vs Computer Use vs Chrome extension, fail-closed binding. | | [`harnesses/grok/permission-modes.md`](harnesses/grok/permission-modes.md) | Grok 会话内权限切换、原生生效确认、失败处理与验证范围。 | ### Copilot @@ -110,12 +114,13 @@ | 文档 | 内容与阅读时机 | | --- | --- | -| [`product/codex-accounts.md`](product/codex-accounts.md) | 设置页中的原生账号与额度布局、外部 Harness 不受 Codex 额度门限制、手动导入 Pi 及导入记录管理;修改账号 UI、发送额度门、查询或凭证导入链路时阅读。 | +| [`product/codex-accounts.md`](product/codex-accounts.md) | 设置页中的原生账号与额度布局、外部 Harness 不受 Codex 额度门限制、外观页「隐藏 Codex 额度警告」(默认开启)、手动导入 Pi 及导入记录管理;修改账号 UI、发送额度门、额度横幅或凭证导入链路时阅读。 | | [`product/usage-metering.md`](product/usage-metering.md) | 外部 Harness 与官方 Codex 的会话用量、插件可选能力、实时刷新、费用、缓存命中、首字延迟及速度限制;修改计量、价格表或 `pricing.json` 时阅读。 | | [`product/usage-statistics.md`](product/usage-statistics.md) | 控制台与 Desktop 内置设置的全局用量统计:独立 Codex 统计插件、各 Harness 原生会话去重、按模型 ID 读时计价与缓存;新增读取器或修改统计口径时阅读。 | | [`product/codex-native-account-switching-design.md`](product/codex-native-account-switching-design.md) | 移除 Codex 多账号切换后的只读边界;修改 Codex 认证或账号路由时阅读。 | | [`architecture/renderer-settings-styling.md`](architecture/renderer-settings-styling.md) | 设置页 Tailwind CSS 使用边界、构建方式与编写规则;新增或改版设置页界面时阅读。 | | [`operations/codex-desktop-upgrade-diagnosis-playbook.md`](operations/codex-desktop-upgrade-diagnosis-playbook.md) | Desktop 更新后 Renderer、Bridge、Agent 和 Model 异常,以及输入性能的诊断流程;兼容性与交互性能回归时阅读。 | +| [`operations/cpa-live-voice-desktop-hook.md`](operations/cpa-live-voice-desktop-hook.md) | Shared Desktop/console Voice settings, audited native Chat lifecycle and external Harness voice: new-chat selection, shared task routing, configured gateway, lifecycle limits, focused smoke tests, attended voice verification, and daily Sparkle pin drift (`.github/workflows/compat-daily.yml`). | ## 平台、进程与远程运行 @@ -139,6 +144,7 @@ | [`operations/remote-thread-read.md`](operations/remote-thread-read.md) | 通过现有 Desktop 远程连接跨 Harness 读取会话;包含部署要求、分页、响应大小与超时限制。 | | [`operations/host-runtime-log.md`](operations/host-runtime-log.md) | Host Runtime 日志文件的位置、内容与边界;排查 Runtime 崩溃或异常退出时阅读。 | | [`operations/codexhost-console.md`](operations/codexhost-console.md) | 本地控制台的打开方式、页面、项目公告、与运行中 Host 的连接、端口、诊断文件、安全与更新边界;排查启动失败或维护 Web 公告时阅读。 | +| [`operations/fork-versioning.md`](operations/fork-versioning.md) | Fork versioning and upstream-sync runbook: dirty-work backup, merge authorization, voice/IAB/compat-daily preservation, Desktop re-audit, and origin-only publishing. | | [`NOTICE.md`](NOTICE.md) | Web 控制台打开或刷新时读取的项目公告,含显示开关、标题、类型与正文;默认关闭,发布规则见控制台文档。 | | [`operations/repository-maintenance.md`](operations/repository-maintenance.md) | PR 标签、CI 评论和发布前检查自动化;修改仓库自动化时阅读。 | diff --git a/docs/operations/codex-desktop-upgrade-diagnosis-playbook.md b/docs/operations/codex-desktop-upgrade-diagnosis-playbook.md index 30ade5e67..6a1399456 100644 --- a/docs/operations/codex-desktop-upgrade-diagnosis-playbook.md +++ b/docs/operations/codex-desktop-upgrade-diagnosis-playbook.md @@ -12,6 +12,102 @@ - `docs/archive/codex-desktop-incidents/26.814-compatibility-debt.md` - `docs/archive/codex-desktop-incidents/26.908-request-manager-wrapper.md`(Fiber hook 把 Request Manager 包进 `{ hostId, manager, status }` 后,连接检查全部失败) +## Recovering “ChatGPT hit a snag” without losing session state + +This screen can be a React render failure while Desktop, Host, and the native +Harness sessions continue running. It does not establish that a session was +lost, that an active Turn stopped, or that the Agent named in another sidebar +Thread caused the failure. An Adapter reporting `ready / request-bridge` also +does not establish that the conversation rendered successfully. + +### Recovery procedure + +1. Preserve the running processes and active work. Record the Desktop version, + build, visible error, and first relevant render stack before making changes. + Discover the current CDP endpoint from process arguments; do not assume a + fixed port. Inspect the primary conversation Renderer, not the pet overlay. +2. Confirm that the error boundary and its **Try again** button are visible. + A matching string or hidden button in the DOM is insufficient: retained + views and rendered messages can contain the same text. Click **Try again** + once. This is the error-boundary retry, not a Composer submission or Desktop + update. Do not install an automatic reload or retry loop. +3. If Desktop returns to project home, reopen the existing Thread from the + sidebar. Verify that history and a visible Composer return and that the error + boundary is gone. For an external Thread, verify that its existing Agent + ownership remains locked and that its Model control is available. Do not + submit a test prompt as part of recovery. +4. Check the existing Turn's state before sending anything. Recovered history + and controls establish view recovery; they do not prove that an interrupted + response resumed or that a new Turn can complete. Mapping files and native + session files establish persistence, not history completeness. +5. If one retry immediately fails again, stop retrying and capture the new + stack. Separate a repeatable render/data failure from a disconnected or dead + native backend. A controlled Desktop/Host restart is a later option when + view retry cannot recover or a process/connection failure requires it, after + checking active Turns and preserving Harness state. Restart only the + necessary component. An unresponsive UI is not permission to cancel work. + +Do not clear histories, delete mappings or native sessions, rewrite transcripts, +replay messages, switch Harness ownership, or update Desktop as a default +recovery step. If there is uncertainty about whether a request was accepted, +inspect the existing Thread before explicitly retrying that request. + +### Image completion state failure in Desktop 26.1007.30912 + +For build `20156` (Chromium `155.0.8059.40`), the observed render stack was +`b0n` in `app-shared-8bd9d0399038.js:529:120057`, then `hQe` and `Qft` in +`app-primary-6f60e9ac4103.js`. These names and coordinates identify this build +only; they are not binding contracts. + +The null value in that comparator is the account's dismissed image-completion +record, not a Thread returned by Grok. The relevant installed function is: + +```js +function sameImageCompletion(latest, dismissed) { + return ( + dismissed?.completedAtMs === latest.completedAtMs && + dismissed.threadId === latest.threadId && + dismissed.turnId === latest.turnId + ); +} +``` + +The dismissed record defaults to null. With a missing `latest.completedAtMs`, +the first comparison is `undefined === undefined`, so evaluation continues to +`dismissed.threadId` and throws. With a numeric completion timestamp, the first +comparison returns false and short-circuits safely. The Composer's image limit +hook calls this comparison whenever the latest image record exists, including +successful completions with no usage-limit failure. + +The image notification listener copies `params.completedAtMs` into that account +record. The installed native `ItemCompletedNotification` contract requires a +Unix timestamp in milliseconds. An experimental image-preview diagnostic sent +an `item/completed` notification without this field before the reported failure. +That malformed notification is a concrete likely trigger: evaluating the exact +installed comparator with the resulting value shape reproduces the reported +exception. The account-state values at the moment of the original failure were +not captured, so the historical causal link remains inferred. + +Production codexhost has no hosted-image completion synthesis on this path. +Its external Item projector supplies `completedAtMs` on completion, and native +notifications pass through the routing bridge. The experimental preview code +was removed. Do not add blanket null suppression, fabricated image completions, +or native account-state writes to compensate for this diagnostic error. Image +presentation investigations should use the native image-tool path or inspect +recovered image files outside Desktop's notification stream. Any future +compatibility producer must validate the full lifecycle notification contract, +including timing, and test downstream consumers in an isolated fixture before +touching a user's running Renderer. + +The successful recovery used **Try again**, then reopened an existing Thread. +The user subsequently confirmed that the original native conversation rendered +again. No restart, session deletion, transcript rewriting, or message replay was +required. A fresh Grok response was not tested. The sandbox `localStorage` +`SecurityError` and usage-reader warning `Codex rollout changed while reading; +refresh to retry` were also observed, but neither is established as the cause. +The latter comes from usage statistics rejecting a file that changed during a +read; it does not itself report session loss. + ## 一、先建立分层模型 ### 历史原生会话不能拖入引用或通过 `@` 搜索会话 @@ -328,7 +424,7 @@ grok ### 检查 Codex 额度门 -ChatGPT 登录的 Codex 额度耗尽时,外部 Agent 的发送按钮仍为 disabled,且 Agent 控件悬停提示「无法将此 Harness 与 Codex 额度限制分离」,说明 `renderer-codex-usage-gate.ts` 没有绑定成功。API Key 登录不会触发这道门,不能用来复现。 +ChatGPT 登录的 Codex 额度耗尽时,外部 Agent 的发送按钮仍为 disabled,且 Agent 控件悬停提示「无法将此 Harness 与 Codex 额度限制分离」,说明 `renderer-codex-usage-gate.ts` 没有绑定成功。API Key 登录不会触发这道门,不能用来复现。额度耗尽横幅是否显示由设置 → 外观「隐藏 Codex 额度警告」控制(默认开启,作用于所有 Composer,包括 Codex 会话);横幅隐藏成功不能代替这道发送门的绑定检查。 在主 Renderer 中只读检查(不修改 getSnapshot,不订阅 store): @@ -454,7 +550,7 @@ ChatGPT 登录的 Codex 额度耗尽时,外部 Agent 的发送按钮仍为 dis - [ ] 点击 DeepSeek Harness 后 Model catalog 可加载 - [ ] 点击 Grok 后 Model catalog 可加载 - [ ] 切回 Codex 后原生 Composer 仍可用 -- [ ] ChatGPT 账号 Codex 额度耗尽时,新 Thread 和已有外部 Thread 均可发送;切回 Codex 后发送按钮恢复 disabled +- [ ] ChatGPT 账号 Codex 额度耗尽时,新 Thread、已有外部 Thread 和 Codex Thread 的发送按钮均可点击(空输入等其他原生限制仍生效) - [ ] 新 Thread 和未锁定 Thread 都完成切换验证 - [ ] 已锁定 Thread 仍按设计保持 locked - [ ] 真实错误不会只被转换为笼统的 `Models unavailable` diff --git a/docs/operations/codexhost-console.md b/docs/operations/codexhost-console.md index 0417e4d4c..821602ee0 100644 --- a/docs/operations/codexhost-console.md +++ b/docs/operations/codexhost-console.md @@ -2,11 +2,17 @@ 控制台是一个本地网页,不依赖 Codex Desktop 启动成功。左侧导航分为: -- **总览**:当前状态与下一步操作、codexhost 版本 / Codex Desktop 版本 / 安装方式三个信息框、最近一次启动的结果;失败时说明卡在哪一步和错误原文,内部阶段时间线折叠在“详细信息”中;注入持续失败时显示原因。成功时不展示时间线,不显示启动历史表格。原启动诊断页已合并,旧的 `#diagnostics` 和 `?view=diagnostics` 链接显示总览。日志不在页面中展示,通过“导出诊断包(含日志)”获取。 -- **设置**:连接、远程连接、账号、会话导入、更新,与 Codex 设置页使用同一套页面和 Host 接口。连接页支持通过运行中的本地 Host 安装 Harness CLI、检查版本和更新,复用安装进度、错误提示及安装后检测;不支持远程 Host 安装。WorkBuddy 仍通过官网手动下载安装桌面应用。Host 不可用时不提供一键安装。 +- **总览**:当前状态与下一步操作、codexhost 版本 / Codex Desktop 版本 / 安装方式三个信息框、最近一次启动的结果;失败时说明卡在哪一步和错误原文,内部阶段时间线折叠在“详细信息”中;注入持续失败时显示原因。成功时不展示时间线,不显示启动历史表格。原启动诊断页已合并,旧的 `#diagnostics` 和 `?view=diagnostics` 链接显示总览。日志不在页面中展示,通过“导出诊断包(含日志)”获取。安装包上的 **codexhost 版本** 来自 `app/codexhost-distribution.json`。`release:prepare` 只同步 `package.json` / lock / Cargo;distribution 文件由随后的 `release:package` 或 `release:npm` 写出。本 fork 上该数字必须等于当前 fork 标签,例如已发 `v0.13.2-3` 时显示 `0.13.2-3`,见 [fork-versioning.md](fork-versioning.md)。热装现有 app 时需重打包或直接改写已安装的 `codexhost-distribution.json`。 +- **设置**:连接、远程连接、账号、会话导入、语音、更新。连接、远程连接、账号、会话导入与 Codex 设置页使用同一套页面和 Host 接口。**语音**与 Desktop 的独立语音页共享路由选择和注入设置,使用明确的应用与重启操作。连接页支持通过运行中的本地 Host 安装 Harness CLI、检查版本和更新,复用安装进度、错误提示及安装后检测;不支持远程 Host 安装。WorkBuddy 仍通过官网手动下载安装桌面应用。Host 不可用时不提供一键安装。 - **其他**:关于。 -外观(思考文本显示、空闲释放)由 Codex 界面保存,只能在 Codex 设置页修改。设计背景见 [`proposals/独立 Web 控制台方案.md`](../proposals/独立%20Web%20控制台方案.md)。 +Appearance settings (thinking text, usage warnings and idle release) remain Desktop-only. **Voice** is shared between Desktop and the console on this Mac. Chat selects ChatGPT or Codex; Codex connection selects Desktop default or configured gateway. Codex Work independently selects Desktop default or gateway; external Harness voice stays on Host → gateway. Existing saved choices migrate once without replacing a newer shared edit. + +The Voice page uses drafts, Cancel and **Apply changes**. Injection changes use **Apply and restart codexhost** with interruption confirmation. Activation is reported separately from saving; a missing Desktop acknowledgment remains unknown or pending. Detected active calls block Apply. **Advanced** contains integration and environment-override details; **Gateway → Configure** reveals manual setup guidance for the local Codex configuration, without displaying credentials. The console remains usable with Host stopped, including saving settings for a later launch. Narrow windows use a compact navigation menu instead of horizontal scrolling. + +The overview restart action remains available for recovery. A different Launcher and ready Controller are required for restart completion; the Voice page additionally checks loaded injection and applied route acknowledgment. See [voice routing and configuration](cpa-live-voice-desktop-hook.md) for contracts and constraints. + +When the selected Grok or Claude Code launcher is managed by grokgod or clawgod, the connection page reads the installed CLI version with background updates disabled. It identifies the patch manager, shows the latest patched version as unknown, and leaves updates to that manager's updater or installer. A version check does not invoke the stock updater through these launchers or compare a patched installation against official releases. Grok Thinking selections use the native ACP `_meta.reasoningEffort` field, alongside `_meta.contextWindow` when selected. ## 项目公告 @@ -21,7 +27,7 @@ title: 版本兼容性提醒 type: warning --- -这里填写公告正文,可使用 **粗体**、列表和 [链接](https://github.com/BytePioneer-AI/codex-host)。 +这里填写公告正文,可使用 **粗体**、列表和 [链接](https://github.com/karlorz/codex-host)。 ``` - `enabled`:仅 `true` 显示,`false` 隐藏;正文为空也隐藏。 @@ -126,12 +132,13 @@ Launcher 启动的本地 Host Runtime 在 `127.0.0.1` 的随机端口开放控 - codexhost 运行中:通过 Host 的更新流程检查与安装,与 Codex 设置页一致。 - codexhost 未运行:控制台下载并准备更新,拉起 Updater 并等待身份校验后的就绪确认才退出;Updater 等待控制台进程退出,再按原流程安装并重新启动 codexhost。 +- Windows 手动安装包使用 Windows Restart Manager 关闭占用本安装包文件的 codexhost Runtime/Launcher 进程,并且不会重新启动旧的控制台;这样直接运行下载的安装包时不需要先手动结束 codexhost。 - npm 安装需通过 npm 命令启动的控制台(`codexhost`、`codexhost console`,或 Codex 设置页)才能更新,因为更新需要 npm 路径环境变量。 - 源码构建不支持在控制台更新。 ## 连接页(codexhost 未运行时) -codexhost 未运行时,“连接”页列出已安装的插件(读取 `manifest.json`,不执行插件代码)及是否启用,并可修改接受自定义安装路径的插件的路径。路径与 Codex 设置页共用 `<数据目录>/harness-launch-settings/<插件>.json`;codexhost 运行中修改时,在重新启动 codexhost 后生效。插件目录与 Host Runtime 一致:发布包的 `app/plugins`,以及 `CODEXHOST_PLUGIN_DIRECTORY` 或 `<数据目录>/plugins`。 +codexhost 未运行时,“连接”页列出已安装的插件(读取 `manifest.json`,不执行插件代码)及是否启用,并可修改接受自定义安装路径的插件的路径。路径与 Codex 设置页共用 `<数据目录>/harness-launch-settings/<插件>.json`;codexhost 运行中修改时,在重新启动 codexhost 后生效。Grok 另有 Computer Use 与 In-app browser 两个开关,与设置页共用 `<数据目录>/grok-computer-use.json`,并写入 `~/.grok/config.toml` 的 `cua_repl`;新的 Grok 线程生效。插件目录与 Host Runtime 一致:发布包的 `app/plugins`,以及 `CODEXHOST_PLUGIN_DIRECTORY` 或 `<数据目录>/plugins`。 ## 诊断包与 Issue diff --git a/docs/operations/cpa-live-voice-desktop-hook.md b/docs/operations/cpa-live-voice-desktop-hook.md new file mode 100644 index 000000000..e8282fb39 --- /dev/null +++ b/docs/operations/cpa-live-voice-desktop-hook.md @@ -0,0 +1,1127 @@ +# CPA live-voice Desktop hook + +CodexHost routes native live-voice call creation through the configured realtime gateway without rewriting `app.asar` or replacing `/Applications/ChatGPT.app`. Desktop **26.1007.30912** uses a call-ownership gate override. Desktop **26.1013.40934**, **26.1016.51333**, **26.1016.61520**, **26.1016.62342**, and **26.1021.31345** use the native client-owned-call flow described below. Native Chat Instant Connect has a separate **26.1021.31345** profile. + +For upstream merges, follow the [Live Voice preservation checklist](fork-versioning.md#preserve-live-voice-when-syncing-upstream), including regression checks and separate Desktop-version qualification. + +## Default + +Desktop and the console expose the same **Voice** page using the existing settings components. Voice is separate from General/Appearance. Choose **Chat → Voice experience → ChatGPT or Codex**, then **Codex connection → Desktop default or Configured gateway** when Codex is selected. **Work → Voice connection** independently chooses Desktop default or gateway for Codex Work. External Harness voice remains Host → configured gateway, including when Codex Work selects Desktop default. + +The local console owns `~/.codexhost/live-voice-inject-v1.json`: version 1 retains `inject` and optionally stores `routes` with `chatExperience`, `chatConnection` and `workConnection`. Desktop reaches this service through the local Host's bounded `codexhost/voice/settings` bridge; it does not open a browser or use cross-origin browser requests. Desktop legacy preferences migrate once, only while shared routes are absent. A console edit takes precedence over later migration. Atomic publication preserves the existing Controller reader. Revision checks reject stale edits from another window. + +ChatGPT is the default Chat experience; its native START and Instant Connect prewarm remain owned by Desktop's browser-voice stack. Codex/Desktop default restores the original Codex voice entry and passes `/wham` through. Codex/gateway forwards call creation through Host. Explicit Chat launch ownership takes precedence over remembered Work/Harness selection. Work defaults to gateway; saved choices are preserved. + +Edits are drafts until **Apply changes**. **Cancel** restores the loaded snapshot. Desktop polls shared choices every two seconds and acknowledges its applied revision; the page reports pending or unknown activation until that acknowledgment. A validated browser cache restores the last applied route before the first Host poll on a new document. Route changes are deferred during a detected active call, and fresh Desktop call reports block setting writes and Voice-page restarts. Runtime reports expire after 12 seconds; an absent report is unknown, not evidence that no call exists. + +**Advanced → Enable voice integration** defaults on. Changing injection uses **Apply and restart codexhost**, with explicit task/call interruption confirmation. Environment `CODEXHOST_CPA_LIVE_VOICE=0|1` controls injection for that launch and locks its saved toggle. The Controller's loaded flag is compared with the desired effective setting; saving and restart completion alone never claim configuration activation. Restart failures retain the saved configuration and permit retry. The normal launch action attaches to an already running Desktop. + +Gateway configuration details are collapsed under **Configuration → Gateway → Configure**. This reveals setup guidance, not a credential editor. The readiness check returns only a boolean from the Host; it does not probe a live call or expose provider credentials. Narrow console layouts use a compact expandable navigation menu and stacked preference controls without a horizontal navigation scrollbar. + +The 26.1013 Work/Chat routing choices apply on the next call. The injection setting and process environment require restart. Host call creation rereads the selected provider and realtime URLs from `CODEX_HOME/config.toml`; Desktop-owned config consumers can require restart. End an existing call before changing its gateway configuration so cleanup uses the correct gateway. + +**Desktop default** leaves Renderer call creation on Desktop's service; it does +not clear the native backend's `experimental_realtime_*` URL overrides. If those +still point at a gateway, native Work can create its call on Desktop and then +attempt to attach through the gateway, producing a WebSocket 404. With gateway +URLs configured, select **Configured gateway** for Codex Work to align creation +and attachment. Returning to the official backend also requires restoring the +native realtime configuration and restarting. The Voice page reports three facts separately: **Gateway configuration** (parsed +credentials), **Preferences applied** (Desktop acknowledged the saved routes), +and **Desktop gateway voice integration** (the version-qualified gateway transport hook). The +**Configured** status checks configuration and credential presence only, not +call connectivity. An unsupported hook means this Desktop build is outside the +audited allowlist; it does not mean the gateway settings were lost. + +Desktop **26.1016.61520** hides native Start voice when Statsig rollout +`3683713372` is on and `model_provider` is not `openai`. CodexHost leaves +`model_provider = "custom"` unchanged. Work + gateway (and owned `codexhost/` +Harness composers) project only that composer’s `voice.realtime` / +`voice.thread` subscriptions. ChatGPT Chat stays on its separate native browser-voice stack. + +The unaudited-Desktop asset gate still fail-closes. `~/.codex/config.toml` realtime keys stay manual. + +Daily CI (`npm run compat:daily`, `.github/workflows/compat-daily.yml`) compares the public Sparkle appcast to the Work/gateway and Chat voice pins. It does not download ChatGPT.app. A pin behind Sparkle latest opens or comments on a `compat-broken` issue on this repository. + +First-start official-backend recovery is covered by `packages/host-runtime/test/native-account-host.test.ts` (failed first generation, delayed second generation, no `unhandledRejection`). Manual cold-start UAT: fully quit ChatGPT and CodexHost, start **CodexHost 0.13.3** from the menu bar, wait ~15s. Pass: no 无法载入组织设定, console 总览 stays 启动成功, Work + Grok still shows Start voice. Fail: org-settings dialog on first launch only, then OK after 重试 — that is the remaining Host-wait gap, not Chat voice. + +The hook only changes Desktop call ownership. Point Codex at the realtime host in `~/.codex/config.toml` as well, then relaunch: + +```toml +experimental_realtime_ws_base_url = "https:///v1" +experimental_realtime_webrtc_call_base_url = "https:///v1" +``` + +## Desktop 26.1021.31345 qualification + +Work/gateway client-owned-call is audited for **26.1021.31345** build **21828**. `/wham/realtime/calls` is `Htr.getInstance().fetch` on `app-initial-feb30b196d11.js`; the message bridge is shared `bIt` and services `Qot`. Overlay is `avatar-overlay-native-page-d549db3706f1.js`. The 26.1016.62342 and 26.1016.61520 Work profiles remain supported. + +The same version profile supplies every Work voice consumer, including new-chat +launch routing and the overlay starter. Capability initialization uses initial +`l0n`, capability family `c0n`, and inference-gate family `Cnr`; the 26.1016 +profiles retain `njr` / `tjr` / `uBr`. Family resolution accepts arbitrary cache +keys, so resolving `3683713372` on an unrelated family does not identify the +inference gate. Missing audited exports fail closed without probing other +families. Both composer and overlay project only their owned subscriptions. +Existing Codex chats send the native product experience `codex`; Work home +starts send `work`. Both use the Work connection choice for explicit local +composer starts. The starter and call transport retain that native product +identity, including when the overlay has no Composer or model metadata. +Starter qualification continues to exclude remote, global-hotkey and +avatar-button requests. + +Composer health also compares the projected subscriber and instance snapshots +with React's committed capability value. A brief Host readiness or routing +interruption can commit the native fallback without changing its underlying +atom. When readiness returns, a stale committed value causes the next +one-second poll to rebind and notify the owned subscriptions. React's equivalent +lazy snapshot wrappers remain valid; account and workspace checks remain native. +Work surface discovery ignores retained zero-size, hidden or inert Chat composers. +A visible Chat composer still prevents that Work fallback from claiming the surface. + +When Desktop omits resource timing and script asset names, the explicit Desktop +product version and platform select the profile and the hook imports its exact native assets. +Observed incomplete or mixed initial/shared pairs still fail closed. Overlay +preparation rechecks that profile and requires the committed starter component +to belong to its exported native overlay page. + +A controlled live check on 25 October 2026 confirmed one enabled **Start new +voice chat** button in an idle Codex Work composer with the gateway selected. +The native capability remained false while the owned subscription read true. +The separate overlay check confirmed gateway readiness through `bIt` and a +committed projected starter snapshot, then restored the native subscription. +Neither check opened a call or captured microphone audio; physical microphone, +gateway call creation, audible reply and hangup acceptance remain unverified on +26.1021. The primary and overlay bundles must both be installed; replacing an +overlay hook after its native starter has registered does not replace that +existing callback. A fresh overlay document or Desktop restart binds it. + +Native Chat Instant Connect uses the full composer START relocated to +`entity-business-profile-reference-b68063dfc05d.js` (`i`, initializer `r`). Its +scope/props/metadata/abort-signal signature preserves native navigation, +availability and session guards. `start-voice-5bbdc12fe620.js` provides a +different starter-prompt entry and cannot substitute for composer START. +`renderer-native-chat-voice-assets.ts` owns the version profiles and normalizes +audited exports to the retained Chat lifecycle contract. Native initializers run +before their mutable exports are captured. + +The current Chat profile uses shared `Bst` / `XVt` / `KNt` for AppScope, React +and ReactDOM; initial `CQn` / `PXn` / `tdn` / `qXt` for RouteScope, +ComposerScope, backend Chat ID and browser voice store; primary `um` / `lm` for +browser initialization and its enable flag; and +`voice-prewarm-d672f1245523.js` for the native prewarm controller. Draft +selection retains the native location/navigation/transcript/intent roles via +initial `Xkn` / `yAn` / `ydn` / `fdn` / `R2` and shared `Fot`. + +A live no-call check confirmed one enabled Chat waveform immediately before +Send, matching size and vertical alignment, and one native prewarm controller. +Microphone capture, native call creation, audible reply and hangup are separate +acceptance checks. + +### Native Voice Settings with a custom provider + +cc-switch's Unified Codex session history intentionally uses `model_provider = +"custom"` for official-subscription and third-party sessions. Keep that provider +identity and its history. Changing it to `openai` is not a voice repair. + +The native Voice Settings page reads the `voice.realtime`, `dictation.composer` +and `dictation.global` capability family members. Its Voice chat group also reads +a derived boolean, and the page and shortcut controls read the native dictation +mode selector. On 26.1021.31345, CodexHost restores the native controls by +projecting only these owned Settings subscriptions when the voice access policy +is `allowed`, root voice capability is ready and capable, and readonly replay +proves the inference-provider condition is the sole blocker. The Voice chat boolean replay must read that exact realtime capability and agree +with its native value before substitution. Dictation mode replay preserves the +native platform choice (`os-global` on macOS/Windows, `app` on eligible Linux) +and all non-provider dictation policy and feature checks. Native atoms, provider configuration and global Statsig values +retain their values. Account/workspace denial, loading, errors, rollout gates +and unrelated restrictions remain authoritative. + +`renderer-voice-settings-availability.ts` identifies committed Settings groups +below the exported page and binds the boolean reader. The existing capability binder handles only the explicitly selected family +members on the exported Settings page. `renderer-dictation-settings.ts` binds +the exact native mode selector within Settings, including newly mounted shortcut +controls, so their save handlers select the correct OS service. Ambiguous ownership or unsupported +subscriptions fail closed, with rollback if a required reader could not be bound. +Navigation, disposal and eligibility changes restore native readers. While a +supported binding is unavailable, the existing text-only explanation can identify +a provider-only rejection; it does not grant access. The hook retries controls +after passive subscription initialization without repeatedly rewriting that text. +The installer also retries native asset discovery during document startup: an +initial chunk observed before its shared chunk cannot permanently skip Settings +installation. Incomplete or mixed asset pairs still fail closed. + +The restored controls keep their native handlers: + +| Control | Native persistence and consumers | Verification limit | +| --- | --- | --- | +| Voice | Account `voice_name`, through `/settings/account_user_setting`; catalog from `/settings/voices` in `wingman` mode. Native Chat browser voice consumes the account voice. Work realtime resolves the same catalog's effective slug, including native alias mapping, and puts it into session audio output. The gateway bootstrap preserves that session body; external Harness voice accepts the native voice parameter. | Picker opens with nine compatible voices. Saving a different voice and audible selection require attended acceptance. | +| Language | Native account `voice_main_language` preference, shared with voice and dictation. Visibility follows the native dictation capabilities or available dictation mode. | Language selection is visible; the existing saved language is retained. | +| Dictation | Native composer/global dictation capabilities and mode selector. The macOS shortcut editor uses the native global-dictation service and retains permission setup. | Shortcut row is visible and its owned mode readers commit `os-global`. Capture, transcription, OS registration and permission grants are not established by restoring Settings. | +| Voice chat hotkey | Desktop `commandKeybindings` service, command `realtimeVoice`; its main-process controller starts `global_hotkey_new_thread` through the avatar overlay. | Editor is enabled. Registration and call launch retain native main-process and overlay capability checks. The gateway starter hook qualifies explicit local composer starts, not global-hotkey starts; restoring Settings does not establish global-hotkey call support with a custom provider. | +| Screen context | Native `realtimeVoiceScreenContextEnabled` preference. The row retains macOS, appshot availability and dynamic-tool configuration checks, plus the native first-use introduction and OS permissions. Work's `capture_screen_context` handler checks this preference again and requires the matching active voice task. | Row and toggle are enabled in the observed eligible environment. Actual foreground capture and permissions were not exercised. Chat browser voice has separate camera/screen-share controls; this toggle does not establish equivalent capture there. | + +Live checks on this build confirmed the native controls, an account policy of +`allowed`, and a native `voice.realtime` capability that stayed false while the +owned Settings readers rendered the controls. Language and Dictation also retain +their native capability values while the owned mode readers commit the platform +mode. Three Settings navigation round +trips retained the controls. No account voice, hotkey or permission was changed. +The Composer capability hook reports `native-ineligible` when its provider-only +replay still cannot make the native capability capable; this is diagnostic state +and leaves the native account, workspace and rollout checks authoritative. +These checks do not establish a complete spoken call. On 25 October 2026, +Karl reported that the instructed manual Voice Settings and existing-Codex-chat +voice test passed after a full restart with the startup and `codex` request +fixes installed. This is operator-reported acceptance; the global voice hotkey, +screen capture, dictation and separate Chat browser-voice call still require +their own live checks. + +| Asset | 62342 | 26.1021.31345 | +| --- | --- | --- | +| Initial | app-initial-dddef47d6ef6.js | app-initial-feb30b196d11.js | +| Shared | app-shared-fc3316d48427.js | app-shared-2cae01273bae.js | +| Work overlay | avatar-overlay-native-page-ef8332c6a2fb.js | avatar-overlay-native-page-d549db3706f1.js | +| Native Chat START | start-voice-7cbf0a39d53b.js | entity-business-profile-reference-b68063dfc05d.js | +| Native Chat prewarm | voice-prewarm-5d107129ac1c.js | voice-prewarm-d672f1245523.js | + +The 26.1021.31345 ASAR SHA-256 is +`130145706a912db74e4d260ad7538f10f1e05e88af955f9afba42cdf01e64606`. +Sanitized evidence is under ignored `.codexhost/update-impact/26.1021.31345/`. + +## Windows MSIX product and platform qualification + +Windows discovery reads the Desktop product version from a bounded ASAR +`package.json` entry and retains the MSIX deployment version in `build` and its +activation identity. The Launcher supplies that product version to the Controller; +the voice bootstrap includes `desktopPlatform`. `CODEXHOST_DESKTOP_VERSION` remains +a diagnostic override. Missing metadata falls back to the deployment version, +which cannot qualify a different product profile by numeric resemblance. + +Product `26.1021.31345` has separate macOS and Windows asset profiles. A complete, +unambiguous pair identifies a profile; empty resource lists require an exact +product/platform match. Mixed assets, partial pairs, unknown versions and platform +contradictions fail closed. Windows uses the public prewarm wrapper, which +initializes and re-exports the native Instant Connect controller. + +Profile selection does not grant account/workspace access or prove a spoken +call. Native Windows Settings excludes screen context; codexhost preserves that +platform restriction. Voice selection and hotkey controls remain subject to native +account and workspace eligibility. The daily Windows CI job verifies MSIX and +profile fixtures, independently of the macOS Sparkle latest-version check. + +On 26 October 2026, the user reported that both Chat and Work voice-chat paths +passed manual testing on the Windows VM after completing official login. Before +login, the configured gateway was ready but native provider-only capability +replay remained ineligible. This acceptance resolves that observed login-related +blocker while preserving the custom provider route and native eligibility checks. +The report does not separately qualify Voice Settings controls, screen context, +external Harness voice, or long-call recovery. + +## Live voice-hook debug on a running Desktop + +Use this procedure when a voice control is missing on an already running +codexhost-controlled Desktop after a hook source fix. Live `Runtime.evaluate` +updates the current document. Composer visibility can be confirmed without a +Desktop restart. A later manual restart remains the user's choice for loading +new document-start source. This is an attended debug path; it is not a generic +hot-reload contract for the Renderer binding. + +For native Chat, use the same attach/build/evaluate procedure with +`installNativeChatVoiceHook` and `__codexhostNativeChatVoiceV1.dispose()`; +`activeCall` must be false before disposal. The Settings controls and fallback explanation have their +own `installVoiceSettingsExplanation` / `__codexhostVoiceSettingsV1` lifecycle. +Verify the actual Chat button and native Settings controls after refresh, not just hook +installation. Both installers are included in the persistent Renderer bundle. + +Read-only attach and status (steps 1–2) come first. Run mutation steps within +the user's requested debug/install scope; preserve any manual-restart +preference. Skip mutation during an active or unknown call. + +### 1. Attach through repository CDP + +Discover the current loopback HTTP origin from the running Controller +`--renderer-cdp-endpoint` or the Desktop `--remote-debugging-port`. The Launcher +runtime descriptor identifies the current process tree; extract only that origin. +Leave ports, PIDs, minified export names, temporary paths, nonce, tokens, query +and hash values, and whole target lists out of notes and command output. + +Attach with `CdpClient` from `packages/desktop-control/dist/index.js` (or +`dist/cdp-client.js`). `listCdpTargets(origin)` reads loopback `/json/list`. +`selectPrimaryRendererTarget` requires exact `app://-/index.html` with empty +search and hash. Overlay targets are distinct URLs that include `avatar-overlay` +(`selectAvatarOverlayRendererTargets`). + +```js +import { execFileSync } from "node:child_process"; +import { + CdpClient, + listCdpTargets, + selectAvatarOverlayRendererTargets, + selectPrimaryRendererTarget, +} from "./packages/desktop-control/dist/index.js"; + +function discoverRendererCdpOrigin() { + const text = execFileSync("/bin/ps", ["-axo", "command="], { encoding: "utf8" }); + let fromDesktop; + for (const line of text.split("\n")) { + const endpoint = + /--renderer-cdp-endpoint\s+(http:\/\/(?:127\.0\.0\.1|localhost|\[::1\]):\d+)\b/.exec(line); + if (endpoint) return endpoint[1]; + const port = /--remote-debugging-port=(\d+)/.exec(line); + if (port && /ChatGPT|Codex\.app|codexhost/i.test(line)) { + fromDesktop = `http://127.0.0.1:${port[1]}`; + } + } + if (fromDesktop) return fromDesktop; + throw new Error("Running Desktop has no loopback Renderer CDP origin"); +} + +async function evaluate(client, expression) { + const response = await client.command("Runtime.evaluate", { + expression, + awaitPromise: true, + returnByValue: true, + }); + if (response?.exceptionDetails) { + throw new Error( + response.exceptionDetails.exception?.description ?? + response.exceptionDetails.text ?? + "Runtime.evaluate failed", + ); + } + return response?.result?.value; +} + +const origin = discoverRendererCdpOrigin(); +const targets = await listCdpTargets(origin); +const primary = selectPrimaryRendererTarget(targets); +const overlay = selectAvatarOverlayRendererTargets(targets)[0] ?? null; +if (!primary) throw new Error("No primary app://-/index.html target"); +``` + +Define `SANITIZED_STATUS` from step 2 before running the attach example there. +`evaluate` allows an `undefined` value, which is the normal `Runtime.evaluate` +result for an IIFE that returns nothing. `CdpClient.evaluate` still checks +`exceptionDetails` but requires a returned value, so use this helper for +injection. Connect `overlay?.webSocketDebuggerUrl` only for steps 5–6. + +### 2. Inspect sanitized status only + +Evaluate counts and booleans. Keep config, credentials, prompts, transcripts, +IDs, and the DOM out of the result. + +```js +const SANITIZED_STATUS = `(() => { + const visible = (el) => { + const box = el.getBoundingClientRect(); + return box.width > 0 && box.height > 0 && !el.closest("[hidden], [inert]"); + }; + const nameOf = (el) => + (el.getAttribute("aria-label") || el.textContent || "").replace(/\\s+/g, " ").trim(); + const routes = JSON.parse(localStorage.getItem("codexhost.voice-routes-cache.v1") || "null"); + let localComposer = false; + try { + const routing = window.__codexhostHostRoutingV1; + const route = routing?.forComposer(); + localComposer = route?.hostId === "local" && routing?.forHost("local") === route; + } catch {} + const workComposers = [...document.querySelectorAll("[data-codex-composer-root]")].filter( + visible, + ); + const workComposer = workComposers.length === 1 ? workComposers[0] : null; + const start = workComposer + ? [...workComposer.querySelectorAll("button")].filter((el) => { + const name = nameOf(el); + return ( + visible(el) && + !el.disabled && + /^(Start new voice chat|Start voice chat|Start voice)$/i.test(name) + ); + }) + : []; + return { + desktopVersion: window.__codexhostCpaLiveVoiceHookV1?.desktopVersion ?? null, + clientOwned: window.__codexhostClientOwnedVoiceV1?.status ?? null, + externalLaunch: window.__codexhostExternalVoiceLaunchV1?.status ?? null, + voiceCapability: window.__codexhostVoiceCapabilityV1?.status ?? null, + voiceStartCapability: window.__codexhostVoiceStartCapabilityV1?.status ?? null, + gatewayReady: window.__codexhostVoicePreferenceStateV1?.gatewayReady?.get(window) === true, + workGateway: routes?.workConnection === "gateway", + chatGateway: routes?.chatExperience === "codex" && routes?.chatConnection === "gateway", + chatgptChat: routes?.chatExperience === "chatgpt", + localComposer, + visibleWorkComposers: workComposers.length, + visibleChatComposers: [...document.querySelectorAll("[data-chatgpt-composer]")].filter(visible) + .length, + enabledStartVoiceButtons: start.length, + stopPresent: [...document.querySelectorAll("button")].some((el) => { + const name = nameOf(el); + return visible(el) && /^(Stop|Stop generating)$/i.test(name); + }), + }; +})()`; +``` + +Count **Start voice** inside the unique visible Work composer +(`[data-codex-composer-root]`). Native Chat **New voice chat** stays out of Work +acceptance. On overlay, `voiceStartCapability` is the overlay hook. Hook +`active` is not proof that the composer button is visible. + +```js +const inspectClient = await CdpClient.connect(primary.webSocketDebuggerUrl); +try { + const status = await evaluate(inspectClient, SANITIZED_STATUS); +} finally { + inspectClient.close(); +} +``` + +### 3. Authorized live refresh of the primary page + +Build an ephemeral browser IIFE from current repository modules with esbuild. +Do not copy vendor chunks. Write the bundle outside tracked source. Evaluate it +once on the primary page. Dispose existing handles first: +`installExternalVoiceLaunchHook` and `installVoiceCapabilityProjection` return +the already-installed object when present. + +Desktop CSP forbids dynamic code generation, so set Zod `jitless` before +install. Refresh only the voice hooks. Leave the production Renderer binding, +native capability atoms, global Statsig, `model_provider`, global `fetch`, and +immutable RPC stubs unchanged. + +```js +import { mkdtemp, readFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { build } from "esbuild"; + +const root = process.cwd(); +const outfile = path.join(await mkdtemp(path.join(os.tmpdir(), "codexhost-voice-")), "primary.js"); +await build({ + bundle: true, + platform: "browser", + format: "iife", + target: "es2024", + outfile, + stdin: { + resolveDir: root, + loader: "ts", + contents: ` + import { z } from "zod"; + import { installExternalVoiceLaunchHook } from "./packages/renderer-extension/src/renderer-external-voice-launch.ts"; + import { installVoiceCapabilityProjection } from "./packages/renderer-extension/src/renderer-voice-capability.ts"; + z.config({ jitless: true }); + window.__codexhostVoiceCapabilityV1?.dispose(); + window.__codexhostExternalVoiceLaunchV1?.dispose(); + installExternalVoiceLaunchHook(); + installVoiceCapabilityProjection(); + `, + }, +}); +const refreshClient = await CdpClient.connect(primary.webSocketDebuggerUrl); +try { + await evaluate(refreshClient, await readFile(outfile, "utf8")); + const deadline = Date.now() + 20_000; + let status; + while (Date.now() < deadline) { + status = await evaluate(refreshClient, SANITIZED_STATUS); + if (status?.externalLaunch === "active" && status?.voiceCapability === "active") break; + await new Promise((resolve) => setTimeout(resolve, 250)); + } + if (status?.externalLaunch !== "active" || status?.voiceCapability !== "active") { + throw new Error( + `Voice hooks did not become active within 20s (externalLaunch=${status?.externalLaunch}, voiceCapability=${status?.voiceCapability})`, + ); + } +} finally { + refreshClient.close(); +} +``` + +Installation is async. The loop is bounded hook initialization: it waits until +`externalLaunch` and `voiceCapability` read `active`, then throws if they never +do. It does not prove a committed React snapshot, owned subscription, or visible +button; that verification is step 4. `Runtime.evaluate` of the IIFE returns +`undefined`; the custom `evaluate` helper allows that. Use the current production +profile from `CLIENT_OWNED_VOICE_ASSETS` for the observed `desktopVersion`. On +exact **26.1021.31345** the audited exports +are `capabilityInit=l0n`, `capabilityFamily=c0n`, `inferenceGateFamily=Cnr`, +service `Qot`, bridge `bIt`, and fetch `Htr` from initial, as recorded in the +qualification section above. Those names document that profile. They are not a +generic discovery contract. Resolving an arbitrary cache key on a family does +not identify the inference gate; an unrelated family can read `active` while +projected `voice.realtime` / `voice.thread` remain false. + +### 4. Verify composer behavior + +Confirm the control, not only hook status. Use an idle, unique, visible **LOCAL** +Work composer with gateway ready and **Work → Voice connection** set to +Configured gateway. Pass: exactly one **Start new voice chat** button is visible +and enabled; native capability stays false; the owned subscription reads true. + +An idle blank composer may be selected for this check without submitting text or +editing existing drafts or turns. A running agent task stays running. A visible +Stop control can hide the microphone while the turn is busy. Native Voice Settings retain their account access policy; restoring their controls +does not establish call eligibility for every voice entry point. + +### 5. Overlay document + +The avatar overlay is a separate JavaScript realm with its own document-start +registration. To replace a pending overlay hook, dispose only +`__codexhostVoiceStartCapabilityV1`, then install from +`renderer-voice-start-capability.ts`: + +```ts +import { z } from "zod"; +import { installVoiceStartCapabilityHook } from "./packages/renderer-extension/src/renderer-voice-start-capability.ts"; +z.config({ jitless: true }); +window.__codexhostVoiceStartCapabilityV1?.dispose(); +installVoiceStartCapabilityHook(); +``` + +That wrap applies to future `registerRealtimeStarter` calls. It cannot replace a +callback already registered on this overlay document. Bind an already-registered +starter with a fresh idle overlay document that received the updated +document-start injection, or with the user's later manual full restart. Refreshing +the primary page does not fix overlay registration. Leave the overlay document +alone during an active or unknown call. A full Desktop restart is not required +to validate primary composer visibility. + +### 6. Optional overlay validation without a call + +A bounded overlay check can confirm gateway readiness and a committed projected +starter snapshot without opening the microphone. Import +`initializeVoiceCapabilityFamilies` and `bindVoiceCapabilitySubscriptions` from +`renderer-voice-capability.ts`, `overlayVoiceStarterFibers` from +`renderer-voice-start-capability.ts`, and `requestVoiceHost` from +`renderer-client-owned-voice.ts`. Use the current profile's exact `bridge` export +and `realtimeOnly: true`. Always `binding.dispose()` in `finally`. + +```js +const ready = await requestVoiceHost( + shared[profile.bridge].getInstance(), + {}, + "codexhost/voice/gateway-status", +); +const { family, gates } = initializeVoiceCapabilityFamilies(initial, profile); +const binding = bindVoiceCapabilitySubscriptions( + () => overlayVoiceStarterFibers(document, overlay.AvatarOverlayNativePage), + family, + gates, + () => true, + true, +); +try { + for (let i = 0; i < 40 && !binding.current(); i += 1) { + await new Promise((resolve) => setTimeout(resolve, 50)); + } + return { + gatewayReady: ready === true, + overlayProjectionCommitted: binding.current() === true, + }; +} finally { + binding.dispose(); +} +``` + +Report those two booleans. Observed pass on 26.1021 was `gatewayReady: true` and +`overlayProjectionCommitted: true` with no call. This check does not replace an +already registered overlay starter. + +### 7. Persist built bundles + +Live evaluation lasts for the current document. For later documents, build and +stage the installed files: + +```sh +npm run build:renderer +``` + +Copy `packages/renderer-extension/dist/production.js` onto the running +installation's `app/renderer-extension.js`, and +`packages/renderer-extension/dist/cpa-live-voice-hook.js` onto `app/cpa-live-voice-hook.js`. +On the macOS app bundle those paths are +`/Applications/codexhost.app/Contents/Resources/app/`. Discover the equivalent +payload from the current Controller `--renderer` argument on other layouts. +Back up both files and hash-verify the copies. Skip `npm run install:local` when +the goal is only to stage assets; that command can restart Desktop. + +The running Controller can keep the previous source in memory. A disk copy does +not replace the active script or change document-start registration for future +pages in this process. A later package update can replace a locally staged fix. +Leave `/Applications/ChatGPT.app` and `app.asar` unchanged. + +### Limits + +- Live injection is the current document only. +- Overlay starter wrapping is future registrations on that overlay document. +- Physical microphone, gateway call creation, audible reply, and hangup remain + separately unverified on 26.1021. +- Keep call IDs, SDP, credentials, and runtime tokens out of docs and output. + +## Desktop 26.1016.62342 qualification + +Work/gateway and native Chat have independently audited **26.1016.62342** build +**21477** profiles. The exact **61520** profiles remain supported, so installing +the host before upgrading Desktop preserves the working baseline. Unknown +versions, version/asset mismatches, incomplete pairs and mixed audited assets +fail closed before native imports. Without version metadata, exactly one +complete audited initial/shared pair is required. The selected profile also +owns its overlay or native Chat START/prewarm imports. + +| Asset | 61520 baseline | 62342 target | +| --- | --- | --- | +| Initial | app-initial-c95ddb62696e.js | app-initial-dddef47d6ef6.js | +| Shared | app-shared-2eda13eb7044.js | app-shared-fc3316d48427.js | +| Primary Chat owner | app-primary-58a09006cc0c.js | app-primary-10e84ec314b8.js | +| Work overlay | avatar-overlay-native-page-fcc17283519a.js | avatar-overlay-native-page-ef8332c6a2fb.js | +| Native Chat START | start-voice-18b08f0acdd7.js | start-voice-7cbf0a39d53b.js | +| Native prewarm entry | voice-prewarm-94271e581c1b.js | voice-prewarm-5d107129ac1c.js | +| Native prewarm controller | voice-prewarm-de985dbe0490.js | voice-prewarm-9695e173952c.js | +| Voice session | voice-session-7d5ee22a0da4.js | voice-session-c6b0f08e69a2.js | + +The 62342 ASAR SHA-256 is +`5b5cf52887cfd4de19cb5862155a8ef6cf58830c0a2fc592b72060cfcc3be869`. +Acquisition reads selected entries through ZIP ranges, then individual ASAR +header offsets; it does not install Desktop or extract the full ASAR. + +After replacing imported asset hash suffixes, the primary owner, overlay, START, +prewarm and voice-session sources equal 61520 byte for byte. Initial/shared +also differ in the embedded build commit: +`d7acb7a1b13f5e0f32a1e399e3fce368b08bd754` becomes +`293b23553a2ef1949121cc87c52983472a85ea23`. +The relevant CSS is byte-identical. Thus the 61520 contract table below also +applies to the corresponding 62342 files at the same formatted line numbers +(Prettier 3.9.6, Babel defaults, printWidth 80). + +Work evidence includes initial:111047/111148 (capability and inference gate), +:682702/682778 (existingCall and WebFetch bootstrap), overlay:4084 (owned realtime +subscription) and :4693 (starter registration). Native Chat evidence includes +START:1119/1173/1194/1260; primary:121258/121370/125082/125124 (RouteScope and +inner ComposerScope/form); shared:84170/84202/84461/104176 (provider chain, +node token, scope ref and AppScope); initial:256888/685663/685754 (store, +runtime flag and VoiceRoom). Export names and the guarded transcript adoption +contract remain unchanged. + +A broader offset comparison covers all 15,305 Renderer JS/CSS files. Aside from +asset references/build metadata, changes are CSS module identifiers in control +and placement UI, and the automation dialog's model-list client version changing +from 0.163.0-alpha.11.1 to .11.2. The consumed composer, routing, usage-gate and +voice contracts have no source change requiring additional integration. + +This is source qualification plus isolated regression coverage. The installed +upstream app remains 61520 until an explicit upgrade. No 62342 live Desktop, +physical microphone, audible reply, transcript adoption or cleanup acceptance is +claimed. Issues [#7](https://github.com/karlorz/codex-host/issues/7) and +[#8](https://github.com/karlorz/codex-host/issues/8) retain functional acceptance; +[#4](https://github.com/karlorz/codex-host/issues/4) retains the original Chat +acceptance limits. A matching compat-daily pin warrants a recovery comment, +independently of those live checks. + +## Desktop 26.1016.61520: client-owned calls + +This Desktop build always creates its WebRTC call in the Renderer before asking +app-server to attach with `transport: { type: "existingCall", callId }`. The old +Statsig gate is absent. The two `experimental_realtime_*_base_url` settings alone +therefore cannot route that initial frontend request. + +On the exact audited `app-initial-c95ddb62696e.js` and +`app-shared-2eda13eb7044.js` assets (Desktop **26.1016.61520**), the hook wraps the loaded native WebFetch +singleton's `fetch` method only for `POST /wham/realtime/calls`. It forwards the +native SDP/session through `codexhost/realtime/call` on the local Host connection +when the matching route uses the gateway. Codex Work honors **Work → Voice connection**; external Harness calls always use Host. **Chat → Codex → Configured gateway** uses the same Host path. **Chat → Codex → Desktop default** leaves `/wham` untouched. **Chat → ChatGPT** invokes native browser voice separately and leaves `/wham` untouched. Other WebFetch requests +pass through with their original receiver and +arguments. +External Harness ownership also comes from an explicit `codexhost/` backend +model in native request metadata, so a cold avatar overlay does not depend on +the main composer's routing state or remembered launch. Explicit Chat ownership +still takes precedence. Safe bootstrap diagnostics report the selected +`desktop` or `gateway` route without model names, identifiers or credentials. +No global fetch, message transport or application archive is replaced. + +Explicit Work launch ownership selects the Work connection preference in the +avatar overlay, where no primary Composer is present. It applies even when the +backend model name has no `codex` marker (for example, `gpt-6.1-sol`) or call +metadata omits the model. Model-name and visible-Composer detection remain +fallbacks for launches without an explicit product experience. Explicit Chat +ownership still selects the Chat connection preference. + +The Host reads the configured realtime URLs and selected provider's +`experimental_bearer_token` or `env_key`. Both URLs must use HTTPS and share an +origin. Header forwarding is allowlisted twice: the Renderer bridge drops every +name outside `REALTIME_CALL_FORWARDED_HEADERS` before the request crosses to the +Host, and the Host applies the same list again before sending. Desktop OAuth, +`ChatGPT-Account-Id`, `Cookie` and every other unlisted header are therefore +excluded at both boundaries. Credentials stay in the Host; only the SDP response, +status and allowed response headers return to the Renderer. Configuration and +transport failures do not fall back to the signed-in ChatGPT account. The +Settings preference for the request's mode is checked again at call time. + +Cancellation aborts an in-flight bootstrap. A known created call is hung up if +startup fails or its answer arrives after cancellation. If no call identity is +available, remote cleanup cannot be confirmed. Existing native app-server +sideband and stop requests continue through their configured gateway route. +External Harness Threads attach to the same existing call without a second +creation or SDP exchange, retaining the selected Harness for new-chat launches. + +The native frontend exports used here are version-specific and must be re-audited +before another Desktop build is enabled. An enabled CPA scheduler only selects +accounts for requests that actually reach CPA. Its configured inventory and +selection counts do not establish successful upstream calls. + +### Work voice capability on 26.1016.61520 + +The audited native capability family requires first-party inference for `voice.realtime` and `voice.thread` when rollout `3683713372` is enabled. A custom provider can therefore hide Start voice even when gateway transport is ready. + +`renderer-voice-capability.ts` projects the owning visible local Work composer's subscriptions to those two capabilities. It requires the exact 26.1016 initial/shared assets, active client-owned and external-launch hooks, enabled injection, a successful live Host gateway-readiness check, and the Work gateway choice or an explicitly owned local external Harness selection. ChatGPT voice, remote Hosts and other capabilities retain native eligibility. The explicit Codex Chat gateway option qualifies the owned Chat form and overlay as described below. Hidden composer roots do not make the visible composer ambiguous; multiple visible roots fail closed. + +Runtime settings reports return `gatewayReady: null` because the console skips its callback to the requesting Host. The Renderer reads the existing local `codexhost/voice/gateway-status` method directly when readiness is omitted. Transient settings or status request failures retain the last successful readiness; an explicit negative result or disposal revokes it. No successful readiness read means the projection stays pending. Route and +readiness caches share the document lifetime with the retained voice-hook globals. +A replacement Renderer bundle updates the same caches; disposing the settings +poll revokes readiness until its replacement completes a successful check. + +The projection replays the native readonly capability dependencies with the first-party inference restriction omitted only for that evaluation. It retains loading/error, access-policy, rollout and config-feature checks. It does not change the provider name, native capability atoms or global Statsig results. Family cache-key, scope-chain, signal and committed subscription ownership must all agree; absent, ambiguous or changed bindings fail closed. Changing scope, routes, readiness or injection restores native subscription getters. + +A controlled live Renderer trial on 26.1016 showed Start voice changing from absent to one visible enabled button in an existing external Harness Work composer. Native capability values and the inference rollout remained unchanged. Disposing the projection restored native visibility. No call, microphone capture, audio connection or remote cleanup was tested. + +### Work gateway call-start capability on 26.1016.61520 + +The composer and the persistent avatar overlay have separate capability readers. +The overlay's native starter reads `voice.realtime` again and emits +`Voice chat is not ready (voice_disabled)` when that reader is false. This check +precedes microphone setup, workspace readiness, call creation and app-server +attachment. Showing the composer button alone does not qualify call startup. + +`renderer-voice-start-capability.ts` wraps the overlay's native +`registerRealtimeStarter` callback through the service reference. It accepts +explicit local Work composer starts using the Work gateway preference, explicit Codex Chat gateway starts, or a +new-thread request carrying an explicit `codexhost/` model. It requires active +gateway transport, a successful local Host readiness read, and the exact initial, +shared and `avatar-overlay-native-page-fcc17283519a.js` assets. Within the exported +native overlay page, exactly one owned `voice.realtime` subscription can qualify. +The hook waits for React's committed capability snapshot before invoking the +native starter. Cancellation, failed startup, completed startup, unregistration +and disposal release the projection. Native access policy, treatment, workspace, +media and busy checks remain authoritative. ChatGPT Chat, remote Hosts, global +hotkeys and unowned starts retain native eligibility. + +The overlay hook-only bundle must be updated alongside the primary Renderer. +An already-running Controller can retain its previously loaded bundle; staging +the installed file does not change that in-memory source. A controlled reload of +an idle voice overlay can qualify the new document without restarting Desktop or +reloading the primary window. The Controller's Zod `jitless` bootstrap must run +before evaluating the bundle under Desktop's Content Security Policy. + +A controlled native Work/Harness trial on 26.1016 returned gateway HTTP 201, +applied the SDP answer, attached app-server with `transport: existingCall`, and +reported connected while `model_provider = "custom"`. Silent generated audio +replaced physical microphone input for the trial. Native stop restored an idle +overlay, native capability false and the unchanged inference rollout. Physical +microphone capture and spoken conversation quality remain separately untested. + +cc-switch can deliberately use `model_provider = "custom"` to unify official and +third-party resume history. Its local provider writer preserves ChatGPT login +separately and can publish provider-scoped gateway credentials. CodexHost reads +that configuration without renaming the provider or modifying history. The +`voice_disabled` rejection is Desktop's capability restriction; it is not an +upstream gateway response or a session-history parsing error. + +### Existing and new ChatGPT conversations: native Instant Connect + +The retained native Chat baseline is **26.1016.61520** build **21421**: +`app-initial-c95ddb62696e.js`, `app-shared-2eda13eb7044.js`, +`start-voice-18b08f0acdd7.js`, and `voice-prewarm-94271e581c1b.js`. +Native Chat uses Desktop's browser Instant Connect stack. Its authentication, +account availability, START/RESTART commands, WM requests, microphone, VoiceRoom +and cleanup remain native. Codex Chat uses the separate Codex overlay and its +own connection preference. Missing or ambiguous native Chat bindings reject +without falling through to that overlay. + +The audited 61520 baseline ASAR SHA-256 is +`c9945f821fcbaf48424933a82860beac4dd4ad8b8a5444a75568fa6e38a1ad4b`. +Only individual assets were read by ASAR offsets. These native file references +use Prettier 3.9.6 Babel formatting with default options (printWidth 80): + +| Contract | 61520 evidence and binding | +| --- | --- | +| Native START | `start-voice-18b08f0acdd7.js:1119` accepts scope, composer props, metadata and signal; `:1173` reads `B1n`; `:1194` dispatches START. Exports `n`/`t` at `:1260`. Native navigation, active-session and availability checks remain authoritative. | +| Scope ownership | `app-primary-58a09006cc0c.js:121258` holds the composer's RouteScope ref (`initial.MOr`). `app-shared-2eda13eb7044.js:84461` defines that ref's get/set and chain. Its AppScope node must match the native context (`shared.Qst`, defined at `:104176`). | +| Saved and draft selection | `app-primary-58a09006cc0c.js:121370` builds an inner chatgpt ComposerScope value with the current selection; `:125082` mounts its provider above the form at `:125124`. `app-shared-2eda13eb7044.js:84170` publishes the chain through that context provider; `:84202` stores the scope descriptor as the node token. Draft IDs originate in useState at `:129002`. The nearest committed ComposerScope provider between the form and owner must carry the native `initial.CDr` token, the owner's RouteScope and AppScope nodes, and the same selected conversation ID. The owner's consumed context can still be the outer new-Chat scope and cannot substitute for the form's provider. | +| Store and runtime | `app-initial-c95ddb62696e.js:256888` exports the native store accessor as SPn. `:685663` initializes browser voice with oT and flag aT; `:685754` mounts native VoiceRoom. `voice-session-7d5ee22a0da4.js:7745` publishes VoiceRoom, START command handling and native cleanup through initializeVoiceSession. | +| Prewarm | `voice-prewarm-94271e581c1b.js:1` re-exports the controller from `voice-prewarm-de985dbe0490.js`. Its AppScope, ComposerScope, selected-system-hint reads, eligibility, abort watchers and cache ownership stay native. React and createRoot are shared.FBt and shared.PNt, the same React/ReactDOM instance used by Desktop. | +| Adopted-draft navigation | `initial.Ycr` is the current location; Nir invokes native AppNavigation. B1n resolves the adopted backend ID; n0n and Y1n resolve the transcript mapping/current node. `shared.Vot` constructs the draft page path. Navigation waits for the connected session, adopted ID and current transcript message; an early server ID alone is insufficient. | +| Primary action presentation | `app-initial-c95ddb62696e.js:202194` places `icon-primary-action text-composer-primary` on the icon, while `:202249` applies disabled opacity/cursor classes to the button. Voice reuses Send geometry and colors, with its own disabled state. | +| Home headline | `app-primary-58a09006cc0c.js:140384` marks the rotating headline with data-headline; `:140770` puts it in the --vt-splash-screen-headline container. During active native voice on that same Chat draft, the unique visible headline container is hidden with visibility to preserve layout and leave room for the native orb. Ending, leaving the eligible composer or disposal restores its prior visibility. Work voice does not suppress the headline. | + +The visible Chat owner must be non-temporary, writable, and allow external +egress. Its props, selection context, RouteScope chain, AppScope context and +inner ComposerScope must agree. Saved Chats require a backend ChatGPT UUID; +unsaved Chats require a valid native local-chatgpt draft locator and no backend +ID. A Desktop Thread-Id is never converted into a ChatGPT UUID. + +For ChatGPT voice, the hook mounts one native prewarm controller per AppScope +under both native contexts and enables the native browser-voice flag. RouteScope +replacement during navigation updates the provider chain without replacing the +AppScope runtime. Starting or active voice retains the runtime while the composer +is hidden. Leaving Chat after voice ends restores the prior flag and unmounts +the controller. Native eligibility determines whether a selection can prewarm. + +The Start Voice waveform sits immediately before the visible native Send button, +including compact Chat home composers without Dictate. It follows Send when its +wrapper changes and is removed while the form has no unique visible submit +action (such as during Stop). +It matches the native voice control row's gap-2 spacing (normally 8px), +supplementing only the spacing missing from its parent and centering the action. +Native voice's microphone/end row uses flex items-center gap-2 in +voice-session-7d5ee22a0da4.js:7563. Existing parent spacing is not doubled. +The waveform uses the native contrasting icon foreground and stays enabled +when an empty message disables Send. Its dimming follows voice startup, +streaming and active-call state independently of Send. +It invokes native START with the current +RouteScope, props, composer metadata and cancellation signal. While present it +hides the competing Codex Start voice entry in that Chat composer. Switching to +Codex voice or leaving Chat restores the original entry. Saved Chats remain +selected. Draft navigation is attempted once, after adoption, and cancels if the +user leaves, the session changes or ends, or the binding is released. Missing +navigation exports leave native voice available with manual transcript selection. + +Home stays selected until native voice has connected, adopted the Chat and +published the current transcript message. Starting while muted or without +microphone input does not by itself satisfy those conditions. The orb and +microphone/end controls alone do not establish transcript readiness. Early page +navigation can interrupt native voice; no timer or early server ID bypasses +the adoption and transcript guards. + +Instant Connect may adopt a prewarmed voice-session ID during startup. The +native `initial.HJt` request-generation counter remains unchanged for that +adoption and advances for a new voice intent. Draft selection tolerates the +startup ID change only with the same proven generation; it still cancels on +a new intent, an established session ID change, navigation, abort or voice end. +If the optional generation reader is absent or invalid, session ID changes +continue to cancel selection. Native START remains available independently. + +The native version/asset and scope guards fail closed on unsupported builds. +Lifecycle tests cover saved and draft starts, prewarm contexts, navigation, +cancellation, retry, hidden composers and disposal. Native Chat microphone, +connected WM audio and account eligibility have **not** been exercised on the +running Desktop: Computer Use refuses access to com.openai.codex. Static source +qualification and tests do not establish those live results. Issue +[#4](https://github.com/karlorz/codex-host/issues/4) retains that acceptance limit. +A matching compat-daily pin clears version drift only. + +### Codex Chat gateway capability + +Chat → Codex → Configured gateway qualifies the visible Chat composer's owned +voice.realtime/voice.thread subscriptions, including the native hooks on the +form's owning component. The owner must match the form's eligible Chat props and +selection context; ambiguous owners or multiple visible forms fail closed. Work +routing state is not required to claim a Chat form. The overlay starter accepts +explicit local Chat starts only when that Codex gateway preference is active. +ChatGPT voice and Codex/Desktop default retain native capability checks. Both +composer and overlay projections preserve non-inference availability checks; +no global Statsig or provider-name override is used. + +The client-owned WebFetch hook initializes shared.Oit before reading its lazy +singleton exports. Call creation for explicit Codex Chat gateway launches uses +Host credentials; native ChatGPT voice does not use this /wham transport. + +Debug events use `__codexhostChatVoiceDebug` and the bounded localStorage key +`codexhost.chat-voice-debug.v1`. They contain event names, timestamps and routing +booleans, with no conversation IDs, SDP, credentials or raw errors. Native store +subscriptions additionally record active/connected/session-started and +server-conversation-ready flags, an allowlisted connection state, and the +allowlisted microphone request state. A +START return is a command acknowledgment, not evidence of a connected call. +These subscriptions are removed with the AppScope binding. + +The production Renderer entry installs this hook on every document. Its polling +rebinds after leaving Chat, returning, or native composer replacement; no manual +session activation is required. The installed codexhost Renderer must contain the +change for it to survive a Desktop restart. A live CDP injection alone does not +update the installed release, and a subsequent codexhost update replaces a local +installation patch. + +Attended verification uses both an existing web-backed Chat and a new Chat draft: +click **Start Voice**, +confirm the conversation remains selected, the native state reports `wm` and +`connected`, and **End Voice** is present. Confirm speech and microphone release +separately. Leave and return to each Chat, then reload Desktop and confirm the +waveform control returns. Then verify Work/Grok voice still reaches the configured +CPA gateway. + +### Account failover: current limitation + +Bounded quota failover is **not** implemented, because no public CPA plugin +contract can observe a pre-call quota result for this route. Evidence from the +deployed baseline: + +- `internal/client/codex/live/live.go:223` selects once via `selectOAuth`, and + `:330` sends that single selection once (`performRequest(selected)`). There is + no retry loop, no `tried`-set and no second `SelectAuthByKind` call on this + path. +- The plugin-facing scheduler contract is pick-only: `SchedulerPickRequest` / + `SchedulerPickResponse` in `sdk/pluginapi/types.go:487-546` carry no outcome or + result field, and the only scheduler ABI method is `scheduler.pick` + (`sdk/pluginabi/types.go:55`). A scheduler plugin cannot learn that its pick + returned `usage_limit_reached`. +- The result-feedback contracts that do exist — `RequestLifecyclePlugin` + (`HandleRequestComplete`), `RequestInterceptor` and `ResponseInterceptor` — + are emitted only by `BaseAPIHandler` execution paths + (`sdk/api/handlers/handlers_execution.go`, `handlers_stream.go`, + `handlers_interceptors.go:160-165`). The Codex live handler is a standalone + `Handler` (`internal/client/codex/live/live.go:49`) that never participates in + interception or lifecycle, so those callbacks never fire for `/v1/realtime/calls`. +- The `QuotaProvider` contract (`sdk/pluginapi/types.go:1577`) reports and resets + quota for management clients; it is not wired into live selection and cannot + skip an exhausted account before a call. +- Disabled-account exclusion is already enforced by core before any plugin sees a + candidate: `sdk/cliproxy/auth/conductor_selection.go:1852` drops + `candidate.Disabled`, and `availableAuthsForSelector` narrows the set the plugin + receives. `cpa-live-voice` intersects that set with `live_auth_ids` + (`selector.go` `PickAuth`), so the disabled Alice Plus credential cannot be + selected — no plugin change is required for exclusion. + +Consequently a retry that skips the exhausted account and re-selects a remaining +priority needs a new CPA core seam: the live handler must record the definitive +`usage_limit_reached` result and re-enter selection with that account excluded. +That is a core change, not a plugin change, and is out of scope here pending a +separate decision. The smallest core seam would be an exported +"exclude these auth IDs and select again" entry point plus a single bounded retry +in `Handle` after a definitive pre-call quota response. + +## Desktop 26.1007.30912: call-ownership gate + +Desktop’s voice-start path sets `clientOwnsCall` when the `hostId` argument is `durable` **or** both of these hold: the host's app-server version supports the `threadRealtimeExistingCall` capability (a version comparison, not a stored flag), and Statsig gate `206132468` passes. That second branch POSTs to ChatGPT `/wham/realtime/calls` and then starts `thread/realtime/start` as `ExistingCall`. CPA never sees creation, so the sideband websocket 404s. + +The hook forces Statsig `checkGate("206132468")` to false on every client registered on `__STATSIG__`. In the audited build that gate has no other reader, so the result is durable-only ownership: a local host gets `clientOwnsCall=false` and `transport=webrtc`, and CPA can `POST /v1/realtime/calls`. Other gates pass through unchanged. Official app-server traffic is not rewritten. Host still passes official `thread/realtime/*` through. + +The legacy gate override reads the Work gateway preference at install time. Per-request routing and the separate Chat gateway switch apply to the 26.1013 client-owned fetch wrap. + +## How it is installed + +1. When the saved injection setting is on (unless overridden by `CODEXHOST_CPA_LIVE_VOICE=0|1`), the Controller prefixes the production Renderer source with a frozen `globalThis.__codexhostCpaLiveVoiceHookV1` record (audited Desktop version and `app-initial` asset name only). +2. `Page.addScriptToEvaluateOnNewDocument` plus immediate `Runtime.evaluate` inject the Host Renderer into the primary `app://-/index.html` page. +3. Voice starts on the avatar overlay (`app://-/index.html?initialRoute=/avatar-overlay`). That page gets a **hook-only** bundle (`cpa-live-voice-hook.js`), not the Host Renderer binding. The overlay loads the same audited `app-initial` asset in its own JavaScript realm with its own Statsig client. The overlay bundle also drops Host `thread/started` and turn/item notifications for Threads that have not hosted voice in that overlay, so a leftover overlay does not upsert CLI-created Threads or rebind the same Turn into the main chat. Known voice Threads keep receiving notifications after hangup because native stream ownership can outlive the call. +4. The Renderer hook no-ops unless the exact audited `app://-/assets/app-initial-6350d04672d0.js` asset is present (performance entries, script src, or link href, including relative URLs). Unknown assets fail closed; matching text in a query string or source-map filename is insufficient. +5. On a matching page it polls for up to 60 seconds until a Statsig client is registered, then wraps that client's `checkGate`. The gate is read when voice starts, so the wrap only has to land before the first voice click. + +Verify before a trial: on the overlay, `__STATSIG__.firstInstance.checkGate("206132468")` returns `false`. + +Overlay discovery, connection, and injection share one timeout. Overlay attempts run concurrently, and a stalled or disappearing overlay does not prevent the primary Host session from becoming ready after that deadline. + +The Controller sets Zod's global `jitless` configuration before evaluating either +the primary Renderer or the overlay bundle. Response validation therefore runs +without JavaScript string compilation under Desktop's Content Security Policy, +including when the bundle was initially evaluated through the debugger. The +policy does not need `unsafe-eval`. + +The hook does not copy OpenAI frontend JavaScript into git. Tests cover the asset gate and Statsig override with synthetic clients. + +## Version gate + +Audited for Desktop **26.1007.30912** (gate path, primary bundle `app-initial-6350d04672d0.js`), **26.1013.40934** (client-owned path), **26.1016.51333**, and **26.1016.61520** (client-owned WebFetch wrap on `app-initial-c95ddb62696e.js` / `app-shared-2eda13eb7044.js`). Native Chat voice is pinned separately to 26.1016.61520 with live microphone/audio acceptance pending. Re-audit before enabling on another build. Fail closed: no wrap, no ASAR patch, keep `~/Applications/ChatGPT-Voice.app` if that copy is still the working path. + +## Invariants + +- Do not overwrite `/Applications/ChatGPT.app` (Developer ID `2DC432GLL2`). +- Do not reuse `install_voice_patch.py`. +- Do not quit or replace the working Voice copy until the CodexHost path proves `POST /v1/realtime/calls` 201. +- Do not put call IDs, SDP, or auth material in docs, git, or command output. +- Do not `git push` this branch or open an upstream PR unless explicitly requested. + +## External Harness voice (local development build) + +New-chat voice launches use the primary Composer's current Harness selection. +On the audited Desktop build, the primary Renderer imports the already-loaded +browser module `app-shared-8bd9d0399038.js` (26.1007, export `fN`) or +`app-shared-2eda13eb7044.js` (26.1016.61520, export `ist`) and wraps only its native +`realtimeVoiceRuntime` service reference. Its RPC stubs themselves +are immutable. For `composer_button_new_thread`, the wrapper snapshots the +matching Host's selected transport carrier into +`activeCollaborationMode.settings.model`. The separate voice overlay forwards +that collaboration mode into native Thread creation, which uses its model for +`thread/start`. This prevents a new Grok voice chat from silently becoming an +official Codex chat. No browser message transport or ASAR code is patched. + +The wrapper requires both exact audited frontend assets. It imports only after +they are present and waits up to 60 seconds for the native services record. +Existing-Thread launches, avatar/global launches, official Codex selection, +ambiguous Composers and mismatched/disconnected Host owners pass through. +Selection is captured before launch, so changing the draft later cannot retarget +an in-flight voice chat. This is generic to the selected Harness, not Grok-specific. +The new-chat correction has focused routing/forwarding tests; installed native +button acceptance is still required. Earlier installed acceptance exposed the +missing new-chat selection handoff despite successful isolated transport trials. + +External `thread/start`, `thread/resume`, and `thread/fork` report +`sandbox: { type: "externalSandbox", networkAccess: "enabled" }`. Codex does not +enforce a sandbox on those Threads; Harness permission modes stay authoritative. +Native voice overlay startup also sends `thread/settings/update` with +`summary: "detailed"` when concurrent reasoning summaries are enabled. External +`thread/settings/update` returns an empty success without applying Codex +sandbox or permission changes; Harness permission modes stay authoritative. +Official Codex sandbox traffic is unchanged. + +External Threads now have a separate Host-owned handler for native WebRTC audio +`thread/realtime/start` and `thread/realtime/stop`. Official Codex Threads continue +through the native app-server unchanged, including their existing gateway routing. +The handler uses the selected external Thread's existing HarnessSession and normal +Turn/approval projection; it does not start a second agent or import a concrete Adapter. + +The transport targets CPA's frameless V3 client-delegation protocol. It posts the +Desktop SDP and client-delegation session to the configured realtime call base, +returns the answer through native `started`/`sdp` notifications, and joins the +configured WebSocket base at `/live/{callId}`. Reopened calls on an existing +External Thread include that Thread's user and agent messages as GPT-Live +`session.initial_items`, using the V3 role-bearing message format. A native +`delegation.created` with task text and `user_bidi_turn_id` starts Harness work +and binds its speakable handle (`item.id`) to that user turn. The service can +wait for the client answer before emitting user `turn.done`, so that final +transcript is not a prerequisite for a linked handoff. A completed user +`turn.done` starts work only if the same turn has not already been registered. +Matching uses protocol turn identifiers, including `turn_id` / `turnId`, not +text similarity or timing. A handle without a turn identifier is ignored for +routing. Both configured +bases must use HTTPS and share an origin. Authentication uses the +selected configured model provider's +`experimental_bearer_token` or `env_key`; OAuth from `auth.json` is never copied. +The Host reads `CODEX_HOME/config.toml` only and does not mutate gateway settings. +Profile/command-line credential overrides are not resolved by this prototype. + +V3 handoff text and user `turn.done` transcripts go to that same external Thread. +An open call also follows ordinary text or attachment tasks in that Thread, +including a task already running when the call opens. Completed agent progress +messages are forwarded during work: explicit `commentary` is eligible at Item +completion; an unphased message is eligible after a later Item starts, proving +that it is not the inferred final answer. The final result waits for successful +Turn completion. Partial messages, private reasoning and tool output are excluded. +Delegation IDs are deduplicated within a call and used only +to address `delegation.context.append` when the handle names the user turn. +Results without a linked handle are appended with the supported +`session.context.append` event. A handoff with no task text or no turn identifier +does not start work and does not bind a later utterance. Partial `turn.created` +and `turn.delta` transcripts do not start work. Transcripts are +projected through native notifications. Closing the call releases voice resources +and suppresses late speech, but does not cancel a running agent task: explicit task +cancellation remains the existing `turn/interrupt` operation. The avatar overlay +keeps delivering known voice Threads' `turn/*` and `item/*` notifications after +hangup, including later text turns. The primary window may remain a native +stream follower of the overlay, so call closure or an idle status does not +release notification delivery. Deleting a Thread removes it from this set. +`turn/interrupt` for a Turn that is already complete, or that is not the active +Turn, returns success without cancelling a newer Turn. + +Context forwarding confirms delivery to the voice control connection. Exact +audible progress and completion still require an attended Desktop call test. + +External Agent Messages completed during a voice call use Desktop's native +`::codex-realtime-inline{}` Markdown presentation. The Host adds the directive +only to Desktop projections; Harness history and speech context retain the +original answer. The Turn mapping records `realtimeInline: true`, so history +reloads and Host restarts preserve the same written presentation. This keeps +exact links and other Markdown visible alongside the speech transcript. + +Grok text Item IDs use the native Session, visible Turn index, Item kind and +message ordinal in both live projection and history replay. Chunk boundaries, +optional native message IDs and message text do not determine those IDs. +History hydration can therefore replace the streamed Item while preserving +separate messages that happen to contain identical text. + +The speech service is instructed to delegate every question and task, including +simple arithmetic; it supplies speech rather than owning task reasoning. +The prototype defaults to Cantonese and the protocol's `cove` voice; supported V3 +voices can be requested. Text/audio append transports, +overlapping-task steering, newer public GPT-Live event dialects, remote-host media +topology, and seamless call recovery are not supported. A rejected control connection +or session-initialization timeout emits error/closed notifications and performs +bounded hangup cleanup. Retry is explicit and never resubmits an agent task. +If the gateway returns success without a usable call identity, the Host rejects +startup and releases the response body, but cannot identify a safe remote hangup +target. Remote cleanup is then unconfirmed. Network failures during hangup are +also reported as `cleanup_unconfirmed`; a closed local overlay is not proof of +remote cleanup. + +Verification includes an isolated real native call through New API/CPA (creation, +WebRTC, data channel, session initialization, sideband, and hangup), a real Host +trial using the public Grok plugin and an existing microphone clip, and Host +fixtures for multiple Harness IDs and official passthrough. The Host trial +observed a completed Grok Turn and the returned spoken-answer transcript, with +no official app-server request; the final trial confirmed hangup with HTTP 404 +(call already gone). One preceding trial returned the answer but did not confirm +hangup. These are isolated headless-page trials, not installed Desktop button +acceptance or proof that every plugin is voice-compatible. Continuous conversation, +speech interruption, task cancellation and approvals need attended acceptance. + +### Automated smoke and attended verification + +From the repository root, run: + +```sh +npm run test:voice:smoke +``` + +This builds the TypeScript packages and packaged Harness plugins, then runs the +focused voice regression files using `tests/vitest.config.js`. It does not launch, +reload or stop Desktop, create a real call, or require gateway credentials. It +covers: + +- Completed public progress, streamed final text, successful Turn completion, + and exclusion of private reasoning and tool output. +- Ordinary text/attachment tasks during an open call, tasks already running + when voice opens, and output held until the control connection is ready. +- Linked handoff deduplication, history seeding, gateway routing and cleanup, + and suppression of late speech after hangup without cancelling Harness work. +- Claude native completed scout results containing quoted background-launch + wording, while preserving genuine asynchronous scout launches. +- Overlay delivery after hangup, new-chat Harness selection, audited call hooks, + preference gating, and CSP-compatible validation setup. + +These files also run in the normal TypeScript CI suite; the named command is a +local focused entry point. Use it instead of `npm test -- `: the root +`test` script invokes the broader TypeScript and Rust suites. + +For attended verification, use the existing configured Desktop installation; +no separate voice hook, mock process or worktree is required: + +1. Open an external Harness Thread and start voice. Keep the call open through + task completion. +2. Submit a normal text task while the call is open, asking for a short read-only + inspection, one separate public progress message before tool work, and a brief + final summary. This exercises ordinary-turn forwarding. A task spoken into + voice exercises the delegation path instead. +3. Verify progress only if the Harness actually emits a separate public message. + A progress heading inside the final answer is one final message and does not + verify intermediate speech. Private reasoning and command output are excluded. +4. Check that the final answer is heard before hangup, its written Markdown and + links appear in the voice timeline, the backend Turn completes, and Desktop + returns to idle. Reopen the chat and check the same links again. Compare a + manual Grok chat with an automatically delegated chat using the same Model + and Thinking option; each answer should appear once. +5. If the UI appears stuck, preserve the call and inspect the backend Turn and + Desktop state before restarting or reloading. An active command is expected + to remain busy. Judge a stale busy state after the final backend reply. + +The attended Claude Code check on Desktop **26.1013.40934** on 19 October 2026 +confirmed an audible final answer, completed Turn and idle Desktop. That run +emitted no separate public progress message, so intermediate audio remains +unverified. Automated transport assertions do not establish physical playback, +universal Model compatibility or recovery from every long-task failure. + +## Harness session rules (Grok ACP `_meta.rules`) + +The Grok Adapter passes Desktop session rules through Grok's official ACP +`session/new` and `session/load` `_meta.rules` field. Grok appends that text in +a `` block. Host Runtime does not write `~/.grok/rules/` or +`~/.claude/rules/`. On startup it removes Host-owned copies of +`codexhost-desktop.md` whose content still matches a previous managed digest; +a user-edited copy is preserved. TUI and Orca Grok sessions therefore stay +free of that file unless the user kept an edited copy. + +Grok appends that text to `system_prompt.txt` inside ``. The +agent never prints tokens, endpoints, or runtime descriptors. + +The rule covers composer Model selection (including that `flash-max` is Gemini +Flash), starting in the real repo, voice versus text tool work, per-Harness MCP +and skills, Codex Desktop chrome / Markdown View, and official Codex live voice +versus Host-owned external `thread/realtime/*`. diff --git a/docs/operations/fork-versioning.md b/docs/operations/fork-versioning.md new file mode 100644 index 000000000..54908ec07 --- /dev/null +++ b/docs/operations/fork-versioning.md @@ -0,0 +1,267 @@ +# Fork versioning (`karlorz/codex-host`) + +This checkout tracks [BytePioneer-AI/codex-host](https://github.com/BytePioneer-AI/codex-host) as `upstream` and publishes only to the fork [karlorz/codex-host](https://github.com/karlorz/codex-host). It does not define upstream BytePioneer release policy. + +Package identity follows the same fork-scope rule as other karlorz forks: a permanent `@karlorz/` rename, not a publish-time override, and not `@codexhost/*`. + +## Preserve Live Voice when syncing upstream + +This is the operational runbook for upstream syncs. Preserve fork patches, merge +upstream on a branch from fork `main`, then verify the fork behavior and Desktop +pins separately. The fork's [Live Voice integration](cpa-live-voice-desktop-hook.md) +and [Grok Computer Use / IAB settings](../harnesses/grok/computer-use-settings.md) +remain fork responsibilities. + +An upstream merge cannot by itself qualify Desktop voice. At the 22 October 2026 +baseline (`origin/main` at `10ba06b7`), Work and Chat pins are **26.1013.40934**; +the reported Sparkle latest is **26.1016.61520** (fork issues +[#3](https://github.com/karlorz/codex-host/issues/3) and +[#4](https://github.com/karlorz/codex-host/issues/4)). The dirty Work overlay and +capability projection target **26.1016.61520** and are not part of that baseline. +Neither merging upstream nor changing a version string qualifies 26.1016.61520. +Recheck the live appcast and exact source pins at each sync. + +### Protect the dirty tree and authorize the merge + +Documentation updates can proceed independently. Before an actual merge, settle +both attended decisions: how to preserve dirty work, and whether to merge now. +The recommended preservation is a named backup branch with a WIP commit after +reviewing the file inventory; do not put an unreviewed bulk feature commit on +fork `main`. Merely creating a branch does not save dirty files. + +1. Record `git status --short --branch`, the fork-main OID, staged changes and + untracked files. Inventory the voice overlay/projection and IAB work as well + as other local changes. Keep secrets, ignored build outputs and local + environment files out of commits. +2. On the approved backup branch, explicitly stage the intended tracked and + untracked source files and create the WIP snapshot. Record its branch and + commit OID. Alternatively, park tracked and untracked work in an explicitly + named `git stash push --include-untracked` entry, record its OID, and inspect + its inventory. Verify that the recovery copy contains the intended files. +3. Return to fork `main` with a clean tracked/untracked source tree before + creating the sync branch. Approved feature work may instead be reviewed and + integrated into fork `main` first. Do not discard remaining dirty changes to + make this precondition pass. +4. Obtain Karl's explicit **merge now** decision. A recommendation to document + the process, silence, or a saved backup does not authorize the merge. + +On a disk-constrained checkout, use Git snapshots and targeted asset reads. Do +not run Docker or extract the full Desktop ASAR. Run Rust validation only when +an approved merge affects Rust and the selected checks require it. + +### Merge on a fork sync branch + +Before an upstream sync: + +1. Verify `origin` is `karlorz/codex-host` and `upstream` is + `BytePioneer-AI/codex-host`, including push URLs. Fetch both. Compare fork + `main` with `origin/main`; reconcile unexpected divergence before continuing. + Record the fork and upstream OIDs used for this sync. +2. Create a uniquely named `sync/upstream-YYYY-MM-DD` branch from fork `main`, + then merge the recorded `upstream/main` OID there. Never reset or replace fork + `main` with upstream. A conflict-free merge still needs behavioral verification. +3. Resolve conflicts and inspect overlapping changes by responsibility. Do not + apply a blanket choice of either side. Preserve the contracts below, or use + an upstream implementation only after equivalent behavior is verified. +4. Restore the intended fork voice, overlay/projection and IAB changes from the + backup, reviewed commits or stash as needed. For a stash, apply the saved OID + and keep the recovery entry until verification finishes. Inspect each diff; + do not blindly replay a mixed WIP snapshot or restore whole upstream files. + Preserve fork package/update identity, `tools/compat-daily/`, the + `compat:daily` / `compat:daily:test` scripts and + `.github/workflows/compat-daily.yml`. IAB preferences must retain their real + supported/unsupported binding semantics. + +If resolution cannot preserve a contract, stop and record the unresolved +ownership or behavior. An in-progress merge can be aborted with `git merge --abort` +before applying saved WIP. Keep the backup/stash; do not use a destructive reset +on fork `main` as recovery. + +### Required fork contracts + +Required voice contracts: + +- **Existing switches:** Console **Inject live-voice hook** (default on, relaunch) + is the process-lifetime wrap. Desktop **Work voice** defaults to **Configured gateway** + and **Chat voice** defaults to **ChatGPT**. Chat Instant Connect on + `https://chatgpt.com/realtime/wm` keeps + Desktop ChatGPT login. Work uses Host → gateway. A disabled Work switch calls + the original `/wham` WebFetch. The usage-warning switch does not route calls. +- **Native call creation:** On the audited client-owned Desktop flow, + `renderer-client-owned-voice.ts` forwards the native call-create request through + `codexhost/realtime/call` to `native-voice-gateway.ts`. Preserve the Host RPC + dispatch, Renderer entry points, shared `realtime-call.ts` contracts and + `realtime-gateway-config.ts`; gateway credentials stay in the Host, request + headers remain allowlisted, and gateway failures do not fall back to Desktop + OAuth. +- **CSP:** `production-controller.ts` sets Zod `jitless` before both the primary + Renderer and overlay bundles are evaluated. Preserve strict CSP without adding + `unsafe-eval`. +- **Lifecycle and Harness:** Preserve cancellation/hangup cleanup, external + existing-call attachment without duplicate creation, and the selected Harness + for new-chat launches. Official Codex realtime handling continues through its + native app-server path. +- **Version boundary:** Preserve exact audited asset checks. After a Desktop + update, re-audit its native exports and call ownership before enabling another + build; a passing source merge does not qualify new Desktop assets. +- **Rollout and scope:** Keep `model_provider = "custom"`. Do not globally + override Statsig `3683713372`. Work capability/start projection must remain + scoped to its audited owner and assets. Chat voice qualification for 26.1016 + stays deferred; preserve the existing qualified Chat path and fail closed on + unqualified builds. +- **Gateway boundary:** Disabled-account selection and any future automatic + quota retry belong to CPA. This fork's voice override does not implement + account failover. Do not edit CPA core or deploy a gateway as part of a routine + codex-host sync. + +### Re-audit pins and verify the sync + +Run `npm run compat:daily` from the candidate tree and record the report. This +fetches only the public Sparkle appcast, reads the local pins and exits nonzero +on drift; it neither updates pins nor qualifies Desktop behavior. Follow +[the compat-daily audit procedure](../../tools/compat-daily/README.md) to inspect +the exact assets/exports and change pins only with matching qualification +evidence. Keep Work and Chat qualification separate. Deferred Chat may continue +to report drift; record that limitation rather than changing its pin to make +the check green. + +After resolving the sync, select checks for the actual merged diff using the +repository pre-push guidance. For voice changes, build the TypeScript packages +before dependent tests: + +```bash +npm run build:typescript +npm run typecheck +npm run lint +npm run build:renderer +npx vitest run --config tests/vitest.config.js \ + packages/desktop-control/test/production-controller.test.ts \ + packages/shared-contracts/test/realtime-call.test.ts \ + packages/renderer-extension/test/renderer-client-owned-voice.test.ts \ + packages/renderer-extension/test/renderer-external-voice-launch.test.ts \ + packages/renderer-extension/test/renderer-cpa-live-voice-hook.test.ts \ + packages/renderer-extension/test/renderer-cpa-live-voice-preference.test.ts \ + packages/host-runtime/test/native-voice-gateway.test.ts \ + packages/host-runtime/test/external-voice-gateway.test.ts \ + packages/host-runtime/test/external-voice-lifecycle.test.ts \ + packages/host-runtime/test/app-server-host.external-voice.test.ts +npm run test:e2e -- renderer-voice-csp.spec.ts +npm run compat:daily:test +npm run compat:daily +git diff --check +``` + +The browser regression requires an installed Playwright browser. If using an +installed Chrome executable, set `CODEXHOST_PLAYWRIGHT_EXECUTABLE_PATH` for that +test command. Broaden checks for any additional packages or Rust code affected by +the sync; the voice set is not a substitute for those checks. + +With restart and live testing authorized, use `npm start` to rebuild and launch +the sync result. Confirm the running overlay hook is active on audited assets, +call creation reaches the configured gateway, WebRTC connects, and hangup is +confirmed without CSP errors. Then verify native-button startup, audible output +and the selected Harness. A silent transport probe alone does not establish +microphone/conversation acceptance. Record the tested commit, Desktop version, +checks and remaining limitations in the wiki without credentials, SDP or call +identities. + +### Review and publish to the fork + +Record the backup/stash OID, fork/upstream OIDs, candidate commit, Desktop +version/assets, checks and any deferred qualification in the wiki. Review the +sync diff and limitations before integrating it into fork `main`; keep the +recovery snapshot until the intended fork work is accounted for. + +Prepare a local commit for review. Push only with explicit authorization, naming +the remote and branch, for example `git push origin sync/upstream-YYYY-MM-DD`. +Use `-R karlorz/codex-host` for any authorized PR. Never push to `upstream` or +force-push `main`. Integrating into fork `main`, releases and production changes +are separate authorized actions. This runbook does not install an automatic +sync guard or change branch protection. + +## Tag rule + +Fork versions stay on the **upstream version that is currently held**, plus a numeric fork counter. + +| Upstream holds | First fork tag | Later fork tags | +| --- | --- | --- | +| `0.13.2` | `v0.13.2-0` | `v0.13.2-1`, `v0.13.2-2`, `v0.13.2-3` | +| `0.13.3` | `v0.13.3-0` | `v0.13.3-1`, `v0.13.3-2`, … | + +- Read upstream from `upstream/main` `package.json` and the upstream GitHub latest release. If they disagree, stop and report; do not guess. +- First fork of an upstream version is `X.Y.Z-0`. Later fork-only releases while that upstream version is still held are `X.Y.Z-1`, `X.Y.Z-2`, …. The shipped `0.13.2` tags already use that single-counter form. +- Do **not** invent a newer minor or patch than upstream, and do **not** add a named suffix such as `0.13.3-cpa.1`. That tag is legacy and is not a fork release channel. +- `v0.13.3-0` is valid SemVer (prerelease `0` of `0.13.3`). Prepare it with `npm run release:prepare -- 0.13.3-0`. +- Numeric prerelease `0.13.3-0` compares **below** upstream `0.13.3`. The in-app updater and Settings command must therefore **not** read BytePioneer `releases/latest` or install `@codexhost/cli`. They use `karlorz/codex-host` and `@karlorz/codexhost-cli` instead. + +The four prepared files (`package.json`, `package-lock.json`, `Cargo.toml`, `Cargo.lock`) must equal that version. Packaged `app/codexhost-distribution.json` is written later by `release:package` / `release:npm`, not by `release:prepare`. The Web console overview **codexhost 版本** reads that distribution file (installer/npm) or workspace `package.json` (source). After prepare, rebuild or recopy the installed `codexhost-distribution.json` so the live console matches the tag. + +## npm and in-app updates + +Fork operators install and update with: + +```bash +npm install -g @karlorz/codexhost-cli@latest +``` + +That is the Settings → Updates manual command. Settings auto-checks `karlorz/codex-host` `/releases/latest` when Updates is opened. A current version that already matches that latest tag is up to date. A higher fork counter is offered only after that version is published as the fork's non-draft, non-prerelease GitHub latest Release. **Update and restart** on an npm install then runs `@karlorz/codexhost-cli` at that GitHub latest version (via the Updater). Installation requires an explicit user action. + +| Channel | Fork identity | Do not use | +| --- | --- | --- | +| npm meta package | `@karlorz/codexhost-cli` | `@codexhost/cli` | +| npm platform packages | `@karlorz/codexhost-cli-darwin-arm64` (and the other five) | `@codexhost/cli-*` | +| GitHub latest / notes / assets | `karlorz/codex-host` | `BytePioneer-AI/codex-host` | + +A fork tag is not shipped until both of these exist for that version: + +1. **npm:** `Release packages` publishes the packed meta package and matching platform tarballs under `@karlorz` with dist-tag `latest` (or `next` / `test` only for named SemVer prereleases). `scripts/release/publish-npm.mjs` fails closed if a plan entry is not `@karlorz/`. After CI logs `+ @karlorz/…@`, npm may still 404 for a few minutes while provenance is processed; recurl the version URL before republishing. +2. **GitHub:** a **latest** (non-draft, non-prerelease) Release on `karlorz/codex-host` for the same tag. The updater reads `/releases/latest` and rejects prereleases, so a prerelease-only fork Release leaves Updates empty or in error instead of offering the fork build. + +Numeric fork counters such as `0.13.3-0` and `0.13.3-1` are GitHub latest and npm `latest`. Named SemVer prereleases (`-rc.1`, `-beta.1`, `-test.1`, `-cpa.1`) stay on `next` / `test` and GitHub prerelease. `packages/repository-automation/src/release.mjs` treats a hyphen plus one or more digit groups (`0.13.3-0`, `0.13.3-1`) as latest, not as `next`. Do not `npm publish` `@codexhost/cli`. Local `npm run install:local` packs and installs tarballs without the registry. + +Shipped latest on this fork is `v0.13.3-0`: GitHub latest plus `@karlorz/codexhost-cli@0.13.3-0` and all six platform packages on npm `latest`. The next unused counter while upstream holds `0.13.3` is `0.13.3-1`. + +## GitHub Actions secrets + +CI and `Release packages` already use the job `GITHUB_TOKEN` (`contents` / `id-token: write`). Do not add `secrets.NPM_TOKEN` or `NODE_AUTH_TOKEN` to those workflows: npm publish is Trusted Publisher (`id-token: write` + `--provenance`), same as upstream. + +Store these repo secrets on `karlorz/codex-host` for attended and future fork-sync work, matching the other karlorz forks: + +| Secret | Same as | Purpose | +| --- | --- | --- | +| `NPM_TOKEN` | `karlorz/llm-wiki` | Operator/local `npm publish` of `@karlorz/*` when OIDC is not in play. Not read by current CI YAML. | +| `FORK_SYNC_TOKEN` | `karlorz/orca` | Classic PAT with `repo` + `workflow`. Needed later for `merge-upstream` / pushing workflow-file updates; `GITHUB_TOKEN` cannot do that. `karlorz/cliproxyapi` names the same kind of PAT `UPSTREAM_SYNC_PUSH_TOKEN`. | + +```bash +# npmjs auth token from ~/.npmrc, value never printed +python3 -c 'from pathlib import Path +for line in Path.home().joinpath(".npmrc").read_text().splitlines(): + if line.startswith("//registry.npmjs.org/:_authToken="): + print(line.split("=",1)[1], end="") + break +' | gh secret set NPM_TOKEN -R karlorz/codex-host + +# classic PAT (repo + workflow), not a gh oauth gho_ token +gh secret set FORK_SYNC_TOKEN -R karlorz/codex-host +``` + +`Release packages` is enabled on `karlorz/codex-host`. npm Trusted Publisher for `@karlorz/codexhost-cli` and the six platform packages is this GitHub repo (`id-token: write` + `--provenance`). A tag push of a numeric fork counter on `main` after that SHA's CI is green is the ship path. Do not disable the workflow for a normal latest release. + +## Release channel on this fork + +- Never `git push` to `upstream`. +- Never publish `@codexhost/*` from this checkout. +- Do not move, delete, or re-push an existing tag unless the operator explicitly requested that rewrite. +- Create GitHub Releases and pull requests with `-R karlorz/codex-host`. Default `gh` in this checkout resolves to BytePioneer-AI/codex-host. `gh pr create` without `-R` opened the wrong-repo PR #547 and must abort. `gh release create --notes-from-tag` does not work with `--repo`; write the tag annotation to a notes file instead. +- After the lock is on `origin/main`, the next command is `npm run release:prepare -- 0.13.3-0` on `main`. Merge of an identity-lock PR is not a release. + +## Next unused counter + +```bash +git fetch upstream --tags --prune +git fetch origin --tags --prune +# example when upstream holds 0.13.3: +git ls-remote --tags origin 'refs/tags/v0.13.3-*' +``` + +Shipped tags include `v0.13.2-0` through `v0.13.2-3` and `v0.13.3-0`. After upstream `0.13.3`, later fork tags on the same upstream are `0.13.3-1`, `0.13.3-2`, …. Then `npm run release:prepare -- 0.13.3-1` on `main`, push the four version files, wait for that SHA's `main` push CI, then push the annotated tag. diff --git a/docs/platforms/linux/linux.md b/docs/platforms/linux/linux.md index 47f8edbe5..788432ab9 100644 --- a/docs/platforms/linux/linux.md +++ b/docs/platforms/linux/linux.md @@ -3,7 +3,7 @@ codexhost supports x64 and ARM64 Linux through the npm package. Install the official ChatGPT App matching the current architecture first, then install codexhost: ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` diff --git a/docs/platforms/linux/linux.zh-CN.md b/docs/platforms/linux/linux.zh-CN.md index be9b0e37d..80fd73151 100644 --- a/docs/platforms/linux/linux.zh-CN.md +++ b/docs/platforms/linux/linux.zh-CN.md @@ -3,7 +3,7 @@ codexhost 通过 npm 包支持 x64 和 ARM64 Linux。请先安装与当前架构匹配的官方 ChatGPT App,再安装 codexhost: ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` diff --git a/docs/platforms/remote/remote-ssh-host.md b/docs/platforms/remote/remote-ssh-host.md index 6cfdcf724..7937a213e 100644 --- a/docs/platforms/remote/remote-ssh-host.md +++ b/docs/platforms/remote/remote-ssh-host.md @@ -90,7 +90,7 @@ Older remote installations cannot report their version; the page shows **Update ```bash codexhost remote stop -npm install -g @codexhost/cli@latest +npm install -g @karlorz/codexhost-cli@latest codexhost remote install codexhost remote start ``` @@ -139,7 +139,7 @@ This applies to external Harness conversations created in SSH projects. Other lo Choose **Uninstall remote service** on the connection card. After confirmation, auto-connect is disabled for this connection, the remote service stops and its connection setup is removed. On Mac, the associated Harness background service is also uninstalled. The SSH connection entry, project files and chat data are kept, so the service can be installed again later. Active remote sessions are interrupted. -The codexhost package is kept by default. Select **Also uninstall the codexhost package** to remove the remote computer's global npm `@codexhost/cli` after service removal succeeds. Codex CLI, Node.js and other Harnesses are kept. Only select this if the remote Desktop no longer needs that package to launch. A mismatch between the active CLI and npm's global installation rejects package removal before stopping the service; choose service-only removal or uninstall the package manually on the remote computer. If package removal fails after service removal, the page explains the partial result and allows retrying. +The codexhost package is kept by default. Select **Also uninstall the codexhost package** to remove the remote computer's global npm `@karlorz/codexhost-cli` after service removal succeeds. Codex CLI, Node.js and other Harnesses are kept. Only select this if the remote Desktop no longer needs that package to launch. A mismatch between the active CLI and npm's global installation rejects package removal before stopping the service; choose service-only removal or uninstall the package manually on the remote computer. If package removal fails after service removal, the page explains the partial result and allows retrying. When you no longer need the remote installation, run these commands on the remote computer: diff --git a/docs/platforms/remote/remote-ssh-host.zh-CN.md b/docs/platforms/remote/remote-ssh-host.zh-CN.md index e4bb2e6c2..19331ea15 100644 --- a/docs/platforms/remote/remote-ssh-host.zh-CN.md +++ b/docs/platforms/remote/remote-ssh-host.zh-CN.md @@ -90,7 +90,7 @@ Web 控制台的「远程连接」(`#remote-connections`)也提供同一套 ```bash codexhost remote stop -npm install -g @codexhost/cli@latest +npm install -g @karlorz/codexhost-cli@latest codexhost remote install codexhost remote start ``` @@ -139,7 +139,7 @@ npm start 0.12.0 在连接卡片中点击「卸载远程服务」。确认后,会关闭此连接的自动连接、停止远程服务并移除远程接入配置;Mac 上还会卸载配套的 Harness 后台服务。SSH 连接记录、项目文件和会话数据保留,之后可重新安装服务。卸载会中断正在进行的远程会话。 -默认保留 codexhost 软件包。勾选「同时卸载 codexhost 软件包」会在服务卸载成功后,再卸载远程电脑的 npm 全局 `@codexhost/cli`,不会卸载 Codex CLI、Node.js 或其他 Harness。勾选前请确认远程桌面端不再需要通过该软件包启动。安装位置与当前 npm 不匹配时,软件包卸载会在停止服务前被拒绝;可改为仅卸载服务,或在远程电脑上手动卸载软件包。软件包卸载失败时,页面会明确提示远程服务已卸载,并允许重试。 +默认保留 codexhost 软件包。勾选「同时卸载 codexhost 软件包」会在服务卸载成功后,再卸载远程电脑的 npm 全局 `@karlorz/codexhost-cli`,不会卸载 Codex CLI、Node.js 或其他 Harness。勾选前请确认远程桌面端不再需要通过该软件包启动。安装位置与当前 npm 不匹配时,软件包卸载会在停止服务前被拒绝;可改为仅卸载服务,或在远程电脑上手动卸载软件包。软件包卸载失败时,页面会明确提示远程服务已卸载,并允许重试。 不再使用远程端时,在远程电脑的终端执行: diff --git a/docs/product/codex-accounts.md b/docs/product/codex-accounts.md index 2435727a1..cdc7af973 100644 --- a/docs/product/codex-accounts.md +++ b/docs/product/codex-accounts.md @@ -24,7 +24,7 @@ ChatGPT 登录的 Codex 订阅额度耗尽时,Desktop 在 Renderer 中用两道账号级布尔门禁用 Composer 提交:账号额度门和 reserve `hardBlocked`。API Key 登录不经过这两道门。它们是界面上的订阅额度预检,不是协议限制;外部 Harness 的 `turn/start` 由 Host 路由,不会发到官方后端。 -因此在单个 Composer 选中外部 Agent、Adapter 就绪且没有 codexhost 自身的提交阻塞时,`renderer-codex-usage-gate.ts` 只把该 Composer 对这两道门的订阅快照投影为 `false`,继续走原生提交链路。不写账号、atom 或额度查询缓存;旁路生效期间只隐藏该 Composer 内的 Codex 额度横幅,恢复原生状态时还原其显示样式。其他 Composer、Codex 路径和空输入、附件、运行中等其他原生限制不受影响。外部 Harness 的真实额度与错误由其自身处理。切回 Codex、Composer 移除或扩展卸载时恢复实时原生结果。 +因此在单个 Composer 没有 codexhost 自身的提交阻塞时(Codex,或外部 Agent 且 Adapter 就绪),`renderer-codex-usage-gate.ts` 只把该 Composer 对这两道门的订阅快照投影为 `false`,继续走原生提交链路。不写账号、atom 或额度查询缓存。设置 → 外观中的「隐藏 Codex 额度警告」默认开启:所有 Composer 上的官方额度耗尽横幅(Composer 内唯一带标题与 CTA 按钮的 `aside`,含 Reset usage / Upgrade;部分 Desktop 版本另带 `role="status"` 与 `aria-live="polite"`)都隐藏,包括 Codex 会话;关闭该开关后横幅恢复原生显示。空输入、附件、运行中等其他原生限制不受影响。官方 Codex 后端仍可能因额度拒绝该 Turn;外部 Harness 的真实额度与错误由其自身处理。Composer 移除或扩展卸载时恢复实时原生结果与横幅。 门按其 selector 实际读取的字段识别,不依赖压缩名或 hook 序号。兼容直接 `[store, atom]` 订阅及 Desktop 26.928 的 readonly signal adapter / lazy snapshot wrapper:以成对的订阅 effect 和原生快照一致性验证 wrapper,限量重放 readonly 布尔 selector 的依赖以识别间接 reserve 门;拒绝循环依赖、追踪型 render、混合门和快照不匹配。Desktop 26.1002 的 Account 门可读取 reserve `active` 作为豁免,并同时由 submit owner 和编辑器到 owner 之间的子组件订阅;这些订阅按组件分别投影。每次重试从 React 当前已提交树重新发现作用域,包括 alternate 切换后才挂载的子组件;无法确认当前作用域时恢复原生限制。释放时分别恢复 subscriber 与 React instance 原来的 getter,不把 wrapper 替换成另一个函数。无法唯一识别时保留原生限制,并在 Agent 控件悬停提示中说明。升级后的诊断步骤见 [Desktop 更新兼容性诊断手册](../operations/codex-desktop-upgrade-diagnosis-playbook.md#检查-codex-额度门)。 diff --git a/docs/project/README.ko.md b/docs/project/README.ko.md index 7fe4fc7f0..8574e21d6 100644 --- a/docs/project/README.ko.md +++ b/docs/project/README.ko.md @@ -30,7 +30,7 @@


-

다운로드 · Agent 간 협업 · 원격 연결 · 微信交流群 · 简体中文 · English

+

다운로드 · Agent 간 협업 · 원격 연결 · 微信交流群 · 简体中文 · English


@@ -66,13 +66,13 @@ https://github.com/user-attachments/assets/c48192d7-23ff-4f6e-b61a-6345a655bb76 **방법 1: npm** (macOS / Windows / Linux) ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` **방법 2: 설치 프로그램** (macOS / Windows) -[Releases](https://github.com/BytePioneer-AI/codex-host/releases)에서 플랫폼에 맞는 설치 프로그램을 다운로드하세요. +[Releases](https://github.com/karlorz/codex-host/releases)에서 플랫폼에 맞는 설치 프로그램을 다운로드하세요. > Linux는 x64 / ARM64를 지원합니다. 자세한 내용은 [Linux 안내](../platforms/linux/linux.md)를 참고하세요. @@ -162,26 +162,41 @@ xattr -dr com.apple.quarantine /Applications/codexhost.app ## 기능 상태 -모든 Harness에서 Codex Desktop 네이티브 Edit Diff, Fork, 메시지 수정, 슬래시 명령을 사용할 수 있습니다. +각 Harness는 자체 기능에 따라 Codex Desktop 기본 UI와 통합됩니다. 지원 범위는 아래 표를 참조하세요.
전체 기능 매트릭스 보기 -| 기능 | Pi | Oh My Pi | Claude Code | OpenCode | Grok | DeepSeek Harness | AGY | CodeBuddy | WorkBuddy | Cursor | Hermes | Qoder | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| 스트리밍 응답 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 도구 상태 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Edit Diff | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 질문 / 취소 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Model / Thinking 선택 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 도구 승인 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | -| 권한 모드 | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Agent 간 작업 협업 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | -| Usage | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | -| Fork | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 컨텍스트 압축 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | — | ✅ | ✅ | -| 슬래시 명령 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 이전 메시지 수정 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 기능 | Pi | Oh My Pi | Claude Code | OpenCode | Grok | DSH | AGY | CodeBuddy | WorkBuddy | Cursor | Hermes | Qoder | Kimi Code | Copilot | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| 스트리밍 응답 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 도구 상태 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Edit Diff | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 질문 / 취소 | ✅ | ✅ | ✅ | ✅ | ◐ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Model / Thinking 선택 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 도구 승인 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 권한 모드 | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Agent 간 작업 협업 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | +| Usage | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | +| Fork | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ◐ | ✅ | +| 컨텍스트 압축 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | +| 슬래시 명령 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| 이전 메시지 수정 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 실시간 음성 | ? | ? | ✅ | ? | ◐ | ? | ? | ? | ? | ? | ? | ? | ? | ? | +| 다중 선택 질문 | — | ◐ | ◐ | ◐ | — | ◐ | — | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | +| 서브에이전트 | ✅ | ✅ | ✅ | — | ✅ | — | ✅ | ✅ | ✅ | ◐ | — | — | — | — | +| Computer Use / 인앱 브라우저 | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | +| 생성 이미지 | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | + +범례: ✅ 통합 지원 · ◐ 부분 지원, 기능 저하 또는 의향만 저장 · — 현재 플러그인에서 투영하지 않음 · ? 미검증. + +- **실시간 음성**은 이 fork의 Desktop 버전별로 검토된 hook과 설정된 gateway를 사용합니다. Host의 `thread/realtime/*` 경로와 새 채팅의 Harness 선택 상속은 범용입니다. Claude Code에는 기록된 유인 Desktop 통화 검증이 있습니다. Grok은 실제 gateway 통화와 공개 플러그인의 격리된 Host 시험까지 검증되었으며 설치된 Desktop 버튼 검증과는 다르므로 ◐입니다. 나머지 Harness는 미검증(?)입니다. 설명: [CPA live-voice Desktop hook](../operations/cpa-live-voice-desktop-hook.md). +- **다중 선택 질문**에는 대응하는 Desktop 다중 선택 카드가 없습니다. 공유 투영은 `multiple`을 생략하지만 Host 계약은 필드를 유지합니다. 사용자가 보고한 위임 Claude Code 시험에서 `AskUserQuestion`에 `multiSelect: true`와 Alpha/Beta/Gamma를 주면 단일 선택으로 표시되고 Host 오류 없이 문자열 하나(`"Alpha"`)를 반환했습니다. Claude Code, OpenCode, DSH, CodeBuddy/WorkBuddy, Cursor, Hermes, Qoder, Kimi Code, Copilot elicitation 폼은 Adapter가 원시 다중 선택 의도를 보존하지만 충실히 투영하지 못하므로 ◐입니다. 실제 시험은 Claude Code만 수행했습니다. OMP의 ◐는 문서에 설명된 원시 단일 선택/텍스트 상호작용의 다단계 조합을 뜻하며 다중 선택 카드는 아닙니다. Pi는 단일 선택 고정이며 Antigravity는 다중 선택을 명시적으로 거부합니다. 설명: [Antigravity 질문 상호작용 회고](../harnesses/antigravity/antigravity-question-interaction-postmortem.md), [OMP 상호작용](../harnesses/omp/omp-interactions.md). +- **Grok 빌드와 질문**: 이 fork는 패치된 **grokgod Grok build**를 사용합니다. 아래의 패치 의존 동작은 stock Grok의 기능 보장이 아닙니다. `0011-ask-question-timeout-action` 패치는 기존 원시 `ask_user_question`에 설정 가능한 시간 초과 선택과 활동 기반 타이머 재설정을 추가하며 Desktop 카드를 연결하지는 않습니다. 사용자가 보고한 grokgod ACP 시험은 `Failed to reach the client for user question: "Method not found": _x.ai/ask_user_question`으로 실패했습니다. Host ACP 클라이언트는 `_x.ai/mcp/elicit` 확장 요청만 처리하므로 질문 카드가 나타나지 않습니다. 질문 / 취소는 ◐입니다. Turn 취소와 제한된 MCP 동의 승인은 별도로 구현된 경로입니다. 설명: [Harness 기능 경계](../harnesses/capability-boundaries.md). +- **Grok MCP 승인 — Always allow**에는 grokgod의 `0025-mcp-elicitation-persist` 패치가 필요합니다. 원시 persist 제안을 전달하고 선택한 범위를 MCP `_meta.persist`로 반환합니다. 요청이 `always`를 포함할 때만 Desktop이 이 옵션을 제공합니다. 패치하지 않은 Grok에는 Allow once / Deny만 있습니다. 원시 session 수명이 검증되지 않아 Allow this conversation은 제공하지 않습니다. 설명: [Grok MCP elicitation](../harnesses/grok/mcp-elicitation.md). +- **서브에이전트**: ✅ 원시 상태와 읽기 전용 하위 Transcript · ◐ 원시 상태만. Pi에는 지원되는 `pi-subagents` 확장 프로토콜이 필요하며 OMP에는 RPC 서브에이전트 구독 지원이 필요합니다. 셀은 플러그인의 투영을 나타내며 상위 Harness 내부에서 서브에이전트를 실행할 수 있는지와는 별개입니다. +- **Computer Use / 인앱 브라우저**는 사용자가 설치한 Grok `cua_repl` MCP를 사용합니다. 이 fork의 Computer Use 통합에는 패치된 grokgod Grok build의 `0026-cua-turn-end-release`와 설치된 ChatGPT Computer Use runtime이 필요합니다. 패치는 Turn 경계 정리와 Mac Sky 원시 해제 브리지를 포함합니다. Stock Grok의 동등한 동작은 검증되지 않았습니다. IAB는 의향만 저장하며 Desktop 승인과 세션 소유 브라우저 경로가 마련될 때까지 Host가 브라우저 바인딩을 비활성화합니다. 설명: [Grok Computer Use 설정](../harnesses/grok/computer-use-settings.md). +- **생성 이미지**는 일반 이미지 파일 생성이 아니라 Grok Imagine을 Desktop `imageGeneration`으로 투영하는 기능입니다. 현재 이 Item을 내보내는 Adapter는 Grok뿐이며 나머지 셀은 전용 갤러리 투영이 없다는 뜻입니다. 투영은 테스트로 다루지만 실제 갤러리 표시는 미검증입니다. 설명: [Grok 생성 이미지](../harnesses/grok/generated-images.md).
@@ -222,7 +237,7 @@ CodexHost는 대상 Harness를 위한 별도의 Native Session을 만듭니다. 1. 제어 대상 컴퓨터에서 설치하고 시작합니다: ```bash - npm install -g @codexhost/cli + npm install -g @karlorz/codexhost-cli codexhost remote install codexhost remote start codexhost remote status @@ -287,7 +302,7 @@ Issue나 PR을 제출하기 전에 [기여 안내](../../CONTRIBUTING.md)를 읽 환경 요구 사항: 공식 Codex Desktop, Node.js 22.19+ 또는 24, Rust. ```bash -git clone https://github.com/BytePioneer-AI/codex-host +git clone https://github.com/karlorz/codex-host cd codex-host npm ci npm start @@ -306,6 +321,8 @@ Pi를 예로 듭니다. 왼쪽에서 오른쪽이 한 번의 요청 호출 체 주요 작업은 플러그인의 Manifest, 팩토리, Adapter, Session 및 네이티브 통신과 변환 로직을 구현하는 것입니다. 현재 Renderer에는 여전히 정적 연결이 있어, 완전한 Desktop 연동은 별도로 처리해야 합니다. Harness를 추가할 때는 코딩 Agent가 저장소의 [codexhost-add-harness Skill](../../.agents/skills/codexhost-add-harness/SKILL.md)을 사용하도록 할 수 있습니다. 플러그인 구조, 공용 Adapter 인터페이스, 기능 구현과 테스트 요구 사항을 설명합니다. +Desktop Grok Thread는 Grok 공식 ACP `session/new` `_meta.rules`로 CodexHost 세션 규칙을 받습니다. Host Runtime은 이전에 설치한 `~/.grok/rules/codexhost-desktop.md` 중 Host 소유 사본만 회수하고, 사용자가 고친 파일은 남깁니다. 새 Harness 플러그인은 home rule을 추가하지 않습니다. 설명: [operations/cpa-live-voice-desktop-hook.md](../operations/cpa-live-voice-desktop-hook.md). + ## 감사의 글 - 지속적인 지원을 보내 주신 [LINUX DO](https://linux.do/) 커뮤니티에 감사드립니다. @@ -315,8 +332,8 @@ Harness를 추가할 때는 코딩 Agent가 저장소의 [codexhost-add-harness CodexHost에 기여해 주신 모든 분께 감사드립니다. - - CodexHost 기여자 + + CodexHost 기여자 ## Star History diff --git a/docs/project/README.zh-CN.md b/docs/project/README.zh-CN.md index 32b1d0c75..0c8ccb5c6 100644 --- a/docs/project/README.zh-CN.md +++ b/docs/project/README.zh-CN.md @@ -31,7 +31,7 @@


-

下载 · 跨 Agent 协作 · 远程连接 · 微信交流群 · English · 한국어

+

下载 · 跨 Agent 协作 · 远程连接 · 微信交流群 · English · 한국어


@@ -67,13 +67,13 @@ https://github.com/user-attachments/assets/c48192d7-23ff-4f6e-b61a-6345a655bb76 **方式一:npm**(macOS / Windows / Linux) ```bash -npm install -g @codexhost/cli +npm install -g @karlorz/codexhost-cli codexhost ``` **方式二:安装包**(macOS / Windows) -从 [Releases](https://github.com/BytePioneer-AI/codex-host/releases) 下载对应平台的安装包。 +从 [Releases](https://github.com/karlorz/codex-host/releases) 下载对应平台的安装包。 > Linux 支持 x64 / ARM64,详见 [Linux 说明](../platforms/linux/linux.zh-CN.md)。 @@ -171,21 +171,36 @@ xattr -dr com.apple.quarantine /Applications/codexhost.app
查看完整功能矩阵 -| 能力 | Pi | Oh My Pi | Claude Code | OpenCode | Grok | DeepSeek Harness | AGY | CodeBuddy | WorkBuddy | Cursor | Hermes | Qoder | Copilot | -| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| 流式回复 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 工具状态 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Edit Diff | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 提问 / 取消 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Model / Thinking 选择 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 工具审批 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 权限模式 | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Agent 间任务协作 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| Usage | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | -| Fork | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| 上下文压缩 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | — | ✅ | ✅ | ✅ | -| 斜杠命令 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | -| 修订上一条消息 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 能力 | Pi | Oh My Pi | Claude Code | OpenCode | Grok | DSH | AGY | CodeBuddy | WorkBuddy | Cursor | Hermes | Qoder | Kimi Code | Copilot | +| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| 流式回复 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 工具状态 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Edit Diff | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 提问 / 取消 | ✅ | ✅ | ✅ | ✅ | ◐ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Model / Thinking 选择 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 工具审批 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 权限模式 | — | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| Agent 间任务协作 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | ✅ | +| Usage | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | +| Fork | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ◐ | ✅ | +| 上下文压缩 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | ✅ | ✅ | — | ✅ | ✅ | ✅ | ✅ | +| 斜杠命令 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| 修订上一条消息 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| 实时语音 | ? | ? | ✅ | ? | ◐ | ? | ? | ? | ? | ? | ? | ? | ? | ? | +| 多选提问 | — | ◐ | ◐ | ◐ | — | ◐ | — | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | ◐ | +| 子代理 | ✅ | ✅ | ✅ | — | ✅ | — | ✅ | ✅ | ✅ | ◐ | — | — | — | — | +| Computer Use / 内置浏览器 | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | +| 生成图片 | — | — | — | — | ◐ | — | — | — | — | — | — | — | — | — | + +图例:✅ 已接入 · ◐ 部分接入、降级或仅保存意向 · — 当前插件未投影该能力 · ? 未验证。 + +- **实时语音**使用本 fork 按 Desktop 版本审计的 hook 与已配置的 gateway。Host 的 `thread/realtime/*` 和新会话继承所选 Harness 的路径是通用的。Claude Code 有已记录的有人值守 Desktop 通话;Grok 标为 ◐,因为验证覆盖真实 gateway 通话和公共插件的隔离 Host 试验,尚不等于已安装 Desktop 按钮验收。其他 Harness 未验证(?)。详见 [CPA live-voice Desktop hook](../operations/cpa-live-voice-desktop-hook.md)。 +- **多选提问**没有对应的 Desktop 多选卡片。共享投影省略 `multiple`,但 Host 契约保留该字段。用户报告的委派 Claude Code 实测中,`AskUserQuestion` 使用 `multiSelect: true` 与 Alpha/Beta/Gamma 时显示单选,返回单个字符串(`"Alpha"`),Host 没有报错。Claude Code、OpenCode、DSH、CodeBuddy/WorkBuddy、Cursor、Hermes、Qoder、Kimi Code 和 Copilot elicitation 表单标为 ◐:Adapter 保留原生多选意图,但不能忠实投影;只有 Claude Code 做过真机验证。OMP 标为 ◐,表示文档所述的原生多次单选/文本交互组合,并非多选卡片。Pi 固定单选;Antigravity 显式拒绝多选。详见 [Antigravity 提问交互复盘](../harnesses/antigravity/antigravity-question-interaction-postmortem.md) 与 [OMP 交互](../harnesses/omp/omp-interactions.md)。 +- **Grok 构建与提问**:本 fork 使用补丁版 **grokgod Grok build**;下述依赖补丁的行为不能视为 stock Grok 的能力保证。补丁 `0011-ask-question-timeout-action` 为已有原生 `ask_user_question` 增加可配置的超时选择及基于活动的超时重置,不会接通 Desktop 卡片。用户报告的 grokgod ACP 实测返回 `Failed to reach the client for user question: "Method not found": _x.ai/ask_user_question`。Host ACP 客户端只处理 `_x.ai/mcp/elicit` 扩展请求,因此不会出现提问卡片。提问 / 取消仍标为 ◐:回合取消和受限 MCP 同意审批走独立的已实现路径。详见 [Harness 能力表边界](../harnesses/capability-boundaries.md)。 +- **Grok MCP 审批 — Always allow**依赖 grokgod 补丁 `0025-mcp-elicitation-persist`,转发原生 persist 选项,并将选中的范围作为 MCP `_meta.persist` 回传。只有请求包含 `always` 时 Desktop 才提供该选项;未打补丁的 Grok 只有 Allow once / Deny。原生 session 生命周期尚未核实,因此不提供 Allow this conversation。详见 [Grok MCP elicitation](../harnesses/grok/mcp-elicitation.md)。 +- **子代理**:✅ 原生状态加只读子 Transcript · ◐ 只有原生状态。Pi 需要受支持的 `pi-subagents` 扩展协议;OMP 需要 RPC 子代理订阅支持。这些单元格描述插件投影,不表示上游 Harness 内部能否运行子代理。 +- **Computer Use / 内置浏览器**使用用户安装的 Grok `cua_repl` MCP。本 fork 的 Computer Use 接入依赖补丁版 grokgod Grok build 的 `0026-cua-turn-end-release`,包括回合边界清理和 Mac Sky 原生释放桥,以及已安装的 ChatGPT Computer Use runtime;未证明 stock Grok 具备同等行为。IAB 仅保存意向:Host 保持浏览器绑定关闭,等待 Desktop 授权和会话所属的浏览器路由。详见 [Grok Computer Use 设置](../harnesses/grok/computer-use-settings.md)。 +- **生成图片**描述 Grok Imagine 到 Desktop `imageGeneration` 的投影,不是一般的图片文件生成。Grok 是当前唯一发出此 Item 的 Adapter;其他单元格表示没有专门的画廊投影。投影有测试覆盖,但真实画廊展示仍未验证。详见 [Grok 生成图片](../harnesses/grok/generated-images.md)。
@@ -226,7 +241,7 @@ CodexHost 会为目标 Harness 创建独立的 Native Session。委派会话将 1. 在被控机器上安装并启动: ```bash - npm install -g @codexhost/cli + npm install -g @karlorz/codexhost-cli codexhost remote install codexhost remote start codexhost remote status @@ -291,7 +306,7 @@ CodexHost 的做法不同: 环境要求:官方 Codex Desktop、Node.js 22.19+ 或 24、Rust。 ```bash -git clone https://github.com/BytePioneer-AI/codex-host +git clone https://github.com/karlorz/codex-host cd codex-host npm ci npm start @@ -313,6 +328,8 @@ npm start 主要实现插件的 Manifest、工厂、Adapter、Session 及原生通信与转换逻辑。当前 Renderer 仍有静态接线,完整 Desktop 接入还需单独处理。 新增 Harness 时,可以让编码 Agent 使用仓库内的 [codexhost-add-harness Skill](../../.agents/skills/codexhost-add-harness/SKILL.md)。它说明了插件结构、公共 Adapter 接口、能力实现与测试要求。 +Desktop Grok Thread 通过 Grok 官方 ACP `session/new` 的 `_meta.rules` 接收 CodexHost 会话规则。Host Runtime 会回收仍匹配历史摘要的 `~/.grok/rules/codexhost-desktop.md`,并保留用户改过的副本。新 Harness 插件不要再写 home rule。说明见 [operations/cpa-live-voice-desktop-hook.md](../operations/cpa-live-voice-desktop-hook.md)。 + ## 鸣谢 - 感谢 [LINUX DO](https://linux.do/) 社区一直以来的支持。 @@ -322,8 +339,8 @@ npm start 感谢所有为 CodexHost 做出贡献的开发者。 - - CodexHost 贡献者 + + CodexHost 贡献者 ## Star History diff --git a/openspec/specs/github-release-background-update/spec.md b/openspec/specs/github-release-background-update/spec.md index 480becd89..00f651054 100644 --- a/openspec/specs/github-release-background-update/spec.md +++ b/openspec/specs/github-release-background-update/spec.md @@ -4,7 +4,7 @@ TBD - created by archiving change implement-github-release-update-flow. Update Purpose after archive. ## Requirements ### Requirement: Host discovers one stable GitHub Release -The update capability SHALL discover the latest stable codexhost version only from `https://api.github.com/repos/BytePioneer-AI/codex-host/releases/latest`, SHALL compare its `v`-prefixed tag with the packaged current SemVer, and SHALL keep check failure non-blocking for normal application launch. It MUST NOT require a custom manifest, checksum sidecar, codexhost server, login, or embedded GitHub credential. +The update capability SHALL discover the latest stable codexhost version only from `https://api.github.com/repos/karlorz/codex-host/releases/latest`, SHALL compare its `v`-prefixed tag with the packaged current SemVer, and SHALL keep check failure non-blocking for normal application launch. It MUST NOT require a custom manifest, checksum sidecar, codexhost server, login, or embedded GitHub credential. #### Scenario: New stable Release exists - **WHEN** GitHub returns a non-draft, non-prerelease Release with a valid newer `v` tag diff --git a/openspec/specs/running-desktop-attachment/spec.md b/openspec/specs/running-desktop-attachment/spec.md index edfb97e3b..1874ba900 100644 --- a/openspec/specs/running-desktop-attachment/spec.md +++ b/openspec/specs/running-desktop-attachment/spec.md @@ -10,6 +10,11 @@ The production Launcher SHALL distinguish stale-launcher recovery, clean Desktop - **WHEN** launcher state exists but its Desktop and control endpoint are both absent - **THEN** Launcher MUST remove only the validated stale state and retry startup +#### Scenario: Leftover Controller after a failed launch +- **WHEN** exclusive Launcher ownership is acquired, no target Desktop is running, and the recorded control endpoint is still listening +- **THEN** Launcher MUST stop only a Controller whose command line matches the recorded attachment port and nonce, remove the matching descriptor, and continue with clean launch +- **AND** it MUST NOT terminate an unrelated listener on that port + #### Scenario: No Desktop is running - **WHEN** no target Codex Desktop process exists - **THEN** Launcher MUST use the existing clean launch with Shim, Host configuration, temporary loopback Chromium Renderer CDP, Renderer, and Controller supervision diff --git a/package-lock.json b/package-lock.json index 701c49ae7..d65de0b92 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "codexhost", - "version": "0.14.0", + "version": "0.14.0-3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "codexhost", - "version": "0.14.0", + "version": "0.14.0-3", "license": "LGPL-3.0-only", "workspaces": [ "packages/*", @@ -6389,7 +6389,8 @@ "@codexhost/shared-contracts": "0.0.0", "@codexhost/update-manager": "0.0.0", "@deepseek-ai/cordis": "4.0.4", - "ws": "8.21.3" + "ws": "8.21.3", + "smol-toml": "^1.8.0" } }, "packages/mapping-store": { diff --git a/package.json b/package.json index 3322fceed..d8b46cffd 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "codexhost", - "version": "0.14.0", + "version": "0.14.0-3", "private": true, "license": "LGPL-3.0-only", "type": "module", @@ -42,6 +42,7 @@ "format": "prettier --write . && cargo fmt --all", "format:check": "prettier --check . && cargo fmt --all --check", "test:typescript": "npm run build:typescript && vitest run --config tests/vitest.config.js", + "test:voice:smoke": "npm run build:typescript && vitest run --config tests/vitest.config.js packages/host-runtime/test/external-voice-results.test.ts packages/host-runtime/test/external-voice-lifecycle.test.ts packages/host-runtime/test/external-voice-session-input.test.ts packages/host-runtime/test/external-voice-gateway.test.ts packages/host-runtime/test/native-voice-gateway.test.ts packages/host-runtime/test/app-server-host.external-voice.test.ts packages/host-runtime/test/app-server-host.voice-text-links.test.ts packages/adapters/claude-code/test/native-message.test.ts packages/renderer-extension/test/renderer-overlay-thread-isolation.test.ts packages/renderer-extension/test/renderer-external-voice-launch.test.ts packages/renderer-extension/test/renderer-cpa-live-voice-hook.test.ts packages/renderer-extension/test/renderer-cpa-live-voice-preference.test.ts packages/renderer-extension/test/renderer-client-owned-voice.test.ts packages/renderer-extension/test/renderer-native-chat-voice.test.ts packages/shared-contracts/test/realtime-call.test.ts packages/desktop-control/test/cpa-live-voice-hook.test.ts packages/desktop-control/test/production-controller.test.ts", "test:deepseek:coverage": "npm run build:typescript && vitest run --config tests/vitest.deepseek-coverage.config.js", "test:rust": "cargo test --workspace --locked --features codexhost-shim/test-utils,codexhost-gate-a-native/gate-tools", "test": "npm run test:typescript && npm run test:rust", @@ -74,6 +75,8 @@ "gate:claude": "node tools/gate-claude-code/run.mjs gate", "probe:renderer-binding": "npm run build:typescript && npm run build:renderer && node tools/renderer-binding/run.mjs", "audit:codex-desktop": "npm run build:typescript && npm run build:renderer && node tools/codex-desktop-contract-audit/run.mjs", + "compat:daily": "node tools/compat-daily/run.mjs", + "compat:daily:test": "node --test tools/compat-daily/run.test.mjs", "check:rust": "cargo fmt --all --check && cargo clippy --workspace --all-targets --locked --features codexhost-shim/test-utils,codexhost-gate-a-native/gate-tools -- -D warnings && cargo test --workspace --locked --features codexhost-shim/test-utils,codexhost-gate-a-native/gate-tools", "check": "npm run format:check && npm run lint && npm run typecheck && npm run test:typescript && npm run check:rust", "clean": "node tools/clean.mjs" diff --git a/packages/adapters/claude-code/src/installation.ts b/packages/adapters/claude-code/src/installation.ts index 673a5a9cf..fe3c9651e 100644 --- a/packages/adapters/claude-code/src/installation.ts +++ b/packages/adapters/claude-code/src/installation.ts @@ -7,6 +7,7 @@ import { installationVersion, newerInstallationVersion, npmInstallation, + readLauncherScript, runInstallationCommand, versionFromOutput, } from "@codexhost/harness-discovery"; @@ -38,6 +39,27 @@ export function createClaudeInstallation(environment: NodeJS.ProcessEnv, command environment, ...(command ? { command } : {}), }); + const launcherScript = await readLauncherScript(executable); + if (launcherScript.includes("CLAWGOD_CLI=")) { + return { + currentVersion: versionFromOutput( + await runInstallationCommand(executable, ["--version"], { + ...environment, + DISABLE_AUTOUPDATER: "1", + }), + ), + latestVersion: "Unknown", + latestVersionKind: "unknown" as const, + updateAvailable: false, + canUpdate: false, + messageCode: "original-installer", + notice: { + en: "Managed by clawgod. Use clawgod's updater or installer to preserve the installed patches. The latest patched version is not checked here.", + "zh-CN": + "此安装由 clawgod 管理。请使用 clawgod 的更新程序或安装程序保留补丁;此处不检查最新补丁版本。", + }, + }; + } const home = environment.HOME ?? environment.USERPROFILE ?? homedir(); const config = environment.CLAUDE_CONFIG_DIR ? path.resolve(home, environment.CLAUDE_CONFIG_DIR) diff --git a/packages/adapters/claude-code/src/native-message.ts b/packages/adapters/claude-code/src/native-message.ts index 048fcb856..15c200a67 100644 --- a/packages/adapters/claude-code/src/native-message.ts +++ b/packages/adapters/claude-code/src/native-message.ts @@ -119,6 +119,12 @@ function subagentBackground(argumentsValue: unknown): boolean { return isRecord(argumentsValue) && argumentsValue.run_in_background === true; } +function subagentModel(argumentsValue: unknown): string | undefined { + if (!isRecord(argumentsValue)) return undefined; + const model = boundedString(argumentsValue.model, SUBAGENT_DESCRIPTION_LIMIT); + return model === "inherit" ? undefined : model; +} + function targetedSubagentId(argumentsValue: unknown): string | undefined { if (!isRecord(argumentsValue)) return undefined; return boundedString( @@ -328,10 +334,15 @@ function subagentContinuesInBackground( outputText: string | undefined, ): boolean { if (isRecord(nativeResult)) { + // Native lifecycle fields take precedence over the agent's report text. + if (typeof nativeResult.status === "string") return nativeResult.status === "async_launched"; if (nativeResult.isAsync === true || nativeResult.is_async === true) return true; - if (nativeResult.status === "async_launched") return true; + if (nativeResult.isAsync === false || nativeResult.is_async === false) return false; } - return outputText?.includes("The agent is working in the background.") ?? false; + // Text-only launch receipts have a fixed prefix; quoted report text is not a receipt. + return /^Async agent launched successfully\.\r?\nagentId: [A-Za-z0-9_-]+\r?\nThe agent is working in the background\.(?:\r?\n|$)/u.test( + outputText ?? "", + ); } function resultText(content: unknown, nativeResult: unknown): string | undefined { @@ -506,6 +517,10 @@ export class ClaudeNativeTurnAccumulator { const parentTool = parentCallId ? this.#tools.get(parentCallId) : undefined; if (parentCallId && parentTool && (message.type === "assistant" || message.type === "user")) { if (parentTool.subagent) { + const model = message.type === "assistant" ? subagentModel(message.message) : undefined; + if (model && model !== "") { + events.push({ type: "subagent.updated", callId: parentCallId, status: "running", model }); + } events.push({ type: "subagent.transcript.changed", callId: parentCallId }); } else if (parentTool.workflow) { events.push({ type: "workflow.activity", callId: parentCallId }); @@ -909,6 +924,8 @@ export class ClaudeNativeTurnAccumulator { } else if (subagent) { const prompt = subagentPrompt(argumentsResult.data); const role = subagentRole(argumentsResult.data); + const model = + block.name === "SendMessage" ? undefined : subagentModel(argumentsResult.data); const agentId = targetedSubagentId(argumentsResult.data); events.push({ type: "subagent.started", @@ -917,6 +934,7 @@ export class ClaudeNativeTurnAccumulator { description: subagentDescription(argumentsResult.data, block.name), ...(prompt ? { prompt } : {}), ...(role ? { role } : {}), + ...(model ? { model } : {}), background: block.name === "SendMessage" || subagentBackground(argumentsResult.data), ...(agentId ? { nativeSubagentId: agentId } : {}), }); diff --git a/packages/adapters/claude-code/src/subagent-lifecycle.ts b/packages/adapters/claude-code/src/subagent-lifecycle.ts index 0753d667d..efaff3549 100644 --- a/packages/adapters/claude-code/src/subagent-lifecycle.ts +++ b/packages/adapters/claude-code/src/subagent-lifecycle.ts @@ -53,6 +53,7 @@ export class ClaudeSubagentLifecycle { ...(event.nativeSubagentId ? { nativeSubagentId: event.nativeSubagentId } : {}), description: event.description, ...(event.role ? { role: event.role } : {}), + ...(event.model ? { model: event.model } : {}), background: event.background, status: "running", }; @@ -78,6 +79,7 @@ export class ClaudeSubagentLifecycle { ...(event.nativeSubagentId ? { nativeSubagentId: event.nativeSubagentId } : {}), ...(event.description ? { description: event.description } : {}), ...(event.role ? { role: event.role } : {}), + ...(event.model ? { model: event.model } : {}), ...(event.resultSummary ? { resultSummary: event.resultSummary } : {}), }; active.item = { ...active.item, subagents: [subagent] }; diff --git a/packages/adapters/claude-code/src/transport.ts b/packages/adapters/claude-code/src/transport.ts index 921a30e90..fabc1b390 100644 --- a/packages/adapters/claude-code/src/transport.ts +++ b/packages/adapters/claude-code/src/transport.ts @@ -128,6 +128,7 @@ export type ClaudeTurnEvent = description: string; prompt?: string; role?: string; + model?: string; background: boolean; nativeSubagentId?: string; } @@ -137,6 +138,7 @@ export type ClaudeTurnEvent = status: "pending" | "running" | "completed" | "failed" | "interrupted"; description?: string; role?: string; + model?: string; nativeSubagentId?: string; resultSummary?: string; } diff --git a/packages/adapters/claude-code/test/claude-code-adapter.test.ts b/packages/adapters/claude-code/test/claude-code-adapter.test.ts index 6365e4302..583622363 100644 --- a/packages/adapters/claude-code/test/claude-code-adapter.test.ts +++ b/packages/adapters/claude-code/test/claude-code-adapter.test.ts @@ -2823,8 +2823,10 @@ describe("Claude Code HarnessAdapter", () => { await session.close(); }); - it("projects Claude Agent delegation as one common Subagent Item", async () => { + it("preserves requested and reported child models through Claude Agent lifecycle", async () => { + const model = "haiku"; const { adapter, transports } = fixture(); + const reportedModel = "claude-sonnet-4-6"; const session = await openSession(adapter); const iterator = session.outputs[Symbol.asyncIterator](); @@ -2843,6 +2845,7 @@ describe("Claude Code HarnessAdapter", () => { callId: "agent-1", description: "Inspect implementation", role: "Explore", + model, background: true, }); expect(await nextEvent(iterator)).toMatchObject({ @@ -2859,6 +2862,7 @@ describe("Claude Code HarnessAdapter", () => { subagentId: "agent-1", description: "Inspect implementation", role: "Explore", + model, background: true, status: "running", }, @@ -2872,6 +2876,7 @@ describe("Claude Code HarnessAdapter", () => { status: "running", nativeSubagentId: "native-agent-1", resultSummary: "Reading files", + model: reportedModel, }); expect(await nextEvent(iterator)).toMatchObject({ type: "item.updated", @@ -2882,6 +2887,7 @@ describe("Claude Code HarnessAdapter", () => { status: "running", nativeSubagentId: "native-agent-1", resultSummary: "Reading files", + model: reportedModel, }, ], }, @@ -2905,13 +2911,22 @@ describe("Claude Code HarnessAdapter", () => { type: "item.updated", update: { type: "subagents.replace", - subagents: [{ status: "completed", resultSummary: "Agent launched successfully" }], + subagents: [ + { + status: "completed", + resultSummary: "Agent launched successfully", + model: reportedModel, + }, + ], }, }); expect(await nextEvent(iterator)).toMatchObject({ type: "item.completed", snapshot: { - item: { type: "subagentDelegation", subagents: [{ status: "completed" }] }, + item: { + type: "subagentDelegation", + subagents: [{ status: "completed", model: reportedModel }], + }, outcome: { status: "succeeded" }, }, }); diff --git a/packages/adapters/claude-code/test/command.test.ts b/packages/adapters/claude-code/test/command.test.ts index 1d55dc9d2..af1c3e4f6 100644 --- a/packages/adapters/claude-code/test/command.test.ts +++ b/packages/adapters/claude-code/test/command.test.ts @@ -147,11 +147,14 @@ describe("Claude Code executable resolution", () => { const homeDirectory = fs.mkdtempSync(path.join(os.tmpdir(), "codexhost-claude-home-")); directories.push(homeDirectory); expect(() => - resolveClaudeCodeExecutable({ - environment: { PATH: "" }, - homeDirectory, - platform: "linux", - }), + resolveClaudeCodeExecutable( + { + environment: { PATH: "" }, + homeDirectory, + platform: "linux", + }, + { isExecutable: () => false }, + ), ).toThrow("not installed"); }); }); diff --git a/packages/adapters/claude-code/test/installation.test.ts b/packages/adapters/claude-code/test/installation.test.ts index c5af23afc..92e52998f 100644 --- a/packages/adapters/claude-code/test/installation.test.ts +++ b/packages/adapters/claude-code/test/installation.test.ts @@ -7,12 +7,14 @@ const mocks = vi.hoisted(() => ({ resolve: vi.fn(), realpath: vi.fn(), read: vi.fn(), + launcher: vi.fn(), })); vi.mock("@codexhost/harness-discovery", async (original) => ({ ...(await original()), runInstallationCommand: mocks.run, fetchInstallationText: mocks.fetch, npmInstallation: mocks.npm, + readLauncherScript: mocks.launcher, })); vi.mock("node:fs/promises", async (original) => ({ ...(await original()), @@ -32,6 +34,7 @@ beforeEach(() => { path.join(environment.HOME, ".local", "share", "claude", "versions", "2.1.0"), ); mocks.read.mockResolvedValue("{}"); + mocks.launcher.mockResolvedValue(""); mocks.npm.mockResolvedValue(null); mocks.fetch.mockResolvedValue("2.2.0"); mocks.run.mockResolvedValue("2.1.0 (Claude Code)"); @@ -39,6 +42,28 @@ beforeEach(() => { afterEach(() => vi.unstubAllGlobals()); describe("Claude installation", () => { + it.each(["check", "update"] as const)( + "keeps clawgod %s read-only without comparing official releases", + async (action) => { + mocks.launcher.mockResolvedValue('#!/bin/bash\nCLAWGOD_CLI="/custom/.clawgod/cli.cjs"'); + const result = await createClaudeInstallation(environment)(action); + expect(result).toMatchObject({ + currentVersion: "2.1.0", + latestVersion: "Unknown", + latestVersionKind: "unknown", + updateAvailable: false, + canUpdate: false, + notice: { en: expect.stringContaining("Managed by clawgod") }, + }); + expect(mocks.run).toHaveBeenCalledExactlyOnceWith(executable, ["--version"], { + ...environment, + DISABLE_AUTOUPDATER: "1", + }); + expect(mocks.fetch).not.toHaveBeenCalled(); + expect(mocks.npm).not.toHaveBeenCalled(); + }, + ); + it("reads the configured stable channel and uses the native policy-aware updater", async () => { mocks.read.mockImplementation(async (file: string) => file === path.join(environment.HOME, ".claude", "settings.json") diff --git a/packages/adapters/claude-code/test/native-message.test.ts b/packages/adapters/claude-code/test/native-message.test.ts index 947b522fa..685b79154 100644 --- a/packages/adapters/claude-code/test/native-message.test.ts +++ b/packages/adapters/claude-code/test/native-message.test.ts @@ -280,7 +280,15 @@ describe("Claude native Turn interpretation", () => { "agent-call", ), ).events, - ).toEqual([{ type: "subagent.transcript.changed", callId: "agent-call" }]); + ).toEqual([ + { + type: "subagent.updated", + callId: "agent-call", + status: "running", + model: "claude-sonnet-4-6", + }, + { type: "subagent.transcript.changed", callId: "agent-call" }, + ]); expect( turn.consume({ type: "assistant", @@ -313,6 +321,58 @@ describe("Claude native Turn interpretation", () => { expect(turn.consume(result()).terminal).toEqual({ status: "succeeded" }); }); + it("uses child-specific response models without exposing nested response text", () => { + const turn = new ClaudeNativeTurnAccumulator(); + turn.consume(toolUse("Agent", "agent-model", { model: "haiku" })); + const child = (model: string, parent = "agent-model") => ({ + type: "assistant", + parent_tool_use_id: parent, + message: { model, content: [{ type: "text", text: "private child response" }] }, + }); + expect(turn.consume(child("claude-sonnet-4-6")).events).toEqual([ + { + type: "subagent.updated", + callId: "agent-model", + status: "running", + model: "claude-sonnet-4-6", + }, + { type: "subagent.transcript.changed", callId: "agent-model" }, + ]); + expect(turn.consume(child("")).events).toEqual([ + { type: "subagent.transcript.changed", callId: "agent-model" }, + ]); + expect(turn.consume(child("claude-opus-4-6", "untracked-agent")).events).toEqual([]); + }); + + it.each(["Agent", "Task"])("preserves explicit %s model aliases", (name) => { + const turn = new ClaudeNativeTurnAccumulator(); + expect( + turn.consume(toolUse(name, "agent-model", { model: " sonnet " })).events[0], + ).toMatchObject({ + type: "subagent.started", + model: "sonnet", + }); + }); + + it.each([undefined, null, "", " ", "inherit", 42])( + "leaves unknown or inherited Agent model %j unspecified", + (model) => { + const turn = new ClaudeNativeTurnAccumulator(); + const event = turn.consume( + toolUse("Agent", "agent-model", model === undefined ? {} : { model }), + ).events[0]; + expect(event).toMatchObject({ type: "subagent.started" }); + expect(event).not.toHaveProperty("model"); + }, + ); + + it("does not interpret a SendMessage argument as child configuration", () => { + const turn = new ClaudeNativeTurnAccumulator(); + const event = turn.consume(toolUse("SendMessage", "send-model", { model: "haiku" })).events[0]; + expect(event).toMatchObject({ type: "subagent.started", operation: "send" }); + expect(event).not.toHaveProperty("model"); + }); + it("maps Root Agent delegation without exposing nested Tool events", () => { const turn = new ClaudeNativeTurnAccumulator(); @@ -323,6 +383,7 @@ describe("Claude native Turn interpretation", () => { subagent_type: "Explore", run_in_background: true, prompt: "private prompt", + model: "haiku", }), ).events, ).toEqual([ @@ -333,6 +394,7 @@ describe("Claude native Turn interpretation", () => { description: "Inspect implementation", prompt: "private prompt", role: "Explore", + model: "haiku", background: true, }, { @@ -379,6 +441,59 @@ describe("Claude native Turn interpretation", () => { expect(turn.consume(result()).terminal).toEqual({ status: "succeeded" }); }); + it.each([ + { status: "completed", agentId: "native-agent-1" }, + { isAsync: false, agentId: "native-agent-1" }, + undefined, + ])( + "does not interpret a completed audit's quoted background text as live status (%j)", + (nativeResult) => { + const turn = new ClaudeNativeTurnAccumulator(); + turn.consume( + toolUse("Agent", "agent-1", { + description: "Audit subagent lifecycle", + run_in_background: false, + }), + ); + const content = + 'The parser checks whether output contains "The agent is working in the background.". This is a source-code finding; the audit is complete.'; + const completed = turn.consume(toolResult("agent-1", { content, nativeResult })); + expect(completed.events).toEqual([ + { + type: "subagent.completed", + callId: "agent-1", + isError: false, + ...(nativeResult ? { nativeSubagentId: "native-agent-1" } : {}), + resultSummary: content, + }, + ]); + expect(turn.consume(result()).terminal).toEqual({ status: "succeeded" }); + }, + ); + + it("prefers structured completed status over a background-looking result body", () => { + const turn = new ClaudeNativeTurnAccumulator(); + turn.consume(toolUse("Agent", "agent-1")); + const content = + "Async agent launched successfully.\nagentId: native-agent-1\nThe agent is working in the background."; + expect( + turn.consume( + toolResult("agent-1", { + content, + nativeResult: { status: "completed", agentId: "native-agent-1" }, + }), + ).events, + ).toEqual([ + { + type: "subagent.completed", + callId: "agent-1", + isError: false, + nativeSubagentId: "native-agent-1", + resultSummary: content, + }, + ]); + }); + it("keeps an async Agent spawn running when its launch Tool Result returns", () => { const turn = new ClaudeNativeTurnAccumulator(); diff --git a/packages/adapters/grok/package.json b/packages/adapters/grok/package.json index 16f1f2013..8d44d3155 100644 --- a/packages/adapters/grok/package.json +++ b/packages/adapters/grok/package.json @@ -25,6 +25,7 @@ "@agentclientprotocol/sdk": "1.3.0", "@codexhost/harness-adapter": "0.0.0", "@codexhost/harness-discovery": "0.0.0", + "@codexhost/harness-plugin-files": "0.0.0", "@codexhost/shared-contracts": "0.0.0", "diff": "8.0.2" } diff --git a/packages/adapters/grok/src/acp-transport.ts b/packages/adapters/grok/src/acp-transport.ts index b03bb5d9b..5d7c0badb 100644 --- a/packages/adapters/grok/src/acp-transport.ts +++ b/packages/adapters/grok/src/acp-transport.ts @@ -48,6 +48,15 @@ import { parseGrokRewindResponse, type GrokRewindParams, } from "./grok-rewind.js"; +import { + GROK_MCP_ELICIT_METHOD, + grokElicitationResolvedToolCallId, + grokMcpElicitCancel, + parseGrokMcpElicitParams, + type GrokMcpElicitFormRequest, + type GrokMcpElicitResponse, +} from "./grok-elicitation.js"; +import { grokCodexhostSessionMeta } from "./grok-codexhost-session-rules.js"; import { decodeGrokPermissionModeId, grokPermissionModeSessionMeta } from "./permission-modes.js"; import { GROK_PERMISSION_MODE_METHOD, @@ -145,7 +154,8 @@ export type GrokTransportEvent = status: "completed" | "failed" | "interrupted"; resultSummary?: string; metadata?: Record; - }; + } + | { type: "elicitation.resolved"; toolCallId: string; metadata?: Record }; export interface GrokPermissionRequest { request: RequestPermissionRequest; @@ -197,6 +207,7 @@ export interface GrokOpenResult { interface ActivePrompt { onEvent(event: GrokTransportEvent): void; onPermission(request: GrokPermissionRequest): Promise; + onElicitation(request: GrokMcpElicitFormRequest): Promise; } interface ActiveCompact { @@ -340,6 +351,14 @@ function transportEvent( if (compaction) return compaction; const subagent = grokSubagentEventFromUpdate(extension); if (subagent) return subagent; + const resolvedToolCallId = grokElicitationResolvedToolCallId(extension); + if (resolvedToolCallId) { + return { + type: "elicitation.resolved", + toolCallId: resolvedToolCallId, + ...(metadata ? { metadata } : {}), + }; + } switch (update.sessionUpdate) { case "user_message_chunk": case "agent_message_chunk": @@ -649,7 +668,7 @@ export class GrokAcpTransport { connection.newSession({ cwd: this.#options.cwd, mcpServers: [], - _meta: grokPermissionModeSessionMeta(permissionMode), + _meta: grokCodexhostSessionMeta(permissionMode), }), this.#options.commandTimeoutMs, "Grok Session creation", @@ -671,6 +690,7 @@ export class GrokAcpTransport { cwd: this.#options.cwd, mcpServers: [], sessionId: forked.newSessionId, + _meta: grokCodexhostSessionMeta(), }), this.#options.commandTimeoutMs, "Grok Session load", @@ -691,7 +711,7 @@ export class GrokAcpTransport { cwd: this.#options.cwd, mcpServers: [], sessionId: input.sessionId, - ...(permissionMode ? { _meta: grokPermissionModeSessionMeta(permissionMode) } : {}), + _meta: grokCodexhostSessionMeta(permissionMode), }), this.#options.commandTimeoutMs, "Grok Session load", @@ -839,6 +859,7 @@ export class GrokAcpTransport { ({ sessionUpdate: (params) => this.#handleUpdate(params), requestPermission: (params) => this.#handlePermission(params), + extMethod: (method, params) => this.#handleExtMethod(method, params), extNotification: (method, params) => this.#handleExtensionNotification(method, params), }) satisfies Client, stream, @@ -880,6 +901,7 @@ export class GrokAcpTransport { text: string, onEvent: ActivePrompt["onEvent"], onPermission: ActivePrompt["onPermission"], + onElicitation: ActivePrompt["onElicitation"], ): Promise { const connection = this.#connection; if (!connection || !this.#sessionId || this.#closed || this.#closing) { @@ -888,7 +910,7 @@ export class GrokAcpTransport { if (this.#activePrompt) throw new Error("Grok ACP Session already has an active Prompt"); if (this.#permissionConfirmation) throw new GrokTransportError("unavailable", "Grok ACP Session has an active operation"); - const active = { onEvent, onPermission }; + const active = { onEvent, onPermission, onElicitation }; this.#activePrompt = active; try { return await connection.prompt({ @@ -956,9 +978,15 @@ export class GrokAcpTransport { { sessionId: this.#sessionId, modelId, - ...(reasoningEffort ? { reasoningEffort } : {}), - ...(contextWindowTokens !== undefined - ? { _meta: { contextWindow: contextWindowTokens } } + ...(reasoningEffort || contextWindowTokens !== undefined + ? { + _meta: { + ...(reasoningEffort ? { reasoningEffort } : {}), + ...(contextWindowTokens !== undefined + ? { contextWindow: contextWindowTokens } + : {}), + }, + } : {}), }, ); @@ -1104,6 +1132,33 @@ export class GrokAcpTransport { return this.#activePrompt.onPermission({ request: params, options: params.options }); } + async #handleExtMethod( + method: string, + params: Record, + ): Promise> { + if (method !== GROK_MCP_ELICIT_METHOD) { + throw RequestError.methodNotFound(method); + } + const parsed = parseGrokMcpElicitParams(params); + if (parsed.status === "invalid") { + throw RequestError.invalidParams(undefined, parsed.message); + } + if ( + parsed.status === "unsupported" || + this.#closed || + this.#closing || + this.#replay !== null || + this.#activeCompact !== null || + !this.#activePrompt + ) { + return { ...grokMcpElicitCancel() }; + } + if (parsed.request.sessionId !== this.#sessionId) { + return { ...grokMcpElicitCancel() }; + } + return { ...(await this.#activePrompt.onElicitation(parsed.request)) }; + } + #fault(error: GrokTransportError): void { this.#permissionConfirmation?.reject(error); if (this.#closing || this.#closed) return; diff --git a/packages/adapters/grok/src/grok-adapter.ts b/packages/adapters/grok/src/grok-adapter.ts index b33619a62..25ccdf189 100644 --- a/packages/adapters/grok/src/grok-adapter.ts +++ b/packages/adapters/grok/src/grok-adapter.ts @@ -1,7 +1,9 @@ import { createGrokUsageStatistics } from "./usage-statistics.js"; import { randomUUID } from "node:crypto"; +import { grokTextItemId } from "./grok-text-item-id.js"; import { readGrokCredentials } from "./grok-credential-export.js"; import path from "node:path"; +import { applyGrokComputerUsePreference } from "@codexhost/harness-plugin-files"; import type { PermissionOption, @@ -88,6 +90,13 @@ import { type GrokPermissionRequest, type GrokTransportEvent, } from "./acp-transport.js"; +import { + grokMcpElicitCancel, + grokMcpElicitResponseForAction, + projectGrokMcpElicitApproval, + type GrokMcpElicitFormRequest, + type GrokMcpElicitResponse, +} from "./grok-elicitation.js"; import { GrokSessionImport } from "./session-import.js"; import { grokMediaResolveRoots, rewriteLocalMediaMarkdown } from "./local-media-markdown.js"; import { GrokSubagentLifecycle } from "./grok-subagent-lifecycle.js"; @@ -104,6 +113,12 @@ import { } from "./grok-subagent.js"; import type { GrokCompactResult } from "./grok-manual-compaction.js"; import { projectGrokFileChanges } from "./grok-file-change.js"; +import { + grokGeneratedImageItem, + mergeGrokGeneratedImage, + projectGrokGeneratedImage, + type GrokGeneratedImage, +} from "./grok-generated-image.js"; import { forkGrokSession } from "./grok-fork.js"; import { mapGrokReplay } from "./grok-history.js"; import { rewindGrokLastTurn } from "./grok-rewind.js"; @@ -176,6 +191,7 @@ export interface GrokAcpTransportLike { text: string, onEvent: (event: GrokTransportEvent) => void, onPermission: (request: GrokPermissionRequest) => Promise, + onElicitation: (request: GrokMcpElicitFormRequest) => Promise, ): Promise; compact( userContext: string | undefined, @@ -192,16 +208,26 @@ export interface GrokAcpTransportLike { interface ActiveTool { item: GrokProjectedToolItem; status?: string; + generatedImage?: GrokGeneratedImage; } interface ActiveApproval { + kind: "permission"; interaction: HostApprovalInteraction; options: Map; resolve(response: RequestPermissionResponse): void; } +interface ActiveElicitation { + kind: "elicitation"; + nativeToolCallId: string; + interaction: HostApprovalInteraction; + resolve(response: GrokMcpElicitResponse): void; +} + interface ActiveTurn { command: TurnStartCommand; + textIdentity: { turnIndex: number; agentMessage: number; reasoning: number }; usage?: GrokTurnUsage; agent: HostAgentMessageItem | null; agentMessageId: string | null; @@ -216,6 +242,7 @@ interface ActiveTurn { subagents: GrokSubagentLifecycle; completedItems: HostItemSnapshot[]; approvals: Map; + elicitations: Map; cancellationRequested: boolean; beforeNativeTurnKeys: Set; completion: Promise; @@ -517,6 +544,7 @@ class GrokHarnessSession implements HarnessSession { }); const active: ActiveTurn = { command, + textIdentity: { turnIndex: this.#snapshot.turns.length, agentMessage: 0, reasoning: 0 }, usage: new GrokTurnUsage(this.#usage, this.#snapshot.turns.length === 0), agent: null, agentMessageId: null, @@ -531,6 +559,7 @@ class GrokHarnessSession implements HarnessSession { subagents: this.#createSubagents(), completedItems: [], approvals: new Map(), + elicitations: new Map(), cancellationRequested: false, beforeNativeTurnKeys: new Set( this.#snapshot.turns.map((turn) => turn.nativeTurnRef.nativeTurnKey), @@ -545,6 +574,7 @@ class GrokHarnessSession implements HarnessSession { text, (event) => this.#handleEvent(active, event), (request) => this.#requestPermission(active, request), + (request) => this.#requestElicitation(active, request), ) .then( (response) => @@ -639,6 +669,7 @@ class GrokHarnessSession implements HarnessSession { }); const active: ActiveTurn = { command: { type: "turn.start", turnId: command.turnId, input: [] }, + textIdentity: { turnIndex: this.#snapshot.turns.length, agentMessage: 0, reasoning: 0 }, agent: null, agentMessageId: null, rawAgentText: "", @@ -652,6 +683,7 @@ class GrokHarnessSession implements HarnessSession { subagents: this.#createSubagents(), completedItems: [], approvals: new Map(), + elicitations: new Map(), cancellationRequested: false, beforeNativeTurnKeys: new Set(), completion, @@ -883,6 +915,7 @@ class GrokHarnessSession implements HarnessSession { for (const approval of active.approvals.values()) { approval.resolve({ outcome: { outcome: "cancelled" } }); } + this.#cancelActiveElicitations(active); try { await this.#transport.cancel(); return { ok: true, value: { cancellationRequested: true } }; @@ -895,8 +928,9 @@ class GrokHarnessSession implements HarnessSession { command: InteractionRespondCommand, ): Promise> { const active = this.#active; - const pending = active?.approvals.get(command.interactionId); - if (!active || !pending) + const permission = active?.approvals.get(command.interactionId); + const elicitation = active?.elicitations.get(command.interactionId); + if (!active || (!permission && !elicitation)) return { ok: false, error: invalidState("Grok Approval is not pending") }; if (command.response.type !== "approval") { return { @@ -908,9 +942,34 @@ class GrokHarnessSession implements HarnessSession { }, }; } - const validation = validateHostApprovalResponse(pending.interaction, command.response); + const response = command.response; + if (elicitation) { + const validation = validateHostApprovalResponse(elicitation.interaction, response); + if (validation) return { ok: false, error: validation }; + const action = elicitation.interaction.actions.find(({ id }) => id === response.actionId); + if (!action) + return { + ok: false, + error: { + code: "invalidRequest", + message: "Grok Approval action is unavailable", + retryable: false, + }, + }; + this.#settleElicitation( + active, + elicitation, + grokMcpElicitResponseForAction( + action.effect === "allowOnce" || action.effect === "allowAlways" ? action.effect : "deny", + ), + "responded", + ); + return { ok: true, value: { accepted: true } }; + } + if (!permission) return { ok: false, error: invalidState("Grok Approval is not pending") }; + const validation = validateHostApprovalResponse(permission.interaction, response); if (validation) return { ok: false, error: validation }; - const option = pending.options.get(command.response.actionId); + const option = permission.options.get(response.actionId); if (!option) return { ok: false, @@ -921,7 +980,7 @@ class GrokHarnessSession implements HarnessSession { }, }; active.approvals.delete(command.interactionId); - pending.resolve({ outcome: { outcome: "selected", optionId: option.optionId } }); + permission.resolve({ outcome: { outcome: "selected", optionId: option.optionId } }); this.#event({ type: "interaction.closed", interactionId: command.interactionId, @@ -954,11 +1013,68 @@ class GrokHarnessSession implements HarnessSession { actions, }; return new Promise((resolve) => { - active.approvals.set(interactionId, { interaction, options, resolve }); + active.approvals.set(interactionId, { + kind: "permission", + interaction, + options, + resolve, + }); this.#channel.emit({ kind: "interaction", interaction }); }); } + #requestElicitation( + active: ActiveTurn, + request: GrokMcpElicitFormRequest, + ): Promise { + if (this.#active !== active || this.#phase !== "open" || active.cancellationRequested) { + return Promise.resolve(grokMcpElicitCancel()); + } + for (const pending of active.elicitations.values()) { + if (pending.nativeToolCallId === request.toolCallId) { + return Promise.resolve(grokMcpElicitCancel()); + } + } + const interactionId = hostInteractionIdSchema.parse(this.#randomUUID()); + const interaction = projectGrokMcpElicitApproval({ + interactionId, + turnId: active.command.turnId, + request, + }); + return new Promise((resolve) => { + active.elicitations.set(interactionId, { + kind: "elicitation", + nativeToolCallId: request.toolCallId, + interaction, + resolve, + }); + this.#channel.emit({ kind: "interaction", interaction }); + }); + } + + #settleElicitation( + active: ActiveTurn, + pending: ActiveElicitation, + response: GrokMcpElicitResponse, + reason: "responded" | "cancelled", + ): void { + if (!active.elicitations.has(pending.interaction.interactionId)) return; + active.elicitations.delete(pending.interaction.interactionId); + pending.resolve(response); + this.#event({ + type: "interaction.closed", + interactionId: pending.interaction.interactionId, + turnId: active.command.turnId, + reason, + }); + } + + #cancelActiveElicitations(active: ActiveTurn): void { + for (const pending of [...active.elicitations.values()]) { + this.#settleElicitation(active, pending, grokMcpElicitCancel(), "cancelled"); + } + } + #handleEvent(active: ActiveTurn, event: GrokTransportEvent): void { if (this.#active !== active || this.#phase !== "open") return; const requestUsage = active.usage?.observe(event); @@ -985,6 +1101,12 @@ class GrokHarnessSession implements HarnessSession { else if (event.type === "compaction.completed") { active.compactionTerminal = event; this.#completeCompaction(active, event); + } else if (event.type === "elicitation.resolved") { + for (const pending of [...active.elicitations.values()]) { + if (pending.nativeToolCallId === event.toolCallId) { + this.#settleElicitation(active, pending, grokMcpElicitCancel(), "cancelled"); + } + } } else if (event.type === "usage" || event.type === "turn.completed") return; } @@ -1072,7 +1194,12 @@ class GrokHarnessSession implements HarnessSession { if (!active.agent) { active.agent = { type: "agentMessage", - itemId: hostItemIdSchema.parse(this.#randomUUID()), + itemId: grokTextItemId( + this.#transport.sessionId, + active.textIdentity.turnIndex, + "agentMessage", + ++active.textIdentity.agentMessage, + ), text: "", }; active.rawAgentText = ""; @@ -1106,7 +1233,12 @@ class GrokHarnessSession implements HarnessSession { if (!active.reasoning) { active.reasoning = { type: "reasoning", - itemId: hostItemIdSchema.parse(this.#randomUUID()), + itemId: grokTextItemId( + this.#transport.sessionId, + active.textIdentity.turnIndex, + "reasoning", + ++active.textIdentity.reasoning, + ), text: "", }; this.#event({ type: "item.started", turnId: active.command.turnId, item: active.reasoning }); @@ -1161,7 +1293,12 @@ class GrokHarnessSession implements HarnessSession { }); const projection = projectGrokToolOutput(event.content, event.rawOutput, this.#toolOutputLimit); if (hasGrokToolProjection(projection)) item = applyGrokToolProjection(item, projection); - active.tools.set(event.callId, { item, ...(event.status ? { status: event.status } : {}) }); + const generatedImage = projectGrokGeneratedImage(event.content, event.rawOutput); + active.tools.set(event.callId, { + item, + ...(event.status ? { status: event.status } : {}), + ...(generatedImage ? { generatedImage } : {}), + }); this.#event({ type: "item.started", turnId: active.command.turnId, item }); if (event.status === "completed" || event.status === "failed") { this.#completeTool(active, event.callId, event.status, event.content, event.rawOutput); @@ -1235,6 +1372,11 @@ class GrokHarnessSession implements HarnessSession { } } if (event.status) tool.status = event.status; + const generatedImage = mergeGrokGeneratedImage( + tool.generatedImage, + projectGrokGeneratedImage(event.content, event.rawOutput), + ); + if (generatedImage) tool.generatedImage = generatedImage; if (event.status === "completed" || event.status === "failed") { this.#completeTool(active, event.callId, event.status, event.content, event.rawOutput); } @@ -1269,15 +1411,24 @@ class GrokHarnessSession implements HarnessSession { if (status !== "completed") return; this.#completeWatchedSubagents(active, tool.item, content, rawOutput); const changes = projectGrokFileChanges(content, this.#cwd); - if (!changes) return; - const fileItem: HostFileChangeItem = { - type: "fileChange", - itemId: hostItemIdSchema.parse(this.#randomUUID()), - sourceItemIds: [tool.item.itemId], - changes, - }; - this.#event({ type: "item.started", turnId: active.command.turnId, item: fileItem }); - this.#completeItem(active, fileItem, { status: "succeeded" }); + if (changes) { + const fileItem: HostFileChangeItem = { + type: "fileChange", + itemId: hostItemIdSchema.parse(this.#randomUUID()), + sourceItemIds: [tool.item.itemId], + changes, + }; + this.#event({ type: "item.started", turnId: active.command.turnId, item: fileItem }); + this.#completeItem(active, fileItem, { status: "succeeded" }); + } + if (!tool.generatedImage) return; + const imageItem = grokGeneratedImageItem( + tool.generatedImage, + hostItemIdSchema.parse(this.#randomUUID()), + tool.item.itemId, + ); + this.#event({ type: "item.started", turnId: active.command.turnId, item: imageItem }); + this.#completeItem(active, imageItem, { status: "succeeded" }); } #completeSubagentTool( @@ -1470,6 +1621,7 @@ class GrokHarnessSession implements HarnessSession { reason: "cancelled", }); } + this.#cancelActiveElicitations(active); this.#active = null; if (usage || this.#usage) this.#publishUsage({ ...usage, ...active.usage?.metrics() }, active.command.turnId, true); @@ -1511,6 +1663,7 @@ class GrokHarnessSession implements HarnessSession { active.cancellationRequested = true; for (const approval of active.approvals.values()) approval.resolve({ outcome: { outcome: "cancelled" } }); + this.#cancelActiveElicitations(active); await this.#transport.cancel().catch(() => undefined); await Promise.race([ active.completion, @@ -1748,6 +1901,19 @@ export class GrokAdapter implements HarnessAdapter { }, }; } + try { + await applyGrokComputerUsePreference(this.#environment ?? process.env); + } catch { + console.warn("Could not apply Grok Computer Use settings"); + return { + ok: false, + error: { + code: "unavailable", + message: "Could not apply Grok Computer Use settings. Retry after fixing Grok config.", + retryable: true, + }, + }; + } const cwd = path.resolve(input.cwd); const parsedRef = input.kind === "resume" ? nativeSessionRefSchema.safeParse(input.nativeRef) : null; diff --git a/packages/adapters/grok/src/grok-codexhost-session-rules.ts b/packages/adapters/grok/src/grok-codexhost-session-rules.ts new file mode 100644 index 000000000..836f5051e --- /dev/null +++ b/packages/adapters/grok/src/grok-codexhost-session-rules.ts @@ -0,0 +1,37 @@ +import type { GrokPermissionMode } from "./permission-modes.js"; +import { grokPermissionModeSessionMeta } from "./permission-modes.js"; + +/** Official Grok ACP `session/new` `_meta.rules` text. CodexHost Grok sessions only. */ +export const GROK_CODEXHOST_SESSION_RULES = `# Codex Desktop / CodexHost + +CodexHost started this Grok ACP session as an external Thread inside Codex Desktop. + +Never print \`CODEXHOST_THREAD_ID\`, \`CODEXHOST_RUNTIME_TOKEN\`, \`CODEXHOST_RUNTIME_ENDPOINT\`, or other runtime descriptors. + +## Session rules + +1. **Model.** The Desktop composer Model is this Harness's Model. Pick that Harness's real Model before starting voice. For Grok, that is grok-4.6 or grok-4.7. \`flash-max\` is a Gemini Flash Model. + +2. **Working directory.** Start the chat in the real repository (\`codex-host\` or the wiki). A Playground \`realtime-voice-chat*\` folder is a scratch cwd. Ask the user to start a new Thread in the real repo. + +3. **Voice vs tools.** Desktop voice is gpt-live speech plus delegated Harness tasks on this same Thread. Do wiki, MCP, and file-heavy work as text in this Thread. + +4. **Harness-owned tools.** This session's MCP servers and skills belong to this Harness. SkillWiki on Grok is Grok's config, not Claude, Pi, or Codex. + +5. **Desktop chrome.** The window is Codex Desktop. Markdown View preview is a Codex file tab (View preview vs View source). \`file://\` browser tabs stay raw. + +6. **Voice routes.** Official Codex live voice (New API/CPA) is a separate route from Host-owned external \`thread/realtime/*\`. This external Thread uses the Host-owned route. + +Keep Grok Build tools, MCP, and rules inside the native session. An \`AGENT_TOOLS_AND_RULES.md\` dump in the cwd is unnecessary. +`; + +export function grokCodexhostSessionMeta(permissionMode?: GrokPermissionMode): { + rules: string; + yoloMode?: boolean; + autoMode?: boolean; +} { + return { + ...(permissionMode ? grokPermissionModeSessionMeta(permissionMode) : {}), + rules: GROK_CODEXHOST_SESSION_RULES, + }; +} diff --git a/packages/adapters/grok/src/grok-elicitation.ts b/packages/adapters/grok/src/grok-elicitation.ts new file mode 100644 index 000000000..bd3198cc2 --- /dev/null +++ b/packages/adapters/grok/src/grok-elicitation.ts @@ -0,0 +1,242 @@ +import type { HostApprovalInteraction } from "@codexhost/harness-adapter"; +import type { HostInteractionId, HostTurnId } from "@codexhost/shared-contracts"; + +/** JSON-RPC wire method. JS ACP SDK 1.3.0 forwards the full `_`-prefixed name to `extMethod`. */ +export const GROK_MCP_ELICIT_METHOD = "_x.ai/mcp/elicit"; + +export const GROK_ELICIT_TITLE_MAX_LENGTH = 150; +export const GROK_ELICIT_DESCRIPTION_MAX_LENGTH = 500; + +const FORM_KEYS = new Set([ + "sessionId", + "toolCallId", + "serverName", + "message", + "mode", + "requestedSchema", + "persist", +]); + +export type GrokMcpElicitPersistScope = "session" | "always"; +const URL_KEYS = new Set([ + "sessionId", + "toolCallId", + "serverName", + "message", + "mode", + "url", + "elicitationId", +]); +const SCHEMA_KEYS = new Set(["type", "properties", "required"]); + +export type GrokMcpElicitOutcome = "accept" | "decline" | "cancel"; + +export interface GrokMcpElicitAcceptResponse { + outcome: "accept"; + content: Record; + persist?: GrokMcpElicitPersistScope; +} + +export interface GrokMcpElicitDeclineResponse { + outcome: "decline"; +} + +export interface GrokMcpElicitCancelResponse { + outcome: "cancel"; +} + +export type GrokMcpElicitResponse = + GrokMcpElicitAcceptResponse | GrokMcpElicitDeclineResponse | GrokMcpElicitCancelResponse; + +export interface GrokMcpElicitFormRequest { + sessionId: string; + toolCallId: string; + serverName: string; + message: string; + mode: "form"; + persist?: GrokMcpElicitPersistScope[]; +} + +/** Sanitized Clock allow shape from `@oai/sky` `withComputerUsePolicy` / `createElicitation`. */ +export const GROK_CLOCK_ELICIT_FIXTURE: GrokMcpElicitFormRequest = { + sessionId: "session-1", + toolCallId: "elicit-clock-1", + serverName: "cua_repl", + message: 'Allow Computer Use to use "Clock"?', + mode: "form", +}; + +export const GROK_CLOCK_ELICIT_PERSIST_FIXTURE: GrokMcpElicitFormRequest = { + ...GROK_CLOCK_ELICIT_FIXTURE, + persist: ["session", "always"], +}; + +export type GrokMcpElicitParseResult = + | { status: "form"; request: GrokMcpElicitFormRequest } + | { status: "unsupported" } + | { status: "invalid"; message: string }; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function nonEmptyString(value: unknown): string | undefined { + return typeof value === "string" && value.trim().length > 0 ? value : undefined; +} + +function hasOnlyKeys(record: Record, allowed: Set): boolean { + return Object.keys(record).every((key) => allowed.has(key)); +} + +function clampText(value: string, max: number): string { + return value.length <= max ? value : value.slice(0, max); +} + +function parsePersistOffer(value: unknown): GrokMcpElicitPersistScope[] | "invalid" | undefined { + if (value === undefined) return undefined; + if (!Array.isArray(value) || value.length === 0) return "invalid"; + let session = false; + let always = false; + const seen = new Set(); + for (const item of value) { + if (typeof item !== "string" || item !== item.trim()) return "invalid"; + if (seen.has(item)) return "invalid"; + seen.add(item); + if (item === "session") session = true; + else if (item === "always") always = true; + else return "invalid"; + } + const out: GrokMcpElicitPersistScope[] = []; + if (session) out.push("session"); + if (always) out.push("always"); + return out.length > 0 ? out : "invalid"; +} + +function schemaIsSupportedEmptyObject(schema: unknown): boolean { + if (!isRecord(schema) || !hasOnlyKeys(schema, SCHEMA_KEYS) || schema.type !== "object") { + return false; + } + if (schema.properties !== undefined) { + if (!isRecord(schema.properties) || Object.keys(schema.properties).length !== 0) return false; + } + if (schema.required !== undefined) { + if (!Array.isArray(schema.required) || schema.required.length !== 0) return false; + } + return true; +} + +export function parseGrokMcpElicitParams(params: unknown): GrokMcpElicitParseResult { + if (!isRecord(params)) + return { status: "invalid", message: "elicitation params must be an object" }; + const sessionId = nonEmptyString(params.sessionId); + const toolCallId = nonEmptyString(params.toolCallId); + const serverName = nonEmptyString(params.serverName); + const message = nonEmptyString(params.message); + if (!sessionId || !toolCallId || !serverName || !message) { + return { + status: "invalid", + message: "elicitation requires sessionId, toolCallId, serverName, and message", + }; + } + if (params.mode === "url") { + if (!hasOnlyKeys(params, URL_KEYS)) { + return { status: "invalid", message: "URL elicitation contains unreviewed fields" }; + } + if (!nonEmptyString(params.url) || !nonEmptyString(params.elicitationId)) { + return { status: "invalid", message: "URL elicitation requires url and elicitationId" }; + } + return { status: "unsupported" }; + } + if (params.mode !== "form") { + return { status: "invalid", message: "elicitation mode must be form or url" }; + } + if (!hasOnlyKeys(params, FORM_KEYS)) { + return { status: "invalid", message: "form elicitation contains unreviewed fields" }; + } + if (params.requestedSchema !== undefined) { + if (!isRecord(params.requestedSchema)) { + return { status: "invalid", message: "requestedSchema must be an object" }; + } + if (!schemaIsSupportedEmptyObject(params.requestedSchema)) return { status: "unsupported" }; + } + const persist = parsePersistOffer(params.persist); + if (persist === "invalid") { + return { + status: "invalid", + message: "persist must be a unique array of session and/or always", + }; + } + return { + status: "form", + request: { + sessionId, + toolCallId, + serverName, + message, + mode: "form", + ...(persist ? { persist } : {}), + }, + }; +} + +export function grokMcpElicitAccept( + persist?: GrokMcpElicitPersistScope, +): GrokMcpElicitAcceptResponse { + return { outcome: "accept", content: {}, ...(persist ? { persist } : {}) }; +} + +export function grokMcpElicitDecline(): GrokMcpElicitDeclineResponse { + return { outcome: "decline" }; +} + +export function grokMcpElicitCancel(): GrokMcpElicitCancelResponse { + return { outcome: "cancel" }; +} + +export function grokMcpElicitResponseForAction( + effect: "allowOnce" | "allowAlways" | "deny" | "cancel", +): GrokMcpElicitResponse { + if (effect === "allowOnce") return grokMcpElicitAccept(); + if (effect === "allowAlways") return grokMcpElicitAccept("always"); + if (effect === "deny") return grokMcpElicitDecline(); + return grokMcpElicitCancel(); +} + +export function grokMcpElicitApprovalActions( + persist?: GrokMcpElicitPersistScope[], +): HostApprovalInteraction["actions"] { + const actions: HostApprovalInteraction["actions"] = [ + { id: "allow-once", label: "Allow once", effect: "allowOnce" }, + ]; + if (persist?.includes("always")) { + actions.push({ id: "allow-always", label: "Always allow", effect: "allowAlways" }); + } + actions.push({ id: "deny", label: "Deny", effect: "deny" }); + return actions; +} + +export function projectGrokMcpElicitApproval(input: { + interactionId: HostInteractionId; + turnId: HostTurnId; + request: GrokMcpElicitFormRequest; +}): HostApprovalInteraction { + const title = clampText( + `${input.request.serverName}: ${input.request.message}`, + GROK_ELICIT_TITLE_MAX_LENGTH, + ); + const description = clampText(input.request.message, GROK_ELICIT_DESCRIPTION_MAX_LENGTH); + return { + type: "approval", + interactionId: input.interactionId, + turnId: input.turnId, + title, + description, + subject: { type: "nativeAction" }, + actions: grokMcpElicitApprovalActions(input.request.persist), + }; +} + +export function grokElicitationResolvedToolCallId(update: unknown): string | undefined { + if (!isRecord(update) || update.sessionUpdate !== "interaction_resolved") return undefined; + return nonEmptyString(update.tool_call_id) ?? nonEmptyString(update.toolCallId); +} diff --git a/packages/adapters/grok/src/grok-file-change.ts b/packages/adapters/grok/src/grok-file-change.ts index 84d090b26..5246302e4 100644 --- a/packages/adapters/grok/src/grok-file-change.ts +++ b/packages/adapters/grok/src/grok-file-change.ts @@ -12,7 +12,7 @@ function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } -function validAbsolutePath(value: unknown): value is string { +export function validAbsolutePath(value: unknown): value is string { return ( typeof value === "string" && path.isAbsolute(value) && diff --git a/packages/adapters/grok/src/grok-generated-image.ts b/packages/adapters/grok/src/grok-generated-image.ts new file mode 100644 index 000000000..909defe8c --- /dev/null +++ b/packages/adapters/grok/src/grok-generated-image.ts @@ -0,0 +1,101 @@ +import path from "node:path"; + +import type { HostGeneratedImageItem } from "@codexhost/harness-adapter"; +import type { HostItemId } from "@codexhost/shared-contracts"; + +import { validAbsolutePath } from "./grok-file-change.js"; +import { mediaFileExists } from "./local-media-markdown.js"; + +const GENERATED_IMAGE_TYPES = new Set(["ImageGen", "ImageEdit"]); + +export interface GrokGeneratedImage { + savedPath: string; + revisedPrompt?: string; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function stringField(value: unknown, key: string): string | undefined { + return isRecord(value) && typeof value[key] === "string" && value[key].length > 0 + ? value[key] + : undefined; +} + +function fromNativeRecord( + value: unknown, + exists: (absolutePath: string) => boolean, +): GrokGeneratedImage | null { + if (!isRecord(value)) return null; + const type = stringField(value, "type"); + const sessionFolder = stringField(value, "session_folder"); + const typed = type !== undefined && GENERATED_IMAGE_TYPES.has(type); + const imageFolder = sessionFolder === "images"; + if (!typed && !imageFolder) return null; + const savedPath = stringField(value, "path"); + if (!validAbsolutePath(savedPath)) return null; + const resolved = path.resolve(savedPath); + if (!exists(resolved)) return null; + const filename = stringField(value, "filename"); + return { + savedPath: resolved, + ...(filename ? { revisedPrompt: filename } : {}), + }; +} + +function parseJsonObject(text: string): unknown { + const trimmed = text.trim(); + if (!(trimmed.startsWith("{") && trimmed.endsWith("}"))) return null; + try { + return JSON.parse(trimmed) as unknown; + } catch { + return null; + } +} + +/** + * Read Grok Imagine ACP output. Only ImageGen/ImageEdit native records qualify; + * tool names and markdown links are ignored. + */ +export function projectGrokGeneratedImage( + content: unknown, + rawOutput: unknown, + exists: (absolutePath: string) => boolean = mediaFileExists, +): GrokGeneratedImage | null { + const fromRaw = fromNativeRecord(rawOutput, exists); + if (fromRaw) return fromRaw; + if (!Array.isArray(content)) return null; + for (const entry of content) { + if (!isRecord(entry)) continue; + const nested = entry.type === "content" && isRecord(entry.content) ? entry.content : entry; + if (isRecord(nested) && nested.type === "text" && typeof nested.text === "string") { + const parsed = fromNativeRecord(parseJsonObject(nested.text), exists); + if (parsed) return parsed; + } + const direct = fromNativeRecord(nested, exists); + if (direct) return direct; + } + return null; +} + +export function mergeGrokGeneratedImage( + previous: GrokGeneratedImage | undefined, + next: GrokGeneratedImage | null, +): GrokGeneratedImage | undefined { + return next ?? previous; +} + +export function grokGeneratedImageItem( + image: GrokGeneratedImage, + itemId: HostItemId, + sourceItemId: HostItemId, +): HostGeneratedImageItem { + return { + type: "generatedImage", + itemId, + savedPath: image.savedPath, + sourceItemIds: [sourceItemId], + ...(image.revisedPrompt ? { revisedPrompt: image.revisedPrompt } : {}), + }; +} diff --git a/packages/adapters/grok/src/grok-history.ts b/packages/adapters/grok/src/grok-history.ts index 6b4596354..f67de5894 100644 --- a/packages/adapters/grok/src/grok-history.ts +++ b/packages/adapters/grok/src/grok-history.ts @@ -17,7 +17,14 @@ import { } from "@codexhost/shared-contracts"; import type { GrokTransportEvent } from "./acp-transport.js"; +import { grokTextItemId } from "./grok-text-item-id.js"; import { projectGrokFileChanges } from "./grok-file-change.js"; +import { + grokGeneratedImageItem, + mergeGrokGeneratedImage, + projectGrokGeneratedImage, + type GrokGeneratedImage, +} from "./grok-generated-image.js"; import { grokMediaResolveRoots, rewriteLocalMediaMarkdown } from "./local-media-markdown.js"; import { grokNativeSubagentId, @@ -122,8 +129,13 @@ export function mapGrokReplay( let nativePromptIndex = 0; let currentPromptIndex: number | null = null; let agent: HostAgentMessageItem | null = null; + let agentMessageId: string | undefined; + let agentIndex = 0; let reasoning: HostReasoningItem | null = null; + let reasoningMessageId: string | undefined; + let reasoningIndex = 0; const tools = new Map(); + const generatedImages = new Map(); const mediaRoots = grokMediaResolveRoots(cwd, sessionDirectory); const subagents = new Map(); const subagentAliases = new Map(); @@ -247,6 +259,7 @@ export function mapGrokReplay( items.push({ item: tool, outcome: { status: "succeeded" } }); } tools.clear(); + generatedImages.clear(); for (const item of subagents.values()) { items.push({ item, outcome: { status: "succeeded" } }); } @@ -263,6 +276,11 @@ export function mapGrokReplay( if (hasGrokToolProjection(projection)) { tools.set(callId, applyGrokToolProjection(tool, projection)); } + const image = mergeGrokGeneratedImage( + generatedImages.get(callId), + projectGrokGeneratedImage(content, rawOutput), + ); + if (image) generatedImages.set(callId, image); }; const completeTool = ( callId: string, @@ -274,6 +292,8 @@ export function mapGrokReplay( const tool = tools.get(callId); if (!tool) return; tools.delete(callId); + const generatedImage = generatedImages.get(callId); + generatedImages.delete(callId); const outcome: HostItemSnapshot["outcome"] = status === "failed" ? { @@ -296,14 +316,24 @@ export function mapGrokReplay( items.push({ item: tool, outcome }); if (status !== "completed") return; const changes = projectGrokFileChanges(content, cwd); - if (!changes) return; - const fileItem: HostFileChangeItem = { - type: "fileChange", - itemId: stableId("file-change", turnIndex, ++messageIndex), - sourceItemIds: [tool.itemId], - changes, - }; - items.push({ item: fileItem, outcome: { status: "succeeded" } }); + if (changes) { + const fileItem: HostFileChangeItem = { + type: "fileChange", + itemId: stableId("file-change", turnIndex, ++messageIndex), + sourceItemIds: [tool.itemId], + changes, + }; + items.push({ item: fileItem, outcome: { status: "succeeded" } }); + } + if (!generatedImage) return; + items.push({ + item: grokGeneratedImageItem( + generatedImage, + stableId("generated-image", turnIndex, ++messageIndex), + tool.itemId, + ), + outcome: { status: "succeeded" }, + }); }; const completeTurn = (outcome: HistoricalTurnOutcome, terminalKey?: string): void => { if (input.length === 0) return; @@ -336,6 +366,8 @@ export function mapGrokReplay( }); turnIndex += 1; messageIndex = 0; + agentIndex = 0; + reasoningIndex = 0; nativeTurnKey = null; currentPromptIndex = null; input = ""; @@ -350,6 +382,8 @@ export function mapGrokReplay( }); turnIndex = turns.length; messageIndex = 0; + agentIndex = 0; + reasoningIndex = 0; nativeTurnKey = null; nativePromptIndex = event.targetPromptIndex; currentPromptIndex = null; @@ -358,6 +392,7 @@ export function mapGrokReplay( agent = null; reasoning = null; tools.clear(); + generatedImages.clear(); subagents.clear(); subagentAliases.clear(); continue; @@ -415,20 +450,26 @@ export function mapGrokReplay( outcome, }); } else if (event.type === "agent.text") { + if (agent && agentMessageId !== event.messageId) completeAgent(); if (!agent) { completeReasoning(); + agentMessageId = event.messageId; + ++messageIndex; agent = { type: "agentMessage", - itemId: stableId("message", turnIndex, ++messageIndex), + itemId: grokTextItemId(sessionId, turnIndex, "agentMessage", ++agentIndex), text: "", }; } agent = { ...agent, text: agent.text + event.text }; } else if (event.type === "agent.thought") { + if (reasoning && reasoningMessageId !== event.messageId) completeReasoning(); if (!reasoning) { + reasoningMessageId = event.messageId; + ++messageIndex; reasoning = { type: "reasoning", - itemId: stableId("reasoning", turnIndex, ++messageIndex), + itemId: grokTextItemId(sessionId, turnIndex, "reasoning", ++reasoningIndex), text: "", }; } diff --git a/packages/adapters/grok/src/grok-text-item-id.ts b/packages/adapters/grok/src/grok-text-item-id.ts new file mode 100644 index 000000000..1e4e189a9 --- /dev/null +++ b/packages/adapters/grok/src/grok-text-item-id.ts @@ -0,0 +1,13 @@ +import { createHash } from "node:crypto"; +import { hostItemIdSchema } from "@codexhost/shared-contracts"; + +/** Live and replay text Items share identity independently of chunk boundaries and text. */ +export function grokTextItemId( + sessionId: string, + turnIndex: number, + kind: "agentMessage" | "reasoning", + ordinal: number, +): ReturnType { + const identity = JSON.stringify([sessionId, turnIndex, kind, ordinal]); + return hostItemIdSchema.parse(`grok-text-${createHash("sha256").update(identity).digest("hex")}`); +} diff --git a/packages/adapters/grok/src/index.ts b/packages/adapters/grok/src/index.ts index 22a8d20da..6ea8e283e 100644 --- a/packages/adapters/grok/src/index.ts +++ b/packages/adapters/grok/src/index.ts @@ -38,7 +38,26 @@ export { decodeGrokPermissionModeId, grokPermissionModeSessionMeta, } from "./permission-modes.js"; +export { + GROK_CODEXHOST_SESSION_RULES, + grokCodexhostSessionMeta, +} from "./grok-codexhost-session-rules.js"; export type { GrokPermissionMode } from "./permission-modes.js"; +export { + GROK_CLOCK_ELICIT_FIXTURE, + GROK_CLOCK_ELICIT_PERSIST_FIXTURE, + GROK_MCP_ELICIT_METHOD, + grokMcpElicitAccept, + grokMcpElicitCancel, + grokMcpElicitDecline, + parseGrokMcpElicitParams, + projectGrokMcpElicitApproval, +} from "./grok-elicitation.js"; +export type { + GrokMcpElicitFormRequest, + GrokMcpElicitParseResult, + GrokMcpElicitResponse, +} from "./grok-elicitation.js"; export { GROK_REWIND_EXECUTE_METHOD, GROK_REWIND_POINTS_METHOD, diff --git a/packages/adapters/grok/src/installation.ts b/packages/adapters/grok/src/installation.ts index c5b3f3d43..ccdc5856f 100644 --- a/packages/adapters/grok/src/installation.ts +++ b/packages/adapters/grok/src/installation.ts @@ -1,7 +1,9 @@ import { createInstallationManager, installationVersion, + readLauncherScript, runInstallationCommand, + versionFromOutput, } from "@codexhost/harness-discovery"; import { resolveGrokExecutable } from "./command.js"; @@ -18,6 +20,29 @@ export function createGrokInstallation(environment: NodeJS.ProcessEnv, command?: let installer = ""; return createInstallationManager({ async check() { + const launcher = await readLauncherScript( + resolveGrokExecutable({ environment, ...(command ? { command } : {}) }), + ); + if (launcher.includes("GROKGOD_HOME=") && launcher.includes("GROKGOD_BIN=")) { + return { + currentVersion: versionFromOutput( + await run(["--version"], undefined, { + GROK_DISABLE_AUTOUPDATER: "1", + GROKGOD_UPDATE_CHECK_DISABLE: "1", + }), + ), + latestVersion: "Unknown", + latestVersionKind: "unknown" as const, + updateAvailable: false, + canUpdate: false, + messageCode: "original-installer", + notice: { + en: "Managed by grokgod. Use grokgod's updater or installer to preserve the installed patches. The latest patched version is not checked here.", + "zh-CN": + "此安装由 grokgod 管理。请使用 grokgod 的更新程序或安装程序保留补丁;此处不检查最新补丁版本。", + }, + }; + } const state = JSON.parse(await run(["update", "--check", "--json"])) as Record< string, unknown diff --git a/packages/adapters/grok/src/local-media-markdown.ts b/packages/adapters/grok/src/local-media-markdown.ts index bbbd6918b..d78eae31e 100644 --- a/packages/adapters/grok/src/local-media-markdown.ts +++ b/packages/adapters/grok/src/local-media-markdown.ts @@ -312,7 +312,7 @@ function hasMediaExtension(value: string): boolean { return MEDIA_EXTENSIONS.has(path.extname(pathname).toLowerCase()); } -function mediaFileExists(absolutePath: string): boolean { +export function mediaFileExists(absolutePath: string): boolean { try { return existsSync(absolutePath) && statSync(absolutePath).isFile(); } catch { diff --git a/packages/adapters/grok/test/grok-acp-elicitation.test.ts b/packages/adapters/grok/test/grok-acp-elicitation.test.ts new file mode 100644 index 000000000..910c4a6e5 --- /dev/null +++ b/packages/adapters/grok/test/grok-acp-elicitation.test.ts @@ -0,0 +1,135 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +import { harnessPermissionModeIdSchema } from "@codexhost/shared-contracts"; +import { describe, expect, it } from "vitest"; + +import { GrokAcpTransport } from "../src/acp-transport.js"; +import { + GROK_CLOCK_ELICIT_FIXTURE, + GROK_MCP_ELICIT_METHOD, + grokMcpElicitAccept, + grokMcpElicitCancel, +} from "../src/grok-elicitation.js"; + +const permissionModeId = harnessPermissionModeIdSchema.parse("ask"); + +async function withFixture( + script: string, + run: (transport: GrokAcpTransport, capture: string) => Promise, +): Promise { + const root = await mkdtemp(path.join(os.tmpdir(), "grok-acp-elicit-")); + const command = path.join(root, "fixture.mjs"); + const capture = path.join(root, "captured.jsonl"); + await writeFile(command, script.replaceAll("__CAPTURE__", JSON.stringify(capture)), { + mode: 0o755, + }); + const transport = new GrokAcpTransport({ + cwd: root, + command, + environment: { HOME: root, GROK_HOME: root }, + commandTimeoutMs: 4000, + closeTimeoutMs: 200, + }); + try { + await run(transport, capture); + } finally { + await transport.close(); + await rm(root, { recursive: true, force: true }); + } +} + +const clockParams = { + ...GROK_CLOCK_ELICIT_FIXTURE, + sessionId: "created", +}; + +describe.skipIf(process.platform === "win32")("Grok ACP MCP elicitation wire", () => { + it("keeps empty clientCapabilities and replies to _x.ai/mcp/elicit with a flat outcome", async () => { + await withFixture( + `#!${process.execPath} +import { appendFileSync } from 'node:fs'; +import { createInterface } from 'node:readline'; +const capture = __CAPTURE__; +const send = (value) => process.stdout.write(JSON.stringify({jsonrpc:'2.0',...value})+'\\n'); +let promptId; +createInterface({input:process.stdin}).on('line', line => { + const request = JSON.parse(line); + appendFileSync(capture, line + '\\n'); + if (request.method === 'initialize') { + send({id:request.id,result:{protocolVersion:1,agentCapabilities:{loadSession:true},authMethods:[]}}); + } else if (request.method === 'session/new') { + send({id:request.id,result:{sessionId:'created'}}); + } else if (request.method === 'session/prompt') { + promptId = request.id; + send({id:'elicit-1', method:'${GROK_MCP_ELICIT_METHOD}', params:${JSON.stringify(clockParams)}}); + } else if (request.id === 'elicit-1') { + send({id:promptId,result:{stopReason:'end_turn'}}); + } +}); +`, + async (transport, capture) => { + await transport.open({ kind: "create", permissionModeId }); + const response = await transport.runTurn( + "open clock", + () => undefined, + async () => ({ outcome: { outcome: "cancelled" } }), + async () => grokMcpElicitAccept(), + ); + expect(response).toEqual({ stopReason: "end_turn" }); + const lines = (await readFile(capture, "utf8")) + .trim() + .split("\n") + .map((line) => JSON.parse(line) as Record); + const initialize = lines.find((line) => line.method === "initialize"); + expect((initialize?.params as { clientCapabilities?: unknown }).clientCapabilities).toEqual( + {}, + ); + const elicitReply = lines.find((line) => line.id === "elicit-1"); + expect(elicitReply).toMatchObject({ + id: "elicit-1", + result: grokMcpElicitAccept(), + }); + expect(elicitReply && "error" in elicitReply).toBe(false); + }, + ); + }); + + it("cancels elicitation with no active prompt and rejects an unknown extension method", async () => { + await withFixture( + `#!${process.execPath} +import { appendFileSync } from 'node:fs'; +import { createInterface } from 'node:readline'; +const capture = __CAPTURE__; +const send = (value) => process.stdout.write(JSON.stringify({jsonrpc:'2.0',...value})+'\\n'); +createInterface({input:process.stdin}).on('line', line => { + const request = JSON.parse(line); + appendFileSync(capture, line + '\\n'); + if (request.method === 'initialize') { + send({id:request.id,result:{protocolVersion:1,agentCapabilities:{loadSession:true},authMethods:[]}}); + } else if (request.method === 'session/new') { + send({id:request.id,result:{sessionId:'created'}}); + send({id:'elicit-idle', method:'${GROK_MCP_ELICIT_METHOD}', params:${JSON.stringify(clockParams)}}); + send({id:'unknown-ext', method:'elicitation/create', params:{}}); + } +}); +`, + async (transport, capture) => { + await transport.open({ kind: "create", permissionModeId }); + await new Promise((resolve) => setTimeout(resolve, 150)); + const lines = (await readFile(capture, "utf8")) + .trim() + .split("\n") + .map((line) => JSON.parse(line) as Record); + expect(lines.find((line) => line.id === "elicit-idle")).toMatchObject({ + result: grokMcpElicitCancel(), + }); + const unknown = lines.find((line) => line.id === "unknown-ext") as { + error?: { code?: number }; + }; + expect(unknown?.error?.code).toBe(-32601); + }, + ); + }); +}); diff --git a/packages/adapters/grok/test/grok-acp-model-selection.test.ts b/packages/adapters/grok/test/grok-acp-model-selection.test.ts new file mode 100644 index 000000000..ee578e699 --- /dev/null +++ b/packages/adapters/grok/test/grok-acp-model-selection.test.ts @@ -0,0 +1,62 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { describe, expect, it } from "vitest"; +import { harnessPermissionModeIdSchema } from "@codexhost/shared-contracts"; +import { GrokAcpTransport } from "../src/acp-transport.js"; + +// The executable fixture uses a POSIX shebang. +describe.skipIf(process.platform === "win32")("Grok ACP model configuration", () => { + it.each([ + { effort: "high", contextWindow: undefined }, + { effort: "max", contextWindow: undefined }, + { effort: "high", contextWindow: 500000 }, + { effort: "max", contextWindow: 500000 }, + { effort: undefined, contextWindow: 500000 }, + { effort: undefined, contextWindow: undefined }, + ])("sends native metadata for $effort / $contextWindow", async ({ effort, contextWindow }) => { + const root = await mkdtemp(path.join(os.tmpdir(), "grok-acp-model-")); + const command = path.join(root, "fixture.mjs"); + await writeFile( + command, + `#!${process.execPath} +import { createInterface } from 'node:readline'; +const expected = ${JSON.stringify({ effort, contextWindow })}; +const send = (value) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', ...value }) + '\\n'); +createInterface({ input: process.stdin }).on('line', (line) => { + const request = JSON.parse(line); + if (request.method === 'initialize') send({ id: request.id, result: { protocolVersion: 1, agentCapabilities: {}, authMethods: [] } }); + else if (request.method === 'session/new') send({ id: request.id, result: { sessionId: 'fixture' } }); + else if (request.method === 'session/set_model') { + const { params } = request; + const valid = params.sessionId === 'fixture' && params.modelId === 'deepseek-v4-flash' + && !Object.hasOwn(params, 'reasoningEffort') + && params._meta?.reasoningEffort === expected.effort + && params._meta?.contextWindow === expected.contextWindow; + send({ id: request.id, result: { _meta: { model: valid ? { Ok: params.modelId } : { Err: 'Invalid native metadata' } } } }); + } +}); +`, + { mode: 0o755 }, + ); + const transport = new GrokAcpTransport({ + cwd: root, + command, + environment: { HOME: root, GROK_HOME: root }, + commandTimeoutMs: 3000, + closeTimeoutMs: 100, + }); + try { + await transport.open({ + kind: "create", + permissionModeId: harnessPermissionModeIdSchema.parse("ask"), + }); + await expect( + transport.setModel("deepseek-v4-flash", effort, contextWindow), + ).resolves.toBeUndefined(); + } finally { + await transport.close(); + await rm(root, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/adapters/grok/test/grok-acp-permissions.test.ts b/packages/adapters/grok/test/grok-acp-permissions.test.ts index da1333861..1ba19df64 100644 --- a/packages/adapters/grok/test/grok-acp-permissions.test.ts +++ b/packages/adapters/grok/test/grok-acp-permissions.test.ts @@ -164,6 +164,7 @@ describe.skipIf(process.platform === "win32")("Grok ACP permission wire", () => async () => ({ outcome: { outcome: "cancelled" }, }), + async () => ({ outcome: "cancel" }), ), ).rejects.toThrow("active operation"); const closing = f.transport.close(); diff --git a/packages/adapters/grok/test/grok-acp-session-rules.test.ts b/packages/adapters/grok/test/grok-acp-session-rules.test.ts new file mode 100644 index 000000000..5407aacd9 --- /dev/null +++ b/packages/adapters/grok/test/grok-acp-session-rules.test.ts @@ -0,0 +1,131 @@ +import { randomUUID } from "node:crypto"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +import { harnessPermissionModeIdSchema } from "@codexhost/shared-contracts"; +import { describe, expect, it } from "vitest"; + +import { GrokAcpTransport } from "../src/acp-transport.js"; +import { + GROK_CODEXHOST_SESSION_RULES, + grokCodexhostSessionMeta, +} from "../src/grok-codexhost-session-rules.js"; + +const permissionModeId = (id: "ask" | "auto" | "always-approve") => + harnessPermissionModeIdSchema.parse(id); + +async function capturedMeta( + root: string, + open: Parameters[0], +): Promise[]> { + const command = path.join(root, "fixture.mjs"); + const capture = path.join(root, `captured-${randomUUID()}.jsonl`); + await writeFile( + command, + `#!${process.execPath} +import { appendFileSync } from 'node:fs'; +import { createInterface } from 'node:readline'; +const capture = process.env.CODEXHOST_ACP_CAPTURE; +const send = (value) => process.stdout.write(JSON.stringify({jsonrpc:'2.0',...value})+'\\n'); +createInterface({input:process.stdin}).on('line', line => { + const request = JSON.parse(line); + if (request.method === 'session/new' || request.method === 'session/load') { + appendFileSync(capture, JSON.stringify({method:request.method, params:request.params})+'\\n'); + } + if (request.method === 'initialize') { + send({id:request.id,result:{protocolVersion:1,agentCapabilities:{loadSession:true},authMethods:[]}}); + } else if (request.method === 'session/new') { + send({id:request.id,result:{sessionId:'created'}}); + } else if (request.method === 'session/load') { + send({id:request.id,result:{sessionId:request.params.sessionId}}); + } else if (request.method === '_x.ai/session/fork') { + send({id:request.id,result:{newSessionId:'forked',parentSessionId:request.params.sourceSessionId}}); + } else if (request.method === '_x.ai/rewind/execute') { + send({id:request.id,result:{success:true,targetPromptIndex:request.params.targetPromptIndex}}); + } +}); +`, + { mode: 0o755 }, + ); + const transport = new GrokAcpTransport({ + cwd: root, + command, + environment: { HOME: root, GROK_HOME: root, CODEXHOST_ACP_CAPTURE: capture }, + commandTimeoutMs: 3000, + closeTimeoutMs: 100, + }); + try { + await transport.open(open); + } finally { + await transport.close(); + } + const text = await readFile(capture, "utf8"); + return text + .trim() + .split("\n") + .map( + (line) => JSON.parse(line) as { method: string; params: { _meta?: Record } }, + ) + .map((entry) => entry.params._meta ?? {}); +} + +describe.skipIf(process.platform === "win32")("Grok ACP session rules wire", () => { + it("sends _meta.rules on create, resume, fork-load, and rewind-load", async () => { + const root = await mkdtemp(path.join(os.tmpdir(), "grok-acp-session-rules-")); + try { + const createAsk = await capturedMeta(root, { + kind: "create", + permissionModeId: permissionModeId("ask"), + }); + const createAuto = await capturedMeta(root, { + kind: "create", + permissionModeId: permissionModeId("auto"), + }); + const createAlways = await capturedMeta(root, { + kind: "create", + permissionModeId: permissionModeId("always-approve"), + }); + const resumeAsk = await capturedMeta(root, { + kind: "resume", + sessionId: "resume-ask", + permissionModeId: permissionModeId("ask"), + }); + const resumeAuto = await capturedMeta(root, { + kind: "resume", + sessionId: "resume-auto", + permissionModeId: permissionModeId("auto"), + }); + const resumeAlways = await capturedMeta(root, { + kind: "resume", + sessionId: "resume-always", + permissionModeId: permissionModeId("always-approve"), + }); + const forkLoad = await capturedMeta(root, { + kind: "fork", + sourceSessionId: "source", + sourceCwd: root, + targetPromptIndex: 0, + }); + const rewindLoad = await capturedMeta(root, { + kind: "rewind", + sessionId: "rewind", + targetPromptIndex: 0, + }); + + expect(createAsk).toEqual([grokCodexhostSessionMeta("ask")]); + expect(createAuto).toEqual([grokCodexhostSessionMeta("auto")]); + expect(createAlways).toEqual([grokCodexhostSessionMeta("always-approve")]); + expect(resumeAsk).toEqual([grokCodexhostSessionMeta("ask")]); + expect(resumeAuto).toEqual([grokCodexhostSessionMeta("auto")]); + expect(resumeAlways).toEqual([grokCodexhostSessionMeta("always-approve")]); + expect(forkLoad).toEqual([grokCodexhostSessionMeta()]); + expect(rewindLoad).toEqual([grokCodexhostSessionMeta()]); + expect(forkLoad[0]?.yoloMode).toBeUndefined(); + expect(rewindLoad[0]?.autoMode).toBeUndefined(); + expect(createAsk[0]?.rules).toBe(GROK_CODEXHOST_SESSION_RULES); + } finally { + await rm(root, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/adapters/grok/test/grok-acp-usage.test.ts b/packages/adapters/grok/test/grok-acp-usage.test.ts index e7221da0f..5f27f2689 100644 --- a/packages/adapters/grok/test/grok-acp-usage.test.ts +++ b/packages/adapters/grok/test/grok-acp-usage.test.ts @@ -51,6 +51,7 @@ createInterface({input:process.stdin}).on('line', line => { "fixture", (event) => events.push(event), async () => ({ outcome: { outcome: "cancelled" } }), + async () => ({ outcome: "cancel" }), ); expect(events).toEqual([ { type: "tool.input.delta", text: '{"command":' }, diff --git a/packages/adapters/grok/test/grok-adapter.test.ts b/packages/adapters/grok/test/grok-adapter.test.ts index 1c39266b6..d973f71f1 100644 --- a/packages/adapters/grok/test/grok-adapter.test.ts +++ b/packages/adapters/grok/test/grok-adapter.test.ts @@ -3,6 +3,15 @@ import type { PromptResponse, RequestPermissionResponse, } from "@agentclientprotocol/sdk"; +import { + GROK_CLOCK_ELICIT_FIXTURE, + GROK_CLOCK_ELICIT_PERSIST_FIXTURE, + grokMcpElicitAccept, + grokMcpElicitCancel, + grokMcpElicitDecline, + type GrokMcpElicitFormRequest, + type GrokMcpElicitResponse, +} from "../src/grok-elicitation.js"; import type { HarnessOutput, HostEvent } from "@codexhost/harness-adapter"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import os from "node:os"; @@ -92,6 +101,8 @@ class FakeGrokTransport implements GrokAcpTransportLike { #onEvent: ((event: GrokTransportEvent) => void) | null = null; #onPermission: ((request: GrokPermissionRequest) => Promise) | null = null; + #onElicitation: ((request: GrokMcpElicitFormRequest) => Promise) | null = + null; #resolve: ((response: PromptResponse) => void) | null = null; #compactResolve: ((result: GrokCompactResult) => void) | null = null; #compactOnEvent: ((event: GrokTransportEvent) => void) | null = null; @@ -172,12 +183,14 @@ class FakeGrokTransport implements GrokAcpTransportLike { text: string, onEvent: (event: GrokTransportEvent) => void, onPermission: (request: GrokPermissionRequest) => Promise, + onElicitation: (request: GrokMcpElicitFormRequest) => Promise, ): Promise { this.lastPromptText = text; this.#activePromptText = text; this.#activePromptEvents = []; this.#onEvent = onEvent; this.#onPermission = onPermission; + this.#onElicitation = onElicitation; return new Promise((resolve) => { this.#resolve = resolve; }); @@ -226,6 +239,13 @@ class FakeGrokTransport implements GrokAcpTransportLike { }); } + elicit( + request: GrokMcpElicitFormRequest = { ...GROK_CLOCK_ELICIT_FIXTURE, sessionId: this.sessionId }, + ): Promise { + if (!this.#onElicitation) throw new Error("No active Grok Prompt"); + return this.#onElicitation(request); + } + finish(response: PromptResponse = { stopReason: "end_turn" }, historyUsage?: unknown): void { if (this.#activePromptText !== null) { const ordinal = this.replay.filter(({ type }) => type === "turn.completed").length + 1; @@ -298,6 +318,38 @@ async function nextEvent( } describe("Grok Adapter ACP projection", () => { + it("does not spawn a session with stale Computer Use config after an apply failure", async () => { + const root = await mkdtemp(path.join(os.tmpdir(), "grok-cua-apply-")); + try { + const data = path.join(root, "data"); + await mkdir(data); + await writeFile(path.join(data, "grok-computer-use.json"), "invalid-json"); + const createTransport = vi.fn(() => new FakeGrokTransport()); + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + try { + const adapter = new GrokAdapter( + { environment: { CODEXHOST_DATA_DIR: data, GROK_HOME: path.join(root, "grok") } }, + { + randomUUID: () => "grok-apply-failure", + createTransport, + fetchCredits: async () => null, + }, + ); + await expect(adapter.open({ kind: "create", cwd: root })).resolves.toMatchObject({ + ok: false, + error: { code: "unavailable", retryable: true }, + }); + expect(createTransport).not.toHaveBeenCalled(); + expect(warn).toHaveBeenCalledWith("Could not apply Grok Computer Use settings"); + await adapter.close(); + } finally { + warn.mockRestore(); + } + } finally { + await rm(root, { recursive: true, force: true }); + } + }); + it("passes per-Session delegation environment to the ACP transport", async () => { const transport = new FakeGrokTransport(); const createTransport = vi.fn(() => transport); @@ -650,7 +702,8 @@ describe("Grok Adapter ACP projection", () => { expect((await nextEvent(liveIterator)).type).toBe("item.started"); expect((await nextEvent(liveIterator)).type).toBe("item.updated"); liveTransport.finish(); - expect((await nextEvent(liveIterator)).type).toBe("item.completed"); + const liveItem = await nextEvent(liveIterator); + if (liveItem.type !== "item.completed") throw new Error("Expected completed Agent Item"); const completed = await nextEvent(liveIterator); if (completed.type !== "turn.completed" || !completed.nativeTurnRef) { throw new Error("Live Grok Turn has no Native identity"); @@ -665,9 +718,67 @@ describe("Grok Adapter ACP projection", () => { throw new Error("Resumed Grok Snapshot has no Turn"); } expect(snapshot.value.turns[0].nativeTurnRef).toEqual(completed.nativeTurnRef); + expect(snapshot.value.turns[0].items[0]?.item).toEqual(liveItem.snapshot.item); await resumed.adapter.close(); }); + it.each([ + { withMessageIds: true, withTool: false }, + { withMessageIds: true, withTool: true }, + { withMessageIds: false, withTool: true }, + ])( + "keeps streamed text Item identities when history is loaded ($withMessageIds, $withTool)", + async ({ withMessageIds, withTool }) => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + await session.execute({ + type: "turn.start", + turnId: hostTurnIdSchema.parse("turn-live-history"), + input: [{ type: "text", text: "Repeat the answer in two separate messages." }], + }); + const identity = (id: string) => (withMessageIds ? { messageId: id } : {}); + transport.event({ type: "agent.thought", text: "checking", ...identity("a") }); + transport.event({ type: "agent.text", text: "same ", ...identity("a") }); + transport.event({ type: "agent.text", text: "answer", ...identity("a") }); + if (withTool) { + transport.event({ type: "tool.call", callId: "tool", title: "Check", status: "completed" }); + } + transport.event({ type: "agent.text", text: "same answer", ...identity("b") }); + transport.finish(); + if (withTool) { + // Native saved history may omit the optional IDs attached to streamed chunks. + transport.replay = transport.replay.map((event) => { + if (event.type !== "agent.text" && event.type !== "agent.thought") return event; + const saved = { ...event }; + delete saved.messageId; + return saved; + }); + } + + const live: Array<{ itemId: string; type: string; text: string }> = []; + for (;;) { + const event = await nextEvent(iterator); + if (event.type === "item.completed") { + const item = event.snapshot.item; + if (item.type === "agentMessage" || item.type === "reasoning") live.push(item); + } + if (event.type === "turn.completed") break; + } + const snapshot = await session.readSnapshot(); + if (!snapshot.ok || !snapshot.value.turns[0]) throw new Error("Expected history snapshot"); + const history = snapshot.value.turns[0].items + .map(({ item }) => item) + .filter((item) => item.type === "agentMessage" || item.type === "reasoning"); + expect(history).toEqual(live); + // Desktop upserts live completions by ID after hydrating saved history. + const visible = new Map(history.map((item) => [item.itemId, item])); + for (const item of live) visible.set(item.itemId, item); + expect([...visible.values()].filter((item) => item.type === "agentMessage")).toHaveLength(2); + await adapter.close(); + }, + ); + it("omits background-task control records without shifting persisted Turn identities", async () => { const transport = new FakeGrokTransport(); transport.replay = [ @@ -910,6 +1021,208 @@ describe("Grok Adapter ACP projection", () => { await adapter.close(); }); + it("projects Clock MCP elicitation as allow-once/deny and accepts with empty content", async () => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-elicit-clock"); + + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "open clock" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + + const elicit = transport.elicit(); + const output = await nextOutput(iterator); + if (output.kind !== "interaction" || output.interaction.type !== "approval") { + throw new Error("Expected Grok MCP elicitation Approval"); + } + expect(output.interaction.actions).toEqual([ + { id: "allow-once", label: "Allow once", effect: "allowOnce" }, + { id: "deny", label: "Deny", effect: "deny" }, + ]); + expect(output.interaction.title).toContain("cua_repl"); + expect(output.interaction.title).toContain("Clock"); + expect(output.interaction.description).toBe(GROK_CLOCK_ELICIT_FIXTURE.message); + + await expect( + session.execute({ + type: "interaction.respond", + interactionId: output.interaction.interactionId, + response: { type: "approval", actionId: "allow-once" }, + }), + ).resolves.toEqual({ ok: true, value: { accepted: true } }); + await expect(elicit).resolves.toEqual(grokMcpElicitAccept()); + expect((await nextEvent(iterator)).type).toBe("interaction.closed"); + + transport.finish(); + for (;;) { + if ((await nextEvent(iterator)).type === "turn.completed") break; + } + await adapter.close(); + }); + + it("maps Always allow of persist-capable MCP elicitation to scalar persist always", async () => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-elicit-always"); + + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "open clock" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + const elicit = transport.elicit({ + ...GROK_CLOCK_ELICIT_PERSIST_FIXTURE, + sessionId: transport.sessionId, + }); + const output = await nextOutput(iterator); + if (output.kind !== "interaction" || output.interaction.type !== "approval") { + throw new Error("Expected persist-capable Grok MCP elicitation"); + } + expect(output.interaction.actions.map(({ effect }) => effect)).toEqual([ + "allowOnce", + "allowAlways", + "deny", + ]); + await expect( + session.execute({ + type: "interaction.respond", + interactionId: output.interaction.interactionId, + response: { type: "approval", actionId: "allow-always" }, + }), + ).resolves.toEqual({ ok: true, value: { accepted: true } }); + await expect(elicit).resolves.toEqual(grokMcpElicitAccept("always")); + transport.finish(); + for (;;) { + if ((await nextEvent(iterator)).type === "turn.completed") break; + } + await adapter.close(); + }); + + it("maps Desktop deny of MCP elicitation to native decline", async () => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-elicit-deny"); + + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "open clock" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + const elicit = transport.elicit(); + const output = await nextOutput(iterator); + if (output.kind !== "interaction") throw new Error("Expected Grok MCP elicitation Approval"); + await session.execute({ + type: "interaction.respond", + interactionId: output.interaction.interactionId, + response: { type: "approval", actionId: "deny" }, + }); + await expect(elicit).resolves.toEqual(grokMcpElicitDecline()); + transport.finish(); + for (;;) { + if ((await nextEvent(iterator)).type === "turn.completed") break; + } + await adapter.close(); + }); + + it("cancels MCP elicitation on native interaction_resolved and ignores a late Desktop answer", async () => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-elicit-withdraw"); + + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "open clock" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + const elicit = transport.elicit({ + ...GROK_CLOCK_ELICIT_FIXTURE, + sessionId: transport.sessionId, + toolCallId: "elicit-clock-withdraw", + }); + const output = await nextOutput(iterator); + if (output.kind !== "interaction") throw new Error("Expected Grok MCP elicitation Approval"); + transport.event({ type: "elicitation.resolved", toolCallId: "elicit-clock-withdraw" }); + await expect(elicit).resolves.toEqual(grokMcpElicitCancel()); + expect((await nextEvent(iterator)).type).toBe("interaction.closed"); + await expect( + session.execute({ + type: "interaction.respond", + interactionId: output.interaction.interactionId, + response: { type: "approval", actionId: "allow-once" }, + }), + ).resolves.toMatchObject({ ok: false }); + transport.finish(); + for (;;) { + if ((await nextEvent(iterator)).type === "turn.completed") break; + } + await adapter.close(); + }); + + it("keeps MCP elicitation distinct from tool permission response shape", async () => { + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-elicit-vs-permission"); + + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "run" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + const elicit = transport.elicit(); + const elicitOutput = await nextOutput(iterator); + if (elicitOutput.kind !== "interaction") throw new Error("Expected elicitation"); + expect(elicitOutput.interaction).toMatchObject({ + type: "approval", + actions: [ + { id: "allow-once", effect: "allowOnce" }, + { id: "deny", effect: "deny" }, + ], + }); + const permission = transport.permission(); + const permissionOutput = await nextOutput(iterator); + if ( + permissionOutput.kind !== "interaction" || + permissionOutput.interaction.type !== "approval" + ) { + throw new Error("Expected tool permission"); + } + expect(permissionOutput.interaction.actions.map(({ effect }) => effect)).toEqual([ + "allowOnce", + "deny", + ]); + await session.execute({ + type: "interaction.respond", + interactionId: elicitOutput.interaction.interactionId, + response: { type: "approval", actionId: "allow-once" }, + }); + await expect(elicit).resolves.toEqual(grokMcpElicitAccept()); + await session.execute({ + type: "interaction.respond", + interactionId: permissionOutput.interaction.interactionId, + response: { type: "approval", actionId: "allow-once" }, + }); + await expect(permission).resolves.toEqual({ + outcome: { outcome: "selected", optionId: "native-allow" }, + }); + transport.finish(); + for (;;) { + if ((await nextEvent(iterator)).type === "turn.completed") break; + } + await adapter.close(); + }); + it("projects Generic Tool arguments and readable results, including raw_output-only tools", async () => { const transport = new FakeGrokTransport(); const { adapter, session } = await openedSession(transport); @@ -1129,6 +1442,68 @@ describe("Grok Adapter ACP projection", () => { await adapter.close(); }); + it("projects ImageGen files as generatedImage items and restores them from history", async () => { + const imageRoot = await mkdtemp(path.join(os.tmpdir(), "codexhost-grok-image-")); + const imagePath = path.join(imageRoot, "1.jpg"); + await writeFile(imagePath, "jpeg"); + const transport = new FakeGrokTransport(); + const { adapter, session } = await openedSession(transport); + const iterator = session.outputs[Symbol.asyncIterator](); + const turnId = hostTurnIdSchema.parse("turn-generated-image"); + try { + await session.execute({ + type: "turn.start", + turnId, + input: [{ type: "text", text: "draw" }], + }); + expect((await nextEvent(iterator)).type).toBe("turn.started"); + transport.event({ + type: "tool.call", + callId: "img-1", + title: "image_gen", + name: "image_gen", + status: "in_progress", + rawInput: { prompt: "cat" }, + }); + expect((await nextEvent(iterator)).type).toBe("item.started"); + transport.event({ + type: "tool.update", + callId: "img-1", + status: "completed", + rawOutput: { + type: "ImageGen", + path: imagePath, + filename: "1.jpg", + session_folder: "images", + }, + }); + expect(await nextEvent(iterator)).toMatchObject({ + type: "item.completed", + snapshot: { item: { type: "toolExecution" }, outcome: { status: "succeeded" } }, + }); + expect(await nextEvent(iterator)).toMatchObject({ + type: "item.started", + item: { + type: "generatedImage", + savedPath: imagePath, + revisedPrompt: "1.jpg", + }, + }); + expect(await nextEvent(iterator)).toMatchObject({ + type: "item.completed", + snapshot: { item: { type: "generatedImage" }, outcome: { status: "succeeded" } }, + }); + transport.finish(); + expect(await nextEvent(iterator)).toMatchObject({ + type: "turn.completed", + outcome: { status: "succeeded" }, + }); + } finally { + await adapter.close(); + await rm(imageRoot, { recursive: true, force: true }); + } + }); + it("keeps failed terminal ACP Diff Content Tool-only", async () => { const transport = new FakeGrokTransport(); const { adapter, session } = await openedSession(transport); diff --git a/packages/adapters/grok/test/grok-codexhost-session-rules.test.ts b/packages/adapters/grok/test/grok-codexhost-session-rules.test.ts new file mode 100644 index 000000000..754418f32 --- /dev/null +++ b/packages/adapters/grok/test/grok-codexhost-session-rules.test.ts @@ -0,0 +1,27 @@ +import { describe, expect, it } from "vitest"; + +import { + GROK_CODEXHOST_SESSION_RULES, + grokCodexhostSessionMeta, +} from "../src/grok-codexhost-session-rules.js"; + +describe("Grok CodexHost ACP session rules", () => { + it("uses official session/new _meta.rules and keeps secrets out of the prompt text", () => { + expect(GROK_CODEXHOST_SESSION_RULES).toContain("CodexHost started this Grok ACP session"); + expect(GROK_CODEXHOST_SESSION_RULES).toContain("Never print"); + expect(GROK_CODEXHOST_SESSION_RULES).toContain("flash-max"); + expect(GROK_CODEXHOST_SESSION_RULES).toContain("thread/realtime/*"); + expect(GROK_CODEXHOST_SESSION_RULES).toContain("AGENT_TOOLS_AND_RULES.md"); + expect(GROK_CODEXHOST_SESSION_RULES).not.toContain("echo $CODEXHOST_THREAD_ID"); + expect(GROK_CODEXHOST_SESSION_RULES).not.toContain("~/.grok/rules"); + }); + + it("adds rules to create metadata and keeps Permission Mode flags", () => { + expect(grokCodexhostSessionMeta("ask")).toEqual({ + yoloMode: false, + autoMode: false, + rules: GROK_CODEXHOST_SESSION_RULES, + }); + expect(grokCodexhostSessionMeta()).toEqual({ rules: GROK_CODEXHOST_SESSION_RULES }); + }); +}); diff --git a/packages/adapters/grok/test/grok-elicitation.test.ts b/packages/adapters/grok/test/grok-elicitation.test.ts new file mode 100644 index 000000000..5c6554133 --- /dev/null +++ b/packages/adapters/grok/test/grok-elicitation.test.ts @@ -0,0 +1,113 @@ +import { describe, expect, it } from "vitest"; +import { hostInteractionIdSchema, hostTurnIdSchema } from "@codexhost/shared-contracts"; + +import { + GROK_CLOCK_ELICIT_FIXTURE, + GROK_CLOCK_ELICIT_PERSIST_FIXTURE, + grokMcpElicitAccept, + grokMcpElicitCancel, + grokMcpElicitDecline, + grokMcpElicitResponseForAction, + parseGrokMcpElicitParams, + projectGrokMcpElicitApproval, +} from "../src/grok-elicitation.js"; + +describe("Grok MCP elicitation parser", () => { + it("accepts the sanitized Clock form and an empty object schema", () => { + expect(parseGrokMcpElicitParams(GROK_CLOCK_ELICIT_FIXTURE)).toEqual({ + status: "form", + request: GROK_CLOCK_ELICIT_FIXTURE, + }); + expect( + parseGrokMcpElicitParams({ + ...GROK_CLOCK_ELICIT_FIXTURE, + requestedSchema: { type: "object", properties: {} }, + }), + ).toMatchObject({ status: "form" }); + }); + + it("cancels URL elicitation and unreviewed non-empty schemas without treating them as forms", () => { + expect( + parseGrokMcpElicitParams({ + sessionId: "s", + toolCallId: "t", + serverName: "cua_repl", + message: "Open a URL", + mode: "url", + url: "https://example.invalid/consent", + elicitationId: "e1", + }), + ).toEqual({ status: "unsupported" }); + expect( + parseGrokMcpElicitParams({ + ...GROK_CLOCK_ELICIT_FIXTURE, + requestedSchema: { + type: "object", + properties: { reason: { type: "string" } }, + required: ["reason"], + }, + }), + ).toEqual({ status: "unsupported" }); + }); + + it("rejects malformed params and unreviewed top-level fields", () => { + expect(parseGrokMcpElicitParams(null)).toMatchObject({ status: "invalid" }); + expect(parseGrokMcpElicitParams({ ...GROK_CLOCK_ELICIT_FIXTURE, mode: "other" })).toMatchObject( + { + status: "invalid", + }, + ); + expect( + parseGrokMcpElicitParams({ ...GROK_CLOCK_ELICIT_FIXTURE, persist: "always" }), + ).toMatchObject({ status: "invalid" }); + expect( + parseGrokMcpElicitParams({ ...GROK_CLOCK_ELICIT_FIXTURE, requestedSchema: "object" }), + ).toMatchObject({ status: "invalid" }); + }); + + it("projects allow-once/deny and maps Host actions to flat outcome replies", () => { + const interaction = projectGrokMcpElicitApproval({ + interactionId: hostInteractionIdSchema.parse("interaction-1"), + turnId: hostTurnIdSchema.parse("turn-1"), + request: GROK_CLOCK_ELICIT_FIXTURE, + }); + expect(interaction.actions.map(({ effect }) => effect)).toEqual(["allowOnce", "deny"]); + expect(interaction.title).toContain("cua_repl"); + expect(grokMcpElicitResponseForAction("allowOnce")).toEqual(grokMcpElicitAccept()); + expect(grokMcpElicitResponseForAction("deny")).toEqual(grokMcpElicitDecline()); + expect(grokMcpElicitResponseForAction("cancel")).toEqual(grokMcpElicitCancel()); + }); + + it("offers Always only when persist includes always and maps it to a scalar accept", () => { + expect(parseGrokMcpElicitParams(GROK_CLOCK_ELICIT_PERSIST_FIXTURE)).toEqual({ + status: "form", + request: { ...GROK_CLOCK_ELICIT_PERSIST_FIXTURE, persist: ["session", "always"] }, + }); + expect( + parseGrokMcpElicitParams({ ...GROK_CLOCK_ELICIT_FIXTURE, persist: ["always", "session"] }), + ).toMatchObject({ + status: "form", + request: { persist: ["session", "always"] }, + }); + expect( + parseGrokMcpElicitParams({ ...GROK_CLOCK_ELICIT_FIXTURE, persist: ["session"] }), + ).toMatchObject({ status: "form", request: { persist: ["session"] } }); + const withAlways = projectGrokMcpElicitApproval({ + interactionId: hostInteractionIdSchema.parse("interaction-2"), + turnId: hostTurnIdSchema.parse("turn-2"), + request: GROK_CLOCK_ELICIT_PERSIST_FIXTURE, + }); + expect(withAlways.actions.map(({ effect }) => effect)).toEqual([ + "allowOnce", + "allowAlways", + "deny", + ]); + const sessionOnly = projectGrokMcpElicitApproval({ + interactionId: hostInteractionIdSchema.parse("interaction-3"), + turnId: hostTurnIdSchema.parse("turn-3"), + request: { ...GROK_CLOCK_ELICIT_FIXTURE, persist: ["session"] }, + }); + expect(sessionOnly.actions.map(({ effect }) => effect)).toEqual(["allowOnce", "deny"]); + expect(grokMcpElicitResponseForAction("allowAlways")).toEqual(grokMcpElicitAccept("always")); + }); +}); diff --git a/packages/adapters/grok/test/grok-generated-image.test.ts b/packages/adapters/grok/test/grok-generated-image.test.ts new file mode 100644 index 000000000..679f9cdd7 --- /dev/null +++ b/packages/adapters/grok/test/grok-generated-image.test.ts @@ -0,0 +1,79 @@ +import { describe, expect, it } from "vitest"; + +import { projectGrokGeneratedImage } from "../src/grok-generated-image.js"; + +const saved = "/Users/karlchow/.grok/sessions/session-1/images/1.jpg"; +const exists = (absolutePath: string) => absolutePath === saved; + +describe("Grok generated image projection", () => { + it("reads ImageGen rawOutput with an existing absolute path", () => { + expect( + projectGrokGeneratedImage( + undefined, + { + type: "ImageGen", + path: saved, + filename: "1.jpg", + session_folder: "images", + }, + exists, + ), + ).toEqual({ savedPath: saved, revisedPrompt: "1.jpg" }); + }); + + it("reads ImageEdit rawOutput the same way", () => { + expect( + projectGrokGeneratedImage( + undefined, + { type: "ImageEdit", path: saved, filename: "1.jpg" }, + exists, + ), + ).toEqual({ savedPath: saved, revisedPrompt: "1.jpg" }); + }); + + it("reads ACP JSON text that uses session_folder images without a type field", () => { + expect( + projectGrokGeneratedImage( + [ + { + type: "content", + content: { + type: "text", + text: JSON.stringify({ + path: saved, + filename: "1.jpg", + session_folder: "images", + message: "Image generated", + }), + }, + }, + ], + undefined, + exists, + ), + ).toEqual({ savedPath: saved, revisedPrompt: "1.jpg" }); + }); + + it("ignores missing files, relative paths, and unrelated JSON", () => { + expect( + projectGrokGeneratedImage( + undefined, + { type: "ImageGen", path: saved, filename: "1.jpg" }, + () => false, + ), + ).toBeNull(); + expect( + projectGrokGeneratedImage( + undefined, + { type: "ImageGen", path: "images/1.jpg", filename: "1.jpg" }, + exists, + ), + ).toBeNull(); + expect( + projectGrokGeneratedImage(undefined, { type: "ReadFile", path: saved }, exists), + ).toBeNull(); + expect( + projectGrokGeneratedImage([{ type: "diff", path: saved }], undefined, exists), + ).toBeNull(); + }); +}); diff --git a/packages/adapters/grok/test/grok-history.test.ts b/packages/adapters/grok/test/grok-history.test.ts index a3230ebe8..f814161ef 100644 --- a/packages/adapters/grok/test/grok-history.test.ts +++ b/packages/adapters/grok/test/grok-history.test.ts @@ -397,3 +397,57 @@ describe("Grok history local media Markdown", () => { } }); }); + +describe("Grok history generated images", () => { + it("restores ImageGen files as generatedImage items", async () => { + const sessionDirectory = await mkdtemp(path.join(os.tmpdir(), "codexhost-grok-image-")); + const imagePath = path.join(sessionDirectory, "images", "1.jpg"); + try { + await mkdir(path.dirname(imagePath), { recursive: true }); + await writeFile(imagePath, "jpeg"); + const snapshot = mapGrokReplay( + [ + { type: "user.text", text: "draw a cat" }, + { + type: "tool.call", + callId: "img-1", + title: "image_gen", + name: "image_gen", + rawInput: { prompt: "cat" }, + }, + { + type: "tool.update", + callId: "img-1", + status: "completed", + rawOutput: { + type: "ImageGen", + path: imagePath, + filename: "1.jpg", + session_folder: "images", + }, + }, + { type: "turn.completed", nativeTurnKey: "prompt-1", stopReason: "end_turn" }, + ], + grokHarnessId, + "session-1", + sessionDirectory, + ); + expect(snapshot.turns[0]?.items).toMatchObject([ + { + item: { type: "toolExecution", toolName: "image_gen" }, + outcome: { status: "succeeded" }, + }, + { + item: { + type: "generatedImage", + savedPath: imagePath, + revisedPrompt: "1.jpg", + }, + outcome: { status: "succeeded" }, + }, + ]); + } finally { + await rm(sessionDirectory, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/adapters/grok/test/installation.test.ts b/packages/adapters/grok/test/installation.test.ts index ee318e957..26e9b29a5 100644 --- a/packages/adapters/grok/test/installation.test.ts +++ b/packages/adapters/grok/test/installation.test.ts @@ -1,18 +1,58 @@ -import { describe, expect, it, vi } from "vitest"; -const mocks = vi.hoisted(() => ({ run: vi.fn(), resolve: vi.fn() })); +import { beforeEach, describe, expect, it, vi } from "vitest"; +const mocks = vi.hoisted(() => ({ run: vi.fn(), resolve: vi.fn(), launcher: vi.fn() })); vi.mock("@codexhost/harness-discovery", async (original) => ({ ...(await original()), runInstallationCommand: mocks.run, + readLauncherScript: mocks.launcher, })); vi.mock("../src/command.js", () => ({ resolveGrokExecutable: mocks.resolve })); import { createGrokInstallation } from "../src/installation.js"; +beforeEach(() => { + vi.resetAllMocks(); + mocks.resolve.mockReturnValue("/chosen/grok"); + mocks.launcher.mockResolvedValue(""); +}); + describe("Grok native version maintenance", () => { + it.each(["check", "update"] as const)( + "keeps grokgod %s read-only and reports patch-manager ownership", + async (action) => { + mocks.launcher.mockResolvedValue( + '#!/bin/sh\nGROKGOD_HOME="${GROKGOD_HOME:-$HOME/.grokgod}"\nGROKGOD_BIN="$GROKGOD_HOME/bin/grok"', + ); + mocks.run.mockResolvedValue("grok 1.0.6 (patched)"); + const result = await createGrokInstallation({ HOME: "/user" })(action); + expect(result).toMatchObject({ + currentVersion: "1.0.6", + latestVersion: "Unknown", + latestVersionKind: "unknown", + canUpdate: false, + updateAvailable: false, + notice: { en: expect.stringContaining("Managed by grokgod") }, + }); + expect(mocks.run).toHaveBeenCalledExactlyOnceWith( + "/chosen/grok", + ["--version"], + { + HOME: "/user", + GROK_DISABLE_AUTOUPDATER: "1", + GROKGOD_UPDATE_CHECK_DISABLE: "1", + }, + undefined, + ); + }, + ); + + it("does not attempt a stock update check if the launcher cannot be read", async () => { + mocks.launcher.mockRejectedValue(new Error("unreadable launcher")); + await expect(createGrokInstallation({})("check")).rejects.toThrow("unreadable launcher"); + expect(mocks.run).not.toHaveBeenCalled(); + }); + it.each(["native", "npm"])( "preserves the %s installer and verifies version after updating", async (installer) => { - mocks.run.mockClear(); - mocks.resolve.mockReturnValue("/chosen/grok"); let updated = false; mocks.run.mockImplementation(async (_command, args) => { if (!args.includes("--check")) { diff --git a/packages/adapters/grok/tsconfig.json b/packages/adapters/grok/tsconfig.json index 263d6e47d..59db295ed 100644 --- a/packages/adapters/grok/tsconfig.json +++ b/packages/adapters/grok/tsconfig.json @@ -16,6 +16,9 @@ }, { "path": "../../harness-discovery/tsconfig.json" + }, + { + "path": "../../harness-plugin-files/tsconfig.json" } ], "include": ["src/**/*.ts"] diff --git a/packages/console-server/src/announcement.ts b/packages/console-server/src/announcement.ts index 4a8df2b4c..141631dd2 100644 --- a/packages/console-server/src/announcement.ts +++ b/packages/console-server/src/announcement.ts @@ -1,7 +1,6 @@ import { consoleAnnouncementSchema, type ConsoleAnnouncement } from "@codexhost/shared-contracts"; -const ANNOUNCEMENT_URL = - "https://raw.githubusercontent.com/BytePioneer-AI/codex-host/main/docs/NOTICE.md"; +const ANNOUNCEMENT_URL = "https://raw.githubusercontent.com/karlorz/codex-host/main/docs/NOTICE.md"; const MAX_DOCUMENT_BYTES = 16 * 1024; const TIMEOUT_MS = 5_000; diff --git a/packages/console-server/src/diagnostic-report.ts b/packages/console-server/src/diagnostic-report.ts index a77e914d0..bdd227d86 100644 --- a/packages/console-server/src/diagnostic-report.ts +++ b/packages/console-server/src/diagnostic-report.ts @@ -8,7 +8,7 @@ import type { } from "./diagnostics.js"; import type { InspectDocument } from "./installation.js"; -export const ISSUE_REPOSITORY_URL = "https://github.com/BytePioneer-AI/codex-host"; +export const ISSUE_REPOSITORY_URL = "https://github.com/karlorz/codex-host"; const ISSUE_ERROR_CHARS = 800; const ISSUE_URL_MAX_LENGTH = 7_000; export const REPORT_LOG_TAIL_BYTES = 32 * 1024; diff --git a/packages/console-server/src/diagnostics.ts b/packages/console-server/src/diagnostics.ts index 01755cbc3..618f143e8 100644 --- a/packages/console-server/src/diagnostics.ts +++ b/packages/console-server/src/diagnostics.ts @@ -35,6 +35,8 @@ export interface RendererIntegrationStatus { export interface ControllerStatus { pid: number; startedAt: number; + voiceInjection?: boolean; + voiceEnvOverride?: "0" | "1" | null; renderer: RendererIntegrationStatus; } @@ -125,6 +127,14 @@ export async function readControllerStatus(filePath: string): Promise; setLaunchPath(id: string, value: string | null): Promise; + setGrokComputerUse(update: GrokComputerUseSet): Promise; } /** @@ -93,6 +98,9 @@ export function createConsoleHarnesses( links: manifest.links ?? null, icon, launchPath, + ...(manifest.id === "grok" + ? { computerUse: await getGrokComputerUseSettings(environment) } + : {}), }); } return harnesses.sort( @@ -115,5 +123,15 @@ export function createConsoleHarnesses( throw new ConsoleHarnessError(400, error instanceof Error ? error.message : String(error)); } }, + async setGrokComputerUse(update) { + const harness = (await list()).find((candidate) => candidate.id === "grok"); + if (!harness) throw new ConsoleHarnessError(404, "Harness 'grok' is not installed"); + try { + const computerUse = await setGrokComputerUse(update, environment); + return { ...harness, computerUse }; + } catch (error) { + throw new ConsoleHarnessError(400, error instanceof Error ? error.message : String(error)); + } + }, }; } diff --git a/packages/console-server/src/host-client.ts b/packages/console-server/src/host-client.ts index 071f4a080..94e2e8aad 100644 --- a/packages/console-server/src/host-client.ts +++ b/packages/console-server/src/host-client.ts @@ -1,3 +1,4 @@ +import { VOICE_GATEWAY_STATUS_METHOD } from "@codexhost/shared-contracts"; import { readdir, readFile, rm } from "node:fs/promises"; import path from "node:path"; @@ -91,7 +92,11 @@ export function createConsoleHostClient( headers: { authorization: `Bearer ${host.token}`, "content-type": "application/json" }, body: JSON.stringify({ method, params: params ?? {} }), signal: AbortSignal.timeout( - method === "codexhost/remote/ssh-setup" ? Math.max(timeoutMs, 330_000) : timeoutMs, + method === VOICE_GATEWAY_STATUS_METHOD + ? Math.min(timeoutMs, 2_000) + : method === "codexhost/remote/ssh-setup" + ? Math.max(timeoutMs, 330_000) + : timeoutMs, ), }); const body = (await response.json().catch(() => null)) as HostReply | null; diff --git a/packages/console-server/src/installation.ts b/packages/console-server/src/installation.ts index 812aefae6..245ec9f19 100644 --- a/packages/console-server/src/installation.ts +++ b/packages/console-server/src/installation.ts @@ -122,16 +122,17 @@ export interface LaunchCommand { export function launchCommand( installation: ConsoleInstallation, environment: NodeJS.ProcessEnv = process.env, + action: "launch" | "restart" = "launch", ): LaunchCommand | null { if (installation.distribution?.distribution === "npm") { const node = environment.CODEXHOST_NPM_NODE_PATH; const wrapper = environment.CODEXHOST_NPM_LAUNCHER_PATH; if (node && wrapper && path.isAbsolute(node) && path.isAbsolute(wrapper)) { - return { command: node, args: [wrapper] }; + return { command: node, args: [wrapper, ...(action === "restart" ? [action] : [])] }; } return null; } return installation.launcherExecutable - ? { command: installation.launcherExecutable, args: ["launch"] } + ? { command: installation.launcherExecutable, args: [action] } : null; } diff --git a/packages/console-server/src/live-voice-inject.ts b/packages/console-server/src/live-voice-inject.ts new file mode 100644 index 000000000..e0af9e5c2 --- /dev/null +++ b/packages/console-server/src/live-voice-inject.ts @@ -0,0 +1,69 @@ +import { randomUUID } from "node:crypto"; +import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"; +import path from "node:path"; +import { voiceRoutesSchema, type VoiceRoutes } from "@codexhost/shared-contracts"; + +const CPA_LIVE_VOICE_ENV = "CODEXHOST_CPA_LIVE_VOICE"; +export const LIVE_VOICE_INJECT_FILE = "live-voice-inject-v1.json"; + +export interface LiveVoiceInjectState { + inject: boolean; + savedInject: boolean; + envOverride: "0" | "1" | null; +} + +function injectPath(dataDirectory: string): string { + return path.join(dataDirectory, LIVE_VOICE_INJECT_FILE); +} + +export async function readLiveVoiceInjectState( + dataDirectory: string, + environment: NodeJS.ProcessEnv = process.env, +): Promise { + const env = environment[CPA_LIVE_VOICE_ENV]; + const envOverride = env === "0" || env === "1" ? env : null; + let inject = true; + try { + const parsed = JSON.parse(await readFile(injectPath(dataDirectory), "utf8")) as { + version?: unknown; + inject?: unknown; + }; + inject = parsed.version !== 1 || parsed.inject !== false; + } catch { + inject = true; + } + const savedInject = inject; + if (envOverride === "0") inject = false; + if (envOverride === "1") inject = true; + return { inject, savedInject, envOverride }; +} + +export async function writeLiveVoiceInjectFile( + dataDirectory: string, + inject: boolean, + routes?: VoiceRoutes, +): Promise { + if (!routes) { + try { + const saved = JSON.parse(await readFile(injectPath(dataDirectory), "utf8")); + if (saved?.version === 1) routes = voiceRoutesSchema.safeParse(saved.routes).data; + } catch { + /* Missing or malformed settings use the injection default. */ + } + } + await mkdir(dataDirectory, { recursive: true, mode: 0o700 }); + const temporaryPath = path.join(dataDirectory, `.${LIVE_VOICE_INJECT_FILE}.${randomUUID()}.tmp`); + try { + await writeFile( + temporaryPath, + `${JSON.stringify({ version: 1, inject, ...(routes ? { routes } : {}) }, null, 2)}\n`, + { + mode: 0o600, + flag: "wx", + }, + ); + await rename(temporaryPath, injectPath(dataDirectory)); + } finally { + await rm(temporaryPath, { force: true }); + } +} diff --git a/packages/console-server/src/restart.ts b/packages/console-server/src/restart.ts new file mode 100644 index 000000000..3b2ee7fd3 --- /dev/null +++ b/packages/console-server/src/restart.ts @@ -0,0 +1,58 @@ +export interface RestartStatus { + phase: "idle" | "restarting" | "succeeded" | "failed"; + error: string | null; +} + +export interface RestartObservation { + launcherPid: number | null; + controllerPid: number | null; + installed: boolean; + failure: string | null; +} + +/** The Launcher owns shutdown. Completion requires a different ready Controller. */ +export class ConsoleRestart { + status: RestartStatus = { phase: "idle", error: null }; + constructor( + private readonly run: () => Promise, + private readonly observe: () => Promise, + private readonly sleep = (ms: number) => + new Promise((resolve) => setTimeout(resolve, ms)), + private readonly now = Date.now, + private readonly timeoutMs = 120_000, + ) {} + + start(previous: RestartObservation): void { + if (this.status.phase === "restarting") throw new Error("Restart is already in progress"); + this.status = { phase: "restarting", error: null }; + void this.restart(previous); + } + + private async restart(previous: RestartObservation): Promise { + try { + await this.run(); + const deadline = this.now() + this.timeoutMs; + while (this.now() < deadline) { + const current = await this.observe(); + if (current.failure) throw new Error(current.failure); + if ( + current.launcherPid !== null && + current.launcherPid !== previous.launcherPid && + current.controllerPid !== null && + current.controllerPid !== previous.controllerPid && + current.installed + ) { + this.status = { phase: "succeeded", error: null }; + return; + } + await this.sleep(250); + } + throw new Error("Restart timed out. Check startup diagnostics before retrying."); + } catch (error) { + this.status = { + phase: "failed", + error: error instanceof Error ? error.message : String(error), + }; + } + } +} diff --git a/packages/console-server/src/server.ts b/packages/console-server/src/server.ts index 4024b0f33..cc78d97bf 100644 --- a/packages/console-server/src/server.ts +++ b/packages/console-server/src/server.ts @@ -1,7 +1,14 @@ +import { ConsoleRestart } from "./restart.js"; import { spawn } from "node:child_process"; import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; -import { isConsoleHostMethod, type ConsoleAnnouncement } from "@codexhost/shared-contracts"; +import { + isConsoleHostMethod, + grokComputerUseSetSchema, + voiceSettingsRequestSchema, + VOICE_GATEWAY_STATUS_METHOD, + type ConsoleAnnouncement, +} from "@codexhost/shared-contracts"; import { readAnnouncement } from "./announcement.js"; import { allowedChange, allowedHost, CONSOLE_REQUEST_HEADER } from "./request-guard.js"; import { @@ -24,6 +31,9 @@ import { HostUnavailableError, type ConsoleHostClient } from "./host-client.js"; import { CONSOLE_PAGE_CSS, CONSOLE_PAGE_HTML } from "./page.js"; import type { ConsolePaths } from "./paths.js"; import { ConsoleUpdateError, type ConsoleUpdates } from "./updates.js"; +import { readLiveVoiceInjectState } from "./live-voice-inject.js"; + +import { VoiceSettingsStore, VoiceSettingsConflict } from "./voice-settings.js"; export const CONSOLE_SERVICE = "codexhost-console"; const MAX_BODY_BYTES = 16 * 1024; @@ -48,6 +58,7 @@ export interface ConsoleServerOptions { idleTimeoutMs?: number; inspect?(launcherExecutable: string): Promise; launch?(command: string, args: string[]): void; + restart?(command: string, args: string[]): Promise; onExit(): void; } @@ -121,11 +132,47 @@ function defaultLaunch(command: string, args: string[]): void { child.unref(); } +function defaultRestart(command: string, args: string[]): Promise { + return new Promise((resolve, reject) => { + const child = spawn(command, args, { detached: true, stdio: "ignore", windowsHide: true }); + child.once("error", () => reject(new Error("Could not start the installed restart command"))); + child.once("spawn", () => { + child.unref(); + resolve(); + }); + }); +} + export function startConsoleServer(options: ConsoleServerOptions): Promise { const environment = options.environment ?? process.env; const inspect = options.inspect ?? inspectInstallation; const launch = options.launch ?? defaultLaunch; const loadAnnouncement = options.announcement ?? readAnnouncement; + const voiceSettings = new VoiceSettingsStore(options.paths.dataDirectory); + let restartRequestedAt = 0; + const restart = new ConsoleRestart( + async () => { + restartRequestedAt = Date.now(); + const command = launchCommand(options.installation, environment, "restart"); + if (!command) throw new Error("This installation cannot be restarted from the console"); + await (options.restart ?? defaultRestart)(command.command, command.args); + }, + async () => { + inspectCache = null; + const collected = await collect(); + const latest = collected.startup[0]; + return { + launcherPid: collected.inspectDocument?.runtime.launcherPid ?? null, + controllerPid: collected.controllerAlive ? (collected.controller?.pid ?? null) : null, + installed: + collected.controllerAlive && collected.controller?.renderer.state === "installed", + failure: + latest?.outcome === "failed" && latest.startedAtMs >= restartRequestedAt + ? (latest.error ?? "Desktop startup failed") + : null, + }; + }, + ); // Updated with the bound port; tests may listen on port 0. let port = options.port; let inspectCache: { at: number; value: Promise } | null = null; @@ -216,6 +263,17 @@ export function startConsoleServer(options: ConsoleServerOptions): Promise = Promise.resolve(); + readonly #runtime = new Map(); + constructor(private readonly directory: string) {} + + async read() { + let document: { version?: unknown; inject?: unknown; routes?: unknown } | null = null; + try { + document = JSON.parse( + await readFile(path.join(this.directory, LIVE_VOICE_INJECT_FILE), "utf8"), + ); + } catch (error) { + if (!(error instanceof SyntaxError) && (error as NodeJS.ErrnoException).code !== "ENOENT") + throw new Error("Could not read voice settings"); + } + const routes = + document?.version === 1 ? voiceRoutesSchema.safeParse(document.routes).data : undefined; + const settings: VoiceSettings = { + ...(routes ?? DEFAULT_VOICE_ROUTES), + inject: document?.version !== 1 || document.inject !== false, + }; + const revision = createHash("sha256") + .update(JSON.stringify({ settings, initialized: !!routes })) + .digest("hex"); + const runtime = [...this.#runtime.values()].filter((entry) => Date.now() - entry.at < 12_000); + return { + settings, + initialized: !!routes, + revision, + activeCall: runtime.length ? runtime.some((entry) => entry.activeCall) : null, + routesActive: runtime.length ? runtime.every((entry) => entry.revision === revision) : null, + }; + } + + report(runtime: { id: string; revision: string; activeCall: boolean }): void { + for (const [id, entry] of this.#runtime) + if (Date.now() - entry.at >= 12_000) this.#runtime.delete(id); + if (this.#runtime.size < 32 || this.#runtime.has(runtime.id)) + this.#runtime.set(runtime.id, { ...runtime, at: Date.now() }); + } + + async setInjection(inject: boolean): Promise { + const operation = this.#writes.then(async () => { + const current = await this.read(); + if (current.activeCall) + throw new VoiceSettingsConflict("End your voice call before applying changes."); + await writeLiveVoiceInjectFile(this.directory, inject); + }); + this.#writes = operation.catch(() => undefined); + await operation; + } + + async write(input: Extract) { + const operation = this.#writes.then(async () => { + const current = await this.read(); + if (input.operation === "migrate" && current.initialized) return current; + if (input.operation === "set") { + if (current.activeCall) + throw new VoiceSettingsConflict("End your voice call before applying changes."); + if (input.revision !== current.revision) + throw new VoiceSettingsConflict( + "Voice settings changed in another window. Reload and try again.", + ); + } + const settings = + input.operation === "set" ? input.settings : { ...current.settings, ...input.routes }; + await writeLiveVoiceInjectFile(this.directory, settings.inject, { + chatExperience: settings.chatExperience, + chatConnection: settings.chatConnection, + workConnection: settings.workConnection, + }); + return this.read(); + }); + this.#writes = operation.catch(() => undefined); + return operation; + } +} diff --git a/packages/console-server/test/announcement.test.ts b/packages/console-server/test/announcement.test.ts new file mode 100644 index 000000000..9cc2d60ff --- /dev/null +++ b/packages/console-server/test/announcement.test.ts @@ -0,0 +1,30 @@ +import { describe, expect, it, vi } from "vitest"; + +import { readAnnouncement } from "../src/announcement.js"; + +describe("console announcement source", () => { + it("reads the fork NOTICE.md through the announcement fetch path", async () => { + const fetcher = vi.fn( + async () => + new Response("---\nenabled: true\ntitle: Fork update\ntype: info\n---\nFork release notes"), + ); + + await expect(readAnnouncement(fetcher)).resolves.toEqual({ + title: "Fork update", + type: "info", + body: "Fork release notes", + }); + expect(fetcher).toHaveBeenCalledOnce(); + expect(fetcher).toHaveBeenCalledWith( + "https://raw.githubusercontent.com/karlorz/codex-host/main/docs/NOTICE.md", + expect.objectContaining({ redirect: "error", cache: "no-store" }), + ); + }); + + it("hides an unavailable fork announcement without retrying", async () => { + const fetcher = vi.fn(async () => new Response(null, { status: 404 })); + + await expect(readAnnouncement(fetcher)).resolves.toBeNull(); + expect(fetcher).toHaveBeenCalledOnce(); + }); +}); diff --git a/packages/console-server/test/harnesses.test.ts b/packages/console-server/test/harnesses.test.ts index 896cb6e85..a7b8a9cf5 100644 --- a/packages/console-server/test/harnesses.test.ts +++ b/packages/console-server/test/harnesses.test.ts @@ -84,4 +84,33 @@ describe("console Harnesses", () => { status: 400, }); }); + + it("toggles Grok Computer Use on the grok plugin", async () => { + const pluginsRoot = path.join(root, "plugins"); + await plugin(pluginsRoot, "grok", false); + await writeFile( + path.join(pluginsRoot, "enabled.json"), + JSON.stringify({ version: 1, enabled: ["grok"] }), + ); + const grokHome = path.join(root, "grok-home"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: path.join(root, "missing-chatgpt.app"), + }; + const harnesses = createConsoleHarnesses([pluginsRoot], environment); + const listed = await harnesses.list(); + expect(listed[0]?.computerUse).toMatchObject({ enabled: false, available: false }); + await expect(harnesses.setGrokComputerUse({ enabled: true })).rejects.toMatchObject({ + status: 400, + }); + await expect(harnesses.setGrokComputerUse({ browser: true })).rejects.toMatchObject({ + status: 400, + }); + await expect(harnesses.setGrokComputerUse({ enabled: false })).resolves.toMatchObject({ + id: "grok", + computerUse: { enabled: false }, + }); + }); }); diff --git a/packages/console-server/test/live-voice-inject.test.ts b/packages/console-server/test/live-voice-inject.test.ts new file mode 100644 index 000000000..dc95ae902 --- /dev/null +++ b/packages/console-server/test/live-voice-inject.test.ts @@ -0,0 +1,82 @@ +import { mkdir, mkdtemp, readdir, readFile, rm, stat, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; + +import { + LIVE_VOICE_INJECT_FILE, + readLiveVoiceInjectState, + writeLiveVoiceInjectFile, +} from "../src/live-voice-inject.js"; + +let directory: string; +beforeEach(async () => { + directory = await mkdtemp(path.join(os.tmpdir(), "codexhost-live-voice-setting-")); +}); +afterEach(async () => { + await rm(directory, { recursive: true, force: true }); +}); + +describe("live voice injection setting", () => { + it("defaults on, persists both values, and leaves only the published file", async () => { + expect(await readLiveVoiceInjectState(directory, {})).toEqual({ + inject: true, + savedInject: true, + envOverride: null, + }); + for (const inject of [false, true]) { + await writeLiveVoiceInjectFile(directory, inject); + expect( + JSON.parse(await readFile(path.join(directory, LIVE_VOICE_INJECT_FILE), "utf8")), + ).toEqual({ version: 1, inject }); + expect(await readLiveVoiceInjectState(directory, {})).toEqual({ + inject, + savedInject: inject, + envOverride: null, + }); + } + expect(await readdir(directory)).toEqual([LIVE_VOICE_INJECT_FILE]); + if (process.platform !== "win32") { + expect((await stat(path.join(directory, LIVE_VOICE_INJECT_FILE))).mode & 0o777).toBe(0o600); + } + }); + + it.each([ + "invalid json", + "null", + "{}", + '{"version":2,"inject":false}', + '{"version":1,"inject":"false"}', + ])("defaults on for malformed or unsupported settings: %s", async (content) => { + await writeFile(path.join(directory, LIVE_VOICE_INJECT_FILE), content); + expect(await readLiveVoiceInjectState(directory, {})).toEqual({ + inject: true, + savedInject: true, + envOverride: null, + }); + }); + + it("gives explicit env overrides precedence without changing the saved value", async () => { + await writeLiveVoiceInjectFile(directory, false); + expect(await readLiveVoiceInjectState(directory, { CODEXHOST_CPA_LIVE_VOICE: "1" })).toEqual({ + inject: true, + savedInject: false, + envOverride: "1", + }); + await writeLiveVoiceInjectFile(directory, true); + expect(await readLiveVoiceInjectState(directory, { CODEXHOST_CPA_LIVE_VOICE: "0" })).toEqual({ + inject: false, + savedInject: true, + envOverride: "0", + }); + expect( + await readLiveVoiceInjectState(directory, { CODEXHOST_CPA_LIVE_VOICE: "other" }), + ).toEqual({ inject: true, savedInject: true, envOverride: null }); + }); + + it("reports publication failure and cleans up the temporary file", async () => { + await mkdir(path.join(directory, LIVE_VOICE_INJECT_FILE)); + await expect(writeLiveVoiceInjectFile(directory, false)).rejects.toThrow(); + expect(await readdir(directory)).toEqual([LIVE_VOICE_INJECT_FILE]); + }); +}); diff --git a/packages/console-server/test/restart.test.ts b/packages/console-server/test/restart.test.ts new file mode 100644 index 000000000..83deaee2b --- /dev/null +++ b/packages/console-server/test/restart.test.ts @@ -0,0 +1,56 @@ +import { describe, expect, it, vi } from "vitest"; +import { ConsoleRestart, type RestartObservation } from "../src/restart.js"; +const previous: RestartObservation = { + launcherPid: 1, + controllerPid: 2, + installed: true, + failure: null, +}; + +describe("managed console restart", () => { + it("does not report success for attach, old status or a new controller still installing", async () => { + const observe = vi + .fn() + .mockResolvedValueOnce(previous) + .mockResolvedValueOnce({ ...previous, launcherPid: 3 }) + .mockResolvedValueOnce({ ...previous, launcherPid: 3, controllerPid: 4, installed: false }) + .mockResolvedValue({ ...previous, launcherPid: 3, controllerPid: 4 }); + const sleep = vi.fn(async () => {}); + const restart = new ConsoleRestart(async () => {}, observe, sleep); + restart.start(previous); + expect(() => restart.start(previous)).toThrow("already"); + await vi.waitFor(() => expect(restart.status.phase).toBe("succeeded")); + expect(observe).toHaveBeenCalledTimes(4); + expect(sleep).toHaveBeenCalledTimes(3); + }); + it("reports spawn, startup and timeout failures and permits retry", async () => { + const run = vi + .fn() + .mockRejectedValueOnce(new Error("spawn failed")) + .mockResolvedValue(undefined); + const restart = new ConsoleRestart(run, async () => ({ + ...previous, + failure: "startup failed", + })); + restart.start(previous); + await vi.waitFor(() => + expect(restart.status).toEqual({ phase: "failed", error: "spawn failed" }), + ); + restart.start(previous); + await vi.waitFor(() => + expect(restart.status).toEqual({ phase: "failed", error: "startup failed" }), + ); + let time = 0; + const timeout = new ConsoleRestart( + async () => {}, + async () => previous, + async () => { + time += 250; + }, + () => time, + 500, + ); + timeout.start(previous); + await vi.waitFor(() => expect(timeout.status.error).toContain("timed out")); + }); +}); diff --git a/packages/console-server/test/server.test.ts b/packages/console-server/test/server.test.ts index abddd4c75..7bd46f1e4 100644 --- a/packages/console-server/test/server.test.ts +++ b/packages/console-server/test/server.test.ts @@ -78,11 +78,27 @@ function fakeHarnesses(): ConsoleHarnesses { ...harness, launchPath: value, })), + setGrokComputerUse: vi.fn(async (update: { enabled?: boolean; browser?: boolean }) => ({ + ...harness, + computerUse: { + enabled: update.enabled ?? false, + available: true, + browser: update.browser ?? false, + browserAvailable: true, + }, + })), }; } async function start( - options: { codexRunning?: boolean; launch?: () => void; host?: ConsoleHostClient } = {}, + options: { + codexRunning?: boolean; + launch?: () => void; + host?: ConsoleHostClient; + environment?: NodeJS.ProcessEnv; + restart?: (command: string, args: string[]) => Promise; + inspect?: () => Promise; + } = {}, ) { const updates = fakeUpdates(); const onExit = vi.fn(); @@ -99,9 +115,11 @@ async function start( harnesses: fakeHarnesses(), pageScript: "window.consoleLoaded = true;\n", host: options.host ?? { available: vi.fn(async () => false), request: vi.fn() }, - inspect: async () => inspectDocument(options.codexRunning ?? false), + inspect: options.inspect ?? (async () => inspectDocument(options.codexRunning ?? false)), launch: options.launch ?? vi.fn(), + ...(options.restart ? { restart: options.restart } : {}), onExit, + environment: { CODEXHOST_DATA_DIR: directory, ...options.environment }, }); const base = `http://127.0.0.1:${running.port}`; return { updates, onExit, base, port: running.port }; @@ -109,6 +127,56 @@ async function start( const CHANGE = { "x-codexhost-console": "1" }; +it("shares validated voice settings, protects writes and blocks changes during a reported call", async () => { + const { base } = await start(); + const call = async (body: unknown) => + fetch(`${base}/api/voice-settings`, { + method: "POST", + headers: { ...CHANGE, "content-type": "application/json" }, + body: JSON.stringify(body), + }); + const initial = await (await call({ operation: "get" })).json(); + expect(initial).toMatchObject({ initialized: false, activeInject: null, gatewayReady: null }); + const settings = { + inject: true, + chatExperience: "codex", + chatConnection: "desktop", + workConnection: "desktop", + }; + expect((await call({ operation: "set", revision: initial.revision, settings })).status).toBe(200); + const saved = await (await call({ operation: "get" })).json(); + expect(saved.settings).toEqual(settings); + expect((await call({ operation: "set", revision: initial.revision, settings })).status).toBe(409); + expect( + ( + await call({ + operation: "set", + revision: saved.revision, + settings: { ...settings, chatExperience: "invalid" }, + }) + ).status, + ).toBe(400); + expect( + ( + await fetch(`${base}/api/voice-settings`, { + method: "POST", + headers: { ...CHANGE, origin: "https://other.example" }, + body: JSON.stringify({ operation: "get" }), + }) + ).status, + ).toBe(403); + await call({ + operation: "get", + runtime: { + id: "5b4ca336-19a2-4d9f-a9bc-2603a0019c3a", + revision: saved.revision, + activeCall: true, + }, + }); + expect((await call({ operation: "set", revision: saved.revision, settings })).status).toBe(409); + expect((await call({ operation: "restart", confirm: true })).status).toBe(409); +}); + function rawRequest(port: number, host: string): Promise { return new Promise((resolve, reject) => { const request = httpRequest( @@ -158,6 +226,8 @@ describe("console server", () => { "startup", "controller", "launchAvailable", + "liveVoice", + "restart", "summary", "issueUrl", "hostAvailable", @@ -167,7 +237,7 @@ describe("console server", () => { expect(overview.summary.state).toBe("stopped"); expect(overview.inspect.desktop?.version).toBe("26.924.20706"); expect(overview.issueUrl).toMatch( - /^https:\/\/github\.com\/BytePioneer-AI\/codex-host\/issues\/new\?/u, + /^https:\/\/github\.com\/karlorz\/codex-host\/issues\/new\?/u, ); expect(decodeURIComponent(overview.issueUrl)).toContain("26.924.20706"); }); @@ -221,6 +291,117 @@ describe("console server", () => { expect(launch).toHaveBeenCalledWith("/opt/codexhost/bin/codexhost", ["launch"]); }); + it("persists live-voice inject and includes it on overview", async () => { + const { base } = await start(); + const first = (await (await fetch(`${base}/api/live-voice-inject`)).json()) as { + inject: boolean; + envOverride: string | null; + }; + expect(first).toEqual({ inject: true, savedInject: true, envOverride: null }); + const saved = await fetch(`${base}/api/live-voice-inject`, { + method: "POST", + headers: { origin: base, "content-type": "application/json", ...CHANGE }, + body: JSON.stringify({ inject: false }), + }); + expect(saved.status).toBe(200); + expect(await saved.json()).toEqual({ inject: false, savedInject: false, envOverride: null }); + const overview = (await (await fetch(`${base}/api/overview`)).json()) as { + liveVoice: { inject: boolean }; + }; + expect(overview.liveVoice.inject).toBe(false); + }); + + it("rejects invalid or foreign voice-setting writes without changing the setting", async () => { + const { base } = await start(); + const post = (origin: string, inject: unknown) => + fetch(`${base}/api/live-voice-inject`, { + method: "POST", + headers: { origin, "content-type": "application/json", ...CHANGE }, + body: JSON.stringify({ inject }), + }); + expect((await post("https://evil.example", false)).status).toBe(403); + for (const inject of ["false", null, 0]) { + expect((await post(base, inject)).status).toBe(400); + } + expect(await (await fetch(`${base}/api/live-voice-inject`)).json()).toEqual({ + inject: true, + savedInject: true, + envOverride: null, + }); + }); + + it("shows an env override while still persisting the requested setting", async () => { + const { base } = await start({ environment: { CODEXHOST_CPA_LIVE_VOICE: "1" } }); + const response = await fetch(`${base}/api/live-voice-inject`, { + method: "POST", + headers: { origin: base, "content-type": "application/json", ...CHANGE }, + body: JSON.stringify({ inject: false }), + }); + expect(await response.json()).toEqual({ inject: true, savedInject: false, envOverride: "1" }); + await running?.close(); + const restarted = await start(); + expect(await (await fetch(`${restarted.base}/api/live-voice-inject`)).json()).toEqual({ + inject: false, + savedInject: false, + envOverride: null, + }); + }); + + it("guards restart, rejects duplicates and verifies a fresh ready controller", async () => { + let launcherPid = 101; + const install = async (pid: number, inject: boolean) => { + const file = consolePaths({ CODEXHOST_DATA_DIR: directory }).controllerStatusFile; + await mkdir(path.dirname(file), { recursive: true }); + await writeFile( + file, + JSON.stringify({ + schemaVersion: 1, + pid, + startedAt: Date.now(), + voiceInjection: inject, + voiceEnvOverride: null, + renderer: { state: "installed", failures: 0, updatedAt: Date.now() }, + }), + ); + }; + await install(process.pid, true); + const gate = Promise.withResolvers(); + const restart = vi.fn(async () => { + await gate.promise; + launcherPid = 102; + await install(process.ppid, false); + }); + const { base } = await start({ + codexRunning: true, + restart, + inspect: async () => ({ + ...inspectDocument(true), + runtime: { running: true, descriptorPath: "/fixture/runtime.json", launcherPid }, + }), + }); + const post = (confirm: boolean, origin = base) => + fetch(`${base}/api/restart`, { + method: "POST", + headers: { origin, ...CHANGE }, + body: JSON.stringify({ confirm }), + }); + expect((await post(true, "https://evil.example")).status).toBe(403); + expect((await post(false)).status).toBe(400); + expect(restart).not.toHaveBeenCalled(); + expect((await post(true)).status).toBe(202); + expect((await post(true)).status).toBe(409); + expect(restart).toHaveBeenCalledExactlyOnceWith("/opt/codexhost/bin/codexhost", ["restart"]); + const read = async () => + (await (await fetch(`${base}/api/overview`)).json()) as { + restart: { phase: string }; + liveVoice: { activeInject: boolean; inject: boolean }; + }; + expect((await read()).restart.phase).toBe("restarting"); + gate.resolve(undefined); + await vi.waitFor(async () => expect((await read()).restart.phase).toBe("succeeded")); + expect((await read()).liveVoice).toMatchObject({ activeInject: false, inject: true }); + }); + it("refuses an update while codexhost is running", async () => { const { base } = await start({ codexRunning: true }); const response = await fetch(`${base}/api/update/start`, { @@ -297,6 +478,26 @@ describe("console server", () => { expect(invalid.status).toBe(400); }); + it("saves the Grok in-app browser switch independently and rejects an empty body", async () => { + const { base } = await start(); + const headers = { origin: base, ...CHANGE, "content-type": "application/json" }; + const saved = await fetch(`${base}/api/harnesses/grok-computer-use`, { + method: "POST", + headers, + body: JSON.stringify({ browser: true }), + }); + expect( + ((await saved.json()) as { harness: { computerUse: { browser: boolean; enabled: boolean } } }) + .harness.computerUse, + ).toMatchObject({ browser: true, enabled: false }); + const empty = await fetch(`${base}/api/harnesses/grok-computer-use`, { + method: "POST", + headers, + body: JSON.stringify({}), + }); + expect(empty.status).toBe(400); + }); + it("forwards only console Host methods and reports a missing Host", async () => { const request = vi.fn(async () => ({ result: { accounts: [] } })); const { base } = await start({ host: { available: vi.fn(async () => true), request } }); diff --git a/packages/console-server/test/updates.test.ts b/packages/console-server/test/updates.test.ts index 2bfd1773c..18ca16797 100644 --- a/packages/console-server/test/updates.test.ts +++ b/packages/console-server/test/updates.test.ts @@ -1,4 +1,4 @@ -import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { mkdir, mkdtemp, readFile, rename, rm, writeFile } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { EventEmitter } from "node:events"; @@ -18,14 +18,24 @@ afterEach(async () => { await rm(root, { recursive: true, force: true }); }); -const release = { - version: "1.1.0", - releaseNotes: "Fixes", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.1.0", - assets: [], -}; +function releaseFor(version: string) { + return { + version, + releaseNotes: "Fixes", + releaseNotesUrl: `https://github.com/karlorz/codex-host/releases/tag/v${version}`, + assets: [], + }; +} -async function npmLayout(platform: NodeJS.Platform = process.platform) { +const release = releaseFor("1.1.0"); + +async function writeWaitingForExit(statusPath: string): Promise { + const status = JSON.parse(await readFile(statusPath, "utf8")); + await writeFile(`${statusPath}.tmp`, JSON.stringify({ ...status, phase: "waiting-for-exit" })); + await rename(`${statusPath}.tmp`, statusPath); +} + +async function npmLayout(platform: NodeJS.Platform = process.platform, version = "1.0.0") { const packageRoot = path.join(root, "package"); const appDirectory = path.join(packageRoot, "app"); await mkdir(path.join(packageRoot, "libexec"), { recursive: true }); @@ -41,7 +51,7 @@ async function npmLayout(platform: NodeJS.Platform = process.platform) { await writeFile(npmCli, ""); await writeFile(wrapper, ""); const target: ConsoleUpdateTarget = { - distribution: { schemaVersion: 1, version: "1.0.0", distribution: "npm", target: "linux-x64" }, + distribution: { schemaVersion: 1, version, distribution: "npm", target: "linux-x64" }, appDirectory, runtimeDescriptorPath: path.join(root, "runtime", "desktop-runtime-v1.json"), codexhostRunning: false, @@ -142,8 +152,29 @@ describe("console updates", () => { await expect(createConsoleUpdates(options).status()).resolves.toEqual({ status: null }); }); - it("hands off to the Updater before returning", async () => { - const { target, environment } = await npmLayout("linux"); + it("reports an equal fork counter as up to date without starting the Updater", async () => { + const { target, environment } = await npmLayout("linux", "0.13.2-1"); + const updates = createConsoleUpdates({ + onHandedOff: vi.fn(), + environment, + stateDirectory: path.join(root, "state"), + fetchLatest: async () => releaseFor("0.13.2-1"), + }); + + await expect(updates.check(target)).resolves.toMatchObject({ + currentVersion: "0.13.2-1", + latestVersion: "0.13.2-1", + updateAvailable: false, + installationAvailable: false, + error: null, + }); + }); + + it.each([ + ["1.0.0", "1.1.0"], + ["0.13.2-1", "0.13.2-2"], + ])("hands off %s → %s to the Updater before returning", async (currentVersion, latestVersion) => { + const { target, environment } = await npmLayout("linux", currentVersion); const spawnUpdater = vi.fn<(executable: string, requestPath: string) => never>( () => Object.assign(new EventEmitter(), { pid: 4321, kill: vi.fn() }) as never, ); @@ -156,30 +187,34 @@ describe("console updates", () => { processId: 777, processExecutable: process.execPath, stateDirectory: path.join(root, "state"), - fetchLatest: async () => release, + fetchLatest: async () => releaseFor(latestVersion), manager: createBackgroundUpdateManager({ platform: "linux", spawnUpdater }), }); - await updates.check(target); + await expect(updates.check(target)).resolves.toMatchObject({ + currentVersion, + latestVersion, + updateAvailable: true, + installationAvailable: true, + }); + expect(spawnUpdater).not.toHaveBeenCalled(); + expect(onHandedOff).not.toHaveBeenCalled(); const starting = updates.start(target); await vi.waitFor(() => expect(spawnUpdater).toHaveBeenCalledOnce()); expect(onHandedOff).not.toHaveBeenCalled(); const call = spawnUpdater.mock.calls[0]; if (!call) throw new Error("Updater was not spawned"); const spawnedRequest = JSON.parse(await readFile(call[1], "utf8")); - const status = JSON.parse(await readFile(spawnedRequest.status_path, "utf8")); - await writeFile( - spawnedRequest.status_path, - JSON.stringify({ ...status, phase: "waiting-for-exit" }), - ); + await writeWaitingForExit(spawnedRequest.status_path); const result = await starting; expect(result.status.phase).toBe("waiting-for-exit"); - expect(result.status).toMatchObject({ version: "1.1.0", installation: "npm" }); + expect(result.status).toMatchObject({ version: latestVersion, installation: "npm" }); expect(onHandedOff).toHaveBeenCalledOnce(); const requestPath = (spawnUpdater.mock.calls[0] as unknown as [string, string])[1]; const request = JSON.parse(await readFile(requestPath, "utf8")) as Record; expect(request).toMatchObject({ + version: latestVersion, wait_pid: 777, wait_executable: process.execPath, runtime_descriptor_path: target.runtimeDescriptorPath, @@ -214,8 +249,7 @@ describe("console updates", () => { const call = spawnUpdater.mock.calls[0]; if (!call) throw new Error("Updater was not spawned"); const request = JSON.parse(await readFile(call[1], "utf8")); - const status = JSON.parse(await readFile(request.status_path, "utf8")); - await writeFile(request.status_path, JSON.stringify({ ...status, phase: "waiting-for-exit" })); + await writeWaitingForExit(request.status_path); await expect(retry).resolves.toMatchObject({ status: { phase: "waiting-for-exit" } }); await expect(updates.status()).resolves.toMatchObject({ status: { phase: "waiting-for-exit", error: null }, diff --git a/packages/console-server/test/voice-settings.test.ts b/packages/console-server/test/voice-settings.test.ts new file mode 100644 index 000000000..115df3622 --- /dev/null +++ b/packages/console-server/test/voice-settings.test.ts @@ -0,0 +1,89 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, expect, it } from "vitest"; +import { VoiceSettingsStore } from "../src/voice-settings.js"; +import { LIVE_VOICE_INJECT_FILE, writeLiveVoiceInjectFile } from "../src/live-voice-inject.js"; + +let directory: string; +let store: VoiceSettingsStore; +beforeEach(async () => { + directory = await mkdtemp(path.join(os.tmpdir(), "voice-shared-")); + store = new VoiceSettingsStore(directory); +}); +afterEach(async () => { + await rm(directory, { recursive: true, force: true }); +}); +const routes = { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", +} as const; + +it("migrates Desktop choices once, preserving injection and subsequent console edits", async () => { + await writeLiveVoiceInjectFile(directory, false); + expect((await store.read()).initialized).toBe(false); + const migrated = await store.write({ operation: "migrate", routes }); + expect(migrated.settings).toEqual({ ...routes, inject: false }); + const saved = await store.write({ + operation: "set", + revision: migrated.revision, + settings: { ...migrated.settings, chatExperience: "codex", chatConnection: "gateway" }, + }); + expect((await store.write({ operation: "migrate", routes })).settings).toEqual(saved.settings); + expect((await new VoiceSettingsStore(directory).read()).settings).toEqual(saved.settings); + await writeLiveVoiceInjectFile(directory, true); + expect((await store.read()).settings).toEqual({ ...saved.settings, inject: true }); +}); + +it("serializes competing migration and edits, and rejects stale revisions", async () => { + const current = await store.read(); + const result = await Promise.allSettled([ + store.write({ + operation: "set", + revision: current.revision, + settings: { ...routes, inject: true }, + }), + store.write({ operation: "migrate", routes: { ...routes, workConnection: "gateway" } }), + store.write({ + operation: "set", + revision: current.revision, + settings: { ...routes, inject: false }, + }), + ]); + expect(result.map((entry) => entry.status)).toEqual(["fulfilled", "fulfilled", "rejected"]); + expect((await store.read()).settings.workConnection).toBe("desktop"); +}); + +it("guards active calls and reports only acknowledged runtime revisions", async () => { + const state = await store.read(); + store.report({ id: "runtime", revision: "old", activeCall: true }); + expect(await store.read()).toMatchObject({ activeCall: true, routesActive: false }); + await expect( + store.write({ + operation: "set", + revision: state.revision, + settings: { ...routes, inject: true }, + }), + ).rejects.toThrow("End your voice call"); + store.report({ id: "runtime", revision: state.revision, activeCall: false }); + expect(await store.read()).toMatchObject({ activeCall: false, routesActive: true }); +}); + +it("defaults malformed routes without dropping a valid injection preference", async () => { + await writeFile( + path.join(directory, LIVE_VOICE_INJECT_FILE), + JSON.stringify({ version: 1, inject: false, routes: { chatExperience: "invalid" } }), + ); + expect(await store.read()).toMatchObject({ + initialized: false, + settings: { inject: false, chatExperience: "chatgpt" }, + }); + const saved = await store.write({ operation: "migrate", routes }); + expect(JSON.parse(await readFile(path.join(directory, LIVE_VOICE_INJECT_FILE), "utf8"))).toEqual({ + version: 1, + inject: false, + routes, + }); + expect(saved.initialized).toBe(true); +}); diff --git a/packages/desktop-control/src/controller-attachment-server.ts b/packages/desktop-control/src/controller-attachment-server.ts index 418c15679..f4db1c16d 100644 --- a/packages/desktop-control/src/controller-attachment-server.ts +++ b/packages/desktop-control/src/controller-attachment-server.ts @@ -18,6 +18,7 @@ export interface StartControllerAttachmentServerOptions { port: number; nonce: string; attach(): Promise; + stop?(): void; openLocalPage?(url: string): Promise; remoteConnections?(request: RemoteConnectionsRequest): Promise; } @@ -142,6 +143,10 @@ export async function startControllerAttachmentServer( void completion.finally(() => pages.delete(completion)); return; } + if (options.stop && line === `STOP ${options.nonce}` && request.slice(newline + 1) === "") { + socket.end("stopping\n", () => options.stop?.()); + return; + } if (line === `ATTACH ${options.nonce}`) { if (attachment) { respond(socket, "busy"); diff --git a/packages/desktop-control/src/controller-status.ts b/packages/desktop-control/src/controller-status.ts index 8bd8012b5..e2fb32916 100644 --- a/packages/desktop-control/src/controller-status.ts +++ b/packages/desktop-control/src/controller-status.ts @@ -27,6 +27,8 @@ export interface DesktopControllerStatusDocument { schemaVersion: 1; pid: number; startedAt: number; + voiceInjection?: boolean; + voiceEnvOverride?: "0" | "1" | null; renderer: RendererIntegrationStatus; } diff --git a/packages/desktop-control/src/cpa-live-voice-hook.ts b/packages/desktop-control/src/cpa-live-voice-hook.ts new file mode 100644 index 000000000..2cac5b4bd --- /dev/null +++ b/packages/desktop-control/src/cpa-live-voice-hook.ts @@ -0,0 +1,72 @@ +import { existsSync } from "node:fs"; +import { execFileSync } from "node:child_process"; +import os from "node:os"; +import path from "node:path"; + +import { readLiveVoiceInjectFile } from "./live-voice-inject.js"; + +export const CPA_LIVE_VOICE_ENV = "CODEXHOST_CPA_LIVE_VOICE"; +export const CPA_LIVE_VOICE_HOOK_GLOBAL = "__codexhostCpaLiveVoiceHookV1"; +export const AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION = "26.1007.30912"; +export const AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET = "app-initial-6350d04672d0.js"; +export const DESKTOP_VERSION_ENV = "CODEXHOST_DESKTOP_VERSION"; + +/** Process env wins for one launch. Otherwise the ~/.codexhost inject file (default on). */ +export function cpaLiveVoiceHookEnabled(environment: NodeJS.ProcessEnv = process.env): boolean { + const value = environment[CPA_LIVE_VOICE_ENV]; + if (value === "0") return false; + if (value === "1") return true; + return readLiveVoiceInjectFile(environment); +} + +/** Sparkle/CFBundle short version, e.g. 26.1021.31345. Env wins. */ +export function detectInstalledDesktopVersion( + environment: NodeJS.ProcessEnv = process.env, +): string | undefined { + const fromEnv = environment[DESKTOP_VERSION_ENV]?.trim(); + if (fromEnv) return fromEnv; + if (process.platform !== "darwin") return undefined; + const home = environment.HOME?.trim() || os.homedir(); + const apps = [ + "/Applications/ChatGPT.app", + "/Applications/Codex.app", + path.join(home, "Applications/ChatGPT.app"), + path.join(home, "Applications/Codex.app"), + ]; + for (const app of apps) { + const plist = path.join(app, "Contents/Info.plist"); + if (!existsSync(plist)) continue; + try { + const version = execFileSync( + "/usr/bin/defaults", + ["read", plist, "CFBundleShortVersionString"], + { + encoding: "utf8", + timeout: 2000, + }, + ).trim(); + if (version) return version; + } catch { + continue; + } + } + return undefined; +} + +export function cpaLiveVoiceHookBootstrap( + environment: NodeJS.ProcessEnv = process.env, + desktopVersion?: string, +): string { + if (!cpaLiveVoiceHookEnabled(environment)) return ""; + const version = + environment[DESKTOP_VERSION_ENV]?.trim() || + desktopVersion || + detectInstalledDesktopVersion(environment); + return `globalThis.${CPA_LIVE_VOICE_HOOK_GLOBAL} = Object.freeze(${JSON.stringify({ + enabled: true, + desktopPlatform: process.platform, + auditedDesktopVersion: AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + auditedAppInitialAsset: AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + ...(version ? { desktopVersion: version } : {}), + })});\n`; +} diff --git a/packages/desktop-control/src/index.ts b/packages/desktop-control/src/index.ts index cb842a2a6..57fa52b46 100644 --- a/packages/desktop-control/src/index.ts +++ b/packages/desktop-control/src/index.ts @@ -30,6 +30,7 @@ export { inspectRendererDom, validateRendererDomInspection } from "./renderer-do export { createRendererCdpControlSession, installRendererCdpControlSession, + selectAvatarOverlayRendererTargets, selectPrimaryRendererTarget, } from "./renderer-cdp-control-session.js"; export type { diff --git a/packages/desktop-control/src/live-voice-inject.ts b/packages/desktop-control/src/live-voice-inject.ts new file mode 100644 index 000000000..dd07675fa --- /dev/null +++ b/packages/desktop-control/src/live-voice-inject.ts @@ -0,0 +1,23 @@ +import { readFileSync } from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +export const LIVE_VOICE_INJECT_FILE = "live-voice-inject-v1.json"; + +/** Missing, malformed or unsupported settings default to injection on. */ +export function readLiveVoiceInjectFile(environment: NodeJS.ProcessEnv = process.env): boolean { + const directory = environment.CODEXHOST_DATA_DIR + ? path.resolve(environment.CODEXHOST_DATA_DIR) + : path.join(os.homedir(), ".codexhost"); + try { + const parsed = JSON.parse( + readFileSync(path.join(directory, LIVE_VOICE_INJECT_FILE), "utf8"), + ) as { + version?: unknown; + inject?: unknown; + }; + return parsed.version !== 1 || parsed.inject !== false; + } catch { + return true; + } +} diff --git a/packages/desktop-control/src/production-controller.ts b/packages/desktop-control/src/production-controller.ts index 0dbf76ca7..84286119a 100644 --- a/packages/desktop-control/src/production-controller.ts +++ b/packages/desktop-control/src/production-controller.ts @@ -15,6 +15,7 @@ import { createRendererStatusReporter, type DesktopControllerStatusDocument, } from "./controller-status.js"; +import { cpaLiveVoiceHookBootstrap } from "./cpa-live-voice-hook.js"; import { installRendererCdpControlSession, type RendererCdpControlSession, @@ -25,6 +26,7 @@ export interface DesktopControllerOptions { rendererPath: string; attachmentPort: number; attachmentNonce: string; + requestStop?(): void; } export interface DesktopControllerReadiness { @@ -38,6 +40,7 @@ export interface DesktopControllerDependencies { install(options: { rendererCdpEndpoint: string; rendererSource: string; + overlayHookSource?: string; enabledAgents?: readonly string[]; timeoutMs: number; signal?: AbortSignal; @@ -224,8 +227,18 @@ export async function runDesktopController( let session: RendererCdpControlSession | undefined; let nextRecoveryAt = 0; let recoveryDelayMs = RECOVERY_RETRY_INITIAL_MS; + const bootstrap = cpaLiveVoiceHookBootstrap(); const status = createRendererStatusReporter( - (document) => dependencies.publishStatus?.(document), + (document) => + dependencies.publishStatus?.({ + ...document, + voiceInjection: Boolean(bootstrap), + voiceEnvOverride: + process.env.CODEXHOST_CPA_LIVE_VOICE === "0" || + process.env.CODEXHOST_CPA_LIVE_VOICE === "1" + ? process.env.CODEXHOST_CPA_LIVE_VOICE + : null, + }), now, ); const recordRecoveryFailure = (error: unknown): void => { @@ -241,13 +254,24 @@ export async function runDesktopController( const createSession = async (): Promise => { status.installing(); startupTrace("reading Renderer bundle"); - const rendererSource = await dependencies.readRenderer(options.rendererPath); + const rendererRead = dependencies.readRenderer(options.rendererPath); + const overlayPath = bootstrap + ? path.join(path.dirname(options.rendererPath), "cpa-live-voice-hook.js") + : undefined; + const [rendererSource, overlayBundle] = overlayPath + ? await Promise.all([rendererRead, dependencies.readRenderer(overlayPath)]) + : [await rendererRead, undefined]; if (rendererSource.trim().length === 0) throw new Error("production Renderer Bundle is empty"); + const overlayHookSource = + bootstrap && overlayBundle && overlayBundle.trim().length > 0 + ? `${RENDERER_CSP_BOOTSTRAP}\n${bootstrap}${overlayBundle}` + : undefined; startupTrace("installing Renderer Session"); const installed = await installProductionSession( { rendererCdpEndpoint: options.rendererCdpEndpoint, - rendererSource: `${RENDERER_CSP_BOOTSTRAP}\n${rendererSource}`, + rendererSource: `${RENDERER_CSP_BOOTSTRAP}\n${bootstrap}${rendererSource}`, + ...(overlayHookSource ? { overlayHookSource } : {}), // External Agents are discovered from each target Host after installation. timeoutMs: PRODUCTION_INSTALL_TIMEOUT_MS, signal, @@ -295,6 +319,7 @@ export async function runDesktopController( attachmentServer = await dependencies.startAttachmentServer({ port: options.attachmentPort, nonce: options.attachmentNonce, + ...(options.requestStop ? { stop: options.requestStop } : {}), remoteConnections: async (request) => { // Serialize session recovery, not the native request: the renderer can await Host // responses while other settings reads run. Never retry a submitted mutation. diff --git a/packages/desktop-control/src/release-main.ts b/packages/desktop-control/src/release-main.ts index 88fab3c3e..be2c15cac 100644 --- a/packages/desktop-control/src/release-main.ts +++ b/packages/desktop-control/src/release-main.ts @@ -6,7 +6,10 @@ process.once("SIGINT", stop); process.once("SIGTERM", stop); try { - await runDesktopController(parseDesktopControllerArguments(process.argv.slice(2)), abort.signal); + await runDesktopController( + { ...parseDesktopControllerArguments(process.argv.slice(2)), requestStop: stop }, + abort.signal, + ); } catch (error) { console.error( `codexhost Desktop Controller: ${error instanceof Error ? error.message : String(error)}`, diff --git a/packages/desktop-control/src/renderer-cdp-control-session.ts b/packages/desktop-control/src/renderer-cdp-control-session.ts index 85e76de67..a99b8de98 100644 --- a/packages/desktop-control/src/renderer-cdp-control-session.ts +++ b/packages/desktop-control/src/renderer-cdp-control-session.ts @@ -6,6 +6,7 @@ import { type CdpTarget, cdpExceptionMessage, } from "./cdp-client.js"; +import { CPA_LIVE_VOICE_HOOK_GLOBAL } from "./cpa-live-voice-hook.js"; import { installRendererDraftPrewarmPolicyDirect, type RendererDraftPrewarmPolicyStatus, @@ -49,6 +50,7 @@ export interface RendererCdpControlSession { export interface InstallRendererCdpControlOptions { rendererCdpEndpoint: string; rendererSource: string; + overlayHookSource?: string; enabledAgents?: readonly string[]; pollIntervalMs?: number; timeoutMs?: number; @@ -124,6 +126,19 @@ function isPrimaryRendererUrl(value: string): boolean { } } +function isAvatarOverlayRendererUrl(value: string): boolean { + try { + const url = new URL(value); + if (url.protocol !== "app:" || url.hostname !== "-") return false; + if (url.pathname === "/avatar-overlay.html") return true; + return ( + url.pathname === "/index.html" && url.searchParams.get("initialRoute") === "/avatar-overlay" + ); + } catch { + return false; + } +} + export function selectPrimaryRendererTarget( targets: readonly CdpTarget[], preferredTargetId?: string, @@ -134,6 +149,10 @@ export function selectPrimaryRendererTarget( return candidates.find((target) => target.id === preferredTargetId) ?? candidates.at(0) ?? null; } +export function selectAvatarOverlayRendererTargets(targets: readonly CdpTarget[]): CdpTarget[] { + return targets.filter((item) => item.type === "page" && isAvatarOverlayRendererUrl(item.url)); +} + class RendererAdapterReadinessError extends Error { constructor( readonly state: string, @@ -249,6 +268,71 @@ async function waitForBinding( throw new Error(`Production Renderer binding did not become ready${detail}`); } +async function overlayHookAlreadyEnabled(renderer: RendererCdpClient): Promise { + try { + const value = await renderer.evaluate( + `globalThis.${CPA_LIVE_VOICE_HOOK_GLOBAL}?.enabled === true`, + ); + return value === true; + } catch { + return false; + } +} + +async function installOverlayHook( + target: CdpTarget, + overlayHookSource: string, + operations: RendererCdpControlOperations, + signal: AbortSignal, +): Promise { + const run = (operation: () => Promise): Promise => { + signal.throwIfAborted(); + return awaitWithSignal(operation(), signal, "Overlay hook installation timed out"); + }; + const renderer = await run(async () => { + const connected = await operations.connect(target.webSocketDebuggerUrl); + if (signal.aborted) { + connected.close(); + signal.throwIfAborted(); + } + return connected; + }); + try { + await run(() => renderer.command("Runtime.enable")); + await run(() => renderer.command("Page.enable")); + if (await run(() => overlayHookAlreadyEnabled(renderer))) return; + await run(() => + renderer.command("Page.addScriptToEvaluateOnNewDocument", { source: overlayHookSource }), + ); + await run(() => evaluateSource(renderer, overlayHookSource)); + } finally { + renderer.close(); + } +} + +async function installOverlayHooks( + endpoint: string, + overlayHookSource: string | undefined, + operations: RendererCdpControlOperations, + signal: AbortSignal, +): Promise { + if (!overlayHookSource) return; + try { + const overlays = selectAvatarOverlayRendererTargets( + await awaitWithSignal( + operations.listTargets(endpoint, signal), + signal, + "Overlay CDP target discovery timed out", + ), + ); + await Promise.allSettled( + overlays.map((overlay) => installOverlayHook(overlay, overlayHookSource, operations, signal)), + ); + } catch { + // Overlay discovery and injection are best-effort and share one deadline. + } +} + async function installTarget( target: CdpTarget, rendererSource: string, @@ -279,6 +363,7 @@ class InstalledRendererCdpControlSession implements RendererCdpControlSession { private renderer: RendererCdpClient, private readonly rendererCdpEndpoint: string, private readonly rendererSource: string, + private readonly overlayHookSource: string | undefined, private readonly enabledAgents: readonly string[] | undefined, private readonly timeoutMs: number, private readonly pollIntervalMs: number, @@ -311,37 +396,41 @@ class InstalledRendererCdpControlSession implements RendererCdpControlSession { this.renderer.close(); this.renderer = replacement.renderer; this.currentSnapshot = replacement.snapshot; - return this.currentSnapshot; - } - - try { - const existing = await readBinding(this.renderer); - if (existing === null) await evaluateSource(this.renderer, this.rendererSource); - // A Host switch can temporarily leave the Adapter installing. Reconcile - // native owners before validating readiness; that is not a lost CDP page. - const draftPrewarmPolicy = await this.operations.installDraftPrewarmPolicy(this.renderer); - const binding = await waitForBinding( - this.renderer, - this.enabledAgents, - this.timeoutMs, - this.pollIntervalMs, - ); - this.currentSnapshot = { target, draftPrewarmPolicy, binding }; - return this.currentSnapshot; - } catch { - const replacement = await installTarget( - target, - this.rendererSource, - this.enabledAgents, - this.timeoutMs, - this.pollIntervalMs, - this.operations, - ); - this.renderer.close(); - this.renderer = replacement.renderer; - this.currentSnapshot = replacement.snapshot; - return this.currentSnapshot; + } else { + try { + const existing = await readBinding(this.renderer); + if (existing === null) await evaluateSource(this.renderer, this.rendererSource); + // A Host switch can temporarily leave the Adapter installing. Reconcile + // native owners before validating readiness; that is not a lost CDP page. + const draftPrewarmPolicy = await this.operations.installDraftPrewarmPolicy(this.renderer); + const binding = await waitForBinding( + this.renderer, + this.enabledAgents, + this.timeoutMs, + this.pollIntervalMs, + ); + this.currentSnapshot = { target, draftPrewarmPolicy, binding }; + } catch { + const replacement = await installTarget( + target, + this.rendererSource, + this.enabledAgents, + this.timeoutMs, + this.pollIntervalMs, + this.operations, + ); + this.renderer.close(); + this.renderer = replacement.renderer; + this.currentSnapshot = replacement.snapshot; + } } + await installOverlayHooks( + this.rendererCdpEndpoint, + this.overlayHookSource, + this.operations, + discoveryTimeoutSignal(this.timeoutMs), + ); + return this.currentSnapshot; } async activateDesktop(): Promise { @@ -391,10 +480,17 @@ export async function createRendererCdpControlSession( pollIntervalMs, operations, ); + await installOverlayHooks( + options.rendererCdpEndpoint, + options.overlayHookSource, + operations, + discoveryTimeoutSignal(timeoutMs, options.signal), + ); return new InstalledRendererCdpControlSession( installed.renderer, options.rendererCdpEndpoint, options.rendererSource, + options.overlayHookSource, enabledAgents, timeoutMs, pollIntervalMs, diff --git a/packages/desktop-control/src/renderer-draft-prewarm-runtime.ts b/packages/desktop-control/src/renderer-draft-prewarm-runtime.ts index 0eff6dd19..cf383c3c5 100644 --- a/packages/desktop-control/src/renderer-draft-prewarm-runtime.ts +++ b/packages/desktop-control/src/renderer-draft-prewarm-runtime.ts @@ -689,6 +689,11 @@ export function createDraftPrewarmPolicyBridge( throw new Error("Renderer request manager is retired"); return manager; }, + selectedModel(): string | null { + if (disposed || (isCurrentManager && !isCurrentManager())) + throw new Error("Renderer request manager is retired"); + return selectedModel; + }, select(model: string | null): boolean { if (disposed || (isCurrentManager && !isCurrentManager())) throw new Error("Renderer request manager is retired"); diff --git a/packages/desktop-control/test/controller-attachment-server.test.ts b/packages/desktop-control/test/controller-attachment-server.test.ts index d49b3fbb0..58424dbe8 100644 --- a/packages/desktop-control/test/controller-attachment-server.test.ts +++ b/packages/desktop-control/test/controller-attachment-server.test.ts @@ -52,6 +52,21 @@ describe("Controller attachment server", () => { } }); + it("stops only on an authenticated single command", async () => { + const port = await availablePort(); + const stop = vi.fn(); + const server = await startControllerAttachmentServer({ port, nonce, attach: vi.fn(), stop }); + try { + expect(await request(port, `STOP ${"0".repeat(32)}\n`)).toBe("rejected\n"); + expect(await request(port, `STOP ${nonce}\nATTACH ${nonce}\n`)).toBe("rejected\n"); + expect(stop).not.toHaveBeenCalled(); + expect(await request(port, `STOP ${nonce}\n`)).toBe("stopping\n"); + expect(stop).toHaveBeenCalledOnce(); + } finally { + await server.close(); + } + }); + it("validates remote settings commands at the socket boundary", async () => { const port = await availablePort(); const remoteConnections = vi.fn(async () => ({ result: [] })); diff --git a/packages/desktop-control/test/cpa-live-voice-hook.test.ts b/packages/desktop-control/test/cpa-live-voice-hook.test.ts new file mode 100644 index 000000000..3cc420481 --- /dev/null +++ b/packages/desktop-control/test/cpa-live-voice-hook.test.ts @@ -0,0 +1,187 @@ +import { execFileSync } from "node:child_process"; +import { existsSync } from "node:fs"; +import type * as NodeFs from "node:fs"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +vi.mock("node:child_process", () => ({ execFileSync: vi.fn() })); +vi.mock("node:fs", async (importOriginal) => ({ + ...(await importOriginal()), + existsSync: vi.fn(), +})); + +import { + AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + CPA_LIVE_VOICE_ENV, + DESKTOP_VERSION_ENV, + cpaLiveVoiceHookBootstrap, + cpaLiveVoiceHookEnabled, + detectInstalledDesktopVersion, +} from "../src/cpa-live-voice-hook.js"; + +let directory: string; +let environment: NodeJS.ProcessEnv; +beforeEach(async () => { + directory = await mkdtemp(path.join(os.tmpdir(), "codexhost-inject-")); + environment = { CODEXHOST_DATA_DIR: directory }; +}); + +describe("installed Desktop version detection", () => { + const platformDescriptor = Object.getOwnPropertyDescriptor(process, "platform"); + if (!platformDescriptor) throw new Error("process.platform descriptor is missing"); + + beforeEach(() => { + Object.defineProperty(process, "platform", { ...platformDescriptor, value: "darwin" }); + vi.mocked(existsSync).mockReset().mockReturnValue(false); + vi.mocked(execFileSync).mockReset(); + }); + + afterEach(() => { + Object.defineProperty(process, "platform", platformDescriptor); + }); + + it.each([ + "/Applications/ChatGPT.app", + "/Applications/Codex.app", + "/test-home/Applications/ChatGPT.app", + "/test-home/Applications/Codex.app", + ])("reads the short version from %s/Contents/Info.plist", (app) => { + const plist = path.join(app, "Contents/Info.plist"); + vi.mocked(existsSync).mockImplementation((candidate) => candidate === plist); + vi.mocked(execFileSync).mockReturnValue("26.1021.31345\n"); + + const version = detectInstalledDesktopVersion({ HOME: "/test-home" }); + + expect(version).toBe("26.1021.31345"); + expect(execFileSync).toHaveBeenCalledExactlyOnceWith( + "/usr/bin/defaults", + ["read", plist, "CFBundleShortVersionString"], + { encoding: "utf8", timeout: 2000 }, + ); + expect(cpaLiveVoiceHookBootstrap(environment, version)).toContain( + '"desktopVersion":"26.1021.31345"', + ); + expect(cpaLiveVoiceHookBootstrap({ ...environment, HOME: "/test-home" })).toContain( + '"desktopVersion":"26.1021.31345"', + ); + }); + + it("does not read installed apps when the hook is disabled", () => { + vi.mocked(existsSync).mockReturnValue(true); + vi.mocked(execFileSync).mockReturnValue("26.1021.31345\n"); + expect(cpaLiveVoiceHookBootstrap({ [CPA_LIVE_VOICE_ENV]: "0" })).toBe(""); + expect(existsSync).not.toHaveBeenCalled(); + expect(execFileSync).not.toHaveBeenCalled(); + }); + + it("uses the trimmed env override without reading installed apps", () => { + expect(detectInstalledDesktopVersion({ [DESKTOP_VERSION_ENV]: " 26.1021.31345 " })).toBe( + "26.1021.31345", + ); + expect(existsSync).not.toHaveBeenCalled(); + expect(execFileSync).not.toHaveBeenCalled(); + }); + + it("tries the next installed app when a plist read fails", () => { + vi.mocked(existsSync).mockReturnValue(true); + vi.mocked(execFileSync) + .mockImplementationOnce(() => { + throw new Error("unreadable plist"); + }) + .mockReturnValueOnce("26.1021.31345\n"); + + expect(detectInstalledDesktopVersion({ HOME: "/test-home" })).toBe("26.1021.31345"); + expect(execFileSync).toHaveBeenNthCalledWith( + 2, + "/usr/bin/defaults", + [ + "read", + path.join("/Applications/Codex.app", "Contents/Info.plist"), + "CFBundleShortVersionString", + ], + { encoding: "utf8", timeout: 2000 }, + ); + }); + + it("returns undefined when no installed plist exists", () => { + expect(detectInstalledDesktopVersion({ HOME: "/test-home" })).toBeUndefined(); + expect(execFileSync).not.toHaveBeenCalled(); + }); + + it("passes Windows product identity and platform to the renderer without using plist discovery", () => { + Object.defineProperty(process, "platform", { ...platformDescriptor, value: "win32" }); + const source = cpaLiveVoiceHookBootstrap(environment, "26.1021.31345"); + expect(source).toContain('"desktopVersion":"26.1021.31345"'); + expect(source).toContain('"desktopPlatform":"win32"'); + expect(existsSync).not.toHaveBeenCalled(); + expect(execFileSync).not.toHaveBeenCalled(); + }); + + it("does not run defaults on other platforms", () => { + Object.defineProperty(process, "platform", { ...platformDescriptor, value: "linux" }); + expect(detectInstalledDesktopVersion({})).toBeUndefined(); + expect(existsSync).not.toHaveBeenCalled(); + expect(execFileSync).not.toHaveBeenCalled(); + }); +}); +afterEach(async () => { + await rm(directory, { recursive: true, force: true }); +}); + +describe("CPA live-voice controller bootstrap", () => { + it("defaults on when no setting exists, and accepts explicit env overrides", () => { + expect(cpaLiveVoiceHookEnabled(environment)).toBe(true); + expect(cpaLiveVoiceHookEnabled({ ...environment, [CPA_LIVE_VOICE_ENV]: "1" })).toBe(true); + expect(cpaLiveVoiceHookEnabled({ [CPA_LIVE_VOICE_ENV]: "0" })).toBe(false); + expect(cpaLiveVoiceHookBootstrap({ [CPA_LIVE_VOICE_ENV]: "0" })).toBe(""); + }); + + it("reads the inject file when the env is unset, and env still wins", async () => { + await writeFile( + path.join(directory, "live-voice-inject-v1.json"), + JSON.stringify({ version: 1, inject: false }), + ); + expect(cpaLiveVoiceHookEnabled(environment)).toBe(false); + expect(cpaLiveVoiceHookBootstrap(environment)).toBe(""); + expect(cpaLiveVoiceHookEnabled({ ...environment, [CPA_LIVE_VOICE_ENV]: "1" })).toBe(true); + expect(cpaLiveVoiceHookEnabled({ ...environment, [CPA_LIVE_VOICE_ENV]: "0" })).toBe(false); + }); + + it.each([ + "invalid json", + "null", + "{}", + '{"version":2,"inject":false}', + '{"version":1,"inject":"false"}', + ])("defaults on for malformed or unsupported settings: %s", async (content) => { + await writeFile(path.join(directory, "live-voice-inject-v1.json"), content); + expect(cpaLiveVoiceHookEnabled(environment)).toBe(true); + }); + + it("emits a frozen enable record by default", () => { + const source = cpaLiveVoiceHookBootstrap(environment); + expect(source.startsWith("globalThis.__codexhostCpaLiveVoiceHookV1 = Object.freeze(")).toBe( + true, + ); + expect(source).toContain(`"auditedDesktopVersion":"${AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION}"`); + expect(source).toContain( + `"auditedAppInitialAsset":"${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}"`, + ); + expect(source).toContain('"enabled":true'); + expect(source).not.toContain("clientOwnsCall"); + expect(source).not.toContain("function pa"); + expect(source).not.toContain("desktopVersion"); + }); + + it("includes the Desktop short version from env", () => { + const source = cpaLiveVoiceHookBootstrap({ + ...environment, + [DESKTOP_VERSION_ENV]: "26.1021.31345", + }); + expect(source).toContain('"desktopVersion":"26.1021.31345"'); + }); +}); diff --git a/packages/desktop-control/test/production-controller.test.ts b/packages/desktop-control/test/production-controller.test.ts index a55a275ff..303c87b51 100644 --- a/packages/desktop-control/test/production-controller.test.ts +++ b/packages/desktop-control/test/production-controller.test.ts @@ -113,7 +113,7 @@ describe("production Desktop Controller", () => { close, }; const ready = vi.fn(); - const install = vi.fn(async () => session); + const install = vi.fn(async () => session); const server = attachmentServer(); let attach: (() => Promise) | undefined; const startAttachmentServer = vi.fn(async (options) => { @@ -131,13 +131,23 @@ describe("production Desktop Controller", () => { await runDesktopController(controllerOptions(), abort.signal, dependencies); - expect(install).toHaveBeenCalledWith({ - rendererCdpEndpoint: "http://127.0.0.1:43123", - rendererSource: - "globalThis.__zod_globalConfig ??= {}; globalThis.__zod_globalConfig.jitless = true;\nproduction renderer", - timeoutMs: 90_000, - signal: abort.signal, - }); + expect(install).toHaveBeenCalledWith( + expect.objectContaining({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + timeoutMs: 90_000, + signal: abort.signal, + }), + ); + const source = install.mock.calls[0]?.[0].rendererSource; + expect(source).toContain( + "globalThis.__zod_globalConfig ??= {}; globalThis.__zod_globalConfig.jitless = true;", + ); + expect(source).toContain("production renderer"); + expect(source).toContain("__codexhostCpaLiveVoiceHookV1"); + expect(install.mock.calls[0]?.[0].overlayHookSource).toContain("__codexhostCpaLiveVoiceHookV1"); + expect(install.mock.calls[0]?.[0].overlayHookSource).toMatch( + /^globalThis\.__zod_globalConfig \?\?= \{\}; globalThis\.__zod_globalConfig\.jitless = true;\n/, + ); expect(startAttachmentServer).toHaveBeenCalledWith({ port: 43124, nonce: attachmentNonce, @@ -156,6 +166,83 @@ describe("production Desktop Controller", () => { expect(attach).toEqual(expect.any(Function)); }); + it("prefixes the CPA live-voice hook by default", async () => { + const previous = process.env.CODEXHOST_CPA_LIVE_VOICE; + delete process.env.CODEXHOST_CPA_LIVE_VOICE; + const abort = new AbortController(); + const snapshot = controllerSnapshot(); + const session: RendererCdpControlSession = { + snapshot, + ensureInstalled: vi.fn(async () => { + abort.abort(); + return snapshot; + }), + activateDesktop: vi.fn(async () => 1), + executeRenderer: vi.fn(), + close: vi.fn(), + }; + const install = vi.fn(async () => session); + try { + await runDesktopController(controllerOptions(), abort.signal, { + readRenderer: vi.fn(async (filePath: string) => + filePath.endsWith("cpa-live-voice-hook.js") ? "overlay hook" : "production renderer", + ), + install, + startAttachmentServer: vi.fn(async () => attachmentServer()), + ready: vi.fn(), + sleep: vi.fn(async () => {}), + monitorIntervalMs: 1, + }); + const source = install.mock.calls[0]?.[0].rendererSource; + expect(source).toContain("globalThis.__codexhostCpaLiveVoiceHookV1 = Object.freeze("); + expect(source).toContain("26.1007.30912"); + expect(source?.endsWith("production renderer")).toBe(true); + expect(install.mock.calls[0]?.[0].overlayHookSource).toContain("overlay hook"); + expect(install.mock.calls[0]?.[0].overlayHookSource).toContain( + "globalThis.__codexhostCpaLiveVoiceHookV1 = Object.freeze(", + ); + } finally { + if (previous === undefined) delete process.env.CODEXHOST_CPA_LIVE_VOICE; + else process.env.CODEXHOST_CPA_LIVE_VOICE = previous; + } + }); + + it("omits the CPA live-voice hook when CODEXHOST_CPA_LIVE_VOICE=0", async () => { + const previous = process.env.CODEXHOST_CPA_LIVE_VOICE; + process.env.CODEXHOST_CPA_LIVE_VOICE = "0"; + const abort = new AbortController(); + const snapshot = controllerSnapshot(); + const session: RendererCdpControlSession = { + snapshot, + ensureInstalled: vi.fn(async () => { + abort.abort(); + return snapshot; + }), + activateDesktop: vi.fn(async () => 1), + executeRenderer: vi.fn(), + close: vi.fn(), + }; + const install = vi.fn(async () => session); + try { + await runDesktopController(controllerOptions(), abort.signal, { + readRenderer: vi.fn(async (filePath: string) => + filePath.endsWith("cpa-live-voice-hook.js") ? "overlay hook" : "production renderer", + ), + install, + startAttachmentServer: vi.fn(async () => attachmentServer()), + ready: vi.fn(), + sleep: vi.fn(async () => {}), + monitorIntervalMs: 1, + }); + const source = install.mock.calls[0]?.[0].rendererSource; + expect(source).not.toContain("globalThis.__codexhostCpaLiveVoiceHookV1"); + expect(install.mock.calls[0]?.[0].overlayHookSource).toBeUndefined(); + } finally { + if (previous === undefined) delete process.env.CODEXHOST_CPA_LIVE_VOICE; + else process.env.CODEXHOST_CPA_LIVE_VOICE = previous; + } + }); + it("publishes readiness before a pending Renderer install finishes", async () => { const abort = new AbortController(); const close = vi.fn(); diff --git a/packages/desktop-control/test/renderer-cdp-control-session.test.ts b/packages/desktop-control/test/renderer-cdp-control-session.test.ts index daab931c8..49c5b59ca 100644 --- a/packages/desktop-control/test/renderer-cdp-control-session.test.ts +++ b/packages/desktop-control/test/renderer-cdp-control-session.test.ts @@ -2,6 +2,7 @@ import { describe, expect, it, vi } from "vitest"; import { createRendererCdpControlSession, + selectAvatarOverlayRendererTargets, selectPrimaryRendererTarget, } from "../src/renderer-cdp-control-session.js"; import type { CdpTarget } from "../src/cdp-client.js"; @@ -62,6 +63,14 @@ describe("Renderer CDP Control Session", () => { expect( selectPrimaryRendererTarget([target("overlay", "app://-/avatar-overlay.html")]), ).toBeNull(); + expect( + selectAvatarOverlayRendererTargets([ + target("overlay", "app://-/index.html?initialRoute=%2Favatar-overlay"), + target("legacy", "app://-/avatar-overlay.html"), + first, + target("query", "app://-/index.html?window=second"), + ]).map((item) => item.id), + ).toEqual(["overlay", "legacy"]); }); it("registers future-document injection before evaluating the current document", async () => { @@ -95,6 +104,189 @@ describe("Renderer CDP Control Session", () => { session.close(); }); + it("injects the hook-only source into the avatar overlay page", async () => { + const primary = rendererClient(); + const overlay = rendererClient(); + overlay.evaluateSpy.mockResolvedValue(false); + const session = await createRendererCdpControlSession({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + rendererSource: "production renderer", + overlayHookSource: "overlay hook only", + pollIntervalMs: 1, + timeoutMs: 100, + operations: { + listTargets: vi.fn(async () => [ + target("page-1"), + target("overlay", "app://-/index.html?initialRoute=%2Favatar-overlay"), + ]), + connect: vi.fn(async (webSocketDebuggerUrl: string) => + webSocketDebuggerUrl.includes("overlay") ? overlay : primary, + ), + installDraftPrewarmPolicy: vi.fn(async () => ({ + state: "ready" as const, + reason: "owned-request-bridge" as const, + })), + }, + }); + + expect(primary.commands).toEqual( + expect.arrayContaining([ + { + method: "Page.addScriptToEvaluateOnNewDocument", + params: { source: "production renderer" }, + }, + ]), + ); + expect(overlay.commands).toEqual([ + { method: "Runtime.enable" }, + { method: "Page.enable" }, + { method: "Page.addScriptToEvaluateOnNewDocument", params: { source: "overlay hook only" } }, + { + method: "Runtime.evaluate", + params: { expression: "overlay hook only", awaitPromise: true }, + }, + ]); + session.close(); + }); + + it("keeps the primary session when overlay CDP connect fails", async () => { + const primary = rendererClient(); + const session = await createRendererCdpControlSession({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + rendererSource: "production renderer", + overlayHookSource: "overlay hook only", + pollIntervalMs: 1, + timeoutMs: 100, + operations: { + listTargets: vi.fn(async () => [ + target("page-1"), + target("overlay", "app://-/index.html?initialRoute=%2Favatar-overlay"), + ]), + connect: vi.fn(async (webSocketDebuggerUrl: string) => { + if (webSocketDebuggerUrl.includes("overlay")) { + throw new Error("overlay closed between list and connect"); + } + return primary; + }), + installDraftPrewarmPolicy: vi.fn(async () => ({ + state: "ready" as const, + reason: "owned-request-bridge" as const, + })), + }, + }); + + expect(session.snapshot.binding).toEqual(readyBinding()); + expect(primary.close).not.toHaveBeenCalled(); + session.close(); + }); + + it("re-injects the hook-only overlay bundle when a reload drops the hook", async () => { + const primary = rendererClient(); + const overlay = rendererClient(); + overlay.evaluateSpy.mockResolvedValue(false); + const session = await createRendererCdpControlSession({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + rendererSource: "production renderer", + overlayHookSource: "overlay hook only", + pollIntervalMs: 1, + timeoutMs: 100, + operations: { + listTargets: vi.fn(async () => [ + target("page-1"), + target("overlay", "app://-/index.html?initialRoute=%2Favatar-overlay"), + ]), + connect: vi.fn(async (webSocketDebuggerUrl: string) => + webSocketDebuggerUrl.includes("overlay") ? overlay : primary, + ), + installDraftPrewarmPolicy: vi.fn(async () => ({ + state: "ready" as const, + reason: "owned-request-bridge" as const, + })), + }, + }); + + overlay.commands.length = 0; + await session.ensureInstalled(); + + expect(overlay.commands).toEqual([ + { method: "Runtime.enable" }, + { method: "Page.enable" }, + { method: "Page.addScriptToEvaluateOnNewDocument", params: { source: "overlay hook only" } }, + { + method: "Runtime.evaluate", + params: { expression: "overlay hook only", awaitPromise: true }, + }, + ]); + expect( + overlay.commands.some( + ({ params }) => + params?.source === "production renderer" || params?.expression === "production renderer", + ), + ).toBe(false); + session.close(); + }); + + it.each(["connect", "command", "evaluate"] as const)( + "bounds stalled overlay %s operations without blocking other overlays", + async (stage) => { + const primary = rendererClient(); + const stalled = rendererClient(); + const healthy = rendererClient(); + healthy.evaluateSpy.mockResolvedValue(false); + let finishConnect: ((client: typeof stalled) => void) | undefined; + const pendingConnect = new Promise((resolve) => { + finishConnect = resolve; + }); + if (stage === "command") { + stalled.command.mockImplementation(() => new Promise(() => {})); + } + if (stage === "evaluate") { + stalled.evaluateSpy.mockImplementation(() => new Promise(() => {})); + } + const session = await createRendererCdpControlSession({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + rendererSource: "production renderer", + overlayHookSource: "overlay hook only", + timeoutMs: 40, + operations: { + listTargets: async () => [ + target("page-1"), + target("stalled", "app://-/avatar-overlay.html"), + target("healthy", "app://-/avatar-overlay.html"), + ], + connect: async (url) => { + if (url.endsWith("/stalled")) return stage === "connect" ? pendingConnect : stalled; + return url.endsWith("/healthy") ? healthy : primary; + }, + installDraftPrewarmPolicy: async () => ({ + state: "ready", + reason: "owned-request-bridge", + }), + }, + }); + try { + expect(session.snapshot.binding).toEqual(readyBinding()); + expect(primary.close).not.toHaveBeenCalled(); + expect(healthy.commands).toContainEqual({ + method: "Runtime.evaluate", + params: { expression: "overlay hook only", awaitPromise: true }, + }); + expect(healthy.close).toHaveBeenCalledOnce(); + if (stage === "connect") { + expect(stalled.close).not.toHaveBeenCalled(); + finishConnect?.(stalled); + await vi.waitFor(() => expect(stalled.close).toHaveBeenCalledOnce()); + expect(stalled.command).not.toHaveBeenCalled(); + } else { + expect(stalled.close).toHaveBeenCalledOnce(); + } + await expect(session.ensureInstalled()).resolves.toMatchObject({ binding: readyBinding() }); + } finally { + session.close(); + } + }, + ); + it("reconciles a changed Host before judging the previous Adapter readiness", async () => { const binding = readyBinding(); const client = rendererClient(binding); @@ -216,6 +408,44 @@ describe("Renderer CDP Control Session", () => { expect(client.close).not.toHaveBeenCalled(); }); + it("keeps the primary session when overlay CDP discovery hangs past its timeout", async () => { + const primary = rendererClient(); + let calls = 0; + const session = await createRendererCdpControlSession({ + rendererCdpEndpoint: "http://127.0.0.1:43123", + rendererSource: "production renderer", + overlayHookSource: "overlay hook only", + pollIntervalMs: 10, + timeoutMs: 40, + operations: { + listTargets: vi.fn((_endpoint, signal) => { + calls += 1; + if (calls % 2 === 1) return Promise.resolve([target("page-1")]); + return new Promise((_resolve, reject) => { + const fail = (): void => reject(new Error("Overlay CDP target discovery timed out")); + if (signal?.aborted) { + fail(); + return; + } + signal?.addEventListener("abort", fail, { once: true }); + }); + }), + connect: vi.fn(async () => primary), + installDraftPrewarmPolicy: vi.fn(async () => ({ + state: "ready" as const, + reason: "owned-request-bridge" as const, + })), + }, + }); + + expect(session.snapshot.binding).toEqual(readyBinding()); + await expect(session.ensureInstalled()).resolves.toMatchObject({ + binding: readyBinding(), + }); + expect(primary.close).not.toHaveBeenCalled(); + session.close(); + }); + it("fails closed when the injected Adapter is unsupported", async () => { const client = rendererClient({ version: 2, diff --git a/packages/harness-adapter/src/index.ts b/packages/harness-adapter/src/index.ts index 1e4c3cb85..b696d506e 100644 --- a/packages/harness-adapter/src/index.ts +++ b/packages/harness-adapter/src/index.ts @@ -62,6 +62,7 @@ export type { HostEvent, HostFileChange, HostFileChangeItem, + HostGeneratedImageItem, HistoricalTurnOutcome, HostInteraction, HostInteractionResponse, diff --git a/packages/harness-adapter/src/testing.ts b/packages/harness-adapter/src/testing.ts index 1e5bb4f82..d66a749f4 100644 --- a/packages/harness-adapter/src/testing.ts +++ b/packages/harness-adapter/src/testing.ts @@ -660,6 +660,18 @@ export class FakeHarnessSession implements HarnessSession { return itemId; } + emitGeneratedImage(savedPath: string, revisedPrompt?: string): HostItemId { + const itemId = this.#nextItemId(); + this.#startItem({ + type: "generatedImage", + itemId, + savedPath, + ...(revisedPrompt ? { revisedPrompt } : {}), + }); + this.completeItem(itemId, { status: "succeeded" }); + return itemId; + } + askQuestion( question: HostQuestion, options: { itemId?: HostItemId; title?: string; expiresAt?: string } = {}, diff --git a/packages/harness-adapter/src/text-session.ts b/packages/harness-adapter/src/text-session.ts index ba857ccbc..3058cf107 100644 --- a/packages/harness-adapter/src/text-session.ts +++ b/packages/harness-adapter/src/text-session.ts @@ -386,6 +386,15 @@ export interface HostFileChangeItem { sourceItemIds?: HostItemId[]; } +/** Desktop `imageGeneration` gallery. Requires an absolute saved file. */ +export interface HostGeneratedImageItem { + type: "generatedImage"; + itemId: HostItemId; + savedPath: string; + revisedPrompt?: string; + sourceItemIds?: HostItemId[]; +} + export type HostSubagentStatus = "pending" | "running" | "completed" | "failed" | "interrupted"; export interface HostSubagentState { @@ -418,6 +427,7 @@ export type HostItem = | HostCommandExecutionItem | HostToolExecutionItem | HostFileChangeItem + | HostGeneratedImageItem | HostSubagentDelegationItem; export type HostItemUpdate = diff --git a/packages/harness-discovery/src/index.ts b/packages/harness-discovery/src/index.ts index 3d3421e20..f1de27d48 100644 --- a/packages/harness-discovery/src/index.ts +++ b/packages/harness-discovery/src/index.ts @@ -29,6 +29,7 @@ export { fetchInstallationText, installationVersion, newerInstallationVersion, + readLauncherScript, runInstallationCommand, versionFromOutput, type InstallationState, diff --git a/packages/harness-discovery/src/installation.ts b/packages/harness-discovery/src/installation.ts index a7853c77e..57cb85fa7 100644 --- a/packages/harness-discovery/src/installation.ts +++ b/packages/harness-discovery/src/installation.ts @@ -1,4 +1,5 @@ import { execFile } from "node:child_process"; +import { open } from "node:fs/promises"; import { homedir } from "node:os"; import { commandInvocation } from "./invocation.js"; import { withNodeRuntimeOnPath } from "./node-runtime.js"; @@ -15,6 +16,19 @@ export interface InstallationState { latestVersionKind?: "unknown" | "tracking-branch"; } +/** Read a bounded launcher header without loading a native executable into memory. */ +export async function readLauncherScript(command: string): Promise { + const file = await open(command, "r"); + try { + const buffer = Buffer.alloc(4096); + const { bytesRead } = await file.read(buffer, 0, buffer.length, 0); + const prefix = buffer.toString("utf8", 0, bytesRead); + return prefix.startsWith("#!") || /\.(cmd|bat|ps1)$/i.test(command) ? prefix : ""; + } finally { + await file.close(); + } +} + /** Fixed adapter-owned commands only. No shell, prompt input, or raw output in errors. */ export function runInstallationCommand( command: string, diff --git a/packages/harness-discovery/test/installation.test.ts b/packages/harness-discovery/test/installation.test.ts index eb1d7680f..9ba545bcc 100644 --- a/packages/harness-discovery/test/installation.test.ts +++ b/packages/harness-discovery/test/installation.test.ts @@ -1,8 +1,12 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; import { describe, expect, it, vi } from "vitest"; import { createInstallationManager, installationVersion, newerInstallationVersion, + readLauncherScript, runInstallationCommand, } from "../src/installation.js"; @@ -14,6 +18,26 @@ const state = (current = "1.0.0", canUpdate = true) => ({ }); describe("Harness installation versions", () => { + it("reads only a launcher prefix and ignores native binary contents", async () => { + const root = await mkdtemp(path.join(os.tmpdir(), "harness-launcher-")); + try { + const launcher = path.join(root, "grok"); + await writeFile(launcher, `#!/bin/sh\nGROKGOD_HOME=custom\n${"x".repeat(8192)}`); + const text = await readLauncherScript(launcher); + expect(text).toHaveLength(4096); + expect(text).toContain("GROKGOD_HOME=custom"); + const native = path.join(root, "native"); + await writeFile(native, Buffer.from([0xcf, 0xfa, 0xed, 0xfe, 0])); + expect(await readLauncherScript(native)).toBe(""); + const windows = path.join(root, "grok.cmd"); + await writeFile(windows, "@echo off\r\nset GROKGOD_HOME=custom\r\n"); + expect(await readLauncherScript(windows)).toContain("GROKGOD_HOME=custom"); + await expect(readLauncherScript(path.join(root, "missing"))).rejects.toThrow(); + } finally { + await rm(root, { recursive: true, force: true }); + } + }); + it("validates versions and never downgrades a newer local build", () => { expect(installationVersion("v1.2.3")).toBe("1.2.3"); expect(newerInstallationVersion("1.2.3", "1.2.2")).toBe(false); diff --git a/packages/harness-plugin-files/src/grok-computer-use-lock.ts b/packages/harness-plugin-files/src/grok-computer-use-lock.ts new file mode 100644 index 000000000..cbf9a9ddf --- /dev/null +++ b/packages/harness-plugin-files/src/grok-computer-use-lock.ts @@ -0,0 +1,40 @@ +import { mkdir, rm } from "node:fs/promises"; +import path from "node:path"; +import { setTimeout as delay } from "node:timers/promises"; + +/** Both preference and Grok config owners participate, including console and Host + * processes. An abandoned lock fails closed; it is never stolen from a writer. + */ +export async function withGrokComputerUseLock( + dataDirectory: string, + grokHome: string, + operation: () => Promise, +): Promise { + const locks = [ + ...new Set([ + path.join(dataDirectory, ".grok-computer-use.lock"), + path.join(grokHome, ".codexhost-cua.lock"), + ]), + ].sort(); + const acquired: string[] = []; + const deadline = Date.now() + 5_000; + try { + for (const lock of locks) { + await mkdir(path.dirname(lock), { recursive: true, mode: 0o700 }); + for (;;) { + try { + await mkdir(lock, { mode: 0o700 }); + acquired.push(lock); + break; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error; + if (Date.now() >= deadline) throw new Error("Grok Computer Use settings are locked"); + await delay(25); + } + } + } + return await operation(); + } finally { + for (const lock of acquired.reverse()) await rm(lock, { recursive: true }); + } +} diff --git a/packages/harness-plugin-files/src/grok-computer-use.ts b/packages/harness-plugin-files/src/grok-computer-use.ts new file mode 100644 index 000000000..bbbf3489b --- /dev/null +++ b/packages/harness-plugin-files/src/grok-computer-use.ts @@ -0,0 +1,431 @@ +import { randomUUID } from "node:crypto"; +import { access, mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import type { GrokComputerUseSettings } from "@codexhost/shared-contracts"; +import { withGrokComputerUseLock } from "./grok-computer-use-lock.js"; + +export const GROK_CUA_REPL_SERVER = "cua_repl"; +/** ChatGPT Desktop does not authorize a Grok-spawned cua_repl for IAB. */ +export const GROK_BROWSER_BINDING_SUPPORTED = false; +const PREFERENCE_FILE = "grok-computer-use.json"; +const TABLE = "[mcp_servers.cua_repl]"; +const ENV_TABLE = "[mcp_servers.cua_repl.env]"; + +export interface GrokComputerUseLaunch { + node: string; + cuaRepl: string; + nodeRepl: string; + nodeModules: string; + codexCli: string; + skyApp: string; + codexHome: string; + browserDesktop: string; +} + +interface Preference { + enabled?: boolean; + browser?: boolean; + applyError?: string; +} + +export interface GrokComputerUseUpdate { + enabled?: boolean | undefined; + browser?: boolean | undefined; +} + +const COMPUTER_UNAVAILABLE = + "ChatGPT Desktop Computer Use files were not found. Install ChatGPT.app with Computer Use, then try again."; +const BROWSER_FILES_UNAVAILABLE = + "ChatGPT Desktop in-app browser files were not found. Update ChatGPT.app, then try again."; +const BROWSER_UNSUPPORTED = + "ChatGPT Desktop does not authorize Grok to use the in-app browser. The preference is saved as intent until Desktop supports an external Harness."; +const APPLY_FAILED = "Could not apply Grok Computer Use settings to Grok config."; + +export class GrokComputerUseError extends Error { + constructor(message: string) { + super(message); + this.name = "GrokComputerUseError"; + } +} + +function dataDirectory(environment: NodeJS.ProcessEnv): string { + return environment.CODEXHOST_DATA_DIR + ? path.resolve(environment.CODEXHOST_DATA_DIR) + : path.join(os.homedir(), ".codexhost"); +} + +function grokHome(environment: NodeJS.ProcessEnv): string { + return environment.GROK_HOME + ? path.resolve(environment.GROK_HOME) + : path.join(environment.HOME ?? os.homedir(), ".grok"); +} + +function chatgptApp(environment: NodeJS.ProcessEnv, platform: string): string | undefined { + if (environment.CODEXHOST_CHATGPT_APP) return path.resolve(environment.CODEXHOST_CHATGPT_APP); + if (platform === "darwin") return "/Applications/ChatGPT.app"; + return undefined; +} + +async function isFile(target: string): Promise { + try { + await access(target); + return true; + } catch { + return false; + } +} + +function isNotFound(error: unknown): boolean { + return Boolean(error && typeof error === "object" && "code" in error && error.code === "ENOENT"); +} + +async function writeAtomically(file: string, content: string): Promise { + await mkdir(path.dirname(file), { recursive: true, mode: 0o700 }); + const temporary = `${file}.${randomUUID()}.tmp`; + try { + await writeFile(temporary, content, { mode: 0o600, flag: "wx" }); + await rename(temporary, file); + } finally { + await rm(temporary, { force: true }); + } +} + +function launchPaths( + environment: NodeJS.ProcessEnv, + platform: string, +): GrokComputerUseLaunch | undefined { + const app = chatgptApp(environment, platform); + if (!app) return undefined; + const resources = path.join(app, "Contents", "Resources"); + const cuaNode = path.join(resources, "cua_node"); + return { + node: path.join(cuaNode, "bin", "node"), + cuaRepl: path.join(cuaNode, "lib", "node_modules", "@oai", "cua-repl", "bin", "cua-repl.mjs"), + nodeRepl: path.join(cuaNode, "bin", "node_repl"), + nodeModules: path.join(cuaNode, "lib", "node_modules"), + codexCli: path.join(resources, "codex-cli", "bin", "codex"), + skyApp: path.join( + environment.HOME ?? os.homedir(), + ".codex", + "computer-use", + "Codex Computer Use.app", + ), + codexHome: path.join(environment.HOME ?? os.homedir(), ".codex"), + browserDesktop: path.join(cuaNode, "lib", "node_modules", "@oai", "browser-desktop"), + }; +} + +async function sharedRuntimePresent(launch: GrokComputerUseLaunch): Promise { + const present = await Promise.all([launch.node, launch.cuaRepl, launch.nodeRepl].map(isFile)); + return present.every(Boolean); +} + +async function computerFilesPresent(launch: GrokComputerUseLaunch): Promise { + const present = await Promise.all([isFile(launch.skyApp), isFile(launch.codexCli)]); + return present.every(Boolean); +} + +async function browserFilesPresent(launch: GrokComputerUseLaunch): Promise { + return isFile(path.join(launch.browserDesktop, "package.json")); +} + +/** Shared cua_node runtime. Sky and browser-desktop are checked separately. */ +export async function discoverGrokComputerUseLaunch( + environment: NodeJS.ProcessEnv = process.env, + platform: string = process.platform, +): Promise { + const launch = launchPaths(environment, platform); + if (!launch || !(await sharedRuntimePresent(launch))) return undefined; + return launch; +} + +function tableRange(text: string, header: string): { start: number; end: number } | undefined { + const lines = text.split(/\n/); + let start = -1; + for (let index = 0; index < lines.length; index += 1) { + if (lines[index] === header) { + start = index; + break; + } + } + if (start < 0) return undefined; + let end = lines.length; + for (let index = start + 1; index < lines.length; index += 1) { + const line = lines[index] ?? ""; + if (line.startsWith("[") && line.endsWith("]")) { + end = index; + break; + } + } + return { start, end }; +} + +function removeTables(text: string, headers: string[]): string { + let current = text; + for (const header of headers) { + const range = tableRange(current, header); + if (!range) continue; + const lines = current.split(/\n/); + lines.splice(range.start, range.end - range.start); + current = lines.join("\n"); + } + return current.replace(/\n{3,}/g, "\n\n"); +} + +function cuaReplEnabledInToml(text: string): boolean { + const range = tableRange(text, TABLE); + if (!range) return false; + const body = text.split(/\n/).slice(range.start, range.end); + for (const line of body) { + if (/^enabled\s*=\s*false\b/u.test(line)) return false; + if (/^enabled\s*=\s*true\b/u.test(line)) return true; + } + return true; +} + +function inferredSurfaces(text: string): { enabled: boolean; browser: boolean } { + if (!cuaReplEnabledInToml(text)) return { enabled: false, browser: false }; + const range = tableRange(text, ENV_TABLE); + const body = range ? text.split(/\n/).slice(range.start, range.end).join("\n") : ""; + const literal = /^CUA_REPL_ENABLED_SURFACES\s*=\s*(["'])([^"']*)\1\s*(?:#.*)?$/mu.exec(body); + if (!literal) return { enabled: true, browser: false }; + const surfaces = new Set(literal[2]?.split(",").map((surface) => surface.trim())); + return { enabled: surfaces.has("computer"), browser: surfaces.has("browser") }; +} + +function stanza( + launch: GrokComputerUseLaunch, + state: Required>, +): string { + const trusted = `${launch.codexHome}:${launch.nodeModules}`; + const services: Record = {}; + if (state.browser) services.browser = "@oai/browser-desktop/service"; + if (state.enabled) services.sky = "@oai/sky/service"; + const surfaces = [state.browser ? "browser" : "", state.enabled ? "computer" : ""] + .filter(Boolean) + .join(","); + const backends = state.browser ? 'BROWSER_USE_AVAILABLE_BACKENDS = "iab"\n' : ""; + const active = state.enabled || state.browser; + return `${TABLE} +command = ${JSON.stringify(launch.node)} +args = [${JSON.stringify(launch.cuaRepl)}] +enabled = ${active ? "true" : "false"} +startup_timeout_sec = 120 + +${ENV_TABLE} +NODE_REPL_TRUSTED_SERVICES = ${JSON.stringify(JSON.stringify(services))} +NODE_REPL_TRUSTED_CODE_PATHS = ${JSON.stringify(trusted)} +NODE_REPL_NODE_PATH = ${JSON.stringify(launch.node)} +CODEX_HOME = ${JSON.stringify(launch.codexHome)} +CUA_REPL_NODE_REPL_PATH = ${JSON.stringify(launch.nodeRepl)} +NODE_REPL_NODE_MODULE_DIRS = ${JSON.stringify(launch.nodeModules)} +NODE_REPL_NATIVE_PIPE_CONNECT_TIMEOUT_MS = "1000" +SKY_CUA_SERVICE_PATH = ${JSON.stringify(launch.skyApp)} +CODEX_CLI_PATH = ${JSON.stringify(launch.codexCli)} +${backends}CUA_REPL_ENABLED_SURFACES = ${JSON.stringify(surfaces)} +`; +} + +async function readPreference(environment: NodeJS.ProcessEnv): Promise { + try { + const document = JSON.parse( + await readFile(path.join(dataDirectory(environment), PREFERENCE_FILE), "utf8"), + ) as { enabled?: unknown; browser?: unknown; applyError?: unknown }; + return { + ...(typeof document.enabled === "boolean" ? { enabled: document.enabled } : {}), + ...(typeof document.browser === "boolean" ? { browser: document.browser } : {}), + ...(typeof document.applyError === "string" && document.applyError + ? { applyError: document.applyError } + : {}), + }; + } catch (error) { + if (isNotFound(error)) return undefined; + throw new GrokComputerUseError("Could not read Grok Computer Use settings"); + } +} + +async function writePreference(environment: NodeJS.ProcessEnv, state: Preference): Promise { + const document: Preference = { + ...(typeof state.enabled === "boolean" ? { enabled: state.enabled } : {}), + ...(typeof state.browser === "boolean" ? { browser: state.browser } : {}), + ...(state.applyError ? { applyError: state.applyError } : {}), + }; + await writeAtomically( + path.join(dataDirectory(environment), PREFERENCE_FILE), + `${JSON.stringify(document)}\n`, + ); +} + +async function readGrokConfig(environment: NodeJS.ProcessEnv): Promise { + try { + return await readFile(path.join(grokHome(environment), "config.toml"), "utf8"); + } catch (error) { + if (isNotFound(error)) return ""; + throw new GrokComputerUseError("Could not read Grok config"); + } +} + +async function writeGrokConfig(environment: NodeJS.ProcessEnv, text: string): Promise { + await writeAtomically( + path.join(grokHome(environment), "config.toml"), + text.endsWith("\n") ? text : `${text}\n`, + ); +} + +async function currentState( + environment: NodeJS.ProcessEnv, + preference?: Preference, +): Promise<{ + enabled: boolean; + browser: boolean; + applyError?: string; +}> { + const resolved = preference ?? (await readPreference(environment)); + const inferred = + resolved?.enabled === undefined + ? inferredSurfaces(await readGrokConfig(environment)) + : undefined; + return { + enabled: resolved?.enabled ?? inferred?.enabled ?? false, + browser: resolved ? (resolved.browser ?? false) : (inferred?.browser ?? false), + ...(resolved?.applyError ? { applyError: resolved.applyError } : {}), + }; +} + +function settingsFrom( + state: { enabled: boolean; browser: boolean; applyError?: string }, + files: { computer: boolean; browser: boolean }, +): GrokComputerUseSettings { + const bindable = files.browser && GROK_BROWSER_BINDING_SUPPORTED; + return { + enabled: state.enabled, + available: files.computer, + ...(files.computer ? {} : { unavailableReason: COMPUTER_UNAVAILABLE }), + browser: state.browser, + browserFilesPresent: files.browser, + browserAvailable: bindable, + ...(bindable + ? {} + : { + browserUnavailableReason: files.browser ? BROWSER_UNSUPPORTED : BROWSER_FILES_UNAVAILABLE, + }), + ...(state.applyError ? { applyFailed: true, applyError: state.applyError } : {}), + }; +} + +async function surfaceFiles( + launch: GrokComputerUseLaunch | undefined, +): Promise<{ computer: boolean; browser: boolean }> { + if (!launch) return { computer: false, browser: false }; + const [computer, browser] = await Promise.all([ + computerFilesPresent(launch), + browserFilesPresent(launch), + ]); + return { computer, browser }; +} + +export async function getGrokComputerUseSettings( + environment: NodeJS.ProcessEnv = process.env, + platform: string = process.platform, +): Promise { + const launch = await discoverGrokComputerUseLaunch(environment, platform); + return settingsFrom(await currentState(environment), await surfaceFiles(launch)); +} + +async function writeConfigForState( + state: { enabled: boolean; browser: boolean }, + launch: GrokComputerUseLaunch | undefined, + files: { computer: boolean; browser: boolean }, + environment: NodeJS.ProcessEnv, +): Promise { + const current = await readGrokConfig(environment); + const range = tableRange(current, TABLE); + const effective = { + enabled: state.enabled && files.computer, + browser: state.browser && files.browser && GROK_BROWSER_BINDING_SUPPORTED, + }; + if (!effective.enabled && !effective.browser && !range) return; + if (!launch) { + if (!range) return; + const lines = current.split(/\n/); + const body = lines.slice(range.start, range.end); + const replaced = body.map((line) => (/^enabled\s*=/u.test(line) ? "enabled = false" : line)); + if (!replaced.some((line) => /^enabled\s*=/u.test(line))) { + replaced.splice(1, 0, "enabled = false"); + } + lines.splice(range.start, range.end - range.start, ...replaced); + await writeGrokConfig(environment, lines.join("\n")); + return; + } + const stripped = removeTables(current, [ENV_TABLE, TABLE]).replace(/\s+$/u, "\n"); + await writeGrokConfig(environment, `${stripped.trimEnd()}\n\n${stanza(launch, effective)}`); +} + +export async function setGrokComputerUse( + update: GrokComputerUseUpdate, + environment: NodeJS.ProcessEnv = process.env, + platform: string = process.platform, +): Promise { + return withGrokComputerUseLock(dataDirectory(environment), grokHome(environment), async () => { + const launch = await discoverGrokComputerUseLaunch(environment, platform); + const files = await surfaceFiles(launch); + if (update.enabled && !files.computer) { + throw new GrokComputerUseError(COMPUTER_UNAVAILABLE); + } + if (update.browser && !files.browser) { + throw new GrokComputerUseError(BROWSER_FILES_UNAVAILABLE); + } + const previous = await currentState(environment); + const state = { + enabled: update.enabled ?? previous.enabled, + browser: update.browser ?? previous.browser, + }; + try { + await writeConfigForState(state, launch, files, environment); + await writePreference(environment, state); + } catch (error) { + await writeConfigForState(previous, launch, files, environment).catch(() => undefined); + const applyError = error instanceof GrokComputerUseError ? error.message : APPLY_FAILED; + await writePreference(environment, { ...previous, applyError }).catch(() => undefined); + if (error instanceof GrokComputerUseError) throw error; + throw new GrokComputerUseError(APPLY_FAILED); + } + return settingsFrom(state, files); + }); +} + +export function setGrokComputerUseEnabled( + enabled: boolean, + environment: NodeJS.ProcessEnv = process.env, + platform: string = process.platform, +): Promise { + return setGrokComputerUse({ enabled }, environment, platform); +} + +/** Apply the saved CodexHost preference to Grok config for a new session. Missing preference is a no-op. */ +export async function applyGrokComputerUsePreference( + environment: NodeJS.ProcessEnv = process.env, + platform: string = process.platform, +): Promise { + await withGrokComputerUseLock(dataDirectory(environment), grokHome(environment), async () => { + const preference = await readPreference(environment); + if (preference === undefined) return; + const launch = await discoverGrokComputerUseLaunch(environment, platform); + const files = await surfaceFiles(launch); + const state = await currentState(environment, preference); + try { + await writeConfigForState(state, launch, files, environment); + if (state.applyError) + await writePreference(environment, { enabled: state.enabled, browser: state.browser }); + } catch (error) { + const applyError = error instanceof GrokComputerUseError ? error.message : APPLY_FAILED; + await writePreference(environment, { + enabled: state.enabled, + browser: state.browser, + applyError, + }).catch(() => undefined); + throw error instanceof GrokComputerUseError ? error : new GrokComputerUseError(APPLY_FAILED); + } + }); +} diff --git a/packages/harness-plugin-files/src/index.ts b/packages/harness-plugin-files/src/index.ts index 6633d6a16..eefd67456 100644 --- a/packages/harness-plugin-files/src/index.ts +++ b/packages/harness-plugin-files/src/index.ts @@ -5,6 +5,16 @@ export { type InstalledHarnessPlugin, } from "./discovery.js"; export { HarnessLaunchSettingsStore } from "./launch-settings.js"; +export { + GROK_BROWSER_BINDING_SUPPORTED, + GROK_CUA_REPL_SERVER, + GrokComputerUseError, + applyGrokComputerUsePreference, + discoverGrokComputerUseLaunch, + getGrokComputerUseSettings, + setGrokComputerUse, + setGrokComputerUseEnabled, +} from "./grok-computer-use.js"; export { pluginResourcePath, readPluginConfiguration, diff --git a/packages/harness-plugin-files/test/grok-computer-use.test.ts b/packages/harness-plugin-files/test/grok-computer-use.test.ts new file mode 100644 index 000000000..26922d881 --- /dev/null +++ b/packages/harness-plugin-files/test/grok-computer-use.test.ts @@ -0,0 +1,365 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +import { afterEach, beforeEach, describe, expect, it } from "vitest"; + +import { + applyGrokComputerUsePreference, + getGrokComputerUseSettings, + setGrokComputerUse, + setGrokComputerUseEnabled, +} from "../src/grok-computer-use.js"; + +let root: string; + +async function chatgptTree(options: { browser?: boolean; sky?: boolean } = {}): Promise { + const app = path.join(root, "ChatGPT.app"); + const cua = path.join(app, "Contents", "Resources", "cua_node"); + await mkdir(path.join(cua, "bin"), { recursive: true }); + await mkdir(path.join(cua, "lib", "node_modules", "@oai", "cua-repl", "bin"), { + recursive: true, + }); + await mkdir(path.join(app, "Contents", "Resources", "codex-cli", "bin"), { recursive: true }); + await writeFile(path.join(cua, "bin", "node"), ""); + await writeFile(path.join(cua, "bin", "node_repl"), ""); + await writeFile( + path.join(cua, "lib", "node_modules", "@oai", "cua-repl", "bin", "cua-repl.mjs"), + "", + ); + await writeFile(path.join(app, "Contents", "Resources", "codex-cli", "bin", "codex"), ""); + if (options.browser) { + const browserDesktop = path.join(cua, "lib", "node_modules", "@oai", "browser-desktop"); + await mkdir(browserDesktop, { recursive: true }); + await writeFile(path.join(browserDesktop, "package.json"), "{}"); + } + if (options.sky !== false) { + await mkdir(path.join(root, "home", ".codex", "computer-use", "Codex Computer Use.app"), { + recursive: true, + }); + } + return app; +} + +beforeEach(async () => { + root = await mkdtemp(path.join(os.tmpdir(), "grok-computer-use-")); +}); + +afterEach(async () => { + await rm(root, { recursive: true, force: true }); +}); + +describe("Grok Computer Use settings", () => { + it("defaults off when the user never opted in", async () => { + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + }; + await expect(getGrokComputerUseSettings(environment, "linux")).resolves.toMatchObject({ + enabled: false, + available: false, + }); + }); + + it("infers enabled from an existing cua_repl stanza and can disable it", async () => { + const grokHome = path.join(root, "grok"); + await mkdir(grokHome, { recursive: true }); + await writeFile( + path.join(grokHome, "config.toml"), + `[ui]\nfoo = 1\n\n[mcp_servers.cua_repl]\ncommand = "/bin/echo"\nenabled = true\n\n[privacy]\nx = 1\n`, + ); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + }; + await expect(getGrokComputerUseSettings(environment, "linux")).resolves.toMatchObject({ + enabled: true, + }); + await setGrokComputerUseEnabled(false, environment, "linux"); + expect(await readFile(path.join(grokHome, "config.toml"), "utf8")).toMatch(/enabled = false/); + expect(await readFile(path.join(grokHome, "config.toml"), "utf8")).toContain("[ui]"); + expect(await readFile(path.join(grokHome, "config.toml"), "utf8")).toContain("[privacy]"); + }); + + it("writes a computer-only cua_repl stanza when ChatGPT Computer Use files exist", async () => { + const app = await chatgptTree(); + const grokHome = path.join(root, "grok"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await expect(setGrokComputerUseEnabled(true, environment, "darwin")).resolves.toMatchObject({ + enabled: true, + browser: false, + available: true, + }); + const toml = await readFile(path.join(grokHome, "config.toml"), "utf8"); + expect(toml).toContain("[mcp_servers.cua_repl]"); + expect(toml).toContain('CUA_REPL_ENABLED_SURFACES = "computer"'); + expect(toml).not.toContain("BROWSER_USE_AVAILABLE_BACKENDS"); + expect(toml).toContain('{\\"sky\\":\\"@oai/sky/service\\"}'); + expect(toml).toContain("cua-repl.mjs"); + await applyGrokComputerUsePreference(environment, "darwin"); + expect(await readFile(path.join(grokHome, "config.toml"), "utf8")).toContain("enabled = true"); + }); + + it("refuses to enable when Computer Use files are missing", async () => { + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + }; + await expect(setGrokComputerUseEnabled(true, environment, "darwin")).rejects.toThrow( + /ChatGPT Desktop Computer Use files were not found/, + ); + }); + it("saves browser intent without enabling the IAB surface until Desktop binding is supported", async () => { + const app = await chatgptTree({ browser: true }); + const grokHome = path.join(root, "grok"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await setGrokComputerUse({ enabled: true }, environment, "darwin"); + await expect( + setGrokComputerUse({ browser: true }, environment, "darwin"), + ).resolves.toMatchObject({ + enabled: true, + browser: true, + browserFilesPresent: true, + browserAvailable: false, + }); + const toml = await readFile(path.join(grokHome, "config.toml"), "utf8"); + expect(toml).toContain('CUA_REPL_ENABLED_SURFACES = "computer"'); + expect(toml).not.toContain("BROWSER_USE_AVAILABLE_BACKENDS"); + expect(toml).not.toContain("browser-desktop"); + }); + + it("does not let an inherited binding override claim IAB availability", async () => { + const app = await chatgptTree({ browser: true }); + const grokHome = path.join(root, "grok"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + CODEXHOST_GROK_IAB_BINDING: "1", + }; + await expect( + setGrokComputerUse({ enabled: true, browser: true }, environment, "darwin"), + ).resolves.toMatchObject({ browser: true, browserAvailable: false }); + const toml = await readFile(path.join(grokHome, "config.toml"), "utf8"); + expect(toml).toContain('CUA_REPL_ENABLED_SURFACES = "computer"'); + expect(toml).not.toContain("browser-desktop"); + expect(toml).not.toContain("BROWSER_USE_AVAILABLE_BACKENDS"); + await setGrokComputerUse({ enabled: false }, environment, "darwin"); + expect(await readFile(path.join(grokHome, "config.toml"), "utf8")).toContain("enabled = false"); + }); + + it("lets browser intent be saved when Sky is absent", async () => { + const app = await chatgptTree({ browser: true, sky: false }); + const grokHome = path.join(root, "grok"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + available: false, + browserFilesPresent: true, + browserAvailable: false, + }); + await expect(setGrokComputerUse({ enabled: true }, environment, "darwin")).rejects.toThrow( + /Computer Use files were not found/, + ); + await expect( + setGrokComputerUse({ browser: true }, environment, "darwin"), + ).resolves.toMatchObject({ browser: true, available: false, browserFilesPresent: true }); + }); + + it("keeps both fields when concurrent partial writes run", async () => { + const app = await chatgptTree({ browser: true }); + const grokHome = path.join(root, "grok"); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await Promise.all([ + setGrokComputerUse({ enabled: true }, environment, "darwin"), + setGrokComputerUse({ browser: true }, environment, "darwin"), + ]); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: true, + browser: true, + }); + }); + + it("merges partial saves from separate Host and console processes", async () => { + const app = await chatgptTree({ browser: true }); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + const moduleUrl = new URL("../dist/grok-computer-use.js", import.meta.url).href; + const run = promisify(execFile); + await Promise.all( + [{ enabled: true }, { browser: true }].map((update) => + run(process.execPath, [ + "--input-type=module", + "-e", + `const {setGrokComputerUse}=await import(process.argv[1]); await setGrokComputerUse(JSON.parse(process.argv[2]), JSON.parse(process.argv[3]), "darwin");`, + moduleUrl, + JSON.stringify(update), + JSON.stringify(environment), + ]), + ), + ); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: true, + browser: true, + browserAvailable: false, + }); + const toml = await readFile(path.join(environment.GROK_HOME, "config.toml"), "utf8"); + expect(toml).toContain('CUA_REPL_ENABLED_SURFACES = "computer"'); + await Promise.all([ + setGrokComputerUse({ enabled: false }, environment, "darwin"), + applyGrokComputerUsePreference(environment, "darwin"), + ]); + expect(await readFile(path.join(environment.GROK_HOME, "config.toml"), "utf8")).toContain( + "enabled = false", + ); + }); + + it("reports a failed config apply without changing saved intent", async () => { + const app = await chatgptTree({ browser: true }); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await setGrokComputerUse({ enabled: false, browser: false }, environment, "darwin"); + await mkdir(path.join(environment.GROK_HOME, "config.toml")); + await expect(setGrokComputerUse({ enabled: true }, environment, "darwin")).rejects.toThrow(); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: false, + browser: false, + applyFailed: true, + }); + await rm(path.join(environment.GROK_HOME, "config.toml"), { recursive: true }); + await applyGrokComputerUsePreference(environment, "darwin"); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: false, + }); + expect((await getGrokComputerUseSettings(environment, "darwin")).applyFailed).toBeUndefined(); + }); + + it("fails closed when in-app browser files are missing", async () => { + const app = await chatgptTree(); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await expect(setGrokComputerUse({ browser: true }, environment, "darwin")).rejects.toThrow( + /in-app browser files were not found/, + ); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + browser: false, + browserAvailable: false, + }); + }); + + it("infers browser-only intent without enabling Computer Use", async () => { + const app = await chatgptTree({ browser: true, sky: false }); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await mkdir(environment.GROK_HOME); + await writeFile( + path.join(environment.GROK_HOME, "config.toml"), + '[mcp_servers.cua_repl]\nenabled = true\n\n[mcp_servers.cua_repl.env]\nCUA_REPL_ENABLED_SURFACES = "browser"\n', + ); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: false, + browser: true, + available: false, + browserAvailable: false, + }); + await setGrokComputerUse({ browser: false }, environment, "darwin"); + expect(await readFile(path.join(environment.GROK_HOME, "config.toml"), "utf8")).toContain( + "enabled = false", + ); + }); + + it("retains saved intent but disables a surface when its files disappear", async () => { + const app = await chatgptTree({ browser: true }); + const environment = { + CODEXHOST_DATA_DIR: path.join(root, "data"), + GROK_HOME: path.join(root, "grok"), + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await setGrokComputerUse({ enabled: true, browser: true }, environment, "darwin"); + await rm(path.join(app, "Contents", "Resources", "codex-cli"), { recursive: true }); + await rm( + path.join( + app, + "Contents", + "Resources", + "cua_node", + "lib", + "node_modules", + "@oai", + "browser-desktop", + ), + { recursive: true }, + ); + await applyGrokComputerUsePreference(environment, "darwin"); + await expect(getGrokComputerUseSettings(environment, "darwin")).resolves.toMatchObject({ + enabled: true, + browser: true, + available: false, + browserFilesPresent: false, + browserAvailable: false, + }); + expect(await readFile(path.join(environment.GROK_HOME, "config.toml"), "utf8")).toContain( + "enabled = false", + ); + }); + + it("keeps a Computer-only preference file working unchanged", async () => { + const app = await chatgptTree({ browser: true }); + const dataDir = path.join(root, "data"); + const grokHome = path.join(root, "grok"); + await mkdir(dataDir, { recursive: true }); + await writeFile(path.join(dataDir, "grok-computer-use.json"), '{"enabled":true}\n'); + const environment = { + CODEXHOST_DATA_DIR: dataDir, + GROK_HOME: grokHome, + HOME: path.join(root, "home"), + CODEXHOST_CHATGPT_APP: app, + }; + await applyGrokComputerUsePreference(environment, "darwin"); + const toml = await readFile(path.join(grokHome, "config.toml"), "utf8"); + expect(toml).toContain('CUA_REPL_ENABLED_SURFACES = "computer"'); + expect(toml).not.toContain("browser-desktop"); + }); +}); diff --git a/packages/host-runtime/package.json b/packages/host-runtime/package.json index e979728cf..2ed4432b2 100644 --- a/packages/host-runtime/package.json +++ b/packages/host-runtime/package.json @@ -26,6 +26,7 @@ "@codexhost/shared-contracts": "0.0.0", "@codexhost/update-manager": "0.0.0", "@deepseek-ai/cordis": "4.0.4", + "smol-toml": "^1.8.0", "ws": "8.21.3" } } diff --git a/packages/host-runtime/scripts/build-release.mjs b/packages/host-runtime/scripts/build-release.mjs index 4a1518c0b..a227dc3e1 100644 --- a/packages/host-runtime/scripts/build-release.mjs +++ b/packages/host-runtime/scripts/build-release.mjs @@ -20,6 +20,7 @@ const allowedRuntimePackages = new Set([ "@deepseek-ai/cordis", "@deepseek-ai/cosmokit", "diff", + "smol-toml", "ws", "zod", ]); diff --git a/packages/host-runtime/src/app-server-host.ts b/packages/host-runtime/src/app-server-host.ts index 28ff005fd..ededea280 100644 --- a/packages/host-runtime/src/app-server-host.ts +++ b/packages/host-runtime/src/app-server-host.ts @@ -1,3 +1,11 @@ +import { + VOICE_SETTINGS_METHOD, + VOICE_GATEWAY_STATUS_METHOD, + voiceSettingsRequestSchema, +} from "@codexhost/shared-contracts"; +import { readRealtimeGateway } from "./realtime-gateway-config.js"; +import { REALTIME_CALL_METHOD } from "@codexhost/shared-contracts"; +import { NativeVoiceGateway } from "./native-voice-gateway.js"; import { DELEGATION_READ_METHOD, delegationReadParamsSchema, @@ -14,7 +22,23 @@ import { import { requestDesktopRemoteConnections } from "@codexhost/desktop-control"; import type { RuntimeMaintenance } from "./runtime-maintenance.js"; import { ExternalThreadPrewarms } from "./external-thread-prewarms.js"; -import { HarnessLaunchSettingsStore } from "@codexhost/harness-plugin-files"; +import { ExternalVoice } from "./external-voice.js"; +import { voiceSessionInputFromTurns } from "./external-voice-session-input.js"; +import { ExternalVoiceResults } from "./external-voice-results.js"; +import { + presentExternalVoiceMessage, + presentExternalVoiceTurn, +} from "./external-voice-presentation.js"; +import { + configuredExternalVoiceGateway, + type ExternalVoiceCallFactory, +} from "./external-voice-gateway.js"; +import { + GrokComputerUseError, + HarnessLaunchSettingsStore, + getGrokComputerUseSettings, + setGrokComputerUse, +} from "@codexhost/harness-plugin-files"; import type { SharedThreadBridge } from "./shared-thread-bridge.js"; import { HARNESS_INSTALLATION_METHOD, @@ -204,6 +228,9 @@ import type { HarnessPluginRegistry } from "./harness-plugin-registry.js"; import { HARNESS_LAUNCH_SETTINGS_GET_METHOD, HARNESS_LAUNCH_SETTINGS_SET_METHOD, + GROK_COMPUTER_USE_GET_METHOD, + GROK_COMPUTER_USE_SET_METHOD, + grokComputerUseSetSchema, harnessLaunchSettingsGetSchema, harnessLaunchSettingsSetSchema, } from "@codexhost/shared-contracts"; @@ -272,6 +299,11 @@ import { import { listSectionThreads, moveThreadSection } from "./external-thread-sections.js"; import { externalThreadListEntries } from "./external-thread-list.js"; import { externalThreadEmptyOfficialResult } from "./external-thread-official-defaults.js"; +import { + ExternalThreadFollowUpQueue, + queuedSubmissionText, + type ExternalQueuedSubmission, +} from "./external-thread-follow-up-queue.js"; import { ModelPriceCatalog } from "./model-prices.js"; import { ObservedSessionUsage } from "./observed-session-usage.js"; import type { UsageStatistics, UsageStatisticsSource } from "./usage-statistics.js"; @@ -361,6 +393,8 @@ export interface AppServerHostOptions { modelPrices?: ModelPriceCatalog; /** Machine-wide usage statistics; one per runtime, read from the loaded Harness plugins. */ usageStatistics?: UsageStatistics; + /** External Harness voice only; official Codex realtime is always native passthrough. */ + createExternalVoiceCall?: ExternalVoiceCallFactory; } interface TurnProjectionGate { @@ -370,6 +404,7 @@ interface TurnProjectionGate { interface ProjectedTurn { projector: CodexTurnProjector; + realtimeInline?: true; } type HostApprovalRequestId = number; @@ -556,26 +591,14 @@ function requestObject(request: JsonRpcRequest): JsonObject { function requestText(params: JsonObject): string { if (!Array.isArray(params.input)) throw new Error("turn/start input must be an array"); - const text = params.input - .filter((item): item is JsonObject => isRecord(item) && item.type === "text") - .map((item) => item.text) - .filter((value): value is string => typeof value === "string") - .join("\n"); + const text = queuedSubmissionText(params.input); if (!text) throw new Error("turn/start must contain text input"); return text; } -function sandboxResult(params: JsonObject): JsonObject { - const sandbox = params.sandbox; - if (sandbox === "read-only") return { type: "readOnly", networkAccess: false }; - if (sandbox === "danger-full-access") return { type: "dangerFullAccess" }; - return { - type: "workspaceWrite", - networkAccess: false, - writableRoots: [], - excludeTmpdirEnvVar: false, - excludeSlashTmp: false, - }; +/** Native protocol type: Codex does not enforce a sandbox on external Threads. */ +function sandboxResult(): JsonObject { + return { type: "externalSandbox", networkAccess: "enabled" }; } function turnProjectionGate(): TurnProjectionGate { @@ -632,6 +655,12 @@ export class AppServerHost { readonly #executionHosts = new WeakMap(); readonly #externalPrewarms = new ExternalThreadPrewarms(); readonly #externalSteering = new ExternalTurnSteering(); + readonly #followUps = new ExternalThreadFollowUpQueue(); + readonly #nativeVoiceGateway: NativeVoiceGateway; + readonly #externalVoice: ExternalVoice; + readonly #externalVoiceResults = new ExternalVoiceResults((threadId, text) => + this.#externalVoice.reportProgress(threadId, text), + ); readonly #liveCommandCache = new LiveCommandCatalogCache(); readonly #modelPrices: ModelPriceCatalog; readonly #observedSessionUsage: ObservedSessionUsage; @@ -694,6 +723,13 @@ export class AppServerHost { return this.#takeConsoleReply(value); }); const environment = this.#options.environment ?? process.env; + this.#nativeVoiceGateway = new NativeVoiceGateway(environment); + this.#externalVoice = new ExternalVoice({ + createCall: options.createExternalVoiceCall ?? configuredExternalVoiceGateway(environment), + notify: (method, params) => this.#writer.json({ method, params }), + diagnose: () => this.#diagnose("External voice cleanup was not confirmed"), + runTask: (threadId, text, signal) => this.#runExternalVoiceTask(threadId, text, signal), + }); this.#launchSettings = new HarnessLaunchSettingsStore( this.#options.pluginContext?.environment ?? environment, ); @@ -788,6 +824,7 @@ export class AppServerHost { idleRelease: { queue: this.#desktopRequests, onClosed: async (thread) => { + await this.#externalVoice.stop(thread.id); for (const pending of [...this.#pendingDesktopApprovals.values()]) { if (pending.thread === thread) await this.#resolveDesktopApproval(pending.interaction.interactionId); @@ -798,6 +835,7 @@ export class AppServerHost { } }, canRelease: (thread) => + !this.#externalVoice.has(thread.id) && !this.#hasRunningSubagents(thread.id) && !thread.session.hasBackgroundWork?.() && !this.#externalSteering.hasPending(thread.id) && @@ -870,6 +908,8 @@ export class AppServerHost { close(): void { if (this.#closeRequested) return; this.#closeRequested = true; + this.#nativeVoiceGateway.close(); + void this.#externalVoice.close(); this.#options.sharedThreads?.close(); this.#externalRuntime.idleRelease.disable(); this.#pluginLoadAbort.abort(); @@ -1012,6 +1052,8 @@ export class AppServerHost { return this.#closeRequested ? 0 : 1; } finally { this.#options.sharedThreads?.close(); + this.#nativeVoiceGateway.close(); + await this.#externalVoice.close(); this.#pluginLoadAbort.abort(); // Stop replacement waiters before waiting for their tracked Host operations. this.#externalSteering.close(); @@ -1308,6 +1350,19 @@ export class AppServerHost { ); return; } + if (request.method === REALTIME_CALL_METHOD && !this.#options.externalOnly) { + try { + const result = await this.#nativeVoiceGateway.request(request.params); + await this.#writer.json(rpcEnvelope(request, { result: jsonValueSchema.parse(result) })); + } catch { + // The Renderer maps every voice failure to one message; never echo request + // payloads, schema details or transport diagnostics back over the bridge. + await this.#writer.json( + rpcError(request, -32090, "Configured voice gateway request failed"), + ); + } + return; + } if (request.method === DELEGATION_READ_METHOD) { const parsed = delegationReadParamsSchema.safeParse(request.params); if (!parsed.success) { @@ -1483,6 +1538,40 @@ export class AppServerHost { this.#dispatchDesktopReply(request, () => this.#handleUpdateRequest(request)); return; } + if ( + request.method === VOICE_SETTINGS_METHOD || + request.method === VOICE_GATEWAY_STATUS_METHOD + ) { + this.#dispatchDesktopReply(request, async () => { + const service = this.#options.consoleOpener; + if (!service?.voiceSettings) { + await this.#writer.json( + rpcError(request, -32090, "Voice settings are available on the local Host only"), + ); + return; + } + if (request.method === VOICE_GATEWAY_STATUS_METHOD) { + const ready = await readRealtimeGateway(this.#options.environment ?? process.env).then( + () => true, + () => false, + ); + await this.#writer.json(rpcEnvelope(request, { result: ready })); + return; + } + const parsed = voiceSettingsRequestSchema.safeParse(request.params); + if (!parsed.success) { + await this.#writer.json(rpcError(request, -32602, "Invalid voice settings request")); + return; + } + try { + const result = await service.voiceSettings(parsed.data); + await this.#writer.json(rpcEnvelope(request, { result: jsonValueSchema.parse(result) })); + } catch (error) { + await this.#writer.json(rpcError(request, -32092, errorMessage(error).slice(0, 500))); + } + }); + return; + } if (request.method === CONSOLE_OPEN_METHOD) { this.#dispatchDesktopReply(request, () => this.#handleConsoleOpen(request)); return; @@ -1620,6 +1709,34 @@ export class AppServerHost { this.#dispatchDesktopReply(request, () => this.#openHarnessWebUi(request)); return; } + if ( + request.method === GROK_COMPUTER_USE_GET_METHOD || + request.method === GROK_COMPUTER_USE_SET_METHOD + ) { + this.#dispatchDesktopReply(request, async () => { + try { + const result = + request.method === GROK_COMPUTER_USE_SET_METHOD + ? await setGrokComputerUse( + grokComputerUseSetSchema.parse(request.params), + process.env, + ) + : await getGrokComputerUseSettings(process.env); + await this.#writer.json(rpcEnvelope(request, { result: jsonValueSchema.parse(result) })); + } catch (error) { + await this.#writer.json( + rpcError( + request, + -32602, + error instanceof GrokComputerUseError + ? error.message + : "Could not read or save Grok Computer Use settings", + ), + ); + } + }); + return; + } if ( request.method === HARNESS_LAUNCH_SETTINGS_GET_METHOD || request.method === HARNESS_LAUNCH_SETTINGS_SET_METHOD @@ -1982,6 +2099,27 @@ export class AppServerHost { this.#pendingOfficialTurnStarts.set(request.id, threadId); } } + if (request.method.startsWith("thread/queue/")) { + const params = routingParams(request); + const threadId = params.threadId; + const resolution = + typeof threadId === "string" + ? await this.#resolveExternalThread(threadId) + : ({ kind: "official" } as const); + if (await this.#writeResolutionError(request, resolution)) return; + if (resolution.kind === "external") { + if (request.method === "thread/queue/start") { + this.#dispatchDesktopReply( + request, + () => this.#startExternalQueuedTurn(request, resolution.thread), + resolution.thread.id, + ); + return; + } + await this.#handleExternalQueueMutation(request, resolution.thread); + return; + } + } if (request.method === "turn/steer") { const params = routingParams(request); const resolution = @@ -2108,6 +2246,22 @@ export class AppServerHost { return; } } + if ( + request.method.startsWith("thread/realtime/") && + isRecord(request.params) && + typeof request.params.threadId === "string" + ) { + const resolution = await this.#resolveExternalThread(request.params.threadId); + if (await this.#writeResolutionError(request, resolution)) return; + if (resolution.kind === "external") { + this.#dispatchDesktopReply( + request, + () => this.#handleExternalVoiceRequest(request, resolution.thread), + resolution.thread.id, + ); + return; + } + } if ( request.method.startsWith("thread/") && !EXPLICIT_EXTERNAL_THREAD_METHODS.has(request.method) && @@ -4132,7 +4286,7 @@ export class AppServerHost { approvalPolicy: typeof params.approvalPolicy === "string" ? params.approvalPolicy : "never", approvalsReviewer: "user", - sandbox: sandboxResult(params), + sandbox: sandboxResult(), reasoningEffort: "medium", serviceTier: "flex", multiAgentMode: "explicitRequestOnly", @@ -4277,9 +4431,6 @@ export class AppServerHost { await this.#writer.json(rpcError(request, result.error.code, result.error.message)); return; } - const params: JsonObject = { - ...(fork.sandbox ? { sandbox: fork.sandbox } : {}), - }; await this.#writer.json( rpcEnvelope(request, { result: threadForkResult(result.responseThread, { @@ -4289,7 +4440,7 @@ export class AppServerHost { ? { runtimeWorkspaceRoots: fork.runtimeWorkspaceRoots } : {}), ...(fork.approvalPolicy ? { approvalPolicy: fork.approvalPolicy } : {}), - sandbox: sandboxResult(params), + sandbox: sandboxResult(), ...(fork.serviceTier ? { serviceTier: fork.serviceTier } : {}), }), }), @@ -4394,6 +4545,7 @@ export class AppServerHost { location: Extract, ): Promise { const thread = location.thread; + await this.#externalVoice.stop(location.record.hostThreadId); try { await this.#repository.removeThread(location.record.hostThreadId); } catch { @@ -4536,15 +4688,19 @@ export class AppServerHost { turns: params.excludeTurns === true ? [] : turns, }; const result = threadForkResult(responseThread, { - model: thread.transportModelId, + model: thread.record.subagent + ? (thread.record.subagentConfiguration?.model ?? "") + : thread.transportModelId, cwd: thread.cwd, runtimeWorkspaceRoots: Array.isArray(params.runtimeWorkspaceRoots) ? params.runtimeWorkspaceRoots.filter((value): value is string => typeof value === "string") : [], approvalPolicy: typeof params.approvalPolicy === "string" ? params.approvalPolicy : "never", - sandbox: sandboxResult(params), + sandbox: sandboxResult(), ...(typeof params.serviceTier === "string" ? { serviceTier: params.serviceTier } : {}), }); + if (thread.record.subagent) + result.reasoningEffort = thread.record.subagentConfiguration?.reasoningEffort ?? null; try { if ( params.initialTurnsPage !== undefined && @@ -4669,6 +4825,191 @@ export class AppServerHost { } } + async #handleExternalVoiceRequest( + request: JsonRpcRequest, + thread: ExternalThread, + ): Promise { + try { + if (request.method === "thread/realtime/start") { + await this.#externalVoice.start( + thread.id, + requestObject(request), + voiceSessionInputFromTurns(thread.turns), + ); + if (thread.activeTurnId && !thread.ephemeralTurnIds.has(thread.activeTurnId)) + this.#followExternalVoiceTurn(thread.id, thread.activeTurnId); + } else if (request.method === "thread/realtime/stop") { + await this.#externalVoice.stop(thread.id); + } else { + await this.#writer.json( + rpcError(request, -32076, "External voice uses the native WebRTC audio transport"), + ); + return; + } + await this.#writer.json(rpcEnvelope(request, { result: {} })); + } catch { + await this.#writer.json( + rpcError(request, -32076, "External voice could not complete this call request"), + ); + } + } + + async #runExternalVoiceTask( + threadId: string, + text: string, + signal: AbortSignal, + ): Promise { + let answer: Promise | undefined; + await this.#desktopRequests.run(threadId, async () => { + if (signal.aborted) throw new Error("Voice session ended"); + const thread = this.#externalRuntime.get(threadId); + if (!thread || this.#closeRequested) throw new Error("External chat is unavailable"); + const started = await this.#beginExternalInputTurn(thread, text); + answer = this.#externalVoiceResults.watch(threadId, started.turnId, signal); + // Attach rejection handling before admitting output; the caller awaits after queue release. + void answer.catch(() => undefined); + started.gate.resolve(); + }); + if (!answer) throw new Error("External voice task did not start"); + return answer; + } + + #followExternalVoiceTurn(threadId: string, turnId: string): void { + if (this.#externalVoiceResults.has(threadId, turnId)) return; + this.#externalVoice.followTurn(threadId, (signal) => + this.#externalVoiceResults.watch(threadId, turnId, signal), + ); + } + + async #notifyFollowUpQueueChanged(threadId: string): Promise { + await this.#writer.json({ + method: "thread/queue/changed", + params: { threadId }, + }); + } + + async #handleExternalQueueMutation( + request: JsonRpcRequest, + thread: ExternalThread, + ): Promise { + const params = requestObject(request); + try { + if (request.method === "thread/queue/list") { + await this.#writer.json(rpcEnvelope(request, { result: this.#followUps.list(thread.id) })); + return; + } + if (request.method === "thread/queue/add") { + if (!Array.isArray(params.input)) + throw new Error("thread/queue/add input must be an array"); + const text = queuedSubmissionText(params.input); + if (!text) throw new Error("thread/queue/add must contain text input"); + const queuedSubmission = this.#followUps.add( + thread.id, + params.input, + typeof params.clientUserMessageId === "string" ? params.clientUserMessageId : undefined, + ); + await this.#writer.json(rpcEnvelope(request, { result: { queuedSubmission } })); + await this.#notifyFollowUpQueueChanged(thread.id); + return; + } + if (request.method === "thread/queue/update") { + if (typeof params.queuedSubmissionId !== "string" || !Array.isArray(params.input)) { + throw new Error("thread/queue/update requires queuedSubmissionId and input"); + } + const queuedSubmission = this.#followUps.update( + thread.id, + params.queuedSubmissionId, + params.input, + ); + if (!queuedSubmission) throw new Error("Queued follow-up was not found"); + await this.#writer.json(rpcEnvelope(request, { result: { queuedSubmission } })); + await this.#notifyFollowUpQueueChanged(thread.id); + return; + } + if (request.method === "thread/queue/delete") { + if (typeof params.queuedSubmissionId !== "string") { + throw new Error("thread/queue/delete requires queuedSubmissionId"); + } + const deleted = this.#followUps.delete(thread.id, params.queuedSubmissionId); + await this.#writer.json(rpcEnvelope(request, { result: { deleted } })); + if (deleted) await this.#notifyFollowUpQueueChanged(thread.id); + return; + } + if (request.method === "thread/queue/reorder") { + if ( + !Array.isArray(params.queuedSubmissionIds) || + params.queuedSubmissionIds.some((id) => typeof id !== "string") + ) { + throw new Error("thread/queue/reorder requires queuedSubmissionIds"); + } + if (!this.#followUps.reorder(thread.id, params.queuedSubmissionIds as string[])) { + throw new Error("Queued follow-up order could not be applied"); + } + await this.#writer.json(rpcEnvelope(request, { result: {} })); + await this.#notifyFollowUpQueueChanged(thread.id); + return; + } + await this.#writer.json( + rpcError(request, -32076, `External Thread does not support ${request.method}`), + ); + } catch (error) { + await this.#writer.json(rpcError(request, -32602, errorMessage(error))); + } + } + + async #startExternalFromFollowUp( + request: JsonRpcRequest, + thread: ExternalThread, + queued: ExternalQueuedSubmission, + sourceHostId?: unknown, + ): Promise { + const text = queuedSubmissionText(queued.input); + if (!text) { + this.#followUps.restore(thread.id, queued); + await this.#writer.json(rpcError(request, -32602, "Queued follow-up was not found")); + return; + } + await this.#notifyFollowUpQueueChanged(thread.id); + try { + const started = await this.#beginExternalInputTurn( + thread, + text, + undefined, + queued.clientUserMessageId, + sourceHostId, + ); + try { + await this.#writer.json(rpcEnvelope(request, { result: { turn: started.turn } })); + } finally { + started.gate.resolve(); + } + } catch (error) { + this.#followUps.restore(thread.id, queued); + await this.#notifyFollowUpQueueChanged(thread.id); + await this.#writer.json( + rpcError( + request, + error instanceof ExternalSteerError || error instanceof ExternalCommandError + ? error.code + : -32073, + errorMessage(error), + ), + ); + } + } + + async #startExternalQueuedTurn(request: JsonRpcRequest, thread: ExternalThread): Promise { + const params = requestObject(request); + const queuedSubmissionId = + typeof params.queuedSubmissionId === "string" ? params.queuedSubmissionId : undefined; + const queued = this.#followUps.take(thread.id, queuedSubmissionId); + if (!queued) { + await this.#writer.json(rpcError(request, -32602, "Queued follow-up was not found")); + return; + } + await this.#startExternalFromFollowUp(request, thread, queued); + } + async #startExternalTurn(request: JsonRpcRequest, thread: ExternalThread): Promise { if (this.#externalThreadBusy(thread) || this.#pendingExternalCommandRequests.has(thread.id)) { await this.#writer.json( @@ -4693,10 +5034,23 @@ export class AppServerHost { } } let text: string; + const clientUserMessageId = + typeof params.clientUserMessageId === "string" ? params.clientUserMessageId : undefined; try { text = requestText(params); } catch (error) { - await this.#writer.json(rpcError(request, -32602, errorMessage(error))); + const queued = + this.#followUps.take(thread.id, clientUserMessageId) ?? this.#followUps.take(thread.id); + if (!queued) { + await this.#writer.json(rpcError(request, -32602, errorMessage(error))); + return; + } + await this.#startExternalFromFollowUp( + request, + thread, + queued, + params[THREAD_REFERENCE_SOURCE_HOST_PARAM], + ); return; } try { @@ -4704,7 +5058,7 @@ export class AppServerHost { thread, text, undefined, - typeof params.clientUserMessageId === "string" ? params.clientUserMessageId : undefined, + clientUserMessageId, params[THREAD_REFERENCE_SOURCE_HOST_PARAM], ); try { @@ -4974,9 +5328,9 @@ export class AppServerHost { !thread.running || thread.activeTurnId !== requestedTurnId ) { - await this.#writer.json( - rpcError(request, -32074, "External turn/interrupt must reference the active Turn"), - ); + // Late overlay stop after the Turn already completed. Desktop retries a + // mismatch error against whichever Turn is now active. + await this.#writer.json(rpcEnvelope(request, { result: {} })); return; } const turnId = thread.activeTurnId; @@ -5177,6 +5531,7 @@ export class AppServerHost { return; } if (event.type === "session.faulted") { + this.#externalVoiceResults.observe(thread.id, event); this.#externalSteering.fault(thread.id, new Error(event.error.message)); thread.stateObserver.fault(new Error(event.error.message)); this.#externalRuntime.markSessionUnavailable(thread); @@ -5229,6 +5584,13 @@ export class AppServerHost { } const projection = this.#projectedTurn(thread, event.turnId); await this.#waitForTurnResponse(thread, event.turnId); + if ( + event.type === "item.completed" && + event.snapshot.item.type === "agentMessage" && + this.#externalVoice.has(thread.id) + ) { + projection.realtimeInline = true; + } if ( event.type === "interaction.closed" && thread.ignoredInteractionIds.delete(event.interactionId) @@ -5262,6 +5624,8 @@ export class AppServerHost { } const result = projection.projector.project(event as ProjectableHostEvent); if (event.type === "turn.started") { + if (!thread.ephemeralTurnIds.has(event.turnId)) + this.#followExternalVoiceTurn(thread.id, event.turnId); await this.#setThreadStatus(thread, { type: "active", activeFlags: [] }); } if (event.type === "turn.completed") { @@ -5271,7 +5635,11 @@ export class AppServerHost { thread.ephemeralTurnIds.delete(event.turnId); this.#manualCompactionTurns.delete(thread); } else { - thread.turns.push(result.completedTurn); + thread.turns.push( + projection.realtimeInline + ? presentExternalVoiceTurn(result.completedTurn) + : result.completedTurn, + ); thread.projectedTerminalTurnId = event.turnId; thread.thread.updatedAt = completedAt; thread.thread.recencyAt = completedAt; @@ -5315,7 +5683,16 @@ export class AppServerHost { ) { thread.projectedTurns.delete(event.turnId); } - for (const message of result.messages) await this.#writer.json(message); + for (const message of result.messages) { + await this.#writer.json( + projection.realtimeInline ? presentExternalVoiceMessage(message) : message, + ); + } + this.#externalVoiceResults.observe( + thread.id, + event, + event.type === "turn.completed" ? result.completedTurn : undefined, + ); if (event.type === "turn.completed") { await this.#setThreadStatus( thread, @@ -5336,6 +5713,15 @@ export class AppServerHost { subagent.status === "pending" || subagent.status === "running" ? "active" : "idle"; const record = await this.#repository.materializeSubagent(parent.record, subagent); if (!record) return subagent; + const loaded = this.#externalRuntime.get(record.hostThreadId); + if (loaded) { + loaded.record = record; + loaded.thread = { + ...loaded.thread, + model: record.subagentConfiguration?.model ?? null, + reasoningEffort: record.subagentConfiguration?.reasoningEffort ?? null, + }; + } if (this.#subagentThreadStatuses.has(record.hostThreadId)) { this.#trackRunningSubagent(parent.id, record.hostThreadId, status); await this.#setSubagentThreadStatus(record.hostThreadId, status); diff --git a/packages/host-runtime/src/codex-runtime/official-runtime-scope.ts b/packages/host-runtime/src/codex-runtime/official-runtime-scope.ts index dcc9952fa..f70df1b55 100644 --- a/packages/host-runtime/src/codex-runtime/official-runtime-scope.ts +++ b/packages/host-runtime/src/codex-runtime/official-runtime-scope.ts @@ -70,6 +70,24 @@ export class OfficialRuntimeScope { return this.#closed; } + /** Wait until this Scope publishes ready, or reject on timeout / close. */ + awaitReady(timeoutMs: number): Promise { + if (this.#closed) return Promise.reject(new OfficialAdmissionError("unavailable")); + if (this.gate.phase === "ready") return Promise.resolve(); + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + unsub(); + reject(new OfficialAdmissionError("unavailable")); + }, timeoutMs); + const unsub = this.gate.subscribe(() => { + if (this.gate.phase !== "ready") return; + clearTimeout(timer); + unsub(); + resolve(); + }); + }); + } + start(): Promise { if (this.#closed) return Promise.reject(new Error("Official Codex is unavailable")); if (this.#starting) return this.#starting; diff --git a/packages/host-runtime/src/codexhost-desktop-rule.ts b/packages/host-runtime/src/codexhost-desktop-rule.ts new file mode 100644 index 000000000..9425bab40 --- /dev/null +++ b/packages/host-runtime/src/codexhost-desktop-rule.ts @@ -0,0 +1,68 @@ +import { createHash } from "node:crypto"; +import { readFile, rm } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +const RULE_FILE = "codexhost-desktop.md"; + +/** Host-owned home-rule copies written before ACP `_meta.rules` replaced them. */ +const MANAGED_DIGESTS: readonly string[] = [ + "5add2d2c7ea03ecbb3552b94e47da7a755909e63f30714878a5609a4e272526f", + "9d8d2762b5f90e6f10a6571651b3461dec882bd45b40f0f0debc432f4833301b", + "4658e68f312686e2134cd282abd9ce434d2583b0a1d23c7c3eb8bd7ec9b76cf3", +]; + +function digest(value: string): string { + return createHash("sha256").update(value).digest("hex"); +} + +function destinations(home: string): string[] { + return [ + path.join(home, ".grok", "rules", RULE_FILE), + path.join(home, ".claude", "rules", RULE_FILE), + ]; +} + +async function readOptional(filePath: string): Promise { + try { + return await readFile(filePath, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; + throw error; + } +} + +export type DesktopRuleRetireStatus = "removed" | "absent" | "conflict"; + +export interface DesktopRuleRetireResult { + path: string; + status: DesktopRuleRetireStatus; + digest: string | null; +} + +/** Remove Host-owned copies. User-edited files stay in place. Does not install a replacement. */ +export async function retireCodexhostDesktopRules( + input: { + homeDirectory?: string; + previousManagedDigests?: readonly string[]; + } = {}, +): Promise { + const home = input.homeDirectory ?? os.homedir(); + const knownDigests = new Set([...MANAGED_DIGESTS, ...(input.previousManagedDigests ?? [])]); + const results: DesktopRuleRetireResult[] = []; + for (const destination of destinations(home)) { + const current = await readOptional(destination); + if (current === null) { + results.push({ path: destination, status: "absent", digest: null }); + continue; + } + const currentDigest = digest(current); + if (!knownDigests.has(currentDigest)) { + results.push({ path: destination, status: "conflict", digest: currentDigest }); + continue; + } + await rm(destination, { force: true }); + results.push({ path: destination, status: "removed", digest: currentDigest }); + } + return results; +} diff --git a/packages/host-runtime/src/console-opener.ts b/packages/host-runtime/src/console-opener.ts index 2b4c22a7e..bf379c93f 100644 --- a/packages/host-runtime/src/console-opener.ts +++ b/packages/host-runtime/src/console-opener.ts @@ -8,6 +8,7 @@ const CONSOLE_URL_PREFIX = "codexhost console: "; export interface HostConsoleOpener { open(): Promise<{ url: string }>; + voiceSettings?(input: unknown): Promise; } /** @@ -65,43 +66,63 @@ export function createHostConsoleOpener( options: CreateHostConsoleOpenerOptions, ): HostConsoleOpener { const spawnProcess = options.spawnProcess ?? spawn; + const address = (command: "open" | "ensure"): Promise<{ url: string }> => + new Promise((resolve, reject) => { + const child = spawnProcess( + options.nodePath ?? process.execPath, + [options.entrypoint, command], + { + env: options.environment, + stdio: ["ignore", "pipe", "pipe"], + windowsHide: true, + }, + ); + let stdout = ""; + let stderr = ""; + const append = (current: string, chunk: Buffer): string => + (current + chunk.toString("utf8")).slice(-OUTPUT_LIMIT); + child.stdout?.on("data", (chunk: Buffer) => (stdout = append(stdout, chunk))); + child.stderr?.on("data", (chunk: Buffer) => (stderr = append(stderr, chunk))); + const timer = setTimeout(() => { + child.kill(); + reject(new Error("codexhost console did not open in time")); + }, OPEN_TIMEOUT_MS); + child.once("error", (error) => { + clearTimeout(timer); + reject(error); + }); + child.once("exit", (code) => { + clearTimeout(timer); + const url = parseConsoleAddress(stdout); + if (code === 0 && url) { + resolve({ url }); + return; + } + const detail = stderr.trim().split(/\r?\n/u).at(-1); + reject(new Error(detail || "codexhost console could not be opened")); + }); + }); + let ready: Promise<{ url: string }> | null = null; return { - open() { - return new Promise((resolve, reject) => { - const child = spawnProcess( - options.nodePath ?? process.execPath, - [options.entrypoint, "open"], - { - env: options.environment, - stdio: ["ignore", "pipe", "pipe"], - windowsHide: true, - }, - ); - let stdout = ""; - let stderr = ""; - const append = (current: string, chunk: Buffer): string => - (current + chunk.toString("utf8")).slice(-OUTPUT_LIMIT); - child.stdout?.on("data", (chunk: Buffer) => (stdout = append(stdout, chunk))); - child.stderr?.on("data", (chunk: Buffer) => (stderr = append(stderr, chunk))); - const timer = setTimeout(() => { - child.kill(); - reject(new Error("codexhost console did not open in time")); - }, OPEN_TIMEOUT_MS); - child.once("error", (error) => { - clearTimeout(timer); - reject(error); + open: () => address("open"), + async voiceSettings(input) { + ready ??= address("ensure"); + try { + const { url } = await ready; + const response = await fetch(new URL("api/voice-settings", url), { + method: "POST", + headers: { "content-type": "application/json", "x-codexhost-console": "1" }, + body: JSON.stringify(input), + redirect: "error", + signal: AbortSignal.timeout(10_000), }); - child.once("exit", (code) => { - clearTimeout(timer); - const url = parseConsoleAddress(stdout); - if (code === 0 && url) { - resolve({ url }); - return; - } - const detail = stderr.trim().split(/\r?\n/u).at(-1); - reject(new Error(detail || "codexhost console could not be opened")); - }); - }); + const body = (await response.json()) as { error?: string }; + if (!response.ok) throw new Error(body.error ?? "Voice settings request failed"); + return body; + } catch (error) { + ready = null; + throw error; + } }, }; } diff --git a/packages/host-runtime/src/external-subagent-threads.ts b/packages/host-runtime/src/external-subagent-threads.ts index bbdae252e..534d81a27 100644 --- a/packages/host-runtime/src/external-subagent-threads.ts +++ b/packages/host-runtime/src/external-subagent-threads.ts @@ -2,10 +2,15 @@ import { createHash, randomUUID } from "node:crypto"; import type { HostSubagentState, HostThreadSnapshot } from "@codexhost/harness-adapter"; import type { StoredThreadRecordV1 } from "@codexhost/mapping-store"; -import { projectHistoricalTurn, type JsonObject } from "@codexhost/protocol-core"; +import { + encodeExternalTransportSelection, + projectHistoricalTurn, + type JsonObject, +} from "@codexhost/protocol-core"; import { hostThreadIdSchema, type NativeSessionRef } from "@codexhost/shared-contracts"; import type { ExternalThreadStore } from "./external-thread-repository.js"; +import { presentExternalVoiceTurn } from "./external-voice-presentation.js"; function childCreateRequest( parent: Pick, @@ -117,13 +122,25 @@ function subagentMaterializer( (previousRequest ? byRequest.get(previousRequest) : undefined) ?? null; } + const configured = (record: StoredThreadRecordV1) => + child.model !== undefined || child.reasoningEffort !== undefined + ? store.updateMetadata(record.hostThreadId, { + subagentConfiguration: { + ...(child.model !== undefined ? { model: child.model } : {}), + ...(child.reasoningEffort !== undefined + ? { reasoningEffort: child.reasoningEffort } + : {}), + }, + }) + : record; if (existing?.state === "ready") { - if (existing.nativeSessionRef?.nativeSessionId === nativeRef.nativeSessionId) return existing; + if (existing.nativeSessionRef?.nativeSessionId === nativeRef.nativeSessionId) + return configured(existing); if (!previousRef) throw new Error("Subagent belongs to a different Native Session"); // Only children retained in a validated rollback Snapshot are rebound. Native // Session IDs remain part of ordinary lookup, so later reused IDs cannot // accidentally select children removed by a previous rollback. - return rebind(existing, parent); + return configured(await rebind(existing, parent)); } // MappingStore atomically deduplicates overlapping live/history creation. @@ -135,7 +152,7 @@ function subagentMaterializer( harnessId: parent.harnessId, cwd: parent.cwd, title: child.description, - transportModelId: parent.transportModelId, + transportModelId: encodeExternalTransportSelection(parent.harnessId, {}), ephemeral: parent.ephemeral, historyMode: "paginated", subagent: { @@ -144,12 +161,14 @@ function subagentMaterializer( ...(child.role ? { role: child.role } : {}), }, })); - return provisional.state === "ready" - ? provisional - : store.commitReady({ - hostThreadId: provisional.hostThreadId, - nativeSessionRef: nativeRef, - }); + return configured( + provisional.state === "ready" + ? provisional + : await store.commitReady({ + hostThreadId: provisional.hostThreadId, + nativeSessionRef: nativeRef, + }), + ); }; } @@ -185,14 +204,13 @@ export async function projectExternalSnapshot( return { ...entry, item: { ...entry.item, subagents } }; }), ); - turns.push( - projectHistoricalTurn({ - threadId: record.hostThreadId, - turnId: mapping.hostTurnId, - cwd: record.cwd, - snapshot: { ...turn, items }, - }), - ); + const projected = projectHistoricalTurn({ + threadId: record.hostThreadId, + turnId: mapping.hostTurnId, + cwd: record.cwd, + snapshot: { ...turn, items }, + }); + turns.push(mapping.realtimeInline ? presentExternalVoiceTurn(projected) : projected); } return turns; } diff --git a/packages/host-runtime/src/external-thread-follow-up-queue.ts b/packages/host-runtime/src/external-thread-follow-up-queue.ts new file mode 100644 index 000000000..d5e1add0a --- /dev/null +++ b/packages/host-runtime/src/external-thread-follow-up-queue.ts @@ -0,0 +1,123 @@ +import { randomUUID } from "node:crypto"; + +import type { JsonObject, JsonValue } from "@codexhost/protocol-core"; + +export type ExternalQueuedSubmission = { + id: string; + clientUserMessageId: string; + input: JsonValue[]; +}; + +function isRecord(value: unknown): value is JsonObject { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +export function queuedSubmissionText(input: JsonValue[]): string { + return input + .filter(isRecord) + .map((item) => item.text) + .filter((value): value is string => typeof value === "string" && value.length > 0) + .join("\n"); +} + +/** Desktop 26.1021 reads `text_elements.length` on every queued `type: "text"` item. */ +export function queuedSubmissionInput(input: JsonValue[]): JsonValue[] { + return input.map((entry) => { + if (!isRecord(entry) || typeof entry.text !== "string" || entry.text.length === 0) { + return entry; + } + if (entry.type === "input_text") { + return { + type: "text", + text: entry.text, + text_elements: Array.isArray(entry.text_elements) ? entry.text_elements : [], + }; + } + if (entry.type !== "text" || Array.isArray(entry.text_elements)) return entry; + return { ...entry, text_elements: [] }; + }); +} + +/** In-memory Desktop follow-up queue for External Threads. Overlay-owned drain uses this. */ +export class ExternalThreadFollowUpQueue { + readonly #threads = new Map(); + + list(threadId: string): { data: ExternalQueuedSubmission[]; nextCursor: null } { + return { data: [...(this.#threads.get(threadId) ?? [])], nextCursor: null }; + } + + add( + threadId: string, + input: JsonValue[], + clientUserMessageId?: string, + ): ExternalQueuedSubmission { + const item: ExternalQueuedSubmission = { + id: randomUUID(), + clientUserMessageId: clientUserMessageId?.trim() || randomUUID(), + input: queuedSubmissionInput(input), + }; + const items = this.#threads.get(threadId) ?? []; + items.push(item); + this.#threads.set(threadId, items); + return item; + } + + update(threadId: string, id: string, input: JsonValue[]): ExternalQueuedSubmission | undefined { + const item = this.#threads.get(threadId)?.find((entry) => entry.id === id); + if (!item) return undefined; + item.input = queuedSubmissionInput(input); + return item; + } + + delete(threadId: string, id: string): boolean { + const items = this.#threads.get(threadId); + if (!items) return false; + const next = items.filter((entry) => entry.id !== id); + if (next.length === items.length) return false; + if (next.length === 0) this.#threads.delete(threadId); + else this.#threads.set(threadId, next); + return true; + } + + reorder(threadId: string, ids: string[]): boolean { + const items = this.#threads.get(threadId); + if (!items) return false; + const remaining = new Map(items.map((entry) => [entry.id, entry])); + const next: ExternalQueuedSubmission[] = []; + for (const id of ids) { + const item = remaining.get(id); + if (!item) return false; + next.push(item); + remaining.delete(id); + } + next.push(...remaining.values()); + this.#threads.set(threadId, next); + return true; + } + + peek(threadId: string, id?: string): ExternalQueuedSubmission | undefined { + const items = this.#threads.get(threadId); + if (!items?.length) return undefined; + return id ? items.find((entry) => this.#matches(entry, id)) : items[0]; + } + + take(threadId: string, id?: string): ExternalQueuedSubmission | undefined { + const items = this.#threads.get(threadId); + if (!items?.length) return undefined; + const index = id ? items.findIndex((entry) => this.#matches(entry, id)) : 0; + if (index < 0) return undefined; + const [item] = items.splice(index, 1); + if (items.length === 0) this.#threads.delete(threadId); + return item; + } + + restore(threadId: string, item: ExternalQueuedSubmission): void { + const items = this.#threads.get(threadId) ?? []; + items.unshift(item); + this.#threads.set(threadId, items); + } + + #matches(entry: ExternalQueuedSubmission, id: string): boolean { + return entry.id === id || entry.clientUserMessageId === id; + } +} diff --git a/packages/host-runtime/src/external-thread-official-defaults.ts b/packages/host-runtime/src/external-thread-official-defaults.ts index 7ec6e0e38..4c051274b 100644 --- a/packages/host-runtime/src/external-thread-official-defaults.ts +++ b/packages/host-runtime/src/external-thread-official-defaults.ts @@ -3,14 +3,13 @@ import type { JsonObject } from "@codexhost/protocol-core"; /** * Official per-Thread requests that clients without the codexhost renderer (such as the ChatGPT * iOS app over SSH) send around every open or send. External Threads have no official - * attachments or queued submissions, and their permissions belong to the Harness, so these answer - * with the empty official response shape rather than an error that aborts the client's flow. + * attachments, and their permissions belong to the Harness, so these answer with the empty + * official response shape rather than an error that aborts the client's flow. * `thread/settings/update` is acknowledged without being applied; the Harness keeps its own - * permission mode. + * permission mode. Follow-up queue list/add/start are handled by the Host queue. */ const EMPTY_RESULTS: ReadonlyMap JsonObject> = new Map([ ["thread/attachment/list", () => ({ data: [], nextCursor: null })], - ["thread/queue/list", () => ({ data: [], nextCursor: null })], ["thread/settings/update", () => ({})], ]); diff --git a/packages/host-runtime/src/external-thread-repository.ts b/packages/host-runtime/src/external-thread-repository.ts index 5e5df525d..3eb578a62 100644 --- a/packages/host-runtime/src/external-thread-repository.ts +++ b/packages/host-runtime/src/external-thread-repository.ts @@ -240,6 +240,7 @@ export class ExternalThreadRepository { hostTurnId: StoredTurnMappingV1["hostTurnId"], nativeTurnRef: NativeTurnRef, nativeCheckpointRef?: NativeCheckpointRef, + realtimeInline?: true, ): Promise { if ( record.state !== "ready" || @@ -257,6 +258,7 @@ export class ExternalThreadRepository { hostTurnId, nativeTurnRef, ...(nativeCheckpointRef ? { nativeCheckpointRef } : {}), + ...(realtimeInline ? { realtimeInline } : {}), }, ]); } @@ -351,6 +353,7 @@ export class ExternalThreadRepository { } return { hostTurnId: previous.hostTurnId, + ...(previous.realtimeInline ? { realtimeInline: true as const } : {}), nativeTurnRef: turn.nativeTurnRef, ...(turn.checkpoint ? { nativeCheckpointRef: turn.checkpoint } : {}), }; @@ -399,6 +402,7 @@ export class ExternalThreadRepository { } return { hostTurnId: previous.hostTurnId, + ...(previous.realtimeInline ? { realtimeInline: true as const } : {}), nativeTurnRef: turn.nativeTurnRef, ...(turn.checkpoint ? { nativeCheckpointRef: turn.checkpoint } : {}), }; @@ -475,6 +479,7 @@ export class ExternalThreadRepository { snapshot: turn, mapping: { hostTurnId: mapping.hostTurnId, + ...(mapping.realtimeInline ? { realtimeInline: true as const } : {}), nativeTurnRef: turn.nativeTurnRef, ...(turn.checkpoint ? { nativeCheckpointRef: turn.checkpoint } : {}), }, @@ -592,6 +597,12 @@ export function externalThreadValue(input: { } : "vscode", threadSource: record.subagent ? "subAgentThreadSpawn" : null, + ...(record.subagent + ? { + model: record.subagentConfiguration?.model ?? null, + reasoningEffort: record.subagentConfiguration?.reasoningEffort ?? null, + } + : {}), modelProvider: "codexhost", cliVersion: "codexhost", createdAt, diff --git a/packages/host-runtime/src/external-thread-runtime.ts b/packages/host-runtime/src/external-thread-runtime.ts index 83a9525fc..71f4766ed 100644 --- a/packages/host-runtime/src/external-thread-runtime.ts +++ b/packages/host-runtime/src/external-thread-runtime.ts @@ -76,7 +76,7 @@ export interface ExternalThread { /** Host metering for the current Session only; replaced with it. */ usageMeter: UsageMeter; usageTurnId: HostTurnId | null; - projectedTurns: Map; + projectedTurns: Map; responseGates: Map; pendingSteerProjections: Set>; ephemeralTurnIds: Set; @@ -769,6 +769,7 @@ export class ExternalThreadRuntime { event.turnId, event.nativeTurnRef, event.outcome.checkpoint, + thread.projectedTurns.get(event.turnId)?.realtimeInline, ); return null; } catch (error) { diff --git a/packages/host-runtime/src/external-voice-gateway.ts b/packages/host-runtime/src/external-voice-gateway.ts new file mode 100644 index 000000000..07d5b9924 --- /dev/null +++ b/packages/host-runtime/src/external-voice-gateway.ts @@ -0,0 +1,147 @@ +import WebSocket from "ws"; +import { + hangupRealtimeCall, + readRealtimeGateway, + realtimeGatewayUrl, +} from "./realtime-gateway-config.js"; +import { isRealtimeCallId, realtimeCallIdFromLocation } from "@codexhost/shared-contracts"; +import type { JsonObject } from "@codexhost/protocol-core"; + +export interface ExternalVoiceCall { + sdp: string; + /** Opens control only after the Host has sent the SDP answer to Desktop. */ + connect(): Promise; + send(event: JsonObject): void; + close(): Promise; +} + +export type ExternalVoiceCallFactory = (input: { + sdp: string; + /** A Desktop-owned call already created through the configured gateway. */ + callId?: string; + voice?: string; + signal: AbortSignal; + /** Prior Thread messages for GPT-Live `session.initial_items`; omitted when empty. */ + input?: JsonObject[]; + onEvent(event: JsonObject): void; + onError(): void; +}) => Promise; + +function object(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** Read only the configured gateway; never borrow OAuth from auth.json. */ +export function configuredExternalVoiceGateway( + environment: NodeJS.ProcessEnv, +): ExternalVoiceCallFactory { + return async (input) => { + const gateway = await readRealtimeGateway(environment); + const { sideband, token, model } = gateway; + const headers = { Authorization: `Bearer ${token}`, "OpenAI-Alpha": "quicksilver=v2" }; + if (input.callId !== undefined && !isRealtimeCallId(input.callId)) + throw new Error("External voice call identity is invalid"); + let callId: string | undefined = input.callId; + let response: Response | undefined; + if (callId === undefined) { + const form = new FormData(); + form.append("sdp", input.sdp); + form.append( + "session", + JSON.stringify({ + model: model ?? "gpt-realtime", + instructions: + "Use natural Hong Kong Cantonese. Delegate every user question and task to the client, including simple arithmetic. " + + "The client agent owns all reasoning and task execution. Do not answer questions yourself or invent task results. " + + "Speak only verified client results; do not read private reasoning or machine identifiers.", + audio: { output: { voice: input.voice ?? "cove" } }, + delegation: { type: "client", ack_filler: false }, + ...(input.input && input.input.length > 0 ? { initial_items: input.input } : {}), + }), + ); + response = await fetch(realtimeGatewayUrl(gateway, "/realtime/calls"), { + method: "POST", + headers, + body: form, + redirect: "error", + signal: AbortSignal.any([input.signal, AbortSignal.timeout(15_000)]), + }); + if (!response.ok) throw new Error("External voice call creation failed"); + callId = realtimeCallIdFromLocation(response.headers.get("location") ?? ""); + if (!callId) { + await response?.body?.cancel().catch(() => undefined); + throw new Error( + "External voice call identity is unavailable; remote cleanup is unconfirmed", + ); + } + } + let socket: WebSocket | undefined; + let closed = false; + let closeTask: Promise | undefined; + const close = (): Promise => + (closeTask ??= (async () => { + closed = true; + socket?.terminate(); + try { + await hangupRealtimeCall(gateway, callId); + } catch { + throw new Error("External voice hangup was not confirmed"); + } + })()); + try { + const sdp = response ? await response.text() : ""; + if (input.signal.aborted) { + await close(); + throw new Error("External voice start cancelled"); + } + return { + sdp, + async connect() { + if (closed || input.signal.aborted) throw new Error("External voice call is closed"); + sideband.protocol = "wss:"; + sideband.pathname = `${sideband.pathname.replace(/\/$/u, "")}/live/${callId}`; + const options: WebSocket.ClientOptions & { autoSelectFamily: boolean } = { + headers, + autoSelectFamily: true, + handshakeTimeout: 12_000, + maxPayload: 1 << 20, + }; + const connection = new WebSocket(sideband, options); + socket = connection; + connection.on("error", () => { + if (!closed) input.onError(); + }); + connection.on("close", () => { + if (!closed) input.onError(); + }); + connection.on("message", (raw) => { + if (closed || input.signal.aborted) return; + try { + const event: unknown = JSON.parse(String(raw)); + if (object(event) && typeof event.type === "string") + input.onEvent(event as JsonObject); + } catch { + input.onError(); + } + }); + await new Promise((resolve, reject) => { + connection.once("open", resolve); + connection.once("error", () => + reject(new Error("External voice control connection failed")), + ); + }); + if (closed) throw new Error("External voice call is closed"); + }, + send(event) { + if (closed || socket?.readyState !== WebSocket.OPEN) + throw new Error("External voice control connection is unavailable"); + socket.send(JSON.stringify(event)); + }, + close, + }; + } catch { + await close(); + throw new Error("External voice call startup failed"); + } + }; +} diff --git a/packages/host-runtime/src/external-voice-presentation.ts b/packages/host-runtime/src/external-voice-presentation.ts new file mode 100644 index 000000000..6b69e308f --- /dev/null +++ b/packages/host-runtime/src/external-voice-presentation.ts @@ -0,0 +1,40 @@ +import type { JsonObject, JsonValue } from "@codexhost/protocol-core"; + +const INLINE_MARKDOWN = "::codex-realtime-inline{}\n"; +const inlineMarker = /^::codex-realtime-inline\{\}\r?\n/u; + +export function externalVoiceContentText(text: string): string { + return text.replace(inlineMarker, ""); +} + +function object(value: JsonValue | undefined): value is JsonObject { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function presentItem(item: JsonValue): JsonValue { + if (!object(item) || item.type !== "agentMessage" || typeof item.text !== "string") return item; + if (!item.text.trim() || inlineMarker.test(item.text)) return item; + return { ...item, text: INLINE_MARKDOWN + item.text }; +} + +/** Desktop's native voice timeline discovers exact Markdown through this directive. */ +export function presentExternalVoiceTurn(turn: JsonObject): JsonObject { + if (!Array.isArray(turn.items)) return turn; + return { ...turn, items: turn.items.map(presentItem) }; +} + +/** Presentation only: neither Harness history nor the speech context receives the directive. */ +export function presentExternalVoiceMessage(message: JsonObject): JsonObject { + const params = message.params; + if (!object(params)) return message; + if ((message.method === "item/started" || message.method === "item/completed") && params.item) { + return { ...message, params: { ...params, item: presentItem(params.item) } }; + } + if ( + (message.method === "turn/started" || message.method === "turn/completed") && + object(params.turn) + ) { + return { ...message, params: { ...params, turn: presentExternalVoiceTurn(params.turn) } }; + } + return message; +} diff --git a/packages/host-runtime/src/external-voice-results.ts b/packages/host-runtime/src/external-voice-results.ts new file mode 100644 index 000000000..22c5863f4 --- /dev/null +++ b/packages/host-runtime/src/external-voice-results.ts @@ -0,0 +1,133 @@ +import type { HostEvent } from "@codexhost/harness-adapter"; +import { externalVoiceContentText } from "./external-voice-presentation.js"; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** Speakable Agent Message text already projected for Desktop, including inferred final answers. */ +export function speakableFromProjectedTurn(turn: unknown): string | undefined { + if (!isRecord(turn) || !Array.isArray(turn.items)) return undefined; + const messages = turn.items.filter( + (item): item is Record & { text: string } => + isRecord(item) && item.type === "agentMessage" && typeof item.text === "string", + ); + return speakableFromItems(messages); +} + +function speakableFromItems( + items: Iterable<{ text: string; phase?: unknown }>, +): string | undefined { + const values = [...items]; + const finals = values.filter((item) => item.phase === "final_answer"); + const text = ( + finals.length ? finals : values.filter((item) => item.phase !== "commentary").slice(-1) + ) + .map((item) => externalVoiceContentText(item.text)) + .join("\n") + .trim(); + return text.length > 0 ? text : undefined; +} + +/** Fan-out from the existing Host consumer; never consumes HarnessSession.outputs. */ +export class ExternalVoiceResults { + constructor(readonly onProgress?: (threadId: string, text: string) => void) {} + + readonly #pending = new Map< + string, + { + threadId: string; + items: Map; + resolve(text: string): void; + reject(error: Error): void; + dispose(): void; + } + >(); + + has(threadId: string, turnId: string): boolean { + return this.#pending.get(turnId)?.threadId === threadId; + } + + watch(threadId: string, turnId: string, signal: AbortSignal): Promise { + if (signal.aborted) return Promise.reject(new Error("Voice session ended")); + return new Promise((resolve, reject) => { + const abort = (): void => { + this.#pending.delete(turnId); + reject(new Error("Voice session ended")); + }; + signal.addEventListener("abort", abort, { once: true }); + this.#pending.set(turnId, { + threadId, + items: new Map(), + resolve, + reject, + dispose: () => signal.removeEventListener("abort", abort), + }); + }); + } + + observe(threadId: string, event: HostEvent, projectedTurn?: unknown): void { + if (event.type === "session.faulted") { + for (const [turnId, pending] of this.#pending) { + if (pending.threadId !== threadId) continue; + this.#pending.delete(turnId); + pending.dispose(); + pending.reject(new Error("External agent session failed")); + } + return; + } + if (!("turnId" in event)) return; + const pending = this.#pending.get(event.turnId); + if (!pending || pending.threadId !== threadId) return; + if (event.type === "item.started") { + // A later Item makes an unphased, completed Agent Message progress rather + // than the Turn's inferred final answer. Never publish partial text. + for (const item of pending.items.values()) { + if (!item.completed || item.phase !== undefined) continue; + item.phase = "commentary"; + if (item.text.trim()) this.onProgress?.(threadId, item.text); + } + } + if (event.type === "item.started" && event.item.type === "agentMessage") { + pending.items.set(event.item.itemId, { + text: event.item.text, + ...(event.item.phase ? { phase: event.item.phase } : {}), + }); + } + if (event.type === "item.updated" && event.update.type === "text.append") { + const item = pending.items.get(event.itemId); + if (item) item.text += event.update.text; + } + if ( + event.type === "item.completed" && + event.snapshot.item.type === "agentMessage" && + event.snapshot.outcome.status === "succeeded" + ) { + const item = event.snapshot.item; + const previous = pending.items.get(item.itemId); + if (item.phase === "commentary" && item.text.trim() && !previous?.completed) { + this.onProgress?.(threadId, item.text); + } + const phase = item.phase ?? previous?.phase; + pending.items.set(item.itemId, { + text: item.text, + ...(phase ? { phase } : {}), + completed: true, + }); + } + if (event.type !== "turn.completed") return; + this.#pending.delete(event.turnId); + pending.dispose(); + if (event.outcome.status !== "succeeded") { + pending.reject(new Error("External agent task did not complete")); + return; + } + const text = + speakableFromItems(pending.items.values()) ?? speakableFromProjectedTurn(projectedTurn); + if (!text) { + pending.reject(new Error("External agent returned no spoken result")); + return; + } + pending.resolve(text); + } +} diff --git a/packages/host-runtime/src/external-voice-session-input.ts b/packages/host-runtime/src/external-voice-session-input.ts new file mode 100644 index 000000000..a2b66c0a6 --- /dev/null +++ b/packages/host-runtime/src/external-voice-session-input.ts @@ -0,0 +1,67 @@ +import type { JsonObject } from "@codexhost/protocol-core"; +import { externalVoiceContentText } from "./external-voice-presentation.js"; + +const MAX_MESSAGES = 32; +const MAX_MESSAGE_CHARS = 2_000; +const MAX_TOTAL_CHARS = 12_000; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function userMessageText(item: Record): string { + if (!Array.isArray(item.content)) return ""; + return item.content + .filter(isRecord) + .map((part) => (part.type === "text" && typeof part.text === "string" ? part.text.trim() : "")) + .filter((text) => text.length > 0) + .join("\n"); +} + +function clip(text: string): string { + return text.length <= MAX_MESSAGE_CHARS ? text : text.slice(0, MAX_MESSAGE_CHARS); +} + +function liveMessage(role: "user" | "assistant", text: string): JsonObject { + return { + type: "message", + role, + content: [{ type: role === "user" ? "input_text" : "output_text", text: clip(text) }], + }; +} + +function messageChars(message: JsonObject): number { + const content = message.content; + if (!Array.isArray(content) || !isRecord(content[0]) || typeof content[0].text !== "string") { + return 0; + } + return content[0].text.length; +} + +/** GPT-Live `session.initial_items` history from an existing External Thread's projected Turns. */ +export function voiceSessionInputFromTurns(turns: readonly unknown[]): JsonObject[] { + const messages: JsonObject[] = []; + for (const turn of turns) { + if (!isRecord(turn) || !Array.isArray(turn.items)) continue; + for (const item of turn.items) { + if (!isRecord(item)) continue; + if (item.type === "userMessage") { + const text = userMessageText(item); + if (text) messages.push(liveMessage("user", text)); + } else if (item.type === "agentMessage" && item.phase !== "commentary") { + const text = + typeof item.text === "string" ? externalVoiceContentText(item.text).trim() : ""; + if (text) messages.push(liveMessage("assistant", text)); + } + } + } + const selected: JsonObject[] = []; + let total = 0; + for (const message of messages.slice(-MAX_MESSAGES).reverse()) { + const size = messageChars(message); + if (total + size > MAX_TOTAL_CHARS) break; + total += size; + selected.push(message); + } + return selected.reverse(); +} diff --git a/packages/host-runtime/src/external-voice.ts b/packages/host-runtime/src/external-voice.ts new file mode 100644 index 000000000..01455bba1 --- /dev/null +++ b/packages/host-runtime/src/external-voice.ts @@ -0,0 +1,373 @@ +import { randomUUID } from "node:crypto"; +import { isRealtimeCallId } from "@codexhost/shared-contracts"; +import type { JsonObject } from "@codexhost/protocol-core"; +import type { ExternalVoiceCall, ExternalVoiceCallFactory } from "./external-voice-gateway.js"; + +interface VoiceTask { + id: string; + text: string; + result?: string; + delegationId?: string; + spokenFor?: string | null; +} + +interface PendingHandle { + id: string; + turnId: string; +} + +interface VoiceSession { + controller: AbortController; + ready: { promise: Promise; resolve(value: undefined): void }; + call?: ExternalVoiceCall; + creation?: Promise; + seen: Set; + tasks: Map; + pendingHandles: PendingHandle[]; + initializationTimeout?: ReturnType; + stopping?: Promise; +} + +function object(value: unknown): value is JsonObject { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +const voices = new Set([ + "juniper", + "maple", + "spruce", + "ember", + "vale", + "breeze", + "arbor", + "sol", + "cove", +]); + +const FAILED_TASK_RESULT = + "The selected agent could not complete this request. No completion is confirmed. Check the chat for its current task status."; + +function protocolId(value: unknown): string | undefined { + return typeof value === "string" && value.length > 0 && value.length <= 256 ? value : undefined; +} + +function userTurnId(event: JsonObject, turn: JsonObject): string | undefined { + return protocolId(turn.id) ?? protocolId(event.event_id); +} + +function delegationTurnId(event: JsonObject, item: JsonObject): string | undefined { + return ( + protocolId(item.user_bidi_turn_id) ?? + protocolId(item.turn_id) ?? + protocolId(item.turnId) ?? + protocolId(event.turn_id) ?? + (object(event.turn) ? protocolId(event.turn.id) : undefined) + ); +} + +/** External-only native call lifecycle. Native Codex requests never enter this owner. */ +export class ExternalVoice { + readonly #sessions = new Map(); + constructor( + readonly options: { + createCall: ExternalVoiceCallFactory; + notify(method: string, params: JsonObject): Promise; + runTask(threadId: string, text: string, signal: AbortSignal): Promise; + diagnose(): void; + }, + ) {} + + has(threadId: string): boolean { + return this.#sessions.has(threadId); + } + + /** Follow work submitted through the chat while this call is open. */ + followTurn(threadId: string, result: (signal: AbortSignal) => Promise): void { + const session = this.#sessions.get(threadId); + if (!session || session.controller.signal.aborted) return; + void (async () => { + let text: string; + try { + text = await result(session.controller.signal); + } catch { + text = FAILED_TASK_RESULT; + } + await this.#reportContext(threadId, session, text); + })().catch(() => this.#fail(threadId, session)); + } + + reportProgress(threadId: string, text: string): void { + const session = this.#sessions.get(threadId); + if (!session || session.controller.signal.aborted) return; + void this.#reportContext(threadId, session, text).catch(() => this.#fail(threadId, session)); + } + + async #reportContext(threadId: string, session: VoiceSession, text: string): Promise { + await session.ready.promise; + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) return; + this.#appendContext(session, text); + } + + async start(threadId: string, params: JsonObject, conversation?: JsonObject[]): Promise { + if (this.has(threadId)) throw new Error("This chat already has a voice call"); + const transport = object(params.transport) ? params.transport : undefined; + const callId = transport?.callId; + const existingCall = transport?.type === "existingCall" && isRealtimeCallId(callId); + const sdp = transport?.type === "webrtc" ? transport.sdp : undefined; + const webrtc = typeof sdp === "string" && sdp.startsWith("v=0") && sdp.length <= 256_000; + if ((!existingCall && !webrtc) || params.outputModality !== "audio") + throw new Error("External voice requires the native WebRTC audio call transport"); + if (params.voice != null && (typeof params.voice !== "string" || !voices.has(params.voice))) { + throw new Error("Selected voice is unavailable for the external live-call protocol"); + } + const session: VoiceSession = { + controller: new AbortController(), + ready: Promise.withResolvers(), + seen: new Set(), + tasks: new Map(), + pendingHandles: [], + }; + this.#sessions.set(threadId, session); + const requireActive = (): void => { + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) + throw new Error("External voice start cancelled"); + }; + try { + session.creation = this.options.createCall({ + sdp: webrtc ? String(sdp) : "", + ...(existingCall ? { callId: String(callId) } : {}), + ...(typeof params.voice === "string" ? { voice: params.voice } : {}), + ...(conversation && conversation.length > 0 ? { input: conversation } : {}), + signal: session.controller.signal, + onEvent: (event) => { + void this.#event(threadId, session, event).catch(() => this.#fail(threadId, session)); + }, + onError: () => { + void this.#fail(threadId, session); + }, + }); + session.call = await session.creation; + if (session.controller.signal.aborted) { + await session.call.close(); + throw new Error("External voice start cancelled"); + } + requireActive(); + await this.options.notify("thread/realtime/started", { + threadId, + realtimeSessionId: + typeof params.realtimeSessionId === "string" ? params.realtimeSessionId : randomUUID(), + version: "v3", + }); + requireActive(); + if (!existingCall) + await this.options.notify("thread/realtime/sdp", { threadId, sdp: session.call.sdp }); + requireActive(); + session.initializationTimeout = setTimeout(() => { + void this.#fail(threadId, session); + }, 20_000); + session.initializationTimeout.unref(); + void session.call.connect().then( + () => session.ready.resolve(undefined), + () => this.#fail(threadId, session), + ); + } catch { + await this.#fail(threadId, session); + throw new Error("External voice could not start"); + } + } + + async stop(threadId: string, reason = "requested"): Promise { + const session = this.#sessions.get(threadId); + if (!session) return; + if (session.stopping) return session.stopping; + session.controller.abort(); + session.ready.resolve(undefined); + clearTimeout(session.initializationTimeout); + session.stopping = (async () => { + let failed = false; + try { + const call = session.call ?? (await session.creation?.catch(() => undefined)); + await call?.close(); + } catch { + failed = true; + this.options.diagnose(); + } finally { + if (this.#sessions.get(threadId) === session) this.#sessions.delete(threadId); + await this.options.notify("thread/realtime/closed", { + threadId, + reason: failed ? "cleanup_unconfirmed" : reason, + }); + } + if (failed) throw new Error("External voice cleanup was not confirmed"); + })(); + return session.stopping; + } + + async close(): Promise { + await Promise.allSettled([...this.#sessions.keys()].map((id) => this.stop(id))); + } + + async #fail(threadId: string, session: VoiceSession): Promise { + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) return; + try { + await this.options.notify("thread/realtime/error", { + threadId, + message: "External voice connection failed. Retry the call explicitly.", + }); + } catch { + this.options.diagnose(); + } finally { + await this.stop(threadId, "error").catch(() => undefined); + } + } + + async #event(threadId: string, session: VoiceSession, event: JsonObject): Promise { + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) return; + if (event.type === "error") { + await this.#fail(threadId, session); + return; + } + if ( + (event.type === "session.started" || event.type === "session.updated") && + object(event.session) && + typeof event.session.id === "string" + ) { + clearTimeout(session.initializationTimeout); + return; + } + if (event.type === "input_transcript.added" || event.type === "output_transcript.added") { + if (object(event.item) && typeof event.item.text === "string") { + await this.options.notify("thread/realtime/transcript/delta", { + threadId, + role: event.type === "input_transcript.added" ? "user" : "assistant", + delta: event.item.text, + }); + } + return; + } + if (event.type === "turn.done" && object(event.turn)) { + const userTask = + String(event.turn.role) === "user" && typeof event.turn.transcript === "string" + ? this.#registerTask(session, userTurnId(event, event.turn), event.turn.transcript) + : undefined; + if ( + ["user", "assistant"].includes(String(event.turn.role)) && + typeof event.turn.transcript === "string" + ) { + await this.options.notify("thread/realtime/transcript/done", { + threadId, + role: String(event.turn.role), + text: event.turn.transcript, + }); + } + if (userTask) await this.#startTask(threadId, session, userTask); + return; + } + if (event.type !== "delegation.created" || !object(event.item)) return; + const item = event.item; + const delegationId = protocolId(item.id); + if (item.type !== "delegation" || item.target !== "client" || !delegationId) return; + if (session.seen.has(delegationId)) return; + if (session.seen.size >= 512) { + await this.#fail(threadId, session); + return; + } + session.seen.add(delegationId); + const turnId = delegationTurnId(event, item); + if (turnId) this.#attachHandle(session, turnId, delegationId); + const text = Array.isArray(item.content) + ? item.content + .filter(object) + .map((part) => + part.type === "input_text" && typeof part.text === "string" ? part.text : "", + ) + .filter(Boolean) + .join("\n") + : ""; + if (turnId && text) { + const task = this.#registerTask(session, turnId, text); + if (task) await this.#startTask(threadId, session, task); + } + } + + #attachHandle(session: VoiceSession, turnId: string, delegationId: string): void { + const byId = session.tasks.get(turnId); + if (byId) { + if (!byId.delegationId) { + byId.delegationId = delegationId; + if (byId.result) this.#speak(session, byId); + } + return; + } + if (session.pendingHandles.length < 512) + session.pendingHandles.push({ id: delegationId, turnId }); + } + + #takePendingHandle(session: VoiceSession, task: VoiceTask): void { + if (task.delegationId) return; + const byTurn = session.pendingHandles.findIndex((handle) => handle.turnId === task.id); + if (byTurn < 0) return; + const [handle] = session.pendingHandles.splice(byTurn, 1); + if (handle) task.delegationId = handle.id; + } + + #registerTask( + session: VoiceSession, + turnId: string | undefined, + text: string, + ): VoiceTask | undefined { + const transcript = text.trim(); + if (!transcript || transcript.length > 32_000) return; + const id = turnId ?? `user-turn:${randomUUID()}`; + if (session.tasks.has(id) || session.tasks.size >= 512) return; + const task: VoiceTask = { id, text: transcript }; + session.tasks.set(id, task); + this.#takePendingHandle(session, task); + return task; + } + + async #startTask(threadId: string, session: VoiceSession, task: VoiceTask): Promise { + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) return; + let result: string; + try { + result = await this.options.runTask(threadId, task.text, session.controller.signal); + } catch { + result = FAILED_TASK_RESULT; + } + task.result = result; + if (session.controller.signal.aborted || this.#sessions.get(threadId) !== session) return; + this.#speak(session, task); + } + + #speak(session: VoiceSession, task: VoiceTask): void { + const result = task.result; + const call = session.call; + if (!result || !call) return; + const spokenFor = task.delegationId ?? null; + if (task.spokenFor === spokenFor) return; + task.spokenFor = spokenFor; + this.#appendContext(session, result, task.delegationId); + } + + #appendContext(session: VoiceSession, text: string, delegationId?: string): void { + const call = session.call; + if (!call) return; + const characters = [...text].slice(0, 8_000); + for (let offset = 0; offset < characters.length; offset += 120) { + const chunk = characters.slice(offset, offset + 120).join(""); + call.send( + delegationId + ? { + type: "delegation.context.append", + delegation_item_id: delegationId, + channel: "speakable", + content: [{ type: "input_text", text: chunk }], + } + : { + type: "session.context.append", + content: [{ type: "input_text", text: chunk }], + }, + ); + } + } +} diff --git a/packages/host-runtime/src/index.ts b/packages/host-runtime/src/index.ts index 471544b40..bdcb8168b 100644 --- a/packages/host-runtime/src/index.ts +++ b/packages/host-runtime/src/index.ts @@ -38,6 +38,7 @@ export { runDelegationCli, DELEGATION_HELP } from "./delegation-cli.js"; export { DelegationControlRegistry } from "./delegation-control-registry.js"; export { startDelegationControlServer } from "./delegation-control-server.js"; export { installDelegationSkills, CODEXHOST_DELEGATION_SKILL } from "./delegation-skill.js"; +export { retireCodexhostDesktopRules } from "./codexhost-desktop-rule.js"; export { DELEGATION_CLI_PATH_ENV, DELEGATION_CLI_NODE_PATH_ENV, diff --git a/packages/host-runtime/src/native-account-host.ts b/packages/host-runtime/src/native-account-host.ts index f5b723cb4..874cc7275 100644 --- a/packages/host-runtime/src/native-account-host.ts +++ b/packages/host-runtime/src/native-account-host.ts @@ -8,7 +8,11 @@ import { type CodexAccountControl, } from "./account/codex-account-control.js"; import { officialEnvironment } from "./app-server-host.js"; -import { OfficialRuntimeScope } from "./codex-runtime/official-runtime-scope.js"; +import { + OfficialRuntimeScope, + type OfficialRuntimeRecoveryOptions, +} from "./codex-runtime/official-runtime-scope.js"; +import { OfficialAdmissionError } from "./codex-runtime/official-work-gate.js"; import { createOwnedLoopbackBackend } from "./codex-runtime/owned-official-backends.js"; export interface PreparedLocalCodex { @@ -29,11 +33,20 @@ async function canonicalCodexHome(home: string): Promise { } } +function sanitizedOfficialCause(error: unknown): string { + const outer = error instanceof Error ? error.name : "unknown"; + const cause = error instanceof Error ? error.cause : undefined; + const inner = cause instanceof Error ? cause.name : undefined; + return inner && inner !== outer ? `${outer}/${inner}` : outer; +} + export async function prepareLocalCodex(input: { stockCodexPath: string; arguments: string[]; environment: NodeJS.ProcessEnv; diagnosticOutput: Writable; + recovery?: OfficialRuntimeRecoveryOptions; + recoveryWaitMs?: number; }): Promise { const home = await canonicalCodexHome( input.environment.CODEX_HOME ?? path.join(homedir(), ".codex"), @@ -43,7 +56,7 @@ export async function prepareLocalCodex(input: { permanentHome: home, diagnosticOutput: input.diagnosticOutput, // Local Desktop connections outlive a failed official backend generation. - recovery: {}, + recovery: input.recovery ?? {}, createBackend: () => createOwnedLoopbackBackend({ stockCodexPath: input.stockCodexPath, @@ -62,9 +75,22 @@ export async function prepareLocalCodex(input: { try { await scope.start(); } catch (error) { - await scope.close(); - identityReader.close(); - throw error; + const waitMs = input.recoveryWaitMs ?? 13_000; + input.diagnosticOutput.write( + `codexhost: official Codex first start failed stage=start cause=${sanitizedOfficialCause(error)}; waiting ${String(waitMs)}ms for recovery\n`, + ); + try { + await scope.awaitReady(waitMs); + input.diagnosticOutput.write( + "codexhost: official Codex recovered after first start failure\n", + ); + } catch { + await scope.close(); + identityReader.close(); + throw error instanceof OfficialAdmissionError + ? error + : new OfficialAdmissionError("unavailable", error); + } } let current: ReturnType = null; const snapshot = () => ({ diff --git a/packages/host-runtime/src/native-voice-gateway.ts b/packages/host-runtime/src/native-voice-gateway.ts new file mode 100644 index 000000000..49f39a666 --- /dev/null +++ b/packages/host-runtime/src/native-voice-gateway.ts @@ -0,0 +1,126 @@ +import { + filterRealtimeCallHeaders, + isRealtimeCallId, + realtimeCallIdFromLocation, + realtimeCallParamsSchema, +} from "@codexhost/shared-contracts"; +import { + hangupRealtimeCall, + readRealtimeGateway, + realtimeGatewayUrl, +} from "./realtime-gateway-config.js"; + +const RESPONSE_HEADERS = ["content-type", "location", "retry-after"]; +const MAX_RESPONSE_BYTES = 16 << 20; +const MAX_PENDING_CALLS = 16; + +/** The `create` body must carry the native SDP offer and session object. */ +function createCallBody(value: string): string { + let payload: unknown; + try { + payload = JSON.parse(value); + } catch { + throw new Error("Invalid voice call request"); + } + if ( + !payload || + typeof payload !== "object" || + !("sdp" in payload) || + typeof payload.sdp !== "string" || + !("session" in payload) || + !payload.session || + typeof payload.session !== "object" + ) + throw new Error("Invalid voice call request"); + return value; +} + +/** Serves Desktop-owned call creation using the configured gateway credential. */ +export class NativeVoiceGateway { + readonly #pending = new Map(); + constructor(private readonly environment: NodeJS.ProcessEnv) {} + + close(): void { + for (const controller of this.#pending.values()) controller.abort(); + this.#pending.clear(); + } + + async request(value: unknown) { + const params = realtimeCallParamsSchema.parse(value); + if (params.operation === "cancel") { + this.#pending.get(params.requestId)?.abort(); + return { status: 204, body: "", headers: {} }; + } + if (params.operation === "hangup") { + const gateway = await readRealtimeGateway(this.environment); + await hangupRealtimeCall(gateway, params.callId); + return { status: 204, body: "", headers: {} }; + } + if (this.#pending.has(params.requestId) || this.#pending.size >= MAX_PENDING_CALLS) + throw new Error("Voice call creation is already pending"); + const controller = new AbortController(); + this.#pending.set(params.requestId, controller); + try { + return await this.#create(params.body, params.headers, controller); + } finally { + this.#pending.delete(params.requestId); + } + } + + async #create( + body: string, + forwarded: Record, + controller: AbortController, + ): Promise<{ status: number; body: string; headers: Record }> { + // Validated before the transport try, so these stay specific for direct callers. + const payload = createCallBody(body); + const gateway = await readRealtimeGateway(this.environment); + const headers = new Headers({ + Authorization: `Bearer ${gateway.token}`, + "Content-Type": "application/json", + }); + for (const [name, value] of Object.entries(filterRealtimeCallHeaders(forwarded))) + headers.set(name, value); + let closeCreatedCall: (() => Promise) | undefined; + try { + controller.signal.throwIfAborted(); + const response = await fetch(realtimeGatewayUrl(gateway, "/realtime/calls"), { + method: "POST", + headers, + body: payload, + redirect: "error", + signal: AbortSignal.any([controller.signal, AbortSignal.timeout(15_000)]), + }); + if (response.ok) { + const callId = realtimeCallIdFromLocation(response.headers.get("location") ?? ""); + if (!isRealtimeCallId(callId)) { + await response.body?.cancel(); + throw new Error("Voice call identity is unavailable; remote cleanup is unconfirmed"); + } + closeCreatedCall = () => hangupRealtimeCall(gateway, callId); + } + const resultHeaders: Record = {}; + for (const name of RESPONSE_HEADERS) { + const value = response.headers.get(name); + if (value !== null) resultHeaders[name] = value; + } + const text = await response.text(); + if (text.length > MAX_RESPONSE_BYTES) throw new Error("Voice gateway response is too large"); + controller.signal.throwIfAborted(); + return { status: response.status, body: text, headers: resultHeaders }; + } catch (error) { + if (closeCreatedCall) { + try { + await closeCreatedCall(); + } catch { + throw new Error("Voice gateway startup failed; remote cleanup is unconfirmed"); + } + } + if (controller.signal.aborted) throw new Error("Voice call creation cancelled"); + // Transport diagnostics must not expose headers, credentials or SDP. + if (error instanceof Error && /^(Invalid voice|Voice call identity)/u.test(error.message)) + throw error; + throw new Error("Configured voice gateway request failed"); + } + } +} diff --git a/packages/host-runtime/src/realtime-gateway-config.ts b/packages/host-runtime/src/realtime-gateway-config.ts new file mode 100644 index 000000000..28048e022 --- /dev/null +++ b/packages/host-runtime/src/realtime-gateway-config.ts @@ -0,0 +1,92 @@ +import { readFile } from "node:fs/promises"; +import { homedir } from "node:os"; +import path from "node:path"; +import { parse } from "smol-toml"; + +function object(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** Gateway credentials belong to the selected provider, never Desktop OAuth. */ +export async function readRealtimeGateway(environment: NodeJS.ProcessEnv) { + let config: Record; + try { + config = parse( + await readFile( + path.join(environment.CODEX_HOME ?? path.join(homedir(), ".codex"), "config.toml"), + "utf8", + ), + ); + } catch { + throw new Error("External voice gateway configuration is unavailable"); + } + const providers = config.model_providers; + const provider = + object(providers) && typeof config.model_provider === "string" + ? providers[config.model_provider] + : undefined; + const token = object(provider) + ? typeof provider.experimental_bearer_token === "string" + ? provider.experimental_bearer_token + : typeof provider.env_key === "string" + ? environment[provider.env_key] + : undefined + : undefined; + const base = config.experimental_realtime_webrtc_call_base_url; + const sidebandBase = config.experimental_realtime_ws_base_url ?? base; + if (typeof base !== "string" || typeof sidebandBase !== "string" || !token) + throw new Error("External voice requires a configured realtime gateway and API credential"); + let url: URL; + let sideband: URL; + try { + url = new URL(base); + sideband = new URL(sidebandBase); + } catch { + throw new Error("External voice gateway URL is invalid"); + } + if ( + url.protocol !== "https:" || + sideband.protocol !== "https:" || + url.origin !== sideband.origin || + url.username || + url.password || + sideband.username || + sideband.password || + url.search || + url.hash || + sideband.search || + sideband.hash + ) + throw new Error("External voice requires matching HTTPS gateway origins"); + return { + url, + sideband, + token, + model: + typeof config.experimental_realtime_ws_model === "string" + ? config.experimental_realtime_ws_model + : undefined, + }; +} + +type RealtimeGateway = Awaited>; + +/** Gateway request URLs share one normalized base; only the path suffix differs. */ +export function realtimeGatewayUrl(gateway: RealtimeGateway, suffix: string): string { + return `${gateway.url.href.replace(/\/$/u, "")}${suffix}`; +} + +/** + * Confirms remote cleanup with the configured credential. A 404 means the call is + * already gone; any other non-OK status leaves cleanup unconfirmed. + */ +export async function hangupRealtimeCall(gateway: RealtimeGateway, callId: string): Promise { + const response = await fetch(realtimeGatewayUrl(gateway, `/realtime/calls/${callId}/hangup`), { + method: "POST", + headers: { Authorization: `Bearer ${gateway.token}` }, + redirect: "error", + signal: AbortSignal.timeout(10_000), + }); + await response.body?.cancel(); + if (!response.ok && response.status !== 404) throw new Error("Realtime call hangup unconfirmed"); +} diff --git a/packages/host-runtime/src/run-host-runtime.ts b/packages/host-runtime/src/run-host-runtime.ts index 39ba74813..9ae37e8e7 100644 --- a/packages/host-runtime/src/run-host-runtime.ts +++ b/packages/host-runtime/src/run-host-runtime.ts @@ -15,6 +15,7 @@ import { createOwnedUnixBackend } from "./codex-runtime/owned-official-backends. import { DelegationControlRegistry } from "./delegation-control-registry.js"; import { installedHarnessPluginOptions } from "./installed-harness-plugins.js"; import { startDelegationControlServer } from "./delegation-control-server.js"; +import { retireCodexhostDesktopRules } from "./codexhost-desktop-rule.js"; import { installDelegationSkills } from "./delegation-skill.js"; import type { DelegationControlRegistration } from "./delegation-types.js"; import { readRemoteDelegationThread } from "./remote-delegation-read.js"; @@ -140,6 +141,24 @@ export function delegationCliEnvironment( }; } +async function installManagedHomeFiles( + install: () => Promise>, + label: string, +): Promise { + try { + const results = await install(); + for (const result of results) { + if (result.status === "conflict") { + process.stderr.write( + `codexhost ${label} conflict: preserving user-managed file at ${result.path}\n`, + ); + } + } + } catch (error) { + process.stderr.write(`codexhost ${label} installation failed: ${String(error)}\n`); + } +} + async function prepareDelegationRuntime(input: { environment: NodeJS.ProcessEnv; hostRuntimePath?: string; @@ -164,19 +183,8 @@ async function prepareDelegationRuntime(input: { [DELEGATION_RUNTIME_ENDPOINT_ENV]: server.endpoint, [DELEGATION_RUNTIME_TOKEN_ENV]: token, }; - await installDelegationSkills() - .then((results) => { - for (const result of results) { - if (result.status === "conflict") { - process.stderr.write( - `codexhost delegation Skill conflict: preserving user-managed file at ${result.path}\n`, - ); - } - } - }) - .catch((error) => { - process.stderr.write(`codexhost delegation Skill installation failed: ${String(error)}\n`); - }); + await installManagedHomeFiles(installDelegationSkills, "delegation Skill"); + await installManagedHomeFiles(retireCodexhostDesktopRules, "Desktop rule retirement"); try { return await input.createHost(environment, (value) => registry.register(value), registry); } finally { diff --git a/packages/host-runtime/test/antigravity-subagents.real.test.ts b/packages/host-runtime/test/antigravity-subagents.real.test.ts index 10205f1a8..fe67546e0 100644 --- a/packages/host-runtime/test/antigravity-subagents.real.test.ts +++ b/packages/host-runtime/test/antigravity-subagents.real.test.ts @@ -131,9 +131,9 @@ describe.skipIf(!RUN_REAL)("Antigravity real Subagent Desktop protocol", () => { (message) => message.method === "item/completed" && params(message).threadId === parentId && - (params(message).item as JsonObject | undefined)?.type === "collabAgentToolCall", + (params(message).item as JsonObject | undefined)?.type === "subAgentActivity", ); - expect((params(card).item as JsonObject).receiverThreadIds).toEqual([childId]); + expect((params(card).item as JsonObject).agentThreadId).toBe(childId); stage = "open child"; // Materialize the child while it runs so progress notifications are exercised. await host.request("thread/turns/list", { threadId: childId, limit: 20, itemsView: "full" }); diff --git a/packages/host-runtime/test/app-server-host-fixture.ts b/packages/host-runtime/test/app-server-host-fixture.ts index 2a476d706..779ea4dd0 100644 --- a/packages/host-runtime/test/app-server-host-fixture.ts +++ b/packages/host-runtime/test/app-server-host-fixture.ts @@ -123,7 +123,7 @@ export function method(message: JsonObject, value: string): boolean { return message.method === value; } -export function requestId(message: JsonObject, id: number): boolean { +export function requestId(message: JsonObject, id: string | number): boolean { return message.id === id; } @@ -289,6 +289,7 @@ export function createFixture( onCreateRequestRoute?: AppServerHostOptions["onCreateRequestRoute"]; modelPrices?: AppServerHostOptions["modelPrices"]; usageStatistics?: AppServerHostOptions["usageStatistics"]; + createExternalVoiceCall?: AppServerHostOptions["createExternalVoiceCall"]; } = {}, ) { const adapter = @@ -358,6 +359,9 @@ export function createFixture( ...(options.consoleOpener ? { consoleOpener: options.consoleOpener } : {}), ...(options.modelPrices ? { modelPrices: options.modelPrices } : {}), ...(options.usageStatistics ? { usageStatistics: options.usageStatistics } : {}), + ...(options.createExternalVoiceCall + ? { createExternalVoiceCall: options.createExternalVoiceCall } + : {}), ...(options.accountControl ? { accountControl: options.accountControl } : {}), ...(options.officialRuntimeScope ? { officialRuntimeScope: options.officialRuntimeScope } : {}), ...(options.onDelegationApi ? { onDelegationApi: options.onDelegationApi } : {}), diff --git a/packages/host-runtime/test/app-server-host.external-sandbox.test.ts b/packages/host-runtime/test/app-server-host.external-sandbox.test.ts new file mode 100644 index 000000000..763175c9d --- /dev/null +++ b/packages/host-runtime/test/app-server-host.external-sandbox.test.ts @@ -0,0 +1,100 @@ +import { describe, expect, it, vi } from "vitest"; +import { + completePiTurn, + createFixture, + requestId, + startExternalThread, + stopFixture, + writeRequest, +} from "./app-server-host-fixture.js"; + +const externalSandbox = { type: "externalSandbox", networkAccess: "enabled" }; + +describe("external Thread sandbox ownership", () => { + it.each(["read-only", "workspace-write", "danger-full-access"])( + "does not claim Codex enforces the requested %s sandbox", + async (sandbox) => { + const fixture = createFixture(); + const open = vi.spyOn(fixture.adapter, "open"); + try { + await startExternalThread(fixture, "codexhost/pi-native", 1, { sandbox }); + expect(await fixture.collector.waitFor((message) => requestId(message, 1))).toMatchObject({ + result: { sandbox: externalSandbox }, + }); + expect(open).toHaveBeenCalledOnce(); + expect(open.mock.calls[0]?.[0]).not.toHaveProperty("sandbox"); + expect(fixture.official.stdin.readableLength).toBe(0); + } finally { + await stopFixture(fixture); + } + }, + ); + + it("keeps sandbox enforcement external after resume and fork", async () => { + const fixture = createFixture(); + try { + const threadId = await startExternalThread(fixture, "codexhost/pi-native"); + await completePiTurn(fixture, threadId, 2); + for (const [id, method] of [ + [3, "thread/resume"], + [4, "thread/fork"], + ] as const) { + writeRequest(fixture.desktopInput, { id, method, params: { threadId } }); + expect(await fixture.collector.waitFor((message) => requestId(message, id))).toMatchObject({ + result: { sandbox: externalSandbox }, + }); + } + expect(fixture.official.stdin.readableLength).toBe(0); + } finally { + await stopFixture(fixture); + } + }); + + it("accepts the reasoning-summary patch used by native voice startup", async () => { + const fixture = createFixture(); + try { + const threadId = await startExternalThread(fixture, "codexhost/pi-native"); + const session = fixture.adapter.sessions[0]; + if (!session) throw new Error("Expected external session"); + const execute = vi.spyOn(session, "execute"); + const before = session.state.effectivePermissionModeId; + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/settings/update", + params: { threadId, summary: "detailed" }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 2))).toMatchObject({ + result: {}, + }); + expect(execute).not.toHaveBeenCalled(); + expect(session.state.effectivePermissionModeId).toBe(before); + expect(fixture.official.stdin.readableLength).toBe(0); + } finally { + await stopFixture(fixture); + } + }); + + it("acknowledges native permission patches without applying them", async () => { + const fixture = createFixture(); + try { + const threadId = await startExternalThread(fixture, "codexhost/pi-native"); + const session = fixture.adapter.sessions[0]; + if (!session) throw new Error("Expected external session"); + const execute = vi.spyOn(session, "execute"); + const before = session.state.effectivePermissionModeId; + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/settings/update", + params: { threadId, sandboxPolicy: { type: "dangerFullAccess" } }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 2))).toMatchObject({ + result: {}, + }); + expect(execute).not.toHaveBeenCalled(); + expect(session.state.effectivePermissionModeId).toBe(before); + expect(fixture.official.stdin.readableLength).toBe(0); + } finally { + await stopFixture(fixture); + } + }); +}); diff --git a/packages/host-runtime/test/app-server-host.external-voice.test.ts b/packages/host-runtime/test/app-server-host.external-voice.test.ts new file mode 100644 index 000000000..42ece2924 --- /dev/null +++ b/packages/host-runtime/test/app-server-host.external-voice.test.ts @@ -0,0 +1,441 @@ +import { FakeHarnessAdapter } from "@codexhost/harness-adapter/testing"; +import { + harnessIdSchema, + hostItemIdSchema, + hostTurnIdSchema, + type JsonObject, +} from "@codexhost/shared-contracts"; +import { describe, expect, it, vi, type Mock } from "vitest"; +import type { ExternalVoiceCallFactory } from "../src/external-voice-gateway.js"; +import { + createFixture, + requestId, + startExternalThread, + stopFixture, + writeRequest, +} from "./app-server-host-fixture.js"; + +function transport() { + let input: Parameters[0]; + const sent: JsonObject[] = []; + const close = vi.fn(async () => {}); + const createCall: Mock = vi.fn(async (options) => { + input = options; + return { + sdp: "v=0\r\nanswer", + connect: async () => {}, + send: (event: JsonObject) => { + sent.push(event); + }, + close, + }; + }); + return { + createCall, + close, + sent, + event: (event: JsonObject) => input.onEvent(event), + fail: () => input.onError(), + }; +} + +describe("external native voice routing", () => { + it.each(["grok", "claude-code"])( + "forwards %s attachment-task progress and completion to an already open call", + async (name) => { + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse(name)); + const call = transport(); + const fixture = createFixture({ + externalAdapters: new Map([[name, adapter]]), + createExternalVoiceCall: call.createCall, + }); + try { + const threadId = await startExternalThread(fixture, `codexhost/${name}-native`); + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/realtime/start", + params: { + threadId, + outputModality: "audio", + transport: { type: "existingCall", callId: "rtc_fixture" }, + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + writeRequest(fixture.desktopInput, { + id: 3, + method: "turn/start", + params: { threadId, input: [{ type: "text", text: "Read the attached audit task." }] }, + }); + const response = await fixture.collector.waitFor((message) => requestId(message, 3)); + const turnId = (response.result as JsonObject).turn as JsonObject; + const session = adapter.sessions[0]; + if (!session) throw new Error("Expected selected Harness session"); + session.startReasoning("PRIVATE REASONING MUST NOT BE SPOKEN"); + const command = session.startCommandExecution("PRIVATE TOOL MUST NOT BE SPOKEN"); + session.completeItem(command, { status: "succeeded" }); + const item = { + type: "agentMessage" as const, + itemId: hostItemIdSchema.parse("44444444-4444-4444-8444-444444444444"), + text: "Both scouts have returned. Checking their findings.", + }; + const activeTurnId = hostTurnIdSchema.parse(turnId.id); + session.emitEvent({ type: "item.started", turnId: activeTurnId, item }); + session.emitEvent({ + type: "item.completed", + turnId: activeTurnId, + snapshot: { item, outcome: { status: "succeeded" } }, + }); + const validation = session.startCommandExecution("PRIVATE VALIDATION MUST NOT BE SPOKEN"); + session.completeItem(validation, { status: "succeeded" }); + await vi.waitFor(() => expect(call.sent).toHaveLength(1)); + session.appendText("Audit complete. The focused tests passed."); + session.succeedTurn(); + await fixture.collector.waitFor( + (message) => + message.method === "turn/completed" && + ((message.params as JsonObject).turn as JsonObject)?.id === turnId.id, + ); + await vi.waitFor(() => expect(call.sent).toHaveLength(2)); + expect(call.sent).toEqual([ + { + type: "session.context.append", + content: [{ type: "input_text", text: item.text }], + }, + { + type: "session.context.append", + content: [{ type: "input_text", text: "Audit complete. The focused tests passed." }], + }, + ]); + } finally { + await stopFixture(fixture); + } + }, + ); + + it.each([ + { hangup: false, label: "forwards completion of work already running when voice opens" }, + { hangup: true, label: "suppresses completion after hangup without cancelling existing work" }, + ])("$label", async ({ hangup }) => { + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse("claude-code")); + const call = transport(); + const fixture = createFixture({ + externalAdapters: new Map([["claude-code", adapter]]), + createExternalVoiceCall: call.createCall, + }); + try { + const threadId = await startExternalThread(fixture, "codexhost/claude-code-native"); + writeRequest(fixture.desktopInput, { + id: 2, + method: "turn/start", + params: { threadId, input: [{ type: "text", text: "Audit the voice lifecycle." }] }, + }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + const params = { + threadId, + outputModality: "audio", + transport: { type: "existingCall", callId: "rtc_fixture" }, + }; + writeRequest(fixture.desktopInput, { id: 3, method: "thread/realtime/start", params }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + if (hangup) { + writeRequest(fixture.desktopInput, { + id: 4, + method: "thread/realtime/stop", + params: { threadId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 4)); + } + const session = adapter.sessions[0]; + if (!session) throw new Error("Expected selected Harness session"); + session.appendText("Audit complete."); + session.succeedTurn(); + await fixture.collector.waitFor((message) => message.method === "turn/completed"); + if (!hangup) { + await vi.waitFor(() => + expect(call.sent).toEqual([ + { + type: "session.context.append", + content: [{ type: "input_text", text: "Audit complete." }], + }, + ]), + ); + return; + } + expect(call.sent).toEqual([]); + writeRequest(fixture.desktopInput, { id: 5, method: "thread/realtime/start", params }); + await fixture.collector.waitFor((message) => requestId(message, 5)); + expect(call.createCall).toHaveBeenCalledTimes(2); + expect(call.sent).toEqual([]); + } finally { + await stopFixture(fixture); + } + }); + + it.each(["grok", "claude-code"])( + "uses the selected %s session and never starts a Codex task", + async (name) => { + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse(name)); + const call = transport(); + const fixture = createFixture({ + externalAdapters: new Map([[name, adapter]]), + createExternalVoiceCall: call.createCall, + }); + try { + const threadId = await startExternalThread(fixture, `codexhost/${name}-native`); + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/realtime/start", + params: { + threadId, + outputModality: "audio", + transport: { type: "webrtc", sdp: "v=0\r\noffer" }, + }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 2))).toMatchObject({ + result: {}, + }); + expect(fixture.collector.messages).toContainEqual( + expect.objectContaining({ + method: "thread/realtime/sdp", + params: { threadId, sdp: "v=0\r\nanswer" }, + }), + ); + const delegation = { + type: "delegation.created", + item: { + type: "delegation", + target: "client", + id: "task-one", + handoff_id: "handoff-one", + user_bidi_turn_id: "user-one", + content: [{ type: "input_text", text: "Three plus two" }], + }, + }; + call.event(delegation); + call.event(delegation); + const session = adapter.sessions[0]; + if (!session) throw new Error("Expected selected Harness session"); + await fixture.collector.waitFor( + (message) => + message.method === "turn/started" && + (message.params as JsonObject)?.threadId === threadId, + ); + session.startReasoning("PRIVATE REASONING MUST NOT BE SPOKEN"); + session.appendText("Five"); + session.succeedTurn(); + await vi.waitFor(() => expect(call.sent).toHaveLength(1)); + expect(call.sent[0]).toEqual({ + type: "delegation.context.append", + delegation_item_id: "task-one", + channel: "speakable", + content: [{ type: "input_text", text: "Five" }], + }); + call.event({ + type: "turn.done", + turn: { id: "user-one", role: "user", transcript: "Three plus two" }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect( + fixture.collector.messages.filter( + (message) => + message.method === "turn/started" && + (message.params as JsonObject)?.threadId === threadId, + ), + ).toHaveLength(1); + expect(adapter.sessions).toHaveLength(1); + expect(fixture.official.stdin.readableLength).toBe(0); + writeRequest(fixture.desktopInput, { + id: 3, + method: "thread/realtime/stop", + params: { threadId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + expect(call.close).toHaveBeenCalledTimes(1); + } finally { + await stopFixture(fixture); + } + }, + ); + + it("still projects command and turn completion after hangup, and late interrupt is a no-op", async () => { + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse("grok")); + const call = transport(); + const fixture = createFixture({ + externalAdapters: new Map([["grok", adapter]]), + createExternalVoiceCall: call.createCall, + }); + try { + const threadId = await startExternalThread(fixture, "codexhost/grok-native"); + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/realtime/start", + params: { + threadId, + outputModality: "audio", + transport: { type: "webrtc", sdp: "v=0\r\noffer" }, + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + call.event({ + type: "turn.done", + turn: { role: "user", transcript: "list files" }, + }); + const session = adapter.sessions[0]; + if (!session) throw new Error("Expected selected Harness session"); + await fixture.collector.waitFor( + (message) => + message.method === "turn/started" && + (message.params as JsonObject)?.threadId === threadId, + ); + const turnStarted = fixture.collector.messages.find( + (message) => + message.method === "turn/started" && + (message.params as JsonObject)?.threadId === threadId, + ); + const turnId = ((turnStarted?.params as JsonObject)?.turn as JsonObject | undefined)?.id; + if (typeof turnId !== "string") throw new Error("Expected voice Turn id"); + const commandId = session.startCommandExecution("ls -la"); + await fixture.collector.waitFor( + (message) => + message.method === "item/started" && + ((message.params as JsonObject).item as JsonObject | undefined)?.type === + "commandExecution", + ); + session.completeItem(commandId, { status: "succeeded" }); + await fixture.collector.waitFor( + (message) => + message.method === "item/completed" && + ((message.params as JsonObject).item as JsonObject | undefined)?.type === + "commandExecution", + ); + writeRequest(fixture.desktopInput, { + id: 3, + method: "thread/realtime/stop", + params: { threadId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + session.appendText("Only empty outputs/ and work/."); + session.succeedTurn(); + const completed = await fixture.collector.waitFor( + (message) => + message.method === "turn/completed" && + (message.params as JsonObject)?.threadId === threadId, + ); + expect(completed).toMatchObject({ params: { turn: { status: "completed" } } }); + expect( + fixture.collector.messages.some( + (message) => + message.method === "item/completed" && + ((message.params as JsonObject).item as JsonObject | undefined)?.type === + "commandExecution", + ), + ).toBe(true); + writeRequest(fixture.desktopInput, { + id: 4, + method: "turn/interrupt", + params: { threadId, turnId }, + }); + await expect(fixture.collector.waitFor((message) => requestId(message, 4))).resolves.toEqual({ + id: 4, + result: {}, + }); + } finally { + await stopFixture(fixture); + } + }); + + it("seeds a reopened call with prior Thread messages and routes a user transcript without delegation", async () => { + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse("grok")); + const call = transport(); + const fixture = createFixture({ + externalAdapters: new Map([["grok", adapter]]), + createExternalVoiceCall: call.createCall, + }); + try { + const threadId = await startExternalThread(fixture, "codexhost/grok-native"); + writeRequest(fixture.desktopInput, { + id: 2, + method: "turn/start", + params: { threadId, input: [{ type: "text", text: "list files" }] }, + }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + const first = adapter.sessions[0]; + if (!first) throw new Error("Expected selected Harness session"); + first.appendText("Only empty outputs/ and work/."); + first.succeedTurn(); + await fixture.collector.waitFor( + (message) => + message.method === "turn/completed" && + (message.params as JsonObject)?.threadId === threadId, + ); + writeRequest(fixture.desktopInput, { + id: 3, + method: "thread/realtime/start", + params: { + threadId, + outputModality: "audio", + transport: { type: "webrtc", sdp: "v=0\r\noffer" }, + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + expect(call.createCall).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.arrayContaining([ + expect.objectContaining({ + role: "assistant", + content: [expect.objectContaining({ text: "Only empty outputs/ and work/." })], + }), + ]), + }), + ); + const startedBefore = fixture.collector.messages.filter( + (message) => message.method === "turn/started", + ).length; + call.event({ + type: "turn.done", + turn: { role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await fixture.collector.waitFor( + (message) => + message.method === "turn/started" && + (message.params as JsonObject)?.threadId === threadId && + fixture.collector.messages.filter((entry) => entry.method === "turn/started").length > + startedBefore, + ); + const second = adapter.sessions[0]; + if (!second) throw new Error("Expected follow-up Harness session"); + second.appendText("The workspace still has only empty outputs/ and work/."); + second.succeedTurn(); + await vi.waitFor(() => + expect(call.sent.some((event) => event.type === "session.context.append")).toBe(true), + ); + } finally { + await stopFixture(fixture); + } + }); + + it("closes a failed call and permits an explicit retry", async () => { + const call = transport(); + const fixture = createFixture({ createExternalVoiceCall: call.createCall }); + try { + const threadId = await startExternalThread(fixture, "codexhost/pi-native"); + const params = { + threadId, + outputModality: "audio", + transport: { type: "webrtc", sdp: "v=0\r\noffer" }, + }; + writeRequest(fixture.desktopInput, { id: 2, method: "thread/realtime/start", params }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + call.fail(); + await fixture.collector.waitFor((message) => message.method === "thread/realtime/closed"); + expect(call.close).toHaveBeenCalledTimes(1); + writeRequest(fixture.desktopInput, { id: 3, method: "thread/realtime/start", params }); + expect(await fixture.collector.waitFor((message) => requestId(message, 3))).toMatchObject({ + result: {}, + }); + expect(call.createCall).toHaveBeenCalledTimes(2); + } finally { + await stopFixture(fixture); + } + }); +}); diff --git a/packages/host-runtime/test/app-server-host.follow-up-queue.test.ts b/packages/host-runtime/test/app-server-host.follow-up-queue.test.ts new file mode 100644 index 000000000..eb7195570 --- /dev/null +++ b/packages/host-runtime/test/app-server-host.follow-up-queue.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, it } from "vitest"; + +import { + createFixture, + requestId, + startPiThread, + startPiTurn, + stopFixture, + turnEvent, + writeRequest, +} from "./app-server-host-fixture.js"; + +describe("external follow-up queue", () => { + it("starts a queued follow-up from thread/queue/start and from an empty turn/start", async () => { + const fixture = createFixture(); + try { + const threadId = await startPiThread(fixture); + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/queue/add", + params: { + threadId, + clientUserMessageId: "follow-up", + input: [{ type: "text", text: "queued from overlay" }], + }, + }); + const added = await fixture.collector.waitFor((message) => requestId(message, 2)); + expect(added.error).toBeUndefined(); + expect(added.result).toMatchObject({ + queuedSubmission: { + clientUserMessageId: "follow-up", + input: [{ type: "text", text: "queued from overlay", text_elements: [] }], + }, + }); + writeRequest(fixture.desktopInput, { + id: 21, + method: "thread/queue/list", + params: { threadId, limit: 100 }, + }); + const listed = await fixture.collector.waitFor((message) => requestId(message, 21)); + expect(listed.error).toBeUndefined(); + expect(listed.result).toMatchObject({ + data: [ + { + clientUserMessageId: "follow-up", + input: [{ type: "text", text: "queued from overlay", text_elements: [] }], + }, + ], + nextCursor: null, + }); + + writeRequest(fixture.desktopInput, { + id: 3, + method: "thread/queue/start", + params: { + threadId, + queuedSubmissionId: (added.result as { queuedSubmission: { id: string } }) + .queuedSubmission.id, + }, + }); + const started = await fixture.collector.waitFor((message) => requestId(message, 3)); + expect(started.error).toBeUndefined(); + const turnId = (started.result as { turn: { id: string } }).turn.id; + expect(turnId).toBeTruthy(); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/started", turnId)); + fixture.adapter.sessions[0]?.succeedTurn(); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/completed", turnId)); + + writeRequest(fixture.desktopInput, { + id: 4, + method: "thread/queue/add", + params: { + threadId, + clientUserMessageId: "empty-start", + input: [{ type: "text", text: "resume after interrupt" }], + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 4)); + writeRequest(fixture.desktopInput, { + id: 5, + method: "turn/start", + params: { threadId, clientUserMessageId: "empty-start", input: [] }, + }); + const resumed = await fixture.collector.waitFor((message) => requestId(message, 5)); + expect(resumed.error).toBeUndefined(); + expect(resumed.result).toMatchObject({ turn: { status: "inProgress" } }); + } finally { + await stopFixture(fixture); + } + }); + + it("resumes an interrupted Thread and starts the queued follow-up from Desktop's empty turn/start", async () => { + const fixture = createFixture(); + try { + const threadId = await startPiThread(fixture); + const turnId = await startPiTurn(fixture, threadId, 2); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/started", turnId)); + const session = fixture.adapter.sessions[0]; + if (!session) throw new Error("Fake Pi Session was not opened"); + writeRequest(fixture.desktopInput, { + id: 3, + method: "turn/interrupt", + params: { threadId, turnId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + session.completeCancellation(); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/completed", turnId)); + + writeRequest(fixture.desktopInput, { + id: 4, + method: "thread/resume", + params: { threadId, excludeTurns: true }, + }); + const resume = await fixture.collector.waitFor((message) => requestId(message, 4)); + expect(resume.error).toBeUndefined(); + expect(resume.result).toMatchObject({ thread: { id: threadId } }); + + writeRequest(fixture.desktopInput, { + id: 5, + method: "thread/queue/add", + params: { + threadId, + clientUserMessageId: "after-resume", + input: [{ type: "text", text: "queued after interrupt" }], + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 5)); + writeRequest(fixture.desktopInput, { + id: 6, + method: "turn/start", + params: { threadId, clientUserMessageId: "after-resume", input: [] }, + }); + const started = await fixture.collector.waitFor((message) => requestId(message, 6)); + expect(started.error).toBeUndefined(); + expect(started.result).toMatchObject({ turn: { status: "inProgress" } }); + } finally { + await stopFixture(fixture); + } + }); + + it("rejects an empty turn/start after interrupt when the Host queue is empty", async () => { + const fixture = createFixture(); + try { + const threadId = await startPiThread(fixture); + const turnId = await startPiTurn(fixture, threadId, 2); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/started", turnId)); + const session = fixture.adapter.sessions[0]; + if (!session) throw new Error("Fake Pi Session was not opened"); + writeRequest(fixture.desktopInput, { + id: 3, + method: "turn/interrupt", + params: { threadId, turnId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + session.completeCancellation(); + await fixture.collector.waitFor((message) => turnEvent(message, "turn/completed", turnId)); + + writeRequest(fixture.desktopInput, { + id: 4, + method: "thread/resume", + params: { threadId, excludeTurns: true }, + }); + await fixture.collector.waitFor((message) => requestId(message, 4)); + writeRequest(fixture.desktopInput, { + id: 5, + method: "turn/start", + params: { threadId, clientUserMessageId: "empty-after-resume", input: [] }, + }); + await expect( + fixture.collector.waitFor((message) => requestId(message, 5)), + ).resolves.toMatchObject({ + error: { code: -32602, message: "turn/start must contain text input" }, + }); + } finally { + await stopFixture(fixture); + } + }); +}); diff --git a/packages/host-runtime/test/app-server-host.official-passthrough.test.ts b/packages/host-runtime/test/app-server-host.official-passthrough.test.ts index beaf01696..e860ed1c2 100644 --- a/packages/host-runtime/test/app-server-host.official-passthrough.test.ts +++ b/packages/host-runtime/test/app-server-host.official-passthrough.test.ts @@ -63,7 +63,24 @@ describe("official Codex passthrough", () => { params: { threadId: "official-thread", gitInfo: { branch: { name: "future" } } }, }, { method: "thread/section/move", params: { threadId: "official-thread", sectionId: 7 } }, + { + method: "thread/settings/update", + params: { threadId: "official-thread", summary: "detailed" }, + }, { method: "turn/start" }, + { + method: "thread/realtime/start", + params: { + threadId: "official-voice", + outputModality: "audio", + transport: { type: "webrtc", sdp: "native-offer" }, + }, + }, + { method: "thread/realtime/stop", params: { threadId: "official-voice" } }, + { + method: "thread/realtime/appendText", + params: { threadId: "official-voice", role: "user", text: "native text" }, + }, { method: "turn/interrupt", params: "future-params-shape" }, { method: "thread/read" }, { method: "future/method", params: { anything: [1, "two"] } }, diff --git a/packages/host-runtime/test/app-server-host.projection-1.test.ts b/packages/host-runtime/test/app-server-host.projection-1.test.ts index a8fed42c8..2206d268a 100644 --- a/packages/host-runtime/test/app-server-host.projection-1.test.ts +++ b/packages/host-runtime/test/app-server-host.projection-1.test.ts @@ -333,9 +333,9 @@ describe("AppServerHost HarnessAdapter projection", () => { { items: expect.arrayContaining([ expect.objectContaining({ - type: "collabAgentToolCall", - senderThreadId: parentId, - receiverThreadIds: [childId], + type: "subAgentActivity", + agentThreadId: childId, + kind: "started", }), ]), }, @@ -427,6 +427,8 @@ describe("AppServerHost HarnessAdapter projection", () => { subagentId: "agent-call", nativeSubagentId: "native-agent-1", description: "Analyze files", + model: "child-model", + reasoningEffort: "high", background: false, status: "running", }); @@ -434,8 +436,22 @@ describe("AppServerHost HarnessAdapter projection", () => { expect(messageParams(childStarted).thread).toMatchObject({ status: { type: "active" }, canAcceptDirectInput: false, + model: "child-model", + reasoningEffort: "high", }); const childThreadId = (messageParams(childStarted).thread as JsonObject).id as string; + writeRequest(fixture.desktopInput, { + id: 901, + method: "thread/resume", + params: { threadId: childThreadId }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 901))).toMatchObject({ + result: { + model: "child-model", + reasoningEffort: "high", + thread: { model: "child-model", reasoningEffort: "high", canAcceptDirectInput: false }, + }, + }); writeRequest(fixture.desktopInput, { id: 98, method: "thread/turns/list", @@ -446,6 +462,32 @@ describe("AppServerHost HarnessAdapter projection", () => { result: { data: [{ items: [expect.objectContaining({ type: "userMessage" })] }] }, }); + const beforeModelChange = fixture.collector.messages.length; + session.replaceSubagents(itemId, [ + { + subagentId: "agent-call", + nativeSubagentId: "native-agent-1", + description: "Analyze files", + model: "reported-child-model", + background: false, + status: "running", + }, + ]); + await fixture.collector.waitFor( + (message) => + fixture.collector.messages.indexOf(message) >= beforeModelChange && + method(message, "item/started") && + (messageParams(message).item as JsonObject | undefined)?.agentThreadId === childThreadId, + ); + writeRequest(fixture.desktopInput, { + id: 902, + method: "thread/read", + params: { threadId: childThreadId, includeTurns: false }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 902))).toMatchObject({ + result: { thread: { model: "reported-child-model", reasoningEffort: "high" } }, + }); + subagentPhase = "temporarily-empty"; session.emitSubagentTranscriptChanged("native-agent-1"); writeRequest(fixture.desktopInput, { @@ -548,11 +590,10 @@ describe("AppServerHost HarnessAdapter projection", () => { const completed = await fixture.collector.waitFor( (message) => method(message, "item/completed") && - (messageParams(message).item as JsonObject | undefined)?.type === "collabAgentToolCall", + (messageParams(message).item as JsonObject | undefined)?.type === "subAgentActivity", ); - const completedChildThreadId = ( - (messageParams(completed).item as JsonObject).receiverThreadIds as string[] - )[0]; + const completedChildThreadId = (messageParams(completed).item as JsonObject) + .agentThreadId as string; expect(completedChildThreadId).toBe(childThreadId); expect(childThreadId).toBeTruthy(); expect(childThreadId).not.toBe("agent-call"); @@ -745,6 +786,11 @@ describe("AppServerHost HarnessAdapter projection", () => { params: { threadId: childThreadId, excludeTurns: true }, }); const opened = await fixture.collector.waitFor((message) => requestId(message, 96)); + expect(opened.result).toMatchObject({ + model: "", + reasoningEffort: null, + thread: { model: null, reasoningEffort: null, canAcceptDirectInput: false }, + }); expect((opened.result as JsonObject).thread).toEqual( expect.objectContaining({ id: childThreadId, status: { type: "active", activeFlags: [] } }), ); @@ -1363,7 +1409,7 @@ describe("AppServerHost HarnessAdapter projection", () => { updateAvailable: true, installationAvailable: true, releaseNotes: "Safer updates", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status: null, error: null, })), diff --git a/packages/host-runtime/test/app-server-host.projection-4.test.ts b/packages/host-runtime/test/app-server-host.projection-4.test.ts index 8f8edab41..c66499ec6 100644 --- a/packages/host-runtime/test/app-server-host.projection-4.test.ts +++ b/packages/host-runtime/test/app-server-host.projection-4.test.ts @@ -1179,7 +1179,68 @@ describe("AppServerHost HarnessAdapter projection", () => { await stopFixture(fixture); }); - it("rejects an interrupt that does not reference the active Pi Turn", async () => { + it("treats interrupt of an already-completed Turn as a no-op success", async () => { + const fixture = createFixture(); + const officialWrite = vi.fn(); + fixture.official.stdin.on("data", officialWrite); + const threadId = await startPiThread(fixture); + const turnId = await startPiTurn(fixture, threadId); + const session = fixture.adapter.sessions[0]; + if (!session) throw new Error("Fake Pi Session was not opened"); + session.appendText("done"); + session.succeedTurn(); + await fixture.collector.waitFor((message) => method(message, "turn/completed")); + + writeRequest(fixture.desktopInput, { + id: 3, + method: "turn/interrupt", + params: { threadId, turnId }, + }); + await expect(fixture.collector.waitFor((message) => requestId(message, 3))).resolves.toEqual({ + id: 3, + result: {}, + }); + expect(officialWrite).not.toHaveBeenCalled(); + await stopFixture(fixture); + }); + + it("does not cancel a newer Turn when interrupt names an earlier Turn", async () => { + const fixture = createFixture(); + const threadId = await startPiThread(fixture); + const firstTurnId = await startPiTurn(fixture, threadId); + const first = fixture.adapter.sessions[0]; + if (!first) throw new Error("Fake Pi Session was not opened"); + first.appendText("first"); + first.succeedTurn(); + await fixture.collector.waitFor((message) => method(message, "turn/completed")); + + const secondTurnId = await startPiTurn(fixture, threadId, 3); + const second = fixture.adapter.sessions[0]; + if (!second) throw new Error("Fake Pi Session was not opened"); + second.startCommandExecution("sleep 10"); + const execute = vi.spyOn(second, "execute"); + + writeRequest(fixture.desktopInput, { + id: 4, + method: "turn/interrupt", + params: { threadId, turnId: firstTurnId }, + }); + await expect(fixture.collector.waitFor((message) => requestId(message, 4))).resolves.toEqual({ + id: 4, + result: {}, + }); + expect(execute).not.toHaveBeenCalled(); + expect( + fixture.collector.messages.some( + (message) => + method(message, "turn/completed") && + ((message.params as JsonObject).turn as JsonObject | undefined)?.id === secondTurnId, + ), + ).toBe(false); + await stopFixture(fixture); + }); + + it("accepts interrupt without an active Turn as already stopped", async () => { const fixture = createFixture(); const officialWrite = vi.fn(); fixture.official.stdin.on("data", officialWrite); @@ -1190,10 +1251,9 @@ describe("AppServerHost HarnessAdapter projection", () => { method: "turn/interrupt", params: { threadId, turnId: "missing-turn" }, }); - await expect( - fixture.collector.waitFor((message) => requestId(message, 2)), - ).resolves.toMatchObject({ - error: { code: -32074, message: "External turn/interrupt must reference the active Turn" }, + await expect(fixture.collector.waitFor((message) => requestId(message, 2))).resolves.toEqual({ + id: 2, + result: {}, }); expect(officialWrite).not.toHaveBeenCalled(); await stopFixture(fixture); diff --git a/packages/host-runtime/test/app-server-host.turn-start-input.test.ts b/packages/host-runtime/test/app-server-host.turn-start-input.test.ts new file mode 100644 index 000000000..d517c923d --- /dev/null +++ b/packages/host-runtime/test/app-server-host.turn-start-input.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from "vitest"; + +import { + createFixture, + requestId, + startPiThread, + stopFixture, + writeRequest, +} from "./app-server-host-fixture.js"; + +describe("external turn/start text input", () => { + it("accepts Desktop input_text items as the Turn prompt", async () => { + const fixture = createFixture(); + try { + const threadId = await startPiThread(fixture); + writeRequest(fixture.desktopInput, { + id: 2, + method: "turn/start", + params: { threadId, input: [{ type: "input_text", text: "queued follow-up" }] }, + }); + const response = await fixture.collector.waitFor((message) => requestId(message, 2)); + expect(response.error).toBeUndefined(); + expect(response.result).toMatchObject({ turn: { status: "inProgress" } }); + } finally { + await stopFixture(fixture); + } + }); + + it("rejects a turn/start whose input array has no text", async () => { + const fixture = createFixture(); + try { + const threadId = await startPiThread(fixture); + writeRequest(fixture.desktopInput, { + id: 2, + method: "turn/start", + params: { threadId, input: [] }, + }); + await expect( + fixture.collector.waitFor((message) => requestId(message, 2)), + ).resolves.toMatchObject({ + error: { code: -32602, message: "turn/start must contain text input" }, + }); + } finally { + await stopFixture(fixture); + } + }); +}); diff --git a/packages/host-runtime/test/app-server-host.voice-settings.test.ts b/packages/host-runtime/test/app-server-host.voice-settings.test.ts new file mode 100644 index 000000000..755226be6 --- /dev/null +++ b/packages/host-runtime/test/app-server-host.voice-settings.test.ts @@ -0,0 +1,56 @@ +import { expect, it, vi } from "vitest"; +import { VOICE_SETTINGS_METHOD, VOICE_GATEWAY_STATUS_METHOD } from "@codexhost/shared-contracts"; +import { createFixture, stopFixture, writeRequest, requestId } from "./app-server-host-fixture.js"; + +it("validates voice settings and proxies only through the local console service", async () => { + const voiceSettings = vi.fn(async () => ({ settings: "fixture" })); + const fixture = createFixture({ + environment: { CODEX_HOME: "/nonexistent-codexhost-voice-test" }, + consoleOpener: { open: vi.fn(), voiceSettings }, + }); + try { + writeRequest(fixture.desktopInput, { + id: 801, + method: VOICE_SETTINGS_METHOD, + params: { operation: "get" }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 801))).toMatchObject({ + result: { settings: "fixture" }, + }); + expect(voiceSettings).toHaveBeenCalledWith({ operation: "get" }); + writeRequest(fixture.desktopInput, { + id: 802, + method: VOICE_SETTINGS_METHOD, + params: { operation: "set", settings: {} }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 802))).toMatchObject({ + error: { code: -32602 }, + }); + expect(voiceSettings).toHaveBeenCalledOnce(); + expect(fixture.official.stdin.readableLength).toBe(0); + expect( + await fixture.host.handleConsoleRequest(VOICE_SETTINGS_METHOD, { operation: "get" }), + ).toMatchObject({ error: { code: -32601 } }); + expect(await fixture.host.handleConsoleRequest(VOICE_GATEWAY_STATUS_METHOD, {})).toEqual({ + result: false, + }); + } finally { + await stopFixture(fixture); + } +}); + +it("rejects voice operations when there is no local console service", async () => { + const fixture = createFixture(); + try { + writeRequest(fixture.desktopInput, { + id: 803, + method: VOICE_SETTINGS_METHOD, + params: { operation: "get" }, + }); + expect(await fixture.collector.waitFor((message) => requestId(message, 803))).toMatchObject({ + error: { code: -32090 }, + }); + } finally { + await stopFixture(fixture); + } +}); diff --git a/packages/host-runtime/test/app-server-host.voice-text-links.test.ts b/packages/host-runtime/test/app-server-host.voice-text-links.test.ts new file mode 100644 index 000000000..d936a27ea --- /dev/null +++ b/packages/host-runtime/test/app-server-host.voice-text-links.test.ts @@ -0,0 +1,120 @@ +import { FakeHarnessAdapter } from "@codexhost/harness-adapter/testing"; +import { harnessIdSchema, hostThreadIdSchema, type JsonObject } from "@codexhost/shared-contracts"; +import { expect, it, vi } from "vitest"; + +import type { ExternalVoiceCallFactory } from "../src/external-voice-gateway.js"; +import { + createFixture, + requestId, + startExternalThread, + stopFixture, + writeRequest, +} from "./app-server-host-fixture.js"; + +it("preserves written links and one Agent item identity when speech returns a shorter transcript", async () => { + const answer = [ + "- [Sannysoft](https://bot.sannysoft.com/)", + "- [Rebrowser](https://bot-detector.rebrowser.net/)", + "- [BrowserLeaks](https://browserleaks.com/javascript)", + ].join("\n"); + const spoken = "Here are the three browser test sites."; + const displayed = `::codex-realtime-inline{}\n${answer}`; + const adapter = new FakeHarnessAdapter(harnessIdSchema.parse("grok")); + let onEvent: Parameters[0]["onEvent"] | undefined; + const sent: JsonObject[] = []; + const fixture = createFixture({ + externalAdapters: new Map([["grok", adapter]]), + createExternalVoiceCall: async (options) => { + onEvent = options.onEvent; + return { + sdp: "v=0\r\nanswer", + connect: async () => {}, + send: (event) => { + sent.push(event); + }, + close: async () => {}, + }; + }, + }); + try { + const threadId = await startExternalThread(fixture, "codexhost/grok-native"); + writeRequest(fixture.desktopInput, { + id: 2, + method: "thread/realtime/start", + params: { + threadId, + outputModality: "audio", + transport: { type: "existingCall", callId: "rtc_fixture" }, + }, + }); + await fixture.collector.waitFor((message) => requestId(message, 2)); + writeRequest(fixture.desktopInput, { + id: 3, + method: "turn/start", + params: { threadId, input: [{ type: "text", text: "List the three test links." }] }, + }); + await fixture.collector.waitFor((message) => requestId(message, 3)); + const session = adapter.sessions[0]; + if (!session) throw new Error("Expected Grok Harness fixture session"); + session.appendText(answer); + session.succeedTurn(); + const terminal = await fixture.collector.waitFor( + (message) => + message.method === "turn/completed" && + (message.params as JsonObject)?.threadId === threadId, + ); + await vi.waitFor(() => + expect( + sent + .flatMap((event) => event.content as JsonObject[]) + .map((part) => part.text) + .join(""), + ).toBe(answer), + ); + expect(sent.every((event) => event.type === "session.context.append")).toBe(true); + + if (!onEvent) throw new Error("Expected speech event handler"); + onEvent({ type: "turn.done", turn: { role: "assistant", transcript: spoken } }); + const transcript = await fixture.collector.waitFor( + (message) => message.method === "thread/realtime/transcript/done", + ); + expect(transcript.params).toMatchObject({ threadId, role: "assistant", text: spoken }); + + const items = ((terminal.params as JsonObject).turn as JsonObject).items as JsonObject[]; + const answers = items.filter((item) => item.type === "agentMessage"); + expect(answers).toHaveLength(1); + expect(answers[0]).toMatchObject({ text: displayed, phase: "final_answer" }); + const completions = fixture.collector.messages + .filter( + (message) => + message.method === "item/completed" && + (message.params as JsonObject)?.threadId === threadId, + ) + .map((message) => (message.params as JsonObject).item as JsonObject) + .filter((item) => item.type === "agentMessage"); + expect(completions.length).toBeGreaterThan(0); + expect(new Set(completions.map((item) => item.id))).toEqual(new Set([answers[0]?.id])); + expect(completions.every((item) => item.text === displayed)).toBe(true); + + const record = await fixture.mappingStore.getThread(hostThreadIdSchema.parse(threadId)); + expect(record?.turnMappings[0]).toMatchObject({ realtimeInline: true }); + writeRequest(fixture.desktopInput, { + id: 4, + method: "thread/realtime/stop", + params: { threadId }, + }); + await fixture.collector.waitFor((message) => requestId(message, 4)); + writeRequest(fixture.desktopInput, { + id: 5, + method: "thread/read", + params: { threadId, includeTurns: true }, + }); + const reloaded = await fixture.collector.waitFor((message) => requestId(message, 5)); + const history = ((reloaded.result as JsonObject).thread as JsonObject).turns as JsonObject[]; + expect( + (history[0]?.items as JsonObject[]).find((item) => item.type === "agentMessage"), + ).toMatchObject({ id: answers[0]?.id, text: displayed }); + } finally { + await stopFixture(fixture); + } +}); diff --git a/packages/host-runtime/test/codexhost-desktop-rule.test.ts b/packages/host-runtime/test/codexhost-desktop-rule.test.ts new file mode 100644 index 000000000..5c5553a6f --- /dev/null +++ b/packages/host-runtime/test/codexhost-desktop-rule.test.ts @@ -0,0 +1,67 @@ +import { mkdir, mkdtemp, readFile, stat, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { retireCodexhostDesktopRules } from "../src/codexhost-desktop-rule.js"; + +async function home(): Promise { + return mkdtemp(path.join(os.tmpdir(), "codexhost-desktop-rule-")); +} + +function grokPath(root: string): string { + return path.join(root, ".grok", "rules", "codexhost-desktop.md"); +} + +function claudePath(root: string): string { + return path.join(root, ".claude", "rules", "codexhost-desktop.md"); +} + +describe("Codex Desktop home-rule retirement", () => { + it("reports absent when no copies exist and does not create them", async () => { + const root = await home(); + const results = await retireCodexhostDesktopRules({ homeDirectory: root }); + expect(results.map((result) => result.status)).toEqual(["absent", "absent"]); + await expect(stat(grokPath(root))).rejects.toMatchObject({ code: "ENOENT" }); + await expect(stat(claudePath(root))).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("removes the shipped Grok home-rule copy and a managed Claude leftover", async () => { + const root = await home(); + const shipped = await readFile(new URL("./fixtures/codexhost-desktop-v3.md", import.meta.url), { + encoding: "utf8", + }); + await mkdir(path.dirname(grokPath(root)), { recursive: true }); + await mkdir(path.dirname(claudePath(root)), { recursive: true }); + await writeFile(grokPath(root), shipped); + await writeFile(claudePath(root), shipped); + const results = await retireCodexhostDesktopRules({ homeDirectory: root }); + expect(results.map((result) => result.status)).toEqual(["removed", "removed"]); + await expect(stat(grokPath(root))).rejects.toMatchObject({ code: "ENOENT" }); + await expect(stat(claudePath(root))).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("removes a previous managed digest supplied for the upgrade case", async () => { + const root = await home(); + const previous = "\nold\n"; + await mkdir(path.dirname(grokPath(root)), { recursive: true }); + await writeFile(grokPath(root), previous); + const { createHash } = await import("node:crypto"); + const results = await retireCodexhostDesktopRules({ + homeDirectory: root, + previousManagedDigests: [createHash("sha256").update(previous).digest("hex")], + }); + expect(results.map((result) => result.status)).toEqual(["removed", "absent"]); + await expect(stat(grokPath(root))).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("preserves a user-edited copy", async () => { + const root = await home(); + await mkdir(path.dirname(grokPath(root)), { recursive: true }); + await writeFile(grokPath(root), "user content\n"); + const results = await retireCodexhostDesktopRules({ homeDirectory: root }); + expect(results.map((result) => result.status)).toEqual(["conflict", "absent"]); + await expect(readFile(grokPath(root), "utf8")).resolves.toBe("user content\n"); + }); +}); diff --git a/packages/host-runtime/test/console-opener.test.ts b/packages/host-runtime/test/console-opener.test.ts index 357616504..118b0228a 100644 --- a/packages/host-runtime/test/console-opener.test.ts +++ b/packages/host-runtime/test/console-opener.test.ts @@ -1,6 +1,7 @@ import { EventEmitter } from "node:events"; import path from "node:path"; import { PassThrough } from "node:stream"; +import { createServer } from "node:http"; import { describe, expect, it, vi } from "vitest"; @@ -25,6 +26,51 @@ function fakeChild(stdout: string, stderr: string, code: number) { } describe("Host console opener", () => { + it("ensures the console without opening a browser and sends bounded settings operations", async () => { + const received: unknown[] = []; + const server = createServer(async (request, response) => { + const chunks: Buffer[] = []; + for await (const chunk of request) chunks.push(Buffer.from(chunk)); + received.push({ + path: request.url, + header: request.headers["x-codexhost-console"], + body: JSON.parse(Buffer.concat(chunks).toString()), + }); + response.setHeader("content-type", "application/json"); + response.end(JSON.stringify({ settings: "fixture" })); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + const address = server.address(); + if (!address || typeof address === "string") throw new Error("Missing listener"); + const spawnProcess = vi.fn(() => + fakeChild(`codexhost console: http://127.0.0.1:${address.port}/\n`, "", 0), + ); + const opener = createHostConsoleOpener({ + entrypoint: "/console.mjs", + environment: {}, + spawnProcess: spawnProcess as never, + }); + try { + await expect(opener.voiceSettings?.({ operation: "get" })).resolves.toEqual({ + settings: "fixture", + }); + await opener.voiceSettings?.({ operation: "migrate", routes: {} }); + expect(spawnProcess).toHaveBeenCalledOnce(); + expect(spawnProcess.mock.calls[0]).toEqual([ + process.execPath, + ["/console.mjs", "ensure"], + expect.any(Object), + ]); + expect(received).toEqual([ + { path: "/api/voice-settings", header: "1", body: { operation: "get" } }, + { path: "/api/voice-settings", header: "1", body: { operation: "migrate", routes: {} } }, + ]); + } finally { + await new Promise((resolve, reject) => + server.close((error) => (error ? reject(error) : resolve())), + ); + } + }); it("finds the console beside a release Host Runtime or in a source checkout", () => { expect(consoleEntrypoint("/opt/codexhost/app/host-runtime.mjs", () => true)).toBe( path.join("/opt/codexhost/app", "console-server.mjs"), diff --git a/packages/host-runtime/test/external-subagent-threads.test.ts b/packages/host-runtime/test/external-subagent-threads.test.ts index 88da1ea68..7ff8f4dee 100644 --- a/packages/host-runtime/test/external-subagent-threads.test.ts +++ b/packages/host-runtime/test/external-subagent-threads.test.ts @@ -1,10 +1,10 @@ -import { mkdtemp, rm } from "node:fs/promises"; +import { mkdtemp, readFile, rm } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import type { HostThreadSnapshot, HostTurnSnapshot } from "@codexhost/harness-adapter"; import { FakeHarnessAdapter } from "@codexhost/harness-adapter/testing"; -import { MappingStore } from "@codexhost/mapping-store"; +import { MappingStore, storedThreadCoreV1Schema } from "@codexhost/mapping-store"; import { decodeThreadListRequest } from "@codexhost/protocol-core"; import { harnessIdSchema, @@ -17,7 +17,10 @@ import { import { afterEach, describe, expect, it, vi } from "vitest"; import { listExternalThreadMetadata } from "../src/external-thread-list.js"; -import { ExternalThreadRepository } from "../src/external-thread-repository.js"; +import { + ExternalThreadRepository, + externalThreadValue, +} from "../src/external-thread-repository.js"; import { ExternalThreadRuntime } from "../src/external-thread-runtime.js"; const directories: string[] = []; @@ -92,6 +95,80 @@ afterEach(async () => { }); describe("External native child identity", () => { + it("persists child configuration independently, upgrades legacy mappings, and never borrows parent settings", async () => { + const { directory, input, repository, store, parent } = await fixture(); + const configuredParent = await repository.setTransportModelId(parentId, "parent-model"); + const child = { + subagentId: "child", + nativeSubagentId: "child", + description: "Scout", + background: false, + status: "running" as const, + }; + let reopened: ExternalThreadRepository | undefined; + try { + const legacyId = hostThreadIdSchema.parse("legacy-child"); + await repository.createProvisional({ + ...input, + hostThreadId: legacyId, + createRequestId: "legacy-child", + transportModelId: "parent-model", + subagent: { parentHostThreadId: parentId, nativeSubagentId: "child" }, + }); + const legacy = await repository.commitNative(legacyId, sourceRef); + const view = (record: typeof legacy | null) => { + if (!record) throw new Error("Missing child mapping"); + return externalThreadValue({ record, turns: [], sessionId: parentId }); + }; + expect(view(legacy)).toMatchObject({ + model: null, + reasoningEffort: null, + canAcceptDirectInput: false, + }); + const known = await repository.materializeSubagent(configuredParent, { + ...child, + model: "sonnet", + reasoningEffort: "high", + }); + expect(known?.hostThreadId).toBe(legacyId); + expect(view(known)).toMatchObject({ + model: "sonnet", + reasoningEffort: "high", + parentThreadId: parentId, + }); + await repository.materializeSubagent(configuredParent, { ...child, model: "reported-model" }); + const missing = await repository.materializeSubagent(configuredParent, child); + expect(view(missing)).toMatchObject({ model: "reported-model", reasoningEffort: "high" }); + const unknown = await repository.materializeSubagent(configuredParent, { + ...child, + subagentId: "other", + nativeSubagentId: "other", + }); + expect(unknown?.transportModelId).not.toBe(configuredParent.transportModelId); + expect(view(unknown)).toMatchObject({ model: null, reasoningEffort: null }); + expect(view(parent)).not.toHaveProperty("model"); + expect(await repository.find(parentId)).toEqual(configuredParent); + // Child observations stay in the metadata sidecar, preserving the strict core format. + expect( + storedThreadCoreV1Schema.safeParse( + JSON.parse(await readFile(path.join(directory, "threads", `${legacyId}.json`), "utf8")), + ).success, + ).toBe(true); + await expect( + store.updateMetadata(parentId, { subagentConfiguration: { model: "wrong" } }), + ).rejects.toThrow("Subagent Thread"); + await repository.close(); + reopened = new ExternalThreadRepository(new MappingStore({ directory })); + await reopened.initialize(); + expect(view(await reopened.find(legacyId))).toMatchObject({ + model: "reported-model", + reasoningEffort: "high", + }); + } finally { + await (reopened ?? repository).close(); + } + }); + it.each([sourceRef, replacementRef])( "retains child and Turn IDs across rollback to $nativeSessionId", async (ref) => { @@ -126,8 +203,8 @@ describe("External native child identity", () => { }); expect(rolledBack.turns[0]?.items).toContainEqual( expect.objectContaining({ - type: "collabAgentToolCall", - receiverThreadIds: [child.hostThreadId], + type: "subAgentActivity", + agentThreadId: child.hostThreadId, }), ); expect( @@ -365,8 +442,8 @@ describe("External native child identity", () => { expect(list).toHaveBeenCalledTimes(1); expect(result.turns[0]?.items).toContainEqual( expect.objectContaining({ - type: "collabAgentToolCall", - receiverThreadIds: ["legacy-a", "legacy-b"], + type: "subAgentActivity", + agentThreadId: "legacy-a", }), ); } finally { diff --git a/packages/host-runtime/test/external-thread-follow-up-queue.test.ts b/packages/host-runtime/test/external-thread-follow-up-queue.test.ts new file mode 100644 index 000000000..ec687b929 --- /dev/null +++ b/packages/host-runtime/test/external-thread-follow-up-queue.test.ts @@ -0,0 +1,44 @@ +import { describe, expect, it } from "vitest"; + +import { + ExternalThreadFollowUpQueue, + queuedSubmissionText, +} from "../src/external-thread-follow-up-queue.js"; + +describe("ExternalThreadFollowUpQueue", () => { + it("adds, lists, starts, and restores a follow-up", () => { + const queue = new ExternalThreadFollowUpQueue(); + const input = [{ type: "text", text: "next task" }]; + const added = queue.add("thread", input, "client-1"); + expect(added.input).toEqual([{ type: "text", text: "next task", text_elements: [] }]); + expect(queue.list("thread").data).toEqual([added]); + expect(queuedSubmissionText(added.input)).toBe("next task"); + expect(queue.peek("thread", "client-1")?.id).toBe(added.id); + const taken = queue.take("thread", added.id); + expect(taken).toEqual(added); + expect(queue.list("thread").data).toEqual([]); + queue.restore("thread", added); + expect(queue.list("thread").data).toEqual([added]); + }); + + it("preserves existing text_elements on queued text items", () => { + const queue = new ExternalThreadFollowUpQueue(); + const input = [ + { + type: "text", + text: "kept", + text_elements: [{ byteRange: { start: 0, end: 4 }, placeholder: "x" }], + }, + ]; + expect(queue.add("thread", input, "client-2").input).toEqual(input); + }); + + it("canonicalizes input_text queue items to type text with text_elements", () => { + const queue = new ExternalThreadFollowUpQueue(); + const added = queue.add("thread", [{ type: "input_text", text: "from desktop" }], "client-3"); + expect(added.input).toEqual([{ type: "text", text: "from desktop", text_elements: [] }]); + const updated = queue.update("thread", added.id, [{ type: "input_text", text: "rewritten" }]); + expect(updated?.input).toEqual([{ type: "text", text: "rewritten", text_elements: [] }]); + expect(queue.list("thread").data[0]?.input).toEqual(updated?.input); + }); +}); diff --git a/packages/host-runtime/test/external-thread-repository.test.ts b/packages/host-runtime/test/external-thread-repository.test.ts index 4311ebd21..bce745576 100644 --- a/packages/host-runtime/test/external-thread-repository.test.ts +++ b/packages/host-runtime/test/external-thread-repository.test.ts @@ -68,6 +68,56 @@ afterEach(async () => { }); describe("ExternalThreadRepository", () => { + it("retains native voice Markdown presentation through reconciliation and a store reopen", async () => { + const directory = await temporaryStoreDirectory(); + let repository = new ExternalThreadRepository(new MappingStore({ directory })); + await repository.initialize(); + try { + await repository.createProvisional({ + hostThreadId, + createRequestId: "voice-presentation", + harnessId, + cwd: "/synthetic", + transportModelId: "codexhost/claude-code-native", + ephemeral: false, + historyMode: "paginated", + }); + const ready = await repository.commitNative(hostThreadId, nativeSessionRef); + const turn = snapshotTurn("voice"); + const text = "[A link](https://example.com/)"; + turn.items = [ + { + item: { type: "agentMessage", itemId: hostItemIdSchema.parse("answer"), text }, + outcome: { status: "succeeded" }, + }, + ]; + const persisted = await repository.persistTurn( + ready, + hostTurnIdSchema.parse("voice-turn"), + turn.nativeTurnRef, + undefined, + true, + ); + const aligned = await repository.alignSnapshot(persisted, { turns: [turn] }); + expect(aligned.record.turnMappings[0]?.realtimeInline).toBe(true); + await repository.close(); + repository = new ExternalThreadRepository(new MappingStore({ directory })); + await repository.initialize(); + const restored = await repository.find(hostThreadId); + if (!restored) throw new Error("Expected saved Thread"); + const reloaded = await repository.alignSnapshot(restored, { turns: [turn] }); + expect(reloaded.turns[0]?.items).toContainEqual( + expect.objectContaining({ + id: "answer", + text: `::codex-realtime-inline{}\n${text}`, + }), + ); + expect(turn.items[0]?.item).toMatchObject({ text }); + } finally { + await repository.close(); + } + }); + it("does not reconcile uncommitted or mismatched native history", async () => { const store = new MappingStore({ directory: await temporaryStoreDirectory() }); const repository = new ExternalThreadRepository(store); @@ -206,9 +256,9 @@ describe("ExternalThreadRepository", () => { }); expect(first.turns[0]?.items).toContainEqual( expect.objectContaining({ - type: "collabAgentToolCall", - senderThreadId: hostThreadId, - receiverThreadIds: [child?.hostThreadId], + type: "subAgentActivity", + agentThreadId: child?.hostThreadId, + kind: "started", }), ); await repository.close(); diff --git a/packages/host-runtime/test/external-voice-gateway.test.ts b/packages/host-runtime/test/external-voice-gateway.test.ts new file mode 100644 index 000000000..3e5758da9 --- /dev/null +++ b/packages/host-runtime/test/external-voice-gateway.test.ts @@ -0,0 +1,242 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import type WebSocket from "ws"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { configuredExternalVoiceGateway } from "../src/external-voice-gateway.js"; + +const connections = vi.hoisted(() => [] as { url: string; options: WebSocket.ClientOptions }[]); + +vi.mock("ws", async () => { + const { EventEmitter } = await import("node:events"); + return { + default: class extends EventEmitter { + static OPEN = 1; + readyState = 1; + send = vi.fn(); + terminate = vi.fn(); + constructor(url: URL, options: WebSocket.ClientOptions) { + super(); + connections.push({ url: String(url), options }); + queueMicrotask(() => this.emit("open")); + } + }, + }; +}); + +const homes: string[] = []; +const input = () => ({ + sdp: "v=0\r\nisolated-offer", + signal: new AbortController().signal, + onEvent: vi.fn(), + onError: vi.fn(), +}); + +async function gateway(extra = "", provider = 'env_key = "TEST_VOICE_KEY"') { + const home = await mkdtemp(path.join(tmpdir(), "host-voice-gateway-")); + homes.push(home); + await writeFile( + path.join(home, "config.toml"), + `model_provider = "selected" +experimental_realtime_webrtc_call_base_url = "https://gateway.example/v1" +${extra} +[model_providers.selected] +${provider} +[model_providers.unselected] +experimental_bearer_token = "wrong-provider-fixture" +`, + ); + return configuredExternalVoiceGateway({ CODEX_HOME: home, TEST_VOICE_KEY: "env-fixture" }); +} + +function created() { + return new Response("v=0\r\nisolated-answer", { + status: 201, + headers: { Location: "/v1/realtime/calls/test-call" }, + }); +} + +afterEach(async () => { + vi.unstubAllGlobals(); + vi.clearAllMocks(); + connections.length = 0; + await Promise.all(homes.splice(0).map((home) => rm(home, { recursive: true, force: true }))); +}); + +describe("configured voice gateway", () => { + it("joins a Desktop-created call using gateway auth without another creation request", async () => { + const request = vi.fn().mockResolvedValue(new Response()); + vi.stubGlobal("fetch", request); + const factory = await gateway(); + const call = await factory({ ...input(), sdp: "", callId: "rtc_fixture" }); + expect(request).not.toHaveBeenCalled(); + await call.connect(); + expect(connections[0]?.url).toBe("wss://gateway.example/v1/live/rtc_fixture"); + await call.close(); + expect(request).toHaveBeenCalledTimes(1); + expect(request.mock.calls[0]?.[0]).toBe( + "https://gateway.example/v1/realtime/calls/rtc_fixture/hangup", + ); + }); + it.each(["http://[invalid", "/unrecognized", ""])( + "rejects an unusable call identity without claiming remote cleanup: %s", + async (location) => { + const response = new Response("answer", { + status: 201, + headers: { Location: location }, + }); + const request = vi.fn().mockResolvedValue(response); + vi.stubGlobal("fetch", request); + const factory = await gateway(); + await expect(factory(input())).rejects.toThrow("remote cleanup is unconfirmed"); + expect(response.body?.locked).toBe(false); + expect(request).toHaveBeenCalledTimes(1); + expect(connections).toHaveLength(0); + }, + ); + + it.each([ + ['env_key = "TEST_VOICE_KEY"', "env-fixture"], + ['experimental_bearer_token = "bearer-fixture"', "bearer-fixture"], + ])( + "uses the selected provider credential for native call and control: %s", + async (provider, token) => { + const request = vi + .fn() + .mockResolvedValueOnce(created()) + .mockResolvedValueOnce(new Response()); + vi.stubGlobal("fetch", request); + const factory = await gateway("", provider); + const callbacks = input(); + const call = await factory(callbacks); + expect(call.sdp).toBe("v=0\r\nisolated-answer"); + const creation = request.mock.calls[0]; + if (!creation) throw new Error("Call creation request is missing"); + const [url, init] = creation; + expect(url).toBe("https://gateway.example/v1/realtime/calls"); + expect(init).toMatchObject({ + method: "POST", + redirect: "error", + headers: { Authorization: `Bearer ${token}`, "OpenAI-Alpha": "quicksilver=v2" }, + }); + const form = init?.body as FormData; + expect(form.get("sdp")).toBe(callbacks.sdp); + expect(JSON.parse(String(form.get("session")))).toMatchObject({ + model: "gpt-realtime", + delegation: { type: "client", ack_filler: false }, + audio: { output: { voice: "cove" } }, + }); + expect(JSON.parse(String(form.get("session")))).not.toHaveProperty("input"); + expect(JSON.parse(String(form.get("session")))).not.toHaveProperty("initial_items"); + await call.connect(); + expect(connections).toHaveLength(1); + expect(connections[0]).toMatchObject({ + url: "wss://gateway.example/v1/live/test-call", + options: { headers: init?.headers, autoSelectFamily: true }, + }); + await Promise.all([call.close(), call.close()]); + expect(request).toHaveBeenCalledTimes(2); + expect(request.mock.calls[1]?.[0]).toBe( + "https://gateway.example/v1/realtime/calls/test-call/hangup", + ); + await expect(call.connect()).rejects.toThrow("closed"); + expect(callbacks.onError).not.toHaveBeenCalled(); + }, + ); + + it.each([ + 'experimental_realtime_ws_base_url = "http://gateway.example/v1"', + 'experimental_realtime_ws_base_url = "https://other.example/v1"', + 'experimental_realtime_ws_base_url = "https://user:password@gateway.example/v1"', + 'experimental_realtime_ws_base_url = "https://gateway.example/v1?key=fixture"', + 'experimental_realtime_ws_base_url = "https://gateway.example/v1#fragment"', + 'experimental_realtime_ws_base_url = "malformed-url"', + ])("rejects unsafe gateway configuration before sending credentials: %s", async (extra) => { + const request = vi.fn(); + vi.stubGlobal("fetch", request); + const factory = await gateway(extra); + await expect(factory(input())).rejects.toThrow(/External voice/); + expect(request).not.toHaveBeenCalled(); + }); + + it("includes resumed conversation history in the native session create request", async () => { + const request = vi + .fn() + .mockResolvedValueOnce(created()) + .mockResolvedValueOnce(new Response()); + vi.stubGlobal("fetch", request); + const factory = await gateway(); + const history = [ + { + type: "message", + role: "user", + content: [{ type: "input_text", text: "list files" }], + }, + { + type: "message", + role: "assistant", + content: [{ type: "output_text", text: "Only empty outputs/ and work/." }], + }, + ]; + await factory({ ...input(), input: history }); + const form = request.mock.calls[0]?.[1]?.body as FormData; + const session = JSON.parse(String(form.get("session"))); + expect(session.initial_items).toEqual(history); + expect(session).not.toHaveProperty("input"); + }); + + it("never substitutes another provider or OAuth when the selected credential is missing", async () => { + const request = vi.fn(); + vi.stubGlobal("fetch", request); + const factory = await gateway("", 'env_key = "MISSING_KEY"'); + await expect(factory(input())).rejects.toThrow( + "configured realtime gateway and API credential", + ); + expect(request).not.toHaveBeenCalled(); + }); + + it("does not expose a rejected gateway response body", async () => { + vi.stubGlobal( + "fetch", + vi.fn().mockResolvedValue(new Response("private-fixture-error", { status: 401 })), + ); + const factory = await gateway(); + await expect(factory(input())).rejects.toThrow(/^External voice call creation failed$/); + }); + + it("hangs up a call cancelled while its SDP answer is still loading", async () => { + const body = Promise.withResolvers(); + const response = created(); + vi.spyOn(response, "text").mockImplementation(() => body.promise); + const request = vi + .fn() + .mockResolvedValueOnce(response) + .mockResolvedValueOnce(new Response(null, { status: 404 })); + vi.stubGlobal("fetch", request); + const controller = new AbortController(); + const factory = await gateway(); + const starting = factory({ ...input(), signal: controller.signal }); + const rejected = expect(starting).rejects.toThrow("startup failed"); + await vi.waitFor(() => expect(response.text).toHaveBeenCalled()); + controller.abort(); + body.resolve("v=0\r\nlate-answer"); + await rejected; + expect(request).toHaveBeenCalledTimes(2); + expect(request.mock.calls[1]?.[0]).toContain("/test-call/hangup"); + expect(request.mock.calls[1]?.[1]?.signal?.aborted).toBe(false); + expect(connections).toHaveLength(0); + }); + + it("cleans up a call whose SDP body fails and reports unconfirmed hangup", async () => { + const response = created(); + vi.spyOn(response, "text").mockRejectedValue(new Error("broken stream")); + const request = vi + .fn() + .mockResolvedValueOnce(response) + .mockResolvedValueOnce(new Response(null, { status: 503 })); + vi.stubGlobal("fetch", request); + const factory = await gateway(); + await expect(factory(input())).rejects.toThrow("hangup was not confirmed"); + expect(request).toHaveBeenCalledTimes(2); + }); +}); diff --git a/packages/host-runtime/test/external-voice-lifecycle.test.ts b/packages/host-runtime/test/external-voice-lifecycle.test.ts new file mode 100644 index 000000000..a1cabdaca --- /dev/null +++ b/packages/host-runtime/test/external-voice-lifecycle.test.ts @@ -0,0 +1,542 @@ +import { describe, expect, it, vi } from "vitest"; +import { ExternalVoice } from "../src/external-voice.js"; +import type { ExternalVoiceCall } from "../src/external-voice-gateway.js"; + +const params = { outputModality: "audio", transport: { type: "webrtc", sdp: "v=0\r\noffer" } }; + +async function voiceSession() { + let event: Parameters< + ConstructorParameters[0]["createCall"] + >[0]["onEvent"] = () => {}; + const send = vi.fn(); + const task = vi.fn(); + const voice = new ExternalVoice({ + createCall: async (input) => { + event = input.onEvent; + return { sdp: "answer", connect: async () => {}, send, close: async () => {} }; + }, + runTask: task, + notify: async () => {}, + diagnose: vi.fn(), + }); + await voice.start("external", params); + return { event, send, task, voice }; +} + +describe("voice lifecycle", () => { + it("holds observed task output until the control connection opens", async () => { + const connected = Promise.withResolvers(); + let open = false; + const send = vi.fn(() => { + if (!open) throw new Error("Control connection not open"); + }); + const voice = new ExternalVoice({ + createCall: async () => ({ + sdp: "answer", + connect: () => connected.promise, + send, + close: async () => {}, + }), + runTask: vi.fn(), + notify: async () => {}, + diagnose: vi.fn(), + }); + try { + await voice.start("external", params); + voice.reportProgress("external", "Checking the audit."); + voice.followTurn("external", async () => "Audit complete."); + await new Promise((resolve) => setImmediate(resolve)); + expect(send).not.toHaveBeenCalled(); + expect(voice.has("external")).toBe(true); + open = true; + connected.resolve(undefined); + await vi.waitFor(() => + expect(send.mock.calls).toEqual([ + [ + { + type: "session.context.append", + content: [{ type: "input_text", text: "Checking the audit." }], + }, + ], + [ + { + type: "session.context.append", + content: [{ type: "input_text", text: "Audit complete." }], + }, + ], + ]), + ); + } finally { + await voice.close(); + } + }); + + it("attaches a Desktop-owned call without creating a second SDP exchange", async () => { + const createCall = vi.fn(async () => ({ + sdp: "", + connect: vi.fn(async () => {}), + send: vi.fn(), + close: vi.fn(async () => {}), + })); + const notify = vi.fn(async () => {}); + const voice = new ExternalVoice({ createCall, notify, runTask: vi.fn(), diagnose: vi.fn() }); + await voice.start("external", { + outputModality: "audio", + realtimeSessionId: "native-session", + transport: { type: "existingCall", callId: "rtc_fixture" }, + }); + expect(createCall).toHaveBeenCalledWith( + expect.objectContaining({ callId: "rtc_fixture", sdp: "" }), + ); + expect(notify).toHaveBeenCalledWith( + "thread/realtime/started", + expect.objectContaining({ realtimeSessionId: "native-session" }), + ); + expect(notify.mock.calls.some(([method]) => method === "thread/realtime/sdp")).toBe(false); + await voice.close(); + }); + it("answers a native handoff before turn.done and deduplicates its later transcript", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Two empty folders."); + session.event({ + type: "turn.created", + turn: { id: "u-native", role: "user", transcript: "What" }, + }); + const handoff = { + type: "delegation.created", + item: { + id: "native-handle", + type: "delegation", + target: "client", + handoff_id: "handoff-native", + user_bidi_turn_id: "u-native", + content: [{ type: "input_text", text: "What files did you just find?" }], + }, + }; + session.event(handoff); + session.event(handoff); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + expect(session.task).toHaveBeenCalledWith( + "external", + "What files did you just find?", + expect.any(AbortSignal), + ); + await vi.waitFor(() => + expect(session.send).toHaveBeenCalledWith({ + type: "delegation.context.append", + delegation_item_id: "native-handle", + channel: "speakable", + content: [{ type: "input_text", text: "Two empty folders." }], + }), + ); + session.event({ + type: "turn.done", + turn: { id: "u-native", role: "user", transcript: "What files did you just find?" }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(session.task).toHaveBeenCalledTimes(1); + expect(session.send).toHaveBeenCalledTimes(1); + await session.voice.close(); + }); + + it.each(["thread/realtime/started", "thread/realtime/sdp"])( + "does not resume startup after hangup while publishing %s", + async (blockedMethod) => { + const published = Promise.withResolvers(); + const release = Promise.withResolvers(); + const connect = vi.fn(async () => {}); + const close = vi.fn(async () => {}); + const notify = vi.fn(async (method) => { + if (method === blockedMethod) { + published.resolve(undefined); + await release.promise; + } + }); + const voice = new ExternalVoice({ + createCall: async () => ({ sdp: "answer", connect, close, send: () => {} }), + notify, + runTask: vi.fn(), + diagnose: vi.fn(), + }); + const starting = voice.start("external", params); + const rejectedStart = expect(starting).rejects.toThrow("could not start"); + await published.promise; + await voice.stop("external"); + const notificationsAtHangup = notify.mock.calls.length; + release.resolve(undefined); + await rejectedStart; + expect(connect).not.toHaveBeenCalled(); + expect(close).toHaveBeenCalledTimes(1); + expect(notify).toHaveBeenCalledTimes(notificationsAtHangup); + expect(notify.mock.calls.at(-1)?.[0]).toBe("thread/realtime/closed"); + expect(voice.has("external")).toBe(false); + }, + ); + + it("waits for cleanup when stopped during call creation and never publishes a late start", async () => { + const creation = Promise.withResolvers(); + const close = vi.fn(async () => {}); + const notify = vi.fn(async () => {}); + const voice = new ExternalVoice({ + createCall: () => creation.promise, + notify, + runTask: vi.fn(), + diagnose: vi.fn(), + }); + const starting = voice.start("external", params); + const rejectedStart = expect(starting).rejects.toThrow("could not start"); + const stopping = voice.stop("external"); + expect(voice.has("external")).toBe(true); + creation.resolve({ sdp: "answer", close, connect: async () => {}, send: () => {} }); + await stopping; + await rejectedStart; + expect(close).toHaveBeenCalled(); + expect(notify.mock.calls.map((call) => call[0])).toEqual(["thread/realtime/closed"]); + expect(voice.has("external")).toBe(false); + }); + + it("starts a Harness task from a user transcript when the speech model does not delegate", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + expect(session.task).toHaveBeenCalledWith( + "external", + "你啱啱搵到咩檔案", + expect.any(AbortSignal), + ); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + expect(session.send).toHaveBeenCalledWith({ + type: "session.context.append", + content: [{ type: "input_text", text: "Only empty outputs/ and work/." }], + }); + }); + + it("starts a new Harness turn when the same question arrives on a distinct user turn", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("ok"); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + session.event({ + type: "turn.done", + turn: { id: "u2", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(2)); + }); + + it("starts distinct rapid user turns without waiting for a quiet window", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("ok"); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + session.event({ + type: "turn.done", + turn: { id: "u2", role: "user", transcript: "你啱啱個任務搵到啲咩" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(2)); + expect(session.task).toHaveBeenNthCalledWith( + 1, + "external", + "你啱啱搵到咩檔案", + expect.any(AbortSignal), + ); + expect(session.task).toHaveBeenNthCalledWith( + 2, + "external", + "你啱啱個任務搵到啲咩", + expect.any(AbortSignal), + ); + }); + + it("keeps one Harness turn and appends session context when a paraphrased delegation has no turn id", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + session.event({ + type: "delegation.created", + item: { + id: "task", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "Summarize the files found by the previous task." }], + }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + expect(session.task).toHaveBeenCalledWith( + "external", + "你啱啱搵到咩檔案", + expect.any(AbortSignal), + ); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + expect(session.send).toHaveBeenCalledWith({ + type: "session.context.append", + content: [{ type: "input_text", text: "Only empty outputs/ and work/." }], + }); + expect(session.send).not.toHaveBeenCalledWith( + expect.objectContaining({ delegation_item_id: "task" }), + ); + }); + + it("attaches a late delegation that names the user turn without a second Harness turn", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + session.event({ + type: "delegation.created", + item: { + id: "late", + type: "delegation", + target: "client", + turn_id: "u1", + content: [{ type: "input_text", text: "Summarize the files found by the previous task." }], + }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(session.task).toHaveBeenCalledTimes(1); + expect(session.send).toHaveBeenCalledWith({ + type: "delegation.context.append", + delegation_item_id: "late", + channel: "speakable", + content: [{ type: "input_text", text: "Only empty outputs/ and work/." }], + }); + }); + + it("runs a linked handoff before the final transcript and deduplicates that transcript", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "delegation.created", + item: { + id: "early", + type: "delegation", + target: "client", + turn_id: "u1", + content: [{ type: "input_text", text: "你啱啱搵到咩檔案" }], + }, + }); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + expect(session.task).toHaveBeenCalledWith( + "external", + "你啱啱搵到咩檔案", + expect.any(AbortSignal), + ); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + expect(session.send).toHaveBeenCalledWith({ + type: "delegation.context.append", + delegation_item_id: "early", + channel: "speakable", + content: [{ type: "input_text", text: "Only empty outputs/ and work/." }], + }); + }); + + it("appends context for a no-id delegation path after one user transcript", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "delegation.created", + item: { + id: "early-noid", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "Summarize the files found by the previous task." }], + }, + }); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + expect(session.send).toHaveBeenCalledWith({ + type: "session.context.append", + content: [{ type: "input_text", text: "Only empty outputs/ and work/." }], + }); + expect(session.send).not.toHaveBeenCalledWith( + expect.objectContaining({ delegation_item_id: "early-noid" }), + ); + }); + + it("does not run a late unlinked delegation after two user utterances", async () => { + const session = await voiceSession(); + session.task.mockImplementation(async (_thread, text) => `answer:${text}`); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + session.event({ + type: "turn.done", + turn: { id: "u2", role: "user", transcript: "你啱啱個任務搵到啲咩" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(2)); + await vi.waitFor(() => expect(session.send).toHaveBeenCalledTimes(2)); + session.event({ + type: "delegation.created", + item: { + id: "late-unlinked", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "Summarize the files found by the previous task." }], + }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(session.task).toHaveBeenCalledTimes(2); + expect(session.send).not.toHaveBeenCalledWith( + expect.objectContaining({ delegation_item_id: "late-unlinked" }), + ); + }); + + it("does not bind an unrelated delegation to an old completed unmatched turn", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("Only empty outputs/ and work/."); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + session.event({ + type: "delegation.created", + item: { + id: "unrelated", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "Start a new investigation." }], + }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(session.task).toHaveBeenCalledTimes(1); + expect(session.send).not.toHaveBeenCalledWith( + expect.objectContaining({ delegation_item_id: "unrelated" }), + ); + }); + + it("does not bind a late no-id handle to a later unrelated user transcript", async () => { + const session = await voiceSession(); + session.task + .mockResolvedValueOnce("Only empty outputs/ and work/.") + .mockResolvedValueOnce("The passphrase is still blue moon."); + session.event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "你啱啱搵到咩檔案" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(1)); + await vi.waitFor(() => expect(session.send).toHaveBeenCalled()); + session.event({ + type: "delegation.created", + item: { + id: "old-handle", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "Summarize the files found by the previous task." }], + }, + }); + session.event({ + type: "turn.done", + turn: { id: "u2", role: "user", transcript: "藍色月亮係咩" }, + }); + await vi.waitFor(() => expect(session.task).toHaveBeenCalledTimes(2)); + await vi.waitFor(() => + expect(session.send).toHaveBeenCalledWith({ + type: "session.context.append", + content: [{ type: "input_text", text: "The passphrase is still blue moon." }], + }), + ); + expect(session.send).not.toHaveBeenCalledWith( + expect.objectContaining({ + delegation_item_id: "old-handle", + content: [{ type: "input_text", text: "The passphrase is still blue moon." }], + }), + ); + }); + + it("passes resumed Thread messages into call creation", async () => { + const createCall = vi.fn(async () => ({ + sdp: "answer", + connect: async () => {}, + send: () => {}, + close: async () => {}, + })); + const voice = new ExternalVoice({ + createCall, + runTask: vi.fn(), + notify: async () => {}, + diagnose: vi.fn(), + }); + const conversation = [ + { + type: "message", + role: "assistant", + content: [{ type: "output_text", text: "Only empty outputs/ and work/." }], + }, + ]; + await voice.start("external", params, conversation); + expect(createCall).toHaveBeenCalledWith(expect.objectContaining({ input: conversation })); + }); + + it("suppresses a late agent answer after hangup without cancelling its work", async () => { + const answer = Promise.withResolvers(); + let event: Parameters< + ConstructorParameters[0]["createCall"] + >[0]["onEvent"] = () => {}; + const send = vi.fn(); + const close = vi.fn(async () => {}); + const task = vi.fn(() => answer.promise); + const voice = new ExternalVoice({ + createCall: async (input) => { + event = input.onEvent; + return { sdp: "answer", connect: async () => {}, send, close }; + }, + runTask: task, + notify: async () => {}, + diagnose: vi.fn(), + }); + await voice.start("external", params); + event({ + type: "turn.done", + turn: { id: "u1", role: "user", transcript: "request" }, + }); + await vi.waitFor(() => expect(task).toHaveBeenCalledTimes(1)); + await voice.stop("external"); + answer.resolve("old result"); + await new Promise((resolve) => setImmediate(resolve)); + expect(send).not.toHaveBeenCalled(); + expect(close).toHaveBeenCalledTimes(1); + }); + + it("does not start Harness work from delegation when no user transcript arrives", async () => { + const session = await voiceSession(); + session.task.mockResolvedValue("should not run"); + session.event({ + type: "delegation.created", + item: { + id: "task", + type: "delegation", + target: "client", + content: [{ type: "input_text", text: "request" }], + }, + }); + await new Promise((resolve) => setImmediate(resolve)); + expect(session.task).not.toHaveBeenCalled(); + expect(session.send).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/host-runtime/test/external-voice-results.test.ts b/packages/host-runtime/test/external-voice-results.test.ts new file mode 100644 index 000000000..0a5f09c5d --- /dev/null +++ b/packages/host-runtime/test/external-voice-results.test.ts @@ -0,0 +1,143 @@ +import { describe, expect, it, vi } from "vitest"; +import { hostItemIdSchema, hostTurnIdSchema } from "@codexhost/shared-contracts"; +import { ExternalVoiceResults, speakableFromProjectedTurn } from "../src/external-voice-results.js"; + +const threadId = "thread-1"; +const turnId = hostTurnIdSchema.parse("11111111-1111-4111-8111-111111111111"); +const itemId = hostItemIdSchema.parse("22222222-2222-4222-8222-222222222222"); + +describe("external voice results", () => { + it("forwards completed commentary once while withholding the final answer", async () => { + const progress = vi.fn(); + const results = new ExternalVoiceResults(progress); + const pending = results.watch(threadId, turnId, new AbortController().signal); + const item = { + type: "agentMessage" as const, + itemId, + text: "Checking the findings.", + phase: "commentary" as const, + }; + results.observe(threadId, { type: "item.started", turnId, item }); + expect(progress).not.toHaveBeenCalled(); + const completed = { + type: "item.completed" as const, + turnId, + snapshot: { item, outcome: { status: "succeeded" as const } }, + }; + results.observe("unrelated-thread", completed); + expect(progress).not.toHaveBeenCalled(); + results.observe(threadId, completed); + results.observe(threadId, completed); + expect(progress.mock.calls).toEqual([[threadId, item.text]]); + results.observe( + threadId, + { type: "turn.completed", turnId, outcome: { status: "succeeded" } }, + { items: [{ type: "agentMessage", text: "Audit complete.", phase: "final_answer" }] }, + ); + await expect(pending).resolves.toBe("Audit complete."); + }); + + it("recognizes unphased progress only after later work starts, without repeating it as a final", async () => { + const progress = vi.fn(); + const results = new ExternalVoiceResults(progress); + const pending = results.watch(threadId, turnId, new AbortController().signal); + const item = { type: "agentMessage" as const, itemId, text: "Both scouts have returned." }; + results.observe(threadId, { type: "item.started", turnId, item }); + const completed = { + type: "item.completed" as const, + turnId, + snapshot: { item, outcome: { status: "succeeded" as const } }, + }; + results.observe(threadId, completed); + expect(progress).not.toHaveBeenCalled(); + const nextItemId = hostItemIdSchema.parse("33333333-3333-4333-8333-333333333333"); + const next = { + type: "item.started" as const, + turnId, + item: { type: "agentMessage" as const, itemId: nextItemId, text: "Audit complete." }, + }; + results.observe(threadId, next); + expect(progress.mock.calls).toEqual([[threadId, item.text]]); + results.observe(threadId, completed); + results.observe(threadId, { + type: "item.completed", + turnId, + snapshot: { item: next.item, outcome: { status: "succeeded" } }, + }); + expect(progress).toHaveBeenCalledTimes(1); + results.observe(threadId, { type: "turn.completed", turnId, outcome: { status: "succeeded" } }); + await expect(pending).resolves.toBe("Audit complete."); + }); + + it("rejects failed work and session faults instead of reporting successful completion", async () => { + const results = new ExternalVoiceResults(); + const controller = new AbortController(); + const failed = results.watch(threadId, turnId, controller.signal); + const failedCheck = expect(failed).rejects.toThrow("did not complete"); + results.observe(threadId, { type: "turn.completed", turnId, outcome: { status: "cancelled" } }); + await failedCheck; + const faulted = results.watch(threadId, turnId, controller.signal); + const faultCheck = expect(faulted).rejects.toThrow("session failed"); + results.observe(threadId, { + type: "session.faulted", + error: { code: "internalError", message: "Session unavailable", retryable: false }, + }); + await faultCheck; + expect(results.has(threadId, turnId)).toBe(false); + }); + + it("speaks the inferred final answer when live item completion is missing", async () => { + const results = new ExternalVoiceResults(); + const controller = new AbortController(); + const pending = results.watch(threadId, turnId, controller.signal); + results.observe( + threadId, + { type: "turn.completed", turnId, outcome: { status: "succeeded" } }, + { + items: [ + { type: "commandExecution", command: "ls", status: "completed" }, + { + type: "agentMessage", + text: "Workspace has empty outputs/ and work/.", + phase: "final_answer", + }, + ], + }, + ); + await expect(pending).resolves.toBe("Workspace has empty outputs/ and work/."); + }); + + it("accumulates streamed agent text before turn completion", async () => { + const results = new ExternalVoiceResults(); + const controller = new AbortController(); + const pending = results.watch(threadId, turnId, controller.signal); + results.observe(threadId, { + type: "item.started", + turnId, + item: { type: "agentMessage", itemId, text: "" }, + }); + results.observe(threadId, { + type: "item.updated", + turnId, + itemId, + update: { type: "text.append", text: "The passphrase is blue moon." }, + }); + results.observe(threadId, { + type: "turn.completed", + turnId, + outcome: { status: "succeeded" }, + }); + await expect(pending).resolves.toBe("The passphrase is blue moon."); + }); + + it("prefers projected final_answer over commentary", () => { + expect( + speakableFromProjectedTurn({ + items: [ + { type: "agentMessage", text: "Checking files.", phase: "commentary" }, + { type: "agentMessage", text: "Only empty directories.", phase: "final_answer" }, + ], + }), + ).toBe("Only empty directories."); + }); +}); diff --git a/packages/host-runtime/test/external-voice-session-input.test.ts b/packages/host-runtime/test/external-voice-session-input.test.ts new file mode 100644 index 000000000..c682612cb --- /dev/null +++ b/packages/host-runtime/test/external-voice-session-input.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, it } from "vitest"; +import { voiceSessionInputFromTurns } from "../src/external-voice-session-input.js"; + +describe("voice session input from Thread Turns", () => { + it("seeds user and agent messages and skips commentary and tools", () => { + expect( + voiceSessionInputFromTurns([ + { + items: [ + { + type: "userMessage", + content: [{ type: "text", text: "what files did you just find?" }], + }, + { type: "commandExecution", command: "ls" }, + { type: "agentMessage", text: "Checking.", phase: "commentary" }, + { + type: "agentMessage", + text: "::codex-realtime-inline{}\nOnly empty outputs/ and work/.", + phase: "final_answer", + }, + ], + }, + ]), + ).toEqual([ + { + type: "message", + role: "user", + content: [{ type: "input_text", text: "what files did you just find?" }], + }, + { + type: "message", + role: "assistant", + content: [{ type: "output_text", text: "Only empty outputs/ and work/." }], + }, + ]); + }); + + it("returns no input for a Thread with no speakable history", () => { + expect(voiceSessionInputFromTurns([])).toEqual([]); + expect(voiceSessionInputFromTurns([{ items: [{ type: "commandExecution" }] }])).toEqual([]); + }); +}); diff --git a/packages/host-runtime/test/fixtures/codexhost-desktop-v3.md b/packages/host-runtime/test/fixtures/codexhost-desktop-v3.md new file mode 100644 index 000000000..87d3f5789 --- /dev/null +++ b/packages/host-runtime/test/fixtures/codexhost-desktop-v3.md @@ -0,0 +1,32 @@ + +# Codex Desktop / CodexHost + +CodexHost installs this file into Grok home rules. Orca and Grok TUI sessions may also load it. + +At the first user turn, if you have a shell, run this once and print only `set` or `unset`: + +```sh +if test -n "${CODEXHOST_THREAD_ID-}"; then echo set; else echo unset; fi +``` + +- `set`: this is a CodexHost external Thread, including Codex Desktop. Apply the session rules even when cwd is a real repository such as `codex-host` or the wiki. +- Working directory is a Playground folder named `realtime-voice-chat` or `realtime-voice-chat-*`: apply the session rules and ask the user to start a new Thread in the real repo. +- `unset` and not a Playground voice cwd: leave the session rules inactive. + +Never print `CODEXHOST_THREAD_ID`, `CODEXHOST_RUNTIME_TOKEN`, `CODEXHOST_RUNTIME_ENDPOINT`, or other runtime descriptors. Do not invent other host signals. + +## Session rules + +1. **Model.** The Desktop composer Model is this Harness's Model. Pick that Harness's real Model before starting voice. For Grok, that is grok-4.6 or grok-4.7. `flash-max` is a Gemini Flash Model. + +2. **Working directory.** Start the chat in the real repository (`codex-host` or the wiki). A Playground `realtime-voice-chat*` folder is a scratch cwd. Ask the user to start a new Thread in the real repo. + +3. **Voice vs tools.** Desktop voice is gpt-live speech plus delegated Harness tasks on this same Thread. Do wiki, MCP, and file-heavy work as text in this Thread. + +4. **Harness-owned tools.** This session's MCP servers and skills belong to this Harness. SkillWiki on Grok is Grok's config, not Claude, Pi, or Codex. + +5. **Desktop chrome.** The window is Codex Desktop. Markdown View preview is a Codex file tab (View preview vs View source). `file://` browser tabs stay raw. + +6. **Voice routes.** Official Codex live voice (New API/CPA) is a separate route from Host-owned external `thread/realtime/*`. This external Thread uses the Host-owned route. + +Keep Grok Build tools, MCP, and rules inside the native session. An `AGENT_TOOLS_AND_RULES.md` dump in the cwd is unnecessary. diff --git a/packages/host-runtime/test/native-account-host.test.ts b/packages/host-runtime/test/native-account-host.test.ts index dbcb66182..f3a204882 100644 --- a/packages/host-runtime/test/native-account-host.test.ts +++ b/packages/host-runtime/test/native-account-host.test.ts @@ -81,13 +81,15 @@ function nativeFixture() { for (const connection of connections) connection.close(); exit.resolve({ code: 0, signal: null }); }); - vi.mocked(createOwnedLoopbackBackend).mockReturnValue({ + const backend = { closed: exit.promise, start: vi.fn(async () => {}), connect, stop, - }); + }; + vi.mocked(createOwnedLoopbackBackend).mockReturnValue(backend); return { + backend, requests, connect, stop, @@ -98,7 +100,25 @@ function nativeFixture() { }; } -async function startLocal() { +async function withUnhandledRejections(run: () => Promise): Promise<{ + value: T; + rejections: string[]; +}> { + const rejections: string[] = []; + const onRejection = (reason: unknown) => { + rejections.push(reason instanceof Error ? reason.name : String(reason)); + }; + process.on("unhandledRejection", onRejection); + try { + const value = await run(); + await Promise.resolve(); + return { value, rejections }; + } finally { + process.off("unhandledRejection", onRejection); + } +} + +async function startLocal(options?: { recoveryWaitMs?: number; delaysMs?: readonly number[] }) { const directory = await mkdtemp(path.join(os.tmpdir(), "codexhost-native-account-host-")); temporaryDirectories.add(directory); const local = await prepareLocalCodex({ @@ -106,6 +126,8 @@ async function startLocal() { arguments: ["app-server"], environment: { CODEX_HOME: path.join(directory, "missing-codex-home") }, diagnosticOutput: new PassThrough(), + ...(options?.delaysMs ? { recovery: { delaysMs: options.delaysMs } } : {}), + ...(options?.recoveryWaitMs !== undefined ? { recoveryWaitMs: options.recoveryWaitMs } : {}), }); prepared.add(local); return local; @@ -157,11 +179,58 @@ describe("local Codex read-only identity startup", () => { } }); - it("closes the owned backend when identity connection initialization fails", async () => { + it("recovers a failed first official start without closing the Scope", async () => { + const first = nativeFixture(); + first.connect.mockRejectedValueOnce(new Error("Synthetic connection failure")); + const firstBackend = first.backend; + let calls = 0; + vi.mocked(createOwnedLoopbackBackend).mockImplementation(() => { + calls += 1; + if (calls === 1) return firstBackend; + return nativeFixture().backend; + }); + const { value: local, rejections } = await withUnhandledRejections(() => + startLocal({ delaysMs: [0], recoveryWaitMs: 2_000 }), + ); + expect(rejections).toEqual([]); + expect(local.officialRuntimeScope.gate.phase).toBe("ready"); + expect(local.officialRuntimeScope.owner.generation).toBe(2); + expect(first.stop).toHaveBeenCalled(); + await local.accountControl.refresh?.(); + }); + + it("recovers a delayed second generation without unhandledRejection", async () => { + const first = nativeFixture(); + first.connect.mockRejectedValueOnce(new Error("Synthetic connection failure")); + const firstBackend = first.backend; + let calls = 0; + vi.mocked(createOwnedLoopbackBackend).mockImplementation(() => { + calls += 1; + if (calls === 1) return firstBackend; + const second = nativeFixture(); + second.backend.start = vi.fn(async () => { + await new Promise((resolve) => setTimeout(resolve, 80)); + }); + return second.backend; + }); + const { value: local, rejections } = await withUnhandledRejections(() => + startLocal({ delaysMs: [20], recoveryWaitMs: 1_000 }), + ); + expect(rejections).toEqual([]); + expect(local.officialRuntimeScope.gate.phase).toBe("ready"); + expect(local.officialRuntimeScope.owner.generation).toBe(2); + }); + + it("closes the owned backend when first start and bounded recovery both fail", async () => { const f = nativeFixture(); - f.connect.mockRejectedValueOnce(new Error("Synthetic connection failure")); - await expect(startLocal()).rejects.toThrow(); - expect(f.stop).toHaveBeenCalledOnce(); + f.connect.mockRejectedValue(new Error("Synthetic connection failure")); + const { rejections } = await withUnhandledRejections(async () => { + await expect(startLocal({ delaysMs: [0], recoveryWaitMs: 30 })).rejects.toThrow( + "unavailable", + ); + }); + expect(rejections).toEqual([]); + expect(f.stop).toHaveBeenCalled(); expect(f.requests).toEqual([]); }); diff --git a/packages/host-runtime/test/native-voice-gateway.test.ts b/packages/host-runtime/test/native-voice-gateway.test.ts new file mode 100644 index 000000000..e141344f1 --- /dev/null +++ b/packages/host-runtime/test/native-voice-gateway.test.ts @@ -0,0 +1,169 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { NativeVoiceGateway } from "../src/native-voice-gateway.js"; +import { REALTIME_CALL_METHOD } from "@codexhost/shared-contracts"; +import { createFixture, requestId, stopFixture, writeRequest } from "./app-server-host-fixture.js"; + +const homes: string[] = []; +async function gateway() { + const home = await mkdtemp(path.join(tmpdir(), "native-voice-gateway-")); + homes.push(home); + await writeFile( + path.join(home, "config.toml"), + 'model_provider="custom"\nexperimental_realtime_webrtc_call_base_url="https://gateway.example/v1"\nexperimental_realtime_ws_base_url="https://gateway.example/v1"\n[model_providers.custom]\nenv_key="VOICE_KEY"\n', + ); + await writeFile( + path.join(home, "auth.json"), + JSON.stringify({ access_token: "desktop-oauth-must-not-be-used" }), + ); + return new NativeVoiceGateway({ CODEX_HOME: home, VOICE_KEY: "gateway-fixture" }); +} +const create = () => ({ + operation: "create", + requestId: "create-fixture", + body: JSON.stringify({ sdp: "offer", session: { model: "gpt-live-1.1-codex" } }), + headers: { + Authorization: "Bearer desktop-oauth", + "ChatGPT-Account-Id": "disabled-account", + "OpenAI-Alpha": "quicksilver=v2", + "Thread-Id": "thread-fixture", + }, +}); +afterEach(async () => { + vi.unstubAllGlobals(); + await Promise.all(homes.splice(0).map((home) => rm(home, { recursive: true, force: true }))); +}); +describe("native voice gateway", () => { + it("handles the Desktop RPC without forwarding it to official Codex", async () => { + const request = vi.fn().mockResolvedValue( + new Response("answer", { + status: 201, + headers: { location: "/v1/realtime/calls/rtc_fixture" }, + }), + ); + vi.stubGlobal("fetch", request); + await gateway(); + const home = homes.at(-1); + if (!home) throw new Error("Missing gateway config"); + const fixture = createFixture({ + environment: { CODEX_HOME: home, VOICE_KEY: "gateway-fixture" }, + }); + try { + writeRequest(fixture.desktopInput, { + id: "voice-create", + method: REALTIME_CALL_METHOD, + params: create(), + }); + const response = await fixture.collector.waitFor((message) => + requestId(message, "voice-create"), + ); + expect(response).toMatchObject({ result: { status: 201, body: "answer" } }); + expect(request).toHaveBeenCalledOnce(); + expect(fixture.official.stdin.readableLength).toBe(0); + } finally { + await stopFixture(fixture); + } + }); + it("hangs up a known created call if reading its answer fails", async () => { + const failedBody = new ReadableStream({ + start(controller) { + controller.error(new Error("transport secret")); + }, + }); + const request = vi + .fn() + .mockResolvedValueOnce( + new Response(failedBody, { + status: 201, + headers: { location: "/v1/realtime/calls/rtc_fixture" }, + }), + ) + .mockResolvedValueOnce(new Response(null, { status: 204 })); + vi.stubGlobal("fetch", request); + await expect((await gateway()).request(create())).rejects.toThrow( + "Configured voice gateway request failed", + ); + expect(request).toHaveBeenCalledTimes(2); + expect(request.mock.calls[1]?.[0]).toBe( + "https://gateway.example/v1/realtime/calls/rtc_fixture/hangup", + ); + }); + it("uses configured URLs and gateway auth, dropping Desktop account credentials", async () => { + const fetcher = vi.fn().mockResolvedValue( + new Response("answer", { + status: 201, + headers: { location: "/v1/realtime/calls/rtc_fixture", "set-cookie": "private" }, + }), + ); + vi.stubGlobal("fetch", fetcher); + const service = await gateway(); + expect(await service.request(create())).toEqual({ + status: 201, + body: "answer", + headers: { + "content-type": "text/plain;charset=UTF-8", + location: "/v1/realtime/calls/rtc_fixture", + }, + }); + const creation = fetcher.mock.calls[0]; + if (!creation) throw new Error("Missing voice call request"); + const [url, init] = creation; + expect(url).toBe("https://gateway.example/v1/realtime/calls"); + const headers = new Headers(init?.headers); + expect(headers.get("authorization")).toBe("Bearer gateway-fixture"); + expect(headers.has("chatgpt-account-id")).toBe(false); + expect(headers.get("thread-id")).toBe("thread-fixture"); + expect(init?.body).toBe(create().body); + expect(init?.redirect).toBe("error"); + }); + it("preserves quota status and retry-after for the native caller", async () => { + vi.stubGlobal( + "fetch", + vi + .fn() + .mockResolvedValue( + new Response("quota", { status: 429, headers: { "retry-after": "60" } }), + ), + ); + expect(await (await gateway()).request(create())).toMatchObject({ + status: 429, + body: "quota", + headers: { "retry-after": "60" }, + }); + }); + it("rejects arbitrary URL operations and malformed payloads before sending", async () => { + const fetcher = vi.fn(); + vi.stubGlobal("fetch", fetcher); + const service = await gateway(); + await expect( + service.request({ operation: "fetch", url: "https://other.example" }), + ).rejects.toThrow(); + await expect(service.request({ ...create(), body: "{}" })).rejects.toThrow("Invalid voice"); + expect(fetcher).not.toHaveBeenCalled(); + }); + it("cancels an in-flight bootstrap without leaking transport diagnostics", async () => { + let ready!: () => void; + const started = new Promise((resolve) => { + ready = resolve; + }); + vi.stubGlobal( + "fetch", + vi.fn( + (_url, options) => + new Promise((_resolve, reject) => { + ready(); + options.signal.addEventListener("abort", () => + reject(new Error("secret upstream detail")), + ); + }), + ), + ); + const service = await gateway(); + const result = service.request(create()); + await started; + await service.request({ operation: "cancel", requestId: "create-fixture" }); + await expect(result).rejects.toThrow("cancelled"); + }); +}); diff --git a/packages/host-runtime/test/runtime-maintenance.test.ts b/packages/host-runtime/test/runtime-maintenance.test.ts index 4ba7d75db..b31967964 100644 --- a/packages/host-runtime/test/runtime-maintenance.test.ts +++ b/packages/host-runtime/test/runtime-maintenance.test.ts @@ -19,7 +19,7 @@ afterEach(async () => { async function fixture(remote = false, updaterStartTimeoutMs?: number) { const root = await mkdtemp(path.join(tmpdir(), "codexhost-maintenance-")); directories.push(root); - const pkg = path.join(root, "node_modules/@codexhost/cli-darwin-arm64"); + const pkg = path.join(root, "node_modules/@karlorz/codexhost-cli-darwin-arm64"); const runtimePath = path.join(pkg, "app/host-runtime.mjs"); await Promise.all( ["app", "libexec", "../cli/bin"].map((dir) => mkdir(path.join(pkg, dir), { recursive: true })), diff --git a/packages/host-runtime/test/update-coordinator.test.ts b/packages/host-runtime/test/update-coordinator.test.ts index ca364e627..a26c6b904 100644 --- a/packages/host-runtime/test/update-coordinator.test.ts +++ b/packages/host-runtime/test/update-coordinator.test.ts @@ -137,7 +137,7 @@ function release(version = "1.2.3"): CodexhostLatestRelease { return { version, releaseNotes: `Release ${version}`, - releaseNotesUrl: `https://github.com/BytePioneer-AI/codex-host/releases/tag/v${version}`, + releaseNotesUrl: `https://github.com/karlorz/codex-host/releases/tag/v${version}`, assets: [], }; } @@ -268,14 +268,14 @@ describe("Host update coordinator", () => { fetchLatest: async () => ({ version: "1.2.3", releaseNotes: "Release 1.2.3", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", assets: [ { name: "codexhost-1.2.3-macos-arm64.dmg", size: bytes.length, digest: `sha256:${digest(bytes)}`, downloadUrl: - "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-macos-arm64.dmg", + "https://github.com/karlorz/codex-host/releases/download/v1.2.3/codexhost-1.2.3-macos-arm64.dmg", }, ], }), diff --git a/packages/mapping-store/src/mapping-store.ts b/packages/mapping-store/src/mapping-store.ts index 70bbc3696..974e3eb07 100644 --- a/packages/mapping-store/src/mapping-store.ts +++ b/packages/mapping-store/src/mapping-store.ts @@ -167,8 +167,8 @@ function threadMetadataOf(record: StoredThreadRecordV1): StoredThreadMetadataV1 /** The record as written to the Thread file, which older releases must still accept. */ function withoutThreadMetadata(record: StoredThreadRecordV1): StoredThreadRecordV1 { - const { projectId, daybreakEnabled, gitInfo, ...core } = record; - void [projectId, daybreakEnabled, gitInfo]; + const { projectId, daybreakEnabled, gitInfo, subagentConfiguration, ...core } = record; + void [projectId, daybreakEnabled, gitInfo, subagentConfiguration]; return core; } @@ -188,6 +188,14 @@ function applyThreadMetadataPatch( patch: ThreadMetadataPatch, ): StoredThreadRecordV1 { const next = { ...current }; + if (patch.subagentConfiguration !== undefined) { + if (!current.subagent) throw new Error("Child configuration requires a Subagent Thread"); + next.subagentConfiguration = { ...current.subagentConfiguration }; + for (const name of ["model", "reasoningEffort"] as const) { + const value = patch.subagentConfiguration[name]; + if (value !== undefined) next.subagentConfiguration[name] = value; + } + } if (patch.projectId !== undefined) { if (patch.projectId === null) delete next.projectId; else next.projectId = patch.projectId; @@ -1069,6 +1077,7 @@ export class MappingStore { if (!hostMatch.nativeCheckpointRef && update.nativeCheckpointRef) { hostMatch.nativeCheckpointRef = update.nativeCheckpointRef; } + if (update.realtimeInline) hostMatch.realtimeInline = true; continue; } const added = { ...update }; diff --git a/packages/mapping-store/src/records.ts b/packages/mapping-store/src/records.ts index aad18b0c9..0a2e91968 100644 --- a/packages/mapping-store/src/records.ts +++ b/packages/mapping-store/src/records.ts @@ -28,6 +28,8 @@ export const storedTurnMappingV1Schema = z hostTurnId: hostTurnIdSchema, nativeTurnRef: nativeTurnRefSchema, nativeCheckpointRef: nativeCheckpointRefSchema.optional(), + /** Desktop presentation for work completed during an external voice call. */ + realtimeInline: z.literal(true).optional(), }) .strict(); @@ -80,12 +82,24 @@ const storedThreadCoreV1Shape = { * Desktop organization metadata, stored separately from the Thread's identity and mappings. */ const storedThreadMetadataFieldsShape = { + subagentConfiguration: z + .object({ + model: nonBlankTextSchema.max(1_024).optional(), + reasoningEffort: nonBlankTextSchema.max(1_024).optional(), + }) + .strict() + .optional(), projectId: nonBlankTextSchema.max(1_024).optional(), daybreakEnabled: z.boolean().optional(), gitInfo: storedGitInfoV1Schema.optional(), }; -export const THREAD_METADATA_FIELDS = ["projectId", "daybreakEnabled", "gitInfo"] as const; +export const THREAD_METADATA_FIELDS = [ + "projectId", + "daybreakEnabled", + "gitInfo", + "subagentConfiguration", +] as const; export const storedThreadMetadataV1Schema = z .object({ @@ -234,6 +248,8 @@ export interface FindRecentDelegationInput { * unchanged; `null` clears the stored value. */ export interface ThreadMetadataPatch { + /** Observed child configuration; omitted fields preserve previous observations. */ + subagentConfiguration?: { model?: string; reasoningEffort?: string }; projectId?: string | null; daybreakEnabled?: boolean; gitInfo?: { diff --git a/packages/protocol-core/src/codex-ui-projector.ts b/packages/protocol-core/src/codex-ui-projector.ts index c2925005c..3a03da02d 100644 --- a/packages/protocol-core/src/codex-ui-projector.ts +++ b/packages/protocol-core/src/codex-ui-projector.ts @@ -450,21 +450,29 @@ function toolContentItems(item: Extract): J ); } -function collabAgentStatus( - status: Extract["subagents"][number]["status"], -): string { - switch (status) { - case "pending": - return "pendingInit"; - case "running": - return "running"; - case "completed": - return "completed"; - case "failed": - return "errored"; - case "interrupted": - return "interrupted"; - } +/** Read-only native activities never advertise a writable spawnAgent receiver. */ +function projectSubagentActivities( + item: Extract, +): JsonObject[] { + return item.subagents.flatMap((child) => { + const activity: JsonObject = { + id: `${item.itemId}-subagent-${encodeURIComponent(child.subagentId)}`, + type: "subAgentActivity", + agentThreadId: child.subagentId, + agentPath: child.description || child.role || child.subagentId, + kind: item.operation === "spawn" ? "started" : "interacted", + }; + // Keep the spawn activity for Desktop's descendant discovery after history reload. + if (child.status === "pending" || child.status === "running") return [activity]; + return [ + activity, + { + ...activity, + id: `${activity.id}-status`, + kind: child.status === "completed" ? "completed" : "interrupted", + }, + ]; + }); } /** `clientId` lets Desktop settle its optimistic steering message for this input. */ @@ -485,7 +493,6 @@ function projectItem( outcome: HostItemOutcome | null, defaultCwd: string, includeCommandOutput = true, - senderThreadId?: string, ): JsonObject { switch (item.type) { case "userMessage": @@ -560,33 +567,20 @@ function projectItem( changes: projectFileChanges(item.changes), status: itemStatus(outcome), }; - case "subagentDelegation": { - const primary = item.subagents[0]; - // Codex exposes one configuration per Item, not per receiver. Do not - // attribute the first child's configuration to a heterogeneous group. - const sameConfiguration = item.subagents.every( - (subagent) => - subagent.model === primary?.model && - subagent.reasoningEffort === primary?.reasoningEffort, - ); + case "generatedImage": return { id: item.itemId, - type: "collabAgentToolCall", - tool: item.operation === "spawn" ? "spawnAgent" : "sendInput", + type: "imageGeneration", status: itemStatus(outcome), - senderThreadId: senderThreadId ?? "", - receiverThreadIds: item.subagents.map(({ subagentId }) => subagentId), - prompt: item.prompt ?? null, - model: sameConfiguration ? (primary?.model ?? null) : null, - reasoningEffort: sameConfiguration ? (primary?.reasoningEffort ?? null) : null, - agentsStates: Object.fromEntries( - item.subagents.map(({ subagentId, status, resultSummary }) => [ - subagentId, - { status: collabAgentStatus(status), message: resultSummary ?? null }, - ]), - ), + savedPath: item.savedPath, + result: "", + revisedPrompt: item.revisedPrompt ?? null, + inputImages: [], + failure: null, + transparentBackground: null, }; - } + case "subagentDelegation": + throw new Error("Subagent delegation requires per-child activity projection"); } } @@ -683,13 +677,11 @@ export function projectHistoricalTurn(input: HistoricalTurnProjectionInput): Jso if (isTodoTool(item.toolName) || todoPlanFromTool(item.toolName, item.arguments)) return []; if (isFileMutatingTool(item.toolName)) return []; } + if (item.type === "subagentDelegation") return projectSubagentActivities(item); if (item.type === "reasoning" && !reasoningDisplayText(item.text)) return []; return item.type === "reasoning" - ? [ - projectItem(item, outcome, cwd, true, input.threadId ?? ""), - projectReasoningTranscriptItem(item, outcome, cwd), - ] - : [projectItem(item, outcome, cwd, true, input.threadId ?? "")]; + ? [projectItem(item, outcome, cwd), projectReasoningTranscriptItem(item, outcome, cwd)] + : [projectItem(item, outcome, cwd)]; }; const entries = snapshot.items.map((entry) => ({ entry, projected: projectEntry(entry) })); const lastVisible = entries.findLast(({ projected }) => projected.length > 0)?.entry; @@ -801,7 +793,7 @@ export class CodexTurnProjector { if (!this.#detached.has(itemId)) continue; const projected = this.#items.get(itemId); if (!projected || projected.outcome) continue; - items.set(itemId, projectItem(projected.item, null, this.#cwd, true, this.#threadId)); + items.set(itemId, projectItem(projected.item, null, this.#cwd)); } return items; } @@ -1039,6 +1031,18 @@ export class CodexTurnProjector { if (isFileMutatingTool(event.item.toolName)) return { messages: [] }; } if (event.item.type === "fileChange") return { messages: this.#fileChangeUpdates(startedAtMs) }; + if (event.item.type === "subagentDelegation") { + projected.wireStarted = true; + this.#wireItemOrder.push(event.item.itemId); + return { + messages: this.#subagentActivityMessages( + event.item, + "item/started", + startedAtMs, + startedAtMs, + ), + }; + } const startedItem = event.item.type === "reasoning" ? { ...event.item, text: "" } : event.item; const messages = [this.#startWireItem(projected, startedItem, startedAtMs)]; if (event.item.type === "reasoning") { @@ -1127,16 +1131,15 @@ export class CodexTurnProjector { } else if (event.update.type === "fileChanges.replace") { messages.push(...this.#fileChangeUpdates(emittedAtMs)); } else if (event.update.type === "subagents.replace") { - messages.push({ - method: "item/started", - emittedAtMs, - params: { - threadId: this.#threadId, - turnId: this.#turnId, - startedAtMs: projected.startedAtMs ?? emittedAtMs, - item: projectItem(next, null, this.#cwd, true, this.#threadId), - }, - }); + if (next.type !== "subagentDelegation") throw new Error("Invalid Subagent update"); + messages.push( + ...this.#subagentActivityMessages( + next, + "item/started", + emittedAtMs, + projected.startedAtMs ?? emittedAtMs, + ), + ); } return { messages }; } @@ -1194,6 +1197,16 @@ export class CodexTurnProjector { } return { messages: [] }; } + if (projected.item.type === "subagentDelegation") { + return { + messages: this.#subagentActivityMessages( + projected.item, + "item/completed", + emittedAtMs, + startedAtMs, + ), + }; + } const messages = [ completedItem( this.#projectItem(projected.item, projected.outcome, !projected.streamedCommandOutput), @@ -1360,6 +1373,25 @@ export class CodexTurnProjector { ]; } + #subagentActivityMessages( + item: Extract, + method: "item/started" | "item/completed", + emittedAtMs: number, + startedAtMs: number, + ): JsonObject[] { + return projectSubagentActivities(item).map((activity) => ({ + method, + emittedAtMs, + params: { + threadId: this.#threadId, + turnId: this.#turnId, + startedAtMs, + ...(method === "item/completed" ? { completedAtMs: emittedAtMs } : {}), + item: activity, + }, + })); + } + #startWireItem(projected: ProjectedItem, item: HostItem, startedAtMs: number): JsonObject { if (projected.wireStarted) throw new Error("Codex Item started more than once"); projected.wireStarted = true; @@ -1384,7 +1416,7 @@ export class CodexTurnProjector { ): JsonObject { return item.type === "userMessage" ? projectUserMessage(item, this.#userMessageClientIds.get(item.itemId) ?? null) - : projectItem(item, outcome, this.#cwd, includeCommandOutput, this.#threadId); + : projectItem(item, outcome, this.#cwd, includeCommandOutput); } #startReasoningTranscript( diff --git a/packages/protocol-core/test/codex-ui-projector.test.ts b/packages/protocol-core/test/codex-ui-projector.test.ts index 904469463..54b6fab10 100644 --- a/packages/protocol-core/test/codex-ui-projector.test.ts +++ b/packages/protocol-core/test/codex-ui-projector.test.ts @@ -15,7 +15,7 @@ import { nativeTurnRefSchema, } from "@codexhost/shared-contracts"; -import { CodexTurnProjector, projectHistoricalTurn } from "../src/index.js"; +import { CodexTurnProjector, projectHistoricalTurn, type JsonObject } from "../src/index.js"; const turnId = hostTurnIdSchema.parse("turn-1"); const itemId = (value: string) => hostItemIdSchema.parse(value); @@ -310,10 +310,14 @@ describe("Codex UI projector", () => { status: "completed", }), expect.objectContaining({ - id: "historical-subagent", - type: "collabAgentToolCall", - status: "completed", - receiverThreadIds: ["historical-agent-1"], + type: "subAgentActivity", + kind: "started", + agentThreadId: "historical-agent-1", + }), + expect.objectContaining({ + type: "subAgentActivity", + kind: "completed", + agentThreadId: "historical-agent-1", }), ], error: null, @@ -614,193 +618,111 @@ describe("Codex UI projector", () => { ]); }); - it("projects Subagent delegation through native collaboration Items", () => { + it("keeps live and restored native children read-only, including multiple receivers", () => { const value = projector(); const delegationId = itemId("delegation-1"); - const startedItem: HostSubagentDelegationItem = { + const item: HostSubagentDelegationItem = { type: "subagentDelegation", itemId: delegationId, operation: "spawn", subagents: [ { - subagentId: "claude-agent-1", - description: "Inspect implementation", - role: "Explore", - model: "grok-4.6", - reasoningEffort: "high", + subagentId: "child-a", + description: "Inspect", + model: "model-a", background: true, + status: "running", + }, + { + subagentId: "child-b", + description: "Review", + model: "model-b", + background: false, status: "pending", }, ], }; value.project({ type: "turn.started", turnId }); - - expect( - value.project({ type: "item.started", turnId, item: startedItem }).messages, - ).toMatchObject([ + const started = value.project({ type: "item.started", turnId, item }).messages; + expect(started).toMatchObject([ { method: "item/started", - params: { - item: { - id: "delegation-1", - type: "collabAgentToolCall", - tool: "spawnAgent", - status: "inProgress", - senderThreadId: "thread-1", - receiverThreadIds: ["claude-agent-1"], - model: "grok-4.6", - reasoningEffort: "high", - agentsStates: { - "claude-agent-1": { status: "pendingInit", message: null }, - }, - }, - }, + params: { item: { type: "subAgentActivity", agentThreadId: "child-a", kind: "started" } }, }, - ]); - const startedSubagent = startedItem.subagents[0]; - if (!startedSubagent) throw new Error("Test delegation has no Subagent"); - const runningSubagent = { ...startedSubagent, status: "running" as const }; - expect( - value.project({ - type: "item.updated", - turnId, - itemId: delegationId, - update: { type: "subagents.replace", subagents: [runningSubagent] }, - }).messages, - ).toMatchObject([ { method: "item/started", - params: { - item: { - type: "collabAgentToolCall", - agentsStates: { - "claude-agent-1": { status: "running", message: null }, - }, - }, - }, + params: { item: { type: "subAgentActivity", agentThreadId: "child-b", kind: "started" } }, }, ]); - const completedItem: HostSubagentDelegationItem = { - ...startedItem, - subagents: [ - { - ...startedSubagent, - status: "completed", - resultSummary: "Inspection complete", - }, - ], + const completed: HostSubagentDelegationItem = { + ...item, + subagents: item.subagents.map((child) => ({ ...child, status: "completed" })), }; - expect( - value.project({ - type: "item.updated", - turnId, - itemId: delegationId, - update: { type: "subagents.replace", subagents: completedItem.subagents }, - }).messages, - ).toMatchObject([ - { - method: "item/started", - params: { - item: { - type: "collabAgentToolCall", - agentsStates: { - "claude-agent-1": { status: "completed", message: "Inspection complete" }, - }, - }, - }, - }, - ]); - expect( - value.project({ - type: "item.completed", - turnId, - snapshot: { item: completedItem, outcome: { status: "succeeded" } }, - }).messages, - ).toMatchObject([ - { - method: "item/completed", - params: { - item: { - type: "collabAgentToolCall", - status: "completed", - agentsStates: { - "claude-agent-1": { status: "completed", message: "Inspection complete" }, - }, - }, - }, + const updated = value.project({ + type: "item.updated", + turnId, + itemId: delegationId, + update: { type: "subagents.replace", subagents: completed.subagents }, + }).messages; + const finished = value.project({ + type: "item.completed", + turnId, + snapshot: { item: completed, outcome: { status: "succeeded" } }, + }).messages; + const wireItems = (messages: typeof started) => + messages.map((message) => (message.params as JsonObject).item); + expect(wireItems(updated)).toEqual(wireItems(finished)); + const historical = projectHistoricalTurn({ + turnId, + cwd: "/workspace", + snapshot: { + nativeTurnRef: nativeTurnRefSchema.parse({ + harnessId: "grok", + nativeSessionId: "parent", + nativeTurnKey: "turn-1", + formatVersion: 1, + }), + input: [], + items: [{ item: completed, outcome: { status: "succeeded" } }], + outcome: { status: "succeeded" }, }, + }); + const activities = (historical.items as JsonObject[]).filter( + (entry) => entry.type === "subAgentActivity", + ); + expect(activities).toEqual(wireItems(finished)); + expect(activities.map((entry) => [entry.agentThreadId, entry.kind])).toEqual([ + ["child-a", "started"], + ["child-a", "completed"], + ["child-b", "started"], + ["child-b", "completed"], ]); + expect(new Set(activities.map((entry) => entry.id)).size).toBe(4); + // Desktop infers writable child chats from collabAgentToolCall/spawnAgent. + expect([...wireItems(started), ...activities]).not.toContainEqual( + expect.objectContaining({ type: "collabAgentToolCall" }), + ); }); - it.each([ - { configurations: [{}], expected: { model: null, reasoningEffort: null } }, - { configurations: [], expected: { model: null, reasoningEffort: null } }, - { - configurations: [{ model: "provider/raw-model-id" }], - expected: { model: "provider/raw-model-id", reasoningEffort: null }, - }, - { - configurations: [{ reasoningEffort: "xhigh" }], - expected: { model: null, reasoningEffort: "xhigh" }, - }, - { - configurations: [ - { model: "grok-4.6", reasoningEffort: "high" }, - { model: "grok-4.6", reasoningEffort: "high" }, - ], - expected: { model: "grok-4.6", reasoningEffort: "high" }, - }, - { - configurations: [{ model: "grok-4.6" }, {}], - expected: { model: null, reasoningEffort: null }, - }, - { - configurations: [{ model: "model-a" }, { model: "model-b" }], - expected: { model: null, reasoningEffort: null }, - }, - { - configurations: [ - { model: "grok-4.6", reasoningEffort: "high" }, - { model: "grok-4.6", reasoningEffort: "low" }, - ], - expected: { model: null, reasoningEffort: null }, - }, - ])( - "preserves native Subagent configuration without formatting or group guesses: %j", - ({ configurations, expected }) => { + it.each(["failed", "interrupted"] as const)( + "projects %s children as terminal read-only activities", + (status) => { const value = projector(); - const item: HostSubagentDelegationItem = { - type: "subagentDelegation", - itemId: itemId("child-config"), - operation: "spawn", - subagents: configurations.map((configuration, index) => ({ - subagentId: `child-${index}`, - description: "Inspect", - background: true, - status: "completed", - ...configuration, - })), - }; value.project({ type: "turn.started", turnId }); - expect(value.project({ type: "item.started", turnId, item }).messages).toMatchObject([ - { method: "item/started", params: { item: expected } }, - ]); - const historical = projectHistoricalTurn({ + const result = value.project({ + type: "item.started", turnId, - cwd: "/workspace", - snapshot: { - nativeTurnRef: nativeTurnRefSchema.parse({ - harnessId: "grok", - nativeSessionId: "parent", - nativeTurnKey: "turn-1", - formatVersion: 1, - }), - input: [{ type: "text", text: "delegate" }], - items: [{ item, outcome: { status: "succeeded" } }], - outcome: { status: "succeeded" }, + item: { + type: "subagentDelegation", + itemId: itemId("send"), + operation: "send", + subagents: [{ subagentId: "child", description: "Child", background: false, status }], }, }); - expect(historical.items).toMatchObject([{ type: "userMessage" }, expected]); + expect(result.messages).toMatchObject([ + { params: { item: { kind: "interacted", agentThreadId: "child" } } }, + { params: { item: { kind: "interrupted", agentThreadId: "child" } } }, + ]); }, ); @@ -1747,6 +1669,68 @@ describe("Codex UI projector", () => { ).toMatchObject({ status: "completed", items: [] }); }); + it("projects generatedImage as Desktop imageGeneration with completedAtMs", () => { + const value = projector(); + const imageId = itemId("image-1"); + const savedPath = "/Users/karlchow/.grok/sessions/session-1/images/1.jpg"; + value.project({ type: "turn.started", turnId }); + expect( + value.project({ + type: "item.started", + turnId, + item: { + type: "generatedImage", + itemId: imageId, + savedPath, + revisedPrompt: "1.jpg", + }, + }).messages, + ).toMatchObject([ + { + method: "item/started", + params: { + item: { + id: "image-1", + type: "imageGeneration", + status: "inProgress", + savedPath, + result: "", + revisedPrompt: "1.jpg", + }, + }, + }, + ]); + expect( + value.project({ + type: "item.completed", + turnId, + snapshot: { + item: { + type: "generatedImage", + itemId: imageId, + savedPath, + revisedPrompt: "1.jpg", + }, + outcome: { status: "succeeded" }, + }, + }).messages, + ).toMatchObject([ + { + method: "item/completed", + params: { + completedAtMs: expect.any(Number), + item: { + id: "image-1", + type: "imageGeneration", + status: "completed", + savedPath, + result: "", + }, + }, + }, + ]); + }); + it("projects failed Turns through the complete Codex error notification shape", () => { const value = projector(); const message = '503: {"message":"Service temporarily unavailable","type":"api_error"}'; diff --git a/packages/renderer-extension/scripts/build.mjs b/packages/renderer-extension/scripts/build.mjs index fd00c89d8..d1f3bf35a 100644 --- a/packages/renderer-extension/scripts/build.mjs +++ b/packages/renderer-extension/scripts/build.mjs @@ -7,6 +7,7 @@ const packageRoot = path.resolve(import.meta.dirname, ".."); const entries = [ { entry: "src/index.ts", format: "esm", outfile: "dist/index.js" }, { entry: "src/production-entry.ts", format: "iife", outfile: "dist/production.js" }, + { entry: "src/cpa-live-voice-entry.ts", format: "iife", outfile: "dist/cpa-live-voice-hook.js" }, { entry: "src/probe-entry.ts", format: "iife", outfile: "dist/renderer-binding-probe.js" }, { entry: "src/audit-entry.ts", format: "iife", outfile: "dist/contract-audit.js" }, { entry: "src/console-entry.ts", format: "iife", outfile: "dist/console.js" }, diff --git a/packages/renderer-extension/src/codex-locale-adapter.ts b/packages/renderer-extension/src/codex-locale-adapter.ts index d715da700..11eaa385f 100644 --- a/packages/renderer-extension/src/codex-locale-adapter.ts +++ b/packages/renderer-extension/src/codex-locale-adapter.ts @@ -60,7 +60,8 @@ type FixedLocaleRequest = export type CodexLocaleOverride = "en-US" | "zh-CN" | null; export type CodexLocaleReadStatus = "ready" | "fallback"; export type CodexLocaleMode = "explicit" | "automatic" | "unavailable"; -export type CodexLocaleSource = "override" | "ide" | "system" | "navigator" | "default"; +export type CodexLocaleSource = + "override" | "ide" | "system" | "document" | "navigator" | "default"; export interface CodexLocaleSettings { readonly status: CodexLocaleReadStatus; @@ -333,13 +334,14 @@ export async function readCodexLocaleSettings( }); } + const documentLocale = canonicalLocale(ownerWindow.document?.documentElement?.lang); return Object.freeze({ status: "fallback", mode: "unavailable", localeOverride: undefined, ideLocale: localeInfo.ideLocale, systemLocale: localeInfo.systemLocale, - preferredLocale: navigatorLocale ?? "en-US", - source: navigatorLocale ? "navigator" : "default", + preferredLocale: documentLocale ?? navigatorLocale ?? "en-US", + source: documentLocale ? "document" : navigatorLocale ? "navigator" : "default", }); } diff --git a/packages/renderer-extension/src/console/app.ts b/packages/renderer-extension/src/console/app.ts index c2afcc274..da7410b56 100644 --- a/packages/renderer-extension/src/console/app.ts +++ b/packages/renderer-extension/src/console/app.ts @@ -1,4 +1,4 @@ -import { hostRequestManager, consoleUpdateClient } from "./api.js"; +import { hostRequestManager, consoleUpdateClient, consolePost } from "./api.js"; import { mountConsoleAnnouncement } from "./announcement.js"; import { startAgentGroupSync } from "../agent-group-sync.js"; import { getSharedAgentGroupPreferenceStore } from "../agent-group-preference.js"; @@ -10,6 +10,7 @@ import { h } from "./dom.js"; import { consoleMessages, type ConsoleMessages } from "./messages.js"; import { createOfflineHarnessesPage } from "./pages/harnesses-offline.js"; import { hostPage } from "./pages/host-required.js"; +import { createVoiceSettingsPage } from "../settings/voice-page.js"; import { createOverviewPage } from "./pages/overview.js"; import { ConsoleState } from "./state.js"; import { createConsoleRemoteConnections } from "./remote-connections.js"; @@ -144,6 +145,8 @@ export function startConsoleApp(document: Document): void { (method, params) => manager.sendRequest(method, params), ); const navigate = (pageId: string): void => { + navigation.classList.remove("is-open"); + menu.setAttribute("aria-expanded", "false"); window.location.hash = pageId; }; const sections: NavigationSection[] = [ @@ -168,6 +171,11 @@ export function startConsoleApp(document: Document): void { hostPage(required(settingsPages, "accounts"), messages, state), hostPage(required(settingsPages, "session-import"), messages, state), hostPage(required(settingsPages, "usage-statistics"), messages, state), + createVoiceSettingsPage( + locale, + (input) => consolePost("/api/voice-settings", input, AbortSignal.timeout(10_000)), + { id: "live-voice" }, + ), required(settingsPages, "updates"), ], }, @@ -249,6 +257,32 @@ export function startConsoleApp(document: Document): void { createRendererSettingsIcon("alert", 16), h(document, "span", {}, `${messages.consoleOffline} · ${messages.consoleOfflineBody}`), ); + const menu = h( + document, + "button", + { + type: "button", + className: "console-menu-button settings-command-button settings-command-button--secondary", + "aria-label": settingsMessages.sectionsLabel, + "aria-expanded": "false", + "aria-controls": "console-navigation", + }, + "☰", + ); + navigation.id = "console-navigation"; + menu.addEventListener("click", () => { + const open = menu.getAttribute("aria-expanded") !== "true"; + menu.setAttribute("aria-expanded", String(open)); + navigation.classList.toggle("is-open", open); + }); + navigation.addEventListener("keydown", (event) => { + if (event.key === "Escape") { + navigation.classList.remove("is-open"); + menu.setAttribute("aria-expanded", "false"); + menu.focus(); + } + }); + brand.append(menu); shadow.append( h( document, @@ -285,6 +319,8 @@ export function startConsoleApp(document: Document): void { const pageId = pages.has(requested) ? requested : "overview"; if (pageId === activeId) return; activeId = pageId; + navigation.classList.remove("is-open"); + menu.setAttribute("aria-expanded", "false"); scope?.dispose(); try { cleanup?.(); diff --git a/packages/renderer-extension/src/console/components.css b/packages/renderer-extension/src/console/components.css index 567b3af9d..e5b0468f5 100644 --- a/packages/renderer-extension/src/console/components.css +++ b/packages/renderer-extension/src/console/components.css @@ -34,6 +34,17 @@ gap: 8px; } +.console-restart { + width: 100%; + min-width: 0; +} + +.console-restart > p { + flex-basis: 100%; + margin: 0; + overflow-wrap: anywhere; +} + .console-muted { margin: 0; color: var(--settings-muted); diff --git a/packages/renderer-extension/src/console/connection-diagnostics.ts b/packages/renderer-extension/src/console/connection-diagnostics.ts index 910ef1d50..3b7437a52 100644 --- a/packages/renderer-extension/src/console/connection-diagnostics.ts +++ b/packages/renderer-extension/src/console/connection-diagnostics.ts @@ -134,6 +134,14 @@ export function createConsoleConnectionDiagnostics( if (!client.setHarnessLaunchSettings) throw new Error("Launch settings are unavailable"); return client.setHarnessLaunchSettings({ harnessId: harnessIdSchema.parse(agent), path }); }, + async getGrokComputerUse() { + if (!client.getGrokComputerUse) throw new Error("Grok Computer Use settings are unavailable"); + return client.getGrokComputerUse(); + }, + async setGrokComputerUse(_hostId, update) { + if (!client.setGrokComputerUse) throw new Error("Grok Computer Use settings are unavailable"); + return client.setGrokComputerUse(update); + }, subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); diff --git a/packages/renderer-extension/src/console/console.css b/packages/renderer-extension/src/console/console.css index 1c150cff0..16f30221e 100644 --- a/packages/renderer-extension/src/console/console.css +++ b/packages/renderer-extension/src/console/console.css @@ -178,7 +178,7 @@ .console-hero { display: grid; - grid-template-columns: auto minmax(0, 1fr) auto; + grid-template-columns: auto minmax(0, 1fr); align-items: center; gap: 16px; padding: 20px; @@ -187,6 +187,10 @@ border-radius: 14px; } +.console-hero > .console-actions { + grid-column: 1 / -1; +} + .console-hero__icon { display: grid; width: 44px; @@ -337,6 +341,11 @@ monospace; } +.console-menu-button { + display: none; + margin-left: auto; +} + /* Empty states */ .console-empty { @@ -435,8 +444,26 @@ } .console-sidebar .settings-nav { + display: none; + flex-direction: column; + overflow-x: hidden; + max-height: 55vh; + overflow-y: auto; + } + + .console-menu-button { + display: inline-flex; + } + .console-brand { + padding: 12px 16px; + } + .console-brand__copy { flex-direction: row; - overflow-x: auto; + gap: 5px; + align-items: baseline; + } + .console-sidebar .settings-nav.is-open { + display: flex; } .console-sidebar .settings-nav-section-label, diff --git a/packages/renderer-extension/src/console/messages.ts b/packages/renderer-extension/src/console/messages.ts index 00407ba4b..0c3605fe1 100644 --- a/packages/renderer-extension/src/console/messages.ts +++ b/packages/renderer-extension/src/console/messages.ts @@ -15,6 +15,13 @@ const ENGLISH = { stopped: "codexhost is not running", stoppedDetail: "Start codexhost to use Codex Desktop with your Harnesses.", start: "Start codexhost", + restart: "Restart codexhost", + restartConfirm: "Restart now", + restartCancel: "Cancel", + restartWarning: + "Restart closes Desktop and may interrupt a voice call or running tasks. Finish or stop them before continuing. Saved chats remain available; Desktop controls which chat reopens.", + restarting: "Restarting codexhost… Waiting for the new Desktop controller.", + restarted: "Restart complete. The new Desktop controller is ready.", starting: "Starting…", startingTitle: "codexhost is starting", startingDetail: "Preparing Codex Desktop. Please wait; there is no need to start it again.", @@ -71,6 +78,16 @@ const ENGLISH = { save: "Save", useAutomatic: "Use automatic", saved: "Saved", + grokComputerUse: "Grok Computer Use", + grokComputerUseHelp: + "Lets Grok control Mac apps through ChatGPT’s Computer Use helper. ChatGPT Settings → Computer use still applies only to official Codex. New Grok threads pick up this switch.", + grokComputerUseUnavailable: "ChatGPT Desktop Computer Use files were not found.", + grokBrowser: "Grok in-app browser", + grokBrowserHelp: + "Lets Grok use ChatGPT’s built-in browser (Browser Use) through cua_repl. Independent of Computer Use; official Codex Browser Use is unchanged. Grok fails closed when the in-app tab cannot be bound.", + grokBrowserUnavailable: "ChatGPT Desktop in-app browser files were not found.", + grokBrowserBlocked: + "Saved as intent. ChatGPT Desktop does not authorize Grok to use the in-app browser.", installGuide: "Install guide", noHarness: "No Harness plugins are installed.", consoleOffline: "The console is not running", @@ -211,6 +228,13 @@ const CHINESE: ConsoleMessages = { stopped: "codexhost 未运行", stoppedDetail: "启动 codexhost 后即可在 Codex Desktop 中使用各个 Harness。", start: "启动 codexhost", + restart: "重新启动 codexhost", + restartConfirm: "立即重新启动", + restartCancel: "取消", + restartWarning: + "重新启动会关闭 Desktop,可能中断语音通话或运行中的任务。请先完成或停止它们。已保存聊天仍可使用;重新打开哪个聊天由 Desktop 决定。", + restarting: "正在重新启动 codexhost… 等待新的 Desktop 控制器。", + restarted: "重新启动完成。新的 Desktop 控制器已就绪。", starting: "正在启动…", startingTitle: "codexhost 正在启动", startingDetail: "正在准备 Codex Desktop,请稍候,无需重复启动。", @@ -266,6 +290,15 @@ const CHINESE: ConsoleMessages = { save: "保存", useAutomatic: "恢复自动查找", saved: "已保存", + grokComputerUse: "Grok Computer Use", + grokComputerUseHelp: + "让 Grok 通过 ChatGPT 的 Computer Use 助手控制 Mac 应用。ChatGPT 设置 → Computer use 仍只作用于官方 Codex。新的 Grok 线程会读取此开关。", + grokComputerUseUnavailable: "未找到 ChatGPT Desktop 的 Computer Use 文件。", + grokBrowser: "Grok 内置浏览器", + grokBrowserHelp: + "让 Grok 通过 cua_repl 使用 ChatGPT 内置浏览器(Browser Use)。与 Computer Use 相互独立,官方 Codex Browser Use 不受影响。无法绑定内置标签页时 Grok 会直接失败。", + grokBrowserUnavailable: "未找到 ChatGPT Desktop 的内置浏览器文件。", + grokBrowserBlocked: "已保存为意向。ChatGPT Desktop 尚未授权 Grok 使用内置浏览器。", installGuide: "安装说明", noHarness: "没有已安装的 Harness 插件。", consoleOffline: "控制台未运行", diff --git a/packages/renderer-extension/src/console/pages/diagnostics.ts b/packages/renderer-extension/src/console/pages/diagnostics.ts index c9f7d1245..3e5af8190 100644 --- a/packages/renderer-extension/src/console/pages/diagnostics.ts +++ b/packages/renderer-extension/src/console/pages/diagnostics.ts @@ -18,6 +18,7 @@ const STAGE_STEPS: Readonly> = { "Codex Desktop installation discovered": "findDesktop", "acquiring Launcher ownership": "ownership", "Launcher ownership acquired": "ownership", + "recovering leftover Desktop Controller": "ownership", "attached to existing controlled Desktop": "ownership", "launching Codex Desktop": "launchDesktop", "Codex Desktop launched": "launchDesktop", diff --git a/packages/renderer-extension/src/console/pages/harnesses-offline.ts b/packages/renderer-extension/src/console/pages/harnesses-offline.ts index 2e3e083ea..9a5c287ab 100644 --- a/packages/renderer-extension/src/console/pages/harnesses-offline.ts +++ b/packages/renderer-extension/src/console/pages/harnesses-offline.ts @@ -1,3 +1,4 @@ +import type { GrokComputerUseSettings } from "@codexhost/shared-contracts"; import { consoleGet, consolePost } from "../api.js"; import { button, h } from "../dom.js"; import type { ConsoleMessages } from "../messages.js"; @@ -16,6 +17,7 @@ interface ConsoleHarness { links: { documentation?: string; installation?: string } | null; icon: string | null; launchPath: string | null; + computerUse?: GrokComputerUseSettings; } /** @@ -35,6 +37,7 @@ export function createOfflineHarnessesPage( const document = context.content.ownerDocument; const list = h(document, "div", { className: "console-harness-list" }); const status = new Map(); + const busy = new Set(); let harnesses: ConsoleHarness[] = []; const render = (): void => { if (harnesses.length === 0) { @@ -118,6 +121,85 @@ export function createOfflineHarnessesPage( ); } } + if (harness.computerUse) { + const computerUse = harness.computerUse; + const pending = busy.has(harness.id); + const post = (update: { enabled: boolean } | { browser: boolean }) => { + busy.add(harness.id); + toggle.disabled = true; + browserToggle.disabled = true; + void consolePost<{ harness: ConsoleHarness }>( + "/api/harnesses/grok-computer-use", + update, + ) + .then(({ harness: saved }) => { + harnesses = harnesses.map((entry) => (entry.id === saved.id ? saved : entry)); + status.set(harness.id, messages.saved); + }) + .catch((error: unknown) => + status.set(harness.id, error instanceof Error ? error.message : String(error)), + ) + .finally(() => { + busy.delete(harness.id); + render(); + }); + }; + const toggle = h(document, "input", { + type: "checkbox", + "aria-label": messages.grokComputerUse, + }); + toggle.checked = computerUse.enabled; + toggle.disabled = pending || (!computerUse.available && !computerUse.enabled); + toggle.addEventListener("change", () => post({ enabled: toggle.checked })); + const browserToggle = h(document, "input", { + type: "checkbox", + "aria-label": messages.grokBrowser, + }); + browserToggle.checked = computerUse.browser === true; + browserToggle.disabled = + pending || + (computerUse.browserFilesPresent !== true && computerUse.browser !== true); + browserToggle.addEventListener("change", () => + post({ browser: browserToggle.checked }), + ); + row.append( + h( + document, + "div", + { className: "console-harness__path" }, + h(document, "label", {}, toggle, ` ${messages.grokComputerUse}`), + ), + h( + document, + "div", + { className: "console-muted" }, + harness.computerUse.available + ? messages.grokComputerUseHelp + : (computerUse.unavailableReason ?? messages.grokComputerUseUnavailable), + ), + h( + document, + "div", + { className: "console-harness__path" }, + h(document, "label", {}, browserToggle, ` ${messages.grokBrowser}`), + ), + h( + document, + "div", + { className: "console-muted" }, + computerUse.browserAvailable === true + ? messages.grokBrowserHelp + : computerUse.browserFilesPresent === true + ? (computerUse.browserUnavailableReason ?? messages.grokBrowserBlocked) + : (computerUse.browserUnavailableReason ?? messages.grokBrowserUnavailable), + ), + ); + if (status.has(harness.id)) { + row.append( + h(document, "div", { className: "console-muted" }, status.get(harness.id) ?? ""), + ); + } + } if (guide) { identity.append( h( diff --git a/packages/renderer-extension/src/console/pages/overview.ts b/packages/renderer-extension/src/console/pages/overview.ts index 64d0628b4..498b87014 100644 --- a/packages/renderer-extension/src/console/pages/overview.ts +++ b/packages/renderer-extension/src/console/pages/overview.ts @@ -1,3 +1,4 @@ +import { renderRestartControl } from "./restart-control.js"; import { consolePost } from "../api.js"; import { button, h } from "../dom.js"; import type { ConsoleMessages } from "../messages.js"; @@ -92,6 +93,8 @@ export function createOverviewPage( start.disabled = launching || !overview.launchAvailable; actions.append(start); } + if (summaryState === "running" || summaryState === "integration-unavailable") + actions.append(renderRestartControl(document, messages, state)); if (state.update?.updateAvailable) { actions.append( button(document, `${messages.viewUpdate} · ${state.update.latestVersion ?? ""}`, () => diff --git a/packages/renderer-extension/src/console/pages/restart-control.ts b/packages/renderer-extension/src/console/pages/restart-control.ts new file mode 100644 index 000000000..24ab5b6d8 --- /dev/null +++ b/packages/renderer-extension/src/console/pages/restart-control.ts @@ -0,0 +1,36 @@ +import { button, h } from "../dom.js"; +import type { ConsoleMessages } from "../messages.js"; +import type { ConsoleState } from "../state.js"; + +export function renderRestartControl( + document: Document, + messages: ConsoleMessages, + state: ConsoleState, +): HTMLElement { + const root = h(document, "div", { className: "console-actions console-restart" }); + const status = state.overview?.restart; + const busy = state.restartSubmitting || status?.phase === "restarting"; + if (state.restartConfirm && !busy) { + root.append( + h(document, "p", { role: "status" }, messages.restartWarning), + button(document, messages.restartConfirm, () => void state.restart(), "primary"), + button(document, messages.restartCancel, () => state.confirmRestart(false)), + ); + } else { + const restart = button(document, messages.restart, () => state.confirmRestart(true)); + restart.disabled = + busy || + state.offline || + !state.overview?.launchAvailable || + !["running", "integration-unavailable"].includes(state.overview.summary.state); + root.append(restart); + } + if (busy || status?.phase === "succeeded") { + root.append( + h(document, "p", { role: "status" }, busy ? messages.restarting : messages.restarted), + ); + } + const error = state.restartError ?? status?.error; + if (error) root.append(h(document, "p", { role: "alert" }, error)); + return root; +} diff --git a/packages/renderer-extension/src/console/state.ts b/packages/renderer-extension/src/console/state.ts index beae529b9..24df8af9a 100644 --- a/packages/renderer-extension/src/console/state.ts +++ b/packages/renderer-extension/src/console/state.ts @@ -37,6 +37,14 @@ export interface ConsoleOverview { }; } | null; launchAvailable: boolean; + liveVoice: { + inject: boolean; + savedInject?: boolean; + envOverride: "0" | "1" | null; + activeInject?: boolean | null; + activeEnvOverride?: "0" | "1" | null; + }; + restart?: { phase: "idle" | "restarting" | "succeeded" | "failed"; error: string | null }; summary: { state: | "starting" @@ -65,6 +73,9 @@ export class ConsoleState { overview: ConsoleOverview | null = null; update: UpdateSummary | null = null; offline = false; + restartConfirm = false; + restartSubmitting = false; + restartError: string | null = null; readonly #listeners = new Set(); #timer: number | undefined; @@ -73,6 +84,28 @@ export class ConsoleState { return () => this.#listeners.delete(listener); } + confirmRestart(confirm: boolean): void { + this.restartConfirm = confirm; + this.#emit(); + } + + async restart(): Promise { + if (!this.restartConfirm || this.restartSubmitting) return; + this.restartConfirm = false; + this.restartSubmitting = true; + this.restartError = null; + this.#emit(); + try { + await consolePost("/api/restart", { confirm: true }); + await this.refresh(); + } catch (error) { + this.restartError = error instanceof Error ? error.message : String(error); + } finally { + this.restartSubmitting = false; + this.#emit(); + } + } + async refresh(): Promise { try { this.overview = await consoleGet("/api/overview"); diff --git a/packages/renderer-extension/src/cpa-live-voice-entry.ts b/packages/renderer-extension/src/cpa-live-voice-entry.ts new file mode 100644 index 000000000..2b34ba62a --- /dev/null +++ b/packages/renderer-extension/src/cpa-live-voice-entry.ts @@ -0,0 +1,12 @@ +import { installCpaLiveVoiceHook } from "./renderer-cpa-live-voice-hook.js"; +import { installClientOwnedVoiceHook } from "./renderer-client-owned-voice.js"; +import { installOverlayExternalFollowUps } from "./renderer-overlay-external-followups.js"; +import { installOverlayThreadIsolation } from "./renderer-overlay-thread-isolation.js"; +import { installVoiceStartCapabilityHook } from "./renderer-voice-start-capability.js"; + +// Document-start: do not wait for body. Voice constructs clientOwnsCall during start. +installCpaLiveVoiceHook(); +installClientOwnedVoiceHook(); +installVoiceStartCapabilityHook(); +installOverlayThreadIsolation(); +installOverlayExternalFollowUps(); diff --git a/packages/renderer-extension/src/production-entry.ts b/packages/renderer-extension/src/production-entry.ts index 2664bbed4..85f4108a7 100644 --- a/packages/renderer-extension/src/production-entry.ts +++ b/packages/renderer-extension/src/production-entry.ts @@ -1,11 +1,24 @@ +import { installNativeChatVoiceHook } from "./renderer-native-chat-voice.js"; +import { installCpaLiveVoiceHook } from "./renderer-cpa-live-voice-hook.js"; +import { installClientOwnedVoiceHook } from "./renderer-client-owned-voice.js"; import { installRendererBinding } from "./install-renderer-binding.js"; +import { installExternalVoiceLaunchHook } from "./renderer-external-voice-launch.js"; +import { installVoiceCapabilityProjection } from "./renderer-voice-capability.js"; +import { installVoiceSettingsExplanation } from "./renderer-voice-settings.js"; -const install = (): void => { +installCpaLiveVoiceHook(); +installClientOwnedVoiceHook(); +installExternalVoiceLaunchHook(); +installNativeChatVoiceHook(); +installVoiceCapabilityProjection(); +installVoiceSettingsExplanation(); + +const installBinding = (): void => { installRendererBinding(); }; if (document.documentElement && document.body) { - install(); + installBinding(); } else { - window.addEventListener("DOMContentLoaded", install, { once: true }); + window.addEventListener("DOMContentLoaded", installBinding, { once: true }); } diff --git a/packages/renderer-extension/src/renderer-binding-probe.ts b/packages/renderer-extension/src/renderer-binding-probe.ts index 89d40c00e..e9fdbbb20 100644 --- a/packages/renderer-extension/src/renderer-binding-probe.ts +++ b/packages/renderer-extension/src/renderer-binding-probe.ts @@ -1,3 +1,4 @@ +import { startVoiceSettingsSync } from "./voice-settings-sync.js"; import { createRemoteConnectionsControl } from "./remote-connections-control.js"; import { isOrbitComposer, orbitComposerKind } from "./renderer-composer-kind.js"; import { handleRemoteConnectionsRequest } from "./remote-connections-request.js"; @@ -58,6 +59,10 @@ import { TRANSCRIPT_ITEM_SELECTOR, } from "./renderer-transcript-dom.js"; import { RendererCodexAccountState } from "./renderer-codex-account-state.js"; +import { + CODEX_USAGE_BANNER_CHANGE_EVENT, + CODEX_USAGE_BANNER_STORAGE_KEY, +} from "./renderer-codex-usage-banner-preference.js"; import { createRendererCodexUsageGate, type RendererCodexUsageGate, @@ -810,12 +815,19 @@ export function installRendererBindingProbe( }); let connectionDiagnostics: RendererConnectionDiagnostics | null = null; const remoteConnections = createRemoteConnectionsControl(window, modelClientForHost); + const requestLocalSettings = async (method: string, params: unknown): Promise => { + const route = window.__codexhostHostRoutingV1?.forHost("local"); + if (disposed || !route) throw new Error("Local Host connection is unavailable"); + return route.manager.sendRequest(method, params); + }; + const stopVoiceSync = startVoiceSettingsSync(window, requestLocalSettings); const usageStatistics = createRendererUsageStatisticsSource({ routing: () => window.__codexhostHostRoutingV1, listConnections: () => remoteConnections.ssh.list(), disposed: () => disposed, }); const settingsLifecycle = installRendererSettingsLifecycle(window, { + requestVoiceSettings: (params) => requestLocalSettings("codexhost/voice/settings", params), getRemoteConnections: () => remoteConnections, getUpdateClient: () => modelControl, getAccountClient: () => modelControl, @@ -1014,7 +1026,7 @@ export function installRendererBindingProbe( const currentCodexAccount = accounts?.accounts.find( ({ accountId }) => accountId === accounts.readyAccountId, ); - const externalSubmissionReady = renderComposerAgentControl( + const usageGateAllowed = renderComposerAgentControl( mounted.control, controller.get(mounted.composer), composerAdapterState(mounted), @@ -1028,7 +1040,7 @@ export function installRendererBindingProbe( currentCodexAccount ?? null, mounted.ownershipStatus === "error", ); - showCodexUsageGateStatus(mounted, mounted.codexUsageGate.update(externalSubmissionReady)); + showCodexUsageGateStatus(mounted, mounted.codexUsageGate.update(usageGateAllowed)); if (mounted.control.usage) { mounted.control.usage.onOpen = () => { void refreshThreadUsage( @@ -2634,6 +2646,18 @@ export function installRendererBindingProbe( if (!client?.setHarnessLaunchSettings) throw new Error("Launch settings are unavailable"); return client.setHarnessLaunchSettings({ harnessId: harnessIdSchema.parse(agent), path }); }, + async getGrokComputerUse(hostId) { + const client = hostId === "local" ? modelClientForHost(hostId) : null; + if (!client?.getGrokComputerUse) + throw new Error("Grok Computer Use settings are unavailable"); + return client.getGrokComputerUse(); + }, + async setGrokComputerUse(hostId, update) { + const client = hostId === "local" ? modelClientForHost(hostId) : null; + if (!client?.setGrokComputerUse) + throw new Error("Grok Computer Use settings are unavailable"); + return client.setGrokComputerUse(update); + }, async openWebUi(hostId: string, agent: ExternalRendererAgent): Promise { const state = hostHarnessAvailabilityState(hostId); const client = hostId === "local" ? modelClientForHost(hostId) : null; @@ -3128,10 +3152,26 @@ export function installRendererBindingProbe( } } }; + const onUsageBannerPreference = (): void => { + for (const mounted of mountedByComposer.values()) { + const hideCodexControls = + controller.isSwitching(mounted.composer) || + controller.get(mounted.composer).agent !== "codex"; + reconcileComposerNativeControls(mounted.control, hideCodexControls, hideCodexControls); + mounted.codexUsageGate.refresh(); + } + }; + const onUsageBannerStorage = (event: StorageEvent): void => { + if (event.key === CODEX_USAGE_BANNER_STORAGE_KEY || event.key === null) { + onUsageBannerPreference(); + } + }; window.addEventListener("codexhost:draft-prewarm-policy-changed", onHostRouteChange); window.addEventListener("codexhost:draft-workspace", onDraftWorkspace); window.addEventListener("codexhost:renderer-adapter-status", onAdapterStatus); window.addEventListener("focus", onWindowFocus); + window.addEventListener(CODEX_USAGE_BANNER_CHANGE_EVENT, onUsageBannerPreference); + window.addEventListener("storage", onUsageBannerStorage); delegationMention = installRendererDelegationMention(document, { onOpen: (editor) => { const composer = composerForElement(editor); @@ -3316,6 +3356,7 @@ export function installRendererBindingProbe( disposeReasoningSoftWrap(); delegationMention?.dispose(); sidebarAgentIcons.dispose(); + stopVoiceSync(); settingsLifecycle.dispose(); document.removeEventListener("beforeinput", onBeforeInput, true); document.removeEventListener("submit", onSubmit, true); @@ -3325,6 +3366,8 @@ export function installRendererBindingProbe( window.removeEventListener("codexhost:draft-workspace", onDraftWorkspace); window.removeEventListener("codexhost:renderer-adapter-status", onAdapterStatus); window.removeEventListener("focus", onWindowFocus); + window.removeEventListener(CODEX_USAGE_BANNER_CHANGE_EVENT, onUsageBannerPreference); + window.removeEventListener("storage", onUsageBannerStorage); for (const state of harnessAvailabilityByHost.values()) { state.requestGeneration += 1; state.codexAccounts?.dispose(); diff --git a/packages/renderer-extension/src/renderer-chat-voice-debug.ts b/packages/renderer-extension/src/renderer-chat-voice-debug.ts new file mode 100644 index 000000000..d03296740 --- /dev/null +++ b/packages/renderer-extension/src/renderer-chat-voice-debug.ts @@ -0,0 +1,25 @@ +const CHAT_VOICE_DEBUG_GLOBAL = "__codexhostChatVoiceDebug"; +const CHAT_VOICE_DEBUG_STORAGE_KEY = "codexhost.chat-voice-debug.v1"; + +export function chatVoiceDebug( + event: string, + detail: Record = {}, +): void { + const entry = { t: Date.now(), event, ...detail }; + const owner = globalThis as unknown as { + console?: { info?: (...args: unknown[]) => void }; + localStorage?: Storage; + } & Record; + const log = Array.isArray(owner[CHAT_VOICE_DEBUG_GLOBAL]) + ? (owner[CHAT_VOICE_DEBUG_GLOBAL] as unknown[]) + : []; + log.push(entry); + while (log.length > 40) log.shift(); + owner[CHAT_VOICE_DEBUG_GLOBAL] = log; + try { + owner.localStorage?.setItem(CHAT_VOICE_DEBUG_STORAGE_KEY, JSON.stringify(log)); + } catch { + /* Overlay storage can be quota-limited. */ + } + owner.console?.info?.("[codexhost-chat-voice]", event, detail); +} diff --git a/packages/renderer-extension/src/renderer-client-owned-voice.ts b/packages/renderer-extension/src/renderer-client-owned-voice.ts new file mode 100644 index 000000000..a5dcabbe2 --- /dev/null +++ b/packages/renderer-extension/src/renderer-client-owned-voice.ts @@ -0,0 +1,473 @@ +import { selectVoiceAssets, supportsVoiceVersion } from "./renderer-voice-assets.js"; +import { chatVoiceDebug } from "./renderer-chat-voice-debug.js"; +import { + REALTIME_CALL_METHOD, + filterRealtimeCallHeaders, + isRealtimeCallId, + realtimeCallIdFromLocation, + realtimeCallResponseSchema, +} from "@codexhost/shared-contracts"; +import { loadedAssetNames, readCpaLiveVoiceHookConfig } from "./renderer-cpa-live-voice-hook.js"; +import { + chatLiveVoicePreferenceAllowsOverride, + cpaLiveVoicePreferenceAllowsInstall, +} from "./renderer-cpa-live-voice-preference.js"; +import { RENDERER_NEW_THREAD_PREFERENCE_KEY } from "./renderer-new-thread-preference.js"; + +export const CLIENT_OWNED_VOICE_VERSION = "26.1021.31345"; +export const CLIENT_OWNED_VOICE_INITIAL_ASSET = "app-initial-feb30b196d11.js"; +export const CLIENT_OWNED_VOICE_SHARED_ASSET = "app-shared-2cae01273bae.js"; +export const CLIENT_OWNED_VOICE_OVERLAY_ASSET = "avatar-overlay-native-page-d549db3706f1.js"; +export const CLIENT_OWNED_VOICE_ASSETS = [ + { + platform: "darwin", + version: CLIENT_OWNED_VOICE_VERSION, + initial: CLIENT_OWNED_VOICE_INITIAL_ASSET, + shared: CLIENT_OWNED_VOICE_SHARED_ASSET, + overlay: CLIENT_OWNED_VOICE_OVERLAY_ASSET, + fetch: "Htr", + fetchFrom: "initial", + bridge: "bIt", + services: "Qot", + capabilityInit: "l0n", + capabilityFamily: "c0n", + inferenceGateFamily: "Cnr", + }, + { + platform: "win32", + version: CLIENT_OWNED_VOICE_VERSION, + initial: "app-initial-438d1df57f70.js", + shared: "app-shared-955d52676f9c.js", + overlay: "avatar-overlay-native-page-1549fe5955e1.js", + fetch: "Htr", + fetchFrom: "initial", + bridge: "bIt", + services: "Qot", + capabilityInit: "l0n", + capabilityFamily: "c0n", + inferenceGateFamily: "Cnr", + }, + { + platform: "darwin", + version: "26.1016.62342", + initial: "app-initial-dddef47d6ef6.js", + shared: "app-shared-fc3316d48427.js", + overlay: "avatar-overlay-native-page-ef8332c6a2fb.js", + init: "Oit", + fetch: "Eit", + bridge: "AFt", + responseError: "Mit", + services: "ist", + capabilityInit: "njr", + capabilityFamily: "tjr", + inferenceGateFamily: "uBr", + }, + { + platform: "darwin", + version: "26.1016.61520", + initial: "app-initial-c95ddb62696e.js", + shared: "app-shared-2eda13eb7044.js", + overlay: "avatar-overlay-native-page-fcc17283519a.js", + init: "Oit", + fetch: "Eit", + bridge: "AFt", + responseError: "Mit", + services: "ist", + capabilityInit: "njr", + capabilityFamily: "tjr", + inferenceGateFamily: "uBr", + }, +]; +const SHARED_INIT = "Oit"; +const SHARED_RESPONSE_ERROR = "Mit"; +const HOOK_GLOBAL = "__codexhostClientOwnedVoiceV1"; +type RecordValue = Record; +function object(value: unknown): value is RecordValue { + return value !== null && typeof value === "object"; +} + +export interface NativeVoiceBridge { + dispatchMessage(type: string, message: RecordValue): void; + subscribe(type: string, listener: (message: unknown) => void): () => void; +} + +export function requestVoiceHost( + bridge: NativeVoiceBridge, + params: RecordValue, + method: string = REALTIME_CALL_METHOD, +): Promise { + const id = `codexhost-voice:${crypto.randomUUID()}`; + return new Promise((resolve, reject) => { + let unsubscribe = () => {}; + const timer = setTimeout(() => { + unsubscribe(); + reject(new Error("Voice gateway did not answer in time")); + }, 30_000); + const done = () => { + clearTimeout(timer); + unsubscribe(); + }; + unsubscribe = bridge.subscribe("mcp-response", (event) => { + if ( + !object(event) || + event.hostId !== "local" || + !object(event.message) || + event.message.id !== id + ) + return; + done(); + if (event.message.error) reject(new Error("Configured voice gateway request failed")); + else resolve(event.message.result); + }); + try { + bridge.dispatchMessage("mcp-request", { + hostId: "local", + request: { id, method, params }, + }); + } catch { + done(); + reject(new Error("Voice gateway Host connection is unavailable")); + } + }); +} + +interface NativeVoiceFetch { + fetch(url: string, options?: RequestInit): Promise; +} + +/** Desktop Chat/Work toggle persistence. Missing means Chat. */ +export const HOME_COMPOSER_MODE_STORAGE_KEY = "home-composer-mode-v1"; + +function backendModelName(headers?: HeadersInit): string { + const raw = new Headers(headers).get("x-codex-turn-metadata"); + if (!raw) return ""; + try { + const parsed = JSON.parse(raw) as { backend_model_name?: unknown }; + return typeof parsed.backend_model_name === "string" ? parsed.backend_model_name : ""; + } catch { + return ""; + } +} + +/** Coding / Harness backend models. Speech `gpt-live*` is used by Chat and Work. */ +function isWorkBackendModel(name: string): boolean { + const model = name.toLowerCase(); + if (!model) return false; + if (model.startsWith("codexhost/")) return true; + return model.includes("codex") && !model.includes("gpt-live"); +} + +function visibleWorkComposer(owner: object): boolean { + const document = (owner as { document?: Document }).document; + if (!document?.querySelectorAll) return false; + const visible = (element: Element) => { + const box = element.getBoundingClientRect?.() ?? { width: 0 }; + return box.width > 0 && !element.closest?.("[hidden], [inert]"); + }; + return ( + [...document.querySelectorAll("[data-codex-composer-root]")].some(visible) && + ![...document.querySelectorAll("[data-chatgpt-composer]")].some(visible) + ); +} + +function selectedWorkTab(owner: object): boolean { + const document = (owner as { document?: Document }).document; + if (!document?.querySelectorAll) return false; + const controls = document.querySelectorAll("button, [role='tab'], a"); + let workSelected = false; + let chatSelected = false; + for (const element of controls) { + const name = (element.textContent ?? "").replace(/\s+/gu, " ").trim(); + if (name !== "Chat" && name !== "Work") continue; + const selected = + element.getAttribute("aria-selected") === "true" || + element.getAttribute("aria-pressed") === "true" || + element.getAttribute("data-state") === "active" || + element.getAttribute("data-state") === "on"; + if (!selected) continue; + if (name === "Work") workSelected = true; + if (name === "Chat") chatSelected = true; + } + return workSelected && !chatSelected; +} + +export function isWorkComposerSurface(owner: object = globalThis): boolean { + const storage = (owner as { localStorage?: Storage }).localStorage; + try { + const raw = storage?.getItem(HOME_COMPOSER_MODE_STORAGE_KEY); + if (raw != null && raw !== "") { + let value: unknown = raw; + if (raw.startsWith("{") || raw.startsWith('"') || raw.startsWith("[")) { + try { + value = JSON.parse(raw) as unknown; + } catch { + value = raw; + } + } + if (value === "work") return true; + if (value === "chat") return false; + } + } catch { + /* Fall through to the visible Work/Chat control. */ + } + return selectedWorkTab(owner) || visibleWorkComposer(owner); +} + +function selectedComposerModel(owner: object): string { + const routing = ( + owner as { + __codexhostHostRoutingV1?: { + forComposer(): { policy: { selectedModel(): string | undefined } } | undefined; + }; + } + ).__codexhostHostRoutingV1; + try { + const model = routing?.forComposer()?.policy.selectedModel(); + return typeof model === "string" ? model : ""; + } catch { + return ""; + } +} + +let lastVoiceProductExperience: string | undefined; + +export function rememberVoiceProductExperience(request: unknown): void { + const experience = object(request) ? request.productExperience : undefined; + lastVoiceProductExperience = + experience === "chat" || experience === "work" || experience === "codex" + ? experience + : undefined; +} + +export function lastVoiceLaunchProductExperience(): string | undefined { + return lastVoiceProductExperience; +} + +export function isExternalHarnessComposer(owner: object = globalThis): boolean { + if (lastVoiceProductExperience === "chat") return false; + if (selectedComposerModel(owner).startsWith("codexhost/")) return true; + if (lastVoiceProductExperience !== "work") return false; + const storage = (owner as { localStorage?: Storage }).localStorage; + if (!storage) return false; + try { + const parsed = JSON.parse(storage.getItem(RENDERER_NEW_THREAD_PREFERENCE_KEY) ?? "") as { + lastAgent?: unknown; + }; + return ( + typeof parsed.lastAgent === "string" && + parsed.lastAgent !== "" && + parsed.lastAgent !== "codex" + ); + } catch { + return false; + } +} + +/** Only the audited native WebFetch voice bootstrap is forwarded. Other native + * requests retain their receiver, arguments, authentication and error behavior. + */ +export function wrapClientOwnedVoiceFetch( + client: NativeVoiceFetch, + bridge: NativeVoiceBridge, + workOverride: () => boolean, + responseError: (response: Response, url: string) => Promise, + chatOverride: () => boolean = () => false, + workSurface: () => boolean = () => isWorkComposerSurface(), + externalHarness: () => boolean = () => isExternalHarnessComposer(), +): () => void { + const original = client.fetch; + const wrapped: NativeVoiceFetch["fetch"] = async function (this: NativeVoiceFetch, url, options) { + if ( + typeof url !== "string" || + url.split("?")[0] !== "/wham/realtime/calls" || + options?.method !== "POST" + ) + return original.call(this, url, options); + const experience = lastVoiceLaunchProductExperience(); + const chat = experience === "chat"; + const model = backendModelName(options.headers); + // The overlay has no Composer; explicit launch ownership also covers Work + // models without "codex" in their name and starts without model metadata. + const work = + !chat && + (experience === "work" || + experience === "codex" || + isWorkBackendModel(model) || + workSurface()); + // The avatar overlay has no primary Composer routing state. The native + // request's explicit Harness model still owns its gateway bootstrap there. + const external = !chat && (model.startsWith("codexhost/") || externalHarness()); + const gateway = external || (work ? workOverride() : chatOverride()); + chatVoiceDebug("wham-create", { + work, + external, + route: gateway ? "gateway" : "desktop", + productExperience: experience ?? "unset", + }); + if (!gateway) { + if (!external && !work) + // Native Chat Instant Connect is Desktop's browser-voice stack + // (fetchVoiceRequest → chatgpt.com/realtime/wm), not Codex /wham. + chatVoiceDebug("chat-native-passthrough", { + hasThreadId: new Headers(options.headers).has("Thread-Id"), + }); + return original.call(this, url, options); + } + if (typeof options.body !== "string") throw new Error("Unsupported native voice call request"); + options.signal?.throwIfAborted(); + const requestId = crypto.randomUUID(); + let cancelled = false; + const cancel = () => { + if (cancelled) return; + cancelled = true; + void requestVoiceHost(bridge, { operation: "cancel", requestId }).catch(() => undefined); + }; + options.signal?.addEventListener("abort", cancel, { once: true }); + try { + const result = realtimeCallResponseSchema.parse( + await requestVoiceHost(bridge, { + operation: "create", + requestId, + body: options.body, + // Drop Desktop OAuth, ChatGPT account identity and cookies at the bridge + // boundary; the Host allowlist repeats this check before sending. + headers: filterRealtimeCallHeaders(Object.fromEntries(new Headers(options.headers))), + }), + ); + if (options.signal?.aborted) { + const callId = realtimeCallIdFromLocation(result.headers.location ?? ""); + if (result.status >= 200 && result.status < 300 && isRealtimeCallId(callId)) + await requestVoiceHost(bridge, { operation: "hangup", callId }); + options.signal.throwIfAborted(); + } + const response = new Response(result.status === 204 ? null : result.body, { + status: result.status, + headers: result.headers, + }); + if (!response.ok) throw await responseError(response, url); + return response; + } catch (error) { + cancel(); + throw error; + } finally { + options.signal?.removeEventListener("abort", cancel); + } + }; + client.fetch = wrapped; + return () => { + if (client.fetch === wrapped) client.fetch = original; + }; +} + +interface VoiceTarget { + [key: string]: unknown; + document?: ParentNode; + performance?: Pick; + setTimeout: typeof setTimeout; + clearTimeout: typeof clearTimeout; +} +interface VoiceHook { + status: "disabled" | "pending" | "active" | "unsupported"; + dispose(): void; +} + +/** Desktop 26.1016+ always creates calls in the Renderer. Its native WebFetch + * singleton and message bridge are audited exports of the loaded shared module. + */ +export function installClientOwnedVoiceHook( + options: { + target?: VoiceTarget; + importModule?: (url: string) => Promise; + } = {}, +): VoiceHook { + const target = options.target ?? (globalThis as unknown as VoiceTarget); + const existing = target[HOOK_GLOBAL] as VoiceHook | undefined; + if (existing) return existing; + const config = readCpaLiveVoiceHookConfig(target); + if (!config) return { status: "disabled", dispose() {} }; + if (!supportsVoiceVersion(CLIENT_OWNED_VOICE_ASSETS, config.desktopVersion)) + return { status: "unsupported", dispose() {} }; + const importModule = options.importModule ?? ((url: string) => import(url)); + let disposed = false; + let timer: ReturnType | undefined; + let restore: (() => void) | undefined; + const started = Date.now(); + const handle: VoiceHook = { + status: "pending", + dispose() { + disposed = true; + if (timer !== undefined) target.clearTimeout(timer); + restore?.(); + if (target[HOOK_GLOBAL] === handle) Reflect.deleteProperty(target, HOOK_GLOBAL); + }, + }; + target[HOOK_GLOBAL] = handle; + const attempt = async (): Promise => { + if (disposed) return; + const names = loadedAssetNames(target.document, target.performance); + const assets = selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ); + if (assets) { + try { + const sharedModule = await importModule(`app://-/assets/${assets.shared}`); + const fetchFrom = "fetchFrom" in assets ? assets.fetchFrom : "shared"; + const fetchModule = + fetchFrom === "initial" + ? await importModule(`app://-/assets/${assets.initial}`) + : sharedModule; + if (disposed) return; + const init = "init" in assets ? assets.init : SHARED_INIT; + const fetchName = assets.fetch; + const bridgeName = assets.bridge; + const errorName = "responseError" in assets ? assets.responseError : SHARED_RESPONSE_ERROR; + if (typeof init === "string" && typeof fetchModule[init] === "function") { + (fetchModule[init] as () => void)(); + } else if (typeof init === "string" && typeof sharedModule[init] === "function") { + (sharedModule[init] as () => void)(); + } + const fetchClass = typeof fetchName === "string" ? fetchModule[fetchName] : undefined; + const bridgeClass = typeof bridgeName === "string" ? sharedModule[bridgeName] : undefined; + if (!object(fetchClass) && typeof fetchClass !== "function") throw new Error(); + if (!object(bridgeClass) && typeof bridgeClass !== "function") throw new Error(); + const client = Reflect.get(fetchClass, "getInstance").call(fetchClass); + const bridge = Reflect.get(bridgeClass, "getInstance").call(bridgeClass); + const responseError = + typeof errorName === "string" && typeof sharedModule[errorName] === "function" + ? (sharedModule[errorName] as (response: Response, url: string) => Promise) + : async (response: Response) => { + throw Object.assign(new Error(await response.text()), { status: response.status }); + }; + if ( + typeof client?.fetch !== "function" || + typeof bridge?.dispatchMessage !== "function" || + typeof bridge?.subscribe !== "function" + ) + throw new Error(); + restore = wrapClientOwnedVoiceFetch( + client, + bridge, + () => cpaLiveVoicePreferenceAllowsInstall(target), + responseError, + () => chatLiveVoicePreferenceAllowsOverride(target), + () => isWorkComposerSurface(target), + () => isExternalHarnessComposer(target), + ); + handle.status = "active"; + } catch { + /* Retry below until the timeout. */ + } + } + if (handle.status === "active") return; + if (Date.now() - started >= 60_000) { + handle.status = "unsupported"; + return; + } + timer = target.setTimeout(() => void attempt(), 250); + }; + void attempt(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-codex-usage-banner-preference.ts b/packages/renderer-extension/src/renderer-codex-usage-banner-preference.ts new file mode 100644 index 000000000..340770707 --- /dev/null +++ b/packages/renderer-extension/src/renderer-codex-usage-banner-preference.ts @@ -0,0 +1,22 @@ +export const CODEX_USAGE_BANNER_STORAGE_KEY = "codexhost.codex-usage-banner-hidden.v1"; +export const CODEX_USAGE_BANNER_CHANGE_EVENT = "codexhost:codex-usage-banner-changed"; + +/** Hide the native Codex out-of-usage card on every Composer. Default on. */ +export function readCodexUsageBannerPreference(owner: Window | null | undefined): boolean { + if (!owner) return true; + try { + return owner.localStorage.getItem(CODEX_USAGE_BANNER_STORAGE_KEY) !== "0"; + } catch { + return true; + } +} + +export function writeCodexUsageBannerPreference(owner: Window, enabled: boolean): boolean { + try { + owner.localStorage.setItem(CODEX_USAGE_BANNER_STORAGE_KEY, enabled ? "1" : "0"); + } catch { + return false; + } + owner.dispatchEvent(new Event(CODEX_USAGE_BANNER_CHANGE_EVENT)); + return true; +} diff --git a/packages/renderer-extension/src/renderer-codex-usage-gate.ts b/packages/renderer-extension/src/renderer-codex-usage-gate.ts index 4ecdb79fc..c64c914a9 100644 --- a/packages/renderer-extension/src/renderer-codex-usage-gate.ts +++ b/packages/renderer-extension/src/renderer-codex-usage-gate.ts @@ -1,14 +1,14 @@ /** - * Lets an external Harness Composer submit while the signed-in ChatGPT Codex - * subscription is out of usage. + * Lets a Composer submit while the signed-in ChatGPT Codex subscription is + * out of usage. * * Desktop blocks Composer submission through two boolean selectors: the * ChatGPT Account rate-limit gate and the reserve `hardBlocked` gate. Both are * Account-wide, only apply to ChatGPT sign-in, and are not a protocol limit. * This module projects `false` for those two subscriptions of one Composer - * only. It never writes Accounts, atoms or query data, so the Codex path and - * other Composers keep the native result. Other native submit blockers are - * unaffected. Anything that cannot be verified keeps the native restriction. + * only. It never writes Accounts, atoms or query data, so other Composers keep + * the native result. Other native submit blockers are unaffected. Anything + * that cannot be verified keeps the native restriction. * * Desktop 26.1002 moved the Account gate into a child of the submit owner and * lets an active reserve exempt it, so gates are collected along the Composer's @@ -17,6 +17,7 @@ import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; import { createRendererCodexUsageBanner } from "./renderer-codex-usage-banner.js"; +import { readCodexUsageBannerPreference } from "./renderer-codex-usage-banner-preference.js"; type RecordValue = Record; type Atom = { read(get: (atom: Atom) => unknown): unknown }; @@ -408,7 +409,7 @@ function isBound(projections: Iterable): boolean { export type RendererCodexUsageGateStatus = "native" | "bypassed" | "unsupported"; export interface RendererCodexUsageGate { - /** `allowed`: this Composer submits to a ready external Harness. */ + /** `allowed`: this Composer should lift the native Codex usage submit gate. */ update(allowed: boolean): RendererCodexUsageGateStatus; /** Re-apply the last decision, rebinding if Desktop replaced the Composer hooks. */ refresh(): RendererCodexUsageGateStatus; @@ -448,11 +449,15 @@ export function createRendererCodexUsageGate(composer: Element): RendererCodexUs } return "bypassed"; }; - // The usage banner only describes the Codex path; hide it while bypassed. + // Fiber banner hide follows Settings → Appearance. Default on. const apply = (): RendererCodexUsageGateStatus => { const status = applyGate(); - if (status === "bypassed" && owner) banner.hide(owner.owner); - else banner.restore(); + const ownerWindow = + composer.ownerDocument?.defaultView ?? + (typeof document !== "undefined" ? document.defaultView : null); + if (status === "bypassed" && owner && readCodexUsageBannerPreference(ownerWindow)) { + banner.hide(owner.owner); + } else banner.restore(); return status; }; const applyGate = (): RendererCodexUsageGateStatus => { diff --git a/packages/renderer-extension/src/renderer-composer-dom.ts b/packages/renderer-extension/src/renderer-composer-dom.ts index d32318af6..9630cf4b7 100644 --- a/packages/renderer-extension/src/renderer-composer-dom.ts +++ b/packages/renderer-extension/src/renderer-composer-dom.ts @@ -54,6 +54,7 @@ import { mountRendererHarnessCommandControl, type RendererHarnessCommandControl, } from "./renderer-harness-command-control.js"; +import { readCodexUsageBannerPreference } from "./renderer-codex-usage-banner-preference.js"; export { CONTROL_ATTRIBUTE }; export type ExternalModelControlView = RendererModelControlView; @@ -95,6 +96,7 @@ export interface ComposerAgentControl { nativeModelControl: NativeModelControlState | null; nativePermissionModeControl: NativePermissionModeControlState | null; nativeContextUsageControl?: NativeControlState | null; + nativeUsageBanner?: NativeControlState | null; nativePermissionModeControlVerified: boolean; credits: RendererCreditsControl; composerIndicators: RendererComposerIndicators; @@ -364,6 +366,24 @@ export function nativeContextUsageControlForComposer(composer: Element): HTMLEle return candidates.length === 1 ? (candidates[0] ?? null) : null; } +/** + * Codex usage / upsell recovery banner above the Composer: an aside with a + * title and CTA buttons. Some Desktop versions also mark it as a polite + * status; others omit those ARIA attributes. Ambiguous matches stay visible. + */ +export function isNativeUsageBannerCandidate(element: Element): boolean { + if (element.localName !== "aside") return false; + if (typeof element.querySelector !== "function") return false; + return element.querySelector("h3") !== null && element.querySelector("button") !== null; +} + +export function nativeUsageBannerForComposer(composer: Element): HTMLElement | null { + const candidates = [...composer.querySelectorAll("aside")].filter( + isNativeUsageBannerCandidate, + ); + return candidates.length === 1 ? (candidates[0] ?? null) : null; +} + function contractElementVisible(element: Element): boolean { const typed = element as HTMLElement; const bounds = typed.getBoundingClientRect?.(); @@ -458,6 +478,13 @@ function refreshNativeContextUsageControl(control: ComposerAgentControl): void { control.nativeContextUsageControl = captureNativeControl(candidate); } +function refreshNativeUsageBanner(control: ComposerAgentControl): void { + const candidate = nativeUsageBannerForComposer(control.composer); + if (candidate === control.nativeUsageBanner?.element) return; + restoreNativeControl(control.nativeUsageBanner); + control.nativeUsageBanner = captureNativeControl(candidate); +} + function refreshNativeModelControl(control: ComposerAgentControl): void { const candidate = nativeModelControlForComposer(control.composer); if (!candidate) return; @@ -646,6 +673,7 @@ export function reconcileComposerNativeControls( // it below, so Credits never reads a stale (e.g. mount-time fallback) // location for it within this same pass. refreshNativePermissionModeControl(control); + refreshNativeUsageBanner(control); refreshTrailingClusterPlacement(control); refreshUsagePlacement(control); refreshCreditsPlacement(control); @@ -660,6 +688,20 @@ export function reconcileComposerNativeControls( // visible when the external Model control is substituted. setNativeControlHidden(control.nativeContextUsageControl, false); setNativeControlHidden(control.nativePermissionModeControl, hidePermissionMode); + // Settings → Appearance, default on: hide the account-wide Codex usage + // aside on every Composer, including Codex threads. The aside reports the + // ChatGPT Account rate limit and upsell, not a per-thread notice. + setNativeControlHidden( + control.nativeUsageBanner, + readCodexUsageBannerPreference(preferenceOwner(control)), + ); +} + +function preferenceOwner(control: ComposerAgentControl): Window | null { + return ( + control.composer.ownerDocument?.defaultView ?? + (typeof document !== "undefined" ? document.defaultView : null) + ); } export function mountComposerAgentControl( @@ -683,6 +725,7 @@ export function mountComposerAgentControl( const nativeContextUsageControl = captureNativeControl( nativeContextUsageControlForComposer(composer), ); + const nativeUsageBanner = captureNativeControl(nativeUsageBannerForComposer(composer)); const semanticNativePermissionModeControl = semanticNativePermissionModeControlForComposer(composer); const nativePermissionModeControl = captureNativeControl(semanticNativePermissionModeControl); @@ -758,6 +801,7 @@ export function mountComposerAgentControl( nativeModelControl, nativePermissionModeControl, nativeContextUsageControl, + nativeUsageBanner, nativePermissionModeControlVerified, credits, composerIndicators, @@ -784,8 +828,8 @@ function nativeSendDisabled(button: HTMLButtonElement, fallback: boolean): boole } /** - * Returns whether this Composer is ready to submit to an external Harness: - * not Codex, Adapter ready, and no codexhost submission blocker. + * Returns whether this Composer should lift the native Codex usage submit + * gate: no codexhost submission blocker, and either Codex or a ready Adapter. */ export function renderComposerAgentControl( control: ComposerAgentControl, @@ -879,7 +923,7 @@ export function renderComposerAgentControl( control.harnessCommands.root.hidden = state.agent === "codex"; control.harnessCommands.root.style.display = state.agent === "codex" ? "none" : "inline-flex"; renderRendererCreditsControl(control.credits, accountCredits, locale); - return state.agent !== "codex" && adapterState === "ready" && !submissionBlocked; + return !submissionBlocked && (state.agent === "codex" || adapterState === "ready"); } export function disposeComposerAgentControl(control: ComposerAgentControl): void { @@ -891,6 +935,7 @@ export function disposeComposerAgentControl(control: ComposerAgentControl): void } restoreNativeControl(control.nativeModelControl); restoreNativeControl(control.nativeContextUsageControl); + restoreNativeControl(control.nativeUsageBanner); restoreNativeControl(control.nativePermissionModeControl); control.composerIndicators?.dispose(); control.credits.dispose(); diff --git a/packages/renderer-extension/src/renderer-cpa-live-voice-hook.ts b/packages/renderer-extension/src/renderer-cpa-live-voice-hook.ts new file mode 100644 index 000000000..0f1efb2d4 --- /dev/null +++ b/packages/renderer-extension/src/renderer-cpa-live-voice-hook.ts @@ -0,0 +1,216 @@ +import { cpaLiveVoicePreferenceAllowsInstall } from "./renderer-cpa-live-voice-preference.js"; + +/** + * Runtime equivalent of durable-only `clientOwnsCall` for Codex Desktop + * 26.1007.30912. Default on. Does not rewrite ASAR or copy Desktop source. + * + * Native: hostId === "durable" OR (app-server supports threadRealtimeExistingCall + * AND Statsig gate 206132468). The capability is a version comparison, so the + * hook forces that gate to false; the gate has no other reader in this build. + */ +export const CPA_LIVE_VOICE_HOOK_GLOBAL = "__codexhostCpaLiveVoiceHookV1"; +export const AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION = "26.1007.30912"; +export const AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET = "app-initial-6350d04672d0.js"; +export const THREAD_REALTIME_EXISTING_CALL_GATE = "206132468"; + +type RecordValue = Record; + +export interface CpaLiveVoiceHookConfig { + enabled: boolean; + auditedDesktopVersion: string; + auditedAppInitialAsset: string; + desktopVersion?: string; + desktopPlatform?: string; +} + +export type CpaLiveVoiceHookStatus = "disabled" | "pending" | "active" | "unsupported"; + +export interface CpaLiveVoiceStatsigClient { + checkGate(name: string, ...rest: unknown[]): unknown; +} + +export interface CpaLiveVoiceHookHandle { + status: CpaLiveVoiceHookStatus; + dispose(): void; +} + +export interface InstallCpaLiveVoiceHookOptions { + config?: CpaLiveVoiceHookConfig | null; + resourceNames?: readonly string[]; + globalThis?: object; + document?: ParentNode; +} + +const originalCheckGate = new WeakMap(); +const DISCOVERY_TIMEOUT_MS = 60_000; + +function isRecord(value: unknown): value is RecordValue { + return typeof value === "object" && value !== null; +} + +export function readCpaLiveVoiceHookConfig(globalObject: object): CpaLiveVoiceHookConfig | null { + const value = (globalObject as Record)[CPA_LIVE_VOICE_HOOK_GLOBAL]; + if (!isRecord(value) || value.enabled !== true) return null; + if ( + typeof value.auditedDesktopVersion !== "string" || + typeof value.auditedAppInitialAsset !== "string" + ) { + return null; + } + return { + enabled: true, + auditedDesktopVersion: value.auditedDesktopVersion, + auditedAppInitialAsset: value.auditedAppInitialAsset, + ...(typeof value.desktopPlatform === "string" + ? { desktopPlatform: value.desktopPlatform } + : {}), + ...(typeof value.desktopVersion === "string" ? { desktopVersion: value.desktopVersion } : {}), + }; +} + +export function auditedAppInitialAssetPresent( + resourceNames: readonly string[], + asset: string, +): boolean { + return resourceNames.some((name) => { + try { + const url = new URL(name, "app://-/"); + return url.protocol === "app:" && url.hostname === "-" && url.pathname === `/assets/${asset}`; + } catch { + return false; + } + }); +} + +function resourceNamesFromPerformance( + source: + (Pick & { getEntries?(): PerformanceEntry[] }) | undefined, +): string[] { + const target = source ?? (typeof performance === "undefined" ? undefined : performance); + if (!target) return []; + const names: string[] = []; + const push = (name: unknown) => { + if (typeof name === "string" && name) names.push(name); + }; + if (typeof target.getEntries === "function") { + for (const entry of target.getEntries()) push(entry.name); + } else if (typeof target.getEntriesByType === "function") { + for (const entry of target.getEntriesByType("resource")) push(entry.name); + } + return names; +} + +/** Every loaded asset name this page exposes: resource timing plus script/link tags. */ +export function loadedAssetNames( + root: ParentNode | undefined, + performanceSource?: Pick & { getEntries?(): PerformanceEntry[] }, +): string[] { + const names = resourceNamesFromPerformance(performanceSource); + if (!root || !("querySelectorAll" in root)) return names; + for (const element of root.querySelectorAll( + 'script[src], link[href], link[rel="modulepreload"], link[rel="preload"]', + )) { + const value = element.getAttribute("src") ?? element.getAttribute("href"); + if (value) names.push(value); + } + return names; +} + +/** Statsig JS clients register on `__STATSIG__` (`instances` by SDK key, plus `firstInstance`). */ +export function statsigClients(globalObject: object): CpaLiveVoiceStatsigClient[] { + const registry = (globalObject as Record).__STATSIG__; + if (!isRecord(registry)) return []; + const clients = new Set( + isRecord(registry.instances) ? Object.values(registry.instances) : [], + ); + if (registry.firstInstance) clients.add(registry.firstInstance); + return [...clients].filter( + (client): client is CpaLiveVoiceStatsigClient => + isRecord(client) && typeof client.checkGate === "function", + ); +} + +export function wrapCheckGate(client: CpaLiveVoiceStatsigClient): boolean { + if (originalCheckGate.has(client)) return false; + const original = client.checkGate; + originalCheckGate.set(client, original); + client.checkGate = function (this: unknown, name: string, ...rest: unknown[]) { + return name === THREAD_REALTIME_EXISTING_CALL_GATE ? false : original.call(this, name, ...rest); + }; + return true; +} + +function restoreCheckGate(client: CpaLiveVoiceStatsigClient): void { + const original = originalCheckGate.get(client); + if (!original) return; + client.checkGate = original; + originalCheckGate.delete(client); +} + +function configAllowsInstall( + config: CpaLiveVoiceHookConfig, + resourceNames: readonly string[], +): boolean { + if ( + config.desktopVersion !== undefined && + config.desktopVersion !== config.auditedDesktopVersion + ) { + return false; + } + if (config.auditedDesktopVersion !== AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION) return false; + if (config.auditedAppInitialAsset !== AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET) return false; + return auditedAppInitialAssetPresent(resourceNames, config.auditedAppInitialAsset); +} + +export function installCpaLiveVoiceHook( + options: InstallCpaLiveVoiceHookOptions = {}, +): CpaLiveVoiceHookHandle { + const globalObject = options.globalThis ?? globalThis; + const config = + options.config === undefined ? readCpaLiveVoiceHookConfig(globalObject) : options.config; + if (!config?.enabled || !cpaLiveVoicePreferenceAllowsInstall(globalObject)) { + return { status: "disabled", dispose() {} }; + } + const root = options.document ?? (globalObject as { document?: ParentNode }).document; + const wrapped: CpaLiveVoiceStatsigClient[] = []; + let disposed = false; + let auditedAssetFound = false; + const handle: CpaLiveVoiceHookHandle = { + status: "pending", + dispose() { + disposed = true; + for (const client of wrapped) restoreCheckGate(client); + wrapped.length = 0; + }, + }; + const attempt = (): boolean => { + if (!auditedAssetFound) { + const resourceNames = options.resourceNames ?? loadedAssetNames(root); + auditedAssetFound = configAllowsInstall(config, resourceNames); + if (!auditedAssetFound) return false; + } + for (const client of statsigClients(globalObject)) { + if (wrapCheckGate(client)) wrapped.push(client); + } + if (wrapped.length === 0) return false; + handle.status = "active"; + return true; + }; + if (attempt()) return handle; + if (options.resourceNames && !configAllowsInstall(config, options.resourceNames)) { + handle.status = "unsupported"; + return handle; + } + // Document-start injection runs before the bundle constructs its Statsig client. + const started = Date.now(); + const tick = (): void => { + if (disposed || attempt()) return; + if (Date.now() - started > DISCOVERY_TIMEOUT_MS) { + handle.status = "unsupported"; + return; + } + globalThis.setTimeout(tick, 250); + }; + globalThis.setTimeout(tick, 250); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-cpa-live-voice-preference.ts b/packages/renderer-extension/src/renderer-cpa-live-voice-preference.ts new file mode 100644 index 000000000..1f95fffd5 --- /dev/null +++ b/packages/renderer-extension/src/renderer-cpa-live-voice-preference.ts @@ -0,0 +1,105 @@ +import { voiceRoutesSchema, type VoiceRoutes } from "@codexhost/shared-contracts"; + +// Voice hook globals survive binding-probe replacement and bundle reinjection. +// Keep their settings in the document realm so each bundle reads the same state. +const voiceStateOwner = globalThis as typeof globalThis & { + __codexhostVoicePreferenceStateV1?: { + sharedRoutes: WeakMap; + gatewayReady: WeakMap; + }; +}; +const { sharedRoutes, gatewayReady } = (voiceStateOwner.__codexhostVoicePreferenceStateV1 ??= { + sharedRoutes: new WeakMap(), + gatewayReady: new WeakMap(), +}); +export function applyVoiceGatewayReadiness(owner: object, ready: boolean): void { + gatewayReady.set(owner, ready); +} +export function voiceGatewayIsReady(owner: object): boolean { + return gatewayReady.get(owner) === true; +} +const SHARED_VOICE_CACHE = "codexhost.voice-routes-cache.v1"; +function cachedRoutes(owner: object): VoiceRoutes | undefined { + const memory = sharedRoutes.get(owner); + if (memory) return memory; + try { + const storage = (owner as { localStorage?: Storage }).localStorage; + const parsed = voiceRoutesSchema.safeParse( + JSON.parse(storage?.getItem(SHARED_VOICE_CACHE) ?? "null"), + ).data; + if (parsed) sharedRoutes.set(owner, parsed); + return parsed; + } catch { + return undefined; + } +} +export function applySharedVoiceRoutes(owner: object, routes: VoiceRoutes): void { + const snapshot = voiceRoutesSchema.parse({ + chatExperience: routes.chatExperience, + chatConnection: routes.chatConnection, + workConnection: routes.workConnection, + }); + sharedRoutes.set(owner, snapshot); + try { + (owner as { localStorage?: Storage }).localStorage?.setItem( + SHARED_VOICE_CACHE, + JSON.stringify(snapshot), + ); + } catch { + /* Memory retains the applied route if browser storage is unavailable. */ + } +} +export function nativeChatVoicePreferenceAllowsStart(owner: object): boolean { + const routes = cachedRoutes(owner); + return routes + ? routes.chatExperience === "chatgpt" + : !readFlag(owner, CHAT_LIVE_VOICE_STORAGE_KEY, false); +} +export function legacyVoiceRoutes(owner: Window): VoiceRoutes { + const chatLegacy = readFlag(owner, CHAT_LIVE_VOICE_STORAGE_KEY, false); + return { + chatExperience: chatLegacy ? "codex" : "chatgpt", + chatConnection: chatLegacy ? "gateway" : "desktop", + workConnection: readFlag(owner, CPA_LIVE_VOICE_STORAGE_KEY, true) ? "gateway" : "desktop", + }; +} + +export const CPA_LIVE_VOICE_STORAGE_KEY = "codexhost.cpa-live-voice.v1"; +export const CHAT_LIVE_VOICE_STORAGE_KEY = "codexhost.chat-live-voice.v1"; + +function readFlag(owner: object, key: string, enabledWhenUnset: boolean): boolean { + const storage = (owner as { localStorage?: Storage }).localStorage; + if (!storage) return enabledWhenUnset; + try { + const value = storage.getItem(key); + return enabledWhenUnset ? value !== "0" : value === "1"; + } catch { + return enabledWhenUnset; + } +} + +/** Codex Work gateway choice; legacy storage is read only until shared migration. */ +export function readCpaLiveVoicePreference(owner: Window): boolean { + const routes = cachedRoutes(owner); + return routes + ? routes.workConnection === "gateway" + : readFlag(owner, CPA_LIVE_VOICE_STORAGE_KEY, true); +} + +/** Gateway is an explicit Codex Chat connection, independent of native ChatGPT. */ +export function readChatLiveVoicePreference(owner: Window): boolean { + const routes = cachedRoutes(owner); + return routes + ? routes.chatExperience === "codex" && routes.chatConnection === "gateway" + : readFlag(owner, CHAT_LIVE_VOICE_STORAGE_KEY, false); +} + +/** Work override state for the injected hook, which only exposes a global object. */ +export function cpaLiveVoicePreferenceAllowsInstall(globalObject: object): boolean { + return readCpaLiveVoicePreference(globalObject as Window); +} + +/** Chat override state for the injected hook, which only exposes a global object. */ +export function chatLiveVoicePreferenceAllowsOverride(globalObject: object): boolean { + return readChatLiveVoicePreference(globalObject as Window); +} diff --git a/packages/renderer-extension/src/renderer-dictation-settings.ts b/packages/renderer-extension/src/renderer-dictation-settings.ts new file mode 100644 index 000000000..b21bbe034 --- /dev/null +++ b/packages/renderer-extension/src/renderer-dictation-settings.ts @@ -0,0 +1,191 @@ +import { hooks, replayVoiceCapability } from "./renderer-voice-capability.js"; + +type Atom = Parameters[1]; +type Store = Parameters[0]; +type Value = Record; +type Mode = "os-global" | "app" | null; +interface Subscriber { + getSnapshot(): unknown; + subscribe(listener: () => void): () => void; + createRender?(): unknown; +} +interface Instance { + getSnapshot(): unknown; +} +const record = (value: unknown): value is Value => value !== null && typeof value === "object"; +const mode = (value: unknown): value is Mode => + value === null || value === "os-global" || value === "app"; + +/** Preserve the native platform selector, replaying only proven dictation capabilities. */ +export function replayDictationSettingsMode( + store: Store, + atom: Atom, + capabilities: readonly Atom[], + gate: Atom, +): Mode | undefined { + try { + const native = store.get(atom); + if (atom.write || !mode(native)) return; + const replacements = new Map( + capabilities.map((capability) => [ + capability, + replayVoiceCapability(store, capability, gate), + ]), + ); + const evaluate = (override: boolean) => { + let remaining = 512; + let hits = 0; + const visiting = new Set(); + const read = (dependency: Atom): unknown => { + if (--remaining < 0) throw new Error("Dictation Settings dependency limit"); + const value = store.get(dependency); + if (replacements.has(dependency)) { + hits += 1; + const projected = replacements.get(dependency); + return override && projected?.isCapable ? projected : value; + } + if (dependency.write || visiting.has(dependency)) return value; + visiting.add(dependency); + try { + return dependency.read(read); + } finally { + visiting.delete(dependency); + } + }; + return { value: read(atom), hits }; + }; + const before = evaluate(false), + after = evaluate(true); + return before.hits > 0 && before.value === native && after.hits > 0 && mode(after.value) + ? after.value + : undefined; + } catch { + return; + } +} + +/** Bind all exact mode readers within Settings, including newly mounted shortcut + * controls. The native mode chooses the OS service used by their save handlers. + */ +export function bindDictationSettingsMode( + currentFibers: () => Set, + store: Store, + atom: Atom, + capabilities: readonly Atom[], + gate: Atom, + allowed: () => boolean, +): { current(): boolean; dispose(): void } { + const readers = () => + [...currentFibers()].flatMap(hooks).filter((hook) => { + const [subscriber, deps] = Array.isArray(hook.memoizedState) ? hook.memoizedState : []; + return ( + record(subscriber) && + typeof subscriber.getSnapshot === "function" && + Array.isArray(deps) && + deps.length === 2 && + deps[0] === store && + deps[1] === atom + ); + }); + const subscriptions = readers(); + if ( + !subscriptions.length || + replayDictationSettingsMode(store, atom, capabilities, gate) === undefined + ) + throw new Error("Native Settings dictation mode unavailable"); + const restores: (() => void)[] = []; + const bindings: { subscriber: Subscriber; instance: Instance }[] = []; + try { + for (const hook of subscriptions) { + const subscriber = (hook.memoizedState as unknown[])[0] as Subscriber; + const instance = hook.next?.queue as Instance | undefined; + const effect = hook.next?.next?.memoizedState; + const native = store.get(atom); + if ( + !instance || + typeof instance.getSnapshot !== "function" || + !record(effect) || + !Array.isArray(effect.deps) || + effect.deps.length !== 1 || + effect.deps[0] !== subscriber.subscribe || + typeof effect.create !== "function" || + subscriber.getSnapshot() !== native || + instance.getSnapshot() !== native || + subscriber.createRender?.() != null + ) + throw new Error("Unsupported Settings dictation mode subscription"); + for (const [object, key] of [ + [store, "sub"], + [subscriber, "getSnapshot"], + [instance, "getSnapshot"], + ] as const) + if (Object.getOwnPropertyDescriptor(object, key)?.writable !== true) + throw new Error("Dictation Settings reader is not writable"); + const sub = store.sub; + let notify: (() => void) | undefined; + try { + store.sub = (selected, listener) => { + if (selected !== atom || notify) + throw new Error("Unexpected dictation Settings subscription"); + notify = listener; + return () => {}; + }; + effect.create(); + } finally { + store.sub = sub; + } + if (!notify) throw new Error("Dictation Settings listener unavailable"); + const original = subscriber.getSnapshot, + originalInstance = instance.getSnapshot; + const snapshot = () => { + if (allowed()) { + const projected = replayDictationSettingsMode(store, atom, capabilities, gate); + if (projected !== undefined) return projected; + } + return original.call(subscriber); + }; + subscriber.getSnapshot = snapshot; + instance.getSnapshot = snapshot; + const onChange = notify; + restores.push(() => { + if (subscriber.getSnapshot === snapshot) subscriber.getSnapshot = original; + if (instance.getSnapshot === snapshot) instance.getSnapshot = originalInstance; + onChange(); + }); + bindings.push({ subscriber, instance }); + onChange(); + } + } catch (error) { + for (const restore of restores.reverse()) restore(); + throw error; + } + return { + current() { + try { + const current = readers(); + return ( + restores.length > 0 && + current.length === bindings.length && + bindings.every(({ subscriber, instance }) => { + const expected = subscriber.getSnapshot(); + return ( + mode(expected) && + instance.getSnapshot() === expected && + current.some( + (hook) => + (hook.memoizedState as unknown[])[0] === subscriber && + hook.next?.queue === instance && + hook.next.memoizedState === expected, + ) + ); + }) + ); + } catch { + return false; + } + }, + dispose() { + for (const restore of restores.splice(0).reverse()) restore(); + }, + }; +} diff --git a/packages/renderer-extension/src/renderer-external-queue.ts b/packages/renderer-extension/src/renderer-external-queue.ts index d7fcd088e..23f9ec1e6 100644 --- a/packages/renderer-extension/src/renderer-external-queue.ts +++ b/packages/renderer-extension/src/renderer-external-queue.ts @@ -3,9 +3,9 @@ function isRecord(value: unknown): value is Record { } /** - * Desktop 26.903.61454 still owns the local follow-up queue and its execution. - * Its serverQueue.isEnabled(threadId) gate must exclude Host-projected external - * Threads: the official app-server queue cannot execute a Harness Turn. + * Desktop 26.1016 overlay-owned External Threads drain through Host + * `thread/queue/*`. Keep the local-queue gate only on Desktops that cannot + * report stream ownership, where follower windows still need that fallback. */ export function installRendererExternalQueue(target: unknown): (() => void) | null { if ( @@ -15,6 +15,7 @@ export function installRendererExternalQueue(target: unknown): (() => void) | nu ) { return null; } + if (typeof target.getStreamRole === "function") return null; const coordinator: unknown = target.getTurnCoordinator(); const queue = isRecord(coordinator) ? coordinator.serverQueue : null; if (!isRecord(queue) || typeof queue.isEnabled !== "function") return null; diff --git a/packages/renderer-extension/src/renderer-external-steering.ts b/packages/renderer-extension/src/renderer-external-steering.ts index 17ae186df..f8f826c05 100644 --- a/packages/renderer-extension/src/renderer-external-steering.ts +++ b/packages/renderer-extension/src/renderer-external-steering.ts @@ -21,6 +21,7 @@ interface SteeringManager { startTurn: RendererMethod; getTurnCoordinator: RendererMethod; getStreamRole?: RendererMethod; + getConversation?: RendererMethod; } function isManager(value: unknown): value is SteeringManager { @@ -32,6 +33,43 @@ function isManager(value: unknown): value is SteeringManager { ); } +function inputText(input: unknown): string { + if (!Array.isArray(input)) return ""; + return input + .filter( + (item): item is Record => isRecord(item) && typeof item.text === "string", + ) + .map((item) => item.text as string) + .join("\n"); +} + +function queuedFollowUpText(message: unknown): string { + if (!isRecord(message)) return ""; + if (typeof message.text === "string" && message.text.trim()) return message.text; + const context = message.context; + if (isRecord(context) && typeof context.prompt === "string" && context.prompt.trim()) { + return context.prompt; + } + return ""; +} + +/** Overlay-owned queued starts can omit type:text input while the local queue still has the prompt. */ +export function restoreQueuedTurnStartText(manager: unknown, params: unknown): unknown { + if (!isManager(manager) || !isRecord(params) || typeof params.threadId !== "string") + return params; + if (inputText(params.input).trim()) return params; + const messageId = params.clientUserMessageId; + if (typeof messageId !== "string" || !messageId) return params; + const coordinator = manager.getTurnCoordinator(); + if (!isRecord(coordinator) || typeof coordinator.readMessages !== "function") return params; + const messages = coordinator.readMessages.call(coordinator, params.threadId); + if (!Array.isArray(messages)) return params; + const match = messages.find((message) => isRecord(message) && message.id === messageId); + const text = queuedFollowUpText(match); + if (!text) return params; + return { ...params, input: [{ type: "text", text, text_elements: [] }] }; +} + function submissionHost(manager: SteeringManager): { getActiveTurnId(threadId: string): unknown; hasPendingTurnStart(threadId: string): unknown; @@ -53,15 +91,17 @@ function submissionHost(manager: SteeringManager): { }; } -type SteeringMethodName = "sendRequest" | "steerTurn"; +type SteeringMethodName = "sendRequest" | "steerTurn" | "getStreamRole"; function installSteeringMethods( manager: SteeringManager, - replacements: Record, + replacements: Partial>, ): (name: SteeringMethodName) => void { const originalPrototype: object | null = Object.getPrototypeOf(manager); const overrides: object = Object.create(originalPrototype); - const names = ["sendRequest", "steerTurn"] as const; + const names = (Object.keys(replacements) as SteeringMethodName[]).filter( + (name) => typeof replacements[name] === "function", + ); const descriptors = new Map( names.map((name) => [name, Object.getOwnPropertyDescriptor(manager, name)]), ); @@ -74,7 +114,7 @@ function installSteeringMethods( configurable: own?.configurable ?? true, enumerable: own?.enumerable ?? false, writable: true, - value: replacements[name], + value: replacements[name] as RendererMethod, }); } if (names.some((name) => !descriptors.get(name))) { @@ -166,6 +206,7 @@ export function installRendererExternalSteering( if ((method === "turn/start" || method === "turn/steer") && prepareTurnInput) { params = prepareTurnInput(params); } + if (method === "turn/start") params = restoreQueuedTurnStartText(manager, params); const messageId = isRecord(params) ? params.clientUserMessageId : null; const route = typeof messageId === "string" && isRecord(params) @@ -294,12 +335,7 @@ export function installRendererExternalSteering( } catch { // Official steering must not depend on our additional presentation binding. } - const text = Array.isArray(input) - ? input - .filter((item) => isRecord(item) && typeof item.text === "string") - .map((item) => (item as Record).text as string) - .join("\n") - : undefined; + const text = Array.isArray(input) ? inputText(input) : undefined; const delivery = await inspectDelivery(threadId, text); if (disposed) throw new Error("External steering binding was disposed"); if (delivery === "official") return originalSteer.apply(manager, args); @@ -407,10 +443,34 @@ export function installRendererExternalSteering( return promise; }; - const restoreMethod = installSteeringMethods(manager, { sendRequest: send, steerTurn: steer }); + const originalRole = manager.getStreamRole; + const getStreamRole: RendererMethod | undefined = + typeof originalRole === "function" + ? (...args: unknown[]) => { + const threadId = args[0]; + if (typeof threadId === "string" && typeof manager.getConversation === "function") { + const conversation: unknown = manager.getConversation(threadId); + if ( + isRecord(conversation) && + conversation.id === threadId && + conversation.modelProvider === "codexhost" + ) { + return { role: "owner" }; + } + } + return originalRole.apply(manager, args); + } + : undefined; + + const restoreMethod = installSteeringMethods(manager, { + sendRequest: send, + steerTurn: steer, + ...(getStreamRole ? { getStreamRole } : {}), + }); return () => { disposed = true; restoreMethod("steerTurn"); + if (getStreamRole) restoreMethod("getStreamRole"); // In-flight starts must fail closed, not accidentally become ordinary start RPCs. if (pending.size === 0) { restoreMethod("sendRequest"); diff --git a/packages/renderer-extension/src/renderer-external-voice-launch.ts b/packages/renderer-extension/src/renderer-external-voice-launch.ts new file mode 100644 index 000000000..fca2740bc --- /dev/null +++ b/packages/renderer-extension/src/renderer-external-voice-launch.ts @@ -0,0 +1,184 @@ +import { selectVoiceAssets, supportsVoiceVersion } from "./renderer-voice-assets.js"; +import { startNativeChatVoiceForRequest } from "./renderer-native-chat-voice.js"; +import { + CLIENT_OWNED_VOICE_ASSETS, + rememberVoiceProductExperience, +} from "./renderer-client-owned-voice.js"; +import type { RendererHostRouting } from "@codexhost/desktop-control/renderer-bindings"; +import { + AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + loadedAssetNames, + readCpaLiveVoiceHookConfig, +} from "./renderer-cpa-live-voice-hook.js"; + +const HOOK_GLOBAL = "__codexhostExternalVoiceLaunchV1"; +const SHARED_ASSET = "app-shared-8bd9d0399038.js"; +const LAUNCH_ASSETS = [ + ...CLIENT_OWNED_VOICE_ASSETS, + { + platform: "darwin", + version: AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + initial: AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + shared: SHARED_ASSET, + services: "fN", + }, +]; +type RecordValue = Record; + +function isRecord(value: unknown): value is RecordValue { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** The native overlay creates its Thread from collaborationMode.settings.model. + * Capture the primary Composer's explicit selection before crossing that boundary. + */ +export function routeExternalVoiceLaunch(request: unknown, routing?: RendererHostRouting): unknown { + rememberVoiceProductExperience(request); + if (!routing || !isRecord(request) || request.source !== "composer_button_new_thread") + return request; + if (request.productExperience === "chat") return request; + const hostId = request.hostId ?? "local"; + if (typeof hostId !== "string" || !hostId) return request; + const route = routing.forComposer(); + if (!route || route.hostId !== hostId || routing.forHost(hostId) !== route) return request; + const model = route.policy.selectedModel(); + if (!model?.startsWith("codexhost/")) return request; + const mode = request.activeCollaborationMode; + if (mode !== undefined && mode !== null && !isRecord(mode)) return request; + const settings = isRecord(mode) ? mode.settings : undefined; + if (settings !== undefined && settings !== null && !isRecord(settings)) return request; + return { + ...request, + activeCollaborationMode: { + ...(isRecord(mode) ? mode : { mode: "default" }), + settings: { + reasoning_effort: null, + developer_instructions: null, + ...(isRecord(settings) ? settings : {}), + model, + }, + }, + }; +} + +interface LaunchHook { + status: "pending" | "active" | "unsupported" | "disabled"; + dispose(): void; +} + +interface LaunchTarget { + [key: string]: unknown; + document?: ParentNode; + performance?: Pick; + setTimeout: typeof globalThis.setTimeout; + clearTimeout: typeof globalThis.clearTimeout; + __codexhostHostRoutingV1?: RendererHostRouting; +} + +interface InstallOptions { + target?: LaunchTarget; + importModule?: (url: string) => Promise; +} + +/** Audited browser module export fN is the native services record. Individual + * service RPC stubs reject mutation; replace only the voice service reference + * with a forwarding Proxy. No Node/Electron APIs or RPC wire interception. + */ +export function installExternalVoiceLaunchHook(options: InstallOptions = {}): LaunchHook { + const target = options.target ?? (globalThis as unknown as LaunchTarget); + const existing = target[HOOK_GLOBAL] as LaunchHook | undefined; + if (existing) return existing; + const config = readCpaLiveVoiceHookConfig(target); + if (!config) return { status: "disabled", dispose() {} }; + if ( + config.auditedDesktopVersion !== AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION || + config.auditedAppInitialAsset !== AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET || + (config.desktopVersion !== undefined && + config.desktopVersion !== config.auditedDesktopVersion && + !supportsVoiceVersion(CLIENT_OWNED_VOICE_ASSETS, config.desktopVersion)) + ) + return { status: "unsupported", dispose() {} }; + const importModule: (url: string) => Promise = + options.importModule ?? ((url: string) => import(url)); + let disposed = false; + let timer: ReturnType | undefined; + let restore: (() => void) | undefined; + let module: RecordValue | undefined; + const started = Date.now(); + const handle: LaunchHook = { + status: "pending", + dispose() { + disposed = true; + if (timer !== undefined) target.clearTimeout(timer); + restore?.(); + if (target[HOOK_GLOBAL] === handle) Reflect.deleteProperty(target, HOOK_GLOBAL); + }, + }; + target[HOOK_GLOBAL] = handle; + const attempt = async (): Promise => { + if (disposed) return; + try { + const names = loadedAssetNames(target.document, target.performance); + const assets = selectVoiceAssets( + LAUNCH_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ); + if (assets) { + module ??= await importModule(`app://-/assets/${assets.shared}`); + if (disposed) return; + const services = module?.[assets.services]; + if (isRecord(services)) { + const original = services.realtimeVoiceRuntime; + if (typeof original === "function" || isRecord(original)) { + const start: unknown = Reflect.get(original, "requestRealtimeStart"); + if (typeof start !== "function") { + handle.status = "unsupported"; + return; + } + const launch = (...args: unknown[]): unknown => { + let request = args[0]; + const chatStart = startNativeChatVoiceForRequest(request, target); + if (chatStart) return chatStart; + try { + request = routeExternalVoiceLaunch(request, target.__codexhostHostRoutingV1); + } catch { + // Retired or disconnected owners cannot claim native traffic. + } + return Reflect.apply(start, original, [request, ...args.slice(1)]); + }; + const wrapped = new Proxy(original, { + get(service, key, receiver) { + return key === "requestRealtimeStart" + ? launch + : Reflect.get(service, key, receiver); + }, + }); + if (!Reflect.set(services, "realtimeVoiceRuntime", wrapped)) { + handle.status = "unsupported"; + return; + } + restore = () => { + if (services.realtimeVoiceRuntime === wrapped) + services.realtimeVoiceRuntime = original; + }; + handle.status = "active"; + return; + } + } + } + } catch { + handle.status = "unsupported"; + return; + } + if (Date.now() - started >= 60_000) { + handle.status = "unsupported"; + return; + } + timer = target.setTimeout(() => void attempt(), 250); + }; + void attempt(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-model-client.ts b/packages/renderer-extension/src/renderer-model-client.ts index 3f5d8fb76..da380d66b 100644 --- a/packages/renderer-extension/src/renderer-model-client.ts +++ b/packages/renderer-extension/src/renderer-model-client.ts @@ -26,6 +26,12 @@ import { type ConsoleOpenResult, HARNESS_LAUNCH_SETTINGS_GET_METHOD, HARNESS_LAUNCH_SETTINGS_SET_METHOD, + GROK_COMPUTER_USE_GET_METHOD, + GROK_COMPUTER_USE_SET_METHOD, + grokComputerUseSetSchema, + grokComputerUseSettingsSchema, + type GrokComputerUseSet, + type GrokComputerUseSettings, harnessLaunchSettingsGetSchema, harnessLaunchSettingsSetSchema, harnessLaunchSettingsSchema, @@ -206,6 +212,8 @@ export interface RendererModelClient extends Partial; getHarnessLaunchSettings?(input: HarnessLaunchSettingsGet): Promise; setHarnessLaunchSettings?(input: HarnessLaunchSettingsSet): Promise; + getGrokComputerUse?(): Promise; + setGrokComputerUse?(update: GrokComputerUseSet): Promise; setIdleReleaseSettings?(settings: IdleReleaseSettings): Promise; listLoadedSessions?(): Promise; currentHostId?(composer?: Element): string | null; @@ -419,6 +427,19 @@ export function createRendererModelClient( ), ); }, + async getGrokComputerUse(): Promise { + return grokComputerUseSettingsSchema.parse( + await manager.sendRequest(GROK_COMPUTER_USE_GET_METHOD, {}), + ); + }, + async setGrokComputerUse(update: GrokComputerUseSet): Promise { + return grokComputerUseSettingsSchema.parse( + await manager.sendRequest( + GROK_COMPUTER_USE_SET_METHOD, + grokComputerUseSetSchema.parse(update), + ), + ); + }, async listLoadedSessions(): Promise { return loadedSessionsSchema.parse(await manager.sendRequest(LOADED_SESSIONS_METHOD, {})); }, diff --git a/packages/renderer-extension/src/renderer-native-chat-voice-assets.ts b/packages/renderer-extension/src/renderer-native-chat-voice-assets.ts new file mode 100644 index 000000000..faa3416d7 --- /dev/null +++ b/packages/renderer-extension/src/renderer-native-chat-voice-assets.ts @@ -0,0 +1,111 @@ +type NativeModule = Record; +export const NATIVE_CHAT_VOICE_VERSION = "26.1021.31345"; +const NATIVE_CHAT_INITIAL_ASSET = "app-initial-feb30b196d11.js"; +const START_ASSET = "entity-business-profile-reference-b68063dfc05d.js"; + +export const NATIVE_CHAT_VOICE_ASSETS = [ + { + platform: "darwin", + version: NATIVE_CHAT_VOICE_VERSION, + initial: NATIVE_CHAT_INITIAL_ASSET, + shared: "app-shared-2cae01273bae.js", + primary: "app-primary-7d7313419f84.js", + start: START_ASSET, + prewarm: "voice-prewarm-d672f1245523.js", + settings: "voice-settings-14000f5c9fc2.js", + }, + { + platform: "win32", + version: NATIVE_CHAT_VOICE_VERSION, + initial: "app-initial-438d1df57f70.js", + shared: "app-shared-955d52676f9c.js", + primary: "app-primary-844e405f2b90.js", + start: "entity-business-profile-reference-241c52d811f4.js", + prewarm: "voice-prewarm-8b47c50a17ae.js", + settings: "voice-settings-b1c1633136a5.js", + }, + { + platform: "darwin", + version: "26.1016.62342", + initial: "app-initial-dddef47d6ef6.js", + shared: "app-shared-fc3316d48427.js", + start: "start-voice-7cbf0a39d53b.js", + prewarm: "voice-prewarm-5d107129ac1c.js", + }, + { + platform: "darwin", + version: "26.1016.61520", + initial: "app-initial-c95ddb62696e.js", + shared: "app-shared-2eda13eb7044.js", + start: "start-voice-18b08f0acdd7.js", + prewarm: "voice-prewarm-94271e581c1b.js", + }, +]; + +/** Normalize audited exports to the baseline Chat lifecycle contract. + * The full composer START can live in a chunk unrelated to its public filename. + * The starter-prompt entry has a different signature and cannot substitute for it. + */ +export async function loadNativeChatVoiceModules( + assets: (typeof NATIVE_CHAT_VOICE_ASSETS)[number], + importModule: (url: string) => Promise, +): Promise<{ + initial: NativeModule; + shared: NativeModule; + start: NativeModule; + prewarm: NativeModule; +}> { + const [sharedModule, initialModule, startModule, prewarm, primary] = await Promise.all( + [assets.shared, assets.initial, assets.start, assets.prewarm, assets.primary].map((asset) => + asset ? importModule(`app://-/assets/${asset}`) : Promise.resolve(undefined), + ), + ); + if (!sharedModule || !initialModule || !startModule || !prewarm) throw new Error(); + const current = assets.version === NATIVE_CHAT_VOICE_VERSION; + const initialize = current ? primary?.um : initialModule.oT; + const start = current ? { t: startModule.r, n: startModule.i } : startModule; + if ( + typeof start.t !== "function" || + typeof start.n !== "function" || + typeof initialize !== "function" || + typeof (current ? initialModule.qXt : initialModule.SPn) !== "function" || + typeof (current ? sharedModule.XVt : sharedModule.FBt) !== "function" || + typeof (current ? sharedModule.KNt : sharedModule.PNt) !== "function" || + typeof prewarm.VoiceInstantConnectCacheController !== "function" + ) + throw new Error(); + initialize(); + start.t(); + // Initializers publish mutable native exports, including the enable flag. + const initial = current + ? { + MOr: initialModule.CQn, + CDr: initialModule.PXn, + B1n: initialModule.tdn, + SPn: initialModule.qXt, + oT: primary?.um, + aT: primary?.lm, + Ycr: initialModule.Xkn, + Nir: initialModule.yAn, + n0n: initialModule.ydn, + Y1n: initialModule.fdn, + HJt: initialModule.R2, + } + : initialModule; + const shared = current + ? { Qst: sharedModule.Bst, FBt: sharedModule.XVt, PNt: sharedModule.KNt, Vot: sharedModule.Fot } + : sharedModule; + const scope = (value: unknown, brand: string) => + value !== null && + typeof value === "object" && + (value as { __scopeBrand?: unknown }).__scopeBrand === brand; + if ( + !scope(shared.Qst, "AppScope") || + !scope(initial.MOr, "RouteScope") || + !scope(initial.CDr, "ComposerScope") || + !initial.B1n || + !initial.aT + ) + throw new Error(); + return { initial, shared, start, prewarm }; +} diff --git a/packages/renderer-extension/src/renderer-native-chat-voice-presentation.ts b/packages/renderer-extension/src/renderer-native-chat-voice-presentation.ts new file mode 100644 index 000000000..4835ccdee --- /dev/null +++ b/packages/renderer-extension/src/renderer-native-chat-voice-presentation.ts @@ -0,0 +1,40 @@ +/** Match native voice's gap-2 action row without doubling an existing flex gap. */ +export function alignNativeChatVoiceButton( + button: HTMLButtonElement, + send: HTMLButtonElement, + doc: Document, +): void { + const parent = send.parentElement; + if (!parent) return; + const layout = doc.defaultView?.getComputedStyle(parent); + const gap = + layout && /flex|grid/.test(layout.display) ? Number.parseFloat(layout.columnGap) || 0 : 0; + const margin = `max(0px, calc(var(--spacing, 0.25rem) * 2 - ${gap}px))`; + if (button.style.marginInlineEnd !== margin) button.style.marginInlineEnd = margin; + if (button.style.alignSelf !== "center") button.style.alignSelf = "center"; +} + +/** Preserve home layout while the native orb occupies the greeting's space. */ +export function hideNativeChatVoiceHomeHeadline(doc: Document): (() => void) | undefined { + const headlines = [ + ...doc.querySelectorAll('[style*="--vt-splash-screen-headline"]'), + ].filter( + (element) => + element.querySelector("h1 [data-headline]") && + element.getBoundingClientRect().width > 0 && + !element.closest("[hidden], [inert]"), + ); + if (headlines.length !== 1) return; + const headline = headlines[0]; + if (!headline) return; + const previous = headline.style.getPropertyValue("visibility"); + const priority = headline.style.getPropertyPriority("visibility"); + headline.style.setProperty("visibility", "hidden", "important"); + return () => { + if ( + headline.style.getPropertyValue("visibility") === "hidden" && + headline.style.getPropertyPriority("visibility") === "important" + ) + headline.style.setProperty("visibility", previous, priority); + }; +} diff --git a/packages/renderer-extension/src/renderer-native-chat-voice-selection.ts b/packages/renderer-extension/src/renderer-native-chat-voice-selection.ts new file mode 100644 index 000000000..26da6bb62 --- /dev/null +++ b/packages/renderer-extension/src/renderer-native-chat-voice-selection.ts @@ -0,0 +1,122 @@ +import { chatVoiceDebug } from "./renderer-chat-voice-debug.js"; +import type { NativeChatVoiceBinding } from "./renderer-native-chat-voice.js"; + +type NativeModule = Record; +const UUID = /^[a-f\d]{8}(?:-[a-f\d]{4}){3}-[a-f\d]{12}$/i; + +/** Select an adopted draft through the native page, which owns its canonical redirect. + * An early voice server ID does not prove that the transcript store owns that Chat. + */ +export function nativeChatVoiceDraftSelection( + binding: NativeChatVoiceBinding, + initial: NativeModule, + shared: NativeModule, + origin: unknown, + signal: AbortSignal, +): () => void { + const draft = binding.props.conversationId; + const navigate = initial.Nir; + const conversationPath = shared.Vot; + if ( + binding.backendConversationId !== undefined || + !draft.startsWith("local-chatgpt:") || + !UUID.test(draft.slice("local-chatgpt:".length)) || + origin == null || + initial.Ycr == null || + initial.n0n == null || + initial.Y1n == null || + typeof navigate !== "function" || + typeof conversationPath !== "function" + ) + return () => {}; + const store = Reflect.apply(initial.SPn as (...args: unknown[]) => unknown, undefined, [ + binding.scope, + ]) as { + getState(): { + isVoiceModeActive: boolean; + isWorkMode: boolean; + conversationId: unknown; + voiceSessionId: unknown; + hasVoiceSessionStarted: boolean; + server: { connectionState: unknown }; + }; + }; + const stateAtStart = store.getState(); + const readIntentVersion = + typeof initial.HJt === "function" + ? () => + Reflect.apply(initial.HJt as (...args: unknown[]) => unknown, undefined, [binding.scope]) + : undefined; + const version = readIntentVersion?.(); + const intentVersion = + typeof version === "number" && Number.isSafeInteger(version) && version > 0 + ? version + : undefined; + let session = stateAtStart.voiceSessionId; + let activeSeen = stateAtStart.isVoiceModeActive; + let sessionStartedSeen = stateAtStart.hasVoiceSessionStarted; + let finished = false; + return () => { + if (finished) return; + try { + const state = store.getState(); + if ( + signal.aborted || + binding.scope.get(initial.Ycr) !== origin || + (intentVersion !== undefined && readIntentVersion?.() !== intentVersion) || + (activeSeen && + (!state.isVoiceModeActive || + (state.voiceSessionId !== session && + (sessionStartedSeen || intentVersion === undefined)))) + ) { + finished = true; + return; + } + if (!state.isVoiceModeActive) return; + if (!activeSeen) { + session = state.voiceSessionId; + activeSeen = true; + } + if (session !== state.voiceSessionId) { + // Native prewarm adopts its cached ID within the same voice intent before + // session startup. A new START increments HJt; established IDs stay fixed. + session = state.voiceSessionId; + chatVoiceDebug("native-chat-selection-startup-id-adopted"); + } + sessionStartedSeen ||= state.hasVoiceSessionStarted; + if (state.isWorkMode !== false || state.conversationId !== draft) { + finished = true; + return; + } + if (!state.hasVoiceSessionStarted || state.server.connectionState !== "connected") return; + const backend = binding.scope.get(initial.B1n, draft); + if (typeof backend !== "string" || !UUID.test(backend)) return; + const mapping = binding.scope.get(initial.n0n, draft); + const currentNode = binding.scope.get(initial.Y1n, draft); + if ( + mapping == null || + typeof mapping !== "object" || + typeof currentNode !== "string" || + !(mapping as Record)[currentNode]?.message + ) + return; + // The voice navigation guard accepts its client draft. The page resolves the + // adopted alias from the same store; never route using earlyServerThreadId. + finished = true; + const path = Reflect.apply(conversationPath, undefined, [draft]); + const accepted = Reflect.apply(navigate, undefined, [binding.scope, path]); + void Promise.resolve(accepted).then( + (result) => + chatVoiceDebug( + result === false + ? "native-chat-selection-declined" + : "native-chat-draft-selection-requested", + ), + () => chatVoiceDebug("native-chat-selection-failed"), + ); + } catch { + finished = true; + chatVoiceDebug("native-chat-selection-failed"); + } + }; +} diff --git a/packages/renderer-extension/src/renderer-native-chat-voice-state.ts b/packages/renderer-extension/src/renderer-native-chat-voice-state.ts new file mode 100644 index 000000000..9ec6fdbae --- /dev/null +++ b/packages/renderer-extension/src/renderer-native-chat-voice-state.ts @@ -0,0 +1,48 @@ +import { chatVoiceDebug } from "./renderer-chat-voice-debug.js"; +import type { ChatVoiceScope } from "./renderer-native-chat-voice.js"; + +/** Observe only audited lifecycle flags; native errors and session payloads stay private. */ +export function observeNativeChatVoiceState( + scope: ChatVoiceScope, + initial: Record, +): () => void { + const store = Reflect.apply(initial.SPn as (...args: unknown[]) => unknown, undefined, [ + scope, + ]) as { + getState(): { + isVoiceModeActive?: boolean; + hasConnectedOnce?: boolean; + hasVoiceSessionStarted?: boolean; + earlyServerThreadId?: unknown; + server?: { connectionState?: unknown }; + }; + subscribe(listener: () => void): () => void; + }; + let last = ""; + const report = () => { + const state = store.getState(); + const connection = state.server?.connectionState; + const detail = { + active: state.isVoiceModeActive === true, + connectedOnce: state.hasConnectedOnce === true, + sessionStarted: state.hasVoiceSessionStarted === true, + serverConversationReady: + typeof state.earlyServerThreadId === "string" && + /^[a-f\d]{8}(?:-[a-f\d]{4}){3}-[a-f\d]{12}$/i.test(state.earlyServerThreadId), + connection: + typeof connection === "string" && + ["disconnected", "connecting", "connected", "reconnecting", "signalReconnecting"].includes( + connection, + ) + ? connection + : "unknown", + }; + const next = JSON.stringify(detail); + if (next === last) return; + last = next; + chatVoiceDebug("native-chat-state", detail); + }; + const unsubscribe = store.subscribe(report); + report(); + return unsubscribe; +} diff --git a/packages/renderer-extension/src/renderer-native-chat-voice.ts b/packages/renderer-extension/src/renderer-native-chat-voice.ts new file mode 100644 index 000000000..6d526158d --- /dev/null +++ b/packages/renderer-extension/src/renderer-native-chat-voice.ts @@ -0,0 +1,534 @@ +import { selectVoiceAssets, supportsVoiceVersion } from "./renderer-voice-assets.js"; +import { + NATIVE_CHAT_VOICE_ASSETS, + loadNativeChatVoiceModules, +} from "./renderer-native-chat-voice-assets.js"; +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; +import { chatVoiceDebug } from "./renderer-chat-voice-debug.js"; +import { loadedAssetNames, readCpaLiveVoiceHookConfig } from "./renderer-cpa-live-voice-hook.js"; +import { nativeChatVoicePreferenceAllowsStart } from "./renderer-cpa-live-voice-preference.js"; +import { observeNativeChatVoiceState } from "./renderer-native-chat-voice-state.js"; +import { nativeChatVoiceDraftSelection } from "./renderer-native-chat-voice-selection.js"; +import { + alignNativeChatVoiceButton, + hideNativeChatVoiceHomeHeadline, +} from "./renderer-native-chat-voice-presentation.js"; + +const HOOK_GLOBAL = "__codexhostNativeChatVoiceV1"; +const BUTTON = "data-codexhost-chat-voice-start"; +const CONVERSATION_UUID = /^[a-f\d]{8}(?:-[a-f\d]{4}){3}-[a-f\d]{12}$/i; +type RecordValue = Record; +const record = (value: unknown): value is RecordValue => + value !== null && typeof value === "object"; + +export interface ChatVoiceScope { + scope: unknown; + node: unknown; + chain: Map; + get(selector: unknown, argument?: unknown): unknown; + set(selector: unknown, value: unknown): void; +} +export interface NativeChatVoiceBinding { + composer: Element; + appNode: object; + props: RecordValue & { conversationId: string }; + scope: ChatVoiceScope; + context: unknown; + composerContext: { context: unknown; chain: Map }; + backendConversationId: string | undefined; +} +interface NativeChatVoiceModules { + scopeDefinition: { id: unknown; __scopeBrand: string }; + ownerScopeDefinition: { id: unknown; __scopeBrand: string }; + backendConversationId: unknown; + composerScopeDefinition: { id: unknown; __scopeBrand: string }; +} + +function scopeRefs(owner: RecordValue, definition: { id: unknown }): Map { + const scopes = new Map(); + let hook = owner.memoizedState; + const seen = new Set(); + while (record(hook) && seen.size < 4096 && !seen.has(hook)) { + seen.add(hook); + const state = hook.memoizedState; + const scope = record(state) ? state.current : undefined; + if ( + record(scope) && + scope.scope === definition && + typeof scope.get === "function" && + typeof scope.set === "function" && + scope.chain instanceof Map && + scope.chain.get(definition.id) === scope.node + ) + scopes.set(scope.node, scope as unknown as ChatVoiceScope); + hook = hook.next; + } + return hook == null ? scopes : new Map(); +} + +/** Resolve the native RouteScope ref and its AppScope/inner ComposerScope chain. + * A Codex locator or Thread-Id is never a Chat ID. + */ +export function detectNativeChatVoiceBinding( + composer: Element, + native: NativeChatVoiceModules, +): NativeChatVoiceBinding | undefined { + const key = Object.getOwnPropertyNames(composer).find((key) => key.startsWith("__reactFiber$")); + if (!key) return; + const ancestors = committedReactAncestors(Object.getOwnPropertyDescriptor(composer, key)?.value); + for (const owner of ancestors) { + const props = owner.memoizedProps; + if ( + !record(props) || + typeof props.isWorkConversation !== "boolean" || + typeof props.isTemporaryChat !== "boolean" || + typeof props.conversationId !== "string" + ) + continue; + if ( + props.isWorkConversation || + props.isTemporaryChat || + props.shouldBlockExternalEgress !== false || + props.readOnly === true || + !record(props.composerSelectionContext) || + props.composerSelectionContext.conversationId !== props.conversationId + ) + return; + const scopes = scopeRefs(owner, native.ownerScopeDefinition); + if (scopes.size !== 1) return; + const scope = [...scopes.values()][0]; + if (!scope) return; + const appNode = scope.chain.get(native.scopeDefinition.id); + if (!record(appNode)) return; + const backendId = scope.get(native.backendConversationId, props.conversationId); + const hasBackendId = typeof backendId === "string" && CONVERSATION_UUID.test(backendId); + // Native drafts have a local locator until voice or a message creates the Chat. + // START accepts an absent backend ID and owns conversation creation. + const isDraft = + backendId == null && + props.conversationId.startsWith("local-chatgpt:") && + CONVERSATION_UUID.test(props.conversationId.slice("local-chatgpt:".length)); + if (!hasBackendId && !isDraft) return; + // The owner creates this provider below itself. Its own subscriptions can + // still consume the outer home/new ComposerScope, which does not own the form. + const provider = ancestors + .slice(0, ancestors.indexOf(owner)) + .find( + (fiber) => + fiber.tag === 10 && + record(fiber.type) && + fiber.type.displayName === native.composerScopeDefinition.__scopeBrand, + ); + if (!provider) return; + const composerChain = record(provider.memoizedProps) ? provider.memoizedProps.value : undefined; + const composerNode = + composerChain instanceof Map + ? composerChain.get(native.composerScopeDefinition.id) + : undefined; + if ( + !(composerChain instanceof Map) || + composerChain.get(native.scopeDefinition.id) !== appNode || + composerChain.get(native.ownerScopeDefinition.id) !== scope.node || + !record(composerNode) || + composerNode.token !== native.composerScopeDefinition || + !record(composerNode.value) || + composerNode.value.kind !== "chatgpt" || + !record(composerNode.value.chatGptSelectionContext) || + composerNode.value.chatGptSelectionContext.conversationId !== props.conversationId + ) + return; + const composerContext = { context: provider.type, chain: composerChain }; + const contexts = new Set(); + for (const fiber of ancestors) { + let dependency = record(fiber.dependencies) ? fiber.dependencies.firstContext : undefined; + const seenDependencies = new Set(); + while ( + record(dependency) && + seenDependencies.size < 4096 && + !seenDependencies.has(dependency) + ) { + seenDependencies.add(dependency); + const context = dependency.context; + const chain = dependency.memoizedValue; + if ( + record(context) && + context.displayName === native.scopeDefinition.__scopeBrand && + chain instanceof Map && + chain.get(native.scopeDefinition.id) === appNode + ) + contexts.add(context); + dependency = dependency.next; + } + } + if (contexts.size !== 1) return; + return { + composer, + appNode, + props: props as NativeChatVoiceBinding["props"], + scope, + context: [...contexts][0], + composerContext, + backendConversationId: hasBackendId ? backendId : undefined, + }; + } +} + +interface ChatVoiceHook { + readonly activeCall?: boolean; + status: "disabled" | "pending" | "active" | "unsupported"; + start(): Promise; + dispose(): void; +} +interface ChatVoiceTarget { + [key: string]: unknown; + document?: Document; + performance?: Pick; + setTimeout: typeof setTimeout; + clearTimeout: typeof clearTimeout; +} + +/** The Chat service request must not enter the Codex new-thread overlay. */ +export function startNativeChatVoiceForRequest( + request: unknown, + target: object = globalThis, +): Promise | undefined { + if ( + !record(request) || + request.productExperience !== "chat" || + !nativeChatVoicePreferenceAllowsStart(target) + ) + return; + const hook = (target as RecordValue)[HOOK_GLOBAL] as ChatVoiceHook | undefined; + if (!hook || hook.status === "disabled" || hook.status === "unsupported") + return Promise.reject(new Error("Native Chat voice is unavailable")); + return hook.start(); +} + +/** Desktop browser voice owns serialization, authentication, WM, events and cleanup. + * aT enables its dormant runtime and native VoiceRoom in the app root. The + * native cache controller is mounted once per AppScope, not once per selection. + */ +export function installNativeChatVoiceHook( + options: { + target?: ChatVoiceTarget; + importModule?: (url: string) => Promise; + } = {}, +): ChatVoiceHook { + const target = options.target ?? (globalThis as unknown as ChatVoiceTarget); + const existing = target[HOOK_GLOBAL] as ChatVoiceHook | undefined; + if (existing) return existing; + const config = readCpaLiveVoiceHookConfig(target); + const unavailable = (status: "disabled" | "unsupported"): ChatVoiceHook => ({ + status, + dispose() {}, + start: async () => { + throw new Error("Native Chat voice is unavailable"); + }, + }); + if (!config) return unavailable("disabled"); + if (!supportsVoiceVersion(NATIVE_CHAT_VOICE_ASSETS, config.desktopVersion)) + return unavailable("unsupported"); + const doc = target.document; + if (!doc) return unavailable("unsupported"); + const importModule = options.importModule ?? ((url: string) => import(url)); + let disposed = false; + let timer: ReturnType | undefined; + let modules: + | { initial: RecordValue; shared: RecordValue; start: RecordValue; prewarm: RecordValue } + | undefined; + let boundScope: ChatVoiceScope | undefined; + let mounted: + { node: unknown; render(binding: NativeChatVoiceBinding): void; dispose(): void } | undefined; + let button: HTMLButtonElement | undefined; + const overlayButtons = new Map(); + let controller: AbortController | undefined; + let starting = false; + let selectDraft: (() => void) | undefined; + let restoreHeadline: (() => void) | undefined; + let renderFailed = false; + const started = Date.now(); + const showHeadline = () => { + restoreHeadline?.(); + restoreHeadline = undefined; + }; + const removeButton = () => { + button?.remove(); + button = undefined; + for (const [control, [display, priority]] of overlayButtons) + if ( + control.style.getPropertyValue("display") === "none" && + control.style.getPropertyPriority("display") === "important" + ) + control.style.setProperty("display", display, priority); + overlayButtons.clear(); + }; + const release = () => { + showHeadline(); + selectDraft = undefined; + controller?.abort(); + controller = undefined; + removeButton(); + mounted?.dispose(); + mounted = undefined; + boundScope = undefined; + }; + const findBinding = () => { + if (!modules || !nativeChatVoicePreferenceAllowsStart(target)) return; + const forms = [...doc.querySelectorAll("[data-chatgpt-composer]")].filter( + (form) => form.getBoundingClientRect().width > 0 && !form.closest("[hidden], [inert]"), + ); + if (forms.length !== 1) return; + const form = forms[0]; + if (!form) return; + return detectNativeChatVoiceBinding(form, { + scopeDefinition: modules.shared.Qst as NativeChatVoiceModules["scopeDefinition"], + ownerScopeDefinition: modules.initial.MOr as NativeChatVoiceModules["ownerScopeDefinition"], + backendConversationId: modules.initial.B1n, + composerScopeDefinition: modules.initial + .CDr as NativeChatVoiceModules["composerScopeDefinition"], + }); + }; + const handle: ChatVoiceHook = { + get activeCall() { + if (starting) return true; + if (!boundScope || !modules || typeof modules.initial.SPn !== "function") return false; + try { + return ( + modules.initial.SPn(boundScope) as { getState(): { isVoiceModeActive: boolean } } + ).getState().isVoiceModeActive; + } catch { + return false; + } + }, + status: "pending", + async start() { + if (disposed || starting) return; + const current = findBinding(); + if (!current || !modules || current.appNode !== mounted?.node) + throw new Error("Select a ChatGPT conversation to start Chat voice"); + starting = true; + controller = new AbortController(); + const signal = controller.signal; + let origin: unknown; + try { + if (modules.initial.Ycr != null) origin = current.scope.get(modules.initial.Ycr); + } catch { + chatVoiceDebug("native-chat-selection-failed"); + } + selectDraft = undefined; + chatVoiceDebug("native-chat-start", { + hasConversationId: current.backendConversationId !== undefined, + }); + try { + // Native start validates availability, navigation, active sessions and START/RESTART + // commands, then reads conversation_id via B1n before dispatching START. + await Reflect.apply(modules.start.n as (...args: unknown[]) => unknown, undefined, [ + current.scope, + current.props, + { event_source: "composer" }, + signal, + ]); + chatVoiceDebug("native-chat-start-returned"); + try { + selectDraft = nativeChatVoiceDraftSelection( + current, + modules.initial, + modules.shared, + origin, + signal, + ); + selectDraft(); + } catch { + chatVoiceDebug("native-chat-selection-failed"); + } + } catch { + chatVoiceDebug("native-chat-start-failed"); + throw new Error("Native Chat voice could not start"); + } finally { + starting = false; + } + }, + dispose() { + disposed = true; + if (timer !== undefined) target.clearTimeout(timer); + release(); + if (target[HOOK_GLOBAL] === handle) Reflect.deleteProperty(target, HOOK_GLOBAL); + }, + }; + target[HOOK_GLOBAL] = handle; + const refresh = async (): Promise => { + if (disposed) return; + try { + if (renderFailed) throw new Error(); + if (!modules) { + const names = loadedAssetNames(doc, target.performance); + const assets = selectVoiceAssets( + NATIVE_CHAT_VOICE_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ); + if (!assets) { + if (Date.now() - started >= 60_000) { + handle.status = "unsupported"; + return; + } + return; + } + const loaded = await loadNativeChatVoiceModules(assets, importModule); + if (disposed) return; + modules = loaded; + } + const current = findBinding(); + selectDraft?.(); + if (!current) { + showHeadline(); + if (handle.activeCall) return; + release(); + handle.status = "pending"; + return; + } + if (mounted && current.appNode !== mounted.node) release(); + boundScope = current.scope; + if (!mounted) { + const { scope } = current; + const initial = modules.initial; + const previous = scope.get(initial.aT); + const container = doc.createElement("div"); + container.setAttribute("data-codexhost-chat-voice-prewarm", ""); + doc.body.append(container); + const react = Reflect.apply(modules.shared.FBt as () => unknown, undefined, []) as { + createElement(type: unknown, props: unknown, ...children: unknown[]): unknown; + }; + const reactDom = Reflect.apply(modules.shared.PNt as () => unknown, undefined, []) as { + createRoot( + container: Element, + options: { onUncaughtError(): void }, + ): { render(element: unknown): void; unmount(): void }; + }; + const root = reactDom.createRoot(container, { + onUncaughtError() { + renderFailed = true; + }, + }); + const stopObserving = observeNativeChatVoiceState(scope, initial); + let appChain: Map | undefined; + let composerChain: Map | undefined; + mounted = { + node: current.appNode, + render(next) { + if (appChain === next.scope.chain && composerChain === next.composerContext.chain) + return; + appChain = next.scope.chain; + composerChain = next.composerContext.chain; + root.render( + react.createElement( + next.context, + { value: appChain }, + react.createElement( + next.composerContext.context, + { value: composerChain }, + react.createElement(modules?.prewarm.VoiceInstantConnectCacheController, null), + ), + ), + ); + }, + dispose() { + stopObserving(); + root.unmount(); + container.remove(); + if (scope.get(initial.aT) === true) scope.set(initial.aT, previous); + chatVoiceDebug("native-chat-unbound"); + }, + }; + scope.set(initial.aT, true); + chatVoiceDebug("native-chat-bound", { + hasConversationId: current.backendConversationId !== undefined, + }); + } + mounted.render(current); + const voiceState = ( + Reflect.apply(modules.initial.SPn as (...args: unknown[]) => unknown, undefined, [ + current.scope, + ]) as { + getState(): { + isVoiceModeActive: boolean; + isWorkMode: boolean; + conversationId: unknown; + }; + } + ).getState(); + if ( + current.props.conversationId.startsWith("local-chatgpt:") && + voiceState.isVoiceModeActive && + voiceState.isWorkMode === false && + voiceState.conversationId === current.props.conversationId + ) + restoreHeadline ??= hideNativeChatVoiceHomeHeadline(doc); + else showHeadline(); + const sends = [ + ...current.composer.querySelectorAll('button[type="submit"]'), + ].filter( + (control) => + control.getBoundingClientRect().width > 0 && !control.closest("[hidden], [inert]"), + ); + const send = sends.length === 1 ? sends[0] : undefined; + if (!send?.parentElement) removeButton(); + else { + if (!button || !current.composer.contains(button)) { + removeButton(); + button = doc.createElement("button"); + button.type = "button"; + button.setAttribute(BUTTON, ""); + button.setAttribute("aria-label", "Start Voice"); + button.title = "Start Voice"; + button.innerHTML = + ''; + button.addEventListener("click", () => { + void handle.start().catch(() => { + if (button) button.title = "Chat voice could not start. Try again."; + }); + }); + } + if (button.nextElementSibling !== send) send.parentElement.insertBefore(button, send); + alignNativeChatVoiceButton(button, send, doc); + } + if (button) + // The Codex entry opens onboarding/optimistic launch state before its + // service RPC. Only the native Chat START control belongs in this composer. + for (const control of current.composer.querySelectorAll( + 'button[aria-label="Start new voice chat"], button[aria-label="Start voice chat"]', + )) { + if (!overlayButtons.has(control)) + overlayButtons.set(control, [ + control.style.getPropertyValue("display"), + control.style.getPropertyPriority("display"), + ]); + control.style.setProperty("display", "none", "important"); + } + if (button && send) { + button.disabled = + starting || current.props.isStreaming === true || voiceState.isVoiceModeActive; + // Empty Send is disabled independently of whether voice can start. + const className = [ + ...send.className + .split(/\s+/) + .filter((name) => name !== "opacity-50" && name !== "cursor-default"), + ...(button.disabled ? ["opacity-50", "cursor-default"] : []), + ].join(" "); + if (button.className !== className) button.className = className; + } + handle.status = "active"; + } catch { + release(); + handle.status = "unsupported"; + chatVoiceDebug("native-chat-binding-failed"); + return; + } finally { + if (!disposed && handle.status !== "unsupported") + timer = target.setTimeout(() => void refresh(), 500); + } + }; + void refresh(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-overlay-external-followups.ts b/packages/renderer-extension/src/renderer-overlay-external-followups.ts new file mode 100644 index 000000000..5528d18b2 --- /dev/null +++ b/packages/renderer-extension/src/renderer-overlay-external-followups.ts @@ -0,0 +1,119 @@ +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; + +import { installRendererExternalQueue } from "./renderer-external-queue.js"; +import { installRendererExternalSteering } from "./renderer-external-steering.js"; +import { isAvatarOverlayHref } from "./renderer-overlay-thread-isolation.js"; + +const DISCOVERY_MS = 60_000; +const INTERVAL_MS = 50; +const HOOK_WALK_LIMIT = 120; + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function isFollowUpManager(value: unknown): value is object { + return ( + isRecord(value) && + ["sendRequest", "steerTurn", "startTurn", "getTurnCoordinator", "getConversation"].every( + (key) => typeof value[key] === "function", + ) + ); +} + +function managerFromHookState(value: unknown): object | null { + if (isFollowUpManager(value)) return value; + return isRecord(value) && isFollowUpManager(value.manager) ? value.manager : null; +} + +function overlayFollowUpManagers(document: Document): object[] { + const managers = new Set(); + const surfaces = [...document.querySelectorAll("[data-avatar-overlay-native-surface-id]")]; + const roots = surfaces.length > 0 ? surfaces : [document.documentElement]; + for (const element of roots) { + const key = Object.getOwnPropertyNames(element).find((name) => + name.startsWith("__reactFiber$"), + ); + const fiber = key ? Object.getOwnPropertyDescriptor(element, key)?.value : undefined; + for (const current of committedReactAncestors(fiber)) { + let hook = current.memoizedState as { memoizedState?: unknown; next?: unknown } | null; + for (let index = 0; hook && index < HOOK_WALK_LIMIT; index += 1) { + const manager = managerFromHookState(hook.memoizedState); + if (manager) managers.add(manager); + hook = hook.next && typeof hook.next === "object" ? (hook.next as typeof hook) : null; + } + } + } + return [...managers]; +} + +export type OverlayExternalFollowUpsStatus = "disabled" | "pending" | "active"; + +export interface OverlayExternalFollowUpsHandle { + status: OverlayExternalFollowUpsStatus; + dispose(): void; +} + +export interface OverlayExternalFollowUpsTarget { + location?: { href: string }; + document?: Document; + setTimeout: typeof setTimeout; + clearTimeout: typeof clearTimeout; +} + +/** + * Avatar overlay is a persistent Desktop webcontents and can own an External + * Thread's stream. This hook installs the same steer wrappers used on the + * primary Renderer, including stream-owner admission for Host queue drain. + */ +export function installOverlayExternalFollowUps( + target: OverlayExternalFollowUpsTarget | null = typeof globalThis === "undefined" + ? null + : (globalThis as unknown as OverlayExternalFollowUpsTarget), +): OverlayExternalFollowUpsHandle { + const href = target?.location?.href; + if (!target || typeof href !== "string" || !isAvatarOverlayHref(href)) { + return { status: "disabled", dispose() {} }; + } + const wrapped = new WeakSet(); + const cleanups = new Set<() => void>(); + let disposed = false; + let timer: ReturnType | undefined; + const started = Date.now(); + const handle: OverlayExternalFollowUpsHandle = { + status: "pending", + dispose() { + disposed = true; + if (timer !== undefined) target.clearTimeout(timer); + for (const cleanup of cleanups) { + try { + cleanup(); + } catch { + /* Release the remaining hooks too. */ + } + } + cleanups.clear(); + }, + }; + const attempt = (): void => { + if (disposed) return; + const document = target.document; + if (document) { + for (const manager of overlayFollowUpManagers(document)) { + if (wrapped.has(manager)) continue; + const queue = installRendererExternalQueue(manager); + const steering = installRendererExternalSteering(manager); + if (!queue && !steering) continue; + wrapped.add(manager); + if (queue) cleanups.add(queue); + if (steering) cleanups.add(steering); + handle.status = "active"; + } + } + if (disposed || handle.status === "active") return; + if (Date.now() - started >= DISCOVERY_MS) return; + timer = target.setTimeout(attempt, INTERVAL_MS); + }; + attempt(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-overlay-thread-isolation.ts b/packages/renderer-extension/src/renderer-overlay-thread-isolation.ts new file mode 100644 index 000000000..c84616208 --- /dev/null +++ b/packages/renderer-extension/src/renderer-overlay-thread-isolation.ts @@ -0,0 +1,109 @@ +/** Overlay-only: leftover avatar-overlay must not ingest other Threads' live work. */ + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +const BLOCKED_METHODS = new Set(["thread/started"]); + +export function isAvatarOverlayHref(href: string): boolean { + try { + const url = new URL(href); + if (url.pathname === "/avatar-overlay.html") return true; + return url.searchParams.get("initialRoute") === "/avatar-overlay"; + } catch { + return false; + } +} + +export function overlayBlocksHostNotification( + method: string, + threadId: string | undefined, + allowedRealtimeThreadIds: ReadonlySet, +): boolean { + if (method.startsWith("thread/realtime/")) { + return false; + } + if (threadId !== undefined && allowedRealtimeThreadIds.has(threadId)) { + return false; + } + if (BLOCKED_METHODS.has(method)) return true; + return method.startsWith("turn/") || method.startsWith("item/"); +} + +export function notificationThreadId(params: unknown): string | undefined { + if (!isRecord(params)) return undefined; + if (typeof params.threadId === "string" && params.threadId.length > 0) return params.threadId; + const thread = params.thread; + if (isRecord(thread) && typeof thread.id === "string" && thread.id.length > 0) { + return thread.id; + } + return undefined; +} + +export interface OverlayIsolationTarget { + location?: { href: string }; + addEventListener( + type: "message", + listener: (event: OverlayIsolationMessageEvent) => void, + options?: boolean | { capture?: boolean }, + ): void; + removeEventListener( + type: "message", + listener: (event: OverlayIsolationMessageEvent) => void, + options?: boolean | { capture?: boolean }, + ): void; +} + +export interface OverlayIsolationMessageEvent { + data?: unknown; + stopImmediatePropagation(): void; +} + +export interface OverlayIsolationHandle { + status: "disabled" | "active"; + dispose(): void; +} + +/** + * Avatar overlay is a persistent Desktop webcontents. Host notifications are + * posted to every window. Without this filter, a leftover overlay upserts + * CLI-created Threads and Desktop rebinds the same Turn twice. + */ +export function installOverlayThreadIsolation( + target: OverlayIsolationTarget | null = typeof globalThis === "undefined" + ? null + : (globalThis as unknown as OverlayIsolationTarget), +): OverlayIsolationHandle { + if (!target || typeof target.addEventListener !== "function") { + return { status: "disabled", dispose() {} }; + } + const href = target.location?.href; + if (typeof href !== "string" || !isAvatarOverlayHref(href)) { + return { status: "disabled", dispose() {} }; + } + // Native stream ownership can outlive the call. A primary follower still + // receives this Thread's updates through the overlay owner after hangup. + const voiceThreadIds = new Set(); + const listener = (event: OverlayIsolationMessageEvent): void => { + const message = event.data; + if (!isRecord(message) || message.type !== "mcp-notification") return; + if (typeof message.method !== "string") return; + const threadId = notificationThreadId(message.params); + if (message.method === "thread/realtime/started" && threadId) { + voiceThreadIds.add(threadId); + return; + } + if (overlayBlocksHostNotification(message.method, threadId, voiceThreadIds)) { + event.stopImmediatePropagation(); + } + if (message.method === "thread/deleted" && threadId) voiceThreadIds.delete(threadId); + }; + target.addEventListener("message", listener, true); + return { + status: "active", + dispose() { + target.removeEventListener("message", listener, true); + }, + }; +} diff --git a/packages/renderer-extension/src/renderer-settings-lifecycle.ts b/packages/renderer-extension/src/renderer-settings-lifecycle.ts index 8cbef501c..079e6808d 100644 --- a/packages/renderer-extension/src/renderer-settings-lifecycle.ts +++ b/packages/renderer-extension/src/renderer-settings-lifecycle.ts @@ -1,3 +1,4 @@ +import type { VoiceSettingsRequestClient } from "./settings/voice-page.js"; import type { RendererUsageStatisticsRequest } from "./console/pages/usage-statistics.js"; import type { RendererUsageStatisticsHostsProvider } from "./console/usage/usage-hosts.js"; import type { RemoteConnectionsControl } from "./remote-connections-control.js"; @@ -35,6 +36,7 @@ export interface RendererSettingsLifecycleOptions { getLoadedSessionsClient?(): LoadedSessionsClient | null; getRemoteConnections?(): RemoteConnectionsControl | null; requestUsageStatistics?: RendererUsageStatisticsRequest; + requestVoiceSettings?: VoiceSettingsRequestClient; /** Machines the usage statistics page may read besides the local Host. */ usageStatisticsHosts?: RendererUsageStatisticsHostsProvider; openImportedThread?: RendererImportedThreadOpener; @@ -52,7 +54,10 @@ export function installRendererSettingsLifecycle( options: RendererSettingsLifecycleOptions = {}, ): RendererSettingsLifecycleControl { const lifecycleController = new AbortController(); - let locale = resolveRendererSettingsLocale(ownerWindow.navigator.languages); + const documentLanguage = ownerWindow.document?.documentElement?.lang; + let locale = resolveRendererSettingsLocale( + documentLanguage ? [documentLanguage] : ownerWindow.navigator.languages, + ); let shell: RendererSettingsShell | null = null; let trigger: RendererSettingsRailTriggerControl | null = null; let localeRequest: Promise | null = null; @@ -87,6 +92,7 @@ export function installRendererSettingsLifecycle( options.getRemoteConnections ?? (() => null), options.requestUsageStatistics, options.usageStatisticsHosts, + options.requestVoiceSettings, ); const nextShell = installRendererSettingsShell(definitions, messages, ownerWindow.document, { onOpenChange(open) { diff --git a/packages/renderer-extension/src/renderer-voice-assets.ts b/packages/renderer-extension/src/renderer-voice-assets.ts new file mode 100644 index 000000000..310f0abb2 --- /dev/null +++ b/packages/renderer-extension/src/renderer-voice-assets.ts @@ -0,0 +1,40 @@ +import { auditedAppInitialAssetPresent } from "./renderer-cpa-live-voice-hook.js"; + +interface VoiceAssets { + version: string; + platform: string; + initial: string; + shared: string; +} + +export function supportsVoiceVersion(profiles: readonly VoiceAssets[], version?: string): boolean { + return version === undefined || profiles.some((profile) => profile.version === version); +} + +/** Select one complete audited pair; version metadata cannot excuse mixed assets. */ +export function selectVoiceAssets( + profiles: readonly T[], + names: readonly string[], + version?: string, + platform?: string, +): T | undefined { + const present = profiles.filter((profile) => + [profile.initial, profile.shared].some((asset) => auditedAppInitialAssetPresent(names, asset)), + ); + const matchesMetadata = (profile: T) => + (version === undefined || profile.version === version) && + (platform === undefined || profile.platform === platform); + if (present.length > 0) { + const profile = present.length === 1 ? present[0] : undefined; + return profile && + matchesMetadata(profile) && + [profile.initial, profile.shared].every((asset) => + auditedAppInitialAssetPresent(names, asset), + ) + ? profile + : undefined; + } + if (version === undefined) return undefined; + const candidates = profiles.filter(matchesMetadata); + return candidates.length === 1 ? candidates[0] : undefined; +} diff --git a/packages/renderer-extension/src/renderer-voice-capability.ts b/packages/renderer-extension/src/renderer-voice-capability.ts new file mode 100644 index 000000000..7f9799711 --- /dev/null +++ b/packages/renderer-extension/src/renderer-voice-capability.ts @@ -0,0 +1,484 @@ +import { selectVoiceAssets, supportsVoiceVersion } from "./renderer-voice-assets.js"; +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; +import type { RendererHostRouting } from "@codexhost/desktop-control/renderer-bindings"; +import { CLIENT_OWNED_VOICE_ASSETS, isWorkComposerSurface } from "./renderer-client-owned-voice.js"; +import { loadedAssetNames, readCpaLiveVoiceHookConfig } from "./renderer-cpa-live-voice-hook.js"; +import { + chatLiveVoicePreferenceAllowsOverride, + cpaLiveVoicePreferenceAllowsInstall, + voiceGatewayIsReady, +} from "./renderer-cpa-live-voice-preference.js"; + +const GLOBAL = "__codexhostVoiceCapabilityV1"; +export const INFERENCE_GATE = "3683713372"; +type RecordValue = Record; +const record = (value: unknown): value is RecordValue => + value !== null && typeof value === "object"; +type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: unknown }; +type Store = { get(atom: Atom): unknown; sub(atom: Atom, listener: () => void): () => void }; +interface Subscriber { + getSnapshot(): unknown; + subscribe(listener: () => void): () => void; + createRender?(): unknown; +} +interface Signal { + atom: Atom; + store: Store; +} +interface Instance { + getSnapshot(): unknown; +} +interface Hook { + memoizedState?: unknown; + queue?: unknown; + next?: Hook | null; +} +interface ScopeNode { + store: Store; + familyBindings: Map>; +} +interface Family { + scope: { id: unknown }; + resolve(node: ScopeNode, chain: Map, parameter: string): { atom: Atom }; +} +type Capability = { isLoading: boolean; isError: boolean; isCapable: boolean }; +const capable = (value: unknown): value is Capability => + record(value) && + typeof value.isLoading === "boolean" && + typeof value.isError === "boolean" && + typeof value.isCapable === "boolean"; +const equal = (a: Capability, b: Capability) => + a.isLoading === b.isLoading && a.isError === b.isError && a.isCapable === b.isCapable; + +/** Replay readonly dependencies for one owned subscription. The inference rollout + * is omitted only in this evaluation; native atoms, provider config and Statsig + * retain their values. Writable state and self-reading leaves remain native. + * This path never writes `model_provider` and never flips Statsig gate 3683713372. + */ +export function replayVoiceCapability(store: Store, atom: Atom, gate: Atom): Capability | null { + const evaluate = (override: boolean) => { + const visiting = new Set(); + let remaining = 512; + let hits = 0; + const read = (dependency: Atom): unknown => { + if (--remaining < 0) throw new Error("Voice capability dependency limit"); + const value = store.get(dependency); + if (dependency === gate) { + hits += 1; + if (typeof value !== "boolean") throw new Error("Voice inference gate unavailable"); + return override ? false : value; + } + if (dependency.write || visiting.has(dependency)) return value; + if (typeof dependency.read !== "function") throw new Error("Voice dependency unavailable"); + visiting.add(dependency); + try { + return dependency.read(read); + } finally { + visiting.delete(dependency); + } + }; + return { value: read(atom), hits }; + }; + try { + const native = store.get(atom); + if (!capable(native) || native.isLoading || native.isError) return null; + const before = evaluate(false); + if (!capable(before.value) || !equal(native, before.value)) return null; + const after = evaluate(true); + return after.hits > 0 && capable(after.value) && !after.value.isLoading && !after.value.isError + ? after.value + : null; + } catch { + return null; + } +} + +function isFamily(value: unknown): value is Family { + return record(value) && record(value.scope) && typeof value.resolve === "function"; +} + +/** Family.resolve accepts arbitrary keys, so probing unrelated families cannot + * identify the inference gate. Only the selected asset profile owns these exports. + */ +export function initializeVoiceCapabilityFamilies( + module: RecordValue, + assets: { capabilityInit: string; capabilityFamily: string; inferenceGateFamily: string }, +): { family: Family; gates: Family } { + const initialize = module[assets.capabilityInit]; + if (typeof initialize !== "function") throw new Error("Voice capability initializer unavailable"); + initialize(); + const family = module[assets.capabilityFamily]; + const gates = module[assets.inferenceGateFamily]; + if (!isFamily(family) || !isFamily(gates)) + throw new Error("Voice capability families unavailable"); + return { family, gates }; +} + +export function hooks(fiber: RecordValue): Hook[] { + const result: Hook[] = []; + const seen = new Set(); + let hook = fiber.memoizedState; + while (record(hook) && result.length < 4096 && !seen.has(hook)) { + seen.add(hook); + result.push(hook as Hook); + hook = hook.next; + } + return hook == null ? result : []; +} + +function composerFibers(composer: Element, chat = false): Set { + const fibers = new Set(); + const rootKey = Object.getOwnPropertyNames(composer).find((key) => + key.startsWith("__reactFiber$"), + ); + const ancestors = rootKey + ? committedReactAncestors(Object.getOwnPropertyDescriptor(composer, rootKey)?.value) + : []; + let root = ancestors[0]; + if (chat) { + // Chat's native realtime hook lives on the form's owning component. + const owners = ancestors.filter( + ({ memoizedProps: props }) => + record(props) && + props.isWorkConversation === false && + props.isTemporaryChat === false && + props.shouldBlockExternalEgress === false && + props.readOnly !== true && + typeof props.conversationId === "string" && + record(props.composerSelectionContext) && + props.composerSelectionContext.conversationId === props.conversationId, + ); + if (owners.length !== 1) return fibers; + root = owners[0]; + } + if (!root) return fibers; + for (const element of [composer, ...composer.querySelectorAll("*")].slice(0, 2048)) { + const key = Object.getOwnPropertyNames(element).find((key) => key.startsWith("__reactFiber$")); + if (!key) continue; + const ancestors = committedReactAncestors(Object.getOwnPropertyDescriptor(element, key)?.value); + const boundary = ancestors.indexOf(root); + if (boundary < 0) continue; + for (const fiber of ancestors.slice(0, boundary + 1)) if (record(fiber)) fibers.add(fiber); + } + return fibers; +} + +/** The native family memo binds the capability cache key, AppScope chain and + * signal to the same component's useSyncExternalStore subscription. Only local + * voice.realtime / voice.thread members within the Composer qualify. + */ +export function bindComposerVoiceCapability( + composer: Element, + family: Family, + gates: Family, + allowed: () => boolean, + chat = false, +): { dispose(): void; current(): boolean; projectedCapable(): boolean } { + return bindVoiceCapabilitySubscriptions( + () => composerFibers(composer, chat), + family, + gates, + allowed, + ); +} + +/** Subscription ownership is supplied by the version-qualified surface owner. */ +export function bindVoiceCapabilitySubscriptions( + currentFibers: () => Set, + family: Family, + gates: Family, + allowed: () => boolean, + realtimeOnly = false, + names: readonly ( + "voice.realtime" | "voice.thread" | "dictation.composer" | "dictation.global" + )[] = ["voice.realtime", "voice.thread"], +): { dispose(): void; current(): boolean; projectedCapable(): boolean } { + const fibers = currentFibers(); + const restores: (() => void)[] = []; + const bindings: { subscriber: Subscriber; signal: Signal; instance: Instance }[] = []; + try { + for (const fiber of fibers) { + const list = hooks(fiber); + for (const memo of list) { + const [signal, deps] = Array.isArray(memo.memoizedState) ? memo.memoizedState : []; + if ( + !record(signal) || + !Array.isArray(deps) || + deps.length !== 5 || + deps[3] !== family || + !(deps[1] instanceof Map) || + deps[2] !== undefined + ) + continue; + const chain = deps[1] as Map; + const node = chain.get(family.scope.id) as ScopeNode | undefined; + const members = node?.familyBindings?.get(family); + if (!node || !(members instanceof Map) || node.store !== signal.store) continue; + const matches = [...members].filter(([, member]) => member.cacheKey === deps[0]); + if (matches.length !== 1 || typeof matches[0]?.[0] !== "string") continue; + const parameter = JSON.parse(matches[0][0]); + if ( + !record(parameter) || + Object.keys(parameter).some((key) => key !== "name" && key !== "hostId") || + !names.some((name) => parameter.name === name) || + (realtimeOnly && parameter.name !== "voice.realtime") || + (parameter.hostId !== undefined && parameter.hostId !== "local") || + (parameter.name === "voice.thread" && parameter.hostId !== "local") + ) + continue; + const subscriptions = list.filter((hook) => { + const state = hook.memoizedState; + return ( + Array.isArray(state) && + Array.isArray(state[1]) && + state[1].length === 2 && + state[1][0] === signal && + state[1][1] === undefined && + typeof state[0]?.getSnapshot === "function" + ); + }); + if (subscriptions.length !== 1) throw new Error("Ambiguous voice subscription"); + const hook = subscriptions[0] as Hook; + const subscriber = (hook.memoizedState as unknown[])[0] as Subscriber; + const instance = hook.next?.queue as Instance | undefined; + const effect = hook.next?.next?.memoizedState; + const store = signal.store as Store; + const gate = gates.resolve(node, chain, INFERENCE_GATE)?.atom; + if ( + !gate || + !instance || + !record(effect) || + !Array.isArray(effect.deps) || + effect.deps[0] !== subscriber.subscribe || + typeof effect.create !== "function" || + typeof instance.getSnapshot !== "function" || + !capable(store.get(signal.atom as Atom)) || + !capable(subscriber.getSnapshot()) || + !capable(instance.getSnapshot()) || + !equal( + store.get(signal.atom as Atom) as Capability, + subscriber.getSnapshot() as Capability, + ) || + !equal(subscriber.getSnapshot() as Capability, instance.getSnapshot() as Capability) || + subscriber.createRender?.() != null + ) + throw new Error("Unsupported voice subscription"); + for (const [object, key] of [ + [store, "sub"], + [subscriber, "getSnapshot"], + [instance, "getSnapshot"], + ] as const) + if (Object.getOwnPropertyDescriptor(object, key)?.writable !== true) + throw new Error("Voice subscription is not writable"); + let notify: (() => void) | undefined; + const sub = store.sub; + try { + store.sub = (atom, listener) => { + if (atom !== signal.atom || notify) throw new Error("Unexpected voice subscription"); + notify = listener; + return () => {}; + }; + effect.create(); + } finally { + store.sub = sub; + } + if (!notify) throw new Error("Voice listener unavailable"); + const original = subscriber.getSnapshot; + const originalInstance = instance.getSnapshot; + let previous: Capability | undefined; + const snapshot = () => { + const native = original.call(subscriber); + if (!allowed()) return native; + const projected = replayVoiceCapability(store, signal.atom as Atom, gate); + if (!projected || !projected.isCapable) return native; + if (!previous || !equal(previous, projected)) previous = projected; + return previous; + }; + subscriber.getSnapshot = snapshot; + instance.getSnapshot = snapshot; + const onChange = notify; + restores.push(() => { + if (subscriber.getSnapshot === snapshot) subscriber.getSnapshot = original; + if (instance.getSnapshot === snapshot) instance.getSnapshot = originalInstance; + onChange(); + }); + bindings.push({ subscriber, signal: signal as unknown as Signal, instance }); + onChange(); + } + } + if (!restores.length) throw new Error("Owned voice capability unavailable"); + if (realtimeOnly && restores.length !== 1) throw new Error("Ambiguous voice starter"); + } catch (error) { + for (const restore of restores.reverse()) restore(); + throw error; + } + const alignedSnapshot = ({ subscriber, instance }: (typeof bindings)[number]) => { + try { + const snapshot = subscriber.getSnapshot(); + const instanceSnapshot = instance.getSnapshot(); + if (!capable(snapshot) || !capable(instanceSnapshot) || !equal(snapshot, instanceSnapshot)) + return undefined; + return snapshot; + } catch { + return undefined; + } + }; + return { + dispose() { + for (const restore of restores.splice(0).reverse()) restore(); + }, + current() { + const current = [...currentFibers()].flatMap(hooks); + return bindings.every((binding) => { + const snapshot = alignedSnapshot(binding); + if (!snapshot) return false; + const { subscriber, signal, instance } = binding; + return current.some((hook) => { + const committed = hook.next?.memoizedState; + return ( + Array.isArray(hook.memoizedState) && + hook.memoizedState[0] === subscriber && + hook.memoizedState[1]?.[0] === signal && + hook.next?.queue === instance && + capable(committed) && + equal(committed, snapshot) && + (!realtimeOnly || snapshot.isCapable) + ); + }); + }); + }, + projectedCapable() { + return ( + bindings.length > 0 && + bindings.every((binding) => alignedSnapshot(binding)?.isCapable === true) + ); + }, + }; +} + +interface Target { + [key: string]: unknown; + document?: Document; + performance?: Pick; + setTimeout: typeof setTimeout; + clearTimeout: typeof clearTimeout; +} + +export function installVoiceCapabilityProjection( + options: { + target?: Target; + importModule?: (url: string) => Promise; + } = {}, +): { status: string; dispose(): void } { + const target = options.target ?? (globalThis as unknown as Target); + if (target[GLOBAL]) return target[GLOBAL] as { status: string; dispose(): void }; + const config = readCpaLiveVoiceHookConfig(target); + if (!config) return { status: "disabled", dispose() {} }; + if (!supportsVoiceVersion(CLIENT_OWNED_VOICE_ASSETS, config.desktopVersion)) + return { status: "unsupported", dispose() {} }; + let timer: ReturnType; + let binding: ReturnType | undefined; + let composer: Element | undefined; + let disposed = false; + const handle = { + status: "pending", + dispose() { + disposed = true; + target.clearTimeout(timer); + binding?.dispose(); + if (target[GLOBAL] === handle) Reflect.deleteProperty(target, GLOBAL); + }, + }; + target[GLOBAL] = handle; + const hookActive = (key: string) => { + const hook = target[key]; + return record(hook) && hook.status === "active"; + }; + const currentRoute = () => { + try { + const routing = target.__codexhostHostRoutingV1 as RendererHostRouting | undefined; + const route = routing?.forComposer(); + return route?.hostId === "local" && routing?.forHost("local") === route ? route : undefined; + } catch { + return undefined; + } + }; + const external = () => { + try { + return currentRoute()?.policy.selectedModel()?.startsWith("codexhost/") === true; + } catch { + return false; + } + }; + const chatGateway = () => + !isWorkComposerSurface(target) && chatLiveVoicePreferenceAllowsOverride(target); + const allowed = () => + !!readCpaLiveVoiceHookConfig(target) && + voiceGatewayIsReady(target) && + hookActive("__codexhostClientOwnedVoiceV1") && + hookActive("__codexhostExternalVoiceLaunchV1") && + (chatGateway() || + (!!currentRoute() && + (isWorkComposerSurface(target) || + (external() && + target.document?.querySelectorAll("[data-chatgpt-composer]").length === 0)) && + (cpaLiveVoicePreferenceAllowsInstall(target) || external()))); + const refresh = async () => { + if (disposed) return; + try { + const names = loadedAssetNames(target.document, target.performance); + const assets = selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ); + if (!assets) { + handle.status = "unsupported"; + } else if (!allowed()) handle.status = "pending"; + else { + const module = await (options.importModule ?? ((url) => import(url)))( + `app://-/assets/${assets.initial}`, + ); + if (disposed) return; + const composers = [ + ...(target.document?.querySelectorAll( + chatGateway() ? "[data-chatgpt-composer]" : "[data-codex-composer-root]", + ) ?? []), + ].filter( + (element) => + element.getBoundingClientRect().width > 0 && !element.closest("[hidden], [inert]"), + ); + if (composers.length !== 1) throw new Error("Voice composer unavailable"); + if ( + composers[0] !== composer || + !binding || + (!binding.current() && binding.projectedCapable()) + ) { + binding?.dispose(); + binding = undefined; + composer = composers[0]; + const { family, gates } = initializeVoiceCapabilityFamilies(module, assets); + binding = bindComposerVoiceCapability( + composer as Element, + family, + gates, + allowed, + chatGateway(), + ); + } + handle.status = binding.projectedCapable() ? "active" : "native-ineligible"; + } + } catch { + handle.status = "unsupported"; + } + if (handle.status !== "active" && handle.status !== "native-ineligible") { + binding?.dispose(); + binding = undefined; + composer = undefined; + } + if (!disposed) timer = target.setTimeout(() => void refresh(), 1000); + }; + void refresh(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-voice-settings-availability.ts b/packages/renderer-extension/src/renderer-voice-settings-availability.ts new file mode 100644 index 000000000..a9b981a49 --- /dev/null +++ b/packages/renderer-extension/src/renderer-voice-settings-availability.ts @@ -0,0 +1,229 @@ +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; +import { hooks, replayVoiceCapability } from "./renderer-voice-capability.js"; + +type Value = Record; +type Store = Parameters[0]; +type Atom = Parameters[1]; +interface Hook { + memoizedState?: unknown; + next?: Hook | null; + queue?: unknown; +} +interface Subscriber { + getSnapshot(): unknown; + subscribe(listener: () => void): () => void; + createRender?(): unknown; +} +interface Instance { + getSnapshot(): unknown; +} +interface Scope { + scope: unknown; + node: { store: Store }; + chain: Map; + get(selector: unknown, argument?: unknown): unknown; +} +const record = (value: unknown): value is Value => value !== null && typeof value === "object"; + +/** Settings groups and their exported page, never AppScope siblings or composers. */ +export function voiceSettingsSurface( + doc: Document, + page: unknown, + appScope: { id: unknown }, +): { scope: Scope; fibers: Set } | undefined { + const pages = new Set(); + const fibers = new Set(); + const elements = new Set(); + for (const section of doc.querySelectorAll("section")) { + elements.add(section); + // Shortcut controls mount their own mode reader beneath the group section. + for (const control of section.querySelectorAll?.("button,input") ?? []) elements.add(control); + } + for (const element of [...elements].slice(0, 512)) { + const key = Object.getOwnPropertyNames(element).find((key) => key.startsWith("__reactFiber$")); + const ancestors = committedReactAncestors( + key && Object.getOwnPropertyDescriptor(element, key)?.value, + ); + const boundary = ancestors.findIndex((fiber) => fiber.type === page); + if (boundary < 0) continue; + pages.add(ancestors[boundary] as Value); + for (const fiber of ancestors.slice(0, boundary + 1)) fibers.add(fiber); + } + if (pages.size !== 1) return; + const scopes = new Map(); + for (const fiber of fibers) + for (const hook of hooks(fiber)) { + const scope = record(hook.memoizedState) ? hook.memoizedState.current : undefined; + if ( + record(scope) && + scope.scope === appScope && + record(scope.node) && + scope.chain instanceof Map && + scope.chain.get(appScope.id) === scope.node && + typeof scope.get === "function" + ) + scopes.set(scope.node, scope as unknown as Scope); + } + const scope = [...scopes.values()][0]; + return scopes.size === 1 && scope ? { scope, fibers } : undefined; +} + +/** Prove that a readonly boolean derives from this exact realtime capability. + * Replaying with the native value must agree before substituting its projection. + */ +function availability(store: Store, atom: Atom, voice: Atom, gate: Atom): boolean | undefined { + const native = store.get(atom); + const projected = replayVoiceCapability(store, voice, gate); + if (typeof native !== "boolean" || atom.write || projected?.isCapable !== true) return; + const evaluate = (override: boolean) => { + let remaining = 512; + let hits = 0; + const visiting = new Set(); + const read = (dependency: Atom): unknown => { + if (--remaining < 0) throw new Error("Settings availability dependency limit"); + const value = store.get(dependency); + if (dependency === voice) { + hits += 1; + return override ? projected : value; + } + if (dependency.write || visiting.has(dependency)) return value; + visiting.add(dependency); + try { + return dependency.read(read); + } finally { + visiting.delete(dependency); + } + }; + return { value: read(atom), hits }; + }; + try { + const before = evaluate(false); + const after = evaluate(true); + return before.hits > 0 && + before.value === native && + after.hits > 0 && + typeof after.value === "boolean" + ? after.value + : undefined; + } catch { + return; + } +} + +/** The native group uses a store/atom boolean reader rather than the capability + * family's signal. Project only its proven dependency on voice.realtime. + */ +export function bindVoiceSettingsAvailability( + currentFibers: () => Set, + store: Store, + voice: Atom, + gate: Atom, + allowed: () => boolean, +): { current(): boolean; dispose(): void } { + const candidates: { hook: Hook; subscriber: Subscriber; atom: Atom }[] = []; + for (const fiber of currentFibers()) + for (const hook of hooks(fiber)) { + const [subscriber, deps] = Array.isArray(hook.memoizedState) ? hook.memoizedState : []; + if ( + !record(subscriber) || + typeof subscriber.getSnapshot !== "function" || + !Array.isArray(deps) || + deps.length !== 2 || + deps[0] !== store || + !record(deps[1]) || + typeof deps[1].read !== "function" + ) + continue; + const atom = deps[1] as Atom; + if (availability(store, atom, voice, gate) === true) + candidates.push({ hook, subscriber: subscriber as unknown as Subscriber, atom }); + } + const candidate = candidates[0]; + if (candidates.length !== 1 || !candidate) + throw new Error("Unavailable or ambiguous Settings availability reader"); + const { hook, subscriber, atom } = candidate; + const instance = hook.next?.queue as Instance | undefined; + const effect = hook.next?.next?.memoizedState; + const native = store.get(atom); + if ( + !instance || + typeof instance.getSnapshot !== "function" || + !record(effect) || + !Array.isArray(effect.deps) || + effect.deps.length !== 1 || + effect.deps[0] !== subscriber.subscribe || + typeof effect.create !== "function" || + subscriber.getSnapshot() !== native || + instance.getSnapshot() !== native || + subscriber.createRender?.() != null + ) + throw new Error("Unsupported Settings availability subscription"); + for (const [object, key] of [ + [store, "sub"], + [subscriber, "getSnapshot"], + [instance, "getSnapshot"], + ] as const) + if (Object.getOwnPropertyDescriptor(object, key)?.writable !== true) + throw new Error("Settings availability reader is not writable"); + let notify: (() => void) | undefined; + const sub = store.sub; + try { + store.sub = (selected, listener) => { + if (selected !== atom || notify) throw new Error("Unexpected Settings subscription"); + notify = listener; + return () => {}; + }; + effect.create(); + } finally { + store.sub = sub; + } + if (!notify) throw new Error("Settings availability listener unavailable"); + const original = subscriber.getSnapshot; + const originalInstance = instance.getSnapshot; + const snapshot = () => + allowed() && availability(store, atom, voice, gate) === true ? true : original.call(subscriber); + subscriber.getSnapshot = snapshot; + instance.getSnapshot = snapshot; + const onChange = notify; + let disposed = false; + const binding = { + current() { + try { + const expected = subscriber.getSnapshot(); + return ( + !disposed && + expected === true && + instance.getSnapshot() === expected && + [...currentFibers()] + .flatMap(hooks) + .some( + (current) => + current.memoizedState && + Array.isArray(current.memoizedState) && + current.memoizedState[0] === subscriber && + current.memoizedState[1]?.[0] === store && + current.memoizedState[1]?.[1] === atom && + current.next?.queue === instance && + current.next.memoizedState === expected, + ) + ); + } catch { + return false; + } + }, + dispose() { + if (disposed) return; + disposed = true; + if (subscriber.getSnapshot === snapshot) subscriber.getSnapshot = original; + if (instance.getSnapshot === snapshot) instance.getSnapshot = originalInstance; + onChange(); + }, + }; + try { + onChange(); + } catch (error) { + binding.dispose(); + throw error; + } + return binding; +} diff --git a/packages/renderer-extension/src/renderer-voice-settings.ts b/packages/renderer-extension/src/renderer-voice-settings.ts new file mode 100644 index 000000000..21d93ac88 --- /dev/null +++ b/packages/renderer-extension/src/renderer-voice-settings.ts @@ -0,0 +1,375 @@ +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; +import { loadedAssetNames, readCpaLiveVoiceHookConfig } from "./renderer-cpa-live-voice-hook.js"; +import { selectVoiceAssets } from "./renderer-voice-assets.js"; +import { + NATIVE_CHAT_VOICE_ASSETS, + NATIVE_CHAT_VOICE_VERSION, +} from "./renderer-native-chat-voice-assets.js"; +import { + INFERENCE_GATE, + bindVoiceCapabilitySubscriptions, + replayVoiceCapability, +} from "./renderer-voice-capability.js"; +import { + bindVoiceSettingsAvailability, + voiceSettingsSurface, +} from "./renderer-voice-settings-availability.js"; +import { bindDictationSettingsMode } from "./renderer-dictation-settings.js"; + +const GLOBAL = "__codexhostVoiceSettingsV1"; +const MESSAGE_ID = "settings.general.realtimeVoice.unavailable.description"; +const LABEL = "Voice settings unavailable with this provider"; +const DESCRIPTION = + "Your account allows voice chat. These Desktop settings require the default inference provider. Start ChatGPT Chat voice from the Chat composer."; +type NativeModule = Record; +type Store = Parameters[0]; +interface Scope { + scope: unknown; + node: { store: Store }; + chain: Map; + get(selector: unknown, argument?: unknown): unknown; +} +interface Family { + resolve( + node: Scope["node"], + chain: Scope["chain"], + argument: unknown, + ): { atom: Parameters[1] }; +} +interface SettingsModules { + appScope: { id: unknown }; + page: unknown; + access: unknown; + capability: { + resolve( + node: Scope["node"], + chain: Scope["chain"], + argument: unknown, + ): { + resolve( + node: Scope["node"], + chain: Scope["chain"], + ): Parameters[1]; + }; + }; + gates: Family; + dictationMode?: { + resolve( + node: Scope["node"], + chain: Scope["chain"], + ): Parameters[1]; + }; +} +const record = (value: unknown): value is NativeModule => + value !== null && typeof value === "object"; +const settled = (value: unknown, isCapable: boolean) => + record(value) && + value.isLoading === false && + value.isError === false && + value.isCapable === isCapable; + +function realtimeAtoms(scope: Scope, modules: SettingsModules) { + const atom = modules.capability + .resolve(scope.node, scope.chain, { name: "voice.realtime" }) + .resolve(scope.node, scope.chain); + return { + atom, + gate: modules.gates.resolve(scope.node, scope.chain, INFERENCE_GATE).atom, + }; +} + +/** Explain only the provider-only rejection; entitlement and native state stay unchanged. */ +export function providerBlocksVoiceSettings(scope: Scope, modules: SettingsModules): boolean { + try { + const access = scope.get(modules.access); + if ( + !record(access) || + access.status !== "allowed" || + !settled(scope.get(modules.capability, { name: "voice" }), true) || + !settled(scope.get(modules.capability, { name: "voice.realtime" }), false) + ) + return false; + const { atom, gate } = realtimeAtoms(scope, modules); + return replayVoiceCapability(scope.node.store, atom, gate)?.isCapable === true; + } catch { + return false; + } +} + +/** Match the native Settings row by message identity and exported page ownership. + * Text-only hosts preserve React's nodes and can be restored when policy changes. + */ +export function explainVoiceSettingsRestriction( + doc: Document, + modules: SettingsModules, +): { current(): boolean; dispose(): void } | undefined { + const descriptions: { element: Element; label: Element; scope: Scope }[] = []; + for (const element of doc.querySelectorAll("section div")) { + if (element.childNodes.length !== 1 || element.firstChild?.nodeType !== 3) continue; + const key = Object.getOwnPropertyNames(element).find((key) => key.startsWith("__reactFiber$")); + const ancestors = committedReactAncestors( + key && Object.getOwnPropertyDescriptor(element, key)?.value, + ); + if (!ancestors.some((fiber) => fiber.type === modules.page)) continue; + const owner = ancestors.find( + ({ memoizedProps: props }) => + record(props) && + record(props.description) && + record(props.description.props) && + props.description.props.id === MESSAGE_ID && + props.control === null, + ); + const props = owner?.memoizedProps; + if ( + !record(props) || + !record(props.description) || + !record(props.description.props) || + !record(props.label) || + !record(props.label.props) || + element.textContent !== props.description.props.defaultMessage + ) + continue; + const labelText = props.label.props.defaultMessage; + if (typeof labelText !== "string") continue; + // The native row's nearest host ancestor contains its label and description. + const row = ancestors.find((fiber) => { + const host = fiber.stateNode as Element | undefined; + return ( + fiber.tag === 5 && + host !== element && + typeof host?.contains === "function" && + host.contains(element) && + host.textContent?.includes(labelText) + ); + }); + const host = row?.stateNode as Element | undefined; + const labels = host + ? [...host.querySelectorAll("div")].filter( + (node) => + node.childNodes.length === 1 && + node.firstChild?.nodeType === 3 && + node.textContent === labelText, + ) + : []; + if (labels.length !== 1) continue; + const scopes = new Map(); + for (const fiber of ancestors) { + let hook = fiber.memoizedState; + const seen = new Set(); + while (record(hook) && seen.size < 4096 && !seen.has(hook)) { + seen.add(hook); + const scope = record(hook.memoizedState) ? hook.memoizedState.current : undefined; + if ( + record(scope) && + scope.scope === modules.appScope && + record(scope.node) && + scope.chain instanceof Map && + scope.chain.get(modules.appScope.id) === scope.node && + typeof scope.get === "function" + ) + scopes.set(scope.node, scope as unknown as Scope); + hook = hook.next; + } + } + const label = labels[0]; + const scope = [...scopes.values()][0]; + if (scopes.size === 1 && label && scope) descriptions.push({ element, label, scope }); + } + const current = descriptions.length === 1 ? descriptions[0] : undefined; + if (!current) return; + if (!providerBlocksVoiceSettings(current.scope, modules)) return; + const changes = [ + [current.element, DESCRIPTION], + [current.label, LABEL], + ] as const; + const restores = changes.map(([element, replacement]) => { + const text = element.firstChild as Text; + const previous = text.data; + text.data = replacement; + return () => { + if (text.data === replacement) text.data = previous; + }; + }); + return { + current() { + return ( + changes.every( + ([element, replacement]) => element.isConnected && element.textContent === replacement, + ) && providerBlocksVoiceSettings(current.scope, modules) + ); + }, + dispose() { + for (const restore of restores) restore(); + }, + }; +} + +/** Restore the native voice and dictation controls through owned Settings readers. + * Account policy and all non-provider capability dependencies remain native. + */ +export function restoreNativeVoiceSettings( + doc: Document, + modules: SettingsModules, + enabled: () => boolean = () => true, +): { current(): boolean; dispose(): void } | undefined { + const surface = voiceSettingsSurface(doc, modules.page, modules.appScope); + if (!surface || !providerBlocksVoiceSettings(surface.scope, modules)) return; + const scope = surface.scope; + const currentFibers = () => { + const current = voiceSettingsSurface(doc, modules.page, modules.appScope); + return current?.scope.node === scope.node ? current.fibers : new Set>(); + }; + const allowed = () => enabled() && providerBlocksVoiceSettings(scope, modules); + const { atom, gate } = realtimeAtoms(scope, modules); + const capability = bindVoiceCapabilitySubscriptions( + () => new Set([...currentFibers()].filter((fiber) => fiber.type === modules.page)), + modules.capability as unknown as Parameters[1], + modules.gates as Parameters[2], + allowed, + false, + ["voice.realtime", "dictation.composer", "dictation.global"], + ); + let availability: ReturnType | undefined; + try { + availability = bindVoiceSettingsAvailability( + currentFibers, + scope.node.store, + atom, + gate, + allowed, + ); + const dictationMode = + modules.dictationMode && + bindDictationSettingsMode( + currentFibers, + scope.node.store, + modules.dictationMode.resolve(scope.node, scope.chain), + ["dictation.composer", "dictation.global", "dictation.chatgpt"].map((name) => + modules.capability + .resolve(scope.node, scope.chain, { name }) + .resolve(scope.node, scope.chain), + ), + gate, + enabled, + ); + const voiceAvailability = availability; + return { + current: () => + allowed() && + capability.current() && + voiceAvailability.current() && + (dictationMode?.current() ?? true), + dispose() { + dictationMode?.dispose(); + voiceAvailability.dispose(); + capability.dispose(); + }, + }; + } catch (error) { + availability?.dispose(); + capability.dispose(); + throw error; + } +} + +export function installVoiceSettingsExplanation( + options: { + target?: typeof globalThis; + importModule?: (url: string) => Promise; + } = {}, +): { dispose(): void } { + const target = options.target ?? globalThis; + const globals = target as unknown as NativeModule; + if (globals[GLOBAL]) return globals[GLOBAL] as { dispose(): void }; + const config = readCpaLiveVoiceHookConfig(target); + if (!config) return { dispose() {} }; + if (config.desktopVersion && config.desktopVersion !== NATIVE_CHAT_VOICE_VERSION) + return { dispose() {} }; + let disposed = false; + let timer: ReturnType | undefined; + let binding: ReturnType; + let modules: SettingsModules | undefined; + let restored = false; + const handle = { + dispose() { + disposed = true; + if (timer !== undefined) target.clearTimeout(timer); + binding?.dispose(); + if (globals[GLOBAL] === handle) Reflect.deleteProperty(globals, GLOBAL); + }, + }; + globals[GLOBAL] = handle; + const tryControls = () => { + if (!modules || !target.document) return; + try { + return restoreNativeVoiceSettings( + target.document, + modules, + () => !!readCpaLiveVoiceHookConfig(target), + ); + } catch { + return undefined; + } + }; + const refresh = async () => { + if (disposed) return; + try { + if (binding?.current()) { + if (restored) return; + // Passive subscription effects can settle after the explanation mounts. + // Retry controls without rewriting the explanatory text on each poll. + const controls = tryControls(); + if (controls) { + binding.dispose(); + binding = controls; + restored = true; + } + return; + } + binding?.dispose(); + binding = undefined; + restored = false; + if (!modules) { + // Document-start can expose one native chunk before its shared dependency. + // Keep the installer alive until the audited pair has finished loading. + const assets = selectVoiceAssets( + NATIVE_CHAT_VOICE_ASSETS, + loadedAssetNames(target.document, target.performance), + config.desktopVersion, + config.desktopPlatform, + ); + if (!assets?.settings) return; + const load = options.importModule ?? ((url: string) => import(url)); + const [initial, shared, settings] = await Promise.all( + [assets.initial, assets.shared, assets.settings].map((asset) => + load(`app://-/assets/${asset}`), + ), + ); + if (disposed) return; + if (typeof initial.l0n !== "function" || typeof settings.VoiceSettings !== "function") + return; + initial.l0n(); + const dictationMode = initial.mW as SettingsModules["dictationMode"]; + modules = { + appScope: shared.Bst as SettingsModules["appScope"], + page: settings.VoiceSettings, + access: initial.k5n, + capability: initial.c0n as SettingsModules["capability"], + gates: initial.Cnr as Family, + ...(dictationMode ? { dictationMode } : {}), + }; + } + if (target.document) { + binding = tryControls(); + restored = !!binding; + if (!binding) binding = explainVoiceSettingsRestriction(target.document, modules); + } + } catch { + /* Unknown Settings layouts retain the native explanation. */ + } finally { + if (!disposed) timer = target.setTimeout(() => void refresh(), 1000); + } + }; + void refresh(); + return handle; +} diff --git a/packages/renderer-extension/src/renderer-voice-start-capability.ts b/packages/renderer-extension/src/renderer-voice-start-capability.ts new file mode 100644 index 000000000..c7883b89d --- /dev/null +++ b/packages/renderer-extension/src/renderer-voice-start-capability.ts @@ -0,0 +1,281 @@ +import { selectVoiceAssets, supportsVoiceVersion } from "./renderer-voice-assets.js"; +import { committedReactAncestors } from "@codexhost/desktop-control/renderer-bindings"; +import { VOICE_GATEWAY_STATUS_METHOD } from "@codexhost/shared-contracts"; +import { + CLIENT_OWNED_VOICE_ASSETS, + rememberVoiceProductExperience, + requestVoiceHost, + type NativeVoiceBridge, +} from "./renderer-client-owned-voice.js"; +import { + bindVoiceCapabilitySubscriptions, + initializeVoiceCapabilityFamilies, +} from "./renderer-voice-capability.js"; +import { loadedAssetNames, readCpaLiveVoiceHookConfig } from "./renderer-cpa-live-voice-hook.js"; +import { + chatLiveVoicePreferenceAllowsOverride, + cpaLiveVoicePreferenceAllowsInstall, +} from "./renderer-cpa-live-voice-preference.js"; +import { isAvatarOverlayHref } from "./renderer-overlay-thread-isolation.js"; + +const GLOBAL = "__codexhostVoiceStartCapabilityV1"; +type RecordValue = Record; +const record = (v: unknown): v is RecordValue => v !== null && typeof v === "object"; +type Start = (...args: unknown[]) => unknown; +type Binding = { dispose(): void; current(): boolean }; +type Attempt = { binding?: Binding }; + +/** Explicit local starts use Chat's choice or Work's choice for Work/Codex. */ +export function gatewayVoiceStartRequest( + request: unknown, + workGateway: boolean, + chatGateway = false, +): boolean { + if ( + !record(request) || + (request.productExperience !== "work" && + request.productExperience !== "codex" && + request.productExperience !== "chat") + ) + return false; + if (request.source === "composer_button_existing_thread") { + if (!record(request.locator) || request.locator.hostId !== "local") return false; + } else if ( + request.source !== "composer_button_new_thread" || + (request.hostId ?? "local") !== "local" + ) + return false; + if (request.productExperience === "chat") return chatGateway; + const mode = request.activeCollaborationMode; + const settings = record(mode) ? mode.settings : undefined; + const model = record(settings) ? settings.model : undefined; + return workGateway || (typeof model === "string" && model.startsWith("codexhost/")); +} + +export function wrapGatewayVoiceStarter( + start: Start, + options: { + qualifies(request: unknown): boolean; + ready(): Promise; + prepare(allowed: () => boolean): Promise; + committed(binding: Binding): Promise; + }, +) { + let active: Attempt | undefined; + return { + async start(...args: unknown[]) { + if (!options.qualifies(args[0])) return start(...args); + if (active) throw new Error("Voice gateway start is already pending"); + const attempt: Attempt = {}; + active = attempt; + const allowed = () => active === attempt && options.qualifies(args[0]); + try { + if (!(await options.ready())) throw new Error("Voice gateway is not ready"); + if (!allowed()) throw new Error("Voice gateway start was cancelled"); + attempt.binding = await options.prepare(allowed); + await options.committed(attempt.binding); + if (!allowed()) throw new Error("Voice gateway start was cancelled"); + if (!attempt.binding.current()) throw new Error("Voice starter subscription changed"); + // Desktop's useEffectEvent now sees the committed capability snapshot. + // Its workspace, media, busy, treatment and account checks remain native. + rememberVoiceProductExperience(args[0]); + return await start(...args); + } finally { + attempt.binding?.dispose(); + if (active === attempt) active = undefined; + } + }, + cancel() { + active?.binding?.dispose(); + active = undefined; + }, + }; +} + +interface Target { + [key: string]: unknown; + location?: { href: string }; + document?: Document; + performance?: Pick; + setTimeout: typeof setTimeout; + clearTimeout: typeof clearTimeout; +} + +/** Native overlay ancestors below the exported page, with exactly one starter + * capability subscription. Composer and unrelated AppScope owners are excluded. + */ +export function overlayVoiceStarterFibers(document: Document, page: unknown): Set { + const fibers = new Set(); + for (const element of document.querySelectorAll("[data-avatar-overlay-native-surface-id]")) { + const key = Object.getOwnPropertyNames(element).find((key) => key.startsWith("__reactFiber$")); + const ancestors = committedReactAncestors( + key && Object.getOwnPropertyDescriptor(element, key)?.value, + ); + const boundary = ancestors.findIndex((fiber) => record(fiber) && fiber.type === page); + if (boundary < 0) continue; + for (const fiber of ancestors.slice(0, boundary)) if (record(fiber)) fibers.add(fiber); + } + return fibers; +} + +/** The overlay registers a native callback through an immutable RPC stub. Wrap + * that service reference, preserving registration arguments and cancellation. + */ +export function installVoiceStartCapabilityHook( + options: { + target?: Target; + importModule?: (url: string) => Promise; + } = {}, +): { status: string; dispose(): void } { + const target = options.target ?? (globalThis as unknown as Target); + if (target[GLOBAL]) return target[GLOBAL] as { status: string; dispose(): void }; + const config = readCpaLiveVoiceHookConfig(target); + if (!config || !isAvatarOverlayHref(target.location?.href ?? "")) + return { status: "disabled", dispose() {} }; + if (!supportsVoiceVersion(CLIENT_OWNED_VOICE_ASSETS, config.desktopVersion)) + return { status: "unsupported", dispose() {} }; + const importModule = options.importModule ?? ((url: string) => import(url)); + let disposed = false; + let timer: ReturnType; + let restore: (() => void) | undefined; + const wrappers = new Set>(); + const started = Date.now(); + const handle = { + status: "pending", + dispose() { + disposed = true; + target.clearTimeout(timer); + for (const wrapper of wrappers) wrapper.cancel(); + restore?.(); + if (target[GLOBAL] === handle) Reflect.deleteProperty(target, GLOBAL); + }, + }; + target[GLOBAL] = handle; + const attempt = async () => { + if (disposed) return; + try { + const names = loadedAssetNames(target.document, target.performance); + const assets = selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ); + if (assets) { + const shared = await importModule(`app://-/assets/${assets.shared}`); + const init = "init" in assets ? assets.init : undefined; + if (typeof init === "string" && typeof shared[init] === "function") { + (shared[init] as () => void)(); + } + const services = shared[assets.services]; + if ( + record(services) && + (record(services.realtimeVoiceRuntime) || + typeof services.realtimeVoiceRuntime === "function") + ) { + const original = services.realtimeVoiceRuntime; + const register = Reflect.get(original, "registerRealtimeStarter"); + if (typeof register !== "function") throw new Error("Voice starter unavailable"); + const wrapped = new Proxy(original, { + get(service, key, receiver) { + if (key === "registerRealtimeStarter") + return (start: Start, cancel: Start, ...rest: unknown[]) => { + if (typeof start !== "function" || typeof cancel !== "function") + throw new Error("Voice starter callback unavailable"); + const wrapper = wrapGatewayVoiceStarter(start, { + qualifies: (request) => + !disposed && + !!readCpaLiveVoiceHookConfig(target) && + gatewayVoiceStartRequest( + request, + cpaLiveVoicePreferenceAllowsInstall(target), + chatLiveVoicePreferenceAllowsOverride(target), + ), + ready: async () => { + const hook = target.__codexhostClientOwnedVoiceV1; + if (!record(hook) || hook.status !== "active") return false; + const bridgeClass = shared[assets.bridge] as { + getInstance(): NativeVoiceBridge; + }; + const result = await requestVoiceHost( + bridgeClass.getInstance(), + {}, + VOICE_GATEWAY_STATUS_METHOD, + ); + return result === true; + }, + prepare: async (allowed) => { + const names = loadedAssetNames(target.document, target.performance); + if ( + selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + names, + config.desktopVersion, + config.desktopPlatform, + ) !== assets + ) + throw new Error("Unaudited voice starter"); + const [initial, overlay] = await Promise.all([ + importModule(`app://-/assets/${assets.initial}`), + importModule(`app://-/assets/${assets.overlay}`), + ]); + if (typeof overlay.AvatarOverlayNativePage !== "function" || !target.document) + throw new Error("Voice starter owner unavailable"); + const document = target.document; + const { family, gates } = initializeVoiceCapabilityFamilies(initial, assets); + return bindVoiceCapabilitySubscriptions( + () => overlayVoiceStarterFibers(document, overlay.AvatarOverlayNativePage), + family, + gates, + allowed, + true, + ); + }, + // Wait for React's committed snapshot, including when the + // persistent overlay is hidden. A stale closure fails closed. + committed: async (binding) => { + for (let i = 0; i < 40 && !disposed && !binding.current(); i += 1) + await new Promise((resolve) => target.setTimeout(resolve, 50)); + }, + }); + wrappers.add(wrapper); + return Reflect.apply(register, original, [ + wrapper.start, + (...args: unknown[]) => { + wrapper.cancel(); + return cancel(...args); + }, + ...rest, + ]); + }; + if (key === "unregisterRealtimeStarter") + return (...args: unknown[]) => { + for (const wrapper of wrappers) wrapper.cancel(); + wrappers.clear(); + return Reflect.apply(Reflect.get(service, key), service, args); + }; + return Reflect.get(service, key, receiver); + }, + }); + if (disposed) return; + if (!Reflect.set(services, "realtimeVoiceRuntime", wrapped)) + throw new Error("Voice service is not writable"); + restore = () => { + if (services.realtimeVoiceRuntime === wrapped) services.realtimeVoiceRuntime = original; + }; + handle.status = "active"; + return; + } + } + if (Date.now() - started >= 60_000) { + handle.status = "unsupported"; + return; + } + timer = target.setTimeout(() => void attempt(), 25); + } catch { + handle.status = "unsupported"; + } + }; + void attempt(); + return handle; +} diff --git a/packages/renderer-extension/src/settings/appearance-page.ts b/packages/renderer-extension/src/settings/appearance-page.ts index 860c6d8df..1ce175059 100644 --- a/packages/renderer-extension/src/settings/appearance-page.ts +++ b/packages/renderer-extension/src/settings/appearance-page.ts @@ -1,3 +1,8 @@ +import { + CODEX_USAGE_BANNER_CHANGE_EVENT, + readCodexUsageBannerPreference, + writeCodexUsageBannerPreference, +} from "../renderer-codex-usage-banner-preference.js"; import { REASONING_SOFT_WRAP_CHANGE_EVENT, readReasoningTranscriptSoftWrap, @@ -35,6 +40,32 @@ export function createAppearanceSettingsPage( description.textContent = messages.appearanceDescription; const { group, card } = createPreferenceGroup(document, messages.appearanceGroup); + const usageBannerId = preferenceId("hide-codex-usage-banner"); + const usageBannerRow = createPreferenceItem(document, { + title: messages.hideCodexUsageBannerTitle, + description: messages.hideCodexUsageBannerDescription, + controlId: usageBannerId, + help: { + label: messages.hideCodexUsageBannerHelpLabel, + lines: messages.hideCodexUsageBannerHelp, + }, + }); + const usageBanner = createPreferenceSwitch( + document, + usageBannerId, + usageBannerRow.description.id, + ); + usageBanner.checked = readCodexUsageBannerPreference(ownerWindow); + usageBanner.addEventListener("change", () => { + writeCodexUsageBannerPreference(ownerWindow, usageBanner.checked); + }); + const syncUsageBanner = (): void => { + usageBanner.checked = readCodexUsageBannerPreference(ownerWindow); + }; + ownerWindow.addEventListener(CODEX_USAGE_BANNER_CHANGE_EVENT, syncUsageBanner); + usageBannerRow.item.append(usageBanner); + card.append(usageBannerRow.item); + const softWrapId = preferenceId("reasoning-soft-wrap"); const row = createPreferenceItem(document, { title: messages.reasoningSoftWrapTitle, @@ -60,6 +91,7 @@ export function createAppearanceSettingsPage( getLoadedSessionsClient, ); return () => { + ownerWindow.removeEventListener(CODEX_USAGE_BANNER_CHANGE_EVENT, syncUsageBanner); ownerWindow.removeEventListener(REASONING_SOFT_WRAP_CHANGE_EVENT, sync); disposeIdleRelease(); }; diff --git a/packages/renderer-extension/src/settings/back-navigation.ts b/packages/renderer-extension/src/settings/back-navigation.ts new file mode 100644 index 000000000..c7a7b641a --- /dev/null +++ b/packages/renderer-extension/src/settings/back-navigation.ts @@ -0,0 +1,114 @@ +const NATIVE_BACK_SELECTOR = '[data-app-shell-titlebar] button[aria-label="Back"]'; +const SETTINGS_BACK_ATTRIBUTE = "data-codexhost-settings-back"; + +/** Settings has a Back action even when the covered native page has no history. */ +export function attachSettingsBackNavigation( + ownerDocument: Document, + onBack: () => void, +): { dispose(): void } { + const ownerWindow = ownerDocument.defaultView; + let source: HTMLButtonElement | null = null; + let button: HTMLButtonElement | null = null; + let disposed = false; + + const onClick = (event: MouseEvent): void => { + event.preventDefault(); + event.stopPropagation(); + onBack(); + }; + const place = (): void => { + if (disposed) return; + const nextSource = + [...ownerDocument.querySelectorAll(NATIVE_BACK_SELECTOR)].find( + (candidate) => { + const bounds = candidate.getBoundingClientRect(); + const parent = candidate.parentElement; + return ( + bounds.width > 0 && + bounds.height > 0 && + !candidate.closest('[aria-hidden="true"]') && + parent !== null && + ownerWindow?.getComputedStyle(parent).visibility !== "hidden" + ); + }, + ) ?? null; + if (nextSource !== source) { + button?.removeEventListener("click", onClick); + button?.remove(); + source = nextSource; + button = source?.cloneNode(true) as HTMLButtonElement | null; + if (button) { + // Copy native styling and its localized label, without native disabled state + // or React handlers. The original control remains owned by Desktop. + for (const element of [button, ...button.querySelectorAll("[id]")]) { + element.removeAttribute("id"); + } + for (const attribute of [ + "disabled", + "aria-disabled", + "data-disabled", + "tabindex", + "aria-describedby", + ]) { + button.removeAttribute(attribute); + } + button.setAttribute(SETTINGS_BACK_ATTRIBUTE, ""); + button.title = source?.getAttribute("aria-label") ?? "Back"; + button.style.position = "fixed"; + button.style.zIndex = "41"; + button.style.margin = "0"; + button.style.setProperty("-webkit-app-region", "no-drag"); + button.addEventListener("click", onClick); + ownerDocument.body.append(button); + } + } + if (!source || !button) return; + const bounds = source.getBoundingClientRect(); + Object.assign(button.style, { + left: `${bounds.left}px`, + top: `${bounds.top}px`, + width: `${bounds.width}px`, + height: `${bounds.height}px`, + }); + }; + const onKeyDown = (event: KeyboardEvent): void => { + if (event.defaultPrevented || event.shiftKey) return; + const macBack = event.metaKey && !event.ctrlKey && !event.altKey && event.key === "["; + const altBack = event.altKey && !event.metaKey && !event.ctrlKey && event.key === "ArrowLeft"; + if (!macBack && !altBack) return; + event.preventDefault(); + event.stopPropagation(); + onBack(); + }; + + const style = ownerDocument.createElement("style"); + // Visibility preserves geometry but removes the covered control from hit + // testing and the accessibility tree. + style.textContent = `${NATIVE_BACK_SELECTOR} { visibility: hidden !important; }`; + ownerDocument.head.append(style); + place(); + const observer = ownerWindow + ? new ownerWindow.MutationObserver((records) => { + const titlebar = source?.closest("[data-app-shell-titlebar]"); + if (!source?.isConnected || records.some((record) => titlebar?.contains(record.target))) { + place(); + } + }) + : null; + observer?.observe(ownerDocument.body, { childList: true, subtree: true }); + ownerWindow?.addEventListener("resize", place); + ownerDocument.addEventListener("keydown", onKeyDown, true); + + return { + dispose() { + if (disposed) return; + disposed = true; + observer?.disconnect(); + ownerWindow?.removeEventListener("resize", place); + ownerDocument.removeEventListener("keydown", onKeyDown, true); + button?.removeEventListener("click", onClick); + button?.remove(); + style.remove(); + }, + }; +} diff --git a/packages/renderer-extension/src/settings/connections-page.ts b/packages/renderer-extension/src/settings/connections-page.ts index 1487a484c..c69c81137 100644 --- a/packages/renderer-extension/src/settings/connections-page.ts +++ b/packages/renderer-extension/src/settings/connections-page.ts @@ -1,5 +1,7 @@ import type { CodexhostError, + GrokComputerUseSet, + GrokComputerUseSettings, HarnessInstallationState, HarnessLaunchSettings, HarnessPluginDescriptor, @@ -17,6 +19,7 @@ import type { RemoteConnectionsControl } from "../remote-connections-control.js" import type { RendererSettingsPageDefinition, RendererSettingsPageMountContext } from "./core.js"; import { createRendererSettingsIcon } from "./icons.js"; import { createHarnessLaunchControls } from "./harness-launch-controls.js"; +import { createGrokComputerUseControls } from "./grok-computer-use-controls.js"; import { harnessHasInstallCommands } from "./harness-installation-guides.js"; import { createHarnessInstallationPanel } from "./harness-installation-panel.js"; import { harnessInstallStore } from "./harness-install-store.js"; @@ -26,8 +29,7 @@ import type { HarnessVersionPanel } from "./harness-version-panel.js"; import { harnessVersionLabel } from "./harness-version-messages.js"; import type { RendererSettingsMessages } from "./localization.js"; -export const CODEXHOST_GITHUB_ISSUES_NEW_URL = - "https://github.com/BytePioneer-AI/codex-host/issues/new"; +export const CODEXHOST_GITHUB_ISSUES_NEW_URL = "https://github.com/karlorz/codex-host/issues/new"; export interface RendererConnectionAgentSnapshot { readonly agent: ExternalRendererAgent; @@ -65,6 +67,8 @@ export interface RendererConnectionDiagnostics { agent: ExternalRendererAgent, path: string | null, ): Promise; + getGrokComputerUse?(hostId: string): Promise; + setGrokComputerUse?(hostId: string, update: GrokComputerUseSet): Promise; subscribe(listener: () => void): () => void; } @@ -634,6 +638,16 @@ function renderConnectionInspector( }); body.append(launchControls); } + const getGrokComputerUse = diagnostics?.getGrokComputerUse?.bind(diagnostics); + const setGrokComputerUse = diagnostics?.setGrokComputerUse?.bind(diagnostics); + if (hostId === "local" && agent === "grok" && getGrokComputerUse && setGrokComputerUse) { + body.append( + createGrokComputerUseControls(document, messages, { + get: () => getGrokComputerUse(hostId), + set: (update) => setGrokComputerUse(hostId, update), + }), + ); + } inspector.append(body); return launchControls; } diff --git a/packages/renderer-extension/src/settings/grok-computer-use-controls.ts b/packages/renderer-extension/src/settings/grok-computer-use-controls.ts new file mode 100644 index 000000000..94ecf924b --- /dev/null +++ b/packages/renderer-extension/src/settings/grok-computer-use-controls.ts @@ -0,0 +1,160 @@ +import type { GrokComputerUseSet, GrokComputerUseSettings } from "@codexhost/shared-contracts"; +import type { RendererSettingsMessages } from "./localization.js"; +import { createPreferenceSwitch } from "./preference-ui.js"; + +export function createGrokComputerUseControls( + document: Document, + messages: RendererSettingsMessages, + settings: { + get(): Promise; + set(update: GrokComputerUseSet): Promise; + }, +): HTMLElement { + const section = document.createElement("section"); + section.className = "settings-grok-computer-use"; + section.dataset.grokComputerUse = "true"; + + const header = document.createElement("div"); + header.className = "settings-grok-computer-use__header"; + const copy = document.createElement("div"); + copy.className = "settings-grok-computer-use__copy"; + const title = document.createElement("h3"); + title.className = "settings-grok-computer-use__title"; + title.id = "grok-computer-use-title"; + title.textContent = messages.grokComputerUseTitle; + const description = document.createElement("p"); + description.className = "settings-grok-computer-use__description"; + description.id = "grok-computer-use-description"; + description.textContent = messages.grokComputerUseDescription; + copy.append(title, description); + const input = createPreferenceSwitch( + document, + "grok-computer-use-enabled", + "grok-computer-use-description grok-computer-use-help grok-computer-use-status", + ); + input.setAttribute("aria-labelledby", title.id); + input.setAttribute("aria-label", messages.grokComputerUseLabel); + header.append(copy, input); + + const help = document.createElement("p"); + help.className = "settings-grok-computer-use__help"; + help.id = "grok-computer-use-help"; + help.textContent = messages.grokComputerUseHelp; + + const status = document.createElement("p"); + status.className = "settings-grok-computer-use__status"; + status.id = "grok-computer-use-status"; + status.setAttribute("role", "status"); + status.setAttribute("aria-live", "polite"); + + const browserHeader = document.createElement("div"); + browserHeader.className = + "settings-grok-computer-use__header settings-grok-computer-use__browser"; + const browserCopy = document.createElement("div"); + browserCopy.className = "settings-grok-computer-use__copy"; + const browserTitle = document.createElement("h3"); + browserTitle.className = "settings-grok-computer-use__title"; + browserTitle.id = "grok-browser-title"; + browserTitle.textContent = messages.grokBrowserTitle; + const browserDescription = document.createElement("p"); + browserDescription.className = "settings-grok-computer-use__description"; + browserDescription.id = "grok-browser-description"; + browserDescription.textContent = messages.grokBrowserDescription; + browserCopy.append(browserTitle, browserDescription); + const browserInput = createPreferenceSwitch( + document, + "grok-browser-enabled", + "grok-browser-description grok-browser-help grok-browser-status", + ); + browserInput.setAttribute("aria-labelledby", browserTitle.id); + browserInput.setAttribute("aria-label", messages.grokBrowserLabel); + browserHeader.append(browserCopy, browserInput); + const browserHelp = document.createElement("p"); + browserHelp.className = "settings-grok-computer-use__help"; + browserHelp.id = "grok-browser-help"; + browserHelp.textContent = messages.grokBrowserHelp; + const browserStatus = document.createElement("p"); + browserStatus.className = "settings-grok-computer-use__status"; + browserStatus.id = "grok-browser-status"; + browserStatus.setAttribute("role", "status"); + browserStatus.setAttribute("aria-live", "polite"); + section.append(header, help, status, browserHeader, browserHelp, browserStatus); + + let busy = true; + let loaded = false; + let last: GrokComputerUseSettings | undefined; + const show = (value: GrokComputerUseSettings) => { + last = value; + input.checked = value.enabled; + input.disabled = busy || !loaded || (!value.available && !value.enabled); + const browserOn = value.browser === true; + const browserFiles = value.browserFilesPresent === true; + browserInput.checked = browserOn; + browserInput.disabled = busy || !loaded || (!browserFiles && !browserOn); + if (!browserFiles) { + browserStatus.dataset.state = "unavailable"; + browserStatus.textContent = value.browserUnavailableReason ?? messages.grokBrowserUnavailable; + } else if (value.browserAvailable !== true) { + browserStatus.dataset.state = browserOn ? "unavailable" : "off"; + browserStatus.textContent = browserOn + ? (value.browserUnavailableReason ?? messages.grokBrowserBlocked) + : messages.grokBrowserOff; + } else { + browserStatus.dataset.state = browserOn ? "on" : "off"; + browserStatus.textContent = browserOn ? messages.grokBrowserOn : messages.grokBrowserOff; + } + if (value.applyFailed) { + status.dataset.state = "unavailable"; + status.textContent = value.applyError ?? messages.grokComputerUseApplyFailed; + return; + } + if (!value.available) { + status.dataset.state = "unavailable"; + status.textContent = value.unavailableReason ?? messages.grokComputerUseUnavailable; + return; + } + status.dataset.state = value.enabled ? "on" : "off"; + status.textContent = value.enabled ? messages.grokComputerUseOn : messages.grokComputerUseOff; + }; + const save = (update: GrokComputerUseSet, target: HTMLElement) => { + if (busy || !loaded) return; + busy = true; + input.disabled = true; + browserInput.disabled = true; + target.textContent = messages.grokComputerUseSaving; + void settings + .set(update) + .then((value) => { + busy = false; + show(value); + }) + .catch(() => { + busy = false; + if (last) show(last); + else { + input.disabled = false; + browserInput.disabled = false; + } + target.textContent = messages.grokComputerUseSaveError; + }); + }; + input.addEventListener("change", () => save({ enabled: input.checked }, status)); + browserInput.addEventListener("change", () => + save({ browser: browserInput.checked }, browserStatus), + ); + status.textContent = messages.grokComputerUseLoading; + browserStatus.textContent = messages.grokBrowserOff; + void settings + .get() + .then((value) => { + loaded = true; + busy = false; + show(value); + }) + .catch(() => { + busy = false; + status.textContent = messages.grokComputerUseLoadError; + browserStatus.textContent = messages.grokComputerUseLoadError; + }); + return section; +} diff --git a/packages/renderer-extension/src/settings/localization.ts b/packages/renderer-extension/src/settings/localization.ts index c685ccb58..a6d929606 100644 --- a/packages/renderer-extension/src/settings/localization.ts +++ b/packages/renderer-extension/src/settings/localization.ts @@ -38,6 +38,10 @@ export interface RendererSettingsMessages { readonly otherSection: string; readonly appearanceDescription: string; readonly appearanceGroup: string; + readonly hideCodexUsageBannerTitle: string; + readonly hideCodexUsageBannerDescription: string; + readonly hideCodexUsageBannerHelpLabel: string; + readonly hideCodexUsageBannerHelp: readonly string[]; readonly loadedSessions: { title: string; description: string; @@ -189,6 +193,26 @@ export interface RendererSettingsMessages { readonly launchPathLoading: string; readonly launchPathLoadError: string; readonly launchPathSaveError: string; + readonly grokComputerUseLabel: string; + readonly grokComputerUseTitle: string; + readonly grokComputerUseDescription: string; + readonly grokComputerUseHelp: string; + readonly grokComputerUseOn: string; + readonly grokComputerUseOff: string; + readonly grokComputerUseSaving: string; + readonly grokComputerUseLoading: string; + readonly grokComputerUseLoadError: string; + readonly grokComputerUseSaveError: string; + readonly grokComputerUseUnavailable: string; + readonly grokBrowserLabel: string; + readonly grokBrowserTitle: string; + readonly grokBrowserDescription: string; + readonly grokBrowserHelp: string; + readonly grokBrowserOn: string; + readonly grokBrowserOff: string; + readonly grokBrowserUnavailable: string; + readonly grokBrowserBlocked: string; + readonly grokComputerUseApplyFailed: string; readonly connectionOfficialWebsite: string; readonly connectionOpenHarnessWeb: string; readonly connectionDeepSeekTestedVersions: string; @@ -297,6 +321,14 @@ const ENGLISH_MESSAGES: RendererSettingsMessages = Object.freeze({ otherSection: "Other", appearanceDescription: "Conversation display and local resource management.", appearanceGroup: "Appearance", + hideCodexUsageBannerTitle: "Hide Codex usage warning", + hideCodexUsageBannerDescription: "On by default. Hide the out-of-usage card on every Composer.", + hideCodexUsageBannerHelpLabel: "About hiding the Codex usage warning", + hideCodexUsageBannerHelp: Object.freeze([ + "The native card appears when the signed-in ChatGPT Codex quota is exhausted.", + "When this is on, that card is hidden on every Composer, including Codex threads. Reset usage and Upgrade stay available in Settings.", + "Turning this off only shows the native card again. Send on this Composer, including Codex threads, is already separated from that quota card.", + ]), loadedSessions: { title: "Loaded sessions", description: @@ -475,6 +507,31 @@ const ENGLISH_MESSAGES: RendererSettingsMessages = Object.freeze({ launchPathLoadError: "Could not load launch settings. Reopen this detail panel to retry.", launchPathSaveError: "Could not save. Enter an existing absolute installation folder on this Host and check configuration permissions.", + grokComputerUseLabel: "Grok Computer Use", + grokComputerUseTitle: "Computer Use", + grokComputerUseDescription: "Let Grok control Mac apps via ChatGPT Computer Use (cua_repl).", + grokComputerUseHelp: + "ChatGPT Settings → Computer use still applies to official Codex threads. Always-allowed apps stay in that list. New Grok threads pick up this switch.", + grokComputerUseOn: "On · available", + grokComputerUseOff: "Off", + grokComputerUseSaving: "Saving…", + grokComputerUseLoading: "Loading Grok Computer Use…", + grokComputerUseLoadError: "Could not load Grok Computer Use. Reopen this panel to retry.", + grokComputerUseSaveError: + "Could not save. Install ChatGPT.app with Computer Use, then try again.", + grokComputerUseUnavailable: "ChatGPT Desktop Computer Use files were not found on this Mac.", + grokBrowserLabel: "Grok in-app browser", + grokBrowserTitle: "In-app browser", + grokBrowserDescription: + "Let Grok use ChatGPT's built-in browser (Browser Use) through cua_repl. Independent of Computer Use.", + grokBrowserHelp: + "Official Codex Browser Use is unchanged. Grok fails closed when ChatGPT does not authorize the connection or no in-app browser tab can be bound. The Chrome extension is a separate path.", + grokBrowserOn: "On · browser surface enabled", + grokBrowserOff: "Off", + grokBrowserUnavailable: "ChatGPT Desktop in-app browser files were not found on this Mac.", + grokBrowserBlocked: + "Saved as intent. ChatGPT Desktop does not authorize Grok to use the in-app browser.", + grokComputerUseApplyFailed: "Could not apply the saved setting to Grok config.", connectionOfficialWebsite: "Visit official website", connectionOpenHarnessWeb: "Open DeepSeek Harness Web", connectionDeepSeekTestedVersions: @@ -592,6 +649,7 @@ const ENGLISH_MESSAGES: RendererSettingsMessages = Object.freeze({ connections: "Connections", "remote-connections": "Remote connections", appearance: "General", + voice: "Voice", accounts: "Accounts", "session-import": "Session Import", "usage-statistics": "Usage statistics", @@ -611,6 +669,14 @@ const CHINESE_MESSAGES: RendererSettingsMessages = Object.freeze({ otherSection: "其他", appearanceDescription: "会话显示与本地资源管理。", appearanceGroup: "外观", + hideCodexUsageBannerTitle: "隐藏 Codex 额度警告", + hideCodexUsageBannerDescription: "默认开启。在所有 Composer 上隐藏额度耗尽卡片。", + hideCodexUsageBannerHelpLabel: "隐藏 Codex 额度警告说明", + hideCodexUsageBannerHelp: Object.freeze([ + "已登录的 ChatGPT Codex 额度耗尽时,Desktop 会显示原生警告卡片。", + "开启后,所有 Composer 都隐藏该卡片,包括 Codex 会话。「重置用量」和「升级」仍可在设置中使用。", + "关闭此开关只会重新显示原生卡片。当前 Composer 的发送(包括 Codex 会话)已与该额度卡片分离。", + ]), loadedSessions: { title: "已加载会话", description: @@ -779,6 +845,29 @@ const CHINESE_MESSAGES: RendererSettingsMessages = Object.freeze({ launchPathLoading: "正在读取启动设置…", launchPathLoadError: "无法读取启动设置,请重新打开此详情面板重试。", launchPathSaveError: "保存失败。请填写此 Host 上实际存在的安装目录绝对路径,并确认配置目录可写。", + grokComputerUseLabel: "Grok Computer Use", + grokComputerUseTitle: "Computer Use", + grokComputerUseDescription: "让 Grok 通过 ChatGPT Computer Use(cua_repl)控制 Mac 应用。", + grokComputerUseHelp: + "ChatGPT 设置 → Computer use 仍只作用于官方 Codex 线程。Always-allowed 应用仍保存在该列表。新的 Grok 线程会读取此开关。", + grokComputerUseOn: "已开启 · 可用", + grokComputerUseOff: "已关闭", + grokComputerUseSaving: "正在保存…", + grokComputerUseLoading: "正在读取 Grok Computer Use…", + grokComputerUseLoadError: "无法读取 Grok Computer Use,请重新打开此面板重试。", + grokComputerUseSaveError: "保存失败。请先安装带 Computer Use 的 ChatGPT.app。", + grokComputerUseUnavailable: "本机未找到 ChatGPT Desktop 的 Computer Use 文件。", + grokBrowserLabel: "Grok 内置浏览器", + grokBrowserTitle: "内置浏览器", + grokBrowserDescription: + "让 Grok 通过 cua_repl 使用 ChatGPT 内置浏览器(Browser Use)。与 Computer Use 相互独立。", + grokBrowserHelp: + "官方 Codex Browser Use 不受影响。ChatGPT 不授权连接或无法绑定内置浏览器标签页时,Grok 会直接失败。Chrome 扩展是另一条路径。", + grokBrowserOn: "已开启 · 浏览器能力已启用", + grokBrowserOff: "已关闭", + grokBrowserUnavailable: "本机未找到 ChatGPT Desktop 的内置浏览器文件。", + grokBrowserBlocked: "已保存为意向。ChatGPT Desktop 尚未授权 Grok 使用内置浏览器。", + grokComputerUseApplyFailed: "无法将已保存的设置写入 Grok 配置。", connectionOfficialWebsite: "访问官网", connectionOpenHarnessWeb: "打开 DeepSeek Harness Web", connectionDeepSeekTestedVersions: @@ -889,6 +978,7 @@ const CHINESE_MESSAGES: RendererSettingsMessages = Object.freeze({ connections: "连接", "remote-connections": "远程连接", appearance: "通用", + voice: "语音", accounts: "账号", "session-import": "会话导入", "usage-statistics": "用量统计", diff --git a/packages/renderer-extension/src/settings/pages.ts b/packages/renderer-extension/src/settings/pages.ts index d08854fb8..81e569765 100644 --- a/packages/renderer-extension/src/settings/pages.ts +++ b/packages/renderer-extension/src/settings/pages.ts @@ -1,3 +1,4 @@ +import { createVoiceSettingsPage, type VoiceSettingsRequestClient } from "./voice-page.js"; import { createUsageStatisticsPage, type RendererUsageStatisticsHostsProvider, @@ -50,9 +51,9 @@ import { runBoundedRendererUpdateRequest, } from "./update-request.js"; -export const CODEXHOST_GITHUB_REPOSITORY_URL = "https://github.com/BytePioneer-AI/codex-host"; +export const CODEXHOST_GITHUB_REPOSITORY_URL = "https://github.com/karlorz/codex-host"; export const CODEXHOST_RELEASES_LATEST_URL = `${CODEXHOST_GITHUB_REPOSITORY_URL}/releases/latest`; -export const CODEXHOST_NPM_MANUAL_UPDATE_COMMAND = "npm install -g @codexhost/cli@latest"; +export const CODEXHOST_NPM_MANUAL_UPDATE_COMMAND = "npm install -g @karlorz/codexhost-cli@latest"; interface RendererUserAgentData { readonly platform?: string; @@ -76,7 +77,7 @@ function windowsInstallerDownloadUrl(window: Window | null | undefined, version: const hints = navigator ? rendererUserAgentData(navigator) : undefined; const identity = `${hints?.architecture ?? ""} ${hints?.platform ?? ""} ${navigator?.platform ?? ""} ${navigator?.userAgent ?? ""}`; const architecture = /arm64|aarch64|\barm\b/iu.test(identity) ? "arm64" : "x64"; - return `https://github.com/BytePioneer-AI/codex-host/releases/download/v${version}/codexhost-${version}-windows-${architecture}.exe`; + return `${CODEXHOST_GITHUB_REPOSITORY_URL}/releases/download/v${version}/codexhost-${version}-windows-${architecture}.exe`; } export const DEFAULT_RENDERER_SETTINGS_PAGE_IDS = [ @@ -86,6 +87,7 @@ export const DEFAULT_RENDERER_SETTINGS_PAGE_IDS = [ "session-import", "usage-statistics", "appearance", + "voice", "updates", "about", ] as const; @@ -698,6 +700,8 @@ export function createDefaultRendererSettingsPages( requestUsageStatistics: RendererUsageStatisticsRequest = () => Promise.reject(new Error("Local Host connection is unavailable")), usageStatisticsHosts?: RendererUsageStatisticsHostsProvider, + requestVoiceSettings: VoiceSettingsRequestClient = () => + Promise.reject(new Error("Local Host connection is unavailable")), ): readonly RendererSettingsPageDefinition[] { return Object.freeze([ createConnectionsSettingsPage(messages, getDiagnostics, undefined, getRemoteConnections), @@ -716,6 +720,7 @@ export function createDefaultRendererSettingsPages( usageStatisticsHosts, ), createAppearanceSettingsPage(messages, getLoadedSessionsClient), + createVoiceSettingsPage(messages.locale, requestVoiceSettings, { desktop: true }), updatesPage(messages, getUpdateClient), aboutPage(messages, getUpdateClient), ]); diff --git a/packages/renderer-extension/src/settings/rail-page.ts b/packages/renderer-extension/src/settings/rail-page.ts index 71909f4a0..581d2491a 100644 --- a/packages/renderer-extension/src/settings/rail-page.ts +++ b/packages/renderer-extension/src/settings/rail-page.ts @@ -1,4 +1,5 @@ import { SETTINGS_TRIGGER_ATTRIBUTE } from "./trigger.js"; +import { attachSettingsBackNavigation } from "./back-navigation.js"; const NAVIGATION_RAIL_SELECTOR = "nav[data-app-navigation-rail]"; const RAIL_DESTINATION_SELECTOR = "[data-sidebar-destination]"; @@ -78,6 +79,7 @@ export function attachRendererSettingsRailPage( if (disposed) return; onNavigateAway(); }; + const backNavigation = attachSettingsBackNavigation(ownerDocument, navigateAway); const checkNavigation = (): void => { if (disposed) return; @@ -158,6 +160,7 @@ export function attachRendererSettingsRailPage( dispose() { if (disposed) return; disposed = true; + backNavigation.dispose(); railMutations?.disconnect(); railResize?.disconnect(); bodyMutations?.disconnect(); diff --git a/packages/renderer-extension/src/settings/shell.css b/packages/renderer-extension/src/settings/shell.css index 0f29126dc..e1d9de045 100644 --- a/packages/renderer-extension/src/settings/shell.css +++ b/packages/renderer-extension/src/settings/shell.css @@ -1359,6 +1359,69 @@ gap: 8px; } +.settings-grok-computer-use { + display: grid; + gap: 10px; + min-width: 0; + margin-top: 4px; + padding: 12px; + background: var(--settings-surface); + border: 1px solid var(--settings-divider); + border-radius: 12px; +} + +.settings-grok-computer-use__header { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 12px; +} + +.settings-grok-computer-use__copy { + display: grid; + gap: 4px; + min-width: 0; +} + +.settings-grok-computer-use__title { + margin: 0; + color: var(--settings-text); + font-size: 14px; + font-weight: 600; + line-height: 20px; +} + +.settings-grok-computer-use__description { + margin: 0; + color: var(--settings-muted); + font-size: 13px; + line-height: 18px; + overflow-wrap: anywhere; +} + +.settings-grok-computer-use__help, +.settings-grok-computer-use__status { + margin: 0; + color: var(--settings-subtle); + font-size: 12px; + line-height: 16px; + overflow-wrap: anywhere; +} + +.settings-grok-computer-use__help { + padding-top: 8px; + border-top: 1px solid var(--settings-divider); +} + +.settings-grok-computer-use__browser { + padding-top: 8px; + border-top: 1px solid var(--settings-divider); +} + +.settings-grok-computer-use__status[data-state="on"] { + color: var(--settings-success); +} + .settings-connection-install-callout { display: grid; grid-template-columns: 28px minmax(0, 1fr); @@ -2579,3 +2642,107 @@ animation: none !important; } } +.settings-voice__select { + flex: none; + width: 190px; + max-width: 100%; + min-height: 34px; + padding: 6px 10px; + border: 1px solid var(--settings-border); + border-radius: 6px; + color: var(--settings-text); + background: var(--settings-panel); + font: inherit; + font-size: 13px; +} + +.settings-voice :disabled { + opacity: 0.45; + cursor: not-allowed; +} +.settings-voice__footer { + display: flex; + align-items: center; + justify-content: space-between; + flex-wrap: wrap; + gap: 16px; + margin-top: 28px; + padding-top: 18px; + border-top: 1px solid var(--settings-divider); + font-size: 13px; +} +.settings-voice__health { + display: grid; + gap: 4px; + margin: 8px 12px 12px; + color: var(--settings-subtle); + font-size: 12px; + line-height: 16px; +} + +.settings-voice__health p { + margin: 0; +} + +.settings-voice__footer p, +.settings-voice__hint { + margin: 4px 0 0; + color: var(--settings-muted); + font-size: 12px; + line-height: 18px; +} +.settings-voice__actions, +.settings-voice__gateway-actions { + display: flex; + align-items: center; + flex-wrap: wrap; + gap: 12px; +} +.settings-voice__gateway-actions { + color: var(--settings-muted); + font-size: 12px; +} +.settings-voice__details { + padding: 12px 16px; + border-top: 1px solid var(--settings-divider); + color: var(--settings-muted); + font-size: 12px; + line-height: 18px; + overflow-wrap: anywhere; +} +.settings-voice__confirmation { + margin-top: 12px; + font-size: 13px; + line-height: 20px; +} +.settings-voice__confirmation button + button { + margin-left: 8px; +} +.settings-voice [role="alert"] { + color: var(--settings-danger); + font-size: 13px; + overflow-wrap: anywhere; +} +.settings-voice { + container-type: inline-size; +} +@container (max-width: 620px) { + .settings-voice__row { + align-items: stretch; + flex-direction: column; + gap: 10px; + } + .settings-voice__select { + width: 100%; + } + .settings-voice__row input[role="switch"] { + align-self: flex-start; + } + .settings-voice__footer { + align-items: stretch; + flex-direction: column; + } + .settings-voice__actions > button { + flex: 1; + } +} diff --git a/packages/renderer-extension/src/settings/voice-messages.ts b/packages/renderer-extension/src/settings/voice-messages.ts new file mode 100644 index 000000000..e9f3305a7 --- /dev/null +++ b/packages/renderer-extension/src/settings/voice-messages.ts @@ -0,0 +1,125 @@ +export function voiceMessages(locale: string) { + const en = { + title: "Voice", + description: "Choose voice for Chat and Work. Shared on this Mac.", + chat: "Chat", + experience: "Voice experience", + chatHint: "ChatGPT stays in your current Chat conversation.", + codexConnection: "Codex connection", + codexHint: "Available when Codex voice is selected.", + work: "Work", + connection: "Voice connection", + workHint: + "Desktop default applies to Codex Work. External Harness voice uses the configured gateway.", + desktop: "Desktop default", + gateway: "Configured gateway", + configuration: "Configuration", + gatewayTitle: "Gateway", + gatewayHint: "Used only when Configured gateway is selected.", + configured: "Configured", + needsSetup: "Needs setup", + unknown: "Not checked", + gatewayStatus: "Gateway configuration", + preferencesStatus: "Preferences applied", + hookStatus: "Desktop gateway voice integration", + preferencesOn: "Applied", + preferencesOff: "Not applied yet", + hookSupported: "Supported", + hookUnsupported: "This Desktop build is not supported for gateway voice", + hookPending: "Attaching…", + hookDisabled: "Off", + configure: "Configure", + setup: "Gateway setup", + setupHint: + "Gateway settings use your local Codex configuration and the selected provider’s credential. End voice before editing configuration. Never paste credentials here.", + advanced: "Advanced", + advancedHint: "Voice integration and diagnostics.", + inject: "Enable voice integration", + injectHint: + "Enables the ChatGPT voice control and gateway routing. Changing this requires restarting codexhost.", + environment: "Voice integration is controlled by this launch environment.", + loading: "Loading voice settings…", + unavailable: "Voice settings are unavailable. Check the local Host connection.", + retry: "Reload", + cancel: "Cancel", + apply: "Apply changes", + applyRestart: "Apply and restart codexhost", + retryRestart: "Retry restart", + saving: "Saving settings…", + restarting: "Restarting codexhost…", + ready: "No pending changes", + edited: "Unsaved changes", + nextCall: "Voice choices apply to the next call.", + pending: "Saved — waiting for Desktop to apply voice choices.", + restartNeeded: "Saved — restart required", + activeUnknown: "Saved — active setting is unknown.", + activeCall: "End your voice call to apply changes.", + integrationOff: "Voice integration is off. Enable it in Advanced to use these choices.", + restartWarning: + "Restart codexhost to apply voice integration? This may interrupt running tasks. End voice before continuing.", + confirm: "Apply and restart", + restartFailed: "Settings are saved, but restart failed.", + conflict: "Settings changed in another window. Reload before applying.", + gatewayWarning: "The selected gateway needs setup before voice can connect.", + }; + if (locale !== "zh-CN") return en; + return { + ...en, + title: "语音", + description: "选择 Chat 和 Work 的语音方式。在此 Mac 上共享。", + experience: "语音体验", + chatHint: "ChatGPT 语音保留在当前 Chat 对话中。", + codexConnection: "Codex 连接", + codexHint: "选择 Codex 语音后可用。", + connection: "语音连接", + workHint: "Desktop 默认适用于 Codex Work。外部 Harness 语音使用已配置的网关。", + desktop: "Desktop 默认", + gateway: "已配置的网关", + configuration: "配置", + gatewayTitle: "网关", + gatewayHint: "仅在选择已配置的网关时使用。", + configured: "已配置", + needsSetup: "需要配置", + unknown: "未检查", + gatewayStatus: "网关配置", + preferencesStatus: "偏好是否已应用", + hookStatus: "Desktop 网关语音集成", + preferencesOn: "已应用", + preferencesOff: "尚未应用", + hookSupported: "支持", + hookUnsupported: "当前 Desktop 版本不支持网关语音", + hookPending: "正在接入…", + hookDisabled: "已关闭", + configure: "配置", + setup: "网关配置", + setupHint: + "网关使用本地 Codex 配置及所选 Provider 的凭据。编辑前请结束语音。请勿在此粘贴凭据。", + advanced: "高级", + advancedHint: "语音集成与诊断。", + inject: "启用语音集成", + injectHint: "启用 ChatGPT 语音按钮与网关路由。更改此项需要重启 codexhost。", + environment: "当前启动环境控制语音集成。", + loading: "正在加载语音设置…", + unavailable: "语音设置不可用,请检查本地 Host 连接。", + retry: "重新加载", + cancel: "取消", + apply: "应用更改", + applyRestart: "应用并重启 codexhost", + retryRestart: "重试重启", + saving: "正在保存设置…", + restarting: "正在重启 codexhost…", + ready: "没有待应用的更改", + edited: "更改尚未保存", + nextCall: "语音选择从下次通话生效。", + pending: "已保存,等待 Desktop 应用语音选择。", + restartNeeded: "已保存,需要重启", + activeUnknown: "已保存,当前生效状态未知。", + activeCall: "请结束语音后再应用更改。", + integrationOff: "语音集成已关闭,请在高级设置中启用后使用这些选项。", + restartWarning: "重启 codexhost 以应用语音集成?这可能中断运行中的任务,请先结束语音。", + confirm: "应用并重启", + restartFailed: "设置已保存,但重启失败。", + conflict: "其他窗口更改了设置,请重新加载后再应用。", + gatewayWarning: "所选网关需要配置后才能连接语音。", + }; +} diff --git a/packages/renderer-extension/src/settings/voice-page.ts b/packages/renderer-extension/src/settings/voice-page.ts new file mode 100644 index 000000000..f69d4d134 --- /dev/null +++ b/packages/renderer-extension/src/settings/voice-page.ts @@ -0,0 +1,420 @@ +import { + voiceSettingsStateSchema, + type VoiceSettings, + type VoiceSettingsRequest, + type VoiceSettingsState, +} from "@codexhost/shared-contracts"; +import { desktopVoiceCallActive } from "../voice-settings-sync.js"; +import { h, button } from "../console/dom.js"; +import type { RendererSettingsPageDefinition, RendererSettingsPageMountContext } from "./core.js"; +import { + createPreferenceGroup, + createPreferenceItem, + createPreferenceSwitch, + preferenceId, +} from "./preference-ui.js"; +import { voiceMessages } from "./voice-messages.js"; + +export type VoiceSettingsRequestClient = (input: VoiceSettingsRequest) => Promise; + +/** Both settings surfaces edit the same snapshot with explicit apply semantics. */ +export function createVoiceSettingsPage( + locale: string, + request: VoiceSettingsRequestClient, + options: { id?: string; desktop?: boolean } = {}, +): RendererSettingsPageDefinition { + const m = voiceMessages(locale); + return Object.freeze({ + id: options.id ?? "voice", + label: m.title, + icon: "settings", + mount(context: RendererSettingsPageMountContext) { + const doc = context.content.ownerDocument; + let state: VoiceSettingsState | null = null; + let base: VoiceSettingsState | null = null; + let draft: VoiceSettings | null = null; + let offline = false; + let busy = false; + let refreshing = false; + let generation = 0; + let disposed = false; + let confirm = false; + let advanced = false; + let setup = false; + let error: string | null = null; + const ids = new Map(); + const controlId = (key: string) => { + const existing = ids.get(key); + if (existing) return existing; + const id = preferenceId(key); + ids.set(key, id); + return id; + }; + const dirty = () => !!draft && JSON.stringify(draft) !== JSON.stringify(base?.settings); + const isActiveCall = () => + state?.activeCall === true || + (options.desktop && !!doc.defaultView && desktopVoiceCallActive(doc.defaultView)); + const needsRestart = () => + !!draft && + !!state && + (state.activeInject === null + ? draft.inject !== base?.settings.inject + : (state.envOverride ? state.envOverride === "1" : draft.inject) !== state.activeInject); + const load = async (reset = false) => { + if (refreshing || busy || disposed) return; + refreshing = true; + const currentGeneration = generation; + try { + const next = voiceSettingsStateSchema.parse(await request({ operation: "get" })); + if (disposed || generation !== currentGeneration) return; + const changed = JSON.stringify(state) !== JSON.stringify(next) || offline || reset; + if (reset || !dirty()) { + base = next; + draft = { ...next.settings }; + } + state = next; + offline = false; + if (reset) { + error = null; + confirm = false; + } + if (changed) render(); + } catch { + if (!disposed && generation === currentGeneration) { + offline = true; + render(); + } + } finally { + refreshing = false; + } + }; + const apply = async (restart: boolean) => { + if (!draft || !base || busy || isActiveCall()) return; + busy = true; + generation += 1; + error = null; + confirm = false; + render(); + try { + const current = voiceSettingsStateSchema.parse(await request({ operation: "get" })); + if (disposed) return; + if (current.activeCall || isActiveCall()) throw new Error(m.activeCall); + if (current.revision !== base.revision) throw new Error(m.conflict); + if (dirty()) { + const saved = voiceSettingsStateSchema.parse( + await request({ + operation: "set", + settings: draft, + revision: base.revision, + }), + ); + if (disposed) return; + state = saved; + base = saved; + draft = { ...saved.settings }; + } + if (restart) await request({ operation: "restart", confirm: true }); + } catch (failure) { + if (!disposed) error = failure instanceof Error ? failure.message : m.unavailable; + } finally { + busy = false; + if (!disposed) { + render(); + void load(); + } + } + }; + const render = () => { + if (disposed) return; + const tree = context.content.getRootNode() as Document | ShadowRoot; + const focused = tree.activeElement; + const focusedId = focused && context.content.contains(focused) ? focused.id : ""; + const root = h(doc, "div", { className: "settings-voice" }); + root.append( + h(doc, "div", { className: "settings-section-label" }, m.title), + h(doc, "p", { className: "settings-page-description" }, m.description), + ); + if (!draft || !state) { + root.append(h(doc, "p", { role: "status" }, offline ? m.unavailable : m.loading)); + if (offline) root.append(button(doc, m.retry, () => void load(true))); + context.content.replaceChildren(root); + return; + } + const unavailable = busy || offline || state.restart.phase === "restarting"; + const row = ( + card: HTMLElement, + title: string, + description: string, + key: "chatExperience" | "chatConnection" | "workConnection", + choices: [string, string][], + disabled = false, + ) => { + const id = controlId(`voice-${key}`); + const item = createPreferenceItem(doc, { title, description, controlId: id }); + item.item.classList.add("settings-voice__row"); + const select = h(doc, "select", { + id, + className: "settings-voice__select", + "aria-describedby": item.description.id, + }); + for (const [value, label] of choices) select.append(h(doc, "option", { value }, label)); + select.value = draft?.[key] ?? ""; + select.disabled = unavailable || disabled; + select.addEventListener("change", () => { + if (!draft) return; + draft = { ...draft, [key]: select.value }; + confirm = false; + render(); + }); + item.item.append(select); + card.append(item.item); + }; + const connections: [string, string][] = [ + ["desktop", m.desktop], + ["gateway", m.gateway], + ]; + const chat = createPreferenceGroup(doc, m.chat); + row(chat.card, m.experience, m.chatHint, "chatExperience", [ + ["chatgpt", "ChatGPT"], + ["codex", "Codex"], + ]); + row( + chat.card, + m.codexConnection, + m.codexHint, + "chatConnection", + connections, + draft.chatExperience !== "codex", + ); + const work = createPreferenceGroup(doc, m.work); + row(work.card, m.connection, m.workHint, "workConnection", connections); + const config = createPreferenceGroup(doc, m.configuration); + const gateway = createPreferenceItem(doc, { + title: m.gatewayTitle, + description: m.gatewayHint, + controlId: preferenceId("gateway-config"), + }); + gateway.item.classList.add("settings-voice__row"); + const configure = button(doc, m.configure, () => { + setup = !setup; + render(); + }); + configure.id = gateway.item.querySelector("label")?.htmlFor ?? ""; + configure.setAttribute("aria-expanded", String(setup)); + const gatewayReadyLabel = + state.gatewayReady === null + ? m.unknown + : state.gatewayReady + ? m.configured + : m.needsSetup; + const gatewayActions = h( + doc, + "div", + { className: "settings-voice__gateway-actions" }, + h(doc, "span", {}, gatewayReadyLabel), + configure, + ); + gateway.item.append(gatewayActions); + config.card.append(gateway.item); + const hook = options.desktop ? desktopVoiceHookStatus(doc.defaultView) : "unknown"; + const hookLabel = + hook === "active" + ? m.hookSupported + : hook === "pending" + ? m.hookPending + : hook === "unsupported" + ? m.hookUnsupported + : hook === "disabled" + ? m.hookDisabled + : m.unknown; + const health = h(doc, "div", { className: "settings-voice__health" }); + health.append( + h(doc, "p", {}, `${m.gatewayStatus}: ${gatewayReadyLabel}`), + h( + doc, + "p", + {}, + `${m.preferencesStatus}: ${ + state.routesActive === true + ? m.preferencesOn + : state.routesActive === false + ? m.preferencesOff + : m.unknown + }`, + ), + h(doc, "p", {}, `${m.hookStatus}: ${hookLabel}`), + ); + config.card.append(health); + if (setup) { + const details = h( + doc, + "div", + { className: "settings-voice__details" }, + h(doc, "strong", {}, m.setup), + h(doc, "p", {}, m.setupHint), + ); + details.append( + h(doc, "code", {}, "~/.codex/config.toml"), + h( + doc, + "p", + {}, + "experimental_realtime_webrtc_call_base_url · experimental_realtime_ws_base_url", + ), + ); + config.card.append(details); + } + const adv = createPreferenceItem(doc, { + title: m.advanced, + description: m.advancedHint, + controlId: preferenceId("voice-advanced"), + }); + const expand = button(doc, advanced ? "⌃" : "›", () => { + advanced = !advanced; + render(); + }); + expand.id = adv.item.querySelector("label")?.htmlFor ?? ""; + expand.setAttribute("aria-label", m.advanced); + expand.setAttribute("aria-expanded", String(advanced)); + adv.item.append(expand); + config.card.append(adv.item); + if (advanced) { + const id = controlId("voice-inject"); + const injection = createPreferenceItem(doc, { + title: m.inject, + description: m.injectHint, + controlId: id, + }); + injection.item.classList.add("settings-voice__row"); + const toggle = createPreferenceSwitch(doc, id, injection.description.id); + toggle.checked = draft.inject; + toggle.disabled = unavailable || !!state.envOverride; + toggle.addEventListener("change", () => { + if (draft) draft = { ...draft, inject: toggle.checked }; + confirm = false; + render(); + }); + injection.item.append(toggle); + config.card.append(injection.item); + if (state.envOverride || state.activeEnvOverride) + config.card.append( + h(doc, "p", { className: "settings-voice__details" }, m.environment), + ); + } + root.append(chat.group, work.group, config.group); + const restarting = state.restart.phase === "restarting"; + const failed = state.restart.phase === "failed"; + const status = busy + ? m.saving + : restarting + ? m.restarting + : isActiveCall() + ? m.activeCall + : offline + ? m.unavailable + : dirty() + ? m.edited + : needsRestart() + ? m.restartNeeded + : state.activeInject === null + ? m.activeUnknown + : state.routesActive === false + ? m.pending + : state.routesActive === null + ? m.activeUnknown + : m.ready; + const footer = h( + doc, + "div", + { className: "settings-voice__footer" }, + h(doc, "div", { role: "status" }, h(doc, "div", {}, status), h(doc, "p", {}, m.nextCall)), + ); + const actions = h(doc, "div", { className: "settings-voice__actions" }); + const cancel = button(doc, m.cancel, () => { + draft = base ? { ...base.settings } : null; + confirm = false; + error = null; + render(); + }); + cancel.disabled = busy || (!dirty() && !confirm); + const submit = button( + doc, + failed && needsRestart() && !dirty() + ? m.retryRestart + : needsRestart() && state.restartAvailable + ? m.applyRestart + : m.apply, + () => { + if (needsRestart() && state?.restartAvailable) { + confirm = true; + render(); + } else void apply(false); + }, + "primary", + ); + submit.disabled = + unavailable || + !!isActiveCall() || + (!dirty() && !(needsRestart() && state.restartAvailable)); + actions.append(cancel, submit); + footer.append(actions); + root.append(footer); + if (confirm) + root.append( + h( + doc, + "div", + { className: "settings-voice__confirmation" }, + h(doc, "p", {}, m.restartWarning), + button(doc, m.confirm, () => void apply(true), "primary"), + button(doc, m.cancel, () => { + confirm = false; + render(); + }), + ), + ); + if (failed) + root.append( + h(doc, "p", { role: "alert" }, `${m.restartFailed} ${state.restart.error ?? ""}`), + ); + if (error) root.append(h(doc, "p", { role: "alert" }, error)); + const gatewaySelected = + draft.workConnection === "gateway" || + (draft.chatExperience === "codex" && draft.chatConnection === "gateway"); + if (gatewaySelected && state.gatewayReady === false) + root.append(h(doc, "p", { className: "settings-voice__hint" }, m.gatewayWarning)); + if (!draft.inject) + root.append(h(doc, "p", { className: "settings-voice__hint" }, m.integrationOff)); + if (offline || base?.revision !== state.revision) + root.append(button(doc, m.retry, () => void load(true))); + context.content.replaceChildren(root); + if (focusedId) + root.querySelector(`#${focusedId}`)?.focus({ preventScroll: true }); + }; + render(); + void load(); + const timer = doc.defaultView?.setInterval(() => void load(), 3_000); + return () => { + disposed = true; + if (timer !== undefined) doc.defaultView?.clearInterval(timer); + }; + }, + }); +} + +function desktopVoiceHookStatus( + win: Window | null, +): "active" | "pending" | "unsupported" | "disabled" | "unknown" { + if (!win) return "unknown"; + const value = (win as unknown as Record).__codexhostClientOwnedVoiceV1; + if (!value || typeof value !== "object") return "unknown"; + const status = (value as { status?: unknown }).status; + if ( + status === "active" || + status === "pending" || + status === "unsupported" || + status === "disabled" + ) + return status; + return "unknown"; +} diff --git a/packages/renderer-extension/src/voice-settings-sync.ts b/packages/renderer-extension/src/voice-settings-sync.ts new file mode 100644 index 000000000..71e7c3130 --- /dev/null +++ b/packages/renderer-extension/src/voice-settings-sync.ts @@ -0,0 +1,74 @@ +import { + VOICE_GATEWAY_STATUS_METHOD, + VOICE_SETTINGS_METHOD, + voiceSettingsStateSchema, +} from "@codexhost/shared-contracts"; +import { + applySharedVoiceRoutes, + applyVoiceGatewayReadiness, + legacyVoiceRoutes, +} from "./renderer-cpa-live-voice-preference.js"; + +export function desktopVoiceCallActive(owner: Window): boolean { + const hook = Reflect.get(owner, "__codexhostNativeChatVoiceV1") as + { activeCall?: boolean } | undefined; + if (hook?.activeCall) return true; + return [ + ...owner.document.querySelectorAll( + 'button[aria-label="End Voice"],button[aria-label="End voice"],button[aria-label="End voice chat"]', + ), + ].some((button) => button.getBoundingClientRect().width > 0); +} + +/** Runtime polls the same console-owned snapshot as the two settings surfaces. */ +export function startVoiceSettingsSync( + owner: Window, + request: (method: string, params: unknown) => Promise, + intervalMs = 2_000, +): () => void { + let disposed = false; + let busy = false; + let revision = ""; + const id = crypto.randomUUID(); + const refresh = async () => { + if (busy || disposed) return; + busy = true; + try { + const activeCall = desktopVoiceCallActive(owner); + let state = voiceSettingsStateSchema.parse( + await request(VOICE_SETTINGS_METHOD, { + operation: "get", + runtime: { id, revision, activeCall }, + }), + ); + if (disposed) return; + // Runtime reports omit the console's Host callback. Query the local Host + // directly so an omitted result is not mistaken for an unconfigured gateway. + const ready = state.gatewayReady ?? (await request(VOICE_GATEWAY_STATUS_METHOD, {})); + if (disposed) return; + applyVoiceGatewayReadiness(owner, ready === true); + if (!state.initialized) + state = voiceSettingsStateSchema.parse( + await request(VOICE_SETTINGS_METHOD, { + operation: "migrate", + routes: legacyVoiceRoutes(owner), + }), + ); + if (!disposed && !activeCall && state.revision !== revision) { + applySharedVoiceRoutes(owner, state.settings); + revision = state.revision; + } + } catch { + /* Keep the last gateway readiness and route while the local Host reconnects. */ + } finally { + busy = false; + } + }; + void refresh(); + const timer = owner.setInterval(() => void refresh(), intervalMs); + return () => { + disposed = true; + applyVoiceGatewayReadiness(owner, false); + owner.clearInterval(timer); + }; +} diff --git a/packages/renderer-extension/test/codex-locale-adapter.test.ts b/packages/renderer-extension/test/codex-locale-adapter.test.ts index 7cde4c32f..6a00fa388 100644 --- a/packages/renderer-extension/test/codex-locale-adapter.test.ts +++ b/packages/renderer-extension/test/codex-locale-adapter.test.ts @@ -347,6 +347,36 @@ describe("Codex locale adapter", () => { }); }); + it.each([ + ["en-US", "zh-TW", "en-US", "document"], + ["zh-CN", "en-US", "zh-CN", "document"], + ["", "zh-TW", "zh-TW", "navigator"], + ["invalid_locale", "en-US", "en-US", "navigator"], + ])( + "uses rendered Desktop language %s when the override endpoint is unavailable", + async (lang, browser, preferredLocale, source) => { + nativeLocale({ ideLocale: browser, systemLocale: browser }); + const ownerWindow = localeWindow({ "vscode://codex/get-setting": handlerMissing }, [browser]); + Object.assign(ownerWindow, { document: { documentElement: { lang } } }); + expect(await readCodexLocaleSettings({ ownerWindow })).toMatchObject({ + status: "fallback", + mode: "unavailable", + preferredLocale, + source, + }); + }, + ); + + it("keeps a readable explicit override ahead of the rendered document language", async () => { + const ownerWindow = localeWindow({ "vscode://codex/get-setting": { value: "en-US" } }); + Object.assign(ownerWindow, { document: { documentElement: { lang: "zh-CN" } } }); + expect(await readCodexLocaleSettings({ ownerWindow, timeoutMs: 10 })).toMatchObject({ + preferredLocale: "en-US", + source: "override", + mode: "explicit", + }); + }); + it("writes only a bounded locale override through the fixed setting endpoint", async () => { const sentRequests: FetchRequest[] = []; const ownerWindow = localeWindow( diff --git a/packages/renderer-extension/test/console/diagnostics-page.test.ts b/packages/renderer-extension/test/console/diagnostics-page.test.ts index db82dd6d8..7fc8da8ae 100644 --- a/packages/renderer-extension/test/console/diagnostics-page.test.ts +++ b/packages/renderer-extension/test/console/diagnostics-page.test.ts @@ -38,6 +38,7 @@ function overview(): ConsoleOverview { ], controller: null, launchAvailable: true, + liveVoice: { inject: true, envOverride: null }, summary: { state: "running", detail: null }, issueUrl: "", hostAvailable: false, @@ -86,7 +87,7 @@ describe("overview startup diagnostics", () => { refresh?.(); const ready = JSON.stringify(replaceChildren.mock.calls.at(-1)); expect(ready).toContain("codexhost 正在运行"); - expect(ready).not.toContain('"tag":"button"'); + expect(ready).toContain("重新启动 codexhost"); }); it("does not present an abandoned starting record as a successful startup", () => { @@ -155,6 +156,7 @@ describe("startup failure step", () => { it("names the step of a stage that was still waiting", () => { expect(failedStep(stages("waiting for Desktop Controller readiness"))).toBe("connectDesktop"); expect(failedStep(stages("waiting for Host chain"))).toBe("startHost"); + expect(failedStep(stages("recovering leftover Desktop Controller"))).toBe("ownership"); }); it("falls back to preparation for unknown or missing stages", () => { diff --git a/packages/renderer-extension/test/renderer-binding-probe-host-catalog.test.ts b/packages/renderer-extension/test/renderer-binding-probe-host-catalog.test.ts index 0189b8c06..f4b2f2cb8 100644 --- a/packages/renderer-extension/test/renderer-binding-probe-host-catalog.test.ts +++ b/packages/renderer-extension/test/renderer-binding-probe-host-catalog.test.ts @@ -224,6 +224,8 @@ function installFakeBrowser(): void { dispatchEvent: listeners.dispatchEvent.bind(listeners), setTimeout, clearTimeout, + setInterval, + clearInterval, open: vi.fn(), }; const document_ = { diff --git a/packages/renderer-extension/test/renderer-binding-probe.test.ts b/packages/renderer-extension/test/renderer-binding-probe.test.ts index b636b8519..539fe24d8 100644 --- a/packages/renderer-extension/test/renderer-binding-probe.test.ts +++ b/packages/renderer-extension/test/renderer-binding-probe.test.ts @@ -42,7 +42,9 @@ import { isComposerVoiceButton, creditsPlacementAnchor, isNativeContextUsageControlCandidate, + isNativeUsageBannerCandidate, nativeContextUsageControlForComposer, + nativeUsageBannerForComposer, reconcileComposerNativeControls, refreshSendButton, trailingActionAnchor, @@ -840,6 +842,150 @@ describe("Renderer Composer DOM behavior", () => { expect(isNativeContextUsageControlCandidate(credits)).toBe(false); }); + it("recognizes only a unique Codex usage recovery aside", () => { + const banner = { + localName: "aside", + getAttribute: (name: string) => + name === "role" ? "status" : name === "aria-live" ? "polite" : null, + querySelector: (selector: string) => (selector === "h3" || selector === "button" ? {} : null), + } as unknown as HTMLElement; + const unmarked = { + localName: "aside", + getAttribute: () => null, + querySelector: (selector: string) => (selector === "h3" || selector === "button" ? {} : null), + } as unknown as HTMLElement; + const notice = { + localName: "aside", + getAttribute: (name: string) => + name === "role" ? "status" : name === "aria-live" ? "polite" : null, + querySelector: () => null, + } as unknown as HTMLElement; + expect(isNativeUsageBannerCandidate(banner)).toBe(true); + expect(isNativeUsageBannerCandidate(unmarked)).toBe(true); + expect(isNativeUsageBannerCandidate(notice)).toBe(false); + const composer = { + querySelectorAll: (selector: string) => (selector === "aside" ? [unmarked] : []), + } as unknown as Element; + expect(nativeUsageBannerForComposer(composer)).toBe(unmarked); + const ambiguous = { + querySelectorAll: () => [banner, unmarked], + } as unknown as Element; + expect(nativeUsageBannerForComposer(ambiguous)).toBeNull(); + }); + + it("hides the Codex usage banner on any Composer by default, including Codex threads", () => { + vi.stubGlobal("document", { activeElement: null }); + const banner = { + localName: "aside", + hidden: false, + getAttribute: (name: string) => { + if (name === "role") return "status"; + if (name === "aria-live") return "polite"; + if (name === "aria-hidden") return banner.ariaHidden; + if (name === "aria-expanded") return null; + return null; + }, + setAttribute: (name: string, value: string) => { + if (name === "aria-hidden") banner.ariaHidden = value; + }, + removeAttribute: (name: string) => { + if (name === "aria-hidden") banner.ariaHidden = null; + }, + querySelector: (selector: string) => (selector === "h3" || selector === "button" ? {} : null), + contains: () => false, + click: vi.fn(), + ariaHidden: null as string | null, + }; + const control = { + composer: { + querySelectorAll: (selector: string) => (selector === "aside" ? [banner] : []), + }, + modelPicker: { root: { parentElement: null }, trigger: {} }, + nativeModelControl: null, + nativePermissionModeControl: null, + nativeContextUsageControl: null, + nativeUsageBanner: null, + credits: { + anchor: null, + place: vi.fn(), + root: { remove: vi.fn() }, + }, + usage: { + anchor: null, + place: vi.fn(), + root: { remove: vi.fn() }, + }, + } as unknown as ComposerAgentControl; + + reconcileComposerNativeControls(control, true, true); + expect(banner.hidden).toBe(true); + expect(banner.ariaHidden).toBe("true"); + expect(control.nativeUsageBanner?.element).toBe(banner); + + reconcileComposerNativeControls(control, false, false); + expect(banner.hidden).toBe(true); + expect(banner.ariaHidden).toBe("true"); + vi.unstubAllGlobals(); + }); + + it("keeps the Codex usage banner when the Settings preference is off", () => { + vi.stubGlobal("document", { activeElement: null, defaultView: null }); + const owner = Object.assign(new EventTarget(), { + localStorage: { getItem: () => "0", setItem: vi.fn() }, + }); + const banner = { + localName: "aside", + hidden: false, + getAttribute: (name: string) => { + if (name === "role") return "status"; + if (name === "aria-live") return "polite"; + if (name === "aria-hidden") return banner.ariaHidden; + if (name === "aria-expanded") return null; + return null; + }, + setAttribute: (name: string, value: string) => { + if (name === "aria-hidden") banner.ariaHidden = value; + }, + removeAttribute: (name: string) => { + if (name === "aria-hidden") banner.ariaHidden = null; + }, + querySelector: (selector: string) => (selector === "h3" || selector === "button" ? {} : null), + contains: () => false, + click: vi.fn(), + ariaHidden: null as string | null, + }; + const control = { + composer: { + ownerDocument: { defaultView: owner }, + querySelectorAll: (selector: string) => (selector === "aside" ? [banner] : []), + }, + modelPicker: { root: { parentElement: null }, trigger: {} }, + nativeModelControl: null, + nativePermissionModeControl: null, + nativeContextUsageControl: null, + nativeUsageBanner: null, + credits: { + anchor: null, + place: vi.fn(), + root: { remove: vi.fn() }, + }, + usage: { + anchor: null, + place: vi.fn(), + root: { remove: vi.fn() }, + }, + } as unknown as ComposerAgentControl; + + reconcileComposerNativeControls(control, true, true); + expect(banner.hidden).toBe(false); + expect(banner.ariaHidden).toBeNull(); + + reconcileComposerNativeControls(control, false, false); + expect(banner.hidden).toBe(false); + expect(banner.ariaHidden).toBeNull(); + vi.unstubAllGlobals(); + }); + it("does not treat attachment controls as submission", () => { const button = (label: string, type = "button") => ({ diff --git a/packages/renderer-extension/test/renderer-client-owned-voice.test.ts b/packages/renderer-extension/test/renderer-client-owned-voice.test.ts new file mode 100644 index 000000000..74914a905 --- /dev/null +++ b/packages/renderer-extension/test/renderer-client-owned-voice.test.ts @@ -0,0 +1,706 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import { REALTIME_CALL_METHOD } from "@codexhost/shared-contracts"; +import { + CLIENT_OWNED_VOICE_INITIAL_ASSET, + CLIENT_OWNED_VOICE_SHARED_ASSET, + installClientOwnedVoiceHook, + isExternalHarnessComposer, + isWorkComposerSurface, + rememberVoiceProductExperience, + wrapClientOwnedVoiceFetch, + type NativeVoiceBridge, +} from "../src/renderer-client-owned-voice.js"; +import { CPA_LIVE_VOICE_HOOK_GLOBAL } from "../src/renderer-cpa-live-voice-hook.js"; +import { wrapGatewayVoiceStarter } from "../src/renderer-voice-start-capability.js"; +import { + CHAT_LIVE_VOICE_STORAGE_KEY, + CPA_LIVE_VOICE_STORAGE_KEY, +} from "../src/renderer-cpa-live-voice-preference.js"; + +function fixture( + result = { status: 201, body: "answer", headers: { location: "/v1/realtime/calls/rtc_fixture" } }, +) { + const listeners = new Set<(event: unknown) => void>(); + const dispatch = vi.fn((_type: string, value: Record) => { + const request = value.request as { id: string; params: { operation: string } }; + queueMicrotask(() => { + for (const listener of listeners) + listener({ + hostId: "local", + message: { + id: request.id, + result: + request.params.operation === "create" + ? result + : { status: 204, body: "", headers: {} }, + }, + }); + }); + }); + const bridge: NativeVoiceBridge = { + dispatchMessage: dispatch, + subscribe: (_type, listener) => { + listeners.add(listener); + return () => { + listeners.delete(listener); + }; + }, + }; + const original = vi.fn<(url: string, options?: RequestInit) => Promise>( + async () => new Response("native"), + ); + const client = { fetch: original as (url: string, options?: RequestInit) => Promise }; + const responseError = vi.fn(async (response: Response) => + Object.assign(new Error(await response.text()), { status: response.status }), + ); + return { bridge, dispatch, listeners, client, original, responseError }; +} +function sharedExports(f: ReturnType) { + const fetch = { getInstance: () => f.client }; + const bridge = { getInstance: () => f.bridge }; + return { + Oit: vi.fn(), + Eit: fetch, + AFt: bridge, + Mit: f.responseError, + Zot: fetch, + Htr: fetch, + bIt: bridge, + Qot: { realtimeVoiceRuntime: vi.fn() }, + }; +} +const url = "/wham/realtime/calls?intent=quicksilver&architecture=avas"; +const body = (model: string | undefined) => + JSON.stringify({ sdp: "offer", ...(model === undefined ? {} : { session: { model } }) }); +const speechBody = () => body("gpt-live-1-codex"); +const chatBody = () => body("gpt-live"); +const workMeta = { + "x-codex-turn-metadata": JSON.stringify({ backend_model_name: "gpt-5.3-codex" }), +}; +const options = (requestBody: string = speechBody(), headers?: HeadersInit) => ({ + method: "POST" as const, + body: requestBody, + ...(headers ? { headers } : {}), +}); +const workOptions = (requestBody: string = speechBody()) => options(requestBody, workMeta); +afterEach(() => { + vi.useRealTimers(); + rememberVoiceProductExperience({ productExperience: "" }); +}); + +describe("client-owned voice gateway", () => { + it("creates the native call through Host rather than Desktop OAuth", async () => { + const f = fixture(); + const dispose = wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + const response = await f.client.fetch(url, workOptions()); + expect(response.status).toBe(201); + expect(await response.text()).toBe("answer"); + expect(response.headers.get("location")).toBe("/v1/realtime/calls/rtc_fixture"); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch).toHaveBeenCalledWith( + "mcp-request", + expect.objectContaining({ + hostId: "local", + request: expect.objectContaining({ + method: REALTIME_CALL_METHOD, + params: expect.objectContaining({ operation: "create", body: speechBody() }), + }), + }), + ); + expect(f.listeners.size).toBe(0); + dispose(); + expect(f.client.fetch).toBe(f.original); + }); + it("strips Desktop account credentials and cookies before crossing the bridge", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await f.client.fetch(url, { + ...workOptions(), + headers: { + ...workMeta, + Authorization: "Bearer desktop-oauth", + Cookie: "session=private", + "ChatGPT-Account-Id": "disabled-account", + "openai-alpha": "quicksilver=v2", + "Thread-Id": "thread-fixture", + }, + }); + const params = f.dispatch.mock.calls + .map( + ([, value]) => (value.request as { params: { headers?: Record } }).params, + ) + .find((entry) => entry.headers !== undefined); + expect(params?.headers).toEqual({ + "openai-alpha": "quicksilver=v2", + "thread-id": "thread-fixture", + "x-codex-turn-metadata": workMeta["x-codex-turn-metadata"], + }); + const serialized = JSON.stringify(f.dispatch.mock.calls); + expect(serialized).not.toContain("desktop-oauth"); + expect(serialized).not.toContain("disabled-account"); + expect(serialized).not.toContain("session=private"); + }); + it("strips Desktop account credentials on an intercepted Chat call too", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => true, + ); + await f.client.fetch(url, { + ...options(chatBody()), + headers: { + Authorization: "Bearer desktop-oauth", + Cookie: "session=private", + "ChatGPT-Account-Id": "disabled-account", + "openai-alpha": "quicksilver=v2", + }, + }); + expect(f.original).not.toHaveBeenCalled(); + const serialized = JSON.stringify(f.dispatch.mock.calls); + expect(serialized).not.toContain("desktop-oauth"); + expect(serialized).not.toContain("disabled-account"); + expect(serialized).not.toContain("session=private"); + expect(serialized).toContain("quicksilver=v2"); + }); + it.each([ + "/other", + "/wham/realtime/calls/rtc_fixture/hangup", + "https://chatgpt.com/wham/realtime/calls", + "/wham/realtime/calls-extra", + ])("preserves unrelated requests: %s", async (path) => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + expect(await (await f.client.fetch(path, options())).text()).toBe("native"); + expect(f.dispatch).not.toHaveBeenCalled(); + }); + it("treats lastAgent grok as external only for a Work voice launch", () => { + const grokOwner = { + localStorage: { + getItem: () => JSON.stringify({ version: 1, lastAgent: "grok", externalByAgent: {} }), + }, + }; + rememberVoiceProductExperience({ productExperience: "chat" }); + expect(isExternalHarnessComposer(grokOwner)).toBe(false); + rememberVoiceProductExperience({ productExperience: "work" }); + expect(isExternalHarnessComposer(grokOwner)).toBe(true); + rememberVoiceProductExperience({ productExperience: "work" }); + expect( + isExternalHarnessComposer({ + localStorage: { + getItem: () => JSON.stringify({ version: 1, lastAgent: "codex", externalByAgent: {} }), + }, + }), + ).toBe(false); + }); + it("sends external Harness voice through Host even when the Work gateway switch is off", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => false, + () => false, + () => true, + ); + const response = await f.client.fetch(url, options(speechBody())); + expect(response.status).toBe(201); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch).toHaveBeenCalledOnce(); + }); + it.each(["codexhost/grok/grok-4", "codexhost/claude/opus"])( + "routes the cold overlay's explicit Harness metadata through Host: %s", + async (model) => { + const f = fixture(); + // No primary Composer, remembered launch, or lastAgent exists in this realm. + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => false, + () => false, + () => false, + ); + await f.client.fetch( + url, + options(speechBody(), { + "x-codex-turn-metadata": JSON.stringify({ backend_model_name: model }), + }), + ); + expect(f.dispatch).toHaveBeenCalledOnce(); + expect(f.original).not.toHaveBeenCalled(); + }, + ); + it("honors the Work preference at call time", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => false, f.responseError); + await f.client.fetch(url, workOptions()); + expect(f.original).toHaveBeenCalledOnce(); + expect(f.dispatch).not.toHaveBeenCalled(); + }); + it.each([ + { experience: "work", model: "gpt-6.1-sol" }, + { experience: "work", model: undefined }, + { experience: "codex", model: "gpt-6.1-sol" }, + { experience: "codex", model: undefined }, + ])( + "routes an explicit $experience overlay start through Host without a Work composer: $model", + async ({ experience, model }) => { + const f = fixture(); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => true, + f.responseError, + () => false, + () => false, + () => false, + ); + const starter = wrapGatewayVoiceStarter( + () => + f.client.fetch( + url, + options( + speechBody(), + model + ? { "x-codex-turn-metadata": JSON.stringify({ backend_model_name: model }) } + : undefined, + ), + ), + { + qualifies: () => true, + ready: async () => true, + prepare: async () => ({ current: () => true, dispose() {} }), + committed: async () => {}, + }, + ); + const response = (await starter.start({ + productExperience: experience, + source: "composer_button_new_thread", + hostId: "local", + })) as Response; + expect(response.status).toBe(201); + expect(f.dispatch).toHaveBeenCalledOnce(); + expect(f.original).not.toHaveBeenCalled(); + }, + ); + it.each(["work", "codex"])( + "uses the Work preference for an explicit %s overlay start when Chat uses gateway", + async (experience) => { + const f = fixture(); + rememberVoiceProductExperience({ productExperience: experience }); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => true, + () => false, + () => false, + ); + await f.client.fetch(url, options(speechBody())); + expect(f.original).toHaveBeenCalledOnce(); + expect(f.dispatch).not.toHaveBeenCalled(); + }, + ); + it.each([ + ["gpt-live", chatBody()], + ["shared speech model gpt-live-1-codex", speechBody()], + ["no session", JSON.stringify({ sdp: "offer" })], + ])( + "leaves Chat %s on native /wham so Desktop Chat voice can own Instant Connect", + async (_name, request) => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + expect(await (await f.client.fetch(url, options(request))).text()).toBe("native"); + expect(f.original).toHaveBeenCalledOnce(); + expect(f.original.mock.calls[0]?.[0]).toBe(url); + expect(f.dispatch).not.toHaveBeenCalled(); + }, + ); + it("records Chat passthrough debug without credentials", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await f.client.fetch(url, { + ...options(chatBody()), + headers: { Authorization: "Bearer secret-token", "Thread-Id": "conv-chat-1" }, + }); + const log = (globalThis as { __codexhostChatVoiceDebug?: unknown[] }).__codexhostChatVoiceDebug; + expect(JSON.stringify(log)).not.toContain("secret-token"); + expect(JSON.stringify(log)).toContain("chat-native-passthrough"); + expect(JSON.stringify(log)).toContain('"hasThreadId":true'); + const persisted = globalThis.localStorage?.getItem("codexhost.chat-voice-debug.v1"); + if (persisted !== undefined && persisted !== null) + expect(persisted).toContain("chat-native-passthrough"); + }); + it("keeps Desktop login headers on native Chat /wham", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await f.client.fetch(url, { + ...options(chatBody()), + headers: { + Authorization: "Bearer desktop-oauth", + "ChatGPT-Account-Id": "acct-fixture", + }, + }); + const forwarded = f.original.mock.calls[0]?.[1]?.headers as Record; + expect(forwarded.Authorization).toBe("Bearer desktop-oauth"); + expect(forwarded["ChatGPT-Account-Id"]).toBe("acct-fixture"); + expect(f.original.mock.calls[0]?.[0]).toBe(url); + expect(f.dispatch).not.toHaveBeenCalled(); + }); + it("treats a Harness backend model as Work even on the Chat surface", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await f.client.fetch( + url, + options(speechBody(), { + "x-codex-turn-metadata": JSON.stringify({ + backend_model_name: "codexhost/plugin-v1@grok-test-model", + }), + }), + ); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch).toHaveBeenCalledOnce(); + }); + it("treats the Work composer surface as Work without backend metadata", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => true, + f.responseError, + () => false, + () => true, + ); + await f.client.fetch(url, options(speechBody())); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch).toHaveBeenCalledOnce(); + }); + it("does not treat a gpt-live backend name as Work", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await f.client.fetch( + url, + options(speechBody(), { + "x-codex-turn-metadata": JSON.stringify({ backend_model_name: "gpt-live-1-codex" }), + }), + ); + expect(f.original).toHaveBeenCalledOnce(); + expect(f.original.mock.calls[0]?.[0]).toBe(url); + expect(f.dispatch).not.toHaveBeenCalled(); + }); + it("routes a Chat call through Host only when the Chat override is on", async () => { + const f = fixture(); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => true, + ); + const response = await f.client.fetch(url, options(chatBody())); + expect(response.status).toBe(201); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch.mock.calls[0]?.[1]).toMatchObject({ + request: { params: { operation: "create", body: chatBody() } }, + }); + }); + it("preserves gateway quota errors without falling back to the signed-in account", async () => { + const f = fixture({ status: 429, body: "usage_limit_reached", headers: { location: "" } }); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + await expect(f.client.fetch(url, workOptions())).rejects.toMatchObject({ + message: "usage_limit_reached", + status: 429, + }); + expect(f.original).not.toHaveBeenCalled(); + }); + it("hangs up an answer that arrives after cancellation", async () => { + const f = fixture(); + const abort = new AbortController(); + wrapClientOwnedVoiceFetch(f.client, f.bridge, () => true, f.responseError); + const creation = f.client.fetch(url, { ...workOptions(), signal: abort.signal }); + abort.abort(); + await expect(creation).rejects.toMatchObject({ name: "AbortError" }); + expect( + f.dispatch.mock.calls.map(([, value]) => (value.request as { params: unknown }).params), + ).toContainEqual({ operation: "hangup", callId: "rtc_fixture" }); + }); + it("installs only after both audited assets are present", async () => { + vi.useFakeTimers(); + const f = fixture(); + const names = [CLIENT_OWNED_VOICE_INITIAL_ASSET]; + const target = { + [CPA_LIVE_VOICE_HOOK_GLOBAL]: { + enabled: true, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: { + querySelectorAll: () => names.map((name) => ({ getAttribute: () => `/assets/${name}` })), + } as unknown as ParentNode, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async () => sharedExports(f)); + const hook = installClientOwnedVoiceHook({ target, importModule }); + expect(importModule).not.toHaveBeenCalled(); + names.push(CLIENT_OWNED_VOICE_SHARED_ASSET); + await vi.advanceTimersByTimeAsync(250); + expect(hook.status).toBe("active"); + expect(importModule).toHaveBeenCalledWith(`app://-/assets/${CLIENT_OWNED_VOICE_SHARED_ASSET}`); + hook.dispose(); + expect(f.client.fetch).toBe(f.original); + }); + it.each([ + ["26.1021.31345", "app-initial-feb30b196d11.js", "app-shared-2cae01273bae.js"], + ["26.1016.62342", "app-initial-dddef47d6ef6.js", "app-shared-fc3316d48427.js"], + ["26.1016.61520", "app-initial-c95ddb62696e.js", "app-shared-2eda13eb7044.js"], + ])( + "routes explicit Work through its qualified %s WebFetch", + async (desktopVersion, initial, shared) => { + vi.useFakeTimers(); + const f = fixture(); + const target = { + [CPA_LIVE_VOICE_HOOK_GLOBAL]: { + enabled: true, + desktopVersion, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: { + querySelectorAll: () => + [initial, shared].map((name) => ({ getAttribute: () => `app://-/assets/${name}` })), + } as unknown as ParentNode, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async () => sharedExports(f)); + const hook = installClientOwnedVoiceHook({ target, importModule }); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(importModule).toHaveBeenCalledWith(`app://-/assets/${shared}`); + rememberVoiceProductExperience({ productExperience: "work" }); + await f.client.fetch( + url, + options(speechBody(), { + "x-codex-turn-metadata": JSON.stringify({ backend_model_name: "gpt-6.1-sol" }), + }), + ); + expect(f.dispatch).toHaveBeenCalledOnce(); + expect(f.original).not.toHaveBeenCalled(); + hook.dispose(); + expect(f.client.fetch).toBe(f.original); + }, + ); + + it("initializes lazy native WebFetch exports before installing on a cold document", async () => { + vi.useFakeTimers(); + const f = fixture(); + const target = { + [CPA_LIVE_VOICE_HOOK_GLOBAL]: { + enabled: true, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: { + querySelectorAll: () => + [CLIENT_OWNED_VOICE_INITIAL_ASSET, CLIENT_OWNED_VOICE_SHARED_ASSET].map((name) => ({ + getAttribute: () => `/assets/${name}`, + })), + } as unknown as ParentNode, + performance: { getEntriesByType: () => [] }, + setTimeout, + clearTimeout, + }; + Object.assign(target[CPA_LIVE_VOICE_HOOK_GLOBAL], { desktopVersion: "26.1016.62342" }); + const module: Record = { + Oit: vi.fn(() => { + module.Eit = { getInstance: () => f.client }; + module.AFt = { getInstance: () => f.bridge }; + module.Mit = f.responseError; + }), + }; + const names = ["app-initial-dddef47d6ef6.js", "app-shared-fc3316d48427.js"]; + target.document = { + querySelectorAll: () => names.map((name) => ({ getAttribute: () => `/assets/${name}` })), + } as unknown as ParentNode; + const hook = installClientOwnedVoiceHook({ target, importModule: async () => module }); + await vi.advanceTimersByTimeAsync(0); + expect(module.Oit).toHaveBeenCalledOnce(); + expect(hook.status).toBe("active"); + await f.client.fetch(url, workOptions()); + expect(f.dispatch).toHaveBeenCalledOnce(); + expect(f.original).not.toHaveBeenCalled(); + hook.dispose(); + }); + it("still installs with the Work override off and applies the Chat override", async () => { + vi.useFakeTimers(); + const f = fixture(); + const values = new Map([ + [CPA_LIVE_VOICE_STORAGE_KEY, "0"], + [CHAT_LIVE_VOICE_STORAGE_KEY, "1"], + ]); + const target = { + [CPA_LIVE_VOICE_HOOK_GLOBAL]: { + enabled: true, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + localStorage: { getItem: (key: string) => values.get(key) ?? null }, + document: { + querySelectorAll: () => + [CLIENT_OWNED_VOICE_INITIAL_ASSET, CLIENT_OWNED_VOICE_SHARED_ASSET].map((name) => ({ + getAttribute: () => `/assets/${name}`, + })), + } as unknown as ParentNode, + performance: { getEntriesByType: () => [] }, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async () => sharedExports(f)); + const hook = installClientOwnedVoiceHook({ target, importModule }); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + await f.client.fetch(url, options(chatBody())); + expect(f.dispatch).toHaveBeenCalledOnce(); + await f.client.fetch(url, workOptions()); + expect(f.original).toHaveBeenCalledOnce(); + hook.dispose(); + }); +}); + +describe("explicit Chat launch ownership", () => { + it("keeps Chat /wham passthrough despite stale Work surface, Harness model and lastAgent", async () => { + const f = fixture(); + rememberVoiceProductExperience({ productExperience: "chat" }); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => true, + f.responseError, + () => false, + () => true, + () => true, + ); + const request = options(speechBody(), { + "x-codex-turn-metadata": JSON.stringify({ backend_model_name: "codexhost/grok/grok-4" }), + }); + await f.client.fetch(url, request); + expect(f.original).toHaveBeenCalledWith(url, request); + expect(f.dispatch).not.toHaveBeenCalled(); + }); + + it("applies the Chat override rather than the Work switch to an explicit Chat launch", async () => { + const f = fixture(); + rememberVoiceProductExperience({ productExperience: "chat" }); + wrapClientOwnedVoiceFetch( + f.client, + f.bridge, + () => false, + f.responseError, + () => true, + () => true, + () => false, + ); + await f.client.fetch(url, workOptions()); + expect(f.original).not.toHaveBeenCalled(); + expect(f.dispatch).toHaveBeenCalledOnce(); + }); +}); + +describe("isWorkComposerSurface", () => { + function owner(mode: string | null, tabs: { name: string; selected?: boolean }[]) { + const storage = { + getItem: (key: string) => (key === "home-composer-mode-v1" ? mode : null), + }; + const nodes = tabs.map((tab) => ({ + textContent: tab.name, + getAttribute: (name: string) => + name === "aria-selected" ? (tab.selected ? "true" : "false") : "", + })); + return { + localStorage: storage, + document: { querySelectorAll: () => nodes }, + }; + } + + it("treats a selected Work tab as Work when composer-mode storage is empty", () => { + expect( + isWorkComposerSurface(owner(null, [{ name: "Chat" }, { name: "Work", selected: true }])), + ).toBe(true); + }); + + it("treats a visible official Work composer as Work when storage is empty", () => { + const composer = { + getBoundingClientRect: () => ({ width: 800 }), + closest: () => null, + }; + expect( + isWorkComposerSurface({ + localStorage: { getItem: () => null }, + document: { + querySelectorAll: (selector: string) => + selector === "[data-codex-composer-root]" + ? [composer] + : selector === "[data-chatgpt-composer]" + ? [] + : [], + }, + }), + ).toBe(true); + }); + + it("does not treat official Chat as Work", () => { + expect( + isWorkComposerSurface(owner(null, [{ name: "Chat", selected: true }, { name: "Work" }])), + ).toBe(false); + }); + + it.each(["zero-size", "hidden", "inert"])( + "ignores a retained %s Chat composer beside the visible Work composer", + (hiddenBy) => { + const work = { getBoundingClientRect: () => ({ width: 800 }), closest: () => null }; + const chat = { + getBoundingClientRect: () => ({ width: hiddenBy === "zero-size" ? 0 : 800 }), + closest: () => (hiddenBy === "zero-size" ? null : {}), + }; + expect( + isWorkComposerSurface({ + localStorage: { getItem: () => null }, + document: { + querySelectorAll: (selector: string) => + selector === "[data-codex-composer-root]" + ? [work] + : selector === "[data-chatgpt-composer]" + ? [chat] + : [], + }, + }), + ).toBe(true); + }, + ); + + it("rejects a Work surface when a visible Chat composer is also present", () => { + const visible = { getBoundingClientRect: () => ({ width: 800 }), closest: () => null }; + expect( + isWorkComposerSurface({ + localStorage: { getItem: () => null }, + document: { + querySelectorAll: (selector: string) => + selector === "[data-codex-composer-root]" || selector === "[data-chatgpt-composer]" + ? [visible] + : [], + }, + }), + ).toBe(false); + }); + + it("keeps explicit chat storage off Work even if a Work tab is selected", () => { + expect(isWorkComposerSurface(owner("chat", [{ name: "Work", selected: true }]))).toBe(false); + }); +}); diff --git a/packages/renderer-extension/test/renderer-codex-usage-banner-preference.test.ts b/packages/renderer-extension/test/renderer-codex-usage-banner-preference.test.ts new file mode 100644 index 000000000..6050d7c29 --- /dev/null +++ b/packages/renderer-extension/test/renderer-codex-usage-banner-preference.test.ts @@ -0,0 +1,56 @@ +import { describe, expect, it, vi } from "vitest"; + +import { + CODEX_USAGE_BANNER_CHANGE_EVENT, + CODEX_USAGE_BANNER_STORAGE_KEY, + readCodexUsageBannerPreference, + writeCodexUsageBannerPreference, +} from "../src/renderer-codex-usage-banner-preference.js"; + +function fixture(initial?: string) { + const values = new Map(); + if (initial !== undefined) values.set(CODEX_USAGE_BANNER_STORAGE_KEY, initial); + const storage = { + getItem: vi.fn((key: string) => values.get(key) ?? null), + setItem: vi.fn((key: string, value: string) => { + values.set(key, value); + }), + }; + const owner = Object.assign(new EventTarget(), { localStorage: storage }) as unknown as Window; + return { owner, values }; +} + +describe("Codex usage banner Settings preference", () => { + it("defaults on, persists off as 0, and stays on without a Window", () => { + const unset = fixture(); + expect(readCodexUsageBannerPreference(unset.owner)).toBe(true); + expect(readCodexUsageBannerPreference(null)).toBe(true); + expect(readCodexUsageBannerPreference(undefined)).toBe(true); + + const events: string[] = []; + unset.owner.addEventListener(CODEX_USAGE_BANNER_CHANGE_EVENT, () => events.push("changed")); + expect(writeCodexUsageBannerPreference(unset.owner, false)).toBe(true); + expect(unset.values.get(CODEX_USAGE_BANNER_STORAGE_KEY)).toBe("0"); + expect(readCodexUsageBannerPreference(unset.owner)).toBe(false); + expect(events).toEqual(["changed"]); + + expect(writeCodexUsageBannerPreference(unset.owner, true)).toBe(true); + expect(unset.values.get(CODEX_USAGE_BANNER_STORAGE_KEY)).toBe("1"); + expect(readCodexUsageBannerPreference(unset.owner)).toBe(true); + }); + + it("stays on when localStorage throws", () => { + const owner = Object.assign(new EventTarget(), { + localStorage: { + getItem: () => { + throw new Error("blocked"); + }, + setItem: () => { + throw new Error("blocked"); + }, + }, + }) as unknown as Window; + expect(readCodexUsageBannerPreference(owner)).toBe(true); + expect(writeCodexUsageBannerPreference(owner, false)).toBe(false); + }); +}); diff --git a/packages/renderer-extension/test/renderer-cpa-live-voice-hook.test.ts b/packages/renderer-extension/test/renderer-cpa-live-voice-hook.test.ts new file mode 100644 index 000000000..c4af69792 --- /dev/null +++ b/packages/renderer-extension/test/renderer-cpa-live-voice-hook.test.ts @@ -0,0 +1,198 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { CPA_LIVE_VOICE_STORAGE_KEY } from "../src/renderer-cpa-live-voice-preference.js"; +import { + AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + THREAD_REALTIME_EXISTING_CALL_GATE, + auditedAppInitialAssetPresent, + installCpaLiveVoiceHook, + readCpaLiveVoiceHookConfig, + statsigClients, + wrapCheckGate, +} from "../src/renderer-cpa-live-voice-hook.js"; + +const AUDITED_ASSET = [`app://-/assets/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}`]; + +const enabledConfig = { + enabled: true as const, + auditedDesktopVersion: AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + auditedAppInitialAsset: AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, +}; + +function statsigClient() { + return { + checkGate(name: string): boolean { + return name.length > 0; + }, + }; +} + +function statsigGlobal(client = statsigClient()) { + return { + client, + globalObject: { __STATSIG__: { instances: { key: client }, firstInstance: client } }, + }; +} + +afterEach(() => { + vi.useRealTimers(); +}); + +describe("CPA live-voice hook gating", () => { + it.each([ + `app://-/assets/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}.map`, + `app://-/assets/other.js?asset=${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}`, + `https://example.test/assets/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}`, + `app://-/other/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}`, + ])("rejects an asset-name substring in %s", (resourceName) => { + expect( + auditedAppInitialAssetPresent([resourceName], AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET), + ).toBe(false); + }); + + it("accepts the exact audited asset as a relative script URL", () => { + expect( + auditedAppInitialAssetPresent( + [`/assets/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}`], + AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + ), + ).toBe(true); + }); + + it("rejects config that substitutes a different app-initial asset", () => { + const { client, globalObject } = statsigGlobal(); + const hook = installCpaLiveVoiceHook({ + config: { ...enabledConfig, auditedAppInitialAsset: "other.js" }, + resourceNames: ["app://-/assets/other.js"], + globalThis: globalObject, + }); + expect(hook.status).toBe("unsupported"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); + + it("stays off when Settings stores the preference as 0", () => { + const { client, globalObject } = statsigGlobal(); + Object.assign(globalObject, { + localStorage: { + getItem: (key: string) => (key === CPA_LIVE_VOICE_STORAGE_KEY ? "0" : null), + }, + }); + const hook = installCpaLiveVoiceHook({ + config: enabledConfig, + resourceNames: AUDITED_ASSET, + globalThis: globalObject, + }); + expect(hook.status).toBe("disabled"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); + + it("stays off without an explicit enable config", () => { + expect(readCpaLiveVoiceHookConfig({})).toBeNull(); + const { client, globalObject } = statsigGlobal(); + const hook = installCpaLiveVoiceHook({ + config: { ...enabledConfig, enabled: false }, + resourceNames: AUDITED_ASSET, + globalThis: globalObject, + }); + expect(hook.status).toBe("disabled"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); + + it("fails closed when the audited app-initial asset is absent", () => { + expect( + auditedAppInitialAssetPresent( + ["app://-/assets/other.js"], + enabledConfig.auditedAppInitialAsset, + ), + ).toBe(false); + const { client, globalObject } = statsigGlobal(); + const hook = installCpaLiveVoiceHook({ + config: enabledConfig, + resourceNames: ["app://-/assets/app-initial-deadbeef.js"], + globalThis: globalObject, + }); + expect(hook.status).toBe("unsupported"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); + + it("fails closed when an injected Desktop version is not the audited build", () => { + const { client, globalObject } = statsigGlobal(); + const hook = installCpaLiveVoiceHook({ + config: { ...enabledConfig, desktopVersion: "26.9999.0" }, + resourceNames: AUDITED_ASSET, + globalThis: globalObject, + }); + expect(hook.status).toBe("unsupported"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); +}); + +describe("CPA live-voice existing-call gate override", () => { + it("reads each registered Statsig client once", () => { + const { client, globalObject } = statsigGlobal(); + expect(statsigClients(globalObject)).toEqual([client]); + expect(statsigClients({})).toEqual([]); + }); + + it("forces only the existing-call gate off and passes other gates through", () => { + const client = statsigClient(); + expect(wrapCheckGate(client)).toBe(true); + expect(wrapCheckGate(client)).toBe(false); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(false); + expect(client.checkGate("other_gate")).toBe(true); + }); + + it("activates on the audited asset and restores on dispose", () => { + const { client, globalObject } = statsigGlobal(); + const hook = installCpaLiveVoiceHook({ + config: enabledConfig, + resourceNames: AUDITED_ASSET, + globalThis: globalObject, + }); + expect(hook.status).toBe("active"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(false); + hook.dispose(); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(true); + }); + + it("wraps a Statsig client registered after document-start injection", () => { + vi.useFakeTimers(); + const globalObject: Record = {}; + const hook = installCpaLiveVoiceHook({ + config: enabledConfig, + resourceNames: AUDITED_ASSET, + globalThis: globalObject, + }); + expect(hook.status).toBe("pending"); + const client = statsigClient(); + globalObject.__STATSIG__ = { firstInstance: client }; + vi.advanceTimersByTime(250); + expect(hook.status).toBe("active"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(false); + hook.dispose(); + }); + + it("stops rescanning assets while waiting for Statsig after an audited match", () => { + vi.useFakeTimers(); + const globalObject: Record = {}; + const querySelectorAll = vi.fn(() => [ + { getAttribute: () => `/assets/${AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET}` }, + ]); + const hook = installCpaLiveVoiceHook({ + config: enabledConfig, + globalThis: globalObject, + document: { querySelectorAll } as unknown as ParentNode, + }); + vi.advanceTimersByTime(1_000); + expect(hook.status).toBe("pending"); + expect(querySelectorAll).toHaveBeenCalledOnce(); + const client = statsigClient(); + globalObject.__STATSIG__ = { firstInstance: client }; + vi.advanceTimersByTime(250); + expect(hook.status).toBe("active"); + expect(client.checkGate(THREAD_REALTIME_EXISTING_CALL_GATE)).toBe(false); + expect(querySelectorAll).toHaveBeenCalledOnce(); + hook.dispose(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-cpa-live-voice-preference.test.ts b/packages/renderer-extension/test/renderer-cpa-live-voice-preference.test.ts new file mode 100644 index 000000000..0e3cf78e8 --- /dev/null +++ b/packages/renderer-extension/test/renderer-cpa-live-voice-preference.test.ts @@ -0,0 +1,111 @@ +import { describe, expect, it, vi } from "vitest"; + +import { + CHAT_LIVE_VOICE_STORAGE_KEY, + CPA_LIVE_VOICE_STORAGE_KEY, + chatLiveVoicePreferenceAllowsOverride, + cpaLiveVoicePreferenceAllowsInstall, + readChatLiveVoicePreference, + readCpaLiveVoicePreference, + applySharedVoiceRoutes, + nativeChatVoicePreferenceAllowsStart, + legacyVoiceRoutes, +} from "../src/renderer-cpa-live-voice-preference.js"; + +function fixture(initial?: string, key = CPA_LIVE_VOICE_STORAGE_KEY) { + const values = new Map(); + if (initial !== undefined) values.set(key, initial); + const storage = { + getItem: vi.fn((entry: string) => values.get(entry) ?? null), + setItem: vi.fn((entry: string, value: string) => { + values.set(entry, value); + }), + }; + const owner = Object.assign(new EventTarget(), { localStorage: storage }) as unknown as Window; + return { owner, values, storage }; +} + +function throwingOwner() { + return Object.assign(new EventTarget(), { + localStorage: { + getItem: () => { + throw new Error("blocked"); + }, + setItem: () => { + throw new Error("blocked"); + }, + }, + }) as unknown as Window; +} + +describe("CPA live-voice Settings preference", () => { + it("reads the old Work preference for migration and defaults on", () => { + expect(readCpaLiveVoicePreference(fixture().owner)).toBe(true); + expect(readCpaLiveVoicePreference(fixture("0").owner)).toBe(false); + expect(readCpaLiveVoicePreference(fixture("1").owner)).toBe(true); + expect(cpaLiveVoicePreferenceAllowsInstall({})).toBe(true); + }); + + it("stays on when localStorage throws", () => { + const owner = throwingOwner(); + expect(readCpaLiveVoicePreference(owner)).toBe(true); + expect(cpaLiveVoicePreferenceAllowsInstall(owner)).toBe(true); + }); +}); + +describe("Chat live-voice Settings preference", () => { + it("distinguishes native ChatGPT, Codex default and gateway without changing Work", () => { + const { owner } = fixture("0"); + expect(legacyVoiceRoutes(owner).workConnection).toBe("desktop"); + for (const [chatExperience, chatConnection, native, gateway] of [ + ["chatgpt", "gateway", true, false], + ["codex", "desktop", false, false], + ["codex", "gateway", false, true], + ] as const) { + applySharedVoiceRoutes(owner, { chatExperience, chatConnection, workConnection: "desktop" }); + expect(nativeChatVoicePreferenceAllowsStart(owner)).toBe(native); + expect(chatLiveVoicePreferenceAllowsOverride(owner)).toBe(gateway); + expect(cpaLiveVoicePreferenceAllowsInstall(owner)).toBe(false); + } + }); + it("restores the last shared route on a fresh renderer document before polling", () => { + const { owner, storage } = fixture(); + applySharedVoiceRoutes(owner, { + chatExperience: "codex", + chatConnection: "desktop", + workConnection: "desktop", + }); + const reloaded = { localStorage: storage } as unknown as Window; + expect(nativeChatVoicePreferenceAllowsStart(reloaded)).toBe(false); + expect(chatLiveVoicePreferenceAllowsOverride(reloaded)).toBe(false); + expect(cpaLiveVoicePreferenceAllowsInstall(reloaded)).toBe(false); + }); + it("defaults off and enables only on the exact 1 value", () => { + const chat = (initial?: string) => fixture(initial, CHAT_LIVE_VOICE_STORAGE_KEY); + expect(readChatLiveVoicePreference(chat().owner)).toBe(false); + expect(readChatLiveVoicePreference(chat("0").owner)).toBe(false); + expect(readChatLiveVoicePreference(chat("true").owner)).toBe(false); + expect(readChatLiveVoicePreference(chat("1").owner)).toBe(true); + expect(chatLiveVoicePreferenceAllowsOverride({})).toBe(false); + expect(chatLiveVoicePreferenceAllowsOverride(chat("1").owner)).toBe(true); + }); + + it("keeps Chat and Work migration independent and handles unavailable browser storage", () => { + const owner = fixture("0").owner; + expect(legacyVoiceRoutes(owner)).toEqual({ + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", + }); + expect(readChatLiveVoicePreference(throwingOwner())).toBe(false); + expect(chatLiveVoicePreferenceAllowsOverride(throwingOwner())).toBe(false); + const blocked = throwingOwner(); + applySharedVoiceRoutes(blocked, { + chatExperience: "codex", + chatConnection: "gateway", + workConnection: "desktop", + }); + expect(chatLiveVoicePreferenceAllowsOverride(blocked)).toBe(true); + expect(cpaLiveVoicePreferenceAllowsInstall(blocked)).toBe(false); + }); +}); diff --git a/packages/renderer-extension/test/renderer-dictation-settings.test.ts b/packages/renderer-extension/test/renderer-dictation-settings.test.ts new file mode 100644 index 000000000..ce311f0b5 --- /dev/null +++ b/packages/renderer-extension/test/renderer-dictation-settings.test.ts @@ -0,0 +1,134 @@ +import { describe, expect, it, vi } from "vitest"; +import { + bindDictationSettingsMode, + replayDictationSettingsMode, +} from "../src/renderer-dictation-settings.js"; +type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: unknown }; +function fixture(platform = "darwin") { + let permitted = true; + let loading = false; + const gate: Atom = { read: () => true, write: true }; + const capability: Atom = { + read: (get) => ({ isLoading: loading, isError: false, isCapable: permitted && !get(gate) }), + }; + const mode: Atom = { + read: (get) => + ["darwin", "win32", "linux"].includes(platform) && + (get(capability) as { isCapable: boolean }).isCapable + ? platform === "linux" + ? "app" + : "os-global" + : null, + }; + const store = { + get: (atom: Atom): unknown => atom.read(store.get), + sub: vi.fn((atom: Atom, listener: () => void) => { + void atom; + void listener; + return () => {}; + }), + }; + function reader() { + const original = () => store.get(mode); + const subscriber = { + getSnapshot: original, + subscribe: (notify: () => void) => store.sub(mode, notify), + createRender: () => undefined, + }; + const instance = { getSnapshot: original }; + const notify = vi.fn(() => { + hook.next.memoizedState = subscriber.getSnapshot(); + }); + const hook = { + memoizedState: [subscriber, [store, mode]], + next: { + memoizedState: original(), + queue: instance, + next: { + memoizedState: { + create: () => subscriber.subscribe(notify), + deps: [subscriber.subscribe], + }, + next: null, + }, + }, + }; + return { original, subscriber, instance, hook, notify }; + } + const page = reader(); + const fibers = new Set>([{ memoizedState: page.hook }]); + return { + gate, + capability, + mode, + store, + page, + fibers, + reader, + bind: () => + bindDictationSettingsMode( + () => fibers, + store, + mode, + [capability], + gate, + () => true, + ), + deny: () => { + permitted = false; + }, + load: () => { + loading = true; + }, + }; +} +describe("native Settings dictation mode", () => { + it.each([ + ["darwin", "os-global"], + ["win32", "os-global"], + ["linux", "app"], + ["unknown", undefined], + ])("retains native platform choice for %s", (platform, expected) => { + const f = fixture(platform as string); + expect(replayDictationSettingsMode(f.store, f.mode, [f.capability], f.gate)).toBe(expected); + expect(f.store.get(f.mode)).toBeNull(); + expect(f.store.get(f.capability)).toMatchObject({ isCapable: false }); + }); + it.each(["deny", "load"] as const)( + "does not invent dictation eligibility during %s", + (change) => { + const f = fixture(); + f[change](); + expect(replayDictationSettingsMode(f.store, f.mode, [f.capability], f.gate)).toBeNull(); + }, + ); + it("binds only the exact mode selector, detects newly mounted shortcut readers, and restores", () => { + const f = fixture(); + const b = f.bind(); + expect(f.page.subscriber.getSnapshot()).toBe("os-global"); + expect(b.current()).toBe(true); + const shortcut = f.reader(); + f.fibers.add({ memoizedState: shortcut.hook }); + expect(b.current()).toBe(false); + b.dispose(); + expect(f.page.subscriber.getSnapshot).toBe(f.page.original); + const rebound = f.bind(); + expect(shortcut.subscriber.getSnapshot()).toBe("os-global"); + expect(rebound.current()).toBe(true); + f.deny(); + expect(shortcut.subscriber.getSnapshot()).toBeNull(); + expect(rebound.current()).toBe(false); + rebound.dispose(); + expect(shortcut.subscriber.getSnapshot).toBe(shortcut.original); + expect(f.store.sub).not.toHaveBeenCalled(); + }); + it("rejects writable or unrelated mode selectors", () => { + const f = fixture(); + expect( + replayDictationSettingsMode(f.store, { read: () => "os-global" }, [f.capability], f.gate), + ).toBeUndefined(); + f.mode.write = true; + expect(() => f.bind()).toThrow(); + expect(f.page.subscriber.getSnapshot).toBe(f.page.original); + }); +}); diff --git a/packages/renderer-extension/test/renderer-external-queue.test.ts b/packages/renderer-extension/test/renderer-external-queue.test.ts index d5efcbdb2..22e335b16 100644 --- a/packages/renderer-extension/test/renderer-external-queue.test.ts +++ b/packages/renderer-extension/test/renderer-external-queue.test.ts @@ -150,6 +150,18 @@ function fixture() { afterEach(() => vi.unstubAllGlobals()); describe("external follow-ups retain Desktop's local queue", () => { + it("leaves the server queue enabled when the Manager reports stream role", () => { + const f = fixture(); + Object.assign(f.manager, { getStreamRole: () => ({ role: "follower" }) }); + const adapter = f.install(); + try { + expect(f.queue.isEnabled).toBe(f.originalGate); + expect(f.queue.isEnabled("external")).toBe(true); + } finally { + adapter.dispose(); + } + }); + it("queues another message during an external Turn without thread/queue/list or add", async () => { const f = fixture(); const adapter = f.install(); diff --git a/packages/renderer-extension/test/renderer-external-steering.test.ts b/packages/renderer-extension/test/renderer-external-steering.test.ts index 4e3c9d904..9fdfe3b33 100644 --- a/packages/renderer-extension/test/renderer-external-steering.test.ts +++ b/packages/renderer-extension/test/renderer-external-steering.test.ts @@ -5,7 +5,12 @@ import { installRendererExternalSteering } from "../src/renderer-external-steeri function fixture(delivery: "newTurn" | "activeTurn" | "official" = "newTurn") { const events: string[] = []; const requestOptions: Record = { timeoutMs: 30_000 }; - let queued: Array<{ id: string; pausedReason?: string }> = []; + let queued: Array<{ + id: string; + text?: string; + context?: { prompt?: string }; + pausedReason?: string; + }> = []; const originalTurn = { turnId: "old", status: "inProgress", @@ -35,7 +40,13 @@ function fixture(delivery: "newTurn" | "activeTurn" | "official" = "newTurn") { steerTurn: vi .fn<(...args: unknown[]) => Promise<{ turnId: string }>>() .mockResolvedValue({ turnId: "official" }), - getStreamRole: vi.fn(() => ({ role: "owner" })), + getStreamRole: vi + .fn<(threadId: string) => { role: string }>() + .mockReturnValue({ role: "owner" }), + getConversation: (threadId: string): { id: string; modelProvider: string } | null => { + void threadId; + return null; + }, getTurnCoordinator: () => ({ loadMessages: async () => undefined, readMessages: () => queued, @@ -75,6 +86,7 @@ function fixture(delivery: "newTurn" | "activeTurn" | "official" = "newTurn") { }), }; const originalSteer = manager.steerTurn; + const originalStreamRole = manager.getStreamRole; const dispose = installRendererExternalSteering(manager); if (!dispose) throw new Error("Synthetic Renderer steering binding was not installed"); const args: unknown[] = [ @@ -95,6 +107,7 @@ function fixture(delivery: "newTurn" | "activeTurn" | "official" = "newTurn") { events, turns, originalSteer, + originalStreamRole, dispose, requestOptions, queue: { @@ -147,6 +160,48 @@ describe("external direction changes use normal Desktop start presentation", () f.dispose(); }); + it("reports owner stream role for External Threads so follower windows can enqueue", () => { + const f = fixture(); + f.manager.getConversation = (threadId: string) => ({ + id: threadId, + modelProvider: "codexhost", + }); + f.originalStreamRole.mockReturnValue({ role: "follower" }); + expect(f.manager.getStreamRole("thread")).toEqual({ role: "owner" }); + f.dispose(); + }); + + it("restores queued follow-up text onto an empty overlay turn/start", async () => { + const f = fixture(); + f.queue.set([ + { + id: "follow-up", + text: "add disallow model list in grok in win10", + context: { prompt: "add disallow model list in grok in win10" }, + }, + ]); + await f.manager.sendRequest("turn/start", { + threadId: "thread", + clientUserMessageId: "follow-up", + input: [], + }); + expect(f.rpc.mock.calls.at(-1)?.slice(0, 2)).toEqual([ + "turn/start", + { + threadId: "thread", + clientUserMessageId: "follow-up", + input: [ + { + type: "text", + text: "add disallow model list in grok in win10", + text_elements: [], + }, + ], + }, + ]); + f.dispose(); + }); + it("passes official steering and unrelated requests through unchanged", async () => { const f = fixture("official"); await expect(f.manager.steerTurn(...f.args)).resolves.toEqual({ turnId: "official" }); @@ -326,7 +381,7 @@ describe("external direction changes use normal Desktop start presentation", () it("lets a follower delegate the original steer to its owning window", async () => { const f = fixture(); - f.manager.getStreamRole.mockReturnValue({ role: "follower" }); + f.originalStreamRole.mockReturnValue({ role: "follower" }); await expect(f.manager.steerTurn(...f.args)).resolves.toEqual({ turnId: "official" }); expect(f.originalSteer).toHaveBeenCalledWith(...f.args); expect(f.rpc).not.toHaveBeenCalled(); diff --git a/packages/renderer-extension/test/renderer-external-voice-launch.test.ts b/packages/renderer-extension/test/renderer-external-voice-launch.test.ts new file mode 100644 index 000000000..36c512fc7 --- /dev/null +++ b/packages/renderer-extension/test/renderer-external-voice-launch.test.ts @@ -0,0 +1,385 @@ +import type { RendererHostRouting } from "@codexhost/desktop-control/renderer-bindings"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { + AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + CPA_LIVE_VOICE_HOOK_GLOBAL, +} from "../src/renderer-cpa-live-voice-hook.js"; +import { CLIENT_OWNED_VOICE_ASSETS } from "../src/renderer-client-owned-voice.js"; +import { + installExternalVoiceLaunchHook, + routeExternalVoiceLaunch, +} from "../src/renderer-external-voice-launch.js"; +import { applySharedVoiceRoutes } from "../src/renderer-cpa-live-voice-preference.js"; + +const carrier = "codexhost/plugin-v1@grok-test-model-permission"; +const source = "composer_button_new_thread"; +const request = () => ({ + source, + activeCollaborationMode: { + mode: "default", + settings: { model: "gpt-6.1-sol", reasoning_effort: "high", developer_instructions: "keep" }, + }, + projectId: "project-1", + pageCompanion: null, +}); + +function routingFixture() { + const policy = { selectedModel: vi.fn<() => string | null>(() => carrier) }; + const route = { hostId: "local", policy }; + const routing = { + forComposer: vi.fn(() => route), + forHost: vi.fn((hostId: string) => (hostId === route.hostId ? route : null)), + } as unknown as RendererHostRouting; + return { policy, route, routing }; +} + +function hookFixture() { + const { routing, policy } = routingFixture(); + const start = vi.fn(function (this: unknown, ...args: unknown[]) { + return { receiver: this, args }; + }); + const other = vi.fn(); + // Native services are a plain decoded record containing immutable RPC stubs. + const runtime = new Proxy( + { requestRealtimeStart: start, registerRealtimeStarter: other }, + { + set() { + throw new Error("Can't assign properties on RPC stubs."); + }, + defineProperty() { + throw new Error("Can't define properties on RPC stubs."); + }, + }, + ); + const services = { realtimeVoiceRuntime: runtime, untouched: {} }; + const names = [AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, "app-shared-8bd9d0399038.js"]; + const target = { + [CPA_LIVE_VOICE_HOOK_GLOBAL]: { + enabled: true, + auditedDesktopVersion: AUDITED_CPA_LIVE_VOICE_DESKTOP_VERSION, + auditedAppInitialAsset: AUDITED_CPA_LIVE_VOICE_APP_INITIAL_ASSET, + }, + __codexhostHostRoutingV1: routing, + document: { + querySelectorAll: () => names.map((name) => ({ getAttribute: () => `/assets/${name}` })), + } as unknown as ParentNode, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async () => ({ fN: services })); + return { policy, services, runtime, start, other, names, target, importModule }; +} + +const settle = async () => { + await Promise.resolve(); + await Promise.resolve(); +}; +afterEach(() => vi.useRealTimers()); + +describe("native new-voice selection", () => { + it.each(CLIENT_OWNED_VOICE_ASSETS)( + "keeps the selected Harness on Desktop $version", + async (assets) => { + const f = hookFixture(); + f.names.splice(0, f.names.length, assets.initial, assets.shared); + Reflect.set(f.target[CPA_LIVE_VOICE_HOOK_GLOBAL], "desktopVersion", assets.version); + const hook = installExternalVoiceLaunchHook({ + target: f.target, + importModule: async () => ({ [assets.services]: f.services }), + }); + await settle(); + expect(hook.status).toBe("active"); + const routed = f.services.realtimeVoiceRuntime.requestRealtimeStart(request()) as { + args: unknown[]; + }; + expect(routed.args[0]).toMatchObject({ + activeCollaborationMode: { settings: { model: carrier } }, + }); + hook.dispose(); + }, + ); + it("does not fall back to legacy launch assets when modern ownership is mixed", async () => { + vi.useFakeTimers(); + const f = hookFixture(); + f.names.push("app-initial-dddef47d6ef6.js", "app-shared-2eda13eb7044.js"); + const hook = installExternalVoiceLaunchHook(f); + await vi.advanceTimersByTimeAsync(60_000); + expect(hook.status).toBe("unsupported"); + expect(f.importModule).not.toHaveBeenCalled(); + expect(f.services.realtimeVoiceRuntime).toBe(f.runtime); + hook.dispose(); + }); + + it("does not move Chat-mode Start Voice onto the selected Harness", () => { + const { routing } = routingFixture(); + const original = { ...request(), productExperience: "chat" }; + expect(routeExternalVoiceLaunch(original, routing)).toBe(original); + }); + it("keeps Work-mode Start Voice on the selected Harness", () => { + const { routing } = routingFixture(); + const routed = routeExternalVoiceLaunch( + { ...request(), productExperience: "work" }, + routing, + ) as ReturnType; + expect(routed.activeCollaborationMode.settings.model).toBe(carrier); + }); + it("snapshots the external route into a separately cloned overlay request", () => { + const { routing, policy } = routingFixture(); + const original = request(); + const routed = structuredClone(routeExternalVoiceLaunch(original, routing)) as ReturnType< + typeof request + >; + // Overlay's native start builder consumes precisely this settings.model. + const threadStart = { model: routed.activeCollaborationMode.settings.model }; + expect(threadStart.model).toBe(carrier); + expect(routed.activeCollaborationMode.settings.developer_instructions).toBe("keep"); + expect(routed.projectId).toBe(original.projectId); + expect(original.activeCollaborationMode.settings.model).toBe("gpt-6.1-sol"); + policy.selectedModel.mockReturnValue(null); + expect(threadStart.model).toBe(carrier); + expect(routeExternalVoiceLaunch(original, routing)).toBe(original); + }); + + it.each(["composer_button_existing_thread", "avatar", "global_hotkey"])( + "preserves %s and its locator even with an external draft", + (entry) => { + const original = { + ...request(), + source: entry, + locator: { hostId: "local", conversationId: "codex-1" }, + }; + const { routing, policy } = routingFixture(); + expect(routeExternalVoiceLaunch(original, routing)).toBe(original); + expect(policy.selectedModel).not.toHaveBeenCalled(); + }, + ); + + it("preserves official requests by identity after clearing external selection", () => { + const { routing, policy } = routingFixture(); + policy.selectedModel.mockReturnValue(null); + const original = request(); + expect(routeExternalVoiceLaunch(original, routing)).toBe(original); + }); + + it("does not borrow another Host or ambiguous Composer's draft", () => { + const { routing, policy } = routingFixture(); + const remote = { ...request(), hostId: "remote-ssh-discovered:linux" }; + expect(routeExternalVoiceLaunch(remote, routing)).toBe(remote); + vi.mocked(routing.forComposer).mockReturnValue(null); + const local = request(); + expect(routeExternalVoiceLaunch(local, routing)).toBe(local); + expect(policy.selectedModel).not.toHaveBeenCalled(); + }); + + it("resolves the matching remote Host without using local selection", () => { + const { routing, route } = routingFixture(); + route.hostId = "remote-ssh-discovered:linux"; + const remote = { ...request(), hostId: route.hostId }; + expect(routeExternalVoiceLaunch(remote, routing)).toMatchObject({ + hostId: route.hostId, + activeCollaborationMode: { settings: { model: carrier } }, + }); + const local = request(); + expect(routeExternalVoiceLaunch(local, routing)).toBe(local); + }); + + it("constructs default collaboration settings when native launch has none", () => { + const { routing } = routingFixture(); + expect( + routeExternalVoiceLaunch({ source, activeCollaborationMode: null }, routing), + ).toMatchObject({ + activeCollaborationMode: { + mode: "default", + settings: { model: carrier, reasoning_effort: null, developer_instructions: null }, + }, + }); + }); +}); + +describe("versioned native voice service hook", () => { + it("wraps the service reference and forwards the native RPC receiver, callbacks and return value", async () => { + const fixture = hookFixture(); + const hook = installExternalVoiceLaunchHook(fixture); + await settle(); + expect(hook.status).toBe("active"); + expect(installExternalVoiceLaunchHook(fixture)).toBe(hook); + const contextProvider = () => "context"; + const result = fixture.services.realtimeVoiceRuntime.requestRealtimeStart( + request(), + "launch-1", + contextProvider, + ); + expect(result.receiver).toBe(fixture.runtime); + expect(result.args).toEqual([ + expect.objectContaining({ + activeCollaborationMode: expect.objectContaining({ + settings: expect.objectContaining({ model: carrier }), + }), + }), + "launch-1", + contextProvider, + ]); + expect(fixture.services.realtimeVoiceRuntime.registerRealtimeStarter).toBe(fixture.other); + expect(fixture.importModule).toHaveBeenCalledOnce(); + expect(fixture.importModule).toHaveBeenCalledWith("app://-/assets/app-shared-8bd9d0399038.js"); + hook.dispose(); + expect(fixture.services.realtimeVoiceRuntime).toBe(fixture.runtime); + }); + + it("passes official launches unchanged and preserves RPC failures", async () => { + const fixture = hookFixture(); + const hook = installExternalVoiceLaunchHook(fixture); + await settle(); + fixture.policy.selectedModel.mockReturnValue(null); + const original = request(); + fixture.services.realtimeVoiceRuntime.requestRealtimeStart(original); + expect(fixture.start.mock.calls[0]?.[0]).toBe(original); + fixture.start.mockImplementation(() => { + throw new Error("native failure"); + }); + expect(() => fixture.services.realtimeVoiceRuntime.requestRealtimeStart(original)).toThrow( + "native failure", + ); + hook.dispose(); + }); + + it("does not import mismatched or absent audited assets", async () => { + vi.useFakeTimers(); + const fixture = hookFixture(); + fixture.names[1] = "app-shared-different.js"; + const hook = installExternalVoiceLaunchHook(fixture); + await vi.advanceTimersByTimeAsync(60_000); + expect(hook.status).toBe("unsupported"); + expect(fixture.importModule).not.toHaveBeenCalled(); + expect(fixture.services.realtimeVoiceRuntime).toBe(fixture.runtime); + hook.dispose(); + }); + + it("waits for native services initialization after importing the audited module", async () => { + vi.useFakeTimers(); + const fixture = hookFixture(); + const module: { fN?: unknown } = {}; + const hook = installExternalVoiceLaunchHook({ ...fixture, importModule: async () => module }); + await settle(); + expect(hook.status).toBe("pending"); + module.fN = fixture.services; + await vi.advanceTimersByTimeAsync(250); + expect(hook.status).toBe("active"); + hook.dispose(); + }); + + it("does not publish a hook after disposal during module import", async () => { + const fixture = hookFixture(); + let resolve!: (value: Record) => void; + const pending = new Promise>((done) => { + resolve = done; + }); + const hook = installExternalVoiceLaunchHook({ ...fixture, importModule: () => pending }); + hook.dispose(); + resolve({ fN: fixture.services }); + await settle(); + expect(fixture.services.realtimeVoiceRuntime).toBe(fixture.runtime); + }); +}); + +describe("native Chat service interception", () => { + it("awaits the native Chat START without entering the overlay's new-thread launcher", async () => { + const f = hookFixture(); + const chatStart = vi.fn().mockResolvedValue(undefined); + Object.assign(f.target, { + __codexhostNativeChatVoiceV1: { status: "active", start: chatStart }, + }); + const hook = installExternalVoiceLaunchHook(f); + await settle(); + await f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "chat", + }); + expect(chatStart).toHaveBeenCalledOnce(); + expect(f.start).not.toHaveBeenCalled(); + await f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "work", + }); + expect(f.start).toHaveBeenCalledOnce(); + expect(f.start.mock.calls[0]?.[0]).toMatchObject({ + activeCollaborationMode: { settings: { model: carrier } }, + }); + hook.dispose(); + }); + + it.each(["absent", "disabled", "unsupported"])( + "rejects native Chat when its binding is %s before the overlay can launch", + async (status) => { + const f = hookFixture(); + const chatStart = vi.fn(); + if (status !== "absent") + Object.assign(f.target, { + __codexhostNativeChatVoiceV1: { status, start: chatStart }, + }); + const hook = installExternalVoiceLaunchHook(f); + try { + await settle(); + await expect( + f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "chat", + }), + ).rejects.toThrow("Native Chat voice is unavailable"); + expect(chatStart).not.toHaveBeenCalled(); + expect(f.start).not.toHaveBeenCalled(); + await f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "work", + }); + expect(f.start).toHaveBeenCalledOnce(); + } finally { + hook.dispose(); + } + }, + ); + + it.each(["desktop", "gateway"] as const)( + "preserves explicit Codex Chat on %s without a native Chat binding", + async (chatConnection) => { + const f = hookFixture(); + applySharedVoiceRoutes(f.target, { + chatExperience: "codex", + chatConnection, + workConnection: "gateway", + }); + const hook = installExternalVoiceLaunchHook(f); + try { + await settle(); + await f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "chat", + }); + expect(f.start).toHaveBeenCalledOnce(); + } finally { + hook.dispose(); + } + }, + ); + + it("does not fall back to a Grok/Codex overlay when native Chat START rejects", async () => { + const f = hookFixture(); + Object.assign(f.target, { + __codexhostNativeChatVoiceV1: { + status: "active", + start: vi.fn().mockRejectedValue(new Error("native start failed")), + }, + }); + const hook = installExternalVoiceLaunchHook(f); + await settle(); + await expect( + f.services.realtimeVoiceRuntime.requestRealtimeStart({ + ...request(), + productExperience: "chat", + }), + ).rejects.toThrow("native start failed"); + expect(f.start).not.toHaveBeenCalled(); + hook.dispose(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-host-routing.test.ts b/packages/renderer-extension/test/renderer-host-routing.test.ts index 176aa34fd..0be9fe234 100644 --- a/packages/renderer-extension/test/renderer-host-routing.test.ts +++ b/packages/renderer-extension/test/renderer-host-routing.test.ts @@ -6,6 +6,7 @@ import { } from "@codexhost/shared-contracts"; import { afterEach, assert, expect, it, vi } from "vitest"; import { installCurrentRendererAdapter } from "../src/versioned-renderer-adapter.js"; +import { routeExternalVoiceLaunch } from "../src/renderer-external-voice-launch.js"; const remoteId = "remote-ssh-discovered:linux"; const piRequest = { harnessId: harnessIdSchema.parse("pi") }; @@ -106,6 +107,32 @@ afterEach(() => { vi.unstubAllGlobals(); }); +it("carries the adapter's selected Harness across native new-voice launch and clears for Codex", async () => { + const fixture = await setup("local"); + try { + const request = { + source: "composer_button_new_thread", + activeCollaborationMode: { mode: "default", settings: { model: "gpt-6.1-sol" } }, + }; + expect(fixture.adapter.applyAgent("grok")).toBe(true); + const expected = encodeHarnessPluginRoute({ harnessId: harnessIdSchema.parse("grok") }); + const overlayRequest = structuredClone( + routeExternalVoiceLaunch(request, window.__codexhostHostRoutingV1), + ); + expect(overlayRequest).toMatchObject({ + activeCollaborationMode: { settings: { model: expected } }, + }); + expect(fixture.adapter.applyAgent("codex")).toBe(true); + expect(routeExternalVoiceLaunch(request, window.__codexhostHostRoutingV1)).toBe(request); + // An owner replacement retires its selection instead of leaking to the new connection. + expect(fixture.adapter.applyAgent("grok")).toBe(true); + fixture.managers.delete("local"); + expect(routeExternalVoiceLaunch(request, window.__codexhostHostRoutingV1)).toBe(request); + } finally { + fixture.adapter.dispose(); + } +}); + it.each(["local", remoteId])( "queries both Hosts and follows switches immediately when %s opens first", async (hostId) => { diff --git a/packages/renderer-extension/test/renderer-model-client.test.ts b/packages/renderer-extension/test/renderer-model-client.test.ts index 800370db2..c99a5b319 100644 --- a/packages/renderer-extension/test/renderer-model-client.test.ts +++ b/packages/renderer-extension/test/renderer-model-client.test.ts @@ -387,7 +387,7 @@ describe("Renderer fixed Model request client", () => { updateAvailable: true, installationAvailable: true, releaseNotes: "Safer updates", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status: null, error: null, }) @@ -408,6 +408,7 @@ describe("Renderer fixed Model request client", () => { "credentialImports", "executeThreadCommand", "forkThread", + "getGrokComputerUse", "getHarnessDisplaySettings", "getHarnessLaunchSettings", "importHarnessSession", @@ -436,6 +437,7 @@ describe("Renderer fixed Model request client", () => { "selectThreadModel", "selectThreadPermissionMode", "selectThreadThinking", + "setGrokComputerUse", "setHarnessDisplaySettings", "setHarnessLaunchSettings", "setIdleReleaseSettings", @@ -884,7 +886,7 @@ describe("Renderer fixed Model request client", () => { updateAvailable: true, installationAvailable: true, releaseNotes: "Safer updates", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status: null, error: null, artifactUrl: "https://example.com/update.exe", diff --git a/packages/renderer-extension/test/renderer-native-chat-voice-selection.test.ts b/packages/renderer-extension/test/renderer-native-chat-voice-selection.test.ts new file mode 100644 index 000000000..d12ee9916 --- /dev/null +++ b/packages/renderer-extension/test/renderer-native-chat-voice-selection.test.ts @@ -0,0 +1,253 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import { nativeChatVoiceDraftSelection } from "../src/renderer-native-chat-voice-selection.js"; +import type { NativeChatVoiceBinding } from "../src/renderer-native-chat-voice.js"; + +function fixture() { + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + const backend = "a7e3282d-f457-4586-9b28-921411364f6b"; + const origin = { pathname: "/", key: "home" }; + const state = { + isVoiceModeActive: true, + isWorkMode: false, + conversationId: draft, + voiceSessionId: "fixture-session", + hasVoiceSessionStarted: true, + earlyServerThreadId: backend, + server: { connectionState: "connected" }, + }; + const initial = { + Ycr: {}, + B1n: {}, + n0n: {}, + Y1n: {}, + SPn: () => ({ getState: () => state }), + Nir: vi.fn(() => true), + HJt: vi.fn(() => 1), + }; + const values = new Map([[initial.Ycr, origin]]); + const binding = { + props: { conversationId: draft }, + backendConversationId: undefined, + scope: { get: vi.fn((selector) => values.get(selector)), set: vi.fn() }, + } as unknown as NativeChatVoiceBinding; + const shared = { Vot: vi.fn((id: string) => `/c/${encodeURIComponent(id)}`) }; + const controller = new AbortController(); + const check = () => + nativeChatVoiceDraftSelection(binding, initial, shared, origin, controller.signal); + const adopt = () => { + values.set(initial.B1n, backend); + values.set(initial.Y1n, "message"); + values.set(initial.n0n, { message: { message: { author: { role: "user" } } } }); + }; + return { + draft, + backend, + origin, + state, + initial, + values, + binding, + shared, + controller, + check, + adopt, + }; +} + +afterEach(() => { + Reflect.deleteProperty(globalThis, "__codexhostChatVoiceDebug"); + vi.restoreAllMocks(); +}); + +describe("native voice-created Chat selection", () => { + it("waits for native draft adoption and transcript, never an early server ID alone", async () => { + const f = fixture(); + const select = f.check(); + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + f.values.set(f.initial.B1n, f.backend); + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + f.adopt(); + // Native navigation recognizes the voice client draft and resolves its adopted + // alias. The backend page is not opened before that mapping and transcript exist. + f.initial.Nir.mockImplementation((_scope?: unknown, path?: unknown) => { + expect(path).toBe(`/c/${encodeURIComponent(f.draft)}`); + expect(f.values.get(f.initial.B1n)).toBe(f.backend); + return true; + }); + select(); + select(); + expect(f.shared.Vot).toHaveBeenCalledExactlyOnceWith(f.draft); + expect(f.initial.Nir).toHaveBeenCalledOnce(); + expect(f.state.isVoiceModeActive).toBe(true); + expect(f.binding.scope.set).not.toHaveBeenCalled(); + await Promise.resolve(); + const log = Reflect.get(globalThis, "__codexhostChatVoiceDebug"); + expect(JSON.stringify(log)).not.toContain(f.draft); + expect(JSON.stringify(log)).not.toContain(f.backend); + expect(JSON.stringify(log)).not.toContain(f.state.voiceSessionId); + }); + + it("waits for native START initialization, then connection and session startup", () => { + const f = fixture(); + f.state.isVoiceModeActive = false; + f.adopt(); + const select = f.check(); + select(); + f.state.voiceSessionId = "new-fixture-session"; + f.state.isVoiceModeActive = true; + f.state.hasVoiceSessionStarted = false; + select(); + f.state.hasVoiceSessionStarted = true; + f.state.server.connectionState = "connecting"; + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + f.state.server.connectionState = "connected"; + select(); + expect(f.initial.Nir).toHaveBeenCalledOnce(); + }); + + it.each([false, true])( + "keeps a cached native session when its first poll is connected: %s", + (connected) => { + const f = fixture(); + f.state.hasVoiceSessionStarted = false; + f.state.server.connectionState = "connecting"; + const select = f.check(); + select(); + // Native Instant Connect adopts the cached ID without issuing a new voice intent. + f.state.voiceSessionId = "cached-session"; + f.state.hasVoiceSessionStarted = connected; + f.state.server.connectionState = connected ? "connected" : "connecting"; + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + f.state.hasVoiceSessionStarted = true; + f.state.server.connectionState = "connected"; + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + f.adopt(); + select(); + select(); + expect(f.initial.Nir).toHaveBeenCalledExactlyOnceWith( + f.binding.scope, + `/c/${encodeURIComponent(f.draft)}`, + ); + }, + ); + + it("rejects an established session ID changing after startup in the same voice intent", () => { + const f = fixture(); + f.state.hasVoiceSessionStarted = false; + const select = f.check(); + f.state.hasVoiceSessionStarted = true; + select(); + f.state.voiceSessionId = "replacement-session"; + f.adopt(); + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }); + + it("cancels a new native voice intent even if it starts the same draft between polls", () => { + const f = fixture(); + f.state.hasVoiceSessionStarted = false; + const select = f.check(); + f.initial.HJt.mockReturnValue(2); + f.state.hasVoiceSessionStarted = true; + f.adopt(); + select(); + f.initial.HJt.mockReturnValue(1); + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }); + + it.each([undefined, null, 0, NaN, "1"])( + "rejects startup session ID changes without a proven native intent version: %s", + (version) => { + const f = fixture(); + f.state.hasVoiceSessionStarted = false; + Reflect.set(f.initial, "HJt", version === undefined ? undefined : () => version); + const select = f.check(); + f.state.voiceSessionId = "replacement-session"; + select(); + f.state.hasVoiceSessionStarted = true; + f.adopt(); + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }, + ); + + it.each(["navigation", "end", "replace", "work", "conversation", "abort"])( + "cancels rather than selecting a stale Chat after %s", + (change) => { + const f = fixture(); + const select = f.check(); + select(); + if (change === "navigation") f.values.set(f.initial.Ycr, { pathname: "/", key: "new-home" }); + if (change === "end") f.state.isVoiceModeActive = false; + if (change === "replace") f.state.voiceSessionId = "replacement"; + if (change === "work") f.state.isWorkMode = true; + if (change === "conversation") f.state.conversationId = "another-chat"; + if (change === "abort") f.controller.abort(); + select(); + f.adopt(); + f.values.set(f.initial.Ycr, f.origin); + f.state.isVoiceModeActive = true; + f.state.isWorkMode = false; + f.state.conversationId = f.draft; + f.state.voiceSessionId = "fixture-session"; + select(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }, + ); + + it.each([null, "local-chatgpt:invalid", "thread-id"])( + "ignores invalid native mapping %s", + (id) => { + const f = fixture(); + f.adopt(); + f.values.set(f.initial.B1n, id); + f.check()(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }, + ); + + it("leaves a saved Chat selected", () => { + const f = fixture(); + f.binding.backendConversationId = f.backend; + f.adopt(); + f.check()(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }); + + it("skips selection when optional private exports are unavailable", () => { + const f = fixture(); + Reflect.deleteProperty(f.initial, "n0n"); + f.check()(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + }); + + it.each(["throw", "reject", "decline"])( + "does not retry or throw into voice on navigation %s", + async (failure) => { + const f = fixture(); + vi.spyOn(console, "info").mockImplementation(() => {}); + f.adopt(); + f.initial.Nir.mockImplementation(() => { + if (failure === "throw") throw new Error("fixture private error"); + if (failure === "reject") + return Promise.reject(new Error("fixture private error")) as never; + return false; + }); + const select = f.check(); + expect(() => select()).not.toThrow(); + await Promise.resolve(); + select(); + expect(f.initial.Nir).toHaveBeenCalledOnce(); + expect(f.state.isVoiceModeActive).toBe(true); + expect(JSON.stringify(Reflect.get(globalThis, "__codexhostChatVoiceDebug"))).not.toContain( + "private error", + ); + }, + ); +}); diff --git a/packages/renderer-extension/test/renderer-native-chat-voice-state.test.ts b/packages/renderer-extension/test/renderer-native-chat-voice-state.test.ts new file mode 100644 index 000000000..61ab47133 --- /dev/null +++ b/packages/renderer-extension/test/renderer-native-chat-voice-state.test.ts @@ -0,0 +1,71 @@ +import { describe, expect, it, vi } from "vitest"; +import { chatVoiceDebug } from "../src/renderer-chat-voice-debug.js"; +import { observeNativeChatVoiceState } from "../src/renderer-native-chat-voice-state.js"; +import type { ChatVoiceScope } from "../src/renderer-native-chat-voice.js"; + +vi.mock("../src/renderer-chat-voice-debug.js", () => ({ chatVoiceDebug: vi.fn() })); + +describe("native Chat lifecycle diagnostics", () => { + it("records connection transitions without native payloads and unsubscribes", () => { + vi.mocked(chatVoiceDebug).mockClear(); + const state = { + isVoiceModeActive: false, + hasConnectedOnce: false, + hasVoiceSessionStarted: false, + earlyServerThreadId: null as unknown, + server: { connectionState: "connecting", messages: ["private transcript"] }, + voiceSessionId: "private session ID", + pendingDisconnect: { error: "private error" }, + }; + const get = vi.fn(); + const scope = { get } as unknown as ChatVoiceScope; + let listener: (() => void) | undefined; + const unsubscribe = vi.fn(() => { + listener = undefined; + }); + const stop = observeNativeChatVoiceState(scope, { + SPn: () => ({ + getState: () => state, + subscribe: (next: () => void) => { + listener = next; + return unsubscribe; + }, + }), + }); + listener?.(); + expect(chatVoiceDebug).toHaveBeenCalledTimes(1); + Object.assign(state, { isVoiceModeActive: true, hasConnectedOnce: true }); + state.server.connectionState = "connected"; + listener?.(); + expect(chatVoiceDebug).toHaveBeenLastCalledWith("native-chat-state", { + active: true, + connectedOnce: true, + sessionStarted: false, + serverConversationReady: false, + connection: "connected", + }); + state.earlyServerThreadId = "a7e3282d-f457-4586-9b28-921411364f6b"; + listener?.(); + expect(chatVoiceDebug).toHaveBeenLastCalledWith( + "native-chat-state", + expect.objectContaining({ serverConversationReady: true }), + ); + expect(JSON.stringify(vi.mocked(chatVoiceDebug).mock.calls)).not.toContain( + state.earlyServerThreadId, + ); + // Unexpected native strings must not enter the persisted diagnostic ring. + state.server.connectionState = "private transport error"; + listener?.(); + expect(chatVoiceDebug).toHaveBeenLastCalledWith( + "native-chat-state", + expect.objectContaining({ + connection: "unknown", + }), + ); + expect(JSON.stringify(vi.mocked(chatVoiceDebug).mock.calls)).not.toContain("private"); + expect(get).not.toHaveBeenCalled(); + stop(); + expect(unsubscribe).toHaveBeenCalledOnce(); + expect(listener).toBeUndefined(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-native-chat-voice.test.ts b/packages/renderer-extension/test/renderer-native-chat-voice.test.ts new file mode 100644 index 000000000..a0ed7b462 --- /dev/null +++ b/packages/renderer-extension/test/renderer-native-chat-voice.test.ts @@ -0,0 +1,1057 @@ +import { + NATIVE_CHAT_VOICE_ASSETS, + NATIVE_CHAT_VOICE_VERSION, +} from "../src/renderer-native-chat-voice-assets.js"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { + detectNativeChatVoiceBinding, + installNativeChatVoiceHook, + startNativeChatVoiceForRequest, +} from "../src/renderer-native-chat-voice.js"; +import { + CHAT_LIVE_VOICE_STORAGE_KEY, + applySharedVoiceRoutes, +} from "../src/renderer-cpa-live-voice-preference.js"; + +function fixture() { + const scopeDefinition = { id: Symbol(), __scopeBrand: "AppScope" }; + const backendConversationId = {}; + const ownerScopeDefinition = { id: Symbol(), __scopeBrand: "RouteScope" }; + const context = { displayName: "AppScope" }; + const composerScopeDefinition = { id: Symbol(), __scopeBrand: "ComposerScope" }; + const composerContext = { displayName: "ComposerScope" }; + const composerNode = { + token: composerScopeDefinition, + value: { + kind: "chatgpt", + chatGptSelectionContext: { + conversationId: "desktop-local-conversation", + }, + }, + }; + const composerChain = new Map([ + [scopeDefinition.id, {}], + [composerScopeDefinition.id, composerNode], + ]); + const appNode = {}; + const node = {}; + const chain = new Map([ + [scopeDefinition.id, appNode], + [ownerScopeDefinition.id, node], + ]); + composerChain.set(scopeDefinition.id, appNode); + composerChain.set(ownerScopeDefinition.id, node); + const outerComposerChain = new Map(composerChain); + outerComposerChain.set(composerScopeDefinition.id, { + value: { kind: "new", entrypoint: "home" }, + }); + const get = vi.fn<(selector: unknown, argument?: unknown) => unknown>( + () => "6ad6391c-9a48-83e8-8a22-21548902d6d4", + ); + const scope = { scope: ownerScopeDefinition, node, chain, get, set: vi.fn() }; + const props = { + conversationId: "desktop-local-conversation", + isWorkConversation: false, + isTemporaryChat: false, + shouldBlockExternalEgress: false, + composerSelectionContext: { conversationId: "desktop-local-conversation" }, + }; + const owner = { + memoizedProps: props, + memoizedState: { memoizedState: { current: scope }, next: null as unknown }, + dependencies: { + firstContext: { + context, + memoizedValue: chain, + next: { context: composerContext, memoizedValue: outerComposerChain, next: null }, + }, + }, + return: null as unknown, + }; + const provider = { + tag: 10, + type: composerContext, + memoizedProps: { value: composerChain }, + return: owner, + }; + const fiber = { return: provider }; + const composer = { __reactFiber$fixture: fiber } as unknown as Element; + return { + scopeDefinition, + composerScopeDefinition, + composerContext, + composerNode, + composerChain, + backendConversationId, + ownerScopeDefinition, + context, + scope, + props, + owner, + provider, + fiber, + composer, + get, + }; +} + +describe("native Chat composer binding", () => { + it("reads the backend Chat ID from the owner's RouteScope, never its desktop locator", () => { + const f = fixture(); + const binding = detectNativeChatVoiceBinding(f.composer, f); + expect(binding).toMatchObject({ scope: f.scope, props: f.props, context: f.context }); + expect(f.get).toHaveBeenCalledWith(f.backendConversationId, "desktop-local-conversation"); + expect(f.scope.set).not.toHaveBeenCalled(); + }); + + it.each([ + { isWorkConversation: true }, + { isTemporaryChat: true }, + { shouldBlockExternalEgress: true }, + { shouldBlockExternalEgress: undefined }, + { readOnly: true }, + { composerSelectionContext: { conversationId: "another-composer" } }, + ])("rejects ineligible or mismatched Chat owners: %j", (overrides) => { + const f = fixture(); + Object.assign(f.props, overrides); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it.each([null, "local-chatgpt:draft", "codex-thread-id", undefined])( + "does not invent a backend conversation ID when the native store returns %s", + (id) => { + const f = fixture(); + f.get.mockReturnValue(id as string); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }, + ); + + it.each([null, undefined])("accepts an unsaved native Chat draft with backend ID %s", (id) => { + const f = fixture(); + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.composerNode.value.chatGptSelectionContext.conversationId = draft; + f.get.mockReturnValue(id); + expect(detectNativeChatVoiceBinding(f.composer, f)).toMatchObject({ + props: f.props, + scope: f.scope, + backendConversationId: undefined, + }); + expect(f.scope.set).not.toHaveBeenCalled(); + }); + + it("rejects an outer new-Chat scope without the inner conversation selection", () => { + const f = fixture(); + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.get.mockReturnValue(null); + Object.assign(f.composerNode.value, { kind: "new", entrypoint: "home" }); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("reads the inner provider below an owner that consumes the outer new-Chat scope", () => { + const f = fixture(); + expect(detectNativeChatVoiceBinding(f.composer, f)?.composerContext).toEqual({ + context: f.composerContext, + chain: f.composerChain, + }); + }); + + it("rejects a consumed ComposerScope without a provider between the form and owner", () => { + const f = fixture(); + f.owner.dependencies.firstContext.next.memoizedValue = f.composerChain; + Object.assign(f.fiber, { return: f.owner }); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("rejects a provider with a different native scope token or RouteScope node", () => { + const f = fixture(); + Object.assign(f.composerNode, { token: { ...f.composerScopeDefinition } }); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + f.composerNode.token = f.composerScopeDefinition; + f.composerChain.set(f.ownerScopeDefinition.id, {}); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("does not skip a nearer empty provider to accept a more distant valid scope", () => { + const f = fixture(); + Object.assign(f.fiber, { + return: { ...f.provider, memoizedProps: { value: new Map() }, return: f.provider }, + }); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("rejects an unrecognized draft locator even when its backend ID is absent", () => { + const f = fixture(); + const draft = "local-chatgpt:draft"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.composerNode.value.chatGptSelectionContext.conversationId = draft; + f.get.mockReturnValue(null); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("rejects absent or mismatched native ComposerScope ownership", () => { + const f = fixture(); + f.composerNode.value.chatGptSelectionContext.conversationId = "different-chat"; + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + f.composerChain.delete(f.composerScopeDefinition.id); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("rejects contexts from another AppScope", () => { + const f = fixture(); + f.composerChain.set(f.scopeDefinition.id, {}); + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("deduplicates native scope refs by their owning RouteScope node", () => { + const f = fixture(); + f.owner.memoizedState.next = { memoizedState: { current: { ...f.scope } }, next: null }; + expect(detectNativeChatVoiceBinding(f.composer, f)?.scope.node).toBe(f.scope.node); + }); + + it("rejects competing RouteScope nodes and cycles in native hook state", () => { + const f = fixture(); + const node = {}; + f.owner.memoizedState.next = { + memoizedState: { + current: { + ...f.scope, + node, + chain: new Map([ + [f.scopeDefinition.id, node], + [f.ownerScopeDefinition.id, node], + ]), + }, + }, + next: null, + }; + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + f.owner.memoizedState.next = f.owner.memoizedState; + expect(detectNativeChatVoiceBinding(f.composer, f)).toBeUndefined(); + }); + + it("uses the published owner after React swaps an alternate", () => { + const f = fixture(); + const staleOwner = { ...f.owner, memoizedProps: { ...f.props, isWorkConversation: true } }; + const stale = { return: staleOwner, alternate: f.fiber }; + const root = { child: f.owner, stateNode: { current: null as unknown } }; + Object.assign(f.owner, { return: root, child: f.provider }); + Object.assign(f.provider, { child: f.fiber }); + root.stateNode.current = root; + staleOwner.return = root; + Object.assign(f.composer, { __reactFiber$fixture: stale }); + expect(detectNativeChatVoiceBinding(f.composer, f)?.props).toBe(f.props); + }); +}); + +describe("Chat voice service launch boundary", () => { + it("leaves Codex Desktop default on the native Codex service without invoking Chat START", () => { + const start = vi.fn(); + const target = { __codexhostNativeChatVoiceV1: { status: "active", start } }; + applySharedVoiceRoutes(target, { + chatExperience: "codex", + chatConnection: "desktop", + workConnection: "gateway", + }); + expect(startNativeChatVoiceForRequest({ productExperience: "chat" }, target)).toBeUndefined(); + expect(start).not.toHaveBeenCalled(); + }); + + it("starts native Chat and propagates failure without invoking a new-thread overlay", async () => { + const start = vi.fn().mockRejectedValue(new Error("native unavailable")); + const target = { __codexhostNativeChatVoiceV1: { status: "active", start } }; + const request = { source: "composer_button_new_thread", productExperience: "chat" }; + await expect(startNativeChatVoiceForRequest(request, target)).rejects.toThrow( + "native unavailable", + ); + expect(start).toHaveBeenCalledOnce(); + }); + + it("leaves Work, Grok and explicit Chat gateway launches on their existing path", () => { + const start = vi.fn(); + const target = { + __codexhostNativeChatVoiceV1: { status: "active", start }, + localStorage: { + getItem: (key: string) => (key === CHAT_LIVE_VOICE_STORAGE_KEY ? "1" : null), + }, + }; + expect(startNativeChatVoiceForRequest({ productExperience: "work" }, target)).toBeUndefined(); + expect(startNativeChatVoiceForRequest({ productExperience: "chat" }, target)).toBeUndefined(); + expect(start).not.toHaveBeenCalled(); + }); + + it("fails through the pending native Chat binding instead of accepting a stale conversation", async () => { + const start = vi.fn().mockRejectedValue(new Error("Select a ChatGPT conversation")); + await expect( + startNativeChatVoiceForRequest( + { productExperience: "chat" }, + { + __codexhostNativeChatVoiceV1: { status: "pending", start }, + }, + ), + ).rejects.toThrow("Select a ChatGPT conversation"); + }); +}); + +function installFixture() { + const f = fixture(); + const enabled = {}; + let browserEnabled: unknown = false; + let active = false; + let connected = true; + let sessionStarted = true; + let serverId: unknown = "a7e3282d-f457-4586-9b28-921411364f6b"; + const listeners = new Set<() => void>(); + const unsubscribe = vi.fn(); + const start = vi.fn<(...args: unknown[]) => Promise>(async () => undefined); + f.scope.get.mockImplementation((selector: unknown) => + selector === enabled ? browserEnabled : "6ad6391c-9a48-83e8-8a22-21548902d6d4", + ); + f.scope.set.mockImplementation((_selector, value) => { + browserEnabled = value; + }); + const render = vi.fn(); + const unmount = vi.fn(); + const createRoot = vi.fn(() => ({ render, unmount })); + const createElement = vi.fn((type, props, ...children) => ({ type, props, children })); + const elements: Record[] = []; + const createElementDom = () => { + const el = { + type: "", + className: "", + title: "", + disabled: false, + style: {}, + setAttribute: vi.fn(), + remove: vi.fn(), + addEventListener: vi.fn(), + }; + elements.push(el); + return el; + }; + let visible = true; + const parent = { + insertBefore: vi.fn((element: Record, anchor: unknown) => { + Object.assign(element, { nextElementSibling: anchor }); + }), + }; + const send = { + type: "submit", + className: "native-send-button", + parentElement: parent, + getBoundingClientRect: () => ({ width: 28 }), + closest: () => null, + }; + const styleValues = new Map([["display", ["inline-flex", ""]]]); + const overlayButton = { + style: { + getPropertyValue: (key: string) => styleValues.get(key)?.[0] ?? "", + getPropertyPriority: (key: string) => styleValues.get(key)?.[1] ?? "", + setProperty: (key: string, value: string, priority: string) => + styleValues.set(key, [value, priority]), + }, + }; + const overlayButtons = [overlayButton]; + Object.assign(f.composer, { + getBoundingClientRect: () => ({ width: visible ? 500 : 0 }), + closest: () => null, + contains: () => true, + querySelector: () => null, + querySelectorAll: vi.fn((selector: string) => + selector === 'button[type="submit"]' ? [send] : overlayButtons, + ), + }); + const initial = { + B1n: f.backendConversationId, + CDr: f.composerScopeDefinition, + MOr: f.ownerScopeDefinition, + aT: enabled, + oT: vi.fn(), + Ycr: {}, + n0n: {}, + Y1n: {}, + Nir: vi.fn(), + SPn: () => ({ + getState: () => ({ + isVoiceModeActive: active, + hasConnectedOnce: connected, + hasVoiceSessionStarted: sessionStarted, + earlyServerThreadId: serverId, + server: { connectionState: connected ? "connected" : null }, + isWorkMode: false, + conversationId: f.props.conversationId, + }), + subscribe: (listener: () => void) => { + listeners.add(listener); + return () => { + listeners.delete(listener); + unsubscribe(); + }; + }, + }), + }; + const shared = { + Qst: f.scopeDefinition, + FBt: () => ({ createElement }), + PNt: () => ({ createRoot }), + Vot: vi.fn((id: string) => `/c/${encodeURIComponent(id)}`), + }; + const assets = ["app-initial-dddef47d6ef6.js", "app-shared-fc3316d48427.js"]; + const doc = { + body: { append: vi.fn() }, + createElement: createElementDom, + querySelectorAll: (selector: string) => + selector === "[data-chatgpt-composer]" + ? [f.composer] + : selector === '[style*="--vt-splash-screen-headline"]' + ? [] + : assets.map((name) => ({ getAttribute: () => `app://-/assets/${name}` })), + } as unknown as Document; + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: "26.1016.62342", + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: doc, + performance: { getEntriesByType: () => [] }, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async (url: string): Promise> => { + if (url.includes("app-shared")) return shared; + if (url.includes("app-initial")) return initial; + if (url.includes("start-voice")) return { t: vi.fn(), n: start }; + return { VoiceInstantConnectCacheController: vi.fn() }; + }); + return { + ...f, + target, + importModule, + start, + render, + unmount, + createRoot, + createElement, + elements, + initial, + shared, + assets, + parent, + send, + overlayButton, + overlayButtons, + browserEnabled: () => browserEnabled, + listeners, + unsubscribe, + setSessionStarted: (value: boolean) => { + sessionStarted = value; + for (const listener of listeners) listener(); + }, + setServerId: (value: unknown) => { + serverId = value; + for (const listener of listeners) listener(); + }, + setConnected: (value: boolean) => { + connected = value; + for (const listener of listeners) listener(); + }, + setActive: (value: boolean) => { + active = value; + for (const listener of listeners) listener(); + }, + setVisible: (value: boolean) => { + visible = value; + }, + }; +} + +afterEach(() => vi.useRealTimers()); +describe("native Chat voice lifecycle installation", () => { + it.each(NATIVE_CHAT_VOICE_ASSETS.filter((p) => p.version === NATIVE_CHAT_VOICE_VERSION))( + "starts $platform Chat through the relocated composer START and primary browser runtime", + async (assets) => { + vi.useFakeTimers(); + const f = installFixture(); + Object.assign(f.target.__codexhostCpaLiveVoiceHookV1, { + desktopVersion: assets.version, + desktopPlatform: assets.platform, + }); + f.assets.splice(0, f.assets.length, assets.initial, assets.shared); + const primary: Record = {}; + const initializeBrowser = vi.fn(() => { + primary.lm = f.initial.aT; + }); + primary.um = initializeBrowser; + const initializeStart = vi.fn(); + f.importModule.mockImplementation(async (url) => { + if (url.endsWith(assets.initial)) + return { + CQn: f.initial.MOr, + PXn: f.initial.CDr, + tdn: f.initial.B1n, + qXt: f.initial.SPn, + Xkn: f.initial.Ycr, + ydn: f.initial.n0n, + fdn: f.initial.Y1n, + yAn: f.initial.Nir, + }; + if (assets.primary && url.endsWith(assets.primary)) return primary; + if (url.endsWith(assets.shared)) + return { Bst: f.shared.Qst, XVt: f.shared.FBt, KNt: f.shared.PNt, Fot: f.shared.Vot }; + if (url.endsWith(assets.start)) return { r: initializeStart, i: f.start }; + if (url.endsWith(assets.prewarm)) return { VoiceInstantConnectCacheController: vi.fn() }; + throw new Error("Unaudited module"); + }); + const hook = installNativeChatVoiceHook(f); + try { + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(initializeBrowser).toHaveBeenCalledOnce(); + expect(initializeStart).toHaveBeenCalledOnce(); + expect(f.browserEnabled()).toBe(true); + expect(f.createRoot).toHaveBeenCalledOnce(); + await hook.start(); + expect(f.start).toHaveBeenCalledExactlyOnceWith( + f.scope, + f.props, + { event_source: "composer" }, + expect.any(AbortSignal), + ); + expect(f.initial.oT).not.toHaveBeenCalled(); + } finally { + hook.dispose(); + } + expect(f.browserEnabled()).toBe(false); + expect(f.listeners.size).toBe(0); + }, + ); + + it("keeps one AppScope runtime when navigation replaces the RouteScope", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + const node = {}; + const chain = new Map(f.scope.chain); + chain.set(f.ownerScopeDefinition.id, node); + const next = { ...f.scope, node, chain }; + f.owner.memoizedState.memoizedState.current = next; + f.composerChain.set(f.ownerScopeDefinition.id, node); + await vi.advanceTimersByTimeAsync(500); + expect(hook.status).toBe("active"); + expect(f.createRoot).toHaveBeenCalledOnce(); + expect(f.unmount).not.toHaveBeenCalled(); + await hook.start(); + expect(f.start).toHaveBeenCalledWith( + next, + f.props, + { event_source: "composer" }, + expect.any(AbortSignal), + ); + hook.dispose(); + }); + it("selects the adopted voice draft while its composer is hidden without releasing the native runtime", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + const backend = "a7e3282d-f457-4586-9b28-921411364f6b"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.composerNode.value.chatGptSelectionContext.conversationId = draft; + let adopted = false; + const nativeGet = f.scope.get.getMockImplementation(); + if (!nativeGet) throw new Error("Fixture native getter is missing"); + f.scope.get.mockImplementation((selector, argument) => { + if (selector === f.initial.B1n) return adopted ? backend : null; + if (selector === f.initial.Y1n) return adopted ? "message" : null; + if (selector === f.initial.n0n) + return adopted ? { message: { message: { author: { role: "user" } } } } : null; + return nativeGet(selector, argument); + }); + f.start.mockImplementation(async () => { + f.setActive(true); + }); + const hook = installNativeChatVoiceHook(f); + try { + await vi.advanceTimersByTimeAsync(0); + await hook.start(); + const signal = f.start.mock.calls[0]?.[3] as AbortSignal; + f.setVisible(false); + await vi.advanceTimersByTimeAsync(1000); + expect(f.initial.Nir).not.toHaveBeenCalled(); + adopted = true; + await vi.advanceTimersByTimeAsync(500); + expect(f.initial.Nir).toHaveBeenCalledExactlyOnceWith( + f.scope, + `/c/${encodeURIComponent(draft)}`, + ); + expect(f.shared.Vot).toHaveBeenCalledExactlyOnceWith(draft); + // Native canonicalization publishes the selected Chat composer in the same AppScope. + f.props.conversationId = backend; + f.props.composerSelectionContext.conversationId = backend; + f.composerNode.value.kind = "chatgpt"; + f.composerNode.value.chatGptSelectionContext.conversationId = backend; + f.setVisible(true); + await vi.advanceTimersByTimeAsync(1000); + expect(f.createRoot).toHaveBeenCalledOnce(); + expect(f.unmount).not.toHaveBeenCalled(); + expect(signal.aborted).toBe(false); + expect(hook.activeCall).toBe(true); + expect(f.initial.Nir).toHaveBeenCalledOnce(); + } finally { + hook.dispose(); + } + }); + + it("starts native voice without private page-navigation exports", async () => { + vi.useFakeTimers(); + const f = installFixture(); + for (const key of ["Nir", "Ycr"]) Reflect.deleteProperty(f.initial, key); + Reflect.deleteProperty(f.shared, "Vot"); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + await hook.start(); + expect(f.start).toHaveBeenCalledOnce(); + hook.dispose(); + }); + + it.each(["saved", "draft"])( + "keeps native voice on its page through connection, session startup and retry: %s", + async (kind) => { + vi.useFakeTimers(); + const f = installFixture(); + if (kind === "draft") { + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.composerNode.value.chatGptSelectionContext.conversationId = draft; + const nativeGet = f.scope.get.getMockImplementation(); + if (!nativeGet) throw new Error("Fixture native getter is missing"); + f.scope.get.mockImplementation((selector, argument) => + selector === f.backendConversationId ? null : nativeGet(selector, argument), + ); + } + f.setConnected(false); + f.setSessionStarted(false); + f.setServerId(null); + f.start.mockImplementation(async () => { + f.setActive(true); + }); + // The real Desktop failure ends voice during a conversation-page transition. + f.initial.Nir.mockImplementation(() => { + f.setActive(false); + }); + const hook = installNativeChatVoiceHook(f); + try { + await vi.advanceTimersByTimeAsync(0); + let acknowledged = false; + const starting = hook.start().then(() => { + acknowledged = true; + }); + await vi.advanceTimersByTimeAsync(0); + expect(acknowledged).toBe(true); + await starting; + f.setConnected(true); + f.setSessionStarted(true); + f.setServerId("a7e3282d-f457-4586-9b28-921411364f6b"); + f.setVisible(false); + await vi.advanceTimersByTimeAsync(1000); + expect(hook.activeCall).toBe(true); + expect(f.unmount).not.toHaveBeenCalled(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + expect(f.shared.Vot).not.toHaveBeenCalled(); + f.setActive(false); + f.setVisible(true); + await vi.advanceTimersByTimeAsync(500); + await hook.start(); + expect(f.start).toHaveBeenCalledTimes(2); + expect(hook.activeCall).toBe(true); + expect(f.initial.Nir).not.toHaveBeenCalled(); + } finally { + hook.dispose(); + } + expect(f.listeners.size).toBe(0); + }, + ); + + it("retains startup through a hidden composer before native START returns", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const gate = Promise.withResolvers(); + f.start.mockImplementation(() => gate.promise); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + const starting = hook.start(); + const signal = f.start.mock.calls[0]?.[3] as AbortSignal; + f.setVisible(false); + await vi.advanceTimersByTimeAsync(1000); + expect(signal.aborted).toBe(false); + expect(f.unmount).not.toHaveBeenCalled(); + expect(f.browserEnabled()).toBe(true); + gate.resolve(undefined); + await starting; + await vi.advanceTimersByTimeAsync(500); + expect(f.unmount).toHaveBeenCalledOnce(); + hook.dispose(); + }); + + it("places native Voice immediately before Send when Dictate is absent", async () => { + vi.useFakeTimers(); + const f = installFixture(); + Object.assign(f.composer, { querySelector: () => null }); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + const voice = f.elements.find((element) => element.title === "Start Voice"); + expect(voice).toBeDefined(); + expect(f.parent.insertBefore).toHaveBeenCalledWith(voice, f.send); + expect(voice?.type).toBe("button"); + await hook.start(); + expect(f.start).toHaveBeenCalledOnce(); + hook.dispose(); + }); + + it("does not place Voice when Send ownership is ambiguous", async () => { + vi.useFakeTimers(); + const f = installFixture(); + Object.assign(f.composer, { + querySelectorAll: (selector: string) => + selector === 'button[type="submit"]' ? [f.send, { ...f.send }] : f.overlayButtons, + }); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(f.parent.insertBefore).not.toHaveBeenCalled(); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("inline-flex"); + hook.dispose(); + }); + + it("shows only native Chat START and restores the overlay control on Work or gateway override", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(f.parent.insertBefore).toHaveBeenCalledOnce(); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("none"); + expect(f.overlayButton.style.getPropertyPriority("display")).toBe("important"); + expect(f.composer.querySelectorAll).toHaveBeenCalledWith( + 'button[aria-label="Start new voice chat"], button[aria-label="Start voice chat"]', + ); + f.props.isWorkConversation = true; + await vi.advanceTimersByTimeAsync(500); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("inline-flex"); + expect(f.overlayButton.style.getPropertyPriority("display")).toBe(""); + f.props.isWorkConversation = false; + await vi.advanceTimersByTimeAsync(500); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("none"); + Object.assign(f.target, { + localStorage: { + getItem: (key: string) => (key === CHAT_LIVE_VOICE_STORAGE_KEY ? "1" : null), + }, + }); + await vi.advanceTimersByTimeAsync(500); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("inline-flex"); + expect(hook.status).toBe("pending"); + hook.dispose(); + }); + + it("keeps the native control visible when the replacement cannot be inserted", async () => { + vi.useFakeTimers(); + const f = installFixture(); + Object.assign(f.composer, { querySelectorAll: () => [] }); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(f.parent.insertBefore).not.toHaveBeenCalled(); + expect(f.overlayButton.style.getPropertyValue("display")).toBe("inline-flex"); + hook.dispose(); + }); + it("starts a new Chat through native START without synthesizing a backend UUID", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const draft = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + f.props.conversationId = draft; + f.props.composerSelectionContext.conversationId = draft; + f.composerNode.value.chatGptSelectionContext.conversationId = draft; + const nativeGet = f.scope.get.getMockImplementation(); + if (!nativeGet) throw new Error("Fixture native getter is missing"); + f.scope.get.mockImplementation((selector, argument) => + selector === f.backendConversationId ? null : nativeGet(selector, argument), + ); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(f.parent.insertBefore).toHaveBeenCalledOnce(); + await hook.start(); + expect(f.start).toHaveBeenCalledWith( + f.scope, + f.props, + { event_source: "composer" }, + expect.any(AbortSignal), + ); + expect(f.props.conversationId).toBe(draft); + expect(f.initial.Nir).not.toHaveBeenCalled(); + expect(f.scope.set).toHaveBeenCalledWith(f.initial.aT, true); + expect(f.scope.set).toHaveBeenCalledTimes(1); + hook.dispose(); + }); + + it("restores Start Voice after leaving Chat, returning, and native composer replacement", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + const firstButton = f.parent.insertBefore.mock.calls[0]?.[0]; + if (!firstButton) throw new Error("Native voice button was not inserted"); + f.props.isWorkConversation = true; + await vi.advanceTimersByTimeAsync(500); + expect(hook.status).toBe("pending"); + expect(firstButton.remove).toHaveBeenCalledOnce(); + expect(f.browserEnabled()).toBe(false); + f.props.isWorkConversation = false; + await vi.advanceTimersByTimeAsync(500); + expect(hook.status).toBe("active"); + expect(f.browserEnabled()).toBe(true); + expect(f.parent.insertBefore).toHaveBeenCalledTimes(2); + Object.assign(f.composer, { contains: () => false }); + await vi.advanceTimersByTimeAsync(500); + expect(f.parent.insertBefore).toHaveBeenCalledTimes(3); + expect(f.createRoot).toHaveBeenCalledTimes(2); + expect(f.start).not.toHaveBeenCalled(); + f.start.mockImplementation(async () => { + f.setActive(true); + return undefined; + }); + await hook.start(); + expect(f.start).toHaveBeenCalledOnce(); + expect(f.initial.Nir).not.toHaveBeenCalled(); + hook.dispose(); + }); + + it("installs again on a fresh document without a session-only activation flag", async () => { + vi.useFakeTimers(); + const first = installFixture(); + const previous = installNativeChatVoiceHook(first); + await vi.advanceTimersByTimeAsync(0); + expect(previous.status).toBe("active"); + previous.dispose(); + const next = installFixture(); + const hook = installNativeChatVoiceHook(next); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(next.browserEnabled()).toBe(true); + expect(next.parent.insertBefore).toHaveBeenCalledOnce(); + hook.dispose(); + }); + + it.each(["26.1013.40934", "26.1016.51333", "26.1016.62343"])( + "rejects unqualified Desktop %s before importing or enabling native Chat", + async (desktopVersion) => { + const f = installFixture(); + f.target.__codexhostCpaLiveVoiceHookV1.desktopVersion = desktopVersion; + // Even cached audited assets must not bypass the installed-version guard. + const hook = installNativeChatVoiceHook(f); + expect(hook.status).toBe("unsupported"); + await expect(hook.start()).rejects.toThrow("Native Chat voice is unavailable"); + expect(f.importModule).not.toHaveBeenCalled(); + expect(f.initial.oT).not.toHaveBeenCalled(); + expect(f.scope.set).not.toHaveBeenCalled(); + expect(f.start).not.toHaveBeenCalled(); + expect(f.createRoot).not.toHaveBeenCalled(); + expect(f.parent.insertBefore).not.toHaveBeenCalled(); + hook.dispose(); + }, + ); + + it.each([ + [ + "26.1016.62342", + "app-initial-dddef47d6ef6.js", + "app-shared-fc3316d48427.js", + "start-voice-7cbf0a39d53b.js", + "voice-prewarm-5d107129ac1c.js", + ], + [ + "26.1016.61520", + "app-initial-c95ddb62696e.js", + "app-shared-2eda13eb7044.js", + "start-voice-18b08f0acdd7.js", + "voice-prewarm-94271e581c1b.js", + ], + ])( + "starts native Chat on %s using only its qualified modules", + async (version, initial, shared, start, prewarm) => { + vi.useFakeTimers(); + const f = installFixture(); + f.target.__codexhostCpaLiveVoiceHookV1.desktopVersion = version; + f.assets.splice(0, f.assets.length, initial, shared); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(f.importModule.mock.calls.map(([url]) => url)).toEqual( + [shared, initial, start, prewarm].map((asset) => `app://-/assets/${asset}`), + ); + await hook.start(); + expect(f.start).toHaveBeenCalledWith( + f.scope, + f.props, + { event_source: "composer" }, + expect.any(AbortSignal), + ); + hook.dispose(); + }, + ); + + it.each([ + ["26.1016.62342", ["app-initial-c95ddb62696e.js", "app-shared-2eda13eb7044.js"]], + ["26.1016.62342", ["app-initial-dddef47d6ef6.js", "app-shared-2eda13eb7044.js"]], + [ + undefined, + ["app-initial-dddef47d6ef6.js", "app-shared-fc3316d48427.js", "app-initial-c95ddb62696e.js"], + ], + ])("rejects mismatched or ambiguous Chat assets (%s, %j)", async (version, names) => { + vi.useFakeTimers(); + const f = installFixture(); + if (version === undefined) + Reflect.deleteProperty(f.target.__codexhostCpaLiveVoiceHookV1, "desktopVersion"); + else f.target.__codexhostCpaLiveVoiceHookV1.desktopVersion = version; + f.assets.splice(0, f.assets.length, ...names); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(60_000); + expect(hook.status).toBe("unsupported"); + expect(f.importModule).not.toHaveBeenCalled(); + expect(f.initial.oT).not.toHaveBeenCalled(); + expect(f.start).not.toHaveBeenCalled(); + hook.dispose(); + }); + + it("rejects stale Chat assets when the installed Desktop version is unavailable", async () => { + vi.useFakeTimers(); + const f = installFixture(); + Reflect.deleteProperty(f.target.__codexhostCpaLiveVoiceHookV1, "desktopVersion"); + f.assets.splice( + 0, + f.assets.length, + "app-initial-90eb42395b7d.js", + "app-shared-52443cbe9ac4.js", + "app-primary-58a09006cc0c.js", + "start-voice-18b08f0acdd7.js", + "voice-prewarm-94271e581c1b.js", + ); + const hook = installNativeChatVoiceHook(f); + expect(hook.status).toBe("pending"); + await vi.advanceTimersByTimeAsync(60_000); + expect(hook.status).toBe("unsupported"); + await expect(hook.start()).rejects.toThrow("Select a ChatGPT conversation"); + expect(f.importModule).not.toHaveBeenCalled(); + expect(f.scope.set).not.toHaveBeenCalled(); + expect(f.start).not.toHaveBeenCalled(); + expect(f.createRoot).not.toHaveBeenCalled(); + expect(f.parent.insertBefore).not.toHaveBeenCalled(); + hook.dispose(); + }); + + it("supplies the native prewarm controller with both AppScope and ComposerScope", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(1000); + const element = f.render.mock.calls[0]?.[0]; + const providers = new Map(); + for (let e = element; e?.props?.value; e = e.children[0]) providers.set(e.type, e.props.value); + expect(providers.get(f.context)).toBe(f.scope.chain); + expect(providers.get(f.composerContext)).toBe(f.composerChain); + expect(hook.status).toBe("active"); + hook.dispose(); + }); + + it("updates native ComposerScope on selection changes without replacing the prewarm root", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + const nextChain = new Map(f.composerChain); + f.provider.memoizedProps.value = nextChain; + await vi.advanceTimersByTimeAsync(500); + expect(f.createRoot).toHaveBeenCalledOnce(); + expect(f.render).toHaveBeenCalledTimes(2); + const element = f.render.mock.calls[1]?.[0]; + const providers = new Map(); + for (let e = element; e?.props?.value; e = e.children[0]) providers.set(e.type, e.props.value); + expect(providers.get(f.composerContext)).toBe(nextChain); + hook.dispose(); + }); + + it("mounts native prewarm once across Chat selections and dispatches the native START with current props", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + expect(hook.status).toBe("active"); + expect(f.browserEnabled()).toBe(true); + expect(f.render).toHaveBeenCalledOnce(); + f.props.conversationId = "second-desktop-chat"; + f.props.composerSelectionContext.conversationId = "second-desktop-chat"; + f.composerNode.value.chatGptSelectionContext.conversationId = "second-desktop-chat"; + await vi.advanceTimersByTimeAsync(1000); + expect(f.createRoot).toHaveBeenCalledOnce(); + expect(f.render).toHaveBeenCalledOnce(); + expect(f.start).not.toHaveBeenCalled(); + await hook.start(); + expect(f.start).toHaveBeenCalledWith( + f.scope, + f.props, + { event_source: "composer" }, + expect.any(AbortSignal), + ); + expect(f.initial.Nir).not.toHaveBeenCalled(); + hook.dispose(); + expect(f.unmount).toHaveBeenCalledOnce(); + expect(f.browserEnabled()).toBe(false); + }); + + it("retains native runtime while voice hides the composer, then restores it when returning to Work", async () => { + vi.useFakeTimers(); + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + await vi.advanceTimersByTimeAsync(0); + f.setActive(true); + f.setVisible(false); + await vi.advanceTimersByTimeAsync(500); + expect(f.unmount).not.toHaveBeenCalled(); + expect(f.browserEnabled()).toBe(true); + f.setActive(false); + f.setVisible(true); + f.props.isWorkConversation = true; + await vi.advanceTimersByTimeAsync(500); + expect(hook.status).toBe("pending"); + expect(f.unmount).toHaveBeenCalledOnce(); + expect(f.browserEnabled()).toBe(false); + await expect(hook.start()).rejects.toThrow("Select a ChatGPT conversation"); + expect(f.start).not.toHaveBeenCalled(); + hook.dispose(); + }); + + it("fails closed before importing native code on an unaudited Desktop", async () => { + const f = installFixture(); + f.target.__codexhostCpaLiveVoiceHookV1.desktopVersion = "26.1020.1"; + const hook = installNativeChatVoiceHook(f); + expect(hook.status).toBe("unsupported"); + expect(f.importModule).not.toHaveBeenCalled(); + expect(f.scope.set).not.toHaveBeenCalled(); + }); + + it("does not mount after disposal during native module loading", async () => { + const f = installFixture(); + const hook = installNativeChatVoiceHook(f); + hook.dispose(); + await Promise.resolve(); + await Promise.resolve(); + expect(f.createRoot).not.toHaveBeenCalled(); + expect(f.scope.set).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-overlay-external-followups.test.ts b/packages/renderer-extension/test/renderer-overlay-external-followups.test.ts new file mode 100644 index 000000000..2d7a6912e --- /dev/null +++ b/packages/renderer-extension/test/renderer-overlay-external-followups.test.ts @@ -0,0 +1,104 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { installOverlayExternalFollowUps } from "../src/renderer-overlay-external-followups.js"; + +const overlayHref = "app://-/index.html?initialRoute=%2Favatar-overlay"; + +function followUpManager() { + const conversations = new Map([["external", { id: "external", modelProvider: "codexhost" }]]); + const isEnabled = vi.fn(() => true); + const queue = { isEnabled }; + const coordinator = { + serverQueue: queue, + getTurnCoordinator() { + return this; + }, + }; + const manager = { + sendRequest: vi.fn(async (method: string, params: unknown) => ({ method, params })), + steerTurn: vi.fn(), + startTurn: vi.fn(), + getStreamRole: vi + .fn<(threadId: string) => { role: string }>() + .mockReturnValue({ role: "follower" }), + getTurnCoordinator: () => coordinator, + getConversation: (threadId: string) => conversations.get(threadId) ?? null, + }; + return { manager, queue, originalGate: isEnabled }; +} + +function overlayDocument(manager?: object): Document { + const surface: Record = {}; + if (manager) { + Object.defineProperty(surface, "__reactFiber$test", { + value: { memoizedState: { memoizedState: manager, next: null }, return: null }, + configurable: true, + }); + } + const attach = (value: object) => { + Object.defineProperty(surface, "__reactFiber$test", { + value: { memoizedState: { memoizedState: value, next: null }, return: null }, + configurable: true, + }); + }; + const document = { + querySelectorAll: (selector: string) => + selector.includes("data-avatar-overlay-native-surface-id") ? [surface] : [], + documentElement: {}, + attach, + }; + return document as unknown as Document; +} + +describe("overlay external follow-up bindings", () => { + afterEach(() => { + vi.useRealTimers(); + }); + + it("stays disabled outside the avatar overlay", () => { + const hook = installOverlayExternalFollowUps({ + location: { href: "app://-/index.html" }, + document: overlayDocument(), + setTimeout, + clearTimeout, + }); + expect(hook.status).toBe("disabled"); + hook.dispose(); + }); + + it("treats overlay-owned External Threads as the stream owner so Host queue drain can run", () => { + const f = followUpManager(); + const hook = installOverlayExternalFollowUps({ + location: { href: overlayHref }, + document: overlayDocument(f.manager), + setTimeout, + clearTimeout, + }); + try { + expect(hook.status).toBe("active"); + expect(f.manager.getStreamRole("external")).toEqual({ role: "owner" }); + expect(f.queue.isEnabled).toBe(f.originalGate); + } finally { + hook.dispose(); + } + expect(f.manager.getStreamRole("external")).toEqual({ role: "follower" }); + }); + + it("keeps polling until the overlay surface publishes a manager", () => { + vi.useFakeTimers(); + const f = followUpManager(); + const root = overlayDocument(); + const hook = installOverlayExternalFollowUps({ + location: { href: overlayHref }, + document: root, + setTimeout, + clearTimeout, + }); + expect(hook.status).toBe("pending"); + (root as unknown as { attach(value: object): void }).attach(f.manager); + vi.advanceTimersByTime(50); + expect(hook.status).toBe("active"); + expect(f.manager.getStreamRole("external")).toEqual({ role: "owner" }); + hook.dispose(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-overlay-thread-isolation.test.ts b/packages/renderer-extension/test/renderer-overlay-thread-isolation.test.ts new file mode 100644 index 000000000..c76ab5937 --- /dev/null +++ b/packages/renderer-extension/test/renderer-overlay-thread-isolation.test.ts @@ -0,0 +1,237 @@ +import { describe, expect, it } from "vitest"; + +import { + installOverlayThreadIsolation, + isAvatarOverlayHref, + notificationThreadId, + overlayBlocksHostNotification, + type OverlayIsolationMessageEvent, + type OverlayIsolationTarget, +} from "../src/renderer-overlay-thread-isolation.js"; + +class FakeOverlayWindow implements OverlayIsolationTarget { + readonly listeners = new Map void>>(); + constructor(readonly location: { href: string }) {} + + addEventListener( + type: "message", + listener: (event: OverlayIsolationMessageEvent) => void, + options?: boolean | { capture?: boolean }, + ): void { + if (type !== "message") return; + const capture = options === true || (typeof options === "object" && options.capture === true); + const set = this.listeners.get(capture) ?? new Set(); + set.add(listener); + this.listeners.set(capture, set); + } + + removeEventListener( + type: "message", + listener: (event: OverlayIsolationMessageEvent) => void, + options?: boolean | { capture?: boolean }, + ): void { + const capture = options === true || (typeof options === "object" && options.capture === true); + this.listeners.get(capture)?.delete(listener); + } + + dispatch(data: unknown): boolean { + let stopped = false; + const event: OverlayIsolationMessageEvent = { + data, + stopImmediatePropagation() { + stopped = true; + }, + }; + for (const listener of this.listeners.get(true) ?? []) { + listener(event); + if (stopped) return true; + } + for (const listener of this.listeners.get(false) ?? []) { + listener(event); + if (stopped) return true; + } + return false; + } +} + +const overlayHref = "app://-/index.html?initialRoute=%2Favatar-overlay"; + +describe("overlay thread isolation", () => { + it("keeps delivering later text turns for a voice Thread after hangup", () => { + const overlay = new FakeOverlayWindow({ href: overlayHref }); + const hook = installOverlayThreadIsolation(overlay); + const dispatch = (method: string, params: Record) => + overlay.dispatch({ type: "mcp-notification", method, params }); + dispatch("thread/realtime/started", { threadId: "known-thread" }); + dispatch("turn/started", { threadId: "known-thread", turn: { id: "voice-turn" } }); + dispatch("thread/realtime/closed", { threadId: "known-thread" }); + dispatch("turn/completed", { threadId: "known-thread", turn: { id: "voice-turn" } }); + dispatch("thread/realtime/started", { threadId: "another-voice-thread" }); + expect( + dispatch("turn/started", { threadId: "known-thread", turn: { id: "later-text-turn" } }), + ).toBe(false); + expect( + dispatch("item/completed", { threadId: "known-thread", turnId: "later-text-turn" }), + ).toBe(false); + expect( + dispatch("turn/completed", { threadId: "known-thread", turn: { id: "later-text-turn" } }), + ).toBe(false); + expect(dispatch("turn/started", { threadId: "unknown-cli-thread" })).toBe(true); + dispatch("thread/deleted", { threadId: "known-thread" }); + expect(dispatch("turn/started", { threadId: "known-thread" })).toBe(true); + hook.dispose(); + }); + + it("recognizes avatar overlay hrefs", () => { + expect(isAvatarOverlayHref(overlayHref)).toBe(true); + expect(isAvatarOverlayHref("app://-/avatar-overlay.html")).toBe(true); + expect(isAvatarOverlayHref("app://-/index.html")).toBe(false); + }); + + it("blocks leftover overlay from ingesting another Thread's live work", () => { + expect(overlayBlocksHostNotification("thread/started", "cli-thread", new Set())).toBe(true); + expect(overlayBlocksHostNotification("turn/started", "cli-thread", new Set())).toBe(true); + expect(overlayBlocksHostNotification("item/completed", "cli-thread", new Set())).toBe(true); + expect( + overlayBlocksHostNotification("thread/realtime/started", "voice-thread", new Set()), + ).toBe(false); + expect( + overlayBlocksHostNotification("item/completed", "voice-thread", new Set(["voice-thread"])), + ).toBe(false); + }); + + it("reads threadId from notification params", () => { + expect(notificationThreadId({ threadId: "abc" })).toBe("abc"); + expect(notificationThreadId({ thread: { id: "def" } })).toBe("def"); + expect(notificationThreadId({})).toBeUndefined(); + }); + + it("does not install on the primary window", () => { + const primary = new FakeOverlayWindow({ href: "app://-/index.html" }); + const hook = installOverlayThreadIsolation(primary); + expect(hook.status).toBe("disabled"); + expect( + primary.dispatch({ + type: "mcp-notification", + method: "thread/started", + params: { thread: { id: "cli" } }, + }), + ).toBe(false); + }); + + it("stops CLI thread/started on leftover overlay and allows the active realtime Thread", () => { + const overlay = new FakeOverlayWindow({ href: overlayHref }); + const hook = installOverlayThreadIsolation(overlay); + expect(hook.status).toBe("active"); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "thread/started", + params: { thread: { id: "cli-thread" } }, + }), + ).toBe(true); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "thread/realtime/started", + params: { threadId: "voice-thread" }, + }), + ).toBe(false); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "item/completed", + params: { threadId: "voice-thread" }, + }), + ).toBe(false); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "item/completed", + params: { threadId: "cli-thread" }, + }), + ).toBe(true); + overlay.dispatch({ + type: "mcp-notification", + method: "thread/realtime/closed", + params: { threadId: "voice-thread" }, + }); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "item/completed", + params: { threadId: "voice-thread" }, + }), + ).toBe(false); + hook.dispose(); + }); + + it("keeps the overlay open for an in-flight Turn after hangup until it completes", () => { + const overlay = new FakeOverlayWindow({ href: overlayHref }); + const hook = installOverlayThreadIsolation(overlay); + overlay.dispatch({ + type: "mcp-notification", + method: "thread/realtime/started", + params: { threadId: "voice-thread" }, + }); + overlay.dispatch({ + type: "mcp-notification", + method: "turn/started", + params: { threadId: "voice-thread", turn: { id: "turn-3" } }, + }); + overlay.dispatch({ + type: "mcp-notification", + method: "item/started", + params: { threadId: "voice-thread", turnId: "turn-3" }, + }); + overlay.dispatch({ + type: "mcp-notification", + method: "thread/realtime/closed", + params: { threadId: "voice-thread" }, + }); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "item/completed", + params: { threadId: "voice-thread", turnId: "turn-3" }, + }), + ).toBe(false); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "turn/completed", + params: { threadId: "voice-thread", turn: { id: "turn-3" } }, + }), + ).toBe(false); + expect( + overlay.dispatch({ + type: "mcp-notification", + method: "item/completed", + params: { threadId: "voice-thread", turnId: "turn-3" }, + }), + ).toBe(false); + hook.dispose(); + }); + + it("delivers final Turn and Item events when idle arrives before completion after hangup", () => { + const overlay = new FakeOverlayWindow({ href: overlayHref }); + const hook = installOverlayThreadIsolation(overlay); + const dispatch = (method: string, params: Record) => + overlay.dispatch({ type: "mcp-notification", method, params }); + dispatch("thread/realtime/started", { threadId: "voice-thread" }); + dispatch("turn/started", { threadId: "voice-thread", turn: { id: "native-turn" } }); + dispatch("thread/realtime/closed", { threadId: "voice-thread" }); + dispatch("thread/status/changed", { threadId: "voice-thread", status: { type: "idle" } }); + + expect(dispatch("item/completed", { threadId: "voice-thread", turnId: "native-turn" })).toBe( + false, + ); + expect( + dispatch("turn/completed", { threadId: "voice-thread", turn: { id: "native-turn" } }), + ).toBe(false); + expect( + dispatch("turn/started", { threadId: "unrelated-thread", turn: { id: "unrelated-turn" } }), + ).toBe(true); + hook.dispose(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-settings-lifecycle.test.ts b/packages/renderer-extension/test/renderer-settings-lifecycle.test.ts index 462619bb7..9303e533c 100644 --- a/packages/renderer-extension/test/renderer-settings-lifecycle.test.ts +++ b/packages/renderer-extension/test/renderer-settings-lifecycle.test.ts @@ -70,6 +70,18 @@ describe("Renderer Settings lifecycle", () => { vi.clearAllMocks(); }); + it("starts in Desktop's rendered language rather than the system language", () => { + const ownerWindow = { + navigator: { languages: ["zh-TW", "en-US"] }, + document: { documentElement: { lang: "en-US" } }, + setTimeout, + clearTimeout, + } as unknown as Window; + const lifecycle = installRendererSettingsLifecycle(ownerWindow); + expect(lifecycle.locale).toBe("en"); + lifecycle.dispose(); + }); + it("notifies consumers when the resolved Codex locale changes", async () => { const onLocaleChange = vi.fn(); const ownerWindow = { diff --git a/packages/renderer-extension/test/renderer-voice-assets.test.ts b/packages/renderer-extension/test/renderer-voice-assets.test.ts new file mode 100644 index 000000000..7958d6cdc --- /dev/null +++ b/packages/renderer-extension/test/renderer-voice-assets.test.ts @@ -0,0 +1,84 @@ +import { describe, expect, it } from "vitest"; +import { CLIENT_OWNED_VOICE_ASSETS } from "../src/renderer-client-owned-voice.js"; +import { selectVoiceAssets, supportsVoiceVersion } from "../src/renderer-voice-assets.js"; + +const latest = ["app-initial-feb30b196d11.js", "app-shared-2cae01273bae.js"] as const; +const windows = ["app-initial-438d1df57f70.js", "app-shared-955d52676f9c.js"] as const; +const previous = ["app-initial-dddef47d6ef6.js", "app-shared-fc3316d48427.js"] as const; +const baseline = ["app-initial-c95ddb62696e.js", "app-shared-2eda13eb7044.js"] as const; +const urls = (assets: readonly string[]) => assets.map((asset) => `app://-/assets/${asset}`); + +describe("audited voice asset ownership", () => { + it("requires platform metadata for an empty same-version asset set", () => { + expect(selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, [], "26.1021.31345")).toBeUndefined(); + expect( + selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, [], "26.1021.31345", "win32"), + ).toMatchObject({ initial: windows[0] }); + }); + + it("rejects mixed platform assets and contradictory platform metadata", () => { + for (const names of [[latest[0], windows[1]], [...latest, ...windows], [windows[0]]]) + expect( + selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, urls(names), "26.1021.31345", "win32"), + ).toBeUndefined(); + expect( + selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, urls(latest), "26.1021.31345", "win32"), + ).toBeUndefined(); + }); + + it.each([ + ["26.1021.31345", latest, "avatar-overlay-native-page-d549db3706f1.js"], + ["26.1021.31345", windows, "avatar-overlay-native-page-1549fe5955e1.js"], + ["26.1016.62342", previous, "avatar-overlay-native-page-ef8332c6a2fb.js"], + ["26.1016.61520", baseline, "avatar-overlay-native-page-fcc17283519a.js"], + ])( + "selects the matching %s overlay with or without version metadata", + (version, assets, overlay) => { + for (const metadata of [version, undefined]) + expect(selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, urls(assets), metadata)).toMatchObject({ + version, + overlay, + }); + }, + ); + + it.each([ + ["26.1021.31345", previous], + ["26.1016.62342", latest], + ["26.1016.62342", baseline], + ["26.1016.61520", latest], + ["unknown", latest], + [undefined, [latest[0], previous[1]]], + [undefined, [...latest, ...previous]], + ["26.1021.31345", [...latest, previous[0]]], + [undefined, [latest[0]]], + ])("rejects incomplete, mixed, ambiguous or mismatched bindings (%s, %j)", (version, assets) => { + expect(selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, urls(assets), version)).toBeUndefined(); + }); + + it("selects the version pin when resource names are empty", () => { + expect( + selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, [], "26.1021.31345", "darwin"), + ).toMatchObject({ + version: "26.1021.31345", + initial: "app-initial-feb30b196d11.js", + }); + expect(selectVoiceAssets(CLIENT_OWNED_VOICE_ASSETS, [], undefined)).toBeUndefined(); + }); + + it("rejects lookalike assets outside the native origin", () => { + expect( + selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + latest.map((asset) => `https://example.com/assets/${asset}`), + ), + ).toBeUndefined(); + expect( + selectVoiceAssets( + CLIENT_OWNED_VOICE_ASSETS, + latest.map((asset) => `app://other/assets/${asset}`), + ), + ).toBeUndefined(); + expect(supportsVoiceVersion(CLIENT_OWNED_VOICE_ASSETS, "26.1021.31346")).toBe(false); + }); +}); diff --git a/packages/renderer-extension/test/renderer-voice-capability.test.ts b/packages/renderer-extension/test/renderer-voice-capability.test.ts new file mode 100644 index 000000000..1ffaefdf3 --- /dev/null +++ b/packages/renderer-extension/test/renderer-voice-capability.test.ts @@ -0,0 +1,660 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import { + bindComposerVoiceCapability, + bindVoiceCapabilitySubscriptions, + installVoiceCapabilityProjection, + replayVoiceCapability, +} from "../src/renderer-voice-capability.js"; +import { + applyVoiceGatewayReadiness, + applySharedVoiceRoutes, +} from "../src/renderer-cpa-live-voice-preference.js"; +import { CLIENT_OWNED_VOICE_ASSETS } from "../src/renderer-client-owned-voice.js"; + +type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: unknown }; +function fixture(name = "voice.realtime", hostId?: string, autoCommit = true) { + const gate: Atom = { read: () => true, write: true }; + let permitted = true; + let loading = false; + let error = false; + const atom: Atom = { + read: (get) => ({ + isLoading: loading, + isError: error, + isCapable: permitted && get(gate) === false, + }), + }; + const store = { + get: (atom: Atom): unknown => atom.read(store.get), + sub: vi.fn((atom: Atom, listener: () => void) => { + void atom; + void listener; + return () => {}; + }), + }; + const signal = { atom, store }; + const original = () => store.get(atom); + const subscriber = { + getSnapshot: original, + subscribe: (listener: () => void) => store.sub(atom, listener), + createRender: () => undefined, + }; + const instance = { getSnapshot: original }; + const notify = vi.fn(() => { + if (autoCommit) subscription.next.memoizedState = subscriber.getSnapshot(); + }); + const subscription = { + memoizedState: [subscriber, [signal, undefined]], + next: { + memoizedState: original(), + queue: instance, + next: { + memoizedState: { create: () => subscriber.subscribe(notify), deps: [subscriber.subscribe] }, + next: null, + }, + }, + }; + const family = { scope: { id: Symbol() }, resolve: () => ({ atom: gate }) }; + const key = JSON.stringify({ name, ...(hostId === undefined ? {} : { hostId }) }); + const node = { + store, + familyBindings: new Map([[family, new Map([[key, { cacheKey: "owned" }]])]]), + }; + const chain = new Map([[family.scope.id, node]]); + const owner = { + memoizedState: { + memoizedState: [signal, ["owned", chain, undefined, family, undefined]], + next: subscription, + }, + return: null, + }; + const composer = { + __reactFiber$fixture: owner, + querySelectorAll: () => [], + getBoundingClientRect: () => ({ width: 100 }), + closest: () => null, + } as unknown as Element; + return { + atom, + gate, + store, + family, + subscriber, + instance, + notify, + original, + owner, + composer, + block: () => { + permitted = false; + }, + load: () => { + loading = true; + }, + fail: () => { + error = true; + }, + }; +} +afterEach(() => vi.useRealTimers()); + +describe("voice capability evaluation", () => { + it("omits only the inference constraint while leaving native gates and other eligibility unchanged", () => { + const f = fixture(); + expect(replayVoiceCapability(f.store, f.atom, f.gate)).toEqual({ + isLoading: false, + isError: false, + isCapable: true, + }); + expect(f.store.get(f.gate)).toBe(true); + expect(f.store.get(f.atom)).toMatchObject({ isCapable: false }); + expect(JSON.stringify(f.store)).not.toMatch(/model_provider/u); + f.block(); + expect(replayVoiceCapability(f.store, f.atom, f.gate)).toBeNull(); + }); + it.each(["load", "fail"] as const)("fails closed on native %s", (flag) => { + const f = fixture(); + f[flag](); + expect(replayVoiceCapability(f.store, f.atom, f.gate)).toBeNull(); + }); + it("rejects a changed replay and an unrelated gate", () => { + const f = fixture(); + expect(replayVoiceCapability(f.store, f.atom, { read: () => false })).toBeNull(); + let calls = 0; + const atom = { + read: () => ({ isLoading: false, isError: false, isCapable: ++calls % 2 === 0 }), + }; + expect(replayVoiceCapability(f.store, atom, f.gate)).toBeNull(); + }); + it("bounds recursive dependencies", () => { + const f = fixture(); + const atom: Atom = { read: (get) => get({ read: atom.read }) }; + expect( + replayVoiceCapability( + { ...f.store, get: () => ({ isLoading: false, isError: false, isCapable: false }) }, + atom, + f.gate, + ), + ).toBeNull(); + }); +}); + +describe("owned composer subscription", () => { + it("requires one committed realtime reader for the native overlay starter", () => { + const f = fixture("voice.realtime", undefined, false); + const binding = bindVoiceCapabilitySubscriptions( + () => new Set([f.owner]), + f.family, + f.family, + () => true, + true, + ); + expect(binding.current()).toBe(false); + Object.assign(f.owner.memoizedState.next.next, { memoizedState: f.instance.getSnapshot() }); + expect(binding.current()).toBe(true); + binding.dispose(); + expect(() => + bindVoiceCapabilitySubscriptions( + () => new Set([f.owner, { ...f.owner }]), + f.family, + f.family, + () => true, + true, + ), + ).toThrow(); + expect(f.instance.getSnapshot).toBe(f.original); + }); + it.each([ + ["voice.realtime", undefined], + ["voice.thread", "local"], + ])("projects %s and restores on preference change and disposal", (name, host) => { + const f = fixture(name, host); + let allowed = true; + const binding = bindComposerVoiceCapability(f.composer, f.family, f.family, () => allowed); + expect(f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + expect(f.instance.getSnapshot()).toBe(f.subscriber.getSnapshot()); + expect(f.notify).toHaveBeenCalledOnce(); + expect(f.store.sub).not.toHaveBeenCalled(); + expect(binding.current()).toBe(true); + allowed = false; + expect(f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + binding.dispose(); + expect(f.subscriber.getSnapshot).toBe(f.original); + expect(f.instance.getSnapshot).toBe(f.original); + expect(f.notify).toHaveBeenCalledTimes(2); + }); + it.each([ + ["chatgpt.image-generation", undefined], + ["voice", undefined], + ["voice.thread", "remote"], + ["voice.thread", undefined], + ])("rejects %s for host %s", (name, host) => { + const f = fixture(name, host); + expect(() => bindComposerVoiceCapability(f.composer, f.family, f.family, () => true)).toThrow(); + expect(f.subscriber.getSnapshot).toBe(f.original); + }); + it("detects replacement of a committed subscription", () => { + const f = fixture(); + const binding = bindComposerVoiceCapability(f.composer, f.family, f.family, () => true); + f.owner.memoizedState.next = null as unknown as typeof f.owner.memoizedState.next; + expect(binding.current()).toBe(false); + binding.dispose(); + }); + + it("accepts React's lazy snapshot wrapper but detects a restored native instance reader", () => { + const f = fixture(); + const binding = bindComposerVoiceCapability(f.composer, f.family, f.family, () => true); + f.instance.getSnapshot = () => f.subscriber.getSnapshot(); + expect(binding.current()).toBe(true); + f.instance.getSnapshot = f.original; + expect(binding.current()).toBe(false); + binding.dispose(); + expect(f.instance.getSnapshot).toBe(f.original); + expect(f.subscriber.getSnapshot).toBe(f.original); + }); + + it.each([ + { isTemporaryChat: true }, + { shouldBlockExternalEgress: true }, + { readOnly: true }, + { composerSelectionContext: { conversationId: "another-chat" } }, + ])("rejects ineligible Chat owner projection: %j", (overrides) => { + const f = fixture(); + Object.assign(f.owner, { + memoizedProps: { + isWorkConversation: false, + isTemporaryChat: false, + shouldBlockExternalEgress: false, + conversationId: "chat-fixture", + composerSelectionContext: { conversationId: "chat-fixture" }, + ...overrides, + }, + }); + Object.assign(f.composer, { __reactFiber$fixture: { return: f.owner, memoizedState: null } }); + expect(() => + bindComposerVoiceCapability(f.composer, f.family, f.family, () => true, true), + ).toThrow(); + expect(f.subscriber.getSnapshot).toBe(f.original); + }); + + it("does not project a capability on an ancestor outside the Composer", () => { + const f = fixture(); + Object.assign(f.composer, { __reactFiber$fixture: { return: f.owner, memoizedState: null } }); + expect(() => bindComposerVoiceCapability(f.composer, f.family, f.family, () => true)).toThrow(); + expect(f.subscriber.getSnapshot).toBe(f.original); + }); +}); + +describe("version and runtime qualification", () => { + function target(assets = CLIENT_OWNED_VOICE_ASSETS[0]) { + if (!assets) throw new Error("Missing fixture profile"); + const f = fixture(); + const values = new Map([["home-composer-mode-v1", "work"]]); + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: assets.version, + desktopPlatform: assets.platform, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + __codexhostClientOwnedVoiceV1: { status: "active" }, + __codexhostExternalVoiceLaunchV1: { status: "active" }, + document: { + querySelectorAll: (selector: string) => + selector === "[data-codex-composer-root]" ? [f.composer] : [], + } as unknown as Document, + performance: { + getEntriesByType: () => + [assets.initial, assets.shared].map((x) => ({ + name: `app://-/assets/${x}`, + })) as PerformanceEntry[], + }, + localStorage: { getItem: (key: string) => values.get(key) ?? null }, + setTimeout, + clearTimeout, + }; + const route = { hostId: "local", policy: { selectedModel: () => "codex-model" } }; + Object.assign(target, { + __codexhostHostRoutingV1: { forComposer: () => route, forHost: () => route }, + }); + applyVoiceGatewayReadiness(target, true); + return { + f, + target, + values, + importModule: vi.fn(async () => ({ + [assets.capabilityInit]: () => {}, + [assets.capabilityFamily]: f.family, + [assets.inferenceGateFamily]: f.family, + })), + }; + } + it("uses the audited 26.1021 capability and inference families despite reused old exports", async () => { + vi.useFakeTimers(); + const f = target(); + const unrelated = { scope: f.f.family.scope, resolve: () => ({ atom: { read: () => false } }) }; + const initialize = vi.fn(); + const importModule = async () => ({ + l0n: initialize, + c0n: f.f.family, + Cnr: f.f.family, + njr: 30_000, + tjr: unrelated, + uBr: unrelated, + }); + const hook = installVoiceCapabilityProjection({ ...f, importModule }); + try { + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + expect(initialize).toHaveBeenCalledOnce(); + } finally { + hook.dispose(); + } + expect(f.f.instance.getSnapshot).toBe(f.f.original); + }); + it.each(CLIENT_OWNED_VOICE_ASSETS)( + "projects the audited $version subscription", + async (assets) => { + vi.useFakeTimers(); + const f = target(assets); + const hook = installVoiceCapabilityProjection(f); + try { + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + } finally { + hook.dispose(); + } + }, + ); + it("reports native ineligibility when provider-only replay cannot make the capability capable", async () => { + vi.useFakeTimers(); + const f = target(); + f.f.block(); + const hook = installVoiceCapabilityProjection(f); + try { + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("native-ineligible"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + expect(f.f.instance.getSnapshot).not.toBe(f.f.original); + const wrapped = f.f.instance.getSnapshot; + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("native-ineligible"); + expect(f.f.instance.getSnapshot).toBe(wrapped); + } finally { + hook.dispose(); + } + }); + it("rejects missing audited exports without probing other scope families", async () => { + vi.useFakeTimers(); + const f = target(); + const resolve = vi.spyOn(f.f.family, "resolve"); + const hook = installVoiceCapabilityProjection({ ...f, importModule: async () => ({}) }); + try { + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("unsupported"); + expect(resolve).not.toHaveBeenCalled(); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + } finally { + hook.dispose(); + } + }); + it("requires injection, gateway readiness, Work ownership and both active hooks", async () => { + vi.useFakeTimers(); + const f = target(); + applyVoiceGatewayReadiness(f.target, false); + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(f.importModule).not.toHaveBeenCalled(); + applyVoiceGatewayReadiness(f.target, true); + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + await vi.advanceTimersByTimeAsync(2000); + expect(f.f.notify).toHaveBeenCalledOnce(); + f.values.set("home-composer-mode-v1", "chat"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + await vi.advanceTimersByTimeAsync(1000); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + hook.dispose(); + }); + it("recovers retained Work subscriptions when a new bundle applies gateway readiness and routes", async () => { + vi.useFakeTimers(); + const f = target(); + applySharedVoiceRoutes(f.target, { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "gateway", + }); + const hook = installVoiceCapabilityProjection(f); + try { + await vi.advanceTimersByTimeAsync(1); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + applyVoiceGatewayReadiness(f.target, false); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + + vi.resetModules(); + const next = await import("../src/renderer-cpa-live-voice-preference.js"); + next.applyVoiceGatewayReadiness(f.target, true); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + next.applySharedVoiceRoutes(f.target, { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", + }); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + next.applySharedVoiceRoutes(f.target, { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "gateway", + }); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + } finally { + hook.dispose(); + } + }); + + it("restores the committed Work capability after readiness recovers without remounting the composer", async () => { + vi.useFakeTimers(); + const f = target(); + const hook = installVoiceCapabilityProjection(f); + try { + await vi.advanceTimersByTimeAsync(1); + const committed = f.f.owner.memoizedState.next.next; + expect(committed.memoizedState).toMatchObject({ isCapable: true }); + // A native notification while the Host is reconnecting commits the fallback. + // Recovery can arrive before the next poll, with no native atom change. + applyVoiceGatewayReadiness(f.target, false); + f.f.notify(); + expect(committed.memoizedState).toMatchObject({ isCapable: false }); + applyVoiceGatewayReadiness(f.target, true); + expect(f.f.subscriber.getSnapshot()).toMatchObject({ isCapable: true }); + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("active"); + expect(committed.memoizedState).toMatchObject({ isCapable: true }); + } finally { + hook.dispose(); + } + }); + + it("projects official Codex Work when the Work tab is selected and composer-mode storage is empty", async () => { + vi.useFakeTimers(); + const f = target(); + f.values.delete("home-composer-mode-v1"); + const work = { + textContent: "Work", + getAttribute: (name: string) => (name === "aria-selected" ? "true" : ""), + }; + const chat = { + textContent: "Chat", + getAttribute: (name: string) => (name === "aria-selected" ? "false" : ""), + }; + const document = f.target.document as unknown as { + querySelectorAll: (selector: string) => unknown; + }; + const composers = document.querySelectorAll; + document.querySelectorAll = (selector: string) => { + if (selector === "[data-codex-composer-root]") return composers.call(document, selector); + if (selector === "[data-chatgpt-composer]") return []; + return [chat, work]; + }; + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + hook.dispose(); + }); + + it("keeps Work Desktop default native, but permits an explicitly owned local external Harness", async () => { + vi.useFakeTimers(); + const f = target(); + applySharedVoiceRoutes(f.target, { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", + }); + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(f.importModule).not.toHaveBeenCalled(); + const route = { hostId: "local", policy: { selectedModel: () => "codexhost/grok/model" } }; + Object.assign(f.target, { + __codexhostHostRoutingV1: { forComposer: () => route, forHost: () => route }, + }); + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("active"); + hook.dispose(); + }); + + it("keeps the visible Work projection when Desktop retains hidden Chat composers", async () => { + vi.useFakeTimers(); + const f = target(); + f.values.delete("home-composer-mode-v1"); + const hiddenChat = { + getBoundingClientRect: () => ({ width: 0 }), + closest: () => null, + }; + let chatVisible = false; + f.target.document.querySelectorAll = ((selector: string) => + selector === "[data-codex-composer-root]" + ? [f.f.composer] + : selector === "[data-chatgpt-composer]" + ? [{ ...hiddenChat, getBoundingClientRect: () => ({ width: chatVisible ? 800 : 0 }) }] + : []) as unknown as Document["querySelectorAll"]; + const hook = installVoiceCapabilityProjection(f); + try { + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + chatVisible = true; + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("pending"); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + chatVisible = false; + await vi.advanceTimersByTimeAsync(1000); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + } finally { + hook.dispose(); + } + }); + it("projects the Chat composer only for Codex gateway, without claiming a Work route", async () => { + vi.useFakeTimers(); + const f = target(); + f.values.set("home-composer-mode-v1", "chat"); + Object.assign(f.f.owner, { + memoizedProps: { + isWorkConversation: false, + isTemporaryChat: false, + shouldBlockExternalEgress: false, + conversationId: "chat-fixture", + composerSelectionContext: { conversationId: "chat-fixture" }, + }, + }); + Object.assign(f.f.composer, { + __reactFiber$fixture: { return: f.f.owner, memoizedState: null }, + }); + Reflect.deleteProperty(f.target, "__codexhostHostRoutingV1"); + f.target.document.querySelectorAll = ((selector: string) => + selector === "[data-chatgpt-composer]" + ? [f.f.composer] + : []) as unknown as Document["querySelectorAll"]; + applySharedVoiceRoutes(f.target, { + chatExperience: "codex", + chatConnection: "gateway", + workConnection: "desktop", + }); + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + for (const [chatExperience, chatConnection] of [ + ["codex", "desktop"], + ["chatgpt", "gateway"], + ] as const) { + applySharedVoiceRoutes(f.target, { + chatExperience, + chatConnection, + workConnection: "gateway", + }); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + await vi.advanceTimersByTimeAsync(1000); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + } + hook.dispose(); + }); + it("binds the visible Work composer when Desktop retains a hidden composer root", async () => { + vi.useFakeTimers(); + const f = target(); + const hidden = { ...f.f.composer, getBoundingClientRect: () => ({ width: 0 }) }; + f.target.document.querySelectorAll = ((selector: string) => + selector === "[data-codex-composer-root]" + ? [hidden, f.f.composer] + : []) as unknown as Document["querySelectorAll"]; + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + hook.dispose(); + }); + it("leaves native capability unchanged when two visible composers are ambiguous", async () => { + vi.useFakeTimers(); + const f = target(); + f.target.document.querySelectorAll = ((selector: string) => + selector === "[data-codex-composer-root]" + ? [f.f.composer, fixture().composer] + : []) as unknown as Document["querySelectorAll"]; + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("unsupported"); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + hook.dispose(); + }); + it.each(["fetch", "launch", "injection", "gateway"])( + "restores native capability immediately when %s becomes unavailable", + async (kind) => { + vi.useFakeTimers(); + const f = target(); + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: true }); + if (kind === "fetch") f.target.__codexhostClientOwnedVoiceV1.status = "unsupported"; + if (kind === "launch") f.target.__codexhostExternalVoiceLaunchV1.status = "unsupported"; + if (kind === "injection") f.target.__codexhostCpaLiveVoiceHookV1.enabled = false; + if (kind === "gateway") applyVoiceGatewayReadiness(f.target, false); + expect(f.f.instance.getSnapshot()).toMatchObject({ isCapable: false }); + await vi.advanceTimersByTimeAsync(1000); + expect(f.f.instance.getSnapshot).toBe(f.f.original); + hook.dispose(); + }, + ); + + it("fails closed on an unaudited Desktop version", () => { + const f = target(); + f.target.__codexhostCpaLiveVoiceHookV1.desktopVersion = "unknown"; + expect(installVoiceCapabilityProjection(f).status).toBe("unsupported"); + }); + + it("uses the version pin when resource timing is empty", async () => { + vi.useFakeTimers(); + const f = target(); + f.target.performance.getEntriesByType = () => []; + const hook = installVoiceCapabilityProjection(f); + await vi.advanceTimersByTimeAsync(1); + expect(hook.status).toBe("active"); + expect(f.importModule).toHaveBeenCalled(); + hook.dispose(); + }); +}); + +describe("Settings-only dictation capability readers", () => { + it.each(["dictation.composer", "dictation.global"])( + "requires an explicit Settings selection for %s", + (name) => { + const f = fixture(name); + expect(() => + bindVoiceCapabilitySubscriptions( + () => new Set([f.owner]), + f.family, + f.family, + () => true, + ), + ).toThrow(); + const binding = bindVoiceCapabilitySubscriptions( + () => new Set([f.owner]), + f.family, + f.family, + () => true, + false, + ["dictation.composer", "dictation.global"], + ); + expect(f.subscriber.getSnapshot()).toMatchObject({ isCapable: true }); + expect(f.store.get(f.atom)).toMatchObject({ isCapable: false }); + f.block(); + expect(f.subscriber.getSnapshot()).toMatchObject({ isCapable: false }); + binding.dispose(); + expect(f.subscriber.getSnapshot).toBe(f.original); + }, + ); +}); diff --git a/packages/renderer-extension/test/renderer-voice-settings-availability.test.ts b/packages/renderer-extension/test/renderer-voice-settings-availability.test.ts new file mode 100644 index 000000000..ec1d86ad3 --- /dev/null +++ b/packages/renderer-extension/test/renderer-voice-settings-availability.test.ts @@ -0,0 +1,196 @@ +import { describe, expect, it, vi } from "vitest"; +import { + bindVoiceSettingsAvailability, + voiceSettingsSurface, +} from "../src/renderer-voice-settings-availability.js"; + +type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: unknown }; +function fixture() { + let access = true; + let loading = false; + let allowed = true; + const gate: Atom = { read: () => true, write: true }; + const voice: Atom = { + read: (get) => ({ isLoading: loading, isError: false, isCapable: access && !get(gate) }), + }; + const boolean: Atom = { read: (get) => (get(voice) as { isCapable: boolean }).isCapable }; + const unrelated: Atom = { read: () => false }; + const store = { + get: (atom: Atom): unknown => atom.read(store.get), + sub: vi.fn((atom: Atom, notify: () => void) => { + void atom; + void notify; + return () => {}; + }), + }; + function subscription(atom: Atom) { + const original = () => store.get(atom); + const subscriber = { + getSnapshot: original, + subscribe: (notify: () => void) => store.sub(atom, notify), + createRender: () => undefined, + }; + const instance = { getSnapshot: () => subscriber.getSnapshot() }; + const notify = vi.fn(() => { + hook.next.memoizedState = subscriber.getSnapshot(); + }); + const hook = { + memoizedState: [subscriber, [store, atom]], + next: { + memoizedState: original(), + queue: instance, + next: { + memoizedState: { + create: () => subscriber.subscribe(notify), + deps: [subscriber.subscribe], + }, + next: null, + }, + }, + }; + return { subscriber, instance, hook, notify, original }; + } + const owned = subscription(boolean); + const other = subscription(unrelated); + const owner = { memoizedState: owned.hook }; + let fibers = new Set>([owner, { memoizedState: other.hook }]); + return { + store, + gate, + voice, + owned, + other, + owner, + bind: () => + bindVoiceSettingsAvailability( + () => fibers, + store, + voice, + gate, + () => allowed, + ), + deny: () => { + access = false; + }, + load: () => { + loading = true; + }, + disable: () => { + allowed = false; + }, + unmount: () => { + fibers = new Set(); + }, + duplicate: () => { + fibers.add({ memoizedState: subscription(boolean).hook }); + }, + }; +} + +describe("owned native Voice Settings availability", () => { + it("restores only the boolean reader derived from realtime voice without writing native gates or capability", () => { + const f = fixture(); + const binding = f.bind(); + expect(f.owned.subscriber.getSnapshot()).toBe(true); + expect(f.other.subscriber.getSnapshot()).toBe(false); + expect(f.store.get(f.voice)).toMatchObject({ isCapable: false }); + expect(f.store.get(f.gate)).toBe(true); + expect(f.store.sub).not.toHaveBeenCalled(); + expect(binding.current()).toBe(true); + binding.dispose(); + expect(f.owned.subscriber.getSnapshot).toBe(f.owned.original); + expect(f.owned.instance.getSnapshot()).toBe(false); + expect(f.owned.notify).toHaveBeenCalledTimes(2); + }); + it.each(["deny", "load", "disable"] as const)("preserves native %s after binding", (change) => { + const f = fixture(); + const binding = f.bind(); + f[change](); + expect(f.owned.subscriber.getSnapshot()).toBe(false); + expect(binding.current()).toBe(false); + binding.dispose(); + }); + it("detects a lost owner and accepts an equivalent React instance wrapper", () => { + const f = fixture(); + const binding = f.bind(); + f.owned.instance.getSnapshot = () => f.owned.subscriber.getSnapshot(); + expect(binding.current()).toBe(true); + f.unmount(); + expect(binding.current()).toBe(false); + binding.dispose(); + }); + it("rebinds while React still holds the previous projected commit after disposal", () => { + const f = fixture(); + const binding = f.bind(); + binding.dispose(); + f.owned.hook.next.memoizedState = true; + const rebound = f.bind(); + expect(rebound.current()).toBe(true); + rebound.dispose(); + }); + + it("rejects duplicate availability readers before mutating either", () => { + const f = fixture(); + f.duplicate(); + expect(f.bind).toThrow(/ambiguous/i); + expect(f.owned.subscriber.getSnapshot).toBe(f.owned.original); + expect(f.owned.notify).not.toHaveBeenCalled(); + }); + it("rejects a writable boolean, an invalid replay and a malformed subscription", () => { + for (const mutate of [ + (f: ReturnType) => { + const deps = f.owned.hook.memoizedState[1] as unknown[]; + (deps[1] as Atom).write = true; + }, + (f: ReturnType) => { + f.owned.subscriber.getSnapshot = () => true; + }, + (f: ReturnType) => { + f.owned.hook.next.next.memoizedState.deps = []; + }, + ]) { + const f = fixture(); + mutate(f); + expect(f.bind).toThrow(); + expect(f.owned.notify).not.toHaveBeenCalled(); + } + }); +}); + +describe("Voice Settings surface ownership", () => { + it("uses only one exported page and its AppScope, excluding ancestors and other pages", () => { + const appScope = { id: Symbol() }; + const node = {}; + const scope = { + scope: appScope, + node, + chain: new Map([[appScope.id, node]]), + get: () => undefined, + }; + const page = () => {}; + const ancestor = { memoizedState: null, return: null }; + const owner = { + type: page, + memoizedState: { memoizedState: { current: scope }, next: null }, + return: ancestor, + }; + const group = { memoizedState: null, return: owner }; + const section = { __reactFiber$fixture: { return: group } }; + const doc = { querySelectorAll: () => [section] } as unknown as Document; + const surface = voiceSettingsSurface(doc, page, appScope); + expect(surface?.scope).toBe(scope); + expect(surface?.fibers.has(group)).toBe(true); + expect(surface?.fibers.has(ancestor)).toBe(false); + owner.type = () => {}; + expect(voiceSettingsSurface(doc, page, appScope)).toBeUndefined(); + owner.type = page; + const second = { __reactFiber$fixture: { return: { ...owner } } }; + expect( + voiceSettingsSurface( + { querySelectorAll: () => [section, second] } as unknown as Document, + page, + appScope, + ), + ).toBeUndefined(); + }); +}); diff --git a/packages/renderer-extension/test/renderer-voice-settings.test.ts b/packages/renderer-extension/test/renderer-voice-settings.test.ts new file mode 100644 index 000000000..5850b6d8b --- /dev/null +++ b/packages/renderer-extension/test/renderer-voice-settings.test.ts @@ -0,0 +1,477 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import { + explainVoiceSettingsRestriction, + installVoiceSettingsExplanation, + providerBlocksVoiceSettings, + restoreNativeVoiceSettings, +} from "../src/renderer-voice-settings.js"; + +type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: unknown }; +function fixture() { + const appScope = { id: Symbol() }; + const page = () => {}; + const access = {}; + let status = "allowed"; + let rootCapable = true; + let otherRestriction = false; + let loading = false; + let error = false; + const gate: Atom = { read: () => true, write: true }; + const atom: Atom = { + read: (get) => ({ + isLoading: loading, + isError: error, + isCapable: !otherRestriction && get(gate) === false, + }), + }; + const store = { + get: (atom: Atom): unknown => atom.read(store.get), + sub: vi.fn((atom: Atom, listener: () => void) => { + void atom; + void listener; + return () => {}; + }), + }; + const capability = { resolve: () => ({ resolve: () => atom }) }; + const gates = { resolve: () => ({ atom: gate }) }; + const modules = { appScope, page, access, capability, gates }; + const node = { store }; + const scope = { + scope: appScope, + node, + chain: new Map([[appScope.id, node]]), + get: (selector: unknown, arg?: unknown) => + selector === access + ? { status } + : (arg as { name: string })?.name === "voice" + ? { isLoading: false, isError: false, isCapable: rootCapable } + : store.get(atom), + }; + const nativeLabel = "Voice chat unavailable"; + const nativeDescription = "Your account or workspace does not have access to voice chat"; + const textElement = (value: string) => { + let data = value; + const text = { + nodeType: 3, + get data() { + return data; + }, + set data(value: string) { + data = value; + }, + }; + return { + isConnected: true, + firstChild: text, + childNodes: [text], + get textContent() { + return text.data; + }, + }; + }; + const label = textElement(nativeLabel); + const description = textElement(nativeDescription); + const host = { + contains: (el: unknown) => el === description || el === label, + get textContent() { + return label.textContent + description.textContent; + }, + querySelectorAll: () => [label, description], + }; + const pageFiber = { + type: page, + memoizedState: { memoizedState: { current: scope }, next: null }, + return: null, + }; + const row = { + memoizedProps: { + control: null, + label: { props: { defaultMessage: nativeLabel } }, + description: { + props: { + id: "settings.general.realtimeVoice.unavailable.description", + defaultMessage: nativeDescription, + }, + }, + }, + return: pageFiber, + }; + Object.assign(description, { + __reactFiber$fixture: { + tag: 5, + stateNode: description, + return: { tag: 5, stateNode: host, return: row }, + }, + }); + const doc = { + querySelectorAll: (selector: string) => (selector === "section div" ? [description] : []), + } as unknown as Document; + return { + scope, + modules, + doc, + label, + description, + nativeLabel, + nativeDescription, + store, + gate, + atom, + pageFiber, + setStatus: (s: string) => { + status = s; + }, + blockRoot: () => { + rootCapable = false; + }, + otherRestriction: () => { + otherRestriction = true; + }, + loading: () => { + loading = true; + }, + error: () => { + error = true; + }, + }; +} + +describe("voice Settings provider explanation", () => { + it("identifies an allowed account blocked only by the inference restriction without changing native state", () => { + const f = fixture(); + expect(providerBlocksVoiceSettings(f.scope, f.modules)).toBe(true); + expect(f.store.get(f.gate)).toBe(true); + expect(f.store.get(f.atom)).toMatchObject({ isCapable: false }); + expect(f.store.sub).not.toHaveBeenCalled(); + }); + it.each(["denied", "loading", "error", "unknown"])("preserves real access state %s", (status) => { + const f = fixture(); + f.setStatus(status); + expect(providerBlocksVoiceSettings(f.scope, f.modules)).toBe(false); + explainVoiceSettingsRestriction(f.doc, f.modules); + expect(f.description.textContent).toBe(f.nativeDescription); + }); + it.each(["blockRoot", "otherRestriction", "loading", "error"] as const)( + "does not misdiagnose another capability restriction: %s", + (change) => { + const f = fixture(); + f[change](); + expect(providerBlocksVoiceSettings(f.scope, f.modules)).toBe(false); + }, + ); + it("changes and restores only the native owned Settings text nodes", () => { + const f = fixture(); + const original = f.description.firstChild; + const restore = explainVoiceSettingsRestriction(f.doc, f.modules); + expect(f.description.firstChild).toBe(original); + expect(f.label.textContent).toBe("Voice settings unavailable with this provider"); + expect(f.description.textContent).toContain("Your account allows voice chat"); + expect(restore?.current()).toBe(true); + f.setStatus("denied"); + expect(restore?.current()).toBe(false); + restore?.dispose(); + expect(f.description.textContent).toBe(f.nativeDescription); + expect(f.label.textContent).toBe(f.nativeLabel); + }); + it("does not change matching transcript text or an ambiguous Settings surface", () => { + const f = fixture(); + const original = f.pageFiber.type; + f.pageFiber.type = () => {}; + explainVoiceSettingsRestriction(f.doc, f.modules); + expect(f.description.textContent).toBe(f.nativeDescription); + f.pageFiber.type = original; + const ambiguous = { + querySelectorAll: () => [f.description, f.description], + } as unknown as Document; + explainVoiceSettingsRestriction(ambiguous, f.modules); + expect(f.description.textContent).toBe(f.nativeDescription); + }); + it("does not overwrite a native text update during restoration", () => { + const f = fixture(); + const restore = explainVoiceSettingsRestriction(f.doc, f.modules); + f.description.firstChild.data = "Checking voice chat access…"; + restore?.dispose(); + expect(f.description.textContent).toBe("Checking voice chat access…"); + }); +}); + +afterEach(() => vi.useRealTimers()); +describe("voice Settings explanation lifecycle", () => { + it("waits for the complete native asset pair during document startup", async () => { + vi.useFakeTimers(); + const f = fixture(); + const names = ["app://-/assets/app-initial-feb30b196d11.js"]; + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: "26.1021.31345", + desktopPlatform: "darwin", + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: f.doc, + performance: { getEntriesByType: () => names.map((name) => ({ name })) }, + setTimeout, + clearTimeout, + } as unknown as typeof globalThis; + const importModule = vi.fn(async (url: string): Promise> => + url.includes("app-initial") + ? { l0n: () => {}, k5n: f.modules.access, c0n: f.modules.capability, Cnr: f.modules.gates } + : url.includes("app-shared") + ? { Bst: f.modules.appScope } + : { VoiceSettings: f.modules.page }, + ); + const hook = installVoiceSettingsExplanation({ target, importModule }); + try { + await vi.advanceTimersByTimeAsync(0); + expect(importModule).not.toHaveBeenCalled(); + names.push("app://-/assets/app-shared-2cae01273bae.js"); + await vi.advanceTimersByTimeAsync(1000); + expect(f.description.textContent).toContain("Your account allows voice chat"); + } finally { + hook.dispose(); + } + expect(vi.getTimerCount()).toBe(0); + }); + it("restores the native denial when account access changes, without rewriting unchanged text on each poll", async () => { + vi.useFakeTimers(); + const f = fixture(); + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: "26.1021.31345", + desktopPlatform: "darwin", + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: f.doc, + performance: { getEntriesByType: () => [] }, + setTimeout, + clearTimeout, + } as unknown as typeof globalThis; + const importModule = vi.fn(async (url: string): Promise> => { + if (url.includes("app-initial")) + return { + l0n: vi.fn(), + k5n: f.modules.access, + c0n: f.modules.capability, + Cnr: f.modules.gates, + }; + if (url.includes("app-shared")) return { Bst: f.modules.appScope }; + return { VoiceSettings: f.modules.page }; + }); + const hook = installVoiceSettingsExplanation({ target, importModule }); + try { + await vi.advanceTimersByTimeAsync(0); + expect(f.description.textContent).toContain("Your account allows voice chat"); + const setter = vi.spyOn(f.description.firstChild, "data", "set"); + await vi.advanceTimersByTimeAsync(1000); + expect(setter).not.toHaveBeenCalled(); + f.setStatus("denied"); + await vi.advanceTimersByTimeAsync(1000); + expect(f.description.textContent).toBe(f.nativeDescription); + } finally { + hook.dispose(); + } + expect(vi.getTimerCount()).toBe(0); + }); +}); + +function controlsFixture(withDictation = false) { + const f = fixture(); + const family = Object.assign(f.modules.capability, { scope: f.modules.appScope }); + const cacheKey = {}; + Object.assign(f.scope.node, { + familyBindings: new Map([ + [family, new Map([[JSON.stringify({ name: "voice.realtime" }), { cacheKey }]])], + ]), + }); + function subscription(atom: typeof f.atom, deps: unknown[]) { + const original = () => f.store.get(atom); + const subscriber = { + getSnapshot: original, + subscribe: (notify: () => void) => f.store.sub(atom, notify), + createRender: () => undefined, + }; + const instance = { getSnapshot: original }; + const hook = { + memoizedState: [subscriber, deps], + next: { + memoizedState: original(), + queue: instance, + next: { + memoizedState: { + create: () => { + f.store.sub(atom, () => { + hook.next.memoizedState = subscriber.getSnapshot(); + }); + }, + deps: [subscriber.subscribe], + }, + next: null, + }, + }, + }; + return { subscriber, instance, hook, original }; + } + const signal = { atom: f.atom, store: f.store }; + const capability = subscription(f.atom, [signal, undefined]); + const boolean = { + read: (get: (atom: typeof f.atom) => unknown) => + (get(f.atom) as { isCapable: boolean }).isCapable, + }; + const availability = subscription(boolean, [f.store, boolean]); + const scopeHook = f.pageFiber.memoizedState; + Object.assign(scopeHook, { + next: { + memoizedState: [signal, [cacheKey, f.scope.chain, undefined, family, undefined]], + next: capability.hook, + }, + }); + const group = { memoizedState: availability.hook, return: f.pageFiber }; + let dictationAllowed = true; + const dictationAtom: Atom = { + read: (get) => ({ + isLoading: false, + isError: false, + isCapable: dictationAllowed && !get(f.gate), + }), + }; + const modeAtom: Atom = { + read: (get) => ((get(dictationAtom) as { isCapable: boolean }).isCapable ? "os-global" : null), + }; + const dictation = subscription(modeAtom, [f.store, modeAtom]); + if (withDictation) { + const resolve = f.modules.capability.resolve; + Object.assign(f.modules.capability, { + resolve: (_node: unknown, _chain: unknown, arg: { name: string }) => + arg.name.startsWith("dictation") ? { resolve: () => dictationAtom } : resolve(), + }); + Object.assign(f.modules, { dictationMode: { resolve: () => modeAtom } }); + let tail = capability.hook.next.next; + for (const name of ["dictation.composer", "dictation.global"]) { + const key = {}; + const members = ( + f.scope.node as typeof f.scope.node & { + familyBindings: Map>; + } + ).familyBindings.get(family); + members?.set(JSON.stringify({ name }), { cacheKey: key }); + const signal = { atom: dictationAtom, store: f.store }; + const reader = subscription(dictationAtom, [signal, undefined]); + Object.assign(tail, { + next: { + memoizedState: [signal, [key, f.scope.chain, undefined, family, undefined]], + next: reader.hook, + }, + }); + tail = reader.hook.next.next; + } + Object.assign(availability.hook.next.next, { next: dictation.hook }); + } + const section = { __reactFiber$controls: { return: group } }; + let controlsReady = true; + const doc = { + querySelectorAll: (selector: string) => + selector === "section" ? (controlsReady ? [section] : []) : f.doc.querySelectorAll(selector), + } as unknown as Document; + return { + ...f, + doc, + capability, + availability, + dictation, + modeAtom, + dictationAtom, + denyDictation: () => { + dictationAllowed = false; + }, + group, + setControlsReady: (ready: boolean) => { + controlsReady = ready; + }, + }; +} + +describe("native Settings controls restoration", () => { + it("commits both owned readers, preserves native policy, and restores on disposal", () => { + const f = controlsFixture(); + const binding = restoreNativeVoiceSettings(f.doc, f.modules); + expect(binding?.current()).toBe(true); + expect(f.capability.subscriber.getSnapshot()).toMatchObject({ isCapable: true }); + expect(f.availability.subscriber.getSnapshot()).toBe(true); + expect(f.scope.get(f.modules.access)).toEqual({ status: "allowed" }); + expect(f.store.get(f.atom)).toMatchObject({ isCapable: false }); + f.setStatus("denied"); + expect(binding?.current()).toBe(false); + expect(f.availability.subscriber.getSnapshot()).toBe(false); + binding?.dispose(); + expect(f.capability.subscriber.getSnapshot).toBe(f.capability.original); + expect(f.availability.subscriber.getSnapshot).toBe(f.availability.original); + }); + it("restores dictation settings and native OS handler mode while preserving independent dictation policy", () => { + const f = controlsFixture(true); + const binding = restoreNativeVoiceSettings(f.doc, f.modules); + expect(binding?.current()).toBe(true); + expect(f.dictation.subscriber.getSnapshot()).toBe("os-global"); + expect(f.store.get(f.modeAtom)).toBeNull(); + expect(f.store.get(f.dictationAtom)).toMatchObject({ isCapable: false }); + f.denyDictation(); + expect(f.dictation.subscriber.getSnapshot()).toBeNull(); + expect(f.availability.subscriber.getSnapshot()).toBe(true); + binding?.dispose(); + expect(f.dictation.subscriber.getSnapshot).toBe(f.dictation.original); + }); + + it("rolls back the capability reader when the group's boolean subscription is unavailable", () => { + const f = controlsFixture(); + f.group.memoizedState = null as unknown as typeof f.group.memoizedState; + expect(() => restoreNativeVoiceSettings(f.doc, f.modules)).toThrow(); + expect(f.capability.subscriber.getSnapshot).toBe(f.capability.original); + expect(f.capability.instance.getSnapshot).toBe(f.capability.original); + }); + it.each(["denied", "loading", "error"])("never binds a genuinely %s account", (status) => { + const f = controlsFixture(); + f.setStatus(status); + expect(restoreNativeVoiceSettings(f.doc, f.modules)).toBeUndefined(); + expect(f.capability.subscriber.getSnapshot).toBe(f.capability.original); + }); +}); + +it("retries controls after passive subscription readiness while retaining the fallback explanation", async () => { + vi.useFakeTimers(); + const f = controlsFixture(); + f.setControlsReady(false); + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: "26.1021.31345", + desktopPlatform: "darwin", + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + document: f.doc, + performance: { getEntriesByType: () => [] }, + setTimeout, + clearTimeout, + } as unknown as typeof globalThis; + const importModule = async (url: string): Promise> => + url.includes("app-initial") + ? { l0n: () => {}, k5n: f.modules.access, c0n: f.modules.capability, Cnr: f.modules.gates } + : url.includes("app-shared") + ? { Bst: f.modules.appScope } + : { VoiceSettings: f.modules.page }; + const hook = installVoiceSettingsExplanation({ target, importModule }); + try { + await vi.advanceTimersByTimeAsync(0); + expect(f.description.textContent).toContain("Your account allows voice chat"); + f.setControlsReady(true); + await vi.advanceTimersByTimeAsync(1000); + expect(f.capability.subscriber.getSnapshot()).toMatchObject({ isCapable: true }); + expect(f.availability.subscriber.getSnapshot()).toBe(true); + expect(f.description.textContent).toBe(f.nativeDescription); + } finally { + hook.dispose(); + } + expect(f.capability.subscriber.getSnapshot).toBe(f.capability.original); +}); diff --git a/packages/renderer-extension/test/renderer-voice-start-capability.test.ts b/packages/renderer-extension/test/renderer-voice-start-capability.test.ts new file mode 100644 index 000000000..8e05c5281 --- /dev/null +++ b/packages/renderer-extension/test/renderer-voice-start-capability.test.ts @@ -0,0 +1,372 @@ +import { describe, expect, it, vi } from "vitest"; +import { + gatewayVoiceStartRequest, + wrapGatewayVoiceStarter, +} from "../src/renderer-voice-start-capability.js"; + +const request = { + source: "composer_button_new_thread", + productExperience: "work", + hostId: "local", +}; +function fixture() { + let projected = false; + let cancelled = false; + const dispose = vi.fn(() => { + projected = false; + }); + const prepare = vi.fn(async (allowed: () => boolean) => { + projected = allowed(); + return { dispose, current: () => true }; + }); + const start = vi.fn(async () => { + if (!projected) throw new Error("voice_disabled"); + if (cancelled) throw new Error("cancelled"); + return "existingCall"; + }); + const options = { + qualifies: () => true, + ready: async () => true, + prepare, + committed: async () => {}, + }; + return { + start, + options, + dispose, + projected: () => projected, + cancel: () => { + cancelled = true; + }, + }; +} +describe("gateway start ownership", () => { + it("qualifies an existing local Codex chat without changing its native product identity", () => { + const codex = { + source: "composer_button_existing_thread", + productExperience: "codex", + locator: { hostId: "local" }, + }; + expect(gatewayVoiceStartRequest(codex, true)).toBe(true); + expect(gatewayVoiceStartRequest(codex, false, true)).toBe(false); + expect(gatewayVoiceStartRequest({ ...codex, locator: { hostId: "remote" } }, true)).toBe(false); + expect(gatewayVoiceStartRequest({ ...codex, source: "global_hotkey_new_thread" }, true)).toBe( + false, + ); + expect(codex.productExperience).toBe("codex"); + }); + it("qualifies explicit local Work requests and preserves Chat, remote and global starts", () => { + expect(gatewayVoiceStartRequest(request, true)).toBe(true); + for (const r of [ + { ...request, productExperience: "chat" }, + { ...request, hostId: "remote" }, + { ...request, source: "global_hotkey_new_thread" }, + { ...request, source: "composer_button_existing_thread", locator: { hostId: "remote" } }, + ]) + expect(gatewayVoiceStartRequest(r, true)).toBe(false); + expect(gatewayVoiceStartRequest(request, false)).toBe(false); + expect( + gatewayVoiceStartRequest( + { ...request, activeCollaborationMode: { settings: { model: "codexhost/owned" } } }, + false, + ), + ).toBe(true); + }); + it("qualifies explicit local Codex Chat only when its gateway connection is selected", () => { + const chat = { ...request, productExperience: "chat" }; + expect(gatewayVoiceStartRequest(chat, false, true)).toBe(true); + expect(gatewayVoiceStartRequest(chat, true, false)).toBe(false); + expect(gatewayVoiceStartRequest({ ...chat, hostId: "remote" }, true, true)).toBe(false); + expect( + gatewayVoiceStartRequest({ ...chat, source: "global_hotkey_new_thread" }, true, true), + ).toBe(false); + expect( + gatewayVoiceStartRequest( + { ...chat, source: "composer_button_existing_thread", locator: { hostId: "local" } }, + false, + true, + ), + ).toBe(true); + }); + it("qualifies the second reader before native start and restores it after existingCall startup", async () => { + const f = fixture(); + await expect(f.start()).rejects.toThrow("voice_disabled"); + const wrapper = wrapGatewayVoiceStarter(f.start, f.options); + await expect(wrapper.start(request, "context")).resolves.toBe("existingCall"); + expect(f.start).toHaveBeenLastCalledWith(request, "context"); + expect(f.projected()).toBe(false); + expect(f.dispose).toHaveBeenCalledOnce(); + }); + it("passes unowned starts through without preparing or reading the gateway", async () => { + const start = vi.fn().mockResolvedValue("native"); + const f = fixture(); + const ready = vi.fn(); + const wrapper = wrapGatewayVoiceStarter(start, { ...f.options, qualifies: () => false, ready }); + await expect(wrapper.start(request)).resolves.toBe("native"); + expect(f.options.prepare).not.toHaveBeenCalled(); + expect(ready).not.toHaveBeenCalled(); + }); + it("fails closed without ready transport and does not fall back to native account creation", async () => { + const f = fixture(); + const wrapper = wrapGatewayVoiceStarter(f.start, { ...f.options, ready: async () => false }); + await expect(wrapper.start(request)).rejects.toThrow(/gateway/u); + expect(f.start).not.toHaveBeenCalled(); + expect(f.options.prepare).not.toHaveBeenCalled(); + }); + it("releases the subscription when native workspace or entitlement checks reject", async () => { + const f = fixture(); + f.start.mockRejectedValue(new Error("workspace_not_ready")); + const wrapper = wrapGatewayVoiceStarter(f.start, f.options); + await expect(wrapper.start(request)).rejects.toThrow("workspace_not_ready"); + expect(f.dispose).toHaveBeenCalledOnce(); + }); + it("cancels before invoking native start if cancellation occurs during qualification", async () => { + const f = fixture(); + let release!: () => void; + const committed = () => + new Promise((resolve) => { + release = resolve; + }); + const wrapper = wrapGatewayVoiceStarter(f.start, { ...f.options, committed }); + const result = wrapper.start(request); + await vi.waitFor(() => expect(release).toBeTypeOf("function")); + wrapper.cancel(); + release(); + await expect(result).rejects.toThrow(/cancel/u); + expect(f.start).not.toHaveBeenCalled(); + expect(f.projected()).toBe(false); + }); + it("allows retry after cancellation while the old native start is still settling", async () => { + const f = fixture(); + let finish!: () => void; + const pending = new Promise((resolve) => { + finish = resolve; + }); + const firstStarted = vi.fn(); + const start = vi + .fn() + .mockImplementationOnce(async () => { + firstStarted(); + await pending; + }) + .mockResolvedValue("retry"); + const wrapper = wrapGatewayVoiceStarter(start, f.options); + const first = wrapper.start(request); + await vi.waitFor(() => expect(firstStarted).toHaveBeenCalledOnce()); + wrapper.cancel(); + await expect(wrapper.start(request)).resolves.toBe("retry"); + finish(); + await first; + }); + it("rejects replaced subscriptions", async () => { + const f = fixture(); + const wrapper = wrapGatewayVoiceStarter(f.start, { + ...f.options, + prepare: async () => ({ dispose: f.dispose, current: () => false }), + }); + await expect(wrapper.start(request)).rejects.toThrow(/changed/u); + expect(f.start).not.toHaveBeenCalled(); + }); +}); + +// Registration is separate from the visible Composer and happens in a different realm. +import { + installVoiceStartCapabilityHook, + overlayVoiceStarterFibers, +} from "../src/renderer-voice-start-capability.js"; +import { CLIENT_OWNED_VOICE_ASSETS } from "../src/renderer-client-owned-voice.js"; + +describe("overlay starter registration", () => { + function registration(assets = CLIENT_OWNED_VOICE_ASSETS[0]) { + if (!assets) throw new Error("Missing fixture profile"); + const register = vi.fn(); + const unregister = vi.fn(); + const native = new Proxy( + { registerRealtimeStarter: register, unregisterRealtimeStarter: unregister }, + { + set() { + throw new Error("immutable RPC"); + }, + }, + ); + const services = { realtimeVoiceRuntime: native }; + const names = [assets.initial, assets.shared]; + const target = { + __codexhostCpaLiveVoiceHookV1: { + enabled: true, + desktopVersion: assets.version, + desktopPlatform: assets.platform, + auditedDesktopVersion: "26.1007.30912", + auditedAppInitialAsset: "app-initial-6350d04672d0.js", + }, + location: { href: "app://-/index.html?initialRoute=/avatar-overlay" }, + document: { + querySelectorAll: () => + names.map((name) => ({ getAttribute: () => `app://-/assets/${name}` })), + } as unknown as Document, + setTimeout, + clearTimeout, + }; + const importModule = vi.fn(async () => ({ + Oit() {}, + ist: services, + Qot: services, + })); + return { register, unregister, native, services, names, target, importModule }; + } + it.each(CLIENT_OWNED_VOICE_ASSETS)( + "starts through the $version bridge with the committed overlay capability and restores it", + async (assets) => { + const f = registration(assets); + // This Desktop can omit all resource timing and script asset names. + f.names.length = 0; + Object.assign(f.target, { __codexhostClientOwnedVoiceV1: { status: "active" } }); + type Atom = { read(get: (atom: Atom) => unknown): unknown; write?: boolean }; + const gate: Atom = { read: () => true, write: true }; + const atom: Atom = { + read: (get) => ({ isLoading: false, isError: false, isCapable: !get(gate) }), + }; + const store = { + get: (a: Atom): unknown => a.read(store.get), + sub: (atom: Atom, listener: () => void) => { + void atom; + void listener; + return () => {}; + }, + }; + const signal = { atom, store }; + const original = () => store.get(atom); + const subscriber = { + getSnapshot: original, + subscribe: (listener: () => void) => store.sub(atom, listener), + }; + const instance = { getSnapshot: original }; + const committed = { memoizedState: original(), queue: instance, next: {} }; + const family = { scope: { id: Symbol() }, resolve: () => ({ atom: gate }) }; + const node = { + store, + familyBindings: new Map([ + [family, new Map([['{"name":"voice.realtime"}', { cacheKey: "voice" }]])], + ]), + }; + const chain = new Map([[family.scope.id, node]]); + Object.assign(committed.next, { + memoizedState: { + create: () => + subscriber.subscribe(() => { + committed.memoizedState = instance.getSnapshot(); + }), + deps: [subscriber.subscribe], + }, + }); + const page = () => {}; + const element = { + __reactFiber$test: { + memoizedState: { + memoizedState: [signal, ["voice", chain, undefined, family, undefined]], + next: { memoizedState: [subscriber, [signal, undefined]], next: committed }, + }, + return: { type: page, return: null }, + }, + }; + const query = f.target.document.querySelectorAll; + f.target.document.querySelectorAll = ((selector: string) => + selector === "[data-avatar-overlay-native-surface-id]" + ? [element] + : query.call(f.target.document, selector)) as Document["querySelectorAll"]; + let reply: ((message: unknown) => void) | undefined; + const dispatchMessage = vi.fn((type, message) => { + expect(type).toBe("mcp-request"); + expect(message.request.method).toBe("codexhost/voice/gateway-status"); + reply?.({ hostId: "local", message: { id: message.request.id, result: true } }); + }); + const bridge = { + subscribe: (_type: string, listener: (message: unknown) => void) => { + reply = listener; + return () => { + reply = undefined; + }; + }, + dispatchMessage, + }; + const initialize = vi.fn(); + const importModule = async (url: string) => + url.endsWith(assets.shared) + ? { [assets.services]: f.services, [assets.bridge]: { getInstance: () => bridge } } + : url.endsWith(assets.overlay) + ? { AvatarOverlayNativePage: page } + : { + [assets.capabilityInit]: initialize, + [assets.capabilityFamily]: family, + [assets.inferenceGateFamily]: family, + }; + const handle = installVoiceStartCapabilityHook({ ...f, importModule }); + try { + await vi.waitFor(() => expect(handle.status).toBe("active")); + const start = vi.fn(async () => { + expect(instance.getSnapshot()).toMatchObject({ isCapable: true }); + expect(store.get(gate)).toBe(true); + return "existingCall"; + }); + f.services.realtimeVoiceRuntime.registerRealtimeStarter(start, vi.fn()); + const wrappedStart = f.register.mock.calls[0]?.[0]; + if (typeof wrappedStart !== "function") throw new Error("Missing registered starter"); + const launch = + assets.version === "26.1021.31345" + ? { + source: "composer_button_existing_thread", + productExperience: "codex", + locator: { hostId: "local" }, + } + : request; + await expect(wrappedStart(launch)).resolves.toBe("existingCall"); + expect(start).toHaveBeenCalledOnce(); + expect(start).toHaveBeenCalledWith(launch); + expect(initialize).toHaveBeenCalledOnce(); + expect(instance.getSnapshot).toBe(original); + } finally { + handle.dispose(); + } + }, + ); + it.each(CLIENT_OWNED_VOICE_ASSETS)( + "wraps the $version callback without modifying RPC stubs or registration readiness", + async (assets) => { + const f = registration(assets); + const handle = installVoiceStartCapabilityHook(f); + await vi.waitFor(() => expect(handle.status).toBe("active")); + const start = vi.fn().mockResolvedValue("native-chat"); + const cancel = vi.fn(); + f.services.realtimeVoiceRuntime.registerRealtimeStarter(start, cancel, true); + const [wrappedStart, wrappedCancel, workspaceReady] = f.register.mock.calls[0] ?? []; + expect(workspaceReady).toBe(true); + expect(wrappedStart).not.toBe(start); + await wrappedStart({ ...request, productExperience: "chat" }); + expect(start).toHaveBeenCalledOnce(); + await expect(wrappedStart(request)).rejects.toThrow(/gateway/u); + expect(start).toHaveBeenCalledOnce(); + wrappedCancel("user_cancelled"); + expect(cancel).toHaveBeenCalledWith("user_cancelled"); + f.services.realtimeVoiceRuntime.unregisterRealtimeStarter(); + expect(f.unregister).toHaveBeenCalledOnce(); + handle.dispose(); + expect(f.services.realtimeVoiceRuntime).toBe(f.native); + }, + ); + it("does not install on the primary page or an unknown version", () => { + const f = registration(); + f.target.location.href = "app://-/index.html"; + expect(installVoiceStartCapabilityHook(f).status).toBe("disabled"); + expect(f.importModule).not.toHaveBeenCalled(); + }); + it("excludes ancestors outside the exported native overlay page", () => { + const page = () => {}; + const unrelated = { type: () => {}, return: null }; + const boundary = { type: page, return: unrelated }; + const starter = { type: () => {}, return: boundary }; + const element = { __reactFiber$test: starter }; + const doc = { querySelectorAll: () => [element] } as unknown as Document; + expect([...overlayVoiceStarterFibers(doc, page)]).toEqual([starter]); + expect(overlayVoiceStarterFibers(doc, () => {}).size).toBe(0); + }); +}); diff --git a/packages/renderer-extension/test/settings/localization.test.ts b/packages/renderer-extension/test/settings/localization.test.ts index fda26a971..c184ce8dd 100644 --- a/packages/renderer-extension/test/settings/localization.test.ts +++ b/packages/renderer-extension/test/settings/localization.test.ts @@ -6,6 +6,7 @@ import { rendererSettingsMessages, resolveRendererSettingsLocale, } from "../../src/settings/localization.js"; +import { voiceMessages } from "../../src/settings/voice-messages.js"; import { createDefaultRendererSettingsPages } from "../../src/settings/pages.js"; describe("Renderer settings localization", () => { @@ -33,6 +34,19 @@ describe("Renderer settings localization", () => { expect(english.title).toBe("Settings"); expect(chinese.title).toBe("设置"); + expect(english.hideCodexUsageBannerTitle).toBe("Hide Codex usage warning"); + expect(chinese.hideCodexUsageBannerTitle).toBe("隐藏 Codex 额度警告"); + expect(english.hideCodexUsageBannerDescription).toContain("On by default"); + expect(chinese.hideCodexUsageBannerDescription).toContain("默认开启"); + const englishVoice = voiceMessages("en"); + const chineseVoice = voiceMessages("zh-CN"); + expect(Object.keys(englishVoice)).toEqual(Object.keys(chineseVoice)); + expect(englishVoice.title).toBe("Voice"); + expect(chineseVoice.title).toBe("语音"); + expect(englishVoice.chatHint).toContain("current Chat conversation"); + expect(englishVoice.workHint).toContain("External Harness voice uses the configured gateway"); + expect(englishVoice.applyRestart).toBe("Apply and restart codexhost"); + expect(chineseVoice.applyRestart).toContain("重启 codexhost"); expect(chinese.openSettings).toBe("打开 codexhost 设置"); expect(chinese.starOnGitHub).toBe("Star 支持"); expect(english.starOnGitHub).toBe("Star to support"); @@ -84,6 +98,6 @@ describe("Renderer settings localization", () => { createDefaultRendererSettingsPages(rendererSettingsMessages("zh-CN")).map( ({ label }) => label, ), - ).toEqual(["连接", "远程连接", "账号", "会话导入", "用量统计", "通用", "更新", "关于"]); + ).toEqual(["连接", "远程连接", "账号", "会话导入", "用量统计", "通用", "语音", "更新", "关于"]); }); }); diff --git a/packages/renderer-extension/test/settings/pages.test.ts b/packages/renderer-extension/test/settings/pages.test.ts index 940906aef..acb0c5f87 100644 --- a/packages/renderer-extension/test/settings/pages.test.ts +++ b/packages/renderer-extension/test/settings/pages.test.ts @@ -39,6 +39,8 @@ import { RendererSessionImportUnavailableError } from "../../src/renderer-sessio const HARNESS_SESSION_LIST_METHOD = "codexhost/harness/session-import/list"; const HARNESS_SESSION_IMPORT_METHOD = "codexhost/harness/session-import/import"; import { + CODEXHOST_GITHUB_REPOSITORY_URL, + CODEXHOST_NPM_MANUAL_UPDATE_COMMAND, CODEXHOST_RELEASES_LATEST_URL, createDefaultRendererSettingsPages, } from "../../src/settings/pages.js"; @@ -67,6 +69,7 @@ class FakeElement { value = ""; tabIndex = 0; disabled = false; + checked = false; focused = false; open = false; parent: FakeElement | undefined; @@ -222,7 +225,7 @@ function updateCheck(status: UpdateStatus | null = null): UpdateCheckResult { updateAvailable: true, installationAvailable: true, releaseNotes: "Safer updates", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status, error: null, }; @@ -1818,6 +1821,91 @@ describe("Renderer Connections page", () => { scope.dispose(); }, ); + + it("shows a Computer Use card in the Grok inspector", async () => { + const messages = rendererSettingsMessages("en"); + const getGrokComputerUse = vi.fn(async () => ({ + enabled: true, + available: true, + browser: false, + browserFilesPresent: true, + browserAvailable: false, + })); + const setGrokComputerUse = vi.fn( + async (_hostId: string, update: { enabled?: boolean; browser?: boolean }) => ({ + enabled: update.enabled ?? true, + available: true, + browser: update.browser ?? false, + browserFilesPresent: true, + browserAvailable: false, + }), + ); + const diagnostics: RendererConnectionDiagnostics = { + snapshot: () => ({ + adapter: { state: "ready", reason: "ready", modelUpdates: 1, hook: "request-bridge" }, + hosts: [ + { + hostId: "local", + active: true, + agents: [ + { + agent: "grok", + plugin: pluginDescriptor("grok"), + availability: "ready", + error: null, + }, + ], + }, + ], + }), + refresh: vi.fn(async () => undefined), + getGrokComputerUse, + setGrokComputerUse, + subscribe: () => () => undefined, + }; + const page = createDefaultRendererSettingsPages( + messages, + () => null, + () => diagnostics, + ).find(({ id }) => id === "connections"); + if (!page) throw new Error("Expected connections page"); + const document = new FakeDocument("MacIntel"); + const content = document.createElement("main"); + const scope = new RendererSettingsPageScope(); + const cleanup = page.mount({ + content: content as unknown as HTMLElement, + signal: scope.signal, + runLatest: (op, handlers) => scope.runLatest(op, handlers), + }); + const row = descendants(content).find(({ dataset }) => dataset.connectionItem === "grok"); + if (!row) throw new Error("Expected Grok row"); + row.dispatch("click", { target: null }); + const card = descendants(content).find(({ dataset }) => dataset.grokComputerUse === "true"); + if (!card) throw new Error("Expected Grok Computer Use card"); + expect(visibleText(card)).toContain(messages.grokComputerUseTitle); + expect(visibleText(card)).toContain(messages.grokComputerUseDescription); + expect(visibleText(card)).toContain(messages.grokComputerUseHelp); + const toggle = descendants(card).find(({ tagName }) => tagName === "input"); + if (!toggle) throw new Error("Expected Computer Use switch"); + await vi.waitFor(() => expect(toggle.disabled).toBe(false)); + expect(toggle.checked).toBe(true); + expect(visibleText(card)).toContain(messages.grokComputerUseOn); + expect(visibleText(card)).toContain(messages.grokBrowserTitle); + expect(visibleText(card)).toContain(messages.grokBrowserOff); + const browserToggle = descendants(card).filter(({ tagName }) => tagName === "input")[1]; + if (!browserToggle) throw new Error("Expected in-app browser switch"); + expect(browserToggle.checked).toBe(false); + browserToggle.checked = true; + browserToggle.dispatch("change", {}); + await vi.waitFor(() => + expect(setGrokComputerUse).toHaveBeenCalledWith("local", { browser: true }), + ); + await vi.waitFor(() => expect(visibleText(card)).toContain(messages.grokBrowserBlocked)); + expect(getGrokComputerUse).toHaveBeenCalledWith("local"); + cleanup?.(); + scope.dispose(); + }); + it("offers automatic installation only for Harnesses with an install command", () => { const installation = vi.fn(async () => { throw new Error("not expected"); @@ -2136,7 +2224,7 @@ describe("Renderer Connections page", () => { expect(report).toContain("stderr:\ncheck ~/.pi/agent/settings.json"); const issueLink = descendants(content).find( ({ tagName, href }) => - tagName === "a" && href === "https://github.com/BytePioneer-AI/codex-host/issues/new", + tagName === "a" && href === "https://github.com/karlorz/codex-host/issues/new", ); expect(issueLink).toBeDefined(); const copyButton = descendants( @@ -2319,6 +2407,16 @@ describe("Renderer Codex Accounts page", () => { }); describe("Renderer Updates page", () => { + it("points Settings Updates at karlorz/codex-host and @karlorz/codexhost-cli", () => { + expect(CODEXHOST_GITHUB_REPOSITORY_URL).toBe("https://github.com/karlorz/codex-host"); + expect(CODEXHOST_RELEASES_LATEST_URL).toBe( + "https://github.com/karlorz/codex-host/releases/latest", + ); + expect(CODEXHOST_NPM_MANUAL_UPDATE_COMMAND).toBe( + "npm install -g @karlorz/codexhost-cli@latest", + ); + }); + it.each(["succeeded", "failed"] as const)( "keeps polling after start/status timeouts until the Host reports %s", async (terminalPhase) => { @@ -2565,9 +2663,7 @@ describe("Renderer Updates page", () => { expect(releaseLink.rel).toBe("noopener noreferrer"); await vi.waitFor(() => { - expect(releaseLink.href).toBe( - "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", - ); + expect(releaseLink.href).toBe("https://github.com/karlorz/codex-host/releases/tag/v1.2.3"); }); const panel = elementWithClass(content, "settings-update-panel"); @@ -2579,9 +2675,7 @@ describe("Renderer Updates page", () => { expect(panel.dataset.updateState).toBe("failed"); }); expect(descendants(content)).toContain(releaseLink); - expect(releaseLink.href).toBe( - "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", - ); + expect(releaseLink.href).toBe("https://github.com/karlorz/codex-host/releases/tag/v1.2.3"); cleanup?.(); scope.dispose(); @@ -2670,13 +2764,13 @@ describe("Renderer Updates page", () => { ).toBeUndefined(); expect(client.startUpdate).not.toHaveBeenCalled(); if (installation === "npm") { - expect(visibleText(content)).toContain("npm install -g @codexhost/cli@latest"); + expect(visibleText(content)).toContain("npm install -g @karlorz/codexhost-cli@latest"); } else { const link = descendants(content).find( ({ tagName, href }) => tagName === "a" && href === - "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", + "https://github.com/karlorz/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", ); expect(link).toMatchObject({ target: "_blank", rel: "noopener noreferrer" }); } @@ -2708,12 +2802,11 @@ describe("Renderer Updates page", () => { expect(visibleText(content)).toContain("codexhost 是一个开源项目"); expect(visibleText(content)).toContain("请给我们一个 Star"); const repository = descendants(content).find( - ({ tagName, href }) => - tagName === "a" && href === "https://github.com/BytePioneer-AI/codex-host", + ({ tagName, href }) => tagName === "a" && href === "https://github.com/karlorz/codex-host", ); expect(repository).toMatchObject({ target: "_blank", rel: "noopener noreferrer" }); expect(visibleNotesText(repository as FakeElement)).toContain( - "https://github.com/BytePioneer-AI/codex-host", + "https://github.com/karlorz/codex-host", ); cleanup?.(); @@ -2823,7 +2916,7 @@ describe("Renderer Updates page", () => { expect(visibleText(starBanner)).toContain("如果 CodexHost 帮到了你,请在 GitHub 点个 Star"); const starLink = descendants(starBanner).find(({ tagName }) => tagName === "a"); expect(starLink).toMatchObject({ - href: "https://github.com/BytePioneer-AI/codex-host", + href: "https://github.com/karlorz/codex-host", target: "_blank", rel: "noopener noreferrer", }); diff --git a/packages/renderer-extension/test/settings/release-notes.test.ts b/packages/renderer-extension/test/settings/release-notes.test.ts index b3454e850..293229478 100644 --- a/packages/renderer-extension/test/settings/release-notes.test.ts +++ b/packages/renderer-extension/test/settings/release-notes.test.ts @@ -76,7 +76,7 @@ describe("Release notes Markdown", () => { "### npm", "", "```bash", - "npm install -g @codexhost/cli", + "npm install -g @karlorz/codexhost-cli", "codexhost", "```", ].join("\n"), @@ -94,13 +94,13 @@ describe("Release notes Markdown", () => { expect(visibleText(root)).toContain("本次发布"); expect(visibleText(root)).not.toContain("##"); expect(visibleText(root)).not.toContain("- 新增"); - expect(visibleText(root)).toContain("npm install -g @codexhost/cli"); + expect(visibleText(root)).toContain("npm install -g @karlorz/codexhost-cli"); const list = descendants(root).find((element) => element.tagName === "ul"); expect(list?.children).toHaveLength(2); const code = descendants(root).find((element) => element.tagName === "code"); expect(code?.className).toBe("language-bash"); - expect(code?.textContent).toBe("npm install -g @codexhost/cli\ncodexhost"); + expect(code?.textContent).toBe("npm install -g @karlorz/codexhost-cli\ncodexhost"); }); it("sets the translation line of a bilingual paragraph apart", () => { @@ -132,7 +132,7 @@ describe("Release notes Markdown", () => { "Use `codexhost` and **restart** after install.", "", "1. Download the package", - "2. Open [Releases](https://github.com/BytePioneer-AI/codex-host/releases)", + "2. Open [Releases](https://github.com/karlorz/codex-host/releases)", ].join("\n"), ); @@ -143,9 +143,7 @@ describe("Release notes Markdown", () => { expect(strong?.textContent).toBe("restart"); const link = descendants(root).find((element) => element.tagName === "a"); expect(link?.textContent).toBe("Releases"); - expect(link?.attributes.get("href")).toBe( - "https://github.com/BytePioneer-AI/codex-host/releases", - ); + expect(link?.attributes.get("href")).toBe("https://github.com/karlorz/codex-host/releases"); expect(link?.attributes.get("target")).toBe("_blank"); expect(link?.attributes.get("rel")).toBe("noopener noreferrer"); }); diff --git a/packages/renderer-extension/test/settings/shell.test.ts b/packages/renderer-extension/test/settings/shell.test.ts index 8c0452c7d..bfb06cae4 100644 --- a/packages/renderer-extension/test/settings/shell.test.ts +++ b/packages/renderer-extension/test/settings/shell.test.ts @@ -24,6 +24,7 @@ describe("Renderer settings foundation", () => { "Session Import", "Usage statistics", "General", + "Voice", "Updates", "About", ]); @@ -34,6 +35,7 @@ describe("Renderer settings foundation", () => { "session-import", "dashboard", "settings", + "settings", "updates", "about", ]); @@ -52,6 +54,7 @@ describe("Renderer settings foundation", () => { "session-import", "usage-statistics", "appearance", + "voice", "updates", "about", ]); diff --git a/packages/renderer-extension/test/voice-settings-sync.test.ts b/packages/renderer-extension/test/voice-settings-sync.test.ts new file mode 100644 index 000000000..41fdbfcc0 --- /dev/null +++ b/packages/renderer-extension/test/voice-settings-sync.test.ts @@ -0,0 +1,195 @@ +import { afterEach, expect, it, vi } from "vitest"; +import { startVoiceSettingsSync } from "../src/voice-settings-sync.js"; +import { VOICE_GATEWAY_STATUS_METHOD } from "@codexhost/shared-contracts"; +import { + nativeChatVoicePreferenceAllowsStart, + chatLiveVoicePreferenceAllowsOverride, + voiceGatewayIsReady, +} from "../src/renderer-cpa-live-voice-preference.js"; + +afterEach(() => { + vi.useRealTimers(); +}); +function fixture() { + vi.useFakeTimers(); + const hook = { activeCall: false }; + const owner = { + document: { querySelectorAll: () => [] }, + setInterval, + clearInterval, + localStorage: { + getItem: (key: string) => (key === "codexhost.cpa-live-voice.v1" ? "0" : null), + }, + __codexhostNativeChatVoiceV1: hook, + } as unknown as Window; + const state = { + settings: { + chatExperience: "codex", + chatConnection: "desktop", + workConnection: "desktop", + inject: true, + }, + initialized: true, + revision: "shared", + activeInject: true, + envOverride: null, + activeEnvOverride: null, + routesActive: false, + activeCall: false, + gatewayReady: false as boolean | null, + restartAvailable: true, + restart: { phase: "idle", error: null }, + }; + return { owner, hook, state }; +} + +it("applies shared Codex default, acknowledges it on the next poll, and stops on dispose", async () => { + const { owner, state } = fixture(); + const request = vi.fn(async () => state); + const stop = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(1); + expect(nativeChatVoicePreferenceAllowsStart(owner)).toBe(false); + expect(chatLiveVoicePreferenceAllowsOverride(owner)).toBe(false); + await vi.advanceTimersByTimeAsync(100); + expect(request).toHaveBeenLastCalledWith( + "codexhost/voice/settings", + expect.objectContaining({ + runtime: expect.objectContaining({ revision: "shared", activeCall: false }), + }), + ); + stop(); + const count = request.mock.calls.length; + await vi.advanceTimersByTimeAsync(1000); + expect(request).toHaveBeenCalledTimes(count); +}); + +it("migrates existing Desktop choices only when the shared routes are absent", async () => { + const { owner, state } = fixture(); + state.initialized = false; + const request = vi.fn(async (_method: string, input: unknown) => { + if ((input as { operation: string }).operation === "migrate") state.initialized = true; + return state; + }); + const stop = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(200); + expect( + request.mock.calls.filter( + ([, input]) => (input as { operation: string }).operation === "migrate", + ), + ).toEqual([ + [ + "codexhost/voice/settings", + { + operation: "migrate", + routes: { chatExperience: "chatgpt", chatConnection: "desktop", workConnection: "desktop" }, + }, + ], + ]); + stop(); +}); + +it("defers route changes during a native voice call", async () => { + const { owner, hook, state } = fixture(); + hook.activeCall = true; + const request = vi.fn(async () => state); + const stop = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(100); + expect(nativeChatVoicePreferenceAllowsStart(owner)).toBe(true); + expect(request).toHaveBeenLastCalledWith( + "codexhost/voice/settings", + expect.objectContaining({ + runtime: expect.objectContaining({ activeCall: true, revision: "" }), + }), + ); + hook.activeCall = false; + await vi.advanceTimersByTimeAsync(100); + expect(nativeChatVoicePreferenceAllowsStart(owner)).toBe(false); + stop(); +}); + +it("qualifies gateway capability only from a successful live settings read", async () => { + const { owner, state } = fixture(); + const request = vi.fn(async () => ({ ...state, gatewayReady: true })); + const stop = startVoiceSettingsSync(owner, request, 100); + expect(voiceGatewayIsReady(owner)).toBe(false); + await vi.advanceTimersByTimeAsync(1); + expect(voiceGatewayIsReady(owner)).toBe(true); + request.mockRejectedValueOnce(new Error("Host disconnected")); + await vi.advanceTimersByTimeAsync(100); + expect(voiceGatewayIsReady(owner)).toBe(true); + await vi.advanceTimersByTimeAsync(100); + expect(voiceGatewayIsReady(owner)).toBe(true); + stop(); + expect(voiceGatewayIsReady(owner)).toBe(false); +}); + +it("reads readiness directly from Host when runtime settings omit the console callback", async () => { + const { owner, state } = fixture(); + state.gatewayReady = null; + let ready = true; + const request = vi.fn(async (method: string) => + method === VOICE_GATEWAY_STATUS_METHOD ? ready : state, + ); + const stop = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(1); + expect(voiceGatewayIsReady(owner)).toBe(true); + expect(request).toHaveBeenCalledWith(VOICE_GATEWAY_STATUS_METHOD, {}); + request.mockRejectedValueOnce(new Error("Host reconnecting")); + await vi.advanceTimersByTimeAsync(100); + expect(voiceGatewayIsReady(owner)).toBe(true); + request.mockImplementationOnce(async () => state); + request.mockRejectedValueOnce(new Error("Gateway status unavailable")); + await vi.advanceTimersByTimeAsync(100); + expect(voiceGatewayIsReady(owner)).toBe(true); + ready = false; + await vi.advanceTimersByTimeAsync(100); + expect(voiceGatewayIsReady(owner)).toBe(false); + stop(); +}); + +it("does not apply a gateway status that arrives after disposal", async () => { + const { owner, state } = fixture(); + state.gatewayReady = null; + let resolveStatus!: (value: boolean) => void; + const request = vi.fn(async (method: string) => + method === VOICE_GATEWAY_STATUS_METHOD + ? new Promise((resolve) => { + resolveStatus = resolve; + }) + : state, + ); + const stop = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(1); + stop(); + resolveStatus(true); + await vi.advanceTimersByTimeAsync(1); + expect(voiceGatewayIsReady(owner)).toBe(false); +}); + +it("updates retained voice hooks after a separately evaluated Renderer bundle replaces settings sync", async () => { + const { owner, state } = fixture(); + state.gatewayReady = true; + const request = vi.fn(async () => state); + const stopFirst = startVoiceSettingsSync(owner, request, 100); + await vi.advanceTimersByTimeAsync(1); + expect(voiceGatewayIsReady(owner)).toBe(true); + stopFirst(); + expect(voiceGatewayIsReady(owner)).toBe(false); + + // Hook globals outlive the binding probe. A reinjected production bundle has + // new module instances but must update the state read by those retained hooks. + vi.resetModules(); + const next = await import("../src/voice-settings-sync.js"); + const stopNext = next.startVoiceSettingsSync(owner, request, 100); + try { + await vi.advanceTimersByTimeAsync(1); + expect(voiceGatewayIsReady(owner)).toBe(true); + state.settings.chatConnection = "gateway"; + state.revision = "updated"; + await vi.advanceTimersByTimeAsync(100); + expect(chatLiveVoicePreferenceAllowsOverride(owner)).toBe(true); + } finally { + stopNext(); + } + expect(voiceGatewayIsReady(owner)).toBe(false); +}); diff --git a/packages/repository-automation/src/release.mjs b/packages/repository-automation/src/release.mjs index fe1aa441e..4217710a5 100644 --- a/packages/repository-automation/src/release.mjs +++ b/packages/repository-automation/src/release.mjs @@ -80,6 +80,7 @@ export async function readReleaseMetadata({ root, tag, expectedRefSha }) { } if (!notes.trim()) throw new Error("annotated tag body must contain release notes"); const prereleaseId = version.split("+")[0].split("-").slice(1).join("-"); + const numericForkCounter = /^\d+(?:-\d+)*$/u.test(prereleaseId); return { tag, version, @@ -87,8 +88,13 @@ export async function readReleaseMetadata({ root, tag, expectedRefSha }) { tagObjectSha, notes, automationSha: automationSha.trim(), - prerelease: Boolean(prereleaseId), - npmTag: !prereleaseId ? "latest" : /^test(?:\.|$)/u.test(prereleaseId) ? "test" : "next", + prerelease: Boolean(prereleaseId) && !numericForkCounter, + npmTag: + !prereleaseId || numericForkCounter + ? "latest" + : /^test(?:\.|$)/u.test(prereleaseId) + ? "test" + : "next", }; } diff --git a/packages/repository-automation/test/release.test.mjs b/packages/repository-automation/test/release.test.mjs index a04f8380d..632b23774 100644 --- a/packages/repository-automation/test/release.test.mjs +++ b/packages/repository-automation/test/release.test.mjs @@ -109,12 +109,18 @@ describe("release source and metadata", () => { }); it.each([ - ["1.2.3-test.1", "test"], - ["1.2.3-beta.1", "next"], - ["1.2.3+build.1", "latest"], - ])("retains npm routing for %s", async (version, npmTag) => { + ["1.2.3", "latest", false], + ["0.13.2-1", "latest", false], + ["0.13.2-2", "latest", false], + ["0.13.3-0", "latest", false], + ["0.13.3-0-1", "latest", false], + ["0.13.3-cpa.1", "next", true], + ["1.2.3-test.1", "test", true], + ["1.2.3-beta.1", "next", true], + ["1.2.3+build.1", "latest", false], + ])("retains npm routing for %s", async (version, npmTag, prerelease) => { const f = await repository({ version }); - expect((await readReleaseMetadata(f)).npmTag).toBe(npmTag); + expect(await readReleaseMetadata(f)).toMatchObject({ npmTag, prerelease }); }); it("rejects lightweight tags and empty release-note bodies", async () => { diff --git a/packages/shared-contracts/src/console.ts b/packages/shared-contracts/src/console.ts index f5852d20d..5c4f0d7ac 100644 --- a/packages/shared-contracts/src/console.ts +++ b/packages/shared-contracts/src/console.ts @@ -33,6 +33,7 @@ export type ConsoleOpenResult = z.infer; * channel: the settings Codex exposes, never Thread or Turn operations. */ export const CONSOLE_HOST_METHODS = Object.freeze([ + "codexhost/voice/gateway-status", "codexhost/console/remote-connections", "codexhost/runtime/status", "codexhost/remote/ssh-setup", @@ -45,6 +46,8 @@ export const CONSOLE_HOST_METHODS = Object.freeze([ "codexhost/harness/installation/status", "codexhost/harness/launch-settings/get", "codexhost/harness/launch-settings/set", + "codexhost/harness/grok-computer-use/get", + "codexhost/harness/grok-computer-use/set", "codexhost/harness/web-ui/open", "codexhost/harness/accounts/sources", "codexhost/harness/accounts/list", diff --git a/packages/shared-contracts/src/grok-computer-use.ts b/packages/shared-contracts/src/grok-computer-use.ts new file mode 100644 index 000000000..414a63a7f --- /dev/null +++ b/packages/shared-contracts/src/grok-computer-use.ts @@ -0,0 +1,31 @@ +import { z } from "zod"; + +export const GROK_COMPUTER_USE_GET_METHOD = "codexhost/harness/grok-computer-use/get"; +export const GROK_COMPUTER_USE_SET_METHOD = "codexhost/harness/grok-computer-use/set"; + +export const grokComputerUseGetSchema = z.object({}).strict(); +export const grokComputerUseSetSchema = z + .object({ + enabled: z.boolean().optional(), + browser: z.boolean().optional(), + }) + .strict() + .refine((value) => value.enabled !== undefined || value.browser !== undefined, { + message: "Expected enabled or browser", + }); +export const grokComputerUseSettingsSchema = z + .object({ + enabled: z.boolean(), + available: z.boolean(), + unavailableReason: z.string().min(1).max(500).optional(), + browser: z.boolean().optional(), + browserFilesPresent: z.boolean().optional(), + browserAvailable: z.boolean().optional(), + browserUnavailableReason: z.string().min(1).max(500).optional(), + applyFailed: z.boolean().optional(), + applyError: z.string().min(1).max(500).optional(), + }) + .strict(); + +export type GrokComputerUseSettings = z.infer; +export type GrokComputerUseSet = z.infer; diff --git a/packages/shared-contracts/src/index.ts b/packages/shared-contracts/src/index.ts index 2c29855f8..424ff8330 100644 --- a/packages/shared-contracts/src/index.ts +++ b/packages/shared-contracts/src/index.ts @@ -7,6 +7,7 @@ export * from "./account-credits-presentation.js"; export * from "./model-price-overrides.js"; export * from "./usage-statistics.js"; export * from "./thread-prewarm.js"; +export * from "./realtime-call.js"; export { CONSOLE_HOST_METHODS, CONSOLE_OPEN_METHOD, @@ -87,6 +88,7 @@ export type { HarnessPluginManifest, } from "./harness-plugins.js"; export * from "./harness-launch-settings.js"; +export * from "./grok-computer-use.js"; export * from "./harness-installation.js"; export { codexhostErrorSchema } from "./errors.js"; export { @@ -380,3 +382,5 @@ export { type CodexSshConnection, type CodexSshDraft, } from "./remote-connections.js"; + +export * from "./voice-settings.js"; diff --git a/packages/shared-contracts/src/realtime-call.ts b/packages/shared-contracts/src/realtime-call.ts new file mode 100644 index 000000000..d7fbf646a --- /dev/null +++ b/packages/shared-contracts/src/realtime-call.ts @@ -0,0 +1,71 @@ +import { z } from "zod"; + +export const REALTIME_CALL_METHOD = "codexhost/realtime/call"; + +/** + * Native voice headers the Host may forward to the configured gateway. Anything + * else — Desktop OAuth, ChatGPT account identity and cookies — is dropped at the + * Renderer bridge and again at the Host before the outbound request. + */ +export const REALTIME_CALL_FORWARDED_HEADERS: readonly string[] = [ + "openai-alpha", + "thread-id", + "session-id", + "originator", + "x-codex-turn-metadata", +]; +const FORWARDED_HEADERS = new Set(REALTIME_CALL_FORWARDED_HEADERS); + +/** Keeps only allowlisted headers with a usable value; all other names are dropped. */ +export function filterRealtimeCallHeaders( + headers: Record | undefined, +): Record { + const result: Record = {}; + for (const [name, value] of Object.entries(headers ?? {})) { + const key = name.toLowerCase(); + if (FORWARDED_HEADERS.has(key) && value.length > 0 && value.length <= 4096) result[key] = value; + } + return result; +} + +/** The single accepted identity form for a gateway-created realtime call. */ +export const REALTIME_CALL_ID_PATTERN = /^rtc_[A-Za-z0-9_-]{1,128}$/u; + +export function isRealtimeCallId(value: unknown): value is string { + return typeof value === "string" && REALTIME_CALL_ID_PATTERN.test(value); +} + +/** + * Reads the created call identity from a gateway `Location` response header: a bare + * identity, or the trailing segment after `/calls/` or `/live/`, ignoring any query + * string or fragment. Callers that require the strict `rtc_` form check the result + * with {@link isRealtimeCallId}. + */ +export function realtimeCallIdFromLocation(location: string): string | undefined { + const path = location.split(/[?#]/u)[0] ?? ""; + if (/^[A-Za-z0-9_-]{1,128}$/u.test(path)) return path; + return /\/(?:calls|live)\/([A-Za-z0-9_-]{1,128})$/u.exec(path)?.[1]; +} + +const requestId = z.string().min(1).max(128); +export const realtimeCallParamsSchema = z.discriminatedUnion("operation", [ + z.object({ + operation: z.literal("create"), + requestId, + body: z + .string() + .min(1) + .max(16 << 20), + headers: z.record(z.string(), z.string()).default({}), + }), + z.object({ operation: z.literal("cancel"), requestId }), + z.object({ + operation: z.literal("hangup"), + callId: z.string().regex(REALTIME_CALL_ID_PATTERN), + }), +]); +export const realtimeCallResponseSchema = z.object({ + status: z.number().int().min(200).max(599), + body: z.string(), + headers: z.record(z.string(), z.string()), +}); diff --git a/packages/shared-contracts/src/updates.ts b/packages/shared-contracts/src/updates.ts index cc4359bcc..f0e1a3ea2 100644 --- a/packages/shared-contracts/src/updates.ts +++ b/packages/shared-contracts/src/updates.ts @@ -46,7 +46,7 @@ const githubReleaseNotesUrlSchema = z .string() .max(300) .regex( - /^https:\/\/github\.com\/BytePioneer-AI\/codex-host\/releases\/tag\/v[0-9A-Za-z.+-]+$/u, + /^https:\/\/github\.com\/karlorz\/codex-host\/releases\/tag\/v[0-9A-Za-z.+-]+$/u, "release notes URL must identify a codexhost GitHub Release", ); diff --git a/packages/shared-contracts/src/voice-settings.ts b/packages/shared-contracts/src/voice-settings.ts new file mode 100644 index 000000000..5cd723dc1 --- /dev/null +++ b/packages/shared-contracts/src/voice-settings.ts @@ -0,0 +1,54 @@ +import { z } from "zod"; + +export const VOICE_SETTINGS_METHOD = "codexhost/voice/settings"; +export const VOICE_GATEWAY_STATUS_METHOD = "codexhost/voice/gateway-status"; +export const voiceRoutesSchema = z.strictObject({ + chatExperience: z.enum(["chatgpt", "codex"]), + chatConnection: z.enum(["desktop", "gateway"]), + workConnection: z.enum(["desktop", "gateway"]), +}); +export type VoiceRoutes = z.infer; +export const DEFAULT_VOICE_ROUTES: VoiceRoutes = Object.freeze({ + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "gateway", +}); +export const voiceSettingsSchema = voiceRoutesSchema.extend({ inject: z.boolean() }); +export type VoiceSettings = z.infer; +export const voiceSettingsRequestSchema = z.discriminatedUnion("operation", [ + z.strictObject({ + operation: z.literal("get"), + runtime: z + .strictObject({ + id: z.string().uuid(), + revision: z.string().max(64), + activeCall: z.boolean(), + }) + .optional(), + }), + z.strictObject({ operation: z.literal("migrate"), routes: voiceRoutesSchema }), + z.strictObject({ + operation: z.literal("set"), + settings: voiceSettingsSchema, + revision: z.string().max(64), + }), + z.strictObject({ operation: z.literal("restart"), confirm: z.literal(true) }), +]); +export type VoiceSettingsRequest = z.infer; +export const voiceSettingsStateSchema = z.strictObject({ + settings: voiceSettingsSchema, + initialized: z.boolean(), + revision: z.string(), + activeInject: z.boolean().nullable(), + envOverride: z.enum(["0", "1"]).nullable(), + activeEnvOverride: z.enum(["0", "1"]).nullable(), + routesActive: z.boolean().nullable(), + activeCall: z.boolean().nullable(), + gatewayReady: z.boolean().nullable(), + restartAvailable: z.boolean(), + restart: z.strictObject({ + phase: z.enum(["idle", "restarting", "succeeded", "failed"]), + error: z.string().nullable(), + }), +}); +export type VoiceSettingsState = z.infer; diff --git a/packages/shared-contracts/test/realtime-call.test.ts b/packages/shared-contracts/test/realtime-call.test.ts new file mode 100644 index 000000000..1f728d687 --- /dev/null +++ b/packages/shared-contracts/test/realtime-call.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it } from "vitest"; + +import { + filterRealtimeCallHeaders, + isRealtimeCallId, + realtimeCallIdFromLocation, + realtimeCallParamsSchema, +} from "@codexhost/shared-contracts"; + +describe("realtime call contract", () => { + it("drops Desktop credentials, cookies and unlisted headers", () => { + expect( + filterRealtimeCallHeaders({ + Authorization: "Bearer desktop-oauth", + Cookie: "session=private", + "ChatGPT-Account-Id": "disabled-account", + "openai-alpha": "quicksilver=v2", + "Thread-Id": "thread-fixture", + "Session-Id": "session-fixture", + }), + ).toEqual({ + "openai-alpha": "quicksilver=v2", + "thread-id": "thread-fixture", + "session-id": "session-fixture", + }); + }); + + it("drops empty and oversized allowlisted header values", () => { + expect( + filterRealtimeCallHeaders({ "openai-alpha": "", "thread-id": "t".repeat(4097) }), + ).toEqual({}); + expect(filterRealtimeCallHeaders(undefined)).toEqual({}); + }); + + it("accepts only the gateway call identity form", () => { + expect(isRealtimeCallId("rtc_fixture-1")).toBe(true); + expect(isRealtimeCallId("fixture")).toBe(false); + expect(isRealtimeCallId("rtc_")).toBe(false); + expect(isRealtimeCallId(undefined)).toBe(false); + }); + + it("reads an identity from bare values and calls/live paths only", () => { + expect(realtimeCallIdFromLocation("rtc_fixture")).toBe("rtc_fixture"); + expect(realtimeCallIdFromLocation("/v1/realtime/calls/rtc_fixture?intent=quicksilver")).toBe( + "rtc_fixture", + ); + expect(realtimeCallIdFromLocation("/v1/live/rtc_fixture#fragment")).toBe("rtc_fixture"); + expect(realtimeCallIdFromLocation("/unrecognized/rtc_fixture")).toBeUndefined(); + expect(realtimeCallIdFromLocation("")).toBeUndefined(); + }); + + it("rejects a malformed or unbounded create request", () => { + expect( + realtimeCallParamsSchema.safeParse({ + operation: "hangup", + callId: "not-a-call", + }).success, + ).toBe(false); + expect( + realtimeCallParamsSchema.safeParse({ + operation: "create", + requestId: "voice", + body: "{}", + }).success, + ).toBe(true); + }); +}); diff --git a/packages/shared-contracts/test/updates.test.ts b/packages/shared-contracts/test/updates.test.ts index c191154d5..f91c8b875 100644 --- a/packages/shared-contracts/test/updates.test.ts +++ b/packages/shared-contracts/test/updates.test.ts @@ -26,7 +26,7 @@ describe("update runtime contracts", () => { updateAvailable: true, installationAvailable: true, releaseNotes: "## Changes\n\n- Safer updates", - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status: null, error: null, }), @@ -111,7 +111,7 @@ describe("update runtime contracts", () => { updateAvailable: true, installationAvailable: true, releaseNotes: "x".repeat(20_001), - releaseNotesUrl: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + releaseNotesUrl: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", status: null, error: null, }).success, diff --git a/packages/update-manager/src/github-cli-release.ts b/packages/update-manager/src/github-cli-release.ts index 2718c2a98..d6e98c1c1 100644 --- a/packages/update-manager/src/github-cli-release.ts +++ b/packages/update-manager/src/github-cli-release.ts @@ -1,9 +1,13 @@ import { execFile } from "node:child_process"; import path from "node:path"; -import { parseLatestGitHubRelease, type CodexhostLatestRelease } from "./github-release.js"; +import { + GITHUB_REPOSITORY, + parseLatestGitHubRelease, + type CodexhostLatestRelease, +} from "./github-release.js"; -const GITHUB_LATEST_RELEASE_ENDPOINT = "repos/BytePioneer-AI/codex-host/releases/latest"; +const GITHUB_LATEST_RELEASE_ENDPOINT = `repos/${GITHUB_REPOSITORY}/releases/latest`; const MAX_RELEASE_RESPONSE_BYTES = 1024 * 1024; const GITHUB_CLI_TIMEOUT_MS = 5_000; diff --git a/packages/update-manager/src/github-release.ts b/packages/update-manager/src/github-release.ts index a8611cb42..326b744f9 100644 --- a/packages/update-manager/src/github-release.ts +++ b/packages/update-manager/src/github-release.ts @@ -1,13 +1,15 @@ import type { ArtifactSource } from "./artifact.js"; import { requireSemanticVersion } from "./status.js"; -export const CODEXHOST_LATEST_RELEASE_URL = - "https://api.github.com/repos/BytePioneer-AI/codex-host/releases/latest"; +export const GITHUB_REPOSITORY = "karlorz/codex-host"; +export const CODEXHOST_LATEST_RELEASE_URL = `https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/latest`; const SHA256_DIGEST_PATTERN = /^sha256:([0-9a-f]{64})$/u; -const RELEASE_NOTES_URL_PATTERN = - /^https:\/\/github\.com\/BytePioneer-AI\/codex-host\/releases\/tag\/(v[0-9A-Za-z.+-]+)$/u; -const DOWNLOAD_URL_PREFIX = "https://github.com/BytePioneer-AI/codex-host/releases/download/"; +const RELEASE_NOTES_URL_PATTERN = new RegExp( + `^https://github\\.com/${GITHUB_REPOSITORY.replaceAll("/", "\\/")}/releases/tag/(v[0-9A-Za-z.+-]+)$`, + "u", +); +const DOWNLOAD_URL_PREFIX = `https://github.com/${GITHUB_REPOSITORY}/releases/download/`; export type InstallerReleaseTarget = "macos-arm64" | "macos-x64" | "windows-x64" | "windows-arm64"; export type ReleaseTarget = InstallerReleaseTarget | "linux-x64" | "linux-arm64"; diff --git a/packages/update-manager/test/github-release.test.ts b/packages/update-manager/test/github-release.test.ts index df760c3e7..8997c7a8f 100644 --- a/packages/update-manager/test/github-release.test.ts +++ b/packages/update-manager/test/github-release.test.ts @@ -9,25 +9,33 @@ import { parseLatestGitHubRelease, selectInstallerReleaseArtifact, type GitHubCliRunner, + type InstallerReleaseTarget, } from "@codexhost/update-manager"; +function installerAsset( + version: string, + target: InstallerReleaseTarget, + overrides: Record = {}, +) { + const name = expectedInstallerAssetName(version, target); + return { + name, + size: 42, + digest: `sha256:${"ab".repeat(32)}`, + browser_download_url: `https://github.com/karlorz/codex-host/releases/download/v${version}/${name}`, + uploader: { login: "karlorz" }, + ...overrides, + }; +} + function release(overrides: Record = {}) { return { tag_name: "v1.2.3", - html_url: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + html_url: "https://github.com/karlorz/codex-host/releases/tag/v1.2.3", draft: false, prerelease: false, body: "## Changes\n\n- Safer updates", - assets: [ - { - name: "codexhost-1.2.3-windows-x64.exe", - size: 42, - digest: `sha256:${"ab".repeat(32)}`, - browser_download_url: - "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", - uploader: { login: "github-actions" }, - }, - ], + assets: [installerAsset("1.2.3", "windows-x64")], author: { login: "github-actions" }, ...overrides, }; @@ -41,7 +49,7 @@ describe("GitHub Release update discovery", () => { expect(selectInstallerReleaseArtifact(parsed, "windows-x64")).toEqual({ name: "codexhost-1.2.3-windows-x64.exe", source: { - url: "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", + url: "https://github.com/karlorz/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", sha256: "ab".repeat(32), size: 42, }, @@ -51,12 +59,62 @@ describe("GitHub Release update discovery", () => { ); }); + it("discovers only karlorz/codex-host latest and rejects BytePioneer assets", () => { + expect(CODEXHOST_LATEST_RELEASE_URL).toBe( + "https://api.github.com/repos/karlorz/codex-host/releases/latest", + ); + expect(() => + parseLatestGitHubRelease( + release({ + html_url: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v1.2.3", + }), + ), + ).toThrow("does not match"); + expect(() => + parseLatestGitHubRelease( + release({ + assets: [ + installerAsset("1.2.3", "windows-x64", { + browser_download_url: + "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", + }), + ], + }), + ), + ).toThrow("invalid"); + }); + + it("parses a fork hyphen version and selects the matching installer asset", () => { + const parsed = parseLatestGitHubRelease( + release({ + tag_name: "v0.13.2-1", + html_url: "https://github.com/karlorz/codex-host/releases/tag/v0.13.2-1", + assets: [installerAsset("0.13.2-1", "macos-arm64")], + }), + ); + expect(parsed.version).toBe("0.13.2-1"); + expect(parsed.releaseNotesUrl).toBe( + "https://github.com/karlorz/codex-host/releases/tag/v0.13.2-1", + ); + expect(expectedInstallerAssetName("0.13.2-1", "macos-arm64")).toBe( + "codexhost-0.13.2-1-macos-arm64.dmg", + ); + expect(selectInstallerReleaseArtifact(parsed, "macos-arm64")).toEqual({ + name: "codexhost-0.13.2-1-macos-arm64.dmg", + source: { + url: "https://github.com/karlorz/codex-host/releases/download/v0.13.2-1/codexhost-0.13.2-1-macos-arm64.dmg", + sha256: "ab".repeat(32), + size: 42, + }, + }); + }); + it("rejects prereleases, mismatched notes, duplicate assets, and unverified selection", () => { expect(() => parseLatestGitHubRelease(release({ prerelease: true }))).toThrow("invalid"); expect(() => parseLatestGitHubRelease( release({ - html_url: "https://github.com/BytePioneer-AI/codex-host/releases/tag/v9.9.9", + html_url: "https://github.com/karlorz/codex-host/releases/tag/v9.9.9", }), ), ).toThrow("does not match"); @@ -67,7 +125,7 @@ describe("GitHub Release update discovery", () => { name: "codexhost-1.2.3-windows-x64.exe", size: 42, browser_download_url: - "https://github.com/BytePioneer-AI/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", + "https://github.com/karlorz/codex-host/releases/download/v1.2.3/codexhost-1.2.3-windows-x64.exe", }, ], }), @@ -95,6 +153,9 @@ describe("GitHub Release update discovery", () => { expect(compareSemanticVersions("1.2.3-test.2", "1.2.3")).toBeLessThan(0); expect(compareSemanticVersions("1.2.4", "1.2.3")).toBeGreaterThan(0); expect(compareSemanticVersions("1.2.3+one", "1.2.3+two")).toBe(0); + expect(compareSemanticVersions("0.13.2-0", "0.13.2-1")).toBeLessThan(0); + expect(compareSemanticVersions("0.13.2-1", "0.13.2-1")).toBe(0); + expect(compareSemanticVersions("0.13.2-1", "0.13.2")).toBeLessThan(0); }); it("requests only the fixed GitHub latest endpoint", async () => { @@ -135,7 +196,7 @@ describe("GitHub Release update discovery", () => { "Accept: application/vnd.github+json", "--header", "X-GitHub-Api-Version: 2022-11-28", - "repos/BytePioneer-AI/codex-host/releases/latest", + "repos/karlorz/codex-host/releases/latest", ], { environment: { PATH: "/usr/bin:/bin" } }, ); diff --git a/scripts/release/macos/package.sh b/scripts/release/macos/package.sh index 93260be10..f9cdd7b92 100755 --- a/scripts/release/macos/package.sh +++ b/scripts/release/macos/package.sh @@ -31,6 +31,7 @@ for relative in \ app/desktop-controller.mjs \ app/host-runtime.mjs \ app/renderer-extension.js \ + app/cpa-live-voice-hook.js \ licenses/Node.js-LICENSE.txt \ licenses/Anthropic-SDK-LICENSE.txt \ licenses/Claude-Agent-SDK-LICENSE.md \ diff --git a/scripts/release/prepare-npm-meta.mjs b/scripts/release/prepare-npm-meta.mjs index 3a7c83030..3b1087f44 100644 --- a/scripts/release/prepare-npm-meta.mjs +++ b/scripts/release/prepare-npm-meta.mjs @@ -7,6 +7,7 @@ import { NPM_PACKAGE_NAME, NPM_PLATFORM_PACKAGE_NAMES, createNpmBinLauncherSource, + npmPackageRepositoryFields, packNpmPackage, resolveNpmPackageVersion, } from "./prepare-npm.mjs"; @@ -30,12 +31,7 @@ export function createNpmMetaPackageManifest({ version }) { Object.values(NPM_PLATFORM_PACKAGE_NAMES).map((packageName) => [packageName, version]), ), keywords: ["codex", "codexhost", "pi", "claude-code", "agent", "harness"], - repository: { - type: "git", - url: "git+https://github.com/BytePioneer-AI/codex-host.git", - }, - bugs: { url: "https://github.com/BytePioneer-AI/codex-host/issues" }, - homepage: "https://github.com/BytePioneer-AI/codex-host#readme", + ...npmPackageRepositoryFields(), publishConfig: { access: "public" }, }; } diff --git a/scripts/release/prepare-npm.mjs b/scripts/release/prepare-npm.mjs index d7583c8a1..e147d7d84 100644 --- a/scripts/release/prepare-npm.mjs +++ b/scripts/release/prepare-npm.mjs @@ -23,15 +23,40 @@ import { hostReleaseTarget, npmReleaseUsage, releaseTargetForHost } from "./targ const repositoryRoot = path.resolve(import.meta.dirname, "../.."); -export const NPM_PACKAGE_NAME = "@codexhost/cli"; +export const NPM_GITHUB_REPOSITORY = "karlorz/codex-host"; +export const NPM_PACKAGE_NAME = "@karlorz/codexhost-cli"; export const NPM_PLATFORM_PACKAGE_NAMES = Object.freeze({ - "macos-arm64": "@codexhost/cli-darwin-arm64", - "macos-x64": "@codexhost/cli-darwin-x64", - "windows-x64": "@codexhost/cli-win32-x64", - "windows-arm64": "@codexhost/cli-win32-arm64", - "linux-x64": "@codexhost/cli-linux-x64", - "linux-arm64": "@codexhost/cli-linux-arm64", + "macos-arm64": "@karlorz/codexhost-cli-darwin-arm64", + "macos-x64": "@karlorz/codexhost-cli-darwin-x64", + "windows-x64": "@karlorz/codexhost-cli-win32-x64", + "windows-arm64": "@karlorz/codexhost-cli-win32-arm64", + "linux-x64": "@karlorz/codexhost-cli-linux-x64", + "linux-arm64": "@karlorz/codexhost-cli-linux-arm64", }); + +export function requireForkNpmIdentity(packageName) { + if (!packageName.startsWith("@karlorz/")) { + throw new Error( + `refusing to publish ${packageName}; fork npm packages must use the @karlorz scope`, + ); + } +} + +requireForkNpmIdentity(NPM_PACKAGE_NAME); +for (const packageName of Object.values(NPM_PLATFORM_PACKAGE_NAMES)) { + requireForkNpmIdentity(packageName); +} + +export function npmPackageRepositoryFields() { + return { + repository: { + type: "git", + url: `git+https://github.com/${NPM_GITHUB_REPOSITORY}.git`, + }, + bugs: { url: `https://github.com/${NPM_GITHUB_REPOSITORY}/issues` }, + homepage: `https://github.com/${NPM_GITHUB_REPOSITORY}#readme`, + }; +} export const NPM_RUNTIME_PLATFORM_PACKAGES = Object.freeze({ "darwin-arm64": NPM_PLATFORM_PACKAGE_NAMES["macos-arm64"], "darwin-x64": NPM_PLATFORM_PACKAGE_NAMES["macos-x64"], @@ -248,6 +273,7 @@ export function expectedNpmPackagePaths(target) { "app/desktop-controller.mjs", "app/host-runtime.mjs", "app/renderer-extension.js", + "app/cpa-live-voice-hook.js", ...preinstalledHarnessPluginPaths(), "licenses/Anthropic-SDK-LICENSE.txt", "licenses/Agent-Client-Protocol-SDK-LICENSE.txt", @@ -288,14 +314,7 @@ export function createNpmPackageManifest({ version, target }) { os: npmPackageOs(target), cpu: npmPackageCpu(target), keywords: ["codex", "codexhost", "pi", "claude-code", "agent", "harness"], - repository: { - type: "git", - url: "git+https://github.com/BytePioneer-AI/codex-host.git", - }, - bugs: { - url: "https://github.com/BytePioneer-AI/codex-host/issues", - }, - homepage: "https://github.com/BytePioneer-AI/codex-host#readme", + ...npmPackageRepositoryFields(), publishConfig: { access: "public", }, @@ -313,7 +332,7 @@ import { fileURLToPath } from "node:url"; const version = ${JSON.stringify(version)}; const userArguments = process.argv.slice(2); -const repositoryUrl = "https://github.com/BytePioneer-AI/codex-host"; +const repositoryUrl = ${JSON.stringify(`https://github.com/${NPM_GITHUB_REPOSITORY}`)}; const startupTraceStartedAt = Date.now(); function startupTrace(stage) { if (process.env.CODEXHOST_STARTUP_TRACE !== "1") return; @@ -392,7 +411,7 @@ try { } if (platformVersion !== version) { fail( - \`platform package version mismatch: '\${platformPackage}' at '\${packageRoot}' has \${JSON.stringify(platformVersion) ?? "no version"}; expected \${version}. Close Codex Desktop, then run: npm install -g @codexhost/cli@\${version} \${platformPackage}@\${version}\`, + \`platform package version mismatch: '\${platformPackage}' at '\${packageRoot}' has \${JSON.stringify(platformVersion) ?? "no version"}; expected \${version}. Close Codex Desktop, then run: npm install -g ${NPM_PACKAGE_NAME}@\${version} \${platformPackage}@\${version}\`, ); } startupTrace("platform package resolved"); @@ -523,7 +542,7 @@ let delegationArguments = null; let consoleArguments = null; if (userArguments.length === 0) { launchArguments = ["launch"]; -} else if (userArguments[0] === "launch") { +} else if (userArguments[0] === "launch" || userArguments[0] === "restart") { launchArguments = userArguments; } else if (userArguments[0] === "inspect") { launchArguments = userArguments; @@ -552,6 +571,7 @@ if (userArguments.length === 0) { " codexhost inspect", " codexhost console", " codexhost update", + " codexhost restart", " codexhost launch [launcher options]", " codexhost remote install|start|stop|status|uninstall", " codexhost broker install|status|stop|uninstall", @@ -571,7 +591,7 @@ if (userArguments.length === 0) { ); } -if (launchArguments?.[0] === "launch") { +if (launchArguments?.[0] === "launch" || launchArguments?.[0] === "restart") { const injected = new Set(); for (let index = 1; index < launchArguments.length; index += 1) { const argument = launchArguments[index]; @@ -594,7 +614,7 @@ if (launchArguments?.[0] === "launch") { extras.push("--desktop-controller", desktopController); } if (!injected.has("--renderer")) extras.push("--renderer", rendererExtension); - launchArguments = ["launch", ...extras, ...launchArguments.slice(1)]; + launchArguments = [launchArguments[0], ...extras, ...launchArguments.slice(1)]; } if (consoleArguments !== null) { @@ -742,7 +762,7 @@ if (consoleArguments !== null) { } process.exit(code ?? 1); }); -} else if (launchArguments?.[0] === "launch") { +} else if (launchArguments?.[0] === "launch" || launchArguments?.[0] === "restart") { // The Launcher prints "ready" once the Desktop, Controller, and Host chain // are up, then detaches from the terminal to keep supervising. Windows // command hosts may clean up a completed command's process tree, so keep the @@ -943,6 +963,7 @@ export async function validateNpmPackage({ packageRoot, target, root }) { "app/console-web.js", "app/desktop-controller.mjs", "app/renderer-extension.js", + "app/cpa-live-voice-hook.js", ].includes(file.relative) ) { forbiddenReferences.push("@anthropic-ai/", "@codexhost/adapter-claude-code"); @@ -1139,6 +1160,11 @@ export async function prepareNpmPackage({ path.join(packageRoot, "app", "renderer-extension.js"), "production Renderer Bundle", ); + await copyReleaseFile( + path.join(root, "packages", "renderer-extension", "dist", "cpa-live-voice-hook.js"), + path.join(packageRoot, "app", "cpa-live-voice-hook.js"), + "CPA live-voice overlay hook Bundle", + ); await copyReleaseFile( path.join(root, "packages", "renderer-extension", "dist", "console.js"), path.join(packageRoot, "app", "console-web.js"), diff --git a/scripts/release/prepare-payload.mjs b/scripts/release/prepare-payload.mjs index 70f9305dc..bc90498ef 100644 --- a/scripts/release/prepare-payload.mjs +++ b/scripts/release/prepare-payload.mjs @@ -276,6 +276,7 @@ export function expectedPayloadPaths(target) { "app/desktop-controller.mjs", "app/host-runtime.mjs", "app/renderer-extension.js", + "app/cpa-live-voice-hook.js", ...preinstalledHarnessPluginPaths(), "licenses/Node.js-LICENSE.txt", "licenses/Copilot-SDK-LICENSE.txt", @@ -343,6 +344,7 @@ export async function validatePayload({ payloadRoot, target, root }) { "app/console-web.js", "app/desktop-controller.mjs", "app/renderer-extension.js", + "app/cpa-live-voice-hook.js", ].includes(file.relative) ) { forbiddenReferences.push("@anthropic-ai/", "@codexhost/adapter-claude-code"); @@ -450,6 +452,11 @@ export async function prepareReleasePayload({ target, root = repositoryRoot }) { path.join(payloadRoot, "app", "renderer-extension.js"), "production Renderer Bundle", ); + await copyReleaseFile( + path.join(root, "packages", "renderer-extension", "dist", "cpa-live-voice-hook.js"), + path.join(payloadRoot, "app", "cpa-live-voice-hook.js"), + "CPA live-voice overlay hook Bundle", + ); await copyReleaseFile( path.join(root, "packages", "renderer-extension", "dist", "console.js"), path.join(payloadRoot, "app", "console-web.js"), diff --git a/scripts/release/publish-npm.mjs b/scripts/release/publish-npm.mjs index df570e480..c56db66e2 100644 --- a/scripts/release/publish-npm.mjs +++ b/scripts/release/publish-npm.mjs @@ -3,7 +3,12 @@ import { readdir } from "node:fs/promises"; import path from "node:path"; import { pathToFileURL } from "node:url"; -import { NPM_PACKAGE_NAME, npmPlatformPackageName, npmTarballFileName } from "./prepare-npm.mjs"; +import { + NPM_PACKAGE_NAME, + npmPlatformPackageName, + npmTarballFileName, + requireForkNpmIdentity, +} from "./prepare-npm.mjs"; import { npmMetaTarballFileName } from "./prepare-npm-meta.mjs"; import { supportedReleaseTargets, releaseTarget } from "./targets.mjs"; @@ -91,6 +96,7 @@ export function npmPublishCommand(entry, { dryRun, provenance, registry, tag }) export function publishNpmPlan(plan, options) { for (const entry of plan) { + requireForkNpmIdentity(entry.packageName); const command = npmPublishCommand(entry, options); console.log( `${options.dryRun ? "checking" : "publishing"} ${entry.packageName}@${entry.version}`, diff --git a/scripts/release/windows/Installer.iss b/scripts/release/windows/Installer.iss index 57b0886fc..9c514ec14 100644 --- a/scripts/release/windows/Installer.iss +++ b/scripts/release/windows/Installer.iss @@ -15,6 +15,12 @@ WizardStyle=modern UninstallDisplayName=codexhost SetupIconFile=..\..\..\crates\launcher\assets\codexhost.ico UninstallDisplayIcon={app}\bin\codexhost-start.exe +; Manual installers must release the bundled runtime before replacing it. Keep +; the filter scoped to codexhost's own files so unrelated Node processes stay +; untouched, and do not relaunch the old console after the upgrade. +CloseApplications=yes +CloseApplicationsFilter=node.exe,codexhost.exe,codexhost-start.exe +RestartApplications=no #if Architecture == "x64" ArchitecturesAllowed=x64compatible diff --git a/tests/e2e/console-restart-layout.spec.ts b/tests/e2e/console-restart-layout.spec.ts new file mode 100644 index 000000000..de8682287 --- /dev/null +++ b/tests/e2e/console-restart-layout.spec.ts @@ -0,0 +1,143 @@ +import { expect, test } from "@playwright/test"; +import { build } from "esbuild"; +import path from "node:path"; +import { tailwindEsbuildPlugin } from "../../packages/renderer-extension/scripts/tailwind-esbuild-plugin.mjs"; + +const browserExecutable = process.env.CODEXHOST_PLAYWRIGHT_EXECUTABLE_PATH; +test.use({ + locale: "en-US", + ...(browserExecutable ? { launchOptions: { executablePath: browserExecutable } } : {}), +}); + +const { outputFiles } = await build({ + entryPoints: ["packages/renderer-extension/src/console-entry.ts"], + absWorkingDir: path.resolve(import.meta.dirname, "../.."), + bundle: true, + format: "iife", + platform: "browser", + target: "es2024", + loader: { ".css": "text", ".png": "dataurl", ".svg": "dataurl" }, + plugins: [tailwindEsbuildPlugin()], + write: false, +}); +const bundle = outputFiles[0]?.text; +if (!bundle) throw new Error("Missing console layout fixture bundle"); + +for (const width of [1920, 640]) { + test(`restart confirmation preserves readable status at ${width}px`, async ({ page }) => { + await page.setViewportSize({ width, height: 1080 }); + let restartRequests = 0; + await page.route("http://console-layout.test/**", async (route) => { + const routePath = new URL(route.request().url()).pathname; + if (routePath === "/") { + await route.fulfill({ contentType: "text/html", body: "" }); + } else if (routePath === "/api/overview") { + await route.fulfill({ + json: { + console: { version: "0.13.3-5", distribution: null }, + inspect: { runtime: { running: true }, desktop: null }, + startup: [], + controller: null, + launchAvailable: true, + liveVoice: { inject: true, envOverride: null }, + summary: { state: "running", detail: null }, + issueUrl: "", + hostAvailable: false, + }, + }); + } else { + if (routePath === "/api/restart") restartRequests += 1; + await route.fulfill({ json: {} }); + } + }); + await page.goto("http://console-layout.test/"); + await page.addScriptTag({ content: bundle }); + await page.getByRole("button", { name: "Restart codexhost", exact: true }).click(); + const copy = page.locator(".console-hero__copy"); + const warning = page.getByRole("status").filter({ hasText: "Restart closes Desktop" }); + await expect(warning).toBeVisible(); + const copyBox = await copy.boundingBox(); + const warningBox = await warning.boundingBox(); + const heroBox = await page.locator(".console-hero").boundingBox(); + if (!copyBox || !warningBox || !heroBox) throw new Error("Missing restart confirmation layout"); + expect(copyBox.width).toBeGreaterThan(200); + expect(copyBox.height).toBeLessThan(100); + expect(warningBox.y).toBeGreaterThanOrEqual(copyBox.y + copyBox.height); + expect(heroBox.height).toBeLessThan(350); + expect(restartRequests).toBe(0); + await page.screenshot({ path: test.info().outputPath("restart-confirmation.png") }); + await page.getByRole("button", { name: "Cancel", exact: true }).click(); + await expect(warning).toHaveCount(0); + expect(restartRequests).toBe(0); + }); +} + +for (const width of [1280, 430]) { + test(`shared console Voice page and navigation fit ${width}px`, async ({ page }) => { + await page.setViewportSize({ width, height: 1100 }); + await page.route("http://console-layout.test/**", async (route) => { + const pathname = new URL(route.request().url()).pathname; + if (pathname === "/") + return route.fulfill({ contentType: "text/html", body: "" }); + if (pathname === "/api/overview") + return route.fulfill({ + json: { + console: { version: "0.13.3-5", distribution: null }, + inspect: { runtime: { running: true }, desktop: null }, + startup: [], + controller: null, + launchAvailable: true, + liveVoice: { inject: true, envOverride: null }, + summary: { state: "running" }, + issueUrl: "", + hostAvailable: false, + }, + }); + if (pathname === "/api/voice-settings") + return route.fulfill({ + json: { + settings: { + inject: true, + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", + }, + initialized: true, + revision: "1", + activeInject: true, + envOverride: null, + activeEnvOverride: null, + activeCall: false, + routesActive: true, + gatewayReady: true, + restartAvailable: true, + restart: { phase: "idle", error: null }, + }, + }); + return route.fulfill({ json: {} }); + }); + await page.goto("http://console-layout.test/#live-voice"); + await page.addScriptTag({ content: bundle }); + await expect(page.getByRole("combobox", { name: "Voice experience", exact: true })).toHaveValue( + "chatgpt", + ); + await expect(page.getByText("Voice", { exact: true }).filter({ visible: true })).toHaveCount( + width < 760 ? 1 : 2, + ); + const overflow = await page + .locator(".console-app") + .evaluate((element) => element.scrollWidth > element.clientWidth); + expect(overflow).toBe(false); + if (width < 760) { + const menu = page.locator(".console-menu-button"); + await menu.click(); + await expect(page.getByRole("button", { name: "Voice", exact: true })).toBeVisible(); + await page.getByRole("button", { name: "Voice", exact: true }).click(); + await expect(menu).toHaveAttribute("aria-expanded", "false"); + } + await page.screenshot({ + path: test.info().outputPath(`console-voice-${width}.png`), + fullPage: true, + }); + }); +} diff --git a/tests/e2e/renderer-binding-startup.spec.ts b/tests/e2e/renderer-binding-startup.spec.ts index 93ca06ec0..bcfb2831c 100644 --- a/tests/e2e/renderer-binding-startup.spec.ts +++ b/tests/e2e/renderer-binding-startup.spec.ts @@ -354,10 +354,10 @@ test("a native Codex draft hides the external Harness command button", async ({ test("a remounted draft keeps explicit Fast through submission, but a new draft starts without it", async ({ page, }) => { - await page.route("http://startup.test/", (route) => + await page.route("https://startup.test/", (route) => route.fulfill({ contentType: "text/html", body: "" }), ); - await page.goto("http://startup.test/"); + await page.goto("https://startup.test/"); await page.evaluate(() => localStorage.setItem( "codexhost.new-thread-preference.v1", diff --git a/tests/e2e/renderer-harness-versions.spec.ts b/tests/e2e/renderer-harness-versions.spec.ts index c29b63b8a..a96456a7c 100644 --- a/tests/e2e/renderer-harness-versions.spec.ts +++ b/tests/e2e/renderer-harness-versions.spec.ts @@ -10,6 +10,13 @@ const { outputFiles } = await build({ stdin: { contents: ` import { createHarnessVersionPanel } from "./packages/renderer-extension/src/settings/harness-version-panel.ts"; + import pi from "./packages/adapters/pi/manifest.json"; + import claude from "./packages/adapters/claude-code/manifest.json"; + import grok from "./packages/adapters/grok/manifest.json"; + import omp from "./packages/adapters/omp/manifest.json"; + import grokIcon from "./packages/adapters/grok/assets/icon.png"; + import ompIcon from "./packages/adapters/omp/assets/icon.svg"; + const plugins = { pi, "claude-code": claude, grok: { ...grok, icon: grokIcon }, omp: { ...omp, icon: ompIcon } }; globalThis.setupLocalizedVersion = (locale, latestVersion, messageCode) => { document.body.replaceChildren(createHarnessVersionPanel( document, rendererSettingsMessages(locale), new AbortController().signal, "hermes", @@ -17,6 +24,7 @@ const { outputFiles } = await build({ currentVersion: "0.21.5+5355.g357f51c.dirty @ 357f51c4", latestVersion, messageCode, updateAvailable: true, canUpdate: false, message: "Use the native updater manually.", + notice: { en: "Needs a clean source checkout", "zh-CN": "要求源码目录无未提交修改" }, }) }, )); }; @@ -24,7 +32,7 @@ const { outputFiles } = await build({ import { createRendererSettingsPageRegistry } from "./packages/renderer-extension/src/settings/core.ts"; import { rendererSettingsMessages } from "./packages/renderer-extension/src/settings/localization.ts"; import { mountRendererSettingsShell } from "./packages/renderer-extension/src/settings/shell.ts"; - globalThis.setupHarnessVersions = () => { + globalThis.setupHarnessVersions = (patched = false) => { const calls = []; let updated = false; let installAttempts = 0; @@ -35,13 +43,26 @@ const { outputFiles } = await build({ hosts: ["local", "remote-test"].map(hostId => ({ hostId, active:hostId === "local", agents: [ {agent:"pi",availability:"ready",error:null}, {agent:"claude-code",availability:"ready",error:null}, + {agent:"grok",availability:"ready",error:null}, {agent:"omp",availability:installedHosts.has(hostId) ? "ready" : "notInstalled",error:null}, - ] })), + ].map(agent => ({ ...agent, plugin: plugins[agent.agent] })) })), }), subscribe: () => () => {}, refresh: async () => {}, installation: async (hostId, agent, action) => { calls.push({hostId,agent,action}); + if (patched && (agent === "grok" || agent === "claude-code")) { + const manager = agent === "grok" ? "grokgod" : "clawgod"; + return { + currentVersion: agent === "grok" ? "1.0.45" : "2.1.263", + latestVersion: "Unknown", latestVersionKind: "unknown", + updateAvailable: false, canUpdate: false, messageCode: "original-installer", + notice: { + en: "Managed by " + manager + ". Use " + manager + "'s updater or installer to preserve the installed patches. The latest patched version is not checked here.", + "zh-CN": "此安装由 " + manager + " 管理。请使用 " + manager + " 的更新程序或安装程序保留补丁;此处不检查最新补丁版本。", + }, + }; + } if (agent === "claude-code") throw {code:-32078}; if (action === "install") { installAttempts++; @@ -115,6 +136,30 @@ test("renders version hints and adapter notes in the selected language", async ( } }); +test("shows patch-manager ownership and unknown latest versions for patched launchers", async ({ + page, +}, testInfo) => { + await page.evaluate(() => { + document.body.replaceChildren(); + ( + globalThis as unknown as { setupHarnessVersions(patched: boolean): void } + ).setupHarnessVersions(true); + }); + for (const [agent, manager, version] of [ + ["grok", "grokgod", "1.0.45"], + ["claude-code", "clawgod", "2.1.263"], + ]) { + await page.locator(`[data-connection-item="${agent}"]`).click(); + const panel = page.locator(".settings-harness-version"); + await expect(panel).toContainText(`当前版本: ${version}`); + await expect(panel).toContainText("最新版本: 未知"); + await expect(panel).toContainText(`此安装由 ${manager} 管理`); + await expect(panel).not.toContainText("操作未完成"); + await expect(panel.getByRole("button", { name: "更新", exact: true })).toBeDisabled(); + await page.screenshot({ path: testInfo.outputPath(`${manager}-versions.png`) }); + } +}); + test("checks automatically and updates explicitly in Connections, with separate remote Host versions", async ({ page, }, testInfo) => { diff --git a/tests/e2e/renderer-live-voice-settings.spec.ts b/tests/e2e/renderer-live-voice-settings.spec.ts new file mode 100644 index 000000000..9524afad9 --- /dev/null +++ b/tests/e2e/renderer-live-voice-settings.spec.ts @@ -0,0 +1,309 @@ +import { expect, test, type Page } from "@playwright/test"; +import { build } from "esbuild"; +import path from "node:path"; +import { tailwindEsbuildPlugin } from "../../packages/renderer-extension/scripts/tailwind-esbuild-plugin.mjs"; + +const executablePath = process.env.CODEXHOST_PLAYWRIGHT_EXECUTABLE_PATH; +if (executablePath) test.use({ launchOptions: { executablePath } }); +const { outputFiles } = await build({ + stdin: { + contents: ` + import { createVoiceSettingsPage } from "./packages/renderer-extension/src/settings/voice-page.ts"; + import { createAppearanceSettingsPage } from "./packages/renderer-extension/src/settings/appearance-page.ts"; + import { rendererSettingsMessages } from "./packages/renderer-extension/src/settings/localization.ts"; + import css from "./packages/renderer-extension/src/settings/shell.css"; + import tailwind from "./packages/renderer-extension/src/settings/tailwind.css"; + const host = document.createElement("div"); host.style.colorScheme = "dark"; document.body.append(host); + const shadow = host.attachShadow({mode:"open"}); + const style = document.createElement("style"); style.textContent = tailwind + css; shadow.append(style); + const surface=document.createElement("section"); surface.className="settings-page"; + const content=document.createElement("main"); content.className="settings-page__content";surface.append(content);shadow.append(surface); + globalThis.setup = (mode) => { + const page = mode === "general" ? createAppearanceSettingsPage(rendererSettingsMessages("en")) : createVoiceSettingsPage("en", async input => { + const response = await fetch("/api/voice-settings", {method:"POST", headers:{"content-type":"application/json"}, body:JSON.stringify(input)}); + const result = await response.json(); if (!response.ok) throw new Error(result.error); return result; + }, {desktop: mode === "desktop"}); + const dispose = page.mount({content}); + globalThis.fixture = {dispose: () => {dispose?.(); content.textContent = "left";}}; + }; + `, + resolveDir: path.resolve(import.meta.dirname, "../.."), + sourcefile: "voice-page-fixture.ts", + loader: "ts", + }, + bundle: true, + format: "iife", + platform: "browser", + target: "es2024", + loader: { ".css": "text", ".png": "dataurl", ".svg": "dataurl" }, + plugins: [tailwindEsbuildPlugin()], + write: false, +}); +const bundle = outputFiles[0]?.text ?? ""; +if (!bundle) throw new Error("Missing voice settings fixture"); +const settings = { + chatExperience: "chatgpt", + chatConnection: "desktop", + workConnection: "desktop", + inject: true, +}; +function initial() { + return { + settings: { ...settings }, + initialized: true, + revision: "1", + activeInject: true, + envOverride: null, + activeEnvOverride: null, + routesActive: true, + activeCall: false, + gatewayReady: true, + restartAvailable: true, + restart: { phase: "idle", error: null }, + }; +} +async function setup(page: Page, mode = "console") { + await page.route("http://voice-settings.test/", (route) => + route.fulfill({ + contentType: "text/html", + body: '', + }), + ); + await page.goto("http://voice-settings.test/"); + await page.addScriptTag({ content: bundle }); + await page.evaluate((value) => Reflect.get(globalThis, "setup")(value), mode); +} + +test("draft edits and Cancel do not save; Apply persists three distinct Chat routes", async ({ + page, +}) => { + const state = initial(); + const writes: unknown[] = []; + await page.route("**/api/voice-settings", (route) => { + const input = route.request().postDataJSON(); + if (input.operation === "set") { + writes.push(input); + state.settings = input.settings; + state.revision = "2"; + } + return route.fulfill({ json: state }); + }); + await setup(page, "desktop"); + const experience = page.getByRole("combobox", { name: "Voice experience", exact: true }); + const connection = page.getByRole("combobox", { name: "Codex connection", exact: true }); + await expect(experience).toHaveValue("chatgpt"); + await expect(connection).toBeDisabled(); + await experience.selectOption("codex"); + await expect(connection).toBeEnabled(); + await connection.selectOption("gateway"); + expect(writes).toEqual([]); + await page.getByRole("button", { name: "Cancel", exact: true }).click(); + await expect(experience).toHaveValue("chatgpt"); + expect(writes).toEqual([]); + await experience.selectOption("codex"); + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await expect(page.getByRole("status")).toContainText("No pending changes"); + expect(writes).toEqual([ + { operation: "set", revision: "1", settings: { ...settings, chatExperience: "codex" } }, + ]); +}); + +test("both settings surfaces load the shared saved values", async ({ page, context }) => { + const state = initial(); + await context.route("**/api/voice-settings", (route) => { + const input = route.request().postDataJSON(); + if (input.operation === "set") { + state.settings = input.settings; + state.revision = "2"; + } + return route.fulfill({ json: state }); + }); + await setup(page, "desktop"); + await page + .getByRole("combobox", { name: "Voice connection", exact: true }) + .selectOption("gateway"); + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await expect(page.getByRole("status")).toContainText("No pending changes"); + const consolePage = await context.newPage(); + await setup(consolePage); + await expect( + consolePage.getByRole("combobox", { name: "Voice connection", exact: true }), + ).toHaveValue("gateway"); +}); + +test("integration changes use one confirmed apply and restart action", async ({ page }) => { + const state = initial(); + const operations: string[] = []; + await page.route("**/api/voice-settings", (route) => { + const input = route.request().postDataJSON(); + if (input.operation !== "get") operations.push(input.operation); + if (input.operation === "set") { + state.settings = input.settings; + state.revision = "2"; + } + if (input.operation === "restart") state.restart.phase = "restarting"; + return route.fulfill({ json: state }); + }); + await setup(page); + await page.getByRole("button", { name: "Advanced", exact: true }).click(); + await page.getByRole("switch", { name: "Enable voice integration", exact: true }).uncheck(); + await page.getByRole("button", { name: "Apply and restart codexhost", exact: true }).click(); + expect(operations).toEqual([]); + await expect(page.getByText(/This may interrupt running tasks/)).toBeVisible(); + await page.getByRole("button", { name: "Apply and restart", exact: true }).click(); + await expect(page.getByRole("status")).toContainText("Restarting codexhost"); + expect(operations).toEqual(["set", "restart"]); +}); + +test("reports a failed restart as saved but inactive and supports retry", async ({ page }) => { + const state = initial(); + let attempts = 0; + await page.route("**/api/voice-settings", (route) => { + const input = route.request().postDataJSON(); + if (input.operation === "set") { + state.settings = input.settings; + state.revision = "2"; + } + if (input.operation === "restart") { + attempts++; + return route.fulfill({ status: 500, json: { error: "restart unavailable" } }); + } + return route.fulfill({ json: state }); + }); + await setup(page); + await page.getByRole("button", { name: "Advanced", exact: true }).click(); + await page.getByRole("switch").uncheck(); + await page.getByRole("button", { name: "Apply and restart codexhost", exact: true }).click(); + await page.getByRole("button", { name: "Apply and restart", exact: true }).click(); + await expect(page.getByRole("alert")).toHaveText("restart unavailable"); + await expect(page.getByRole("status")).toContainText("Saved — restart required"); + expect(state.settings.inject).toBe(false); + expect(attempts).toBe(1); + await expect(page.getByText("Restart complete", { exact: false })).toHaveCount(0); + await expect( + page.getByRole("button", { name: "Apply and restart codexhost", exact: true }), + ).toBeEnabled(); +}); + +test("guards active calls and locks only integration under an environment override", async ({ + page, +}) => { + const state = initial(); + state.activeCall = true; + state.envOverride = "1" as never; + await page.route("**/api/voice-settings", (route) => route.fulfill({ json: state })); + await setup(page); + await page.getByRole("button", { name: "Advanced", exact: true }).click(); + await expect(page.getByRole("switch")).toBeDisabled(); + await page.getByRole("combobox", { name: "Voice experience", exact: true }).selectOption("codex"); + await expect(page.getByRole("button", { name: "Apply changes", exact: true })).toBeDisabled(); + await expect(page.getByRole("status")).toContainText("End your voice call"); +}); + +test("save failure keeps the draft and permits retry", async ({ page }) => { + const state = initial(); + await page.route("**/api/voice-settings", (route) => + route.request().postDataJSON().operation === "set" + ? route.fulfill({ status: 500, json: { error: "read-only setting" } }) + : route.fulfill({ json: state }), + ); + await setup(page); + await page.getByRole("combobox", { name: "Voice experience", exact: true }).selectOption("codex"); + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await expect(page.getByRole("alert")).toHaveText("read-only setting"); + await expect(page.getByRole("combobox", { name: "Voice experience", exact: true })).toHaveValue( + "codex", + ); + await expect(page.getByRole("button", { name: "Apply changes", exact: true })).toBeEnabled(); +}); + +test("rejects a stale edit without overwriting another window", async ({ page }) => { + const state = initial(); + let writes = 0; + await page.route("**/api/voice-settings", (route) => { + if (route.request().postDataJSON().operation === "set") writes++; + return route.fulfill({ json: state }); + }); + await setup(page); + await page.getByRole("combobox", { name: "Voice experience", exact: true }).selectOption("codex"); + state.revision = "other"; + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await expect(page.getByRole("alert")).toContainText("Settings changed in another window"); + expect(writes).toBe(0); + await page.getByRole("button", { name: "Reload", exact: true }).click(); + await expect(page.getByRole("combobox", { name: "Voice experience", exact: true })).toHaveValue( + "chatgpt", + ); +}); + +test("General no longer contains voice preferences", async ({ page }) => { + await setup(page, "general"); + await expect(page.getByRole("combobox")).toHaveCount(0); + await expect(page.getByText("Hide Codex usage warning", { exact: true })).toBeVisible(); +}); + +test("console saves integration for a later launch while Desktop is stopped", async ({ page }) => { + const state = { ...initial(), activeInject: null, restartAvailable: false, routesActive: null }; + const operations: string[] = []; + await page.route("**/api/voice-settings", (route) => { + const input = route.request().postDataJSON(); + if (input.operation !== "get") operations.push(input.operation); + if (input.operation === "set") { + state.settings = input.settings; + state.revision = "2"; + } + return route.fulfill({ json: state }); + }); + await setup(page); + await page.getByRole("button", { name: "Advanced", exact: true }).click(); + await page.getByRole("switch").uncheck(); + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await expect(page.getByRole("status")).toContainText("active setting is unknown"); + expect(operations).toEqual(["set"]); +}); + +for (const width of [1280, 430]) + test(`Voice page fits upstream layout at ${width}px`, async ({ page }) => { + await page.setViewportSize({ width, height: 1100 }); + await page.route("**/api/voice-settings", (route) => route.fulfill({ json: initial() })); + await setup(page); + await expect( + page.getByRole("combobox", { name: "Voice experience", exact: true }), + ).toBeVisible(); + const overflow = await page.evaluate(() => document.documentElement.scrollWidth > innerWidth); + expect(overflow).toBe(false); + await page.screenshot({ path: test.info().outputPath(`voice-${width}.png`), fullPage: true }); + }); + +test("leaving while Apply checks current state prevents a later save or restart", async ({ + page, +}) => { + let getCount = 0; + const mutations: string[] = []; + let finish: () => void = () => {}; + const gate = new Promise((resolve) => { + finish = resolve; + }); + await page.route("**/api/voice-settings", async (route) => { + const input = route.request().postDataJSON(); + if (input.operation === "get") { + getCount++; + if (getCount === 2) await gate; + } else mutations.push(input.operation); + await route.fulfill({ json: initial() }); + }); + await setup(page); + await page.getByRole("combobox", { name: "Voice experience", exact: true }).selectOption("codex"); + const checking = page.waitForRequest( + (request) => + request.url().endsWith("/api/voice-settings") && request.postDataJSON()?.operation === "get", + ); + await page.getByRole("button", { name: "Apply changes", exact: true }).click(); + await checking; + await page.evaluate(() => Reflect.get(globalThis, "fixture").dispose()); + const completed = page.waitForResponse("**/api/voice-settings"); + finish(); + await completed; + await expect(page.locator("main")).toHaveText("left"); + expect(mutations).toEqual([]); +}); diff --git a/tests/e2e/renderer-native-chat-voice-button.spec.ts b/tests/e2e/renderer-native-chat-voice-button.spec.ts new file mode 100644 index 000000000..c17475ae5 --- /dev/null +++ b/tests/e2e/renderer-native-chat-voice-button.spec.ts @@ -0,0 +1,274 @@ +import { expect, test } from "@playwright/test"; +import { build } from "esbuild"; +import path from "node:path"; + +const browserExecutable = process.env.CODEXHOST_PLAYWRIGHT_EXECUTABLE_PATH; +if (browserExecutable) test.use({ launchOptions: { executablePath: browserExecutable } }); + +const { outputFiles } = await build({ + stdin: { + contents: ` + import { installNativeChatVoiceHook } from "./packages/renderer-extension/src/renderer-native-chat-voice.ts"; + const app = { id: Symbol(), __scopeBrand: "AppScope" }; + const route = { id: Symbol(), __scopeBrand: "RouteScope" }; + const composerScope = { id: Symbol(), __scopeBrand: "ComposerScope" }; + const appNode = {}; + const routeNode = {}; + const conversationId = "local-chatgpt:bf2d4b2f-43ee-49df-b802-b55c9fb345d7"; + const selection = { conversationId }; + const chain = new Map([[app.id, appNode], [route.id, routeNode]]); + const composerChain = new Map([[app.id, appNode], [route.id, routeNode], [composerScope.id, { + token: composerScope, + value: { kind: "chatgpt", chatGptSelectionContext: selection }, + }]]); + const composerContext = { displayName: "ComposerScope" }; + const outerChain = new Map(chain); + outerChain.set(composerScope.id, { token: composerScope, value: { kind: "new", entrypoint: "home" } }); + const backend = {}; + const enabled = {}; + let runtime = false; + const scope = { scope: route, node: routeNode, chain, + get: (selector) => selector === enabled ? runtime : null, + set: (selector, value) => { if (selector === enabled) runtime = value; }, + }; + const form = document.querySelector("form"); + const owner = { + memoizedProps: { conversationId, composerSelectionContext: selection, + isWorkConversation: false, isTemporaryChat: false, shouldBlockExternalEgress: false }, + memoizedState: { memoizedState: { current: scope }, next: null }, + dependencies: { firstContext: { + context: { displayName: "AppScope" }, memoizedValue: chain, next: { + context: composerContext, memoizedValue: outerChain, next: null, + }, + } }, return: null, + }; + const provider = { tag: 10, type: composerContext, + memoizedProps: { value: composerChain }, return: owner }; + form.__reactFiber$fixture = { return: provider }; + globalThis.chatProvider = provider; + globalThis.chatComposerChain = composerChain; + globalThis.nativeVoiceState = { isVoiceModeActive: false, isWorkMode: false, conversationId }; + const initial = { B1n: backend, CDr: composerScope, MOr: route, aT: enabled, + oT() {}, SPn: () => ({ + getState: () => globalThis.nativeVoiceState, subscribe: () => () => {}, + }), + }; + const shared = { Qst: app, FBt: () => ({ createElement() {} }), + PNt: () => ({ createRoot: () => ({ render() {}, unmount() {} }) }), + }; + globalThis.__codexhostCpaLiveVoiceHookV1 = { enabled: true, desktopVersion: "26.1016.62342", auditedDesktopVersion: "26.1016.62342", auditedAppInitialAsset: "app-initial-dddef47d6ef6.js" }; + globalThis.voiceStarts = 0; + globalThis.voiceHook = installNativeChatVoiceHook({ importModule: async (url) => + url.includes("app-initial") ? initial : url.includes("app-shared") ? shared : + url.includes("start-voice") ? { t() {}, n: async () => { globalThis.voiceStarts++; } } : + { VoiceInstantConnectCacheController() {} }, + }); + `, + resolveDir: path.resolve(import.meta.dirname, "../.."), + sourcefile: "native-chat-voice-button-fixture.ts", + loader: "ts", + }, + bundle: true, + format: "iife", + platform: "browser", + target: "es2024", + write: false, +}); +const browserBundle = outputFiles[0]?.text; +if (!browserBundle) throw new Error("Chat voice button fixture was not generated"); + +for (const { theme, dictate } of ["light", "dark"].flatMap((theme) => + [false, true].map((dictate) => ({ theme, dictate })), +)) { + test(`places Chat Voice to the left of Send ${dictate ? "with" : "without"} Dictate in ${theme}`, async ({ + page, + }) => { + await page.route("http://codexhost.test/", (route) => + route.fulfill({ + contentType: "text/html", + body: ` + + + +
+

Ready when you are.

+
+
+ +
Ask ChatGPT
+
+ + ${dictate ? '' : ""} + +
+
0 +
+ + +
`, + }), + ); + await page.route("http://codexhost.test/assets/**", (route) => + route.fulfill({ contentType: "text/javascript", body: "" }), + ); + await page.goto("http://codexhost.test/"); + await page.evaluate(() => { + const form = document.querySelector("form"); + const output = document.querySelector("output"); + if (!form || !output) throw new Error("Chat form fixture is missing"); + form.addEventListener("submit", (event) => { + event.preventDefault(); + output.textContent = "submitted"; + }); + }); + await page.addScriptTag({ content: browserBundle }); + const voice = page.getByRole("button", { name: "Start Voice", exact: true }); + await expect(voice).toHaveCount(1); + await expect(voice).toBeEnabled(); + await expect(voice).toHaveCSS("opacity", "1"); + await expect(voice).toHaveCSS("cursor", "pointer"); + await expect(voice.locator("svg")).toHaveCSS( + "color", + theme === "dark" ? "rgb(17, 17, 17)" : "rgb(238, 238, 238)", + ); + await expect(voice.locator("svg path")).toHaveCSS( + "stroke", + theme === "dark" ? "rgb(17, 17, 17)" : "rgb(238, 238, 238)", + ); + const position = await voice.evaluate((button) => { + const send = button.nextElementSibling; + if (!send) throw new Error("Voice is not followed by Send"); + const a = button.getBoundingClientRect(); + const b = send.getBoundingClientRect(); + return { + send: send.getAttribute("aria-label"), + sameParent: button.parentElement === send.parentElement, + gap: b.left - a.right, + centerDelta: Math.abs(a.top + a.height / 2 - (b.top + b.height / 2)), + }; + }); + expect(position.send).toBe("Send"); + expect(position.sameParent).toBe(true); + const referenceGap = await page + .getByRole("button", { name: "Reference microphone" }) + .evaluate((mic) => { + const end = mic.nextElementSibling; + if (!end) throw new Error("Reference end control is missing"); + return end.getBoundingClientRect().left - mic.getBoundingClientRect().right; + }); + expect(position.gap).toBeCloseTo(referenceGap, 1); + expect(position.centerDelta).toBeLessThan(1); + await page.screenshot({ + path: `.codexhost/update-impact/26.1016.62342/chat-voice-button-${theme}-${dictate ? "with" : "without"}-dictate.png`, + }); + await voice.click(); + await expect.poll(() => page.evaluate(() => Reflect.get(globalThis, "voiceStarts"))).toBe(1); + await expect(page.locator("output")).toHaveText("0"); + + const headline = page.locator("[data-headline]"); + const composerBox = await page.locator("form").boundingBox(); + await expect(headline).toBeVisible(); + await page.evaluate(() => { + Object.assign(Reflect.get(globalThis, "nativeVoiceState"), { + isVoiceModeActive: true, + isWorkMode: true, + }); + }); + await expect(voice).toBeDisabled(); + await expect(headline).toBeVisible(); + await page.evaluate(() => { + Reflect.get(globalThis, "nativeVoiceState").isWorkMode = false; + }); + await expect(headline).toBeHidden(); + expect(await page.locator("form").boundingBox()).toEqual(composerBox); + await expect(voice).toBeDisabled(); + await expect(voice).toHaveCSS("opacity", "0.5"); + // Leaving the composer restores the greeting even while native voice is active. + await page.locator("form").evaluate((form) => { + form.setAttribute("hidden", ""); + }); + await expect(headline).toBeVisible(); + await page.locator("form").evaluate((form) => { + form.removeAttribute("hidden"); + }); + await expect(headline).toBeHidden(); + await page.evaluate(() => { + Reflect.get(globalThis, "nativeVoiceState").isVoiceModeActive = false; + }); + await expect(headline).toBeVisible(); + await expect(voice).toBeEnabled(); + + // An empty nearer provider must hide Voice rather than reuse the outer selection. + await page.evaluate(() => { + Reflect.get(globalThis, "chatProvider").memoizedProps.value = new Map(); + }); + await expect(voice).toHaveCount(0); + await page.evaluate(() => { + Reflect.get(globalThis, "chatProvider").memoizedProps.value = Reflect.get( + globalThis, + "chatComposerChain", + ); + }); + await expect(voice).toHaveCount(1); + + // Native draft updates can replace the action wrapper while retaining the form. + await page.getByRole("button", { name: "Send", exact: true }).evaluate((send) => { + const wrapper = document.createElement("span"); + wrapper.className = "actions"; + wrapper.append(send.cloneNode(true)); + send.replaceWith(wrapper); + }); + await expect + .poll(() => voice.evaluate((button) => button.nextElementSibling?.getAttribute("aria-label"))) + .toBe("Send"); + await expect(voice).toHaveCount(1); + await voice.evaluate((button) => { + if (button.parentElement) button.parentElement.style.gap = "8px"; + }); + await expect + .poll(() => + voice.evaluate((button) => { + const send = button.nextElementSibling; + if (!send) throw new Error("Send is missing after replacement"); + return send.getBoundingClientRect().left - button.getBoundingClientRect().right; + }), + ) + .toBeCloseTo(referenceGap, 1); + await page.getByRole("button", { name: "Send", exact: true }).evaluate((send) => { + (send as HTMLButtonElement).disabled = false; + }); + await page.getByRole("button", { name: "Send", exact: true }).click(); + await expect(page.locator("output")).toHaveText("submitted"); + + // A streaming Stop action is not a Send insertion target. + await page.getByRole("button", { name: "Send", exact: true }).evaluate((send) => { + send.setAttribute("type", "button"); + send.setAttribute("aria-label", "Stop"); + }); + await expect(voice).toHaveCount(0); + await page.getByRole("button", { name: "Stop", exact: true }).evaluate((stop) => { + stop.setAttribute("type", "submit"); + stop.setAttribute("aria-label", "Send"); + }); + await expect(voice).toHaveCount(1); + await page.evaluate(() => { + Reflect.get(globalThis, "nativeVoiceState").isVoiceModeActive = true; + }); + await expect(headline).toBeHidden(); + await page.evaluate(() => Reflect.get(globalThis, "voiceHook").dispose()); + await expect(voice).toHaveCount(0); + await expect(headline).toBeVisible(); + }); +} diff --git a/tests/e2e/renderer-settings-page.spec.ts b/tests/e2e/renderer-settings-page.spec.ts index 6d8cb395c..ad640340a 100644 --- a/tests/e2e/renderer-settings-page.spec.ts +++ b/tests/e2e/renderer-settings-page.spec.ts @@ -41,6 +41,10 @@ const { outputFiles } = await build({ // returning from another destination restores the previous conversation. const nativeNavigations = []; history.replaceState({}, "", "/thread/settings-test"); + document.querySelector('[data-app-shell-titlebar] button').addEventListener("click", () => { + nativeNavigations.push("back"); + history.pushState({}, "", "/thread/previous"); + }); document.body.addEventListener("click", (event) => { const destination = event.target.closest("[data-sidebar-destination]"); if (!destination) return; @@ -78,6 +82,8 @@ if (!bundle) throw new Error("Settings page fixture bundle missing"); // [data-selected]::before and expose the current destination via aria-current. const nativeShell = ` +
+ +