diff --git a/.env.example b/.env.example index 87f7a6b..f3c94ea 100644 --- a/.env.example +++ b/.env.example @@ -7,6 +7,6 @@ X_SOURCE=mock HOST=127.0.0.1 PORT=8787 -# Optional now; required before exposing the gateway unless Cloudflare Access is used. +# Required before exposing the gateway. Generate a private 43-character base64url key. GATEWAY_BEARER_TOKEN= ALLOWED_ORIGINS=http://127.0.0.1:5173,http://localhost:5173 diff --git a/AGENTS.md b/AGENTS.md index ca937b4..6f5c388 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -32,7 +32,7 @@ switch package managers without an explicit migration task. ## Non-negotiable boundaries - Never commit or print X cookies, browser profiles, access keys, Cloudflare - credentials, relay catalogs, `.env` files, generated `dist/`, or `.ehpk` files. + credentials, relay catalogues, `.env` files, generated `dist/`, or `.ehpk` files. - Keep `twitter_api_safe_relay` loopback-only. Only the bearer-authenticated Doge gateway may be exposed through a tunnel. - Public G2 builds must not contain a maintainer Gateway URL or access key. Users @@ -40,16 +40,18 @@ switch package managers without an explicit migration task. - Validate external data at boundaries using the shared contract. Do not pass raw X responses, cookies, or internal headers to the G2 client. - A build does not update a running production Gateway. Restart it, then verify - authenticated and tokenless behavior before claiming deployment success. -- Simulator, package, Hub upload, and physical-glasses behavior are separate + authenticated and tokenless behaviour before claiming deployment success. +- Simulator, package, Hub upload, and physical-glasses behaviour are separate evidence. State clearly which one was actually verified. ## Working agreement - Preserve unrelated changes and keep each change reviewable. -- Add or update regression tests for behavior changes. Prefer observable +- Write all maintained documentation in British English; follow the language + guidance in `docs/development.md`. +- Add or update regression tests for behaviour changes. Prefer observable contract tests over implementation-detail assertions. -- Run focused checks while iterating, then `npm run verify` before finalizing. +- Run focused checks while iterating, then `npm run verify` before finalising. - For release-affecting changes, run `npm run verify:release` and report the package hash; never stage the generated package. - Record lasting architecture or operational knowledge in the owning document diff --git a/README.md b/README.md index d5cf32b..49b674f 100644 --- a/README.md +++ b/README.md @@ -1,223 +1,179 @@ -# Doge for Even G2 - -Even G2でXのHome、Following、Bookmarks、threadを読み、Like、Repost、Bookmarkを切り替えるTypeScript製アプリです。 - -X API keyは使いません。Mac上のログイン済みブラウザを[`twitter_api_safe_relay`](https://github.com/fa0311/twitter_api_safe_relay)が操作し、このプロジェクトのgatewayが必要な結果だけを固定DTOへ変換してEven G2へ渡します。Xが`promotedMetadata`を付けたtimeline entryは広告として正規化前に除外します。 - -> [!IMPORTANT] -> `twitter_api_safe_relay`をインターネットへ直接公開しないでください。Tunnelへ接続するのはBearer認証付きgatewayだけです。write操作はLike、Repost、Bookmarkの有効化・解除だけに制限し、投稿、返信、Follow、削除などは公開しません。 - -## 構成 - -```text -Even G2 / iPhone WebView - │ authenticated GET / PUT / DELETE - ▼ -Doge gateway :8787 - │ allowlist済み10操作だけ - ▼ -twitter_api_safe_relay :6900 (localhost only) - │ - ▼ -ログイン済みX browser profile -``` - -- `apps/g2` — Even Hub SDK、576×288 glasses UI、投稿者icon・投稿画像、iPhone companion UI -- `apps/gateway` — Hono/Node.js gateway、固定reaction routes、Xレスポンス正規化、avatar・投稿画像proxy -- `packages/contracts` — frontendとgatewayで共有するZod schema -- `scripts` — relay catalog同期、preview、maintainer用deployment - -## Agent harness - -このrepositoryはCodex、Claude Code、GitHub Copilotなど特定vendorに依存しないrepository-native harnessを備えています。agentは最初に[`AGENTS.md`](AGENTS.md)と[`docs/INDEX.md`](docs/INDEX.md)を読み、変更対象に応じてpackage-local `AGENTS.md`へ進みます。設計意図は[`docs/agent-harness.md`](docs/agent-harness.md)にあります。 - -検証の定義はagent用proseではなくnpm scriptsとCIへ集約しています。 - -```bash -npm run verify # format、repo invariants、types、tests、build -npm run verify:release # 上記+production EHPK+artifact境界検査 -``` - -GitHub Actionsもclean checkoutから同じ`npm run verify:release`を実行します。長期的に残す未完了事項だけを[`Backlog.md`](Backlog.md)へ記録し、session logや一時planには使いません。 - -## 操作 - -起動直後はHome、Following、Bookmarksのview選択画面です。 - -| 場所 | 入力 | 動作 | -| ----------- | ---------- | ---------------------------------- | -| view選択 | scroll | Home / Following / Bookmarks選択 | -| view選択 | tap | 選択したviewを開く | -| view選択 | double tap | Dogeを終了 | -| 投稿view | 下スワイプ | 本文をnative scroll / 読了後に次へ | -| 投稿view | 上スワイプ | 本文をnative scroll / 境界で前へ | -| 投稿view | tap | 右側のaction menuを開く | -| 投稿view | double tap | view選択へ戻る | -| action menu | scroll | Like / Repost / Bookmark / thread | -| action menu | tap | 選択を実行し、成功後menuを閉じる | - -G2のscrollはcontentを引っ張るnatural/inverse方式ではありません。投稿viewでは、進みたい方向へ指をslideします(下方向で次の本文page/post、上方向で前の本文page/post)。view選択とaction menuはG2 native listのscrollに従います。 - -長い本文は1行ごとの再描画をせず、G2 firmwareの`TextContainer` overflow scrollで連続的に読みます。本文は`textContainerUpgrade`の上限である最大2000文字のchunkへ分割し、文字は省略しません。layout rebuild時はSDKの1000文字上限内でcontainerを作成した直後、完全なchunkへin-place更新します。複数chunkのpostでは左下にchunk位置、右下にtimeline内のpost位置を表示します。G2画面内には常設の操作guideを置かず、その領域も本文とview選択listに使います。画像付きpostでは、本文末尾の文書blockとして1〜4枚を縦横比を維持したgridで表示します。本文が短いほど画像領域を大きく取り、各slotは取得中だけ独立したloading placeholderを表示します。Galleryは画像だけを最大表示する別modeです。動画・animated GIFは静止posterを再生マーク付きで表示し、動画データの取得・再生は行いません。 - -取得済みmediaのBlobとG2用に変換済みのPNG tileはWebView memory内のLRU cacheへ短期間保持します。同じpostやGallery画像へ戻った場合はnetwork取得・decode・再変換を省略します。ただしEven SDKには眼鏡側の画像cacheをIDで再利用するAPIがないため、page rebuild後のBLE再送自体は必要です。icon、投稿者画像、投稿画像はG2 bridgeへencoded PNG/JPEGのbyte列として順番に渡します。高速にpostやviewを切り替えた場合、古いavatar取得結果は破棄し、最新renderだけをimage containerへ反映します。 - -`apps/g2/public/doge-icon.png` はiPhone側のweb app iconと、view選択後の初回loading画面で使います。特定の写真やTwitter/Xの旧logoを複製しない、このproject用のoriginal illustrationです。Even Hubの掲載iconには同じファイルを手動でuploadしてください。詳細は同directoryの `doge-icon.LICENSE.md` を参照してください。 - -## 必要環境 - -- Node.js 22以降(agentとCIの標準versionは`.node-version`に固定) -- Even Hub対応のEven Appとpairing済みEven G2 -- live X利用時のみ、別途起動した`twitter_api_safe_relay`とログイン済みbrowser profile - -## すぐ試す(mock) - -mockにはXへの接続が不要です。 - -```bash -npm install -npm run dev -``` - -gatewayは`http://127.0.0.1:8787`、Viteは`http://127.0.0.1:5173`で起動します。iPhone側画面はVite URL、Even Hub simulatorにも同じVite URLを指定してください。 - -初回はphone画面のGateway settingsへ`http://127.0.0.1:8787`と43文字の開発用access keyを入力して`Save and test connection`を押します。認証に成功した組だけが保存され、以後は自動復元されます。 - -検証とpackaging: - -```bash -npm run verify -npm run pack:g2 -``` - -生成物は`apps/g2/doge.ehpk`です。`.ehpk`とbuild成果物はGit管理しません。 - -## live X relayへ切り替える - -このprojectはHeliumとは別のPlaywright Chromium profileを使います。初回だけ専用browserでXへログインし、以後は`var/relay-profile`に保存されたsessionを使います。profileにはX cookieとLocal Storageが入るため、directory全体をGitから除外しています。 - -最初に専用Chromiumをinstallします。 - -```bash -npx playwright install chromium +

+ Doge: a Shiba Inu wearing smart glasses +

+ +

Doge

+ +

An X reader for Even G2. Your timeline, a glance away.

+ +

+ Quick start · + Reader guide · + Development · + Deployment +

+ +--- + +Doge brings Home, Following, Bookmarks, threads and profiles to the Even G2's +576 × 288 display. Read full posts, browse images and manage reactions with the +glasses' touch controls. An iPhone companion handles gateway pairing. + +For use on the glasses, you need an Even G2 paired with an Even Hub-compatible +Even App on your iPhone. You can try the reader locally in the simulator without +glasses or an X account. + +Live data comes from your own signed-in X browser session through a self-hosted +gateway. No X API key is required. The Mac running the gateway and browser must +remain online while you read. + +## Features + +- **Three feeds, one reader.** Switch between Home, Following and Bookmarks; + open a thread or an author's profile from any post. +- **Full text.** Read long posts with firmware-native scrolling. Longer text is + split into bounded chunks without truncation. +- **Images in context.** View up to four images alongside a post or open the + gallery for a larger view. Videos and animated GIFs appear as still posters. +- **Reactions on the glasses.** Like, repost and bookmark, or undo each action. +- **Your own gateway.** Pair an HTTPS gateway and access key on your device. + Public packages contain no preset server or maintainer credentials. +- **Local development.** Explore the reader with deterministic mock data before + connecting an X account. + +> [!NOTE] +> Physical-device validation is tracked in [Backlog.md](Backlog.md), including +> background recovery, phone locking and use away from home. A successful build +> or simulator run does not establish those behaviours. + +## Quick start + +This path runs the source checkout with mock data in the Even Hub simulator on +**macOS**. You need Git, **Node.js 24.15.0** (see [`.node-version`](.node-version)) +and **npm 11.12.1** (pinned in [`package.json`](package.json)). `npm ci` installs +the simulator alongside the app's other development dependencies. Ports 8787 +and 5173 must be available. + +Clone the repository and install its dependencies: + +```sh +git clone https://github.com/hauntedfail/Doge.git +cd Doge +npm ci ``` -Safe Relayと専用browserを起動します。 +Once installation succeeds, create a temporary development key and start the +mock gateway and companion in the same terminal: -```bash -npm run relay:login +```sh +export GATEWAY_BEARER_TOKEN="$( + node -p 'require("node:crypto").randomBytes(32).toString("base64url")' +)" +printf '%s' "$GATEWAY_BEARER_TOKEN" | pbcopy +X_SOURCE=mock npm run dev ``` -表示されたChromiumでXへログインし、`https://x.com/home`が表示されたらwindowを開いたままにします。別terminalで状態を確認できます。 +This starts the mock gateway at `http://127.0.0.1:8787` and the companion at +`http://127.0.0.1:5173`. Keep this terminal running. The `pbcopy` command puts the +key on the macOS clipboard without printing it. Keep it private; it is not an +X API key. -```bash -npm run relay:check -``` - -続いて現在のquery IDとfeature flagsを取得し、gatewayをrelay modeで起動します。 +In a second terminal, change to the same `Doge` directory and launch the installed +simulator: -```bash -npm run relay:sync -X_SOURCE=relay npm run dev:gateway -npm run dev:g2 +```sh +npx --no-install evenhub-simulator http://127.0.0.1:5173 ``` -Safe Relayは`127.0.0.1:6900`、gatewayは`127.0.0.1:8787`を使います。`3000`はこのMacのTraumaが使用中なので使いません。Safe Relayのbrowserを閉じるか`Ctrl-C`するとrelayも停止します。 +In the simulator's companion window: -relay catalogはX側の変更に追従するversion lockです。`var/requests.ndjson`はaccount由来のIDやcursorを含み得るため、`.gitignore`で除外しています。gatewayは起動後もcatalogを読み直すため、同期後の再buildは不要です。 +1. Open **Gateway settings**, enter `http://127.0.0.1:8787` and paste the key. +2. Select **Save and test connection**. The confirmation should read + **Connected to http://127.0.0.1:8787. Settings were saved.** +3. Select **Home** using the companion's test controls or the simulator's glasses + controls. The first post should read: -### 認証付き実機preview + > Doge is running in mock mode with deterministic timeline data. -Safe Relayへログイン済みで、`npm run build`と`npm run relay:sync`が完了していれば、実Xデータ用の一時previewを起動できます。 - -```bash -npm run preview:live -``` +A regular browser can show the companion layout, but it cannot complete pairing: +settings wait for the Even SDK bridge supplied by the simulator or Even App. +If **Save and test connection** stays disabled, check that you opened the URL in +the simulator. -このcommandは256-bitの一時tokenを生成し、Bearer認証を必須にしたscoped gatewayと使い捨てCloudflare Quick Tunnelを起動します。tokenはterminalへ出力せず、権限`600`の一時QR画像にだけ埋め込みます。URL fragmentはCloudflareへ送られません。phone画面でQRと同じHTTPS originをGateway URLとして保存・検証すると、以後のAPI requestへAuthorization headerが付与されます。`Ctrl-C`でgatewayとTunnelを終了し、QR画像を削除してtokenを失効させます。 - -Quick Tunnelは実機開発専用で、可用性保証や固定URLはありません。本番運用では`doge.h1ka.ru`のnamed Tunnelを使います。 - -## Public buildとGateway pairing - -public buildは特定のGateway URLを含みません。Even Hub manifestはuserがphone画面で選ぶ任意のHTTPS serviceへの通信を許可し、Dogeは次の順序でpairingします。 - -1. userがGatewayのHTTPS URLと43文字のaccess keyを入力 -2. DogeがBearer key付きで`GET /api/v1/session`を呼ぶ -3. 認証済みDoge Gateway protocol v1応答を確認 -4. 成功したURLとkeyの組だけをEven App SDKのdevice local storageへ保存 -5. 次回起動時は保存値を復元し、同じsession endpointで再確認 - -未設定時にtimelineやmedia requestは送信しません。Gateway URLの入力draftは接続成否にかかわらずEven SDK storageへ別keyで保持し、失敗後や再起動後も入力欄へ復元します。access keyは認証成功後に専用のEven SDK storage keyへ保存し、書き戻し確認が成功してから接続済みとして扱います。起動時はdevice storageの復元が完了するまで設定操作を無効化し、保存済みURL+keyがあれば再入力なしで起動します。keyを変更するときだけ新しい値を入力し、空欄なら保存済みkeyを維持します。password欄が空でも`Saved access key is active`と表示されていればkeyは非表示のまま有効です。`Forget access key`はURLを入力欄へ残したままkeyを削除します。旧WebView storageまたは旧形式に保存済みのURL+keyも初回にdevice storageへ移行します。 - -公開packageを作る場合: - -```bash -npm run verify -npm run pack:g2:production -``` +To stop, quit the simulator and press `Ctrl-C` in the development terminal. +Restarting from the same shell retains the development key; if you generate a +new key, update the companion pairing. -`apps/g2/app.production.json`はGit管理し、network whitelistを`https://`に固定します。build時環境変数、配布package、frontend sourceのいずれにも運用者のGateway URLやaccess keyを埋め込みません。Gateway実装の互換contractは[`docs/gateway-protocol.md`](docs/gateway-protocol.md)です。 +Loopback addresses work only on the machine running Doge. For an iPhone and +physical glasses, use the [authenticated preview](docs/development.md#live-device-preview) +or a [deployed gateway](docs/deployment.md). -各userはこのcontractを実装したGatewayをHTTPSで公開し、phone画面からpairingします。X cookieはGateway hostから出ません。 +## How it works -## Maintainer deployment - -このrepositoryのmaintainer用named Tunnelは現在`https://doge.h1ka.ru`を使います。これはserver運用scriptの設定であり、Doge public buildのdefault接続先ではありません。 - -Cloudflare Tunnelで公開するのはgatewayの`127.0.0.1:8787`だけです。Safe Relayのport `6900`はTunnelへ接続しません。 - -初回にDoge access keyを生成し、クリップボードへコピーします。key自体はterminalへ表示せず、`var/doge-access-key`へ権限`600`で保存します。 - -```bash -npm run production:key +```text +Even G2 + iPhone companion + │ HTTPS + bearer authentication + ▼ + Doge gateway :8787 + │ Allowlisted operations over loopback + ▼ + Safe Relay :6900 + │ + ▼ + Dedicated X browser session ``` -Even AppのDoge画面で`https://doge.h1ka.ru`とkeyを入力し、`Save and test connection`を押します。 - -Safe Relayが起動中であることを確認して本番gatewayとnamed Tunnelを起動します。 +[`twitter_api_safe_relay`](https://github.com/fa0311/twitter_api_safe_relay) +uses the signed-in browser session. Doge validates and normalises the results +into a shared wire contract before sending them to the glasses. Timeline entries +marked by X with `promotedMetadata` are removed before normalisation. -```bash -npm run production:start +> [!IMPORTANT] +> Keep Safe Relay on loopback. Expose only the bearer-authenticated Doge gateway +> through a tunnel. Supported write actions are limited to enabling and removing +> likes, reposts and bookmarks. See the [security model](docs/security.md). + +## Documentation + +| Guide | Contents | +| -------------------------------------------- | --------------------------------------------------------- | +| [Reader guide](docs/reader-guide.md) | Controls, text, images and saved pairing | +| [Development](docs/development.md) | Local setup, live X data, preview and verification | +| [Deployment](docs/deployment.md) | HTTPS hosting, production operation and Even Hub packages | +| [Gateway protocol](docs/gateway-protocol.md) | Pairing handshake, routes and compatibility | +| [Security](docs/security.md) | Authentication, relay restrictions and data handling | +| [Documentation index](docs/INDEX.md) | Architecture sources and documentation ownership | + +## Contributing + +Start with [`AGENTS.md`](AGENTS.md) and the [documentation index](docs/INDEX.md). +Use the nearest package's instructions when changing the G2 app or gateway. +Write documentation in British English and add regression coverage for behaviour +changes. + +For a concrete place to help, see the outstanding device checks in +[`Backlog.md`](Backlog.md). When reporting a device issue, include the app version, +Even App version, reproduction steps and whether it occurs in the simulator or +on physical glasses. Keep access keys and X session data out of reports. + +```sh +npm run verify # Formatting, repository checks, types, tests and builds +npm run verify:release # Also package and check the production EHPK ``` -Gateway sourceを更新して`npm run build`した場合、実行中の`production:start`も再起動してください。Node processは起動時に読み込んだ`dist/server.js`を使い続けるため、buildだけではprofileなどの新routeは反映されません。 - -外出中もMacの電源・ネット接続、Safe Relay、`production:start`を維持してください。現在の構成ではiPhoneとG2だけでXへ直接接続するわけではなく、自宅Macがbackendです。 - -## Private buildとBeta build - -推奨は**Beta build**です。Betaは公開Store審査なしで、自分のEven accountだけをtester groupへ追加できます。公開版と同じlifecycleで動き、phone lockやbackground遷移も含めた外出利用を試せます。Private buildも自分だけでinstallできますが、配布できず、lifecycleは公開版と完全には同じではありません。 - -1. `npm run verify && npm run pack:g2:production` -2. Even HubでBeta tester groupを作り、自分のEven account emailを追加 -3. `apps/g2/doge.ehpk`をbuildとしてuploadし、そのgroupへpush -4. iPhoneのEven Appで`Me → Beta tester → Doge → Install` -5. glasses homeからDogeを起動し、phone画面でaccess keyを一度だけpair - -公開versionはrelease後に同じversion番号で差し替えられないため、Betaでnative settings復元、background復帰、実機profile表示を確認してから提出します。掲載iconにはrepository同梱のoriginal assetを使えます。 - -公開審査が発生するのはStoreへsubmissionする段階です。詳細はEven Hub公式の[Beta testing](https://hub.evenrealities.com/docs/test/beta-testing)と[Private testing](https://hub.evenrealities.com/docs/test/private-testing)を参照してください。 - -## Security defaults - -- public client routeは認証確認、timeline、thread、profile、avatar・投稿画像の`GET`と、Like、Repost、Bookmark専用の`PUT`/`DELETE`のみ -- feed、cursor、post IDをschema検証 -- relay base URLはloopback HTTPのみ許可 -- relay operationは4つのread操作とFavorite/Unfavorite、Create/Delete Retweet、Create/Delete Bookmarkの6つだけ -- XのGraphQL errorをHTTP 200でも失敗として扱う -- upstream timeout 15秒、response上限5 MB -- avatar proxyは`pbs.twimg.com/profile_images`のHTTPS画像だけを許可し、redirect禁止、5秒timeout、512 KB上限、画像content-type必須 -- 投稿画像proxyは`pbs.twimg.com`の写真および動画posterの固定pathだけを許可し、redirect禁止、5秒timeout、4 MB上限、JPEG/PNG/WebPのcontent-typeとfile signature一致を必須化。`video.twimg.com`やMP4/HLSは取得しない -- responseは固定DTOへ変換し、X cookieや内部headerをclientへ返さない -- production responseは`no-store`、security headers付き -- 本番gatewayはBearer token必須。tokenを`.ehpk`へ埋め込まず、各iPhoneで一度だけpair -- installed WebViewのoriginが変わってもBearer認証を必須にしたままCORS responseを返す - -## Gitに入れないもの +The same release verification runs in CI. See [development](docs/development.md#verification) +for focused commands and [agent harness design](docs/agent-harness.md) for the +repository's approach to coding agents. -`.env*`、Cloudflare credentials、browser profile、relay catalog、build output、coverage、`.ehpk`を除外しています。X cookieやCloudflare tokenをrepository内へ置かないでください。 +| Path | Responsibility | +| ------------------------------------------ | ---------------------------------------------------------------- | +| [`apps/g2`](apps/g2) | Even Hub app, glasses display and iPhone companion | +| [`apps/gateway`](apps/gateway) | Hono/Node.js gateway, X response normalisation and image proxies | +| [`packages/contracts`](packages/contracts) | Shared Zod request and response schemas | +| [`scripts`](scripts) | Relay setup, preview, production operation and repository checks | -## License +## Licence -[GNU Affero General Public License v3.0 only](LICENSE)。Trauma projectと同じライセンスです。 +[GNU Affero General Public License v3.0 only](LICENSE). +The Doge illustration is an original project asset; see its +[asset licence](apps/g2/public/doge-icon.LICENSE.md). diff --git a/apps/g2/AGENTS.md b/apps/g2/AGENTS.md index 1944570..f304ef6 100644 --- a/apps/g2/AGENTS.md +++ b/apps/g2/AGENTS.md @@ -6,8 +6,8 @@ The glasses UI is rendered through the Even Hub SDK, not through the phone DOM. ## Read first - Root `AGENTS.md` and `docs/INDEX.md` -- `README.md` sections “操作”, “Public buildとGateway pairing”, and - “Private buildとBeta build” +- `docs/reader-guide.md` for controls and pairing +- `docs/deployment.md` for packaging and Even Hub testing - `app.json` and `app.production.json` before changing permissions or versions ## Invariants diff --git a/apps/gateway/AGENTS.md b/apps/gateway/AGENTS.md index 7d3fd54..0121e74 100644 --- a/apps/gateway/AGENTS.md +++ b/apps/gateway/AGENTS.md @@ -7,7 +7,7 @@ G2 client and `twitter_api_safe_relay` on loopback. - Root `AGENTS.md` and `docs/INDEX.md` - `docs/gateway-protocol.md` -- `README.md` sections “Security defaults” and “Maintainer deployment” +- `docs/security.md` and `docs/deployment.md` ## Invariants @@ -29,7 +29,7 @@ G2 client and `twitter_api_safe_relay` on loopback. - Focused route/security test: `npm test -- apps/gateway/src/.test.ts` - Type check: `npm run check --workspace @even-g2-x-reader/gateway` - Build: `npm run build --workspace @even-g2-x-reader/gateway` -- Run root `npm run verify` before finalizing. +- Run root `npm run verify` before finalising. After production source changes, rebuilding is insufficient: restart `npm run production:start`, then verify authenticated `/api/v1/session` returns diff --git a/docs/INDEX.md b/docs/INDEX.md index 56c096c..7112330 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -5,8 +5,11 @@ smallest relevant reading set instead of loading the entire repository history. ## Product and architecture -- [`README.md`](../README.md) — product behavior, input model, development, - pairing, deployment, distribution, and security defaults. +- [`README.md`](../README.md) — product overview, quick start and contribution + entry points. +- [`reader-guide.md`](reader-guide.md) — controls, text, images and pairing. +- [`security.md`](security.md) — authentication, relay restrictions and data + handling. - [`gateway-protocol.md`](gateway-protocol.md) — public Gateway protocol and compatibility contract. - [`../apps/g2/AGENTS.md`](../apps/g2/AGENTS.md) — G2/WebView implementation @@ -18,11 +21,10 @@ smallest relevant reading set instead of loading the entire repository history. ## Operations -- README “すぐ試す(mock)” — deterministic local startup. -- README “live X relayへ切り替える” — login-bound Safe Relay workflow. -- README “Maintainer deployment” — named Tunnel production operation. -- README “Private buildとBeta build” — Even Hub distribution and device test - boundaries. +- [`development.md`](development.md) — deterministic local startup, live X + relay setup, authenticated preview and verification. +- [`deployment.md`](deployment.md) — HTTPS hosting, maintainer operation, + Even Hub distribution and device test boundaries. - [`.env.example`](../.env.example) — non-secret configuration names and safe loopback defaults. @@ -31,7 +33,7 @@ smallest relevant reading set instead of loading the entire repository history. - `npm run verify` — formatting, repository invariants, TypeScript, all tests, and all builds. This is the required local and CI baseline. - `npm run verify:release` — baseline verification, production EHPK packaging, - and artifact boundary checks. + and artefact boundary checks. - [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) — clean-install CI using the same release verification entry point. - [`../Backlog.md`](../Backlog.md) — durable outcomes that still require real @@ -39,7 +41,10 @@ smallest relevant reading set instead of loading the entire repository history. ## Knowledge ownership -- Put current user-visible behavior and operator guidance in `README.md`. +- Keep the product overview and quick start in `README.md`; put detailed reader, + development, deployment and security guidance in the linked guides above. +- Write maintained documentation in British English. Preserve exact identifiers, + commands, UI labels and third-party legal text. - Put wire compatibility rules in `gateway-protocol.md` and executable schemas. - Put package-specific agent constraints in the nearest `AGENTS.md`. - Put durable unfinished outcomes in `Backlog.md`; keep it under 20 items. diff --git a/docs/agent-harness.md b/docs/agent-harness.md index 706a8da..9c7a5f5 100644 --- a/docs/agent-harness.md +++ b/docs/agent-harness.md @@ -15,7 +15,7 @@ does not depend on a particular agent vendor or a hidden prompt bundle. 4. **Fast feedback** — focused Vitest and workspace type/build commands support short edit-test loops. 5. **Admission control** — `npm run verify` is the one baseline definition of - done; `npm run verify:release` adds package and artifact checks. + done; `npm run verify:release` adds package and artefact checks. 6. **Independent enforcement** — GitHub Actions installs from the lockfile and invokes the same release verification command on a clean runner. 7. **Reproducible dependencies** — `.node-version` and `packageManager` pin the @@ -34,7 +34,7 @@ does not depend on a particular agent vendor or a hidden prompt bundle. as interchangeable. - The structure remains lightweight enough for this small monorepo. Add a new harness layer only after a measured failure shows that current tooling cannot - express or enforce the needed behavior. + express or enforce the needed behaviour. ## Maintenance rule diff --git a/docs/deployment.md b/docs/deployment.md new file mode 100644 index 0000000..69f87f5 --- /dev/null +++ b/docs/deployment.md @@ -0,0 +1,109 @@ +# Deployment and distribution + +## Hosting model + +Live Doge requires a running Mac, a signed-in Safe Relay browser and a Doge +gateway. Keep the Mac powered and connected to the network when using the +glasses away from home. The iPhone and glasses reach that backend over HTTPS. + +Expose only the gateway at `127.0.0.1:8787`. Safe Relay must remain on +`127.0.0.1:6900` and must never be connected directly to a tunnel. See the +[security model](security.md). + +Public G2 packages contain no preset gateway origin or access key. Each user +provides an HTTPS gateway implementing the [v1 contract](gateway-protocol.md) +and pairs it through the phone companion. X cookies remain on the gateway host. + +## Maintainer deployment + +The repository's `production:*` helpers target the maintainer's configured +named Cloudflare Tunnel at `https://doge.h1ka.ru`. This is a server-side +operational setting, not a default destination for public builds. The start +script expects the existing tunnel credentials on the maintainer's Mac; +it does not provision a new tunnel for another host. + +For your own installation, provide your own HTTPS tunnel or reverse proxy to +the gateway. Configure `GATEWAY_BEARER_TOKEN` and accept the installed Even +WebView's origin. `ALLOW_BEARER_CORS=1` supports varying WebView origins while +keeping bearer authentication mandatory. The +[server entry point](../apps/gateway/src/server.ts) and +[configuration example](../.env.example) define the available settings. + +On the configured maintainer host, first generate or reuse the persistent Doge +key and copy it to the clipboard: + +```sh +npm run production:key +``` + +The script stores the key in `var/doge-access-key` with file mode `600` and +does not print it. In the Even App's Doge companion, enter the configured +HTTPS origin, paste the key and select **Save and test connection** once the +gateway is running. + +With Safe Relay running and signed in, prepare the catalogue and build, then +start the gateway and named tunnel: + +```sh +npm run relay:sync +npm run build +npm run production:start +``` + +The start script checks the local authenticated timeline, public health, +rejection of a tokenless timeline request and success of an authenticated +timeline request. It requires port 8787 to be free. + +After changing gateway source, rebuild and restart `production:start`. A +running Node process keeps the code loaded from `dist/server.js`; building +alone does not update it. Confirm that authenticated `/api/v1/session` returns +HTTP 200 and a tokenless request returns HTTP 401 before treating an update +as deployed. + +## Packaging and distribution + +For local development packaging: + +```sh +npm run verify +npm run pack:g2 +``` + +For a package intended for distribution: + +```sh +npm run verify:release +``` + +Both write `apps/g2/doge.ehpk`. Generated packages and build outputs are +excluded from Git. Record the package hash when reporting a release build. + +The development manifest permits only the loopback mock gateway. The tracked +production manifest, `apps/g2/app.production.json`, allows user-selected HTTPS +services through its `https://` network whitelist. Do not embed a maintainer +origin or access key in frontend source, build-time variables or packages. + +## Even Hub testing + +Use a Beta tester group containing only your own Even account to evaluate the +installed app before submitting it to the Store: + +1. Run `npm run verify:release`. +2. Create a Beta tester group in Even Hub and add your account's email address. +3. Upload `apps/g2/doge.ehpk` as a build and push it to the group. +4. In the iPhone Even App, open **Me → Beta tester → Doge → Install**. +5. Start Doge from the glasses home screen and pair your gateway in the phone + companion. + +Validate saved settings restoration, recovery from the background, phone +locking and profile rendering on physical glasses before Store submission. +Use the bundled [Doge icon](../apps/g2/public/doge-icon.png) for the listing. + +Beta and Private testing are distinct Hub workflows. Consult the official +[Beta testing](https://hub.evenrealities.com/docs/test/beta-testing) and +[Private testing](https://hub.evenrealities.com/docs/test/private-testing) +guides for current eligibility, distribution and lifecycle rules. + +Package verification, Hub upload, simulator testing and physical-device +testing are separate outcomes. Track outstanding hardware checks in +[`Backlog.md`](../Backlog.md). diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..63abbbd --- /dev/null +++ b/docs/development.md @@ -0,0 +1,127 @@ +# Development + +## Requirements + +- Node.js from [`.node-version`](../.node-version) and npm from the + `packageManager` field in [`package.json`](../package.json). +- For physical-device testing, an Even Hub-compatible Even App paired with + Even G2 glasses. +- For local pairing and interaction, the Even Hub simulator installed by + `npm ci`. A regular browser alone does not provide the required Even SDK bridge. +- For live X data, Safe Relay and a dedicated signed-in browser session. +- For the live-device preview, `cloudflared` available on your path. The + preview and production helper scripts target macOS. + +Run commands from the repository root. Install dependencies with `npm ci`; +retain `package-lock.json` and the existing package manager. + +## Mock development + +Follow the [quick start](../README.md#quick-start) to generate a development +bearer key, start both services and pair the companion inside the simulator. +The quick start sets `X_SOURCE=mock` explicitly, so an existing shell setting +cannot select live X data. Mock mode requires no X connection. + +| Service | Local address | +| ----------------------------------- | ----------------------- | +| Gateway | `http://127.0.0.1:8787` | +| Vite companion | `http://127.0.0.1:5173` | +| Safe Relay, when started separately | `http://127.0.0.1:6900` | + +The development key is generated in the shell and passed to the gateway through +`GATEWAY_BEARER_TOKEN`. Keep that shell open to reuse it. A new key requires +updating the companion pairing. Never reuse a development key for a public +gateway. + +See [`.env.example`](../.env.example) for configuration names. The commands here +pass values through the shell; the gateway entry point reads `process.env`. + +## Connect live X data + +Doge uses a dedicated Playwright Chromium profile rather than your everyday +browser profile. Install Chromium and start Safe Relay: + +```sh +npx playwright install chromium +npm run relay:login +``` + +Sign in to X in the new Chromium window. Once `https://x.com/home` is visible, +leave the window open. Closing the browser or stopping the command with +`Ctrl-C` also stops the relay. + +The session is stored in `var/relay-profile`. It contains X cookies and local +storage, so the entire directory is excluded from Git. + +In a second terminal, check the relay and refresh its query catalogue: + +```sh +npm run relay:check +npm run relay:sync +``` + +Stop the mock gateway before starting the live gateway on the same port. In +the shell holding your development key, run: + +```sh +X_SOURCE=relay npm run dev:gateway +``` + +Start the companion in another terminal: + +```sh +npm run dev:g2 +``` + +The catalogue records the current X query IDs and feature flags. Refresh it +when those change. `var/requests.ndjson` may contain account-derived IDs and +cursors and must remain untracked. The gateway reloads the catalogue during +operation, so catalogue synchronisation does not require a rebuild. + +## Live-device preview + +With Safe Relay signed in and its catalogue synchronised, build the app and +start the temporary authenticated preview: + +```sh +npm run build +npm run preview:live +``` + +Stop any other gateway on port 8787 first. The preview creates a random +256-bit bearer token, starts the gateway and opens a disposable Cloudflare +Quick Tunnel. It checks the authenticated live timeline before opening a QR +image on the Mac. + +The token is embedded in the QR's URL fragment, which is not sent to the tunnel +server. It is not printed in the terminal. The temporary image has file mode +`600`; keep it private. Scan it for the preview, then save and test the same +HTTPS origin as the Gateway URL in the phone companion. API calls use the +bearer token in the `Authorization` header. + +Press `Ctrl-C` to stop the gateway and tunnel and remove the QR image. The +temporary token stops providing access when that gateway exits. Quick Tunnels +have no fixed URL or availability guarantee; use the +[deployment guide](deployment.md) for persistent operation. + +## Verification + +```sh +npm test -- apps/g2/src/input.test.ts +npm run check --workspace @even-g2-x-reader/g2 +npm run check --workspace @even-g2-x-reader/gateway +npm run verify +``` + +`npm run verify` checks formatting, repository invariants and TypeScript, then +runs all tests and builds all workspaces. Run it before finalising code or +documentation changes. + +For a distributable package, use `npm run verify:release`. It also creates the +production EHPK and checks artefact boundaries. See +[packaging and distribution](deployment.md#packaging-and-distribution) for the +distinction between local and public manifests. + +Write all maintained documentation in British English, including README files, +guides, agent instructions and backlog entries. Preserve exact API names, code +identifiers, commands, UI labels and third-party legal text. diff --git a/docs/reader-guide.md b/docs/reader-guide.md new file mode 100644 index 0000000..2d5fd07 --- /dev/null +++ b/docs/reader-guide.md @@ -0,0 +1,92 @@ +# Reader guide + +Doge starts with a choice of Home, Following or Bookmarks. Select a feed to +begin reading. + +## Controls + +| Context | Input | Action | +| ------------------ | ---------- | ----------------------------------------------------------------------- | +| Feed selector | Scroll | Select Home, Following or Bookmarks | +| Feed selector | Tap | Open the selected feed | +| Feed selector | Double tap | Exit Doge | +| Post | Swipe down | Scroll forwards; at the boundary, advance to the next chunk or post | +| Post | Swipe up | Scroll backwards; at the boundary, return to the previous chunk or post | +| Post | Tap | Open the action menu | +| Timeline | Double tap | Return to the feed selector | +| Thread | Double tap | Return to the previous reader view | +| Action menu | Scroll | Select an action | +| Action menu | Tap | Run the selected action | +| Action menu | Double tap | Close the menu | +| Gallery or profile | Double tap | Return to the reader | + +Post navigation follows the finger's direction: down advances and up goes back. +It does not drag the content as a phone's natural scrolling would. The feed +selector and action menu use native G2 list scrolling. + +The action menu offers Like, Repost and Bookmark, with the corresponding undo +action when active, followed by Gallery when images are available, Reload, +Open thread, Profile and Close. Within a thread, Open thread becomes Close +thread. A successful reaction closes the menu. + +## Text and images + +Long posts use the G2 firmware's native text-container overflow scrolling, +without redrawing the text one line at a time. Text is split into chunks of at +most 2,000 UTF-16 code units without truncation or splitting surrogate pairs. +When creating or rebuilding a layout, Doge first supplies a prefix within the +1,000-code-unit creation limit, then upgrades the container to the full chunk. + +For posts spanning multiple frames, the bottom-left indicator shows the current +chunk or media-frame position; the bottom-right indicator shows the post's +position in the timeline. The display reserves its space for content rather +than a permanent control guide. Physical scrolling and boundary transitions +still require the checks listed in [`Backlog.md`](../Backlog.md). + +Posts with images show one to four images in an aspect-ratio-preserving grid at +the end of the post. The final text lines share a frame with the image grid; +earlier text stays in scrollable chunks. Shorter text leaves more room for +images. Each image has its own loading placeholder. Gallery provides a +separate, larger image view. + +Videos and animated GIFs use still posters with a play indicator. Doge does not +fetch or play video streams. + +Fetched media blobs and converted PNG tiles are held briefly in bounded +least-recently-used WebView memory caches. Revisiting an image can avoid another +download, decode and conversion. The current bridge path still sends image +bytes again after a page rebuild. Icons, avatars and post images are sent +sequentially as encoded PNG or JPEG bytes; stale avatar results are discarded +when the reader moves on. + +## Gateway pairing + +Public builds have no preset gateway. In the phone companion, enter your +gateway's HTTPS origin and 43-character access key, then select +**Save and test connection**. Local desktop development also accepts loopback +HTTP addresses. + +Doge calls `GET /api/v1/session` with the bearer key and checks the +[protocol identity](gateway-protocol.md#pairing-handshake). It saves the pairing +only after validation succeeds. No timeline or media requests are sent before +pairing. + +The Even SDK's device storage keeps the URL and access key. Doge verifies that +the saved key can be read back before treating the connection as ready. At +startup, settings controls remain disabled until storage restoration completes; +the saved pairing is then checked again without requiring re-entry. + +- The URL draft is stored separately, so it survives a failed connection or a + restart. +- Leave the key field blank to retain the saved key. **Saved access key is + active** means it remains available even though the password field is empty. +- Enter a new key only when replacing the saved one. +- **Forget access key** removes the key and leaves the URL in the form. +- Earlier WebView and legacy pairing values are migrated to device storage. + +## App icon + +[`doge-icon.png`](../apps/g2/public/doge-icon.png) is used for the phone web app +and the initial loading screen after feed selection. Upload the same image +manually when setting the Even Hub listing icon. Its origin and terms are +documented in the [asset licence](../apps/g2/public/doge-icon.LICENSE.md). diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..facd742 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,59 @@ +# Security model + +Doge's public boundary is its authenticated gateway. Safe Relay and the X +browser session stay on the host. The client receives validated data and image +bytes, never X cookies or internal upstream headers. + +## Authentication and pairing + +- Public gateways require a bearer access key for data and reaction requests + under `/api/v1/*`. Health checks and CORS preflight responses remain public. +- Public builds contain no maintainer origin or access key. Each device pairs + a user-selected HTTPS origin through the [session handshake](gateway-protocol.md#pairing-handshake). +- The gateway can accept varying installed WebView origins while retaining + bearer authentication. CORS is not an authentication mechanism. +- Set `GATEWAY_BEARER_TOKEN` before exposing a gateway. The raw development + server permits an unset token; the production and live-preview helpers + supply one explicitly. +- API responses use `no-store` and security headers. + +## Relay boundary + +The relay base URL must be loopback HTTP. An explicit allowlist permits six +read operations and six reaction operations: + +| Purpose | Relay operations | +| -------------------- | ------------------------------------------------- | +| Feeds | `HomeTimeline`, `HomeLatestTimeline`, `Bookmarks` | +| Threads and profiles | `TweetDetail`, `UserByScreenName`, `UserTweets` | +| Likes | `FavoriteTweet`, `UnfavoriteTweet` | +| Reposts | `CreateRetweet`, `DeleteRetweet` | +| Bookmarks | `CreateBookmark`, `DeleteBookmark` | + +The gateway exposes no arbitrary X request endpoint and no routes for creating +posts, replying, following accounts or deleting posts. Feed names, cursors and +post IDs are validated against schemas. Upstream responses are normalised into +the shared contract, and GraphQL errors count as failures even when returned +with HTTP 200. Relay requests have a 15-second timeout and a 5 MB response limit. + +## Image proxies + +| Proxy | Allowed source | Limits | +| ---------- | -------------------------------------------------------------- | ---------------------------------------------------------------------------- | +| Avatar | HTTPS `pbs.twimg.com/profile_images/` paths | 5 seconds; 512 KiB; JPEG, PNG or WebP content type | +| Post media | Allowlisted HTTPS `pbs.twimg.com` photo and video-poster paths | 5 seconds; 4 MiB; matching JPEG, PNG or WebP content type and file signature | + +Both proxies reject redirects. Media fetching excludes `video.twimg.com`, MP4 +and HLS; video and animated GIF posts use still posters only. + +## Files that stay local + +Never commit X cookies, access keys, browser profiles, Cloudflare credentials, +relay catalogues or real `.env` files. The repository also excludes generated +`dist/`, coverage and `.ehpk` files. Keep secrets out of logs and review output. + +Non-secret configuration examples such as [`.env.example`](../.env.example) +are tracked deliberately. The [repository checks](../scripts/check-repository.mjs) +guard tracked-file boundaries, and production +[artefact checks](../scripts/check-production-artifact.mjs) inspect packages for +fixed gateway origins and the local access key when available. diff --git a/package-lock.json b/package-lock.json index 2d3f82f..1162d75 100644 --- a/package-lock.json +++ b/package-lock.json @@ -2747,7 +2747,9 @@ } }, "node_modules/js-yaml": { - "version": "4.3.1", + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.2.tgz", + "integrity": "sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA==", "dev": true, "funding": [ {