diff --git a/CHANGELOG.md b/CHANGELOG.md index 053c5c30..b254ec70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,11 +10,15 @@ - Malformed fixture `citations` or `files` (not an array, a missing or mistyped field, a non-integer offset, an offset outside the text, or citations on a turn with no text: tool calls only, `blocks` with no text block, or empty content) answer a 500 before any content is written: `aimock_invalid_fixture_citation` or `aimock_invalid_fixture_file`. A non-`image/*` file on the OpenAI Responses API answers a 500 with `aimock_unsupported_generated_file`. This is the same on every wire that reads these keys, with or without a misbehavior fault: the fixture is checked on its original turn before any fault applies (only an invalid misbehavior config answers its own 400 or 501 first). A valid fixture under a fault that changes the reply text is served without its `citations`. Only Chat Completions, Anthropic Messages, Gemini and OpenAI Responses read these keys; other wires ignore them. - Fixture load validation warns about malformed `citations` and `files`, with the same problem description the 500 carries (the 500 message adds an `aimock: invalid fixture` prefix; the warning adds which 500 the requests answer). The fixture still loads. - `InvalidFixtureOutputError` (the error behind `aimock_invalid_fixture_citation` and `aimock_invalid_fixture_file`) is exported from `@copilotkit/aimock`. +- Opt-in fixture `nonStreamingTimingAllProviders: true` delays non-streaming replies on OpenAI chat, Anthropic, Responses, Gemini/Vertex, Gemini Interactions, Cohere, Bedrock invoke/converse and Ollama; `nonStreamingTiming` is unchanged (OpenAI chat only); error replies are never delayed. On the OpenAI Responses API the wait also counts the stream frames that fixture `citations` and generated `files` add ### Fixed - Misbehavior faults on Chat Completions, Anthropic Messages, Gemini and the OpenAI Responses API (HTTP and WebSocket): a fixture the wire rejects (generated `files` on Chat Completions or Anthropic, a non-`image/*` file on Responses, malformed `citations` or `files`) is never faulted. It takes the normal path, is journaled as a 500 that keeps its fixture, and consumes no fault `times` budget or seed ordinal; `aimock_misbehavior_total` does not count it. A non-stream Chat Completions fault is journaled before its timing wait. - Journal accuracy: a client that hangs up during the timing wait of any non-stream Chat Completions reply (faulted or not) is journaled as interrupted, not as served. +- Client disconnects stop delivery (no more chunks or waits) on OpenAI chat, Anthropic, Responses, Gemini/Vertex, Gemini Interactions, Cohere, Bedrock, Ollama, transcription streams and the Responses, Realtime and Gemini Live WebSockets +- Client disconnects on those surfaces are journaled as `interrupted` with `interruptReason: "client aborted"` (HTTP `status: 0` when nothing was sent); aimock-caused teardowns record `"disconnect"`. Journal snapshots of hung-up requests now carry these fields +- WebSocket: a client TCP FIN without a close frame now closes the connection at once (1006) and ends the turn - Docker image: the base image is pinned to `node:22.23.4-alpine` by digest. Node 22.23.4 fixes CVE-2026-111497 (Brotli decompression memory bug); the 1.46.0 image shipped Node 22.23.3. ## [1.46.0] - 2026-10-20 diff --git a/docs/aws-bedrock/index.html b/docs/aws-bedrock/index.html index c155f91f..4b84bd61 100644 --- a/docs/aws-bedrock/index.html +++ b/docs/aws-bedrock/index.html @@ -107,6 +107,16 @@

URL Patterns

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + InvokeModel and Converse replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

Request Format

Bedrock Claude requests use the Anthropic Messages format. The diff --git a/docs/chat-completions/index.html b/docs/chat-completions/index.html index ac7f3aaa..d337fa28 100644 --- a/docs/chat-completions/index.html +++ b/docs/chat-completions/index.html @@ -73,6 +73,16 @@

Endpoint

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + stream: false replies, set nonStreamingTiming: true or + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + Non-Streaming Responses). +

+

Unit Test: Text Response

Using the programmatic API with vitest, register a fixture and assert on the response. diff --git a/docs/cohere/index.html b/docs/cohere/index.html index 5c4ae1b4..ec1c4f76 100644 --- a/docs/cohere/index.html +++ b/docs/cohere/index.html @@ -73,6 +73,16 @@

Endpoint

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + stream: false replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

Key Features

+ +

Client Disconnects

+

+ When the client hangs up before the reply is complete, aimock stops delivery. It sends no + more chunks and skips the remaining waits. This applies to these endpoints, streaming and + non-streaming: +

+ +

+ On WebSocket, a client that closes its TCP connection without sending a close frame is + also a disconnect. aimock closes the connection at once (code 1006, "Connection lost"), + stops the turn and journals it. +

+

The journal entry records how the reply ended:

+ diff --git a/docs/fixtures/index.html b/docs/fixtures/index.html index 47d55b48..c6331240 100644 --- a/docs/fixtures/index.html +++ b/docs/fixtures/index.html @@ -1049,10 +1049,23 @@

Fixture Options

Also delay stream: false OpenAI Chat Completions replies by the fixture's streamingProfile or recordedTimings (default - false). See + false). Other providers ignore it (use + nonStreamingTimingAllProviders). See Non-Streaming Responses + + nonStreamingTimingAllProviders + boolean + + Delay stream: false replies the same way on OpenAI Chat Completions, + Anthropic Messages, OpenAI Responses, Gemini, Vertex AI, Gemini Interactions, + Cohere, Bedrock InvokeModel and Converse, and Ollama (default false). + Needs streamingProfile or recordedTimings. Error replies + and audio transcription are not delayed. See + All Providers + + chaos object diff --git a/docs/gemini-interactions/index.html b/docs/gemini-interactions/index.html index 07d03450..2995a865 100644 --- a/docs/gemini-interactions/index.html +++ b/docs/gemini-interactions/index.html @@ -75,6 +75,16 @@

Endpoint

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + stream: false replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

Quick Start

Set up a minimal mock that responds to a Gemini Interactions request:

diff --git a/docs/gemini/index.html b/docs/gemini/index.html index 0ec1e515..c5ed266b 100644 --- a/docs/gemini/index.html +++ b/docs/gemini/index.html @@ -98,6 +98,16 @@

Endpoints

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). Gemini Live records + the turn the same way. To delay generateContent replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

Model Catalog

The native catalog contains Google IDs and unrecognized IDs from fixtures, in first-seen diff --git a/docs/ollama/index.html b/docs/ollama/index.html index c8037263..ae3b5089 100644 --- a/docs/ollama/index.html +++ b/docs/ollama/index.html @@ -93,6 +93,16 @@

Endpoints

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + "stream": false replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

Embeddings

Use the Ollama SDK's embed({ model, input }) with a string or array of diff --git a/docs/openrouter-chat/index.html b/docs/openrouter-chat/index.html index 04d9fa09..645e414f 100644 --- a/docs/openrouter-chat/index.html +++ b/docs/openrouter-chat/index.html @@ -107,6 +107,16 @@

Endpoints

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + stream: false replies, set nonStreamingTiming: true or + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + Non-Streaming Responses). +

+

Point the OpenAI SDK at aimock

The dominant real-world integration path is the OpenAI SDK with its diff --git a/docs/responses-api/index.html b/docs/responses-api/index.html index e82005b7..1d68aa42 100644 --- a/docs/responses-api/index.html +++ b/docs/responses-api/index.html @@ -77,6 +77,16 @@

Endpoints

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). The WebSocket + transport records the turn the same way. To delay HTTP stream: false replies, + set nonStreamingTimingAllProviders: true with a + streamingProfile or recordedTimings (see + All Providers). +

+

Unit Test: HTTP SSE Text Response

diff --git a/docs/streaming-physics/index.html b/docs/streaming-physics/index.html index 84fbf4ee..82733cd9 100644 --- a/docs/streaming-physics/index.html +++ b/docs/streaming-physics/index.html @@ -195,9 +195,65 @@

Non-Streaming Responses

nonStreamingTiming has an effect only on OpenAI Chat Completions, and only when the fixture also sets streamingProfile or recordedTimings. - Non-streaming responses from other providers are not delayed. + Non-streaming responses from other providers are not delayed by it.

+

All Providers

+

+ To delay stream: false replies on other providers too, set + nonStreamingTimingAllProviders: true on the fixture. It uses the same formula + as nonStreamingTiming: true, and it also needs a + streamingProfile or recordedTimings. It applies to these + non-streaming endpoints: +

+ +

+ Audio transcription replies are not delayed. nonStreamingTiming is unchanged: + it still applies to OpenAI Chat Completions only. Setting + nonStreamingTimingAllProviders: false does not turn off + nonStreamingTiming: true. +

+ +
+
+ fixtures/slow-non-streaming-all.json json +
+ 
{
+  "fixtures": [
+    {
+      "match": { "userMessage": "think carefully" },
+      "response": { "content": "Let me think about this..." },
+      "streamingProfile": { "ttft": 800, "tps": 50 },
+      "nonStreamingTimingAllProviders": true
+    }
+  ]
+}
+
+ +

Errors and Disconnects

+ +

Interaction with latency

diff --git a/docs/transcription/index.html b/docs/transcription/index.html index 7072f042..3ca88b92 100644 --- a/docs/transcription/index.html +++ b/docs/transcription/index.html @@ -73,6 +73,13 @@

Endpoint

+

+ If the client hangs up during a streamed transcription (stream=true), aimock + stops the stream and journals the request with + interruptReason: "client aborted" (see + Client Disconnects). +

+

Unit Test: Simple Transcription

Using the programmatic API with vitest, register a fixture and assert on the response. diff --git a/docs/vertex-ai/index.html b/docs/vertex-ai/index.html index 88f2ef95..779b221c 100644 --- a/docs/vertex-ai/index.html +++ b/docs/vertex-ai/index.html @@ -82,6 +82,16 @@

Endpoints

+

+ If the client hangs up before the reply is complete, aimock stops delivery and journals + the request with interruptReason: "client aborted" (see + Client Disconnects). To delay + generateContent replies, set + nonStreamingTimingAllProviders: true with a streamingProfile or + recordedTimings (see + All Providers). +

+

URL Pattern Difference

The key difference between consumer Gemini and Vertex AI is the URL routing. Consumer diff --git a/docs/websocket/index.html b/docs/websocket/index.html index d4288ada..fac4f432 100644 --- a/docs/websocket/index.html +++ b/docs/websocket/index.html @@ -88,6 +88,13 @@

Endpoints

+

+ On OpenAI Responses, OpenAI Realtime and Gemini Live, if the client closes the connection + during a reply, aimock stops sending and journals the turn with + interruptReason: "client aborted" (see + Client Disconnects). +

+

OpenAI GPT-Live

GPT-Live uses a primary WebSocket upgrade: GET /v1/live/sessions. Connect to diff --git a/etc/aimock-a2a.import.api.md b/etc/aimock-a2a.import.api.md index b2482cfa..7c4306d9 100644 --- a/etc/aimock-a2a.import.api.md +++ b/etc/aimock-a2a.import.api.md @@ -335,6 +335,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-a2a.require.api.md b/etc/aimock-a2a.require.api.md index b2482cfa..7c4306d9 100644 --- a/etc/aimock-a2a.require.api.md +++ b/etc/aimock-a2a.require.api.md @@ -335,6 +335,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-agui.import.api.md b/etc/aimock-agui.import.api.md index 2d106358..d3a8449b 100644 --- a/etc/aimock-agui.import.api.md +++ b/etc/aimock-agui.import.api.md @@ -964,6 +964,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-agui.require.api.md b/etc/aimock-agui.require.api.md index 2d106358..d3a8449b 100644 --- a/etc/aimock-agui.require.api.md +++ b/etc/aimock-agui.require.api.md @@ -964,6 +964,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-index.import.api.md b/etc/aimock-index.import.api.md index 5fea0202..242db462 100644 --- a/etc/aimock-index.import.api.md +++ b/etc/aimock-index.import.api.md @@ -2105,6 +2105,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -2223,6 +2224,7 @@ export interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-index.require.api.md b/etc/aimock-index.require.api.md index 5fea0202..242db462 100644 --- a/etc/aimock-index.require.api.md +++ b/etc/aimock-index.require.api.md @@ -2105,6 +2105,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -2223,6 +2224,7 @@ export interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-jest.import.api.md b/etc/aimock-jest.import.api.md index 2a553853..add7814e 100644 --- a/etc/aimock-jest.import.api.md +++ b/etc/aimock-jest.import.api.md @@ -271,6 +271,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -383,6 +384,7 @@ interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-jest.require.api.md b/etc/aimock-jest.require.api.md index 2a553853..add7814e 100644 --- a/etc/aimock-jest.require.api.md +++ b/etc/aimock-jest.require.api.md @@ -271,6 +271,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -383,6 +384,7 @@ interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-mcp.import.api.md b/etc/aimock-mcp.import.api.md index d821ab97..8d4b6a43 100644 --- a/etc/aimock-mcp.import.api.md +++ b/etc/aimock-mcp.import.api.md @@ -203,6 +203,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-mcp.require.api.md b/etc/aimock-mcp.require.api.md index d821ab97..8d4b6a43 100644 --- a/etc/aimock-mcp.require.api.md +++ b/etc/aimock-mcp.require.api.md @@ -203,6 +203,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-vector.import.api.md b/etc/aimock-vector.import.api.md index 5fd533c2..1a6ad453 100644 --- a/etc/aimock-vector.import.api.md +++ b/etc/aimock-vector.import.api.md @@ -197,6 +197,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-vector.require.api.md b/etc/aimock-vector.require.api.md index 5fd533c2..1a6ad453 100644 --- a/etc/aimock-vector.require.api.md +++ b/etc/aimock-vector.require.api.md @@ -197,6 +197,7 @@ interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-vitest.import.api.md b/etc/aimock-vitest.import.api.md index cb667af8..de442b65 100644 --- a/etc/aimock-vitest.import.api.md +++ b/etc/aimock-vitest.import.api.md @@ -271,6 +271,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -383,6 +384,7 @@ interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/etc/aimock-vitest.require.api.md b/etc/aimock-vitest.require.api.md index cb667af8..de442b65 100644 --- a/etc/aimock-vitest.require.api.md +++ b/etc/aimock-vitest.require.api.md @@ -271,6 +271,7 @@ export interface Fixture { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; @@ -383,6 +384,7 @@ interface FixtureFileEntry { // (undocumented) misbehavior?: MisbehaviorConfig | MisbehaviorFaultId; nonStreamingTiming?: boolean; + nonStreamingTimingAllProviders?: boolean; openRouterProcessing?: boolean; // (undocumented) recordedTimings?: RecordedTimings; diff --git a/skills/write-fixtures/SKILL.md b/skills/write-fixtures/SKILL.md index 3d1405c2..436e4696 100644 --- a/skills/write-fixtures/SKILL.md +++ b/skills/write-fixtures/SKILL.md @@ -666,9 +666,9 @@ Custom tool calls (`customToolCalls` / `customToolCall` blocks in `responsesBloc 2. **`arguments` accepts both objects and strings** — `"arguments": {"key":"value"}` (preferred, auto-stringified) or `"arguments": "{\"key\":\"value\"}"` (legacy). The same applies to `content` fields that contain JSON. The fixture loader detects `typeof === "object"` and calls `JSON.stringify()` automatically. -3. **Latency is per-chunk, not total** — `latency: 100` means 100ms between each SSE chunk, not 100ms total response time. It never delays a non-streaming (`stream: false`) response, also with `nonStreamingTiming: true`. Similarly, `truncateAfterChunks` and `disconnectAfterMs` are for simulating stream interruptions (added in v1.3.0). +3. **Latency is per-chunk, not total** — `latency: 100` means 100ms between each SSE chunk, not 100ms total response time. It never delays a non-streaming (`stream: false`) response, also with `nonStreamingTiming: true` or `nonStreamingTimingAllProviders: true`. Similarly, `truncateAfterChunks` and `disconnectAfterMs` are for simulating stream interruptions (added in v1.3.0). -4. **`streamingProfile` takes precedence over `latency`** — when both are set on a fixture, `streamingProfile` controls timing. Use one or the other. By default, `streamingProfile` and `recordedTimings` delay only streamed replies: a `stream: false` reply returns immediately, also for recorded fixtures. Set `nonStreamingTiming: true` on the fixture to also delay `stream: false` OpenAI Chat Completions replies (text, tool calls, combined content and tool calls, and `blocks`) by the sum of the per-chunk delays a streamed reply at the same `chunkSize` would get. Other providers' non-streaming replies are not delayed. +4. **`streamingProfile` takes precedence over `latency`** — when both are set on a fixture, `streamingProfile` controls timing. Use one or the other. By default, `streamingProfile` and `recordedTimings` delay only streamed replies: a `stream: false` reply returns immediately, also for recorded fixtures. Set `nonStreamingTiming: true` on the fixture to also delay `stream: false` OpenAI Chat Completions replies (text, tool calls, combined content and tool calls, and `blocks`) by the sum of the per-chunk delays a streamed reply at the same `chunkSize` would get. `nonStreamingTiming` does not delay other providers. To delay `stream: false` replies with the same formula on OpenAI Chat Completions, Anthropic Messages, OpenAI Responses, Gemini and Vertex AI `generateContent`, Gemini Interactions (`stream: false`), Cohere v2 chat, Bedrock InvokeModel and Converse, and Ollama `/api/chat` and `/api/generate`, set `nonStreamingTimingAllProviders: true` instead. Audio transcription replies are not delayed. Error replies are never delayed. If the client disconnects during the wait, nothing is written and the journal entry gets `interrupted: true`, `interruptReason: "client aborted"` and `status: 0`. 5. **Tool result messages don't change the user message** — after a tool call, the client sends the same conversation + tool result. Matching on `userMessage` will hit the SAME fixture again → infinite loop. Always use `predicate` checking `role === "tool"` for tool results. Note: a whole-conversation `role === "tool"` check (e.g. `req.messages.some((m) => m.role === "tool")`) diverges from `hasToolResult`'s current-turn scoping in multi-turn flows — the built-in `hasToolResult` matcher only looks after the last user message, so a later turn whose history carries an earlier tool result still reads `false`. diff --git a/src/__tests__/__snapshots__/frame-codec-golden/aws-eventstream.json b/src/__tests__/__snapshots__/frame-codec-golden/aws-eventstream.json new file mode 100644 index 00000000..ed54e8a1 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/aws-eventstream.json @@ -0,0 +1,523 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello \"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 151, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"world\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello \"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 151, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"world\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello \"}}" + } + ] + }, + { + "op": "return", + "completed": false + } + ], + "journal": null + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello \"}}" + } + ] + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "frames": [ + { + "total": 151, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"world\"}}" + } + ] + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello \"}}" + } + ] + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "frames": [ + { + "total": 151, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"world\"}}" + } + ] + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/bedrock-converse-eventstream.json b/src/__tests__/__snapshots__/frame-codec-golden/bedrock-converse-eventstream.json new file mode 100644 index 00000000..29b08275 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/bedrock-converse-eventstream.json @@ -0,0 +1,1946 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello g\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"olden w\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"orld, t\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"his is \"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"a chara\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"cteriza\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"tion ru\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 148, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"n.\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 191, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{\"toolUse\":{\"toolUseId\":\"call_g1\",\"name\":\"get_weather\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"toolUse\":{\"input\":\"{\\\"city\\\"\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"toolUse\":{\"input\":\":\\\"NYC\\\"}\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"tool_use\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Checkin\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"g now.\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 191, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"start\":{\"toolUse\":{\"toolUseId\":\"call_g2\",\"name\":\"get_weather\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\"{\\\"city\\\"\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 167, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\":\\\"SF\\\"}\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"tool_use\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Lead te\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 149, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"xt.\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 188, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"start\":{\"toolUse\":{\"toolUseId\":\"call_g3\",\"name\":\"get_time\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 161, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\"{}\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":2,\"start\":{}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":2,\"delta\":{\"text\":\"Tail te\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 149, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":2,\"delta\":{\"text\":\"xt.\"}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":2}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"tool_use\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 191, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{\"toolUse\":{\"toolUseId\":\"call_m1\",\"name\":\"get_weather\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"toolUse\":{\"input\":\"{\\\"city\\\"\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 188, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"start\":{\"toolUse\":{\"toolUseId\":\"call_m2\",\"name\":\"get_time\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\"{\\\"zone\\\"\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\":\\\"UTC\\\"}\"}}}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"tool_use\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 182, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":3,\"outputTokens\":10,\"totalTokens\":13},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello g\"}}" + } + ] + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"olden w\"}}" + } + ] + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"orld, t\"}}" + } + ] + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"his is \"}}" + } + ] + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"a chara\"}}" + } + ] + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"cteriza\"}}" + } + ] + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"tion ru\"}}" + } + ] + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "frames": [ + { + "total": 148, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"n.\"}}" + } + ] + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Checkin\"}}" + } + ] + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "frames": [ + { + "total": 152, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"g now.\"}}" + } + ] + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "frames": [ + { + "total": 191, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"start\":{\"toolUse\":{\"toolUseId\":\"call_g2\",\"name\":\"get_weather\"}}}" + } + ] + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "frames": [ + { + "total": 168, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\"{\\\"city\\\"\"}}}" + } + ] + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "frames": [ + { + "total": 167, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1,\"delta\":{\"toolUse\":{\"input\":\":\\\"SF\\\"}\"}}}" + } + ] + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":1}" + } + ] + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"tool_use\"}" + } + ] + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "frames": [ + { + "total": 118, + "headersLength": 82, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStart", + ":message-type": "event" + }, + "payload": "{\"role\":\"assistant\"}" + } + ] + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "frames": [ + { + "total": 137, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStart", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"start\":{}}" + } + ] + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"Hello g\"}}" + } + ] + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"olden w\"}}" + } + ] + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"orld, t\"}}" + } + ] + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"his is \"}}" + } + ] + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"a chara\"}}" + } + ] + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"cteriza\"}}" + } + ] + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "frames": [ + { + "total": 153, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"tion ru\"}}" + } + ] + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "frames": [ + { + "total": 148, + "headersLength": 87, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockDelta", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0,\"delta\":{\"text\":\"n.\"}}" + } + ] + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "write", + "frames": [ + { + "total": 125, + "headersLength": 86, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "contentBlockStop", + ":message-type": "event" + }, + "payload": "{\"contentBlockIndex\":0}" + } + ] + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "write", + "frames": [ + { + "total": 122, + "headersLength": 81, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "messageStop", + ":message-type": "event" + }, + "payload": "{\"stopReason\":\"end_turn\"}" + } + ] + }, + { + "op": "timer", + "ms": 26.5 + }, + { + "op": "write", + "frames": [ + { + "total": 180, + "headersLength": 78, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "metadata", + ":message-type": "event" + }, + "payload": "{\"usage\":{\"inputTokens\":0,\"outputTokens\":0,\"totalTokens\":0},\"metrics\":{\"latencyMs\":0}}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/bedrock-invoke-eventstream.json b/src/__tests__/__snapshots__/frame-codec-golden/bedrock-invoke-eventstream.json new file mode 100644 index 00000000..2e07190c --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/bedrock-invoke-eventstream.json @@ -0,0 +1,1946 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Hello g\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"olden w\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"orld, t\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"his is \\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"a chara\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"cteriza\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"tion ru\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"n.\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"end_turn\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 267, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_g1\\\",\\\"name\\\":\\\"get_weather\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{\\\\\\\"city\\\\\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\":\\\\\\\"NYC\\\\\\\"}\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"tool_use\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Checkin\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"g now.\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 267, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":1,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_g2\\\",\\\"name\\\":\\\"get_weather\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{\\\\\\\"city\\\\\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 239, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\":\\\\\\\"SF\\\\\\\"}\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":1}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"tool_use\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Lead te\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"xt.\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 263, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":1,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_g3\\\",\\\"name\\\":\\\"get_time\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 231, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{}\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":1}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":2,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":2,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Tail te\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":2,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"xt.\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":2}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"tool_use\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":3,\\\"output_tokens\\\":10}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 267, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_m1\\\",\\\"name\\\":\\\"get_weather\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 231, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{}\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 263, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":1,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_m2\\\",\\\"name\\\":\\\"get_time\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{\\\\\\\"zone\\\\\\\"\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\":\\\\\\\"UTC\\\\\\\"}\\\"}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":1}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 251, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"tool_use\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":10}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Hello g\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"olden w\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"orld, t\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"his is \\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"a chara\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"cteriza\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"tion ru\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"n.\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"end_turn\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Checkin\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"g now.\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "frames": [ + { + "total": 267, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":1,\\\"content_block\\\":{\\\"type\\\":\\\"tool_use\\\",\\\"id\\\":\\\"call_g2\\\",\\\"name\\\":\\\"get_weather\\\",\\\"input\\\":{}}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "frames": [ + { + "total": 243, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\"{\\\\\\\"city\\\\\\\"\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "frames": [ + { + "total": 239, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":1,\\\"delta\\\":{\\\"type\\\":\\\"input_json_delta\\\",\\\"partial_json\\\":\\\":\\\\\\\"SF\\\\\\\"}\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":1}\"}" + } + ] + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"tool_use\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/vnd.amazon.eventstream" + }, + { + "op": "setHeader", + "name": "Transfer-Encoding", + "value": "chunked" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "frames": [ + { + "total": 439, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_start\\\",\\\"message\\\":{\\\"id\\\":\\\"\\\",\\\"type\\\":\\\"message\\\",\\\"role\\\":\\\"assistant\\\",\\\"content\\\":[],\\\"model\\\":\\\"anthropic.claude-3-5-sonnet-20241022-v2:0\\\",\\\"stop_reason\\\":null,\\\"stop_sequence\\\":null,\\\"usage\\\":{\\\"input_tokens\\\":0,\\\"output_tokens\\\":0}}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_start\\\",\\\"index\\\":0,\\\"content_block\\\":{\\\"type\\\":\\\"text\\\",\\\"text\\\":\\\"\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"Hello g\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"olden w\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"orld, t\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"his is \\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"a chara\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"cteriza\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "frames": [ + { + "total": 219, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"tion ru\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "frames": [ + { + "total": 215, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_delta\\\",\\\"index\\\":0,\\\"delta\\\":{\\\"type\\\":\\\"text_delta\\\",\\\"text\\\":\\\"n.\\\"}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "write", + "frames": [ + { + "total": 155, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"content_block_stop\\\",\\\"index\\\":0}\"}" + } + ] + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "write", + "frames": [ + { + "total": 247, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_delta\\\",\\\"delta\\\":{\\\"stop_reason\\\":\\\"end_turn\\\",\\\"stop_sequence\\\":null},\\\"usage\\\":{\\\"output_tokens\\\":0}}\"}" + } + ] + }, + { + "op": "timer", + "ms": 26.5 + }, + { + "op": "write", + "frames": [ + { + "total": 135, + "headersLength": 75, + "preludeOk": true, + "messageOk": true, + "headers": { + ":content-type": "application/json", + ":event-type": "chunk", + ":message-type": "event" + }, + "payload": "{\"bytes\":\"{\\\"type\\\":\\\"message_stop\\\"}\"}" + } + ] + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/claude-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/claude-sse.json new file mode 100644 index 00000000..f5adb0a6 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/claude-sse.json @@ -0,0 +1,769 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Hello g\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"olden w\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"orld, t\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"his is \"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"a chara\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"cteriza\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"tion ru\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"n.\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_g1\",\"name\":\"get_weather\",\"input\":{}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"city\\\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\":\\\"NYC\\\"}\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Checkin\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"g now.\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":1,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_g2\",\"name\":\"get_weather\",\"input\":{}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"city\\\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\":\\\"SF\\\"}\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":1}\n\n" + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Lead te\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"xt.\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":1,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_g3\",\"name\":\"get_time\",\"input\":{}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{}\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":1}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":2,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":2,\"delta\":{\"type\":\"text_delta\",\"text\":\"Tail te\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":2,\"delta\":{\"type\":\"text_delta\",\"text\":\"xt.\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":2}\n\n" + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":3,\"output_tokens\":10}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_m1\",\"name\":\"get_weather\",\"input\":{}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{}\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":1,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_m2\",\"name\":\"get_time\",\"input\":{}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"zone\\\"\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\":\\\"UTC\\\"}\"}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":1}\n\n" + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":10}}\n\n" + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Hello g\"}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"olden w\"}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"orld, t\"}}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"his is \"}}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"a chara\"}}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"cteriza\"}}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"tion ru\"}}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"n.\"}}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Checkin\"}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"g now.\"}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":1,\"content_block\":{\"type\":\"tool_use\",\"id\":\"call_g2\",\"name\":\"get_weather\",\"input\":{}}}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\"{\\\"city\\\"\"}}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":{\"type\":\"input_json_delta\",\"partial_json\":\":\\\"SF\\\"}\"}}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":1}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"content\":[],\"model\":\"claude-3-5-sonnet-20241022\",\"stop_reason\":null,\"stop_sequence\":null,\"usage\":{\"input_tokens\":0,\"output_tokens\":0}}}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Hello g\"}}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"olden w\"}}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"orld, t\"}}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"his is \"}}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"a chara\"}}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"cteriza\"}}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"tion ru\"}}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"n.\"}}\n\n" + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "write", + "data": "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "write", + "data": "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":0}}\n\n" + }, + { + "op": "timer", + "ms": 26.5 + }, + { + "op": "write", + "data": "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/cohere-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/cohere-sse.json new file mode 100644 index 00000000..dcc80e7a --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/cohere-sse.json @@ -0,0 +1,737 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Hello g\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"olden w\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"orld, t\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"his is \"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"a chara\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"cteriza\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"tion ru\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"n.\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"COMPLETE\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-plan-delta\ndata: {\"type\":\"tool-plan-delta\",\"delta\":{\"message\":{\"tool_plan\":\"I will use the requested tool.\"}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\"{\\\"city\\\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"TOOL_CALL\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Checkin\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"g now.\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: tool-plan-delta\ndata: {\"type\":\"tool-plan-delta\",\"delta\":{\"message\":{\"tool_plan\":\"I will use the requested tool.\"}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"id\":\"call_g2\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\"{\\\"city\\\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\":\\\"SF\\\"}\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"TOOL_CALL\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Lead te\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"xt.\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: tool-plan-delta\ndata: {\"type\":\"tool-plan-delta\",\"delta\":{\"message\":{\"tool_plan\":\"I will use the requested tool.\"}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"id\":\"call_g3\",\"type\":\"function\",\"function\":{\"name\":\"get_time\",\"arguments\":\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\"{}\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":1,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":1,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Tail te\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":1,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"xt.\"}}}}\n\n" + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":1}\n\n" + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"TOOL_CALL\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-plan-delta\ndata: {\"type\":\"tool-plan-delta\",\"delta\":{\"message\":{\"tool_plan\":\"I will use the requested tool.\"}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"id\":\"call_m1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\"{}\"}}}}}\n\n" + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0}\n\n" + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"TOOL_CALL\",\"usage\":{\"billed_units\":{\"input_tokens\":3,\"output_tokens\":5,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":3,\"output_tokens\":5}}}}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Hello g\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"olden w\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"orld, t\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"his is \"}}}}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"a chara\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"cteriza\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"tion ru\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"n.\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"COMPLETE\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Checkin\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"g now.\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: tool-plan-delta\ndata: {\"type\":\"tool-plan-delta\",\"delta\":{\"message\":{\"tool_plan\":\"I will use the requested tool.\"}}}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"id\":\"call_g2\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}}}}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\"{\\\"city\\\"\"}}}}}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0,\"delta\":{\"message\":{\"tool_calls\":{\"function\":{\"arguments\":\":\\\"SF\\\"}\"}}}}}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: tool-\ndata: {\"type\":\"tool-\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"TOOL_CALL\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "event: message-start\ndata: {\"id\":\"\",\"type\":\"message-start\",\"delta\":{\"message\":{\"role\":\"assistant\",\"content\":[],\"tool_plan\":\"\",\"tool_calls\":[],\"citations\":[]}}}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "event: content-start\ndata: {\"type\":\"content-start\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"Hello g\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"olden w\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"orld, t\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"his is \"}}}}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"a chara\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"cteriza\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"tion ru\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "event: content-delta\ndata: {\"type\":\"content-delta\",\"index\":0,\"delta\":{\"message\":{\"content\":{\"type\":\"text\",\"text\":\"n.\"}}}}\n\n" + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "write", + "data": "event: content-end\ndata: {\"type\":\"content-end\",\"index\":0}\n\n" + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "write", + "data": "event: message-end\ndata: {\"type\":\"message-end\",\"delta\":{\"finish_reason\":\"COMPLETE\",\"usage\":{\"billed_units\":{\"input_tokens\":0,\"output_tokens\":0,\"search_units\":0,\"classifications\":0},\"tokens\":{\"input_tokens\":0,\"output_tokens\":0}}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/gemini-interactions-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/gemini-interactions-sse.json new file mode 100644 index 00000000..84c96412 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/gemini-interactions-sse.json @@ -0,0 +1,713 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Hello g\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"olden w\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"orld, t\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"his is \"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"a chara\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"cteriza\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"tion ru\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"n.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"completed\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"function_call\",\"id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"requires_action\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Checkin\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"g now.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":1,\"step\":{\"type\":\"function_call\",\"id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":1,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":1,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"requires_action\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Lead te\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"xt.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":1,\"step\":{\"type\":\"function_call\",\"id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":1,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":1,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":2,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":2,\"delta\":{\"type\":\"text\",\"text\":\"Tail te\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":2,\"delta\":{\"type\":\"text\",\"text\":\"xt.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":2,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"requires_action\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"function_call\",\"id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":1,\"step\":{\"type\":\"function_call\",\"id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":1,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":1,\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"requires_action\",\"usage\":{\"total_input_tokens\":3,\"total_output_tokens\":10,\"total_tokens\":13}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Hello g\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"olden w\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Hello g\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"olden w\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"orld, t\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"his is \"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"a chara\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"cteriza\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"tion ru\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"n.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"completed\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Checkin\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"g now.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":1,\"step\":{\"type\":\"function_call\",\"id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":{}},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":1,\"delta\":{\"type\":\"arguments_delta\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":1,\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"requires_action\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.created\",\"interaction\":{\"id\":\"\",\"status\":\"in_progress\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.start\",\"index\":0,\"step\":{\"type\":\"model_output\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"Hello g\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"olden w\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"orld, t\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"his is \"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"a chara\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"cteriza\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"tion ru\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.delta\",\"index\":0,\"delta\":{\"type\":\"text\",\"text\":\"n.\"},\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"step.stop\",\"index\":0,\"event_id\":\"\"}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"event_type\":\"interaction.completed\",\"interaction\":{\"id\":\"\",\"status\":\"completed\",\"usage\":{\"total_input_tokens\":0,\"total_output_tokens\":0,\"total_tokens\":0}},\"event_id\":\"\"}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/gemini-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/gemini-sse.json new file mode 100644 index 00000000..4203f17f --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/gemini-sse.json @@ -0,0 +1,477 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Hello g\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"olden w\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"orld, t\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"his is \"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"a chara\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"cteriza\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"tion ru\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"n.\"}]},\"index\":0,\"finishReason\":\"STOP\"}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"functionCall\":{\"name\":\"get_weather\",\"args\":{\"city\":\"NYC\"},\"id\":\"call_g1\"}}]},\"finishReason\":\"FUNCTION_CALL\",\"index\":0}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Checkin\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"g now.\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"functionCall\":{\"name\":\"get_weather\",\"args\":{\"city\":\"SF\"},\"id\":\"call_g2\"}}]},\"finishReason\":\"FUNCTION_CALL\",\"index\":0}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Lead te\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"xt.\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"functionCall\":{\"name\":\"get_time\",\"args\":{},\"id\":\"call_g3\"}}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Tail te\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"xt.\"}]},\"finishReason\":\"FUNCTION_CALL\",\"index\":0}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"index\":0,\"finishReason\":\"MALFORMED_FUNCTION_CALL\",\"finishMessage\":\"Malformed function call: get_weather({\\\"city\\\")\"}],\"usageMetadata\":{\"promptTokenCount\":3,\"candidatesTokenCount\":1,\"totalTokenCount\":4}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Hello g\"}]},\"index\":0}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"olden w\"}]},\"index\":0}]}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Hello g\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"olden w\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"orld, t\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"his is \"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"a chara\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"cteriza\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"tion ru\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"n.\"}]},\"index\":0,\"finishReason\":\"STOP\"}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Checkin\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"g now.\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"functionCall\":{\"name\":\"get_weather\",\"args\":{\"city\":\"SF\"},\"id\":\"call_g2\"}}]},\"finishReason\":\"FUNCTION_CALL\",\"index\":0}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"Hello g\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"olden w\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"orld, t\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"his is \"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"a chara\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"cteriza\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"tion ru\"}]},\"index\":0}]}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"candidates\":[{\"content\":{\"role\":\"model\",\"parts\":[{\"text\":\"n.\"}]},\"index\":0,\"finishReason\":\"STOP\"}],\"usageMetadata\":{\"promptTokenCount\":0,\"candidatesTokenCount\":0,\"totalTokenCount\":0}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/ndjson.json b/src/__tests__/__snapshots__/frame-codec-golden/ndjson.json new file mode 100644 index 00000000..574503f0 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/ndjson.json @@ -0,0 +1,238 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"golden \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"world.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true}\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"golden \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"world.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true}\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"golden \"},\"done\":false}\n" + }, + { + "op": "return", + "completed": false + } + ], + "journal": null + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"golden \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"world.\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true}\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"golden \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"world.\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true}\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/ollama-ndjson.json b/src/__tests__/__snapshots__/frame-codec-golden/ollama-ndjson.json new file mode 100644 index 00000000..a2b73f80 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/ollama-ndjson.json @@ -0,0 +1,517 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello g\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"olden w\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"orld, t\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"his is \"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"a chara\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"cteriza\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"tion ru\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"n.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"function\":{\"name\":\"get_weather\",\"arguments\":{\"city\":\"NYC\"}}}]},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Checkin\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"g now.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"function\":{\"name\":\"get_weather\",\"arguments\":{\"city\":\"SF\"}}}]},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Lead te\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"xt.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"function\":{\"name\":\"get_time\",\"arguments\":{}}}]},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Tail te\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"xt.\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"error\":\"error parsing tool call: raw='{\\\"city\\\"', err=Expected ':' after property name in JSON at position 7\"}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello g\"},\"done\":false}\n" + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"olden w\"},\"done\":false}\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello g\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"olden w\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"orld, t\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"his is \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"a chara\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"cteriza\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"tion ru\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"n.\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Checkin\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"g now.\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\",\"tool_calls\":[{\"function\":{\"name\":\"get_weather\",\"arguments\":{\"city\":\"SF\"}}}]},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "application/x-ndjson" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"Hello g\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"olden w\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"orld, t\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"his is \"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"a chara\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"cteriza\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"tion ru\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"n.\"},\"done\":false}\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "{\"model\":\"llama3\",\"created_at\":\"\",\"message\":{\"role\":\"assistant\",\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":0,\"load_duration\":0,\"prompt_eval_count\":0,\"prompt_eval_duration\":0,\"eval_count\":0,\"eval_duration\":0}\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/openai-chat-handler.json b/src/__tests__/__snapshots__/frame-codec-golden/openai-chat-handler.json new file mode 100644 index 00000000..ca47195e --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/openai-chat-handler.json @@ -0,0 +1,1118 @@ +{ + "stream text": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream tool": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream combined": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Checkin\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"g now.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g2\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"SF\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream blocks": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Lead te\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"xt.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g3\",\"type\":\"function\",\"function\":{\"name\":\"get_time\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Tail te\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"xt.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_m1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":1,\"id\":\"call_m2\",\"type\":\"function\",\"function\":{\"name\":\"get_time\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":1,\"function\":{\"arguments\":\"{\\\"zone\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":1,\"function\":{\"arguments\":\":\\\"UTC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream truncate2": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "stream delay-text": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Checkin\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"g now.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g2\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"SF\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "stream delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "non-stream text": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "application/json" + } + }, + { + "op": "write", + "data": "{\"id\":\"\",\"object\":\"chat.completion\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"Hello golden world, this is a characterization run.\",\"refusal\":null},\"logprobs\":null,\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":13,\"total_tokens\":16}}" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "non-stream tool": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "application/json" + } + }, + { + "op": "write", + "data": "{\"id\":\"\",\"object\":\"chat.completion\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":null,\"refusal\":null,\"tool_calls\":[{\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":7,\"total_tokens\":10}}" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "non-stream combined": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "application/json" + } + }, + { + "op": "write", + "data": "{\"id\":\"\",\"object\":\"chat.completion\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"Checking now.\",\"refusal\":null,\"tool_calls\":[{\"id\":\"call_g2\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":10,\"total_tokens\":13}}" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "non-stream blocks": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "application/json" + } + }, + { + "op": "write", + "data": "{\"id\":\"\",\"object\":\"chat.completion\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"Lead text.Tail text.\",\"refusal\":null,\"tool_calls\":[{\"id\":\"call_g3\",\"type\":\"function\",\"function\":{\"name\":\"get_time\",\"arguments\":\"{}\"}}]},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":8,\"total_tokens\":11}}" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "non-stream delay-text": { + "log": [ + { + "op": "setHeader", + "name": "X-Request-Id", + "value": "" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Origin", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Methods", + "value": "GET, POST, PUT, DELETE, OPTIONS" + }, + { + "op": "setHeader", + "name": "Access-Control-Allow-Headers", + "value": "*" + }, + { + "op": "setHeader", + "name": "Access-Control-Expose-Headers", + "value": "X-Total-Count, X-Request-Id" + }, + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "application/json" + } + }, + { + "op": "write", + "data": "{\"id\":\"\",\"object\":\"chat.completion\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"Hello golden world, this is a characterization run.\",\"refusal\":null},\"logprobs\":null,\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":13,\"total_tokens\":16}}" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/openai-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/openai-sse.json new file mode 100644 index 00000000..54ffe623 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/openai-sse.json @@ -0,0 +1,1175 @@ +{ + "plain text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "plain tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "plain truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "return", + "completed": false + } + ], + "journal": null + }, + "plain delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "plain delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "usageChunk text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":5,\"total_tokens\":8}}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "usageChunk tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":5,\"total_tokens\":8}}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "usageChunk truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "return", + "completed": false + } + ], + "journal": null + }, + "usageChunk delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":5,\"total_tokens\":8}}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "usageChunk delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":5,\"total_tokens\":8}}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "openRouterProcessing text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": ": OPENROUTER PROCESSING\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "openRouterProcessing tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": ": OPENROUTER PROCESSING\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":null},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_g1\",\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\"arguments\":\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{\\\"city\\\"\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\":\\\"NYC\\\"}\"}}]},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"tool_calls\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "openRouterProcessing truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": ": OPENROUTER PROCESSING\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "return", + "completed": false + } + ], + "journal": null + }, + "openRouterProcessing delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": ": OPENROUTER PROCESSING\n\n" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "openRouterProcessing delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": ": OPENROUTER PROCESSING\n\n" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"Hello g\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"olden w\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"orld, t\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"his is \"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"a chara\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"cteriza\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"tion ru\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"n.\"},\"logprobs\":null,\"finish_reason\":null}]}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "data: {\"id\":\"\",\"object\":\"chat.completion.chunk\",\"created\":\"\",\"model\":\"gpt-4o\",\"choices\":[{\"index\":0,\"delta\":{},\"logprobs\":null,\"finish_reason\":\"stop\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + }, + { + "op": "return", + "completed": true + } + ], + "journal": null + }, + "already-ended": { + "log": [ + { + "op": "return", + "completed": true + } + ], + "journal": null + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/responses-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/responses-sse.json new file mode 100644 index 00000000..ccfe6fe4 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/responses-sse.json @@ -0,0 +1,917 @@ +{ + "text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\"{\\\"city\\\"\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\":\\\"NYC\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":0,\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Checkin\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"g now.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Checking now.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"city\\\"\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"SF\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Lead te\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"xt.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Lead text.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"{}\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":2,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"delta\":\"Tail te\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"delta\":\"xt.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"text\":\"Tail text.\"}\n\n" + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":2,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"{}\",\"status\":\"completed\"},{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\"{\\\"city\\\"\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":0,\"arguments\":\"{\\\"city\\\"\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\"\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"zone\\\"\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"UTC\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\"}\n\n" + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\"\",\"status\":\"completed\"},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":3,\"output_tokens\":10,\"total_tokens\":13}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}\n\n" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}\n\n" + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}\n\n" + }, + { + "op": "timer", + "ms": 59 + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 61 + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "timer", + "ms": 67 + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Checkin\"}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"g now.\"}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Checking now.\"}\n\n" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}\n\n" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"city\\\"\"}\n\n" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "write", + "data": "event: response.function_call_arguments.delta\ndata: {\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"SF\\\"}\"}\n\n" + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "write", + "data": "event: response.function_call_arguments.done\ndata: {\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"}\n\n" + }, + { + "op": "timer", + "ms": 59 + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}}\n\n" + }, + { + "op": "timer", + "ms": 61 + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "setHeader", + "name": "Content-Type", + "value": "text/event-stream" + }, + { + "op": "setHeader", + "name": "Cache-Control", + "value": "no-cache" + }, + { + "op": "setHeader", + "name": "Connection", + "value": "keep-alive" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "event: response.in_progress\ndata: {\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "event: response.content_part.added\ndata: {\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}\n\n" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}\n\n" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}\n\n" + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}\n\n" + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "write", + "data": "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}\n\n" + }, + { + "op": "timer", + "ms": 26.5 + }, + { + "op": "write", + "data": "event: response.output_text.done\ndata: {\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}\n\n" + }, + { + "op": "timer", + "ms": 29.5 + }, + { + "op": "write", + "data": "event: response.content_part.done\ndata: {\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}\n\n" + }, + { + "op": "timer", + "ms": 30.5 + }, + { + "op": "write", + "data": "event: response.output_item.done\ndata: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}\n\n" + }, + { + "op": "timer", + "ms": 33.5 + }, + { + "op": "write", + "data": "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/transcription-sse.json b/src/__tests__/__snapshots__/frame-codec-golden/transcription-sse.json new file mode 100644 index 00000000..b9e0a8af --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/transcription-sse.json @@ -0,0 +1,258 @@ +{ + "text": { + "log": [ + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + } + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"olden w\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"his is \"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"a trans\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"criptio\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"n.\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.done\",\"text\":\"Hello golden world, this is a transcription.\",\"languages\":[{\"code\":\"en\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + } + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"olden w\"}\n\n" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + } + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"olden w\"}\n\n" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"his is \"}\n\n" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"a trans\"}\n\n" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"criptio\"}\n\n" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"n.\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.done\",\"text\":\"Hello golden world, this is a transcription.\",\"languages\":[{\"code\":\"en\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "writeHead", + "status": 200, + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + } + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"Hello g\"}\n\n" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"olden w\"}\n\n" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"orld, t\"}\n\n" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"his is \"}\n\n" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"a trans\"}\n\n" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"criptio\"}\n\n" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.delta\",\"delta\":\"n.\"}\n\n" + }, + { + "op": "write", + "data": "data: {\"type\":\"transcript.text.done\",\"text\":\"Hello golden world, this is a transcription.\",\"languages\":[{\"code\":\"en\"}]}\n\n" + }, + { + "op": "write", + "data": "data: [DONE]\n\n" + }, + { + "op": "end" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/ws-gemini-live.json b/src/__tests__/__snapshots__/frame-codec-golden/ws-gemini-live.json new file mode 100644 index 00000000..38d145be --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/ws-gemini-live.json @@ -0,0 +1,382 @@ +{ + "text": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Hello g\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"olden w\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"orld, t\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"his is \"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"a chara\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"cteriza\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"tion ru\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"n.\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"toolCall\":{\"functionCalls\":[{\"name\":\"get_weather\",\"args\":{\"city\":\"NYC\"},\"id\":\"call_g1\"}]}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Checkin\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"g now.\"}]}}}" + }, + { + "op": "send", + "data": "{\"toolCall\":{\"functionCalls\":[{\"name\":\"get_weather\",\"args\":{\"city\":\"SF\"},\"id\":\"call_g2\"}]}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Lead te\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"xt.\"}]}}}" + }, + { + "op": "send", + "data": "{\"toolCall\":{\"functionCalls\":[{\"name\":\"get_time\",\"args\":{},\"id\":\"call_g3\"}]}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Tail te\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"xt.\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true,\"turnCompleteReason\":\"MALFORMED_FUNCTION_CALL\"}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Hello g\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"olden w\"}]}}}" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Hello g\"}]}}}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"olden w\"}]}}}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"orld, t\"}]}}}" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"his is \"}]}}}" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"a chara\"}]}}}" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"cteriza\"}]}}}" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"tion ru\"}]}}}" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"n.\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Checkin\"}]}}}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"g now.\"}]}}}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"toolCall\":{\"functionCalls\":[{\"name\":\"get_weather\",\"args\":{\"city\":\"SF\"},\"id\":\"call_g2\"}]}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "send", + "data": "{\"setupComplete\":{}}" + }, + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"Hello g\"}]}}}" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"olden w\"}]}}}" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"orld, t\"}]}}}" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"his is \"}]}}}" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"a chara\"}]}}}" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"cteriza\"}]}}}" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"tion ru\"}]}}}" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "send", + "data": "{\"serverContent\":{\"modelTurn\":{\"parts\":[{\"text\":\"n.\"}]}}}" + }, + { + "op": "send", + "data": "{\"serverContent\":{\"turnComplete\":true}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/ws-realtime.json b/src/__tests__/__snapshots__/frame-codec-golden/ws-realtime.json new file mode 100644 index 00000000..8b724667 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/ws-realtime.json @@ -0,0 +1,507 @@ +{ + "text": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.session\",\"model\":\"gpt-4o-realtime-preview\",\"expires_at\":1767326645,\"modalities\":[\"text\"],\"instructions\":\"\",\"tools\":[],\"tool_choice\":\"auto\",\"temperature\":0.8,\"max_response_output_tokens\":\"inf\",\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null},\"output\":{\"format\":null,\"voice\":null}},\"type\":\"realtime\",\"reasoning\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"previous_item_id\":null,\"item\":{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"golden probe\"}],\"id\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"in_progress\",\"status_details\":null,\"output\":[],\"usage\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"in_progress\",\"content\":[],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.done\",\"item\":{\"id\":\"\",\"object\":\"realtime.item\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.done\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"completed\",\"output\":[{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]}],\"usage\":{\"total_tokens\":0,\"input_tokens\":0,\"output_tokens\":0}},\"event_id\":\"\"}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-text": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.session\",\"model\":\"gpt-4o-realtime-preview\",\"expires_at\":1767326645,\"modalities\":[\"text\"],\"instructions\":\"\",\"tools\":[],\"tool_choice\":\"auto\",\"temperature\":0.8,\"max_response_output_tokens\":\"inf\",\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null},\"output\":{\"format\":null,\"voice\":null}},\"type\":\"realtime\",\"reasoning\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"previous_item_id\":null,\"item\":{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"golden probe\"}],\"id\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"in_progress\",\"status_details\":null,\"output\":[],\"usage\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"in_progress\",\"content\":[],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.done\",\"item\":{\"id\":\"\",\"object\":\"realtime.item\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.done\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"completed\",\"output\":[{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]}],\"usage\":{\"total_tokens\":0,\"input_tokens\":0,\"output_tokens\":0}},\"event_id\":\"\"}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "profile": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.session\",\"model\":\"gpt-4o-realtime-preview\",\"expires_at\":1767326645,\"modalities\":[\"text\"],\"instructions\":\"\",\"tools\":[],\"tool_choice\":\"auto\",\"temperature\":0.8,\"max_response_output_tokens\":\"inf\",\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null},\"output\":{\"format\":null,\"voice\":null}},\"type\":\"realtime\",\"reasoning\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"previous_item_id\":null,\"item\":{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"golden probe\"}],\"id\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"in_progress\",\"status_details\":null,\"output\":[],\"usage\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"in_progress\",\"content\":[],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.done\",\"item\":{\"id\":\"\",\"object\":\"realtime.item\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.done\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"completed\",\"output\":[{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"completed\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\"}]}],\"usage\":{\"total_tokens\":0,\"input_tokens\":0,\"output_tokens\":0}},\"event_id\":\"\"}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.session\",\"model\":\"gpt-4o-realtime-preview\",\"expires_at\":1767326645,\"modalities\":[\"text\"],\"instructions\":\"\",\"tools\":[],\"tool_choice\":\"auto\",\"temperature\":0.8,\"max_response_output_tokens\":\"inf\",\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null},\"output\":{\"format\":null,\"voice\":null}},\"type\":\"realtime\",\"reasoning\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"previous_item_id\":null,\"item\":{\"type\":\"message\",\"role\":\"user\",\"content\":[{\"type\":\"input_text\",\"text\":\"golden probe\"}],\"id\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"realtime.response\",\"status\":\"in_progress\",\"status_details\":null,\"output\":[],\"usage\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"response_id\":\"\",\"output_index\":0,\"item\":{\"id\":\"\",\"type\":\"message\",\"role\":\"assistant\",\"status\":\"in_progress\",\"content\":[],\"phase\":\"final_answer\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\"},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"response_id\":\"\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\",\"event_id\":\"\"}" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "transcription profile": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"transcription_session.updated\",\"session\":{\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":{\"model\":\"gpt-live-transcribe-2026-07-01\"},\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"input_audio_buffer.committed\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"item\":{\"type\":\"message\",\"id\":\"\",\"role\":\"user\",\"content\":[{\"type\":\"input_audio\",\"transcript\":null}]},\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 5 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"Live go\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 20 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"lden ca\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 20 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"ption f\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 20 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"or the \",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 20 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"profile\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 20 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\" path.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.completed\",\"item_id\":\"\",\"content_index\":0,\"transcript\":\"Live golden caption for the profile path.\",\"usage\":{\"type\":\"duration\",\"seconds\":0},\"event_id\":\"\"}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "transcription delay-text": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"transcription_session.updated\",\"session\":{\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":{\"model\":\"gpt-live-transcribe-2026-07-01\"},\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"input_audio_buffer.committed\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"item\":{\"type\":\"message\",\"id\":\"\",\"role\":\"user\",\"content\":[{\"type\":\"input_audio\",\"transcript\":null}]},\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"Live go\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"lden ca\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"ption f\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"or the \",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"profile\",\"event_id\":\"\"}" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\" path.\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.completed\",\"item_id\":\"\",\"content_index\":0,\"transcript\":\"Live golden caption for the profile path.\",\"usage\":{\"type\":\"duration\",\"seconds\":0},\"event_id\":\"\"}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "transcription truncate2": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"session.created\",\"session\":{\"id\":\"\",\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":null,\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"transcription_session.updated\",\"session\":{\"object\":\"realtime.transcription_session\",\"type\":\"transcription\",\"expires_at\":1767326645,\"audio\":{\"input\":{\"format\":null,\"noise_reduction\":null,\"transcription\":{\"model\":\"gpt-live-transcribe-2026-07-01\"},\"turn_detection\":null}},\"include\":null},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"input_audio_buffer.committed\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.added\",\"item\":{\"type\":\"message\",\"id\":\"\",\"role\":\"user\",\"content\":[{\"type\":\"input_audio\",\"transcript\":null}]},\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"Live go\",\"event_id\":\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"conversation.item.input_audio_transcription.delta\",\"item_id\":\"\",\"content_index\":0,\"delta\":\"lden ca\",\"event_id\":\"\"}" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + } +} diff --git a/src/__tests__/__snapshots__/frame-codec-golden/ws-responses.json b/src/__tests__/__snapshots__/frame-codec-golden/ws-responses.json new file mode 100644 index 00000000..77975bd8 --- /dev/null +++ b/src/__tests__/__snapshots__/frame-codec-golden/ws-responses.json @@ -0,0 +1,758 @@ +{ + "text": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "tool": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\"{\\\"city\\\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\":\\\"NYC\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":0,\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\",\"status\":\"completed\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"NYC\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "combined": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Checkin\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"g now.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Checking now.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"city\\\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"SF\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "blocks": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Lead te\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"xt.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Lead text.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"{}\",\"status\":\"completed\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":2,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"delta\":\"Tail te\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"delta\":\"xt.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"text\":\"Tail text.\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":2,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":2,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Lead text.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g3\",\"name\":\"get_time\",\"arguments\":\"{}\",\"status\":\"completed\"},{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Tail text.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "misbehavior": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":0,\"delta\":\"{\\\"city\\\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":0,\"arguments\":\"{\\\"city\\\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\"\",\"status\":\"completed\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"zone\\\"\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"UTC\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\"}" + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\",\"status\":\"completed\"}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m1\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\"\",\"status\":\"completed\"},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_m2\",\"name\":\"get_time\",\"arguments\":\"{\\\"zone\\\":\\\"UTC\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":3,\"output_tokens\":10,\"total_tokens\":13}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "truncate2": { + "log": [ + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "destroy" + } + ], + "journal": { + "status": 200, + "interrupted": true, + "interruptReason": "truncateAfterChunks" + } + }, + "delay-text": { + "log": [ + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}" + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}" + }, + { + "op": "timer", + "ms": 59 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 61 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}" + }, + { + "op": "timer", + "ms": 67 + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-combined": { + "log": [ + { + "op": "timer", + "ms": 7 + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 11 + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 13 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "timer", + "ms": 17 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 19 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Checkin\"}" + }, + { + "op": "timer", + "ms": 23 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"g now.\"}" + }, + { + "op": "timer", + "ms": 29 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Checking now.\"}" + }, + { + "op": "timer", + "ms": 31 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 37 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]}}" + }, + { + "op": "timer", + "ms": 41 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"\",\"status\":\"in_progress\"}}" + }, + { + "op": "timer", + "ms": 43 + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\"{\\\"city\\\"\"}" + }, + { + "op": "timer", + "ms": 47 + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.delta\",\"item_id\":\"\",\"output_index\":1,\"delta\":\":\\\"SF\\\"}\"}" + }, + { + "op": "timer", + "ms": 53 + }, + { + "op": "send", + "data": "{\"type\":\"response.function_call_arguments.done\",\"item_id\":\"\",\"output_index\":1,\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\"}" + }, + { + "op": "timer", + "ms": 59 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}}" + }, + { + "op": "timer", + "ms": 61 + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Checking now.\",\"annotations\":[]}]},{\"type\":\"function_call\",\"id\":\"\",\"call_id\":\"call_g2\",\"name\":\"get_weather\",\"arguments\":\"{\\\"city\\\":\\\"SF\\\"}\",\"status\":\"completed\"}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + }, + "delay-replay2": { + "log": [ + { + "op": "timer", + "ms": 3.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.created\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 5.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.in_progress\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"in_progress\",\"output\":[]}}" + }, + { + "op": "timer", + "ms": 6.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"in_progress\",\"role\":\"assistant\",\"content\":[]}}" + }, + { + "op": "timer", + "ms": 8.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.added\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 9.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"Hello g\"}" + }, + { + "op": "timer", + "ms": 11.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"olden w\"}" + }, + { + "op": "timer", + "ms": 14.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"orld, t\"}" + }, + { + "op": "timer", + "ms": 15.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"his is \"}" + }, + { + "op": "timer", + "ms": 18.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"a chara\"}" + }, + { + "op": "timer", + "ms": 20.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"cteriza\"}" + }, + { + "op": "timer", + "ms": 21.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"tion ru\"}" + }, + { + "op": "timer", + "ms": 23.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.delta\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"delta\":\"n.\"}" + }, + { + "op": "timer", + "ms": 26.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_text.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"text\":\"Hello golden world, this is a characterization run.\"}" + }, + { + "op": "timer", + "ms": 29.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.content_part.done\",\"item_id\":\"\",\"output_index\":0,\"content_index\":0,\"part\":{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}}" + }, + { + "op": "timer", + "ms": 30.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}}" + }, + { + "op": "timer", + "ms": 33.5 + }, + { + "op": "send", + "data": "{\"type\":\"response.completed\",\"response\":{\"id\":\"\",\"object\":\"response\",\"created_at\":\"\",\"model\":\"gpt-4o\",\"status\":\"completed\",\"output\":[{\"type\":\"message\",\"id\":\"\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"Hello golden world, this is a characterization run.\",\"annotations\":[]}]}],\"usage\":{\"input_tokens\":0,\"output_tokens\":0,\"total_tokens\":0}}}" + } + ], + "journal": { + "status": 200, + "interrupted": null, + "interruptReason": null + } + } +} diff --git a/src/__tests__/aws-event-stream.test.ts b/src/__tests__/aws-event-stream.test.ts index 6245fbdb..a01c998f 100644 --- a/src/__tests__/aws-event-stream.test.ts +++ b/src/__tests__/aws-event-stream.test.ts @@ -316,6 +316,31 @@ describe("writeEventStream", () => { expect(headers()["Content-Type"]).toBeUndefined(); }); + it("returns true when res.writableEnded is already true and there are no events", async () => { + const { res } = makeMockResponse(); + Object.defineProperty(res, "writableEnded", { get: () => true }); + expect(await writeEventStream(res, [])).toBe(true); + }); + + it("returns true when the response ends mid-stream (only an abort returns false)", async () => { + const { res } = makeMockResponse(); + let ended = false; + Object.defineProperty(res, "writableEnded", { get: () => ended }); + const events = [ + { eventType: "test", payload: { n: 1 } }, + { eventType: "test", payload: { n: 2 } }, + ]; + let sent = 0; + const result = await writeEventStream(res, events, { + onChunkSent: () => { + sent++; + ended = true; + }, + }); + expect(result).toBe(true); + expect(sent).toBe(1); + }); + it("supports streaming profile delays", async () => { vi.useFakeTimers(); const { res } = makeMockResponse(); diff --git a/src/__tests__/chunk-size-invalid-served.test.ts b/src/__tests__/chunk-size-invalid-served.test.ts new file mode 100644 index 00000000..8195bd87 --- /dev/null +++ b/src/__tests__/chunk-size-invalid-served.test.ts @@ -0,0 +1,107 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { LLMock } from "../llmock.js"; +import type { Fixture } from "../types.js"; + +// Base (508dd9b9) served every one of these chunkSize values: the callers +// clamp with Math.max(1, ...), so 0 and -1 become 1, and a value that is not +// a number becomes NaN, which sized a zero non-streaming wait. These rows hold +// that behavior over HTTP, through code-built fixtures and server options. + +// A fixture or server option built in code (or loaded from JSON) can carry a +// non-numeric chunkSize at runtime, even though the type says number. +const NOT_A_NUMBER = "abc" as unknown as number; + +const VALUES: Array<[string, number | undefined]> = [ + ["NaN", Number.NaN], + ['"abc"', NOT_A_NUMBER], + ["0", 0], + ["-1", -1], + ["undefined", undefined], +]; +const PROVIDERS = ["openai", "anthropic"] as const; +const TIMINGS = ["off", "nonStreamingTiming", "nonStreamingTimingAllProviders"] as const; +const WHERE = ["fixture", "server"] as const; + +const TTFT = 300; + +type Provider = (typeof PROVIDERS)[number]; + +const rows = WHERE.flatMap((where) => + VALUES.flatMap(([label, value]) => + PROVIDERS.flatMap((provider) => + [false, true].flatMap((stream) => + TIMINGS.map((timing) => ({ where, label, value, provider, stream, timing })), + ), + ), + ), +); + +function request(url: string, provider: Provider, stream: boolean): Promise { + const isOpenAI = provider === "openai"; + return fetch(`${url}${isOpenAI ? "/v1/chat/completions" : "/v1/messages"}`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + model: isOpenAI ? "gpt-4o" : "claude-3", + ...(isOpenAI ? {} : { max_tokens: 10 }), + stream, + messages: [{ role: "user", content: "hi" }], + }), + }); +} + +function nonStreamingContent(provider: Provider, json: unknown): unknown { + if (provider === "openai") { + return (json as { choices: Array<{ message: { content: unknown } }> }).choices[0].message + .content; + } + return (json as { content: Array<{ text: unknown }> }).content[0].text; +} + +describe("an invalid chunkSize is served as base served it", () => { + let mock: LLMock | null = null; + + afterEach(async () => { + if (mock) { + await mock.stop(); + mock = null; + } + }); + + it.each(rows)( + "$where chunkSize $label, $provider, stream=$stream, timing=$timing", + async ({ where, value, provider, stream, timing }) => { + mock = new LLMock({ port: 0, ...(where === "server" && { chunkSize: value }) }); + const fixture: Fixture = { + match: { userMessage: "hi" }, + response: { content: "hello world" }, + ...(where === "fixture" && value !== undefined && { chunkSize: value }), + ...(timing !== "off" && { streamingProfile: { ttft: TTFT, tps: 1000 } }), + ...(timing === "nonStreamingTiming" && { nonStreamingTiming: true }), + ...(timing === "nonStreamingTimingAllProviders" && { + nonStreamingTimingAllProviders: true, + }), + }; + mock.addFixture(fixture); + await mock.start(); + + const started = Date.now(); + const res = await request(mock.url, provider, stream); + const text = await res.text(); + const elapsed = Date.now() - started; + + expect(res.status).toBe(200); + if (stream) return; + expect(nonStreamingContent(provider, JSON.parse(text))).toBe("hello world"); + if (timing === "off") return; + // A non-numeric chunkSize clamps to NaN, which base sized as a zero wait. + const nonNumeric = value !== undefined && Number.isNaN(Number(value)); + const waits = + !nonNumeric && + (timing === "nonStreamingTimingAllProviders" || + (timing === "nonStreamingTiming" && provider === "openai")); + if (waits) expect(elapsed).toBeGreaterThanOrEqual(TTFT - 50); + else expect(elapsed).toBeLessThan(TTFT - 50); + }, + ); +}); diff --git a/src/__tests__/delay-abort-listener-leak.test.ts b/src/__tests__/delay-abort-listener-leak.test.ts new file mode 100644 index 00000000..06683527 --- /dev/null +++ b/src/__tests__/delay-abort-listener-leak.test.ts @@ -0,0 +1,145 @@ +import { describe, it, expect, afterEach } from "vitest"; +import { getEventListeners } from "node:events"; +import { PassThrough } from "node:stream"; +import type * as http from "node:http"; +import { delay, writeSSEStream } from "../sse-writer.js"; +import { createServer, type ServerInstance } from "../server.js"; +import type { SSEChunk } from "../types.js"; + +function abortListenerCount(signal: AbortSignal): number { + return getEventListeners(signal, "abort").length; +} + +function makeChunk(i: number): SSEChunk { + return { + id: `chunk-${i}`, + object: "chat.completion.chunk", + created: 1700000000, + model: "gpt-4", + choices: [{ index: 0, delta: { content: `t${i}` }, logprobs: null, finish_reason: null }], + }; +} + +function makeSink(): http.ServerResponse { + const stream = new PassThrough(); + stream.resume(); + let ended = false; + const res = { + setHeader() {}, + writeHead() {}, + write(data: string) { + stream.write(data); + }, + end(data?: string) { + if (data !== undefined) stream.write(data); + ended = true; + stream.end(); + }, + get writableEnded() { + return ended; + }, + }; + return res as unknown as http.ServerResponse; +} + +describe("delay() abort listener cleanup", () => { + it("removes its abort listener when the timer fires normally", async () => { + const controller = new AbortController(); + for (let i = 0; i < 25; i++) { + await delay(1, controller.signal); + } + expect(abortListenerCount(controller.signal)).toBe(0); + }); + + it("still resolves early on abort and leaves no listener behind", async () => { + const controller = new AbortController(); + const start = Date.now(); + const p = delay(10_000, controller.signal); + expect(abortListenerCount(controller.signal)).toBe(1); + controller.abort(); + await p; + expect(Date.now() - start).toBeLessThan(1000); + expect(abortListenerCount(controller.signal)).toBe(0); + }); + + it("a 60-frame profiled stream does not accumulate abort listeners", async () => { + const controller = new AbortController(); + const chunks = Array.from({ length: 60 }, (_, i) => makeChunk(i)); + const counts: number[] = []; + const result = await writeSSEStream(makeSink(), chunks, { + // tps 1000 → ~1ms per frame, so every frame goes through delay(signal). + streamingProfile: { ttft: 1, tps: 1000 }, + signal: controller.signal, + onChunkSent: () => counts.push(abortListenerCount(controller.signal)), + }); + expect(result).toBe(true); + expect(counts).toHaveLength(60); + expect(Math.max(...counts)).toBe(0); + expect(abortListenerCount(controller.signal)).toBe(0); + }); +}); + +describe("profiled HTTP stream: no MaxListenersExceededWarning", () => { + let instance: ServerInstance | null = null; + const warnings: Error[] = []; + const onWarning = (w: Error) => warnings.push(w); + + // Peak abort-listener count seen on any single AbortSignal during the request. + let peak = 0; + const originalAdd = EventTarget.prototype.addEventListener; + const trackAbortListeners = (): void => { + peak = 0; + Object.defineProperty(AbortSignal.prototype, "addEventListener", { + configurable: true, + writable: true, + value: function (this: AbortSignal, ...args: Parameters) { + originalAdd.apply(this, args); + if (args[0] === "abort") peak = Math.max(peak, abortListenerCount(this)); + }, + }); + }; + + afterEach(async () => { + Reflect.deleteProperty(AbortSignal.prototype, "addEventListener"); + process.off("warning", onWarning); + warnings.length = 0; + if (instance) { + await new Promise((resolve) => instance!.server.close(() => resolve())); + instance = null; + } + }); + + it("streams 60+ delayed frames over the real server without a listener-leak warning", async () => { + process.on("warning", onWarning); + trackAbortListeners(); + instance = await createServer([ + { + match: { userMessage: "leak" }, + response: { content: "x".repeat(64) }, + chunkSize: 1, + streamingProfile: { ttft: 1, tps: 1000 }, + }, + ]); + const res = await fetch(`${instance.url}/v1/chat/completions`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + model: "gpt-4", + stream: true, + messages: [{ role: "user", content: "leak" }], + }), + }); + expect(res.status).toBe(200); + const body = await res.text(); + const frames = body.split("\n\n").filter((f) => f.startsWith("data: {")); + expect(frames.length).toBeGreaterThanOrEqual(60); + expect(body).toContain("data: [DONE]"); + // Let any deferred process 'warning' emission land. + await new Promise((r) => setImmediate(r)); + const leak = warnings.filter((w) => w.name === "MaxListenersExceededWarning"); + expect(leak.map((w) => w.message)).toEqual([]); + // Node 26 never emits the warning, so also bound the per-signal peak directly. + expect(peak).toBeGreaterThan(0); + expect(peak).toBeLessThanOrEqual(3); + }); +}); diff --git a/src/__tests__/delivery-lifecycle/anthropic.test.ts b/src/__tests__/delivery-lifecycle/anthropic.test.ts new file mode 100644 index 00000000..247ae28c --- /dev/null +++ b/src/__tests__/delivery-lifecycle/anthropic.test.ts @@ -0,0 +1,39 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const messages = [{ role: "user", content: HARNESS_MATCH }]; +const event = (type: string, data: Record) => + `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`; + +describeDeliveryLifecycle({ + name: "Anthropic Messages", + streamPath: "/v1/messages", + nonStreamPath: "/v1/messages", + streamBody: { model: "claude-3-5-sonnet-20241022", max_tokens: 256, messages, stream: true }, + nonStreamBody: { model: "claude-3-5-sonnet-20241022", max_tokens: 256, messages, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "anthropic", + contentType: "text/event-stream", + frames: [ + event("message_start", { + message: { + id: "msg_up", + type: "message", + role: "assistant", + content: [], + model: "claude-3-5-sonnet-20241022", + stop_reason: null, + usage: { input_tokens: 1, output_tokens: 0 }, + }, + }), + event("content_block_start", { index: 0, content_block: { type: "text", text: "" } }), + event("content_block_delta", { index: 0, delta: { type: "text_delta", text: "alpha " } }), + event("content_block_delta", { index: 0, delta: { type: "text_delta", text: "beta" } }), + event("content_block_stop", { index: 0 }), + event("message_delta", { delta: { stop_reason: "end_turn" }, usage: { output_tokens: 2 } }), + event("message_stop", {}), + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/bedrock-converse.test.ts b/src/__tests__/delivery-lifecycle/bedrock-converse.test.ts new file mode 100644 index 00000000..6b40214a --- /dev/null +++ b/src/__tests__/delivery-lifecycle/bedrock-converse.test.ts @@ -0,0 +1,37 @@ +import { encodeEventStreamMessage } from "../../aws-event-stream.js"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const MODEL = "anthropic.claude-3-5-sonnet-20241022-v2:0"; +const body = { messages: [{ role: "user", content: [{ text: HARNESS_MATCH }] }] }; + +describeDeliveryLifecycle({ + name: "Bedrock Converse", + streamPath: `/model/${MODEL}/converse-stream`, + nonStreamPath: `/model/${MODEL}/converse`, + streamBody: body, + nonStreamBody: body, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "bedrock", + contentType: "application/vnd.amazon.eventstream", + frames: [ + encodeEventStreamMessage("messageStart", { role: "assistant" }), + encodeEventStreamMessage("contentBlockDelta", { + contentBlockIndex: 0, + delta: { text: "alpha " }, + }), + encodeEventStreamMessage("contentBlockDelta", { + contentBlockIndex: 0, + delta: { text: "beta" }, + }), + encodeEventStreamMessage("contentBlockStop", { contentBlockIndex: 0 }), + encodeEventStreamMessage("messageStop", { stopReason: "end_turn" }), + encodeEventStreamMessage("metadata", { + usage: { inputTokens: 1, outputTokens: 2, totalTokens: 3 }, + metrics: { latencyMs: 0 }, + }), + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/bedrock-invoke.test.ts b/src/__tests__/delivery-lifecycle/bedrock-invoke.test.ts new file mode 100644 index 00000000..0d2c2f6f --- /dev/null +++ b/src/__tests__/delivery-lifecycle/bedrock-invoke.test.ts @@ -0,0 +1,58 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; +import { encodeEventStreamMessage } from "../../aws-event-stream.js"; + +const MODEL = "anthropic.claude-3-5-sonnet-20241022-v2:0"; +const body = { + anthropic_version: "bedrock-2023-05-31", + max_tokens: 256, + messages: [{ role: "user", content: HARNESS_MATCH }], +}; +// One InvokeModel stream frame: a `chunk` event whose payload is a base64 +// PayloadPart wrapping a Claude stream event, as the real service sends it. +const chunk = (event: Record): Buffer => + encodeEventStreamMessage("chunk", { + bytes: Buffer.from(JSON.stringify(event), "utf8").toString("base64"), + }); + +describeDeliveryLifecycle({ + name: "Bedrock InvokeModel", + streamPath: `/model/${MODEL}/invoke-with-response-stream`, + nonStreamPath: `/model/${MODEL}/invoke`, + streamBody: body, + nonStreamBody: body, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "bedrock", + contentType: "application/vnd.amazon.eventstream", + frames: [ + chunk({ + type: "message_start", + message: { + id: "msg_up", + type: "message", + role: "assistant", + content: [], + model: MODEL, + stop_reason: null, + usage: { input_tokens: 1, output_tokens: 0 }, + }, + }), + chunk({ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }), + chunk({ + type: "content_block_delta", + index: 0, + delta: { type: "text_delta", text: "alpha " }, + }), + chunk({ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "beta" } }), + chunk({ type: "content_block_stop", index: 0 }), + chunk({ + type: "message_delta", + delta: { stop_reason: "end_turn" }, + usage: { output_tokens: 2 }, + }), + chunk({ type: "message_stop" }), + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/cohere.test.ts b/src/__tests__/delivery-lifecycle/cohere.test.ts new file mode 100644 index 00000000..195c80cf --- /dev/null +++ b/src/__tests__/delivery-lifecycle/cohere.test.ts @@ -0,0 +1,83 @@ +import { expect, test } from "vitest"; +import { LLMock } from "../helpers/misbehavior-enabled.js"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const messages = [{ role: "user", content: HARNESS_MATCH }]; +const event = (type: string, data: Record) => + `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`; + +describeDeliveryLifecycle({ + name: "Cohere v2 Chat", + streamPath: "/v2/chat", + nonStreamPath: "/v2/chat", + streamBody: { model: "command-r-plus", messages, stream: true }, + nonStreamBody: { model: "command-r-plus", messages, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "cohere", + contentType: "text/event-stream", + frames: [ + event("message-start", { + id: "msg_up", + delta: { + message: { role: "assistant", content: [], tool_plan: "", tool_calls: [], citations: [] }, + }, + }), + event("content-start", { + index: 0, + delta: { message: { content: { type: "text", text: "" } } }, + }), + event("content-delta", { index: 0, delta: { message: { content: { text: "alpha " } } } }), + event("content-delta", { index: 0, delta: { message: { content: { text: "beta" } } } }), + event("content-end", { index: 0 }), + event("message-end", { + delta: { + finish_reason: "COMPLETE", + usage: { tokens: { input_tokens: 1, output_tokens: 2 } }, + }, + }), + ], + }, +}); + +test("a K4 duplicate-id reply extends the recorded gaps before sizing the non-streaming wait", async () => { + // The same hand-computed total as the OpenAI chat K4 test: two `weather` + // calls with `{"city":"Paris"}` (7 + 16 chars each) are 46 chars, 5 chunks at + // chunkSize 10. The recorded gaps [0, 60] extend to 4 gaps by repeating the + // last one: 0 + 0 + 3 * 60 = 180ms. Without the extension the 2 extra + // chunks would average the recorded gaps (30ms): 120ms. + const timings = { ttftMs: 0, interChunkDelaysMs: [0, 60], totalDurationMs: 60 }; + const mock = new LLMock({ port: 0, logLevel: "silent" }); + mock.addFixture({ + match: { userMessage: "weather" }, + misbehavior: "tool-call-id-duplicate", + response: { toolCalls: [{ name: "weather", arguments: '{"city":"Paris"}' }] }, + recordedTimings: timings, + nonStreamingTimingAllProviders: true, + chunkSize: 10, + }); + const url = await mock.start(); + try { + const started = performance.now(); + const res = await fetch(`${url}/v2/chat`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "command-r-plus", + messages: [{ role: "user", content: "weather" }], + stream: false, + }), + }); + const body = (await res.json()) as { message: { tool_calls: Array<{ id: string }> } }; + const elapsed = performance.now() - started; + expect(res.status).toBe(200); + const calls = body.message.tool_calls; + expect(calls).toHaveLength(2); + expect(calls[0].id).toBe(calls[1].id); + expect(elapsed).toBeGreaterThanOrEqual(175); + } finally { + await mock.stop(); + } +}, 10000); diff --git a/src/__tests__/delivery-lifecycle/gemini-interactions.test.ts b/src/__tests__/delivery-lifecycle/gemini-interactions.test.ts new file mode 100644 index 00000000..67980b59 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/gemini-interactions.test.ts @@ -0,0 +1,33 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const event = (data: Record) => `data: ${JSON.stringify(data)}\n\n`; + +describeDeliveryLifecycle({ + name: "Gemini Interactions", + streamPath: "/v1beta/interactions", + nonStreamPath: "/v1beta/interactions", + streamBody: { model: "gemini-2.5-flash", input: HARNESS_MATCH, stream: true }, + nonStreamBody: { model: "gemini-2.5-flash", input: HARNESS_MATCH, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + // Gemini Interactions proxies to the configured Gemini upstream. + provider: "gemini", + contentType: "text/event-stream", + frames: [ + event({ + event_type: "interaction.created", + interaction: { id: "int_up", status: "in_progress" }, + }), + event({ event_type: "step.start", index: 0, step: { type: "model_output" } }), + event({ event_type: "step.delta", index: 0, delta: { type: "text", text: "alpha " } }), + event({ event_type: "step.delta", index: 0, delta: { type: "text", text: "beta" } }), + event({ event_type: "step.stop", index: 0 }), + event({ + event_type: "interaction.completed", + interaction: { id: "int_up", status: "completed" }, + }), + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/gemini.test.ts b/src/__tests__/delivery-lifecycle/gemini.test.ts new file mode 100644 index 00000000..00b2ae76 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/gemini.test.ts @@ -0,0 +1,41 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const body = { contents: [{ role: "user", parts: [{ text: HARNESS_MATCH }] }] }; +const frame = (text: string, finishReason?: string) => + `data: ${JSON.stringify({ + candidates: [ + { + content: { role: "model", parts: [{ text }] }, + ...(finishReason && { finishReason }), + index: 0, + }, + ], + })}\r\n\r\n`; +const upstreamFrames = [frame("alpha "), frame("beta "), frame("gamma "), frame("", "STOP")]; + +describeDeliveryLifecycle({ + name: "Gemini generateContent", + streamPath: "/v1beta/models/gemini-2.0-flash:streamGenerateContent?alt=sse", + nonStreamPath: "/v1beta/models/gemini-2.0-flash:generateContent", + streamBody: body, + nonStreamBody: body, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { provider: "gemini", contentType: "text/event-stream", frames: upstreamFrames }, +}); + +const vertexModel = + "/v1/projects/p/locations/us-central1/publishers/google/models/gemini-2.0-flash"; + +describeDeliveryLifecycle({ + name: "Vertex AI generateContent", + streamPath: `${vertexModel}:streamGenerateContent?alt=sse`, + nonStreamPath: `${vertexModel}:generateContent`, + streamBody: body, + nonStreamBody: body, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { provider: "vertexai", contentType: "text/event-stream", frames: upstreamFrames }, +}); diff --git a/src/__tests__/delivery-lifecycle/ollama.test.ts b/src/__tests__/delivery-lifecycle/ollama.test.ts new file mode 100644 index 00000000..9ba3bd18 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/ollama.test.ts @@ -0,0 +1,209 @@ +import * as http from "node:http"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; +import { LLMock } from "../helpers/misbehavior-enabled.js"; +import type { Fixture } from "../../types.js"; + +const messages = [{ role: "user", content: HARNESS_MATCH }]; +const line = (obj: Record) => JSON.stringify(obj) + "\n"; +const created_at = "2024-01-01T00:00:00Z"; + +describeDeliveryLifecycle({ + name: "Ollama chat", + streamPath: "/api/chat", + nonStreamPath: "/api/chat", + streamBody: { model: "llama3", messages }, + // Ollama streams unless the request sends `stream: false` explicitly. + nonStreamBody: { model: "llama3", messages, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "ollama", + contentType: "application/x-ndjson", + frames: [ + line({ + model: "llama3", + created_at, + message: { role: "assistant", content: "alpha " }, + done: false, + }), + line({ + model: "llama3", + created_at, + message: { role: "assistant", content: "beta " }, + done: false, + }), + line({ + model: "llama3", + created_at, + message: { role: "assistant", content: "gamma" }, + done: false, + }), + line({ + model: "llama3", + created_at, + message: { role: "assistant", content: "" }, + done: true, + }), + ], + }, +}); + +describeDeliveryLifecycle({ + name: "Ollama generate", + streamPath: "/api/generate", + nonStreamPath: "/api/generate", + streamBody: { model: "llama3", prompt: HARNESS_MATCH }, + nonStreamBody: { model: "llama3", prompt: HARNESS_MATCH, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "ollama", + contentType: "application/x-ndjson", + frames: [ + line({ model: "llama3", created_at, response: "alpha ", done: false }), + line({ model: "llama3", created_at, response: "beta ", done: false }), + line({ model: "llama3", created_at, response: "gamma", done: false }), + line({ model: "llama3", created_at, response: "", done: true }), + ], + }, +}); + +// ─── Misbehavior non-stream replies (plan n1) ────────────────────────────── + +interface Reply { + status: number; + elapsedMs: number; + body: string; +} + +function post(url: string, pathName: string, body: Record): Promise { + const start = Date.now(); + return new Promise((resolve, reject) => { + const req = http.request(new URL(pathName, url), { + method: "POST", + agent: false, + headers: { "content-type": "application/json" }, + }); + req.on("response", (res) => { + const chunks: Buffer[] = []; + res.on("data", (c: Buffer) => chunks.push(c)); + res.on("end", () => + resolve({ + status: res.statusCode ?? 0, + elapsedMs: Date.now() - start, + body: Buffer.concat(chunks).toString("utf8"), + }), + ); + res.on("error", reject); + }); + req.on("error", reject); + req.end(JSON.stringify(body)); + }); +} + +describe("delivery lifecycle: Ollama chat misbehavior non-stream replies", () => { + let mock: LLMock; + const weatherTool = { + type: "function", + function: { + name: "weather", + parameters: { type: "object", properties: { city: { type: "string" } } }, + }, + }; + const request = { + model: "llama3", + messages, + tools: [weatherTool], + stream: false, + }; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", logLevel: "silent" }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + + const serve = async (extra: Partial, misbehavior: Fixture["misbehavior"]) => { + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { toolCalls: [{ name: "weather", arguments: '{"city":"Paris"}' }] }, + misbehavior, + ...extra, + }); + const reply = await post(mock.url, "/api/chat", request); + return { reply, entry: mock.getRequests().at(-1) }; + }; + + it("never delays a misbehavior reply whose status is not 200, and keeps its bytes", async () => { + const fault: Fixture["misbehavior"] = { + faults: [{ fault: "tool-args-invalid-json", style: "truncated", tool: "weather" }], + }; + const base = await serve({}, fault); + const timed = await serve( + { nonStreamingTimingAllProviders: true, streamingProfile: { ttft: 2000 } }, + fault, + ); + expect(base.reply.status).toBe(500); + expect(timed.reply.status).toBe(500); + expect(timed.reply.elapsedMs).toBeLessThan(1000); + expect(timed.reply.body).toBe(base.reply.body); + expect(timed.entry?.response.status).toBe(500); + expect(timed.entry?.response.interrupted).toBeUndefined(); + }, 10000); + + it("delays a 200 misbehavior reply when nonStreamingTimingAllProviders is set", async () => { + const fault: Fixture["misbehavior"] = { + faults: [{ fault: "tool-unknown-name", tool: "weather", name: "missing_weather" }], + }; + const instant = await serve({ streamingProfile: { ttft: 300, tps: 10000 } }, fault); + const timed = await serve( + { nonStreamingTimingAllProviders: true, streamingProfile: { ttft: 300, tps: 10000 } }, + fault, + ); + expect(instant.reply.status).toBe(200); + expect(instant.reply.elapsedMs).toBeLessThan(250); + expect(timed.reply.status).toBe(200); + expect(timed.reply.elapsedMs).toBeGreaterThanOrEqual(250); + expect(timed.reply.body).toContain("missing_weather"); + expect(timed.entry?.response.misbehavior).toMatchObject({ applied: true }); + }, 10000); +}); + +// /api/generate serves a text fixture's content and never reads its blocks +// (fixture-error-parity.test.ts). The wait is sized from that served text, so +// a block list the chat wire would reject never fails a timed generate reply. +describe("delivery lifecycle: Ollama generate sizes the wait from the served text", () => { + let mock: LLMock; + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", logLevel: "silent" }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + + it("delays content plus unread invalid blocks and serves the content", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "hi", blocks: [{ type: "toolcall", name: "f", arguments: "{}" }] }, + nonStreamingTimingAllProviders: true, + streamingProfile: { ttft: 300, tps: 10000 }, + } as Fixture); + const reply = await post(mock.url, "/api/generate", { + model: "llama3", + prompt: HARNESS_MATCH, + stream: false, + }); + expect(reply.status, reply.body).toBe(200); + expect(reply.body).toContain("hi"); + expect(reply.elapsedMs).toBeGreaterThanOrEqual(250); + }, 10000); +}); diff --git a/src/__tests__/delivery-lifecycle/openai-chat.test.ts b/src/__tests__/delivery-lifecycle/openai-chat.test.ts new file mode 100644 index 00000000..9fccd423 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/openai-chat.test.ts @@ -0,0 +1,33 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const messages = [{ role: "user", content: HARNESS_MATCH }]; +const chunk = (delta: Record, finish: string | null = null) => + `data: ${JSON.stringify({ + id: "chatcmpl-up", + object: "chat.completion.chunk", + created: 1, + model: "gpt-4o", + choices: [{ index: 0, delta, finish_reason: finish }], + })}\n\n`; + +describeDeliveryLifecycle({ + name: "OpenAI Chat Completions", + streamPath: "/v1/chat/completions", + nonStreamPath: "/v1/chat/completions", + streamBody: { model: "gpt-4o", messages, stream: true }, + nonStreamBody: { model: "gpt-4o", messages, stream: false }, + bodyKind: "json", + transport: "http", + surface: "openai-chat", + proxy: { + provider: "openai", + contentType: "text/event-stream", + frames: [ + chunk({ role: "assistant", content: "alpha" }), + chunk({ content: " beta" }), + chunk({ content: " gamma" }), + chunk({}, "stop"), + "data: [DONE]\n\n", + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/responses.test.ts b/src/__tests__/delivery-lifecycle/responses.test.ts new file mode 100644 index 00000000..af4f067c --- /dev/null +++ b/src/__tests__/delivery-lifecycle/responses.test.ts @@ -0,0 +1,42 @@ +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; + +const input = [{ role: "user", content: HARNESS_MATCH }]; +const event = (type: string, data: Record) => + `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`; +const upstreamResponse = { + id: "resp_up", + object: "response", + created_at: 1, + model: "gpt-4o", + status: "in_progress", + output: [], +}; +const delta = (text: string) => + event("response.output_text.delta", { + item_id: "msg_up", + output_index: 0, + content_index: 0, + delta: text, + }); + +describeDeliveryLifecycle({ + name: "OpenAI Responses", + streamPath: "/v1/responses", + nonStreamPath: "/v1/responses", + streamBody: { model: "gpt-4o", input, stream: true }, + nonStreamBody: { model: "gpt-4o", input, stream: false }, + bodyKind: "json", + transport: "http", + surface: "other", + proxy: { + provider: "openai", + contentType: "text/event-stream", + frames: [ + event("response.created", { response: upstreamResponse }), + delta("alpha "), + delta("beta "), + delta("gamma"), + event("response.completed", { response: { ...upstreamResponse, status: "completed" } }), + ], + }, +}); diff --git a/src/__tests__/delivery-lifecycle/transcription.test.ts b/src/__tests__/delivery-lifecycle/transcription.test.ts new file mode 100644 index 00000000..bbbfb6f0 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/transcription.test.ts @@ -0,0 +1,175 @@ +import * as fs from "node:fs"; +import * as http from "node:http"; +import * as os from "node:os"; +import * as path from "node:path"; +import type { AddressInfo } from "node:net"; +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import { LLMock } from "../../llmock.js"; +import { describeDeliveryLifecycle } from "../helpers/delivery-lifecycle-harness.js"; + +// Transcription streams only (non-streaming timing is out of scope, plan T14), +// so the harness runs the abort, healthy and truncate rows on the multipart +// request. Each delta carries `chunkSize` (8) characters of the text. +const streamBody = { model: "gpt-4o-transcribe", stream: "true" }; + +describeDeliveryLifecycle({ + name: "Audio transcription", + streamPath: "/v1/audio/transcriptions", + nonStreamPath: "/v1/audio/transcriptions", + streamBody, + nonStreamBody: { model: "gpt-4o-transcribe" }, + bodyKind: "multipart", + transport: "http", + surface: "other", + nonStream: false, + match: { endpoint: "transcription" }, + streamResponse: { + transcription: { + text: "The quick brown fox jumps over the lazy dog, again and again, for a long time.", + }, + }, +}); + +// The harness's stream abort lands between frames. This one lands in the +// first-frame wait: nothing is written after it, and since no byte of the +// response reached the socket, the journal status is 0. +describe("delivery lifecycle: Audio transcription, abort during the first-frame wait", () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1" }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + }); + + it("stops the wait, writes nothing and journals the client abort", async () => { + mock.addFixture({ + match: { endpoint: "transcription" }, + response: { transcription: { text: "a transcript that never gets sent" } }, + streamingProfile: { ttft: 2000 }, + }); + const boundary = "----aimock-transcription-wait"; + const payload = + `--${boundary}\r\nContent-Disposition: form-data; name="model"\r\n\r\ngpt-4o-transcribe\r\n` + + `--${boundary}\r\nContent-Disposition: form-data; name="stream"\r\n\r\ntrue\r\n` + + `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="a.wav"\r\n` + + `Content-Type: audio/wav\r\n\r\nRIFF\r\n--${boundary}--\r\n`; + let gotBytes = 0; + const req = http.request(new URL("/v1/audio/transcriptions", mock.url), { + method: "POST", + agent: false, + headers: { "content-type": `multipart/form-data; boundary=${boundary}` }, + }); + req.on("error", () => undefined); + req.on("response", (res) => { + res.on("data", (c: Buffer) => { + gotBytes += c.length; + }); + res.on("error", () => undefined); + }); + req.end(payload); + await new Promise((resolve) => setTimeout(resolve, 150)); + let writes = 0; + for (const method of ["writeHead", "write", "end"] as const) { + const original = http.ServerResponse.prototype[method]; + vi.spyOn(http.ServerResponse.prototype, method).mockImplementation(function ( + this: http.ServerResponse, + ...args: unknown[] + ) { + writes++; + return Reflect.apply(original, this, args); + }); + } + req.destroy(); + + await expect + .poll(() => mock.getRequests().at(-1)?.response.interrupted, { timeout: 1000 }) + .toBe(true); + const entry = mock.getRequests().at(-1)!; + expect(entry.response.interruptReason).toBe("client aborted"); + expect(entry.response.status).toBe(0); + // Past the full 2000ms wait: nothing was ever written. + await new Promise((resolve) => setTimeout(resolve, 2100)); + expect(writes).toBe(0); + expect(gotBytes).toBe(0); + }, 10000); +}); + +// Record mode relays a fixture miss to the real provider. A client that hangs +// up mid-relay is journaled as a client abort, as on every other handler. +describe("delivery lifecycle: Audio transcription, client hang-up during a record-mode relay", () => { + let upstream: http.Server; + let mock: LLMock; + let fixturePath: string; + + beforeAll(async () => { + // An upstream that sends one transcription delta every 200ms. + upstream = http.createServer((req, res) => { + req.resume(); + res.writeHead(200, { "Content-Type": "text/event-stream" }); + let i = 0; + const next = (): void => { + if (res.destroyed) return; + if (i >= 10) { + res.end(`data: ${JSON.stringify({ type: "transcript.text.done", text: "done" })}\n\n`); + return; + } + res.write( + `data: ${JSON.stringify({ type: "transcript.text.delta", delta: `w${i++} ` })}\n\n`, + ); + setTimeout(next, 200); + }; + next(); + }); + await new Promise((resolve) => upstream.listen(0, "127.0.0.1", () => resolve())); + const { port } = upstream.address() as AddressInfo; + fixturePath = fs.mkdtempSync(path.join(os.tmpdir(), "aimock-transcription-hangup-")); + mock = new LLMock({ + port: 0, + host: "127.0.0.1", + record: { providers: { openai: `http://127.0.0.1:${port}` }, fixturePath }, + }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + upstream.closeAllConnections(); + await new Promise((resolve) => upstream.close(() => resolve())); + fs.rmSync(fixturePath, { recursive: true, force: true }); + }); + + it("journals the client abort", async () => { + const boundary = "----aimock-transcription-record"; + const payload = + `--${boundary}\r\nContent-Disposition: form-data; name="model"\r\n\r\ngpt-4o-transcribe\r\n` + + `--${boundary}\r\nContent-Disposition: form-data; name="stream"\r\n\r\ntrue\r\n` + + `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="a.wav"\r\n` + + `Content-Type: audio/wav\r\n\r\nRIFF\r\n--${boundary}--\r\n`; + const req = http.request(new URL("/v1/audio/transcriptions", mock.url), { + method: "POST", + agent: false, + headers: { "content-type": `multipart/form-data; boundary=${boundary}` }, + }); + req.on("error", () => undefined); + const firstBytes = new Promise((resolve) => { + req.on("response", (res) => { + res.on("error", () => undefined); + res.once("data", () => resolve()); + }); + }); + req.end(payload); + await firstBytes; + req.destroy(); + + await expect.poll(() => mock.getRequests().length, { timeout: 5000 }).toBeGreaterThan(0); + const entry = mock.getRequests().at(-1)!; + expect(entry.response.source).toBe("proxy"); + expect(entry.response.interrupted).toBe(true); + expect(entry.response.interruptReason).toBe("client aborted"); + }, 10000); +}); diff --git a/src/__tests__/delivery-lifecycle/ws-gemini-live.test.ts b/src/__tests__/delivery-lifecycle/ws-gemini-live.test.ts new file mode 100644 index 00000000..975d8848 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/ws-gemini-live.test.ts @@ -0,0 +1,274 @@ +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { LLMock } from "../../llmock.js"; +import type { Fixture, JournalEntry } from "../../types.js"; +import { WebSocketConnection } from "../../ws-framing.js"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; +import { BAD_FRAME, closeFrame, closeOnSend, textFrame } from "../helpers/ws-close-parity.js"; +import { connectWebSocket, type WSTestClient } from "../ws-test-client.js"; + +const LIVE_PATH = "/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent"; +const LONG_CONTENT = + "The quick brown fox jumps over the lazy dog, again and again, for a long time. ".repeat(4); + +const wsRow = { + streamPath: LIVE_PATH, + nonStreamPath: LIVE_PATH, + streamBody: {}, + nonStreamBody: {}, + bodyKind: "json", + transport: "ws", + wsTransport: "gemini-live", + surface: "other", + nonStream: false, +} as const; + +// Text streaming (the default long-text response). +describeDeliveryLifecycle({ name: "Gemini Live", ...wsRow }); + +// Content + tool calls: the text loop, then the tool-call delay. +describeDeliveryLifecycle({ + name: "Gemini Live (content + toolCalls)", + ...wsRow, + streamResponse: { + content: LONG_CONTENT, + toolCalls: [{ name: "lookup", arguments: '{"q":1}' }], + }, +}); + +// Ordered blocks: text, tool call, text. +describeDeliveryLifecycle({ + name: "Gemini Live (blocks)", + ...wsRow, + streamResponse: { + blocks: [ + { type: "text", text: LONG_CONTENT }, + { type: "toolCall", name: "lookup", arguments: '{"q":1}' }, + { type: "text", text: LONG_CONTENT }, + ], + }, +}); + +// The tool-call-only and empty-response misbehavior branches send nothing +// before their first delay, so the client leaves before the first message. +// It leaves with a bare TCP drop (FIN, no close frame). `WebSocketConnection` +// treats the FIN as a disconnect at once, so the turn stops during the delay, +// well before the delay ends, and not at a later failed write. +describe("delivery lifecycle: Gemini Live close before the first message", () => { + let mock: LLMock; + let sends = 0; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", enableMisbehavior: true }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + beforeEach(() => { + const realSend = WebSocketConnection.prototype.send; + vi.spyOn(WebSocketConnection.prototype, "send").mockImplementation(function ( + this: WebSocketConnection, + data: string, + ) { + sends++; + realSend.call(this, data); + }); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + }); + + const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); + const lastFixtureEntry = (): JournalEntry | undefined => + mock + .getRequests() + .filter((e) => e.response.fixture) + .at(-1); + + async function startTurn(): Promise { + const client = await connectWebSocket(mock.url, LIVE_PATH); + client.send(JSON.stringify({ setup: { model: "models/gemini-2.0-flash-live-001" } })); + await client.waitForMessages(1); // setupComplete + client.send( + JSON.stringify({ + clientContent: { + turns: [{ role: "user", parts: [{ text: HARNESS_MATCH }] }], + turnComplete: true, + }, + }), + ); + return client; + } + + const cases: Array<[string, Partial]> = [ + ["tool call", { response: { toolCalls: [{ name: "lookup", arguments: '{"q":1}' }] } }], + [ + "empty-response misbehavior", + { response: { content: LONG_CONTENT }, misbehavior: "empty-response" }, + ], + ]; + + for (const [label, extra] of cases) { + it(`stops the turn promptly on a TCP drop during the ${label} delay`, async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: LONG_CONTENT }, + latency: 1500, + ...extra, + }); + const client = await startTurn(); + await expect.poll(() => lastFixtureEntry() !== undefined, { timeout: 2000 }).toBe(true); + const before = sends; + client.destroy(); + // Well inside the 1500 ms delay, so the drop is seen at once. The + // interruption reason is not asserted here. + await expect + .poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 750, interval: 10 }) + .toBe(true); + // Past the end of the delay: the stopped turn sends nothing. + await sleep(1700); + expect(sends - before).toBe(0); + }, 10000); + } +}); + +/** + * Closed-socket parity with WS Realtime. aimock's own close (a bad client + * frame, or a send over the write limit) marks the socket closed at once and + * emits `close` about 100ms later; a turn that sees it is journaled with + * "disconnect". A client that left before the reply is "client aborted". In + * both cases nothing more is sent. + */ +describe("delivery lifecycle: Gemini Live closed-socket parity", () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", enableMisbehavior: true }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + }); + + const lastFixtureEntry = (): JournalEntry | undefined => + mock + .getRequests() + .filter((e) => e.response.fixture) + .at(-1); + + const contentMsg = JSON.stringify({ + clientContent: { + turns: [{ role: "user", parts: [{ text: HARNESS_MATCH }] }], + turnComplete: true, + }, + }); + + async function setup(): Promise { + const client = await connectWebSocket(mock.url, LIVE_PATH); + client.send(JSON.stringify({ setup: { model: "models/gemini-2.0-flash-live-001" } })); + await client.waitForMessages(1); // setupComplete + return client; + } + + async function expectInterrupted(reason: string): Promise { + await expect.poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 3000 }).toBe(true); + const entry = lastFixtureEntry()!; + expect(entry.response.interruptReason).toBe(reason); + expect(entry.response.status).toBe(200); + return entry; + } + + it("journals a bad frame mid text turn as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: LONG_CONTENT }, + chunkSize: 8, + latency: 30, + }); + const client = await setup(); + client.send(contentMsg); + await client.waitForMessages(2); // first chunk + client.sendRawFrame(BAD_FRAME); + await expectInterrupted("disconnect"); + }); + + it("journals an audio turn whose client left before the reply as client aborted", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { audio: "SGVsbG8=", format: "mp3" }, + }); + const client = await setup(); + const sent = closeOnSend(() => false); + client.sendRawFrame(Buffer.concat([textFrame(contentMsg), closeFrame()])); + await expectInterrupted("client aborted"); + expect(sent).toEqual([]); + }); + + it("journals an audio reply send that closes the socket as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { audio: "SGVsbG8=", format: "mp3" }, + }); + const client = await setup(); + closeOnSend((data) => data.includes("inlineData")); + client.send(contentMsg); + await expectInterrupted("disconnect"); + }); + + it("journals a text turn whose turnComplete send closes the socket as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "ABCDEFGH" }, + chunkSize: 4, + }); + const client = await setup(); + closeOnSend((data) => data.includes('"turnComplete":true')); + client.send(contentMsg); + await expectInterrupted("disconnect"); + }); + + const toolCases: Array<[string, Fixture["response"]]> = [ + ["tool-call", { toolCalls: [{ name: "lookup", arguments: '{"q":1}' }] }], + ["content + tool-call", { content: "ABCD", toolCalls: [{ name: "lookup", arguments: "{}" }] }], + [ + "blocks", + { + blocks: [ + { type: "text", text: "ABCD" }, + { type: "toolCall", name: "lookup", arguments: "{}" }, + ], + }, + ], + ]; + for (const [label, response] of toolCases) { + it(`journals a ${label} turn whose toolCall send closes the socket as disconnect`, async () => { + mock.addFixture({ match: { userMessage: HARNESS_MATCH }, response, chunkSize: 4 }); + const client = await setup(); + const sent = closeOnSend((data) => data.includes('"toolCall"')); + client.send(contentMsg); + await expectInterrupted("disconnect"); + expect(sent.some((m) => m.includes("turnComplete"))).toBe(false); + }); + } + + it("journals an empty-response turn whose turnComplete send closes the socket as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: LONG_CONTENT }, + misbehavior: "empty-response", + }); + const client = await setup(); + closeOnSend((data) => data.includes('"turnComplete":true')); + client.send(contentMsg); + await expectInterrupted("disconnect"); + }); +}); diff --git a/src/__tests__/delivery-lifecycle/ws-realtime.test.ts b/src/__tests__/delivery-lifecycle/ws-realtime.test.ts new file mode 100644 index 00000000..56b47d77 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/ws-realtime.test.ts @@ -0,0 +1,282 @@ +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import { LLMock } from "../../llmock.js"; +import type { Fixture, JournalEntry } from "../../types.js"; +import { WebSocketConnection } from "../../ws-framing.js"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; +import { BAD_FRAME, closeOnSend, textFrame } from "../helpers/ws-close-parity.js"; +import { connectWebSocket, type WSTestClient } from "../ws-test-client.js"; + +const REALTIME_PATH = "/v1/realtime?model=gpt-4o-realtime-preview"; +/** The per-chunk delay the client drops the socket in. */ +const DELAY_MS = 1500; + +describeDeliveryLifecycle({ + name: "WS Realtime", + streamPath: REALTIME_PATH, + nonStreamPath: REALTIME_PATH, + streamBody: {}, + nonStreamBody: {}, + bodyKind: "json", + transport: "ws", + wsTransport: "realtime", + surface: "other", + nonStream: false, +}); + +/** + * Realtime has five per-turn delivery loops (live transcription, text, + * tool calls, content + tool calls, blocks). The harness above drives the text + * loop. These cases drive the others: a client that drops the socket mid-turn + * is journaled as interrupted, and the turn sends nothing after that. + * + * The drop is a bare TCP FIN with no close frame, made during a long delay. + * `WebSocketConnection` treats the FIN as a disconnect at once, so the turn + * must stop well before the delay ends. The interruption reason is not + * asserted here. + */ +describe("delivery lifecycle: WS Realtime turn kinds", () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", chunkSize: 8 }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + }); + + const sleep = (ms: number): Promise => new Promise((r) => setTimeout(r, ms)); + + const lastFixtureEntry = (): JournalEntry | undefined => + mock + .getRequests() + .filter((e) => e.response.fixture !== null) + .at(-1); + + const typeOf = (m: string): string => (JSON.parse(m) as { type: string }).type; + + async function waitFor(client: WSTestClient, type: string): Promise { + await expect + .poll(() => client.getMessages().some((m) => typeOf(m) === type), { + timeout: 8000, + interval: 20, + }) + .toBe(true); + } + + async function startConversationTurn(client: WSTestClient): Promise { + await client.waitForMessages(1); // session.created + client.send( + JSON.stringify({ + type: "conversation.item.create", + item: { + type: "message", + role: "user", + content: [{ type: "input_text", text: HARNESS_MATCH }], + }, + }), + ); + client.send(JSON.stringify({ type: "response.create" })); + } + + /** Drop the socket, then expect a prompt interruption and no send after it. */ + async function expectClientAbort(client: WSTestClient): Promise { + let sends = 0; + const send = WebSocketConnection.prototype.send; + vi.spyOn(WebSocketConnection.prototype, "send").mockImplementation(function ( + this: WebSocketConnection, + data: string, + ) { + sends++; + send.call(this, data); + }); + client.destroy(); + // Well inside the delay (DELAY_MS), so the FIN is seen at once. + await expect + .poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 750, interval: 10 }) + .toBe(true); + const before = sends; + // Past the end of the delay: the stopped turn sends nothing. + await sleep(DELAY_MS + 200); + expect(sends - before).toBe(0); + } + + const turnKinds: Array<{ name: string; response: Fixture["response"]; firstEvent: string }> = [ + { + name: "tool-call turn", + response: { + toolCalls: [{ name: "get_weather", arguments: JSON.stringify({ city: "x".repeat(80) }) }], + }, + firstEvent: "response.output_item.added", + }, + { + name: "content + tool-call turn", + response: { + content: "Let me check that for you, it will take a moment or two.", + toolCalls: [{ name: "get_weather", arguments: JSON.stringify({ city: "x".repeat(80) }) }], + }, + firstEvent: "response.content_part.added", + }, + { + name: "blocks turn", + response: { + blocks: [ + { type: "text", text: "Here is the weather, read out slowly, one chunk at a time." }, + { type: "toolCall", name: "get_weather", arguments: '{"city":"NYC"}', id: "call_1" }, + ], + }, + firstEvent: "response.created", + }, + ]; + + for (const kind of turnKinds) { + it(`journals a client close mid-turn on a ${kind.name}`, async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: kind.response, + chunkSize: 8, + latency: DELAY_MS, + }); + const client = await connectWebSocket(mock.url, REALTIME_PATH); + await startConversationTurn(client); + await waitFor(client, kind.firstEvent); + await expectClientAbort(client); + }, 15000); + } + + it("journals a client close mid-transcription with a streaming profile", async () => { + mock.addFixture({ + match: { endpoint: "realtime-transcription" }, + response: { transcription: { text: "A long live caption that streams out slowly." } }, + chunkSize: 4, + streamingProfile: { ttft: 0, tps: 1000 / DELAY_MS }, + }); + const client = await connectWebSocket( + mock.url, + "/v1/realtime?model=gpt-realtime&intent=transcription", + ); + await client.waitForMessages(1); // session.created + client.send( + JSON.stringify({ + type: "transcription_session.update", + session: { input_audio_transcription: { model: "gpt-live-transcribe" } }, + }), + ); + await client.waitForMessages(2); // transcription_session.updated + client.send(JSON.stringify({ type: "input_audio_buffer.commit" })); + await waitFor(client, "conversation.item.input_audio_transcription.delta"); + await expectClientAbort(client); + }, 15000); +}); + +/** + * aimock's own close (a bad client frame, or a send over the write limit) + * marks the socket closed at once and emits `close` about 100ms later. A turn + * that sees it is journaled with "disconnect", the reason for an aimock-caused + * teardown, never "client aborted". + */ +describe("delivery lifecycle: WS Realtime aimock-side close", () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1" }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + }); + + const lastFixtureEntry = (): JournalEntry | undefined => + mock + .getRequests() + .filter((e) => e.response.fixture !== null) + .at(-1); + + const itemCreate = JSON.stringify({ + type: "conversation.item.create", + item: { + type: "message", + role: "user", + content: [{ type: "input_text", text: HARNESS_MATCH }], + }, + }); + const responseCreate = JSON.stringify({ type: "response.create" }); + + async function expectDisconnect(): Promise { + await expect.poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 3000 }).toBe(true); + const entry = lastFixtureEntry()!; + expect(entry.response.interruptReason).toBe("disconnect"); + expect(entry.response.status).toBe(200); + } + + it("journals a bad frame mid text turn as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "x".repeat(400) }, + chunkSize: 8, + latency: 30, + }); + const client = await connectWebSocket(mock.url, REALTIME_PATH); + await client.waitForMessages(1); // session.created + client.send(itemCreate); + client.send(responseCreate); + await expect + .poll(() => client.getMessages().some((m) => m.includes("response.output_text.delta")), { + timeout: 3000, + }) + .toBe(true); + client.sendRawFrame(BAD_FRAME); + await expectDisconnect(); + }); + + it("journals a bad frame that lands before the turn starts as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "x".repeat(400) }, + chunkSize: 8, + latency: 30, + }); + const client = await connectWebSocket(mock.url, REALTIME_PATH); + await client.waitForMessages(1); // session.created + client.send(itemCreate); + await client.waitForMessages(2); // conversation.item.created + client.sendRawFrame(Buffer.concat([textFrame(responseCreate), BAD_FRAME])); + await expectDisconnect(); + }); + + it("journals a transcription whose last delta send closes the socket as disconnect", async () => { + mock.addFixture({ + match: { endpoint: "realtime-transcription" }, + response: { transcription: { text: "ABCDEFGH" } }, + chunkSize: 4, + }); + const sent = closeOnSend((data) => data.includes('"delta":"EFGH"')); + const client = await connectWebSocket( + mock.url, + "/v1/realtime?model=gpt-realtime&intent=transcription", + ); + await client.waitForMessages(1); // session.created + client.send( + JSON.stringify({ + type: "transcription_session.update", + session: { input_audio_transcription: { model: "gpt-live-transcribe" } }, + }), + ); + await client.waitForMessages(2); // transcription_session.updated + client.send(JSON.stringify({ type: "input_audio_buffer.commit" })); + await expectDisconnect(); + expect(sent.some((m) => m.includes("input_audio_transcription.completed"))).toBe(false); + }); +}); diff --git a/src/__tests__/delivery-lifecycle/ws-responses.test.ts b/src/__tests__/delivery-lifecycle/ws-responses.test.ts new file mode 100644 index 00000000..f84dcd31 --- /dev/null +++ b/src/__tests__/delivery-lifecycle/ws-responses.test.ts @@ -0,0 +1,89 @@ +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import { LLMock } from "../../llmock.js"; +import type { JournalEntry } from "../../types.js"; +import { describeDeliveryLifecycle, HARNESS_MATCH } from "../helpers/delivery-lifecycle-harness.js"; +import { BAD_FRAME, textFrame } from "../helpers/ws-close-parity.js"; +import { connectWebSocket } from "../ws-test-client.js"; + +describeDeliveryLifecycle({ + name: "WebSocket Responses", + streamPath: "/v1/responses", + nonStreamPath: "/v1/responses", + streamBody: {}, + nonStreamBody: {}, + bodyKind: "json", + transport: "ws", + wsTransport: "responses", + surface: "other", + nonStream: false, +}); + +/** + * aimock's own close (a bad client frame here) marks the socket closed at + * once and emits `close` about 100ms later. A turn that sees the closed socket + * in that window is journaled with "disconnect", the reason for an + * aimock-caused teardown, as WS Realtime does. + */ +describe("delivery lifecycle: WS Responses aimock-side close", () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1" }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + }); + + const lastFixtureEntry = (): JournalEntry | undefined => + mock + .getRequests() + .filter((e) => e.response.fixture !== null) + .at(-1); + + const createMsg = JSON.stringify({ + type: "response.create", + model: "gpt-4", + input: [{ role: "user", content: HARNESS_MATCH }], + }); + + it("journals a bad frame mid-turn as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "x".repeat(400) }, + chunkSize: 8, + latency: 30, + }); + const client = await connectWebSocket(mock.url, "/v1/responses"); + client.send(createMsg); + await expect + .poll(() => client.getMessages().some((m) => m.includes("response.output_text.delta")), { + timeout: 3000, + }) + .toBe(true); + client.sendRawFrame(BAD_FRAME); + await expect.poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 3000 }).toBe(true); + const entry = lastFixtureEntry()!; + expect(entry.response.interruptReason).toBe("disconnect"); + expect(entry.response.status).toBe(200); + }); + + it("journals a bad frame that lands before the turn starts as disconnect", async () => { + mock.addFixture({ + match: { userMessage: HARNESS_MATCH }, + response: { content: "x".repeat(400) }, + chunkSize: 8, + latency: 30, + }); + const client = await connectWebSocket(mock.url, "/v1/responses"); + client.sendRawFrame(Buffer.concat([textFrame(createMsg), BAD_FRAME])); + await expect.poll(() => lastFixtureEntry()?.response.interrupted, { timeout: 3000 }).toBe(true); + expect(lastFixtureEntry()!.response.interruptReason).toBe("disconnect"); + }); +}); diff --git a/src/__tests__/exported-writers-base-contract.test.ts b/src/__tests__/exported-writers-base-contract.test.ts new file mode 100644 index 00000000..7759c0b2 --- /dev/null +++ b/src/__tests__/exported-writers-base-contract.test.ts @@ -0,0 +1,35 @@ +import { describe, it, expect } from "vitest"; +import { readFileSync } from "node:fs"; +import { writeSSEStream } from "../sse-writer.js"; +import { writeNDJSONStream } from "../ndjson-writer.js"; +import { writeEventStream } from "../aws-event-stream.js"; +import { + MATRIX_CASES, + caseId, + runCase, + type ExportedWriters, + type WriterObservation, +} from "./helpers/exported-writer-matrix.js"; + +// No breaking changes: the PUBLIC exported writers must behave exactly like +// base 508dd9b9 for every edge case: same bytes, same headers, same return +// value, same end-of-response behavior. The expected observations were +// captured by running this matrix against the 508dd9b9 build; they are data, +// not a snapshot, so they never update themselves. +const BASE = JSON.parse( + readFileSync(new URL("./fixtures/exported-writers-base-508dd9b9.json", import.meta.url), "utf8"), +) as Record; + +const writers: ExportedWriters = { writeSSEStream, writeNDJSONStream, writeEventStream }; + +describe("exported writers keep the 508dd9b9 contract", () => { + it("has a base observation for every matrix case", () => { + expect(Object.keys(BASE).sort()).toEqual(MATRIX_CASES.map(caseId).sort()); + }); + + for (const c of MATRIX_CASES) { + it(caseId(c), async () => { + expect(await runCase(writers, c)).toEqual(BASE[caseId(c)]); + }); + } +}); diff --git a/src/__tests__/fixture-loader.test.ts b/src/__tests__/fixture-loader.test.ts index b6b5a32f..f88a7aab 100644 --- a/src/__tests__/fixture-loader.test.ts +++ b/src/__tests__/fixture-loader.test.ts @@ -286,6 +286,24 @@ describe("loadFixtureFile", () => { expect(fixtures[0].nonStreamingTiming).toBe(true); }); + it("nonStreamingTimingAllProviders passthrough from JSON", () => { + const filePath = writeJson(tmpDir, "non-streaming-timing-all.json", { + fixtures: [ + { + match: { userMessage: "profile" }, + response: { content: "Hello!" }, + streamingProfile: { ttft: 50, tps: 100 }, + nonStreamingTimingAllProviders: true, + }, + ], + }); + + const fixtures = loadFixtureFile(filePath); + expect(fixtures).toHaveLength(1); + expect(fixtures[0].nonStreamingTimingAllProviders).toBe(true); + expect(fixtures[0].nonStreamingTiming).toBeUndefined(); + }); + it("chaos config passthrough from JSON", () => { const filePath = writeJson(tmpDir, "chaos.json", { fixtures: [ @@ -2087,6 +2105,46 @@ function functionCall(tc: FixtureToolCall | undefined): ToolCall { return tc; } +describe("validateFixtures: non-streaming timing flags must be booleans", () => { + let tmpDir: string; + + beforeEach(() => { + tmpDir = makeTmpDir(); + }); + + afterEach(() => { + rmSync(tmpDir, { recursive: true, force: true }); + }); + + for (const flag of ["nonStreamingTimingAllProviders", "nonStreamingTiming"] as const) { + for (const bad of ["true", 1, null] as const) { + it(`warns on ${flag}: ${JSON.stringify(bad)} loaded from a file, and keeps the value`, () => { + const filePath = writeJson(tmpDir, `${flag}.json`, { + fixtures: [{ match: { userMessage: "hi" }, response: { content: "Hello" }, [flag]: bad }], + }); + const fixtures = loadFixtureFile(filePath); + expect(fixtures).toHaveLength(1); + // The loader passes the value through unchanged (not coerced, not dropped). + expect(fixtures[0][flag]).toBe(bad); + const results = validateFixtures(fixtures); + expect(results).toEqual([ + { + severity: "warning", + fixtureIndex: 0, + message: `${flag} must be a boolean, got ${typeof bad} — the opt-in is ignored`, + }, + ]); + }); + } + + for (const good of [true, false]) { + it(`does not warn on ${flag}: ${good}`, () => { + expect(validateFixtures([makeFixture({ [flag]: good })])).toEqual([]); + }); + } + } +}); + describe("auto-stringify JSON objects in fixture entries", () => { it("stringifies object arguments in toolCalls", () => { const entry: FixtureFileEntry = { diff --git a/src/__tests__/fixtures/exported-writers-base-508dd9b9.json b/src/__tests__/fixtures/exported-writers-base-508dd9b9.json new file mode 100644 index 00000000..e7b848e2 --- /dev/null +++ b/src/__tests__/fixtures/exported-writers-base-508dd9b9.json @@ -0,0 +1,690 @@ +{ + "sse: empty frames": { + "returned": true, + "bytes": "646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: two frames": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: two frames, latency": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: empty frames, signal aborted before": { + "returned": true, + "bytes": "646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: two frames, signal aborted before": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, signal aborted before, latency": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, abort after first frame": { + "returned": false, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, abort after last frame": { + "returned": false, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, abort during delay": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: empty frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: empty frames, ended before, signal aborted": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, ended after first frame": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, ended after last frame": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: two frames, ended during delay": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: empty frames, destroyed": { + "returned": true, + "bytes": "646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: two frames, destroyed": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: two frames, no writeHead": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: empty frames, no writeHead, signal aborted": { + "returned": true, + "bytes": "646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: numeric latency": { + "returned": true, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227431227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: usage + openRouterProcessing": { + "returned": true, + "bytes": "3a204f50454e524f555445522050524f43455353494e470a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b5d2c227573616765223a7b2270726f6d70745f746f6b656e73223a312c22636f6d706c6574696f6e5f746f6b656e73223a322c22746f74616c5f746f6b656e73223a337d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: empty frames, usage + openRouterProcessing, signal aborted before": { + "returned": true, + "bytes": "3a204f50454e524f555445522050524f43455353494e470a0a646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b5d2c227573616765223a7b2270726f6d70745f746f6b656e73223a312c22636f6d706c6574696f6e5f746f6b656e73223a322c22746f74616c5f746f6b656e73223a337d7d0a0a646174613a205b444f4e455d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "sse: openRouterProcessing, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "sse: usage, abort after last frame": { + "returned": false, + "bytes": "646174613a207b226964223a2263222c226f626a656374223a22636861742e636f6d706c6574696f6e2e6368756e6b222c2263726561746564223a312c226d6f64656c223a226d222c2263686f69636573223a5b7b22696e646578223a302c2264656c7461223a7b22636f6e74656e74223a227430227d2c226c6f6770726f6273223a6e756c6c2c2266696e6973685f726561736f6e223a6e756c6c7d5d7d0a0a", + "headers": { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: empty frames": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: two frames": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: two frames, latency": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: empty frames, signal aborted before": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: two frames, signal aborted before": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, signal aborted before, latency": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, abort after first frame": { + "returned": false, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, abort after last frame": { + "returned": false, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, abort during delay": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: empty frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: empty frames, ended before, signal aborted": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, ended after first frame": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, ended after last frame": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: two frames, ended during delay": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "ndjson: empty frames, destroyed": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: two frames, destroyed": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: two frames, no writeHead": { + "returned": true, + "bytes": "7b2269223a302c22646f6e65223a66616c73657d0a7b2269223a312c22646f6e65223a66616c73657d0a", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "ndjson: empty frames, no writeHead, signal aborted": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + "Connection": "keep-alive" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: empty frames": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: two frames": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: two frames, latency": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: empty frames, signal aborted before": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: two frames, signal aborted before": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, signal aborted before, latency": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, abort after first frame": { + "returned": false, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b24", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, abort after last frame": { + "returned": false, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, abort during delay": { + "returned": false, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": false, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: empty frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, ended before": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: empty frames, ended before, signal aborted": { + "returned": true, + "bytes": "", + "headers": {}, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, ended after first frame": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b24", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, ended after last frame": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: two frames, ended during delay": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 0, + "writeHeadCalls": 0 + }, + "eventStream: empty frames, destroyed": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: two frames, destroyed": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: two frames, no writeHead": { + "returned": true, + "bytes": "0000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a307d30ba1b240000006e00000057b62fd2120d3a636f6e74656e742d747970650700106170706c69636174696f6e2f6a736f6e0b3a6576656e742d74797065070011636f6e74656e74426c6f636b44656c74610d3a6d6573736167652d747970650700056576656e747b2269223a317d29a12a65", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + }, + "eventStream: empty frames, no writeHead, signal aborted": { + "returned": true, + "bytes": "", + "headers": { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked" + }, + "writableEnded": true, + "endCalls": 1, + "writeHeadCalls": 0 + } +} diff --git a/src/__tests__/frame-codec-golden.test.ts b/src/__tests__/frame-codec-golden.test.ts new file mode 100644 index 00000000..b0ba7501 --- /dev/null +++ b/src/__tests__/frame-codec-golden.test.ts @@ -0,0 +1,1022 @@ +/** + * Characterization goldens for every streaming writer (spec F0, plan T1). + * + * Two layers are pinned, and both must stay unchanged by the shared + * frame-loop / delivery-lifecycle refactor: + * + * 1. Bytes and call order. Latency-0 fixtures run through each writer (the + * public writers directly, the module-private writers through their + * handlers) against a recording fake `res` / fake `ws`. Every + * `setHeader` / `writeHead` / `write` / `end` / `destroy` / `send` call is + * logged in order, plus the final journal interruption state. + * 2. Delays at the timer layer. Fixtures carry `recordedTimings` with a + * distinct gap at every index, the writer runs under fake timers, and every + * scheduled `setTimeout` duration is logged in order, interleaved with the + * write log. A refactor that calls `calculateDelay` the same way but sleeps + * differently fails here. + * + * Volatile values (generated ids, timestamps) are masked before comparison. + * Fixture-authored ids (`AUTHORED_IDS`) are never masked, so a lost or + * rewritten authored id fails the comparison. + * + * The goldens live in `__snapshots__/frame-codec-golden/*.json`. They are + * written only when `FRAME_CODEC_GOLDEN_WRITE=1` is set. Write mode merges + * the captured entries into the existing file (a `-t` filtered run never + * drops entries it did not run) and drops entries no test declares. In + * compare mode a missing or different entry fails, and so does a file whose + * entries are not exactly the declared cases (stale or extra entries). + */ +import { EventEmitter } from "node:events"; +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import type * as http from "node:http"; +import { crc32 } from "node:zlib"; +import { fileURLToPath } from "node:url"; +import { Readable } from "node:stream"; +import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { writeSSEStream } from "../sse-writer.js"; +import { writeNDJSONStream } from "../ndjson-writer.js"; +import { writeEventStream } from "../aws-event-stream.js"; +import { createInterruptionSignal } from "../interruption.js"; +import { buildTextChunks, buildToolCallChunks, buildUsageChunk } from "../helpers.js"; +import { handleMessages } from "../messages.js"; +import { handleResponses } from "../responses.js"; +import { handleGemini } from "../gemini.js"; +import { + handleGeminiInteractions, + resetEventIdCounter, + resetInteractionCounter, +} from "../gemini-interactions.js"; +import { handleCohere } from "../cohere.js"; +import { handleBedrockStream } from "../bedrock.js"; +import { handleConverseStream } from "../bedrock-converse.js"; +import { handleOllama } from "../ollama.js"; +import { handleTranscription } from "../transcription.js"; +import { handleWebSocketResponses } from "../ws-responses.js"; +import { handleWebSocketRealtime } from "../ws-realtime.js"; +import { handleWebSocketGeminiLive } from "../ws-gemini-live.js"; +import { createServer } from "../server.js"; +import type { WebSocketConnection } from "../ws-framing.js"; +import { Journal } from "../journal.js"; +import { setFixtureMisbehaviorPosition } from "../misbehavior.js"; +import { Logger } from "../logger.js"; +import type { Fixture, FixtureResponse, HandlerDefaults, RecordedTimings } from "../types.js"; + +// ─── Recording doubles ───────────────────────────────────────────────────── + +type LogEntry = Record; + +const VOLATILE_KEY = + /"(id|[A-Za-z]*_id|[a-z]+Id|created|created_at|createdAt|signature)":("(?:[^"\\]|\\.)*"|-?\d+(?:\.\d+)?)/g; +// A whole id token, never a JSON key (`"call_id":` stays as written). +const GENERATED_ID = + /\b(chatcmpl|msg|toolu|resp|call|fc|rs|aimock-int|aimock-evt|item|req|evt|event|sess|gen|conv|ws)[-_][A-Za-z0-9_-]+(?![A-Za-z0-9_-]|"\s*:)/g; + +/** Tool-call ids the fixtures author. Output must carry them verbatim. */ +const AUTHORED_IDS = { + g1: "call_g1", + g2: "call_g2", + g3: "call_g3", + m1: "call_m1", + m2: "call_m2", +} as const; +const AUTHORED_ID = new RegExp(`(?:${Object.values(AUTHORED_IDS).join("|")})(?![A-Za-z0-9_-])`); + +// V8's `JSON.parse` message carries a " (line N column M)" suffix on Node 22+ +// but not on Node 20; drop it so a writer that echoes the parse error compares the same everywhere. +const V8_PARSE_LOCATION = /( in JSON at position \d+) \(line \d+ column \d+\)/g; + +/** Mask generated ids and timestamps; keep any value that carries an authored id. */ +function mask(text: string): string { + return text + .replace(V8_PARSE_LOCATION, "$1") + .replace(VOLATILE_KEY, (whole: string, key: string, value: string) => + AUTHORED_ID.test(value) ? whole : `"${key}":""`, + ) + .replace(GENERATED_ID, (id: string) => (AUTHORED_ID.test(id) ? id : "")); +} + +/** Decode concatenated AWS event-stream frames, checking both CRCs. */ +function decodeEventStream(buf: Buffer): LogEntry[] { + const frames: LogEntry[] = []; + let off = 0; + while (off + 12 <= buf.length) { + const total = buf.readUInt32BE(off); + const headersLength = buf.readUInt32BE(off + 4); + const preludeOk = crc32(buf.subarray(off, off + 8)) >>> 0 === buf.readUInt32BE(off + 8); + const messageOk = + crc32(buf.subarray(off, off + total - 4)) >>> 0 === buf.readUInt32BE(off + total - 4); + const headers: Record = {}; + let h = off + 12; + const headersEnd = h + headersLength; + while (h < headersEnd) { + const nameLength = buf.readUInt8(h); + const name = buf.subarray(h + 1, h + 1 + nameLength).toString("utf8"); + h += 1 + nameLength + 1; + const valueLength = buf.readUInt16BE(h); + headers[name] = buf.subarray(h + 2, h + 2 + valueLength).toString("utf8"); + h += 2 + valueLength; + } + let payload = buf.subarray(headersEnd, off + total - 4).toString("utf8"); + const parsed: unknown = JSON.parse(payload); + if ( + typeof parsed === "object" && + parsed !== null && + typeof (parsed as { bytes?: unknown }).bytes === "string" + ) { + const inner = Buffer.from((parsed as { bytes: string }).bytes, "base64").toString("utf8"); + payload = JSON.stringify({ bytes: `${mask(inner)}` }); + } else { + payload = mask(payload); + } + frames.push({ total, headersLength, preludeOk, messageOk, headers, payload }); + off += total; + } + if (off !== buf.length) frames.push({ trailingBytes: buf.length - off }); + return frames; +} + +class RecordingRes extends EventEmitter { + writableEnded = false; + writableFinished = false; + destroyed = false; + headersSent = false; + statusCode = 200; + /** Called once the response ends or is destroyed (not logged). */ + onDone: (() => void) | null = null; + private readonly headers: Record = {}; + constructor(private readonly log: LogEntry[]) { + super(); + } + setHeader(name: string, value: string | number | string[]): this { + // The server sets a generated `X-Request-Id`; every other header is kept verbatim. + this.log.push({ + op: "setHeader", + name, + value: name.toLowerCase() === "x-request-id" ? "" : value, + }); + this.headers[name.toLowerCase()] = value; + return this; + } + getHeader(name: string): string | number | string[] | undefined { + return this.headers[name.toLowerCase()]; + } + writeHead(status: number, headers?: Record): this { + this.log.push({ op: "writeHead", status, headers: headers ?? null }); + this.statusCode = status; + this.headersSent = true; + return this; + } + write(chunk: string | Buffer): boolean { + if (Buffer.isBuffer(chunk)) { + this.log.push({ op: "write", frames: decodeEventStream(chunk) }); + } else { + this.log.push({ op: "write", data: mask(chunk) }); + } + this.headersSent = true; + return true; + } + end(chunk?: string | Buffer): this { + if (chunk !== undefined) this.write(chunk); + this.log.push({ op: "end" }); + this.writableEnded = true; + this.writableFinished = true; + this.headersSent = true; + this.onDone?.(); + return this; + } + destroy(): this { + this.log.push({ op: "destroy" }); + this.destroyed = true; + this.onDone?.(); + return this; + } +} + +class RecordingWs extends EventEmitter { + private closed = false; + constructor(private readonly log: LogEntry[]) { + super(); + } + send(data: string): void { + this.log.push({ op: "send", data: mask(data) }); + } + close(code?: number, reason?: string): void { + this.log.push({ op: "close", code: code ?? null, reason: reason ?? null }); + this.closed = true; + } + destroy(): void { + this.log.push({ op: "destroy" }); + this.closed = true; + } + get isClosed(): boolean { + return this.closed; + } +} + +function asRes(res: RecordingRes): http.ServerResponse { + return res as unknown as http.ServerResponse; +} +function asWs(ws: RecordingWs): WebSocketConnection { + return ws as unknown as WebSocketConnection; +} +function fakeReq(url: string, headers: Record = {}): http.IncomingMessage { + return { method: "POST", url, headers } as unknown as http.IncomingMessage; +} + +// ─── Fixtures ────────────────────────────────────────────────────────────── + +const MATCH = "golden probe"; +const PRIMES = [ + 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, + 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, + 223, 227, 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311, +]; +const RECORDED: RecordedTimings = { ttftMs: 7, interChunkDelaysMs: PRIMES, totalDurationMs: 0 }; + +const RESPONSES: Record = { + text: { content: "Hello golden world, this is a characterization run." }, + tool: { toolCalls: [{ name: "get_weather", arguments: '{"city":"NYC"}', id: AUTHORED_IDS.g1 }] }, + combined: { + content: "Checking now.", + toolCalls: [{ name: "get_weather", arguments: '{"city":"SF"}', id: AUTHORED_IDS.g2 }], + }, + blocks: { + blocks: [ + { type: "text", text: "Lead text." }, + { type: "toolCall", name: "get_time", arguments: "{}", id: AUTHORED_IDS.g3 }, + { type: "text", text: "Tail text." }, + ], + }, +}; + +type CaseName = + | "text" + | "tool" + | "combined" + | "blocks" + | "misbehavior" + | "truncate2" + | "delay-text" + | "delay-combined" + | "delay-replay2"; + +const BYTE_CASES: CaseName[] = ["text", "tool", "combined", "blocks", "misbehavior", "truncate2"]; +const DELAY_CASES: CaseName[] = ["delay-text", "delay-combined", "delay-replay2"]; + +function fixtureFor( + name: CaseName, + match: Fixture["match"], + misbehaviorFault: string, + textResponse: FixtureResponse = RESPONSES.text, +): Fixture { + const base = { match, chunkSize: 7 }; + switch (name) { + case "text": + case "tool": + case "combined": + case "blocks": + return { ...base, latency: 0, response: name === "text" ? textResponse : RESPONSES[name] }; + case "misbehavior": { + const fixture: Fixture = { + ...base, + latency: 0, + response: { + toolCalls: [ + { name: "get_weather", arguments: '{"city":"NYC"}', id: AUTHORED_IDS.m1 }, + { name: "get_time", arguments: '{"zone":"UTC"}', id: AUTHORED_IDS.m2 }, + ], + }, + misbehavior: misbehaviorFault as Fixture["misbehavior"], + }; + setFixtureMisbehaviorPosition(fixture, "golden#0"); + return fixture; + } + case "truncate2": + return { ...base, latency: 0, response: textResponse, truncateAfterChunks: 2 }; + case "delay-text": + return { ...base, response: textResponse, recordedTimings: RECORDED }; + case "delay-combined": + return { ...base, response: RESPONSES.combined, recordedTimings: RECORDED }; + case "delay-replay2": + return { ...base, response: textResponse, recordedTimings: RECORDED, replaySpeed: 2 }; + } +} + +function makeDefaults(journal: Journal): HandlerDefaults { + return { + latency: 0, + chunkSize: 7, + replaySpeed: 1, + logger: new Logger("silent"), + misbehavior: { byTestId: new Map() }, + misbehaviorCounters: journal, + }; +} + +// ─── Runner ──────────────────────────────────────────────────────────────── + +interface Capture { + log: LogEntry[]; + journal: LogEntry | null; +} + +interface CaptureOptions { + /** Read the journal state from this journal (a server's) instead of a fresh one. */ + journal?: Journal; + /** + * Keep running timers until `body` settles. For bodies that wait on I/O + * (a request body stream) before they schedule their first timer. + */ + untilSettled?: boolean; +} + +/** + * Run `body` under fake timers, logging every scheduled setTimeout duration + * into the same ordered log the doubles write to. + */ +async function capture( + body: (log: LogEntry[], journal: Journal) => Promise | unknown, + options: CaptureOptions = {}, +): Promise { + const log: LogEntry[] = []; + const journal = options.journal ?? new Journal(); + const fakeSetTimeout = globalThis.setTimeout; + const spy = vi.spyOn(globalThis, "setTimeout").mockImplementation((( + fn: () => void, + ms?: number, + ...args: unknown[] + ) => { + log.push({ op: "timer", ms: ms ?? 0 }); + return fakeSetTimeout(fn, ms, ...args); + }) as unknown as typeof setTimeout); + try { + const pending = Promise.resolve().then(() => body(log, journal)); + // Observed now so a rejection is not reported as unhandled while timers run. + pending.catch(() => undefined); + if (options.untilSettled) { + let settled = false; + pending.then( + () => (settled = true), + () => (settled = true), + ); + // Each pass yields one real macrotask, so stream I/O makes progress. + for (let pass = 0; !settled; pass++) { + if (pass >= 1000) throw new Error("capture: body did not settle"); + await vi.runAllTimersAsync(); + } + } else { + await vi.runAllTimersAsync(); + } + await pending; + await vi.runAllTimersAsync(); + } finally { + spy.mockRestore(); + } + const entries = journal.getAll(); + const last = entries.at(-1); + return { + log, + journal: last + ? { + status: last.response.status, + interrupted: last.response.interrupted ?? null, + interruptReason: last.response.interruptReason ?? null, + } + : null, + }; +} + +const GOLDEN_DIR = fileURLToPath(new URL("./__snapshots__/frame-codec-golden/", import.meta.url)); +const WRITE = process.env.FRAME_CODEC_GOLDEN_WRITE === "1"; +const pendingWrites = new Map>(); +const loaded = new Map>(); +/** + * Every golden entry a test declares, in declaration order. Filled while the + * suite is collected, so it is complete even when `-t` runs only some tests. + */ +const declared = new Map(); + +function goldenPath(file: string): string { + return `${GOLDEN_DIR}${file}.json`; +} + +function readGolden(file: string): Record | null { + const path = goldenPath(file); + if (!existsSync(path)) return null; + return JSON.parse(readFileSync(path, "utf8")) as Record; +} + +/** Declare the golden entry `name` in `.json`. Call at collection time. */ +function declare(file: string, name: string): void { + const names = declared.get(file) ?? []; + if (names.includes(name)) throw new Error(`golden entry declared twice: ${file} › ${name}`); + names.push(name); + declared.set(file, names); +} + +/** + * Compare one capture with its entry in `.json`. In write mode the + * captures are collected and merged into the file after the suite runs. + */ +function record(file: string, name: string, value: Capture): void { + expect(declared.get(file)?.includes(name), `golden entry not declared: ${file} › ${name}`).toBe( + true, + ); + const normalized: unknown = JSON.parse(JSON.stringify(value)); + if (WRITE) { + const group = pendingWrites.get(file) ?? {}; + group[name] = normalized; + pendingWrites.set(file, group); + return; + } + let golden = loaded.get(file); + if (!golden) { + const read = readGolden(file); + expect(read, `golden file missing: ${goldenPath(file)}`).not.toBeNull(); + golden = read ?? {}; + loaded.set(file, golden); + } + expect(Object.hasOwn(golden, name), `golden entry missing: ${file} › ${name}`).toBe(true); + expect(normalized).toEqual(golden[name]); +} + +/** + * Compare mode: the file exists and its entries are exactly the declared + * cases, so a stale entry (a case that was renamed or removed) or an extra + * one fails. Write mode checks nothing here; the merge runs in `afterAll`. + */ +function finishGolden(file: string): void { + if (WRITE) return; + const golden = readGolden(file); + expect(golden, `golden file missing: ${goldenPath(file)}`).not.toBeNull(); + const expected = declared.get(file) ?? []; + const actual = Object.keys(golden ?? {}); + const stale = actual.filter((name) => !expected.includes(name)); + const missing = expected.filter((name) => !actual.includes(name)); + expect({ stale, missing }, `golden entries out of date: ${file}`).toEqual({ + stale: [], + missing: [], + }); +} + +/** + * Write mode: merge the captured entries into each file. Entries this run did + * not capture keep their stored value, entries no test declares are dropped, + * and the declared order is kept. A file with no captures is not touched. + */ +afterAll(() => { + if (!WRITE) return; + mkdirSync(GOLDEN_DIR, { recursive: true }); + for (const [file, captured] of pendingWrites) { + const existing = readGolden(file) ?? {}; + const merged: Record = {}; + for (const name of declared.get(file) ?? []) { + if (Object.hasOwn(captured, name)) merged[name] = captured[name]; + else if (Object.hasOwn(existing, name)) merged[name] = existing[name]; + } + writeFileSync(goldenPath(file), JSON.stringify(merged, null, 2) + "\n"); + } +}); + +function installFakeClock(): void { + vi.useFakeTimers(); + vi.setSystemTime(new Date("2026-01-02T03:04:05.000Z")); +} + +beforeEach(() => { + installFakeClock(); + resetInteractionCounter(); + resetEventIdCounter(); +}); +afterEach(() => { + vi.useRealTimers(); +}); + +// ─── HTTP handler rows ───────────────────────────────────────────────────── + +const noCors = (): void => {}; + +interface HandlerRow { + file: string; + url: string; + body: Record; + misbehaviorFault: string; + run: ( + req: http.IncomingMessage, + res: http.ServerResponse, + raw: string, + fixtures: Fixture[], + journal: Journal, + defaults: HandlerDefaults, + ) => Promise; +} + +const BEDROCK_MODEL = "anthropic.claude-3-5-sonnet-20241022-v2:0"; + +const HANDLER_ROWS: HandlerRow[] = [ + { + file: "claude-sse", + url: "/v1/messages", + body: { + model: "claude-3-5-sonnet-20241022", + max_tokens: 256, + stream: true, + messages: [{ role: "user", content: MATCH }], + }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => handleMessages(req, res, raw, f, j, d, noCors), + }, + { + file: "responses-sse", + url: "/v1/responses", + body: { model: "gpt-4o", stream: true, input: [{ role: "user", content: MATCH }] }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => handleResponses(req, res, raw, f, j, d, noCors), + }, + { + file: "gemini-sse", + url: "/v1beta/models/gemini-2.0-flash:streamGenerateContent?alt=sse", + body: { contents: [{ role: "user", parts: [{ text: MATCH }] }] }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => + handleGemini(req, res, raw, "gemini-2.0-flash", true, f, j, d, noCors), + }, + { + file: "gemini-interactions-sse", + url: "/v1beta/interactions", + body: { model: "gemini-2.5-flash", input: MATCH, stream: true }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => handleGeminiInteractions(req, res, raw, f, j, d, noCors), + }, + { + file: "cohere-sse", + url: "/v2/chat", + body: { model: "command-r-plus", stream: true, messages: [{ role: "user", content: MATCH }] }, + // stop-length-mid-tool is the only path that turns on terminalSentinel. + misbehaviorFault: "stop-length-mid-tool", + run: (req, res, raw, f, j, d) => handleCohere(req, res, raw, f, j, d, noCors), + }, + { + file: "bedrock-invoke-eventstream", + url: `/model/${BEDROCK_MODEL}/invoke-with-response-stream`, + body: { + anthropic_version: "bedrock-2023-05-31", + max_tokens: 256, + messages: [{ role: "user", content: MATCH }], + }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => + handleBedrockStream(req, res, raw, BEDROCK_MODEL, f, j, d, noCors), + }, + { + file: "bedrock-converse-eventstream", + url: `/model/${BEDROCK_MODEL}/converse-stream`, + body: { messages: [{ role: "user", content: [{ text: MATCH }] }] }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => + handleConverseStream(req, res, raw, BEDROCK_MODEL, f, j, d, noCors), + }, + { + file: "ollama-ndjson", + url: "/api/chat", + body: { model: "llama3", stream: true, messages: [{ role: "user", content: MATCH }] }, + misbehaviorFault: "tool-args-invalid-json", + run: (req, res, raw, f, j, d) => handleOllama(req, res, raw, f, j, d, noCors), + }, +]; + +describe("frame codec goldens: handler-driven writers", () => { + for (const row of HANDLER_ROWS) { + describe(row.file, () => { + const cases = [...BYTE_CASES, ...DELAY_CASES]; + for (const name of cases) { + declare(row.file, name); + it(name, async () => { + const fixture = fixtureFor(name, { userMessage: MATCH }, row.misbehaviorFault); + const raw = JSON.stringify(row.body); + const result = await capture(async (log, journal) => { + const res = new RecordingRes(log); + await row.run( + fakeReq(row.url), + asRes(res), + raw, + [fixture], + journal, + makeDefaults(journal), + ); + }); + record(row.file, name, result); + }); + } + it("has its golden file", () => { + finishGolden(row.file); + }); + }); + } +}); + +// ─── Public writers driven directly ──────────────────────────────────────── + +function openAIChunks(name: CaseName) { + if (name === "tool" || name === "misbehavior") { + return buildToolCallChunks( + [{ name: "get_weather", arguments: '{"city":"NYC"}', id: AUTHORED_IDS.g1 }], + "gpt-4o", + 7, + ); + } + return buildTextChunks("Hello golden world, this is a characterization run.", "gpt-4o", 7); +} + +function writerTiming(name: CaseName): { recordedTimings?: RecordedTimings; replaySpeed?: number } { + if (name === "delay-text" || name === "delay-combined") return { recordedTimings: RECORDED }; + if (name === "delay-replay2") return { recordedTimings: RECORDED, replaySpeed: 2 }; + return {}; +} + +const DIRECT_CASES: CaseName[] = ["text", "tool", "truncate2", "delay-text", "delay-replay2"]; + +describe("frame codec goldens: writeSSEStream", () => { + const variants = ["plain", "usageChunk", "openRouterProcessing"] as const; + for (const variant of variants) { + for (const name of DIRECT_CASES) { + declare("openai-sse", `${variant} ${name}`); + it(`${variant} ${name}`, async () => { + const result = await capture(async (log) => { + const res = new RecordingRes(log); + const chunks = openAIChunks(name); + const interruption = + name === "truncate2" + ? createInterruptionSignal({ + match: {}, + response: { content: "" }, + truncateAfterChunks: 2, + }) + : null; + const completed = await writeSSEStream(asRes(res), chunks, { + latency: 0, + ...writerTiming(name), + signal: interruption?.signal, + onChunkSent: interruption?.tick, + usageChunk: + variant === "usageChunk" + ? buildUsageChunk(chunks[0].id, "gpt-4o", chunks[0].created, { + prompt_tokens: 3, + completion_tokens: 5, + total_tokens: 8, + }) + : undefined, + openRouterProcessing: variant === "openRouterProcessing", + }); + interruption?.cleanup(); + log.push({ op: "return", completed }); + }); + record("openai-sse", `${variant} ${name}`, result); + }); + } + } + declare("openai-sse", "already-ended"); + it("returns true and writes nothing when the response already ended", async () => { + const result = await capture(async (log) => { + const res = new RecordingRes(log); + res.writableEnded = true; + log.push({ op: "return", completed: await writeSSEStream(asRes(res), openAIChunks("text")) }); + }); + record("openai-sse", "already-ended", result); + }); + it("has its golden file", () => { + finishGolden("openai-sse"); + }); +}); + +describe("frame codec goldens: writeNDJSONStream", () => { + for (const name of DIRECT_CASES) { + declare("ndjson", name); + it(name, async () => { + const result = await capture(async (log) => { + const res = new RecordingRes(log); + const chunks = [ + { model: "llama3", message: { role: "assistant", content: "Hello " }, done: false }, + { model: "llama3", message: { role: "assistant", content: "golden " }, done: false }, + { model: "llama3", message: { role: "assistant", content: "world." }, done: false }, + { model: "llama3", message: { role: "assistant", content: "" }, done: true }, + ]; + const interruption = + name === "truncate2" + ? createInterruptionSignal({ + match: {}, + response: { content: "" }, + truncateAfterChunks: 2, + }) + : null; + const completed = await writeNDJSONStream(asRes(res), chunks, { + latency: 0, + ...writerTiming(name), + signal: interruption?.signal, + onChunkSent: interruption?.tick, + }); + interruption?.cleanup(); + log.push({ op: "return", completed }); + }); + record("ndjson", name, result); + }); + } + it("has its golden file", () => { + finishGolden("ndjson"); + }); +}); + +describe("frame codec goldens: writeEventStream", () => { + for (const name of DIRECT_CASES) { + declare("aws-eventstream", name); + it(name, async () => { + const result = await capture(async (log) => { + const res = new RecordingRes(log); + const events = [ + { eventType: "messageStart", payload: { role: "assistant" } }, + { + eventType: "contentBlockDelta", + payload: { contentBlockIndex: 0, delta: { text: "Hello " } }, + }, + { + eventType: "contentBlockDelta", + payload: { contentBlockIndex: 0, delta: { text: "world" } }, + }, + { eventType: "contentBlockStop", payload: { contentBlockIndex: 0 } }, + { eventType: "messageStop", payload: { stopReason: "end_turn" } }, + ]; + const interruption = + name === "truncate2" + ? createInterruptionSignal({ + match: {}, + response: { content: "" }, + truncateAfterChunks: 2, + }) + : null; + const completed = await writeEventStream(asRes(res), events, { + latency: 0, + ...writerTiming(name), + signal: interruption?.signal, + onChunkSent: interruption?.tick, + }); + interruption?.cleanup(); + log.push({ op: "return", completed }); + }); + record("aws-eventstream", name, result); + }); + } + it("has its golden file", () => { + finishGolden("aws-eventstream"); + }); +}); + +// ─── Transcription (multipart, writeHead up front) ───────────────────────── + +describe("frame codec goldens: transcription stream", () => { + const boundary = "----goldenboundary"; + const raw = [ + `--${boundary}\r\nContent-Disposition: form-data; name="model"\r\n\r\ngpt-4o-transcribe\r\n`, + `--${boundary}\r\nContent-Disposition: form-data; name="stream"\r\n\r\ntrue\r\n`, + `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="a.wav"\r\nContent-Type: audio/wav\r\n\r\nRIFF\r\n`, + `--${boundary}--\r\n`, + ].join(""); + const transcript: FixtureResponse = { + transcription: { + text: "Hello golden world, this is a transcription.", + languages: [{ code: "en" }], + }, + }; + for (const name of ["text", "truncate2", "delay-text", "delay-replay2"] as CaseName[]) { + declare("transcription-sse", name); + it(name, async () => { + const fixture = fixtureFor(name, { endpoint: "transcription" }, "", transcript); + const result = await capture(async (log, journal) => { + const res = new RecordingRes(log); + await handleTranscription( + fakeReq("/v1/audio/transcriptions", { + "content-type": `multipart/form-data; boundary=${boundary}`, + }), + asRes(res), + raw, + [fixture], + journal, + makeDefaults(journal), + noCors, + ); + }); + record("transcription-sse", name, result); + }); + } + it("has its golden file", () => { + finishGolden("transcription-sse"); + }); +}); + +// ─── WebSocket transports ────────────────────────────────────────────────── + +describe("frame codec goldens: WS Responses sendEvents", () => { + for (const name of [...BYTE_CASES, ...DELAY_CASES]) { + declare("ws-responses", name); + it(name, async () => { + const fixture = fixtureFor(name, { userMessage: MATCH }, "tool-args-invalid-json"); + const result = await capture(async (log, journal) => { + const ws = new RecordingWs(log); + handleWebSocketResponses(asWs(ws), [fixture], journal, { + ...makeDefaults(journal), + model: "gpt-4o", + }); + ws.emit( + "message", + JSON.stringify({ + type: "response.create", + model: "gpt-4o", + input: [{ role: "user", content: MATCH }], + }), + ); + }); + record("ws-responses", name, result); + }); + } + it("has its golden file", () => { + finishGolden("ws-responses"); + }); +}); + +describe("frame codec goldens: WS Realtime", () => { + const rows: Array<[string, Partial]> = [ + ["text", { latency: 0 }], + ["delay-text", { recordedTimings: RECORDED }], + ["profile", { streamingProfile: { ttft: 5, tps: 50 } }], + ["truncate2", { latency: 0, truncateAfterChunks: 2 }], + ]; + for (const [name, extra] of rows) { + declare("ws-realtime", name); + it(name, async () => { + const fixture: Fixture = { + match: { userMessage: MATCH }, + response: RESPONSES.text, + chunkSize: 7, + ...extra, + }; + const result = await capture(async (log, journal) => { + const ws = new RecordingWs(log); + handleWebSocketRealtime(asWs(ws), [fixture], journal, { + ...makeDefaults(journal), + model: "gpt-4o-realtime-preview", + }); + ws.emit( + "message", + JSON.stringify({ + type: "conversation.item.create", + item: { + type: "message", + role: "user", + content: [{ type: "input_text", text: MATCH }], + }, + }), + ); + ws.emit("message", JSON.stringify({ type: "response.create" })); + }); + record("ws-realtime", name, result); + }); + } + // The live-transcription delta loop is the Realtime path that passes the + // fixture's streamingProfile to calculateDelay; the text-response loops pass + // `undefined` (the "profile" row above pins that they ignore it). + const transcriptionRows: Array<[string, Partial]> = [ + ["transcription profile", { streamingProfile: { ttft: 5, tps: 50 } }], + ["transcription delay-text", { recordedTimings: RECORDED }], + ["transcription truncate2", { latency: 0, truncateAfterChunks: 2 }], + ]; + for (const [name, extra] of transcriptionRows) { + declare("ws-realtime", name); + it(name, async () => { + const fixture: Fixture = { + match: { endpoint: "realtime-transcription" }, + response: { transcription: { text: "Live golden caption for the profile path." } }, + chunkSize: 7, + ...extra, + }; + const result = await capture(async (log, journal) => { + const ws = new RecordingWs(log); + handleWebSocketRealtime(asWs(ws), [fixture], journal, { + ...makeDefaults(journal), + model: "gpt-realtime", + transcriptionIntent: true, + }); + ws.emit( + "message", + JSON.stringify({ + type: "transcription_session.update", + session: { input_audio_transcription: { model: "gpt-live-transcribe-2026-07-01" } }, + }), + ); + ws.emit("message", JSON.stringify({ type: "input_audio_buffer.commit" })); + }); + record("ws-realtime", name, result); + }); + } + it("has its golden file", () => { + finishGolden("ws-realtime"); + }); +}); + +// ─── OpenAI Chat Completions handler (through the server dispatcher) ────── + +/** + * `handleCompletions` is module-private, so these rows drive the server's own + * request listener with a body stream and a recording response. The server is + * created before the timer spy starts, so only the request's timers are + * logged; the journal state is read from the server's journal. + */ +async function captureChat(fixture: Fixture, body: Record): Promise { + // Real timers while the server starts: its sweep intervals would make + // `runAllTimersAsync` loop forever under the fake clock. + vi.useRealTimers(); + const instance = await createServer([fixture], { port: 0, logLevel: "silent" }); + installFakeClock(); + try { + const [listener] = instance.server.listeners("request") as Array< + (req: http.IncomingMessage, res: http.ServerResponse) => void + >; + return await capture( + (log) => + new Promise((resolve) => { + const res = new RecordingRes(log); + res.onDone = resolve; + const req = Object.assign(Readable.from([Buffer.from(JSON.stringify(body))]), { + method: "POST", + url: "/v1/chat/completions", + headers: { "content-type": "application/json", host: "127.0.0.1" }, + }); + listener(req as unknown as http.IncomingMessage, asRes(res)); + }), + { journal: instance.journal, untilSettled: true }, + ); + } finally { + await new Promise((resolve) => instance.server.close(() => resolve())); + } +} + +describe("frame codec goldens: OpenAI chat completions handler", () => { + const file = "openai-chat-handler"; + const request = (stream: boolean) => ({ + model: "gpt-4o", + stream, + messages: [{ role: "user", content: MATCH }], + }); + const streamCases: CaseName[] = [...BYTE_CASES, ...DELAY_CASES]; + const nonStreamCases: CaseName[] = ["text", "tool", "combined", "blocks", "delay-text"]; + for (const [mode, cases] of [ + ["stream", streamCases], + ["non-stream", nonStreamCases], + ] as const) { + for (const name of cases) { + declare(file, `${mode} ${name}`); + it(`${mode} ${name}`, async () => { + const fixture = fixtureFor(name, { userMessage: MATCH }, "tool-args-invalid-json"); + const result = await captureChat(fixture, request(mode === "stream")); + record(file, `${mode} ${name}`, result); + }); + } + } + it("has its golden file", () => { + finishGolden(file); + }); +}); + +// ─── Gemini Live (WebSocket) ─────────────────────────────────────────────── + +describe("frame codec goldens: Gemini Live", () => { + const file = "ws-gemini-live"; + for (const name of [...BYTE_CASES, ...DELAY_CASES]) { + declare(file, name); + it(name, async () => { + const fixture = fixtureFor(name, { userMessage: MATCH }, "tool-args-invalid-json"); + const result = await capture(async (log, journal) => { + const ws = new RecordingWs(log); + handleWebSocketGeminiLive(asWs(ws), [fixture], journal, { + ...makeDefaults(journal), + model: "gemini-2.0-flash-live-001", + }); + ws.emit( + "message", + JSON.stringify({ setup: { model: "models/gemini-2.0-flash-live-001" } }), + ); + ws.emit( + "message", + JSON.stringify({ + clientContent: { + turns: [{ role: "user", parts: [{ text: MATCH }] }], + turnComplete: true, + }, + }), + ); + }); + record(file, name, result); + }); + } + it("has its golden file", () => { + finishGolden(file); + }); +}); diff --git a/src/__tests__/helpers/delivery-lifecycle-harness.ts b/src/__tests__/helpers/delivery-lifecycle-harness.ts new file mode 100644 index 00000000..a6b3f908 --- /dev/null +++ b/src/__tests__/helpers/delivery-lifecycle-harness.ts @@ -0,0 +1,602 @@ +/** + * Shared real-socket harness for the delivery lifecycle (plan T5). Each + * provider's test file calls {@link describeDeliveryLifecycle} with one row + * describing its paths and bodies, and gets the same cases: + * + * - a client abort mid-stream is journaled (`interrupted`, `"client aborted"`, + * status 200) and nothing is written after it; + * - a healthy stream is not marked; + * - a fixture interruption keeps its own reason; + * - `nonStreamingTimingAllProviders: true` delays a `stream: false` reply + * (regression row c); + * - `nonStreamingTiming: true` on a shared fixture delays OpenAI chat only + * (regression rows a and b); + * - the non-stream default (profile and recorded timings), bare latency and + * error fixtures stay instant; + * - a client abort during the non-stream wait writes nothing and is journaled + * with status 0 (no status line was sent); + * - optionally, a proxied stream the client abandons is journaled. + * + * WebSocket rows (`transport: "ws"`) get the three stream cases through + * the per-transport handshake, plus a bare TCP drop during a delay, which must + * stop the turn promptly. Real sockets only: one LLMock per file on port + * 0, `expect.poll` on the journal, one-sided time bounds, never fake timers. + */ +import * as http from "node:http"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import type { AddressInfo, Socket } from "node:net"; +import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest"; +import { LLMock } from "../../llmock.js"; +import { WebSocketConnection } from "../../ws-framing.js"; +import type { Fixture, JournalEntry, RecordProviderKey } from "../../types.js"; +import { connectWebSocket, type WSTestClient } from "../ws-test-client.js"; + +/** The user message every harness fixture matches. Rows put it in their bodies. */ +export const HARNESS_MATCH = "delivery lifecycle probe"; +const LONG_CONTENT = + "The quick brown fox jumps over the lazy dog, again and again, for a long time. ".repeat(4); + +export type WsTransport = "responses" | "realtime" | "gemini-live"; + +export interface DeliveryLifecycleRow { + name: string; + /** HTTP path, or the WebSocket upgrade path for `transport: "ws"`. */ + streamPath: string; + nonStreamPath: string; + /** JSON body, or the multipart form fields (a small audio file is added). */ + streamBody: Record; + nonStreamBody: Record; + bodyKind: "json" | "multipart"; + headers?: Record; + /** How the stream rows run: an HTTP request, or a WebSocket turn. */ + transport: "http" | "ws"; + surface: "openai-chat" | "other"; + /** A record-mode passthrough row: the provider key and wire framing of a slow upstream. */ + proxy?: { provider: RecordProviderKey; contentType: string; frames: Array }; + /** False for stream-only surfaces (no non-streaming rows). */ + nonStream?: false; + /** The handshake for a WebSocket row. */ + wsTransport?: WsTransport; + /** The fixture `match` (default: the harness user message). */ + match?: Fixture["match"]; + /** The fixture response for the stream rows (default: long text content). */ + streamResponse?: Fixture["response"]; +} + +interface Exchange { + status: number; + elapsedMs: number; + body: Buffer; + /** Resolves once the connection has closed. */ + closed: Promise; +} + +let writeCalls = 0; + +/** Count every HTTP response write and every WebSocket send from here on. */ +function countServerWrites(): void { + const targets: Array<[object, string]> = [ + [http.ServerResponse.prototype, "writeHead"], + [http.ServerResponse.prototype, "write"], + [http.ServerResponse.prototype, "end"], + [WebSocketConnection.prototype, "send"], + ]; + for (const [proto, method] of targets) { + const record = proto as Record unknown>; + const original = record[method]; + vi.spyOn(record, method).mockImplementation(function (this: unknown, ...args: unknown[]) { + writeCalls++; + return original.apply(this, args); + }); + } +} + +const BOUNDARY = "----aimock-lifecycle-boundary"; + +function encodeBody( + row: DeliveryLifecycleRow, + body: Record, +): { payload: Buffer; contentType: string } { + if (row.bodyKind === "json") { + return { payload: Buffer.from(JSON.stringify(body)), contentType: "application/json" }; + } + const parts: Buffer[] = []; + for (const [name, value] of Object.entries(body)) { + parts.push( + Buffer.from( + `--${BOUNDARY}\r\nContent-Disposition: form-data; name="${name}"\r\n\r\n${String(value)}\r\n`, + ), + ); + } + parts.push( + Buffer.from( + `--${BOUNDARY}\r\nContent-Disposition: form-data; name="file"; filename="a.wav"\r\n` + + `Content-Type: audio/wav\r\n\r\nRIFF\r\n--${BOUNDARY}--\r\n`, + ), + ); + return { + payload: Buffer.concat(parts), + contentType: `multipart/form-data; boundary=${BOUNDARY}`, + }; +} + +function send( + baseUrl: string, + row: DeliveryLifecycleRow, + pathName: string, + body: Record, +): http.ClientRequest { + const { payload, contentType } = encodeBody(row, body); + const req = http.request(new URL(pathName, baseUrl), { + method: "POST", + agent: false, + headers: { "content-type": contentType, ...row.headers }, + }); + req.on("error", () => undefined); + req.end(payload); + return req; +} + +function exchange( + baseUrl: string, + row: DeliveryLifecycleRow, + pathName: string, + body: Record, +): Promise { + const start = Date.now(); + return new Promise((resolve, reject) => { + const req = send(baseUrl, row, pathName, body); + const closed = connectionClosed(req); + req.on("response", (res) => { + const chunks: Buffer[] = []; + res.on("data", (c: Buffer) => chunks.push(c)); + res.on("end", () => + resolve({ + status: res.statusCode ?? 0, + elapsedMs: Date.now() - start, + body: Buffer.concat(chunks), + closed, + }), + ); + res.on("error", reject); + }); + req.on("error", reject); + }); +} + +/** Send a stream request, wait for the first body bytes, then hang up. */ +function abortAfterFirstChunk( + baseUrl: string, + row: DeliveryLifecycleRow, + pathName: string, + body: Record, +): Promise { + return new Promise((resolve, reject) => { + const req = send(baseUrl, row, pathName, body); + const timer = setTimeout(() => { + req.destroy(); + reject(new Error("no first chunk within 5s")); + }, 5000); + req.on("response", (res) => { + res.once("data", () => { + clearTimeout(timer); + req.destroy(); + resolve(); + }); + res.on("error", () => undefined); + }); + }); +} + +/** Send a request and hang up after `ms`. Resolves with the bytes received. */ +function abortAfter( + baseUrl: string, + row: DeliveryLifecycleRow, + pathName: string, + body: Record, + ms: number, +): Promise<{ gotBytes: number }> { + return new Promise((resolve) => { + let gotBytes = 0; + const req = send(baseUrl, row, pathName, body); + req.on("response", (res) => { + res.on("data", (c: Buffer) => { + gotBytes += c.length; + }); + res.on("error", () => undefined); + }); + setTimeout(() => { + req.destroy(); + resolve({ gotBytes }); + }, ms); + }); +} + +/** Resolve once the client side of `req`'s connection has closed. */ +function connectionClosed(req: http.ClientRequest): Promise { + return new Promise((resolve) => { + const onSocket = (socket: Socket): void => { + if (socket.destroyed) resolve(); + else socket.once("close", () => resolve()); + }; + if (req.socket) onSocket(req.socket); + else req.once("socket", onSocket); + }); +} + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +function lastEntry(mock: LLMock): JournalEntry | undefined { + return mock.getRequests().at(-1); +} + +function lastFixtureEntry(mock: LLMock): JournalEntry | undefined { + return mock + .getRequests() + .filter((e) => e.response.fixture !== null) + .at(-1); +} + +// ─── WebSocket handshakes ────────────────────────────────────────────────── + +function typeOf(message: string): string { + const parsed = JSON.parse(message) as Record; + return typeof parsed.type === "string" ? parsed.type : (Object.keys(parsed)[0] ?? ""); +} + +const WS_CONTENT: Record boolean> = { + responses: (m) => typeOf(m) === "response.output_text.delta", + realtime: (m) => ["response.output_text.delta", "response.text.delta"].includes(typeOf(m)), + "gemini-live": (m) => typeOf(m) === "serverContent", +}; + +const WS_DONE: Record boolean> = { + responses: (m) => typeOf(m) === "response.completed", + realtime: (m) => typeOf(m) === "response.done", + "gemini-live": (m) => { + const parsed = JSON.parse(m) as { serverContent?: { turnComplete?: boolean } }; + return parsed.serverContent?.turnComplete === true; + }, +}; + +/** Run the transport's handshake up to the request that starts the turn. */ +async function startWsTurn(client: WSTestClient, transport: WsTransport): Promise { + switch (transport) { + case "responses": + client.send( + JSON.stringify({ + type: "response.create", + model: "gpt-4o", + input: [{ role: "user", content: HARNESS_MATCH }], + }), + ); + return; + case "realtime": + await client.waitForMessages(1); // session.created + client.send(JSON.stringify({ type: "session.update", session: { modalities: ["text"] } })); + await client.waitForMessages(2); // session.updated + client.send( + JSON.stringify({ + type: "conversation.item.create", + item: { + type: "message", + role: "user", + content: [{ type: "input_text", text: HARNESS_MATCH }], + }, + }), + ); + client.send(JSON.stringify({ type: "response.create" })); + return; + case "gemini-live": + client.send(JSON.stringify({ setup: { model: "models/gemini-2.0-flash-live-001" } })); + await client.waitForMessages(1); // setupComplete + client.send( + JSON.stringify({ + clientContent: { + turns: [{ role: "user", parts: [{ text: HARNESS_MATCH }] }], + turnComplete: true, + }, + }), + ); + return; + } +} + +async function waitForWsMessage( + client: WSTestClient, + predicate: (m: string) => boolean, + timeoutMs = 8000, +): Promise { + await expect + .poll(() => client.getMessages().some(predicate), { timeout: timeoutMs, interval: 20 }) + .toBe(true); +} + +// ─── The cases ───────────────────────────────────────────────────────────── + +export function describeDeliveryLifecycle(row: DeliveryLifecycleRow): void { + describe(`delivery lifecycle: ${row.name}`, () => { + let mock: LLMock; + + beforeAll(async () => { + mock = new LLMock({ port: 0, host: "127.0.0.1", chunkSize: 8 }); + await mock.start(); + }); + afterAll(async () => { + await mock.stop(); + }); + afterEach(() => { + vi.restoreAllMocks(); + mock.clearFixtures(); + mock.clearRequests(); + mock.resetMatchCounts(); + mock.disableRecording(); + writeCalls = 0; + }); + + const use = (extra: Partial, response?: Fixture["response"]): void => { + mock.addFixture({ + match: row.match ?? { userMessage: HARNESS_MATCH }, + response: response ?? row.streamResponse ?? { content: LONG_CONTENT }, + chunkSize: 8, + ...extra, + }); + }; + + if (row.transport === "ws") { + const transport = row.wsTransport; + if (!transport) throw new Error(`WS row ${row.name} needs a wsTransport`); + + it("journals a client close mid-turn and sends nothing after it", async () => { + use({ latency: 300 }); + const client = await connectWebSocket(mock.url, row.streamPath, row.headers); + await startWsTurn(client, transport); + await waitForWsMessage(client, WS_CONTENT[transport]); + client.destroy(); + countServerWrites(); + await expect + .poll(() => lastFixtureEntry(mock)?.response.interrupted, { timeout: 2000 }) + .toBe(true); + const entry = lastFixtureEntry(mock)!; + expect(entry.response.interruptReason).toBe("client aborted"); + expect(entry.response.status).toBe(200); + const before = writeCalls; + await sleep(900); + expect(writeCalls - before).toBe(0); + }, 15000); + + it("stops the turn promptly when the client drops TCP during a delay", async () => { + // A bare FIN with no close frame, while the turn waits out a long delay. + // The turn stops long before the delay ends, so the drop is seen at once, + // not at the next failed write. The reason is not asserted here. + use({ latency: 1500 }); + const client = await connectWebSocket(mock.url, row.streamPath, row.headers); + await startWsTurn(client, transport); + await waitForWsMessage(client, WS_CONTENT[transport]); + countServerWrites(); + client.destroy(); + await expect + .poll(() => lastFixtureEntry(mock)?.response.interrupted, { timeout: 750, interval: 10 }) + .toBe(true); + const before = writeCalls; + await sleep(1700); + expect(writeCalls - before).toBe(0); + }, 15000); + + it("does not mark a healthy turn", async () => { + use({ latency: 0 }); + const client = await connectWebSocket(mock.url, row.streamPath, row.headers); + await startWsTurn(client, transport); + await waitForWsMessage(client, WS_DONE[transport]); + const entry = lastFixtureEntry(mock)!; + expect(entry.response.interrupted).toBeUndefined(); + expect(entry.response.interruptReason).toBeUndefined(); + // A close after a complete turn is not an interruption either. + client.close(); + await sleep(300); + expect(entry.response.interrupted).toBeUndefined(); + expect(entry.response.interruptReason).toBeUndefined(); + }, 10000); + + it("keeps the fixture's own interruption reason", async () => { + use({ latency: 0, truncateAfterChunks: 2 }); + const client = await connectWebSocket(mock.url, row.streamPath, row.headers); + await startWsTurn(client, transport); + await expect + .poll(() => lastFixtureEntry(mock)?.response.interrupted, { timeout: 2000 }) + .toBe(true); + expect(lastFixtureEntry(mock)!.response.interruptReason).toBe("truncateAfterChunks"); + expect(lastFixtureEntry(mock)!.response.status).toBe(200); + client.destroy(); + }, 10000); + return; + } + + it("journals a client abort mid-stream and writes nothing after it", async () => { + use({ streamingProfile: { ttft: 0, tps: 5 } }); + await abortAfterFirstChunk(mock.url, row, row.streamPath, row.streamBody); + countServerWrites(); + await expect.poll(() => lastEntry(mock)?.response.interrupted, { timeout: 2000 }).toBe(true); + const entry = lastEntry(mock)!; + expect(entry.response.interruptReason).toBe("client aborted"); + expect(entry.response.status).toBe(200); + const before = writeCalls; + await sleep(600); + expect(writeCalls - before).toBe(0); + }, 10000); + + it("does not mark a healthy stream", async () => { + use({ latency: 0 }); + const res = await exchange(mock.url, row, row.streamPath, row.streamBody); + expect(res.status).toBe(200); + const entry = lastEntry(mock)!; + expect(entry.response.interrupted).toBeUndefined(); + expect(entry.response.interruptReason).toBeUndefined(); + // A close after a complete stream is not an interruption either. + await res.closed; + await sleep(100); + expect(entry.response.interrupted).toBeUndefined(); + expect(entry.response.interruptReason).toBeUndefined(); + }, 10000); + + it("keeps the fixture's own interruption reason", async () => { + use({ latency: 0, truncateAfterChunks: 2 }); + await exchange(mock.url, row, row.streamPath, row.streamBody).catch(() => undefined); + await expect.poll(() => lastEntry(mock)?.response.interrupted, { timeout: 2000 }).toBe(true); + expect(lastEntry(mock)!.response.interruptReason).toBe("truncateAfterChunks"); + expect(lastEntry(mock)!.response.status).toBe(200); + }, 10000); + + if (row.proxy) { + const proxy = row.proxy; + it("journals a client abort on a proxied stream and stops relaying", async () => { + // The upstream writes through the unspied methods, so the write count + // below sees only what aimock writes to its client. + const upstreamWrite = http.ServerResponse.prototype.write; + const upstreamEnd = http.ServerResponse.prototype.end; + const upstream = http.createServer((_req, res) => { + res.writeHead(200, { "Content-Type": proxy.contentType }); + let i = 0; + const next = (): void => { + if (res.destroyed) return; + if (i >= proxy.frames.length) { + Reflect.apply(upstreamEnd, res, []); + return; + } + Reflect.apply(upstreamWrite, res, [proxy.frames[i++]]); + setTimeout(next, 500); + }; + next(); + }); + await new Promise((resolve) => upstream.listen(0, "127.0.0.1", () => resolve())); + const fixturePath = fs.mkdtempSync(path.join(os.tmpdir(), "aimock-lifecycle-proxy-")); + try { + const port = (upstream.address() as AddressInfo).port; + mock.enableRecording({ + providers: { [proxy.provider]: `http://127.0.0.1:${port}` }, + fixturePath, + }); + await abortAfterFirstChunk(mock.url, row, row.streamPath, row.streamBody); + countServerWrites(); + await expect + .poll(() => lastEntry(mock)?.response.source, { timeout: 3000 }) + .toBe("proxy"); + const entry = lastEntry(mock)!; + expect(entry.response.interrupted).toBe(true); + expect(entry.response.interruptReason).toBe("client aborted"); + // The upstream keeps sending a frame every 500ms for over 2s: none + // of it reaches the client that left. + await sleep(1200); + expect(writeCalls).toBe(0); + } finally { + upstream.closeAllConnections(); + await new Promise((resolve) => upstream.close(() => resolve())); + fs.rmSync(fixturePath, { recursive: true, force: true }); + } + }, 10000); + } + + if (row.nonStream === false) return; + + it("nonStreamingTimingAllProviders: true delays the non-streaming reply (row c)", async () => { + use( + { nonStreamingTimingAllProviders: true, streamingProfile: { ttft: 300, tps: 10000 } }, + { content: "short" }, + ); + const res = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + expect(res.status).toBe(200); + expect(res.elapsedMs).toBeGreaterThanOrEqual(250); + }, 10000); + + it("nonStreamingTiming: true on a shared fixture delays OpenAI chat only (rows a and b)", async () => { + use({ nonStreamingTiming: true, streamingProfile: { ttft: 2000 } }, { content: "short" }); + const shared = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + expect(shared.status).toBe(200); + if (row.surface === "openai-chat") { + // Row b: the shipped delay (ttft 2000, no tps: exactly 2000ms in total). + expect(shared.elapsedMs).toBeGreaterThanOrEqual(1995); + mock.clearFixtures(); + mock.clearRequests(); + use( + { nonStreamingTimingAllProviders: true, streamingProfile: { ttft: 2000 } }, + { content: "short" }, + ); + const all = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + // Row c on OpenAI pays the same computed total. + expect(all.elapsedMs).toBeGreaterThanOrEqual(1995); + } else { + // Row a: other providers ignore nonStreamingTiming: true. + expect(shared.elapsedMs).toBeLessThan(1000); + } + }, 15000); + + it("stays instant by default with a profile or recorded timings", async () => { + use({ streamingProfile: { ttft: 2000 } }, { content: "short" }); + const a = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + mock.clearFixtures(); + use( + { + recordedTimings: { ttftMs: 2000, interChunkDelaysMs: [500, 500], totalDurationMs: 3000 }, + }, + { content: "short" }, + ); + const b = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + expect(a.status).toBe(200); + expect(b.status).toBe(200); + expect(a.elapsedMs).toBeLessThan(1000); + expect(b.elapsedMs).toBeLessThan(1000); + }, 10000); + + it("never delays on bare latency, even when opted in", async () => { + use({ latency: 2000, nonStreamingTimingAllProviders: true }, { content: "short" }); + const res = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + expect(res.status).toBe(200); + expect(res.elapsedMs).toBeLessThan(1000); + }, 10000); + + it("never delays an error fixture, and keeps its status", async () => { + use( + { nonStreamingTimingAllProviders: true, streamingProfile: { ttft: 2000 } }, + { error: { message: "rate limited", type: "rate_limit_error" }, status: 429 }, + ); + const res = await exchange(mock.url, row, row.nonStreamPath, row.nonStreamBody); + expect(res.status).toBe(429); + expect(res.elapsedMs).toBeLessThan(1000); + }, 10000); + + const nonStreamAbortCases: Array<[string, Partial]> = [ + ["nonStreamingTimingAllProviders", { nonStreamingTimingAllProviders: true }], + ]; + if (row.surface === "openai-chat") { + nonStreamAbortCases.push(["nonStreamingTiming", { nonStreamingTiming: true }]); + } + for (const [label, optIn] of nonStreamAbortCases) { + it(`a client abort during the ${label} wait writes nothing and is journaled`, async () => { + use({ ...optIn, streamingProfile: { ttft: 2000 } }, { content: "short" }); + countServerWrites(); + const { gotBytes } = await abortAfter( + mock.url, + row, + row.nonStreamPath, + row.nonStreamBody, + 100, + ); + expect(gotBytes).toBe(0); + await expect + .poll(() => lastEntry(mock)?.response.interrupted, { timeout: 1000 }) + .toBe(true); + const entry = lastEntry(mock)!; + expect(entry.response.interruptReason).toBe("client aborted"); + // No status line was ever sent, so the journal says 0, not 200. + expect(entry.response.status).toBe(0); + // Past the full 2000ms wait: nothing was ever written. + await sleep(2200); + expect(writeCalls).toBe(0); + }, 10000); + } + }); +} diff --git a/src/__tests__/helpers/exported-writer-matrix.ts b/src/__tests__/helpers/exported-writer-matrix.ts new file mode 100644 index 00000000..a93f6c29 --- /dev/null +++ b/src/__tests__/helpers/exported-writer-matrix.ts @@ -0,0 +1,310 @@ +import type * as http from "node:http"; +import type { writeSSEStream } from "../../sse-writer.js"; +import type { writeNDJSONStream } from "../../ndjson-writer.js"; +import type { writeEventStream } from "../../aws-event-stream.js"; +import type { SSEChunk } from "../../types.js"; + +/** + * Edge-case matrix for the PUBLIC exported stream writers. Each case drives a + * writer against a recording response and captures everything a caller can + * observe: the bytes written, the headers set, the return value, + * `writableEnded`, and how often `end()` was called. The base-contract test + * pins the result of every case to the pre-foundation (508dd9b9) behavior. + */ + +export interface ExportedWriters { + writeSSEStream: typeof writeSSEStream; + writeNDJSONStream: typeof writeNDJSONStream; + writeEventStream: typeof writeEventStream; +} + +export interface WriterObservation { + returned: boolean; + /** Written bytes as hex (the event stream is binary). */ + bytes: string; + headers: Record; + writableEnded: boolean; + endCalls: number; + writeHeadCalls: number; +} + +interface RecordingResponse { + res: http.ServerResponse; + /** Mark the response ended from outside the writer (no `end()` call). */ + endExternally(): void; + observe(returned: boolean): WriterObservation; +} + +interface ResponseSetup { + endedBefore?: boolean; + destroyed?: boolean; + /** Omit `writeHead` from the response, as a bare stand-in might. */ + noWriteHead?: boolean; +} + +function recordingResponse(setup: ResponseSetup = {}): RecordingResponse { + const parts: Buffer[] = []; + const headers: Record = {}; + let ended = setup.endedBefore === true; + let endCalls = 0; + let writeHeadCalls = 0; + const target: Record = { + destroyed: setup.destroyed === true, + setHeader(name: string, value: string) { + headers[name] = String(value); + }, + write(chunk: string | Buffer) { + parts.push(typeof chunk === "string" ? Buffer.from(chunk, "utf8") : Buffer.from(chunk)); + return true; + }, + end(chunk?: string | Buffer) { + endCalls++; + if (chunk !== undefined) { + parts.push(typeof chunk === "string" ? Buffer.from(chunk, "utf8") : Buffer.from(chunk)); + } + ended = true; + }, + }; + if (!setup.noWriteHead) { + target.writeHead = (_status: number, hs?: Record) => { + writeHeadCalls++; + if (hs) for (const [k, v] of Object.entries(hs)) headers[k] = String(v); + }; + } + Object.defineProperty(target, "writableEnded", { get: () => ended }); + const res = target as unknown as http.ServerResponse; + return { + res, + endExternally: () => { + ended = true; + }, + observe: (returned) => ({ + returned, + bytes: Buffer.concat(parts).toString("hex"), + headers: { ...headers }, + writableEnded: ended, + endCalls, + writeHeadCalls, + }), + }; +} + +type WriterName = "sse" | "ndjson" | "eventStream"; + +interface CaseContext { + signal: AbortSignal; + abort(): void; + endExternally(): void; +} + +interface MatrixCase { + name: string; + writer: WriterName; + frames: number; + setup?: ResponseSetup; + abortBefore?: boolean; + /** Options built per run; may hook `onChunkSent` to act mid-stream. */ + options?: (ctx: CaseContext) => { + latency?: number; + signal?: AbortSignal; + onChunkSent?: () => void; + usageChunk?: SSEChunk; + openRouterProcessing?: boolean; + }; + /** Act asynchronously while the writer is waiting (ms after start). */ + during?: { afterMs: number; act: (ctx: CaseContext) => void }; + /** Pass the legacy numeric latency argument (SSE only). */ + numericLatency?: number; +} + +function sseChunk(i: number): SSEChunk { + return { + id: "c", + object: "chat.completion.chunk", + created: 1, + model: "m", + choices: [{ index: 0, delta: { content: `t${i}` }, logprobs: null, finish_reason: null }], + }; +} + +const USAGE: SSEChunk = { + id: "c", + object: "chat.completion.chunk", + created: 1, + model: "m", + choices: [], + usage: { prompt_tokens: 1, completion_tokens: 2, total_tokens: 3 }, +}; + +function onNth(n: number, act: () => void): () => void { + let sent = 0; + return () => { + sent++; + if (sent === n) act(); + }; +} + +function casesFor(writer: WriterName): MatrixCase[] { + const cases: MatrixCase[] = [ + { name: "empty frames", writer, frames: 0 }, + { name: "two frames", writer, frames: 2 }, + { name: "two frames, latency", writer, frames: 2, options: () => ({ latency: 1 }) }, + { + name: "empty frames, signal aborted before", + writer, + frames: 0, + abortBefore: true, + options: (c) => ({ signal: c.signal }), + }, + { + name: "two frames, signal aborted before", + writer, + frames: 2, + abortBefore: true, + options: (c) => ({ signal: c.signal }), + }, + { + name: "two frames, signal aborted before, latency", + writer, + frames: 2, + abortBefore: true, + options: (c) => ({ signal: c.signal, latency: 5 }), + }, + { + name: "two frames, abort after first frame", + writer, + frames: 2, + options: (c) => ({ signal: c.signal, onChunkSent: onNth(1, c.abort) }), + }, + { + name: "two frames, abort after last frame", + writer, + frames: 2, + options: (c) => ({ signal: c.signal, onChunkSent: onNth(2, c.abort) }), + }, + { + name: "two frames, abort during delay", + writer, + frames: 2, + options: (c) => ({ signal: c.signal, latency: 40 }), + during: { afterMs: 10, act: (c) => c.abort() }, + }, + { name: "empty frames, ended before", writer, frames: 0, setup: { endedBefore: true } }, + { name: "two frames, ended before", writer, frames: 2, setup: { endedBefore: true } }, + { + name: "empty frames, ended before, signal aborted", + writer, + frames: 0, + setup: { endedBefore: true }, + abortBefore: true, + options: (c) => ({ signal: c.signal }), + }, + { + name: "two frames, ended after first frame", + writer, + frames: 2, + options: (c) => ({ onChunkSent: onNth(1, c.endExternally) }), + }, + { + name: "two frames, ended after last frame", + writer, + frames: 2, + options: (c) => ({ onChunkSent: onNth(2, c.endExternally) }), + }, + { + name: "two frames, ended during delay", + writer, + frames: 2, + options: () => ({ latency: 40 }), + during: { afterMs: 10, act: (c) => c.endExternally() }, + }, + { name: "empty frames, destroyed", writer, frames: 0, setup: { destroyed: true } }, + { name: "two frames, destroyed", writer, frames: 2, setup: { destroyed: true } }, + { name: "two frames, no writeHead", writer, frames: 2, setup: { noWriteHead: true } }, + { + name: "empty frames, no writeHead, signal aborted", + writer, + frames: 0, + setup: { noWriteHead: true }, + abortBefore: true, + options: (c) => ({ signal: c.signal }), + }, + ]; + if (writer === "sse") { + cases.push( + { name: "numeric latency", writer, frames: 2, numericLatency: 1 }, + { + name: "usage + openRouterProcessing", + writer, + frames: 1, + options: () => ({ usageChunk: USAGE, openRouterProcessing: true }), + }, + { + name: "empty frames, usage + openRouterProcessing, signal aborted before", + writer, + frames: 0, + abortBefore: true, + options: (c) => ({ signal: c.signal, usageChunk: USAGE, openRouterProcessing: true }), + }, + { + name: "openRouterProcessing, ended before", + writer, + frames: 1, + setup: { endedBefore: true }, + options: () => ({ openRouterProcessing: true }), + }, + { + name: "usage, abort after last frame", + writer, + frames: 1, + options: (c) => ({ signal: c.signal, usageChunk: USAGE, onChunkSent: onNth(1, c.abort) }), + }, + ); + } + return cases; +} + +export const MATRIX_CASES: readonly MatrixCase[] = [ + ...casesFor("sse"), + ...casesFor("ndjson"), + ...casesFor("eventStream"), +]; + +export function caseId(c: MatrixCase): string { + return `${c.writer}: ${c.name}`; +} + +export async function runCase(writers: ExportedWriters, c: MatrixCase): Promise { + const rec = recordingResponse(c.setup); + const controller = new AbortController(); + const ctx: CaseContext = { + signal: controller.signal, + abort: () => controller.abort(), + endExternally: rec.endExternally, + }; + if (c.abortBefore) controller.abort(); + const opts = c.options?.(ctx) ?? {}; + const timer = c.during ? setTimeout(() => c.during?.act(ctx), c.during.afterMs) : undefined; + let returned: boolean; + try { + if (c.writer === "sse") { + const chunks = Array.from({ length: c.frames }, (_, i) => sseChunk(i)); + returned = + c.numericLatency !== undefined + ? await writers.writeSSEStream(rec.res, chunks, c.numericLatency) + : await writers.writeSSEStream(rec.res, chunks, opts); + } else if (c.writer === "ndjson") { + const chunks = Array.from({ length: c.frames }, (_, i) => ({ i, done: false })); + returned = await writers.writeNDJSONStream(rec.res, chunks, opts); + } else { + const events = Array.from({ length: c.frames }, (_, i) => ({ + eventType: "contentBlockDelta", + payload: { i }, + })); + returned = await writers.writeEventStream(rec.res, events, opts); + } + } finally { + if (timer) clearTimeout(timer); + } + return rec.observe(returned); +} diff --git a/src/__tests__/helpers/ws-close-parity.ts b/src/__tests__/helpers/ws-close-parity.ts new file mode 100644 index 00000000..b4b75fc6 --- /dev/null +++ b/src/__tests__/helpers/ws-close-parity.ts @@ -0,0 +1,64 @@ +import { randomBytes } from "node:crypto"; +import { vi } from "vitest"; +import { WebSocketConnection } from "../../ws-framing.js"; + +/** + * An unmasked client text frame. aimock rejects it with its own + * `close(1002, "Invalid WebSocket frame")`, which marks the socket closed at + * once and emits `close` about 100ms later: the window these tests probe. + */ +export const BAD_FRAME = Buffer.from([0x81, 0x00]); + +/** A masked client frame, so several frames can go out in one TCP write. */ +export function maskedFrame(opcode: number, payload: Buffer): Buffer { + const maskKey = randomBytes(4); + const masked = Buffer.from(payload); + for (let i = 0; i < masked.length; i++) masked[i] ^= maskKey[i % 4]; + let header: Buffer; + if (payload.length < 126) { + header = Buffer.from([0x80 | opcode, 0x80 | payload.length]); + } else { + header = Buffer.alloc(4); + header[0] = 0x80 | opcode; + header[1] = 0x80 | 126; + header.writeUInt16BE(payload.length, 2); + } + return Buffer.concat([header, maskKey, masked]); +} + +/** A masked text frame. */ +export function textFrame(data: string): Buffer { + return maskedFrame(0x1, Buffer.from(data, "utf-8")); +} + +/** A masked close frame with code 1000. */ +export function closeFrame(): Buffer { + const payload = Buffer.alloc(2); + payload.writeUInt16BE(1000, 0); + return maskedFrame(0x8, payload); +} + +/** + * Make the first server send whose payload matches `matches` behave as a send + * over aimock's write limit does: aimock closes the socket with + * `close(1009, …)` and the frame is not written. Every other send is real. + * Returns the payloads that were really sent. Undo with `vi.restoreAllMocks()`. + */ +export function closeOnSend(matches: (data: string) => boolean): string[] { + const sent: string[] = []; + const realSend = WebSocketConnection.prototype.send; + let fired = false; + vi.spyOn(WebSocketConnection.prototype, "send").mockImplementation(function ( + this: WebSocketConnection, + data: string, + ) { + if (!fired && !this.isClosed && matches(data)) { + fired = true; + this.close(1009, "WebSocket write limit exceeded"); + return; + } + if (!this.isClosed) sent.push(data); + realSend.call(this, data); + }); + return sent; +} diff --git a/src/__tests__/misbehavior-anthropic.test.ts b/src/__tests__/misbehavior-anthropic.test.ts index 97446e0f..be1d40b8 100644 --- a/src/__tests__/misbehavior-anthropic.test.ts +++ b/src/__tests__/misbehavior-anthropic.test.ts @@ -1,7 +1,9 @@ import Anthropic from "@anthropic-ai/sdk"; -import { expect, test } from "vitest"; +import { expect, test, vi } from "vitest"; import { withClaudeFault } from "./helpers/misbehavior-claude-stage3.js"; import type { FixtureFileResponse, MisbehaviorFault } from "../types.js"; +import * as deliver from "../response/deliver.js"; +import * as timing from "../response/timing.js"; const request: Anthropic.MessageCreateParamsNonStreaming = { model: "claude-sonnet-4-20250514", @@ -713,3 +715,45 @@ test.each(["truncateAfterChunks", "disconnectAfterMs"] as const)( ); }, ); + +test("Claude non-streaming K4 duplicate-id sizes its wait from the served reasoning and extends the gaps", async () => { + // Hand-computed: the duplicated plan serves two `lookup` calls with + // `{"city":"Paris"}` (6 + 16 chars each) and the 10-char fixture reasoning: + // 54 chars at the mock's chunkSize 2 is 27 chunks. The recorded gaps [0, 10] + // are extended to 26 gaps by repeating the last one, so the wait is + // ttft 0 + 0 + 25 * 10 = 250ms. Without the extension the 25 extra chunks + // would average the recorded gaps (5ms): 0 + 0 + 10 + 24 * 5 = 130ms. + const deliverJson = vi.spyOn(deliver, "deliverJson"); + const delayMs = vi.spyOn(timing, "nonStreamingDelayMs"); + try { + await withClaudeFault( + { faults: [{ fault: "tool-call-id-duplicate" }] }, + async ({ client, mock }) => { + const fixture = mock.getFixtures()[0]; + if (!fixture) throw new Error("Missing fixture"); + fixture.nonStreamingTimingAllProviders = true; + fixture.recordedTimings = { ttftMs: 0, interChunkDelaysMs: [0, 10], totalDurationMs: 10 }; + const started = performance.now(); + await client.messages.create(request); + const elapsed = performance.now() - started; + expect(deliverJson).toHaveBeenCalledOnce(); + expect(deliverJson.mock.calls[0]?.[4]).toMatchObject({ + surface: "other", + payloadLength: 10 + 2 * (6 + 16), + chunkSize: 2, + duplicateId: true, + }); + expect(delayMs).toHaveBeenCalledOnce(); + expect(delayMs.mock.calls[0]?.[3]?.interChunkDelaysMs).toEqual([0, ...Array(25).fill(10)]); + expect(delayMs.mock.results[0]?.value).toBe(250); + expect(elapsed).toBeGreaterThanOrEqual(245); + // The fixture's own timings are not changed by the extension. + expect(fixture.recordedTimings.interChunkDelaysMs).toEqual([0, 10]); + }, + { reasoning: "think hard", toolCalls: [{ name: "lookup", arguments: { city: "Paris" } }] }, + ); + } finally { + delayMs.mockRestore(); + deliverJson.mockRestore(); + } +}); diff --git a/src/__tests__/misbehavior-composition.test.ts b/src/__tests__/misbehavior-composition.test.ts index 16a04fae..f3b28e38 100644 --- a/src/__tests__/misbehavior-composition.test.ts +++ b/src/__tests__/misbehavior-composition.test.ts @@ -320,9 +320,9 @@ test("P1d time-disconnected applied stream retains the full prepared usage of it chunkSize: 8, streamingProfile: { ttft: 0, tps: 50 }, }; - // Call-through observation of the real writer's prepared usage argument. + // Call-through observation of the real SSE codec's prepared usage argument. // This is server-side accounting, not evidence that an interrupted client received usage. - const writer = vi.spyOn(sseWriter, "writeSSEStream"); + const writer = vi.spyOn(sseWriter, "sseCodec"); try { const observations = []; for (const disconnectAfterMs of [undefined, 80]) { @@ -353,10 +353,7 @@ test("P1d time-disconnected applied stream retains the full prepared usage of it } } expect(writer).toHaveBeenCalledTimes(2); - const options = writer.mock.calls.map((call) => call[2]); - const preparedUsage = options.map((option) => - typeof option === "object" ? option.usageChunk?.usage : undefined, - ); + const preparedUsage = writer.mock.calls.map(([option]) => option.usageChunk?.usage); const expectedUsage = { prompt_tokens: Math.ceil(prompt.length / 4), completion_tokens: Math.ceil(("weather" + faultedArguments).length / 4), diff --git a/src/__tests__/misbehavior-fault-journal-order.test.ts b/src/__tests__/misbehavior-fault-journal-order.test.ts index 6884bc98..00b15b8d 100644 --- a/src/__tests__/misbehavior-fault-journal-order.test.ts +++ b/src/__tests__/misbehavior-fault-journal-order.test.ts @@ -10,11 +10,32 @@ * /metrics. */ import http from "node:http"; -import { afterEach, describe, expect, it } from "vitest"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { InvalidFixtureOutputError } from "../helpers.js"; import type { ServerInstance } from "../server.js"; import type { Fixture, FixtureResponse, MisbehaviorFault, TextResponse, WireId } from "../types.js"; import { createServer } from "./helpers/misbehavior-enabled.js"; +// The planner pre-check rejects every malformed `citations` and `files` value +// before a fault is chosen, so no fixture alone can make the Responses +// misbehavior builder throw. This switch makes the builder's usage renderer +// throw the same coded error the fixture checks throw. It is off by default, +// so every other test sees the real renderer. +const responsesBuild = vi.hoisted(() => ({ fail: false, calls: 0 })); +vi.mock("../response/usage/codecs/responses.js", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + render: (...args: Parameters) => { + responsesBuild.calls++; + if (responsesBuild.fail) { + throw new InvalidFixtureOutputError("aimock_invalid_fixture_citation", "forced by test"); + } + return actual.render(...args); + }, + }; +}); + const png = { mimeType: "image/png", data: "iVBORw0KGgo=" }; const badCitations = [{ url: "https://example.com", startIndex: 0, endIndex: 999 }]; @@ -429,3 +450,48 @@ describe("a non-stream Chat reply records its fault before the timing wait", () }); } }); + +describe("a non-stream Responses reply that throws while it is built records no applied fault", () => { + // The fixture passes the planner pre-check; the builder then throws. The + // request was not served with the fault, so the journal and the counter must + // not record it as applied, whether the reply is built before writeHead + // (untimed) or after the timing wait (timed). + const route = responsesCite[0]; + for (const timed of [false, true]) { + it(`${route.name}, ${timed ? "nonStreamingTimingAllProviders" : "untimed"}`, async () => { + const fixture: Fixture = { + match: {}, + response: { content: text, toolCalls }, + misbehavior: { faults: [{ fault: "tool-unknown-name", rate: 1 }] }, + ...(timed + ? { streamingProfile: { ttft: 50, tps: 1_000_000 }, nonStreamingTimingAllProviders: true } + : {}), + }; + instance = await createServer([fixture], { metrics: true }); + responsesBuild.fail = true; + responsesBuild.calls = 0; + let raw: string; + let status: number; + try { + const res = await post(`${instance.url}${route.path}`, route.body); + raw = await res.text(); + status = res.status; + } finally { + responsesBuild.fail = false; + } + // The forced throw came from the builder, not from anything before it. + expect(responsesBuild.calls).toBe(1); + expect(status).toBe(500); + expect(raw).toContain(CITE); + + const entries = instance.journal.getAll(); + expect(entries).toHaveLength(1); + expect(entries[0].response.status).toBe(500); + expect(entries[0].response.fixture).toBe(fixture); + expect(entries[0].response.misbehavior).toBeUndefined(); + + const metrics = await (await fetch(`${instance.url}/metrics`)).text(); + expect(misbehaviorLines(metrics)).toEqual([]); + }); + } +}); diff --git a/src/__tests__/misbehavior-openai.test.ts b/src/__tests__/misbehavior-openai.test.ts index 3c321da2..394376f1 100644 --- a/src/__tests__/misbehavior-openai.test.ts +++ b/src/__tests__/misbehavior-openai.test.ts @@ -31,6 +31,7 @@ import { import type { FixtureResponse, ChatCompletionRequest } from "../types.js"; import { withFaultFixture } from "./helpers/misbehavior-server.js"; import { LLMock, createServer } from "./helpers/misbehavior-enabled.js"; +import * as deliver from "../response/deliver.js"; const cases = [ { cell: "control", fault: undefined }, @@ -2041,6 +2042,51 @@ test("A2 real K4 extra chunks repeat the last recorded gap without mutating timi ); }); +test("A2 non-streaming K4 duplicate-id extends the recorded gaps before sizing the wait", async () => { + // Hand-computed: two `weather` calls with `{"city":"Paris"}` (7 + 16 chars + // each) are 46 chars, 5 chunks at chunkSize 10. The recorded gaps [0, 60] + // are extended to 4 gaps by repeating the last one: 0 + 0 + 3 * 60 = 180ms. + // Without the extension the 2 extra chunks would average the recorded gaps + // (30ms): 0 + 0 + 60 + 2 * 30 = 120ms. + const timings = { ttftMs: 0, interChunkDelaysMs: [0, 60], totalDurationMs: 60 }; + const snapshot = structuredClone(timings); + const deliverJson = vi.spyOn(deliver, "deliverJson"); + try { + await withRuntimeServer( + [ + { + match: {}, + misbehavior: "tool-call-id-duplicate", + response: { toolCalls: [{ name: "weather", arguments: '{"city":"Paris"}' }] }, + recordedTimings: timings, + nonStreamingTiming: true, + chunkSize: 10, + }, + ], + async ({ url }) => { + const client = new OpenAI({ apiKey: "local", baseURL: `${url}/v1`, maxRetries: 0 }); + const started = performance.now(); + const completion = await client.chat.completions.create(request); + const elapsed = performance.now() - started; + const calls = completion.choices[0].message.tool_calls ?? []; + expect(calls).toHaveLength(2); + expect(calls[0].id).toBe(calls[1].id); + expect(deliverJson).toHaveBeenCalledOnce(); + expect(deliverJson.mock.calls[0]?.[4]).toMatchObject({ + surface: "openai-chat", + payloadLength: 2 * (7 + 16), + chunkSize: 10, + duplicateId: true, + }); + expect(elapsed).toBeGreaterThanOrEqual(175); + expect(timings).toEqual(snapshot); + }, + ); + } finally { + deliverJson.mockRestore(); + } +}); + async function withDirectRuntime( fixture: Fixture, run: (instance: ServerInstance) => Promise, diff --git a/src/__tests__/proxy-hangup.test.ts b/src/__tests__/proxy-hangup.test.ts new file mode 100644 index 00000000..27f41fa3 --- /dev/null +++ b/src/__tests__/proxy-hangup.test.ts @@ -0,0 +1,203 @@ +/** + * Proxy hang-up detection on real sockets: a real front server runs the + * record-mode relay (`proxyAndRecord`) inside `withHangUpDetection` against a + * slow local upstream, and journals with `proxyInterruption`. + * + * - A client that sends its body, half-closes its write side and keeps reading + * is NOT a hang-up: the stream completes and nothing is marked, and if the + * upstream later dies mid-stream the tear-down is the proxy's, not the client's. + * - A client whose socket is already gone when the helper is entered IS a + * hang-up, and is journaled as a client abort. + */ +import * as http from "node:http"; +import * as net from "node:net"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import type { AddressInfo } from "node:net"; +import { afterEach, describe, expect, it } from "vitest"; +import { Logger } from "../logger.js"; +import { proxyAndRecord } from "../recorder.js"; +import { CLIENT_ABORTED } from "../response/lifecycle.js"; +import { proxyInterruption, withHangUpDetection } from "../response/proxy.js"; +import type { ChatCompletionRequest } from "../types.js"; + +const sse = (content: string): string => + `data: ${JSON.stringify({ + id: "chatcmpl-up", + object: "chat.completion.chunk", + created: 1, + model: "gpt-4o", + choices: [{ index: 0, delta: { content }, finish_reason: null }], + })}\n\n`; +const FRAMES = [sse("alpha"), sse(" beta"), sse(" gamma"), "data: [DONE]\n\n"]; + +const chatBody = JSON.stringify({ + model: "gpt-4o", + messages: [{ role: "user", content: "proxy hang-up probe" }], + stream: true, +}); +const rawRequest = + `POST /v1/chat/completions HTTP/1.1\r\nHost: 127.0.0.1\r\n` + + `Content-Type: application/json\r\nContent-Length: ${Buffer.byteLength(chatBody)}\r\n` + + `Connection: close\r\n\r\n${chatBody}`; + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +async function listen(server: http.Server): Promise { + await new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve())); + cleanups.push(async () => { + server.closeAllConnections(); + await new Promise((resolve) => server.close(() => resolve())); + }); + return (server.address() as AddressInfo).port; +} + +/** + * An upstream that writes one SSE frame every `gapMs`. With `dieAfter`, it + * destroys its socket after that many frames instead of finishing. + */ +async function slowUpstream(gapMs: number, dieAfter?: number): Promise { + const upstream = http.createServer((req, res) => { + req.resume(); + res.writeHead(200, { "Content-Type": "text/event-stream" }); + let i = 0; + const next = (): void => { + if (res.destroyed) return; + if (dieAfter !== undefined && i >= dieAfter) { + res.socket?.destroy(); + return; + } + if (i >= FRAMES.length) { + res.end(); + return; + } + res.write(FRAMES[i++]); + setTimeout(next, gapMs); + }; + next(); + }); + return `http://127.0.0.1:${await listen(upstream)}`; +} + +interface Journaled { + clientHungUp: boolean; + fields: ReturnType; +} + +/** + * A front server that relays every request to `upstreamUrl` the way the record + * arm does. `enterAfterClose` enters the helper only once the client is gone. + */ +async function frontServer( + upstreamUrl: string, + opts: { allowHalfOpen: boolean; enterAfterClose?: boolean }, +): Promise<{ port: number; journaled: Promise }> { + const fixturePath = fs.mkdtempSync(path.join(os.tmpdir(), "aimock-proxy-hangup-")); + cleanups.push(() => fs.rmSync(fixturePath, { recursive: true, force: true })); + let settle!: (j: Journaled) => void; + let fail!: (err: unknown) => void; + const journaled = new Promise((resolve, reject) => { + settle = resolve; + fail = reject; + }); + const server = http.createServer((req, res) => { + const relay = (): void => { + withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + JSON.parse(chatBody) as ChatCompletionRequest, + "openai", + "/v1/chat/completions", + [], + { record: { providers: { openai: upstreamUrl }, fixturePath }, logger: new Logger() }, + chatBody, + ), + ).then( + ({ clientHungUp }) => + settle({ clientHungUp, fields: proxyInterruption(res, clientHungUp) }), + fail, + ); + }; + req.resume(); + req.on("end", () => { + if (opts.enterAfterClose) res.once("close", relay); + else relay(); + }); + }); + // Node reads this undeclared flag on a peer FIN: true keeps the write side + // open for a half-closed client, false (the default) ends the response. + Object.assign(server, { httpAllowHalfOpen: opts.allowHalfOpen }); + return { port: await listen(server), journaled }; +} + +/** Send the request; on the first response bytes run `onFirstData`. Resolves with all bytes read. */ +function rawClient(port: number, onFirstData: (socket: net.Socket) => void): Promise { + return new Promise((resolve) => { + const chunks: Buffer[] = []; + const socket = net.connect(port, "127.0.0.1", () => socket.write(rawRequest)); + socket.on("data", (d: Buffer) => { + chunks.push(d); + if (chunks.length === 1) onFirstData(socket); + }); + socket.on("error", () => undefined); + socket.on("close", () => resolve(Buffer.concat(chunks).toString())); + }); +} + +describe("proxy hang-up detection", () => { + it("a client that half-closes and keeps reading gets the whole stream and is not a hang-up", async () => { + const { port, journaled } = await frontServer(await slowUpstream(100), { + allowHalfOpen: true, + }); + // Half-close: the request is complete, the read side stays open. + const received = await rawClient(port, (socket) => socket.end()); + expect(received).toContain(" gamma"); + expect(received).toContain("data: [DONE]"); + const { clientHungUp, fields } = await journaled; + expect(clientHungUp).toBe(false); + expect(fields).toEqual({}); + }, 10000); + + it("a half-closed client is not blamed when the upstream dies mid-stream", async () => { + const { port, journaled } = await frontServer(await slowUpstream(100, 2), { + allowHalfOpen: true, + }); + const received = await rawClient(port, (socket) => socket.end()); + expect(received).toContain("alpha"); + expect(received).not.toContain("data: [DONE]"); + const { clientHungUp, fields } = await journaled; + expect(clientHungUp).toBe(false); + expect(fields).toEqual({ interrupted: true, interruptReason: "proxy stream destroyed" }); + }, 10000); + + it("a client that hangs up mid-stream is journaled as a client abort", async () => { + const { port, journaled } = await frontServer(await slowUpstream(200), { + allowHalfOpen: true, + }); + await rawClient(port, (socket) => socket.destroy()); + const { clientHungUp, fields } = await journaled; + expect(clientHungUp).toBe(true); + expect(fields).toEqual({ interrupted: true, interruptReason: CLIENT_ABORTED }); + }, 10000); + + it("a client already gone when the proxy helper is entered is journaled as a client abort", async () => { + const { port, journaled } = await frontServer(await slowUpstream(100), { + allowHalfOpen: false, + enterAfterClose: true, + }); + const socket = net.connect(port, "127.0.0.1", () => { + socket.write(rawRequest); + setTimeout(() => socket.destroy(), 50); + }); + socket.on("error", () => undefined); + const { clientHungUp, fields } = await journaled; + expect(clientHungUp).toBe(true); + // Nothing of the response reached the socket: no status line, status 0. + expect(fields).toEqual({ status: 0, interrupted: true, interruptReason: CLIENT_ABORTED }); + }, 10000); +}); diff --git a/src/__tests__/response-deliver.test.ts b/src/__tests__/response-deliver.test.ts new file mode 100644 index 00000000..8b1eb002 --- /dev/null +++ b/src/__tests__/response-deliver.test.ts @@ -0,0 +1,894 @@ +import { EventEmitter } from "node:events"; +import * as http from "node:http"; +import type { AddressInfo } from "node:net"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { deliverJson, deliverStream } from "../response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "../response/proxy.js"; +import { CLIENT_ABORTED, teardownReason } from "../response/lifecycle.js"; +import { applyChaosAction } from "../chaos.js"; +import { Journal } from "../journal.js"; +import type { FrameCodec } from "../response/frame-loop.js"; +import type { Fixture, JournalEntry, RecordedTimings } from "../types.js"; + +type Op = Record; + +class FakeRes extends EventEmitter { + ops: Op[] = []; + destroyed = false; + writableEnded = false; + writableFinished = false; + headersSent = false; + statusCode = 200; + setHeader(name: string, value: string): void { + this.ops.push({ op: "setHeader", name, value }); + } + writeHead(status: number, headers?: Record): this { + this.ops.push({ op: "writeHead", status, headers }); + this.headersSent = true; + return this; + } + write(chunk: string): boolean { + this.ops.push({ op: "write", chunk }); + // A real write commits the status line and headers. + this.headersSent = true; + return true; + } + end(chunk?: string): this { + this.ops.push(chunk === undefined ? { op: "end" } : { op: "end", chunk }); + this.writableEnded = true; + this.writableFinished = true; + return this; + } + destroy(): this { + this.ops.push({ op: "destroy" }); + this.destroyed = true; + queueMicrotask(() => this.emit("close")); + return this; + } +} + +const asRes = (res: FakeRes): http.ServerResponse => res as unknown as http.ServerResponse; + +function entry(): JournalEntry { + return { + id: "e1", + timestamp: 0, + method: "POST", + path: "/x", + headers: {}, + body: null, + response: { status: 200, fixture: null }, + }; +} + +function fixture(extra: Partial = {}): Fixture { + return { match: {}, response: { content: "hello" }, ...extra }; +} + +const codec: FrameCodec = { + headers: { "Content-Type": "text/event-stream" }, + encode: (frame) => `data: ${frame}\n\n`, +}; + +const jsonOpts = { surface: "other" as const, payloadLength: 10, chunkSize: 5 }; + +beforeEach(() => { + vi.useFakeTimers(); + vi.setSystemTime(new Date("2026-01-01T00:00:00.000Z")); +}); +afterEach(() => { + vi.useRealTimers(); +}); + +describe("deliverJson default path", () => { + it("writes the same writeHead/end calls as today and calls the thunk exactly once", async () => { + const res = new FakeRes(); + const body = vi.fn(() => ({ a: 1 })); + await deliverJson(asRes(res), entry(), fixture(), body, jsonOpts); + expect(body).toHaveBeenCalledOnce(); + expect(res.ops).toEqual([ + { op: "writeHead", status: 200, headers: { "Content-Type": "application/json" } }, + { op: "end", chunk: '{"a":1}' }, + ]); + }); + + it("does not wait when the fixture has a profile but no opt-in", async () => { + const res = new FakeRes(); + const promise = deliverJson( + asRes(res), + entry(), + fixture({ streamingProfile: { ttft: 5000 } }), + () => ({}), + jsonOpts, + ); + await promise; + expect(res.ops).toHaveLength(2); + expect(vi.getTimerCount()).toBe(0); + }); + + it("surface other with nonStreamingTiming: true does not wait (M1 row a)", async () => { + const res = new FakeRes(); + const promise = deliverJson( + asRes(res), + entry(), + fixture({ nonStreamingTiming: true, streamingProfile: { ttft: 5000 } }), + () => ({}), + jsonOpts, + ); + // Flush microtasks only: no timer was armed and the reply is already written. + await vi.advanceTimersByTimeAsync(0); + expect(vi.getTimerCount()).toBe(0); + expect(res.ops).toHaveLength(2); + await vi.runAllTimersAsync(); + await promise; + }); + + it("a throwing thunk propagates and writeHead is never called", async () => { + const res = new FakeRes(); + await expect( + deliverJson( + asRes(res), + entry(), + fixture(), + () => { + throw new Error("builder failed"); + }, + jsonOpts, + ), + ).rejects.toThrow("builder failed"); + expect(res.ops).toEqual([]); + }); + + it("status and headers (#549): headers merged over Content-Type", async () => { + const res = new FakeRes(); + await deliverJson(asRes(res), entry(), fixture(), () => ({ ok: true }), { + ...jsonOpts, + status: 201, + headers: { "x-a": "1", "Content-Type": "application/x-json" }, + }); + expect(res.ops[0]).toEqual({ + op: "writeHead", + status: 201, + headers: { "Content-Type": "application/x-json", "x-a": "1" }, + }); + }); +}); + +describe("deliverJson timed path", () => { + const timed = fixture({ + nonStreamingTimingAllProviders: true, + streamingProfile: { ttft: 300, tps: 10 }, + }); + + it("waits the nonStreamingDelayMs total, then calls the thunk after the timer fires", async () => { + const res = new FakeRes(); + // Hand-computed: 10 chars at chunkSize 5 is 2 chunks, ttft 300 + 1000 / 10 tps. + const expected = 400; + let readAt = -1; + const body = vi.fn(() => { + readAt = Date.now(); + return { created: Math.floor(Date.now() / 1000) }; + }); + const start = Date.now(); + const promise = deliverJson(asRes(res), entry(), timed, body, jsonOpts); + await vi.advanceTimersByTimeAsync(expected - 1); + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([]); + await vi.advanceTimersByTimeAsync(1); + await promise; + expect(body).toHaveBeenCalledOnce(); + expect(readAt - start).toBe(expected); + expect(res.ops.map((o) => o.op)).toEqual(["writeHead", "end"]); + }); + + it("a client abort during the wait never calls the thunk, writes nothing, and marks the entry", async () => { + const res = new FakeRes(); + const e = entry(); + const body = vi.fn(() => ({})); + const promise = deliverJson(asRes(res), e, timed, body, jsonOpts); + await vi.advanceTimersByTimeAsync(100); + res.destroyed = true; + res.emit("close"); + await promise; + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([]); + // No status line was ever sent: the server-wide rule journals status 0. + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + expect(vi.getTimerCount()).toBe(0); + expect(res.listenerCount("close")).toBe(0); + }); + + it("serves a NaN chunkSize (a non-numeric user value after Math.max) with no wait, as base did", async () => { + const res = new FakeRes(); + const body = vi.fn(() => ({ ok: true })); + await deliverJson(asRes(res), entry(), timed, body, { ...jsonOpts, chunkSize: Number.NaN }); + expect(body).toHaveBeenCalledOnce(); + expect(res.ops.map((o) => o.op)).toEqual(["writeHead", "end"]); + expect(vi.getTimerCount()).toBe(0); + }); + + it.each([0, -1])( + "rejects chunkSize %s as a programming error and writes nothing", + async (chunkSize) => { + const res = new FakeRes(); + const body = vi.fn(() => ({})); + await expect( + deliverJson(asRes(res), entry(), timed, body, { ...jsonOpts, chunkSize }), + ).rejects.toThrow(/chunkSize must be >= 1/); + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([]); + expect(vi.getTimerCount()).toBe(0); + }, + ); + + it("a client already gone at the start skips the wait, never calls the thunk, and marks the entry", async () => { + const res = new FakeRes(); + res.destroyed = true; + const e = entry(); + const body = vi.fn(() => ({})); + // No timers are advanced: the call must finish without waiting. + await deliverJson(asRes(res), e, timed, body, jsonOpts); + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([]); + expect(e.response.interrupted).toBe(true); + expect(vi.getTimerCount()).toBe(0); + }); + + it("a response destroyed during the wait without a close event writes nothing and marks the entry", async () => { + const res = new FakeRes(); + const e = entry(); + const body = vi.fn(() => ({})); + const promise = deliverJson(asRes(res), e, timed, body, jsonOpts); + await vi.advanceTimersByTimeAsync(100); + res.destroyed = true; + await vi.runAllTimersAsync(); + await promise; + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([]); + // No close reached the lifecycle, so the reason comes from the teardown + // fallback; no status line was sent, so the status is 0. + expect(e.response.status).toBe(0); + expect(e.response.interrupted).toBe(true); + expect(e.response.interruptReason).toBe(teardownReason(asRes(res))); + expect(e.response.interruptReason).toBeDefined(); + }); + + it("a thunk that throws after the wait propagates, writes nothing, and disposes", async () => { + const res = new FakeRes(); + const promise = deliverJson( + asRes(res), + entry(), + timed, + () => { + throw new Error("late builder failure"); + }, + jsonOpts, + ); + const assertion = expect(promise).rejects.toThrow("late builder failure"); + await vi.runAllTimersAsync(); + await assertion; + expect(res.ops).toEqual([]); + expect(res.listenerCount("close")).toBe(0); + }); + + it("surface openai-chat honors nonStreamingTiming: true", async () => { + const res = new FakeRes(); + const f = fixture({ nonStreamingTiming: true, streamingProfile: { ttft: 400 } }); + const promise = deliverJson(asRes(res), entry(), f, () => ({}), { + ...jsonOpts, + surface: "openai-chat", + }); + await vi.advanceTimersByTimeAsync(399); + expect(res.ops).toEqual([]); + await vi.advanceTimersByTimeAsync(1); + await promise; + expect(res.ops).toHaveLength(2); + }); + + it("applies the duplicate-id extension (hand-computed)", async () => { + // 16 chars at chunkSize 3 is 6 chunks. The recorded gaps [1, 9] are + // extended to 5 gaps by repeating the last one: 5 + 1 + 4 * 9 = 42ms. + // Without the extension the 3 extra chunks would average the recorded + // gaps (5ms): 5 + 1 + 9 + 3 * 5 = 30ms. + const recorded: RecordedTimings = { + ttftMs: 5, + interChunkDelaysMs: [1, 9], + totalDurationMs: 15, + }; + const f = fixture({ nonStreamingTimingAllProviders: true, recordedTimings: recorded }); + const res = new FakeRes(); + const promise = deliverJson(asRes(res), entry(), f, () => ({}), { + surface: "other", + payloadLength: 16, + chunkSize: 3, + recordedTimings: recorded, + duplicateId: true, + replaySpeed: 1, + }); + await vi.advanceTimersByTimeAsync(41); + expect(res.ops).toEqual([]); + await vi.advanceTimersByTimeAsync(1); + await promise; + expect(res.ops).toHaveLength(2); + expect(recorded.interChunkDelaysMs).toEqual([1, 9]); + }); + + it("an empty payload still waits one chunk: the ttft (the Math.max(1, …) clamp)", async () => { + // ceil(0 / 3) is 0 chunks; the clamp makes it 1, so the wait is ttft 5ms. + const recorded: RecordedTimings = { + ttftMs: 5, + interChunkDelaysMs: [1, 9], + totalDurationMs: 15, + }; + const f = fixture({ nonStreamingTimingAllProviders: true, recordedTimings: recorded }); + const res = new FakeRes(); + const promise = deliverJson(asRes(res), entry(), f, () => ({}), { + surface: "other", + payloadLength: 0, + chunkSize: 3, + recordedTimings: recorded, + duplicateId: true, + replaySpeed: 1, + }); + await vi.advanceTimersByTimeAsync(4); + expect(res.ops).toEqual([]); + await vi.advanceTimersByTimeAsync(1); + await promise; + expect(res.ops).toHaveLength(2); + }); +}); + +describe("deliverStream", () => { + it("streams the frames with the codec and leaves a healthy entry unmarked", async () => { + const res = new FakeRes(); + const e = entry(); + await deliverStream(asRes(res), e, fixture(), codec, ["a", "b"], {}); + expect(res.ops).toEqual([ + { op: "setHeader", name: "Content-Type", value: "text/event-stream" }, + { op: "write", chunk: "data: a\n\n" }, + { op: "write", chunk: "data: b\n\n" }, + { op: "end" }, + ]); + expect(e.response.interrupted).toBeUndefined(); + }); + + it("on a client abort: destroy, then mark, then dispose", async () => { + const res = new FakeRes(); + const e = entry(); + const order: string[] = []; + const origDestroy = res.destroy.bind(res); + res.destroy = () => { + order.push(`destroy:${String(e.response.interrupted)}`); + return origDestroy(); + }; + // dispose() removes the close listener through `off`: record whether the + // entry was already marked at that point. + const origOff = res.off.bind(res); + res.off = (event: string | symbol, listener: (...args: unknown[]) => void) => { + if (event === "close") order.push(`dispose:${String(e.response.interrupted)}`); + return origOff(event, listener); + }; + const promise = deliverStream(asRes(res), e, fixture(), codec, ["a", "b", "c"], { + latency: 50, + }); + await vi.advanceTimersByTimeAsync(50); + res.emit("close"); + await promise; + order.push(`listeners:${res.listenerCount("close")}`); + expect(order).toEqual(["destroy:undefined", "dispose:true", "listeners:0"]); + expect(e.response).toMatchObject({ + status: 200, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(1); + }); + + it("a truncateAfterChunks reason wins over the close its destroy causes", async () => { + // The close is emitted synchronously inside destroy(), while the lifecycle + // still listens, so the client cause really reaches the lifecycle. + class SyncCloseRes extends FakeRes { + closesSeen = 0; + override destroy(): this { + this.ops.push({ op: "destroy" }); + this.destroyed = true; + this.closesSeen = this.listenerCount("close"); + this.emit("close"); + return this; + } + } + const res = new SyncCloseRes(); + const e = entry(); + await deliverStream( + asRes(res), + e, + fixture({ truncateAfterChunks: 2 }), + codec, + ["a", "b", "c"], + {}, + ); + await vi.runAllTimersAsync(); + expect(res.closesSeen).toBe(1); + expect(e.response.interruptReason).toBe("truncateAfterChunks"); + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(2); + expect(res.ops.at(-1)).toEqual({ op: "destroy" }); + }); + + it("a response destroyed mid-stream without a close event is journaled through the teardown fallback", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream(asRes(res), e, fixture(), codec, ["a", "b", "c"], { + latency: 50, + }); + await vi.advanceTimersByTimeAsync(50); + // Gone, but no close event: the lifecycle never fired, so it has no reason. + res.destroyed = true; + await vi.runAllTimersAsync(); + await promise; + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(1); + expect(e.response.interrupted).toBe(true); + expect(e.response.interruptReason).toBe(teardownReason(asRes(res))); + expect(e.response.interruptReason).toBeDefined(); + // The first frame committed the status line, so the status is kept. + expect(e.response.status).toBe(200); + }); + + it("does not destroy an already-ended response when the loop reports an abort", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream(asRes(res), e, fixture({ disconnectAfterMs: 10 }), codec, ["a"], { + latency: 20, + }); + await vi.advanceTimersByTimeAsync(5); + res.writableEnded = true; + await vi.runAllTimersAsync(); + await promise; + expect(res.ops.some((o) => o.op === "destroy")).toBe(false); + expect(e.response.interruptReason).toBe("disconnectAfterMs"); + }); + + it("status and headers (#549): writeHead(status, merged) before the first frame", async () => { + const res = new FakeRes(); + await deliverStream( + asRes(res), + entry(), + fixture(), + codec, + ["a"], + {}, + { + status: 207, + headers: { "x-a": "1" }, + }, + ); + expect(res.ops[0]).toEqual({ + op: "writeHead", + status: 207, + headers: { "Content-Type": "text/event-stream", "x-a": "1" }, + }); + }); + + it("a response chaos already tore down is journaled with the chaos reason and status 0", async () => { + // The chaos `disconnect` destroyed the response before streaming and no + // `close` reached the lifecycle: the loop stops on a gone sink alone. + const res = new FakeRes(); + res.destroy = function (this: FakeRes) { + this.destroyed = true; + return this; + }; + applyChaosAction( + "disconnect", + asRes(res), + null, + new Journal(), + { method: "POST", path: "/x", headers: {}, body: null }, + "fixture", + undefined, + undefined, + ); + const e = entry(); + await deliverStream(asRes(res), e, fixture(), codec, ["a", "b"], {}); + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(0); + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: "disconnect", + }); + }); + + it("a server-side destroy is not attributed to the client", async () => { + // A request socket that saw no FIN and no RST: aimock tore it down. + const res = new FakeRes(); + Object.assign(res, { req: { socket: { writableEnded: false, errored: null } } }); + res.destroyed = true; + const e = entry(); + await deliverStream(asRes(res), e, fixture(), codec, ["a"], {}); + expect(e.response).toMatchObject({ status: 0, interrupted: true }); + expect(e.response.interruptReason).not.toBe(CLIENT_ABORTED); + expect(e.response.interruptReason).toBe("disconnect"); + }); + + it("a client that leaves before the first write journals status 0", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream(asRes(res), e, fixture(), codec, ["a", "b"], { latency: 50 }); + await vi.advanceTimersByTimeAsync(20); + res.destroyed = true; + res.emit("close"); + await promise; + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(0); + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + }); + + it("on a real socket, a server destroy mid-stream is journaled as aimock's teardown", async () => { + vi.useRealTimers(); + const e = entry(); + let done: Promise = Promise.resolve(); + const server = http.createServer((_req, res) => { + done = deliverStream(res, e, fixture(), codec, ["a", "b", "c", "d"], { latency: 60 }); + setTimeout(() => res.destroy(), 150); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve())); + try { + const port = (server.address() as AddressInfo).port; + await new Promise((resolve) => { + const req = http.request({ host: "127.0.0.1", port, method: "POST", agent: false }); + req.on("response", (r) => { + r.on("data", () => undefined); + r.on("error", () => resolve()); + r.on("close", () => resolve()); + }); + req.on("error", () => resolve()); + req.end(); + }); + await done; + expect(e.response).toMatchObject({ + status: 200, + interrupted: true, + interruptReason: "disconnect", + }); + } finally { + server.closeAllConnections(); + await new Promise((resolve) => server.close(() => resolve())); + } + }); + + it("on a real socket, a client hang-up mid-stream is still the client's", async () => { + vi.useRealTimers(); + const e = entry(); + let done: Promise = Promise.resolve(); + const server = http.createServer((_req, res) => { + done = deliverStream(res, e, fixture(), codec, ["a", "b", "c", "d"], { latency: 60 }); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve())); + try { + const port = (server.address() as AddressInfo).port; + await new Promise((resolve) => { + const req = http.request({ host: "127.0.0.1", port, method: "POST", agent: false }); + req.on("response", (r) => { + r.once("data", () => { + req.destroy(); + resolve(); + }); + r.on("error", () => undefined); + }); + req.on("error", () => undefined); + req.end(); + }); + await expect.poll(() => e.response.interrupted, { timeout: 2000 }).toBe(true); + await done; + expect(e.response).toMatchObject({ status: 200, interruptReason: CLIENT_ABORTED }); + } finally { + server.closeAllConnections(); + await new Promise((resolve) => server.close(() => resolve())); + } + }); + + it("omitted status and headers give the default setHeader calls", async () => { + const a = new FakeRes(); + const b = new FakeRes(); + await deliverStream(asRes(a), entry(), fixture(), codec, ["a"], {}); + await deliverStream(asRes(b), entry(), fixture(), codec, ["a"], {}, {}); + expect(a.ops).toEqual(b.ops); + }); +}); + +describe("proxy helpers", () => { + class FakeSocket extends EventEmitter { + destroyed = false; + errored: Error | null = null; + writableEnded = false; + } + + it("withHangUpDetection reports a client-caused close before the response ended, and removes its listener", async () => { + for (const close of [ + (s: FakeSocket) => (s.errored = new Error("ECONNRESET")), + (s: FakeSocket) => (s.writableEnded = true), + ]) { + const socket = new FakeSocket(); + const req = { socket } as unknown as http.IncomingMessage; + const result = await withHangUpDetection(req, asRes(new FakeRes()), async () => { + close(socket); + socket.destroyed = true; + socket.emit("close"); + return "relayed"; + }); + expect(result).toEqual({ outcome: "relayed", clientHungUp: true }); + expect(socket.listenerCount("close")).toBe(0); + } + }); + + it("withHangUpDetection ignores a half-close and our own tear-down", async () => { + const socket = new FakeSocket(); + const req = { socket } as unknown as http.IncomingMessage; + const result = await withHangUpDetection(req, asRes(new FakeRes()), async () => { + socket.emit("end"); + socket.destroyed = true; + socket.emit("close"); + return 1; + }); + expect(result.clientHungUp).toBe(false); + }); + + it("withHangUpDetection ignores a close after the response ended", async () => { + const socket = new FakeSocket(); + const req = { socket } as unknown as http.IncomingMessage; + const res = new FakeRes(); + res.writableEnded = true; + const result = await withHangUpDetection(req, asRes(res), async () => { + socket.errored = new Error("x"); + socket.emit("close"); + return 1; + }); + expect(result.clientHungUp).toBe(false); + }); + + it("withHangUpDetection reports a client already gone on entry", async () => { + for (const gone of [ + (s: FakeSocket) => (s.destroyed = true), + (_s: FakeSocket, r: FakeRes) => (r.destroyed = true), + ]) { + // The client's socket closed (Node ended our side on its FIN) or reset. + for (const evidence of [ + (s: FakeSocket) => (s.writableEnded = true), + (s: FakeSocket) => (s.errored = new Error("ECONNRESET")), + ]) { + const socket = new FakeSocket(); + const res = new FakeRes(); + gone(socket, res); + evidence(socket); + const req = { socket } as unknown as http.IncomingMessage; + const result = await withHangUpDetection(req, asRes(res), async () => 1); + expect(result.clientHungUp).toBe(true); + } + } + }); + + it("withHangUpDetection does not blame the client for a response aimock destroyed before entry", async () => { + for (const gone of [ + (s: FakeSocket) => (s.destroyed = true), + (_s: FakeSocket, r: FakeRes) => (r.destroyed = true), + ]) { + const socket = new FakeSocket(); + const res = new FakeRes(); + gone(socket, res); + const req = { socket } as unknown as http.IncomingMessage; + const result = await withHangUpDetection(req, asRes(res), async () => 1); + expect(result.clientHungUp).toBe(false); + } + }); + + it("withHangUpDetection removes its listener when run throws", async () => { + const socket = new FakeSocket(); + const req = { socket } as unknown as http.IncomingMessage; + await expect( + withHangUpDetection(req, asRes(new FakeRes()), async () => { + throw new Error("boom"); + }), + ).rejects.toThrow("boom"); + expect(socket.listenerCount("close")).toBe(0); + }); + + it("proxyInterruption names the side that tore down a mid-stream response", () => { + const res = new FakeRes(); + expect(proxyInterruption(asRes(res), true)).toEqual({}); + res.destroyed = true; + // No status line sent: status 0. + expect(proxyInterruption(asRes(res), true)).toEqual({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + res.headersSent = true; + expect(proxyInterruption(asRes(res), true)).toEqual({ + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + expect(proxyInterruption(asRes(res), false)).toEqual({ + interrupted: true, + interruptReason: "proxy stream destroyed", + }); + // `end()` called on a response already torn down never finished it. + res.writableEnded = true; + expect(proxyInterruption(asRes(res), true)).toEqual({ + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + res.writableFinished = true; + expect(proxyInterruption(asRes(res), true)).toEqual({}); + }); +}); + +describe("delivery edges: answered elsewhere and fixture interruptions", () => { + const timedFixture = fixture({ + nonStreamingTimingAllProviders: true, + streamingProfile: { ttft: 5000 }, + }); + + it("deliverJson destroys a response whose head was sent but whose body never ended", async () => { + const res = new FakeRes(); + res.writeHead(202); + res.statusCode = 202; // A real writeHead sets it; the fake does not. + res.ops = []; + const e = entry(); + const body = vi.fn(() => ({})); + await deliverJson(asRes(res), e, fixture(), body, jsonOpts); + expect(body).not.toHaveBeenCalled(); + expect(res.ops).toEqual([{ op: "destroy" }]); + expect(e.response).toMatchObject({ + status: 202, + interrupted: true, + interruptReason: "disconnect", + }); + }); + + it("deliverJson does not destroy a response that already ended", async () => { + const res = new FakeRes(); + res.writeHead(200); + res.end("x"); + res.ops = []; + const e = entry(); + await deliverJson(asRes(res), e, fixture(), () => ({}), jsonOpts); + expect(res.ops).toEqual([]); + expect(e.response).toMatchObject({ interrupted: true, interruptReason: "disconnect" }); + }); + + it("with timing on, a response whose head was already sent is journaled at once, with no wait", async () => { + const res = new FakeRes(); + res.writeHead(201); + res.statusCode = 201; // A real writeHead sets it; the fake does not. + res.ops = []; + const e = entry(); + const body = vi.fn(() => ({})); + let settled = false; + const promise = deliverJson(asRes(res), e, timedFixture, body, jsonOpts).then(() => { + settled = true; + }); + await vi.advanceTimersByTimeAsync(0); + expect(settled).toBe(true); + expect(vi.getTimerCount()).toBe(0); + await promise; + expect(body).not.toHaveBeenCalled(); + expect(e.response).toMatchObject({ + status: 201, + interrupted: true, + interruptReason: "disconnect", + }); + }); + + it("with timing on, a response that already ended is journaled at once, with no wait", async () => { + const res = new FakeRes(); + res.writeHead(200); + res.end("x"); + res.ops = []; + const e = entry(); + let settled = false; + const promise = deliverJson(asRes(res), e, timedFixture, () => ({}), jsonOpts).then(() => { + settled = true; + }); + await vi.advanceTimersByTimeAsync(0); + expect(settled).toBe(true); + expect(vi.getTimerCount()).toBe(0); + await promise; + expect(res.ops).toEqual([]); + expect(e.response).toMatchObject({ interrupted: true, interruptReason: "disconnect" }); + }); + + it("a fixture disconnectAfterMs before the first byte keeps the journaled status (base: 200)", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream(asRes(res), e, fixture({ disconnectAfterMs: 10 }), codec, ["a"], { + latency: 50, + }); + await vi.runAllTimersAsync(); + await promise; + expect(res.ops.some((o) => o.op === "write")).toBe(false); + expect(e.response).toMatchObject({ + status: 200, + interrupted: true, + interruptReason: "disconnectAfterMs", + }); + }); + + it("a client that leaves before the first byte still journals status 0", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream( + asRes(res), + e, + fixture({ disconnectAfterMs: 100 }), + codec, + ["a"], + { + latency: 50, + }, + ); + await vi.advanceTimersByTimeAsync(10); + res.destroyed = true; + res.emit("close"); + await vi.runAllTimersAsync(); + await promise; + expect(e.response).toMatchObject({ status: 0, interruptReason: CLIENT_ABORTED }); + }); + + it("deliverStream journals a response that other code ended before the stream", async () => { + const res = new FakeRes(); + res.writeHead(200); + res.end("x"); + res.ops = []; + const e = entry(); + await deliverStream(asRes(res), e, fixture(), codec, ["a"], {}); + expect(res.ops).toEqual([]); + expect(e.response).toMatchObject({ interrupted: true, interruptReason: "disconnect" }); + }); + + it("deliverStream journals a response that other code ended during the stream", async () => { + const res = new FakeRes(); + const e = entry(); + const promise = deliverStream(asRes(res), e, fixture(), codec, ["a", "b"], { latency: 50 }); + await vi.advanceTimersByTimeAsync(50); + res.end(); + await vi.runAllTimersAsync(); + await promise; + expect(res.ops.filter((o) => o.op === "write")).toHaveLength(1); + expect(res.ops.some((o) => o.op === "destroy")).toBe(false); + expect(e.response).toMatchObject({ + status: 200, + interrupted: true, + interruptReason: "disconnect", + }); + }); + + it("deliverStream on a response whose head was already sent does not throw: it journals and destroys", async () => { + // A real ServerResponse throws ERR_HTTP_HEADERS_SENT from setHeader/writeHead. + class HeadSentRes extends FakeRes { + override setHeader(): void { + throw Object.assign(new Error("headers sent"), { code: "ERR_HTTP_HEADERS_SENT" }); + } + override writeHead(): this { + throw Object.assign(new Error("headers sent"), { code: "ERR_HTTP_HEADERS_SENT" }); + } + } + const res = new HeadSentRes(); + res.headersSent = true; + res.statusCode = 203; + const e = entry(); + await expect( + deliverStream(asRes(res), e, fixture(), codec, ["a"], {}), + ).resolves.toBeUndefined(); + expect(res.ops).toEqual([{ op: "destroy" }]); + expect(e.response).toMatchObject({ + status: 203, + interrupted: true, + interruptReason: "disconnect", + }); + }); +}); diff --git a/src/__tests__/response-frame-loop.test.ts b/src/__tests__/response-frame-loop.test.ts new file mode 100644 index 00000000..f87e2c68 --- /dev/null +++ b/src/__tests__/response-frame-loop.test.ts @@ -0,0 +1,618 @@ +import { EventEmitter } from "node:events"; +import type * as http from "node:http"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + httpSink, + runFrameLoop, + wsSink, + type FrameCodec, + type FrameSink, +} from "../response/frame-loop.js"; +import { httpLifecycle } from "../response/lifecycle.js"; +import { sseCodec, typedSseCodec } from "../sse-writer.js"; +import { ndjsonCodec } from "../ndjson-writer.js"; +import { eventStreamCodec, encodeEventStreamMessage } from "../aws-event-stream.js"; +import type { WebSocketConnection } from "../ws-framing.js"; +import type { Fixture } from "../types.js"; + +type Op = Record; + +class FakeSink implements FrameSink { + ops: Op[] = []; + ended = false; + isGone = false; + gone(): boolean { + return this.isGone; + } + writableEnded(): boolean { + return this.ended; + } + write(chunk: string | Buffer): void { + this.ops.push({ op: "write", chunk: Buffer.isBuffer(chunk) ? chunk.toString("hex") : chunk }); + } + end(): void { + this.ops.push({ op: "end" }); + this.ended = true; + } + setHeader(name: string, value: string): void { + this.ops.push({ op: "setHeader", name, value }); + } + writeHead(status: number, headers: Record): void { + this.ops.push({ op: "writeHead", status, headers }); + } +} + +const textCodec: FrameCodec = { + headers: { "Content-Type": "text/plain" }, + encode: (frame) => `${frame}|`, +}; + +let timers: number[]; +beforeEach(() => { + vi.useFakeTimers(); + timers = []; + const fake = globalThis.setTimeout; + vi.spyOn(globalThis, "setTimeout").mockImplementation((( + fn: () => void, + ms?: number, + ...args: unknown[] + ) => { + timers.push(ms ?? 0); + return fake(fn, ms, ...args); + }) as unknown as typeof setTimeout); +}); +afterEach(() => { + vi.restoreAllMocks(); + vi.useRealTimers(); +}); + +/** + * An array whose iteration is observable: `pulled` lists the frames the loop + * took, and `exhausted()` tells whether it asked for one past the last. + */ +function trackedFrames(items: string[]): { + frames: string[]; + pulled: string[]; + exhausted: () => boolean; +} { + const pulled: string[] = []; + let done = false; + const frames = [...items]; + Object.defineProperty(frames, Symbol.iterator, { + value: function* () { + for (const item of items) { + pulled.push(item); + yield item; + } + done = true; + }, + }); + return { frames, pulled, exhausted: () => done }; +} + +async function run(promise: Promise): Promise { + await vi.runAllTimersAsync(); + return promise; +} + +describe("runFrameLoop ordering", () => { + it("sets headers, writes the prelude, then delays → writes each frame, then the trailer and end", async () => { + const sink = new FakeSink(); + const codec: FrameCodec = { + ...textCodec, + prelude: () => "P|", + trailer: (aborted) => [`T${aborted ? "!" : ""}|`], + }; + const ticks: number[] = []; + const done = await run( + runFrameLoop(sink, ["a", "b"], codec, { latency: 5 }, { onChunkSent: () => ticks.push(1) }), + ); + expect(done).toBe(true); + expect(sink.ops).toEqual([ + { op: "setHeader", name: "Content-Type", value: "text/plain" }, + { op: "write", chunk: "P|" }, + { op: "write", chunk: "a|" }, + { op: "write", chunk: "b|" }, + { op: "write", chunk: "T|" }, + { op: "end" }, + ]); + expect(timers).toEqual([5, 5]); + expect(ticks).toHaveLength(2); + }); + + it("returns true without writing when the sink already ended", async () => { + const sink = new FakeSink(); + sink.ended = true; + expect(await run(runFrameLoop(sink, ["a"], textCodec, {}))).toBe(true); + expect(sink.ops).toEqual([]); + }); + + it("returns true at the writableEnded check after a delay, without the trailer", async () => { + const sink = new FakeSink(); + const codec: FrameCodec = { ...textCodec, trailer: () => ["T|"] }; + const promise = runFrameLoop(sink, ["a", "b"], codec, { latency: 5 }); + await vi.advanceTimersByTimeAsync(5); + sink.ended = true; + expect(await run(promise)).toBe(true); + expect(sink.ops.filter((o) => o.op === "write")).toEqual([{ op: "write", chunk: "a|" }]); + }); + + it("does not advance the index or tick for frames that do not count", async () => { + const sink = new FakeSink(); + const codec: FrameCodec = { ...textCodec, counts: (f) => f.startsWith("d") }; + const ticks: string[] = []; + await run( + runFrameLoop( + sink, + ["start", "d1", "mid", "d2", "end"], + codec, + { recordedTimings: { ttftMs: 3, interChunkDelaysMs: [7, 11], totalDurationMs: 0 } }, + { onChunkSent: () => ticks.push("t") }, + ), + ); + // start@0, d1@0, mid@1, d2@1, end@2: framing frames pay the current index. + expect(timers).toEqual([3, 3, 7, 7, 11]); + expect(ticks).toHaveLength(2); + }); + + it("stops on a signal abort with no write after it, returning false", async () => { + const sink = new FakeSink(); + const controller = new AbortController(); + const codec: FrameCodec = { ...textCodec, trailer: () => ["T|"] }; + const promise = runFrameLoop( + sink, + ["a", "b", "c"], + codec, + { latency: 10 }, + { + signal: controller.signal, + }, + ); + await vi.advanceTimersByTimeAsync(10); + controller.abort(); + expect(await run(promise)).toBe(false); + expect(sink.ops.filter((o) => o.op === "write")).toEqual([{ op: "write", chunk: "a|" }]); + expect(sink.ops.some((o) => o.op === "end")).toBe(false); + expect(vi.getTimerCount()).toBe(0); + }); + + it("re-checks the abort after the tick, so a truncating tick stops before the next frame", async () => { + const sink = new FakeSink(); + const controller = new AbortController(); + const frames = trackedFrames(["a", "b"]); + const done = await run( + runFrameLoop( + sink, + frames.frames, + textCodec, + {}, + { + signal: controller.signal, + onChunkSent: () => controller.abort(), + }, + ), + ); + expect(done).toBe(false); + expect(sink.ops.filter((o) => o.op === "write")).toHaveLength(1); + // The loop stopped right after the tick: it never took the next frame. + expect(frames.pulled).toEqual(["a"]); + }); + + it("re-checks the abort after the last frame's tick, so the stream is not ended", async () => { + // On the last frame there is no next iteration: only the check after the + // tick stops the loop before the end-of-stream path. + const sink = new FakeSink(); + const controller = new AbortController(); + const frames = trackedFrames(["a", "b"]); + const done = await run( + runFrameLoop( + sink, + frames.frames, + textCodec, + {}, + { + signal: controller.signal, + onChunkSent: () => { + if (sink.ops.filter((o) => o.op === "write").length === 2) controller.abort(); + }, + }, + ), + ); + expect(done).toBe(false); + expect(sink.ops.filter((o) => o.op === "write")).toHaveLength(2); + expect(sink.ops.some((o) => o.op === "end")).toBe(false); + // The loop returned from inside the iteration: it never asked for more. + expect(frames.exhausted()).toBe(false); + }); + + it("without a lifecycle, a gone sink does not stop the loop (exported-writer contract)", async () => { + const sink = new FakeSink(); + sink.isGone = true; + expect(await run(runFrameLoop(sink, ["a", "b"], textCodec, {}))).toBe(true); + expect(sink.ops.filter((o) => o.op === "write")).toHaveLength(2); + }); +}); + +describe("runFrameLoop with a lifecycle", () => { + class FakeRes extends EventEmitter { + destroyed = false; + writableEnded = false; + writableFinished = false; + headersSent = false; + writes: string[] = []; + setHeader(): void {} + write(chunk: string): boolean { + this.writes.push(chunk); + return true; + } + end(): void { + this.writableEnded = true; + this.writableFinished = true; + } + } + const fixture: Fixture = { match: {}, response: { content: "x" } }; + + it("returns false at entry when the response is already destroyed", async () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + res.destroyed = true; + const done = await run( + runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a"], + textCodec, + {}, + { + lifecycle, + }, + ), + ); + expect(done).toBe(false); + expect(res.writes).toEqual([]); + lifecycle.dispose(); + }); + + it("stops at entry for a destroyed response, before the headers or the prelude", async () => { + const res = new FakeRes(); + const setHeader = vi.spyOn(res, "setHeader"); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + res.destroyed = true; + await run( + runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a"], + { ...textCodec, prelude: () => "P|" }, + {}, + { lifecycle }, + ), + ); + expect(setHeader).not.toHaveBeenCalled(); + expect(res.writes).toEqual([]); + lifecycle.dispose(); + }); + + it("a close during a delay clears the timer and stops before the next write", async () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + const promise = runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a", "b", "c"], + textCodec, + { latency: 1000 }, + { lifecycle }, + ); + await vi.advanceTimersByTimeAsync(1000); + expect(res.writes).toEqual(["a|"]); + res.destroyed = true; + res.emit("close"); + expect(vi.getTimerCount()).toBe(0); + expect(await run(promise)).toBe(false); + expect(res.writes).toEqual(["a|"]); + lifecycle.dispose(); + }); + + it("returns false (not completed) for a response already ended at entry", async () => { + const res = new FakeRes(); + res.writableEnded = true; + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + const done = await run( + runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a"], + textCodec, + {}, + { lifecycle }, + ), + ); + expect(done).toBe(false); + expect(res.writes).toEqual([]); + lifecycle.dispose(); + }); + + it("returns false (not completed) for a response ended by other code during the stream", async () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + const promise = runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a", "b"], + textCodec, + { latency: 10 }, + { lifecycle }, + ); + await vi.advanceTimersByTimeAsync(10); + res.end(); + expect(await run(promise)).toBe(false); + expect(res.writes).toEqual(["a|"]); + lifecycle.dispose(); + }); + + it("returns false without touching the head when the head was already sent", async () => { + const res = new FakeRes(); + res.headersSent = true; + const setHeader = vi.spyOn(res, "setHeader"); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + const done = await run( + runFrameLoop( + httpSink(res as unknown as http.ServerResponse), + ["a"], + textCodec, + {}, + { lifecycle }, + ), + ); + expect(done).toBe(false); + expect(setHeader).not.toHaveBeenCalled(); + expect(res.writes).toEqual([]); + lifecycle.dispose(); + }); + + it("checks sink.gone() next to the abort checks", async () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(res as unknown as http.ServerResponse, fixture, { + streaming: true, + }); + const sink = httpSink(res as unknown as http.ServerResponse); + const done = await run( + runFrameLoop( + sink, + ["a", "b"], + textCodec, + {}, + { + lifecycle, + onChunkSent: () => { + // Destroyed without a close event (yet): still gone. + res.destroyed = true; + }, + }, + ), + ); + expect(done).toBe(false); + expect(res.writes).toEqual(["a|"]); + lifecycle.dispose(); + }); +}); + +describe("runFrameLoop status/headers (#549 contract)", () => { + it("status given: writeHead(status, merged headers) before the first frame", async () => { + const sink = new FakeSink(); + await run(runFrameLoop(sink, ["a"], textCodec, {}, { status: 207, headers: { "x-a": "1" } })); + expect(sink.ops[0]).toEqual({ + op: "writeHead", + status: 207, + headers: { "Content-Type": "text/plain", "x-a": "1" }, + }); + }); + + it("headers alone are merged over the codec defaults through setHeader", async () => { + const sink = new FakeSink(); + await run( + runFrameLoop(sink, ["a"], textCodec, {}, { headers: { "Content-Type": "x/y", "x-b": "2" } }), + ); + expect(sink.ops.slice(0, 2)).toEqual([ + { op: "setHeader", name: "Content-Type", value: "x/y" }, + { op: "setHeader", name: "x-b", value: "2" }, + ]); + }); + + it("writeHeadImmediately writes the head with an implicit 200", async () => { + const sink = new FakeSink(); + await run(runFrameLoop(sink, ["a"], { ...textCodec, writeHeadImmediately: true }, {})); + expect(sink.ops[0]).toEqual({ + op: "writeHead", + status: 200, + headers: { "Content-Type": "text/plain" }, + }); + }); +}); + +describe("runFrameLoop on a WS sink", () => { + class FakeWs extends EventEmitter { + isClosed = false; + sent: string[] = []; + send(data: string): void { + this.sent.push(data); + } + } + + it("checks gone before the delay, writes no headers, and never ends", async () => { + const ws = new FakeWs(); + const sink = wsSink(ws as unknown as WebSocketConnection); + const codec: FrameCodec = { headers: {}, encode: (f) => f, checkGoneBeforeDelay: true }; + const promise = runFrameLoop(sink, ["a", "b"], codec, { latency: 10 }); + await vi.advanceTimersByTimeAsync(10); + expect(ws.sent).toEqual(["a"]); + ws.isClosed = true; + expect(await run(promise)).toBe(false); + expect(ws.sent).toEqual(["a"]); + // Frame 1's delay started before the close; the gone re-check after it + // stops the loop before the write. + expect(timers).toEqual([10, 10]); + }); + + it("checks gone before the delay: a close between frames starts no further delay", async () => { + const ws = new FakeWs(); + const sink = wsSink(ws as unknown as WebSocketConnection); + const codec: FrameCodec = { headers: {}, encode: (f) => f, checkGoneBeforeDelay: true }; + await run( + runFrameLoop( + sink, + ["a", "b"], + codec, + { latency: 10 }, + { + onChunkSent: () => { + ws.isClosed = true; + }, + }, + ), + ); + expect(ws.sent).toEqual(["a"]); + // Only frame 0's delay ran: the check before frame 1's delay stopped it. + expect(timers).toEqual([10]); + }); + + it("returns false when the socket closes during the delay", async () => { + const ws = new FakeWs(); + const sink = wsSink(ws as unknown as WebSocketConnection); + const codec: FrameCodec = { headers: {}, encode: (f) => f, checkGoneBeforeDelay: true }; + const promise = runFrameLoop(sink, ["a"], codec, { latency: 10 }); + ws.isClosed = true; + expect(await run(promise)).toBe(false); + expect(ws.sent).toEqual([]); + }); +}); + +describe("codecs match the writer goldens' bytes", () => { + const chunk = { id: "c", choices: [{ delta: { content: "hi" } }] }; + + it("sseCodec writes data-only frames, and [DONE] only when done is true", () => { + const withDone = sseCodec({ done: true, usageChunk: { ...chunk, id: "u" } as never }); + expect(withDone.encode(chunk as never)).toBe(`data: ${JSON.stringify(chunk)}\n\n`); + expect(withDone.trailer?.(false)).toEqual([ + `data: ${JSON.stringify({ ...chunk, id: "u" })}\n\n`, + "data: [DONE]\n\n", + ]); + expect(sseCodec({ done: false }).trailer?.(false) ?? []).toEqual([]); + expect(sseCodec({ done: true, openRouterProcessing: true }).prelude?.()).toBe( + ": OPENROUTER PROCESSING\n\n", + ); + expect(sseCodec({ done: true }).prelude?.()).toBeUndefined(); + expect(sseCodec({ done: true }).headers).toEqual({ + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + Connection: "keep-alive", + }); + }); + + it("typedSseCodec writes typed frames and the sentinel only with terminalSentinel", () => { + const event = { type: "content-delta", x: 1 }; + expect(typedSseCodec().encode(event)).toBe( + `event: content-delta\ndata: ${JSON.stringify(event)}\n\n`, + ); + expect(typedSseCodec().trailer?.(false) ?? []).toEqual([]); + expect(typedSseCodec({ terminalSentinel: true }).trailer?.(false)).toEqual([ + "data: [DONE]\n\n", + ]); + }); + + it("ndjsonCodec writes one JSON line per frame", () => { + expect(ndjsonCodec().encode({ a: 1 })).toBe('{"a":1}\n'); + expect(ndjsonCodec().headers).toEqual({ + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + Connection: "keep-alive", + }); + }); + + it("eventStreamCodec encodes with encodeEventStreamMessage, or with the given encoder", () => { + const event = { eventType: "chunk", payload: { a: 1 } }; + expect(eventStreamCodec().encode(event)).toEqual(encodeEventStreamMessage("chunk", { a: 1 })); + const custom = eventStreamCodec((e) => encodeEventStreamMessage(`x-${e.eventType}`, e.payload)); + expect(custom.encode(event)).toEqual(encodeEventStreamMessage("x-chunk", { a: 1 })); + expect(eventStreamCodec().headers).toEqual({ + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked", + }); + }); +}); + +describe("runFrameLoop contract (truthful completion)", () => { + it("an abort before start with no frames returns false and writes no trailer or end", async () => { + const sink = new FakeSink(); + const controller = new AbortController(); + controller.abort(); + const trailerCalls: boolean[] = []; + const codec: FrameCodec = { + ...textCodec, + trailer: (aborted) => { + trailerCalls.push(aborted); + return ["T|"]; + }, + }; + const done = await run(runFrameLoop(sink, [], codec, {}, { signal: controller.signal })); + expect(done).toBe(false); + expect(trailerCalls).toEqual([]); + expect(sink.ops.some((o) => o.op === "write" || o.op === "end")).toBe(false); + }); + + it("a completed delivery calls the trailer with aborted=false", async () => { + const sink = new FakeSink(); + const trailerCalls: boolean[] = []; + const codec: FrameCodec = { + ...textCodec, + trailer: (aborted) => { + trailerCalls.push(aborted); + return []; + }, + }; + expect(await run(runFrameLoop(sink, [], codec, {}))).toBe(true); + expect(trailerCalls).toEqual([false]); + }); + + it("rejects a lifecycle and a signal passed together as a programming error", async () => { + const res = new EventEmitter(); + const lifecycle = httpLifecycle( + res as unknown as http.ServerResponse, + { match: {}, response: { content: "x" } }, + { streaming: true }, + ); + const controller = new AbortController(); + const sink = new FakeSink(); + await expect( + runFrameLoop( + sink, + ["a"], + textCodec, + {}, + // @ts-expect-error -- a lifecycle and a signal are mutually exclusive + { lifecycle, signal: controller.signal }, + ), + ).rejects.toThrow(TypeError); + expect(sink.ops).toEqual([]); + lifecycle.dispose(); + }); + + it("rejects a status on a sink without writeHead instead of dropping it", async () => { + const ws = Object.assign(new EventEmitter(), { isClosed: false, send: () => {} }); + const sink = wsSink(ws as unknown as WebSocketConnection); + await expect(runFrameLoop(sink, ["a"], textCodec, {}, { status: 201 })).rejects.toThrow( + TypeError, + ); + await expect( + runFrameLoop(sink, ["a"], { ...textCodec, writeHeadImmediately: true }, {}), + ).rejects.toThrow(TypeError); + }); +}); diff --git a/src/__tests__/response-lifecycle.test.ts b/src/__tests__/response-lifecycle.test.ts new file mode 100644 index 00000000..afff3752 --- /dev/null +++ b/src/__tests__/response-lifecycle.test.ts @@ -0,0 +1,363 @@ +import { EventEmitter } from "node:events"; +import type * as http from "node:http"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + CLIENT_ABORTED, + httpLifecycle, + markInterrupted, + wsLifecycle, +} from "../response/lifecycle.js"; +import { applyChaosAction } from "../chaos.js"; +import { Journal } from "../journal.js"; +import type { Fixture, JournalEntry } from "../types.js"; +import type { WebSocketConnection } from "../ws-framing.js"; + +class FakeRes extends EventEmitter { + writableFinished = false; + writableEnded = false; + destroyed = false; + headersSent = false; +} + +class FakeWs extends EventEmitter { + isClosed = false; + destroy(): void { + if (this.isClosed) return; + this.isClosed = true; + this.emit("close", 1006, "Connection destroyed"); + } +} + +const asRes = (res: FakeRes): http.ServerResponse => res as unknown as http.ServerResponse; +const asWs = (ws: FakeWs): WebSocketConnection => ws as unknown as WebSocketConnection; + +/** + * The request socket state the lifecycle reads. `readableEnded` is the bare + * FIN, which is not a hang-up; Node ending our write side on it + * (`writableEnded`) or a reset (`errored`) is. + */ +interface FakeSocketState { + readableEnded: boolean; + writableEnded: boolean; + errored: Error | null; +} + +function withSocket(res: FakeRes, socket: FakeSocketState): FakeRes { + Object.assign(res, { req: { socket } }); + return res; +} + +const plain: Fixture = { match: {}, response: { content: "hi" } }; +const truncate2: Fixture = { ...plain, truncateAfterChunks: 2 }; +const disconnect50: Fixture = { ...plain, disconnectAfterMs: 50 }; + +function entry(): JournalEntry { + return { + id: "e1", + timestamp: 0, + method: "POST", + path: "/x", + headers: {}, + body: null, + response: { status: 200, fixture: null }, + }; +} + +beforeEach(() => { + vi.useFakeTimers(); +}); +afterEach(() => { + vi.useRealTimers(); +}); + +describe("httpLifecycle", () => { + it("aborts with cause client when the response closes before it finished", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(false); + res.emit("close"); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("client"); + expect(lifecycle.reason()).toBe(CLIENT_ABORTED); + lifecycle.dispose(); + }); + + it("ignores a close after the response finished", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + res.writableFinished = true; + res.emit("close"); + expect(lifecycle.signal.aborted).toBe(false); + expect(lifecycle.cause()).toBeUndefined(); + expect(lifecycle.reason()).toBeUndefined(); + lifecycle.dispose(); + }); + + it("keeps the fixture reason when the fixture abort is followed by the resulting close", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), truncate2, { streaming: true }); + lifecycle.tick(); + lifecycle.tick(); + expect(lifecycle.signal.aborted).toBe(true); + res.destroyed = true; + res.emit("close"); + expect(lifecycle.cause()).toBe("fixture"); + expect(lifecycle.reason()).toBe("truncateAfterChunks"); + lifecycle.dispose(); + }); + + it("keeps the client cause when the client closes first and the fixture aborts after", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), truncate2, { streaming: true }); + res.emit("close"); + expect(lifecycle.cause()).toBe("client"); + lifecycle.tick(); + lifecycle.tick(); + expect(lifecycle.cause()).toBe("client"); + expect(lifecycle.reason()).toBe(CLIENT_ABORTED); + lifecycle.dispose(); + }); + + it("aborts at once when the client is already gone at construction", () => { + const res = new FakeRes(); + res.destroyed = true; + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("client"); + lifecycle.dispose(); + }); + + it("never classifies an already-ended response as a client abort", () => { + const res = new FakeRes(); + res.writableEnded = true; + res.destroyed = true; + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(false); + expect(lifecycle.cause()).toBeUndefined(); + lifecycle.dispose(); + }); + + it("never classifies a chaos-disconnected response as a client abort", () => { + const res = new FakeRes(); + const destroy = vi.fn(() => { + res.destroyed = true; + }); + Object.assign(res, { destroy }); + applyChaosAction( + "disconnect", + asRes(res), + null, + new Journal(), + { method: "POST", path: "/x", headers: {}, body: null }, + "fixture", + undefined, + undefined, + ); + expect(destroy).toHaveBeenCalledOnce(); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("server"); + expect(lifecycle.reason()).toBe("disconnect"); + lifecycle.dispose(); + }); + + it("attributes a close with no client FIN or RST on the socket to aimock", () => { + const res = withSocket(new FakeRes(), { + readableEnded: false, + writableEnded: false, + errored: null, + }); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + res.destroyed = true; + res.emit("close"); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("server"); + expect(lifecycle.reason()).toBe("disconnect"); + lifecycle.dispose(); + }); + + it("attributes a close after the client FIN, which Node answers by ending our side, to the client", () => { + const socket: FakeSocketState = { readableEnded: false, writableEnded: false, errored: null }; + const res = withSocket(new FakeRes(), socket); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + socket.readableEnded = true; + socket.writableEnded = true; + res.destroyed = true; + res.emit("close"); + expect(lifecycle.cause()).toBe("client"); + expect(lifecycle.reason()).toBe(CLIENT_ABORTED); + lifecycle.dispose(); + }); + + it("does not treat a FIN alone as a client hang-up", () => { + const socket: FakeSocketState = { readableEnded: true, writableEnded: false, errored: null }; + const res = withSocket(new FakeRes(), socket); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + res.destroyed = true; + res.emit("close"); + expect(lifecycle.cause()).toBe("server"); + expect(lifecycle.reason()).toBe("disconnect"); + lifecycle.dispose(); + }); + + it("attributes a close after a client RST (socket error) to the client", () => { + const res = withSocket(new FakeRes(), { + readableEnded: false, + writableEnded: false, + errored: new Error("read ECONNRESET"), + }); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + res.destroyed = true; + res.emit("close"); + expect(lifecycle.cause()).toBe("client"); + expect(lifecycle.reason()).toBe(CLIENT_ABORTED); + lifecycle.dispose(); + }); + + it("aborts with cause server when aimock already destroyed the response at construction", () => { + const res = withSocket(new FakeRes(), { + readableEnded: false, + writableEnded: false, + errored: null, + }); + res.destroyed = true; + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("server"); + expect(lifecycle.reason()).toBe("disconnect"); + lifecycle.dispose(); + }); + + it("creates no fixture interruption when not streaming", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), truncate2, { streaming: false }); + lifecycle.tick(); + lifecycle.tick(); + lifecycle.tick(); + expect(lifecycle.signal.aborted).toBe(false); + expect(vi.getTimerCount()).toBe(0); + const timed = httpLifecycle(asRes(res), disconnect50, { streaming: false }); + expect(vi.getTimerCount()).toBe(0); + lifecycle.dispose(); + timed.dispose(); + }); + + it("aborts with the fixture reason when disconnectAfterMs fires", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), disconnect50, { streaming: true }); + vi.advanceTimersByTime(50); + expect(lifecycle.cause()).toBe("fixture"); + expect(lifecycle.reason()).toBe("disconnectAfterMs"); + lifecycle.dispose(); + }); + + it("dispose removes the close listener and clears the disconnectAfterMs timer", () => { + const res = new FakeRes(); + const lifecycle = httpLifecycle(asRes(res), disconnect50, { streaming: true }); + expect(res.listenerCount("close")).toBe(1); + expect(vi.getTimerCount()).toBe(1); + lifecycle.dispose(); + expect(res.listenerCount("close")).toBe(0); + expect(vi.getTimerCount()).toBe(0); + res.emit("close"); + expect(lifecycle.signal.aborted).toBe(false); + }); + + it("does not throw for a partial double without once/off", () => { + const partial = { writableFinished: false } as unknown as http.ServerResponse; + const lifecycle = httpLifecycle(partial, null, { streaming: true }); + expect(lifecycle.signal.aborted).toBe(false); + expect(() => lifecycle.tick()).not.toThrow(); + expect(() => lifecycle.dispose()).not.toThrow(); + }); + + it("removes through removeListener when off is absent", () => { + const res = new FakeRes(); + const remover = vi.spyOn(res, "removeListener"); + Object.defineProperty(res, "off", { value: undefined }); + const lifecycle = httpLifecycle(asRes(res), plain, { streaming: true }); + lifecycle.dispose(); + expect(remover).toHaveBeenCalledWith("close", expect.any(Function)); + expect(res.listenerCount("close")).toBe(0); + }); +}); + +describe("wsLifecycle", () => { + it("aborts with cause client on a WS close", () => { + const ws = new FakeWs(); + const lifecycle = wsLifecycle(asWs(ws), plain); + ws.emit("close", 1006, "gone"); + expect(lifecycle.cause()).toBe("client"); + expect(lifecycle.reason()).toBe(CLIENT_ABORTED); + lifecycle.dispose(); + }); + + it("aborts at once when the socket is already closed", () => { + const ws = new FakeWs(); + ws.isClosed = true; + const lifecycle = wsLifecycle(asWs(ws), plain); + expect(lifecycle.signal.aborted).toBe(true); + expect(lifecycle.cause()).toBe("client"); + lifecycle.dispose(); + }); + + it("keeps the fixture reason when truncateAfterChunks fires and ws.destroy() closes", () => { + const ws = new FakeWs(); + const lifecycle = wsLifecycle(asWs(ws), truncate2); + lifecycle.tick(); + lifecycle.tick(); + ws.destroy(); + expect(lifecycle.cause()).toBe("fixture"); + expect(lifecycle.reason()).toBe("truncateAfterChunks"); + lifecycle.dispose(); + }); + + it("keeps the fixture reason when disconnectAfterMs fires and ws.destroy() closes", () => { + const ws = new FakeWs(); + const lifecycle = wsLifecycle(asWs(ws), disconnect50); + vi.advanceTimersByTime(50); + ws.destroy(); + expect(lifecycle.cause()).toBe("fixture"); + expect(lifecycle.reason()).toBe("disconnectAfterMs"); + lifecycle.dispose(); + }); + + it("always owns the fixture signal, and dispose clears its timer and listener", () => { + const ws = new FakeWs(); + const lifecycle = wsLifecycle(asWs(ws), disconnect50); + expect(vi.getTimerCount()).toBe(1); + expect(ws.listenerCount("close")).toBe(1); + lifecycle.dispose(); + expect(vi.getTimerCount()).toBe(0); + expect(ws.listenerCount("close")).toBe(0); + }); +}); + +describe("markInterrupted", () => { + it("sets interrupted and the reason, and never touches status", () => { + const e = entry(); + markInterrupted(e, CLIENT_ABORTED); + expect(e.response).toEqual({ + status: 200, + fixture: null, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + }); + + it("never overwrites an existing reason, and never erases it with undefined", () => { + const e = entry(); + markInterrupted(e, "truncateAfterChunks"); + markInterrupted(e, CLIENT_ABORTED); + markInterrupted(e, undefined); + expect(e.response.interruptReason).toBe("truncateAfterChunks"); + expect(e.response.status).toBe(200); + }); + + it("marks interrupted without a reason when none is known", () => { + const e = entry(); + markInterrupted(e, undefined); + expect(e.response.interrupted).toBe(true); + expect("interruptReason" in e.response).toBe(false); + }); +}); diff --git a/src/__tests__/response-timing.test.ts b/src/__tests__/response-timing.test.ts new file mode 100644 index 00000000..4c7168fc --- /dev/null +++ b/src/__tests__/response-timing.test.ts @@ -0,0 +1,462 @@ +import { describe, expect, it } from "vitest"; +import { + extendRecordedTimings, + misbehaviorPayloadLength, + nonStreamingDelayMs, + nonStreamingTimingEnabled, + responsesOutputFrames, + responsesPayloadLength, + servedPayloadLength, +} from "../response/timing.js"; +import { calculateTotalDelay } from "../sse-writer.js"; +import { LLMock } from "../llmock.js"; +import type { Fixture, FixtureResponse, RecordedTimings } from "../types.js"; + +const profile = { ttft: 300, tps: 10 }; +const recorded: RecordedTimings = { + ttftMs: 100, + interChunkDelaysMs: [10, 20, 30], + totalDurationMs: 160, +}; + +function fixture(extra: Partial): Fixture { + return { match: {}, response: { content: "hi" }, ...extra }; +} + +describe("nonStreamingTimingEnabled", () => { + const surfaces = ["openai-chat", "other"] as const; + + it("is off on both surfaces when neither field is set or both are false", () => { + for (const surface of surfaces) { + expect(nonStreamingTimingEnabled(fixture({ streamingProfile: profile }), surface)).toBe( + false, + ); + expect( + nonStreamingTimingEnabled( + fixture({ + streamingProfile: profile, + nonStreamingTiming: false, + nonStreamingTimingAllProviders: false, + }), + surface, + ), + ).toBe(false); + } + }); + + it("nonStreamingTiming: true is on for openai-chat only", () => { + const f = fixture({ streamingProfile: profile, nonStreamingTiming: true }); + expect(nonStreamingTimingEnabled(f, "openai-chat")).toBe(true); + expect(nonStreamingTimingEnabled(f, "other")).toBe(false); + }); + + it("nonStreamingTimingAllProviders: true is on for both, with or without nonStreamingTiming", () => { + for (const nonStreamingTiming of [undefined, false, true]) { + const f = fixture({ + recordedTimings: recorded, + nonStreamingTimingAllProviders: true, + nonStreamingTiming, + }); + expect(nonStreamingTimingEnabled(f, "openai-chat")).toBe(true); + expect(nonStreamingTimingEnabled(f, "other")).toBe(true); + } + }); + + it("nonStreamingTimingAllProviders: false never turns off nonStreamingTiming: true on OpenAI", () => { + const f = fixture({ + streamingProfile: profile, + nonStreamingTiming: true, + nonStreamingTimingAllProviders: false, + }); + expect(nonStreamingTimingEnabled(f, "openai-chat")).toBe(true); + expect(nonStreamingTimingEnabled(f, "other")).toBe(false); + }); + + it("treats non-boolean values of either field as off", () => { + const f = { + ...fixture({ streamingProfile: profile }), + nonStreamingTiming: "yes", + nonStreamingTimingAllProviders: "yes", + } as unknown as Fixture; + expect(nonStreamingTimingEnabled(f, "openai-chat")).toBe(false); + expect(nonStreamingTimingEnabled(f, "other")).toBe(false); + }); + + it("is off without a streamingProfile or recordedTimings, even when opted in", () => { + const f = fixture({ + latency: 500, + nonStreamingTiming: true, + nonStreamingTimingAllProviders: true, + }); + expect(nonStreamingTimingEnabled(f, "openai-chat")).toBe(false); + expect(nonStreamingTimingEnabled(f, "other")).toBe(false); + }); +}); + +describe("nonStreamingDelayMs", () => { + it("wraps calculateTotalDelay with the fixture profile and the given timings", () => { + const f = fixture({ streamingProfile: profile }); + expect(nonStreamingDelayMs(f, 25, 10, undefined, 1)).toBe( + calculateTotalDelay(25, 10, profile, undefined, 1), + ); + expect(nonStreamingDelayMs(fixture({}), 25, 10, recorded, 2)).toBe( + calculateTotalDelay(25, 10, undefined, recorded, 2), + ); + }); + + it("gives 0 for bare latency", () => { + expect(nonStreamingDelayMs(fixture({ latency: 2000 }), 100, 5, undefined, 1)).toBe(0); + }); +}); + +// The OpenAI chat formulas this must equal (src/server.ts at the base SHA): +// text (effReasoning?.length ?? 0) + response.content.length +// tool (effReasoning?.length ?? 0) + Σ(tc.name + tc.arguments) +// combined (effReasoning?.length ?? 0) + content + Σ(tc.name + tc.arguments), +// blocks resolved through resolveFixtureBlockOutcome +describe("servedPayloadLength", () => { + it("text: reasoning + content", () => { + expect(servedPayloadLength({ content: "Hello world" })).toBe(11); + expect(servedPayloadLength({ content: "Hello world" }, "Think.")).toBe(17); + }); + + it("tool calls: reasoning + each name + arguments", () => { + const r: FixtureResponse = { + toolCalls: [ + { name: "get_weather", arguments: '{"city":"NYC"}' }, + { name: "t", arguments: "{}" }, + ], + }; + expect(servedPayloadLength(r)).toBe(11 + 14 + 1 + 2); + expect(servedPayloadLength(r, "abc")).toBe(3 + 11 + 14 + 1 + 2); + }); + + it("combined: content + each tool call", () => { + const r: FixtureResponse = { + content: "Checking.", + toolCalls: [{ name: "get_weather", arguments: '{"city":"SF"}' }], + }; + expect(servedPayloadLength(r, "r")).toBe(1 + 9 + 11 + 13); + }); + + it("blocks: resolved text plus each tool block, ignoring the legacy fields", () => { + const r: FixtureResponse = { + blocks: [ + { type: "text", text: "Lead." }, + { type: "toolCall", name: "get_time", arguments: "{}" }, + { type: "text", text: "Tail." }, + ], + }; + expect(servedPayloadLength(r)).toBe(5 + 5 + 8 + 2); + expect(servedPayloadLength(r, "abc")).toBe(3 + 5 + 5 + 8 + 2); + }); + + it("blocks win over the legacy content and toolCalls fields", () => { + const r: FixtureResponse = { + content: "legacy content", + toolCalls: [{ name: "legacy_tool", arguments: '{"a":1}' }], + blocks: [{ type: "text", text: "Block." }], + }; + expect(servedPayloadLength(r, "rr")).toBe(2 + 6); + }); + + it("gives just the reasoning length for a response with no served text", () => { + expect(servedPayloadLength({ error: { message: "x", type: "y" } }, "ab")).toBe(2); + }); +}); + +describe("responsesPayloadLength", () => { + it("counts customToolCalls after toolCalls: each name + input", () => { + const r: FixtureResponse = { + toolCalls: [{ name: "f", arguments: "{}" }], + customToolCalls: [{ type: "custom", name: "apply_patch", input: "patch body" }], + }; + expect(responsesPayloadLength(r, "ab")).toBe(2 + 1 + 2 + 11 + 10); + // OpenAI chat's length leaves the custom call out. + expect(servedPayloadLength(r, "ab")).toBe(2 + 1 + 2); + }); + + it("resolves responsesBlocks, custom calls included", () => { + const r: FixtureResponse = { + responsesBlocks: [ + { type: "text", text: "Lead." }, + { type: "customToolCall", name: "run", input: "ls -la" }, + ], + }; + expect(responsesPayloadLength(r)).toBe(5 + 3 + 6); + }); + + it("takes responsesBlocks over blocks, as the Responses wire serves", () => { + const r: FixtureResponse = { + blocks: [{ type: "text", text: "x" }], + responsesBlocks: [{ type: "text", text: "Served text." }], + }; + expect(responsesPayloadLength(r)).toBe(12); + expect(servedPayloadLength(r)).toBe(1); + }); + + it("equals servedPayloadLength for a fixture without the Responses keys", () => { + const rows: FixtureResponse[] = [ + { content: "Hello world" }, + { toolCalls: [{ name: "t", arguments: "{}" }] }, + { content: "c", toolCalls: [{ name: "t", arguments: "{}" }] }, + { blocks: [{ type: "toolCall", name: "get_time", arguments: "{}" }] }, + ]; + for (const r of rows) expect(responsesPayloadLength(r, "r")).toBe(servedPayloadLength(r, "r")); + }); + + it("is the length misbehaviorPayloadLength uses when passed", () => { + const plan = { + response: { + toolCalls: [], + customToolCalls: [{ type: "custom" as const, name: "run", input: "ls" }], + } as FixtureResponse, + }; + expect(misbehaviorPayloadLength(plan, undefined, responsesPayloadLength)).toBe(5); + expect(misbehaviorPayloadLength(plan, undefined)).toBe(0); + }); +}); + +describe("responsesOutputFrames", () => { + const png = { mimeType: "image/png", data: "iVBORw0KGgo=" }; + const url = { url: "https://example.com/a" }; + + it("counts one frame per citation and five per generated file on a text turn", () => { + const r: FixtureResponse = { content: "Cited.", citations: [url, url], files: [png] }; + expect(responsesOutputFrames(r)).toBe(2 + 5); + }); + + it("counts them on a combined turn", () => { + const r: FixtureResponse = { + content: "Cited.", + toolCalls: [{ name: "t", arguments: "{}" }], + citations: [url], + files: [png, png], + }; + expect(responsesOutputFrames(r)).toBe(1 + 10); + }); + + it("counts files but no citations on a tool-only turn", () => { + const r: FixtureResponse = { toolCalls: [{ name: "t", arguments: "{}" }], files: [png] }; + expect(responsesOutputFrames(r)).toBe(5); + }); + + it("is 0 without citations or files, and never throws on a malformed value", () => { + expect(responsesOutputFrames({ content: "plain" })).toBe(0); + const malformed = { content: "x", citations: "nope", files: 7 } as unknown as FixtureResponse; + expect(responsesOutputFrames(malformed)).toBe(0); + expect(responsesOutputFrames({ error: { message: "x", type: "y" } })).toBe(0); + }); +}); + +describe("misbehaviorPayloadLength", () => { + const text: FixtureResponse = { content: "The full answer." }; + const tools: FixtureResponse = { toolCalls: [{ name: "f", arguments: "{}" }] }; + + it("matches OpenAI's selection for the three stop plans on a text response", () => { + expect( + misbehaviorPayloadLength({ response: text, stop: "refusal", refusal: "No." }, "rr"), + ).toBe(3); + expect(misbehaviorPayloadLength({ response: text, stop: "refusal" }, "rr")).toBe(0); + expect(misbehaviorPayloadLength({ response: text, stop: "content_filter" }, "rr")).toBe(0); + expect(misbehaviorPayloadLength({ response: text, reasoning: "thinking..." }, "rr")).toBe(11); + }); + + it("refusal wins over reasoning, as in OpenAI's selection order", () => { + expect( + misbehaviorPayloadLength( + { response: text, stop: "refusal", refusal: "No", reasoning: "long reasoning" }, + undefined, + ), + ).toBe(2); + }); + + it("gives the served length when the response is not a text response", () => { + expect( + misbehaviorPayloadLength({ response: tools, stop: "refusal", refusal: "No." }, "rr"), + ).toBe(servedPayloadLength(tools, "rr")); + expect(misbehaviorPayloadLength({ response: tools, reasoning: "xyz" }, undefined)).toBe( + servedPayloadLength(tools), + ); + }); + + it("gives the served length for other faults on a text response", () => { + expect(misbehaviorPayloadLength({ response: text, stop: "length" }, "rr")).toBe( + servedPayloadLength(text, "rr"), + ); + }); +}); + +describe("extendRecordedTimings", () => { + it("extends the gaps to frameCount - 1 by repeating the last gap (hand-computed)", () => { + const empty = { ...recorded, interChunkDelaysMs: [] }; + const cases: Array<[RecordedTimings, number, number[]]> = [ + [recorded, 0, [10, 20, 30]], + [recorded, 1, [10, 20, 30]], + [recorded, 4, [10, 20, 30]], + [recorded, 6, [10, 20, 30, 30, 30]], + [recorded, 9, [10, 20, 30, 30, 30, 30, 30, 30]], + [empty, 0, []], + [empty, 4, [0, 0, 0]], + ]; + for (const [t, frames, gaps] of cases) { + expect(extendRecordedTimings(t, frames, true)).toEqual({ ...t, interChunkDelaysMs: gaps }); + } + expect(extendRecordedTimings(undefined, 9, true)).toBeUndefined(); + expect(extendRecordedTimings(undefined, 9, false)).toBeUndefined(); + }); + + it("returns the same object when not a duplicate plan", () => { + expect(extendRecordedTimings(recorded, 9, false)).toBe(recorded); + }); + + it("gives the hand-computed non-stream total for a K4 plan on OpenAI chat and on another provider", () => { + // The shared misbehavior path: chunkCount = max(1, ceil(len / chunkSize)), + // then extendRecordedTimings, then nonStreamingDelayMs. OpenAI chat and a + // non-OpenAI provider feed it the same fixture-level inputs. + const f = fixture({ + recordedTimings: { ttftMs: 5, interChunkDelaysMs: [1, 9], totalDurationMs: 15 }, + nonStreamingTimingAllProviders: true, + }); + const plan = { + response: { + toolCalls: [ + { name: "a", arguments: '{"x":1}', id: "c1" }, + { name: "b", arguments: '{"y":2}', id: "c1" }, + ], + } as FixtureResponse, + }; + // "a" + '{"x":1}' + "b" + '{"y":2}' = 16 chars; at chunkSize 3 that is + // 6 chunks, so the recorded gaps [1, 9] are extended to 5 gaps. + const length = misbehaviorPayloadLength(plan, undefined); + expect(length).toBe(16); + const extended = extendRecordedTimings(f.recordedTimings, Math.max(1, Math.ceil(16 / 3)), true); + expect(extended?.interChunkDelaysMs).toEqual([1, 9, 9, 9, 9]); + // An empty payload is clamped to one chunk: no gap is added. + const emptyExtended = extendRecordedTimings(f.recordedTimings, Math.max(1, 0), true); + expect(emptyExtended?.interChunkDelaysMs).toEqual([1, 9]); + for (const surface of ["openai-chat", "other"] as const) { + expect(nonStreamingTimingEnabled(f, surface)).toBe(true); + // ttft 5 + 1 + 4 gaps of 9 = 42. Unextended, the 3 extra chunks would + // average the recorded gaps (5): 5 + 1 + 9 + 3 * 5 = 30. + expect(nonStreamingDelayMs(f, length, 3, extended, 1)).toBe(42); + expect(nonStreamingDelayMs(f, length, 3, f.recordedTimings, 1)).toBe(30); + // An empty payload still pays one chunk (the Math.max(1, …) clamp): ttft 5. + expect(nonStreamingDelayMs(f, 0, 3, emptyExtended, 1)).toBe(5); + } + }); +}); + +describe("recordedTimings with a missing optional field, over HTTP", () => { + // A fixture built in code can omit `interChunkDelaysMs` (the file loader + // fills it in; `addFixture` does not). Each row is what base 508dd9b9 + // answered for the same request. Base never waited on a non-streaming + // reply unless `nonStreamingTiming: true` was set on OpenAI chat, so every + // other non-streaming row must answer 200. + const content = "x".repeat(40); + const noGaps: Partial = { ttftMs: 1, totalDurationMs: 1 }; + const noTtft: Partial = { interChunkDelaysMs: [1, 1], totalDurationMs: 2 }; + const flags = { + off: {}, + nonStreamingTiming: { nonStreamingTiming: true }, + allProviders: { nonStreamingTimingAllProviders: true }, + both: { nonStreamingTiming: true, nonStreamingTimingAllProviders: true }, + } as const; + type FlagName = keyof typeof flags; + type Row = { + provider: "openai" | "anthropic"; + stream: boolean; + flag: FlagName; + timings: "noGaps" | "noTtft"; + duplicate?: boolean; + status: number; + }; + + // Base 500s only where it summed the gaps: OpenAI chat, stream:false, + // nonStreamingTiming: true, with more than one chunk. + const baseStatus = (r: Omit): number => + r.provider === "openai" && + !r.stream && + r.timings === "noGaps" && + (r.flag === "nonStreamingTiming" || r.flag === "both") + ? 500 + : 200; + const rows: Row[] = []; + for (const provider of ["openai", "anthropic"] as const) + for (const stream of [false, true]) + for (const flag of Object.keys(flags) as FlagName[]) + for (const timings of ["noGaps", "noTtft"] as const) { + const row = { provider, stream, flag, timings }; + rows.push({ ...row, status: baseStatus(row) }); + } + for (const flag of Object.keys(flags) as FlagName[]) { + const row = { + provider: "openai" as const, + stream: false, + flag, + timings: "noGaps" as const, + duplicate: true, + }; + rows.push({ ...row, status: baseStatus(row) }); + } + + const key = (r: Omit) => + `${r.provider}-${r.stream ? "stream" : "json"}-${r.flag}-${r.timings}${r.duplicate ? "-dup" : ""}`; + + it.each(rows.map((r) => [key(r), r] as const))("%s answers as base did", async (_, row) => { + const timings = row.timings === "noGaps" ? noGaps : noTtft; + const mock = new LLMock({ port: 0, logLevel: "silent" }); + mock.addFixture({ + match: { userMessage: key(row) }, + response: row.duplicate + ? { + toolCalls: [ + { name: "a", arguments: content }, + { name: "b", arguments: content }, + ], + } + : { content }, + chunkSize: 4, + recordedTimings: timings as RecordedTimings, + ...(row.duplicate ? { misbehavior: "tool-call-id-duplicate" as const } : {}), + ...flags[row.flag], + }); + const url = await mock.start(); + try { + const messages = [{ role: "user", content: key(row) }]; + const res = + row.provider === "openai" + ? await fetch(`${url}/v1/chat/completions`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ model: "gpt-4o", stream: row.stream, messages }), + }) + : await fetch(`${url}/v1/messages`, { + method: "POST", + headers: { + "Content-Type": "application/json", + "x-api-key": "k", + "anthropic-version": "2023-06-01", + }, + body: JSON.stringify({ + model: "claude-3-5-sonnet-20241022", + max_tokens: 64, + stream: row.stream, + messages, + }), + }); + expect(res.status).toBe(row.status); + // A streamed reply with no gaps is cut short on base too (the frame + // loop reads the missing array); only its status is pinned here. + if (row.stream && row.timings === "noGaps") { + await res.body?.cancel(); + return; + } + const body = await res.text(); + if (row.status === 200 && !row.duplicate) + expect(body).toContain(row.stream ? "xxxx" : content); + } finally { + await mock.stop(); + } + }); +}); diff --git a/src/__tests__/responses-ollama-nonstream-timing.test.ts b/src/__tests__/responses-ollama-nonstream-timing.test.ts new file mode 100644 index 00000000..637a5599 --- /dev/null +++ b/src/__tests__/responses-ollama-nonstream-timing.test.ts @@ -0,0 +1,155 @@ +/** + * Opt-in non-streaming timing (`nonStreamingTimingAllProviders: true`) on the + * OpenAI Responses and Ollama wires, over real HTTP with real timing. + * + * 1. The Responses JSON wait is sized from what the Responses wire serves + * (custom tool calls, `responsesBlocks` over `blocks`, citations and + * generated files), so it tracks the + * timed stream of the same fixture within a few frames of overhead. + * 2. A misbehavior-applied JSON reply is built after the wait, so its + * `created_at` is not earlier than the request time plus the delay. + */ +import { afterEach, describe, expect, it } from "vitest"; +import { LLMock } from "./helpers/misbehavior-enabled.js"; +import type { Fixture } from "../types.js"; + +let mock: LLMock | null = null; +afterEach(async () => { + await mock?.stop(); + mock = null; +}); + +async function start(fixtures: Fixture[]): Promise { + mock = new LLMock({ port: 0, logLevel: "silent", responsesTools: "extended" }); + mock.addFixtures(fixtures); + return mock.start(); +} + +/** Milliseconds from sending the request to reading the whole body. */ +async function timedResponses(url: string, input: string, stream: boolean): Promise { + const t0 = performance.now(); + const res = await fetch(`${url}/v1/responses`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ model: "gpt-5", input, stream }), + }); + const text = await res.text(); + expect(res.status, text).toBe(200); + return performance.now() - t0; +} + +// 40 frames per second: 25 ms per frame (chunk 0 included, no ttft). +const timing = { + streamingProfile: { tps: 40 }, + nonStreamingTimingAllProviders: true, + chunkSize: 20, +}; +const INPUT = "p".repeat(400); + +describe("Responses non-streaming wait matches the timed stream", () => { + const cases: Array<[string, Fixture["response"]]> = [ + [ + "customToolCalls", + { toolCalls: [], customToolCalls: [{ type: "custom", name: "apply_patch", input: INPUT }] }, + ], + [ + "responsesBlocks with a custom call", + { responsesBlocks: [{ type: "customToolCall", name: "apply_patch", input: INPUT }] }, + ], + [ + "blocks plus responsesBlocks", + { + blocks: [{ type: "text", text: "x" }], + responsesBlocks: [{ type: "text", text: INPUT }], + }, + ], + ]; + + // One text chunk, then frames with no text: 10 annotation.added frames and + // 2 image_generation_call items of 5 frames each. The JSON wait counts those + // 20 frames as chunks; without them it counted one chunk (about 25 ms) and + // trailed the stream by about 750 ms. + it("citations and generated files", async () => { + const name = "cited"; + const text = "Cited text."; + const response: Fixture["response"] = { + content: text, + citations: Array.from({ length: 10 }, (_, i) => ({ + url: `https://example.com/${i}`, + startIndex: 0, + endIndex: text.length, + })), + files: Array.from({ length: 2 }, () => ({ mimeType: "image/png", data: "iVBORw0KGgo=" })), + }; + const url = await start([{ match: { userMessage: name }, response, ...timing }]); + const streamMs = await timedResponses(url, name, true); + const jsonMs = await timedResponses(url, name, false); + expect(streamMs).toBeGreaterThan(500); + expect(jsonMs).toBeGreaterThan(450); + expect(streamMs - jsonMs).toBeLessThanOrEqual(350); + expect(jsonMs).toBeLessThanOrEqual(streamMs + 100); + }); + + it.each(cases)("%s", async (name, response) => { + const url = await start([{ match: { userMessage: name }, response, ...timing }]); + const streamMs = await timedResponses(url, name, true); + const jsonMs = await timedResponses(url, name, false); + // The stream sends 20 delta frames plus 6 to 8 envelope frames (650 to + // 700 ms). The JSON wait counts only the served payload chunks (about + // 500 to 525 ms), so it trails the stream by the envelope frames plus + // HTTP overhead: at most 350 ms. Before the fix it counted one chunk + // (about 25 ms), so it trailed by about 700 ms. + expect(streamMs).toBeGreaterThan(500); + expect(streamMs - jsonMs).toBeLessThanOrEqual(350); + expect(jsonMs).toBeLessThanOrEqual(streamMs + 100); + }); +}); + +describe("misbehavior JSON reply is built after the wait", () => { + it("Responses: created_at is at or after the request time plus the delay", async () => { + const url = await start([ + { + match: { userMessage: "late" }, + response: { content: "hi" }, + misbehavior: "empty-response", + recordedTimings: { ttftMs: 1500, interChunkDelaysMs: [], totalDurationMs: 1500 }, + nonStreamingTimingAllProviders: true, + }, + ]); + const t0 = Date.now(); + const res = await fetch(`${url}/v1/responses`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ model: "gpt-5", input: "late" }), + }); + const body = (await res.json()) as { created_at: number }; + expect(res.status).toBe(200); + expect(body.created_at).toBeGreaterThanOrEqual(Math.floor((t0 + 1500) / 1000)); + }); + + it("Ollama: created_at is at or after the request time plus the delay", async () => { + const url = await start([ + { + match: { userMessage: "late" }, + response: { content: "hi" }, + misbehavior: "empty-response", + recordedTimings: { ttftMs: 400, interChunkDelaysMs: [], totalDurationMs: 400 }, + nonStreamingTimingAllProviders: true, + }, + ]); + const t0 = Date.now(); + const res = await fetch(`${url}/api/chat`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + model: "llama3", + stream: false, + messages: [{ role: "user", content: "late" }], + }), + }); + const body = (await res.json()) as { created_at: string }; + expect(res.status).toBe(200); + // Timer slack: a 400 ms timer can fire a millisecond early. + expect(Date.parse(body.created_at)).toBeGreaterThanOrEqual(t0 + 400 - 5); + }); +}); diff --git a/src/__tests__/teardown-attribution.test.ts b/src/__tests__/teardown-attribution.test.ts new file mode 100644 index 00000000..6abc3b6f --- /dev/null +++ b/src/__tests__/teardown-attribution.test.ts @@ -0,0 +1,370 @@ +/** + * One teardown-attribution rule for the shared delivery code, on real sockets + * and aimock's real server settings (`http.createServer` defaults: + * `httpAllowHalfOpen` is false, so Node ends the response on a client FIN). + * + * - A client hang-up is the client's socket closing (Node ended our write side + * in answer to its FIN) or resetting (`errored`). A FIN alone is not one. + * - A teardown aimock caused is `"disconnect"` (`"proxy stream destroyed"` on + * the proxy relay). + * - "No status line sent" means nothing of this response reached the socket. + */ +import { EventEmitter } from "node:events"; +import * as fs from "node:fs"; +import * as http from "node:http"; +import * as net from "node:net"; +import * as os from "node:os"; +import * as path from "node:path"; +import type { AddressInfo } from "node:net"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { Logger } from "../logger.js"; +import { proxyAndRecord } from "../recorder.js"; +import { deliverJson, deliverStream } from "../response/deliver.js"; +import type { FrameCodec } from "../response/frame-loop.js"; +import { AIMOCK_DISCONNECTED, CLIENT_ABORTED, httpLifecycle } from "../response/lifecycle.js"; +import { proxyInterruption, withHangUpDetection } from "../response/proxy.js"; +import type { ChatCompletionRequest, Fixture, JournalEntry } from "../types.js"; + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +/** Listen on a server built with aimock's own settings (no `httpAllowHalfOpen`). */ +async function listen(handler: http.RequestListener): Promise { + const server = http.createServer(handler); + await new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve())); + cleanups.push(async () => { + server.closeAllConnections(); + await new Promise((resolve) => server.close(() => resolve())); + }); + return (server.address() as AddressInfo).port; +} + +const chatBody = JSON.stringify({ + model: "gpt-4o", + messages: [{ role: "user", content: "teardown probe" }], + stream: true, +}); +const rawRequest = + `POST /v1/chat/completions HTTP/1.1\r\nHost: 127.0.0.1\r\n` + + `Content-Type: application/json\r\nContent-Length: ${Buffer.byteLength(chatBody)}\r\n\r\n${chatBody}`; + +/** + * A raw client: sends the request, runs `act` after `afterMs`, and resolves + * with the bytes it read once its socket closes. + */ +function rawClient(port: number, afterMs: number, act: (s: net.Socket) => void): Promise { + return new Promise((resolve) => { + const chunks: Buffer[] = []; + const socket = net.connect(port, "127.0.0.1", () => { + socket.write(rawRequest); + setTimeout(() => act(socket), afterMs); + }); + socket.on("data", (d: Buffer) => chunks.push(d)); + socket.on("error", () => undefined); + socket.on("close", () => resolve(Buffer.concat(chunks).toString())); + }); +} + +function entry(): JournalEntry { + return { + id: "e1", + timestamp: 0, + method: "POST", + path: "/x", + headers: {}, + body: null, + response: { status: 200, fixture: null }, + }; +} + +const plain: Fixture = { match: {}, response: { content: "hello" } }; +const timed: Fixture = { + ...plain, + nonStreamingTimingAllProviders: true, + streamingProfile: { ttft: 400, tps: 10 }, +}; +const codec: FrameCodec = { + headers: { "Content-Type": "text/event-stream" }, + encode: (frame) => `data: ${frame}\n\n`, +}; +const jsonOpts = { surface: "other" as const, payloadLength: 10, chunkSize: 5 }; + +describe("(1) a FIN alone is not a client hang-up", () => { + it("a socket showing only the client FIN when aimock tears down is aimock's teardown, in the lifecycle and the proxy alike", async () => { + let seen!: { readableEnded: boolean; writableEnded: boolean; lifecycle: string | undefined }; + let proxied!: Promise<{ clientHungUp: boolean }>; + const port = await listen((req, res) => { + req.resume(); + // aimock destroys the response in the tick the FIN is read, before Node + // ends the write side for it: at that instant the socket shows the FIN + // and nothing else, and both helpers judge it then. + req.socket.prependListener("end", () => { + res.destroy(); + const lifecycle = httpLifecycle(res, plain, { streaming: true }); + seen = { + readableEnded: req.socket.readableEnded, + writableEnded: req.socket.writableEnded, + lifecycle: lifecycle.reason(), + }; + lifecycle.dispose(); + proxied = withHangUpDetection(req, res, async () => undefined); + }); + }); + await rawClient(port, 50, (s) => s.end()); + expect(seen).toEqual({ + readableEnded: true, + writableEnded: false, + lifecycle: AIMOCK_DISCONNECTED, + }); + expect((await proxied).clientHungUp).toBe(false); + }); + + it("a client FIN that makes Node end the response is still the client's", async () => { + const e = entry(); + let delivered!: Promise<{ clientHungUp: boolean }>; + const port = await listen((req, res) => { + req.resume(); + delivered = withHangUpDetection(req, res, () => + deliverStream(res, e, plain, codec, ["a", "b"], { latency: 300 }), + ); + }); + await rawClient(port, 50, (s) => s.end()); + const { clientHungUp } = await delivered; + expect(clientHungUp).toBe(true); + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + }); +}); + +/** An upstream that answers only after `headDelayMs`, so no status line goes out before then. */ +async function lateUpstream(headDelayMs: number): Promise { + const port = await listen((req, res) => { + req.resume(); + const timer = setTimeout(() => { + res.writeHead(200, { "Content-Type": "application/json" }); + res.end(JSON.stringify({ id: "x", object: "chat.completion", choices: [] })); + }, headDelayMs); + res.on("close", () => clearTimeout(timer)); + }); + return `http://127.0.0.1:${port}`; +} + +/** A front server running the record-mode relay inside the hang-up helper. */ +async function proxyFront( + upstreamUrl: string, + before?: (res: http.ServerResponse, relay: () => void) => void, +): Promise<{ + port: number; + journaled: Promise & { clientHungUp: boolean }>; +}> { + const fixturePath = fs.mkdtempSync(path.join(os.tmpdir(), "aimock-teardown-")); + cleanups.push(() => fs.rmSync(fixturePath, { recursive: true, force: true })); + let settle!: (v: ReturnType & { clientHungUp: boolean }) => void; + const journaled = new Promise & { clientHungUp: boolean }>( + (resolve) => (settle = resolve), + ); + const port = await listen((req, res) => { + const relay = (): void => { + void withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + JSON.parse(chatBody) as ChatCompletionRequest, + "openai", + "/v1/chat/completions", + [], + { + record: { providers: { openai: upstreamUrl }, fixturePath }, + logger: new Logger("silent"), + }, + chatBody, + ), + ).then(({ clientHungUp }) => + settle({ clientHungUp, ...proxyInterruption(res, clientHungUp) }), + ); + }; + req.resume(); + req.on("end", () => (before ? before(res, relay) : relay())); + }); + return { port, journaled }; +} + +describe("(2) proxy: no status line sent journals status 0", () => { + it("a client that hangs up before the upstream answers is journaled with status 0", async () => { + const { port, journaled } = await proxyFront(await lateUpstream(400)); + const received = await rawClient(port, 100, (s) => s.destroy()); + expect(received).toBe(""); + expect(await journaled).toEqual({ + clientHungUp: true, + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + }); +}); + +describe("(3) proxy: a response already gone at entry is blamed by evidence", () => { + it("a response aimock destroyed before the relay is not a client hang-up", async () => { + const { port, journaled } = await proxyFront(await lateUpstream(50), (res, relay) => { + res.destroy(); + res.once("close", relay); + }); + await rawClient(port, 1000, (s) => s.destroy()); + const fields = await journaled; + expect(fields.clientHungUp).toBe(false); + expect(fields.interruptReason).toBe("proxy stream destroyed"); + }); + + it("a client gone before the relay is still the client's", async () => { + const { port, journaled } = await proxyFront(await lateUpstream(50), (res, relay) => { + res.once("close", relay); + }); + await rawClient(port, 50, (s) => s.destroy()); + const fields = await journaled; + expect(fields.clientHungUp).toBe(true); + expect(fields.interruptReason).toBe(CLIENT_ABORTED); + }); +}); + +describe("(4) deliverJson untimed: a response already gone is not written", () => { + it("aimock's own teardown: nothing written, the thunk never runs, journaled as a disconnect with status 0", async () => { + const e = entry(); + const body = vi.fn(() => ({ ok: true })); + let delivered!: Promise; + const errors: unknown[] = []; + const port = await listen((req, res) => { + req.resume(); + res.on("error", (err) => errors.push(err)); + req.on("end", () => { + res.destroy(); + res.once("close", () => { + delivered = deliverJson(res, e, plain, body, jsonOpts); + }); + }); + }); + const received = await rawClient(port, 1000, (s) => s.destroy()); + await vi.waitFor(() => expect(delivered).toBeDefined()); + await delivered; + expect(received).toBe(""); + expect(errors).toEqual([]); + expect(body).not.toHaveBeenCalled(); + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: AIMOCK_DISCONNECTED, + }); + }); + + it("a client gone first: journaled as a client abort with status 0", async () => { + const e = entry(); + const body = vi.fn(() => ({ ok: true })); + let delivered!: Promise; + const port = await listen((req, res) => { + req.resume(); + res.on("error", () => undefined); + res.once("close", () => { + delivered = deliverJson(res, e, plain, body, jsonOpts); + }); + }); + await rawClient(port, 50, (s) => s.destroy()); + await vi.waitFor(() => expect(delivered).toBeDefined()); + await delivered; + expect(body).not.toHaveBeenCalled(); + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + }); +}); + +describe("(5) deliverJson timed: a response ended during the wait", () => { + it("is not written again and does not throw ERR_HTTP_HEADERS_SENT", async () => { + const e = entry(); + const body = vi.fn(() => ({ ok: true })); + let delivered!: Promise; + const port = await listen((req, res) => { + req.resume(); + delivered = deliverJson(res, e, timed, body, jsonOpts); + setTimeout(() => { + res.writeHead(503, { "Content-Type": "application/json" }); + res.end("{}"); + }, 50); + }); + const received = await rawClient(port, 1200, (s) => s.destroy()); + await expect(delivered).resolves.toBeUndefined(); + expect(received).toContain("503"); + expect(body).not.toHaveBeenCalled(); + expect(e.response).toMatchObject({ + status: 503, + interrupted: true, + interruptReason: AIMOCK_DISCONNECTED, + }); + }); +}); + +describe("(6) status 0 is judged by bytes reaching the socket, not headersSent", () => { + /** A response whose `writeHead` commits the head without any byte reaching the socket. */ + class HeadOnlyRes extends EventEmitter { + destroyed = false; + writableEnded = false; + writableFinished = false; + headersSent = false; + statusCode = 200; + socket = { bytesWritten: 0, errored: null as Error | null, writableEnded: false }; + writeHead(status: number): this { + this.statusCode = status; + this.headersSent = true; + return this; + } + setHeader(): void {} + write(): boolean { + this.socket.bytesWritten += 10; + return true; + } + end(): this { + this.writableEnded = true; + this.writableFinished = true; + return this; + } + destroy(): this { + this.destroyed = true; + return this; + } + } + + it("a client that leaves after writeHead but before any byte reached the socket journals status 0", async () => { + vi.useFakeTimers(); + try { + const res = new HeadOnlyRes(); + const e = entry(); + const headFirst: FrameCodec = { ...codec, writeHeadImmediately: true }; + const promise = deliverStream( + res as unknown as http.ServerResponse, + e, + plain, + headFirst, + ["a", "b"], + { latency: 50 }, + ); + await vi.advanceTimersByTimeAsync(20); + expect(res.headersSent).toBe(true); + res.socket.errored = new Error("read ECONNRESET"); + res.destroyed = true; + res.emit("close"); + await promise; + expect(e.response).toMatchObject({ + status: 0, + interrupted: true, + interruptReason: CLIENT_ABORTED, + }); + } finally { + vi.useRealTimers(); + } + }); +}); diff --git a/src/__tests__/timing-replay.test.ts b/src/__tests__/timing-replay.test.ts index 43ba191c..7337e2a2 100644 --- a/src/__tests__/timing-replay.test.ts +++ b/src/__tests__/timing-replay.test.ts @@ -2,7 +2,7 @@ import { describe, it, expect, afterEach, vi } from "vitest"; import http from "node:http"; import { createServer, type ServerInstance } from "../server.js"; import type { Fixture, SSEChunk, ChatCompletionRequest, RecordedTimings } from "../types.js"; -import * as sseWriter from "../sse-writer.js"; +import * as deliver from "../response/deliver.js"; // --------------------------------------------------------------------------- // Helpers @@ -61,6 +61,7 @@ function chatRequest( let instance: ServerInstance | null = null; afterEach(async () => { + vi.restoreAllMocks(); if (instance) { await new Promise((resolve) => instance!.server.close(() => resolve())); instance = null; @@ -125,14 +126,17 @@ describe("timing-aware replay through handleCompletions", () => { chunkSize: 5, }); - const writeSSEStream = vi.spyOn(sseWriter, "writeSSEStream"); + const deliverStream = vi.spyOn(deliver, "deliverStream"); const res = await httpPost(`${instance.url}/v1/chat/completions`, chatRequest("speed-test")); expect(res.status).toBe(200); const chunks = parseSSEResponse(res.body); expect(chunks.length).toBeGreaterThan(1); - expect(writeSSEStream).toHaveBeenCalledWith( + expect(deliverStream).toHaveBeenCalledWith( + expect.anything(), + expect.anything(), + expect.anything(), expect.anything(), expect.anything(), expect.objectContaining({ recordedTimings: timings, replaySpeed: 2 }), diff --git a/src/__tests__/ws-framing.test.ts b/src/__tests__/ws-framing.test.ts index 7477edec..c9d99364 100644 --- a/src/__tests__/ws-framing.test.ts +++ b/src/__tests__/ws-framing.test.ts @@ -993,3 +993,88 @@ describe("received transport activity", () => { expect(activities).toBe(1); }); }); + +describe("client half-close (TCP FIN without a close frame)", () => { + async function upgradedPair(): Promise<{ + socket: net.Socket; + ws: WebSocketConnection; + serverSocket: Promise; + events: string[]; + closedAt: Promise; + }> { + const server = http.createServer(); + let resolveServerSocket: (socket: net.Socket) => void = () => {}; + const serverSocket = new Promise((resolve) => { + resolveServerSocket = resolve; + }); + let resolveWs: (ws: WebSocketConnection) => void = () => {}; + const wsPromise = new Promise((resolve) => { + resolveWs = resolve; + }); + server.on("connection", (socket) => { + socket.on("error", () => {}); + }); + server.on("upgrade", (req, socket) => { + socket.on("error", () => {}); + resolveServerSocket(socket as net.Socket); + resolveWs(upgradeToWebSocket(req, socket as net.Socket)); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + const { socket, response } = rawConnect((server.address() as net.AddressInfo).port); + trackCleanup(server, socket); + await response; + const ws = await wsPromise; + ws.on("error", () => {}); + const events: string[] = []; + ws.on("message", (message: string) => events.push(`message:${message}`)); + const closedAt = new Promise((resolve) => { + ws.on("close", (code: number, reason: string) => { + events.push(`close:${code}:${reason}`); + resolve(performance.now()); + }); + }); + return { socket, ws, serverSocket, events, closedAt }; + } + + it("emits close promptly and exactly once when the client sends FIN", async () => { + const { socket, ws, serverSocket, events, closedAt } = await upgradedPair(); + const finAt = performance.now(); + socket.end(); + + const elapsed = (await closedAt) - finAt; + expect(elapsed).toBeLessThan(1000); + expect(ws.isClosed).toBe(true); + expect((await serverSocket).destroyed).toBe(true); + // The server socket's own close follows; it must not emit a second close. + await new Promise((r) => setTimeout(r, 200)); + expect(events).toEqual(["close:1006:Connection lost"]); + }); + + it("delivers a message sent before FIN, then closes once", async () => { + const { socket, events, closedAt } = await upgradedPair(); + socket.end(createMaskedFrame(OP_TEXT, Buffer.from("last words"))); + + await closedAt; + await new Promise((r) => setTimeout(r, 200)); + expect(events).toEqual(["message:last words", "close:1006:Connection lost"]); + }); + + it("keeps the close-frame handshake unchanged when FIN follows the close frame", async () => { + const { socket, ws, events, closedAt } = await upgradedPair(); + const echo = readServerFrame(socket); + const serverFin = new Promise((resolve) => socket.once("end", () => resolve())); + const closePayload = Buffer.alloc(2 + 3); + closePayload.writeUInt16BE(1000, 0); + closePayload.write("bye", 2); + socket.end(createMaskedFrame(OP_CLOSE, closePayload)); + + await closedAt; + const frame = await echo; + expect(frame.opcode).toBe(OP_CLOSE); + expect(frame.payload.equals(closePayload)).toBe(true); + await serverFin; + await new Promise((r) => setTimeout(r, 200)); + expect(events).toEqual(["close:1000:bye"]); + expect(ws.isClosed).toBe(true); + }); +}); diff --git a/src/aws-event-stream.ts b/src/aws-event-stream.ts index b370e7c0..00862504 100644 --- a/src/aws-event-stream.ts +++ b/src/aws-event-stream.ts @@ -18,7 +18,7 @@ import { crc32 } from "node:zlib"; import type * as http from "node:http"; import type { StreamingProfile, RecordedTimings } from "./types.js"; -import { delay, calculateDelay } from "./sse-writer.js"; +import { httpSink, runFrameLoop, type FrameCodec } from "./response/frame-loop.js"; // ─── Header encoding ──────────────────────────────────────────────────────── @@ -128,34 +128,38 @@ export async function writeEventStream( }, ): Promise { const opts = options ?? {}; - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "application/vnd.amazon.eventstream"); - res.setHeader("Transfer-Encoding", "chunked"); - - let chunkIndex = 0; - for (const event of events) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) { - await delay(chunkDelay, signal); - } - if (signal?.aborted) return false; - if (res.writableEnded) return true; - - const frame = encodeEventStreamMessage(event.eventType, event.payload); - res.write(frame); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } + // Base contract (public API): with no frames, an aborted signal does not + // stop the writer; it still finishes the stream (any trailer, then end()) + // and returns true. The frame loop returns false there, so pass it no + // signal when there are no frames (it reads the signal only per frame and + // after the loop). + return runFrameLoop( + httpSink(res), + events, + eventStreamCodec(), + { + latency: opts.latency, + streamingProfile: opts.streamingProfile, + recordedTimings: opts.recordedTimings, + replaySpeed: opts.replaySpeed, + }, + { signal: events.length > 0 ? opts.signal : undefined, onChunkSent: opts.onChunkSent }, + ); +} - if (!res.writableEnded) { - res.end(); - } - return true; +// Internal (not part of the public API): the AWS binary event-stream codec. +// Each event becomes one complete frame, encoded by encodeEventStreamMessage +// unless an `encode` argument replaces it. There is no prelude or trailer. +// These are line comments, not JSDoc, so the bundled .d.ts does not attach +// them to a public declaration. +export function eventStreamCodec( + encode?: (event: F) => Buffer, +): FrameCodec { + return { + headers: { + "Content-Type": "application/vnd.amazon.eventstream", + "Transfer-Encoding": "chunked", + }, + encode: encode ?? ((event) => encodeEventStreamMessage(event.eventType, event.payload)), + }; } diff --git a/src/bedrock-converse.ts b/src/bedrock-converse.ts index 5e7f983b..1ecdc2d9 100644 --- a/src/bedrock-converse.ts +++ b/src/bedrock-converse.ts @@ -44,8 +44,10 @@ import { } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; import { writeErrorResponse } from "./sse-writer.js"; -import { writeEventStream } from "./aws-event-stream.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { eventStreamCodec } from "./aws-event-stream.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { misbehaviorPayloadLength, servedPayloadLength } from "./response/timing.js"; import type { Journal } from "./journal.js"; import type { Logger } from "./logger.js"; import { mapAnthropicFinish } from "./response/finish.js"; @@ -1028,15 +1030,8 @@ export async function handleConverse( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "bedrock", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "bedrock", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1045,7 +1040,12 @@ export async function handleConverse( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1111,6 +1111,14 @@ export async function handleConverse( ); return; } + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize: Math.max(1, fixture.chunkSize ?? defaults.chunkSize), + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); if (misbehavior.kind === "applied") { const prepared = prepareConverseMisbehavior( @@ -1121,16 +1129,29 @@ export async function handleConverse( req.headers, ); misbehavior = prepared.plan; + // Built before the journal entry, as before: a builder that throws leaves + // no entry. The reply carries no timestamp, so building early is safe. const body = buildConverseMisbehaviorResponse(prepared); - addResponseEntry({ + const entry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, response: { status: 200, fixture }, }); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + entry, + fixture, + () => body, + jsonOpts( + misbehaviorPayloadLength( + prepared.plan, + prepared.plan.reasoning === undefined ? prepared.reasoning : undefined, + ), + !!prepared.plan.duplicateId, + ), + ); return; } @@ -1172,7 +1193,7 @@ export async function handleConverse( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), @@ -1180,16 +1201,21 @@ export async function handleConverse( response: { status: 200, fixture }, }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Bedrock Converse"); - const body = buildConverseContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - logger, - effReasoning, - overrides, - response.blocks, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildConverseContentWithToolCallsResponse( + response.content ?? "", + functionToolCalls, + logger, + effReasoning, + overrides, + response.blocks, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); return; } @@ -1207,16 +1233,20 @@ export async function handleConverse( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, response: { status: 200, fixture }, }); - const body = buildConverseTextResponse(response.content, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildConverseTextResponse(response.content, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); return; } @@ -1234,7 +1264,7 @@ export async function handleConverse( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), @@ -1242,9 +1272,13 @@ export async function handleConverse( response: { status: 200, fixture }, }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Bedrock Converse"); - const body = buildConverseToolCallResponse(functionToolCalls, logger, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildConverseToolCallResponse(functionToolCalls, logger, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); return; } @@ -1424,15 +1458,8 @@ export async function handleConverseStream( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "bedrock", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "bedrock", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1441,7 +1468,12 @@ export async function handleConverseStream( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1507,8 +1539,13 @@ export async function handleConverseStream( ); return; } - const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const streamTiming = { + latency: fixture.latency ?? defaults.latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }; if (misbehavior.kind === "applied") { const prepared = prepareConverseMisbehavior( @@ -1527,21 +1564,7 @@ export async function handleConverseStream( body: completionReq, response: { status: 200, fixture }, }); - const interruption = createInterruptionSignal(fixture); - const completed = await writeEventStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, entry, fixture, eventStreamCodec(), events, streamTiming); return; } @@ -1600,21 +1623,7 @@ export async function handleConverseStream( overrides, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeEventStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, eventStreamCodec(), events, streamTiming); return; } @@ -1645,21 +1654,7 @@ export async function handleConverseStream( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeEventStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, eventStreamCodec(), events, streamTiming); return; } @@ -1692,21 +1687,7 @@ export async function handleConverseStream( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeEventStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, eventStreamCodec(), events, streamTiming); return; } diff --git a/src/bedrock.ts b/src/bedrock.ts index 17a4fac3..b18db8ee 100644 --- a/src/bedrock.ts +++ b/src/bedrock.ts @@ -54,8 +54,10 @@ import { } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; import { writeErrorResponse } from "./sse-writer.js"; -import { writeEventStream } from "./aws-event-stream.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { encodeEventStreamMessage, eventStreamCodec } from "./aws-event-stream.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { misbehaviorPayloadLength, servedPayloadLength } from "./response/timing.js"; import type { Journal } from "./journal.js"; import type { Logger } from "./logger.js"; import { applyChaosAsync } from "./chaos.js"; @@ -547,15 +549,8 @@ export async function handleBedrock( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "bedrock", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "bedrock", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -564,7 +559,12 @@ export async function handleBedrock( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -630,12 +630,21 @@ export async function handleBedrock( writeErrorResponse(res, misbehavior.status, renderedError.body); return; } + const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); if (misbehavior.kind === "applied") { const prepared = prepareClaudeMisbehavior(misbehavior, { request: completionReq, stream: false, - chunkSize: Math.max(1, fixture.chunkSize ?? defaults.chunkSize), + chunkSize, logger, strict: resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, citations: false, @@ -649,8 +658,20 @@ export async function handleBedrock( response: { status: 200, fixture }, }); recordMisbehaviorOutcome({ entry, summary: prepared.plan.summary, defaults, testId }); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(prepared.body)); + const preparedBody = prepared.body; + await deliverJson( + res, + entry, + fixture, + () => preparedBody, + jsonOpts( + misbehaviorPayloadLength( + misbehavior, + misbehavior.reasoning === undefined ? prepared.reasoning : undefined, + ), + !!misbehavior.duplicateId, + ), + ); return; } @@ -684,7 +705,7 @@ export async function handleBedrock( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), @@ -692,46 +713,52 @@ export async function handleBedrock( response: { status: 200, fixture }, }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Bedrock InvokeModel"); - if (response.blocks && response.blocks.length > 0) { + const blocks = response.blocks; + if (blocks && blocks.length > 0) { // NEW PATH: honor the fixture's ordered `blocks` in the positionally // observable Anthropic-style content array (tool-first expressible). - const blocksBody = buildBedrockBlocksResponse( - response.blocks, - completionReq.model, - logger, - effReasoning, - overrides, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildBedrockBlocksResponse(blocks, completionReq.model, logger, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(blocksBody)); return; } - const textBody = buildBedrockTextResponse( - response.content ?? "", - completionReq.model, - effReasoning, - overrides, - ); - const toolBody = buildBedrockToolCallResponse( - functionToolCalls, - completionReq.model, - logger, - // Reasoning is rendered by the text response in this merged path; pass - // undefined here to avoid emitting a duplicate thinking block. - undefined, - overrides, + await deliverJson( + res, + journalEntry, + fixture, + () => { + const textBody = buildBedrockTextResponse( + response.content ?? "", + completionReq.model, + effReasoning, + overrides, + ); + const toolBody = buildBedrockToolCallResponse( + functionToolCalls, + completionReq.model, + logger, + // Reasoning is rendered by the text response in this merged path; pass + // undefined here to avoid emitting a duplicate thinking block. + undefined, + overrides, + ); + // Merge: take the text response as base, append tool_use blocks, set stop_reason to tool_use + return { + ...(textBody as Record), + content: [ + ...((textBody as Record).content as object[]), + ...((toolBody as Record).content as object[]), + ], + stop_reason: bedrockStopReason(overrides?.finishReason, "tool_use"), + }; + }, + jsonOpts(servedPayloadLength(response, effReasoning)), ); - // Merge: take the text response as base, append tool_use blocks, set stop_reason to tool_use - const merged = { - ...(textBody as Record), - content: [ - ...((textBody as Record).content as object[]), - ...((toolBody as Record).content as object[]), - ], - stop_reason: bedrockStopReason(overrides?.finishReason, "tool_use"), - }; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(merged)); return; } @@ -747,21 +774,21 @@ export async function handleBedrock( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, response: { status: 200, fixture }, }); - const body = buildBedrockTextResponse( - response.content, - completionReq.model, - effReasoning, - overrides, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildBedrockTextResponse(response.content, completionReq.model, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); return; } @@ -777,7 +804,7 @@ export async function handleBedrock( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); - addResponseEntry({ + const journalEntry = addResponseEntry({ method: req.method ?? "POST", path: urlPath, headers: flattenHeaders(req.headers), @@ -785,15 +812,20 @@ export async function handleBedrock( response: { status: 200, fixture }, }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Bedrock InvokeModel"); - const body = buildBedrockToolCallResponse( - functionToolCalls, - completionReq.model, - logger, - effReasoning, - overrides, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildBedrockToolCallResponse( + functionToolCalls, + completionReq.model, + logger, + effReasoning, + overrides, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); return; } @@ -819,20 +851,11 @@ export async function handleBedrock( const BEDROCK_INVOKE_STREAM_EVENT_TYPE = "chunk"; /** Serialize Claude events as InvokeModel PayloadPart, leaving generic AWS framing unchanged. */ -function writeBedrockInvokeStream( - res: http.ServerResponse, - events: Parameters[1], - options: Parameters[2], -): Promise { - return writeEventStream( - res, - events.map((event) => ({ - eventType: event.eventType, - payload: { bytes: Buffer.from(JSON.stringify(event.payload), "utf8").toString("base64") }, - })), - options, - ); -} +const bedrockInvokeCodec = eventStreamCodec<{ eventType: string; payload: object }>((event) => + encodeEventStreamMessage(event.eventType, { + bytes: Buffer.from(JSON.stringify(event.payload), "utf8").toString("base64"), + }), +); function buildBedrockInvokeMessageStart( model: string, @@ -1456,15 +1479,8 @@ export async function handleBedrockStream( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "bedrock", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "bedrock", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1473,7 +1489,12 @@ export async function handleBedrockStream( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1539,8 +1560,13 @@ export async function handleBedrockStream( writeErrorResponse(res, misbehavior.status, renderedError.body); return; } - const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const streamTiming = { + latency: fixture.latency ?? defaults.latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }; if (misbehavior.kind === "applied") { const prepared = prepareClaudeMisbehavior(misbehavior, { @@ -1564,24 +1590,7 @@ export async function handleBedrockStream( eventType: BEDROCK_INVOKE_STREAM_EVENT_TYPE, payload: event, })); - const interruption = createInterruptionSignal(fixture); - try { - const completed = await writeBedrockInvokeStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - } finally { - interruption?.cleanup(); - } + await deliverStream(res, entry, fixture, bedrockInvokeCodec, events, streamTiming); return; } @@ -1633,21 +1642,7 @@ export async function handleBedrockStream( overrides, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeBedrockInvokeStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, bedrockInvokeCodec, events, streamTiming); return; } @@ -1677,21 +1672,7 @@ export async function handleBedrockStream( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeBedrockInvokeStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, bedrockInvokeCodec, events, streamTiming); return; } @@ -1723,21 +1704,7 @@ export async function handleBedrockStream( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeBedrockInvokeStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, bedrockInvokeCodec, events, streamTiming); return; } diff --git a/src/cohere.ts b/src/cohere.ts index 3b24e6df..3ae5afd8 100644 --- a/src/cohere.ts +++ b/src/cohere.ts @@ -16,9 +16,7 @@ import type { Fixture, FixtureBlock, HandlerDefaults, - RecordedTimings, ResponseOverrides, - StreamingProfile, ToolCall, ToolDefinition, } from "./types.js"; @@ -49,8 +47,10 @@ import { servedToolArgs, } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; -import { writeErrorResponse, delay, calculateDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { writeErrorResponse, typedSseCodec } from "./sse-writer.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { servedPayloadLength } from "./response/timing.js"; import type { Journal } from "./journal.js"; import type { Logger } from "./logger.js"; import { applyChaosAsync } from "./chaos.js"; @@ -819,55 +819,6 @@ function buildCohereContentWithToolCallsStreamEvents( return events; } -// ─── SSE writer for Cohere typed events ───────────────────────────────────── - -interface CohereStreamOptions { - terminalSentinel?: boolean; - latency?: number; - streamingProfile?: StreamingProfile; - recordedTimings?: RecordedTimings; - replaySpeed?: number; - signal?: AbortSignal; - onChunkSent?: () => void; -} - -async function writeCohereSSEStream( - res: http.ServerResponse, - events: CohereSSEEvent[], - optionsOrLatency?: number | CohereStreamOptions, -): Promise { - const opts: CohereStreamOptions = - typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const event of events) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal?.aborted) return false; - if (res.writableEnded) return true; - res.write(`event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } - - if (!res.writableEnded) { - if (opts.terminalSentinel && !signal?.aborted) res.write("data: [DONE]\n\n"); - res.end(); - } - return true; -} - // ─── Request handler ──────────────────────────────────────────────────────── export async function handleCohere( @@ -1052,15 +1003,17 @@ export async function handleCohere( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "cohere", - req.url ?? "/v2/chat", - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + completionReq, + "cohere", + req.url ?? "/v2/chat", + fixtures, + defaults, + raw, + ), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1069,7 +1022,12 @@ export async function handleCohere( path: req.url ?? "/v2/chat", headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1160,6 +1118,25 @@ export async function handleCohere( const empty = prepared?.summary.fault === "empty-response"; const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + const codec = typedSseCodec({ terminalSentinel: observedK5Stream }); + // Cohere has no separate misbehavior reply: an applied plan is merged into + // `response` above and served by the branches below, so every non-streaming + // site sizes its wait from the served payload and carries the K4 flag. + const jsonOpts = (payloadLength: number) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId: !!prepared?.duplicateId, + replaySpeed, + }); // Error response if (isErrorResponse(response)) { @@ -1213,16 +1190,21 @@ export async function handleCohere( }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Cohere"); if (cohereReq.stream !== true) { - const body = buildCohereContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - logger, - effReasoning, - overrides, - response.blocks, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildCohereContentWithToolCallsResponse( + response.content ?? "", + functionToolCalls, + logger, + effReasoning, + overrides, + response.blocks, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = buildCohereContentWithToolCallsStreamEvents( response.content ?? "", @@ -1233,22 +1215,7 @@ export async function handleCohere( overrides, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeCohereSSEStream(res, events, { - terminalSentinel: observedK5Stream, - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, codec, events, streamTiming); } return; } @@ -1282,20 +1249,25 @@ export async function handleCohere( testId, }); if (cohereReq.stream !== true) { - const body = empty - ? { - ...buildCohereTextResponse("", undefined, overrides), - message: { - role: "assistant", - content: [], - tool_calls: [], - tool_plan: "", - citations: [], - }, - } - : buildCohereTextResponse(response.content, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => + empty + ? { + ...buildCohereTextResponse("", undefined, overrides), + message: { + role: "assistant", + content: [], + tool_calls: [], + tool_plan: "", + citations: [], + }, + } + : buildCohereTextResponse(response.content, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const events = buildCohereTextStreamEvents( response.content, @@ -1303,22 +1275,7 @@ export async function handleCohere( effReasoning, overrides, ).filter((event) => !empty || event.type === "message-start" || event.type === "message-end"); - const interruption = createInterruptionSignal(fixture); - const completed = await writeCohereSSEStream(res, events, { - terminalSentinel: observedK5Stream, - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, codec, events, streamTiming); } return; } @@ -1353,9 +1310,13 @@ export async function handleCohere( }); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Cohere"); if (cohereReq.stream !== true) { - const body = buildCohereToolCallResponse(functionToolCalls, logger, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildCohereToolCallResponse(functionToolCalls, logger, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const events = buildCohereToolCallStreamEvents( functionToolCalls, @@ -1364,22 +1325,7 @@ export async function handleCohere( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeCohereSSEStream(res, events, { - terminalSentinel: observedK5Stream, - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, codec, events, streamTiming); } return; } diff --git a/src/fixture-loader.ts b/src/fixture-loader.ts index 2ee7b9e2..b9707db6 100644 --- a/src/fixture-loader.ts +++ b/src/fixture-loader.ts @@ -187,6 +187,9 @@ export function entryToFixture( ...(entry.nonStreamingTiming !== undefined && { nonStreamingTiming: entry.nonStreamingTiming, }), + ...(entry.nonStreamingTimingAllProviders !== undefined && { + nonStreamingTimingAllProviders: entry.nonStreamingTimingAllProviders, + }), ...(entry.metadata !== undefined && { metadata: entry.metadata }), }; @@ -1563,6 +1566,19 @@ export function validateFixtures( message: "disconnectAfterMs must be >= 0", }); } + // The non-streaming timing opt-ins are read as `=== true`, so a non-boolean + // from JSON (`"true"`, 1) silently does nothing. Warn, but do not reject: + // the loader passes the value through and rejecting it would be breaking. + for (const flag of ["nonStreamingTiming", "nonStreamingTimingAllProviders"] as const) { + const value: unknown = f[flag]; + if (value !== undefined && typeof value !== "boolean") { + results.push({ + severity: "warning", + fixtureIndex: i, + message: `${flag} must be a boolean, got ${typeof value} — the opt-in is ignored`, + }); + } + } if (f.streamingProfile !== undefined) { const sp = f.streamingProfile; if (sp.ttft !== undefined && sp.ttft < 0) { diff --git a/src/gemini-interactions.ts b/src/gemini-interactions.ts index fc7e8144..709bb623 100644 --- a/src/gemini-interactions.ts +++ b/src/gemini-interactions.ts @@ -43,8 +43,11 @@ import { resolveOpenAIChatMisbehaviorUsage, } from "./helpers.js"; import { matchFixtureDiagnostic } from "./router.js"; -import { writeErrorResponse, delay, calculateDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { writeErrorResponse, sseCodec } from "./sse-writer.js"; +import { httpSink, runFrameLoop, type FrameCodec } from "./response/frame-loop.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { misbehaviorPayloadLength, servedPayloadLength } from "./response/timing.js"; import type { Journal } from "./journal.js"; import type { Logger } from "./logger.js"; import { applyChaosAsync } from "./chaos.js"; @@ -768,6 +771,15 @@ export function buildInteractionsContentWithToolCallsSSEEvents( // ─── SSE writer for Interactions streaming ──────────────────────────────── +// Data-only SSE (no event: prefix, no [DONE]). Only step deltas count for the +// chunk index and truncateAfterChunks: framing events (interaction.created, +// step.start, step.stop, interaction.completed) pay the delay at the current +// index but do not consume chunk budget or trigger the chunk-sent callback. +const interactionsCodec: FrameCodec = { + ...sseCodec({ done: false }), + counts: (event) => event.event_type === "step.delta", +}; + interface InteractionsStreamOptions { latency?: number; streamingProfile?: StreamingProfile; @@ -784,39 +796,21 @@ export async function writeGeminiInteractionsSSEStream( ): Promise { const opts: InteractionsStreamOptions = typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const event of events) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal?.aborted) return false; - if (res.writableEnded) return true; - // Data-only SSE (no event: prefix, no [DONE]) - res.write(`data: ${JSON.stringify(event)}\n\n`); - // Only count step deltas for truncateAfterChunks — framing events - // (interaction.created, step.start, step.stop, interaction.completed) - // should not consume chunk budget or trigger the chunk-sent callback. - if (event.event_type === "step.delta") { - onChunkSent?.(); - chunkIndex++; - } - if (signal?.aborted) return false; - } - - if (!res.writableEnded) { - res.end(); - } - return true; + // No lifecycle: a caller gets close-awareness only through its own `signal`. + // With no events an aborted signal does not stop the writer (it still ends + // the response and returns true), so the loop gets no signal then. + return runFrameLoop( + httpSink(res), + events, + interactionsCodec, + { + latency: opts.latency, + streamingProfile: opts.streamingProfile, + recordedTimings: opts.recordedTimings, + replaySpeed: opts.replaySpeed, + }, + { signal: events.length > 0 ? opts.signal : undefined, onChunkSent: opts.onChunkSent }, + ); } // ─── Request handler ────────────────────────────────────────────────────── @@ -1019,15 +1013,17 @@ export async function handleGeminiInteractions( return; } if (decisions.upstream && defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "gemini-interactions", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + completionReq, + "gemini-interactions", + urlPath, + fixtures, + defaults, + raw, + ), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1040,6 +1036,7 @@ export async function handleGeminiInteractions( status: res.statusCode ?? 200, fixture: null, source: "proxy", + ...proxyInterruption(res, clientHungUp), }, }); return; @@ -1112,6 +1109,29 @@ export async function handleGeminiInteractions( const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + // Non-streaming replies size their (opt-in) wait from the served payload. + // An applied plan is merged into `response`, so every JSON site below is a + // misbehavior path when one applied. Interactions serves no fixture + // reasoning, so there is no effective reasoning to count. + const applied = misbehavior.kind === "applied" ? misbehavior : undefined; + const jsonOpts = () => ({ + surface: "other" as const, + payloadLength: applied + ? misbehaviorPayloadLength(applied, undefined) + : servedPayloadLength(response), + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId: !!applied?.duplicateId, + replaySpeed, + }); + const markIncomplete = (body: T): T | (T & { status: "incomplete" }) => + applied?.summary.fault === "stop-length-mid-tool" ? { ...body, status: "incomplete" } : body; // Error response if (isErrorResponse(response)) { @@ -1174,9 +1194,12 @@ export async function handleGeminiInteractions( isCombinedFixtureResponse(response) ? extractOverrides(response) : undefined; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end( - JSON.stringify({ + const reasoning = misbehavior.reasoning ?? ""; + await deliverJson( + res, + journalEntry, + fixture, + () => ({ id: interactionId, model: overrides?.model ?? model, status: "incomplete", @@ -1184,11 +1207,12 @@ export async function handleGeminiInteractions( { type: "thought", signature: "", - summary: [{ type: "text", text: misbehavior.reasoning ?? "" }], + summary: [{ type: "text", text: reasoning }], }, ], usage, }), + jsonOpts(), ); return; } @@ -1219,21 +1243,7 @@ export async function handleGeminiInteractions( event_id: nextEventId(), }, ]; - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiInteractionsSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, interactionsCodec, events, streamTiming); return; } @@ -1264,22 +1274,25 @@ export async function handleGeminiInteractions( const functionToolCalls = requireEmittedFunctionToolCalls(response, "Gemini Interactions"); const interactionId = nextInteractionId(); if (!streaming) { - const body = buildInteractionsContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - model, - interactionId, - logger, - overrides, - response.blocks, - ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end( - JSON.stringify( - misbehavior.kind === "applied" && misbehavior.summary.fault === "stop-length-mid-tool" - ? { ...body, status: "incomplete" } - : body, - ), + const content = response.content ?? ""; + const blocks = response.blocks; + await deliverJson( + res, + journalEntry, + fixture, + () => + markIncomplete( + buildInteractionsContentWithToolCallsResponse( + content, + functionToolCalls, + model, + interactionId, + logger, + overrides, + blocks, + ), + ), + jsonOpts(), ); } else { const events = buildInteractionsContentWithToolCallsSSEEvents( @@ -1291,25 +1304,14 @@ export async function handleGeminiInteractions( overrides, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiInteractionsSSEStream( + await deliverStream( res, + journalEntry, + fixture, + interactionsCodec, finalizeInteractionsMisbehavior(events, misbehavior), - { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }, + streamTiming, ); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); } return; } @@ -1340,14 +1342,14 @@ export async function handleGeminiInteractions( }); const interactionId = nextInteractionId(); if (!streaming) { - const body = buildInteractionsTextResponse(response.content, model, interactionId, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end( - JSON.stringify( - misbehavior.kind === "applied" && misbehavior.summary.fault === "stop-length-mid-tool" - ? { ...body, status: "incomplete" } - : body, - ), + const content = response.content; + await deliverJson( + res, + journalEntry, + fixture, + () => + markIncomplete(buildInteractionsTextResponse(content, model, interactionId, overrides)), + jsonOpts(), ); } else { const events = buildInteractionsTextSSEEvents( @@ -1356,25 +1358,14 @@ export async function handleGeminiInteractions( chunkSize, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiInteractionsSSEStream( + await deliverStream( res, + journalEntry, + fixture, + interactionsCodec, finalizeInteractionsMisbehavior(events, misbehavior), - { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }, + streamTiming, ); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); } return; } @@ -1406,20 +1397,21 @@ export async function handleGeminiInteractions( const functionToolCalls = requireEmittedFunctionToolCalls(response, "Gemini Interactions"); const interactionId = nextInteractionId(); if (!streaming) { - const body = buildInteractionsToolCallResponse( - functionToolCalls, - model, - interactionId, - logger, - overrides, - ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end( - JSON.stringify( - misbehavior.kind === "applied" && misbehavior.summary.fault === "stop-length-mid-tool" - ? { ...body, status: "incomplete" } - : body, - ), + await deliverJson( + res, + journalEntry, + fixture, + () => + markIncomplete( + buildInteractionsToolCallResponse( + functionToolCalls, + model, + interactionId, + logger, + overrides, + ), + ), + jsonOpts(), ); } else { const events = buildInteractionsToolCallSSEEvents( @@ -1428,25 +1420,14 @@ export async function handleGeminiInteractions( logger, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiInteractionsSSEStream( + await deliverStream( res, + journalEntry, + fixture, + interactionsCodec, finalizeInteractionsMisbehavior(events, misbehavior), - { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }, + streamTiming, ); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); } return; } diff --git a/src/gemini.ts b/src/gemini.ts index 73fd35b8..45458f53 100644 --- a/src/gemini.ts +++ b/src/gemini.ts @@ -16,10 +16,8 @@ import type { Fixture, FixtureBlock, HandlerDefaults, - RecordedTimings, RecordProviderKey, ResponseOverrides, - StreamingProfile, ToolCall, ToolDefinition, } from "./types.js"; @@ -50,14 +48,16 @@ import { surrogateSafeChunks, } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; -import { writeErrorResponse, delay, calculateDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { writeErrorResponse, sseCodec } from "./sse-writer.js"; import type { Journal } from "./journal.js"; -import type { Logger } from "./logger.js"; +import { Logger } from "./logger.js"; import { applyChaosAsync } from "./chaos.js"; import { proxyAndRecord } from "./recorder.js"; import { planMisbehavior, recordMisbehaviorOutcome, type MisbehaviorPlan } from "./misbehavior.js"; import { mapGeminiFinish } from "./response/finish.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { misbehaviorPayloadLength, servedPayloadLength } from "./response/timing.js"; import { render as renderGeminiUsage } from "./response/usage/codecs/gemini.js"; // ─── Gemini request types ─────────────────────────────────────────────────── @@ -984,50 +984,36 @@ function buildGeminiAudioStreamChunks( // ─── SSE writer for Gemini streaming ──────────────────────────────────────── -interface GeminiStreamOptions { - latency?: number; - streamingProfile?: StreamingProfile; - recordedTimings?: RecordedTimings; - replaySpeed?: number; - signal?: AbortSignal; - onChunkSent?: () => void; -} +// Gemini streams data-only SSE frames (no `event:` prefix) and sends no +// `[DONE]` trailer. +const GEMINI_SSE = sseCodec({ done: false }); -async function writeGeminiSSEStream( - res: http.ServerResponse, - chunks: GeminiResponseChunk[], - optionsOrLatency?: number | GeminiStreamOptions, -): Promise { - const opts: GeminiStreamOptions = - typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const chunk of chunks) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal?.aborted) return false; - if (res.writableEnded) return true; - // Gemini uses data-only SSE (no event: prefix, no [DONE]) - res.write(`data: ${JSON.stringify(chunk)}\n\n`); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } +const SILENT_LOGGER = new Logger("silent"); - if (!res.writableEnded) { - res.end(); +/** + * The reasoning a misbehavior reply's non-streaming wait is sized with: the + * plan response's reasoning through the same model and strict gate the + * builders apply. The gate runs with a silent logger, so the builder's own + * warning stays the only one. A reasoning-only plan carries its reasoning on + * the plan, and `misbehaviorPayloadLength` reads it there. + */ +function misbehaviorPayloadReasoning( + plan: MisbehaviorPlan, + model: string | undefined, + strict: boolean, +): string | undefined { + const response = plan.response; + if ( + plan.reasoning !== undefined || + !( + isCombinedFixtureResponse(response) || + isTextResponse(response) || + isToolCallResponse(response) + ) + ) { + return undefined; } - return true; + return resolveReasoningForModel(response.reasoning, model, strict, SILENT_LOGGER); } /** Prepare native fault output once; the journal describes these exact parts. */ @@ -1341,15 +1327,8 @@ export async function handleGemini( } if (decisions.upstream && defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - providerKey, - path, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, providerKey, path, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1358,7 +1337,12 @@ export async function handleGemini( path, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1425,6 +1409,20 @@ export async function handleGemini( const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed, + }); if (misbehavior.kind === "applied") { // Journal first, as the success branches do, so a fixture error thrown @@ -1437,34 +1435,36 @@ export async function handleGemini( body: completionReq, response: { status: 200, fixture }, }); + const strictReasoning = resolveRequestDecisions( + defaults.strict, + req.headers, + ).reasoningSuppression; const prepared = prepareGeminiMisbehavior( misbehavior, completionReq, streaming, chunkSize, logger, - resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, + strictReasoning, ); recordMisbehaviorOutcome({ entry, summary: prepared.summary, defaults, testId }); if (!streaming) { - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(prepared.chunks[0])); + const preparedBody = prepared.chunks[0]; + await deliverJson( + res, + entry, + fixture, + () => preparedBody, + jsonOpts( + misbehaviorPayloadLength( + misbehavior, + misbehaviorPayloadReasoning(misbehavior, completionReq.model, strictReasoning), + ), + !!misbehavior.duplicateId, + ), + ); } else { - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiSSEStream(res, prepared.chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, entry, fixture, GEMINI_SSE, prepared.chunks, streamTiming); } return; } @@ -1523,26 +1523,16 @@ export async function handleGemini( ? requireServedFunctionToolCalls(response, wire) : []; if (!streaming) { - const body = buildGeminiAudioResponse(response, audioToolCalls, logger, effReasoning); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildGeminiAudioResponse(response, audioToolCalls, logger, effReasoning), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const chunks = buildGeminiAudioStreamChunks(response, audioToolCalls, logger, effReasoning); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, GEMINI_SSE, chunks, streamTiming); } return; } @@ -1578,16 +1568,21 @@ export async function handleGemini( }); const functionToolCalls = requireEmittedFunctionToolCalls(response, wire); if (!streaming) { - const body = buildGeminiContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - logger, - effReasoning, - overrides, - response.blocks, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildGeminiContentWithToolCallsResponse( + response.content ?? "", + functionToolCalls, + logger, + effReasoning, + overrides, + response.blocks, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const chunks = buildGeminiContentWithToolCallsStreamChunks( response.content ?? "", @@ -1598,21 +1593,7 @@ export async function handleGemini( overrides, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, GEMINI_SSE, chunks, streamTiming); } return; } @@ -1647,9 +1628,13 @@ export async function handleGemini( testId, }); if (!streaming) { - const body = buildGeminiTextResponse(response.content, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildGeminiTextResponse(response.content, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const chunks = buildGeminiTextStreamChunks( response.content, @@ -1657,21 +1642,7 @@ export async function handleGemini( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, GEMINI_SSE, chunks, streamTiming); } return; } @@ -1707,9 +1678,13 @@ export async function handleGemini( }); const functionToolCalls = requireEmittedFunctionToolCalls(response, wire); if (!streaming) { - const body = buildGeminiToolCallResponse(functionToolCalls, logger, effReasoning, overrides); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildGeminiToolCallResponse(functionToolCalls, logger, effReasoning, overrides), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const chunks = buildGeminiToolCallStreamChunks( functionToolCalls, @@ -1718,21 +1693,7 @@ export async function handleGemini( effReasoning, overrides, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeGeminiSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, GEMINI_SSE, chunks, streamTiming); } return; } diff --git a/src/messages.ts b/src/messages.ts index d8d8219b..b4321133 100644 --- a/src/messages.ts +++ b/src/messages.ts @@ -16,10 +16,8 @@ import type { Fixture, FixtureResponse, HandlerDefaults, - RecordedTimings, FixtureBlock, ResponseOverrides, - StreamingProfile, ToolCall, ToolDefinition, } from "./types.js"; @@ -51,8 +49,10 @@ import { strictNoMatchLogLine, } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; -import { writeErrorResponse, delay, calculateDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { writeErrorResponse, typedSseCodec } from "./sse-writer.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { misbehaviorPayloadLength, servedPayloadLength } from "./response/timing.js"; import type { Journal } from "./journal.js"; import type { Logger } from "./logger.js"; import { applyChaosAsync } from "./chaos.js"; @@ -1225,6 +1225,8 @@ function buildClaudeContentWithToolCallsResponse( /** Internal Claude-family prepared output; the transport owns delivery and journaling. */ export type PreparedClaudeMisbehavior = { plan: MisbehaviorPlan; + /** The reasoning the reply serves: the fault's for reasoning-only, else the model-gated fixture reasoning. */ + reasoning: string | undefined; usage: { input_tokens: number; output_tokens: number }; } & ({ stream: true; events: ClaudeSSEEvent[] } | { stream: false; body: object }); @@ -1354,7 +1356,7 @@ export function prepareClaudeMisbehavior( stop_sequence: null, usage, }; - if (!context.stream) return { stream: false, body, plan: preparedPlan, usage }; + if (!context.stream) return { reasoning, stream: false, body, plan: preparedPlan, usage }; const events: ClaudeSSEEvent[] = [ { type: "message_start", message: { ...body, content: [], stop_reason: null } }, { @@ -1381,7 +1383,7 @@ export function prepareClaudeMisbehavior( }, { type: "message_stop" }, ); - return { stream: true, events, plan: preparedPlan, usage }; + return { reasoning, stream: true, events, plan: preparedPlan, usage }; } const empty = plan.summary.fault === "empty-response" || plan.summary.fault === "refusal"; if (empty) { @@ -1407,7 +1409,7 @@ export function prepareClaudeMisbehavior( ...detail, usage, }; - if (!context.stream) return { stream: false, body, plan: preparedPlan, usage }; + if (!context.stream) return { reasoning, stream: false, body, plan: preparedPlan, usage }; const events: ClaudeSSEEvent[] = [ { type: "message_start", @@ -1424,7 +1426,7 @@ export function prepareClaudeMisbehavior( }, { type: "message_stop" }, ]; - return { stream: true, events, plan: preparedPlan, usage }; + return { reasoning, stream: true, events, plan: preparedPlan, usage }; } if (context.stream) { const args = [ @@ -1462,6 +1464,7 @@ export function prepareClaudeMisbehavior( event.content_block.type === "tool_use", )[plan.target?.index ?? 0]?.index; return { + reasoning, stream: true, events: events.filter( (event) => !(event.type === "content_block_stop" && event.index === cutIndex), @@ -1470,7 +1473,7 @@ export function prepareClaudeMisbehavior( usage, }; } - return { stream: true, events, plan: preparedPlan, usage }; + return { reasoning, stream: true, events, plan: preparedPlan, usage }; } const args = [ reasoning, @@ -1490,75 +1493,7 @@ export function prepareClaudeMisbehavior( : isToolCallResponse(preparedResponse) ? buildClaudeToolCallResponse(toolCalls, model, context.logger, ...args) : buildClaudeTextResponse(content, model, ...args); - return { stream: false, body, plan: preparedPlan, usage }; -} - -// ─── SSE writer for Claude Messages API ───────────────────────────────────── - -interface ClaudeStreamOptions { - latency?: number; - streamingProfile?: StreamingProfile; - recordedTimings?: RecordedTimings; - replaySpeed?: number; - signal?: AbortSignal; - onChunkSent?: () => void; -} - -async function writeClaudeSSEStream( - res: http.ServerResponse, - events: ClaudeSSEEvent[], - optionsOrLatency?: number | ClaudeStreamOptions, -): Promise { - const opts: ClaudeStreamOptions = - typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - if (res.destroyed) return false; - - // Stop streaming when either the fixture's interruption signal fires or the - // client disconnects before the response finishes. Without the close - // listener, a dropped client leaves the loop sleeping through per-chunk - // latency and writing into a dead socket, and the journal records the - // request as complete. - const controller = new AbortController(); - const signal = controller.signal; - const abort = () => controller.abort(); - const onClose = () => { - if (!res.writableFinished) abort(); - }; - res.once("close", onClose); - opts.signal?.addEventListener("abort", abort, { once: true }); - if (opts.signal?.aborted) abort(); - - try { - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const event of events) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal.aborted || res.destroyed) return false; - if (res.writableEnded) return true; - res.write(`event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`); - onChunkSent?.(); - if (signal.aborted || res.destroyed) return false; - chunkIndex++; - } - - if (!res.writableEnded) { - res.end(); - } - return true; - } finally { - res.off("close", onClose); - opts.signal?.removeEventListener("abort", abort); - } + return { reasoning, stream: false, body, plan: preparedPlan, usage }; } // ─── Request handler ──────────────────────────────────────────────────────── @@ -1826,15 +1761,17 @@ export async function handleMessages( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "anthropic", - req.url ?? "/v1/messages", - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + completionReq, + "anthropic", + req.url ?? "/v1/messages", + fixtures, + defaults, + raw, + ), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1843,7 +1780,12 @@ export async function handleMessages( path: req.url ?? "/v1/messages", headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1907,6 +1849,21 @@ export async function handleMessages( } const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed, + }); if (misbehavior.kind === "applied") { // Journal first, as the success branches do, so a fixture error thrown @@ -1933,27 +1890,22 @@ export async function handleMessages( }); recordMisbehaviorOutcome({ entry, summary: prepared.plan.summary, defaults, testId }); if (!prepared.stream) { - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(prepared.body)); + const preparedBody = prepared.body; + await deliverJson( + res, + entry, + fixture, + () => preparedBody, + jsonOpts( + misbehaviorPayloadLength( + misbehavior, + misbehavior.reasoning === undefined ? prepared.reasoning : undefined, + ), + !!misbehavior.duplicateId, + ), + ); } else { - const interruption = createInterruptionSignal(fixture); - try { - const completed = await writeClaudeSSEStream(res, prepared.events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason() ?? "client aborted"; - } - } finally { - interruption?.cleanup(); - } + await deliverStream(res, entry, fixture, typedSseCodec(), prepared.events, streamTiming); } return; } @@ -2024,19 +1976,24 @@ export async function handleMessages( rejectGeneratedFiles(response, "Anthropic Messages"); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Anthropic Messages"); if (claudeReq.stream !== true) { - const body = buildClaudeContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - completionReq.model, - logger, - effReasoning, - overrides, - effReasoningSignature, - effRedactedThinking, - response.blocks, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildClaudeContentWithToolCallsResponse( + response.content ?? "", + functionToolCalls, + completionReq.model, + logger, + effReasoning, + overrides, + effReasoningSignature, + effRedactedThinking, + response.blocks, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = buildClaudeContentWithToolCallsStreamEvents( response.content ?? "", @@ -2050,21 +2007,7 @@ export async function handleMessages( effRedactedThinking, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeClaudeSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason() ?? "client aborted"; - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } @@ -2111,16 +2054,21 @@ export async function handleMessages( // Generated `files` have no Anthropic Messages projection: reject before any byte is written. rejectGeneratedFiles(response, "Anthropic Messages"); if (claudeReq.stream !== true) { - const body = buildClaudeTextResponse( - response.content, - completionReq.model, - effReasoning, - overrides, - effReasoningSignature, - effRedactedThinking, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildClaudeTextResponse( + response.content, + completionReq.model, + effReasoning, + overrides, + effReasoningSignature, + effRedactedThinking, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = buildClaudeTextStreamEvents( response.content, @@ -2131,21 +2079,7 @@ export async function handleMessages( effReasoningSignature, effRedactedThinking, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeClaudeSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason() ?? "client aborted"; - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } @@ -2193,17 +2127,22 @@ export async function handleMessages( rejectGeneratedFiles(response, "Anthropic Messages"); const functionToolCalls = requireEmittedFunctionToolCalls(response, "Anthropic Messages"); if (claudeReq.stream !== true) { - const body = buildClaudeToolCallResponse( - functionToolCalls, - completionReq.model, - logger, - effReasoning, - overrides, - effReasoningSignature, - effRedactedThinking, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildClaudeToolCallResponse( + functionToolCalls, + completionReq.model, + logger, + effReasoning, + overrides, + effReasoningSignature, + effRedactedThinking, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = buildClaudeToolCallStreamEvents( functionToolCalls, @@ -2215,21 +2154,7 @@ export async function handleMessages( effReasoningSignature, effRedactedThinking, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeClaudeSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason() ?? "client aborted"; - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } diff --git a/src/ndjson-writer.ts b/src/ndjson-writer.ts index ac6103a4..1fdc19d4 100644 --- a/src/ndjson-writer.ts +++ b/src/ndjson-writer.ts @@ -7,7 +7,7 @@ import type * as http from "node:http"; import type { StreamingProfile, RecordedTimings } from "./types.js"; -import { delay, calculateDelay } from "./sse-writer.js"; +import { httpSink, runFrameLoop, type FrameCodec } from "./response/frame-loop.js"; export interface NDJSONStreamOptions { latency?: number; @@ -24,33 +24,35 @@ export async function writeNDJSONStream( options?: NDJSONStreamOptions, ): Promise { const opts = options ?? {}; - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "application/x-ndjson"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const chunk of chunks) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) { - await delay(chunkDelay, signal); - } - if (signal?.aborted) return false; - if (res.writableEnded) return true; - res.write(JSON.stringify(chunk) + "\n"); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } + // Base contract (public API): with no frames, an aborted signal does not + // stop the writer; it still finishes the stream (any trailer, then end()) + // and returns true. The frame loop returns false there, so pass it no + // signal when there are no frames (it reads the signal only per frame and + // after the loop). + return runFrameLoop( + httpSink(res), + chunks, + ndjsonCodec(), + { + latency: opts.latency, + streamingProfile: opts.streamingProfile, + recordedTimings: opts.recordedTimings, + replaySpeed: opts.replaySpeed, + }, + { signal: chunks.length > 0 ? opts.signal : undefined, onChunkSent: opts.onChunkSent }, + ); +} - if (!res.writableEnded) { - res.end(); - } - return true; +// Internal (not part of the public API): newline-delimited JSON, one object +// per line, no trailer (Ollama). Line comments, not JSDoc, so the bundled +// .d.ts does not attach this comment to a public declaration. +export function ndjsonCodec(): FrameCodec { + return { + headers: { + "Content-Type": "application/x-ndjson", + "Cache-Control": "no-cache", + Connection: "keep-alive", + }, + encode: (frame) => JSON.stringify(frame) + "\n", + }; } diff --git a/src/ollama.ts b/src/ollama.ts index 59843167..fd7b0cfe 100644 --- a/src/ollama.ts +++ b/src/ollama.ts @@ -54,8 +54,14 @@ import { } from "./helpers.js"; import { matchFixtureDiagnostic, recordMatchOptions } from "./router.js"; import { writeErrorResponse } from "./sse-writer.js"; -import { writeNDJSONStream } from "./ndjson-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { ndjsonCodec } from "./ndjson-writer.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { + misbehaviorPayloadLength, + nonStreamingTimingEnabled, + servedPayloadLength, +} from "./response/timing.js"; import type { Journal } from "./journal.js"; import { applyChaosAsync } from "./chaos.js"; import { planMisbehavior, recordMisbehaviorOutcome, type MisbehaviorPlan } from "./misbehavior.js"; @@ -540,7 +546,8 @@ function prepareOllamaMisbehavior( ? [...buildOllamaChatTextChunks(priorContent, model, chunkSize).slice(0, -1), { error }] : [{ error }]; return { - chunks, + build: () => chunks, + reasoning: undefined, status: stream ? 200 : 500, summary: { ...plan.summary, servedToolCalls: [] }, }; @@ -553,48 +560,56 @@ function prepareOllamaMisbehavior( if (parsed.kind === "verbatim") throw new InvalidToolArgumentsError(call); return { name: call.name, arguments: parsed.text }; }); - let chunks: object[]; - let reasoning = ""; - if (plan.summary.fault === "reasoning-only") { - reasoning = plan.reasoning ?? ""; - const createdAt = new Date().toISOString(); - const base = { model, created_at: createdAt }; - if (stream) { - chunks = []; - for (let i = 0; i < reasoning.length; i += chunkSize) { + const reasoning = + plan.summary.fault === "reasoning-only" + ? (plan.reasoning ?? "") + : (resolveReasoningForModel( + "reasoning" in response ? response.reasoning : undefined, + model, + effectiveStrict, + logger, + ) ?? ""); + const completion = + content + reasoning + servedToolCalls.map((call) => call.name + call.arguments).join(""); + // Built on demand, so a timed non-streaming reply is built after its wait + // and its `created_at` is the write time. + const build = (): object[] => { + let chunks: object[]; + if (plan.summary.fault === "reasoning-only") { + const createdAt = new Date().toISOString(); + const base = { model, created_at: createdAt }; + if (stream) { + chunks = []; + for (let i = 0; i < reasoning.length; i += chunkSize) { + chunks.push({ + ...base, + message: { + role: "assistant", + content: "", + thinking: reasoning.slice(i, i + chunkSize), + }, + done: false, + }); + } chunks.push({ ...base, - message: { role: "assistant", content: "", thinking: reasoning.slice(i, i + chunkSize) }, - done: false, - }); - } - chunks.push({ - ...base, - message: { role: "assistant", content: "" }, - done: true, - ...DURATION_FIELDS, - done_reason: "length", - }); - } else { - chunks = [ - { - ...base, - message: { role: "assistant", content: "", thinking: reasoning }, + message: { role: "assistant", content: "" }, done: true, ...DURATION_FIELDS, done_reason: "length", - }, - ]; - } - } else { - reasoning = - resolveReasoningForModel( - "reasoning" in response ? response.reasoning : undefined, - model, - effectiveStrict, - logger, - ) ?? ""; - if (combined) { + }); + } else { + chunks = [ + { + ...base, + message: { role: "assistant", content: "", thinking: reasoning }, + done: true, + ...DURATION_FIELDS, + done_reason: "length", + }, + ]; + } + } else if (combined) { chunks = stream ? buildOllamaChatContentWithToolCallsChunks( content, @@ -622,15 +637,14 @@ function prepareOllamaMisbehavior( ? buildOllamaChatTextChunks(content, model, chunkSize, reasoning) : [buildOllamaChatTextResponse(content, model, reasoning)]; } - } - const completion = - content + reasoning + servedToolCalls.map((call) => call.name + call.arguments).join(""); - const terminal = chunks.length - 1; - chunks[terminal] = { - ...chunks[terminal], - ...renderUsage({ kind: "applied-chat-estimate", messages: request.messages, completion }), + const terminal = chunks.length - 1; + chunks[terminal] = { + ...chunks[terminal], + ...renderUsage({ kind: "applied-chat-estimate", messages: request.messages, completion }), + }; + return chunks; }; - return { chunks, status: 200, summary: { ...plan.summary, servedToolCalls } }; + return { build, reasoning, status: 200, summary: { ...plan.summary, servedToolCalls } }; } // ─── Response builders: /api/generate ──────────────────────────────────────── @@ -862,15 +876,8 @@ export async function handleOllama( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "ollama", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "ollama", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -879,7 +886,12 @@ export async function handleOllama( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -940,6 +952,21 @@ export async function handleOllama( } const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed, + }); // Ollama defaults to streaming when stream is absent or true const streaming = ollamaReq.stream !== false; @@ -953,6 +980,12 @@ export async function handleOllama( resolveRequestDecisions(defaults.strict, req.headers).reasoningSuppression, logger, ); + // A timed non-streaming reply is built after the wait (inside deliverJson, + // still before writeHead), so `created_at` is the write time. Every other + // reply is built here, before journaling, as before. + const timedBody = + !streaming && prepared.status === 200 && nonStreamingTimingEnabled(fixture, "other"); + const chunks = timedBody ? undefined : prepared.build(); const journalEntry = journal.add({ method: req.method ?? "POST", path: urlPath, @@ -961,25 +994,24 @@ export async function handleOllama( response: { status: prepared.status, fixture }, }); recordMisbehaviorOutcome({ entry: journalEntry, summary: prepared.summary, defaults, testId }); - if (!streaming) { + if (!streaming && prepared.status !== 200) { + // A modeled error reply (plan n1): written as before, never delayed. res.writeHead(prepared.status, { "Content-Type": "application/json" }); - res.end(JSON.stringify(prepared.chunks[0])); - } else { - const interruption = createInterruptionSignal(fixture); - const completed = await writeNDJSONStream(res, prepared.chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, + res.end(JSON.stringify((chunks ?? prepared.build())[0])); + } else if (!streaming) { + await deliverJson(res, journalEntry, fixture, () => (chunks ?? prepared.build())[0], { + ...jsonOpts( + misbehaviorPayloadLength( + misbehavior, + misbehavior.reasoning === undefined ? prepared.reasoning : undefined, + ), + !!misbehavior.duplicateId, + ), + status: prepared.status, }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + } else { + const frames = chunks ?? prepared.build(); + await deliverStream(res, journalEntry, fixture, ndjsonCodec(), frames, streamTiming); } return; } @@ -1037,15 +1069,20 @@ export async function handleOllama( logger, ); if (!streaming) { - const body = buildOllamaChatContentWithToolCallsResponse( - response.content ?? "", - functionToolCalls, - completionReq.model, - effReasoning, - response.blocks, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildOllamaChatContentWithToolCallsResponse( + response.content ?? "", + functionToolCalls, + completionReq.model, + effReasoning, + response.blocks, + ), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const chunks = buildOllamaChatContentWithToolCallsChunks( response.content ?? "", @@ -1055,21 +1092,7 @@ export async function handleOllama( effReasoning, response.blocks, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeNDJSONStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, ndjsonCodec(), chunks, streamTiming); } return; } @@ -1104,9 +1127,13 @@ export async function handleOllama( logger, ); if (!streaming) { - const body = buildOllamaChatTextResponse(response.content, completionReq.model, effReasoning); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); + await deliverJson( + res, + journalEntry, + fixture, + () => buildOllamaChatTextResponse(response.content, completionReq.model, effReasoning), + jsonOpts(servedPayloadLength(response, effReasoning)), + ); } else { const chunks = buildOllamaChatTextChunks( response.content, @@ -1114,21 +1141,7 @@ export async function handleOllama( chunkSize, effReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeNDJSONStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, ndjsonCodec(), chunks, streamTiming); } return; } @@ -1164,13 +1177,13 @@ export async function handleOllama( logger, ); if (!streaming) { - const body = buildOllamaChatToolCallResponse( - functionToolCalls, - completionReq.model, - effReasoning, + await deliverJson( + res, + journalEntry, + fixture, + () => buildOllamaChatToolCallResponse(functionToolCalls, completionReq.model, effReasoning), + jsonOpts(servedPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const chunks = buildOllamaChatToolCallChunks( functionToolCalls, @@ -1178,21 +1191,7 @@ export async function handleOllama( chunkSize, effReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeNDJSONStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, ndjsonCodec(), chunks, streamTiming); } return; } @@ -1370,15 +1369,8 @@ export async function handleOllamaGenerate( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "ollama", - urlPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord(req, res, completionReq, "ollama", urlPath, fixtures, defaults, raw), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -1387,7 +1379,12 @@ export async function handleOllamaGenerate( path: urlPath, headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -1449,6 +1446,21 @@ export async function handleOllamaGenerate( } const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); + const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed, + }; + const jsonOpts = (payloadLength: number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId, + replaySpeed, + }); // Ollama defaults to streaming when stream is absent or true const streaming = generateReq.stream !== false; @@ -1502,13 +1514,15 @@ export async function handleOllamaGenerate( defaults.logger, ); if (!streaming) { - const body = buildOllamaGenerateTextResponse( - response.content, - completionReq.model, - effReasoning, + await deliverJson( + res, + journalEntry, + fixture, + () => buildOllamaGenerateTextResponse(response.content, completionReq.model, effReasoning), + // /api/generate serves only the text content and never reads blocks, + // so the wait is sized from that text, not from the fixture's blocks. + jsonOpts(servedPayloadLength({ content: response.content }, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const chunks = buildOllamaGenerateTextChunks( response.content, @@ -1516,21 +1530,7 @@ export async function handleOllamaGenerate( chunkSize, effReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeNDJSONStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixture.recordedTimings, - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, ndjsonCodec(), chunks, streamTiming); } return; } diff --git a/src/response/deliver.ts b/src/response/deliver.ts new file mode 100644 index 00000000..b48d2634 --- /dev/null +++ b/src/response/deliver.ts @@ -0,0 +1,234 @@ +import type * as http from "node:http"; +import type { Fixture, JournalEntry, RecordedTimings } from "../types.js"; +import { delay } from "../sse-writer.js"; +import { httpSink, runFrameLoop, type FrameCodec, type FrameTiming } from "./frame-loop.js"; +import { responseGoneReason } from "../chaos.js"; +import { + AIMOCK_DISCONNECTED, + closedByClient, + httpLifecycle, + markInterrupted, + noteDeliveryStart, + statusLineSent, + teardownReason, + requestSocket, + type DeliveryLifecycle, +} from "./lifecycle.js"; +import { + extendRecordedTimings, + nonStreamingDelayMs, + nonStreamingRecordedTimings, + nonStreamingTimingEnabled, + type NonStreamingSurface, +} from "./timing.js"; + +/** The lifecycle fields the teardown journal reads. */ +type TeardownSource = Pick; + +/** + * Journal a delivery that ended early. The reason is always set: the + * lifecycle's first cause when it fired, else {@link teardownReason} for the + * response that is gone. When no byte of the response reached the socket + * ({@link statusLineSent}), the status is 0, the rule the server and the + * chaos `disconnect` apply to a torn socket. A fixture interruption + * (`disconnectAfterMs`, `truncateAfterChunks`) keeps the status it was + * journaled with, as base recorded it. + */ +function journalTeardown( + entry: JournalEntry, + res: http.ServerResponse, + lifecycle: TeardownSource | undefined, +): void { + markInterrupted(entry, lifecycle?.reason() ?? teardownReason(res)); + if (lifecycle?.cause() !== "fixture" && !statusLineSent(res)) entry.response.status = 0; +} + +/** + * Whether the response was torn down: the lifecycle fired, the response is + * destroyed or chaos disconnected it, or the client closed the socket. + */ +function tornDown(res: http.ServerResponse, lifecycle: TeardownSource | undefined): boolean { + const socket = requestSocket(res); + const gone = responseGoneReason(res); + return ( + lifecycle?.reason() !== undefined || + gone === "client-gone" || + gone === "chaos-disconnected" || + (!res.writableEnded && !!socket && closedByClient(socket)) + ); +} + +/** + * Journal a response that something else on aimock's side already answered + * (head committed or body ended) as aimock's teardown, with the status the + * client actually got. + */ +function journalAnsweredElsewhere(entry: JournalEntry, res: http.ServerResponse): void { + markInterrupted(entry, AIMOCK_DISCONNECTED); + entry.response.status = statusLineSent(res) ? res.statusCode : 0; +} + +/** + * Whether `deliverJson` can no longer write its reply, journaling why when it + * cannot. Torn down ({@link tornDown}): {@link journalTeardown}. Already + * answered by something else on aimock's side (head committed or body + * ended): writing again would throw `ERR_HTTP_HEADERS_SENT`, so it is + * journaled by {@link journalAnsweredElsewhere}, and a body that never ended + * is destroyed so the client does not hang. + */ +function journalUnwritable( + entry: JournalEntry, + res: http.ServerResponse, + lifecycle: TeardownSource | undefined, +): boolean { + if (tornDown(res, lifecycle)) { + journalTeardown(entry, res, lifecycle); + return true; + } + if (res.headersSent || res.writableEnded) { + journalAnsweredElsewhere(entry, res); + if (!res.writableEnded) res.destroy(); + return true; + } + return false; +} + +/** + * Deliver one streamed reply and journal how it ended. Builds the HTTP + * lifecycle (client close + fixture interruption) and runs the frame loop on + * it. When the loop stops early it destroys `res` (if it has not ended) and + * marks the entry with the first cause's reason (or, when no cause fired, who + * tore the response down). A response other code already answered (head + * committed before the stream, or body ended) is journaled as + * {@link deliverJson} journals it. It disposes the lifecycle on every path. + */ +export async function deliverStream( + res: http.ServerResponse, + entry: JournalEntry, + fixture: Fixture, + codec: FrameCodec, + frames: readonly F[], + timing: FrameTiming, + opts?: { + status?: number; // When set, sent with `writeHead` before the first frame (issue #549). + headers?: Record; // Merged over the codec's headers (issue #549). + }, +): Promise { + noteDeliveryStart(res); + const headSentBefore = res.headersSent; + const lifecycle = httpLifecycle(res, fixture, { streaming: true }); + try { + const completed = await runFrameLoop(httpSink(res), frames, codec, timing, { + lifecycle, + onChunkSent: lifecycle.tick, + status: opts?.status, + headers: opts?.headers, + }); + if (!completed) { + // Read before the destroy below, which makes any response look torn down. + const answeredElsewhere = (headSentBefore || res.writableEnded) && !tornDown(res, lifecycle); + if (!res.writableEnded) res.destroy(); + if (answeredElsewhere) journalAnsweredElsewhere(entry, res); + else journalTeardown(entry, res, lifecycle); + } + } finally { + lifecycle.dispose(); + } +} + +export interface DeliverJsonOptions { + /** + * Passed to `nonStreamingTimingEnabled` and `nonStreamingRecordedTimings`: + * OpenAI chat, or any other provider. + */ + surface: NonStreamingSurface; + status?: number; // The response status, 200 when unset (issue #549). + headers?: Record; // Merged over the JSON `Content-Type` header (issue #549). + /** The served payload length the wait is sized from. */ + payloadLength: number; + /** + * The wait is sized in chunks of this many characters. Callers pass + * `Math.max(1, )`: 1 or more, or `NaN` for a + * non-numeric value. `NaN` gives a zero wait. + */ + chunkSize: number; + recordedTimings?: RecordedTimings; + /** A K4 duplicate-id plan: extend the recorded gaps before sizing the wait. */ + duplicateId?: boolean; + replaySpeed?: number; +} + +/** + * Deliver one non-streaming JSON reply. The body is a thunk. + * + * On both paths it first checks {@link journalUnwritable}, before any wait. A + * response that can no longer take the reply (torn down, or already answered) + * is not written, the thunk is never called, and the entry is marked + * interrupted. An already-answered response whose body never ended is + * destroyed, so the client does not hang. + * + * With non-streaming timing off (the default) it then runs + * `JSON.stringify(body())`, then `writeHead`, then `end`: the body is built + * before anything is written, so a builder that throws leaves the response + * unwritten. + * + * With timing on, it waits the non-streaming delay on the client's close + * signal, checks {@link journalUnwritable} again (the response can be torn + * down or answered during the wait), then runs the thunk, so a `created` + * timestamp reflects the write time. + * + * Rejects with a `RangeError` when `chunkSize` is a number below 1: that is + * a programming error, and the delay math divides by it. No user input reaches + * it, because every caller clamps with `Math.max(1, ...)`. A user value that + * is not a number clamps to `NaN`: the reply is still served, with a zero + * wait when timing is on. + */ +export async function deliverJson( + res: http.ServerResponse, + entry: JournalEntry, + fixture: Fixture, + body: () => unknown, + opts: DeliverJsonOptions, +): Promise { + if (opts.chunkSize < 1) { + throw new RangeError(`deliverJson: chunkSize must be >= 1, got ${opts.chunkSize}`); + } + const write = (): void => { + const text = JSON.stringify(body()); + res.writeHead(opts.status ?? 200, { "Content-Type": "application/json", ...opts.headers }); + res.end(text); + }; + noteDeliveryStart(res); + // Checked before the lifecycle and the wait on both paths: a response that + // cannot take the reply is journaled at once, with no wait. + if (journalUnwritable(entry, res, undefined)) return; + if (!nonStreamingTimingEnabled(fixture, opts.surface)) { + write(); + return; + } + const lifecycle = httpLifecycle(res, fixture, { streaming: false }); + try { + const fixtureTimings = nonStreamingRecordedTimings(fixture, opts.surface, opts.recordedTimings); + const recordedTimings = opts.duplicateId + ? extendRecordedTimings( + fixtureTimings, + Math.max(1, Math.ceil(opts.payloadLength / opts.chunkSize)), + true, + ) + : fixtureTimings; + await delay( + nonStreamingDelayMs( + fixture, + opts.payloadLength, + opts.chunkSize, + recordedTimings, + opts.replaySpeed, + ), + lifecycle.signal, + ); + if (journalUnwritable(entry, res, lifecycle)) return; + write(); + } finally { + lifecycle.dispose(); + } +} diff --git a/src/response/frame-loop.ts b/src/response/frame-loop.ts new file mode 100644 index 00000000..ddfe85e2 --- /dev/null +++ b/src/response/frame-loop.ts @@ -0,0 +1,213 @@ +import type * as http from "node:http"; +import type { RecordedTimings, StreamingProfile } from "../types.js"; +import type { WebSocketConnection } from "../ws-framing.js"; +import { calculateDelay, delay } from "../sse-writer.js"; +import type { DeliveryLifecycle } from "./lifecycle.js"; + +/** + * How one wire frames a stream. The frame loop is the same for every wire; + * only these parts differ. + */ +export interface FrameCodec { + /** Response headers, set before the first write (skipped if the response already ended). */ + headers: Record; + /** Written once before the first frame, when it returns a value (the OpenRouter keepalive in `sseCodec`). */ + prelude?: () => string | Buffer | undefined; + /** The bytes for one frame. */ + encode(frame: F): string | Buffer; + /** + * Whether a frame advances the chunk index and the fixture tick (default + * true). A frame that does not count still pays the delay at the current + * index. + */ + counts?(frame: F): boolean; + /** + * Written after the last frame, before `end()`, when the response has not + * ended. Only a completed delivery reaches it: an aborted one returns `false` + * with no trailer and no `end()`, so `aborted` is always `false` here. + */ + trailer?(aborted: boolean): Array; + /** Send the head with `writeHead` up front instead of `setHeader`. */ + writeHeadImmediately?: boolean; + /** + * Check `sink.gone()` before each delay as well as after it, with or + * without a lifecycle, and return `false` when gone. + */ + checkGoneBeforeDelay?: boolean; +} + +/** + * Where frames go: an HTTP response or a WebSocket connection. A sink with no + * HTTP head (WebSocket) omits `setHeader` and `writeHead`; asking such a sink + * for a status (`status` or `writeHeadImmediately`) is a programming error. + */ +export interface FrameSink { + gone(): boolean; + writableEnded(): boolean; + write(chunk: string | Buffer): void; + end(): void; + /** Whether the head was already committed (HTTP only). */ + headersSent?(): boolean; + setHeader?(name: string, value: string): void; + writeHead?(status: number, headers: Record): void; +} + +/** Per-frame delay inputs, passed straight to `calculateDelay`. */ +export interface FrameTiming { + latency?: number; + streamingProfile?: StreamingProfile; + recordedTimings?: RecordedTimings; + replaySpeed?: number; +} + +interface FrameLoopCommonOptions { + onChunkSent?: () => void; + /** When given, `writeHead(status, headers)` is sent before the first frame. */ + status?: number; + /** Extra headers, merged over the codec's headers. */ + headers?: Record; +} + +/** + * The stop source is a `lifecycle` or a bare `signal`, never both: passing both + * is a programming error and rejects with a `TypeError`. + */ +export type FrameLoopOptions = FrameLoopCommonOptions & + ( + | { + /** + * Its signal stops the loop and `sink.gone()` is checked next to every + * abort check. + */ + lifecycle: DeliveryLifecycle; + signal?: never; + } + | { + lifecycle?: never; + /** + * Without a lifecycle, `signal` stops the loop, and a gone sink stops it + * only when the codec sets `checkGoneBeforeDelay`. + */ + signal?: AbortSignal; + } + ); + +/** An HTTP response as a frame sink. Gone means destroyed. */ +export function httpSink(res: http.ServerResponse): FrameSink { + return { + gone: () => res.destroyed, + writableEnded: () => res.writableEnded, + headersSent: () => res.headersSent, + write: (chunk) => { + res.write(chunk); + }, + end: () => { + res.end(); + }, + setHeader: (name, value) => { + res.setHeader(name, value); + }, + writeHead: (status, headers) => { + res.writeHead(status, headers); + }, + }; +} + +/** A WebSocket connection as a frame sink. Gone means closed; it never ends. */ +export function wsSink(ws: WebSocketConnection): FrameSink { + return { + gone: () => ws.isClosed, + writableEnded: () => false, + write: (chunk) => { + ws.send(typeof chunk === "string" ? chunk : chunk.toString("utf8")); + }, + end: () => {}, + }; +} + +/** + * The shared streaming loop. Per frame, in order: gone check (with + * `checkGoneBeforeDelay`) → `calculateDelay` → `delay(d, signal)` → abort + * check → gone check (with `checkGoneBeforeDelay`) → `writableEnded` (return + * `true`) → write → advance the index and tick (counting frames only) → abort + * re-check. After the last frame the trailer and `end()` run if the response + * has not ended. Returns `false` when stopped by an abort, or by a gone sink + * (with a lifecycle or `checkGoneBeforeDelay`), including an abort before the + * first frame or with no frames, and `true` otherwise. Rejects with a + * `TypeError` when given both a lifecycle and a signal, or a status for a sink + * without `writeHead`. + * + * With a lifecycle, a response that other code already answered is not a + * completed delivery: an ended response (at entry or at the `writableEnded` + * check) returns `false`, and so does a head already committed at entry, + * which `setHeader`/`writeHead` would reject with `ERR_HTTP_HEADERS_SENT`. + * Without a lifecycle (the exported writers) an ended response still returns + * `true`, as base did. + */ +export async function runFrameLoop( + sink: FrameSink, + frames: readonly F[], + codec: FrameCodec, + timing: FrameTiming, + opts: FrameLoopOptions = {}, +): Promise { + const { lifecycle } = opts; + if (lifecycle !== undefined && opts.signal !== undefined) { + throw new TypeError("runFrameLoop: pass a lifecycle or a signal, not both"); + } + const signal = lifecycle?.signal ?? opts.signal; + const stopped = (): boolean => + signal?.aborted === true || (lifecycle !== undefined && sink.gone()); + + // With a lifecycle, a response that other code answered is not a completed delivery. + const endedIsCompleted = lifecycle === undefined; + if (sink.writableEnded()) return endedIsCompleted; + if (lifecycle && sink.gone()) return false; + if (lifecycle && sink.headersSent?.() === true) return false; + + const headers = + opts.headers === undefined ? codec.headers : { ...codec.headers, ...opts.headers }; + if (opts.status !== undefined || codec.writeHeadImmediately) { + if (!sink.writeHead) { + throw new TypeError("runFrameLoop: a status was requested but the sink has no writeHead"); + } + sink.writeHead(opts.status ?? 200, headers); + } else if (sink.setHeader) { + for (const [name, value] of Object.entries(headers)) sink.setHeader(name, value); + } + + if (codec.prelude && !sink.writableEnded()) { + const prelude = codec.prelude(); + if (prelude !== undefined) sink.write(prelude); + } + + let chunkIndex = 0; + for (const frame of frames) { + if (codec.checkGoneBeforeDelay && sink.gone()) return false; + const chunkDelay = calculateDelay( + chunkIndex, + timing.streamingProfile, + timing.latency ?? 0, + timing.recordedTimings, + timing.replaySpeed, + ); + if (chunkDelay > 0) await delay(chunkDelay, signal); + if (stopped()) return false; + if (codec.checkGoneBeforeDelay && sink.gone()) return false; + if (sink.writableEnded()) return endedIsCompleted; + sink.write(codec.encode(frame)); + if (codec.counts?.(frame) !== false) { + chunkIndex++; + opts.onChunkSent?.(); + } + if (stopped()) return false; + } + + // An abort the loop never observed (no frames) still stops the delivery. + if (stopped()) return false; + if (!sink.writableEnded()) { + for (const part of codec.trailer?.(false) ?? []) sink.write(part); + sink.end(); + } + return true; +} diff --git a/src/response/lifecycle.ts b/src/response/lifecycle.ts new file mode 100644 index 00000000..0095234d --- /dev/null +++ b/src/response/lifecycle.ts @@ -0,0 +1,301 @@ +import type * as http from "node:http"; +import type { Fixture, JournalEntry } from "../types.js"; +import type { WebSocketConnection } from "../ws-framing.js"; +import { createInterruptionSignal, type InterruptionControl } from "../interruption.js"; +import { responseGoneReason } from "../chaos.js"; + +/** The journal `interruptReason` for a client that left before the reply finished. */ +export const CLIENT_ABORTED = "client aborted"; + +/** + * The journal `interruptReason` for a response aimock tore down itself: the + * chaos `disconnect` action or a server-side `destroy`. It is the name the + * chaos journal entry records (`chaosAction: "disconnect"`), so a teardown + * aimock caused is never blamed on the client. + */ +export const AIMOCK_DISCONNECTED = "disconnect"; + +/** + * Which side stopped a delivery first: the client, the fixture's own + * interruption, or aimock tearing the response down (`"server"`). + */ +export type InterruptCause = "client" | "fixture" | "server"; + +/** + * One delivery's stop signal. It merges the fixture interruption + * (`truncateAfterChunks` / `disconnectAfterMs`) with the client going away + * and with aimock tearing the response down (`"server"`). + * The first cause wins: a later cause never reclassifies it. + */ +export interface DeliveryLifecycle { + /** Aborts on the first cause: fixture, client, or server. */ + readonly signal: AbortSignal; + /** The cause that aborted `signal`, or `undefined` while it has not aborted. */ + cause(): InterruptCause | undefined; + /** + * The fixture's reason, {@link CLIENT_ABORTED}, {@link AIMOCK_DISCONNECTED}, + * or `undefined` while running. + */ + reason(): string | undefined; + /** Forwards a sent chunk to the fixture interruption (counts toward `truncateAfterChunks`). */ + tick(): void; + /** Removes the close and fixture-abort listeners and clears the fixture's `disconnectAfterMs` timer. */ + dispose(): void; +} + +/** The part of an EventEmitter a close source may expose. Test doubles may carry less. */ +interface CloseSource { + once?: (event: "close", listener: () => void) => unknown; + off?: (event: "close", listener: () => void) => unknown; + removeListener?: (event: "close", listener: () => void) => unknown; +} + +/** + * Subscribe to `close`, guarded the same way as `awaitChaosLatency`: a partial + * test double without `once` is not subscribed, and only a listener that was + * added is ever removed, through a remover that exists. + */ +function subscribeClose(source: CloseSource, onClose: () => void): (() => void) | undefined { + if (typeof source.once !== "function") return undefined; + source.once("close", onClose); + if (typeof source.off === "function") return () => source.off?.("close", onClose); + if (typeof source.removeListener === "function") + return () => source.removeListener?.("close", onClose); + return undefined; +} + +function createLifecycle( + interruption: InterruptionControl | null, + goneAtStart: InterruptCause | undefined, + subscribe: (onGone: (cause: InterruptCause) => void) => (() => void) | undefined, +): DeliveryLifecycle { + const controller = new AbortController(); + let firstCause: InterruptCause | undefined; + const abortWith = (cause: InterruptCause): void => { + if (controller.signal.aborted) return; + firstCause = cause; + controller.abort(); + }; + const onFixtureAbort = (): void => abortWith("fixture"); + interruption?.signal.addEventListener("abort", onFixtureAbort, { once: true }); + let unsubscribe: (() => void) | undefined; + if (goneAtStart !== undefined) abortWith(goneAtStart); + else unsubscribe = subscribe(abortWith); + return { + signal: controller.signal, + cause: () => firstCause, + reason: () => { + if (firstCause === "fixture") return interruption?.reason(); + if (firstCause === "client") return CLIENT_ABORTED; + if (firstCause === "server") return AIMOCK_DISCONNECTED; + return undefined; + }, + tick: () => { + interruption?.tick(); + }, + dispose: () => { + unsubscribe?.(); + unsubscribe = undefined; + interruption?.signal.removeEventListener("abort", onFixtureAbort); + interruption?.cleanup(); + }, + }; +} + +/** The request socket fields the teardown rule and the status-line rule read. */ +export interface PeerSocket { + readonly errored?: Error | null; + readonly writableEnded?: boolean; + readonly bytesWritten?: number; +} + +/** Where a response's request socket can be read from. Test doubles may carry neither. */ +interface SocketOwner { + readonly req?: { readonly socket?: PeerSocket | null }; + readonly socket?: PeerSocket | null; +} + +/** The request socket of `res`, or `undefined` when it carries none. */ +export function requestSocket(res: http.ServerResponse): PeerSocket | undefined { + const owner: SocketOwner = res; + return owner.req?.socket ?? owner.socket ?? undefined; +} + +/** + * THE client hang-up rule, shared by the delivery lifecycle and the proxy + * relay: the client's socket closed or reset. + * + * - A reset, or a write into a socket the peer closed, leaves `errored` set + * (ECONNRESET / EPIPE). + * - aimock's server keeps Node's default `httpAllowHalfOpen: false`, so a + * client FIN makes Node end our write side: the socket is `writableEnded` + * while the response is not. That is the evidence the client closed. + * + * A FIN alone (`readableEnded` with neither of those) is not a hang-up: it is + * what the socket shows when aimock tore the response down before Node + * answered the FIN. A real client FIN that Node has not answered yet looks the + * same, so this rule also classifies it as not a hang-up. aimock's own + * `destroy()` leaves neither signal. + */ +export function closedByClient(socket: PeerSocket): boolean { + return (socket.errored ?? null) !== null || socket.writableEnded === true; +} + +/** + * Who tore down a response that is gone without having ended. After the fact + * `res.destroyed` reads the same either way, so this reads the evidence: + * + * - a chaos `disconnect` marked the response before destroying it: aimock; + * - the request socket meets {@link closedByClient}: the client; + * - neither: aimock destroyed it server-side. + * + * `socket` defaults to the response's request socket. With no socket to read + * (a partial test double) the client attribution is kept. + */ +export function teardownCause( + res: http.ServerResponse, + socket: PeerSocket | null | undefined = requestSocket(res), +): "client" | "server" { + if (responseGoneReason(res) === "chaos-disconnected") return "server"; + if (!socket) return "client"; + return closedByClient(socket) ? "client" : "server"; +} + +/** + * The request socket's `bytesWritten` when each delivery started, so the + * status-line rule counts only this response's bytes on a keep-alive socket. + */ +const deliveryStartBytes = new WeakMap(); + +/** + * Record where this response's bytes start on its socket. A response whose + * head was already committed when the delivery started is not recorded: + * {@link statusLineSent} keeps the `headersSent` rule for it. + */ +export function noteDeliveryStart(res: http.ServerResponse): void { + if (deliveryStartBytes.has(res) || res.headersSent) return; + const bytes = requestSocket(res)?.bytesWritten; + if (typeof bytes === "number") deliveryStartBytes.set(res, bytes); +} + +/** + * Whether this response's status line reached the client's socket: a byte of + * it was handed to the socket after {@link noteDeliveryStart}. `headersSent` + * is not that test: `writeHead` sets it before any byte is written. With no + * recorded start, or no socket to read, `headersSent` is the fallback. + */ +export function statusLineSent(res: http.ServerResponse): boolean { + const start = deliveryStartBytes.get(res); + const bytes = requestSocket(res)?.bytesWritten; + if (start === undefined || typeof bytes !== "number") return res.headersSent; + return bytes > start; +} + +/** The journal `interruptReason` for {@link teardownCause}. */ +export function teardownReason(res: http.ServerResponse): string { + return teardownCause(res) === "client" ? CLIENT_ABORTED : AIMOCK_DISCONNECTED; +} + +/** + * HTTP delivery lifecycle. A `close` before `res.writableFinished` aborts it, + * and {@link teardownCause} decides whose teardown it was: the client + * (`"client"`) or aimock itself (`"server"`). A response that is already gone + * at construction aborts at once with the same attribution. A chaos + * `disconnect` destroys without ending, so no `writableEnded` check stops a + * writer for it. Only an already-ended response does not abort: the frame + * loop exits on its `writableEnded` check. The fixture interruption is + * created only when `streaming` is true (it is streaming-only). + */ +export function httpLifecycle( + res: http.ServerResponse, + fixture: Fixture | null, + opts: { streaming: boolean }, +): DeliveryLifecycle { + const interruption = opts.streaming && fixture ? createInterruptionSignal(fixture) : null; + const gone = responseGoneReason(res); + const goneAtStart = gone === null || gone === "already-ended" ? undefined : teardownCause(res); + return createLifecycle(interruption, goneAtStart, (onGone) => + subscribeClose(res, () => { + if (!res.writableFinished) onGone(teardownCause(res)); + }), + ); +} + +/** + * Per watched connection: whether its `close` event has fired. Only sockets + * watched while open are recorded, so an entry is evidence, not a guess. + */ +const wsCloseEmitted = new WeakMap(); + +/** + * Start recording when `ws` emits `close`, so {@link wsClosedCause} can tell + * aimock's own close from the client's. Each WS transport calls it when the + * connection opens. Idempotent, and a no-op once closed. + */ +export function watchWsClose(ws: WebSocketConnection): void { + if (ws.isClosed || wsCloseEmitted.has(ws)) return; + const source: CloseSource = ws; + if (typeof source.once !== "function") return; + wsCloseEmitted.set(ws, false); + source.once("close", () => wsCloseEmitted.set(ws, true)); +} + +/** + * Who closed a socket that reads `isClosed`, or `undefined` while it is open. + * + * `WebSocketConnection` marks itself closed and emits `close` in the same step + * for a client close frame, a lost connection, and `destroy()`. Only aimock's + * own `close()` (a write or buffer limit, a bad frame, a strict miss, shutdown) + * marks it closed at once and emits `close` about 100ms later. So a socket + * watched while open ({@link watchWsClose}) that is closed with no `close` yet + * was closed by aimock (`"server"`). Any other closed socket reads as the + * client leaving, as before. + */ +export function wsClosedCause(ws: WebSocketConnection): "client" | "server" | undefined { + if (!ws.isClosed) return undefined; + return wsCloseEmitted.get(ws) === false ? "server" : "client"; +} + +/** + * WebSocket turn lifecycle. A WS turn is always a stream, so it always owns + * the fixture interruption. A `close` means the client left. A socket already + * closed at construction aborts at once with {@link wsClosedCause}. Callers + * never create their own interruption object. + */ +export function wsLifecycle(ws: WebSocketConnection, fixture: Fixture | null): DeliveryLifecycle { + const interruption = fixture ? createInterruptionSignal(fixture) : null; + return createLifecycle(interruption, wsClosedCause(ws), (onGone) => + subscribeClose(ws, () => onGone("client")), + ); +} + +/** + * Journal a WS turn that stopped because its lifecycle aborted or its socket + * closed. This is the closed-socket rule every WS transport uses: the + * lifecycle's reason when it has one, else the closed socket's cause + * ({@link wsClosedCause}): {@link AIMOCK_DISCONNECTED} for aimock's own close, + * {@link CLIENT_ABORTED} for the client. A turn with no lifecycle (it never + * waits) passes none. + */ +export function markWsTurnStopped( + entry: JournalEntry, + ws: WebSocketConnection, + lifecycle?: DeliveryLifecycle, +): void { + const cause = wsClosedCause(ws); + const socketReason = + cause === "server" ? AIMOCK_DISCONNECTED : cause === "client" ? CLIENT_ABORTED : undefined; + markInterrupted(entry, lifecycle?.reason() ?? socketReason); +} + +/** + * Mark a journal entry interrupted. The reason is set only when the entry has + * none, so a later `undefined` never erases an earlier reason. The status is + * never changed: the entry keeps the status it was journaled with. + */ +export function markInterrupted(entry: JournalEntry, reason: string | undefined): void { + entry.response.interrupted = true; + if (entry.response.interruptReason === undefined && reason !== undefined) { + entry.response.interruptReason = reason; + } +} diff --git a/src/response/proxy.ts b/src/response/proxy.ts new file mode 100644 index 00000000..808da94c --- /dev/null +++ b/src/response/proxy.ts @@ -0,0 +1,76 @@ +import type * as http from "node:http"; +import { CLIENT_ABORTED, noteDeliveryStart, statusLineSent, teardownCause } from "./lifecycle.js"; + +/** + * Responses torn down before they finished, judged at entry, on `close`, and + * when the relay returns. Reading `res.writableFinished` only when journaling + * is too late: on Node 20 and 22 a relay that calls `end()` on a destroyed + * response makes it read `true` (Node 24+ keeps it `false`). + */ +const tornDownBeforeFinish = new WeakSet(); + +/** + * Run a proxy relay while watching for a client hang-up. + * + * WHO killed a mid-flight stream decides what the journal says, and after the + * fact the two look identical: `req.aborted`, `req.destroyed` and + * `res.destroyed` all read the same whether the peer hung up or the recorder + * tore the response down. So the hang-up is decided when the request socket + * actually closes before the response ended, by the same rule the delivery + * lifecycle uses ({@link teardownCause}): the client's socket closed or reset. + * A FIN alone is not a hang-up. A response already gone when this is entered + * is judged by that rule too, so a response aimock destroyed is not blamed on + * the client. The listener is attached before `run` and removed after it: a + * keep-alive socket outlives this request. + */ +export async function withHangUpDetection( + req: http.IncomingMessage, + res: http.ServerResponse, + run: () => Promise, +): Promise<{ outcome: T; clientHungUp: boolean }> { + const socket = req.socket; + noteDeliveryStart(res); + const clientCaused = (): boolean => !res.writableEnded && teardownCause(res, socket) === "client"; + let clientHungUp = (res.destroyed || socket?.destroyed === true) && clientCaused(); + const noteSocketClose = (): void => { + if (clientCaused()) clientHungUp = true; + }; + const noteTearDown = (): void => { + if (res.destroyed && !res.writableFinished) tornDownBeforeFinish.add(res); + }; + noteTearDown(); + socket?.on("close", noteSocketClose); + res.on("close", noteTearDown); + try { + const outcome = await run(); + return { outcome, clientHungUp }; + } finally { + // The relay may have destroyed the response in its last tick, before + // `close` is emitted. + noteTearDown(); + socket?.off("close", noteSocketClose); + res.off("close", noteTearDown); + } +} + +/** + * The journal fields for a proxied response. A stream torn down before it + * finished (destroyed, even when the relay called `end()` on it afterwards) + * is marked interrupted and names the side that killed it: a client hang-up + * is normal caller behaviour, an upstream/recorder tear-down is a fault worth + * chasing. When no byte of the response reached the socket + * ({@link statusLineSent}) the status is 0: the client never got a status + * line. A response that finished, or was never torn down, gets no fields. + */ +export function proxyInterruption( + res: http.ServerResponse, + clientHungUp: boolean, +): { status?: 0; interrupted?: true; interruptReason?: string } { + const tornDown = res.destroyed && (tornDownBeforeFinish.has(res) || !res.writableFinished); + if (!tornDown) return {}; + return { + ...(statusLineSent(res) ? {} : { status: 0 as const }), + interrupted: true, + interruptReason: clientHungUp ? CLIENT_ABORTED : "proxy stream destroyed", + }; +} diff --git a/src/response/timing.ts b/src/response/timing.ts new file mode 100644 index 00000000..51d5da42 --- /dev/null +++ b/src/response/timing.ts @@ -0,0 +1,217 @@ +import type { + Fixture, + FixtureResponse, + FixtureToolCall, + RecordedTimings, + ToolCall, +} from "../types.js"; +import type { MisbehaviorCandidate } from "../misbehavior.js"; +import { calculateTotalDelay } from "../sse-writer.js"; +import { + isCombinedFixtureResponse, + isTextResponse, + isToolCallResponse, + resolveFixtureBlockOutcome, + resolveServedBlockOutcome, + servedToolCalls, +} from "../helpers.js"; + +/** Which non-streaming surface asks: OpenAI Chat Completions, or any other provider. */ +export type NonStreamingSurface = "openai-chat" | "other"; + +/** + * Whether a non-streaming (`stream: false`) reply on `surface` waits. It is + * opt-in per fixture: `nonStreamingTimingAllProviders: true` on every + * surface that calls this, or the shipped `nonStreamingTiming: true` on OpenAI Chat + * Completions only. Either one also needs an explicit `streamingProfile` or + * `recordedTimings`. Only the boolean `true` counts. + */ +export function nonStreamingTimingEnabled(fixture: Fixture, surface: NonStreamingSurface): boolean { + const optedIn = + fixture.nonStreamingTimingAllProviders === true || + (fixture.nonStreamingTiming === true && surface === "openai-chat"); + return optedIn && !!(fixture.streamingProfile || fixture.recordedTimings); +} + +/** + * The recorded timings a non-streaming wait on `surface` is sized from. `t` is + * returned as given when it is `undefined`, when its `interChunkDelaysMs` is + * an array, or for `nonStreamingTiming: true` on OpenAI Chat Completions. + * Otherwise (a fixture built in code can omit `interChunkDelaysMs`) it is + * returned with `interChunkDelaysMs: []`, so the wait counts no recorded gaps + * and the reply is still served. + * + * The `nonStreamingTiming: true` OpenAI Chat Completions case is passed + * through unchanged so that the shipped option keeps its behavior: with no + * `streamingProfile`, sizing the wait for a reply of more than one chunk reads + * the missing `interChunkDelaysMs` and throws. With `deliverJson`'s + * `duplicateId` set, that path throws on a missing `interChunkDelaysMs` in any + * case, because {@link extendRecordedTimings} reads the list first. + */ +export function nonStreamingRecordedTimings( + fixture: Fixture, + surface: NonStreamingSurface, + t: RecordedTimings | undefined, +): RecordedTimings | undefined { + if (!t || Array.isArray(t.interChunkDelaysMs)) return t; + if (fixture.nonStreamingTiming === true && surface === "openai-chat") return t; + return { ...t, interChunkDelaysMs: [] }; +} + +/** + * The total wait before a non-streaming reply: `calculateTotalDelay` over the + * content chunks `payloadLength` makes at `chunkSize`. There is no latency + * input, so bare `latency` never delays a non-streaming reply. + */ +export function nonStreamingDelayMs( + fixture: Fixture, + payloadLength: number, + chunkSize: number, + recordedTimings: RecordedTimings | undefined, + replaySpeed: number | undefined, +): number { + return calculateTotalDelay( + payloadLength, + chunkSize, + fixture.streamingProfile, + recordedTimings, + replaySpeed, + ); +} + +function toolCallsLength(calls: ToolCall[]): number { + return calls.map((tc) => tc.name + tc.arguments).join("").length; +} + +/** + * The payload a reply serves, as OpenAI Chat Completions sizes its + * non-streaming wait: reasoning, then text content, then each tool call's + * name and arguments. Non-empty `blocks` are authoritative and resolve + * through `resolveFixtureBlockOutcome`. The inputs are fixture-level values, + * so the length is the same on every provider. + */ +export function servedPayloadLength(response: FixtureResponse, reasoning?: string): number { + const reasoningLength = reasoning?.length ?? 0; + if (isCombinedFixtureResponse(response)) { + if (response.blocks && response.blocks.length > 0) { + const outcome = resolveFixtureBlockOutcome(response.blocks); + return reasoningLength + outcome.content.length + toolCallsLength(outcome.toolCalls); + } + return ( + reasoningLength + (response.content ?? "").length + toolCallsLength(response.toolCalls ?? []) + ); + } + if (isTextResponse(response)) return reasoningLength + response.content.length; + if (isToolCallResponse(response)) return reasoningLength + toolCallsLength(response.toolCalls); + return reasoningLength; +} + +/** Each served call's name plus its function `arguments` or custom `input`. */ +function servedCallsLength(calls: readonly FixtureToolCall[]): number { + return calls.map((tc) => tc.name + (tc.type === "custom" ? tc.input : tc.arguments)).join("") + .length; +} + +/** + * The payload the OpenAI Responses API serves, sized as + * {@link servedPayloadLength} sizes it but from the Responses carriers: a + * non-empty `responsesBlocks` (else a non-empty `blocks`) through + * `resolveServedBlockOutcome`, and `toolCalls` plus `customToolCalls` through + * `servedToolCalls`. A custom call counts its name and `input`; its stream + * sends the input in `chunkSize` deltas, as a function call's arguments. + */ +export function responsesPayloadLength(response: FixtureResponse, reasoning?: string): number { + const reasoningLength = reasoning?.length ?? 0; + if (isCombinedFixtureResponse(response)) { + const outcome = resolveServedBlockOutcome(response); + if (outcome) { + return reasoningLength + outcome.content.length + servedCallsLength(outcome.toolCalls); + } + return ( + reasoningLength + + (response.content ?? "").length + + servedCallsLength(servedToolCalls(response)) + ); + } + if (isTextResponse(response)) return reasoningLength + response.content.length; + if (isToolCallResponse(response)) { + return reasoningLength + servedCallsLength(servedToolCalls(response)); + } + return reasoningLength; +} + +/** + * The stream frames of one `image_generation_call` item: `output_item.added`, + * the `in_progress`, `generating` and `completed` events, and + * `output_item.done`. + */ +export const IMAGE_GENERATION_FRAMES = 5; + +/** + * The frames the OpenAI Responses stream sends for a fixture's `citations` + * and generated `files`, which carry no text for {@link responsesPayloadLength} + * to count: one `output_text.annotation.added` per citation on a turn with + * text, and {@link IMAGE_GENERATION_FRAMES} per file (its + * `image_generation_call` item has no deltas; these frames are its whole + * stream). The non-streaming wait counts each as one chunk, as the stream + * counts each as one frame, so the JSON wait is not undersized for them. A + * malformed value counts as none: this never throws, and the builder rejects + * it with its coded error before any byte is written. + */ +export function responsesOutputFrames(response: FixtureResponse): number { + const textTurn = isCombinedFixtureResponse(response) || isTextResponse(response); + if (!textTurn && !isToolCallResponse(response)) return 0; + const files = Array.isArray(response.files) ? response.files.length * IMAGE_GENERATION_FRAMES : 0; + // A tool-only turn has no text to annotate: its builder rejects citations. + const citations = textTurn && Array.isArray(response.citations) ? response.citations.length : 0; + return files + citations; +} + +/** + * The payload length for a misbehavior-applied reply. For the three fault + * terminals on a text response (refusal, content filter, reasoning-only) it is + * OpenAI's selection: the refusal text, nothing, or the reasoning. In every + * other case, including a stop plan on a tool-call, combined or blocks + * response, it is `servedLength`: {@link servedPayloadLength} by default, or + * {@link responsesPayloadLength} on the OpenAI Responses API. + */ +export function misbehaviorPayloadLength( + plan: Pick, + effReasoning: string | undefined, + servedLength: (response: FixtureResponse, reasoning?: string) => number = servedPayloadLength, +): number { + if ( + isTextResponse(plan.response) && + (plan.stop === "refusal" || plan.stop === "content_filter" || plan.reasoning !== undefined) + ) { + if (plan.stop === "refusal") return (plan.refusal ?? "").length; + if (plan.stop === "content_filter") return 0; + return (plan.reasoning ?? "").length; + } + return servedLength(plan.response, effReasoning); +} + +/** + * The K4 duplicate-id gap extension: a duplicate-id plan can add frames, so + * with `duplicate` true the recorded gaps grow to at least `frameCount - 1` + * entries by repeating the last gap (0 when the list is empty). A longer + * list is kept whole, except when `frameCount` is `NaN` (a `NaN` chunk size): + * the list is then truncated to empty. The wait is 0 either way, because a + * `NaN` chunk size gives no chunks to wait for. With `duplicate` false or no + * `t`, `t` is returned unchanged. + */ +export function extendRecordedTimings( + t: RecordedTimings | undefined, + frameCount: number, + duplicate: boolean, +): RecordedTimings | undefined { + if (!duplicate || !t) return t; + const gaps = t.interChunkDelaysMs; + return { + ...t, + interChunkDelaysMs: Array.from( + { length: Math.max(gaps.length, frameCount - 1) }, + (_, index) => gaps[index] ?? gaps.at(-1) ?? 0, + ), + }; +} diff --git a/src/responses.ts b/src/responses.ts index a7cb95f3..d342f3cc 100644 --- a/src/responses.ts +++ b/src/responses.ts @@ -17,11 +17,11 @@ import type { Fixture, FixtureBlock, FixtureGeneratedFile, + FixtureResponse, FixtureToolCall, HandlerDefaults, ResponseOverrides, ResponsesFixtureBlock, - StreamingProfile, ToolCall, ToolDefinition, } from "./types.js"; @@ -60,9 +60,16 @@ import { import type { MisbehaviorPlan } from "./misbehavior.js"; import { isReasoningModel } from "./model-utils.js"; import { matchFixtureDiagnostic, recordMatchOptions, setResponsesOfferedTools } from "./router.js"; -import { writeErrorResponse, delay, calculateDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; -import type { RecordedTimings } from "./types.js"; +import { writeErrorResponse, typedSseCodec } from "./sse-writer.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { + extendRecordedTimings, + misbehaviorPayloadLength, + nonStreamingTimingEnabled, + responsesOutputFrames, + responsesPayloadLength, +} from "./response/timing.js"; import type { Journal } from "./journal.js"; import { applyChaosAsync } from "./chaos.js"; import { proxyAndRecord } from "./recorder.js"; @@ -1873,53 +1880,6 @@ function buildContentWithToolCallsResponse( return buildResponseEnvelope(model, output, overrides); } -// ─── SSE writer for Responses API ─────────────────────────────────────────── - -interface ResponsesStreamOptions { - latency?: number; - streamingProfile?: StreamingProfile; - recordedTimings?: RecordedTimings; - replaySpeed?: number; - signal?: AbortSignal; - onChunkSent?: () => void; -} - -async function writeResponsesSSEStream( - res: http.ServerResponse, - events: ResponsesSSEEvent[], - optionsOrLatency?: number | ResponsesStreamOptions, -): Promise { - const opts: ResponsesStreamOptions = - typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); - - let chunkIndex = 0; - for (const event of events) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal?.aborted) return false; - if (res.writableEnded) return true; - res.write(`event: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } - - if (!res.writableEnded) { - res.end(); - } - return true; -} - // ─── Request handler ──────────────────────────────────────────────────────── export async function handleResponses( @@ -2125,15 +2085,17 @@ export async function handleResponses( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - completionReq, - "openai", - req.url ?? "/v1/responses", - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + completionReq, + "openai", + req.url ?? "/v1/responses", + fixtures, + defaults, + raw, + ), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -2142,7 +2104,12 @@ export async function handleResponses( path: req.url ?? "/v1/responses", headers: flattenHeaders(req.headers), body: completionReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -2213,6 +2180,28 @@ export async function handleResponses( const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); const fixtureTimings = fixture.recordedTimings; const effectiveReplaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; + const streamTiming = { + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixtureTimings, + replaySpeed: effectiveReplaySpeed, + }; + // The payload length only sizes the opt-in wait. With timing off it is not + // read, so it is not computed: a malformed fixture then fails in its builder, + // with the same error as before. + const timedJson = nonStreamingTimingEnabled(fixture, "other"); + // The JSON reply's payload: the served text and calls, plus one chunk per + // stream frame its citations and generated files add (responsesOutputFrames). + const jsonPayloadLength = (r: FixtureResponse, reasoning?: string): number => + responsesPayloadLength(r, reasoning) + responsesOutputFrames(r) * chunkSize; + const jsonOpts = (payloadLength: () => number, duplicateId = false) => ({ + surface: "other" as const, + payloadLength: timedJson ? payloadLength() : 0, + chunkSize, + recordedTimings: fixtureTimings, + duplicateId, + replaySpeed: effectiveReplaySpeed, + }); if (misbehavior.kind === "applied") { // Journal first, as the success branches do, so a fixture error thrown @@ -2226,42 +2215,47 @@ export async function handleResponses( response: { status: 200, fixture }, }); const prepared = prepareResponsesMisbehavior(misbehavior); - const output = buildResponsesMisbehavior( - prepared, - completionReq, - chunkSize, - emitEncryptedReasoning, - ); - recordMisbehaviorOutcome({ entry, summary: prepared.summary, defaults, testId }); + const build = () => + buildResponsesMisbehavior(prepared, completionReq, chunkSize, emitEncryptedReasoning); + const record = () => + recordMisbehaviorOutcome({ entry, summary: prepared.summary, defaults, testId }); + // A timed JSON reply is built after the wait (inside deliverJson, still + // before writeHead), so `created_at` is the write time, and its outcome is + // recorded there, once the build succeeds. Every other reply is built + // here, before the outcome is recorded. + const output = timedJson && responsesReq.stream !== true ? undefined : build(); + if (output) record(); if (responsesReq.stream !== true) { - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(output.response)); + // The reasoning buildResponsesMisbehavior serves: the fault's, else the fixture's. + const servedReasoning = + prepared.reasoning ?? + ("reasoning" in prepared.response ? prepared.response.reasoning : undefined); + await deliverJson( + res, + entry, + fixture, + () => { + if (output) return output.response; + const built = build(); + record(); + return built.response; + }, + jsonOpts( + () => misbehaviorPayloadLength(prepared, servedReasoning, jsonPayloadLength), + !!prepared.duplicateId, + ), + ); } else { - const appliedTimings = - prepared.duplicateId && fixtureTimings - ? { ...fixtureTimings, interChunkDelaysMs: [...fixtureTimings.interChunkDelaysMs] } - : fixtureTimings; - if (prepared.duplicateId && appliedTimings) { - const lastGap = appliedTimings.interChunkDelaysMs.at(-1) ?? 0; - while (appliedTimings.interChunkDelaysMs.length < output.events.length - 1) { - appliedTimings.interChunkDelaysMs.push(lastGap); - } - } - const interruption = createInterruptionSignal(fixture); - const completed = await writeResponsesSSEStream(res, output.events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: appliedTimings, - replaySpeed: effectiveReplaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, + // Always built above for a stream. + const { events } = output ?? build(); + await deliverStream(res, entry, fixture, typedSseCodec(), events, { + ...streamTiming, + recordedTimings: extendRecordedTimings( + fixtureTimings, + events.length, + !!prepared.duplicateId, + ), }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); } return; } @@ -2321,19 +2315,24 @@ export async function handleResponses( if (!ordered) assertCustomToolCalls(response); const toolCalls = servedToolCalls(response); if (responsesReq.stream !== true) { - const body = buildContentWithToolCallsResponse( - response.content ?? "", - toolCalls, - completionReq.model, - effReasoning, - response.webSearches, - overrides, - ordered, - emitEncryptedReasoning, - synthesizeSummarylessReasoning, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildContentWithToolCallsResponse( + response.content ?? "", + toolCalls, + completionReq.model, + effReasoning, + response.webSearches, + overrides, + ordered, + emitEncryptedReasoning, + synthesizeSummarylessReasoning, + ), + jsonOpts(() => jsonPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = contentWithToolCallsStreamEvents( response.content ?? "", @@ -2347,21 +2346,7 @@ export async function handleResponses( emitEncryptedReasoning, synthesizeSummarylessReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeResponsesSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixtureTimings, - replaySpeed: effectiveReplaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } @@ -2394,17 +2379,22 @@ export async function handleResponses( testId, }); if (responsesReq.stream !== true) { - const body = buildTextResponse( - response.content, - completionReq.model, - effReasoning, - response.webSearches, - overrides, - emitEncryptedReasoning, - synthesizeSummarylessReasoning, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildTextResponse( + response.content, + completionReq.model, + effReasoning, + response.webSearches, + overrides, + emitEncryptedReasoning, + synthesizeSummarylessReasoning, + ), + jsonOpts(() => jsonPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = buildTextStreamEvents( response.content, @@ -2416,21 +2406,7 @@ export async function handleResponses( emitEncryptedReasoning, synthesizeSummarylessReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeResponsesSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixtureTimings, - replaySpeed: effectiveReplaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } @@ -2464,17 +2440,22 @@ export async function handleResponses( }); assertCustomToolCalls(response); if (responsesReq.stream !== true) { - const body = buildToolCallResponse( - servedToolCalls(response), - completionReq.model, - effReasoning, - response.webSearches, - overrides, - emitEncryptedReasoning, - synthesizeSummarylessReasoning, + await deliverJson( + res, + journalEntry, + fixture, + () => + buildToolCallResponse( + servedToolCalls(response), + completionReq.model, + effReasoning, + response.webSearches, + overrides, + emitEncryptedReasoning, + synthesizeSummarylessReasoning, + ), + jsonOpts(() => jsonPayloadLength(response, effReasoning)), ); - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(body)); } else { const events = toolCallStreamEvents( servedToolCalls(response), @@ -2486,21 +2467,7 @@ export async function handleResponses( emitEncryptedReasoning, synthesizeSummarylessReasoning, ); - const interruption = createInterruptionSignal(fixture); - const completed = await writeResponsesSSEStream(res, events, { - latency, - streamingProfile: fixture.streamingProfile, - recordedTimings: fixtureTimings, - replaySpeed: effectiveReplaySpeed, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream(res, journalEntry, fixture, typedSseCodec(), events, streamTiming); } return; } diff --git a/src/server.ts b/src/server.ts index 592c6b52..41670c07 100644 --- a/src/server.ts +++ b/src/server.ts @@ -52,8 +52,11 @@ import { responsesToolsExtendedView, responsesToolsViewSource, } from "./fixture-loader.js"; -import { writeSSEStream, writeErrorResponse, delay, calculateTotalDelay } from "./sse-writer.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { writeErrorResponse, sseCodec } from "./sse-writer.js"; +import { deliverJson, deliverStream } from "./response/deliver.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { CLIENT_ABORTED } from "./response/lifecycle.js"; +import { extendRecordedTimings, misbehaviorPayloadLength } from "./response/timing.js"; import { prepareOpenAIChatMisbehavior, resolveOpenAIChatMisbehaviorUsage, @@ -1636,7 +1639,7 @@ async function handleCompletions( // and chaos `disconnect` do, never a status the client did not get. const interruptReason = bodyLimitDisconnect ? "request body exceeded size limit" - : "client aborted"; + : CLIENT_ABORTED; defaults.logger.warn( bodyLimitDisconnect ? `${route}: ${msg}` : `${route}: client aborted mid-body (${msg})`, ); @@ -2210,22 +2213,10 @@ async function handleCompletions( } : undefined; - // WHO killed a mid-flight stream decides what the journal says, and - // after the fact the two look identical: `req.aborted`, `req.destroyed` - // and `res.destroyed` all read the same whether the peer hung up or the - // recorder tore the response down. The one signal that separates them is - // the peer's FIN/RST, which arrives on the request socket as `end` (or - // `error`) and only ever when the CLIENT went away. Removed afterwards: - // a keep-alive socket outlives this request. - let clientHungUp = false; - const noteClientHangUp = (): void => { - if (!res.writableEnded) clientHungUp = true; - }; - req.socket?.on("end", noteClientHangUp); - req.socket?.on("error", noteClientHangUp); - let outcome: Awaited>; - try { - outcome = await proxyAndRecord( + // A client hang-up and a recorder tear-down look identical after the + // fact; withHangUpDetection watches the request socket while relaying. + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( req, res, body, @@ -2235,19 +2226,17 @@ async function handleCompletions( defaults, raw, hookOptions, - ); - } finally { - req.socket?.off("end", noteClientHangUp); - req.socket?.off("error", noteClientHangUp); - } + ), + ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { - // A stream that died mid-flight still carries the 200 it opened with; - // mark it interrupted so the journal shows what the client actually - // got rather than a clean success, and name the side that killed it — - // a client hang-up is normal caller behaviour, an upstream/recorder - // tear-down is a fault worth chasing. - const destroyedMidStream = res.destroyed && !res.writableEnded; + // A proxied response torn down before it finished is marked + // interrupted, so the journal shows what the client actually got + // rather than a clean success, and names the side that killed it — a + // client hang-up is normal caller behaviour, an upstream/recorder + // tear-down is a fault worth chasing. Its status is the one it opened + // with (`res.statusCode`, e.g. 200) when its status line reached the + // socket, and 0 when it did not: the client never got a status. journal.add({ method: req.method ?? "POST", path: req.url ?? COMPLETIONS_PATH, @@ -2257,12 +2246,7 @@ async function handleCompletions( status: res.statusCode ?? 200, fixture: null, source: "proxy", - ...(destroyedMidStream - ? { - interrupted: true, - interruptReason: clientHungUp ? "client aborted" : "proxy stream destroyed", - } - : {}), + ...proxyInterruption(res, clientHungUp), }, }); recordBypass("proxied"); @@ -2423,54 +2407,45 @@ async function handleCompletions( ): { prompt_tokens: number; completion_tokens: number; total_tokens: number } => appliedUsage ?? resolveUsage(overrides, streamingPromptText, completionText); - const resolveRecordedTimings = (chunkCount: number) => { - const timings = fixture.recordedTimings; - if (!appliedPlan?.duplicateId || !timings) return timings; - const gaps = timings.interChunkDelaysMs; - return { - ...timings, - interChunkDelaysMs: Array.from( - { length: Math.max(gaps.length, chunkCount - 1) }, - (_, index) => gaps[index] ?? gaps.at(-1) ?? 0, - ), - }; - }; - // Non-streaming (`stream: false`) timing is opt-in per fixture: only a - // fixture with `nonStreamingTiming: true` AND an explicit `streamingProfile` - // or `recordedTimings` delays the single JSON write, by the total a streamed - // reply at the same chunkSize would take. Without the flag the reply returns - // immediately (the pre-#545 default). Bare `latency` / `--latency` is the gap - // between SSE chunks only and never delays this path. - const awaitNonStreamingTiming = (contentLength: number): Promise => { - if (fixture.nonStreamingTiming !== true) return Promise.resolve(); - if (!fixture.streamingProfile && !fixture.recordedTimings) return Promise.resolve(); - const chunkCount = Math.max(1, Math.ceil(contentLength / chunkSize)); - return delay( - calculateTotalDelay( - contentLength, - chunkSize, - fixture.streamingProfile, - resolveRecordedTimings(chunkCount), - fixture.replaySpeed ?? defaults.replaySpeed, - ), - ); - }; + // fixture with `nonStreamingTiming: true` (OpenAI chat only) or + // `nonStreamingTimingAllProviders: true`, AND an explicit `streamingProfile` + // or `recordedTimings`, delays the single JSON write, by the total a streamed + // reply at the same chunkSize would take. Without it the reply returns + // immediately. Bare `latency` / `--latency` is the gap between SSE chunks + // only and never delays this path. deliverJson owns the wait, which a client + // hang-up cancels. A K4 duplicate-id plan extends the recorded gaps. + const nonStreamingOpts = (payloadLength: number) => ({ + surface: "openai-chat" as const, + payloadLength, + chunkSize, + recordedTimings: fixture.recordedTimings, + duplicateId: !!appliedPlan?.duplicateId, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); + const streamTiming = (frameCount: number) => ({ + latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: extendRecordedTimings( + fixture.recordedTimings, + frameCount, + !!appliedPlan?.duplicateId, + ), + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); // A built non-stream reply is committed to be served: record the outcome - // before the timing wait, so the journal carries it while the reply waits. - // A client that hangs up during the wait got nothing, so the entry records a - // torn socket (status 0, interrupted), as routeError does for one that hangs - // up before its body arrives. False when there is no one left to answer. - const commitNonStream = async (entry: JournalEntry, contentLength: number): Promise => { + // before deliverJson's timing wait, so the journal carries it while the reply + // waits. deliverJson journals a client that hangs up during the wait as a + // torn socket (status 0, interrupted, "client aborted"), as routeError does + // for one that hangs up before its body arrives, and then writes nothing. + const commitNonStream = async ( + entry: JournalEntry, + reply: unknown, + payloadLength: number, + ): Promise => { recordOutcome(entry); - await awaitNonStreamingTiming(contentLength); - const gone = responseGoneReason(res); - if (gone === null) return true; - entry.response.status = 0; - entry.response.interrupted = true; - entry.response.interruptReason = gone === "client-gone" ? "client aborted" : gone; - return false; + await deliverJson(res, entry, fixture, () => reply, nonStreamingOpts(payloadLength)); }; // OpenRouter response shaping (no-op for OpenAI callers). Applied as a @@ -2595,15 +2570,11 @@ async function handleCompletions( overrides, body.messages, ); - const nonStreamedContent = - appliedPlan.stop === "refusal" - ? (appliedPlan.refusal ?? "") - : appliedPlan.stop === "content_filter" - ? "" - : (appliedPlan.reasoning ?? ""); - if (!(await commitNonStream(entry, nonStreamedContent.length))) return; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(shapeORCompletion(completion, overrides))); + await commitNonStream( + entry, + shapeORCompletion(completion, overrides), + misbehaviorPayloadLength(appliedPlan, appliedReasoning), + ); } else { const chunks = appliedPlan.stop === "refusal" @@ -2628,23 +2599,14 @@ async function handleCompletions( : undefined; shapeORChunks(chunks, usageChunk, overrides); recordOutcome(entry); - const interruption = createInterruptionSignal(fixture); - const completed = await writeSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - usageChunk, - recordedTimings: resolveRecordedTimings(chunks.length), - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - openRouterProcessing, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream( + res, + entry, + fixture, + sseCodec({ done: true, usageChunk, openRouterProcessing }), + chunks, + streamTiming(chunks.length), + ); } return; } @@ -2711,9 +2673,7 @@ async function handleCompletions( body.messages, ); const contentLength = (effReasoning?.length ?? 0) + completionText.length; - if (!(await commitNonStream(journalEntry, contentLength))) return; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(shapeORCompletion(completion, overrides))); + await commitNonStream(journalEntry, shapeORCompletion(completion, overrides), contentLength); } else { const chunks = buildContentWithToolCallsChunks( blockOutcome?.content ?? response.content ?? "", @@ -2736,23 +2696,14 @@ async function handleCompletions( : undefined; shapeORChunks(chunks, usageChunk, overrides); recordOutcome(journalEntry); - const interruption = createInterruptionSignal(fixture); - const completed = await writeSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - usageChunk, - recordedTimings: resolveRecordedTimings(chunks.length), - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - openRouterProcessing, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream( + res, + journalEntry, + fixture, + sseCodec({ done: true, usageChunk, openRouterProcessing }), + chunks, + streamTiming(chunks.length), + ); } return; } @@ -2795,9 +2746,7 @@ async function handleCompletions( body.messages, ); const contentLength = (effReasoning?.length ?? 0) + response.content.length; - if (!(await commitNonStream(journalEntry, contentLength))) return; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(shapeORCompletion(completion, overrides))); + await commitNonStream(journalEntry, shapeORCompletion(completion, overrides), contentLength); } else { const chunks = buildTextChunks( response.content, @@ -2817,23 +2766,14 @@ async function handleCompletions( : undefined; shapeORChunks(chunks, usageChunk, overrides); recordOutcome(journalEntry); - const interruption = createInterruptionSignal(fixture); - const completed = await writeSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - usageChunk, - recordedTimings: resolveRecordedTimings(chunks.length), - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - openRouterProcessing, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream( + res, + journalEntry, + fixture, + sseCodec({ done: true, usageChunk, openRouterProcessing }), + chunks, + streamTiming(chunks.length), + ); } return; } @@ -2878,9 +2818,7 @@ async function handleCompletions( body.messages, ); const contentLength = (effReasoning?.length ?? 0) + toolCallText.length; - if (!(await commitNonStream(journalEntry, contentLength))) return; - res.writeHead(200, { "Content-Type": "application/json" }); - res.end(JSON.stringify(shapeORCompletion(completion, overrides))); + await commitNonStream(journalEntry, shapeORCompletion(completion, overrides), contentLength); } else { const chunks = buildToolCallChunks( toolCalls, @@ -2901,23 +2839,14 @@ async function handleCompletions( : undefined; shapeORChunks(chunks, usageChunk, overrides); recordOutcome(journalEntry); - const interruption = createInterruptionSignal(fixture); - const completed = await writeSSEStream(res, chunks, { - latency, - streamingProfile: fixture.streamingProfile, - signal: interruption?.signal, - onChunkSent: interruption?.tick, - usageChunk, - recordedTimings: resolveRecordedTimings(chunks.length), - replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, - openRouterProcessing, - }); - if (!completed) { - if (!res.writableEnded) res.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverStream( + res, + journalEntry, + fixture, + sseCodec({ done: true, usageChunk, openRouterProcessing }), + chunks, + streamTiming(chunks.length), + ); } return; } @@ -3483,7 +3412,7 @@ async function startServer( if ((clientAbort || bodyLimitDisconnect) && !res.headersSent) { const interruptReason = bodyLimitDisconnect ? "request body exceeded size limit" - : "client aborted"; + : CLIENT_ABORTED; // Nothing reached the client and nothing crashed: the entry records a // torn socket (status 0, same as chaos `disconnect` before headers), and // there is no socket left to answer on. @@ -3573,7 +3502,7 @@ async function startServer( // `interrupted` means the client did not get a complete body. Once // `end()` has run it did, so a crash after that point is recorded in // `error` with the status that was delivered, never as an interruption. - const midStreamReason = clientAbort ? "client aborted" : "handler crashed mid-stream"; + const midStreamReason = clientAbort ? CLIENT_ABORTED : "handler crashed mid-stream"; if (existing) { if (res.writableEnded) { if (existing.response.error === undefined) existing.response.error = msg; diff --git a/src/sse-writer.ts b/src/sse-writer.ts index 3754ffc7..6f2f488d 100644 --- a/src/sse-writer.ts +++ b/src/sse-writer.ts @@ -1,18 +1,21 @@ import type * as http from "node:http"; import type { SSEChunk, StreamingProfile, RecordedTimings } from "./types.js"; +import { httpSink, runFrameLoop, type FrameCodec } from "./response/frame-loop.js"; export function delay(ms: number, signal?: AbortSignal): Promise { if (ms <= 0 || signal?.aborted) return Promise.resolve(); return new Promise((resolve) => { - const timer = setTimeout(resolve, ms); - signal?.addEventListener( - "abort", - () => { - clearTimeout(timer); - resolve(); - }, - { once: true }, - ); + const onAbort = (): void => { + clearTimeout(timer); + resolve(); + }; + // Remove the abort listener when the timer fires, so a long stream sharing + // one lifecycle signal does not keep one listener per delayed frame. + const timer = setTimeout(() => { + signal?.removeEventListener("abort", onAbort); + resolve(); + }, ms); + signal?.addEventListener("abort", onAbort, { once: true }); }); } @@ -123,46 +126,72 @@ export async function writeSSEStream( ): Promise { const opts: StreamOptions = typeof optionsOrLatency === "number" ? { latency: optionsOrLatency } : (optionsOrLatency ?? {}); - const latency = opts.latency ?? 0; - const profile = opts.streamingProfile; - const { recordedTimings, replaySpeed } = opts; - const signal = opts.signal; - const onChunkSent = opts.onChunkSent; - - if (res.writableEnded) return true; - res.setHeader("Content-Type", "text/event-stream"); - res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); + // No lifecycle: an external caller gets close-awareness only through its own + // `signal` (a gone client still returns true). + // Base contract (public API): with no frames, an aborted signal does not + // stop the writer; it still finishes the stream (any trailer, then end()) + // and returns true. The frame loop returns false there, so pass it no + // signal when there are no frames (it reads the signal only per frame and + // after the loop). + return runFrameLoop( + httpSink(res), + chunks, + sseCodec({ + done: true, + usageChunk: opts.usageChunk, + openRouterProcessing: opts.openRouterProcessing, + }), + { + latency: opts.latency, + streamingProfile: opts.streamingProfile, + recordedTimings: opts.recordedTimings, + replaySpeed: opts.replaySpeed, + }, + { signal: chunks.length > 0 ? opts.signal : undefined, onChunkSent: opts.onChunkSent }, + ); +} - // OpenRouter emits exactly ONE `: OPENROUTER PROCESSING` keepalive comment - // first — before any data frame (opt-in; see or-capture/chat-stream.sse). A - // faithful client discards `:`-prefixed lines before JSON-parsing a `data:`. - if (opts.openRouterProcessing && !res.writableEnded) { - res.write(OPENROUTER_PROCESSING_COMMENT); - } +const SSE_HEADERS: Record = { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + Connection: "keep-alive", +}; - let chunkIndex = 0; - for (const chunk of chunks) { - const chunkDelay = calculateDelay(chunkIndex, profile, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) { - await delay(chunkDelay, signal); - } - if (signal?.aborted) return false; - if (res.writableEnded) return true; - res.write(`data: ${JSON.stringify(chunk)}\n\n`); - onChunkSent?.(); - if (signal?.aborted) return false; - chunkIndex++; - } +// Internal (not part of the public API): data-only SSE (`data: \n\n`). +// `done: true` writes the optional usage chunk then `data: [DONE]` after the +// last frame (OpenAI chat); `done: false` writes no trailer. +// `openRouterProcessing` writes the OpenRouter keepalive comment first. Line +// comments, not JSDoc: see calculateTotalDelay. +export function sseCodec(opts: { + done: boolean; + usageChunk?: SSEChunk; + openRouterProcessing?: boolean; +}): FrameCodec { + return { + headers: SSE_HEADERS, + ...(opts.openRouterProcessing ? { prelude: () => OPENROUTER_PROCESSING_COMMENT } : {}), + encode: (frame) => `data: ${JSON.stringify(frame)}\n\n`, + trailer: () => { + if (!opts.done) return []; + const parts: string[] = []; + if (opts.usageChunk) parts.push(`data: ${JSON.stringify(opts.usageChunk)}\n\n`); + parts.push("data: [DONE]\n\n"); + return parts; + }, + }; +} - if (!res.writableEnded) { - if (opts.usageChunk) { - res.write(`data: ${JSON.stringify(opts.usageChunk)}\n\n`); - } - res.write("data: [DONE]\n\n"); - res.end(); - } - return true; +// Internal: typed SSE (`event: \ndata: \n\n`), used by Anthropic. +// `terminalSentinel` writes `data: [DONE]` after the last frame, only when the +// stream was not aborted. +export function typedSseCodec(opts?: { + terminalSentinel?: boolean; +}): FrameCodec { + return { + headers: SSE_HEADERS, + encode: (frame) => `event: ${frame.type}\ndata: ${JSON.stringify(frame)}\n\n`, + trailer: () => (opts?.terminalSentinel ? ["data: [DONE]\n\n"] : []), + }; } /** diff --git a/src/transcription.ts b/src/transcription.ts index 111b24fb..b1828c65 100644 --- a/src/transcription.ts +++ b/src/transcription.ts @@ -13,11 +13,19 @@ import { strictNoMatchLogLine, } from "./helpers.js"; import { matchFixtureDiagnostic } from "./router.js"; -import { calculateDelay, delay, writeErrorResponse } from "./sse-writer.js"; +import { sseCodec, writeErrorResponse } from "./sse-writer.js"; import type { Journal } from "./journal.js"; import { applyChaosAsync } from "./chaos.js"; import { proxyAndRecord } from "./recorder.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { proxyInterruption, withHangUpDetection } from "./response/proxy.js"; +import { deliverStream } from "./response/deliver.js"; +import type { FrameCodec } from "./response/frame-loop.js"; + +/** One `transcript.text.delta` frame of a streamed transcription. */ +interface TranscriptDelta { + type: "transcript.text.delta"; + delta: string; +} /** * Extract the multipart boundary string from a Content-Type header. @@ -191,15 +199,17 @@ export async function handleTranscription( return; } if (defaults.record) { - const outcome = await proxyAndRecord( - req, - res, - syntheticReq, - "openai", - req.url ?? defaultPath, - fixtures, - defaults, - raw, + const { outcome, clientHungUp } = await withHangUpDetection(req, res, () => + proxyAndRecord( + req, + res, + syntheticReq, + "openai", + req.url ?? defaultPath, + fixtures, + defaults, + raw, + ), ); if (outcome === "handled_by_hook") return; if (outcome !== "not_configured") { @@ -208,7 +218,12 @@ export async function handleTranscription( path, headers: flattenHeaders(req.headers), body: syntheticReq, - response: { status: res.statusCode ?? 200, fixture: null, source: "proxy" }, + response: { + status: res.statusCode ?? 200, + fixture: null, + source: "proxy", + ...proxyInterruption(res, clientHungUp), + }, }); return; } @@ -292,46 +307,25 @@ export async function handleTranscription( ...(t.languages !== undefined ? { languages: t.languages } : {}), ...(t.usage !== undefined ? { usage: t.usage } : {}), }; - const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); - const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; - res.writeHead(200, { - "Content-Type": "text/event-stream", - "Cache-Control": "no-cache", - Connection: "keep-alive", - }); - let chunkIndex = 0; - const interruption = createInterruptionSignal(fixture); + const frames: TranscriptDelta[] = []; for (let index = 0; index < t.text.length; index += chunkSize) { - const chunkDelay = calculateDelay( - chunkIndex, - fixture.streamingProfile, - latency, - fixture.recordedTimings, - replaySpeed, - ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) break; - res.write( - `data: ${JSON.stringify({ type: "transcript.text.delta", delta: t.text.slice(index, index + chunkSize) })}\n\n`, - ); - interruption?.tick(); - chunkIndex++; - } - if (interruption?.signal.aborted) { - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption.reason(); - interruption.cleanup(); - res.destroy(); - return; + frames.push({ type: "transcript.text.delta", delta: t.text.slice(index, index + chunkSize) }); } - res.write(`data: ${JSON.stringify(done)}\n\n`); // The live `gpt-transcribe&stream=true` stream ends with the `[DONE]` // sentinel after `transcript.text.done`. Omitting it leaves clients that // loop until the sentinel waiting on a stream that never terminates. - res.write("data: [DONE]\n\n"); - res.end(); - interruption?.cleanup(); + const codec: FrameCodec = { + ...sseCodec({ done: false }), + writeHeadImmediately: true, + trailer: () => [`data: ${JSON.stringify(done)}\n\n`, "data: [DONE]\n\n"], + }; + await deliverStream(res, journalEntry, fixture, codec, frames, { + latency: fixture.latency ?? defaults.latency, + streamingProfile: fixture.streamingProfile, + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); return; } diff --git a/src/types.ts b/src/types.ts index e60cb4f8..db585486 100644 --- a/src/types.ts +++ b/src/types.ts @@ -999,9 +999,26 @@ export interface Fixture { * non-streaming (`stream: false`) OpenAI chat completion replies (default OFF). * When enabled, the single JSON reply waits about as long as the streamed * reply would. Without it, non-streaming replies return immediately. Bare - * `latency` never delays non-streaming replies. Other providers ignore it. + * `latency` never delays non-streaming replies. Other providers ignore it; to + * delay them too, use {@link Fixture.nonStreamingTimingAllProviders}. */ nonStreamingTiming?: boolean; + /** + * Opt into `streamingProfile` / `recordedTimings` timing on this fixture's + * non-streaming (`stream: false`) replies on these providers (default OFF): + * OpenAI Chat Completions, Anthropic Messages, OpenAI Responses, Gemini and + * Vertex AI `generateContent`, Gemini Interactions, Cohere v2 chat, Bedrock + * InvokeModel and Converse, Ollama `/api/chat` and `/api/generate`, and + * OpenRouter chat completions (`/api/v1/chat/completions`), which is served + * as OpenAI Chat Completions, so `nonStreamingTiming` also delays it. + * Audio transcription replies are not delayed. The delay is the same as + * {@link Fixture.nonStreamingTiming} gives on OpenAI Chat Completions: the + * single JSON reply waits about as long as the streamed reply would. It + * needs a `streamingProfile` or `recordedTimings`. Bare `latency` never delays + * non-streaming replies, and error replies are never delayed. Setting it to + * `false` does not turn off `nonStreamingTiming: true`. + */ + nonStreamingTimingAllProviders?: boolean; metadata?: { systemHash?: string; toolsHash?: string; @@ -1208,6 +1225,8 @@ export interface FixtureFileEntry { openRouterProcessing?: boolean; /** See {@link Fixture.nonStreamingTiming}. */ nonStreamingTiming?: boolean; + /** See {@link Fixture.nonStreamingTimingAllProviders}. */ + nonStreamingTimingAllProviders?: boolean; metadata?: { systemHash?: string; toolsHash?: string; diff --git a/src/ws-framing.ts b/src/ws-framing.ts index 1fa2f154..a49daec3 100644 --- a/src/ws-framing.ts +++ b/src/ws-framing.ts @@ -71,12 +71,17 @@ export class WebSocketConnection extends EventEmitter { } }); - socket.on("close", () => { - if (!this.closed) { - this.closed = true; - this.release(new Error("WebSocket closed")); - this.emit("close", 1006, "Connection lost"); - } + socket.on("close", () => this.lost()); + + // A peer FIN without a close frame is a disconnect. HTTP upgrade sockets + // allow half-open, so without this the loss surfaces only on a later + // failed write. Data events precede "end", so frames that arrive just + // before the FIN still parse, but any turn they start is cancelled at once + // because the connection is then lost. + socket.on("end", () => { + if (this.closed) return; + this.socket.destroy(); + this.lost(); }); socket.on("error", (err: Error) => { @@ -135,6 +140,14 @@ export class WebSocketConnection extends EventEmitter { }); } + /** Mark an unannounced connection loss and emit close once. */ + private lost(): void { + if (this.closed) return; + this.closed = true; + this.release(new Error("WebSocket closed")); + this.emit("close", 1006, "Connection lost"); + } + private release(error: Error): void { this.buffer = Buffer.alloc(0); this.fragments = []; diff --git a/src/ws-gemini-live.ts b/src/ws-gemini-live.ts index e7674bdb..5ef6aab6 100644 --- a/src/ws-gemini-live.ts +++ b/src/ws-gemini-live.ts @@ -45,7 +45,7 @@ import { strictNoMatchMessage, strictNoMatchLogLine, } from "./helpers.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { markWsTurnStopped, watchWsClose, wsLifecycle } from "./response/lifecycle.js"; import { delay, calculateDelay } from "./sse-writer.js"; import { DEFAULT_TEST_ID, type Journal } from "./journal.js"; import type { Logger } from "./logger.js"; @@ -349,6 +349,7 @@ export function handleWebSocketGeminiLive( }, ): void { const { logger } = defaults; + watchWsClose(ws); const session: SessionState = { setupDone: false, model: defaults.model, @@ -629,14 +630,16 @@ async function processMessage( misbehavior.summary.fault === "empty-response") ) { const journalEntry = addResponseEntry(200); - const interruption = createInterruptionSignal(fixture); - const markInterrupted = () => { - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); + const lifecycle = wsLifecycle(ws, fixture); + const interruptTurn = () => { + markWsTurnStopped(journalEntry, ws, lifecycle); ws.destroy(); }; try { - if (ws.isClosed) return; + if (ws.isClosed) { + interruptTurn(); + return; + } const chunkDelay = calculateDelay( 0, undefined, @@ -644,12 +647,11 @@ async function processMessage( fixture.recordedTimings, fixture.replaySpeed ?? defaults.replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { - markInterrupted(); + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted || ws.isClosed) { + interruptTurn(); return; } - if (ws.isClosed) return; const malformed = misbehavior.summary.fault === "tool-args-invalid-json"; ws.send( JSON.stringify({ @@ -658,18 +660,22 @@ async function processMessage( : { modelTurn: { parts: [{ text: "" }] } }, }), ); - interruption?.tick(); - if (interruption?.signal.aborted) { - markInterrupted(); + lifecycle.tick(); + if (lifecycle.signal.aborted || ws.isClosed) { + interruptTurn(); return; } // Like ordinary text streaming, completion follows the counted content chunk. - if (!malformed && !ws.isClosed) { + if (!malformed) { ws.send(JSON.stringify({ serverContent: { turnComplete: true } })); + if (ws.isClosed) { + interruptTurn(); + return; + } session.conversationHistory.push({ role: "assistant", content: "" }); } } finally { - interruption?.cleanup(); + lifecycle.dispose(); } return; } @@ -748,6 +754,14 @@ async function processMessage( resolvedId: tc.id ?? generateToolCallId(), })); + // This branch never waits, so it holds no lifecycle. A closed socket is + // journaled by the shared WS rule, and the turn stops there. + const stoppedByClose = (): boolean => { + if (!ws.isClosed) return false; + markWsTurnStopped(journalEntry, ws); + return true; + }; + if (stoppedByClose()) return; if (resolvedToolCalls.length === 0) { ws.send( JSON.stringify({ @@ -756,19 +770,16 @@ async function processMessage( ); } else { ws.send(JSON.stringify({ serverContent: { modelTurn: { parts } } })); - - if (!ws.isClosed) { - const functionCalls = resolvedToolCalls.map((tc) => { - const argsObj = liveToolArguments(tc); - return { name: tc.name, args: argsObj, id: tc.resolvedId }; - }); - ws.send(JSON.stringify({ toolCall: { functionCalls } })); - } - - if (!ws.isClosed) { - ws.send(JSON.stringify({ serverContent: { turnComplete: true } })); - } + if (stoppedByClose()) return; + const functionCalls = resolvedToolCalls.map((tc) => { + const argsObj = liveToolArguments(tc); + return { name: tc.name, args: argsObj, id: tc.resolvedId }; + }); + ws.send(JSON.stringify({ toolCall: { functionCalls } })); + if (stoppedByClose()) return; + ws.send(JSON.stringify({ serverContent: { turnComplete: true } })); } + if (stoppedByClose()) return; session.conversationHistory.push({ role: "assistant", @@ -801,7 +812,7 @@ async function processMessage( // payload — a silent drop. Legacy fixtures (no `blocks`) skip this entirely. if (response.blocks && response.blocks.length > 0) { const resolvedBlocks = preflightBlocks(response, journalEntry); - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; let chunkIndex = 0; @@ -826,8 +837,8 @@ async function processMessage( recordedTimings, replaySpeed, ); - if (tcDelay > 0) await delay(tcDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (tcDelay > 0) await delay(tcDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -855,8 +866,8 @@ async function processMessage( type: "function" as const, function: { name: block.name, arguments: block.arguments }, }); - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -882,8 +893,8 @@ async function processMessage( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break outer; } @@ -902,8 +913,8 @@ async function processMessage( break outer; } chunkIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break outer; } @@ -911,19 +922,19 @@ async function processMessage( } } - if (interrupted) { + if (interrupted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); - // Send turnComplete - if (!ws.isClosed) { - ws.send(JSON.stringify({ serverContent: { turnComplete: true } })); + ws.send(JSON.stringify({ serverContent: { turnComplete: true } })); + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; } session.conversationHistory.push({ @@ -941,7 +952,7 @@ async function processMessage( chunkList.push(content.slice(i, i + chunkSize)); } - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; let interrupted = false; @@ -961,8 +972,8 @@ async function processMessage( for (let i = 0; i < chunkList.length; i++) { if (ws.isClosed) break; const chunkDelay = calculateDelay(i, undefined, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -980,19 +991,18 @@ async function processMessage( defaults.logger.debug("[gemini-live] send failed during text streaming, closing", err); break; } - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } } } - if (interrupted) { + if (interrupted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1011,12 +1021,11 @@ async function processMessage( recordedTimings, replaySpeed, ); - if (tcDelay > 0) await delay(tcDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (tcDelay > 0) await delay(tcDelay, lifecycle.signal); + if (lifecycle.signal.aborted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1030,26 +1039,27 @@ async function processMessage( }); ws.send(JSON.stringify({ toolCall: { functionCalls } })); - interruption?.tick(); + lifecycle.tick(); } - if (interruption?.signal.aborted) { + if (lifecycle.signal.aborted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); // Send turnComplete - if (!ws.isClosed) { - ws.send( - JSON.stringify({ - serverContent: { turnComplete: true }, - }), - ); + ws.send( + JSON.stringify({ + serverContent: { turnComplete: true }, + }), + ); + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; } // Add to conversation history using the same resolved IDs from the wire message @@ -1075,7 +1085,12 @@ async function processMessage( const content = response.content; if (content.length === 0) { - if (ws.isClosed) return; + // No wait and no lifecycle here: as in the audio branch, a closed + // socket is journaled by the shared WS rule, and the turn stops there. + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws); + return; + } // Empty content: send empty modelTurn, then separate turnComplete ws.send( JSON.stringify({ @@ -1084,11 +1099,16 @@ async function processMessage( }, }), ); + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws); + return; + } ws.send( JSON.stringify({ serverContent: { turnComplete: true }, }), ); + if (ws.isClosed) markWsTurnStopped(journalEntry, ws); return; } @@ -1098,7 +1118,7 @@ async function processMessage( chunks.push(content.slice(i, i + chunkSize)); } - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; let interrupted = false; @@ -1107,8 +1127,8 @@ async function processMessage( for (let i = 0; i < chunks.length; i++) { if (ws.isClosed) break; const chunkDelay = calculateDelay(i, undefined, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -1126,30 +1146,31 @@ async function processMessage( defaults.logger.debug("[gemini-live] send failed during text streaming, closing", err); break; } - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } } - if (interrupted) { + if (interrupted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); // Send separate turnComplete message - if (!ws.isClosed) { - ws.send( - JSON.stringify({ - serverContent: { turnComplete: true }, - }), - ); + ws.send( + JSON.stringify({ + serverContent: { turnComplete: true }, + }), + ); + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; } // Add assistant response to conversation history @@ -1163,25 +1184,21 @@ async function processMessage( const functionToolCalls = preflightToolArguments(response, journalEntry); - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } const tcDelay = calculateDelay(0, undefined, latency, recordedTimings, replaySpeed); - if (tcDelay > 0) await delay(tcDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (tcDelay > 0) await delay(tcDelay, lifecycle.signal); + if (lifecycle.signal.aborted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); - return; - } - if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1201,25 +1218,26 @@ async function processMessage( }); ws.send(JSON.stringify({ toolCall: { functionCalls } })); - interruption?.tick(); + lifecycle.tick(); - if (interruption?.signal.aborted) { + if (lifecycle.signal.aborted || ws.isClosed) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); // Send turnComplete after tool call - if (!ws.isClosed) { - ws.send( - JSON.stringify({ - serverContent: { turnComplete: true }, - }), - ); + ws.send( + JSON.stringify({ + serverContent: { turnComplete: true }, + }), + ); + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; } // Add assistant tool_calls to conversation history using the same resolved IDs diff --git a/src/ws-realtime.ts b/src/ws-realtime.ts index de648009..17758a05 100644 --- a/src/ws-realtime.ts +++ b/src/ws-realtime.ts @@ -38,7 +38,12 @@ import { strictNoMatchLogLine, } from "./helpers.js"; import { planMisbehavior, recordMisbehaviorOutcome, type MisbehaviorPlan } from "./misbehavior.js"; -import { createInterruptionSignal } from "./interruption.js"; +import { + markInterrupted, + markWsTurnStopped, + watchWsClose, + wsLifecycle, +} from "./response/lifecycle.js"; import { delay, calculateDelay } from "./sse-writer.js"; import { DEFAULT_TEST_ID, type Journal } from "./journal.js"; import type { Logger } from "./logger.js"; @@ -563,6 +568,7 @@ export function handleWebSocketRealtime( ): void { const { logger } = defaults; const sessionId = realtimeId("sess"); + watchWsClose(ws); const isBeta = defaults.upgradeHeaders?.["openai-beta"] ? String(defaults.upgradeHeaders["openai-beta"]).includes("realtime=v1") @@ -1091,7 +1097,7 @@ async function emitLiveTranscriptionEvents( const latency = fixture.latency ?? defaults.latency; const chunkSize = Math.max(1, fixture.chunkSize ?? defaults.chunkSize); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); let chunkIndex = 0; for (let index = 0; index < transcript.length; index += chunkSize) { const chunkDelay = calculateDelay( @@ -1101,14 +1107,15 @@ async function emitLiveTranscriptionEvents( fixture.recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted || ws.isClosed) { - if (interruption?.signal.aborted) { - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption.reason(); + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted || ws.isClosed) { + if (lifecycle.signal.aborted) { + markInterrupted(journalEntry, lifecycle.reason()); ws.destroy(); + } else { + markWsTurnStopped(journalEntry, ws, lifecycle); } - interruption?.cleanup(); + lifecycle.dispose(); return; } sendEvent( @@ -1121,14 +1128,13 @@ async function emitLiveTranscriptionEvents( }, isBeta, ); - interruption?.tick(); + lifecycle.tick(); chunkIndex++; } - if (interruption?.signal.aborted) { - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption.reason(); + if (lifecycle.signal.aborted || ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); ws.destroy(); - interruption.cleanup(); + lifecycle.dispose(); return; } sendEvent( @@ -1145,7 +1151,7 @@ async function emitLiveTranscriptionEvents( }, isBeta, ); - interruption?.cleanup(); + lifecycle.dispose(); } function sendLiveTranscriptionFailure( @@ -1573,7 +1579,7 @@ async function handleResponseCreate( isBeta, ); - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); let interrupted = false; const allOutputItems: unknown[] = []; @@ -1641,8 +1647,8 @@ async function handleResponseCreate( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -1666,8 +1672,8 @@ async function handleResponseCreate( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -1675,14 +1681,14 @@ async function handleResponseCreate( if (interrupted) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markInterrupted(journalEntry, lifecycle.reason()); + lifecycle.dispose(); return; } if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1701,7 +1707,8 @@ async function handleResponseCreate( ); if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1720,7 +1727,8 @@ async function handleResponseCreate( ); if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1754,7 +1762,8 @@ async function handleResponseCreate( ); if (ws.isClosed) { - interruption?.cleanup(); + markWsTurnStopped(journalEntry, ws, lifecycle); + lifecycle.dispose(); return; } @@ -1809,8 +1818,8 @@ async function handleResponseCreate( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -1837,8 +1846,8 @@ async function handleResponseCreate( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -1901,15 +1910,17 @@ async function handleResponseCreate( if (interrupted) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markInterrupted(journalEntry, lifecycle.reason()); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); - if (ws.isClosed) return; + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; + } // response.done sendEvent( @@ -2010,7 +2021,7 @@ async function handleResponseCreate( const content = response.content ?? ""; const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); let interrupted = false; let eventIndex = 0; @@ -2023,8 +2034,8 @@ async function handleResponseCreate( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2048,8 +2059,8 @@ async function handleResponseCreate( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2057,15 +2068,17 @@ async function handleResponseCreate( if (interrupted) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markInterrupted(journalEntry, lifecycle.reason()); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); - if (ws.isClosed) return; + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; + } // response.output_text.done sendEvent( @@ -2181,7 +2194,7 @@ async function handleResponseCreate( ); const outputItems: unknown[] = []; - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; const { recordedTimings } = fixture; let interrupted = false; @@ -2233,8 +2246,8 @@ async function handleResponseCreate( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2261,8 +2274,8 @@ async function handleResponseCreate( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2321,15 +2334,17 @@ async function handleResponseCreate( if (interrupted) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markInterrupted(journalEntry, lifecycle.reason()); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); - if (ws.isClosed) return; + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; + } // response.done sendEvent( @@ -2407,7 +2422,7 @@ async function streamRealtimeBlocks( isBeta, ); - const interruption = createInterruptionSignal(fixture); + const lifecycle = wsLifecycle(ws, fixture); let interrupted = false; const allOutputItems: unknown[] = []; const replaySpeed = fixture.replaySpeed ?? defaults.replaySpeed; @@ -2476,8 +2491,8 @@ async function streamRealtimeBlocks( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2504,8 +2519,8 @@ async function streamRealtimeBlocks( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2616,8 +2631,8 @@ async function streamRealtimeBlocks( recordedTimings, replaySpeed, ); - if (chunkDelay > 0) await delay(chunkDelay, interruption?.signal); - if (interruption?.signal.aborted) { + if (chunkDelay > 0) await delay(chunkDelay, lifecycle.signal); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2644,8 +2659,8 @@ async function streamRealtimeBlocks( break; } eventIndex++; - interruption?.tick(); - if (interruption?.signal.aborted) { + lifecycle.tick(); + if (lifecycle.signal.aborted) { interrupted = true; break; } @@ -2703,15 +2718,17 @@ async function streamRealtimeBlocks( if (interrupted) { ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - interruption?.cleanup(); + markInterrupted(journalEntry, lifecycle.reason()); + lifecycle.dispose(); return; } - interruption?.cleanup(); + lifecycle.dispose(); - if (ws.isClosed) return; + if (ws.isClosed) { + markWsTurnStopped(journalEntry, ws, lifecycle); + return; + } // response.done sendEvent( diff --git a/src/ws-responses.ts b/src/ws-responses.ts index b2615177..5976f207 100644 --- a/src/ws-responses.ts +++ b/src/ws-responses.ts @@ -6,7 +6,7 @@ * handler, but as individual WebSocket text frames. */ -import type { Fixture, HandlerDefaults } from "./types.js"; +import type { Fixture, HandlerDefaults, JournalEntry, RecordedTimings } from "./types.js"; import { planMisbehavior, recordMisbehaviorOutcome } from "./misbehavior.js"; import { matchFixtureDiagnostic } from "./router.js"; import { @@ -41,12 +41,13 @@ import { servedToolCalls, withoutLegacyNamespaces, } from "./helpers.js"; -import { createInterruptionSignal } from "./interruption.js"; -import { delay, calculateDelay } from "./sse-writer.js"; import { DEFAULT_TEST_ID, type Journal } from "./journal.js"; import type { WebSocketConnection } from "./ws-framing.js"; import { resolveRequestDecisions, strictOverrideField } from "./response/policy.js"; import { buildOpenAIError } from "./response/errors.js"; +import { markWsTurnStopped, watchWsClose, wsLifecycle } from "./response/lifecycle.js"; +import { runFrameLoop, wsSink, type FrameCodec } from "./response/frame-loop.js"; +import { extendRecordedTimings } from "./response/timing.js"; type ResponsesWebSocketDefaults = Omit & { replaySpeed?: number; @@ -96,6 +97,7 @@ export function handleWebSocketResponses( defaults: ResponsesWebSocketDefaults, ): void { const { logger } = defaults; + watchWsClose(ws); // Serialize message processing to prevent event interleaving let pending = Promise.resolve(); ws.on("message", (raw: string) => { @@ -310,35 +312,17 @@ async function processMessage( defaults: misbehaviorDefaults, testId, }); - const timings = fixture.recordedTimings; - const gaps = timings?.interChunkDelaysMs; // K4 can add events; extend only its replay using the last recorded gap. - const appliedTimings = - prepared.duplicateId && timings && gaps - ? { - ...timings, - interChunkDelaysMs: Array.from( - { length: Math.max(gaps.length, output.events.length - 1) }, - (_, index) => gaps[index] ?? gaps.at(-1) ?? 0, - ), - } - : timings; - const interruption = createInterruptionSignal(fixture); - const completed = await sendEvents( - ws, - output.events, - latency, - interruption?.signal, - interruption?.tick, - appliedTimings, - fixture.replaySpeed ?? defaults.replaySpeed, + const appliedTimings = extendRecordedTimings( + fixture.recordedTimings, + output.events.length, + !!prepared.duplicateId, ); - if (!completed) { - ws.destroy(); - entry.response.interrupted = true; - entry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + await deliverTurn(ws, entry, fixture, output.events, { + latency, + recordedTimings: appliedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); return; } @@ -407,22 +391,11 @@ async function processMessage( throw error; } - const interruption = createInterruptionSignal(fixture); - const completed = await sendEvents( - ws, - events, + await deliverTurn(ws, journalEntry, fixture, events, { latency, - interruption?.signal, - interruption?.tick, - fixture.recordedTimings, - fixture.replaySpeed ?? defaults.replaySpeed, - ); - if (!completed) { - ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); return; } @@ -456,22 +429,11 @@ async function processMessage( synthesizeSummarylessReasoning, ), ); - const interruption = createInterruptionSignal(fixture); - const completed = await sendEvents( - ws, - events, + await deliverTurn(ws, journalEntry, fixture, events, { latency, - interruption?.signal, - interruption?.tick, - fixture.recordedTimings, - fixture.replaySpeed ?? defaults.replaySpeed, - ); - if (!completed) { - ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); return; } @@ -511,22 +473,11 @@ async function processMessage( if (isFixtureToolCallError(error)) journalEntry.response.error = error.message; throw error; } - const interruption = createInterruptionSignal(fixture); - const completed = await sendEvents( - ws, - events, + await deliverTurn(ws, journalEntry, fixture, events, { latency, - interruption?.signal, - interruption?.tick, - fixture.recordedTimings, - fixture.replaySpeed ?? defaults.replaySpeed, - ); - if (!completed) { - ws.destroy(); - journalEntry.response.interrupted = true; - journalEntry.response.interruptReason = interruption?.reason(); - } - interruption?.cleanup(); + recordedTimings: fixture.recordedTimings, + replaySpeed: fixture.replaySpeed ?? defaults.replaySpeed, + }); return; } @@ -546,26 +497,38 @@ async function processMessage( ); } -async function sendEvents( +/** WS Responses frames: one JSON text message per event, no head, no trailer. */ +const wsEventCodec: FrameCodec = { + headers: {}, + encode: (event) => JSON.stringify(event), + checkGoneBeforeDelay: true, +}; + +/** + * Send one turn's events under a WS lifecycle. The lifecycle owns the fixture + * interruption and also stops the turn when the client closes, so a pending + * delay ends at once. A stopped turn marks the entry by the shared WS rule + * ({@link markWsTurnStopped}) and destroys the socket. + * `streamingProfile` is not applied on this transport, as before. + */ +async function deliverTurn( ws: WebSocketConnection, + entry: JournalEntry, + fixture: Fixture, events: ResponsesSSEEvent[], - latency: number, - signal?: AbortSignal, - onChunkSent?: () => void, - recordedTimings?: import("./types.js").RecordedTimings, - replaySpeed?: number, -): Promise { - let eventIndex = 0; - for (const event of events) { - if (ws.isClosed) return false; - const chunkDelay = calculateDelay(eventIndex, undefined, latency, recordedTimings, replaySpeed); - if (chunkDelay > 0) await delay(chunkDelay, signal); - if (signal?.aborted) return false; - if (ws.isClosed) return false; - ws.send(JSON.stringify(event)); - eventIndex++; - onChunkSent?.(); - if (signal?.aborted) return false; + timing: { latency: number; recordedTimings?: RecordedTimings; replaySpeed?: number }, +): Promise { + const lifecycle = wsLifecycle(ws, fixture); + try { + const completed = await runFrameLoop(wsSink(ws), events, wsEventCodec, timing, { + lifecycle, + onChunkSent: lifecycle.tick, + }); + if (!completed) { + markWsTurnStopped(entry, ws, lifecycle); + ws.destroy(); + } + } finally { + lifecycle.dispose(); } - return true; }