diff --git a/README.md b/README.md index 3ef0fca6..e8d01692 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ once; without it the run fails partway through with a 401/403 from the Hub. ## Drive it from an agent -Then just ask. The agent has 62 tools covering the whole surface — the +Then just ask. The agent has 65 tools covering the whole surface — the workflow catalog, the real diffusers pipeline signatures, the job queue, the gallery, the model cache: diff --git a/docs/ACCELERATION.md b/docs/ACCELERATION.md index 1d64c59c..e864eb7b 100644 --- a/docs/ACCELERATION.md +++ b/docs/ACCELERATION.md @@ -8,14 +8,29 @@ For ready-made configurations that combine these levers per model family, see [R Applied at pipeline load time via the `cache` configuration. Hooks auto-reset between runs. +`validate_workflow` warns (it never refuses) about two cache settings that cost a render to find out: + +- **`cache_short_schedule`: a threshold cache on a short schedule.** `first_block` and `mag` decide each step's skip on a residual threshold, and the consecutive steps of a short schedule differ too much for it: under 20 `num_inference_steps` such a cache will likely skip nothing and still perturbs the result. On MiniMax-H3 a `first_block` cache at `0.05` skipped nothing at 12 steps and took 22% off at 30 ([FirstBlockCache](#firstblockcache)). Drop the cache or run 20 steps or more. A turbo schedule (9 steps or fewer) never benefits. `taylorseer` and `faster` skip on a fixed interval and `text_kv` skips no step, so the rule does not apply to them. +- **`taylorseer_state_unestimated`: TaylorSeer's state is not in the VRAM estimate.** Said whenever `taylorseer` is set - see [TaylorSeerCache](#taylorseercache). + ### Models diffusers has not registered `first_block`, `mag` and `layer_skip` look a model's transformer block class up in diffusers' own registry and raise when it is absent, which is how a model that supports `enable_cache()` ends up with no usable cache. [cache_blocks.json](../dw/cache_blocks.json) fills those gaps in, registering the missing block metadata on demand; entries become -redundant, not wrong, once diffusers registers the same class upstream. MiniMax-H3 and -LTX-2 are listed there today. +redundant, not wrong, once diffusers registers the same class upstream. MiniMax-H3, +LTX-2 and Qwen-Image-2.1 are listed there today. + +Qwen-Image-2.1's pipeline keeps a key/value cache of its own (`use_kv_cache`, on by +default) that drops the text tokens from the sequence after the first step. `first_block` +and `mag` compare one step's residual against the previous one, so the sequence length +changing under them fails the second step; pass `"use_kv_cache": false` in the step's +`arguments` to cache it. + +A transformer that does not support `enable_cache()` at all - Z-Image's, for one - takes +no cache of any type. The step still runs, uncached, and the job carries a warning that the +cache was not applied. LTX-2's block needs one thing more than the metadata diffusers defines. It returns two streams - video and audio - and diffusers reads the second one back out of a forward @@ -40,7 +55,7 @@ Simplest and broadest support. Compares first-block residuals to decide whether } ``` -Higher threshold = more speedup, more quality loss. Start with `0.05` and increase to taste. +Higher threshold = more speedup, more quality loss. Start with `0.05` and increase to taste. On MiniMax-H3 (RTX 3090, 1344x768, 4 references) `0.05` took 22% off a 30-step base-model job and nothing off a 12-step or turbo one, which it never skips; `torch.compile` there cost 289 s on the first pass for ~1% per pass, `text_kv` did nothing on Ref2VA, and `taylorseer` ran out of memory. Every MiniMax-H3 template takes a `cache` argument, `null` (off) by default and wired to the H3 step only: pass `{"type": "first_block", "threshold": 0.05}` rather than editing the JSON, and only at ~20 or more base steps, never with the turbo LoRAs. **Example:** [step-caching.json](../workflows/templates/step-caching.json) @@ -63,7 +78,7 @@ Magnitude-based caching with error accumulation. Requires `num_inference_steps` | -------- | ------- | ----------- | | `mag_ratios` | required | Preset name or explicit per-step ratio array — see below | | `threshold` | 0.06 | Accumulated error threshold for skipping | -| `num_inference_steps` | required | Must match pipeline arguments | +| `num_inference_steps` | required | The transformer forwards the call runs: the pipeline's `num_inference_steps`, except where the scheduler runs fewer (MiniMax-H3 runs one fewer) | | `max_skip_steps` | 3 | Max consecutive steps to skip | | `retention_ratio` | 0.2 | Fraction of initial steps where skipping is disabled | | `calibrate` | false | Measure ratios for a new model instead of skipping — see below | @@ -75,7 +90,7 @@ MagCache needs to know how each denoising step's output magnitude typically beha - **A preset name** — `"mag_ratios": "flux"` resolves to the ratios diffusers ships for Flux. Any preset a later diffusers release adds is usable by name without a change here. - **An explicit array** — `"mag_ratios": [1.0, 0.98, 0.96, ...]`. The array is interpolated automatically when its length differs from `num_inference_steps`, so ratios measured at one step count can be reused at another. -For a model with no preset, run once with `"calibrate": true`. Calibration skips nothing and logs the measured ratios at the end of the run; paste that array into `mag_ratios` and drop the `calibrate` flag for subsequent runs. +For a model with no preset, run once with `"calibrate": true`. Calibration skips nothing. At the end of the run the measured ratios appear as a job warning and a `log` event (`get_job`'s `warnings`, `get_job_events`, `export_job`): `MagCache calibration (N steps): [ ... ] — paste into mag_ratios`. While calibrating, the run's own `num_inference_steps` is the step count, so the cache's can be left at its default (a mismatch is reported as a warning). The cache attaches to a pipeline's `transformer`, and only one that supports `enable_cache()`: a UNet pipeline such as SD 1.5 has no transformer, and Z-Image's transformer has no cache support, so both get a warning that the cache was not applied and no calibration. Paste that array into `mag_ratios` and drop the `calibrate` flag for subsequent runs. `N` is the forwards the run actually made, which is not always the call's `num_inference_steps`: MiniMax-H3's scheduler turns N sigmas into N - 1 evaluations, so a 30-step H3 call reports 29 ratios. Set the cache's `num_inference_steps` to that `N` when you paste them, so each ratio lines up with its forward. A pipeline that runs classifier-free guidance as two transformer passes in their own `cond` and `uncond` cache contexts (Qwen-Image-2.1 with `true_cfg_scale` above 1 and a `negative_prompt`, or the other Qwen-Image pipelines) keeps a separate state for each, so it emits **two** lists, each labeled `[cond]` or `[uncond]`; use the `cond` one, since `mag_ratios` takes one array. A guidance-distilled pipeline such as Flux, or one that batches both passes into one forward emits one unlabeled list. ```json "cache": { "type": "mag", "calibrate": true, "num_inference_steps": 28 } @@ -96,7 +111,9 @@ Taylor series approximation of cached outputs: | Property | Default | Description | | -------- | ------- | ----------- | | `cache_interval` | 5 | Full computation every N steps | -| `max_order` | 1 | Taylor series order (higher = better approximation, more memory) | +| `max_order` | 1 | Taylor series order (higher = better approximation, and more state kept on the GPU) | + +TaylorSeer keeps per-block state on the GPU that no `vram_estimate` charges, so a job that fits without the cache can run out of memory with it. On MiniMax-H3 at 1344x768, 30 steps and `max_order` 1, a run that `validate_workflow` had passed ran out of memory 2 min in on a 24 GB card. Validate says so as `taylorseer_state_unestimated` whenever `taylorseer` is set, with the job's estimate and the card's capacity when both are known; the state itself is on top of that figure, and is not sized. ### FasterCache @@ -110,6 +127,10 @@ Experimental, video-oriented. Uses FFT frequency decomposition: Best for video models like CogVideoX. No additional parameters needed for basic use. +| Property | Default | Description | +| -------- | ------- | ----------- | +| `is_guidance_distilled` | false | Set `true` for a guidance-distilled model (FLUX.1-dev, FLUX.1-Krea-dev), where `guidance_scale` is an embedding and the transformer sees a batch of 1. FasterCache otherwise assumes a cond/uncond batch of 2 and fails at the first step with `tuple index out of range` (#895). Validate cannot tell a distilled model from a CFG one, so it does not check this | + ### TextKVCache Caches the transformer's key/value projections of the (unchanging) text @@ -123,6 +144,10 @@ embeddings across denoising steps, recomputing only what the latents need: No parameters. +### Reading the skip counts + +A step that set a `cache` reports `timing.cache` on its manifest entry (fields in [WORKSPACES.md](WORKSPACES.md#step-timing)). `hit_rate` is the fraction of denoiser forwards whose remaining blocks were skipped. It counts forwards, not steps: classifier-free guidance runs two per step, so a 30-step run is 60 forwards. Only `first_block` and `mag` are counted; `taylorseer`, `faster` and `text_kv` report null counts. A cache with zero skips can still change the output, so a 0-skip run is not guaranteed bit-identical to an uncached one. With `first_block` the head block always runs, and enabling any cache wraps the blocks in hooks that recompute the residual (the tail's output minus the head's, in the transformer's dtype), which changes the order and precision of operations; the MiniMax-H3 matrix run P2 saw a changed output with nothing skipped. Judge quality against an uncached run, not against `skipped == 0`. A threshold sweep (#875) reads `hit_rate` against quality. + ## Attention Backends Select the attention implementation diffusers uses for the duration of each pipeline call, via a context manager wrapped around `pipeline(...)`: @@ -136,15 +161,15 @@ Select the attention implementation diffusers uses for the duration of each pipe Common values: `"flash"`, `"flash_hub"`, `"sage"`, `"sage_hub"`, `"native"`, `"flex"`. The full set is diffusers' `AttentionBackendName` enum - availability depends on what's installed (`flash-attn`, `sageattention`, etc.) and the platform. `_hub`-suffixed backends are fetched from the Hugging Face Hub kernel registry on first use, which needs the `kernels` package installed (`pip install kernels`) - it is not a dw dependency, and no bundled workflow sets a backend, so each runs on a plain install. diffusers also refuses a Hub kernel whose publisher it does not list as trusted - `sage_hub` among them, since SageAttention 2 comes from `SageAttention/sage-attention` - until `DIFFUSERS_TRUST_REMOTE_KERNELS=true`, which allows remote code from any such publisher. On an RTX 3090 `sage_hub` measured about 30% faster per MiniMax-H3 step (13.8 s against 19.2 s at 960x544x124, 54.7 s against 81.2 s at 345 frames); the local `sage` backend needs `sageattention>=2.1.1`, which PyPI does not carry. -The MiniMax-H3 baselines expose this as a variable: `attention_backend` on -[video-with-audio.json](../workflows/templates/minimax/video-with-audio.json), -[video-with-audio-768p.json](../workflows/templates/minimax/video-with-audio-768p.json), -[image-to-video.json](../workflows/templates/minimax/image-to-video.json) and -[reference-to-video.json](../workflows/templates/minimax/reference-to-video.json), pinned -on the transformer and `null` by default. Pass `attention_backend=sage_hub` to a box with -the `kernels` package and `DIFFUSERS_TRUST_REMOTE_KERNELS=true` in its `.env`; on MPS -leave it null. The other H3 templates take the same key under `components.transformer` -(`transformer_ref` for a ref2va workflow). LTX-2.5 cannot use it: its transformer +Every MiniMax-H3 template in [templates/minimax/](../workflows/templates/minimax/) +exposes this as a variable: `attention_backend`, pinned on the H3 step's transformer +(`components.transformer`, or `components.transformer_ref` for a ref2va workflow) and +`null` by default. That covers the shots, chains and episodes - `dialogue-short`, +`shots-batch`, `music-video` and `storyboard` among them - and only their H3 steps: a +Z-Image still or Music3 song beside an H3 shot keeps the process default. Pass +`attention_backend=sage_hub` to a box with the `kernels` package and +`DIFFUSERS_TRUST_REMOTE_KERNELS=true` in its `.env`; on MPS leave it null, since a +CUDA-only kernel fails there at load (validation warns, below). LTX-2.5 cannot use it: its transformer passes an attention mask, and sage attention refuses one ("`attn_mask` is not supported for sage attention", measured 2026-10-10), so the LTX templates expose no backend. @@ -160,6 +185,8 @@ A component can also pin its backend persistently instead, via `set_attention_ba Prefer the pinned form for a compiled component - the per-call context manager switches implementations under the compiled graph and forces a recompile on every run. +The flash, flash-3, sage and xformers backends need CUDA, `_hub` variants included. On a server whose device is not CUDA (`mps`, `cpu`), `validate_workflow` warns `attention_backend_unavailable` for one set at pipeline or component level, naming the backend and the device: the job would fail at load. It never refuses, and a `native`-family backend never warns. A backend diffusers adds later is not on the list until dw's is updated, and draws no warning. + Either form stays on what it names. diffusers' `set_attention_backend` also makes its backend the process-wide default, which the persistent worker would keep for every later model; dw restores the default after pinning, so an H3 `sage_hub` pin cannot reach a Music3 or LTX job that follows it. The server-wide default is `DIFFUSERS_ATTN_BACKEND` in `.env` - leave it `native`, since it reaches every model and sage cannot serve all of them. ## Attention Slicing @@ -201,10 +228,11 @@ Compile a component once it is fully configured - the graph captures final dtype Typical gains are 1.3-1.5x on diffusion transformers, and compilation stacks with the caches above. Notes: - **First run pays the compile cost.** The [server](SERVER.md)'s persistent worker keeps compiled pipelines loaded between runs, so the cost is paid once per session rather than once per generation. A new session still retraces, but the kernels it built come back from the on-disk cache, which dw keeps across reboots (`TORCHINDUCTOR_CACHE_DIR`, see [Environment Defaults](#environment-defaults)). SDNQ's `triton.jit`/autotune kernels are cached separately by Triton itself (`TRITON_CACHE_DIR`, else `$TRITON_HOME/.triton/cache`, else `~/.triton/cache`); dw sets no `TRITON_*` variable and leaves that cache where it is, shared with any other Triton user on the machine. Both caches' sizes show in `get_server_info`'s `caches` and `GET /api/caches`, and `DELETE /api/caches/{name}` empties one ([SERVER.md](SERVER.md)); the next first step of each shape pays the compile or re-autotune again (100-300 s per shape on MiniMax-H3 for Triton). +- **Measure the cold cost from `timing`.** A pipeline step's manifest entry carries `timing` ([WORKSPACES.md](WORKSPACES.md#step-timing)): on a cold run the compile cost is about `first_pass_s - pass_s`, and `compiled` says whether compile was on. `pass_s` is wall time at step boundaries, not kernel time - on CUDA kernels run asynchronously, so a pass boundary is when the host saw it. - **Pin the attention backend** on a compiled component (`"attention_backend"` in the same `components` entry) rather than using the pipeline-level per-call context manager, which forces recompiles. - **Quantize-on-load time is a separate cost.** An SDNQ on-the-fly component re-reads and re-quantizes its bf16 weights on every cold load; `DW_QUANTIZED_CACHE=1` saves the quantized result and reads it back instead - see [QUANTIZATION.md](QUANTIZATION.md#cached-on-the-fly-dw_quantized_cache). -- **Composes with offloading**: apply `group_offload` and `compile` on the same component and the offload hooks are installed first, as required. Skipped with a warning on MPS. -- **Don't combine `fullgraph` with a `cache`**: the cache hooks decide skip-or-compute per step, a data-dependent branch diffusers wraps in `torch.compiler.disable` - it needs the graph break that `fullgraph: true` forbids. Compile with the default (partial) graph mode when a cache is active. +- **Composes with offloading**: apply `group_offload` and `compile` on the same component and the offload hooks are installed first, as required. Skipped with a warning on MPS. The compile cost is paid again every time the worker releases the pipeline ("Workflow changed - releasing cached models"), i.e. on any workflow change, so it pays off only across many runs of one workflow at one shape. +- **Don't combine `fullgraph` with a `cache`**: the cache hooks decide skip-or-compute per step, a data-dependent branch diffusers wraps in `torch.compiler.disable` - it needs the graph break that `fullgraph: true` forbids. Compile with the default (partial) graph mode when a cache is active. `validate_workflow` warns `compile_fullgraph_with_cache` on a component that sets `fullgraph: true` in a pipeline with a `cache`. - **TorchAO quantization needs compile to be fast** - see [QUANTIZATION.md](QUANTIZATION.md#torchao). **Example:** [flux-dev-compile.json](../workflows/models/flux-dev-compile.json), [flux-torchao.json](../workflows/models/flux-torchao.json) @@ -306,7 +334,7 @@ For faster model downloads, optionally `pip install hf_transfer` and set `HF_HUB Apple Silicon has narrower acceleration support than CUDA: -- No flash-attn, no Triton, no bitsandbytes - `attention_backend` is effectively CUDA-only; use `"native"`-family backends or leave it unset on MPS. `compile` is skipped with a warning (inductor support on MPS is immature). A workflow that loads a bitsandbytes checkpoint (`flux2-dev`, `multi-image-reference`) or a TorchAO int4 config (`flux-torchao`) has no Mac path yet. +- No flash-attn, no Triton, no bitsandbytes - `attention_backend` is effectively CUDA-only; use `"native"`-family backends or leave it unset on MPS. Validate warns `attention_backend_unavailable` for a CUDA-only one ([Attention Backends](#attention-backends)). `compile` is skipped with a warning (inductor support on MPS is immature). A workflow that loads a bitsandbytes checkpoint (`flux2-dev`, `multi-image-reference`) or a TorchAO int4 config (`flux-torchao`) has no Mac path yet. - Workflows written for CUDA are adapted rather than refused: a `cuda` device becomes `mps`, SDNQ's `quantization_device` follows it and its quantized matmul is switched off (it runs through `torch._int_mm`, ~500x slower on MPS), group-offload CUDA streams are dropped, and `"offload": "sequential"` becomes `"model"`. Each change is logged as a warning. - Attention slicing is automatic; set `disable_attention_slicing` for SDXL-family models (head dim 64), where it is 2.4x slower in attention. It helps SD 1.5. - `float16` can produce NaN values on Apple Silicon - use `bfloat16`; dw only warns, it doesn't override the dtype for you. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 43f13588..4cc5c11c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -31,12 +31,14 @@ one to open. | Quantized component cache | `dw/quantized_cache.py`: `load_components`, `from_pretrained`, `cache_key` | A key that can't pin the weights isn't cached, the cache never fails a job, and the save comes before pruning, so an entry is the whole component. | `tests/test_quantized_cache.py` | | LoRAs | `dw/pipeline_processors/adapters.py`: `active_loras`, `load_loras` | A `loras` entry whose `model_name` is null is switched off. | `tests/test_lora_disable.py::TestLoadLoras::test_an_all_null_entry_loads_nothing` | | Step progress | `dw/pipeline_processors/progress.py`: `reported_progress_bars` | A pipeline with no step callback reports its denoise steps through its progress bars. | `tests/test_modular_progress.py::test_every_advance_of_the_bar_is_reported` | +| Step timing | `dw/pipeline_processors/timing.py`: `StepTimer`, `step_timer`, `timed_step`, `mark` | Only a pipeline step that ran is timed, on the worker's monotonic clock: `step_timer` opens a `StepTimer` for a step with a `pipeline` or `pipeline_reference` and None for any other, which also hides an outer step's timer from a composed child. Load, call, denoise start, each pass and save reach the open timer through `mark`, and nowhere else. Denoise start is never taken inside a compiled region: a whole-compiled denoiser (`_compiled_call_impl` set) has that callable wrapped for the call instead of a forward pre-hook, which `fullgraph: true` cannot trace. A span with a missing mark is null, never guessed. `record_step` puts the reading on the step's manifest entry and `step_end`, and a reused step gets no `timing`. A chain sums its pipeline calls into one `timing` with `segments: N`. `ManifestEntry.timing` declares the shape (`docs/WORKSPACES.md` *Step timing*). | `tests/test_step_timing.py::test_a_missing_denoise_start_nulls_encode_and_first_pass`, `tests/test_step_timing.py::test_a_reused_step_carries_no_timing_even_when_one_is_passed`, `tests/test_step_timing.py::test_a_chain_sums_its_segments_and_takes_one_median_over_both`, `tests/test_step_timing.py::test_a_task_step_run_through_the_step_loop_has_no_timing`, `tests/test_step_timing.py::test_the_denoise_hook_marks_the_first_forward_once_and_removes_itself`, `tests/test_step_timing.py::test_a_fullgraph_compiled_denoiser_runs_and_still_gets_the_start_mark` | +| Step-cache skip counts | `dw/pipeline_processors/timing.py`: `CacheCounter`, `counting_forwards`, `report_cache_stats`, `StepTimer.cache_counted`; `dw/pipeline_processors/components.py`: `stateful_cache_context`, `get_cache_transformer` | For `first_block` and `mag`, a forward hook on the cache transformer counts computed or skipped once per forward, reading the leader hook's `should_compute` inside the cond/uncond `cache_context` the pipeline opened, so classifier-free guidance counts both forwards. `stateful_cache_context` registers the hook for one pipeline call; its `finally` emits `cache_stats` before `_reset_stateful_cache()` and removes the hook after the event. Private diffusers state that has moved gives null counts and one `cache_stats_unavailable` warning, and never fails the run; `taylorseer`, `faster` and `text_kv` report type and config only, with no warning. The counts reach the step's `timing.cache` only through `StepTimer.cache_counted`; a chain sums its calls' counts, and a null in any call nulls the sum. A step with no `cache` has no `cache` key and no event (`docs/ACCELERATION.md` *Reading the skip counts*). | `tests/test_cache_stats.py::test_cfg_counts_two_forwards_per_step_and_they_add_up`, `tests/test_cache_stats.py::test_the_event_names_the_step_and_segment_and_goes_out_before_the_reset`, `tests/test_cache_stats.py::test_moved_hook_state_nulls_the_counts_warns_once_and_the_run_finishes`, `tests/test_cache_stats.py::test_the_private_diffusers_names_are_where_the_counter_reads_them`, `tests/test_cache_stats.py::test_the_timer_sums_counts_and_nulls_on_any_null`, `tests/test_cache_stats.py::test_types_without_a_decision_report_type_only_and_no_warning` | | Community pipelines | `dw/community_pipelines/` (e.g. `pipeline_ltx2_refine.py`: `LTX2RefinePipeline`), `dw/introspection.py`: `community_pipelines`, `load_allowed_class` | A pipeline dw ships is named by its dotted path (`dw.community_pipelines..`), loads untrusted because it is under `dw`, and is listed by `list_pipelines` and resolved by `get_pipeline_signature` - only the modules in that directory, never a caller-chosen import. `LTX2RefinePipeline` VAE-encodes `video` at `width` x `height` and passes it unnormalized as `latents`, because `LTX2Pipeline.prepare_latents` normalizes 5-D latents itself. | `tests/test_ltx2_refine_pipeline.py::TestIntrospection::test_only_the_shipped_modules_resolve`, `tests/test_ltx2_refine_pipeline.py::TestEncode::test_the_latents_are_left_unnormalized_for_the_parent` | | Adding a task | `dw/tasks/registry.py`: `register_command`, `RegistryTable` | A task is a function registered with `@register_command`, and the signature of its implementation is its argument schema. Its validate-time rules are declared on the same decorator - `domains`, `choices`, `static_check`, `whole_numbers` and `media_arguments` - and `TASK_ARGUMENT_DOMAINS`, `TASK_ARGUMENT_CHOICES`, `TASK_STATIC_CHECKS`, `TASK_WHOLE_NUMBER_ARGUMENTS` and `TASK_MEDIA_ARGUMENTS` are read-only views derived from them, so there is no second table to forget (#692). The registry lives in `dw/tasks/registry.py`; `dw/tasks/task.py` re-exports it, and a task module may register its own handler, as `dw/tasks/beats.py` does, provided `dw/tasks/task.py` imports it (`docs/TASKS.md` *Adding a task*). The built-in handlers are grouped by family the same way - `dw/tasks/audio_handlers.py`, `dw/tasks/video_handlers.py`, `dw/tasks/finish_handlers.py` and `dw/tasks/model_handlers.py` (#790) - and `dw/tasks/task.py` keeps the general-purpose commands, the processor dispatch and `Task`. | `tests/test_task_discovery.py::TestDescribeTask::test_signature_becomes_the_schema`, `tests/test_task_registry_rules.py::test_every_derived_table_is_the_registrys_view`, `tests/test_task_registry_rules.py::test_every_static_check_is_registered_to_exactly_one_command`, `tests/test_task_registry_rules.py::test_every_media_argument_is_a_parameter_of_its_command` | | Task argument domains | `dw/task_domains.py`: `check_arguments`, `task_argument_errors`; `dw/task_problems.py`: `face_track_errors`, `beats_errors`, `cuts_errors`, `lut_errors` | A numeric domain that a signature cannot express is declared on the command's `@register_command(domains=...)`, and it is checked at validation and again at run time. A numeric argument is read by `whole_number` or `real_number`, the only numeric coercion in `dw/tasks/`, and validation applies the same rule (`number_problem`), so a value is refused at both or at neither: `"3.0"` is a whole number, `"3.5"` for one of `whole_numbers` is not, and `true` is no number (#774). `Task.run` coerces every declared argument through `coerce_arguments` before dispatch, so a handler sees a number, and an argument that has no range besides being a number declares `finite`. A command's cross-argument rules are its registered `static_check`, which `task_argument_errors` runs at validation (`attribute_voices`' `voices_argument_errors`, kept in `dw/tasks/voice_attribution.py`, among them). A literal enum choice (`choice_errors`) is refused at validation by the same module. The per-family rules past a single domain live in `dw/task_problems.py`, which reads `task_domains`' helpers and is never imported by it (`task_argument_errors` reaches them only through the registered `static_check`, so `import_cycles` stays 0; #790): the `ingredients_grid` image count against `max_images` (`ingredients_grid_errors`), and `crop_face_track`'s cross-argument rules (`face_track_problems`, `face_track_errors`: crop size a multiple of the step's `multiple` (32 by default), `remainder` below `modulus`, `gate_zero` above `gate_full`, padding and confidence limits), which the task checks again at run time, as `analyze_beats`' are too (`beats_problems`, `beats_errors`: `min_bpm` below `max_bpm`, and anchors that are one kind per list, ascending, with whole beat indexes; `tests/test_analyze_beats.py`), and `plan_cuts`' (`cuts_problems`, `cuts_errors`: a bare-string transcript (`transcript_problem`, which `plan_cuts` also calls at run time), `min_scene_s` above `max_scene_s`, `segment_by: "beat"` without beats), and `apply_lut`'s (`lut_source_problem`, `palette_problem`, `lut_errors`: exactly one of `lut` or `palette`, and a palette of 2..16 `#rrggbb` colours, re-checked at run time by `check_lut_source`), and `check_face_detector_source` vets its Hub repo and `.onnx` file before any download. `select`'s index stays in `task_domains` with the other media-timing rules (`select_index_problem`, `select_rule_problems`: a whole number, at least 0, and below the candidate count when the count is known, i.e. at run time or when `candidates` is a written list). | `tests/test_task_domains.py::TestTheStaticPass::test_a_negative_frame_count_is_refused_at_its_path`, `tests/test_numeric_coercion.py::TestGradeExposure::test_a_string_number_grades_like_the_number`, `tests/test_task_domains.py::TestAtRunTime::test_slice_audio_refuses_a_negative_count`, `tests/test_ingredients_grid.py::TestStaticValidation::test_bad_layout_and_fit`, `tests/test_face_track.py::TestStaticValidation::test_bad_literal_is_an_error_at_its_argument`, `tests/test_face_track.py::TestCropFaceTrack::test_bad_arguments_are_refused_at_run_time`, `tests/test_face_track.py::TestStaticValidation::test_a_declared_grid_is_accepted`, `tests/test_face_track.py::TestDetectorSource::test_traversing_repo_is_refused`, `tests/test_select_validation.py::TestSelectErrors::test_an_out_of_range_literal_index_names_both_ends`, `tests/test_lut.py::TestPaletteAtValidate::test_both_refuse_naming_the_two`, `tests/test_lut.py::TestPalette::test_a_bad_palette_refuses_naming_the_entry` | | Variables | `dw/variables.py`: `set_variables`, `argument_errors`, `resolve_variable_values` | `argument_errors` folds a caller's arguments in exactly as `set_variables` does when a run starts, so a bad name or value is refused before anything is queued. | `tests/test_variables.py::test_argument_errors_reports_a_dict_passed_for_a_string_variable` | | List variables from the CLI | `dw/run.py`, `dw/variables.py`: `get_value` | A `name=value` string given for a list variable is split on commas, so a list of objects can only be passed as JSON over the API or MCP. | `tests/test_variables.py::test_set_variables_list_default_splits_on_comma` | -| `for_each` expansion | `dw/for_each.py`: `expand_for_each` | Each entry becomes an ordinary step named `@`, after substitution and before the reference check. | `tests/test_for_each.py::TestNaming::test_member_name_joins_with_at` | +| `for_each` expansion | `dw/for_each.py`: `expand_for_each`, `group_of` | Each entry becomes an ordinary step named `@`, after substitution and before the reference check. | `tests/test_for_each.py::TestNaming::test_member_name_joins_with_at` | | Optional entry field | `dw/for_each.py`: `_optional_item`, `_item_fields` | `{"item": "field", "default": X}` reads an entry field with a fallback; without `default` it is the plain required read, and `list_fields` / `entry_field_warnings` count the field as one entries take. The editor's string scan needs no twin code, only the shared case file. | `tests/test_for_each.py::TestOptionalItem::test_a_present_field_wins_and_an_absent_one_takes_the_default`, `tests/test_ui_twins.py::test_the_engine_decides_each_shared_reference_case` | | Previous results | `dw/previous_results.py`: `get_iterations`, `previous_result_reference_errors` | Several `previous_result:` references in one step multiply into a cartesian product. | `tests/test_previous_results.py::TestGetIterations::test_multiple_references_create_cartesian_product` | | Previous-result check | `dw/previous_results.py`: `previous_result_reference_errors` | A literal reference to a step that is not earlier in the workflow is a validation error. | `tests/test_previous_results.py::TestStaticReferenceChecking::test_a_step_cannot_reference_itself_or_a_later_one` | @@ -57,6 +59,7 @@ one to open. | Packaged builtins | `dw/library.py`: `builtin_root`, `resolve_sub_workflow_reference` | `builtin:` names the packaged `dw/workflows/`, not the top-level `workflows/` examples. | `tests/test_sub_workflow_resolver.py::TestTheResolver::test_a_builtin_resolves_in_the_packaged_root` | | Sub-workflow paths | `dw/library.py`: `resolve_sub_workflow_reference`, `resolve_sub_workflow` | A sub-workflow path is confined to the root it resolves in. The search order is in the docstring of `resolve_sub_workflow`. | `tests/test_sub_workflow_resolver.py::TestTheResolver::test_a_path_escaping_its_root_is_refused` | | Workspace | `dw/workspace.py`: `resolve_workspace`, `set_workspace` | `--workspace` beats `DW_WORKSPACE`, which beats the setting, which beats a working directory that looks like a workspace, which beats `~/diffusers-workspace`. | `tests/test_workspace.py::TestResolution::test_a_flag_wins_over_everything`, `tests/test_workspace.py::TestResolution::test_a_bare_working_directory_falls_back_to_the_home_workspace` | +| Workspace copy | `dw/workspace_copy.py`: `copy_between` | Copies runs and asset-library entries between workspaces under one root: media hard-linked, sidecars always copied, never a symlink; an asset keeps its library-relative path and its note and kept-shots entry are merged into the destination folder's sidecars; all or nothing, a differing asset refusing runs too; `vN` renumbered by `runs.copy_runs_in` under the run lock; entries checked by `security.validate_output_prefix`. | `tests/test_workspace_copy.py` | | Realized workflow | `dw/realize.py`: `realize_workflow`, `dw/runs.py`: `write_realized_workflow` | Each run directory holds `workflow.json`, with the run's arguments, seed and stored prompt text pinned. | `tests/test_realize.py::TestVariablesAndSeed::test_arguments_become_the_variable_defaults`, `tests/test_runs.py::TestRealizedWorkflow::test_the_run_directory_holds_a_realized_copy` | ## Runs and outputs @@ -64,7 +67,9 @@ one to open. | Concept | Owner | Rule | Enforced by | | --- | --- | --- | --- | | Run directories | `dw/runs.py`: `open_run`, `workflow_identity`, `new_run_id` | Each execution writes into `///` with a `manifest.json`, unless the flat layout is chosen (the test pins the flat-layout exception). | `tests/test_runs.py::TestRunDirectories::test_the_flat_layout_writes_where_it_always_did` | -| Run versions | `dw/runs.py`: `open_run`, `run_versions`, `record_run_versions` | A run takes its number once, when it opens, so deleting a middle run leaves a gap rather than renumbering. | `tests/test_runs.py::TestRunVersions::test_a_deleted_middle_run_leaves_a_gap_rather_than_renumbering`, `tests/test_runs.py::TestRunVersions::test_runs_started_in_the_same_second_still_number_upward` | +| A file's role (final / intermediate / input) | `dw/roles.py` | Decided per step: result.role, then a final/intermediate subfolder, then data flow (a step a media-writing or unsaved consumer reads is intermediate). Engine records it at run end, with `role_declared` where result.role stated it; the server reports only recorded roles, so an older run has none. At read time the gallery (`GET /api/gallery`, its `role=` filter, the metadata route) also demotes a file a later run of the same workspace read - an `output:` input (an unpinned `latest` is the newest run started before the reader) or an `asset:` keep_output kept from it - unless its role was declared (`consumed_outputs`, `listed_role`, once per listing; no manifest is rewritten). | `tests/test_roles.py`, `tests/test_roles.py::test_a_file_a_later_run_read_is_consumed`, `tests/test_roles.py::test_listed_role`, `tests/test_gallery_roles.py::test_a_file_a_later_run_read_lists_as_intermediate`, `tests/test_gallery_roles.py::test_a_declared_final_stays_final` | +| Notes on runs, files, assets, workspaces | dw/notes.py | notes.json beside what it describes (run dir or asset dir; a file's own note else the run's), workspace.json at a workspace root; only the server writes them, never the engine; ≤4000 chars, plain text. | tests/test_notes.py | +| Run versions | `dw/runs.py`: `open_run`, `run_versions`, `record_run_versions`, `copy_runs_in` | A run takes its number once, when it opens (or, copied in from another workspace, when `copy_runs_in` places it), so deleting a middle run leaves a gap rather than renumbering. | `tests/test_runs.py::TestRunVersions::test_a_deleted_middle_run_leaves_a_gap_rather_than_renumbering`, `tests/test_runs.py::TestRunVersions::test_runs_started_in_the_same_second_still_number_upward` | | Run-version surfaces | `dw/runs.py`: `run_versions`, `dw/server/routes/gallery.py`, `dw/server/outputs.py`, `dw_mcp/tools_catalog.py`: `list_gallery`, `ui/src/lib/pages/GalleryPage.svelte` | Only `dw/runs.py` computes the number, and every other module reads it back. | `tests/test_runs.py::TestRunVersions::test_an_output_reference_can_name_a_run_by_its_version` | | Result subfolders | `dw/subfolders.py`: `subfolder_errors`, `dw/security.py`: `SUBFOLDER_PATTERN` | A step's `result.subfolder` is checked for shape at validation. | `tests/test_subfolders.py::TestValidationErrorsIntegration::test_validation_errors_reports_a_bad_subfolder` | | Template subfolder roles | `workflows/templates/`, `dw/workflows/` | Every saving step of a template is marked `final` or `intermediate`, with at least one `final`, and packaged builtins stay unmarked. | `tests/test_template_subfolders.py::test_every_saving_step_of_a_template_names_its_role`, `tests/test_template_subfolders.py::test_the_packaged_builtins_stay_unmarked` | @@ -86,20 +91,22 @@ one to open. | Concept | Owner | Rule | Enforced by | | --- | --- | --- | --- | | Validation registry | `dw/validation.py`: `ERROR_CHECKS`, `WARNING_CHECKS`, `run_checks` | A check is one `Check` in a registry, and a check that raises becomes one `internal` finding while the rest still run. | `tests/test_validation.py::TestExceptionPolicy::test_a_raising_check_is_one_internal_error_and_the_rest_still_run` | -| Step-value checks | `dw/step_value_checks.py`: `fps_errors`, `null_media_errors`, `select_errors`, `chain_prompts_errors` | The only caller of these checks is the error registry. | `dw/validation.py`: `ERROR_CHECKS` | +| Step-value checks | `dw/step_value_checks.py`: `fps_errors`, `cache_errors`, `null_media_errors`, `select_errors`, `chain_prompts_errors` | The only caller of these checks is the error registry. | `dw/validation.py`: `ERROR_CHECKS` | | Admission | `dw/server/admission.py`: `admit` | Validate, submit, rerun and enhance load and check a request once, and `JobManager.submit` does not check it again. | `tests/test_admission.py::test_a_submit_expands_once` | | Variable constraints | `dw/variable_constraints.py`: `apply_constraints`; the grid arithmetic `aligned` / `aligned_down` | A model's rule about a value is declared in the workflow's `variable_constraints`, and it is checked at validation and again before anything loads. `plan_cuts` (and its static check) rounds render lengths through `aligned` / `aligned_down`, never its own copy, so a planned `num_frames` is one the template's constraint accepts; `crop_face_track` pads its crops to the step's `modulus`/`remainder` grid through `aligned` too (`padding_to_grid`). | `tests/test_variable_constraints.py::TestTheGrid::test_without_snap_an_off_grid_value_is_refused_not_rounded`, `tests/test_variable_constraints.py::TestTheGrid::test_plan_cuts_rounds_through_the_constraint_owner`, `tests/test_face_track.py::TestPadding::test_padding_to_grid_rounds_through_the_constraint_owner`, `tests/test_variable_constraints.py::TestTheRunTimePass::test_an_illegal_value_is_refused_before_anything_loads` | -| Bound acknowledgement | `dw/plan.py`: `fingerprint`, `dw/server/admission.py`: `check_bound_acknowledgement` | A bound `acknowledged_cost` is refused with 409 when the plan's fingerprint or its required downloads changed. | `tests/test_server.py::TestBoundRerun::test_a_rerun_bound_to_a_stale_plan_is_refused`, `tests/test_server.py::TestBoundAcknowledgement::test_a_matching_fingerprint_queues` | +| Bound acknowledgement | `dw/plan.py`: `fingerprint`, `dw/server/admission.py`: `check_bound_acknowledgement` | A bound `acknowledged_cost` is refused with 409 when the plan's fingerprint or its required downloads changed. The fingerprint is keyed only by what can change the output: `DOCUMENTATION_KEYS` (`cost`, `cost_drivers`, `vram_estimate`, descriptions, catalog shape/traits), a constraint's `reason` and a result's role, subfolder and file name are left out, of the workflow and of each composed child; every schema key is sorted into left out or kept. A number digests by value, not spelling (`12.0` as `12`), and the seed stays out (#85). | `tests/test_plan.py::TestFingerprintKeys::test_the_exclusion_lists_are_pinned`, `tests/test_plan.py::TestTheCatalogH3Template::test_a_client_round_trip_keeps_it`, `tests/test_plan.py::TestFingerprintKeys::test_every_top_level_schema_key_is_sorted`, `tests/test_server.py::TestBoundRerun::test_a_rerun_bound_to_a_stale_plan_is_refused`, `tests/test_server.py::TestBoundAcknowledgement::test_a_matching_fingerprint_queues` | | Per-repo gating | `dw/plan.py`: `build_plan` | Each gated repo, including a `loras` entry's repo, is probed and reported on its own. | `tests/test_plan.py::TestDownloadsRequired::test_a_gated_repo_this_token_lacks_access_to_is_blocked` | | Observed cost | `dw/server/observed_cost.py`: `declared_drivers`, `ObservedCosts._on_this_card`, `ObservedCosts.card` | `observed` comes only from this box's finished jobs on the named card (the one `route` names, for `plan.estimate`), else the server's own card; `ObservedCosts.card` owns a card's (kind, name, default). Rows are matched by card name whatever index they ran on (a job from before jobs carried a `device` counts only while the server is on the default card), and never writes `cost`. Cold and warm runs are reported separately. | `tests/test_observed_cost.py::TestColdIsNotWarm::test_the_two_are_reported_separately_each_with_its_runs`, `tests/test_observed_cost.py::TestByCard::test_only_rows_from_this_card_count_whichever_index_they_had`, `tests/test_observed_cost.py::TestByCard::test_rows_from_before_jobs_carried_a_device_do_not_count_off_the_default_card` | -| The estimate | `dw/plan.py`: `estimate`, `_tempered` | `plan.estimate` quotes the observed figure ahead of the curated one, and blends it toward the curated figure below three runs. | `tests/test_plan.py::TestObservedEstimate::test_history_beats_a_curated_figure`, `tests/test_plan.py::TestLowConfidenceObservedEstimate::test_a_single_run_blends_toward_the_curated_figure` | +| The estimate | `dw/plan.py`: `estimate`, `_tempered`; the driver scaling is `dw/cost_scaling.py`: `scale_curated`, `_driver_scaling` | `plan.estimate` quotes the observed figure ahead of the curated one, and blends it toward the curated figure below three runs. A `cost` entry is measured at the workflow's variable defaults; a run moving `num_inference_steps`, `width`/`height` or `num_frames` (per list entry, summed) scales it in proportion (`scale_curated`, which `dw/server/deps.py` reuses for an inherited figure), basis `derived` with `scaled` naming each axis, times the list-length re-price, and that scaled figure is what a thin observed one blends toward (#866). | `tests/test_plan.py::TestObservedEstimate::test_history_beats_a_curated_figure`, `tests/test_plan.py::TestLowConfidenceObservedEstimate::test_a_single_run_blends_toward_the_curated_figure`, `tests/test_plan.py::TestDriverScaling::test_steps_canvas_and_frames_multiply`, `tests/test_plan.py::TestScalingAccuracy::test_within_thirty_percent` | | Cost drivers | `dw/server/observed_cost.py`: `declared_drivers` | A cost driver that names no declared variable is dropped. | `tests/test_observed_cost.py::TestComparability::test_a_driver_naming_no_variable_is_dropped`, `tests/test_observed_cost.py::TestTheCatalogsDriversAreReal::test_every_declared_driver_is_a_variable_of_its_workflow` | | Raw workflow GET | `dw/server/routes/library.py`: `get_workflow` | The raw workflow GET is served verbatim, with no `observed`, because the editor saves what it reads. | — | | VRAM projection | `dw/vram_estimate.py`, `dw/guides.py`: `guide_lengths` | A template's `vram_estimate` is projected per step after `for_each` expansion, and only the largest step over the ceiling is reported. With `bytes_per_guide_voxel` declared, each guide the step lays in (its own non-null `guides` plus a guide chain's) adds its snapped frames times the canvas; an unprobeable clip is charged at `num_frames`, and validate, admission and run project the same number. | `tests/test_vram_estimate.py::test_only_the_largest_over_ceiling_step_is_reported_once_per_cost_entry`, `tests/test_vram_guides.py::test_validate_apply_and_required_agree_on_a_guided_definition` | | H3 Ref2VA VRAM numbers | `workflows/templates/minimax/` | Every Ref2VA template declares `base_gb` 16.0, `bytes_per_voxel` 28.71 and `gb_per_reference` 1.0. | `tests/test_h3_vram_ceiling.py::test_every_ref2va_minimax_template_declares_gb_per_reference` | | Inherited VRAM ceiling | `dw/vram_inheritance.py`, `dw/server/deps.py`: `ceiling_index` (the catalog parse is `_catalog_listing`, shared with the cost lookup below) | A workflow with no `vram_estimate` is matched against the catalog by pipeline identity, and is warned, never refused. | `tests/test_vram_inheritance.py::test_every_template_declaring_one_identity_declares_the_same_numbers` | -| Inherited cost | `dw/server/deps.py`: `inherited_observed`, `dw/vram_inheritance.py`: `inherited_differences`, `dw/plan.py`: `_observed` (basis `inherited`), `dw/server/routes/jobs.py`: `_inherited_cost_warnings` | An inline workflow (no catalog name) is priced from the observed runs of the catalog template with the same `component_type` + `model_name` + `workflow` key as `vram_inheritance`; the template with the most cold runs wins, then name. `differs` lists offload, quantization and frame count set differently from the template. It is warned, never refused. | `tests/test_inherited_cost.py::TestPricedByPipelineIdentity::test_an_inline_copy_inherits_the_templates_runs`, `tests/test_inherited_cost.py::TestDifferences::test_offload_quantization_and_frames_are_named`, `tests/test_inherited_cost.py::TestThePlanReportsIt::test_the_estimate_is_inherited_and_the_validate_answer_warns` | -| H3 adapter partition | `dw/adapter_compatibility.py` | An FL2VA LoRA on a reference step is refused, and a LoRA whose file name says neither `ref2v` nor `fl2v` is warned. | `tests/test_h3_adapters.py::TestWhatIsRefused::test_a_keyframe_adapter_on_the_reference_path`, `tests/test_h3_adapters.py::TestWhatIsRefused::test_an_unrecognised_name_is_a_warning_not_a_refusal` | +| Inherited cost | `dw/server/deps.py`: `inherited_observed`, `dw/vram_inheritance.py`: `inherited_differences`, `dw/plan.py`: `_observed` (basis `inherited`), `dw/server/routes/jobs.py`: `_inherited_cost_warnings` | An inline workflow (no catalog name) is priced from the observed runs of the catalog template with the same `component_type` + `model_name` + `workflow` key as `vram_inheritance`; the template with the most cold runs wins, then name. `differs` lists offload, quantization, frame count, attention backend, cache, LoRA and reference count (the `references` list lengths, per-entry `item:` ones read off the `for_each` list; named, not scaled, #915) set differently from the template, and sets `low_confidence` when it lists any (#866). More pipeline steps than the template run (a `for_each` counted per member, against the list the bucket ran, so a fallback to the default bucket re-prices a longer list) re-price its minutes linearly, and the mean frames, inference steps, width and height per pipeline step are scaled by `scale_curated` itself - the one rule the catalog figure uses, so an inline copy and its template quote one request alike - with `scaled` naming why and `low_confidence` set (#851, #866). It is warned, never refused. | `tests/test_inherited_cost.py::TestPricedByPipelineIdentity::test_an_inline_copy_inherits_the_templates_runs`, `tests/test_inherited_cost.py::TestDifferences::test_offload_quantization_and_frames_are_named`, `tests/test_inherited_cost.py::TestDifferences::test_a_reference_count_other_than_the_templates_is_named`, `tests/test_inherited_cost.py::TestThePlanReportsIt::test_the_estimate_is_inherited_and_the_validate_answer_warns`, `tests/test_inherited_cost.py::TestTheTemplateAndAnInlineCopyAgree::test_the_same_minutes_within_rounding`, `tests/test_inherited_cost.py::TestAListLongerThanTheFallbackBucket::test_an_inline_six_shot_copy_scales_by_members` | +| Projected host-RAM peak | `dw/server/host_ram_need.py`: `memory_key`, `projected_peak`, `host_memory_block`; the fit is `dw/host_memory_projection.py`: `projected_mb`; history by key is `dw/server/job_history.py`: `peaks_for_key` | A run's memory key is the sorted pipeline identity of its pipeline steps plus every `quantization_config` / `offload` / `group_offload` value, with arguments substituted through `replace_variables`; never the workflow's name, so a renamed or inline copy shares the template's history. `observed` is the max peak over this machine's successful same-key runs and comes ahead of the list-run fit, unless the shape holds its pipeline across the list and the fit is the larger (the key leaves list length out, #910); with neither there is no figure. It never refuses; dispatch may delay a start on it (`dw/server/dispatch.py`: `Dispatcher._next_dispatch`, #869) (`Admission.host_ram_need`, validate's `plan.host_memory`). | `tests/test_host_ram_need.py::TestMemoryKey::test_an_inline_copy_without_its_metadata_has_the_same_key`, `tests/test_host_ram_need.py::TestMemoryKey::test_a_different_weights_dtype_is_a_different_key`, `tests/test_host_ram_need.py::TestProjectedPeak::test_an_observed_peak_beats_the_fit`, `tests/test_host_ram_need.py::TestALongerListThanObserved::test_an_accumulating_list_takes_the_larger_fit`, `tests/test_host_ram_need.py::TestProjectedPeak::test_a_failed_run_is_not_observed`, `tests/test_host_ram_need.py::TestValidateRoute::test_an_observed_run_is_the_plans_host_memory` | +| Acceleration warnings | `dw/acceleration_warnings.py`: `acceleration_warnings`, `CACHE_MIN_USEFUL_STEPS`, `CUDA_ONLY_ATTENTION_BACKENDS` (the last entry of `WARNING_CHECKS`) | An acceleration setting that can't pay off (a `first_block`/`mag` cache under `CACHE_MIN_USEFUL_STEPS`, any `taylorseer` cache, `compile.fullgraph` with a cache, a cache on a pipeline class with no transformer or whose transformer lacks `enable_cache()`, a CUDA-only `attention_backend` on a non-CUDA device) is a warning and never a refusal, and an unresolvable value (steps, backend, device) leaves it silent. | `tests/test_validation.py::TestAccelerationWarnings::test_warnings_never_refuse`, `tests/test_validation.py::TestAccelerationWarnings::test_steps_that_resolve_to_nothing_are_silent`, `tests/test_validation.py::TestAccelerationWarnings::test_an_unknown_device_is_silent`, `tests/test_validation.py::TestAccelerationWarnings::test_an_unresolvable_backend_is_silent_on_mps` | +| H3 adapter partition | `dw/adapter_compatibility.py`; the server's index is `dw/server/deps.py` `lora_index` | The LoRA catalog decides first, on the server only: a `proven` entry's `workflow` refuses a step in the other partition, a `trial`/`rejected` entry's only warns. A file no entry places falls to its name: an FL2VA LoRA on a reference step is refused, and one whose name says neither `ref2v` nor `fl2v` is warned. | `tests/test_h3_adapters.py::TestWhatIsRefused::test_a_keyframe_adapter_on_the_reference_path`, `tests/test_h3_adapters.py::TestWhatIsRefused::test_an_unrecognised_name_is_a_warning_not_a_refusal`, `tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_proven_entry_refuses_the_other_partition`, `tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_weaker_entry_only_warns`, `tests/test_server_loras.py::TestTheCatalogPlacesAnAdapter::test_a_trial_entry_warns_and_promoting_it_refuses` | | H3 audio hold, refine and guide layout | Five modules, imported one way: `dw/pipeline_processors/h3_rules.py` holds the names and rules, torch- and diffusers-free (`refine_problems`, the per-argument refine rules asked before validation and before the call; `not_h3`, the step rule; `snap_guide_length`, `guide_frame_problem`, `guide_chain_problems`, `guide_end_problem`; `CHAIN_CONTINUITY_MODES`; the chunk constants), `dw/pipeline_processors/h3_hold.py` puts the hold and refine blocks in (`insert_audio_hold`, `core_denoise_sequences`, `refines`), and `dw/pipeline_processors/h3_guides.py` the guide layout (`LAYOUT_ANCHORS`, `insert_guides`), which imports `h3_hold` and never the reverse; the blocks themselves, and the arithmetic they run, are `dw/pipeline_processors/h3_hold_steps.py` (`encode_audio_span`, `refine_sigmas`) and `dw/pipeline_processors/h3_guide_steps.py` (`splice_guide_rows`), which import diffusers when they load, so the two owners import them only inside `blocks()` and `guide_blocks()` (#790), and `h3_guide_steps` imports `h3_hold_steps` and never the reverse (`_with_guides` in `dw/pipeline_processors/pipeline.py` adds its blocks for a `guides` call), `dw/guides.py`: the `guides` validation check and the chain `continuity: "guide"` check, `guide_chain_errors`, which asks the `guides` check's own step rule (`_takes_guides_problem`, built on `h3_rules.not_h3` and `REFERENCE_WORKFLOWS`) and reads the mode names from `h3_rules.CHAIN_CONTINUITY_MODES`, the names `chain.py`'s `CONTINUITY_MODES` is zipped against (both registered in `dw/validation.py`; a `previous_result:` guide is checked at run time), `dw/hold_audio.py`: `hold_audio_errors`, `refine_strength_errors` | The one place dw modifies a diffusers modular pipeline's block graph: three blocks go in, before `set_timesteps`, `denoise` and `after_denoise`, in every H3 core-denoise sequence (the whole graph and the `workflow=`-pruned flat graph), and they are no-ops without `hold_audio` / `refine_strength`. A `hold_audio` path is opened only after `locations.validate_media_path` against the step's workflow directory. `dw/introspection.py` builds the same graph without weights, hold included, so `get_pipeline_signature` lists a modular pipeline's block inputs. The copies of diffusers behaviour are pinned to the installed diffusers: `shifted_sigma_grid` (the spacing `refine_sigmas` rests on) to stock `set_timesteps`, `GUIDE_FRAMES_PER_CHUNK`/`GUIDE_LATENTS_PER_CHUNK` to the H3 video VAE's `clip_length` and latents per clip, and every name in `LAYOUT_ANCHORS` (`_fill_audio_positions` among them) to `before_denoise`. | `tests/test_h3_hold_audio.py::TestAnchors::test_inserted_next_to_the_anchors_in_every_shape`, `tests/test_h3_hold_audio.py::TestHoldAudioLocation::test_a_path_outside_the_roots_is_refused_at_the_call`, `tests/test_h3_hold_audio.py::TestRefusals::test_a_non_h3_step_is_refused`, `tests/test_h3_hold_audio.py::TestSignature::test_the_h3_signature_shows_hold_audio`, `tests/test_h3_refine.py::TestAnchors::test_inserted_before_denoise_in_every_shape`, `tests/test_h3_refine.py::TestSigmaGridDrift::test_the_full_grid_is_stock_set_timesteps`, `tests/test_h3_guides.py::TestChunkDrift::test_frames_per_chunk_is_the_vae_clip_length`, `tests/test_h3_guides.py::TestChunkDrift::test_latents_per_chunk_is_the_vae_tokens_per_clip`, `tests/test_h3_guides.py::TestAnchors::test_installed_diffusers_has_every_anchor`, `tests/test_guide_chain_validation.py::TestRefused::test_the_step_rule_is_the_guides_rule`, `tests/test_guide_chain_validation.py::TestRefused::test_every_run_mode_is_known_to_validate`, `tests/test_h3_rules.py::test_h3_rules_imports_neither_torch_nor_diffusers`, `tests/test_h3_rules.py::test_the_block_owners_import_no_diffusers` | | IC-LoRA reference scale | `workflows/templates/ltx2/` | The three conditioning templates run at `reference_downscale_factor: 1`, and the two upscalers at 2. | `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_reference_is_encoded_at_the_output_resolution`, `tests/test_ltx2_ic_loras.py::TestTheGenerativeUpscaleMatchesTheSameCard::test_it_loads_the_upscaler_at_factor_two` | | IC-LoRA numbers | `workflows/templates/ltx2/` | Every number in the IC-LoRA templates comes from the vendor's model card. | `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_strength_is_the_cards_default`, `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_defaults_are_the_trained_bucket` | @@ -109,10 +116,10 @@ one to open. | Concept | Owner | Rule | Enforced by | | --- | --- | --- | --- | -| Step cache | `dw/step_cache.py`, `dw/workflow_run.py`: `cache_lookup` | A seeded rerun with unchanged inputs is served from the cache, marked `reused: true`, and writes nothing; `POST /api/memory/clear` (`dw/server/routes/system.py`: `clear_memory`) drops it. | `tests/test_workflow_step_cache.py::test_cache_hit_marks_its_manifest_entry_and_event_reused`, `tests/test_workflow_step_cache.py::test_second_run_with_unchanged_step_reuses_cached_result` | +| Step cache | `dw/step_cache.py`, `dw/workflow_run.py`: `cache_lookup` | A seeded rerun with unchanged inputs is served from the cache, marked `reused: true`, and writes nothing; `POST /api/memory/clear` (`dw/server/routes/system.py`: `clear_memory`) drops it. A step's key is only what can change its output: workflow metadata (`cost_drivers`, `vram_estimate`, descriptions) never reaches it and a result's `role` is dropped (`NON_OUTPUT_RESULT_KEYS`); attention backend, cache config, dtype and LoRA stay, and so do `subfolder` and `file_base_name`, because a hit republishes the cached paths. | `tests/test_workflow_step_cache.py::TestOutputKeys::test_the_exclusion_list_is_pinned`, `tests/test_workflow_step_cache.py::TestOutputKeys::test_metadata_alone_is_a_hit`, `tests/test_workflow_step_cache.py::test_cache_hit_marks_its_manifest_entry_and_event_reused`, `tests/test_workflow_step_cache.py::test_second_run_with_unchanged_step_reuses_cached_result` | | No seed, no cache | `dw/workflow_run.py`: `prepare_run` | A workflow that sets no `seed` skips the cache. | `tests/test_workflow_step_cache.py::TestCacheHits::test_an_unseeded_workflow_has_no_hits` | | A different result | `dw/server/routes/jobs.py`, `dw_mcp/tools_jobs.py`: `rerun_job` | `rerun` with `new_seed` draws a fresh seed into the workflow's seed variable. | `tests/test_rerun_new_seed.py::test_a_rerun_with_a_new_seed_draws_one_into_that_variable` | -| Worker protocol | `dw/worker_protocol.py`: `parse_reply` | Every command and reply is a frozen dataclass that travels as a wire dict. | `tests/test_worker_messages.py::test_from_wire_inverts_to_wire`, `tests/test_worker_messages.py::test_an_unknown_reply_type_is_kept_whole_rather_than_raised` | +| Worker protocol | `dw/worker_protocol.py`: `parse_reply` | Every command and reply is a frozen dataclass that travels as a wire dict. A run event's keys sit beside the wire's own `type`, so an event's `type` key (`cache_stats`' cache type) crosses as `event_type` and comes back as `type`. | `tests/test_worker_messages.py::test_from_wire_inverts_to_wire`, `tests/test_cache_stats.py::test_the_event_crosses_the_worker_wire_with_its_cache_type`, `tests/test_worker_messages.py::test_an_unknown_reply_type_is_kept_whole_rather_than_raised` | | Persistent worker | `dw/worker.py`, `dw/worker_manager.py`, `dw/serve.py` | Jobs run in spawned workers (one per card) that keep models loaded, so a change to engine code needs a server restart. | — | | Worker card pinning | `dw/devices.py`: `worker_environment`, `pinned_environment`, `resolve_serve_devices`, `check_device_present`, `label_of` | A worker named with a CUDA index is spawned with `CUDA_VISIBLE_DEVICES` set to that card and `DW_DEVICE=cuda` (a bare `cuda` pins nothing), so index-0 reads inside it measure its own card; `--devices`/`devices` takes one entry per worker (a card named twice, or a bare `cuda` in a list of several, is refused), a card the box lacks is refused at startup naming the cards present, and a job stores the card it ran on as `device_ordinal` and `device_card` (#693), job and history alike: rerun affinity reads the ordinal, observed cost buckets by the card, and the `device` label `"cuda:N "` is only `label_of` joining the two - nothing splits a label back apart. | `tests/test_worker_manager.py::TestWorkerPinning::test_a_cuda_index_pins_the_worker_through_the_environment`, `tests/test_worker_manager.py::TestWorkerPinning::test_a_bare_cuda_sets_no_pin`, `tests/test_serve_main.py::TestConfigureDevices::test_a_refusal_exits_with_code_2`, `tests/test_serve_main.py::TestResolveServeDevices::test_a_card_the_machine_lacks_is_refused_naming_the_cards_present`, `tests/test_serve_main.py::TestResolveServeDevices::test_two_entries_are_both_kept`, `tests/test_serve_main.py::TestResolveServeDevices::test_a_card_named_twice_is_refused` | | Failed-run reporting | `dw/worker.py`, `dw/worker_protocol.py`: `Failed`, `Cancelled` | A failed or cancelled run's reply still carries the manifest of the steps that ran. | `tests/test_worker_execute.py::test_failure_carries_the_manifest_of_the_steps_that_ran`, `tests/test_worker_execute.py::test_cancellation_carries_the_manifest_too` | @@ -124,9 +131,11 @@ one to open. | Concept | Owner | Rule | Enforced by | | --- | --- | --- | --- | | Opening media | `dw/media.py` | This is the only module that calls `av.open`. It imports no torch and no step types. | `tests/test_media_layering.py::test_av_open_appears_only_in_media`, `tests/test_media_layering.py::test_media_imports_no_step_types` (no torch: —) | +| Frame moments and crops | `dw/media_frames.py`: `moment_index`, `parse_crop` | One `at` grammar ("frame:N", or seconds at the clip's fps; negative counts back from the end for `get_output_frames` and is refused by `compare_outputs`) and one `[x, y, width, height]` crop parse serve `get_output_frames` and `compare_outputs`. Only the bounds policy is the caller's: `get_output_frames` refuses a moment past the end and clamps a crop (`_moment_to_index`, `resolve_crop_box`); `compare_outputs` blanks a past-end cell and refuses a crop that doesn't fit an output. | `tests/test_media_compare.py::test_resolve_moments_refuses_what_get_output_frames_refuses`, `tests/test_media_compare.py::test_crop_box_refuses_a_malformed_box_as_resolve_crop_box_does` | +| Output comparison | `dw/media_compare.py`: `compare`, `psnr`, `ssim`, `mean_abs_diff`, `check_request`, `crop_box` | Metrics always compare the same pixels: one shared crop is the measured region, and per-output crops are display only, leaving metrics on the full frame. An output whose size differs from the baseline's, or a moment past its end, gets null metrics with a reason; nothing is resized to force a number. SSIM is on Rec. 601 luma (SSIM's convention, not `image_ops.luma`). It runs in the server process and decodes through `dw/media.py` and `open_bounded`. | `tests/test_compare_route.py::test_a_shared_crop_is_the_region_measured`, `tests/test_compare_route.py::test_per_output_crops_are_display_only`, `tests/test_compare_route.py::test_a_size_mismatch_gets_no_numbers_and_says_why`, `tests/test_compare_route.py::test_a_moment_past_the_shorter_clip_is_a_blank_cell` | | Signal processing | `dw/dsp.py` | It measures and transforms waveforms, decides nothing, and imports nothing from `dw`. | `tests/test_media_layering.py::test_dsp_imports_nothing_from_dw` | | Assessment probes | `dw/tasks/assess.py`, `dw/assessment_rules.py` | Exactly three registered commands are assessment probes. Each measures a finished file, and every rule names a real probe field. | `tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_assessment_is_exactly_the_three_probes`, `tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_every_rule_names_a_registered_json_command` | -| Script check | `dw/tasks/script_check.py`: `check`; `dw/script_lines.py`: `parse_lines`, `parse_shots`, `shot_names_error` | `check_script` aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with `assessment_rules.finding()` and reads the guard's floor and window (`DEAD_AIR_FLOOR_DBFS`, `DEAD_AIR_WINDOW`) rather than copying them; the loudest-window measure is `dsp.loudest_window_dbfs` (`script_check.word_level` wraps it), which `transcribe_audio`'s `suspect: "silent"` mark reads with the same floor and window; the repetition-loop detector (`script_lines.looped_texts`, `REPEAT_RUN_MIN`, `REPEAT_MAX_PERIOD`) lives in `dw/script_lines.py` and backs both `check_script`'s `repetition` discard and `transcribe_audio`'s `suspect: "repetition"`, and `script_lines.suspect_runs`/`SUSPECT_LEFT_OUT` are what `plan_cuts` groups and leaves out suspect chunks by - so neither task imports the other. It is not an assessment probe and adds nothing to `RULES`. Its literal `lines` is refused at validation through `task_problems` (`script_lines_errors`), which calls the same `parse_lines` the task runs, and a literal `shots` through the same `parse_shots` and `shot_names_error`; the parsers live in `dw/script_lines.py`, not the task, so that `task_problems` reaches them without importing `script_check` (the `import_cycles` ratchet in `scripts/arch_metrics.py`). Its shot map comes from `assess.resolve_shots`, the one the probes use, never a copy; it places each shot on the soundtrack with `assess.sample_span`, decides repeated shot names with `shots.duplicate_shot_names`, and asks `locations.is_http_url` whether the take is a URL. | `tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag`, `tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found`, `tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference` | +| Script check | `dw/tasks/script_check.py`: `check`; `dw/script_lines.py`: `parse_lines`, `parse_shots`, `shot_names_error` | `check_script` aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with `assessment_rules.finding()` and reads the guard's floor and window (`DEAD_AIR_FLOOR_DBFS`, `DEAD_AIR_WINDOW`) rather than copying them; the loudest-window measure is `dsp.loudest_window_dbfs` (`script_check.word_level` wraps it), which `transcribe_audio`'s `suspect: "silent"` mark reads with the same floor and window; the repetition-loop detector (`script_lines.looped_texts`, `REPEAT_RUN_MIN`, `REPEAT_MAX_PERIOD`) lives in `dw/script_lines.py` and backs both `check_script`'s `repetition` discard and `transcribe_audio`'s `suspect: "repetition"`, and `script_lines.suspect_runs`/`SUSPECT_LEFT_OUT` are what `plan_cuts` groups and leaves out suspect chunks by - so neither task imports the other. It is not an assessment probe and adds nothing to `RULES`. Its literal `lines` is refused at validation through `task_problems` (`script_lines_errors`), which calls the same `parse_lines` the task runs, and a literal `shots` through the same `parse_shots` and `shot_names_error`; the parsers live in `dw/script_lines.py`, not the task, so that `task_problems` reaches them without importing `script_check` (the `import_cycles` ratchet in `scripts/arch_metrics.py`). A line's `shot` may be the short `@name` of the one recorded shot that carries it (`script_lines.matching_shot_names`, `with_full_shot_names`, shared by the task and the literal-`shots` check); which member key a recorded name carries comes from `references.member_keys`, beside `MEMBER_SEPARATOR` and `for_each.member_name`, never a re-derived pattern: a key ends at the first character outside `references.VARIABLE_NAME_CHARACTERS`, the set `security.validate_variable_name` (which checks a for_each entry's name) builds its pattern from. With `audio` an `output:`/`asset:`/path whose run manifest records shots, `dw/shot_span_preflight.py: script_shot_name_errors` (the `script_shot_names` error check) runs `shot_names_error` at validation, reading the map through `runs.shots_beside`. Its shot map comes from `assess.resolve_shots`, the one the probes use, never a copy; it places each shot on the soundtrack with `assess.sample_span`, decides repeated shot names with `shots.duplicate_shot_names`, and asks `locations.is_http_url` whether the take is a URL. | `tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag`, `tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found`, `tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference`, `tests/test_script_check.py::TestShotValidation::test_a_short_member_name_matches_the_one_shot_carrying_it`, `tests/test_script_check.py::TestShotValidation::test_a_valid_variable_name_comes_back_whole_as_a_member_key`, `tests/test_shot_span_preflight.py::TestScriptShotNames::test_a_wrong_shot_name_is_an_error_at_validate_time` | | Audio sync against a source slice | `dw/tasks/measure_sync.py`: `measure_sync`; `dw/dsp.py`: `envelope_lag` | `measure_sync` cross-correlates the onset envelopes of a shot's audio and its source slice and answers the lag and a 0-1 confidence. A positive lag means the audio is LATE (its events fall after the reference's), negative early; `lag_seconds` is None for a silent or flat track. It builds its findings (`sync_unmeasurable`, `sync_low_confidence`, `audio_out_of_sync`) with `assessment_rules.finding()` over module-local rule dicts and adds nothing to `RULES`; it reads the silence floor from `beats.SILENT_DBFS`. `envelope_lag` imports nothing from `dw`. | `tests/test_measure_sync.py::TestLag::test_late_audio_is_a_positive_lag`, `tests/test_measure_sync.py::TestLag::test_early_audio_is_a_negative_lag`, `tests/test_measure_sync.py::TestUnmeasurable::test_a_silent_reference_has_no_lag` | ## Security and architecture guardrails @@ -150,12 +159,12 @@ one to open. | App factory | `dw/server/app.py`: `create_app` | `create_app` builds the JobManager and `app.state`, installs the middleware and registers the routes, and all state lives in the JobManager. | — | | Routers | `dw/server/routes/*.py`, `dw/server/routes/__init__.py`: `ROUTERS`, `include_routers`, `include_file_routes` | Routers register in `ROUTERS` order, because a greedy `{name:path}` route must come after its specific siblings. The files router comes later, after `/mcp`. | `tests/test_server_downloads.py::test_download_workflow_sets_content_disposition_attachment` | | Job queue | `dw/server/jobs.py`: `JobManager`; `dw/server/job_results.py`: `consume_results` | A dispatcher thread hands queued jobs, in queue order, to the pool's workers (`Worker pool dispatch`); each worker runs one job at a time, and `consume_results` folds that worker's replies into the job, naming files relative to the job's own `output_dir`. A job carries its own `output_dir`, `asset_dir` and `workflow_dir`, so it stays in its workspace. | `tests/test_server_workspaces.py::TestRunning::test_rerun_stays_in_the_workspace_it_ran_in` (FIFO: —) | -| Worker pool dispatch | `dw/server/jobs.py`: `JobManager._next_dispatch`; `dw/server/pool.py`: `WorkerSlot`, `check_fits`, `largest_ceiling_gb`; `dw/vram_estimate.py`: `required_vram_gb`; `dw/server/admission.py`: `pool_capacity` | `--devices` runs one worker per card; a queued job goes to a free card that fits what it needs (none starts while a cache clear holds dispatch, `dw disk caches`), a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, and cancel or a crash touches only the worker running that job. On a backend no `cost` entry describes, a declared `vram_estimate` is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. | `tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once`, `tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest`, `tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job`, `tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job`, `tests/test_worker_pool.py::TestOomRanking::test_the_later_started_worker_is_ranked_first`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted` | +| Worker pool dispatch | `dw/server/dispatch.py`: `Dispatcher._next_dispatch`, `Dispatcher._host_ram_wait` (mixed into `JobManager`); `dw/server/pool.py`: `WorkerSlot`, `check_fits`, `largest_ceiling_gb`, `host_ram_wait`; `dw/vram_estimate.py`: `required_vram_gb`; `dw/server/admission.py`: `pool_capacity` | `--devices` runs one worker per card; a queued job goes to a free card that fits what it needs (none starts while a cache clear holds dispatch, `dw disk caches`), a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, a job that fits a free card is also held while its projected host-RAM peak exceeds `MemAvailable` less what running jobs have yet to add plus the idle worker's own RSS (a job alone is never held, no figure or no RAM reading means no gate, a smaller job behind a held one starts, and the dispatcher re-checks every 10 s; #869), and cancel or a crash touches only the worker running that job. On a backend no `cost` entry describes, a declared `vram_estimate` is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. | `tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once`, `tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest`, `tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job`, `tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job`, `tests/test_worker_pool.py::TestOomRanking::test_each_dispatch_scores_its_worker_highest`, `tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted`, `tests/test_worker_pool.py::TestHostRamGate::test_a_job_the_host_cannot_hold_beside_the_running_one_is_held`, `tests/test_worker_pool.py::TestHostRamGate::test_a_job_alone_is_never_held`, `tests/test_worker_pool.py::TestHostRamGate::test_a_smaller_job_starts_past_a_held_one` | | Card affinity | `dw/server/pool.py`: `choose_slot`; `dw/server/jobs.py`: `JobManager.route`; `dw/worker_protocol.py`: `workflow_identity` | Among the free cards a job fits it goes to the rerun's original card, else the card whose worker last ran its workflow identity (`workflow_identity`, the one rule the worker caches by too), else one with no worker running, else the first in `--devices` order; a cache probe and `plan.estimate` (`priced_for`) ask about the card `route` names, and memory is read and cleared per card. | `tests/test_worker_pool.py::TestIdentityAffinity::test_a_second_run_lands_on_the_card_that_ran_the_workflow_first`, `tests/test_worker_pool.py::TestRerunAffinity::test_a_rerun_lands_on_the_original_card_when_it_is_free`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle` | | Per-card memory and cache | `dw/server/worker_memory.py`: `memory_status`, `clear_memory`, `probe_cache` | Each card keeps its own last memory reading; an idle card's worker is asked live, a busy one answers with the run's last report and says why it is not live, and a clear leaves a card running a job alone. Every request takes the card's lock with a bounded wait, and a card's lock is never taken while the JobManager's lock is held. | `tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle`, `tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to`, `tests/test_server.py::test_memory_says_why_a_reading_is_not_the_worker_s` | | dw disk caches | `dw/disk_caches.py`; `dw/server/jobs.py`: `JobManager.hold_dispatch`; `dw/server/routes/system.py`: `clear_cache`, `delete_cache_entry` | Members `inductor`, `triton`, `quantized`. Clear only a root carrying the `.dw-cache` marker, never while a job is running or queued (`quantized` also not during a model download); an entry delete takes only a 64-hex id, checked before any path is built; clear renames into `.clearing-*` then deletes. The busy check and the dispatch hold are one step in `JobManager.hold_dispatch`, kept for the whole clear or entry delete: a job submitted meanwhile is accepted but stays queued until it ends. | `tests/test_disk_caches.py`, `tests/test_disk_caches.py::test_job_submitted_during_a_clear_waits_for_it` | | Job-to-run link | `dw/server/job_record.py`, `dw/server/jobs.py`: `JobManager.realized` | A job records `run_id`, `run_dir` and `run_version`, and `JobManager.realized` reads that run's `workflow.json`, confined to the output root. | `tests/test_server_jobs.py::test_realized_reads_the_file_the_run_wrote`, `tests/test_server_jobs.py::test_realized_refuses_a_run_dir_that_escapes_the_output_root` | -| Job history | `dw/server/job_history.py`: `JobHistory` | Finished jobs persist in `jobs.sqlite`, in WAL mode, so the Jobs view survives a restart. | `tests/test_server.py::test_job_history_survives_restart_and_reruns`, `tests/test_job_history_wal.py::test_the_jobs_database_uses_wal_mode` | +| Job history | `dw/server/job_history.py`: `JobHistory`, `_ADDED_COLUMNS` | Finished jobs persist in `jobs.sqlite`, in WAL mode, so the Jobs view survives a restart. A database from an older release gains each column it lacks from the `_ADDED_COLUMNS` table when it is opened. | `tests/test_server.py::test_job_history_survives_restart_and_reruns`, `tests/test_job_history_wal.py::test_the_jobs_database_uses_wal_mode`, `tests/test_server_jobs.py::test_a_v0_13_0_database_gains_the_columns_added_since` | | HTTP security | `dw/server/http_security.py`: `install_middleware`, `query_token_ok` | Every API route needs the token, and only a route marked `query_token_ok` takes it as `?token=`. | `tests/test_security_auth.py::TestTheTokenGate::test_every_api_spelling_needs_the_token`, `tests/test_security_auth.py::TestTheTokenGate::test_the_query_token_is_refused_where_it_is_not_allowed` | | Prompt enhancement | `dw/server/enhancers.py`: `PRESETS`, `_T2I_SYSTEM_PROMPT` | `PRESETS` is the Enhance panel's menu (label, curated LLMs, preselect hints); a preset's family-specific knowledge, its spec, lives in a builtin under `dw/workflows/`, never in Python. `t2i`'s generic, family-agnostic system prompt stays in Python. A third preset moves the menu into builtin metadata (#696). | `tests/test_prompt_library.py::test_every_enhancer_preset_targets_a_known_family`, `tests/test_server.py::TestEnhance::test_presets_are_listed`, `tests/test_server.py::TestEnhance::test_enhance_workflows_are_schema_valid`, `tests/test_server.py::TestEnhance::test_an_unknown_preset_is_a_client_error` | @@ -190,6 +199,6 @@ one to open. | The editors' document | `ui/src/lib/editorShell.svelte.ts`: `DocumentEditor` | The workflow and prompt editors keep one rule each for the view (remembered per editor), the JSON draft (a failed parse pins it), dirty against the last load or save, the save path, and the tab-close guard. | `ui/src/lib/editorShell.test.ts`, `ui/src/lib/pages/PromptEditorPage.test.ts`, `ui/src/lib/pages/EditorPage.test.ts` | | UI polling | `ui/src/lib/poll.ts`: `poll`, `sleep` | A page that refreshes on a timer starts it through `poll` and stops it with the function `poll` returns. | `ui/src/lib/poll.test.ts` | | Workspace state in the UI | `ui/src/lib/workspaceState.svelte.ts`: `workspace` | The current workspace is one `$state`; `api.ts` reads it from here, so the request layer imports no page state. | `import_cycles` in `ui/scripts/arch-metrics.mjs` | -| The UI's response contract | `dw/server/api_models.py`: `ApiModel`, `send_rejected_responses`; `scripts/dump_openapi.py` | Every JSON route `api.ts` calls declares a response model, and `types.ts` re-exports the types generated from it (`ui/src/lib/generated/`). The payload does not change: an absent key stays absent (`response_model_exclude_unset`), a sometimes-sent key is `sometimes()`. Runtime is lenient - an undeclared key passes and a rejected response is logged and sent as built; tests, the e2e server and the dump run strict (`DW_STRICT_RESPONSES=1`). The document is generated under the FastAPI and Pydantic `constraints-openapi.txt` pins: CI installs them, and the dump refuses to write under others. | `tests/test_api_contract.py`, `tests/test_ui_twins.py::test_every_json_route_the_ui_calls_declares_its_response`, CI's "Response contract" step | +| The UI's response contract | `dw/server/api_models.py` (the models; it re-exports `dw/server/api_models_base.py`: `ApiModel`, `send_rejected_responses`, and the library routes' models in `dw/server/api_models_library.py`); `scripts/dump_openapi.py` | Every JSON route `api.ts` calls declares a response model, and `types.ts` re-exports the types generated from it (`ui/src/lib/generated/`). The payload does not change: an absent key stays absent (`response_model_exclude_unset`), a sometimes-sent key is `sometimes()`. Runtime is lenient - an undeclared key passes and a rejected response is logged and sent as built; tests, the e2e server and the dump run strict (`DW_STRICT_RESPONSES=1`). The document is generated under the FastAPI and Pydantic `constraints-openapi.txt` pins: CI installs them, and the dump refuses to write under others. | `tests/test_api_contract.py`, `tests/test_ui_twins.py::test_every_json_route_the_ui_calls_declares_its_response`, CI's "Response contract" step | | Where `--live` appears | `ui/src/app.css` (its header) | The state colour marks machine state only, in the files `ui/scripts/design-rules.test.ts` lists. | `ui/scripts/design-rules.test.ts` | | UI architecture ratchet | `ui/scripts/arch-metrics.mjs` | No metric rises above `docs/stabilization/ui/baseline.json`. | the `ui` job in `.github/workflows/ci.yml`; `npm run preflight` | diff --git a/docs/LORAS.md b/docs/LORAS.md index 788cf6cb..5aa164c0 100644 --- a/docs/LORAS.md +++ b/docs/LORAS.md @@ -165,8 +165,14 @@ exact - an adapter on the wrong base usually loads and is quietly worse. (the turbo keyframe files are `["t2va", "fl2va"]`). `proven` and `rejected` entries need `evidence`; a rejected entry's first note is why. The schema is `GET /api/lora-schema`. +The server's validate enforces that `workflow` (#870): a step whose workflow +sits in the other partition is refused when the entry is `proven`, and only +warned when it is `trial` or `rejected`. Without a covering entry the file +name decides (`ref2v` / `fl2v`), as in `dw/adapter_compatibility.py`. + Promotion: a trial that worked is saved with `save_lora` (or -`PUT /api/loras/{name}`) as `proven`, its job id in `evidence`. +`PUT /api/loras/{name}`) as `proven`, its job id in `evidence`. Promoting +it is also what makes its `workflow` refuse a mismatched step. ## Finding LoRAs on the Hub diff --git a/docs/MCP.md b/docs/MCP.md index 820c4831..436c0bbb 100644 --- a/docs/MCP.md +++ b/docs/MCP.md @@ -187,7 +187,7 @@ Nothing in this sequence costs GPU time. ## Tool reference -62 tools in six groups. Names and arguments below are transcribed from +65 tools in six groups. Names and arguments below are transcribed from `dw_mcp/tools_*.py` — nothing here is renamed or reshaped for the docs. ### Catalog (read-only) @@ -203,7 +203,9 @@ a request nothing on disk covers. The instructions also say the engine runs **one job per GPU** - one worker per `dw.serve --devices` card, each running its own job from the one shared queue - and point at `get_health`'s `workers` for what each card is running. -A job record's `device` (`get_job`, `list_jobs`) names the card it ran on. +A job record's `device` (`get_job`, `list_jobs`) names the card it ran on, +and a job started by `rerun_job` carries `rerun_of`, the id of the job it +reran (absent on a fresh submit). A request usually names a *subject* ("a lego movie trailer set in the marvel universe") while the catalog is written in *shapes* - a single image, an image @@ -230,8 +232,8 @@ when no single workflow covers it. | `get_memory()` | `device` | Get VRAM and RAM statistics: `workers` has one reading per card (the top level repeats the first card's), and `device` (`cuda:1`) asks for that card alone. `gpu_*` is the card; `host_memory_rss_mb` / `host_memory_peak_rss_mb` are the worker process's resident and high-water host memory, beside the machine's `host_memory_total_mb` / `host_memory_available_mb`; on Linux `host_memory_rss_anon_mb` / `host_memory_rss_file_mb` split the resident figure into memory the worker owns and mapped-file pages (memory-mapped checkpoint weights) the kernel can drop, so a loader that maps rather than copies raises the resident and peak figures without raising committed memory (#709) - read both, since an offloading workflow keeps its weights in host RAM and the card says little about what it holds (a host field is absent, not null, where the platform cannot measure it). `host_pinned_reserved_mb` / `host_pinned_allocated_mb`, when present, are torch's pinned-host cache - the staging buffers group offloading moves weights through, part of `host_memory_rss_mb` and invisible in every `gpu_*` figure, which is what a worker holding GB after releasing every model is usually holding (#98). `live: true` means `info` was measured now and is the worker's own memory; only live readings are comparable with each other. `live: false` with `info: null` (and `stale: false`) means nothing has been measured because nothing is resident; `live: false` with a populated `info` is a cached earlier reading, with `reason` (`job_running`, `worker_stopped`, `worker_busy`, `worker_unreachable`) and `age_seconds` - one cached while a job loads a model understates what is resident, so ask again when the card is idle rather than comparing it with a live figure | | `get_health()` | — | Check that the server is alive, and which machine answered: `version`, `device`, whether any model process is resident (`worker_alive`), and the queue depth. The engine runs one job per GPU: `workers` has one entry per card - `device`, `name`, `vram_gb`, `alive`, its own `current_job`, and the worker's `host_memory_rss_mb` while it runs - and the top-level `current_job` is the longest-running job. `worker_alive: false` is the normal idle state on a server that has not run a job since startup or the last memory clear - not a fault - the on-demand worker starts with the next job (#206) | | `get_server_info()` | — | What this installation can do and where it keeps things: `device` (the accelerator a run will use), `version`, the `workspace` this session is working in and the workflow/asset/output/prompt `directories` of *that* workspace, the bind address and port, whether a token is required, and whether MCP is mounted. Check the device before authoring - a CUDA-only choice (bitsandbytes, `torch.compile`, flash attention) is not available on an `mps` or `cpu` server. `runtime` (#222) reports the Python version, torch version and the CUDA version torch was built against, the NVIDIA driver version (when `nvidia-smi` is reachable), and the installed versions of diffusers, transformers, accelerate, bitsandbytes, peft, safetensors and sentencepiece (`null` for one not installed) - for diagnosing an environment mismatch between boxes without shelling in. `caches` lists dw's disk caches (`inductor`, `triton`, `quantized`), each with `name`, `bytes`, `files`, `enabled`, `clearable` and `disk_free`, plus `entry_count` for a cache with entries, and `cache_entries=true` lists them (`id`, `model`, `subfolder`, `component`, `bytes`, `files`, `unreadable`, `created`, `last_hit`; the quantizer and dtype are not in `/api/caches`); it is omitted when the server has no `/api/caches`. There is no MCP tool to clear one - that is `DELETE /api/caches/{name}` or `.../entries/{entry}` ([SERVER.md](SERVER.md)) | -| `list_jobs(limit=20, status=None, workspace=None)` | optional `limit` (newest N), `status` (one state or a comma-separated set of `queued`, `running`, `succeeded`, `failed`, `cancelled`), `workspace` | List queued, running and recent jobs, **newest first**. Bounded by default: the unbounded listing was over a client's tool-result limit on a server with a few months of history, which made it a tool that could not be called at all. `total` says how many matched and `truncated`/`next` say so when the answer was cut - raise `limit` or narrow with `status`. Without `workspace`, a named workspace lists its own jobs and the default one lists every job the server holds | -| `list_gallery(limit=50, subfolder=None, only_orphans=False, workspace=None, folder=None, version=None)` | `limit`, `subfolder`, `only_orphans`, `workspace`, `folder`, `version` | List generated output files, newest first. A name is `//`, where `` may sit in the subfolder the step chose (`final/episode.mp4`); each entry carries `folder` (the workflow) and `subfolder` (by convention `final` or `intermediate`, `''` when the step chose none, any path the workflow wrote otherwise), and `subfolder="final"` lists only deliverables. Each entry carries `run_id` and `version` - that run's ordinal among the workflow's runs, which is how one of several runs that wrote the same basename is named to a person: the web UI labels the same file `v5`. The number is assigned when the run opens and never renumbered, so deleting a run leaves a gap rather than sliding the rest down (a failed run, or a rerun that reused every step, leaves one too - it took a number and may have nothing to list), and it is `null` under the flat output layout, which has no runs. `folder` with `version` lists that one run's files, and `output:/v5/` names one in a workflow; every other tool takes `name`. Each entry also carries a ready-made `url`, already scoped to the workspace that made it - a hand-built `/outputs/` URL 404s for anything but the default workspace. `only_orphans=True` inverts the call: instead of files, it returns run directories with no media anywhere under them (`runs`, each `{name, mtime}`) - a run whose output was deleted before `delete_output` could remove it by name, or one that failed before writing anything; `subfolder` does not apply in this mode, and `name` is exactly what `delete_output` accepts (#170). `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | +| `list_jobs(limit=20, status=None, workspace=None)` | optional `limit` (newest N), `status` (one state or a comma-separated set of `queued`, `running`, `succeeded`, `failed`, `cancelled`), `workspace` | List queued, running and recent jobs, **newest first** (a queued job held for host RAM carries `waiting_for`). Bounded by default: the unbounded listing was over a client's tool-result limit on a server with a few months of history, which made it a tool that could not be called at all. `total` says how many matched and `truncated`/`next` say so when the answer was cut - raise `limit` or narrow with `status`. Without `workspace`, a named workspace lists its own jobs and the default one lists every job the server holds | +| `list_gallery(limit=50, subfolder=None, only_orphans=False, workspace=None, folder=None, version=None, media=False, role=None)` | `limit`, `subfolder`, `only_orphans`, `workspace`, `folder`, `version`, `media`, `role` | List generated output files, newest first. A name is `//`, where `` may sit in the subfolder the step chose (`final/episode.mp4`); each entry carries `folder` (the workflow) and `subfolder` (by convention `final` or `intermediate`, `''` when the step chose none, any path the workflow wrote otherwise), and `subfolder="final"` lists only deliverables. Each entry also carries `role`, `final` or `intermediate` (null for a flat-layout file with no run; `intermediate` once a later step or a later run read the file, unless `result.role` declared it); `role=["final"]` lists only those, and the `role` filter defaults to every file. See `dw/roles.py` for how a role is decided. A file starred in the web UI's gallery (its keep mark) carries `starred: true`; an unstarred one carries no `starred`. Each entry carries `run_id` and `version` - that run's ordinal among the workflow's runs, which is how one of several runs that wrote the same basename is named to a person: the web UI labels the same file `v5`. The number is assigned when the run opens and never renumbered, so deleting a run leaves a gap rather than sliding the rest down (a failed run, or a rerun that reused every step, leaves one too - it took a number and may have nothing to list), and it is `null` under the flat output layout, which has no runs. `folder` with `version` lists that one run's files, and `output:/v5/` names one in a workflow; every other tool takes `name`. Each entry also carries a ready-made `url`, already scoped to the workspace that made it - a hand-built `/outputs/` URL 404s for anything but the default workspace. `only_orphans=True` inverts the call: instead of files, it returns run directories with no media anywhere under them (`runs`, each `{name, mtime}`) - a run whose output was deleted before `delete_output` could remove it by name, or one that failed before writing anything; `subfolder` does not apply in this mode, and `name` is exactly what `delete_output` accepts (#170). `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | | `get_gallery_metadata(name, envelope=False, workspace=None, interval_seconds=None)` | `name`, `workspace` | Get the metadata embedded in a generated file — or, when `name` is an `asset:` reference, what an *input* asset holds (`source` says which; `job` is null for an asset). Reading an input's duration, frame count, fps and sample rate before a run is how a caller learns the `total_frames`, `fps` and `sample_rate` a workflow expects it to supply: the exact workflow and arguments that produced it, and, for audio/video, a `media` block (duration, rate, channels, fps, size, peak/mean dBFS) and `findings` - each level problem the server measured against `dw/audio_qc.py`'s thresholds (`full_scale`, `near_silent`), with the fix. Only an image (PNG/JPEG/WebP) embeds `metadata` this way — it is always null for audio and video, and `next` then points at `get_job_workflow(job_id)` when `job` is known, or says a kept asset carries no provenance at all when it isn't. `envelope=true` adds `media.envelope` — `rms_dbfs` and `peak_dbfs` one entry per second, or per `interval_seconds` (default 1.0, 0.05 to 3600; 0.1 places a vocal onset; a finer-than-default width that would need over 2400 bins is refused with the interval that fits) — which is what locates something in a track rather than measuring the whole of it. `media.shots` is set on an output joined from shots: one `{name, start_frame, num_frames, start_sample, num_samples}` per shot, as the join measured them (see the run manifest in docs/WORKFLOW_GUIDE.md), else null. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | ### Media @@ -243,6 +245,7 @@ when no single workflow covers it. | `get_output_frames(name, at=None, seams=None, count=None, boundaries=None, names=None, max_dimension=512, hear=None, workspace=None)` | `name`, `at`, `seams`, `count`, `boundaries`, `names`, `max_dimension`, `hear`, `workspace` | See a generated video as frames, since there is no video content type over MCP (#193). One selector per call: `count` for an evenly spaced contact sheet, `at` for moments (seconds or `"frame:N"`), `seams` (true, or seam numbers from 1) for the last frame before and first frame after each join side by side (each pair carries `difference`, the mean pixel change across the join, 0-255 - rank seams by it and look at the worst). `boundaries` and `names` are modifiers of `seams` only - passing either alongside `at` or `count` is refused, naming the offending argument(s). On an output joined from shots (`concat_videos`, `dissolve_videos`, a chained step) `seams` alone is enough: the seams and their names are the `media.shots` its run recorded. For any other file pass `boundaries` - each later shot's first frame, the running sum of the shots' `frame_count` from `get_gallery_metadata` on their own `intermediate/` files - and `names` to name them; either one given overrides the recorded value. Tiles are fitted to `max_dimension` and, when the set would exceed the 4MB budget, shrunk together rather than dropped; the text part lists each tile and says so. `hear=N` also returns N seconds of soundtrack centred on each `at` moment, after its image - the way to check a hit point or lip-sync without reconciling two clocks; a mute clip keeps its frames and says `no soundtrack` | | `get_output_text(name, max_characters=20000, workspace=None)` | `name`, `max_characters`, `workspace` | Read a text output — a prompt enhancement, or any step whose result is `text/plain` or JSON. Reports the file's real length and whether it was truncated. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | | `assess_output(name, probe=None, detail=False, workspace=None)` | `name` (a gallery name or `asset:`), `probe`, `detail`, `workspace` | Measure a finished cut on the server, without queueing: the assessment probes (`analyze_shots`, `analyze_seams`, `analyze_sync_drift`) run in the server process over one decode of the file, beside whatever job holds the GPU. Shot boundaries are the ones its run recorded (the manifest for an output, the `keep_output` sidecar for an asset). The answer merges every applicable probe's `findings` (`{rule, severity, at, value, threshold, says}`) with `rules_applied`, `rules_skipped` and `not_applicable` (`{probe: why}` - a still, a mute file, a file with no recorded shots); `detail=true` adds each probe's full body under `probes`, and `probe=` returns that one probe's full body. `probe` is checked against the three names before anything else is read, and an unknown one is refused naming them. Findings are places to look, not verdicts: drill in with `get_output_frames(seams=[n])` / `get_output_audio(start, duration)`. See *Assessing a run's output* in docs/WORKFLOW_GUIDE.md | +| `compare_outputs(names, at=None, crop=None, baseline=None, max_dimension=384, workspace=None)` | `names`, `at`, `crop`, `baseline`, `max_dimension`, `workspace` | Read-only, queues nothing: 2-8 outputs (gallery names or `asset:`) as one grid, a row per output (labelled with its row number and short name, `(baseline)` on the baseline) and a column per moment, each cell stamped `frame N`, with PSNR, SSIM and mean absolute difference against `baseline` (default the first name). `at` is 1-8 moments (seconds or `"frame:N"`, never negative: unlike `get_output_frames`, a negative moment is a 400, not counted back from the end); they resolve to one frame index against the baseline (the first video if the baseline is a still) and every output is read at that same frame number; default 4 evenly spaced frames of the shortest video. A frame past an output's end is a blank cell and its metrics cell says `past end (N frames)`; `fps_differs` flags videos with different frame rates, where the same frame number is not the same second. All stills: one column, `at` ignored with a note; a still among videos repeats in every column. Metrics are per non-baseline output at source resolution: `psnr_db` (`identical` when equal), `ssim` (luma, 11x11 Gaussian window, sigma 1.5) and `mean_abs_diff` (0-255), per column and as mean/min/max. `crop` is `[x,y,w,h]`: one shared box is the region shown and measured (`region` reads `crop [...]`); a list of one box per name is display only and metrics stay on the full frame (no crop is `full frame`). A crop must lie wholly inside every output's frame, else 400 (stricter than `get_output_frames`, which clamps). An output whose size differs from the baseline gets null metrics with the reason `size differs from baseline: WxH vs WxH` - it is never resized. `max_dimension` is the longest side of each cell; the grid is held to 4 MiB and shrunk whole when over (never below 64 px per cell), `downscaled_to` null when nothing shrank. Returns the grid image and a text summary. Errors: 400 bad shape, out-of-bounds crop, every moment past every end, or not an image or video; 404 unknown name; 413 over the decode pixel limit; 422 undecodable file. See *Assessing a run's output* in [the workflow guide](WORKFLOW_GUIDE.md#assessing-a-runs-output) | | `download_output(name, destination=None, overwrite=False, workspace=None)` | `name`, `destination`, `overwrite`, `workspace` | Save one output file to local disk, of any content type. `destination` may be a full path or a directory; `~` expands and missing parent directories are created. `overwrite=True` is required to replace a file already at the resolved path. Over the stdio `dw-mcp`, omitting `destination` saves under the output's own name in the current working directory. Over a `dw.serve --mcp` endpoint the file lands on the server confined to that workspace (a relative path is joined onto it), and `destination` is required there - an omitted one is refused rather than dropped loose in the workspace root, where nothing can find or delete it later (#353); use the `url` `list_gallery` reports, `get_output_image`/`get_output_audio`/`get_output_frames` for inline content, or `keep_output` to make it a named asset instead. Returns nothing to the conversation but where the file landed — unlike the other media tools, the point is a file on disk, not a payload in context. Writes on the machine running the MCP server - over `dw.serve --mcp` that is the GPU box. A write that fails there (a path that exists only on the client, for instance) comes back as an error naming the server-side write and the client-side alternatives, not as an anonymous tool failure. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | | `delete_output(name=None, workspace=None, job_id=None)` | exactly one of `name` / `job_id`, `workspace` | Permanently remove one generated file from the output directory, or - with `name` a `/` run directory, or with `job_id` - a whole run. By `job_id` the server reads the run directory from the job record and the root the job ran in (`DELETE /api/jobs/{id}/run`), and the reply is `job_id`, `run_dir`, `deleted` and `run_swept`; a job that never wrote a run directory, an unknown one, or one still queued or running is an error. `workspace` names the workspace for a `name` delete without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | @@ -274,14 +277,16 @@ tools) rather than relying on the session pin. | Tool | Arguments | Purpose | | --- | --- | --- | -| `validate_workflow(workflow=None, name=None, workspace=None, arguments=None)` | exactly one of `workflow` (inline definition) or `name` (a stored workflow, as `list_workflows` reports it), optional `workspace`, optional `arguments` | Check a workflow against the schema and against real pipeline signatures. Free and instant. Validating by name uses the workflow file's own directory as the base directory, so it sees what a run would. Returns every schema violation in `errors`, each with the JSON path it sits at, so a draft is fixed in one pass, and a `previous_result:` that names no earlier step is one of them. `warnings` covers what still runs but is probably wrong - a signature mismatch, and, for a list-driven variable, an entry key no step reads, at the entry's path. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's. Pass the same `arguments` you will pass to `run_workflow` and they are checked too - an undeclared or renamed variable name, a value that will not coerce to the declared type, and an `asset:`, `prompt:` or `output:` reference that names nothing this workspace can reach, each reported at `arguments.`. `checked_arguments` lists what was covered, so a `valid: true` about the stored defaults cannot be mistaken for one about your values. A reference set the model would refuse - too many images, videos or audio clips, or, for MiniMax-H3, audio as the only reference - is an error here too, rather than a failure minutes into a run you acknowledged. `run_workflow` makes the same check and refuses a bad argument rather than queuing a job that fails on its first step. A valid answer carries `plan` - the fingerprint, step count, list lengths, `downloads_required`, `estimate` (with `basis`) and `elided_steps` for the arguments given; quote from it. `steps` counts what will run: a step nothing reads and which saves no file does not run, and is named in `elided_steps` instead | +| `validate_workflow(workflow=None, name=None, workspace=None, arguments=None)` | exactly one of `workflow` (inline definition) or `name` (a stored workflow, as `list_workflows` reports it), optional `workspace`, optional `arguments` | Check a workflow against the schema and against real pipeline signatures. Free and instant. Validating by name uses the workflow file's own directory as the base directory, so it sees what a run would. Returns every schema violation in `errors`, each with the JSON path it sits at, so a draft is fixed in one pass, and a `previous_result:` that names no earlier step is one of them. `warnings` covers what still runs but is probably wrong - a signature mismatch, and, for a list-driven variable, an entry key no step reads, at the entry's path. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's. Pass the same `arguments` you will pass to `run_workflow` and they are checked too - an undeclared or renamed variable name, a value that will not coerce to the declared type, and an `asset:`, `prompt:` or `output:` reference that names nothing this workspace can reach, each reported at `arguments.`. `checked_arguments` lists what was covered, so a `valid: true` about the stored defaults cannot be mistaken for one about your values. A reference set the model would refuse - too many images, videos or audio clips, or, for MiniMax-H3, audio as the only reference - is an error here too, rather than a failure minutes into a run you acknowledged. `run_workflow` makes the same check and refuses a bad argument rather than queuing a job that fails on its first step. A valid answer carries `plan` - the fingerprint, step count, list lengths, `downloads_required`, `estimate` (with `basis`), `host_memory` (`{projected_mb, basis, runs}`, the projected host-RAM peak; `basis` is `observed` from same-key successful runs on this machine or `fit`; absent when there is no figure) and `elided_steps` for the arguments given; quote from it. `steps` counts what will run: a step nothing reads and which saves no file does not run, and is named in `elided_steps` instead | | `list_workspaces()` | — | The server's workspaces and which one this session is using. Each has its own workflows, assets and outputs; the prompt library is shared by all of them | | `use_workspace(name)` | `name` | Work in that workspace for the rest of the session - every later call reads and writes there. This is how to keep your work out of another agent's namespace rather than sharing the default one. Checked against the server, so a typo fails here rather than scoping every later call to nothing. On a `dw.serve --mcp` endpoint the pin is shared by every connected client (#298, see the note above the table) - a `warning` field appears when this call overwrote a pin already pointed away from `default` | | `create_workspace(name, use=False)` | `name`, `use` | Create a workspace. Pass use=true to switch this session to it as well; otherwise the session stays where it was and the result says so. The result names the workspace (`name`, `default`, `current`, `next`), not its folders - `list_workspaces(detail=true)` is the opt-in for those. `use=true` carries the same mounted-server caveat and `warning` field as `use_workspace` | +| `copy_to_workspace(destination, runs=None, assets=None, source=None)` | `destination`, `runs`, `assets`, `source` | Copy runs and assets from one workspace into another. Each `runs` entry is `/` as `list_gallery` names it, or a bare `` for all its runs; each `assets` entry is a path as after `asset:`, or a folder (`proj/`) for every file under it, keeping its path, note and kept shots. `source` defaults to this session's workspace. Free: no `acknowledged_cost`. Media is hard-linked (a copy across devices falls back to a real copy) and JSON sidecars are always copied, so an edit in one workspace never reaches the other. All or nothing; a run or identical asset the destination already has is skipped as `already_present`, a different file at an asset's path refuses the whole request, so a repeat is idempotent, and a `vN` it already uses is renumbered after its highest and reported in `renumbered`. Warnings: `unresolved_output_refs`, `absolute_paths`, `no_jobs`. A copied run has no job record - it shows in `list_gallery` and resolves as an `output:` reference, not in `list_jobs`. A run whose job is still queued or running is refused. To move, copy then `delete_output` the source runs and `delete_asset` the source assets. See [Workspaces](WORKSPACES.md#copying-between-workspaces) | | `delete_workspace(name, acknowledged_cost=False)` | `name`, `acknowledged_cost` | Permanently delete a workspace and everything in it. Refuses without the acknowledgement, reporting what it would remove | | `list_assets(limit=50, prefix)` | `limit`, `prefix` | The input media on the server, each with the `asset:` reference a workflow argument carries. Look here before asking for a file - what a workflow needs may already be there. `libraries` names the roots searched and which are writable; `shadowed` lists names a nearer library hides. Newest first; `prefix` keeps names starting with it (e.g. `cast/`), `limit` bounds the answer, `total` is the count matched before the cut | -| `keep_output(name, asset_name=None, overwrite=False, shared=False, workspace=None)` | `name`, optional `asset_name`, `overwrite`, `shared`, `workspace` | Keep a generated file as an input asset under a stable `asset:` name, so a later workflow can rely on it. The copy happens on the server: nothing is downloaded or re-uploaded. `asset_name` may name a folder and takes the kept file's extension when it has none; `shared=true` keeps it in the library every workspace shares, which is where a recurring cast belongs. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | -| `upload_asset(file_path=None, content=None, asset_name=None, shared=False)` | exactly one of `file_path` or `content` (base64), optional `asset_name`, `shared` | Push an image, video or audio file, or a `.cube` 3D LUT for `apply_lut`, into the server's asset library and get back its `asset:` reference. `file_path` is read from the machine the MCP server runs on, so this is how an input reaches a dw.serve running somewhere else - but only when the caller shares a filesystem with that machine. `content` is the alternative for an agent that does not: the bytes travel inline, base64-encoded, in the tool call itself, capped at 4MB (well under `file_path`'s 200MB) because inline bytes compete with the calling agent's own context budget rather than being a bulk-transfer path (#203). `asset_name` is required with `content` (there is no file name to infer one from) and, either way, stores it under a readable name (`cast/priya-voice.wav`) instead of a random one; `shared=true` puts it in the library every workspace shares | +| `keep_output(name, asset_name=None, overwrite=False, shared=False, workspace=None, note=None)` | `name`, optional `asset_name`, `overwrite`, `shared`, `workspace`, `note` | Keep a generated file as an input asset under a stable `asset:` name, so a later workflow can rely on it. The copy happens on the server: nothing is downloaded or re-uploaded. `note` is kept with the asset; left out, the source file's note is copied. `asset_name` may name a folder and takes the kept file's extension when it has none; `shared=true` keeps it in the library every workspace shares, which is where a recurring cast belongs. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | +| `set_note(target, note=None, workspace=None)` | `target` (a run `/`, a gallery file name, `asset:` or `workspace:`), `note`, `workspace` | Set the note on a run, an output file, an asset or a workspace; null or `""` clears it. Plain text up to 4000 characters saying what the thing is for - what a run tested, why a file is the keeper. Free: no `acknowledged_cost`. Returns `{target, note, note_source}`. `run_workflow(note=...)` sets a run's note at submission and it shows on every file the run wrote; `list_gallery`, `list_assets`, `list_workspaces` and `list_jobs` carry each note clipped to 200 characters (`note_truncated: true` when cut), while `get_job` and `get_gallery_metadata` return it whole. `get_job` shows the run's current note, so one set here after submission replaces the submitted one. `workspace` names the workspace a run or file name is read in for this one call without switching the session to it - the same pin `run_workflow` takes | +| `upload_asset(file_path=None, content=None, asset_name=None, shared=False, note=None)` | exactly one of `file_path` or `content` (base64), optional `asset_name`, `shared`, `note` (kept with the asset) | Push an image, video or audio file, or a `.cube` 3D LUT for `apply_lut`, into the server's asset library and get back its `asset:` reference. `file_path` is read from the machine the MCP server runs on, so this is how an input reaches a dw.serve running somewhere else - but only when the caller shares a filesystem with that machine. `content` is the alternative for an agent that does not: the bytes travel inline, base64-encoded, in the tool call itself, capped at 4MB (well under `file_path`'s 200MB) because inline bytes compete with the calling agent's own context budget rather than being a bulk-transfer path (#203). `asset_name` is required with `content` (there is no file name to infer one from) and, either way, stores it under a readable name (`cast/priya-voice.wav`) instead of a random one; `shared=true` puts it in the library every workspace shares | | `delete_asset(name)` | `name` | Permanently remove one file from the asset library, by the name `list_assets` reports. Deletes from whichever library holds it - this workspace's own before the shared one; one from a read-only examples library is refused. Any workflow still carrying that `asset:` reference stops loading | | `save_workflow(name, workflow)` | `name`, `workflow` | Save a workflow into the server's writable workflow directory, overwriting any existing workflow of that name there. A name that currently resolves to a read-only source (an examples directory) is not overwritten - the copy lands in the writable directory and shadows it | | `delete_workflow(name)` | `name` | Permanently delete a stored workflow | @@ -310,12 +315,12 @@ references written in the same session. | Tool | Arguments | Purpose | | --- | --- | --- | -| `run_workflow(workflow_path=None, inline_workflow=None, arguments=None, acknowledged_cost=False, workspace=None, wait_seconds=0)` | exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path to a workflow file on the server) or `inline_workflow`, optional `arguments`, `acknowledged_cost`, `workspace`, `wait_seconds` | Queue a workflow for generation. Returns as soon as the job is queued - unless `wait_seconds` is above 0, in which case the call then waits on the queued job exactly as `wait_for_job(job_id, timeout_seconds=wait_seconds)` would (same per-call cap, clamped not honoured) and the result carries the queued-job fields plus that wait's (`status`, `still_running`, `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds`, `timeout_capped`, the slim `job`); when the cap covers the job's runtime one call is the run and the wait, and a `still_running: true` result is followed with `wait_for_job` as before. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's - `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and the message carries the new estimate, and nothing is waited on | -| `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback; each manifest entry's `subfolder` is the in-run subfolder the step declared - by convention `final` for the deliverable, `intermediate` for scratch, `''` for none. A running job also carries `progress` (below) | +| `run_workflow(workflow_path=None, inline_workflow=None, arguments=None, acknowledged_cost=False, workspace=None, wait_seconds=0, note=None)` | exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path to a workflow file on the server) or `inline_workflow`, optional `arguments`, `acknowledged_cost`, `workspace`, `wait_seconds`, `note` | Queue a workflow for generation. Returns as soon as the job is queued - unless `wait_seconds` is above 0, in which case the call then waits on the queued job exactly as `wait_for_job(job_id, timeout_seconds=wait_seconds)` would (same per-call cap, clamped not honoured) and the result carries the queued-job fields plus that wait's (`status`, `still_running`, `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds`, `timeout_capped`, the slim `job`); when the cap covers the job's runtime one call is the run and the wait, and a `still_running: true` result is followed with `wait_for_job` as before. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's - `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and the message carries the new estimate, and nothing is waited on. `note` says what the run is for (see `set_note`) | +| `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback; each manifest entry's `subfolder` is the in-run subfolder the step declared - by convention `final` for the deliverable, `intermediate` for scratch, `''` for none. A running job also carries `progress` (below); a queued job held for host RAM carries `waiting_for` (#869) | | `get_job_workflow(job_id)` | `job_id` | The REST equivalent is `GET /api/jobs/{id}/workflow` (see [SERVER.md](SERVER.md#jobs-api)). The workflow the job actually ran. `realized: true` means every mutable input is pinned (arguments, seed, prompts, `output:latest`); `false` means the job predates run tracking and this is the definition as submitted. Pass it to `save_workflow` to keep it under a name | | `export_job(job_id, overwrite=False)` | `job_id`, `overwrite` | Gather one finished job into `/exports//` on the server: the realized workflow, the run's manifest, the job row, a README, and copies of the assets, earlier-run inputs and outputs. Returns the directory, a zip URL, the file list with sizes and the total. The three JSON files are in the zip, not repeated here - get_job_workflow and get_job serve them individually. **The directory is on the machine running the server**, like `download_output`'s destination. The zip needs no token (`/exports/*.zip` is ungated like `/outputs`, #592), so `auth_required` is false and the agent fetches `open_url` itself (prefixing a relative one with the server address) and unpacks it into `exports/` under the session's working directory (a deliverable, not a temp file) - the archive already unpacks into one folder named after the job id, so do not create that folder first; only an agent without HTTP gives the user `open_url`. `true` is kept as a forward guard for a gated zip, when `open_url` is handed to the person instead | | `get_job_events(job_id, after=-1, limit=200)` | `job_id`, `after`, `limit` | Get a page of a job's progress events | -| `wait_for_job(job_id, timeout_seconds=20)` | `job_id`, `timeout_seconds` | Block until a job reaches a terminal status, or `timeout_seconds` elapses. **One call blocks for at most the server's cap** — 55 seconds unless the deployment sets `DW_MCP_MAX_WAIT_SECONDS` higher, and the tool's description states the live value; a larger `timeout_seconds` is clamped, not honoured. Ask for the job's `plan.estimate` plus a margin: under the cap that is one call for the whole job, over it one call per cap. A deployment raises the cap only as far as its clients (and anything between them and the server) hold one silent HTTP request open - Claude Code's limit is `MCP_TOOL_TIMEOUT`. Every reply carries `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds` and `timeout_capped`, so a capped return is distinguishable from an elapsed one. Use instead of hand-polling `get_job`/`get_job_events` in a loop; if it returns `still_running: true`, call it again. Returns a slim job - status, warnings, error, `run_id` and `run_version` (the run's `v5`, as the gallery labels it), and the manifest once finished - without the arguments; `get_job` has those. A running job also carries `progress` (below) | +| `wait_for_job(job_id, timeout_seconds=20)` | `job_id`, `timeout_seconds` | Block until a job reaches a terminal status, or `timeout_seconds` elapses. **One call blocks for at most the server's cap** — 55 seconds unless the deployment sets `DW_MCP_MAX_WAIT_SECONDS` higher, and the tool's description states the live value; a larger `timeout_seconds` is clamped, not honoured. Ask for the job's `plan.estimate` plus a margin: under the cap that is one call for the whole job, over it one call per cap. A deployment raises the cap only as far as its clients (and anything between them and the server) hold one silent HTTP request open - Claude Code's limit is `MCP_TOOL_TIMEOUT`. Every reply carries `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds` and `timeout_capped`, so a capped return is distinguishable from an elapsed one. Use instead of hand-polling `get_job`/`get_job_events` in a loop; if it returns `still_running: true`, call it again. Returns a slim job - status, warnings, error, `run_id` and `run_version` (the run's `v5`, as the gallery labels it), and the manifest once finished - without the arguments; `get_job` has those. A running job also carries `progress` (below); a queued job held for host RAM carries `waiting_for` | | `cancel_job(job_id)` | `job_id` | Ask a queued or running job to stop | | `clear_memory()` | `device` | Drop every loaded pipeline and the step cache on each idle card - `device`'s alone when named - freeing VRAM/RAM immediately instead of waiting for the next job to evict one model for another. Also drops the step cache, so a seeded workflow that would otherwise reuse cached results regenerates on its next run. A card running a job is skipped (`workers` reports it `job_running`); refused with a 409 when no card asked about is idle, so wait and retry rather than expecting this call to block (#221, #677) | | `rerun_job(job_id, acknowledged_cost=False, new_seed=False)` | `job_id`, `acknowledged_cost`, `new_seed` | Queue a fresh job from a previous job's stored specification. Costs GPU time, so it passes the same gate as `run_workflow`. `new_seed=true` draws a fresh seed into the workflow's seed variable — without it a seeded workflow's rerun repeats its arguments exactly and the step cache serves the whole run from the earlier one's files (`reused: true`), generating nothing. `get_job_workflow`'s `seed_variable` says whether there is one - `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and the message carries the new estimate | @@ -328,7 +333,7 @@ references written in the same session. | `list_models()` | — | (Catalog) List what the Hugging Face cache holds, largest first | | `download_model(repo_id, acknowledged_cost=False)` | `repo_id`, `acknowledged_cost` | Fetch a model repo into the cache. Costs disk and bandwidth, so it passes the gate. Returns as soon as the download starts | | `list_downloads()` | — | List downloads the server is running or recently ran | -| `cancel_download(download_id)` | `download_id` | Stop a running download. Partial files stay cached and resume on a retry | +| `cancel_download(download_id)` | `download_id` | Stop a running download; the record shows `cancel_requested: true` until it is `cancelled`. Partial files stay cached and resume on a retry | | `delete_model(repo, acknowledged_cost=False)` | `repo`, `acknowledged_cost` | Delete every cached revision of one repo. Not recoverable locally | | `get_diffusers_state()` | — | Installed diffusers version, its git commit, and any update in flight | | `update_diffusers(acknowledged_cost=False)` | `acknowledged_cost` | Upgrade diffusers to GitHub HEAD in the background | @@ -398,9 +403,12 @@ The intended loop: terminal/not-terminal status. Each event carries `at`, seconds since the job started, so where a step's time went is a subtraction between two events - `step_start` to `generating` is the lead-in a reused pipeline - still pays, `generating` to the first `pipeline_step` the encoding + still pays; `generating` is emitted when the pipeline call starts, so + `generating` to the first `pipeline_step` is the encoding plus the first + denoise pass (`timing.encode_s` on the manifest entry is the encoding alone) 4. `get_job(job_id)` for the finished manifest (or the error and traceback, - if it failed) + if it failed). A pipeline step's entry carries `timing` - load, encode, + denoise, decode and save seconds ([WORKSPACES.md](WORKSPACES.md#step-timing)) 5. `get_output_image(name)` to look at a result image 6. `get_output_frames(name, count=12)` to look at a result video, and `get_output_audio(name, start, duration)` to hear it diff --git a/docs/RECIPES_24GB.md b/docs/RECIPES_24GB.md index 5d1346f1..1ed6c2c8 100644 --- a/docs/RECIPES_24GB.md +++ b/docs/RECIPES_24GB.md @@ -116,7 +116,7 @@ pipeline-level `offload`. The working 24GB configuration at 960x544: | `text_encoder` | `remove_modules: ["lm_head"]` - the encoder path never calls it | | `text_encoder.model` | SDNQ int4, `truncate_layers: {"language_model.layers": 51}`, `group_offload` `leaf_level` | | `vae`, `audio_vae` | SDNQ int8, `device: "cuda"`, `residency: "on_demand"` | -| pipeline | `cache: first_block` (`threshold: 0.1`) on the 20-step ref2va workflows only - measured on the 9-step turbo schedule it never skips (consecutive distilled steps differ too much for the threshold), so the turbo examples omit it rather than hold cache state for nothing | +| pipeline | `cache: first_block` (`threshold: 0.05`) only on a ~30-step base-model run (`lora_model_name: null`): -22% denoise there, identical staging and faces. It skips nothing at 12 steps or on a turbo schedule (consecutive distilled steps differ too much), and every H3 template takes it as a `cache` argument (null by default): `{"type": "first_block", "threshold": 0.05}`. Measured on a 3090 with `sage_hub`; see the `minimax-h3` skill's `references/quality-and-speed.md` | The VAEs are the piece worth calling out. They hold roughly 3GiB, are used only to encode references and decode the result, and group offloading them is worse than useless diff --git a/docs/RELEASING.md b/docs/RELEASING.md index e1fcf87c..997e7f1a 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -7,6 +7,115 @@ notes from commits at tag time (see below). This section is a scratch pad for items a branch's author wants the next release note to name; clear it when a release ships. +### 0.14.0 + + + +A release about what a run is and what it will cost. Gallery files say +whether they are a final or an intermediate, anything can carry a note, and +runs copy between workspaces. The cost estimate scales with steps, canvas +and frames instead of quoting a flat figure that was up to 12x off, and a +queued job waits for host RAM rather than being OOM-killed. + +**Roles and notes** (#855-#858) + +- Gallery files are tagged final or intermediate (inferred when a workflow + doesn't say), and the gallery opens on finals. A run lists what it was + Made from. +- A file a later run in the workspace read, through `output:` or an asset + kept from it, lists as intermediate, so a project's shots, narration and + score step aside once the cut that uses them exists. A step that sets + `result.role: "final"` stays final. The `script-to-video` and + `series-episodes` skills have ingredient workflows declare + `"role": "intermediate"` (#928). +- Runs, files, assets and workspaces take an optional note: `note` on + `run_workflow`, `upload_asset` and `keep_output`, and MCP `set_note`. + Every listing carries notes; the web UI edits them in place as Markdown. +- Starring a file keeps it. A rerun records the job it reran (`rerun_of`). +- `copy_to_workspace` copies whole runs, with manifests, and asset-library + entries into another workspace (#853, #859, #860). + +**Cost and host RAM** (#866, #869, #881) + +- The estimate scales as steps x canvas x frames against the template's + defaults (`basis: "derived"`, `scaled` names the axes). Any other driver + that moves keeps the figure and sets `low_confidence`; H3 adds weights, + LoRA, attention and cache as drivers. On the H3 matrix it lands within + 30% (B3->L5: 32.3 min quoted, 29.5 run), and an inline copy prices + within 2% of its template (#906-#908, #915, #916, #921). +- A queued job is held at dispatch while its projected host-RAM peak won't + fit, and stays `queued` with `waiting_for` and a `queue_wait` event. It + is never held with nothing else running. On a 125 GB, 2x 3090 host a + second int8 H3 job waited ("needs ~87 GB, ~57 GB free") while two int4 + jobs ran at once (#909, #910). +- An OOM kill now takes the newest dispatch, not the earlier job (#881). + +**Timing and caches** (#867, #868, #905) + +- Manifest entries carry per-step timing (`first_pass_s`, `pass_s`) and + step-cache skip counts (`cache_stats`). A `first_pass_slow` warning names + a warm pipeline's slow first pass (#924: 202 s against 37 s/pass). +- Validate warns on acceleration that can't pay off or won't run: a cache + on a short schedule or with no transformer to attach to, `fullgraph` + with a cache, a CUDA-only attention backend, and an unbraced type name + under a `*_dtype` key (#887, #904, #926). +- The plan fingerprint and step cache key only on what changes the output, + so metadata edits and `12.0` vs `12` no longer miss the cache (#905, + #911). +- Every H3 template exposes `attention_backend` and `cache` (#883, #884). + MagCache calibration now reports its ratios on H3 (#927); with them, + MagCache 0.06 ran the cinema final in 18.1 min, 39% under L5 (#875). + +**H3 quality ramp** (#861, #862, #882) + +- The `minimax-h3` skill has measured draft -> final rungs. TV: L1 (~5.4 + min) -> L7 (~7.2 min), changing only `weights_dtype`. Cinema: B3 (~12.9 + min) -> L5 (~29.5 min), changing only steps. Staging carries at the same + seed; the skill says what to approve at each tier. +- A proven LoRA catalog entry decides which H3 step an adapter fits, + ahead of its filename (#870, #888). + +**Compare, sweeps and check-script** (#871, #872, #919-#925) + +- `compare_outputs` puts 2-4 outputs side by side with PSNR/SSIM; the web + UI's Compare page plays them in sync and lists the differing arguments. + The workspaces guide has a Running a sweep section. +- check-script refuses a wrong shot name at validate, adds + `unscripted_speech` and `speech_not_transcribed`, and levels its input + so the verdict no longer depends on loudness. + +**Web UI** + +- A running-jobs chip, a tab title that tracks the running job, measured + run times on cards, drag-and-drop upload, and keyboard culling in the + gallery. + +**Fixes** + +- `cancel_download` stops queued files and shows `cancel_requested` + (#899); a model download records `refs/main`, so it reads complete. + `cancel_job` takes effect during a load (#892). +- A kept asset keeps its lineage and its job survives the workspace's + deletion (#902, #913, #917, #918). +- `get_gallery_metadata` measures an image, and `include_workflow=false` + omits the embedded workflow (#922). +- An inline copy of a list-driven template is no longer quoted the + template's default-length price unscaled. +- The host-RAM gate no longer under-projects a list-driven run whose peak + grows with the list. +- Whole-module `compile: {fullgraph: true}` no longer fails at the first + denoise step after the timing hook. +- FasterCache runs on FLUX.1-dev (#897); denoise time on mps is no longer + billed to decode (#896). +- The gallery's sort no longer stalls each keystroke on a large library. + +**Dependencies and deploy** + +- No dependency floors changed. +- `scripts/dw-serve.service` sets `OOMPolicy=continue`, so one worker's + OOM kill no longer stops the server. Reinstall the unit to pick it up. +- The jobs history gains a `memory_key` column, migrated on start. + ### 0.13.0 @@ -140,9 +249,10 @@ hallucinations as lyrics. **Fixes** -- The inherited cost estimate is withheld when an inline workflow runs - more steps or frames than the template it came from, instead of - quoting the template's number (#821: a 7x under-quote). +- The inherited cost estimate is re-priced linearly when an inline + workflow runs more steps or frames than the template it came from, + instead of quoting the template's number (#821: a 7x under-quote); the + estimate carries `scaled` (why) and `low_confidence` (#851). - `assess_output` shot findings carry `index` and `start_frame`, so a repeated shot name is unambiguous (#835). - `dialogue-short` and `music-video` expose `match_levels_dbfs` (#836). diff --git a/docs/SERVER.md b/docs/SERVER.md index 9e403bcb..5dd42d32 100644 --- a/docs/SERVER.md +++ b/docs/SERVER.md @@ -201,17 +201,17 @@ running: | Route | What it does | | --- | --- | | `POST /api/jobs` | Queue a run: `{"workflow_path": ...}` or an inline `{"workflow": {...}, "base_dir": ...}`, plus `arguments` for variable overrides. `workflow_path` accepts a stored workflow name as listed by `/api/workflows` (with or without `.json`, nested names included), or a relative/absolute path that still resolves under `--workflow-dir` - confined the same way the `/api/workflows` CRUD routes are; a path that names a real file outside that directory is rejected with 400, not opened. Answers with argument warnings from signature checking. Takes an optional `acknowledged_cost`: `true` is recorded as `acknowledged: boolean`; the object `{fingerprint, minutes, downloads}` from a validate answer's `plan` is `bound` - the server re-plans the run for the arguments given and answers **409** when the fingerprint differs or a repo in `downloads_required` is not in `downloads` (a download that has since vanished is not a refusal); the body is `{"detail": {message, reason: "fingerprint" \| "downloads" \| "unplannable", acknowledged, plan, acknowledge}}` with the current plan, so the caller re-quotes from it, and `acknowledge` - the `{fingerprint, minutes, downloads}` to resend - whenever there is a plan. A `null` in `downloads` (a URL with no repo) is ignored. `minutes` is recorded, never compared. Nothing is required: the web UI and every caller that sends nothing are `acknowledged: none`, and every job answer and history row carries `acknowledged` (and `acknowledged_cost` when bound). `POST /api/jobs/{id}/rerun` takes the same field and checks against the stored spec; a fresh seed does not change a fingerprint. | -| `GET /api/jobs?workspace=&status=&limit=` | Queue + history summaries, oldest first, with `total` beside them. `status` narrows to one state or a comma-separated set (`queued`, `running`, `succeeded`, `failed`, `cancelled`; anything else is a 400); `limit` keeps the newest N, and `total` still reports how many matched, so a bounded answer cannot be mistaken for a complete one. Each summary carries `device`, the card the job ran on (`"cuda:1 NVIDIA GeForce RTX 3090"`; null while queued and for older jobs). No parameters means every job, which is what the web UI polls | -| `GET /api/jobs/{id}` | Full detail: spec, events, manifest, error. A manifest entry for a step served from the step cache carries `reused: true`. Every entry carries `subfolder` - the in-run subfolder the step's `result.subfolder` chose, `''` for none. A `for_each` step appears in the manifest as its members (`shot@wide_open`, `shot@closeup`), because the manifest records what ran; the run's `workflow.json` keeps the `for_each` form, because it records what was asked Carries `output_kinds`: each manifest file mapped to its media kind (`image`, `video`, `audio`, `text`, or null), and each file-carrying run event (`step_end`) carries the same map for its own files. | +| `GET /api/jobs?workspace=&status=&limit=` | Queue + history summaries, oldest first, with `total` beside them. `status` narrows to one state or a comma-separated set (`queued`, `running`, `succeeded`, `failed`, `cancelled`; anything else is a 400); `limit` keeps the newest N, and `total` still reports how many matched, so a bounded answer cannot be mistaken for a complete one. Each summary carries `device`, the card the job ran on (`"cuda:1 NVIDIA GeForce RTX 3090"`; null while queued and for older jobs), and `rerun_of`, present on a job started by rerun: the id of the job it reran; absent for a fresh submit. No parameters means every job, which is what the web UI polls | +| `GET /api/jobs/{id}` | Full detail: spec, events, manifest, error. A manifest entry for a step served from the step cache carries `reused: true`. Every entry carries `subfolder` - the in-run subfolder the step's `result.subfolder` chose, `''` for none. A `for_each` step appears in the manifest as its members (`shot@wide_open`, `shot@closeup`), because the manifest records what ran; the run's `workflow.json` keeps the `for_each` form, because it records what was asked Carries `output_kinds`: each manifest file mapped to its media kind (`image`, `video`, `audio`, `text`, or null), and each file-carrying run event (`step_end`) carries the same map for its own files. The entry of a pipeline step that ran carries `timing` (see [Step timing](WORKSPACES.md#step-timing)). Carries `rerun_of` - present on a job started by rerun: the id of the job it reran; absent for a fresh submit. | | `DELETE /api/jobs/{id}/run` | Delete the run directory a finished job wrote, whole, from the output root the job ran against (no workspace selector applies): `{job_id, run_dir, deleted, run_swept}`. 404 for an unknown job or one with no run directory, 409 for one still queued or running | | `GET /api/jobs/{id}/workflow` | The workflow the job ran: `{id, definition, realized, seed_variable}`. `seed_variable` names the variable a `new_seed` rerun would draw into (null when the workflow has none), read from the workflow as written rather than the realized copy, whose seed is pinned. `realized: true` is the copy the run itself wrote (`workflow.json` in its run directory), with arguments, seed, prompts and `output:latest` pinned; `false` falls back to the submitted definition, which is what a job from before run tracking has. 404 means neither is readable - the job itself still is. The equivalent MCP tool is `get_job_workflow` (see [MCP.md](MCP.md#diagnose)) | | `POST /api/jobs/{id}/export?workspace=&overwrite=` | Gather one finished job into `/exports//`: `workflow.json`, `manifest.json`, `job.json`, `README.md`, `assets/`, `inputs/`, `outputs/`. 201 with the file list, total bytes, anything it could not find, a `zip_url`, and the three JSON files inline. 404 unknown job, 409 for a job still running or an existing export without `overwrite` | | `GET /exports/{id}.zip?workspace=` | The same tree as one archive, built on request rather than kept as a second copy. Entries are named `/`. Ungated exactly as `/outputs` is | -| `GET /api/jobs/{id}/events` | Server-sent events stream; `?after=N` / `Last-Event-ID` replay missed events, so reconnects are lossless. Every event carries `seq` and `at` - seconds since the job started (since it was queued, for the events before that) - so a step's cost is a subtraction: `step_start` to `generating` is what a reused pipeline still pays before it runs, `generating` to the first `pipeline_step` is the prompt and reference encoding | +| `GET /api/jobs/{id}/events` | Server-sent events stream; `?after=N` / `Last-Event-ID` replay missed events, so reconnects are lossless. Every event carries `seq` and `at` - seconds since the job started (since it was queued, for the events before that) - so a step's cost is a subtraction: `step_start` to `generating` is what a reused pipeline still pays before it runs; `generating` is emitted when the pipeline call starts, so `generating` to the first `pipeline_step` is the prompt and reference encoding plus the first denoise pass (`timing.encode_s` in the manifest is the encoding alone; see [Step timing](WORKSPACES.md#step-timing)). The server keeps only the last 200 events, so on a long job the early ones are gone | | `GET /api/jobs/{id}/event-log?after=-1&limit=200` | The same events as the SSE stream, as one JSON page: `{id, status, events, last_seq, truncated, note}`. `after` is exclusive; page by passing back the previous `last_seq`. A job restored from history serves the bounded event tail persisted with it; a job that finished before events were retained returns an empty list and a `note` saying so. | | `POST /api/jobs/{id}/cancel` | Cooperative cancel (takes effect at the next step boundary or denoise step) | -| `POST /api/jobs/{id}/rerun` | Re-queue a finished job's spec. Body `{"new_seed": true}` draws a fresh seed into the workflow's seed variable instead of repeating the original arguments; 400 when the workflow pins its seed to a literal or names none. A plain rerun of a seeded workflow is served whole from the step cache — the earlier run's files, republished with `reused: true`, generating nothing | -| `POST /api/jobs/{id}/move` | Reorder a queued job: `{"direction": "up"\|"down"\|"front"\|"back"}`. Job listings carry each waiting job's `queue_position`. | +| `POST /api/jobs/{id}/rerun` | Re-queue a finished job's spec. Body `{"new_seed": true}` draws a fresh seed into the workflow's seed variable instead of repeating the original arguments; 400 when the workflow pins its seed to a literal or names none. A plain rerun of a seeded workflow is served whole from the step cache — the earlier run's files, republished with `reused: true`, generating nothing. The new job's summary and detail carry `rerun_of`: the id of the job it reran (absent on a fresh submit) | +| `POST /api/jobs/{id}/move` | Reorder a queued job: `{"direction": "up"\|"down"\|"front"\|"back"}`. Job listings carry each waiting job's `queue_position`, and `waiting_for` while one is held for host RAM. | Each worker runs one job at a time, so with one device (the default) one job runs at a time; submissions queue in order, and the waiting portion of the @@ -228,10 +228,34 @@ on one card); else any card. A card's size is its GiB rounded up (a 3090 counts as 24), which a `cost` figure is compared to; a declared estimate is held to the card's GiB to one decimal (a 3090's 23.6), the same ceiling admission checks it against. Cancel reaches only the worker running that job, and a worker -crash fails only its own job. The worker started later gets a higher Linux -`oom_score_adj` (+100 over the highest other live worker), so a host-RAM -squeeze kills the later job; nothing is written with one worker. There is no -host-RAM gate at dispatch: two big jobs can still exhaust host RAM together. +crash fails only its own job. Each dispatch sets its worker's Linux +`oom_score_adj` to 1000 and every other live worker's back to the value it was +spawned with (the server's own, e.g. 200 under a systemd user unit - the kernel +refuses an unprivileged write below that), so a host-RAM squeeze kills the +newest job whatever the workers' sizes; nothing is written with one worker, and +a write the kernel refuses is a warning on the job. Under systemd the unit needs +`OOMPolicy=continue` (`scripts/dw-serve.service` sets it): the default `stop` +answers one worker's OOM kill by stopping the whole server, every other job +with it. A +queued job may also carry a projected host-RAM peak (see +`host_ram_need`, #869). When it fits a free card, dispatch checks host RAM +too: available is `MemAvailable` less, for each running job, what that job's +projected peak still has to add beyond its worker's resident size now, and +the job is held while its peak exceeds available plus the resident size of +the idle worker it would reuse. A running job with no figure is charged +nothing extra, since `MemAvailable` already reflects what it holds. Backfill +applies: a smaller job behind a held one may start. With no other job +running a job is never held, so it waits only for running jobs, never +forever; a job with no figure is never held; and where the host's RAM cannot +be read (macOS without psutil) there is no gate. With one card only one job +runs, so the gate is never exercised. A held job is re-checked every 10 s, +since memory can free outside dw. The hold emits one `queue_wait` event +(`reason: "host_ram"`, message `waiting for host RAM: needs ~N GB, ~M GB +free`; not a `warning`, so nothing lands in the job's `warnings`), and job +detail and summaries carry `waiting_for` - that message - only while the job +is queued and held. The web UI shows it beside "#n in line". The gate never +refuses a submit (#243); it only delays a start, and the OOM ranking above +(#881) stays the backstop for a wrong projection. Among the free cards a job fits, dispatch prefers the one it has an affinity for: a rerun goes to the card that ran the original, and any job to the card whose worker last ran the same workflow (the same file, or an inline @@ -244,6 +268,13 @@ card the job would be dispatched to now, and the estimate names it in `priced_for`. Memory is per card: `GET /api/memory` and `POST /api/memory/clear` take an optional `device`. +### Step timing + +A pipeline step's manifest entry carries `timing`, where its time went; the +fields are in [WORKSPACES.md](WORKSPACES.md#step-timing), which MCP serves as +`get_guide("workspaces", section="Step timing")`. A step that set a `cache` also +carries `timing.cache`, its skip counts (see the `cache_stats` event below). + ### Progress events Every event in the stream carries a `seq` and an `event` name: @@ -260,7 +291,8 @@ Every event in the stream carries a `seq` and an `event` name: | `pipeline_step` | each denoise step | `step`, `total_steps`. Emitted for a pipeline that takes a `callback_on_step_end`, and for a `ModularPipeline` (H3, LTX-2, Qwen-Image), which takes none - there the denoise block's own progress bar is what reports | | `phase` | the step changes what it is doing | `phase`, `detail` | | `pipeline_released` | a step with `release_pipeline` drops its pipeline | `step`, `index`, `gpu_memory_allocated_mb` and `gpu_memory_allocated_before_mb` (both `null` where the backend cannot say). Emitted between the step's generation and its files being written, which is where the release happens - so the ordering is readable off the event stream rather than by trying to poll memory through a sub-second write | -| `warning` | a step finds something wrong with what it is about to write | `message`, plus a `kind` and the figures behind it (`level_spread`: `spread_db`, `measure`, `command`; `fps_mismatch`: `declared_fps`, `source_fps`; `audio_no_headroom`: `file`, `peak_dbfs` - a deliverable at or above -0.5 dBFS, which an mp3 or AAC encode decodes over full scale; `step_elided`: `step`, `overridden_by` when a supplied argument is what made it unreferenced). Also appended to the job's `warnings`, prefixed with the step it fired in - the event keeps the moment, `warnings` keeps it where a caller polling the finished job will look, since a warning about the artifact outlives the run that noticed it | +| `warning` | a step finds something wrong with what it is about to write | `message`, plus a `kind` and the figures behind it (`level_spread`: `spread_db`, `measure`, `command`; `fps_mismatch`: `declared_fps`, `source_fps`; `audio_no_headroom`: `file`, `peak_dbfs` - a deliverable at or above -0.5 dBFS, which an mp3 or AAC encode decodes over full scale; `step_elided`: `step`, `overridden_by` when a supplied argument is what made it unreferenced; `cache_stats_unavailable`: `cache_type` - the installed diffusers moved the private hook state dw reads, so the step's cache counts are null; the run is unaffected). Also appended to the job's `warnings`, prefixed with the step it fired in - the event keeps the moment, `warnings` keeps it where a caller polling the finished job will look, since a warning about the artifact outlives the run that noticed it | +| `cache_stats` | a pipeline call that set a `cache` finishes its denoise, once per call | `step`, `segment` (only on a chain segment, e.g. `segment 1/3`), and the fields of `timing.cache`: `type`, `threshold`, `forwards`, `computed`, `skipped`, `hit_rate`. Counts are null for a cache diffusers keeps no per-forward decision for. For watching live (`get_job_events(kinds=["cache_stats"])`); the manifest entry's `timing.cache` is the record | | `workflow_end` | the run finishes | `manifest` | A step spends most of its wall clock outside the denoise loop, and @@ -373,10 +405,19 @@ The editor's forms come from these; they are just as usable from scripts: `weight_name` carrying neither `ref2v` nor `fl2v` cannot be placed, so it is a `warnings` entry naming the rule rather than a refusal - a reference-trained checkpoint nobody has named yet still gets through. + The LoRA catalog is read first (#870): a `proven` entry for that + `model_name` and `weight_name` whose `workflow` sits in one partition + decides it, so a `ref2va` step loading a file the catalog proves for + `t2va`/`fl2va` is refused, whatever its name says. A `trial` or + `rejected` entry's `workflow` only warns; promoting the entry to `proven` + is what makes it refuse. The CLI and the worker have no catalog and keep + the file-name rule alone. A valid answer also carries `plan`, what the run will execute for those arguments: `fingerprint` (`sha256:…` over the realized, expanded - definition with the seed and the documentation keys removed and + definition with the seed and every key that cannot change the output + (cost, `cost_drivers`, `vram_estimate`, descriptions, a result's role, + subfolder and file name) removed and `output:…/latest/…` left unpinned - the same work hashes the same, a longer list or an edited stored prompt does not); `steps`, the expanded member count; `list_entries`, `{variable: length}` for each `for_each` @@ -415,7 +456,16 @@ The editor's forms come from these; they are just as usable from scripts: ones it was measured with), `per_entry` (re-priced from a measured per-entry rate, when the entry carries `per_entry`), `derived` (the stored total extrapolated linearly over a list whose length the caller - changed - an estimate, not a measurement), `other_device` (no entry for + changed, or scaled in proportion along `num_inference_steps`, + `width` x `height` and `num_frames` - per entry in a list, summed - that + the caller moved off the variable defaults a `cost` entry is measured at, + with `scaled` naming each moved axis; an estimate, not a measurement; a + non-numeric value of one of those is `unknown`). Any other declared + driver moved off its default - H3's `weights_dtype` or LoRA, a music + `audio_duration` - has no rule to scale by: the (scaled) figure is kept + under whatever basis it has, with `low_confidence` true and `differs` + naming the drivers that moved, because it was measured at another setting + of them; `other_device` (no entry for the serving backend; the first entry's figure, which is a warning rather than a quote) or `unknown` (no cost block, or more than one list changed so there is nothing honest to extrapolate along) or `inherited` (an @@ -423,11 +473,21 @@ The editor's forms come from these; they are just as usable from scripts: `observed` runs of the catalog template that loads the same pipeline - `component_type` + `model_name` + `workflow`, as the VRAM ceiling is inherited - with `inherited_from` naming it and `differs` listing what the - workflow sets differently among offload, quantization and frame count; - validate warns when it does, and never refuses); a composed child's + workflow sets differently among offload, quantization, frame count, + attention backend, cache and LoRA; more pipeline steps, frames, inference + steps or canvas pixels than the template's run re-price it in proportion, + as the catalog figure is scaled, with `scaled` naming each axis; an + inherited figure has `low_confidence` true when `differs` lists anything + or the figure was scaled; validate warns when it does, and never + refuses); a composed child's cost is added to a curated figure and `partial` is true when a child has none - an `observed` figure already measured the whole run, children - included, so nothing is added to it and `partial` is false. `plan` is `null` when + included, so nothing is added to it and `partial` is false; and + `host_memory`, `{projected_mb, basis, runs}`, the run's projected host-RAM + peak - `basis` is `observed` (the largest job peak RSS of this machine's + successful runs with the same memory key) or `fit` (the list-run projection + from the workflow's own history), and the key is absent when there is no + figure. `plan` is `null` when it could not be built; an invalid answer carries no `plan` key. ## Files and models @@ -533,7 +593,13 @@ The editor's forms come from these; they are just as usable from scripts: `?subfolder=` filter independently (`?version=` too - with `?folder=`, the one run the gallery labels `v4`), and the reply's `folders` and `subfolders` list every distinct value over the whole tree, `''` always a - member of each so root-level files stay selectable + member of each so root-level files stay selectable. Each entry also + carries `role`, `final` or `intermediate` (null for a flat-layout file with + no run; `intermediate` once a later run read the file, unless the step's + `result.role` declared it), and `GET /api/gallery?role=final` lists only + those (repeatable; a null role counts as final). See `dw/roles.py` for how + a role is decided; + the metadata route reports the file's `role` and the run's `inputs` too - `GET /api/gallery/{name:path}/download` — download an output file - `GET /api/builtin-workflows` — the packaged fragments a sub-workflow step names as `builtin:.json` (`dw/workflows/`), which no workflow listing @@ -551,6 +617,33 @@ The editor's forms come from these; they are just as usable from scripts: they apply `X-DW-Crop` and `X-DW-Downscaled-To`. Takes `?token=` - `GET /api/gallery/{name}/frames` takes `max_total_bytes`: over it, every tile shrinks to one shared size and `downscaled_to` says which +- `POST /api/gallery/compare` — two to eight outputs side by side, and how far each is + from a baseline; read-only, queues nothing. Body: `names` (2-8 gallery names or + `asset:` references, one row each), `at` (1-8 moments, seconds or `"frame:N"`; + default 4 evenly spaced frames of the shortest video), `crop` (`[x, y, w, h]`: the + region shown and measured; or one box per name, shown only), `baseline` (default + the first name), `max_dimension` (default 384, longest side of each cell) and + `max_total_bytes`. Moments resolve to one frame index against the baseline (the + first video if the baseline is a still) and every output is read at that frame + number; a frame past an output's end is a blank cell. A still among videos repeats + in every column; all stills give one column and `at` is ignored with a note. A crop + must lie wholly inside every output's frame (400 otherwise). Reply: `image` (the + grid as base64 PNG: `data`, `mime_type`, `width`, `height`; a row per output under a + label band, each cell stamped `frame N`), `downscaled_to` (over `max_total_bytes` the + whole grid shrinks, never below 64 px per cell; null when nothing shrank), + `baseline`, `region` (`full frame`, `crop [...]`, or `full frame (per-output crops + are display only)`), `fps_differs` (videos with different frame rates, so one + frame number is not one second), `columns` (`{moment, frame}`), `rows` (name, kind, + size, crop, and for a video `frame_count` and `fps`), `metrics` and `notes`. A + `metrics` entry per non-baseline output holds `cells` (one per column: + `psnr_db`, `ssim`, `mean_abs_diff`, `reason`) and `psnr_db`/`ssim`/`mean_abs_diff` + summaries (`mean`, `min`, `max`), measured at source resolution; PSNR is + `identical` when equal and SSIM is on luma (11x11 Gaussian window, sigma 1.5). An + output whose size differs from the baseline gets null metrics and the `reason` + `size differs from baseline: WxH vs WxH`, never a resize; a cell past an end says + `past end (N frames)`. Errors: 400 bad request, out-of-bounds crop, every moment + past every end, or not an image or video; 404 unknown name; 413 over the decode + pixel limit; 422 undecodable file - `POST /api/gallery/archive` — `{"names": [...]}` (1-1000) bundles a multi-file selection into one zip, named by each file's gallery-relative path so output subfolders survive. A browser cannot zip on its own and @@ -573,6 +666,21 @@ The editor's forms come from these; they are just as usable from scripts: `exports/` sits beside the workspace's own folders, holding one directory per exported job. It is a reserved name: no workspace can be called `exports`, and the folder is never listed as one. +- `POST /api/workspaces/{destination}/copy` (`{"source": ..., "runs": [...], + "assets": [...]}`) — copy runs and assets from one workspace into another. + A `runs` entry is `/`, or a bare `` for all its + runs; an `assets` entry is a library-relative path as after `asset:`, or a + folder (`proj/`) for every file under it. Naming neither, a source equal to + the destination, a path that escapes or reaches a symlink, or a sidecar + named directly is a 400, an unknown workspace, run or asset a 404, and a + server without a workspace root, a destination that is not writable, a run + whose job is still queued or running, or a different file already at an + asset's path a 409 (`detail.conflicts` names every one). All or + nothing. The reply is `{source, destination, copied: {runs, assets}, + already_present, renumbered, warnings: {unresolved_output_refs, + absolute_paths, no_jobs}, bytes_linked, bytes_copied}`; `renumbered` maps + `"/"` to `{"from": 3, "to": 7}`. What is copied and how is + in [WORKSPACES.md](WORKSPACES.md#copying-between-workspaces). - `GET /api/assets` — the asset library: input media, each with the `asset:` reference a workflow carries rather than a path, since a path only means something on the server's own machine. Empty rather than an @@ -707,8 +815,8 @@ The editor's forms come from these; they are just as usable from scripts: machine answered. `current_job` is the longest-running job and `worker_alive` is true if any worker is alive; `workers` has one entry per card (one with a single device): `{device, name, vram_gb, current_job, alive, - host_memory_rss_mb}` - `device` is the ordinal (`cuda:1`) and `name` the - GPU's own name, the two halves of a job's `device` - the last absent when the worker process isn't running + host_memory_rss_mb, pid, oom_score_adj}` - `device` is the ordinal (`cuda:1`) and `name` the + GPU's own name, the two halves of a job's `device` - the last three absent when the worker process isn't running (`oom_score_adj` is read from `/proc` at call time, so it shows the kernel's value: 1000 for the latest dispatch's worker in a pool, the spawn-time value - the server's own - for the others) - `POST /api/memory/clear` (#221) — drops every loaded pipeline and the step cache on each idle card (MCP `clear_memory`), and returns the memory reading taken right after. A card running a job is left alone: `workers` @@ -728,6 +836,60 @@ The editor's forms come from these; they are just as usable from scripts: without shelling in; the machine's non-loopback `addresses`; a client composes its URLs from an address, the port and the MCP path +## Notes + +A note is optional plain text on a run, on a file in a run, on an asset or +on a workspace, saying what it is for. At most 4,000 characters (longer is +a 400); `null` or `""` clears it. Only the server writes notes: a run's and +its files' live in `notes.json` in the run directory, an asset's in +`notes.json` in its library directory, and a workspace's in `workspace.json` +at its root (`dw/notes.py`). + +`PUT /api/notes?workspace=` with `{"target": ..., "note": ...}` sets or +clears one. `target` is one of: + +- a gallery file name (`output:` accepted) - that file's own note. A + flat-layout file has no run directory and answers 409 `notes need a run + directory`; +- a run directory, `/` - the run note; +- `asset:` - resolved down the asset search path the way the asset + delete resolves it, a read-only examples tree answering 403; +- `workspace:`, `workspace:default` for the root's own (409 on a + server with no workspace root). + +The answer is `{target, note, note_source}`: the target's effective note +after the write, and whether it is the file's own (`"file"`) or its run's +(`"run"`); `note_source` is null for a workspace and when there is no note. +A file's **effective note** is its own, else its run's. An unknown target or +one that would leave its root is a 404 (a name a workspace cannot take, or +an invalid asset name, a 400), and a write that fails a 500. Notes are free +metadata and need no `acknowledged_cost`. + +Notes are set at creation too: + +- `POST /api/jobs` takes `note`. It is kept on the job (`note` in every job + summary and detail, and in history) and written as the run note when the + worker reports the run directory (`run_start`); a rerun carries it + forward. +- `POST /api/uploads?note=` keeps it beside the upload. An upload over an + existing name takes its own note or none, never the old file's. +- `POST /api/assets/keep` takes `note`; without one the kept asset takes the + source file's effective note. + +They are read back on every listing: a gallery entry's `note` is the +effective one; the metadata route carries `note`, `note_source` and the +run's `run_note`; an asset entry and a workspace entry carry `note`. Each is +absent when unset. Deleting an output drops its note and its star; deleting +an asset drops its note and its kept provenance and shots, so a later file +under the same name inherits none of them. + +`PUT /api/stars` with `{"name": , "starred": true|false}` +stars or unstars one output file - the "keep" mark for culling a run. A star +is stored in the run's `notes.json` (`"starred": [...]`) beside its notes, and +the gallery listing carries `"starred": true` on a starred file. Only a file +in a run directory can be starred: a flat-layout file has no run directory +and answers 409, and a run, asset or workspace target is a 400. + ## Security model The server is built to serve **your own GPU to your own browser**, not the diff --git a/docs/TASKS.md b/docs/TASKS.md index 6bcf24af..974c80d3 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -3007,7 +3007,7 @@ The step's `result.content_type` must be `"application/json"`; read the answer w | Argument | Required | Description | | -------- | -------- | ----------- | | `audio` | Yes | The take - a path, `asset:`/`output:` reference, or an earlier step's audio or video (its soundtrack is taken) | -| `lines` | Yes | The expected lines in order: a list of strings or `{text, shot}` objects, `shot` naming the take's shot the line belongs in. Markup is stripped (below). `[]` means no speech is expected | +| `lines` | Yes | The expected lines in order: a list of strings or `{text, shot}` objects, `shot` naming the take's shot the line belongs in; a shot name the map holds more than once (a cut joined from episodes sharing names) also needs `start_frame` on the line, `{text, shot, start_frame}`, and findings call it `name#start_frame`. Markup is stripped (below). `[]` means no speech is expected | | `shots` | No | The take's shot map, `[{name, start_frame, num_frames, start_sample, num_samples}]`. Default: the take's own `.shots` (an earlier step's video), else the run manifest or `keep_output` sidecar beside an `asset:`/`output:` file - resolved as the assessment probes resolve them (`assess.resolve_shots`). The argument wins over both | | `similarity` | No | Least similarity, 0..1, of a line's heard words to its expected words before it is a `line_mismatch` (default `0.85`) | | `model_name` | No | HuggingFace ID of a Whisper-class ASR model (default `openai/whisper-base`) | @@ -3023,8 +3023,10 @@ Markup stripped from an expected line: `[Language] ...`, ``, | `line_clipped_at_end` | A line's last heard word ends inside the final `CLIP_TAIL_SECONDS` of its shot (when the line has one) or of the file, and that tail is still above `GUARD_FLOOR_DBFS` | The tail's level in dBFS / `GUARD_FLOOR_DBFS` | `{line, seconds, word, shot}` - `shot` when the line has one | | `speech_where_silent` | `lines` is `[]` and words were heard above the floor and outside a repetition loop | Number of words heard / 0 | `{line: null, seconds, word}` - the first; `shot` when shots are known | | `speech_in_silent_shot` | Shots are known, at least one line names a shot, and words were heard above the floor and outside a repetition loop in a shot no line names. A word is in the shot its midpoint falls in; one a line in another shot matched (a word straddling the cut) is that line's, not counted | Number of words heard in the shot / 0 | `{line: null, seconds, word, shot}` - the shot's first counted word, `seconds` no earlier than the shot's start | +| `unscripted_speech` | Shots are known and a line names the shot; words were heard inside it that align to no line (a line spoken twice, an ad-lib) | Number of unmatched words in the shot / 0 | `{line: null, seconds, end, word, shot, repeats_line}` - the first such word; `repeats_line` is true when those words repeat or contain one of the shot's lines | +| `speech_not_transcribed` | Between two words of one matched line there is a gap of `UNHEARD_MIN_SECONDS` or more, or one of its words lasts `UNHEARD_MIN_SECONDS` + 2 `UNHEARD_MARGIN_SECONDS` or more, and the interior (trimmed by `UNHEARD_MARGIN_SECONDS` each side) holds a window above `UNHEARD_FLOOR_DBFS`: audible speech the ASR left out of its text, which the line's timing absorbed (#923) | The gap's length in seconds / `UNHEARD_FLOOR_DBFS` | `{line, seconds, end, shot}` - `shot` when the line has one | -Every finding is severity `warn`; `at.shot` is set whenever the shot is known. A rule that does not apply to the call is listed in `rules_skipped` with its reason: `speech_in_silent_shot` is skipped when no shots are known (`line_clipped_at_end` then looks at the file's end only), when the shots carry neither a sample span nor a frame span the take's frame rate can place, or when no line names a shot (no shot is known to be meant silent). +Every finding is severity `warn`; `at.shot` is set whenever the shot is known. A rule that does not apply to the call is listed in `rules_skipped` with its reason: `speech_in_silent_shot` and `unscripted_speech` are skipped when no shots are known (`line_clipped_at_end` then looks at the file's end only), when the shots carry neither a sample span nor a frame span the take's frame rate can place, or when no line names a shot (no shot is known to be meant silent). The thresholds are module constants in `dw/tasks/script_check.py`, deliberately not in `RULES` (`dw/assessment_rules.py`): every rule there must fire on a synthetic file that holds no script. The answer echoes them under `thresholds`. @@ -3032,6 +3034,7 @@ The thresholds are module constants in `dw/tasks/script_check.py`, deliberately | -------- | ----- | --------------------- | | `DEFAULT_SIMILARITY` | 0.85 | A line below this similarity is a `line_mismatch` (the `similarity` argument overrides it) | | `GUARD_FLOOR_DBFS` | -65 dBFS (= `DEAD_AIR_FLOOR_DBFS`, `shot_dead_air`'s floor) | A heard word whose loudest window is at or below it is discarded as unheard; also the level a clipped tail must exceed | +| `UNHEARD_MIN_SECONDS` / `UNHEARD_MARGIN_SECONDS` / `UNHEARD_FLOOR_DBFS` | 1.0 s / 0.25 s / -40 dBFS | A gap in a line's words (or a stretched word) this long, with a window above the floor in its margin-trimmed interior, is a `speech_not_transcribed` | | `GUARD_WINDOW_SECONDS` | 0.05 s (= `DEAD_AIR_WINDOW`, `shot_dead_air`'s window) | The window a heard word is measured in - its loudest one, so a span overhanging a pause does not average a real word away | | `CLIP_TAIL_SECONDS` | 0.25 s | The final stretch of the file a line's last word must end in to be `line_clipped_at_end` | | `REPEAT_RUN_MIN` | 6 | Heard words repeating the same unit this many times over, back to back, are a repetition loop: every chunk holding one is discarded (`reason: "repetition"`), however loud | diff --git a/docs/WORKFLOW_GUIDE.md b/docs/WORKFLOW_GUIDE.md index 94a43ab4..c69a7fe6 100644 --- a/docs/WORKFLOW_GUIDE.md +++ b/docs/WORKFLOW_GUIDE.md @@ -270,6 +270,15 @@ produces the shape it needs, and is about to write JSON. `get_schema` says what is *well-formed*; this section says what the engine *does* with a well-formed document, which is where a draft that validates still fails. +When exploring variations, pass `note` to `run_workflow` saying what each run +tests ("video_shift 5 against 3"), and call `set_note` to mark the keeper: the +note is shown with every file the run made. Each file carries a `role` +(`final` or `intermediate`) inferred from how a step's output is used - a file +a later run reads (`output:`, or an `asset:` kept from it) lists as +intermediate too; set `result.role` on a step to override it when the default +is wrong, and set `"role": "intermediate"` on a workflow whose deliverable is an +ingredient a later assembly consumes. + ### References An argument value is a reference when it begins with one of these prefixes. @@ -702,7 +711,9 @@ by the source, and `validate_workflow` names the entries to add or drop. a measured per-entry rate re-priced for your list; `catalog` is a measured total for a run whose lists are the ones it was measured with; `derived` is that total extrapolated over a list you changed the length - of - say it is an estimate; `inherited` is an inline workflow priced + of, or scaled along the steps, canvas or frames you moved off the + defaults it was measured at (`scaled` names each axis that moved) - say + it is an estimate; `inherited` is an inline workflow priced from this box's runs of the catalog template with the same pipeline (`inherited_from`) - say it is an approximation, and relay the warning if offload, quantization or frame count differ; `other_device` is a figure from another @@ -761,7 +772,7 @@ by the source, and `validate_workflow` names the entries to add or drop. arguments={"input_audio": "output:", "lines": [...]})` first (free; it takes an audio file or a video's muxed soundtrack directly, and `lines` is the script in order, H3 markup allowed, `{text, shot}` to name - a line's shot, `[]` for a take that should be silent), then + a line's shot (plus `start_frame` when the cut repeats that shot's name), `[]` for a take that should be silent), then `run_workflow(..., acknowledged_cost={"fingerprint": ..., "minutes": ..., "downloads": [...]})` bound to that plan with `wait_seconds=60`, then `get_output_text` on the result, and `delete_output(job_id=...)` the @@ -971,7 +982,12 @@ longer than that is truncated with an ellipsis in every listing. Declare `shape`, `traits` or `summary` at the top level only when derivation gets it wrong; a declaration that merely repeats the derivation is noise that rots when the rules change, and the repo's catalog tests refuse it. `cost` is -never derived — leave it absent until a run has been measured. That makes it +never derived — leave it absent until a run has been measured. A `cost` entry +is measured at the workflow's variable defaults: a run that moves +`num_inference_steps`, `width`/`height` or `num_frames` (per entry in a list, +summed) is quoted scaled in proportion to them, so a figure measured at some +other size or step count is re-priced to the defaults before it is written +(dialogue-short's 42 min at 1344x768 is 21.2 at its 960x544). That makes it the catalog's verified marker as well: an entry carrying `cost` has been run to completion on the device it names, and one without has only been authored — its description may still say what it has not been able to check (VRAM at @@ -986,7 +1002,12 @@ reports the result as `observed` beside the curated `cost`, so a 345-frame run never informs a 124-frame figure and a list driver buckets on its length. Declaring none is not neutral: the figure then falls back to runs that overrode nothing at all, which most real runs do, so a measured workflow with -no drivers keeps answering "unknown". Each name must be a variable the +no drivers keeps answering "unknown". A driver the estimate cannot scale +along - anything but steps, width, height and frames, such as a weights +dtype, a LoRA name or an audio duration - still splits the buckets (a text +value buckets by its exact value). When a run moves one off its default, the +curated figure is kept but carries `low_confidence: true`, and `differs` +names the drivers, since it was measured at the default. Each name must be a variable the workflow declares — `tests/test_observed_cost.py` sweeps the catalog for one that is not, since a driver bucketing on nothing looks exactly like a driver that works. @@ -1067,6 +1088,23 @@ story wanted is the cut working, and a level step into a quieter scene can be the scene. Tell the person what you confirmed, not what the probe reported. +**Comparing outputs.** `compare_outputs(names)` (`POST /api/gallery/compare`) +lays 2-8 outputs (variations, reruns, the same shot with and without a speed +setting) in one grid, a row per output and a column per moment, every output +read at the same frame number, and measures each against a `baseline` (default +the first). Per non-baseline output it reports PSNR (dB, higher is closer; +`identical` when equal), SSIM (structure on luma, 1.0 identical) and mean +absolute difference (0-255, lower is closer), at source resolution, per column +and as mean/min/max. A shared `crop` is the region both shown and measured; one +box per name is display only and the metrics stay on the full frame; the reply's +`region` says which. Outputs of a different size get no numbers (never resized). +A bare PSNR means little alone; these bands, measured on full frames, are a +reading aid, not thresholds: about 44 dB is rerun noise (same settings, +nondeterministic kernels); 23-27 dB means the numerics changed (a different +cache, attention backend or quantization) but the shot is the same; about 19 dB +means it was re-staged (the composition changed). Look at the grid before +trusting a number. + **Remediation.** A *recut* reruns only the join over the shots the run already made: each entry in `videos` is `output:` + the run's `intermediate/` shot file. That is cheap, and generates nothing new. A @@ -1219,6 +1257,9 @@ For components the pipeline loads itself — which is all of a modular pipeline' `diffusion_decoder`, for example). - `attention_backend` — a persistent `set_attention_backend` on one component, which a compiled component needs (the pipeline-level `attention_backend` applies per call). + Every MiniMax-H3 template wires an `attention_backend` variable (default `null`) here, + on its H3 step's `transformer` or `transformer_ref`; see + [ACCELERATION.md](ACCELERATION.md) for `sage_hub` and what it needs. - `attn_processor_type` — the attention processor the component runs, constructed with no arguments and handed to `set_attn_processor`. The `unet` and `transformer` blocks cover those two; this covers any other component that carries attention (LTX-2.5's @@ -1516,10 +1557,18 @@ A `cache` block speeds up inference by skipping redundant computation: `cache` wraps diffusers' own cache hooks - `type` is one of `first_block`, `faster`, `mag`, `taylorseer` or `text_kv`, each with its own tuning fields (`threshold`, `num_inference_steps`, `max_skip_steps`, `retention_ratio`, `cache_interval`, -`max_order`, `mag_ratios`, `calibrate` — see [dw/workflow_schema.json](../dw/workflow_schema.json) for which +`max_order`, `mag_ratios`, `calibrate`, `is_guidance_distilled` — see [dw/workflow_schema.json](../dw/workflow_schema.json) for which fields apply to which type). See [workflows/templates/step-caching.json](../workflows/templates/step-caching.json). +`cache` can also be a whole-object `variable:` reference, which must resolve to a cache +object or to `null`. A variable resolving to `null` drops the key: no cache, and the only +true "off", since an installed `first_block` hook changes the output even when it skips +nothing. The substituted object is checked against the cache shape at validate, so +`{"type": "bogus"}` or a bare `"first_block"` is refused, naming the field. Every +MiniMax-H3 template exposes a `cache` variable (default `null`) on its H3 step: +`"arguments": {"cache": {"type": "first_block", "threshold": 0.05}}`. + ### Device and Dtype Device is auto-detected (CUDA > MPS > CPU). Dtype is set per-component: @@ -2609,6 +2658,26 @@ is pinned to, not for every workspace that happens to hold a matching run. Deleting a workspace takes its cache entries with it, the same as deleting its `outputs/` directory would. +The step cache is also per card and per workflow identity. Identity is the stored +workflow's name, or an inline workflow's `id`; entries are keyed on the definition `id` +and step name (`StepCache.get` in `dw/step_cache.py`), and an inline job's identity is +`("inline", id)` (`workflow_identity` in `dw/worker_protocol.py`). Switching identity on a +card empties that card's cache (the worker's `_cleanup_all` calls `step_cache.clear()`), +so: + +- running any other workflow on the card in between starts the next run cold: A, then B, + then A runs the second A cold. This is the common case. On a multi-card server + warm-identity routing (`choose_slot` in `dw/server/pool.py`) mostly avoids it unless + that card is busy; on a single card it happens every time. +- a renamed copy, a new inline `id`, or switching between a stored workflow and its + inline copy also starts cold. + +To reuse cached upstream steps, rerun the same workflow (same name, or the same inline +`id`), for example with `rerun_job` or changed arguments. Only arguments that reach +*later* steps leave the earlier steps cached: changing an argument the expensive first +step reads re-runs that step. To check before running, `validate_workflow`'s +`plan.cached_steps` says how many steps would be reused. + ## Type System Dynamic type conversion applies to certain values: diff --git a/docs/WORKSPACES.md b/docs/WORKSPACES.md index f7964b40..4bfdf31d 100644 --- a/docs/WORKSPACES.md +++ b/docs/WORKSPACES.md @@ -212,6 +212,11 @@ run fails part way, since the files it did write are on disk either way. A sub-workflow is part of its parent's run: it writes into the same directory and rolls up into the same manifest. +A pipeline step that ran also carries `timing` in its `manifest.json` entry +(and in the same entry of the job's manifest from the API or MCP `get_job`): +load, encode, denoise, decode and save seconds. A reused, task or composed +step has none. See [Step timing](#step-timing). + An unchanged rerun still reuses the step cache: it writes no new files and its manifest reports the earlier run's, marked `"reused": true`. The cache is validated against the output *root* a run writes into, which is the pinned @@ -230,11 +235,65 @@ A later workflow names what an earlier run made with an `output:` reference — longer needs files copied back by hand. See [Output References](WORKFLOW_GUIDE.md#output-references). +Each step's entry in `manifest.json` records its `role`, `final` or +`intermediate` (and `role_declared` when the step's `result.role` stated it), +from that one run's data flow. The gallery reads across runs as well: a file a +later run in the same workspace read - by an `output:` reference, or by an +`asset:` that `keep_output` kept from it - lists as `intermediate`, so a +project's shots, narration and score stop reading as finals once the assembly +that cuts them has run. Nothing is rewritten; a role `result.role` declared +`final` stays final, and a run from before roles stays unknown. See +`dw/roles.py`. + To keep the previous layout — everything at the output root, with only a `workflows/`-mirroring subfolder — use `--output-layout flat`, `DW_OUTPUT_LAYOUT=flat`, or `"output_layout": "flat"` in settings. Scripts that glob the output directory are the reason to. +### Step timing + +A manifest entry for a pipeline step that ran - in `GET /api/jobs/{id}`, in +MCP `get_job`, and on that step's `step_end` event - carries `timing`: where +the step's time went. It is measured in the worker on its monotonic clock, not +taken from event timestamps, because the job keeps only the last 200 events and +the load and encode events are gone from a long job. Every `_s` is seconds to +0.1. A field whose mark never happened is null, never guessed. + +| Field | Meaning | +|---|---| +| `load_s` | Loading or fetching the step's pipeline, cached or not | +| `load_cached` | `true` if the pipeline came from the worker's cache, `false` if any part loaded fresh, null if the step loaded nothing | +| `encode_s` | Call start to the first denoiser forward (or a modular pipeline's denoise bar opening): prompt and reference encoding and preparation | +| `denoise_s` | First denoiser forward to the last pass's end. The whole call when the pipeline reports no passes; `encode_s` and `decode_s` are then null | +| `passes` | Denoise passes the pipeline actually ran: one per iteration of its denoise loop, so one per scheduler timestep. This is not always `num_inference_steps`: SD 1.5's default PNDM scheduler runs 21 passes at 20 steps (N+1 from 2 steps up). `denoise_s` ≈ `first_pass_s + (passes − 1) × pass_s` | +| `first_pass_s` | The first pass, which carries any compile warm-up | +| `pass_s` | Median of passes 2..n; null with fewer than two passes | +| `compiled` | The denoiser runs through `torch.compile` | +| `decode_s` | Last pass's end to the call's return: VAE decode and post-processing | +| `save_s` | Writing the step's files | +| `segments` | Pipeline calls summed into this timing. A chain step sums its segments into one timing with `segments: N`; a step that iterates counts each call. 1 for a plain step | + +A step that set a `cache` also carries `timing.cache`: `{type, threshold, forwards, computed, skipped, hit_rate}`. `forwards` counts denoiser forwards, not steps - classifier-free guidance runs two per step (cond and uncond), so it is a whole multiple of `passes` - and `computed + skipped == forwards`; `hit_rate` is `skipped / forwards` to 3 places. A chain sums the counts over its segments. Counted for `first_block` and `mag` (`mag` also reports `num_inference_steps`, `max_skip_steps`, `retention_ratio`, `calibrate`); `taylorseer` (with `cache_interval`, `max_order`), `faster` and `text_kv` keep no per-forward decision in diffusers, so they report `type` and their config keys with `threshold` and every count null. A step with no cache has no `cache` key. If the installed diffusers moved the hook state dw reads, the counts are null and one `warning` event with `kind: "cache_stats_unavailable"` is emitted; the run never fails for it. The same fields arrive live as a `cache_stats` event ([SERVER.md](SERVER.md#progress-events)). See [Reading the skip counts](ACCELERATION.md#reading-the-skip-counts). + +A step served from the step cache (`reused: true`) has no `timing`, nor does a +task step or a composed (sub-workflow) step. A failed or cancelled step has no +manifest entry, so no timing. Job summaries and history are unchanged. + +### Host-memory history + +Each finished job's history row records a `memory_key`: a digest of the +run's pipeline identities (`component_type`, `model_name`, `workflow` of +every pipeline step) and every `quantization_config`, `offload` and +`group_offload` value, with the caller's arguments resolved. The +workflow's name is not in it, so a renamed or inline copy of a template +shares the template's history, and a different `weights_dtype` or offload +is a different key. From it the server projects a run's host-RAM peak: +`observed` (the largest job peak RSS among this machine's successful runs +with the same key), else `fit` (the list-run projection from the +workflow's own history), else no figure. Rows recorded before the column +existed have no key and count as no data. Validate reports it as +`plan.host_memory`. + ## Several workspaces on one server Everything above describes one workspace. `dw.serve` goes one step @@ -324,6 +383,125 @@ A workspace is a **namespace, not a security boundary**. The API token is all-or-nothing: anything that can reach the server can name any workspace on it. Use them to keep work apart, not to keep it private. +## Copying between workspaces + +A run is self-contained on disk, so copying its folder into another +workspace's `outputs/` is what makes it resolvable there. `copy_to_workspace` +(MCP) and `POST /api/workspaces/{destination}/copy` ([SERVER.md](SERVER.md)) +do that for runs, named `/` or, for all of a workflow's runs, +a bare ``, and for assets, named as after `asset:` (`cast/iris.png`, +or a folder such as `proj/` for every file under it). An asset keeps its +library-relative path, so `uploads/` stays `uploads/` and an `asset:` reference +means the same file in both workspaces. Its note and its kept-shots entry (the +folder's `notes.json` and `manifest.json`, including where it was kept from) +are merged into the destination folder's own sidecars, which are written +fresh: naming a sidecar directly is a 400, since it travels with its files. + +Media is hard-linked, so a copy costs almost no disk, falling back to a real +copy across devices. JSON sidecars (the manifest, `workflow.json`, notes, +any `.json`) are always copied, never linked: they are edited in place, and a +link would carry an edit in one workspace into the other. Nothing follows or +creates a symlink. + +The copy is all or nothing: every entry is checked before anything is written. +A run whose run id the destination already has is the same run, so it is +skipped and listed as `already_present`, and repeating a copy changes nothing. +A run whose job is still queued or running is refused, since its manifest is +not final. An asset the destination already holds with the same bytes is +skipped as `already_present` too; a different file at that path is a conflict, +never replaced. Every conflict, runs and assets alike, is named in one 409 +before anything is written. + +Because a copied asset is a hard link, nothing edits media in place: an upload +over an existing name writes a new file and renames it into place, and +`keep_output` and `delete_asset` unlink first, so an edit in one workspace +never reaches the other. + +A run keeps its `vN` unless the destination already uses that number for the +workflow. Then this request's runs of that workflow are renumbered after the +destination's highest, in run-id order, under the run lock so a concurrent run +cannot take the same number. `renumbered` reports each `from` and `to`. Pinned +`output:` references name run ids, so none of them break. + +The reply carries warnings, not refusals: `unresolved_output_refs` (a run's +`output:` input that does not resolve in the destination), `absolute_paths` +(manifest paths that still point into the source workspace, reported relative +to the source workspace's root) and `no_jobs`. A copied run has no job record, so it appears in +the gallery and resolves as an `output:` reference, but not in `list_jobs`. + +A move is a copy, then a delete of the source runs (`delete_output`, or +`DELETE` on the gallery runs) and assets (`delete_asset`); there is no move that could lose a run halfway. + +## Running a sweep + +A sweep answers one quality or speed question by rendering the same request +under several configurations, changing one factor at a time, and keeping the +results together. There is no sweep tool: every step is one call to a tool +dw already has. + +1. **One workspace per experiment.** `create_workspace` a workspace for it, + then `use_workspace`. Put the question and the factor list in its + workspace note, so the experiment explains itself to the next session: + `set_note("workspace:", "Q: does steps change the face? factors: steps 4/8/12")`. + +2. **A base workflow with every factor as a variable.** If a catalog + template already exposes the factor as a variable (`get_workflow(name, + variables_only=true)`), run the template. Otherwise `get_workflow` it, + add a variable for each factor, put `variable:` where the value + was, and `save_workflow` the copy into the experiment's workspace. + `text-to-image` fixes `num_inference_steps` at 25, so a steps sweep saves a copy + with `"steps": 25` under `variables` and `"num_inference_steps": + "variable:steps"` in the step. A factor that is a dtype or a quantization + name keeps its braces in the default, `"{nf4}"`, so it reaches the + pipeline as a string ([Types and escaping](WORKFLOW_GUIDE.md#types-and-escaping)). + Keep the template's `cost` block in the copy, or `plan.estimate` has no + basis to quote from. + +3. **Pin the seed, change one factor per variant.** Every variant passes the + same `seed` in `arguments`, and differs from the base in one factor only. + A factor can change the cost, so `validate_workflow` each variant with its + `arguments`, quote each one's `plan.estimate`, and get one go-ahead + for the whole list. + +4. **One `run_workflow` per variant, with a `note`.** The note's first line + is the overrides, `steps=8` or `steps=20 attention=sage`, so a listing + that clips notes still shows which variant is which. Call `wait_for_job` + until the variant finishes before submitting the next. The server holds a + second large variant until the host can hold it beside the first (see + [SERVER.md](SERVER.md)), so waiting is for ordering, not for memory. + +5. **Only the latest run of a step is cached.** The step cache holds one + entry per workflow step: its most recent run. A variant whose step and + seed match that run, such as the same variant submitted twice in a row, + is served from the cache: its manifest marks the step `"reused": true` + and lists that run's files ([Runs](#runs)). Going back to an earlier + variant after another one has run renders it again. Each card's worker + keeps its own cache, and a server restart empties them. Either way a + re-render is expected, not a failed variant. + +6. **The results table.** `list_jobs(workspace=)` gives each variant's + job id, status and note. `get_job` on each gives its arguments, step + `timing` ([Step timing](#step-timing)) and files. + +7. **Assess each output with `set_note`.** Put the note on the output file + (its gallery name, as `list_gallery` gives it). The first line is the + score and who judged it, then one `key: value` line per facet: + + ``` + score 4/5 · judged_by claude-opus-5-5 + faces: sharp, eyes aligned + staging: matches the brief + defects: none + ``` + + `get_gallery_metadata` returns the whole note. `list_gallery` and + `list_jobs` clip a note to its first 200 characters, so keep the + first line short: it is what a listing shows. + +8. **Keep the winner.** `keep_output` the winning file under a stable + `asset:` name, and put the conclusion (the answer, and the variant that + gave it) in the workspace note. + ## Where this is going The resolver, the workflow search path with writes confined to the writable diff --git a/docs/proposals/complete/acceleration-warnings-complete.md b/docs/proposals/complete/acceleration-warnings-complete.md new file mode 100644 index 00000000..b6435857 --- /dev/null +++ b/docs/proposals/complete/acceleration-warnings-complete.md @@ -0,0 +1,128 @@ +# Validate warnings for acceleration settings that can't pay off (#868) + +Written by model `claude-opus-5-5` via provider `anthropic` (close-out, +2026-10-14). Don approved plan v2 on 2026-10-14. Stage A, #887, shipped the +same day at `develop` @ `c96e9531` and was verified on mini-ai (mps). + +## The idea + +Don's H3 quality matrix on lem (2x RTX 3090 24 GB, workspace `h3-quality`) +had three acceleration settings that each took a render to show they were +wrong, though all three could be known at validate time: + +1. **P2:** a `first_block` cache at 0.05 on 12 base steps skipped nothing + (12.79 vs 12.89 min) and still changed the output (PSNR 25.6 dB). +2. **P5:** `taylorseer` (`cache_interval` 3, `max_order` 1, 30 steps, + 1344x768, 4 refs) passed validate and ran out of memory 2 min in (job + a28aa8ffb931). +3. **P3:** `compile: {repeated_blocks: true}` on a group-offloaded int8 + transformer added 289 s on the first pass and saved about 0.6 s per pass + after that. The worker released the compiled pipeline on every workflow + change, so the cost was paid again each time. + +## Verdict and scope + +**Build smaller, in one stage.** #1 and #2 became validate warnings. The +same check picked up two related rules: `fullgraph` with a cache +(`ACCELERATION.md` stated it and nothing checked it) and a CUDA-only +attention backend on a server that isn't CUDA (#865's agreed follow-up). +#3 is docs only. + +Corrections the plan made to the proposal: +- **No turbo-LoRA clause.** Every turbo schedule in the catalog is 9 steps + or fewer, so `steps < 20` already covers it. Detecting turbo would mean + matching filenames in the engine, which CLAUDE.md forbids. +- **Threshold caches only** (`first_block`, `mag`). `taylorseer` and + `faster` skip on a fixed interval, and `text_kv` skips no steps. +- **TaylorSeer is not charged in the estimate.** `vram_estimate` errors + are hard and `required_vram_gb` drives dispatch, so charging the state + would turn a warning into a refusal. The engine also holds no + architecture data to size it with. So #2 is a separate warning, and + neither the code nor the docs give a state size. + +## Decisions (Don's review of plan v1, 2026-10-14) + +- **Q1.** #3 is docs only, no warning. +- **Q2.** `mag` is included in the short-schedule rule. +- **Q3.** The `fullgraph` + cache warning is included. +- **Q4.** The TaylorSeer warning always fires when TaylorSeer is set. +- **Changes:** add `attention_backend_unavailable`, keyed on a named set of + backends rather than the `_hub` suffix. Shrink the #3 docs to one fact + (compile is paid again when the worker releases the pipeline). Give no + TaylorSeer state size. Keep 20 steps as the single threshold: nothing + between 12 and 30 steps had been measured, and #875 measures 20. + +## What was built + +### Stage A, #887: `acceleration_warnings` + +- **Engine:** `dw/acceleration_warnings.py`. It is registered as the last + entry of `WARNING_CHECKS` in `dw/validation.py`, and it resolves values + through variables and arguments. Strings use the existing + `: : ` form, so no wire shape changed. The codes: + - `cache_short_schedule`: a `first_block`/`mag` cache where the resolved + `num_inference_steps` is below `CACHE_MIN_USEFUL_STEPS` (20). + - `taylorseer_state_unestimated`: always fires on `type: taylorseer`, and + names `max_order`. It adds "estimated at X of Y GB" only when the + workflow declares a `vram_estimate` and the device capacity is known. + - `compile_fullgraph_with_cache`: a component with `compile.fullgraph: + true` in a pipeline that has a cache. + - `attention_backend_unavailable`: a backend in + `CUDA_ONLY_ATTENTION_BACKENDS` (the flash, flash-3, sage and xformers + families plus `aiter_fa2_hub`, Hub kernels included), at pipeline or component + level, on a known device that isn't CUDA. The device is compared with + `get_device_type()`. + - A value that can't be resolved, or an unknown device, keeps its + warning silent. No warning ever refuses. +- **Save path:** `_write_validated` (`dw/server/routes/library.py`) runs + only selected warning checks, not the whole registry. It now also runs + `acceleration_warnings`, so `save_workflow` returns the same codes as + `validate_workflow`. This was the tester's bounce (see below). +- **Map:** `docs/ARCHITECTURE.md`, *Validation, plan and cost*, has an + *Acceleration warnings* row. +- **Docs:** `docs/ACCELERATION.md` names all four codes, gives the P5 OOM + configuration with no state size, says the compile cost is paid again + whenever the worker releases the pipeline, covers `fullgraph` with a + cache, and has the backend warning in the MPS notes. It also removes the + old line saying that validation doesn't check the backend. +- **Tests:** `tests/test_validation.py::TestAccelerationWarnings` covers + every boundary through the real `validate` path, checks that no catalog + workflow emits the three device-independent codes, and checks that none + of the codes refuses. + `tests/test_validate_arguments.py::test_save_workflow_carries_the_acceleration_warnings` + covers the save route. + +Deviations from the plan, all recorded on #887: the check lives in its own +module instead of inline in `validation.py`; `aiter_fa2_hub` is in the +CUDA-only set; a `for_each` step's members share one path, so each code +appears once per path. + +## Verified + +On mini-ai (mps), over MCP only: C-F446 to C-F453. The case that +`attention_backend_unavailable` stays silent on CUDA needs a CUDA server. +On mps it is covered only by unit tests with a patched device. + +## Bounces per stage + +- **Stage A, #887: 2.** + - **Architecture review (1):** the new module had no row in + `docs/ARCHITECTURE.md`. The row was added (`547a82c6`). + - **Tester (1):** C-F451 step 2. `save_workflow` didn't return the + acceleration warnings, because the plan wrongly assumed save runs the + whole `WARNING_CHECKS` registry. Fixed in `bf448a2a` (`c96e9531`). + +The stage cost isn't recorded, because the stage comments give no `usage:` +figures. The plan estimated $4-5. + +## Deferred + +- **#3 as a warning** (Q1). It is docs only. Reconsider if an agent keeps + paying the compile cost across workflow changes. +- **Sizing TaylorSeer state** in the estimate. The engine has no + architecture data, and the estimate drives dispatch and refusal. +- **A `mag.num_inference_steps` / pipeline-steps mismatch warning.** The + schema says the two must match, and nothing checks it. It hasn't yet been + shown to cost a render. Don decides whether it becomes its own issue. +- **The 20-step threshold** is a heuristic. #875's threshold experiments + may move it, and it is a single constant. diff --git a/docs/proposals/complete/catalog-adapter-check-complete.md b/docs/proposals/complete/catalog-adapter-check-complete.md new file mode 100644 index 00000000..7f832ee7 --- /dev/null +++ b/docs/proposals/complete/catalog-adapter-check-complete.md @@ -0,0 +1,133 @@ +# Catalog-backed H3 adapter partition check (#870) + +Written by model `claude-opus-5-5` via provider `anthropic` (close-out, +2026-10-14). Plan v2 approved by Don as "build smaller" on 2026-10-14; +one stage, #888 (A), verified on lem. + +## The report + +On the H3 quality matrix (lem, workspace `h3-quality`), the fal Realism +People LoRA (`fal/MiniMax-H3-Realism-People-LoRA`, +`h3-realism-people-t2v-i2v-r2v.safetensors`) was put on a Ref2VA +(`transformer_ref`) step to fix faces. It was inert and cost a full render +to find out. `dw/adapter_compatibility.py` decided from the filename tokens +`ref2v` / `fl2v`; this name has neither (its `r2v` even suggests reference +support), so validate only warned "names neither". The file's safetensors +`__metadata__` says `base_model: minimax-h3-fl2va`. On FL2VA/I2VA it works +(R3: 4/5 faces). + +The ask was to read that header during validation and prefer `base_model` +over the filename. + +## Decisions + +- **Plan v1: build smaller.** The catalog already said it: + `loras/minimax-h3/realism-people.json` had `"workflow": "t2va"`, + `"status": "proven"`. The check just never asked the catalog. Wire the + catalog in; cut the header read, because: + 1. it is I/O on the hottest validation path (every validate, admit, + `for_each` member and composed child), which is free and offline today; + 2. a local-path read during validate is a directory-existence oracle + (`plan.py` refuses exactly that); + 3. a ranged Hub GET breaks "`lora_hub.py` is the only place dw searches + the Hub"; + 4. `__metadata__` is uploader-written: it shouldn't turn a warning into a + refusal; + 5. only one vendor key (`base_model`) was evidence. +- **Q1:** a catalog entry that disagrees with the filename token warns, + never refuses. **Q2:** the revision is ignored. **Q3:** not needed (if a + `save_lora` pre-fill comes back, trust `base_model` only). +- **Build stop on v1 (#888, 2026-10-14):** v1 assumed the catalog was + layered per workspace. It isn't: `server_lora_library(state)` is one + library shared by every workspace, so one agent-written trial entry would + refuse a file server-wide. +- **Don's ruling (a) → plan v2:** keep the one shared catalog, add no + layering. **Only a `proven` entry can refuse**; a `trial` or `rejected` + entry that conflicts with the step's partition warns, naming the entry + and its status. No new evidence requirement on `proven`. Scope narrower + than v1. +- **Non-goals held:** no header reads, no Hub request from validation, no + non-H3 base-model checks, no change to the filename rule, no per-workspace + catalogs, no MCP/REST/syntax change. + +## What was built + +**Stage A, #888** (server + data + docs), `feat/870-A-catalog-adapter-check` +(27674293, 687e288a), merged to `develop` as `9a210585`: + +- **`adapter_compatibility.catalog_index(entries)`**: `(model_name, + weight_name) -> (entry, workflows, status)`, revision ignored. With no + index the result is the old rule. +- **Partition, not workflow** (`_catalog_partition`): all of + `{t2va, fl2va}` is `transformer`, `ref2va` is `transformer_ref`, a mix + decides nothing. +- **Status order:** a `proven` conflict is an error naming the entry + ("catalog entry 'minimax-h3/realism-people' (proven) lists it for 't2va, + fl2va', which denoises against 'transformer'…"); a `trial`/`rejected` + conflict is a warning naming the entry and status; an agreeing entry + clears "names neither" but a `trial` one never lifts a filename-token + error; a `proven` entry that disagrees with a token is a warning naming + both. +- **Callers:** `deps.lora_index(state)` is built from the shared + `catalog_entries(state)` (moved from `routes/loras.py` to `deps.py`, so + `list_loras` and validation read one function), cached on + `state.lora_indexes` keyed on the listing's (name, path, mtime). Passed by + `admit_for` (`/api/validate`, submit, rerun, enhance-admit) and the save + path in `routes/library.py`. The CLI `dw.validate`, the worker and + `warn_adapters` pass none: the server is stricter than the worker, never + looser. +- **Data:** `realism-people.json` `workflow` is `["t2va","fl2va"]`, + `use_when` says "not reference-to-video", new `evidence` note (#870); it + stays `proven`. +- **Docs:** the module docstring, `docs/ARCHITECTURE.md`, `docs/SERVER.md`, + `docs/LORAS.md` (a `proven` entry's `workflow` is enforced on the server; + a `trial` one only warns; promoting is what makes it refuse), and the + `minimax-h3` skill's `checkpoints-and-loras.md` ("validate refuses it on + `ref2va`"). +- **Tests:** `tests/test_h3_adapters.py` (`test_a_proven_*`, + `test_a_weaker_*`, `test_no_index_is_todays_rule`, null/mixed fallback, + `arguments.` path, `test_one_build_per_listing_for_every_member`, + `test_a_trial_entry_warns_and_promoting_it_refuses`); the + `workflows/**/*.json` sweep also runs with the packaged catalog. + +**Deviations from the plan**, both accepted at verify: +- Two entries covering the same file: the index keeps the stronger status + (proven > trial > rejected). +- A `trial` conflict on a no-token file gives the entry's warning *in place + of* "names neither", not both. + +## Bounces per stage + +| Stage | Bounces | Cause | +|---|---|---| +| A, #888 | 0 (1 build stop) | The build stopped on v1's per-workspace catalog assumption; re-planned as v2 (Don's ruling (a)), re-decomposed and re-specced. Verified first time on v2. | + +No stage comment names `usage:` figures, so per-stage cost is not recorded. +The plan estimated ~$3-5. + +## Verification notes + +- Verified over MCP on lem `develop` @ f73088a5: M-F142-M-F145, M-F147, + M-F152 and M-F153 pass. M-F142(c): `run_workflow` with Realism on + `reference-to-video` is refused at `arguments.lora_weight_name` with no + job queued. +- M-F144(b): the `ref2v` turbo on `fl2va` is refused by the proven catalog + message (`minimax-h3/turbo-reference (proven)`), not the filename message; + v2 lets a proven entry decide. +- M-F146 (v1 workspace layering) contradicts v2 and was not run; retiring it + is dkackman/harnest#122. +- mini-ai (MPS) was not checked, and its restart is Don's. + +## Deferred + +- **Reading safetensors headers**, reopened only by a second render lost to + a LoRA the catalog doesn't cover, or a run of uncatalogued H3 trials that + keep saying "names neither". It would go into `save_lora` / + `recommend_loras` (which already read Hub headers via `lora_hub.py`) to + *pre-fill* an entry's `workflow`, trusting `base_model` only; the + validator stays free. +- **Edges with no shipped entry** (a `proven` entry against its own token, a + mixed list, a `rejected` entry with a `workflow`): unit-tested only, since + probing them over MCP needs a `save_lora` write to the shared catalog. +- **CLI `dw.validate` stays on the filename rule** (documented; the safe + direction). diff --git a/docs/proposals/complete/compare-outputs-complete.md b/docs/proposals/complete/compare-outputs-complete.md new file mode 100644 index 00000000..49abc5dd --- /dev/null +++ b/docs/proposals/complete/compare-outputs-complete.md @@ -0,0 +1,131 @@ +# `compare_outputs`: aligned grids and diff metrics across runs (#871) + +Written by model `claude-opus-5-5` via provider `anthropic` (close-out session, +2026-10-14). The idea came from the H3 quality matrix on lem (workspace +`h3-quality`). Every judgement there needed local tooling: ffmpeg contact +sheets, `tools/facegrid2.py` face crops and `ffmpeg psnr` against a baseline. +A text-only MCP client can do none of that. Plan v1 was designed on #871. +Don's review made v2, which says metrics always compare the same pixels and +moves the plugin pointer out of the full `SKILL.md`. Don approved v2. It was +built as one stage, #889. The plan comment on #871 stays the full design; +this is the record of what shipped. + +## The problem + +Judging "did this knob change the picture, and how much" across runs took +N `get_output_frames` calls, aligned by eye, and nothing in dw computed PSNR +or SSIM. The numeric half could not be done from a client without ffmpeg. + +## Verdict: build smaller + +Cut from the proposal, with what would bring each back: +- **`crops: "faces"`** (stage B, deferred and not filed). YuNet is already in + `dw/tasks/face_track.py`, so it needs no new dependency. *Trigger:* a field + report showing per-output coordinates are still the bottleneck. *To settle + before filing:* where the YuNet weights are downloaded. `face_track` + downloads them in the worker; stage B would otherwise fetch them in the + server process. +- **`same_staging`.** dw reports findings, not verdicts. The matrix's bands + are in the guide as a reading aid instead: ~44 dB is rerun noise, 23–27 dB + is changed numerics, ~19 dB is a re-staged shot. *Trigger:* two or more + comparison sessions read the bands the same way and want it as a field. +- **`layout`.** A sheet and a grid are the same output with more or fewer + columns. +- **Saving the grid.** dw has no run-less output. The grid comes back + inline, like `get_output_frames`. *Trigger:* someone asks to keep one, as a + `keep_as` that writes to the asset library. + +Non-goals stayed out: `latest`/`vN` expansion in names, audio comparison, +and resizing outputs to make different sizes comparable. + +## Plan history + +- **v1:** per-output crops computed metrics whenever the boxes were the same + size, and a pointer line went in `minimax-h3/SKILL.md`. +- **v2 (approved):** Don's review settled that **metrics always compare the + same pixels**. One shared crop is both what is shown and what is measured. + Per-output crops are display only, and metrics then cover the full frame. + The plugin pointer moved to `minimax-h3/references/quality-and-speed.md`, + because `SKILL.md` was at 12,282 of 12,288 bytes. The route became a sync + `def` so decode and SSIM run in FastAPI's threadpool. Q1: a new tool. Q2: + per-output crops, with the rule above. Q3: raise `SURFACE_BUDGET` with a + dated ledger entry. + +## What was built + +### Stage A (#889): `compare_outputs` and `POST /api/gallery/compare` + +Commits f15ca771 (the route and `dw/media_compare.py`), 9c3c23a2 (the MCP +tool and tests), 74700668 (docs and the OpenAPI snapshot), 93d06dbe (map +rows and the single owners for the `at` and crop parses) and 99f04d67 +(negative moments refused, `max` in the reply). +- **Engine:** `dw/media_compare.py` owns `compare`, `psnr`, `ssim` (601 + luma, 11×11 Gaussian window, σ 1.5, numpy + `cv2.GaussianBlur`, no + `skimage`), `mean_abs_diff`, `check_request` and `crop_box`. The grid + reuses `dw/media_frames.py`'s compositor. It runs in the server process, + needs no GPU and queues nothing. +- **Reply:** one grid image, with rows for outputs in the given order + (labelled, `(baseline)` marked) and columns for moments (each cell stamped + `frame N`). It also has a text summary per non-baseline output, per column + and as mean/min/max, with `identical_columns` counted apart so ∞ doesn't + skew the mean. It names the `region` measured. +- **Nulls with a reason, never a resize:** `size differs from baseline: WxH + vs WxH`. A moment past an output's end gives a blank cell and `past end (N + frames)`. `fps_differs` flags clips compared by frame number at different + rates. +- **Shared owners:** `media_frames.moment_index` owns the `at` grammar and + `media_frames.parse_crop` owns the `[x, y, w, h]` parse. They are used by + both `get_output_frames` and `compare_outputs`, and each tool keeps its own + bounds policy. `docs/ARCHITECTURE.md` *Media and DSP* has an **Output + comparison** row and a **Frame moments and crops** row. +- **Surface:** the MCP tool count went from 64 to 65. `SURFACE_BUDGET` is + 14,990, with a dated ledger entry for the measured 14,989.25. +- **Docs:** `docs/MCP.md` (a tool row), `docs/SERVER.md` (the route), + `docs/WORKFLOW_GUIDE.md` *Assessing a run's output* → "Comparing outputs" + (the metrics, the region rule and the dB bands as a reading aid), and + `quality-and-speed.md` (one line). +- **Tests:** engine tests in `tests/test_media_compare.py`, route tests on + real PyAV-encoded clips and PNGs in `tests/test_compare_route.py`, and MCP + pins and summary tests in `tests/test_mcp_server.py`. + +**Deviations from the plan:** +1. **A crop must lie inside every output's frame, or it is a 400.** + `get_output_frames` clamps instead. Clamping per output would measure a + different region in each row. +2. **The summary gives mean, min and max**, more than the plan's "mean and + min". `max` was dropped from the MCP text at first and restored in + 99f04d67. +3. **A grid that can't shrink without going under the 64 px per-cell floor + comes back unshrunk**, with `downscaled_to` null and a note that it is + over budget. It does not claim a shrink that didn't happen. +4. **Negative moments are refused** (`'frame:-1' is negative - moments count + from the start of the clip`). `get_output_frames` still counts them back + from the end. That makes two bounds-policy differences from frames' + grammar: this one and past-end. It came from acceptance case C-F463, + through `moment_index(from_end=False)`. + +## Verification and bounces + +| Stage | Bounces | What bounced | +|---|---|---| +| #889 | 2 | Architecture review: there was no map row for `dw/media_compare.py`, and it had second owners of the `at` grammar and the crop parse (fixed 93d06dbe). Verify on mini-ai: C-F463 call 9, where `frame:-1` and `-1.0` wrapped to the end of the clip instead of being refused (fixed 99f04d67). | + +- #889 was verified on mps (mini-ai, develop @ 5ff05cf5) against C-F454 to + C-F466 and SE-F049. All 14 cases pass. Nothing in the stage is + CUDA-specific. +- Cost is not recorded: the stage comments name no `usage:` figures. + +## Found along the way + +- **The fixture literal in C-F454/C-F459:** `qr_code` squares its output, so + S is 640×640, not 640×384. The tester filed a suite amendment on harnest. +- **Cosmetic, not changed:** in a one-column grid (stills), row labels are + clipped at the cell width. +- **Not a server issue:** C-F466's tool-count check is vacuous over MCP, + because no tool reply states the count. + +## Follow-up + +#861, #875, #862 and #865 each end in "is variant X the same as baseline?". +They can now answer that over MCP with `compare_outputs` instead of local +ffmpeg. diff --git a/docs/proposals/complete/estimate-scaling-complete.md b/docs/proposals/complete/estimate-scaling-complete.md new file mode 100644 index 00000000..6196db5f --- /dev/null +++ b/docs/proposals/complete/estimate-scaling-complete.md @@ -0,0 +1,115 @@ +# Scale the curated estimate instead of dropping or repeating it (#866) + +Written by model `claude-opus-5-5` via provider `anthropic` (close-out, +2026-10-15). Plan v4, approved by Don as "build smaller" on 2026-10-14 (v1 → v2: +Q1-Q5 and the fingerprint ruling. v2 → v3: the stage 0 re-plan. v3 → v4: two +fixes to stage A's acceptance). Four stages: #905 (0), #906 (A), #907 (B) and #908 (C), +all verified on lem. + +## The report + +`POST /api/validate` quoted ~42 min for the ep 6 dialogue-short no matter what +changed. The H3 quality matrix (lem, workspace `h3-quality`, 15 rungs of one +Ref2VA shot, 124 frames, 4 refs) measured the same shot at 3.5 to 44.2 min, +depending on canvas, weights, LoRA, steps, attention and cache. Agents quote +`plan.estimate` before every run, so the quote was off by up to 12x in either +direction. It was the tenth estimate bug in this class (#267, #255, #301, +#589, #590, #593, #796, #821, #851). + +The code check (plan v1, at a27ad5aa) found that the flat figure had three +sources, not one: +- **Catalog path.** A move of steps, canvas or frames off the defaults made + `_own_price` return `basis: unknown`. It was not scaled. +- **Default bucket.** Runs that differed only in weights, LoRA, attention or + cache landed in it, because those were not drivers. +- **Inherited path.** `_scale_past_template` scaled by pipeline-step count + and frames only, not by `num_inference_steps` or the canvas. + +## What was built + +- **Stage 0 (#905): the fingerprint and step cache are keyed only by what + changes the output.** + - `cost_drivers`, `vram_estimate` and the other documentation and cost + metadata no longer reach the plan fingerprint or the step cache keys. The + exclusion list is pinned by a test. + - Integral floats are canonicalized in the fingerprint, so `12.0` and `12` + digest alike. That was the inline vs. by-name mismatch. + - Seed stays out of the fingerprint (the #85 contract). The step cache keys + on it. + - The step cache keeps `subfolder` and `file_base_name`, because a hit + replays files to their recorded paths. +- **Stage A (#906): the catalog figure scales.** + - `scale_curated` gives + `minutes = curated × steps/steps₀ × (w·h)/(w₀·h₀) × frames/frames₀` on the + four generic driver names (`num_inference_steps`, `width`, `height`, + `num_frames`). `num_frames` is summed across a list's entries. + - The result is `basis: "derived"`, and `scaled` names each axis that moved. + - It multiplies with list-length repricing (`_repriced`). `_tempered` blends + toward the scaled figure. + - A non-numeric driver value still gives `unknown`. + - The rule "a `cost` entry is measured at the workflow's variable defaults" + is written in the schema and WORKFLOW_GUIDE. + - dialogue-short's `cost` was re-priced from 42 to 21.2 for its 960x544 + defaults. + - The 30% accuracy bar is `TestScalingAccuracy`: B4→B1 16.5 vs 15.1, B4→B5 + 13.0 vs 11.8, B3→L5 32.3 vs 29.5. + - The UI's `derived` wording shows `scaled`. +- **Stage B (#907): unscalable drivers.** + - Any other declared scalar driver that moves off its default keeps the + (scaled) figure, sets `low_confidence` and is listed in `differs`. This + holds for every template, with no engine allow-list. + - The H3 templates add `weights_dtype`, `lora_model_name`, + `attention_backend` and `cache` to `cost_drivers`, so an int8 or native + attention run no longer pools with the default bucket. LoRA buckets by its + exact name. + - Thanks to stage 0, the new drivers leave H3 fingerprints unchanged. +- **Stage C (#908): the inherited path scales the same way.** + - `_scale_past_template` reuses `scale_curated`'s steps and canvas ratios, + so there is one owner and inline and template quotes agree (C-F484: + within 2%). + - `inherited_differences` compares attention, cache and LoRA too, through + `replace_variables` (no second owner of substitution). + - A reference the definition doesn't declare compares as null. + - Numbers compare by value: `same_number` moved to `dw/variables.py` and is + shared with the fingerprint. + +## What was cut or deferred, and why + +- **Compile as a cost driver.** No H3 template exposes a `compile` variable, + and a driver must be a declared variable. It comes back if a template + exposes compile. +- **A learned fixed + passes × s/pass fit.** Deferred by Don. Plain + proportional scaling over-quotes by about 9-10% on the matrix pairs, + because it ignores load, encode and decode time. It is also slightly + under-superlinear in pixels (2.27x measured for 1.98x pixels). Both are + inside the 30% bar. A miss past 30% is the trigger to build the fit. +- **`passes = steps − 1`.** That is H3 scheduler knowledge, so it can't live + in the engine. The engine uses a plain step ratio (Q2). +- **Price precedence for a saved copy with its own `cost` and no runs** + (catalog vs. inherited) is unchanged (v3 review). +- **The step cache across identities.** A renamed or inline copy can't hit + the cache: it is per card and cleared on an identity switch. That is a + memory-policy change, filed as #912 (idea). +- **The validate probe ignores the card's warm identity**, so it can report + `cached_steps` that the run's identity switch then clears. Filed as #911 + (bug). + +## Bounces per stage + +| Stage | Verify bounces | Architecture bounces | Notes | +|---|---|---|---| +| #905 (0) | 1 | 0 | The bounce showed that v2's "a metadata-only copy hits the step cache" can't hold under the cache's per-identity scope. Re-planned as v3 (in-place edit only), plus the integral-float fix. | +| #906 (A) | 0 | 0 | | +| #907 (B) | 0 | 0 | | +| #908 (C) | 3 | 2 | Architecture: a second owner of the scaling rule, then a second owner of `variable:` substitution. Both were folded into the single owners. Verify: C-F485 `differs: ["LoRA"]` at the defaults, three times. Parked after the 4th hand-off; Don ruled that the plan stands. The causes were an undeclared null-default variable and then `lora_scale` 1.0 vs 1 after a client's JSON round trip. | + +Cost per stage: the stage comments record no `usage:` figures, so none is +given here. The plan estimated ~$2 / $5 / $3 / $4. + +## Rules this feature added + +- A `cost` entry is measured at the workflow's variable defaults. +- The plan fingerprint and the step cache are keyed only by what can change + the output (pinned in `docs/ARCHITECTURE.md` and its tests). +- A number is its value, not its spelling, in the fingerprint and in the + inherited-cost comparison. diff --git a/docs/proposals/complete/h3-keyframe-audio-complete.md b/docs/proposals/complete/h3-keyframe-audio-complete.md new file mode 100644 index 00000000..8e694ad3 --- /dev/null +++ b/docs/proposals/complete/h3-keyframe-audio-complete.md @@ -0,0 +1,103 @@ +# A locked H3 frame that lip-syncs to a song (#852) + +Written by model `claude-opus-5-5` via provider `anthropic` (close-out, +2026-10-14). Plan v0 (the issue body's *Proposed*) approved by Don on +2026-10-13 and capped at four H3 renders on lem; three stages: #877 (A) and +#878 (B) verified, #879 (C) closed not planned by its own condition. + +## The report + +A field report: for a music-video finale, an agent had to choose between a +still camera and real lip-sync. `first-and-last-frame` (FL2VA) pins the +frames and holds a wide two-shot, but takes no audio. `reference-to-video` / +`music-video` (Ref2VA) lip-syncs to the song, but the camera drifts on wide +two-shots. It rendered the finale as solos. + +MiniMax's Ref2VA prompt guide lists `keyframe completion` as a task type +(an image as the target video's first frame, keyframe or last frame), and +task types combine with ` + `. So `[keyframe completion + audio reuse]` is +a valid Ref2VA task. Whether Ref2VA, which treats the keyframe as a +reference anchor, holds a locked two-shot as FL2VA does was untested. + +## Decisions + +- **Don's ruling (2026-10-13):** approved as proposed, capped at four H3 + renders on lem, all on one wide two-shot and one ~8 s song slice, each + with its `plan.estimate` quoted first. Either outcome becomes a + `minimax-h3` skill line; a template variant only if the recipe holds and + needs a shape of its own. +- **Routing (2026-10-14):** first routed to the tester, whose handoff + session couldn't run it; Don re-routed it to the lead as an approved + experiment. + +## What was found (stage A, #877) + +Same still (`asset:h3-keyframe-audio-two-shot.jpg`) and 8 s duet slice +(`asset:h3-keyframe-audio-duet-8s.wav`, 192 frames at 24 fps, 960x544) for +all four. The man sings 0-2.8 s, the woman 4.4-7.3 s. + +| # | Job | Variant | Time | Camera hold | Lip-sync | +|---|-----|---------|------|-------------|----------| +| 1 | `03c511a226bd` | Ref2VA, `` first frame + `