ContributeDevelopment workflow, testing requirements, and contribution policy.
Bugs and feature requestsSearch existing reports or open an issue with a reproducible example.
@@ -47,17 +44,10 @@ uv sync --package flashdreams-wan21 --extra dev
### Which Python API should I use?
-Use `flashdreams.api_v2`; the
-[Demo API reference](../documentation/demo_api/index.md) maps its
-application, session, and loop contracts.
-
-### How do I add a model?
-
-Start with
-[Create a model](../documentation/inferencing_api/guides/create_model.md), then
-[integrate it with a demo](../documentation/demo_api/guides/integrate_model.md).
+Make demos using world-models with our [Demo API](../documentation/demo_api/index.md).
+Implement world-models with our [Inferencing API](../documentation/inferencing_api/index.md).
-### Can I use FlashDreams commercially?
+### How do I implement a model with flashdreams?
-The Apache 2.0 license permits commercial use. Model weights and datasets may
-have separate licenses.
+1. [Implement a model in FlashDreams](../documentation/inferencing_api/guides/create_model.md), then
+2. [Integrate your new model with a demo](../documentation/demo_api/guides/integrate_model_adapter_for_a_demo.md).
diff --git a/docs/source/demos/.nav.yml b/docs/source/demos/.nav.yml
index 7a1aa40a1..8664b90b8 100644
--- a/docs/source/demos/.nav.yml
+++ b/docs/source/demos/.nav.yml
@@ -6,5 +6,6 @@ nav:
- Model Interfaces:
- Action2V: action2v.md
- Cam2V: cam2v.md
+ - Minimal T2V: minimal_t2v.md
- T2V: t2v.md
- V2V: v2v.md
diff --git a/docs/source/demos/index.md b/docs/source/demos/index.md
index 025f07577..f5b50600a 100644
--- a/docs/source/demos/index.md
+++ b/docs/source/demos/index.md
@@ -31,4 +31,5 @@ For how to run a particular **demo/model pair**, refer to the format listed here
+
diff --git a/docs/source/demos/minimal_t2v.md b/docs/source/demos/minimal_t2v.md
new file mode 100644
index 000000000..646569c8f
--- /dev/null
+++ b/docs/source/demos/minimal_t2v.md
@@ -0,0 +1,25 @@
+---
+title: 'Minimal T2V'
+---
+
+
+
+
+Minimal T2V is the tutorial application built in [Create a demo](../documentation/demo_api/guides/create_demo.md).
+It owns an ImGui prompt field, a model loop, and a per-prompt generation cache.
+
+## Quick Start:
+
+
+ [Wan 2.1](../models/wan21.md#minimal-t2v){ .fd-button }
+
+
+## Demo Arguments
+
+Edit the prompt and click **Send** to start a new clip. Empty text pauses
+generation. The window remains open between clips; close it to stop.
+
+| App argument | Default | Purpose |
+| --- | --- | --- |
+| `--default-prompt` | Empty | Initial prompt; omit it to wait for input. |
+| `--device` | `cuda` | Device on which to load the model. |
diff --git a/docs/source/documentation/cli.md b/docs/source/documentation/cli.md
index 00515332c..af47d1066 100644
--- a/docs/source/documentation/cli.md
+++ b/docs/source/documentation/cli.md
@@ -53,20 +53,8 @@ These options override the default demo session values requested by a particular
| `--pixel-height N` | Override generated-frame height. |
| `--fps N` | Override the generated-frame playback rate. |
| `--layout {tchw,btchw,bcthw,bvtchw}` | Override generated tensor layout. |
-| `--backpressure-mode {block,drop_oldest}` | Choose how the model thread handles a full presentation queue. |
-| `--presentation-mode {on_demand,continuous}` | Choose when the presentation loop renders. |
-
-#### Backpressure Mode
-
-- `block` waits for queue capacity so generated chunks are retained.
-
-- `drop_oldest` discards the oldest queued chunk to favor recent output.
-
-#### Presentation Mode
-
-- `on_demand` presents each selected model frame when it arrives.
-
-- `continuous` keeps the UI responsive between model frames by reusing the newest frame.
+| `--backpressure-mode {block,drop_oldest}` | Choose full-queue behavior: `block` waits for capacity to retain generated chunks; `drop_oldest` discards the oldest queued chunk to favor recent output. |
+| `--presentation-mode {on_demand,continuous}` | Choose rendering behavior: `on_demand` presents each selected model frame when it arrives; `continuous` keeps the UI responsive between model frames by reusing the newest frame. |
### Examples
@@ -110,7 +98,5 @@ uv run flashdreams-run-v2 DEMO_MODEL_SLUG --mode native-window --window-title Fl
## See also
- [Quickstart](../quickstart/index.md)
-- [Demo configuration](demo_api/guides/configuration.md)
-- [Application slugs](demo_api/guides/application_slugs.md)
- [Demo API](demo_api/index.md)
- [Offline Program Packager](tools/offline_program_packager.md)
diff --git a/docs/source/documentation/demo_api/api_reference/application.md b/docs/source/documentation/demo_api/api_reference/application.md
index 3ba277494..83fa0e31e 100644
--- a/docs/source/documentation/demo_api/api_reference/application.md
+++ b/docs/source/documentation/demo_api/api_reference/application.md
@@ -8,424 +8,26 @@ title: 'Demo Application API Reference'
This page documents the public application protocols in `flashdreams.api_v2`
and the runtime-owned value types those protocols exchange. For implementation
workflows, start with [Create a demo](../guides/create_demo.md) or
-[Integrate a model with a demo](../guides/integrate_model.md).
+[Integrate model adapter for a demo](../guides/integrate_model_adapter_for_a_demo.md).
## Application and session
-### `flashdreams.api_v2.application.IApplication`
+::: flashdreams.api_v2.application.IApplication
-One application, for as long as the process runs.
-
-Parses its own arguments and holds whatever its sessions share, such as a
-checkpoint or a compiled pipeline. It outlives every session it creates, so
-that shared state is loaded once here and released in `close`.
-
-An application module implements this and `ISession`. The runtime
-creates everything else and passes it in.
-
-[View source](https://github.com/NVIDIA/flashdreams/blob/main/flashdreams/flashdreams/api_v2/application.py#L14)
-
-#### `init`
-
-```python
-def init(self, commandline_args: Sequence[str]) -> None
-```
-
-Parse application arguments and validate startup state.
-
-#### `session_desc`
-
-```python
-def session_desc(self) -> SessionDesc | None
-```
-
-Return the description of a session this application would generate.
-
-A caller has to describe a session before there is one to describe, and
-only the application knows what its model was trained for. Asked before
-`init`, so describing a session costs nothing.
-
-**Returns**
-
-The session to create when nobody asks for another, or `None`,
-the default, from an application that generates whatever it is
-asked for. Its caller describes the session instead.
-
-#### `create_session`
-
-```python
-def create_session(self, session_desc: SessionDesc) -> ISession
-```
-
-Create one isolated, uninitialized session for `session_desc`.
-
-**Parameters**
-
-- `session_desc`: Session the runtime is asking for.
-
-**Returns**
-
-A session for `session_desc`, resolved to what this application can
-actually produce.
-
-**Raises**
-
-- `ValueError`: The application cannot honour `session_desc`.
-
-#### `close`
-
-```python
-def close(self) -> None
-```
-
-Release whatever the application holds.
-
-Not abstract, and does nothing by default, so an application with nothing
-to release does not implement it.
-
-### `flashdreams.api_v2.session.ISession`
-
-One application run with a model loop and a UI loop.
-
-Register the model loop in `init`. If no UI loop is registered,
-the runtime uses `BlitModelOutputToScreenLoop`.
-
-[View source](https://github.com/NVIDIA/flashdreams/blob/main/flashdreams/flashdreams/api_v2/session.py#L21)
-
-- `_registered_ui_loop: IUILoop[Any] | None`
-
-- `_registered_model_loop: IModelLoop[Any] | None`
-
-#### `parallel_context`
-
-```python
-def parallel_context(self) -> ParallelContext | None
-```
-
-Return the model mesh before initialization, or `None` for a local session.
-
-The runtime synchronizes model steps and inputs across this world.
-Only rank zero owns a client window and output sinks.
-
-#### `init`
-
-```python
-def init(self) -> None
-```
-
-Initialize state and register this session's loops.
-
-A model loop is required. Registering a UI loop is optional; without one
-the runtime uses `BlitModelOutputToScreenLoop`.
-
-**Raises**
-
-- `RuntimeError`: No model loop was registered by the time the runtime
- asks for the loops.
-
-#### `session_desc`
-
-```python
-def session_desc(self) -> SessionDesc
-```
-
-Return the description used to configure the runtime.
-
-#### `register_ui_loop`
-
-```python
-def register_ui_loop(self, loop_type: type[IUILoop[Any]], *, state: Any=None, **kwargs: Any) -> IUILoop[Any]
-```
-
-Create and register the UI loop, and return it.
-
-Omit this call to use the default UI. The loop is paced at the session's
-`frames_per_second_for_ui`.
-
-**Parameters**
-
-- `loop_type`: UI loop class to instantiate.
-- `state`: State the loop owns. Unlike a model loop, a UI loop is
- allowed to hold none.
-- `**kwargs`: Passed to `loop_type`.
-
-**Raises**
-
-- `RuntimeError`: The runtime already took the loops, or a UI loop was
- registered already.
-- `TypeError`: `loop_type` does not derive from `IUILoop`.
-
-#### `register_model_loop`
-
-```python
-def register_model_loop(self, loop_type: type[IModelLoop[Any]], *, state: Any, **kwargs: Any) -> IModelLoop[Any]
-```
-
-Create and register the model loop, and return it.
-
-The loop is paced at the session's `frames_per_second_for_step`.
-
-**Parameters**
-
-- `loop_type`: Model loop class to instantiate.
-- `state`: State the loop owns. Required, since a model loop with no
- state has nothing to generate from.
-- `**kwargs`: Passed to `loop_type`.
-
-**Raises**
-
-- `RuntimeError`: The runtime already took the loops, or a model loop
- was registered already.
-- `TypeError`: `loop_type` does not derive from `IModelLoop`.
-
-#### `ui_loop`
-
-```python
-def ui_loop(self) -> IUILoop[Any]
-```
-
-Return the registered UI loop.
-
-#### `model_loop`
-
-```python
-def model_loop(self) -> IModelLoop[Any]
-```
-
-Return the registered model-generation loop.
-
-#### `close`
-
-```python
-def close(self) -> None
-```
-
-Release resources owned by the session.
+::: flashdreams.api_v2.session.ISession
## Model and UI loops
-### `flashdreams.api_v2.loop.IModelLoop`
-
-Loop that generates model results on the model thread.
-
-`ILoop.step` must return `list[StepResult]` here, one entry per
-channel, with every channel reporting the same `frame_count`. An empty
-list means the step produced no presentable output.
-
-Every admitted rank executes the same model step and ordered collectives.
-The integration owns TP/CP partitioning and must gather all CP shards needed
-for publication before the presenting rank returns non-empty results.
-Workers return `[]` only after all required collectives, including gathers.
-Gather before decoding when it needs the complete latent/view sequence;
-decoding may run only on the presenting rank when the model permits it.
-TP reductions that already replicate a complete value need no extra gather.
-
-A run whose steps all present nothing still ends when the model loop does.
-Returning a bare `StepResult` or `None` raises `TypeError`.
-
-[View source](https://github.com/NVIDIA/flashdreams/blob/main/flashdreams/flashdreams/api_v2/loop.py#L289)
+::: flashdreams.api_v2.loop.IModelLoop
-#### `step`
+::: flashdreams.api_v2.loop.IUILoop
-```python
-def step(self, step_index: int, events: UserInputEvents) -> list[StepResult]
-```
-
-We expect IModelLoop's to return a list of StepResults, one result per channel.
-
-#### `inference_state`
-
-```python
-def inference_state(self) -> ModelInferenceState
-```
-
-Return whether this model loop has started, is running, or finished.
-
-### `flashdreams.api_v2.loop.IUILoop`
-
-Loop whose output is sent to the client window.
-
-`ILoop.step` must return `list[StepResult]` here: one frame to
-present, or an empty list to present nothing this step. A list longer than
-one raises `TypeError`. Model frames to draw come from
-`presented_model_frame` and `presented_model_frames` rather
-than from the model loop directly.
-`has_pending_model_frames` and `presented_model_frame_count`
-report whether more model frames are waiting and how many have been
-selected.
-
-[View source](https://github.com/NVIDIA/flashdreams/blob/main/flashdreams/flashdreams/api_v2/loop.py#L424)
-
-#### `step`
-
-```python
-def step(self, step_index: int, events: UserInputEvents) -> list[StepResult]
-```
-
-Return zero or one `StepResult` to present this tick.
-
-#### `register_session_ui_loop_objects`
-
-```python
-def register_session_ui_loop_objects(self, *, session_desc: SessionDesc, presentation_manager: PresentationManager) -> None
-```
-
-Store UI objects supplied when this loop is registered with a session.
-
-**Parameters**
-
-- `session_desc`: Description of the session this UI presents.
-- `presentation_manager`: Buffer containing model frames.
-
-#### `model_inference_state`
-
-```python
-def model_inference_state(self) -> ModelInferenceState
-```
-
-Return whether model inference has started, is running, or finished.
-
-#### `request_new_session`
-
-```python
-def request_new_session(self, session_desc: SessionDesc) -> None
-```
-
-Ask the runtime to replace this session after the current UI step.
-
-**Parameters**
-
-- `session_desc`: Fully resolved description for the replacement session.
-
-#### `request_hide_cursor`
-
-```python
-def request_hide_cursor(self, hide_cursor: bool) -> None
-```
-
-Request that the client window show or hide its cursor.
-
-#### `request_lock_cursor_to_window`
-
-```python
-def request_lock_cursor_to_window(self, lock_cursor_to_window: bool) -> None
-```
-
-Request that the client window release or capture pointer motion.
-
-#### `request_new_window_size`
-
-```python
-def request_new_window_size(self, new_window_size: tuple[int, int]) -> None
-```
-
-Request a positive downstream client-window width and height.
-
-**Parameters**
-
-- `new_window_size`: Requested `(width, height)` in pixels.
-
-**Raises**
-
-- `TypeError`: The size is not a pair of integers.
-- `ValueError`: Either dimension is not positive.
-
-#### `get_or_create_ui_loop_requests`
-
-```python
-def get_or_create_ui_loop_requests(self) -> UILoopRequests
-```
-
-#### `flush_ui_loop_requests`
-
-```python
-def flush_ui_loop_requests(self) -> UILoopRequests | None
-```
-
-return UI loop requests and clear the internal state of requests.
-
-#### `presented_model_frame`
-
-```python
-def presented_model_frame(self, channel_index: int=0) -> Tensor | None
-```
-
-Return the current frame from one model-result channel.
-
-**Parameters**
-
-- `channel_index`: Channel to read, indexed as the model loop returned
- them.
-
-**Returns**
-
-A `[C, H, W]` frame with one, three or four channels, or `None`
-before the first model result has been presented.
-
-**Raises**
-
-- `IndexError`: The presented result has no such channel.
-- `ValueError`: The presentation stream is on a different CUDA device
- from the presented result.
-
-#### `presented_model_frames`
-
-```python
-def presented_model_frames(self) -> tuple[Tensor, ...]
-```
-
-Return the current frame from every model-result channel.
-
-**Returns**
-
-One `[C, H, W]` frame per channel, bottom channel first, or an
-empty tuple before the first model result has been presented.
-
-**Raises**
-
-- `ValueError`: The presentation stream is on a different CUDA device
- from a presented result.
-
-#### `presented_model_frame_count`
-
-```python
-def presented_model_frame_count(self) -> int
-```
-
-Return how many model frames have been selected in this generation.
-
-#### `has_pending_model_frames`
-
-```python
-def has_pending_model_frames(self) -> bool
-```
-
-Return whether another model frame is ready to present.
+::: flashdreams.runtime_v2.imgui_ui_loop.ImGuiUILoop
## Session description
-### `flashdreams.runtime_v2.session_desc.SessionDesc`
-
-Description of a session, passed to create one and to open a window on it.
-
-The runtime fills this in to ask an application for a session, and the
-session reports back what it resolved to. The same description then
-configures the client window through `OutputSink.open`.
-
-[View source](https://github.com/NVIDIA/flashdreams/blob/main/flashdreams/flashdreams/runtime_v2/session_desc.py#L35)
-
-- `output_layout: VideoTensorLayout`
-
-- `backpressure_mode: BackpressureMode`
-
-- `presentation_mode: PresentationMode`
-
-- `frames_per_second_for_ui: int`
-
-- `frames_per_second_for_step: int`
-
-- `video_width: int`
+::: flashdreams.runtime_v2.session_desc.SessionDesc
-- `video_height: int`
+::: flashdreams.runtime_v2.session_desc.BackpressureMode
-- `metadata: dict[str, Any]`
+::: flashdreams.runtime_v2.session_desc.PresentationMode
diff --git a/docs/source/documentation/demo_api/guides/application_slugs.md b/docs/source/documentation/demo_api/guides/application_slugs.md
deleted file mode 100644
index 4c00df5b3..000000000
--- a/docs/source/documentation/demo_api/guides/application_slugs.md
+++ /dev/null
@@ -1,59 +0,0 @@
----
-title: 'Application slugs and model adapters'
----
-
-
-
-
-An application slug identifies one installed, zero-argument
-`flashdreams.api_v2.IApplication` factory. It selects both an interaction and a
-model binding; it is not a pipeline-config registry key.
-
-## Register an application
-
-Register factories in the model integration's `pyproject.toml`:
-
-```toml
-[project.entry-points."flashdreams.applications_v2"]
-"t2v-self-forcing-wan2.1-t2v-1.3b" = "self_forcing.apps.t2v.adapter:create_app"
-"t2v-self-forcing-wan2.1-t2v-1.3b-taehv" = "self_forcing.apps.t2v.adapter:create_app_taehv"
-```
-
-Each target must be callable without arguments and return an uninitialized
-`IApplication`. Importing the module or calling the factory must not load a
-checkpoint, initialize CUDA, or open a window; `IApplication.init` owns that
-work.
-
-Use a stable, lowercase, hyphenated slug. Existing integrations generally use
-`
Create a demoBuild a model-independent application against the Demo API.
- Integrate a modelConnect a model implementation to an existing demo.
- Demo configurationDefine arguments, defaults, and application configuration.
- Application slugs and dispatchRegister application names and route model adapters.
- Interactive servingServe responsive sessions through supported client windows.
- Local benchmarksMeasure application performance in a local environment.
+ Integrate model adapter for a demoConnect a model to an existing demo.
+ Interactive servingRun a demo in a browser or local window.
Profiling a demoCollect Nsight Systems traces and per-step performance metrics.
diff --git a/docs/source/documentation/inferencing_api/api_reference/accelerated.md b/docs/source/documentation/inferencing_api/api_reference/accelerated.md
new file mode 100644
index 000000000..952797752
--- /dev/null
+++ b/docs/source/documentation/inferencing_api/api_reference/accelerated.md
@@ -0,0 +1,81 @@
+---
+title: 'Accelerated API Reference'
+---
+
+The `flashdreams.accelerated` package provides optimized attention, quantization,
+and shared inference components. See [Accelerated building blocks](../guides/accelerated.md)
+for usage and integration guidance.
+
+## Common components
+
+::: flashdreams.accelerated.common.NonPersistentLinear
+
+::: flashdreams.accelerated.common.rms_norm
+
+## Quantization
+
+::: flashdreams.accelerated.quantization.quantizer.Granularity
+
+::: flashdreams.accelerated.quantization.quantizer.quantize
+
+::: flashdreams.accelerated.quantization.quantizer.dequantize
+
+::: flashdreams.accelerated.quantization.linear.WeightGranularity
+
+::: flashdreams.accelerated.quantization.linear.QuantizedNonPersistentLinear
+
+## Attention contracts
+
+::: flashdreams.accelerated.multi_head_attention.AttentionType
+
+::: flashdreams.accelerated.multi_head_attention.AttentionConfig
+
+::: flashdreams.accelerated.multi_head_attention.AttentionMask
+
+::: flashdreams.accelerated.multi_head_attention.QKNormScope
+
+::: flashdreams.accelerated.multi_head_attention.RoPEStyle
+
+::: flashdreams.accelerated.multi_head_attention.RoPEScope
+
+::: flashdreams.accelerated.multi_head_attention.RoPEConfig
+
+::: flashdreams.accelerated.multi_head_attention.MultiHeadAttention
+
+## Attention implementations
+
+::: flashdreams.accelerated.multi_head_attention.torch.TorchMultiHeadAttention
+
+::: flashdreams.accelerated.multi_head_attention.optimized.SDPABackend
+
+::: flashdreams.accelerated.multi_head_attention.optimized.QKVFusionOption
+
+::: flashdreams.accelerated.multi_head_attention.optimized.QuantizationOption
+
+::: flashdreams.accelerated.multi_head_attention.optimized.OptimizedImplConfig
+
+::: flashdreams.accelerated.multi_head_attention.optimized.OptimizedMultiHeadAttention
+
+::: flashdreams.accelerated.multi_head_attention.flex.FlexAttentionOptions
+
+::: flashdreams.accelerated.multi_head_attention.flex.compiled_flex_attention
+
+::: flashdreams.accelerated.multi_head_attention.MoTRoute
+
+::: flashdreams.accelerated.multi_head_attention.mot.MoTMultiHeadAttention
+
+## Masked attention functions
+
+::: flashdreams.accelerated.multi_head_attention.functional.masked_attention
+
+::: flashdreams.accelerated.multi_head_attention.reference.reference_masked_attention
+
+## KV arena
+
+::: flashdreams.accelerated.multi_head_attention.kv_arena.KVSegmentLayout
+
+::: flashdreams.accelerated.multi_head_attention.kv_arena.allocate_kv_arena
+
+::: flashdreams.accelerated.multi_head_attention.kv_arena.validate_kv_arena_segments
+
+::: flashdreams.accelerated.multi_head_attention.kv_arena.stage_current_kv_
diff --git a/docs/source/documentation/inferencing_api/api_reference/core.md b/docs/source/documentation/inferencing_api/api_reference/core.md
index d88643b93..6f4f5461d 100644
--- a/docs/source/documentation/inferencing_api/api_reference/core.md
+++ b/docs/source/documentation/inferencing_api/api_reference/core.md
@@ -3,7 +3,6 @@ title: 'Core API Reference'
---
-
The `flashdreams.core` package collects the low-level attention primitives,
@@ -14,368 +13,18 @@ KV-cache implementations, and process-group utilities that integrations share.
The attention package provides the kernels used by the transformer and
the block-structured KV cache that backs streaming inference.
-
-### `NativeAttention`
-
-```text
-class NativeAttention ( * args : Any , ** kwargs : Any )
-```
-
-Bases: `Module`
-
-Native attention module with configurable QKV layout and SDPA backend.
-
-
-#### `set_context_parallel_group`
-
-```text
-set_context_parallel_group ( cp_group : ProcessGroup | None ) → None
-```
-
-Enable or disable context parallelism for ring attention.
-
-##### ``
-
-```text
-Parameters :
-```
-
-**cp\_group** – Process group for context parallel; use None to disable.
-
-#### `is_context_parallel_enabled`
-
-```text
-is_context_parallel_enabled ( ) → bool
-```
-
-Return True if context parallelism is active.
-
-#### `context_parallel_size`
-
-```text
-context_parallel_size ( ) → int
-```
-
-Return the context parallel world size, or 1 if disabled.
-
-#### `forward`
-
-```text
-forward ( query : torch.Tensor , key : torch.Tensor , value : torch.Tensor ) → torch.Tensor
-```
-
-Run context-parallel SDPA (or single-rank SDPA when CP is disabled).
-
-##### ``
-
-```text
-Parameters :
-```
-
-* **query** – Query tensor in configured `qkv_format`.
-* **key** – Key tensor in configured `qkv_format`.
-* **value** – Value tensor in configured `qkv_format`.
-
-
-### `ContextParallelAttention`
-
-```text
-class ContextParallelAttention ( * args : Any , ** kwargs : Any )
-```
-
-Bases: `NativeAttention`
-
-Context-parallel attention with selectable method and SDPA backend.
-
-
-### `BlockKVCache`
-
-```text
-class BlockKVCache ( k_shape : tuple [ int , ... ] , v_shape : tuple [ int , ... ] , seq_dim : int , chunk_size : int , window_size : int , sink_size : int = 0 , device : torch.device | str = torch.device , dtype : torch.dtype = torch.float16 , _prev_chunk_idx : int = -1 , _curr_chunk_idx : int | None = None , _n_cached : int = 0 )
-```
-
-Bases: `object`
-
-KV cache for causal attention with a fixed-size local window, CUDA-graph compatible.
-
-Keys and values can have arbitrary shape `[..., total_size, ...]`; the sequence
-(rolling) dimension is given by `seq_dim` (dimension index, can be negative).
-Layout along that dimension: [sink tokens | local window tokens]. Sink tokens are
-never evicted; the local window rolls left as new chunks are added if full. Chunks are
-non-overlapping: each update adds one chunk of `chunk_size` tokens at the
-next logical position in the full sequence.
-
-#### ``
-
-```text
-Phases :
-```
-
-* **- Filling** – cache not yet full; tokens are written contiguously;
- `cached_k()` / `cached_v()` return only the valid prefix.
-* **- Steady-state** – if adding a chunk would exceed the fixed cache size, the
- local window rolls left by the overflow amount and the new chunk
- overwrites the rightmost positions; `cached_k()` / `cached_v()`
- return the full buffer.
-The argument `chunk_idx` (0, 1, 2, …) is the index of the new chunk in the full
-sequence (not an index into the cache). If `chunk_idx` is greater than
-the previous one, the chunk is appended (or, in steady-state, written after
-the roll). If `chunk_idx` equals the previous one, the same cache positions
-are overwritten.
-
-#### ``
-
-```text
-Per-step usage:
-```
-
-1. before\_update(chunk\_idx) — prepare (roll local window if steady-state).
-2. update(k, v) — write the new chunk’s keys/values into the cache.
-3. cached\_k() / cached\_v() — get cached keys/values for attention.
-4. after\_update(chunk\_idx) — update internal bookkeeping.
-
-#### `k_shape`
-
-```text
-k_shape : tuple [ int , ... ]
-```
-
-Shape of the keys. Must be the same as the values shape except for the last dimension.
-
-#### `v_shape`
-
-```text
-v_shape : tuple [ int , ... ]
-```
-
-Shape of the values. Must be the same as the keys shape except for the last dimension.
-
-#### `seq_dim`
-
-```text
-seq_dim : int
-```
-
-Sequence dimension that will be rolled. Can be negative.
-
-#### `chunk_size`
-
-```text
-chunk_size : int
-```
-
-Number of tokens processed each time.
-
-#### `window_size`
-
-```text
-window_size : int
-```
-
-Size of the local attention window (excluding sink tokens).
-
-#### `sink_size`
-
-```text
-sink_size : int = 0
-```
-
-Number of sink tokens at the start of the cache that are never evicted. Defaults to 0.
-
-#### `device`
-
-```text
-device : torch.device | str = 'cuda'
-```
-
-Device to store the cache on.
-
-#### `dtype`
-
-```text
-dtype : torch.dtype
-```
+::: flashdreams.core.attention.NativeAttention
-Data type to store the cache in.
-
-#### `size`
+::: flashdreams.core.attention.ContextParallelAttention
-```text
-property size : int
-```
-
-Number of valid cached tokens visible to attention.
-
-#### `write_end`
-
-```text
-property write_end : int
-```
-
-Right edge of the current chunk in the physical cache layout.
-
-#### `from_tensor`
-
-```text
-classmethod from_tensor ( k : torch.Tensor , v : torch.Tensor , seq_dim : int ) → Self
-```
-
-Build a single-chunk cache pre-filled with the given key and value tensors.
-
-#### `is_steady_state`
-
-```text
-is_steady_state ( ) → bool
-```
-
-Return True if the cache is full (steady-state phase).
-
-#### `before_update`
-
-```text
-before_update ( chunk_idx : int ) → None
-```
-
-Prepare the cache before writing new tokens.
-
-If `chunk_idx` equals the previous chunk index, this is a no-op. Otherwise,
-we expect the `chunk_idx` to be +1 from the previous chunk index. In this case,
-we will roll the local window left if the cache is in steady-state, or no op
-if the cache is in filling phase.
-
-##### ``
-
-```text
-Parameters :
-```
-
-**chunk\_idx** – Chunk index of the new chunk in the full sequence.
-
-#### `update`
-
-```text
-update ( k : torch.Tensor , v : torch.Tensor ) → None
-```
-
-Write the new chunk’s keys and values into the cache.
-
-Must be called after `before_update()` and before `after_update()`.
-
-##### ``
-
-```text
-Parameters :
-```
-
-* **k** – Keys; shape must match cached keys except at seq\_dim, where length must be chunk\_size.
-* **v** – Values; shape must match cached values except at seq\_dim, where length must be chunk\_size.
-
-#### `after_update`
-
-```text
-after_update ( chunk_idx : int ) → None
-```
-
-Finalize bookkeeping after writing new tokens.
-
-Updates `_prev_chunk_idx` and, in filling phase, `_n_cached`.
-
-##### ``
-
-```text
-Parameters :
-```
-
-**chunk\_idx** – The index of the new chunk in the full sequence.
-
-#### `cached_k`
-
-```text
-cached_k ( ) → torch.Tensor
-```
-
-Return cached keys for attention (valid prefix in filling phase, full buffer in steady-state).
-
-#### `cached_v`
-
-```text
-cached_v ( ) → torch.Tensor
-```
-
-Return cached values for attention (valid prefix in filling phase, full buffer in steady-state).
-
-#### `reset`
-
-```text
-reset ( ) → None
-```
-
-Reset bookkeeping while preserving the allocated tensor storage.
-
-#### `clone_kv`
-
-```text
-clone_kv ( ) → tuple [ torch.Tensor , torch.Tensor ]
-```
-
-Return clones of the full physical K/V buffers.
-
-#### `overwrite_kv_`
-
-```text
-overwrite_kv_ ( k : torch.Tensor , v : torch.Tensor ) → None
-```
-
-Overwrite the full K/V buffers without changing their addresses.
-
-##### ``
-
-```text
-Parameters :
-```
-
-* **k** – Replacement keys with the exact cache shape.
-* **v** – Replacement values with the exact cache shape.
+::: flashdreams.core.attention.BlockKVCache
## Distributed
-Helpers for multi-GPU and multi-node inference. `init` boots the NCCL
-process group with NVML-derived CPU affinity, a configurable heartbeat timeout,
-and a larger L2 fetch granularity.
-
-
-### `init`
-
-```text
-init ( ) → int | None
-```
-
-Initialize distributed training.
-
-
-### `Device`
-
-```text
-class Device ( device_idx : int )
-```
-
-Bases: `object`
-
-Lightweight wrapper around an NVML device handle for CPU-affinity queries.
-
-
-#### `get_name`
-
-```text
-get_name ( ) → str
-```
-
-Return the marketing name reported by NVML for this device.
-
-#### `get_cpu_affinity`
+Helpers for multi-GPU / multi-node inference. `init` boots the NCCL process
+group with sensible defaults and replaces the boilerplate at the top of
+example launchers.
-```text
-get_cpu_affinity ( ) → list [ int ]
-```
+::: flashdreams.core.distributed.init
-Return the indices of CPUs ideally affined to this GPU per NVML.
+::: flashdreams.core.distributed.Device
diff --git a/docs/source/documentation/inferencing_api/api_reference/pipeline.md b/docs/source/documentation/inferencing_api/api_reference/pipeline.md
index fb173d69e..78ffdad83 100644
--- a/docs/source/documentation/inferencing_api/api_reference/pipeline.md
+++ b/docs/source/documentation/inferencing_api/api_reference/pipeline.md
@@ -3,7 +3,6 @@ title: 'Pipeline API Reference'
---
-
The `flashdreams.infra` package defines the swappable abstractions that
@@ -19,59 +18,11 @@ built via `config.setup()`. `PrintableConfig` provides readable
configuration dumps, while `derive_config` deep-copies an existing
config before applying nested overrides.
-
-### `PrintableConfig`
-
-```text
-class PrintableConfig
-```
-
-Bases: `object`
-
-Config base class providing a multi-line `__str__` for human-readable dumps.
-
-
-### `InstantiateConfig`
-
-```text
-class InstantiateConfig ( _target : type )
-```
-
-Bases: `PrintableConfig`
-
-Config carrying a `_target` class plus its kwargs, instantiable via `setup`.
-
-
-#### `setup`
-
-```text
-setup ( ** kwargs : Any ) → Any
-```
-
-Instantiate the configured object.
-
-
-### `derive_config`
-
-```text
-derive_config ( base_config : T , ** changes : Any ) → T
-```
+::: flashdreams.infra.config.PrintableConfig
-Deep-copy a base config and apply nested keyword overrides.
+::: flashdreams.infra.config.InstantiateConfig
-Nested `dict` values walk into both dataclass attributes and nested
-dicts; leaf values overwrite directly. Raises `KeyError` on unknown
-paths.
-
-Example:
-
-```
-new_config = derive_config(
- base_config,
- tokenizer=WanVAEInterfaceConfig(checkpoint_path=...),
- dit=dict(len_t=3, checkpoint_path=...),
-)
-```
+::: flashdreams.infra.config.derive_config
## Pipeline
@@ -81,1464 +32,80 @@ the diffusion model, and an optional decoder, with per-session caches for the
configured streaming components. Call `finalize()` after `generate()` to
advance the diffusion model's autoregressive cache.
-
-### `StreamInferencePipelineConfig`
-
-```text
-class StreamInferencePipelineConfig ( * , _target: type[StreamInferencePipeline] =
Create a modelImplement and register a model integration with FlashDreams.
Stream inference pipelineBuild stateful generation with streaming pipeline contracts.
- Latency tuningProfile and reduce model, decode, transfer, and presentation latency.
Profiling a pipelineMeasure pipeline stages with CUDA event or NVTX profilers.
Accelerated building blocksUse optimized attention, quantization, caching, and execution components.
@@ -26,7 +19,7 @@ the lower-level pipeline and model building blocks.
## API Reference
- Inference Runtime APIReference for model loops, sessions, and runtime contracts.
Pipeline APIReference for inference pipelines and reusable infrastructure.
Core APIReference for model-agnostic core components and utilities.
+ Accelerated APIReference for optimized attention, quantization, and shared inference components.
diff --git a/flashdreams/tools/benchmarks/README.md b/docs/source/documentation/tools/benchmarking.md
similarity index 96%
rename from flashdreams/tools/benchmarks/README.md
rename to docs/source/documentation/tools/benchmarking.md
index 983880594..14d4b8422 100644
--- a/flashdreams/tools/benchmarks/README.md
+++ b/docs/source/documentation/tools/benchmarking.md
@@ -1,3 +1,7 @@
+---
+title: 'Benchmarking'
+---
+
-Choose a tool for packaging applications or maintaining the documentation website.
+Tools for packaging applications, building documentation, running tests, and benchmarking models.
Offline Program PackagerPackage an application for offline use without Python, uv, or network access.
Build WebsiteValidate, build, and preview the FlashDreams documentation website.
+ TestingTest discovery, CI tier markers, and local commands.
+ BenchmarkingGenerate model benchmarks, score clips, and compare runs against a baseline.
diff --git a/docs/source/documentation/tools/testing.md b/docs/source/documentation/tools/testing.md
new file mode 100644
index 000000000..4a4e36d6b
--- /dev/null
+++ b/docs/source/documentation/tools/testing.md
@@ -0,0 +1,69 @@
+---
+title: 'Testing'
+---
+
+
+
+
+Pytest collection is configured in the root `pyproject.toml`. There is
+no `testpaths` setting, so `pytest` from the repo root walks the tree:
+
+- Files named `test_*.py`. Pytest's default also collects `*_test.py`;
+ don't use that name here.
+- Classes named `Test*` (pytest default). Some modules use classes,
+ some only module-level functions.
+- Functions and methods named `test_*` (pytest default), including
+ `async def test_*`.
+
+Same test filename in different folders is fine (several
+`test_application.py` files exist). Root `pyproject.toml` already sets
+`--import-mode=importlib` so pytest keeps them separate. Do not remove
+that flag.
+
+A `test_*.py` next to the code it validates is collected with no CI
+change. The root `pyproject.toml` skips `parity_check`,
+`parity_check_v2`, `baseline_fastvideo`, and `baseline_lightx2v` via
+`norecursedirs`.
+
+Every test function must be marked with exactly one **CI tier marker**.
+A pytest plugin (`flashdreams._pytest_plugins.marker_enforcement`)
+enforces this at collection time -- tests without a marker are
+rejected, and tests with both `ci_cpu` and `ci_gpu` are rejected.
+
+| Marker | When to use | CI runner |
+|--------|-------------|-----------|
+| `@pytest.mark.ci_cpu` | Pure CPU logic, no GPU or `libGL` needed | CPU runner |
+| `@pytest.mark.ci_gpu` | Needs CUDA, `libGL`, or transitive `cv2` | GPU runner (RTX Pro 6000) |
+| `@pytest.mark.manual` | Heavy (OOM risk), flaky, needs credentials or large downloads | Not run in CI |
+
+Use a module-level `pytestmark` when every test in a file shares the
+same marker:
+
+```python
+import pytest
+
+pytestmark = pytest.mark.ci_cpu
+```
+
+Use per-function markers when tests in the same file have different
+tiers:
+
+```python
+@pytest.mark.ci_cpu
+def test_basic_math(): ...
+
+@pytest.mark.ci_gpu
+def test_cudagraph_path(): ...
+```
+
+Running tests locally:
+
+```bash
+uv run pytest -m ci_cpu # CPU-safe tests only
+uv run pytest -m ci_gpu # GPU tests only (needs CUDA)
+uv run pytest -m "not manual" # everything that runs in CI
+uv run pytest # all tests including manual
+```
+
+For local and Docker test-runner scripts, see
+[tests/README.md](https://github.com/NVIDIA/flashdreams/blob/main/tests/README.md).
diff --git a/docs/source/index.md b/docs/source/index.md
index f3ae0fe85..9f39b95a8 100644
--- a/docs/source/index.md
+++ b/docs/source/index.md
@@ -9,6 +9,8 @@ hide:
+
+
FlashDreams
diff --git a/docs/source/models/index.md b/docs/source/models/index.md index fb4b18607..2554d7100 100644 --- a/docs/source/models/index.md +++ b/docs/source/models/index.md @@ -67,7 +67,7 @@ These models enhance an existing video stream rather than generating a scene fro Follow [Create a model](../documentation/inferencing_api/guides/create_model.md) for the model package, then -[Integrate a model with a demo](../documentation/demo_api/guides/integrate_model.md) +[Integrate model adapter for a demo](../documentation/demo_api/guides/integrate_model_adapter_for_a_demo.md) to register a runnable application. ## Related diff --git a/docs/source/models/wan21.md b/docs/source/models/wan21.md index 6e46f26ab..b9b57036b 100644 --- a/docs/source/models/wan21.md +++ b/docs/source/models/wan21.md @@ -35,10 +35,22 @@ uv run --no-sync flashdreams-run-v2 \ ``` +### Minimal T2V + +```bash +uv sync --package flashdreams-wan21 --inexact +uv run --no-sync flashdreams-run-v2 minimal-t2v-wan21 --mode native-window -- --default-prompt "A cat surfing" +``` + + +## Demo Presets @@ -51,6 +63,14 @@ uv run --no-sync flashdreams-run-v2 \ | --- | --- | | `t2v-wan21-t2v-1.3b-480p` | Wan 2.1 T2V 1.3B at 480p with a single bidirectional block. | + + +### Minimal T2V + +| Preset | Description | +| --- | --- | +| `minimal-t2v-wan21` | [Minimal T2V app](../demos/minimal_t2v.md) with Wan 2.1 T2V 1.3B at 480p, generating one block per prompt. | +
## Developer Details diff --git a/docs/source/quickstart/index.md b/docs/source/quickstart/index.md index 44e6a28f9..ac1475e67 100644 --- a/docs/source/quickstart/index.md +++ b/docs/source/quickstart/index.md @@ -63,6 +63,19 @@ See our [How to Run a Model](../models/index.md#running-a-model) for more inform - See our [Omnidreams Overview](../models/omnidreams.md) for a brief Omnidreams world-model overview. - See our [Interactive Drive Overview](../demos/interactive_drive.md) for more information on the demo. +## Install one integration + +Install only the model package you need with `--package`. Add `--extra dev` +when working on its code: + +```bash +uv sync --package flashdreams-omnidreams --extra dev --inexact +``` + +Use the distribution name from the integration's `pyproject.toml`. +See [Models](../models/index.md) for available integrations, requirements, and +setup commands. See [Testing](../documentation/tools/testing.md) for test commands. + ## Where to next - [Model Gallery](../models/index.md): Every FlashDreams default-shipped model & how to run models! diff --git a/docs/zensical.toml b/docs/zensical.toml index 8fcb3c864..ba55481b6 100644 --- a/docs/zensical.toml +++ b/docs/zensical.toml @@ -45,9 +45,24 @@ md_in_html = {} tables = {} pymdownx.details = {} pymdownx.superfences = {} +pymdownx.snippets = { base_path = ["."], check_paths = true } pymdownx.arithmatex.generic = true [project.plugins.awesome-nav] +[project.plugins.mkdocstrings.handlers.python] +paths = ["../flashdreams"] + +[project.plugins.mkdocstrings.handlers.python.options] +docstring_style = "google" +members_order = "source" +merge_init_into_class = true +separate_signature = true +show_root_full_path = true +show_root_heading = true +show_signature_annotations = true +show_source = true +signature_crossrefs = true + [project.extra] generator = false diff --git a/flashdreams/flashdreams/accelerated/__init__.py b/flashdreams/flashdreams/accelerated/__init__.py new file mode 100644 index 000000000..52a7a9daf --- /dev/null +++ b/flashdreams/flashdreams/accelerated/__init__.py @@ -0,0 +1,2 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 diff --git a/flashdreams/flashdreams/accelerated/quantization/__init__.py b/flashdreams/flashdreams/accelerated/quantization/__init__.py new file mode 100644 index 000000000..52a7a9daf --- /dev/null +++ b/flashdreams/flashdreams/accelerated/quantization/__init__.py @@ -0,0 +1,2 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 diff --git a/flashdreams/flashdreams/infra/__init__.py b/flashdreams/flashdreams/infra/__init__.py new file mode 100644 index 000000000..9ef615b94 --- /dev/null +++ b/flashdreams/flashdreams/infra/__init__.py @@ -0,0 +1,4 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Shared infrastructure for model integrations.""" diff --git a/flashdreams/flashdreams/infra/diffusion/__init__.py b/flashdreams/flashdreams/infra/diffusion/__init__.py new file mode 100644 index 000000000..592a63868 --- /dev/null +++ b/flashdreams/flashdreams/infra/diffusion/__init__.py @@ -0,0 +1,4 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Diffusion model, transformer, and scheduler interfaces.""" diff --git a/flashdreams/flashdreams/runtime_v2/README.md b/flashdreams/flashdreams/runtime_v2/README.md index 064974c7c..6f2cd1398 100644 --- a/flashdreams/flashdreams/runtime_v2/README.md +++ b/flashdreams/flashdreams/runtime_v2/README.md @@ -218,7 +218,7 @@ then the new session receives zero remaining time and stops at its first boundar results as they are published, not the UI loop's output, so a benchmark measures what the model generated while the window still sees one composited frame per tick. See [`configs/v2_model_benchmarks.json`](../../../configs/v2_model_benchmarks.json) -and the [benchmark README](../../tools/benchmarks/README.md) for the suite. +and the [benchmarking guide](https://nvidia.github.io/flashdreams/main/documentation/tools/benchmarking/) for the suite. The runtime augments those model records with step wall time and presentation-queue depth/publish-wait measurements under the reserved `runtime_` metric prefix. UI and window timings are not folded into a later diff --git a/integrations_v2/README.md b/integrations_v2/README.md index 33a715db2..21ea59067 100644 --- a/integrations_v2/README.md +++ b/integrations_v2/README.md @@ -6,10 +6,10 @@ SPDX-License-Identifier: Apache-2.0 Applications built on the v2 API. Each directory is a standalone package that depends on `flashdreams` and holds no framework code of its own. -This is the guide to writing one. If the model generates video from a prompt, -read [`apps/t2v`](../apps/t2v/README.md) -instead, that path is a subclass and a `pyproject.toml`, and most of what -follows is already done for you. +For new work, follow the website's +[Create a model](../docs/source/documentation/inferencing_api/guides/create_model.md) +and [Integrate model adapter for a demo](../docs/source/documentation/demo_api/guides/integrate_model_adapter_for_a_demo.md) +guides. They are the canonical authoring instructions. ## What is here @@ -42,7 +42,6 @@ follows is already done for you. ```text integrations_v2/