diff --git a/README.md b/README.md index 1883a45..e38cdd9 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,8 @@ All four covers are **AI-generated editorial illustrations**, not footage, measu [DVIDIA Observation Lab pilot](reports/DVIDIA%20Observation%20Lab%20pilot.md) publishes our local video detection and review workflow: sampled object/palm proposals, editable timeline revisions and JSON export. Each recorded engineering run completed six public excerpts (19 seconds and 38 sampled frames); independent annotation accuracy remains unmeasured. [Read the paper](https://research.dvidia.org/papers/dvidia-observation-lab/), [install the versioned open-source release](https://github.com/Dvidia-Inference/dvidia-training/releases/tag/observation-lab-v0.1.0), or [try the synthetic browser review demo](https://huggingface.co/spaces/Dvidia/observation-lab). The hosted demo uses authored clips and boxes; real detection runs locally with separately provisioned upstream models. No action captions, validated depth or robot skill are established. +The [image-motion source alpha](https://github.com/Dvidia-Inference/dvidia-training/pull/7) adds manually seeded causal image tracking, pixel speed, short-horizon forecasts and local replay. Its six authored cases cover 720 frames and process in 1.454 seconds for 24 source seconds on one CPU run; occlusion/low-texture losses and an 89.975 px worst 500 ms reversal forecast remain explicit. [Exact source, benchmark and installation](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md). The v0.1 release/Space remain separate; real-scene accuracy, metric speed, live robot latency and the full v0.2 are not established. + [Robotics research pipeline](reports/Robotics%20research%20pipeline.md) maps the path from Skillspace originals and synchronized robot measurements to reviewed episodes, local training, frozen simulation evaluation and compatible skill installation. The diagram distinguishes implemented simulation components from proposed automatic labeling and a learned task supervisor. A new annotation-labor pilot and feedback-supervisor experiment are specified; neither has been run, and physical robot qualification remains open. [Read the public architecture](https://research.dvidia.org/papers/robotics-research-pipeline/). [Skillspace footage training pipeline](reports/Skillspace%20footage%20training%20pipeline.md) introduces the standalone [DVIDIA Training repository](https://github.com/Dvidia-Inference/dvidia-training): audit actual footage, freeze connected source groups, fit visual weights locally and export inspectable results. Its optional movement branch requires aligned numerical actions and a supported simulation adapter. The separate [frozen sample-count benchmark](https://github.com/Dvidia-Inference/dvidia-training/blob/main/benchmarks/sample-count-19047/README.md) compares two, four and seven training groups on the same twelve layouts: nominal completion is 8/8, 7/8 and 8/8; actuator-stress completion is 3/4 for each, and all 36 matched open-jaw controls fail. This single composition-order pilot establishes no sufficient video count, human-video action bridge or hardware skill. [Inspect the protocol, outcomes and recorded evidence release](https://github.com/Dvidia-Inference/dvidia-training/releases/tag/benchmark-2026-10-08). diff --git a/docs/downloads/reports/DVIDIA Observation Lab pilot.md b/docs/downloads/reports/DVIDIA Observation Lab pilot.md index efc7d68..0e65496 100644 --- a/docs/downloads/reports/DVIDIA Observation Lab pilot.md +++ b/docs/downloads/reports/DVIDIA Observation Lab pilot.md @@ -42,3 +42,23 @@ This lab is upstream of robot learning. Image rectangles supply no calibrated XY The public [Hugging Face Observation Lab Space](https://huggingface.co/spaces/Dvidia/observation-lab) reuses our review interface with two authored geometric videos and authored image boxes. It is a synthetic workflow demonstration: no detector, VLM or robot policy runs in the hosted page. Revisions stay in each visitor's browser, with a visible temporary-memory fallback if browser storage or cross-tab locking is unavailable. Visitors can edit, save and export their own review without server writes. The actual detector pipeline is installed and run locally on permitted footage. All sixteen hosted files were read anonymously at [Space commit 8547b78](https://huggingface.co/spaces/Dvidia/observation-lab/tree/8547b78f312e39d9a0b299ebe055c401f8135848) and compared with the staged hashes. The public runtime was observed running with its synthetic scope labels visible. Publication preparation reran 311 repository tests and separately checked 22 publisher tests, six synthetic preparation/privacy/media tests and browser-local revision/conflict behavior. Both pinned ONNX detectors loaded and ran on a neutral image in the local CPU runtime; that smoke test measures neither detection accuracy nor task understanding. [Release source](https://github.com/Dvidia-Inference/dvidia-training/tree/3cc4a3b03ca6d61479c9f6a86cebf7ccc77fd117) preserves the original pilot receipts and clarifies the requested-versus-reported thread discrepancy. Original code uses MIT and the detector adapter retains its Apache-2.0 notices. Production uploads, accounts, storage and application analysis remain separate from this research release. + +## Image-motion source alpha estimates speed and future paths + +On 10 October 2026, the next source alpha adds causal image motion and passive forecasting to our local research tools. [Candidate pull request 7](https://github.com/Dvidia-Inference/dvidia-training/pull/7) and [exact source commit ade2b54](https://github.com/Dvidia-Inference/dvidia-training/tree/ade2b541d8ca4a785bbd1effdfd33eb2a306f150) identify this addition. The published `observation-lab-v0.1.0` release and Hugging Face review demo remain separate; neither includes these motion capabilities. The complete Observation Lab v0.2 remains planned. ([Installation and motion contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md), [next-version plan](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/observation-v0.2-plan.md)) + +The user supplies one first-frame image region and label. Sparse CPU optical flow translates its initial center using robust feature displacement; two contiguous observations estimate velocity, and constant-velocity forecasts project that image anchor 100, 250 and 500 ms ahead. Integer input presentation timestamps multiplied by the declared rational time base preserve the source clock. Initial labels and regions are human-authored, not detector outputs; no new weights are trained or downloaded. Position and speed are analysis-image pixels and pixels/second. The anchor is not a physical centroid, and this fixed-camera baseline has no calibrated metric speed, contact model or robot command authority. A lost track remains lost without interpolation or reacquisition. ([Exact implementation contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md)) + +The frozen authored benchmark includes stationary motion, constant velocity, stop/start, direction change, brief occlusion and low texture. Each generated clip is four seconds at 640 × 360 and 30 fps; the batch contains **six cases, 720 attempted frames and 24 source seconds**. Future authored reference trajectories enter only the scorer, after analysis; they do not enter earlier tracking or forecasts. These six cases share one generator and texture rather than independent real-world task attempts. ([Frozen protocol](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/protocol.json), [aggregate scores and denominators](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/result.json)) + +| Recorded result | Scope | +| --- | --- | +| Six of six pipeline cases completed | Engineering processing, not six successful physical tasks | +| 1.454 seconds processing for 24 source seconds | One offline CPU batch; excludes 1.514 seconds of fixture generation and provisioning | +| Occlusion: 45/100 visible reference frames tracked | Loss remains visible after reappearance; no hidden reacquisition | +| Low texture: 0/120 visible reference frames tracked | All rows abstain; missing predictions are not zero-error answers | +| Direction change: 89.975 px worst endpoint error at 500 ms | A failure tail despite small anchor error and a lower whole-case forecast mean | + +The batch ran once on macOS ARM64 with Python 3.12.9, OpenCV 5.0.0 and NumPy 2.5.3. Processing covers intake, probing, streaming decode, tracking/forecast, writes and scoring; the offline processing/source-duration ratio is 0.0606. One OpenCV thread was requested while its runtime reported ten; actual concurrency and an enforced thread limit remain unmeasured. Fast replay does not establish live capture-to-command delay or a reliable 30 Hz robot loop. The 250 ms forecast falls between source frames and uses a disclosed nearest-frame reference, so it is not an exact-time accuracy result. Heuristic uncertainty envelopes remain uncalibrated. ([Measured run and limitations](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/README.md)) + +Local validation recorded **378 repository tests and six browser workflow checks** for this candidate. These are software/interface checks; no CI completion count, independent real-motion accuracy, live sensing or physical control result follows from them. Original DVIDIA motion, fixture and scoring code uses MIT, while existing upstream detector notices retain their separate terms. Next evidence is independently reviewed permitted motion footage, measured observation-only live streaming and calibrated geometry, followed by synchronized robot actions/state for interaction-conditioned prediction. ([Reviewable candidate and validation](https://github.com/Dvidia-Inference/dvidia-training/pull/7), [motion and interaction gates](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-prediction-plan.md)) diff --git a/docs/feed.xml b/docs/feed.xml index c1e3ee0..0db5aef 100644 --- a/docs/feed.xml +++ b/docs/feed.xml @@ -1,2 +1,2 @@ -DVIDIA Researchhttps://research.dvidia.org/Research footprints on demonstrations, calibrated simulation and robot skills.enDVIDIA Observation Lab pilothttps://research.dvidia.org/papers/dvidia-observation-lab/https://research.dvidia.org/papers/dvidia-observation-lab/Sat, 10 Oct 2026 12:00:00 GMTSix public excerpts processed · accuracy not assessed · no robot actions. DVIDIA Observation Lab supplies a local detector and palm-observation workflow with sampled timestamps, source/model provenance, editable timeline revisions and saved-review export. Two engineering attempts each process the same six public excerpts: 19 seconds and 38 sampled frames. Processing takes 4.835 and 4.635 seconds on one warm macOS ARM64 machine, excluding model setup, UI and human review. Software and browser checks establish integration; independent annotation accuracy, correction time, complete cup-placement coverage, depth, action learning and hardware competence remain unmeasured.Robotics research pipelinehttps://research.dvidia.org/papers/robotics-research-pipeline/https://research.dvidia.org/papers/robotics-research-pipeline/Fri, 09 Oct 2026 12:00:00 GMTArchitecture proposal · current simulation alpha · hardware unqualified. An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.Skillspace footage training pipelinehttps://research.dvidia.org/papers/skillspace-footage-training/https://research.dvidia.org/papers/skillspace-footage-training/Thu, 08 Oct 2026 12:00:00 GMTPublic CPU package · one native scene qualified · simulation-only. The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.Skill capsule acquisition and optimizationhttps://research.dvidia.org/papers/skill-capsule-acquisition/https://research.dvidia.org/papers/skill-capsule-acquisition/Wed, 07 Oct 2026 12:00:00 GMTSimulation-only · teacher/student 7/8 · no overall speedup. A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.Skillspace arm benchmarks and evolutionhttps://research.dvidia.org/papers/skillspace-arm-benchmarks/https://research.dvidia.org/papers/skillspace-arm-benchmarks/Wed, 07 Oct 2026 12:00:00 GMTMixed held-out result · 30/32 baseline, 29/32 updated. Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.Installing a Skillspace on a simulated armhttps://research.dvidia.org/papers/skillspace-arm-installation/https://research.dvidia.org/papers/skillspace-arm-installation/Wed, 07 Oct 2026 12:00:00 GMTPublic skill link to authored adapter · 6/6 scene placements. The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.Offline cable training environmenthttps://research.dvidia.org/papers/offline-cable-training-environment/https://research.dvidia.org/papers/offline-cable-training-environment/Wed, 07 Oct 2026 12:00:00 GMTCPU simulation-only · Controller search and frozen evaluation executed. The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.Fast accurate robot training simulationhttps://research.dvidia.org/papers/fast-accurate-robot-training-simulation/https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/Wed, 07 Oct 2026 12:00:00 GMT54 CPU fixture trials · GPU and physical transfer untested. A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.Grok robotics research on Xhttps://research.dvidia.org/papers/grok-robotics-research-on-x/https://research.dvidia.org/papers/grok-robotics-research-on-x/Wed, 07 Oct 2026 12:00:00 GMTPrimary sources checked · Experiments proposed. Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.Spatially grounded downloadable robot skillshttps://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/Wed, 07 Oct 2026 12:00:00 GMTSource review · DVIDIA experiments not run. A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.Clef and grounded video observationshttps://research.dvidia.org/papers/clef-and-grounded-video-observations/https://research.dvidia.org/papers/clef-and-grounded-video-observations/Tue, 06 Oct 2026 12:00:00 GMTFive licensed excerpts · Accuracy evaluation outstanding. A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.From Skillspace to robot skillhttps://research.dvidia.org/papers/from-skillspace-to-robot-skill/https://research.dvidia.org/papers/from-skillspace-to-robot-skill/Tue, 06 Oct 2026 12:00:00 GMTGraduation and robot training not run. An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.Turn physical footage into testable evidencehttps://research.dvidia.org/papers/physical-skill-research-agenda/https://research.dvidia.org/papers/physical-skill-research-agenda/Tue, 06 Oct 2026 12:00:00 GMTFour baseline protocols proposed. A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct. +DVIDIA Researchhttps://research.dvidia.org/Research footprints on demonstrations, calibrated simulation and robot skills.enDVIDIA Observation Lab pilothttps://research.dvidia.org/papers/dvidia-observation-lab/https://research.dvidia.org/papers/dvidia-observation-lab/Sat, 10 Oct 2026 12:00:00 GMTLocal review release · causal motion source alpha · real accuracy not assessed. DVIDIA Observation Lab supplies local sampled object/palm proposals, provenance, timeline revisions and review export. Two engineering attempts each process six public excerpts: 19 seconds and 38 frames; annotation accuracy remains unmeasured. A separate source alpha adds manually seeded causal image tracking, pixel speed and short-horizon forecasts. Six authored cases cover 720 frames and 24 source seconds, processed in 1.454 seconds on one offline CPU run. Occlusion and low-texture losses remain visible; the worst 500 ms direction-change forecast misses by 89.975 pixels. The v0.1 release and hosted review demo remain separate. Real-scene motion accuracy, calibrated metric speed, live robot latency and hardware competence are unestablished.Robotics research pipelinehttps://research.dvidia.org/papers/robotics-research-pipeline/https://research.dvidia.org/papers/robotics-research-pipeline/Fri, 09 Oct 2026 12:00:00 GMTArchitecture proposal · current simulation alpha · hardware unqualified. An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.Skillspace footage training pipelinehttps://research.dvidia.org/papers/skillspace-footage-training/https://research.dvidia.org/papers/skillspace-footage-training/Thu, 08 Oct 2026 12:00:00 GMTPublic CPU package · one native scene qualified · simulation-only. The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.Skill capsule acquisition and optimizationhttps://research.dvidia.org/papers/skill-capsule-acquisition/https://research.dvidia.org/papers/skill-capsule-acquisition/Wed, 07 Oct 2026 12:00:00 GMTSimulation-only · teacher/student 7/8 · no overall speedup. A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.Skillspace arm benchmarks and evolutionhttps://research.dvidia.org/papers/skillspace-arm-benchmarks/https://research.dvidia.org/papers/skillspace-arm-benchmarks/Wed, 07 Oct 2026 12:00:00 GMTMixed held-out result · 30/32 baseline, 29/32 updated. Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.Installing a Skillspace on a simulated armhttps://research.dvidia.org/papers/skillspace-arm-installation/https://research.dvidia.org/papers/skillspace-arm-installation/Wed, 07 Oct 2026 12:00:00 GMTPublic skill link to authored adapter · 6/6 scene placements. The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.Offline cable training environmenthttps://research.dvidia.org/papers/offline-cable-training-environment/https://research.dvidia.org/papers/offline-cable-training-environment/Wed, 07 Oct 2026 12:00:00 GMTCPU simulation-only · Controller search and frozen evaluation executed. The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.Fast accurate robot training simulationhttps://research.dvidia.org/papers/fast-accurate-robot-training-simulation/https://research.dvidia.org/papers/fast-accurate-robot-training-simulation/Wed, 07 Oct 2026 12:00:00 GMT54 CPU fixture trials · GPU and physical transfer untested. A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.Grok robotics research on Xhttps://research.dvidia.org/papers/grok-robotics-research-on-x/https://research.dvidia.org/papers/grok-robotics-research-on-x/Wed, 07 Oct 2026 12:00:00 GMTPrimary sources checked · Experiments proposed. Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.Spatially grounded downloadable robot skillshttps://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/https://research.dvidia.org/papers/spatially-grounded-downloadable-robot-skills/Wed, 07 Oct 2026 12:00:00 GMTSource review · DVIDIA experiments not run. A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.Clef and grounded video observationshttps://research.dvidia.org/papers/clef-and-grounded-video-observations/https://research.dvidia.org/papers/clef-and-grounded-video-observations/Tue, 06 Oct 2026 12:00:00 GMTFive licensed excerpts · Accuracy evaluation outstanding. A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.From Skillspace to robot skillhttps://research.dvidia.org/papers/from-skillspace-to-robot-skill/https://research.dvidia.org/papers/from-skillspace-to-robot-skill/Tue, 06 Oct 2026 12:00:00 GMTGraduation and robot training not run. An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.Turn physical footage into testable evidencehttps://research.dvidia.org/papers/physical-skill-research-agenda/https://research.dvidia.org/papers/physical-skill-research-agenda/Tue, 06 Oct 2026 12:00:00 GMTFour baseline protocols proposed. A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct. diff --git a/docs/index.html b/docs/index.html index 1e88fbb..0e8be60 100644 --- a/docs/index.html +++ b/docs/index.html @@ -1,4 +1,4 @@ Research journal · DVIDIA Research -

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

13 papers and working notes

· Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

DVIDIA Observation Lab supplies a local detector and palm-observation workflow with sampled timestamps, source/model provenance, editable timeline revisions and saved-review export. Two engineering attempts each process the same six public excerpts: 19 seconds and 38 sampled frames. Processing takes 4.835 and 4.635 seconds on one warm macOS ARM64 machine, excluding model setup, UI and human review. Software and browser checks establish integration; independent annotation accuracy, correction time, complete cup-placement coverage, depth, action learning and hardware competence remain unmeasured.

Six public excerpts processed · accuracy not assessed · no robot actions

· Architecture proposal and staged validation roadmap

Robotics research pipeline

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

Architecture proposal · current simulation alpha · hardware unqualified

· Product implementation and measured synthetic pilot

Skillspace footage training pipeline

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

Public CPU package · one native scene qualified · simulation-only

· Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

Simulation-only · teacher/student 7/8 · no overall speedup

· Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

Mixed held-out result · 30/32 baseline, 29/32 updated

· Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

Public skill link to authored adapter · 6/6 scene placements

· Product prototype and local qualification

Offline cable training environment

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

CPU simulation-only · Controller search and frozen evaluation executed

· Decision report and local experiment

Fast accurate robot training simulation

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

54 CPU fixture trials · GPU and physical transfer untested

· Source review and thesis comparison

Grok robotics research on X

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

Primary sources checked · Experiments proposed

· Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

Source review · DVIDIA experiments not run

· Engineering feasibility pilot

Clef and grounded video observations

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

Five licensed excerpts · Accuracy evaluation outstanding

· Source review and implementation proposal

From Skillspace to robot skill

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

Graduation and robot training not run

· Research agenda

Turn physical footage into testable evidence

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

Four baseline protocols proposed

Research footprints

  1. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  2. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  3. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  4. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  5. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  6. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  7. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

Follow new publications through RSS or inspect the source and revisions on GitHub.

+

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

13 papers and working notes

· Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

DVIDIA Observation Lab supplies local sampled object/palm proposals, provenance, timeline revisions and review export. Two engineering attempts each process six public excerpts: 19 seconds and 38 frames; annotation accuracy remains unmeasured. A separate source alpha adds manually seeded causal image tracking, pixel speed and short-horizon forecasts. Six authored cases cover 720 frames and 24 source seconds, processed in 1.454 seconds on one offline CPU run. Occlusion and low-texture losses remain visible; the worst 500 ms direction-change forecast misses by 89.975 pixels. The v0.1 release and hosted review demo remain separate. Real-scene motion accuracy, calibrated metric speed, live robot latency and hardware competence are unestablished.

Local review release · causal motion source alpha · real accuracy not assessed

· Architecture proposal and staged validation roadmap

Robotics research pipeline

An updated architecture connects immutable Skillspace recordings, reviewed task-label proposals and synchronized measured telemetry to grouped datasets, separate visual and motor learners, and a proposed local state supervisor. The implemented CPU baseline retains authored phases, gripper decisions and recovery around its learned joint-delta head. New annotation and supervisor pilots, fresh frozen confirmation, physics calibration and compatible skill-installation gates define the next milestones. Automatic labeling, learned supervision, human-video action recovery and physical arm qualification remain unestablished; this publication adds no new training or hardware result.

Architecture proposal · current simulation alpha · hardware unqualified

· Product implementation and measured synthetic pilot

Skillspace footage training pipeline

The public Dvidia-Inference/dvidia-training repository provides a standalone CPU pipeline that audits Skillspace media, freezes grouped splits and exports inspectable artifacts. Ten synthetic six-second clips train a visual predictor with a 0.5106-second median pipeline time across three repeats on one macOS host. Aligned native simulator telemetry supplies 4,029 training labels for a learned joint-delta head, evaluated on one held-out recording. Its capsule passes native placement qualification in that recording's exact scene; identical joint targets with open jaws fail. Installation and hardware guides accompany the package. Human-video action recovery, shoe organization, demonstration sufficiency, independent scene generalization and physical robot competence remain unestablished.

Public CPU package · one native scene qualified · simulation-only

· Learned movement artifact and installation prototype

Skill capsule acquisition and optimization

A compact learned CPU movement head replaces joint-target computation inside an authored placement supervisor. Teacher and student both pass 7/8 in a new frozen evaluation, while episode wall time increases from 6.273 to 6.360 seconds. A 39,845-byte JSON capsule imports into the local lab, checks its exact runtime and qualifies the current scene through native placement and a recorded-arm/open-jaw control. Scene changes require a fresh test. One known scene and repeat pass native network denial; perception, complete task learning, other embodiments and hardware remain unqualified.

Simulation-only · teacher/student 7/8 · no overall speedup

· Paired simulation benchmark and product assessment

Skillspace arm benchmarks and evolution

Two controller revisions improve development completion from 14/20 to 17/20, but the frozen paired test declines from 30/32 to 29/32: both pass all 24 normal-strength cases while actuator-stress completion falls from 6/8 to 5/8. Instrumented CPU throughput also declines. We publish the regression, unchanged contact and success rules, exact controller snapshots, richer physical inputs, runtime-bound installation and a proposed staged Skillspace contract. Contact forces and penetration remain uncalibrated; learned perception and physical qualification are open.

Mixed held-out result · 30/32 baseline, 29/32 updated

· Product prototype and native contact qualification

Installing a Skillspace on a simulated arm

The first link-to-arm product path installs an existing DVIDIA placement envelope, binds a local authored controller to one simulated arm, and reuses it in independently grounded scenes. Native jaw contacts carry a free box; six layouts complete, while no-motion and identical joint commands with open jaws each complete zero of six. Installed execution passes process-level offline checks. Task metadata contains no learned policy; perception, cup geometry, full arm collisions and physical qualification remain open.

Public skill link to authored adapter · 6/6 scene placements

· Product prototype and local qualification

Offline cable training environment

The first runnable product loop combines an anchored cable environment, local controller-parameter search, frozen evaluation, offline replay and a manifest-backed skill pack. The selected controller and a fixed feedback baseline both completed 10/10 held-out simulated episodes; zero-force and release controls completed 0/10. A native macOS network-denial run completed 3/3 additional episodes. Ideal endpoint attachment and privileged state bound the result; physical rope calibration, gripping, sensors and GPU throughput remain untested.

CPU simulation-only · Controller search and frozen evaluation executed

· Decision report and local experiment

Fast accurate robot training simulation

A design for an offline, self-hosted training environment that turns demonstrations into calibrated practice and qualified robot skills. We examine contact, deformation, sensor feedback and the performance–accuracy tradeoff, then report 54 deterministic native MuJoCo CPU fixture trials. Fine endpoint agreement did not establish accurate contact transients, and coarse timesteps missed collisions. Minimum GPU memory, rope accuracy and physical transfer remain unmeasured.

54 CPU fixture trials · GPU and physical transfer untested

· Source review and thesis comparison

Grok robotics research on X

Three Grok 4.7 research passes surface observation-to-action, synthetic-practice and reusable-repair ideas. Independent primary-source checks compare those findings with our thesis: observation supplies goals and state, language retrieves a skill, spatial expectations guide attention, and executable feedback with evaluation establishes competence. X posts remain discovery leads. No universal demonstration count or arbitrary-robot transfer follows from the reviewed results.

Primary sources checked · Experiments proposed

· Research synthesis and proposed experiments

Spatially grounded downloadable robot skills

A synthesis of learning from human demonstrations, language-conditioned spatial attention, simulation expansion and robot-specific skill delivery. Spatial priors can accelerate inspection but do not determine contact state or control. The proposed downloadable skill packages a controller, initiation and termination conditions, compatible sensing and hardware, recovery behavior and measured evidence. Separate experiments would test attention, data coverage, feedback and transfer.

Source review · DVIDIA experiments not run

· Engineering feasibility pilot

Clef and grounded video observations

A preregistered image-API feasibility run processed five licensed three-second POV excerpts using four timestamped frames, a Gemma caption and a Clef-flash check. All five completed at a price-derived inference estimate of $0.00140634. Post-run inspection found an object error accepted by the checker and omitted state changes. API and schema compatibility were demonstrated; annotation accuracy and whole-task success were not.

Five licensed excerpts · Accuracy evaluation outstanding

· Source review and implementation proposal

From Skillspace to robot skill

An Argus review and a proposal to connect original recordings, reviewed episodes, immutable datasets, training runs, held-out evaluation and robot-tested releases. The review separates video annotation from action supervision and defines compatibility beyond a robot’s degree-of-freedom count. A proposed 40-episode annotation pilot and evidence-based graduation criteria remain unrun.

Graduation and robot training not run

· Research agenda

Turn physical footage into testable evidence

A research agenda covering visible capture evidence, relations over time, evaluation provenance and physical grounding. Each question has a bounded protocol and a failure condition. The aim is to measure whether structure and physical signals improve a defined task before scaling collection. Strategic opinions, author-reported results and our untested proposals are kept distinct.

Four baseline protocols proposed

Research footprints

  1. Observe image motion and forecast its path

    A manually seeded CPU source alpha records pixel speed and causal forecasts. Six authored cases process 720 frames in 1.454 seconds; occlusion and low-texture losses remain visible, and a 500 ms reversal forecast misses by nearly 90 pixels. The v0.1 release/demo remain separate; real-scene accuracy, metric speed and live robot control are unqualified.

    Read the source alpha and measured limitations
  2. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  3. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  4. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  5. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  6. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  7. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  8. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

Follow new publications through RSS or inspect the source and revisions on GitHub.

diff --git a/docs/papers/dvidia-observation-lab/index.html b/docs/papers/dvidia-observation-lab/index.html index cdb0509..3769a2f 100644 --- a/docs/papers/dvidia-observation-lab/index.html +++ b/docs/papers/dvidia-observation-lab/index.html @@ -1,7 +1,7 @@ -DVIDIA Observation Lab pilot · DVIDIA Research -

Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

· DVIDIA

Six public excerpts processed · accuracy not assessed · no robot actions

Source & history · Download Markdown

Abstract

DVIDIA Observation Lab supplies a local detector and palm-observation workflow with sampled timestamps, source/model provenance, editable timeline revisions and saved-review export. Two engineering attempts each process the same six public excerpts: 19 seconds and 38 sampled frames. Processing takes 4.835 and 4.635 seconds on one warm macOS ARM64 machine, excluding model setup, UI and human review. Software and browser checks establish integration; independent annotation accuracy, correction time, complete cup-placement coverage, depth, action learning and hardware competence remain unmeasured.

In this paper

Engineering integration passed on six public excerpts; annotation accuracy remains unmeasured. On 10 October 2026, we ran a local observation pipeline that decodes video, proposes sampled object and palm boxes, associates nearby detections and opens an editable review timeline. Review changes persist as revisions and export with their source identity. This establishes a working research tool, not reliable automatic task labeling, a cup-placement benchmark or a trained robot skill. The published implementation is anchored to DVIDIA Training commit d355a4a.

+DVIDIA Observation Lab pilot · DVIDIA Research +

Local observation workflow and engineering feasibility pilot

DVIDIA Observation Lab pilot

· DVIDIA

Local review release · causal motion source alpha · real accuracy not assessed

Source & history · Download Markdown

Abstract

DVIDIA Observation Lab supplies local sampled object/palm proposals, provenance, timeline revisions and review export. Two engineering attempts each process six public excerpts: 19 seconds and 38 frames; annotation accuracy remains unmeasured. A separate source alpha adds manually seeded causal image tracking, pixel speed and short-horizon forecasts. Six authored cases cover 720 frames and 24 source seconds, processed in 1.454 seconds on one offline CPU run. Occlusion and low-texture losses remain visible; the worst 500 ms direction-change forecast misses by 89.975 pixels. The v0.1 release and hosted review demo remain separate. Real-scene motion accuracy, calibrated metric speed, live robot latency and hardware competence are unestablished.

In this paper

Engineering integration passed on six public excerpts; annotation accuracy remains unmeasured. On 10 October 2026, we ran a local observation pipeline that decodes video, proposes sampled object and palm boxes, associates nearby detections and opens an editable review timeline. Review changes persist as revisions and export with their source identity. This establishes a working research tool, not reliable automatic task labeling, a cup-placement benchmark or a trained robot skill. The published implementation is anchored to DVIDIA Training commit d355a4a.

Install the Observation Lab release · GitHub release and downloads · Try the synthetic review demo · Frozen engineering receipts

The pilot produces reviewable 2D evidence

The local adapter uses the pinned OpenCV Zoo YOLOX COCO object detector and MediaPipe palm detector. It samples at two frames per second, up to 120 frames and a 640-pixel maximum image side per clip. Proposals retain decoded presentation timestamps, frame and source hashes, raw labels and scores. Same-class image overlap associates detections; palm/object overlap proposes 2D proximity. Neither association establishes physical identity nor overlap proves touching. Palm rectangles are not full-hand segmentation. No optical flow, depth reconstruction, natural-language action captioner or language model participates in this pilot. (Observation contract, frozen model and recipe)

@@ -21,4 +21,11 @@

The release k

This lab is upstream of robot learning. Image rectangles supply no calibrated XYZ, contact forces, joint actions, gripper commands or robot policy. No automatic physical action is produced, and the existing visual predictor and movement head gain no new skill from this release. The next proposed study collects 30–50 complete permission-cleared cup-placement attempts with explicit session/person grouping, failures, occlusion and recovery. Independent blinded references should precede comparisons of detector-only, temporal-caption, combined and manual annotation at matched quality, correction time and full cost. Depth, a supervised action bridge and closed-loop qualification require separate experiments. (Uncollected cohort and gates)

The public Hugging Face Observation Lab Space reuses our review interface with two authored geometric videos and authored image boxes. It is a synthetic workflow demonstration: no detector, VLM or robot policy runs in the hosted page. Revisions stay in each visitor's browser, with a visible temporary-memory fallback if browser storage or cross-tab locking is unavailable. Visitors can edit, save and export their own review without server writes. The actual detector pipeline is installed and run locally on permitted footage. All sixteen hosted files were read anonymously at Space commit 8547b78 and compared with the staged hashes. The public runtime was observed running with its synthetic scope labels visible.

Publication preparation reran 311 repository tests and separately checked 22 publisher tests, six synthetic preparation/privacy/media tests and browser-local revision/conflict behavior. Both pinned ONNX detectors loaded and ran on a neutral image in the local CPU runtime; that smoke test measures neither detection accuracy nor task understanding. Release source preserves the original pilot receipts and clarifies the requested-versus-reported thread discrepancy. Original code uses MIT and the detector adapter retains its Apache-2.0 notices. Production uploads, accounts, storage and application analysis remain separate from this research release.

+

Image-motion source alpha estimates speed and future paths

+

On 10 October 2026, the next source alpha adds causal image motion and passive forecasting to our local research tools. Candidate pull request 7 and exact source commit ade2b54 identify this addition. The published observation-lab-v0.1.0 release and Hugging Face review demo remain separate; neither includes these motion capabilities. The complete Observation Lab v0.2 remains planned. (Installation and motion contract, next-version plan)

+

The user supplies one first-frame image region and label. Sparse CPU optical flow translates its initial center using robust feature displacement; two contiguous observations estimate velocity, and constant-velocity forecasts project that image anchor 100, 250 and 500 ms ahead. Integer input presentation timestamps multiplied by the declared rational time base preserve the source clock. Initial labels and regions are human-authored, not detector outputs; no new weights are trained or downloaded. Position and speed are analysis-image pixels and pixels/second. The anchor is not a physical centroid, and this fixed-camera baseline has no calibrated metric speed, contact model or robot command authority. A lost track remains lost without interpolation or reacquisition. (Exact implementation contract)

+

The frozen authored benchmark includes stationary motion, constant velocity, stop/start, direction change, brief occlusion and low texture. Each generated clip is four seconds at 640 × 360 and 30 fps; the batch contains six cases, 720 attempted frames and 24 source seconds. Future authored reference trajectories enter only the scorer, after analysis; they do not enter earlier tracking or forecasts. These six cases share one generator and texture rather than independent real-world task attempts. (Frozen protocol, aggregate scores and denominators)

+
Recorded resultScope
Six of six pipeline cases completedEngineering processing, not six successful physical tasks
1.454 seconds processing for 24 source secondsOne offline CPU batch; excludes 1.514 seconds of fixture generation and provisioning
Occlusion: 45/100 visible reference frames trackedLoss remains visible after reappearance; no hidden reacquisition
Low texture: 0/120 visible reference frames trackedAll rows abstain; missing predictions are not zero-error answers
Direction change: 89.975 px worst endpoint error at 500 msA failure tail despite small anchor error and a lower whole-case forecast mean
+

The batch ran once on macOS ARM64 with Python 3.12.9, OpenCV 5.0.0 and NumPy 2.5.3. Processing covers intake, probing, streaming decode, tracking/forecast, writes and scoring; the offline processing/source-duration ratio is 0.0606. One OpenCV thread was requested while its runtime reported ten; actual concurrency and an enforced thread limit remain unmeasured. Fast replay does not establish live capture-to-command delay or a reliable 30 Hz robot loop. The 250 ms forecast falls between source frames and uses a disclosed nearest-frame reference, so it is not an exact-time accuracy result. Heuristic uncertainty envelopes remain uncalibrated. (Measured run and limitations)

+

Local validation recorded 378 repository tests and six browser workflow checks for this candidate. These are software/interface checks; no CI completion count, independent real-motion accuracy, live sensing or physical control result follows from them. Original DVIDIA motion, fixture and scoring code uses MIT, while existing upstream detector notices retain their separate terms. Next evidence is independently reviewed permitted motion footage, measured observation-only live streaming and calibrated geometry, followed by synchronized robot actions/state for interaction-conditioned prediction. (Reviewable candidate and validation, motion and interaction gates)

diff --git a/docs/publications.json b/docs/publications.json index bff679c..d1830d4 100644 --- a/docs/publications.json +++ b/docs/publications.json @@ -11,13 +11,13 @@ "source": "reports/DVIDIA Observation Lab pilot.md", "date": "2026-10-10", "type": "Local observation workflow and engineering feasibility pilot", - "status": "Six public excerpts processed · accuracy not assessed · no robot actions", + "status": "Local review release · causal motion source alpha · real accuracy not assessed", "topics": [ "capture-quality", "evaluation-provenance", "physical-grounding" ], - "abstract": "DVIDIA Observation Lab supplies a local detector and palm-observation workflow with sampled timestamps, source/model provenance, editable timeline revisions and saved-review export. Two engineering attempts each process the same six public excerpts: 19 seconds and 38 sampled frames. Processing takes 4.835 and 4.635 seconds on one warm macOS ARM64 machine, excluding model setup, UI and human review. Software and browser checks establish integration; independent annotation accuracy, correction time, complete cup-placement coverage, depth, action learning and hardware competence remain unmeasured." + "abstract": "DVIDIA Observation Lab supplies local sampled object/palm proposals, provenance, timeline revisions and review export. Two engineering attempts each process six public excerpts: 19 seconds and 38 frames; annotation accuracy remains unmeasured. A separate source alpha adds manually seeded causal image tracking, pixel speed and short-horizon forecasts. Six authored cases cover 720 frames and 24 source seconds, processed in 1.454 seconds on one offline CPU run. Occlusion and low-texture losses remain visible; the worst 500 ms direction-change forecast misses by 89.975 pixels. The v0.1 release and hosted review demo remain separate. Real-scene motion accuracy, calibrated metric speed, live robot latency and hardware competence are unestablished." }, { "slug": "robotics-research-pipeline", diff --git a/publications.json b/publications.json index 25bf587..06e1fef 100644 --- a/publications.json +++ b/publications.json @@ -11,13 +11,13 @@ "source": "reports/DVIDIA Observation Lab pilot.md", "date": "2026-10-10", "type": "Local observation workflow and engineering feasibility pilot", - "status": "Six public excerpts processed \u00b7 accuracy not assessed \u00b7 no robot actions", + "status": "Local review release \u00b7 causal motion source alpha \u00b7 real accuracy not assessed", "topics": [ "capture-quality", "evaluation-provenance", "physical-grounding" ], - "abstract": "DVIDIA Observation Lab supplies a local detector and palm-observation workflow with sampled timestamps, source/model provenance, editable timeline revisions and saved-review export. Two engineering attempts each process the same six public excerpts: 19 seconds and 38 sampled frames. Processing takes 4.835 and 4.635 seconds on one warm macOS ARM64 machine, excluding model setup, UI and human review. Software and browser checks establish integration; independent annotation accuracy, correction time, complete cup-placement coverage, depth, action learning and hardware competence remain unmeasured." + "abstract": "DVIDIA Observation Lab supplies local sampled object/palm proposals, provenance, timeline revisions and review export. Two engineering attempts each process six public excerpts: 19 seconds and 38 frames; annotation accuracy remains unmeasured. A separate source alpha adds manually seeded causal image tracking, pixel speed and short-horizon forecasts. Six authored cases cover 720 frames and 24 source seconds, processed in 1.454 seconds on one offline CPU run. Occlusion and low-texture losses remain visible; the worst 500 ms direction-change forecast misses by 89.975 pixels. The v0.1 release and hosted review demo remain separate. Real-scene motion accuracy, calibrated metric speed, live robot latency and hardware competence are unestablished." }, { "slug": "robotics-research-pipeline", diff --git a/reports/DVIDIA Observation Lab pilot.md b/reports/DVIDIA Observation Lab pilot.md index 3a1ed35..9997a86 100644 --- a/reports/DVIDIA Observation Lab pilot.md +++ b/reports/DVIDIA Observation Lab pilot.md @@ -42,3 +42,23 @@ This lab is upstream of robot learning. Image rectangles supply no calibrated XY The public [Hugging Face Observation Lab Space](https://huggingface.co/spaces/Dvidia/observation-lab) reuses our review interface with two authored geometric videos and authored image boxes. It is a synthetic workflow demonstration: no detector, VLM or robot policy runs in the hosted page. Revisions stay in each visitor's browser, with a visible temporary-memory fallback if browser storage or cross-tab locking is unavailable. Visitors can edit, save and export their own review without server writes. The actual detector pipeline is installed and run locally on permitted footage. All sixteen hosted files were read anonymously at [Space commit 8547b78](https://huggingface.co/spaces/Dvidia/observation-lab/tree/8547b78f312e39d9a0b299ebe055c401f8135848) and compared with the staged hashes. The public runtime was observed running with its synthetic scope labels visible. Publication preparation reran 311 repository tests and separately checked 22 publisher tests, six synthetic preparation/privacy/media tests and browser-local revision/conflict behavior. Both pinned ONNX detectors loaded and ran on a neutral image in the local CPU runtime; that smoke test measures neither detection accuracy nor task understanding. [Release source](https://github.com/Dvidia-Inference/dvidia-training/tree/3cc4a3b03ca6d61479c9f6a86cebf7ccc77fd117) preserves the original pilot receipts and clarifies the requested-versus-reported thread discrepancy. Original code uses MIT and the detector adapter retains its Apache-2.0 notices. Production uploads, accounts, storage and application analysis remain separate from this research release. + +## Image-motion source alpha estimates speed and future paths + +On 10 October 2026, the next source alpha adds causal image motion and passive forecasting to our local research tools. [Candidate pull request 7](https://github.com/Dvidia-Inference/dvidia-training/pull/7) and [exact source commit ade2b54](https://github.com/Dvidia-Inference/dvidia-training/tree/ade2b541d8ca4a785bbd1effdfd33eb2a306f150) identify this addition. The published `observation-lab-v0.1.0` release and Hugging Face review demo remain separate; neither includes these motion capabilities. The complete Observation Lab v0.2 remains planned. ([Installation and motion contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md), [next-version plan](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/observation-v0.2-plan.md)) + +The user supplies one first-frame image region and label. Sparse CPU optical flow translates its initial center using robust feature displacement; two contiguous observations estimate velocity, and constant-velocity forecasts project that image anchor 100, 250 and 500 ms ahead. Integer input presentation timestamps multiplied by the declared rational time base preserve the source clock. Initial labels and regions are human-authored, not detector outputs; no new weights are trained or downloaded. Position and speed are analysis-image pixels and pixels/second. The anchor is not a physical centroid, and this fixed-camera baseline has no calibrated metric speed, contact model or robot command authority. A lost track remains lost without interpolation or reacquisition. ([Exact implementation contract](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-observations.md)) + +The frozen authored benchmark includes stationary motion, constant velocity, stop/start, direction change, brief occlusion and low texture. Each generated clip is four seconds at 640 × 360 and 30 fps; the batch contains **six cases, 720 attempted frames and 24 source seconds**. Future authored reference trajectories enter only the scorer, after analysis; they do not enter earlier tracking or forecasts. These six cases share one generator and texture rather than independent real-world task attempts. ([Frozen protocol](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/protocol.json), [aggregate scores and denominators](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/result.json)) + +| Recorded result | Scope | +| --- | --- | +| Six of six pipeline cases completed | Engineering processing, not six successful physical tasks | +| 1.454 seconds processing for 24 source seconds | One offline CPU batch; excludes 1.514 seconds of fixture generation and provisioning | +| Occlusion: 45/100 visible reference frames tracked | Loss remains visible after reappearance; no hidden reacquisition | +| Low texture: 0/120 visible reference frames tracked | All rows abstain; missing predictions are not zero-error answers | +| Direction change: 89.975 px worst endpoint error at 500 ms | A failure tail despite small anchor error and a lower whole-case forecast mean | + +The batch ran once on macOS ARM64 with Python 3.12.9, OpenCV 5.0.0 and NumPy 2.5.3. Processing covers intake, probing, streaming decode, tracking/forecast, writes and scoring; the offline processing/source-duration ratio is 0.0606. One OpenCV thread was requested while its runtime reported ten; actual concurrency and an enforced thread limit remain unmeasured. Fast replay does not establish live capture-to-command delay or a reliable 30 Hz robot loop. The 250 ms forecast falls between source frames and uses a disclosed nearest-frame reference, so it is not an exact-time accuracy result. Heuristic uncertainty envelopes remain uncalibrated. ([Measured run and limitations](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/benchmarks/motion-alpha-20261010/README.md)) + +Local validation recorded **378 repository tests and six browser workflow checks** for this candidate. These are software/interface checks; no CI completion count, independent real-motion accuracy, live sensing or physical control result follows from them. Original DVIDIA motion, fixture and scoring code uses MIT, while existing upstream detector notices retain their separate terms. Next evidence is independently reviewed permitted motion footage, measured observation-only live streaming and calibrated geometry, followed by synchronized robot actions/state for interaction-conditioned prediction. ([Reviewable candidate and validation](https://github.com/Dvidia-Inference/dvidia-training/pull/7), [motion and interaction gates](https://github.com/Dvidia-Inference/dvidia-training/blob/ade2b541d8ca4a785bbd1effdfd33eb2a306f150/docs/motion-prediction-plan.md)) diff --git a/scripts/build-papers.mjs b/scripts/build-papers.mjs index d462794..652c985 100644 --- a/scripts/build-papers.mjs +++ b/scripts/build-papers.mjs @@ -186,7 +186,7 @@ export async function buildPublications() { pages.push({ path: page, date: papers.reduce((latest, paper) => paper.date > latest ? paper.date : latest, topic.updated) }); } const paperRows = catalog.publications.map(p => `

· ${escapeHtml(p.type)}

${escapeHtml(p.title)}

${escapeHtml(p.abstract)}

${escapeHtml(p.status)}

`).join(""); - const content = `

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

${catalog.publications.length} papers and working notes

${paperRows}

Research footprints

  1. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  2. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  3. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  4. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  5. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  6. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  7. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

Follow new publications through RSS or inspect the source and revisions on GitHub.

`; + const content = `

An open research notebook

From observation
to physical skill.

We study how demonstrations become usable evidence, how simulation becomes useful practice, and when a robot skill earns a release.

Published by DVIDIA · hello@dvidia.org

Publications

${catalog.publications.length} papers and working notes

${paperRows}

Research footprints

  1. Observe image motion and forecast its path

    A manually seeded CPU source alpha records pixel speed and causal forecasts. Six authored cases process 720 frames in 1.454 seconds; occlusion and low-texture losses remain visible, and a 500 ms reversal forecast misses by nearly 90 pixels. The v0.1 release/demo remain separate; real-scene accuracy, metric speed and live robot control are unqualified.

    Read the source alpha and measured limitations
  2. Publish the robotics research pipeline

    A diagram connects footage review, measured robot state, local training, frozen simulation evaluation and compatible skill installation. Automatic annotation and a learned task supervisor are proposed; their companion validation studies and physical robot qualification remain unrun.

    Read the architecture and validation gates
  3. A public skill link reaches a simulated arm

    One-click local installation binds a DVIDIA placement envelope to an authored contact controller. Six scene layouts pass; identical joint commands with open jaws and no-motion controls each pass 0/6. Physical transfer and learned policies remain open.

    Read the installation contract and arm replay
  4. The offline cable environment is runnable

    A local controller-training loop and simulation-only skill pack now run on CPU. Trained and baseline feedback each pass 10/10 simulated seeds; zero force and early release each pass 0/10. Real gripping, calibration and GPU qualification remain open.

    Read the first product build and replay
  5. Contact accuracy meets runtime

    54 deterministic CPU fixture trials recorded. Endpoint agreement coexists with unqualified force transients; coarse wall tests expose missed contact. No GPU or physical-transfer claim.

    Read the measurements
  6. Compare the observation-to-skill thesis with prior work

    Spatial attention, demonstration coverage, executable simulation and reusable repairs are reviewed separately. Three parallel Grok 4.7 searches support discovery; primary sources support the claims.

    Read the comparison
  7. Five licensed POV excerpts processed

    Ten image calls completed. A checker accepted an object error and captions omitted visible state changes. The separate accuracy pilot remains unrun.

    Read the feasibility run
  8. Define the evidence and release questions

    Four baseline protocols and a Skillspace graduation proposal establish the questions to test. A proposed protocol is distinct from an executed experiment.

    Read the agenda

Open questions

Follow new publications through RSS or inspect the source and revisions on GitHub.

`; await writePage("index.html", layout({ page: "index.html", title: "Research journal", description: "An open notebook on demonstrations, physical grounding, calibrated simulation and qualified robot skills.", content, catalog, topics, active: "index" })); pages.unshift({ path: "index.html", date: catalog.publications.reduce((latest, paper) => paper.date > latest ? paper.date : latest, "2026-10-07") }); const rss = `\nDVIDIA Research${ORIGIN}/Research footprints on demonstrations, calibrated simulation and robot skills.en${catalog.publications.map(p => { const url = `${ORIGIN}/papers/${p.slug}/`; return `${escapeHtml(p.title)}${url}${url}${new Date(`${p.date}T12:00:00Z`).toUTCString()}${escapeHtml(`${p.status}. ${p.abstract}`)}`; }).join("")}\n`;