From 7883c5e1ead3821e491b87dc4f06280414fd5f21 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Sat, 17 Oct 2026 18:11:02 +0100 Subject: [PATCH 1/3] docs: link /prm close-outs and refresh receipts after 2026-10-17 merges Record the human /prm of 2026-10-17 across the affected campaigns and tasks: PRs merged, issues closed, and the matching Mind completion records. Live ingest refreshes the lens and fit receipts (lens at autolens_profiling c40e33c8, 112 records). last_checkin unchanged; this is a ledger link pass, not a check-in. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01QBeyBLN7ntDChx9v1fmt6M --- campaigns.yaml | 141 ++++++++++-------- checkins/2026-10-17.md | 10 ++ dashboard.html | 4 +- dashboard.md | 24 +-- receipts/fit.json | 2 +- receipts/lens.json | 2 +- tasks/delaunay_nn_cpu_imaging.md | 6 + tasks/imaging_production_over_sampling.md | 7 + tasks/interferometer_large_mask_aris.md | 8 + ...ometer_nnls_memo_scattered_stream_guard.md | 6 + tasks/interferometer_streaming_scaling.md | 6 + .../interferometer_w_tilde_fft_size_levers.md | 6 + .../point_source_image_plane_gpu_breakdown.md | 6 + ...t_source_source_plane_chi_squared_speed.md | 6 + tasks/runtime_cell_single_jit_gpu_warmup.md | 6 + 15 files changed, 163 insertions(+), 77 deletions(-) diff --git a/campaigns.yaml b/campaigns.yaml index acdac10..f424857 100644 --- a/campaigns.yaml +++ b/campaigns.yaml @@ -27,22 +27,25 @@ campaigns: title: Point-source image plane · A100 status: parked updated: '2026-10-17' - next: 'Parked as research, human decision 2026-10-17 (chat) ("just research to file"). Reference measurement autolens_profiling#434 - (issue #427, OPEN): job 412519 euclid-ral-gpu-2 exclusive A100, jax 0.10.2, fp64, 2026.10.17.1 private clone: - single call 0.934 ms (pre-#768 0.910), vmap-256 0.0125 ms (0.0120), compile 6.54 s; forward gradient finite 16/16, - equal to reverse to 1.6e-11; logL bit-exact; +2.7% not attributed (different node/date). A100 per-step breakdown - rows unusable. Phase-2 inference fit measurement dropped; no baseline admitted.' + next: 'Parked as research, human decision 2026-10-17 (chat) ("just research to file"). Reference measurement + autolens_profiling#434 (issue #427 closed) merged 2026-10-17 (human /prm): job 412519 euclid-ral-gpu-2 exclusive + A100, jax 0.10.2, fp64, 2026.10.17.1 private clone: single call 0.934 ms (pre-#768 0.910), vmap-256 0.0125 ms + (0.0120), compile 6.54 s; forward gradient finite 16/16, equal to reverse to 1.6e-11; logL bit-exact; +2.7% not + attributed (different node/date). A100 per-step breakdown rows unusable. Phase-2 inference fit measurement + dropped; no baseline admitted. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/point-source-image-gpu-admission.md.' evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/point_source_gpu_breakdown.md - id: point-source-source title: Point-source source plane status: complete updated: '2026-10-17' next: 'Closed as research, human decision 2026-10-17 (chat): source-plane speed-ups "not worth pursuing again". - Evidence: autolens_inference#29 (issue #25) + autolens_profiling#432 (issue #428), both OPEN at PR-open: MultiStartAdam - forward gradients, 48 starts, 5 seeds, 2026.10.17.1 wheels (jax 0.11.2), fp64; likelihood+gradient ≤3.7% of fit - wall, A100 slower end to end than 8 CPU cores; default 300 steps stops 4/5 seeds on the limit, 3000 converges. - Runtime cell medians A100 0.233 ms / RAL CPU 0.147 ms. Blackjax stays parked; Adam step limit recorded as a PyAutoFit - search-settings note.' + Evidence: autolens_inference#29 (issue #25) + autolens_profiling#432 (issue #428), both merged 2026-10-17 (human + /prm), issues closed: MultiStartAdam forward gradients, 48 starts, 5 seeds, 2026.10.17.1 wheels (jax 0.11.2), + fp64; likelihood+gradient ≤3.7% of fit wall, A100 slower end to end than 8 CPU cores; default 300 steps stops 4/5 + seeds on the limit, 3000 converges. Runtime cell medians A100 0.233 ms / RAL CPU 0.147 ms. Blackjax stays parked; + Adam step limit recorded as a PyAutoFit search-settings note. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ps-source-gradient-admission.md.' evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/point_source_source_plane.md - id: cluster-pointsolver title: Cluster PointSolver robustness and performance @@ -58,16 +61,18 @@ campaigns: title: Interferometer likelihood and streaming status: parked updated: '2026-10-17' - next: 'Evening 2026-10-17: human decision 2026-10-17 (chat): interferometer CPU work is exhaustive and closed; curvature - preload parked ("we will come back to it soon"); GPU levers "file wiki research but not make source code changes - for now", so no library prompt for JAX FFT pruning. Parked at knowledge-complete. Open PRs (human''s to merge - or close): L2 symmetric direct_conv PyAutoArray#636 (pending-release; real Aris RAL 1.49×/1.53×/1.58× at 1,459/2,577/5,746 - vertices, logL to round-off; decision record direct-conv-symmetric-gate) + record autolens_profiling#425; A100 - FFT levers #433 (pruned FFT 1.08×/1.18×/1.41× whole likelihood at alma/alma_high/jvla; pixel-scale coarsening - fails the 0.5-nat bar); streaming 1e8 CPU/A100 + memo after-measurement #435. Route-constant refit (RAL FFT/full - direct_conv 0.68 vs 0.89 predicted) deferred. Merge #425 after #636. Six open autolens_profiling PRs (#425, #431, - #432, #433, #434, #435) all regenerate the shared catalogue/dashboard: each merge after the first needs a rebase - + re-render; suggested order #425, #432, #431, #433, #434, #435. #425 after PyAutoArray#636.' + next: 'Evening 2026-10-17: human decision 2026-10-17 (chat): interferometer CPU work is exhaustive and closed; + curvature preload parked ("we will come back to it soon"); GPU levers "file wiki research but not make source code + changes for now", so no library prompt for JAX FFT pruning. Parked at knowledge-complete. All merged 2026-10-17 + (human /prm), pending release for the library PRs: L2 symmetric direct_conv PyAutoArray#636 (issue #635 closed, + pending-release; real Aris RAL 1.49×/1.53×/1.58× at 1,459/2,577/5,746 vertices, logL to round-off; decision record + direct-conv-symmetric-gate) + record autolens_profiling#425; A100 FFT levers #433 (issue #429 closed) (pruned FFT + 1.08×/1.18×/1.41× whole likelihood at alma/alma_high/jvla; pixel-scale coarsening fails the 0.5-nat bar); + streaming 1e8 CPU/A100 + memo after-measurement #435 (issues #368, #426 closed). Route-constant refit (RAL + FFT/full direct_conv 0.68 vs 0.89 predicted) deferred. Merged in order #636, #425, #432, #431, #433, #434, #435. + Records: https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/direct-conv-symmetric.md, + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ral-route-constants-fft-levers.md, + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/interferometer-streaming-memo-after.md.' evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/fd29902bd09f/wiki/campaigns/interferometer_likelihood.md - id: linear-solver title: Linear-solver accuracy and cost @@ -84,22 +89,24 @@ campaigns: title: Production imaging and likelihood breakdown status: active updated: '2026-10-17' - next: 'Production over-sampling phase 1 at PR-open: autolens_profiling#431 (issue #430, OPEN, CPU smoke 8/8): --over-sampling - {flat1,production} on the 4 JAX Delaunay/DelaunayNN cells, euclid/hst presets. Findings: HST flat-1 adapt_split - pin fails on unmodified main (29155.000 vs 29140.296); numba cells'' production threshold never reaches sub 4 - for HST; rectangular JAX cells uncovered. Phase 2 (A100 production re-measure) needs-decision (human decision - 2026-10-17 (chat): undecided). DelaunayNN imaging moved to the parked delaunay-nn-cpu campaign. Post-certified, - quick-update plotting and other tasks unchanged.' + next: 'Production over-sampling phase 1 merged 2026-10-17 (human /prm): autolens_profiling#431 (issue #430 closed, + CPU smoke 8/8): --over-sampling {flat1,production} on the 4 JAX Delaunay/DelaunayNN cells, euclid/hst presets. + Findings: HST flat-1 adapt_split pin fails on unmodified main (29155.000 vs 29140.296); numba cells'' production + threshold never reaches sub 4 for HST; rectangular JAX cells uncovered. Phase 2 (A100 production re-measure) + needs-decision (human decision 2026-10-17 (chat): undecided). DelaunayNN imaging moved to the parked + delaunay-nn-cpu campaign. Post-certified, quick-update plotting and other tasks unchanged. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/imaging-production-over-sampling.md.' evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/4db4d988e520a82219152743748c3188d485a4ba/wiki/campaigns/post_certified_breakdown.md - id: delaunay-nn-cpu title: DelaunayNN (Sibson) mesh at Delaunay cost on the numba/NumPy CPU path status: parked updated: '2026-10-17' - next: 'Created parked, human decision 2026-10-17 (chat): "file in research to come back to in the future". PyAutoArray#637 - (issue #629, OPEN, pending-release, CI 3 legs green): RAL compute-10-4 exclusive 1 thread A-B-B-A ×2, 1,030 vertices - / 15,380 px: NN sub4 0.569→0.392 s (2.20× Delaunay, was 3.00×), sub2 0.235→0.189 s (1.75×); logL bit-identical; - fixes the size-0 Sibson NaN case. ~1.5× target not met; remaining cost is Sibson (~0.16 s) + wider curvature rows. - Target and two hazards recorded as future work, not filed.' + next: 'Created parked, human decision 2026-10-17 (chat): "file in research to come back to in the future". + PyAutoArray#637 (issue #629 closed) merged 2026-10-17 (human /prm), pending-release: RAL compute-10-4 exclusive 1 + thread A-B-B-A ×2, 1,030 vertices / 15,380 px: NN sub4 0.569→0.392 s (2.20× Delaunay, was 3.00×), sub2 0.235→0.189 + s (1.75×); logL bit-identical; fixes the size-0 Sibson NaN case. ~1.5× target not met; remaining cost is Sibson + (~0.16 s) + wider curvature rows. Target and two hazards recorded as future work, not filed. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/delaunaynn-wide-row-overhead.md.' evidence: https://github.com/PyAutoLabs/PyAutoArray/pull/637 - id: measurement-tools title: Measurement reliability and profiling tools @@ -176,8 +183,10 @@ tasks: priority: low path: tasks/runtime_cell_single_jit_gpu_warmup.md updated: '2026-10-17' - next: 'Fresh medians at PR-open, autolens_profiling#432 (OPEN), 2026.10.17.1 wheels: A100 0.233 ms (single_jit 0.347), - RAL CPU 0.147 ms (0.177); pins pass. Remaining release-sweep cells gain the field at the next full sweep.' + next: 'Fresh medians in autolens_profiling#432, merged 2026-10-17 (human /prm), 2026.10.17.1 wheels: A100 0.233 ms + (single_jit 0.347), RAL CPU 0.147 ms (0.177); pins pass. Remaining release-sweep cells gain the field at the next + full sweep. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ps-source-gradient-admission.md.' issue: https://github.com/PyAutoLabs/autolens_profiling/issues/371 evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/719459fefb40c7e5be157f652818dbf923830557/scripts/point_source_source/source_plane/likelihood_runtime_solved.py - id: timing_noise_audit @@ -298,10 +307,11 @@ tasks: priority: normal path: tasks/imaging_production_over_sampling.md updated: '2026-10-17' - next: 'Phase 1 at PR-open: autolens_profiling#431 (issue #430, OPEN, CPU smoke 8/8). Phase 2 A100 production re-measure - needs-decision (human decision 2026-10-17 (chat): undecided); requires #431 merged and compute authorization. - Findings: HST flat-1 adapt_split pin fails on main; numba HST threshold never reaches sub 4; rectangular JAX cells - uncovered.' + next: 'Phase 1 merged 2026-10-17 (human /prm): autolens_profiling#431 (issue #430 closed, CPU smoke 8/8). Phase 2 + A100 production re-measure needs-decision (human decision 2026-10-17 (chat): undecided); requires compute + authorization (#431 now merged). Findings: HST flat-1 adapt_split pin fails on main; numba HST threshold never + reaches sub 4; rectangular JAX cells uncovered. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/imaging-production-over-sampling.md.' evidence: https://github.com/PyAutoLabs/PyAutoArray/pull/606 - id: interferometer_decision_matrix_last_cell campaign: interferometer @@ -333,11 +343,12 @@ tasks: priority: medium path: tasks/interferometer_nnls_memo_scattered_stream_guard.md updated: '2026-10-17' - next: 'After-measurement at PR-open autolens_profiling#435 (issue #426, OPEN; job 412526, ral compute-10-2, load - ~10, 1 thread, 6 rounds, paired bootstrap 90% CI): Delaunay iid #332 flags 1.918 [1.887,1.925] (2.17 in #332); - 15 timed solves 1.018 [1.013,1.023]; Delaunay walk 0.46; rect iid 0.887/0.987; rect walk 0.15; FoM identical. - Not evidence #613 is fixed. Research owed: Nautilus-replay witness, varied-seed rounds. Parked, human decision - 2026-10-17 (chat).' + next: 'After-measurement autolens_profiling#435 merged 2026-10-17 (human /prm) (issue #426 closed; job 412526, ral + compute-10-2, load ~10, 1 thread, 6 rounds, paired bootstrap 90% CI): Delaunay iid #332 flags 1.918 [1.887,1.925] + (2.17 in #332); 15 timed solves 1.018 [1.013,1.023]; Delaunay walk 0.46; rect iid 0.887/0.987; rect walk 0.15; FoM + identical. Not evidence #613 is fixed. Research owed: Nautilus-replay witness, varied-seed rounds. Parked, human + decision 2026-10-17 (chat). Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/interferometer-streaming-memo-after.md.' issue: https://github.com/PyAutoLabs/PyAutoArray/issues/613 evidence: https://github.com/PyAutoLabs/PyAutoArray/pull/615 - id: interferometer_streaming_scaling @@ -347,10 +358,11 @@ tasks: priority: high path: tasks/interferometer_streaming_scaling.md updated: '2026-10-17' - next: 'Phase 2 at PR-open autolens_profiling#435 (issue #368 reopened, OPEN): from_stream chunk 65536 fp64, 1e8 - vis RAL CPU 707.5 s / 1.78 GB RSS, A100 157.3 s / 1.48 GB device; compile ~3 s constant. In-memory fails at 2.5e5 - (RAL 10 GB cap) and 4e6 (A100); uncapped CPU ~7.5 kB/vis. Optional 2e8 rows are research; library candidate (chunk - transformer.image_from) unfiled per human decision 2026-10-17 (chat). Parked.' + next: 'Phase 2 autolens_profiling#435 merged 2026-10-17 (human /prm) (issue #368 closed): from_stream chunk 65536 + fp64, 1e8 vis RAL CPU 707.5 s / 1.78 GB RSS, A100 157.3 s / 1.48 GB device; compile ~3 s constant. In-memory fails + at 2.5e5 (RAL 10 GB cap) and 4e6 (A100); uncapped CPU ~7.5 kB/vis. Optional 2e8 rows are research; library + candidate (chunk transformer.image_from) unfiled per human decision 2026-10-17 (chat). Parked. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/interferometer-streaming-memo-after.md.' issue: https://github.com/PyAutoLabs/autolens_profiling/issues/368 evidence: https://github.com/PyAutoLabs/autolens_profiling/blob/719459fefb40c7e5be157f652818dbf923830557/wiki/campaigns/interferometer_streaming.md - id: interferometer_large_mask_aris @@ -361,10 +373,12 @@ tasks: path: tasks/interferometer_large_mask_aris.md updated: '2026-10-17' next: 'L2 shipped to PR: PyAutoArray#636 (head 62a1e3ac, pending-release) + autolens_profiling#425 (results note), - both OPEN, CI green. RAL real-data witness 1.49×/1.53×/1.58× at 1,459/2,577/5,746 vertices (pred 1.48×/1.50×/1.53×), - logL to round-off; peak RSS 43 GB without NUFFT chunking. Decision record direct-conv-symmetric-gate added. Route-constant - refit (0.68 measured vs 0.89 predicted) deferred: CPU lane closed, human decision 2026-10-17 (chat). Remaining: - human merge of #636 then #425, and release.' + both merged 2026-10-17 (human /prm); issue #635 closed. RAL real-data witness 1.49×/1.53×/1.58× at + 1,459/2,577/5,746 vertices (pred 1.48×/1.50×/1.53×), logL to round-off; peak RSS 43 GB without NUFFT chunking. + Decision record direct-conv-symmetric-gate added. Route-constant refit (0.68 measured vs 0.89 predicted) deferred: + CPU lane closed, human decision 2026-10-17 (chat). Remaining: release of #636; shadow row owed in Mind + autonomy_log (glance tier; human to confirm merged-unchanged). Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/direct-conv-symmetric.md.' issue: https://github.com/PyAutoLabs/PyAutoPulse/issues/62 evidence: https://github.com/PyAutoLabs/PyAutoArray/issues/626 - id: interferometer_w_tilde_fft_size_levers @@ -375,9 +389,10 @@ tasks: priority: normal path: tasks/interferometer_w_tilde_fft_size_levers.md updated: '2026-10-17' - next: 'A100 research filed at PR-open autolens_profiling#433 (issue #429, OPEN; job 412515): pruned padded FFT 1.08×/1.18×/1.41× - whole likelihood at alma/alma_high/jvla, F exact to 1.7e-15; pixel-scale coarsening shifts log evidence by +1,310 - to +2,757 nats (fails). No library prompt, human decision 2026-10-17 (chat). Parked.' + next: 'A100 research autolens_profiling#433 merged 2026-10-17 (human /prm) (issue #429 closed; job 412515): pruned + padded FFT 1.08×/1.18×/1.41× whole likelihood at alma/alma_high/jvla, F exact to 1.7e-15; pixel-scale coarsening + shifts log evidence by +1,310 to +2,757 nats (fails). No library prompt, human decision 2026-10-17 (chat). Parked. + Records: https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ral-route-constants-fft-levers.md.' - id: point_solver_profiling_cells campaign: cluster-pointsolver title: 'PointSolver profiling cells: lensed quasar → cluster runtime tier → single/multi-source → multiplane' @@ -394,8 +409,10 @@ tasks: priority: high path: tasks/point_source_image_plane_gpu_breakdown.md updated: '2026-10-17' - next: 'Parked, human decision 2026-10-17 (chat). Reference A100 measurement autolens_profiling#434 (OPEN, job 412519): - 0.934 ms single call, 0.0125 ms at vmap-256, forward gradient fixed; phase-2 inference fit measurement dropped.' + next: 'Parked, human decision 2026-10-17 (chat). Reference A100 measurement autolens_profiling#434 (merged + 2026-10-17 (human /prm), issue #427 closed, job 412519): 0.934 ms single call, 0.0125 ms at vmap-256, forward + gradient fixed; phase-2 inference fit measurement dropped. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/point-source-image-gpu-admission.md.' issue: https://github.com/PyAutoLabs/PyAutoLens/issues/767 evidence: https://github.com/PyAutoLabs/PyAutoLens/pull/768 - id: point_source_source_plane_chi_squared_speed @@ -405,9 +422,10 @@ tasks: priority: low path: tasks/point_source_source_plane_chi_squared_speed.md updated: '2026-10-17' - next: 'Complete as research, human decision 2026-10-17 (chat). Admission evidence autolens_inference#29 + autolens_profiling#432 - (OPEN): likelihood+gradient ≤3.7% of MultiStartAdam fit wall, so a likelihood speed-up caps at a few percent. - Blackjax stays parked.' + next: 'Complete as research, human decision 2026-10-17 (chat). Admission evidence autolens_inference#29 + + autolens_profiling#432 (merged 2026-10-17 (human /prm); issues #25, #428 closed): likelihood+gradient ≤3.7% of + MultiStartAdam fit wall, so a likelihood speed-up caps at a few percent. Blackjax stays parked. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ps-source-gradient-admission.md.' evidence: https://github.com/PyAutoLabs/autolens_inference/pull/17 - id: pointsolver_cpu_speed_campaign_remainder campaign: point-source-image-cpu @@ -476,8 +494,9 @@ tasks: priority: normal path: tasks/delaunay_nn_cpu_imaging.md updated: '2026-10-17' - next: 'Parked, human decision 2026-10-17 (chat). PyAutoArray#637 (OPEN, pending-release) at PR-open: sub4 2.20× - Delaunay (was 3.00×), sub2 1.75×; logL bit-identical. Future (unfiled): ~1.5× target; hazards sub_slim_indexes_for_pix_index - padding and Sibson rows with mid-row gaps.' + next: 'Parked, human decision 2026-10-17 (chat). PyAutoArray#637 merged 2026-10-17 (human /prm) (issue #629 + closed, pending-release): sub4 2.20× Delaunay (was 3.00×), sub2 1.75×; logL bit-identical. Future (unfiled): ~1.5× + target; hazards sub_slim_indexes_for_pix_index padding and Sibson rows with mid-row gaps. Records: + https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/delaunaynn-wide-row-overhead.md.' issue: https://github.com/PyAutoLabs/PyAutoArray/issues/629 evidence: https://github.com/PyAutoLabs/PyAutoArray/pull/637 diff --git a/checkins/2026-10-17.md b/checkins/2026-10-17.md index e373f58..39692d1 100644 --- a/checkins/2026-10-17.md +++ b/checkins/2026-10-17.md @@ -88,3 +88,13 @@ and independent; merge #425 after #636. - `/mnt/ral/jnightin/scratch/fft_levers_stack.tgz`, `fft_levers_jobs/`, `fft_levers_results/` (results JSON is committed in #433) - `/mnt/ral/jnightin/profiling/autolens_profiling_wt/ps-image-admission_stale_*` - Worker private clones under `profiling/` and `inference/` are kept for reuse. + +## Post-/prm + +The human ran `/prm` on 2026-10-17 after this review; every PR above merged. + +- **Merged:** PyAutoArray [#636](https://github.com/PyAutoLabs/PyAutoArray/pull/636) and [#637](https://github.com/PyAutoLabs/PyAutoArray/pull/637) (both pending-release); autolens_profiling [#425](https://github.com/PyAutoLabs/autolens_profiling/pull/425), [#432](https://github.com/PyAutoLabs/autolens_profiling/pull/432), [#431](https://github.com/PyAutoLabs/autolens_profiling/pull/431), [#433](https://github.com/PyAutoLabs/autolens_profiling/pull/433), [#434](https://github.com/PyAutoLabs/autolens_profiling/pull/434), [#435](https://github.com/PyAutoLabs/autolens_profiling/pull/435) (main now c40e33c8, the merge of #435); autolens_inference [#29](https://github.com/PyAutoLabs/autolens_inference/pull/29); PyAutoPulse [#66](https://github.com/PyAutoLabs/PyAutoPulse/pull/66) and [#67](https://github.com/PyAutoLabs/PyAutoPulse/pull/67). +- **Issues closed:** PyAutoArray #635, #629; autolens_profiling #427, #428, #429, #430, #368, #426; autolens_inference #25. +- **Mind completion records:** [`direct-conv-symmetric`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/direct-conv-symmetric.md), [`delaunaynn-wide-row-overhead`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/delaunaynn-wide-row-overhead.md), [`ps-source-gradient-admission`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ps-source-gradient-admission.md), [`imaging-production-over-sampling`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/imaging-production-over-sampling.md), [`point-source-image-gpu-admission`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/point-source-image-gpu-admission.md), [`ral-route-constants-fft-levers`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/ral-route-constants-fft-levers.md), [`interferometer-streaming-memo-after`](https://github.com/PyAutoLabs/PyAutoMind/blob/main/complete/2026/10/interferometer-streaming-memo-after.md). +- **Owed:** shadow row in Mind `autonomy_log` for the Aris L2 work (glance tier; human to confirm merged-unchanged). +- Library speed-ups (#636, #637) are still unreleased; the figures above remain PR-time measurements until a release carries them. This section links close-outs and does not count as a check-in. diff --git a/dashboard.html b/dashboard.html index ba5e83a..fa12821 100644 --- a/dashboard.html +++ b/dashboard.html @@ -645,8 +645,8 @@ Work on GitHub: - PyAutoPulse: https://github.com/PyAutoLabs/PyAutoPulse - autolens_profiling: https://github.com/PyAutoLabs/autolens_profiling -- autofit_profiling: https://github.com/PyAutoLabs/autofit_profiling
Last updated 2026-10-17 17:05:42 UTC
↻ Update

-

Active campaigns

CampaignStatusLinks
Setup baseline collection and scientific acceptance
2 tasks · next step

Review the Pulse collection choices (candidate input hashes and 31-worker resource plan verified without execution) alongside project correctness audit #417. Resolve data origin/suitability, named host/stack/full settings, witness/tolerances and background-load method. No freeze, compute or acceptance authorized. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.

Reviewed 2026-10-17

Active tasks

Freeze the setup baseline specification, then collect and review evidence↗

needs-decision · high

HST/MGE raw fixed-parameter CPU runner merged in #416; Pulse handoff #55 merged and Mind implementation closed. Resolve exact production host/stack/data, full settings, independent correctness witness/tolerances and background-load method. No collection, campaign freeze or acceptance authorized; global compile dispatch remains unavailable.

Prepare and review the HST/MGE fp64 CPU pilot↗

needs-decision · high

Review the Pulse collection choices (candidate input hashes and 31-worker resource plan verified without execution) alongside project correctness audit #417. Resolve data origin/suitability, named host/stack/full settings, witness/tolerances and background-load method. No freeze, compute or acceptance authorized.

needs-decision
↗☷
Point-source image plane · CPU
1 task · next step

No new committed campaign measurements. Phases 1–4 remain shipped; epic closure is a human decision. Carried leftovers remain pending. RAL queue empty at review, but transfer preview timed out: cleanup folders and unpulled results unverified. Select one owed cleanup/measurement phase only after review. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement. RAL queue holds only euclid_dr1 jobs; cleanup folders and unpulled results still unverified.

Reviewed 2026-10-17

Active tasks

active
↗☷
Point-source image plane · A100
1 task · next step

Parked as research, human decision 2026-10-17 (chat) ("just research to file"). Reference measurement autolens_profiling#434 (issue #427, OPEN): job 412519 euclid-ral-gpu-2 exclusive A100, jax 0.10.2, fp64, 2026.10.17.1 private clone: single call 0.934 ms (pre-#768 0.910), vmap-256 0.0125 ms (0.0120), compile 6.54 s; forward gradient finite 16/16, equal to reverse to 1.6e-11; logL bit-exact; +2.7% not attributed (different node/date). A100 per-step breakdown rows unusable. Phase-2 inference fit measurement dropped; no baseline admitted.

Reviewed 2026-10-17

Active tasks

parked
↗☷
Cluster PointSolver robustness and performance
2 tasks · next step

No campaign wiki/results change or new matching implementation claim found. Representative cluster baseline and observable-overflow policy remain unstarted. Choose one bounded phase; inherited single-source results do not qualify a representative cluster baseline. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.

Reviewed 2026-10-17

Active tasks

active
↗☷
Interferometer likelihood and streaming
5 tasks · next step

Evening 2026-10-17: human decision 2026-10-17 (chat): interferometer CPU work is exhaustive and closed; curvature preload parked ("we will come back to it soon"); GPU levers "file wiki research but not make source code changes for now", so no library prompt for JAX FFT pruning. Parked at knowledge-complete. Open PRs (human's to merge or close): L2 symmetric direct_conv PyAutoArray#636 (pending-release; real Aris RAL 1.49×/1.53×/1.58× at 1,459/2,577/5,746 vertices, logL to round-off; decision record direct-conv-symmetric-gate) + record autolens_profiling#425; A100 FFT levers #433 (pruned FFT 1.08×/1.18×/1.41× whole likelihood at alma/alma_high/jvla; pixel-scale coarsening fails the 0.5-nat bar); streaming 1e8 CPU/A100 + memo after-measurement #435. Route-constant refit (RAL FFT/full direct_conv 0.68 vs 0.89 predicted) deferred. Merge #425 after #636. Six open autolens_profiling PRs (#425, #431, #432, #433, #434, #435) all regenerate the shared catalogue/dashboard: each merge after the first needs a rebase + re-render; suggested order #425, #432, #431, #433, #434, #435. #425 after PyAutoArray#636.

Reviewed 2026-10-17

Active tasks

fnnls warm-start memo: stop it slowing scattered evaluation streams (interferometer CPU, autolens_profiling#332)↗

parked · medium

After-measurement at PR-open autolens_profiling#435 (issue #426, OPEN; job 412526, ral compute-10-2, load ~10, 1 thread, 6 rounds, paired bootstrap 90% CI): Delaunay iid #332 flags 1.918 [1.887,1.925] (2.17 in #332); 15 timed solves 1.018 [1.013,1.023]; Delaunay walk 0.46; rect iid 0.887/0.987; rect walk 0.15; FoM identical. Not evidence #613 is fixed. Research owed: Nautilus-replay witness, varied-seed rounds. Parked, human decision 2026-10-17 (chat).

Campaign: interferometer streaming (array-free) vs in-memory — memory and time scaling to 2e8 visibilities↗

parked · high

Phase 2 at PR-open autolens_profiling#435 (issue #368 reopened, OPEN): from_stream chunk 65536 fp64, 1e8 vis RAL CPU 707.5 s / 1.78 GB RSS, A100 157.3 s / 1.48 GB device; compile ~3 s constant. In-memory fails at 2.5e5 (RAL 10 GB cap) and 4e6 (A100); uncapped CPU ~7.5 kB/vis. Optional 2e8 rows are research; library candidate (chunk transformer.image_from) unfiled per human decision 2026-10-17 (chat). Parked.

Interferometer large-mask likelihood (Aris 512×512): sparse data-vector memory, AdaptSplit log-det backend parity, numba route-gate cost model↗

active · high

L2 shipped to PR: PyAutoArray#636 (head 62a1e3ac, pending-release) + autolens_profiling#425 (results note), both OPEN, CI green. RAL real-data witness 1.49×/1.53×/1.58× at 1,459/2,577/5,746 vertices (pred 1.48×/1.50×/1.53×), logL to round-off; peak RSS 43 GB without NUFFT chunking. Decision record direct-conv-symmetric-gate added. Route-constant refit (0.68 measured vs 0.89 predicted) deferred: CPU lane closed, human decision 2026-10-17 (chat). Remaining: human merge of #636 then #425, and release.

parked
↗☷
Linear-solver accuracy and cost
0 tasks · next step

Retain the 2026-10-08 human decision: Jacobi for Mapper, raw+polish for MGE-only, no default change. Project wiki still says decision pending (stale). No new solver measurements after the recorded phase-5 results; release 2026.10.12.1 does not supply a rerun. Wide-prior/interferometer gaps remain evidence gaps, not authorized tasks. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.

Reviewed 2026-10-17

Active tasks

No open tasks.

active
↗☷
Production imaging and likelihood breakdown
3 tasks · next step

Production over-sampling phase 1 at PR-open: autolens_profiling#431 (issue #430, OPEN, CPU smoke 8/8): --over-sampling {flat1,production} on the 4 JAX Delaunay/DelaunayNN cells, euclid/hst presets. Findings: HST flat-1 adapt_split pin fails on unmodified main (29155.000 vs 29140.296); numba cells' production threshold never reaches sub 4 for HST; rectangular JAX cells uncovered. Phase 2 (A100 production re-measure) needs-decision (human decision 2026-10-17 (chat): undecided). DelaunayNN imaging moved to the parked delaunay-nn-cpu campaign. Post-certified, quick-update plotting and other tasks unchanged.

Reviewed 2026-10-17

Active tasks

Profile imaging pixelizations at production over-sampling

needs-decision · normal

Phase 1 at PR-open: autolens_profiling#431 (issue #430, OPEN, CPU smoke 8/8). Phase 2 A100 production re-measure needs-decision (human decision 2026-10-17 (chat): undecided); requires #431 merged and compute authorization. Findings: HST flat-1 adapt_split pin fails on main; numba HST threshold never reaches sub 4; rectangular JAX cells uncovered.

active
↗☷
DelaunayNN (Sibson) mesh at Delaunay cost on the numba/NumPy CPU path
1 task · next step

Created parked, human decision 2026-10-17 (chat): "file in research to come back to in the future". PyAutoArray#637 (issue #629, OPEN, pending-release, CI 3 legs green): RAL compute-10-4 exclusive 1 thread A-B-B-A ×2, 1,030 vertices / 15,380 px: NN sub4 0.569→0.392 s (2.20× Delaunay, was 3.00×), sub2 0.235→0.189 s (1.75×); logL bit-identical; fixes the size-0 Sibson NaN case. ~1.5× target not met; remaining cost is Sibson (~0.16 s) + wider curvature rows. Target and two hazards recorded as future work, not filed.

Reviewed 2026-10-17

Active tasks

parked
↗☷
Measurement reliability and profiling tools
8 tasks · next step

Timing audit #410–#412 and raw HST/MGE pilot producer #416 merged; implementation issues #362/#415 closed. No fresh production measurements or scientific acceptance. C1/C11 repeat counts, C5 estimator policy, memo interpretation, fresh runtime repeats and other tools remain pending. #418 and #417 open. Evening 2026-10-17 merge-order note: Six open autolens_profiling PRs (#425, #431, #432, #433, #434, #435) all regenerate the shared catalogue/dashboard: each merge after the first needs a rebase + re-render; suggested order #425, #432, #431, #433, #434, #435. #425 after PyAutoArray#636.

Reviewed 2026-10-17

Active tasks

jax_compile/probe.py lost its cell builder with the searches tier — give profiling its own

active · high

Manual HST/MGE fp64 jit builder #414 and separate raw pilot runner #416 merged. Bounded Mind implementations complete; wider builder coverage/global compile dispatch still unavailable. Agree a next capability or separately authorize a qualified pilot; no production compute authorized. Check-in 2026-10-17: autolens_profiling#418 (pilot qualification design) open since 2026-10-13 with CI green and no review; #417 open; no new measurements.

active
↗☷
PyAutoFit profiling bootstrap
1 task · next step

B4a (autofit_profiling#3) and B4b (autofit_profiling#5, issue #4 closed, merged 2026-10-15) shipped: search.fit breakdown cell, profiling-summary@2 exporter, EP/graphical N3/N10/N30 baselines and analytic parity cell; #6 repointed RAL paths to the category layout (2026-10-15). Pulse fit fixture/campaign merged (#60, #61); fit receipt ok at autofit_profiling 58e56b9 (31 records, fetched 2026-10-15T22:58Z), so the post-deploy refresh is done. Epic 2 not opened: next bounded step is a human-scoped epic-2 campaign row; B5c inference work belongs to Insight, not Pulse.

Reviewed 2026-10-17

Active tasks

autofit_profiling: bootstrap the repo + general PyAutoFit profiling epic

active · normal

B4a (autofit_profiling#3) and B4b (autofit_profiling#5, issue #4 closed, merged 2026-10-15) shipped: search.fit breakdown cell, profiling-summary@2 exporter, EP/graphical N3/N10/N30 baselines and analytic parity cell; #6 repointed RAL paths to the category layout (2026-10-15). Pulse fit fixture/campaign merged (#60, #61); fit receipt ok at autofit_profiling 58e56b9 (31 records, fetched 2026-10-15T22:58Z), so the post-deploy refresh is done. Epic 2 not opened: next bounded step is a human-scoped epic-2 campaign row; B5c inference work belongs to Insight, not Pulse.

active
↗☷
Critical curves and evaluation grids
0 tasks · next step

No new campaign measurement. Phase 3b cap fix remains shipped; project wiki still incorrectly says #646 awaits merge/release. Seed coverage and path completion remain unresolved; retain marching-squares decision and ask the human whether to scope the next phase. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.

Reviewed 2026-10-17

Active tasks

No open tasks.

needs-decision
↗☷
Certified positive solver
0 tasks · next step

Parked unchanged. No new campaign evidence; existing C2/library/workspace drafts remain pending. Do not infer admission from solver timing, and do not change default or restart the campaign. Check-in 2026-10-17: no project wiki/results/issue change since 2026-10-13 beyond autolens_profiling#420/#422/#424 (interferometer only); releases 2026.10.16.1/.2 and 2026.10.17.1 supply no new measurement.

Reviewed 2026-10-17

Active tasks

No open tasks.

parked
↗☷

Open means tracked, not necessarily running. Full ledger ↗

Profiling Results

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default

Decision History

Gate the symmetric direct_conv kernel at work fraction 0.7, with ymin column orderingRoute interferometer curvature by a calibrated two-constant cost modelExclude zero-weight mappings from AdaptSplit pixel signals on Delaunay meshes onlyKeep Jacobi PDIP as the Mapper default and raw+polish as the MGE-only default