diff --git a/dashboard.html b/dashboard.html index 6503db1..74d24a1 100644 --- a/dashboard.html +++ b/dashboard.html @@ -333,7 +333,7 @@ - Open autolens_inference: https://github.com/PyAutoLabs/autolens_inference - Open autofit_inference: https://github.com/PyAutoLabs/autofit_inference - Open euclid_detailed: https://github.com/PyAutoLabs/euclid_detailed -- Open cowls: https://github.com/PyAutoLabs/cowls
Last updated 2026-10-16 10:04:26 UTC
↻ Update

+- Open cowls: https://github.com/PyAutoLabs/cowls
Last updated 2026-10-16 11:38:08 UTC
↻ Update

@@ -655,18 +655,18 @@

Runs (6)

RAL:
/mnt/ral/jnightin/inference/autofit_inference

Current position

-

B5c (2026-10-15, autofit_inference#14): a second model family, correlated_nd (correlated Gaussians D = 20/50/100, condition number 10^3, and a D = 20 hybrid Rosenbrock banana, exact posteriors and ln Z), pre-registered as correlated_nd@1 (first committed 9ec37795f404; amendment A1 before any row) with seeded datasets and eight exact references that pass the section 6 agreement test. Its 930-run pilot RAL 410866 finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G on corr_gaussian_d50, carried to B5c-2; 409957 failed at import on the disturbed shared clone and was cancelled) on --partition=ral from the private copy inference/autofit_inference_b5c at the pin df6d218f10cd; it ranks nothing and is catalogued apart from gaussian_x3 (nothing ranked across families). gaussian_x3 wave 2 stays closed and judged under gaussian_x3@2. Next: B5c-2 pulls the pilot (results/ only) from the private copy, settles the 4 OOM cells (more memory or excluded), judges it under @1, freezes correlated_nd@2 and submits the scored wave.

+

B5c-2a (2026-10-16, autofit_inference#17): the correlated_nd pilot (RAL 410866) is pulled from the private copy and judged under correlated_nd@1 (1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred; 307 accepted, 623 rejected; ranks nothing, nothing ranked across families), and correlated_nd@2 is frozen in autofit_inference 44c5e77f1fb0 (protocol section 11 A2) before any scored row: sigma-ratio bands per task (evidence [0.85, 1.1] on 79 rows, posterior kept at [0.5, 2.0] on 1 row), ESS/R-hat bars kept, 75 timed + 8 probe-first cells, 21 cells deferred with reasons (incl. d50 Nautilus n_live_100 by the cap rule and all d100 Nautilus by A2), memory 2x the pilot peak (d50 Nautilus n_live_400 16 G, d100 cold NUTS 17 G). Calibration: autofit_inference results/calibration/correlated_nd_wave1.json. gaussian_x3 wave 2 stays closed under gaussian_x3@2. Next: B5c-2b (draft/research/autofit_inference/search_ext_b5c2b_scored_wave_submit.md) adds realisations 2-3 and their exact references, the 12 450-run scored-wave manifest and its probe, and submits on --partition=ral from a private pinned copy; nothing is submitted until then.

Runs (0)

nothing on the cluster

-
Recent log β€” 5 of 27 entries +
Recent log β€” 5 of 28 entries

full log

active Β· ral Β· autofit_inference#8

diff --git a/dashboard.md b/dashboard.md index 4337512..c8a7f80 100644 --- a/dashboard.md +++ b/dashboard.md @@ -524,7 +524,7 @@ active Β· both Β· [autolens_inference#4](https://github.com/PyAutoLabs/autolens_ **Current position** -B5c (2026-10-15, autofit_inference#14): a second model family, correlated_nd (correlated Gaussians D = 20/50/100, condition number 10^3, and a D = 20 hybrid Rosenbrock banana, exact posteriors and ln Z), pre-registered as correlated_nd@1 (first committed 9ec37795f404; amendment A1 before any row) with seeded datasets and eight exact references that pass the section 6 agreement test. Its 930-run pilot RAL 410866 finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G on corr_gaussian_d50, carried to B5c-2; 409957 failed at import on the disturbed shared clone and was cancelled) on --partition=ral from the private copy inference/autofit_inference_b5c at the pin df6d218f10cd; it ranks nothing and is catalogued apart from gaussian_x3 (nothing ranked across families). gaussian_x3 wave 2 stays closed and judged under gaussian_x3@2. Next: B5c-2 pulls the pilot (results/ only) from the private copy, settles the 4 OOM cells (more memory or excluded), judges it under @1, freezes correlated_nd@2 and submits the scored wave. +B5c-2a (2026-10-16, autofit_inference#17): the correlated_nd pilot (RAL 410866) is pulled from the private copy and judged under correlated_nd@1 (1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred; 307 accepted, 623 rejected; ranks nothing, nothing ranked across families), and correlated_nd@2 is frozen in autofit_inference 44c5e77f1fb0 (protocol section 11 A2) before any scored row: sigma-ratio bands per task (evidence [0.85, 1.1] on 79 rows, posterior kept at [0.5, 2.0] on 1 row), ESS/R-hat bars kept, 75 timed + 8 probe-first cells, 21 cells deferred with reasons (incl. d50 Nautilus n_live_100 by the cap rule and all d100 Nautilus by A2), memory 2x the pilot peak (d50 Nautilus n_live_400 16 G, d100 cold NUTS 17 G). Calibration: autofit_inference results/calibration/correlated_nd_wave1.json. gaussian_x3 wave 2 stays closed under gaussian_x3@2. Next: B5c-2b (draft/research/autofit_inference/search_ext_b5c2b_scored_wave_submit.md) adds realisations 2-3 and their exact references, the 12 450-run scored-wave manifest and its probe, and submits on --partition=ral from a private pinned copy; nothing is submitted until then.
πŸ“‹ @@ -538,13 +538,13 @@ Use the cortex skill. β€” resume autofit_inference: read PyAutoCortex projects/a _nothing on the cluster_ -
Recent log β€” 5 of 27 entries +
Recent log β€” 5 of 28 entries +- 2026-10-16 β€” *result* β€” correlated_nd pilot RAL 410866 judged under correlated_nd@1 (B5c-2a, autofit_inference#17): results/ pulled from the private copy inference/autofit_inference_b5c (930 rows; exact references byte-identical); 1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred, reconciled with sacct; 307 accepted, 623 rejected; ranks nothing. Failures: 80 Emcee/Zeus (d20, banana) crashed in the harness's chain diagnostics after the search (relabelling by g0.centre; fixed, no verdict moves); 60 d100 Nautilus (n_live_100: needs n_live > D; n_live_200/400: nautilus HDF5 checkpoint cannot store a D = 100 bound); 10 d100 warm NUTS (provider); 27 at the 4 h cap (d50 Nautilus n_live_100, d50 warm NUTS). The 4 OUT_OF_MEMORY tasks 263/273/304/383 (MaxRSS 7.6/7.7/3.1/8.4 G at 8 G) had been given cap stubs by the exit-137 path and are now an oom attempt class. A2 ruling: the OOM cells d50 Nautilus n_live_100 and d50 warm NUTS are deferred by the cap rule (no attempt ended on its own); d50 Nautilus n_live_400 gets 16 G and d100 cold NUTS 17 G (2x the measured peak, 8 G for an OOM class); d100 Nautilus n_live_200 (and 100, 400) is deferred (fails at the pin before sampling, affordability unmeasured); an oom run is rerun once at 2x. correlated_nd@2 frozen in autofit_inference 44c5e77f1fb0 (44c5e77f1fb0779f675a8f51247d812183e93b48) before any scored row; nothing submitted. - 2026-10-16 β€” *run* β€” 410866_[0-929] finished β€” wall 4:16: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 β€” 926/930 COMPLETED, 4 OUT_OF_MEMORY at 8 G, all corr_gaussian_d50: task 263 nautilus n_live_100 numpy seed 3, 273 nautilus n_live_100 jax_cpu seed 3, 304 nautilus n_live_400 numpy seed 4, 383 blackjax_nuts_warm warmup_200_samples_500 jax_cpu seed 3; not rerun in stage 1 (human decision), B5c-2 raises their memory or excludes them; results unpulled in the private copy, pulled and judged in B5c-2 - 2026-10-15 β€” *run* β€” 410866_[0-929] submitted: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 - 2026-10-15 β€” *run* β€” 409957_[0-929] failed β€” wall 0:06: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2 β€” every task failed at import (ModuleNotFoundError _autofit_inference_cli): the shared RAL clone had been switched to an old main by another session (stash pre-main-switch 19:24 UTC) and lost the CLI and config/ as the array started; 800 tasks exited, the rest scancelled; no row written; resubmitted from the private copy inference/autofit_inference_b5c as 410866 - 2026-10-15 β€” *run* β€” 409957_[0-929] submitted: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2 -- 2026-10-15 β€” *run* β€” 409930_[0-19] finished β€” wall 1:30: wave-2 probe-first remainder (B5b): Zeus JAX + NSS n_live_200_mcmc_steps_50, both datasets, 980 runs after the probe under probe-derived @2 timeouts (NSS 600 s, Zeus 1168 s); %20; submit_wave2_remainder; pin df6d218f10cd; pulled in B5b-2 β€” 20/20 COMPLETED (16:08-17:38 UTC); 980 remainder rows pulled by per-directory tar (20 probe rows byte-identical); all admitted under gaussian_x3@2, 0 missing; judged B5b-2 [full log](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/autofit_inference.md) diff --git a/projects/autofit_inference.md b/projects/autofit_inference.md index 3b53179..1053aab 100644 --- a/projects/autofit_inference.md +++ b/projects/autofit_inference.md @@ -5,13 +5,14 @@ Issue: autofit_inference#8 ## Now -B5c (2026-10-15, autofit_inference#14): a second model family, correlated_nd (correlated Gaussians D = 20/50/100, condition number 10^3, and a D = 20 hybrid Rosenbrock banana, exact posteriors and ln Z), pre-registered as correlated_nd@1 (first committed 9ec37795f404; amendment A1 before any row) with seeded datasets and eight exact references that pass the section 6 agreement test. Its 930-run pilot RAL 410866 finished 2026-10-16 (926/930 completed, 4 OUT_OF_MEMORY at 8 G on corr_gaussian_d50, carried to B5c-2; 409957 failed at import on the disturbed shared clone and was cancelled) on --partition=ral from the private copy inference/autofit_inference_b5c at the pin df6d218f10cd; it ranks nothing and is catalogued apart from gaussian_x3 (nothing ranked across families). gaussian_x3 wave 2 stays closed and judged under gaussian_x3@2. Next: B5c-2 pulls the pilot (results/ only) from the private copy, settles the 4 OOM cells (more memory or excluded), judges it under @1, freezes correlated_nd@2 and submits the scored wave. +B5c-2a (2026-10-16, autofit_inference#17): the correlated_nd pilot (RAL 410866) is pulled from the private copy and judged under correlated_nd@1 (1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred; 307 accepted, 623 rejected; ranks nothing, nothing ranked across families), and correlated_nd@2 is frozen in autofit_inference 44c5e77f1fb0 (protocol section 11 A2) before any scored row: sigma-ratio bands per task (evidence [0.85, 1.1] on 79 rows, posterior kept at [0.5, 2.0] on 1 row), ESS/R-hat bars kept, 75 timed + 8 probe-first cells, 21 cells deferred with reasons (incl. d50 Nautilus n_live_100 by the cap rule and all d100 Nautilus by A2), memory 2x the pilot peak (d50 Nautilus n_live_400 16 G, d100 cold NUTS 17 G). Calibration: autofit_inference results/calibration/correlated_nd_wave1.json. gaussian_x3 wave 2 stays closed under gaussian_x3@2. Next: B5c-2b (draft/research/autofit_inference/search_ext_b5c2b_scored_wave_submit.md) adds realisations 2-3 and their exact references, the 12 450-run scored-wave manifest and its probe, and submits on --partition=ral from a private pinned copy; nothing is submitted until then. ## Runs ## Log +- 2026-10-16 β€” result β€” correlated_nd pilot RAL 410866 judged under correlated_nd@1 (B5c-2a, autofit_inference#17): results/ pulled from the private copy inference/autofit_inference_b5c (930 rows; exact references byte-identical); 1040 expected = 749 completed rows + 177 failed + 4 oom + 0 missing + 110 deferred, reconciled with sacct; 307 accepted, 623 rejected; ranks nothing. Failures: 80 Emcee/Zeus (d20, banana) crashed in the harness's chain diagnostics after the search (relabelling by g0.centre; fixed, no verdict moves); 60 d100 Nautilus (n_live_100: needs n_live > D; n_live_200/400: nautilus HDF5 checkpoint cannot store a D = 100 bound); 10 d100 warm NUTS (provider); 27 at the 4 h cap (d50 Nautilus n_live_100, d50 warm NUTS). The 4 OUT_OF_MEMORY tasks 263/273/304/383 (MaxRSS 7.6/7.7/3.1/8.4 G at 8 G) had been given cap stubs by the exit-137 path and are now an oom attempt class. A2 ruling: the OOM cells d50 Nautilus n_live_100 and d50 warm NUTS are deferred by the cap rule (no attempt ended on its own); d50 Nautilus n_live_400 gets 16 G and d100 cold NUTS 17 G (2x the measured peak, 8 G for an OOM class); d100 Nautilus n_live_200 (and 100, 400) is deferred (fails at the pin before sampling, affordability unmeasured); an oom run is rerun once at 2x. correlated_nd@2 frozen in autofit_inference 44c5e77f1fb0 (44c5e77f1fb0779f675a8f51247d812183e93b48) before any scored row; nothing submitted. - 2026-10-16 β€” run β€” 410866_[0-929] finished β€” wall 4:16: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 β€” 926/930 COMPLETED, 4 OUT_OF_MEMORY at 8 G, all corr_gaussian_d50: task 263 nautilus n_live_100 numpy seed 3, 273 nautilus n_live_100 jax_cpu seed 3, 304 nautilus n_live_400 numpy seed 4, 383 blackjax_nuts_warm warmup_200_samples_500 jax_cpu seed 3; not rerun in stage 1 (human decision), B5c-2 raises their memory or excludes them; results unpulled in the private copy, pulled and judged in B5c-2 - 2026-10-15 β€” run β€” 410866_[0-929] submitted: correlated_nd@1 pilot (B5c), resubmission of 409957 from the PRIVATE copy /mnt/ral/jnightin/inference/autofit_inference_b5c (delivered files sha256-verified from 245a3ef, own copy of the pin): 930 runs, 10 seeds x realisation 1 x 93 planned cells, one run per task, 4 h process cap, %100; pin df6d218f10cd; pull results/ from the private copy in B5c-2 - 2026-10-15 β€” run β€” 409957_[0-929] failed β€” wall 0:06: correlated_nd@1 pilot (B5c, second model family; protocol first committed 9ec37795f404, A1 before any row): 930 runs, 10 seeds x realisation 1 x 93 planned cells over corr_gaussian_d20/d50/d100 + rosenbrock_d20, one run per task, 4 h process cap, %100; submit_correlated_nd_pilot; pin df6d218f10cd; pulled in B5c-2 β€” every task failed at import (ModuleNotFoundError _autofit_inference_cli): the shared RAL clone had been switched to an old main by another session (stash pre-main-switch 19:24 UTC) and lost the CLI and config/ as the array started; 800 tasks exited, the rest scancelled; no row written; resubmitted from the private copy inference/autofit_inference_b5c as 410866 diff --git a/state.json b/state.json index 60525f7..ff78462 100644 --- a/state.json +++ b/state.json @@ -4,7 +4,7 @@ "repo": "PyAutoCortex", "status": "yellow", "headline": "YELLOW \u2014 11 active \u00b7 22 running", - "updated": "2026-10-16T10:04:26Z", + "updated": "2026-10-16T11:38:08Z", "pages_url": "https://pyautolabs.github.io/PyAutoCortex/", "items": [ {