From 031b02f3da6b9deee233ecc3b4f186d7cc95de75 Mon Sep 17 00:00:00 2001 From: Emily Boudreaux Date: Thu, 17 Sep 2026 17:53:07 -0400 Subject: [PATCH] docs: clarify preconditioner experiment results and timing --- .../PRECONDITIONER_IMPROVEMENT_2026-09-18.md | 226 ++++++++++-------- 1 file changed, 121 insertions(+), 105 deletions(-) diff --git a/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md b/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md index e7ca6e4..15cc925 100644 --- a/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md +++ b/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md @@ -1,9 +1,9 @@ # Preconditioner improvement: matched frozen Newton systems This experiment changes preconditioning only. The complete Newton Jacobian, -normalization, physical residual, mesh, and linear acceptance gate are held fixed. -The date in branch and artifact names is an experiment identifier; provenance -files contain the actual UTC execution times. +normalization, physical residual, mesh, and linear acceptance gate are held +fixed. The date in branch and artifact names is an experiment identifier; +provenance files contain the actual UTC execution times. ## Reproduction and scope @@ -15,18 +15,20 @@ Integration branch: `research/preconditioner-benchmarks-20260918`, based on The experiment workspace is `/Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d`. -Evidence is under `diagnostics/preconditioner_improvement_2026-09-18` there. -Each campaign retains its clean source commit, build configuration, copied -executable and hash, mesh hash, exact commands, input/output hashes, and logs. -Runs are serial, without overlapping builds or FEM jobs during timing. +Evidence is stored under `diagnostics/preconditioner_improvement_2026-09-18` +in that workspace. Each campaign retains its clean source commit, build +configuration, copied executable and hash, mesh hash, exact commands, +input/output hashes, and logs. Runs are serial, without overlapping builds or +FEM jobs during timing. -Frozen comparisons start FGMRES from zero on the same saved state and right-hand -side. Each role constructs a fresh inverse, including its own specification -border responses. The restart length is 160, the initial cap is 200 iterations, -and the required relative full-J true defect is 0.03. The independently applied -full Jacobian decides acceptance; the Krylov recurrence flag alone does not. -All accepted/trial state buffers, normalization factors, preparation generation, -and a deterministic full-J probe must remain unchanged. +Frozen comparisons start FGMRES from zero on the same saved state and +right-hand side. Each role constructs a fresh inverse, including its own +specification border responses. The restart length is 160, the initial cap is +200 iterations, and the required relative full-J true defect is 0.03. The +independently applied full Jacobian decides acceptance; the Krylov recurrence +flag alone does not. All accepted/trial state buffers, normalization factors, +preparation generation, and a deterministic full-J probe must remain +unchanged. Costs include inverse construction, normalized binding, Krylov setup, Krylov solve, and full-J verification. Extra diagnostic applications and artifact I/O @@ -37,30 +39,32 @@ physical convergence. ## Existing two-way coupling: negative controls -`screen_v1` uses source `018fc1c`, before the cached cross-action optimization. +`screen_v1` uses source `018fc1c`, before the cached cross-action +optimization. | Saved state | Inverse | Iterations | Full-J relative defect | Complete verified cost (s) | |---|---|---:|---:|---:| -| n=1.5, J=0 plateau | independent | 200 | 0.0590170471860 | 87.9302 | +| n = 1.5, J = 0 plateau | independent | 200 | 0.0590170471860 | 87.9302 | | same | stellar LDU | 200 | 0.0600668598825 | 599.5473 | -| n=1, J=0.1 plateau | independent | 200 | 0.0826534298794 | 86.9409 | +| n = 1, J = 0.1 plateau | independent | 200 | 0.0826534298794 | 86.9409 | | same | stellar LDU | 200 | 0.0720550396371 | 594.2053 | -All four miss the 0.03 gate. Stellar LDU spends 6.82–6.83 times the total cost; -it slightly worsens the n=1.5 result and reduces the rotating defect by 12.82%. -Mean inverse application costs rise from approximately 0.19 s to 2.7 s, while -full-J actions remain approximately 0.23 s. The uncached gravity-to-material -force column explains the application-cost anomaly. +All four solves fail the 0.03 gate. Stellar LDU incurs 6.82–6.83 times the +total cost; it slightly worsens the n = 1.5 result and reduces the rotating +defect by 12.82%. Mean inverse application time rises from approximately 0.19 +s to 2.7 s, while full-J actions remain approximately 0.23 s. The uncached +gravity-to-material force column explains the application-cost anomaly. -Poisson contributes 95.53% of the squared n=1.5 baseline defect and 98.28% with -stellar LDU. For the rotating state those fractions are 85.00% and 98.10%; LDU -reduces the gradient-row norm by 94.16% but the Poisson norm by only 6.35%. -These are global normalized block diagnostics, not proof of an exclusively -exterior error. No exterior weighting is changed. +Poisson contributes 95.53% of the squared n = 1.5 baseline defect and 98.28% +with stellar LDU. For the rotating state those fractions are 85.00% and +98.10%; LDU reduces the gradient-row norm by 94.16% but the Poisson norm by +only 6.35%. These are global normalized block diagnostics, not proof of an +exclusively exterior error. No exterior weighting is changed. The independent artifact audit passes 65 provenance checks and 93 checks per -case. Both saved states and the normalization data exactly match their producer -artifacts. The two independent baselines reproduce the earlier failed solves. +case. Both saved states and the normalization data exactly match their +producer artifacts. The two independent baselines reproduce the earlier failed +solves. ## Candidate mechanisms @@ -83,103 +87,115 @@ artifacts. The two independent baselines reproduce the earlier failed solves. ## First candidate screen -`candidates_v1` uses source `dc8d4b0` and the same n=1.5, J=0 plateau state. -Every fresh independent control reproduces relative defect 0.059017047186 at -200 iterations. Complete control costs remain between 87.53 and 88.25 s. +`candidates_v1` uses source `dc8d4b0` and the same n = 1.5, J = 0 plateau +state. Every fresh independent control reproduces relative defect +0.059017047186 at 200 iterations. Complete control costs remain between 87.53 +and 88.25 s. | Candidate | Iterations | Full-J relative defect | Gate | Complete verified cost (s) | |---|---:|---:|---|---:| | Cached stellar LDU | 200 | 0.0600668598779 | fail | 117.4845 | -| Consistent gravity Schur, k=1, omega=0.5 | 132 | 0.0292091549685 | pass | 72.3981 | +| Consistent gravity Schur, k = 1, omega = 0.5 | 132 | 0.0292091549685 | pass | 72.3981 | | Cached stellar LDU + consistent gravity Schur | 121 | 0.0285619160766 | pass | 84.7104 | | Nine-mode coupled shape coarse correction | 200 | 0.0668657989082 | fail | 97.0564 | -The cached LDU is 5.1032 times faster than its old execution path at the same -iteration cap, while mean inverse application cost drops 8.0425 times. This is -an execution improvement, not a convergence improvement. Saved inputs and the -full-J probe agree exactly; the final normalized correction changes by only -2.7051e-11 relatively. Both LDU solves fail, and the cached version remains -34.2% more expensive than the independent control on this state. +At the same iteration cap, the cached LDU reduces total execution time by a +factor of 5.1032 relative to the old path, while mean inverse application time +drops by a factor of 8.0425. This is an execution improvement, not a +convergence improvement. Saved inputs and the full-J probe agree exactly; the +final normalized correction changes by only 2.7051e-11 relatively. Both LDU +solves fail, and the cached version remains 34.2% more expensive than the +independent control on this state. -Gravity refinement is the first candidate to cross the original gate. It lowers -the Poisson-row defect by 55.4% and the gradient-row defect by 15.0%. The squared -Poisson share drops from 95.53% to 77.59%; surface and hydrostatic defects grow, -so the improvement is judged by the verified global norm. Work changes from -400 to 396 mass-inverse applications, 200 to 264 AMG Schur applications, and -200 to 132 outer inverse applications. The default has not yet passed the same -gate, so dividing its failed 88 s attempt by 72.4 s is not a convergence speedup. -The combined LDU saves another eleven iterations but is 17% slower overall. +Gravity refinement is the first candidate to cross the original gate. It +lowers the Poisson-row defect by 55.4% and the gradient-row defect by 15.0%. +The squared Poisson share drops from 95.53% to 77.59%; surface and hydrostatic +defects grow, so the improvement is judged by the verified global norm. Work +changes from 400 to 396 mass-inverse applications, 200 to 264 AMG Schur +applications, and 200 to 132 outer inverse applications. At this iteration +cap, the default does not pass the same gate, so dividing its failed 88 s +attempt by 72.4 s is not a convergence speedup. The combined LDU saves another +eleven iterations relative to gravity refinement alone but takes 17% more time +overall. The coarse candidate has all nine modes active, minimum overlap lower bound -0.79465, and no fallback. Its extra construction costs 8.6012 s, including mode -completion and eighteen setup Jacobian applications; it stores 28,491,680 bytes -in persistent vectors. The actual FEM subspace check passes. Nevertheless, its -200-iteration defect worsens by 13.3%. This is a negative numerical result for -these modes on this state, not evidence about every possible coarse space or -rapidly rotating model. The prototype remains confined to experiments. +0.79465, and no fallback. Its extra construction costs 8.6012 s, including +mode completion and eighteen setup Jacobian applications; it stores 28,491,680 +bytes in persistent vectors. The actual FEM subspace check passes. +Nevertheless, its 200-iteration defect worsens by 13.3%. This is a negative +numerical result for these modes on this state, not evidence about every +possible coarse space or rapidly rotating model. The prototype remains +confined to experiments. ## Validation status The integrated library and experiment driver compile with the repository's -Release/ThinLTO configuration. Source dc8d4b0 passes 20 focused serial test cases -(441,978 assertions), including FEM equivalence, geometry/discretization refresh, -fixed-work algebra, and coarse inverse tests. Distributed unit and coarse tests -pass on two ranks (13 cases and 213 assertions per rank). +Release/ThinLTO configuration. Source `dc8d4b0` passes 20 focused serial test +cases (441,978 assertions), including FEM equivalence, geometry/discretization +refresh, fixed-work algebra, and coarse inverse tests. Distributed unit and +coarse tests pass on two ranks (13 cases and 213 assertions per rank). -The broad real-FEM MPI run aborts because the existing METIS partition gives one -rank no compactified vacuum element, which the mapped mass constructor requires. -The failing log is retained; this campaign does not claim distributed FEM Schur -validation. See the gravity refinement note for the exact scope and evidence. +The broad real-FEM MPI run aborts because the existing METIS partition gives +one rank no compactified vacuum element, which the mapped mass constructor +requires. The failing log is retained; this campaign does not claim +distributed FEM Schur validation. See the gravity refinement note for the +exact scope and evidence. The evidence checker passes 22 lightweight tests. Independent reviews of the -cached LDU and gravity-only cases validate provenance, state/operator invariants, -true gates, and child-work accounting. Reviews are stored outside immutable -campaign directories. +cached LDU and gravity-only cases validate provenance, state/operator +invariants, true gates, and child-work accounting. Reviews are stored outside +immutable campaign directories. -Source fcb7833 adds owning experiment-only Newton recipes through the existing -ADL preparation interface. They use the same concrete prepared inverses as the -frozen benchmark and the existing context refresh/border lifecycle. The driver -supports default, gravity-Schur, and stellar-LDU recipes for ordinary Newton, -positivity, curvature, and positivity-curvature runs. This adds no public -physics-facing solver API. The updated driver builds successfully at `87569c0`. +Source `fcb7833` adds owning experiment-only Newton recipes through the +existing ADL preparation interface. They use the same concrete prepared +inverses as the frozen benchmark and the existing context refresh/border +lifecycle. The driver supports default, gravity-Schur, and stellar-LDU recipes +for ordinary Newton, positivity, curvature, and positivity-curvature runs. +This adds no public physics-facing solver API. The updated driver builds +successfully at `87569c0`. ## Equal-accuracy confirmation -`confirmation_v1` uses source `87569c0`, restart160 and a maximum of600 iterations -for both roles. The target remains a freshly verified relative defect of0.03. +`confirmation_v1` uses source `87569c0`, a restart length of 160, and a +maximum of 600 iterations for both roles. The target remains a freshly +verified relative defect of 0.03. | State | Inverse | Iterations | Full-J relative defect | Gate | Complete verified cost (s) | |---|---|---:|---:|---|---:| -| n=1.5, J=0 plateau | independent | 231 | 0.0290033752814 | pass | 100.9943 | -| same | gravity Schur k=1, omega=0.5 | 132 | 0.0292091549685 | pass | 72.3727 | -| n=1, J=0.1 plateau | independent | 600 | 0.0739197073237 | fail | 257.9399 | -| same | gravity Schur k=1, omega=0.5 | 600 | 0.0408505998829 | fail | 321.5918 | +| n = 1.5, J = 0 plateau | independent | 231 | 0.0290033752814 | pass | 100.9943 | +| same | gravity Schur k = 1, omega = 0.5 | 132 | 0.0292091549685 | pass | 72.3727 | +| n = 1, J = 0.1 plateau | independent | 600 | 0.0739197073237 | fail | 257.9399 | +| same | gravity Schur k = 1, omega = 0.5 | 600 | 0.0408505998829 | fail | 321.5918 | -The n=1.5 speedup to the same requested accuracy is **1.39548x**, or28.34% less -complete cost and42.86% fewer iterations. The candidate repeats its earlier132 -iterations and72.3981s timing closely. Both endpoints overshoot the same stopping -threshold slightly; neither tolerance nor the full-J check was relaxed. +For n = 1.5, the frozen linear solve speedup to the same requested accuracy is +**1.39548x**, with 28.34% less complete cost and 42.86% fewer iterations. The +candidate repeats its earlier 132 iterations, with timing close to the earlier +72.3981 s. Both endpoints finish slightly below the same stopping threshold; +neither tolerance nor the full-J check was relaxed. -Rotation behaves differently. Refinement reduces the600-step defect by44.74%, -but costs24.68% more and still fails. The remaining gradient and Poisson defects -contribute0.02664 and0.03018 to the global relative norm. The older rotating LDU -control reduced the gradient contribution to0.001813 while leaving a large -Poisson contribution. This supports one complementary LDU+Schur follow-up; it -neither proves a gradient floor nor permits multiplying the separate gains. +Rotation behaves differently. Refinement reduces the 600-step defect by +44.74%, but costs 24.68% more and still fails the gate. The remaining gradient +and Poisson defects contribute 0.02664 and 0.03018 to the global relative +norm. The older rotating LDU control reduced the gradient contribution to +0.001813 while leaving a large Poisson contribution. This supports one +complementary LDU+Schur follow-up; it neither proves a gradient floor nor +permits multiplying the separate gains. -All73 provenance and186 case checks pass. The reproducible figure and plotting -provenance are in `confirmation_v1_figures`; reported Krylov histories are shown -separately from independently verified final points. Plotting used an existing -compatible Python environment and installed no dependencies. +All 73 provenance checks and 186 case checks pass. The reproducible figure and +plotting provenance are in `confirmation_v1_figures`; reported Krylov +histories are shown separately from independently verified final points. +Plotting used an existing compatible Python environment and installed no +dependencies. ## Easy nonlinear and physical regression -The first two `newton_v1` cases use source `bed1893`, the same n=1, J=0 model with -no central-density constraint, positivity strategy, and identical tolerances. -Both converge in six accepted full steps, with no repairs or rejections. Initial -physical diagnostics and layout are byte-identical; all linear true gates pass. +The first two `newton_v1` cases use source `bed1893`, the same n = 1, J = 0 +model with no central-density constraint, positivity strategy, and identical +tolerances. Both converge in six accepted full steps, with no repairs or +rejections. Initial physical diagnostics and layout are byte-identical; all +linear true gates pass. -| Measure | Independent | Gravity Schur k=1, omega=0.5 | +| Measure | Independent | Gravity Schur k = 1, omega = 0.5 | |---|---:|---:| | Total linear iterations | 307 | 258 | | Summed Newton iteration time (s) | 191.068 | 199.375 | @@ -187,17 +203,17 @@ physical diagnostics and layout are byte-identical; all linear true gates pass. | Complete diagnostic-run wall time (s) | 251.720 | 261.285 | | Final nonlinear residual | 1.65157e-13 | 1.05839e-13 | -Both final residuals are below the shared1.81187e-12 nonlinear target. Fewer -iterations do not produce an easy-case speedup: Newton time rises4.35% and full -run time3.80%. The full run includes physical validation and artifact I/O; it is -not presented as pure solver cost. +Both final residuals are below the shared 1.81187e-12 nonlinear target. Fewer +iterations do not produce an easy-case speedup: Newton time rises by 4.35% and +full run time by 3.80%. The full run includes physical validation and artifact +I/O; it is not presented as pure solver cost. -All eight virial errors (four definitions at quadrature14 and18) lie between -2.61811e-7 and3.73092e-7 for both models, comfortably below1e-5. Maximum endpoint -coefficient difference is2.679e-11; radius difference is3.48e-14. The independent -paired audit also checks mass, density/enthalpy admissibility and local EOS -metrics. This preserves the existing physical regression, rather than adding -independent ESTER validation. +All eight virial errors (four definitions at quadrature orders 14 and 18) lie +between 2.61811e-7 and 3.73092e-7 for both models, comfortably below 1e-5. The +maximum endpoint coefficient difference is 2.679e-11; the radius difference is +3.48e-14. The independent paired audit also checks mass, density/enthalpy +admissibility and local EOS metrics. This preserves the existing physical +regression, rather than adding independent ESTER validation. The two hard nonlinear cases and the targeted rotating coupling follow-up are still running or pending. Production defaults remain unchanged: the stronger