diff --git a/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md b/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md index 623272b..0dba933 100644 --- a/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md +++ b/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md @@ -1,9 +1,19 @@ -# Preconditioner improvement: matched frozen Newton systems +# Preconditioner improvement: frozen systems and nonlinear checks -This experiment changes preconditioning only. The complete Newton Jacobian, -normalization, physical residual, mesh, and linear acceptance gate are held -fixed. The date in branch and artifact names is an experiment identifier; -provenance files contain the actual UTC execution times. +This experiment changes preconditioning only. The physical equations, +complete Newton Jacobian construction, normalization rules, mesh, and linear +acceptance gate are unchanged. Within each frozen comparison, the numerical +Jacobian, right-hand side, and normalization factors are held fixed; nonlinear +runs rebuild them along their respective trajectories. The date in branch +and artifact names is an experiment identifier; provenance files contain the +actual UTC execution times. + +The strongest equal-accuracy timing result is a 1.39548x frozen linear solve +speedup for n = 1.5, J = 0. The combined material/gravity inverse also passes +a saved rotating system that the default cannot solve within 600 iterations. +Neither result establishes convergence of the hard nonlinear models. The +easy n = 1, J = 0 physical regression retains all virial errors below 3.74e-7, +but takes approximately 4% longer with the stronger inverse. ## Reproduction and scope @@ -149,10 +159,11 @@ immutable campaign directories. Source `fcb7833` adds owning experiment-only Newton recipes through the existing ADL preparation interface. They use the same concrete prepared inverses as the frozen benchmark and the existing context refresh/border -lifecycle. The driver supports default, gravity-Schur, and stellar-LDU recipes -for ordinary Newton, positivity, curvature, and positivity-curvature runs. -This adds no public physics-facing solver API. The updated driver builds -successfully at `87569c0`. +lifecycle. The driver supports default, gravity-Schur, stellar-LDU, and +combined stellar-LDU/gravity-Schur recipes for ordinary Newton, positivity, +curvature, and positivity-curvature runs. The combined Newton recipe was +added at `7b4f674`; the driver builds successfully at `5298d88`. This adds no +public physics-facing solver API. ## Equal-accuracy confirmation @@ -211,9 +222,13 @@ I/O; it is not presented as pure solver cost. All eight virial errors (four definitions at quadrature orders 14 and 18) lie between 2.61811e-7 and 3.73092e-7 for both models, comfortably below 1e-5. The maximum endpoint coefficient difference is 2.679e-11; the radius difference is -3.48e-14. The independent paired audit also checks mass, density/enthalpy -admissibility and local EOS metrics. This preserves the existing physical -regression, rather than adding independent ESTER validation. +3.48e-14. The independent paired audit also checks mass, sampled +density/enthalpy admissibility and local EOS metrics. At quadrature order 18, +the density EOS mismatch, scaled by the sampled maximum density, has RMS +8.5654e-5 and maximum 4.2355e-4. The small virial error therefore does not +establish pointwise accuracy to 1e-5 or spatial mesh convergence. This +preserves the existing physical regression; it does not add independent +ESTER validation. ## Higher-index nonlinear follow-through @@ -249,11 +264,12 @@ artifact verification. ## Rotating nonlinear follow-through -The n = 1, J = 0.1 positivity-curvature run accepts one full Newton step after -an initial positivity repair. Its first linear solve passes in thirteen -iterations with relative true defect 0.023583. The second reaches 200 -iterations with defect 0.0315391 and fails the unchanged 0.03 gate, before a -second trial can be tested. There are no line-search rejections. This case +The n = 1, J = 0.1 positivity-curvature run accepts one full trial step after +a positivity repair modifies its search direction. Its first raw linear +solve passes in thirteen iterations with relative true defect 0.023583; +that defect does not certify the subsequently repaired direction. The second +reaches 200 iterations with defect 0.0315391 and fails the unchanged 0.03 gate, +before a second trial can be tested. There are no line-search rejections. This case therefore retains a linear obstruction as well as an unconverged nonlinear state. @@ -283,10 +299,94 @@ Its fresh independent control reproduces the original 200-iteration defect failed at 600 iterations, so there is no measured finite baseline time-to-convergence with which to form a speedup ratio. +The independent audit verifies byte-identical saved states, right-hand side, +normalization factors, layout, and deterministic Jacobian probe against the +earlier rotating campaigns. Its fresh baseline correction and exact residual +also match the original 200-iteration control byte for byte. The combination +reduces the Poisson defect norm by 63.33% and the gradient defect norm by +94.47%. Surface and hydrostatic defects increase; acceptance still uses the +complete norm. Mean inverse application cost rises from 0.18409 to 0.43046 s, +so iteration reduction alone must not be presented as an application speedup. + This combination succeeds where either isolated change failed on this frozen -rotating problem. It does not establish nonlinear convergence. One bounded -eight-step positivity-curvature run is pending to check that distinction; -its budget matches the historical rotating run. +rotating problem. The follow-through in `rotation_newton_v1`, source +`83afd82`, demonstrates the limit of that result. With the historical +eight-step budget and otherwise matching controls, its first raw linear +solve passes in twelve iterations at relative true defect 0.0171651. A +positivity repair changes the search direction; the accepted full trial +reduces the nonlinear residual from 0.0858812 to 0.00685562. At this new state, +the next linear solve reaches 200 iterations with defect 0.0339685 and fails. +There is no second trial and no nonlinear convergence. + +Summed Newton time is 172.396 s, including 141.631 s in outer linear solves; +full diagnostic-run wall time is 234.738 s. The quadrature-18 enthalpy-pressure +energy and force virial errors are 0.00712711 and 0.00709672. Both fail 1e-5. +Density-pressure virials are undefined, with 1975 negative density samples +and negative mass fraction 4.04801e-5. The successful frozen linear solve +therefore does not certify a physically acceptable rotating solution, or +even passage through every linear system along a new Newton trajectory. + +The historical independent endpoint has a larger nonlinear residual, +0.00840387, but a smaller enthalpy energy virial error, 0.000544539. Both +physical endpoints fail. This is direct evidence that improvement in the +normalized nonlinear residual need not improve virial balance along distinct +trajectories. The combined run passes all 38 current-case artifact checks and +32 historical-control checks, including matching initial diagnostics, +layout, physics and solver settings. Production defaults remain unchanged: the stronger inverse is an optional typed policy, and the coarse candidate remains an experiment. + +Here J = 0.1 denotes the angular-momentum input, not a measured fraction of +critical rotation. This campaign establishes neither a 50% nor a 99% critical +rotating equilibrium. + +## Integration decision and reproducibility + +Keep the prepared cross-action optimization: it preserves the mathematical +action and removes the demonstrated cost of rebuilding geometry inside the +two-way stellar inverse. Keep consistent gravity Schur refinement as an +optional typed policy. The measured setting is one sweep at relaxation 0.5; +use it alone for the tested n = 1.5 state and combine it with stellar LDU for +the tested rotating state. These are measured choices for these states, not +universal parameter recommendations. The nine-mode coarse prototype has not +earned promotion, and no production default is changed. + +The production policy and cross-action files, with their tests, match their +isolated hypothesis branches. Changes are committed on the integration +branch; the original working checkout and its unrelated quadrature formatting +edit are preserved. No central-density constraint or physics-facing API +requirement was added. + +The retained campaign source revisions and tracked recipes are: + +| Evidence directory | Source | Recipe in `experiments/` | +|---|---|---| +| `screen_v1` | `018fc1c` | `preconditioner_screen_cases_2026-09-18.json` | +| `candidates_v1` | `dc8d4b0` | `preconditioner_candidates_2026-09-18.json` | +| `confirmation_v1` | `87569c0` | `preconditioner_confirmation_2026-09-18.json` | +| `newton_v1` | `bed1893` | `preconditioner_newton_cases_2026-09-18.json` | +| `rotation_coupling_v1` | `5298d88` | `preconditioner_rotation_coupling_2026-09-18.json` | +| `rotation_newton_v1` | `83afd82` | `preconditioner_rotation_newton_2026-09-18.json` | + +Use `run_plateau_campaign.py` with the selected recipe, a build configured +from the recorded `provenance.json`, and a fresh output directory. It requires +a clean committed source tree, checks the build's source directory, copies +the executable before running cases, records hashes, and refuses to overwrite +an earlier campaign. The exact build and execution commands are retained in +each campaign. The mesh is versioned as `sandbox.smesh`; saved-state input +paths and hashes identify the earlier producer artifacts. + +For frozen campaigns, `check_preconditioner_benchmark.py` independently checks +the artifacts. Place its `--output` outside the campaign directory to retain +the original evidence unchanged. The Newton audits and their source scripts +are retained alongside the campaigns, with solver and physical failures +reported separately from artifact-integrity checks. Six lightweight audit +tests cover failed solves, raw versus repaired directions, incorrect policy +labels, hash failures, incomplete producers, and unmatched Newton budgets. + +The final comparison figure, script, and input hashes are in +`final_comparison_figures/` under the evidence directory. It shows reported +Krylov histories and independently verified endpoints separately, includes +setup and verification cost, and labels the unequal rotating iteration caps +and failed controls explicitly.