MeanField/experiments/PRECONDITIONER_IMPROVEMENT_2026-09-18.md

Preconditioner improvement: matched frozen Newton systems

This experiment changes preconditioning only. The complete Newton Jacobian, normalization, physical residual, mesh, and linear acceptance gate are held fixed. The date in branch and artifact names is an experiment identifier; provenance files contain the actual UTC execution times.

Reproduction and scope

Integration branch: research/preconditioner-benchmarks-20260918, based on 10adf7e. Hypotheses are retained separately on research/gravity-consistent-schur-20260918, research/preconditioner-cached-cross-actions-20260918, and research/coupled-shape-coarse-20260918.

The experiment workspace is /Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d. Evidence is stored under diagnostics/preconditioner_improvement_2026-09-18 in that workspace. Each campaign retains its clean source commit, build configuration, copied executable and hash, mesh hash, exact commands, input/output hashes, and logs. Runs are serial, without overlapping builds or FEM jobs during timing.

Frozen comparisons start FGMRES from zero on the same saved state and right-hand side. Each role constructs a fresh inverse, including its own specification border responses. The restart length is 160, the initial cap is 200 iterations, and the required relative full-J true defect is 0.03. The independently applied full Jacobian decides acceptance; the Krylov recurrence flag alone does not. All accepted/trial state buffers, normalization factors, preparation generation, and a deterministic full-J probe must remain unchanged.

Costs include inverse construction, normalized binding, Krylov setup, Krylov solve, and full-J verification. Extra diagnostic applications and artifact I/O are recorded separately. A speedup to convergence is reported only if both methods meet the same true gate. Less work spent on a failed solve is not a time-to-convergence improvement. A frozen linear pass is not nonlinear or physical convergence.

Existing two-way coupling: negative controls

screen_v1 uses source 018fc1c, before the cached cross-action optimization.

Saved state	Inverse	Iterations	Full-J relative defect	Complete verified cost (s)
n = 1.5, J = 0 plateau	independent	200	0.0590170471860	87.9302
same	stellar LDU	200	0.0600668598825	599.5473
n = 1, J = 0.1 plateau	independent	200	0.0826534298794	86.9409
same	stellar LDU	200	0.0720550396371	594.2053

All four solves fail the 0.03 gate. Stellar LDU incurs 6.82–6.83 times the total cost; it slightly worsens the n = 1.5 result and reduces the rotating defect by 12.82%. Mean inverse application time rises from approximately 0.19 s to 2.7 s, while full-J actions remain approximately 0.23 s. The uncached gravity-to-material force column explains the application-cost anomaly.

Poisson contributes 95.53% of the squared n = 1.5 baseline defect and 98.28% with stellar LDU. For the rotating state those fractions are 85.00% and 98.10%; LDU reduces the gradient-row norm by 94.16% but the Poisson norm by only 6.35%. These are global normalized block diagnostics, not proof of an exclusively exterior error. No exterior weighting is changed.

The independent artifact audit passes 65 provenance checks and 93 checks per case. Both saved states and the normalization data exactly match their producer artifacts. The two independent baselines reproduce the earlier failed solves.

Candidate mechanisms

1. Prepared cross action. Reuse the complete prepared gravity force with zero density and displacement directions to apply its gravity-gradient column. This is the same mathematical coupling with cached geometry; it does not strengthen the inverse. See PRECONDITIONER_CACHED_CROSS_ACTION_2026-09-18.md.
2. Consistent gravity Schur refinement. Keep the existing AMG action, but apply a fixed number of corrections against B R B^T, where R is the same mass inverse used in flux recovery. Fixed work preserves linear stationarity for stationary child inverses. Damping does not guarantee contraction; it must earn its additional cost. See GRAVITY_CONSISTENT_SCHUR_2026-09-18.md.
3. Experimental coupled shape coarse inverse. Use nine fixed geometric modes, complete their material/gravity responses, and construct a checked low-rank correction to the normalized inverse. Modes do not depend on the Newton right-hand side or previous Krylov results. A rank/overlap guard explicitly falls back to the underlying inverse when the correction cannot be trusted. Coarse setup and storage are counted, and actual FEM subspace identities are independently checked before solving.

First candidate screen

candidates_v1 uses source dc8d4b0 and the same n = 1.5, J = 0 plateau state. Every fresh independent control reproduces relative defect 0.059017047186 at 200 iterations. Complete control costs remain between 87.53 and 88.25 s.

Candidate	Iterations	Full-J relative defect	Gate	Complete verified cost (s)
Cached stellar LDU	200	0.0600668598779	fail	117.4845
Consistent gravity Schur, k = 1, omega = 0.5	132	0.0292091549685	pass	72.3981
Cached stellar LDU + consistent gravity Schur	121	0.0285619160766	pass	84.7104
Nine-mode coupled shape coarse correction	200	0.0668657989082	fail	97.0564

At the same iteration cap, the cached LDU reduces total execution time by a factor of 5.1032 relative to the old path, while mean inverse application time drops by a factor of 8.0425. This is an execution improvement, not a convergence improvement. Saved inputs and the full-J probe agree exactly; the final normalized correction changes by only 2.7051e-11 relatively. Both LDU solves fail, and the cached version remains 34.2% more expensive than the independent control on this state.

Gravity refinement is the first candidate to cross the original gate. It lowers the Poisson-row defect by 55.4% and the gradient-row defect by 15.0%. The squared Poisson share drops from 95.53% to 77.59%; surface and hydrostatic defects grow, so the improvement is judged by the verified global norm. Work changes from 400 to 396 mass-inverse applications, 200 to 264 AMG Schur applications, and 200 to 132 outer inverse applications. At this iteration cap, the default does not pass the same gate, so dividing its failed 88 s attempt by 72.4 s is not a convergence speedup. The combined LDU saves another eleven iterations relative to gravity refinement alone but takes 17% more time overall.

The coarse candidate has all nine modes active, minimum overlap lower bound 0.79465, and no fallback. Its extra construction costs 8.6012 s, including mode completion and eighteen setup Jacobian applications; it stores 28,491,680 bytes in persistent vectors. The actual FEM subspace check passes. Nevertheless, its 200-iteration defect worsens by 13.3%. This is a negative numerical result for these modes on this state, not evidence about every possible coarse space or rapidly rotating model. The prototype remains confined to experiments.

Validation status

The integrated library and experiment driver compile with the repository's Release/ThinLTO configuration. Source dc8d4b0 passes 20 focused serial test cases (441,978 assertions), including FEM equivalence, geometry/discretization refresh, fixed-work algebra, and coarse inverse tests. Distributed unit and coarse tests pass on two ranks (13 cases and 213 assertions per rank).

The broad real-FEM MPI run aborts because the existing METIS partition gives one rank no compactified vacuum element, which the mapped mass constructor requires. The failing log is retained; this campaign does not claim distributed FEM Schur validation. See the gravity refinement note for the exact scope and evidence.

The evidence checker passes 22 lightweight tests. Independent reviews of the cached LDU and gravity-only cases validate provenance, state/operator invariants, true gates, and child-work accounting. Reviews are stored outside immutable campaign directories.

Source fcb7833 adds owning experiment-only Newton recipes through the existing ADL preparation interface. They use the same concrete prepared inverses as the frozen benchmark and the existing context refresh/border lifecycle. The driver supports default, gravity-Schur, and stellar-LDU recipes for ordinary Newton, positivity, curvature, and positivity-curvature runs. This adds no public physics-facing solver API. The updated driver builds successfully at 87569c0.

Equal-accuracy confirmation

confirmation_v1 uses source 87569c0, a restart length of 160, and a maximum of 600 iterations for both roles. The target remains a freshly verified relative defect of 0.03.

State	Inverse	Iterations	Full-J relative defect	Gate	Complete verified cost (s)
n = 1.5, J = 0 plateau	independent	231	0.0290033752814	pass	100.9943
same	gravity Schur k = 1, omega = 0.5	132	0.0292091549685	pass	72.3727
n = 1, J = 0.1 plateau	independent	600	0.0739197073237	fail	257.9399
same	gravity Schur k = 1, omega = 0.5	600	0.0408505998829	fail	321.5918

For n = 1.5, the frozen linear solve speedup to the same requested accuracy is 1.39548x, with 28.34% less complete cost and 42.86% fewer iterations. The candidate repeats its earlier 132 iterations, with timing close to the earlier 72.3981 s. Both endpoints finish slightly below the same stopping threshold; neither tolerance nor the full-J check was relaxed.

Rotation behaves differently. Refinement reduces the 600-step defect by 44.74%, but costs 24.68% more and still fails the gate. The remaining gradient and Poisson defects contribute 0.02664 and 0.03018 to the global relative norm. The older rotating LDU control reduced the gradient contribution to 0.001813 while leaving a large Poisson contribution. This supports one complementary LDU+Schur follow-up; it neither proves a gradient floor nor permits multiplying the separate gains.

All 73 provenance checks and 186 case checks pass. The reproducible figure and plotting provenance are in confirmation_v1_figures; reported Krylov histories are shown separately from independently verified final points. Plotting used an existing compatible Python environment and installed no dependencies.

Easy nonlinear and physical regression

The first two newton_v1 cases use source bed1893, the same n = 1, J = 0 model with no central-density constraint, positivity strategy, and identical tolerances. Both converge in six accepted full steps, with no repairs or rejections. Initial physical diagnostics and layout are byte-identical; all linear true gates pass.

Measure	Independent	Gravity Schur k = 1, omega = 0.5
Total linear iterations	307	258
Summed Newton iteration time (s)	191.068	199.375
Summed linear solve time (s)	125.035	132.028
Complete diagnostic-run wall time (s)	251.720	261.285
Final nonlinear residual	1.65157e-13	1.05839e-13

Both final residuals are below the shared 1.81187e-12 nonlinear target. Fewer iterations do not produce an easy-case speedup: Newton time rises by 4.35% and full run time by 3.80%. The full run includes physical validation and artifact I/O; it is not presented as pure solver cost.

All eight virial errors (four definitions at quadrature orders 14 and 18) lie between 2.61811e-7 and 3.73092e-7 for both models, comfortably below 1e-5. The maximum endpoint coefficient difference is 2.679e-11; the radius difference is 3.48e-14. The independent paired audit also checks mass, density/enthalpy admissibility and local EOS metrics. This preserves the existing physical regression, rather than adding independent ESTER validation.

The two hard nonlinear cases and the targeted rotating coupling follow-up are still running or pending. Production defaults remain unchanged: the stronger inverse is an optional typed policy, and the coarse candidate remains an experiment.