MeanField/experiments/COUPLED_TRIAL_CORRECTION_2026-09-18.md

Bounded coupled second-order trial correction

Branch research/coupled-trial-correction-20260918, based on the completed nonlinear obstruction investigation at 2944dd5. The configured worktree retains its existing name, worktrees/preconditioner-improvement, so that build reuse does not disturb the preceding branch or immutable executables.

Hypothesis

At the recorded n = 1.5, J = 0 endpoint, tenfold tighter linear accuracy did not improve the useful trial range. The full directional derivative was consistent with the prepared residual. Gravity-only correction reduced the remainder norm but increased its adverse projection through hydrostatic, EOS and surface balance. The next hypothesis is that correcting the entire trial remainder can preserve the intended Newton response over larger steps.

For the captured base residual F0 and verified Newton direction p, preserve L(alpha) = F0 + alpha J0 p. At a prepared candidate y, solve

J(y) c = -(F(y) - L(alpha)),

then prepare and evaluate y+c. All equations, normalization factors and ordinary full-residual acceptance thresholds remain unchanged. A separate auxiliary true-defect check is required; the base Newton report is never replaced by a correction report. The locally second-order argument requires sufficient residual smoothness and a bounded inverse, neither of which is assumed to hold uniformly in this model.

Implementation contracts

• A third, defaulted compile-time trial-path policy composes with the established enthalpy and gravity policies. Existing two-policy recipes retain their types and behavior. The new policy is incompatible with the eliminated-gravity path until a corresponding reduced-manifold contract is designed; unsupported combinations fail at compile time.
• The auxiliary solve has its own backend, a finite iteration budget and an independent application of the complete candidate Jacobian. Its initial options are relative tolerance 0.03, absolute tolerance 1e-14, at most160 iterations and restart160.
• Limit the correction norm to alpha||p|| by default. The initial work limit is one actual auxiliary solve per Newton iteration, and one solve per selected candidate. Runtime expert controls change these bounds explicitly.
• Select correction only after a prepared candidate fails the existing acceptance predicate. Reuse the exact configured metric for selecting between original and corrected candidates. An already acceptable step incurs no coupled solve; an unprepared candidate cannot invoke one.
• Recheck the same selected admissibility policy and actual geometry after correction. Retain the original candidate if the auxiliary solve fails, the norm cap rejects it, preparation/admissibility fails, or its metric worsens. Restoring a fallback includes fresh operator preparation, not merely copying state vectors.
• The exported numerical core and trial transaction are shared by production and frozen diagnostics. The physics-facing model API is unchanged.

Initial screen

The committed coupled_trial_screen_2026-09-18.json records complete controls and immutable input paths. All heavy builds and numerical cases run serially. Evidence is written to diagnostics/coupled_trial_2026-09-18/ in the task workspace; individual campaign directories are immutable.

Use the previously saved late eta0.03 direction, independently reapplying the full Jacobian at the restored base before using it. This avoids conflating a new linear solve with the new trial map. Test both raw and gravity-corrected starting candidates at alpha1/16,1/64,1/256,1/1024. The middle two previously prepared but failed the merit test. The largest failed pressure preparation; the smallest already decreased the residual and must not trigger correction.

The seed control uses its saved eta0.03 direction and alpha1 with the gravity-corrected path. Its candidate already passes; it should preserve that result without auxiliary coupled work. Each frozen alpha is an independent trial with a fresh per-iteration budget, not a simulated live line search.

Record inputs/factors/layout, F0,p,J0p, the actual auxiliary RHS and J(y)c, candidate and returned state/residual vectors, true defects, norm-cap gates, selection/fallback decisions, restoration invariants and complete work. Useful frozen progress is a prerequisite for a bounded nonlinear campaign. Any accepted model must also be checked for virial balance and internal admissibility; a lower residual alone is insufficient.

Result and present recommendation

The opt-in implementation is complete and tested. It creates a newly Armijo-acceptable frozen candidate at alpha = 1/256 for both trial paths, but does not establish a production convergence or speed improvement. Do not enable it by default on the strength of these measurements.

The nominal fourfold increase over the previously accepted alpha = 1/1024 overstates the realized displacement. The coupled correction largely reverses the original Newton direction. Its own nonlinear curvature is much larger than the checked auxiliary linear error, and the useful candidate remains internally nonphysical despite passing enthalpy-based virial checks.

Each frozen alpha has a fresh one-attempt budget. In a live search that spends the default budget at the unsuccessful 1/64 trial, the later 1/256 trial cannot use this correction. Thus these independent samples are not a measured live line-search speedup. The existing seed protocol projects radial profiles and regenerates fields and shape; it cannot faithfully resume the arbitrary saved late state. No diagnostic mutation of accepted state was used to bypass that restriction, and no new initialization API was added.

Numerical results

The common late state has ||F0|| = 1.3966991043973936e-5 and the saved outer direction again passes the full-J gate with relative defect 0.02955918315. All four distinct auxiliary solves pass the independent 0.03 relative gate and the norm cap. No auxiliary failure or relaxed acceptance threshold explains the outcomes below.

| Prepared path | alpha | Before correction: ||F||/||F0|| | Corrected: ||F||/||F0|| | Selection | Auxiliary iterations | Added transaction time | |---|---:|---:|---:|---|---:|---:| | Raw | 1/64 | 1.80726810 | 1.03360191 | Improved, still fails Armijo | 34 | 27.33 s | | Raw | 1/256 | 1.00250966 | 0.99746464 | Passes Armijo | 52 | 37.59 s | | Gravity-corrected | 1/64 | 1.08032296 | 1.09566246 | Worse; original restored | 55 | 47.60 s | | Gravity-corrected | 1/256 | 1.00202482 | 0.99810801 | Passes Armijo | 62 | 43.53 s |

At 1/16, both raw preparations fail at the pressure-force thermodynamic-domain check, before any coupled solve. At 1/1024, the original candidates already pass Armijo and receive no coupled solve: their residual ratios are 0.9991695118 (raw) and 0.9993941386 (gravity-corrected).

The Lane–Emden seed's gravity-corrected full step likewise remains acceptable without a coupled solve. All frozen probes preserve accepted buffers, normalization factors and direction buffers; the final residual and Jp at the restored base agree exactly with the initially captured values.

Raw timings above come from the uncontended repeat. An initially launched artifact audit accidentally rebuilt a gravity-index set for every vector entry and overlapped the first raw case. It was stopped, the checker was made linear in state size, and the raw case was repeated. Its trial vectors and residuals reproduce the first run. The original evidence is retained; its raw timing is excluded. All 50 saved vector files are byte-identical in the repeat. See AUDIT_RUN_NOTES.md in the evidence folder.

The added times include refresh, solve, independent verification, candidate preparation, and any fallback restoration. They do not include the original trial preparation or captured base action. At 1/256 the full candidate-path costs are 43.86 s (raw) and 63.45 s (gravity-first), versus 6.30 s and 18.59 s for their already acceptable 1/1024 controls. Outer Newton work, previous rejected trials, base capture and physical postprocessing are separate. Timings are single observations, not statistical performance estimates.

Why a verified solve does not ensure useful correction

For the actual candidate y, let b = L - F(y), delta = J(y)c - b and E = F(y+c) - F(y) - J(y)c. The exact saved-vector identity is

F(y+c) - L = delta + E.

The auxiliary gate bounds delta. It does not bound E. At alpha = 1/256, ||E|| is about 11.28 times the verified defect for the raw path and 254 times for the gravity-first path. For the two gravity-first attempts, ||E||/||c||^2 is 13.5756 and 13.5358; the raw 1/256 value is 13.5271. This is evidence that the correction's own quadratic curvature, rather than inaccurate solution of its linear equation, dominates its remaining nonlinear error.

On the gravity-first path at 1/256, the affine-target remainder actually grows from 8.87747e-8 to 6.23769e-7 (7.03 times), despite the improved merit. The raw path reduces that remainder by about 3.165 times. Gravity completion before the coupled solve therefore does not help this particular comparison.

The correction/direction cosines are -0.9999330 (raw) and -0.9999942 (gravity-first). The realized displacement norms at 1/256 are only 1.7596 and 1.2342 times their respective raw-direction 1/1024 displacement norms. On the gravity-first path, ||c||/||b|| increases from 480.8 to 2418.1 when alpha decreases fourfold. These samples do not demonstrate a stable small second-order correction regime. They do not identify a null mode either. Tightening the auxiliary tolerance is consequently a weak next target.

Near collinearity is insufficient to conclude that scalar damping is equivalent: small transverse displacements can have large Jacobian action. The final scalar controls below address that attribution directly.

Matched scalar controls

For each selected 1/256 candidate z, compute t = dot(z-x,p)/dot(p,p), then freshly prepare x+t*p through the existing --replay-candidate path. The input generator validates and hashes the source artifacts before writing normalized candidates. No new linear solve, repair, or acceptance threshold is introduced.

| Coupled candidate being matched | Scalar t | Scalar ||F||/||F0|| | Coupled ||F||/||F0|| | Scalar residual decrease | Coupled residual decrease | |---|---:|---:|---:|---:|---:| | Raw + coupled | 0.00171818301767 | 0.9988437736 | 0.9974646400 | 0.115623% | 0.253536% | | Gravity-first + coupled | 0.00120521815790 | 0.9990306561 | 0.9981080052 | 0.096934% | 0.189199% |

Both scalar controls prepare and pass the ordinary Armijo criterion at t. Their normalized base and full base residual reproduce the coupled screen byte for byte. The replay uses the independent preconditioner configuration because that is the existing replay interface; no inverse or linear solve is used in evaluating these candidates. The equation operator and normalization are the same. The campaign executable is byte-identical to both screens.

The coupled candidate produces 2.19 times the residual decrease of the matched scalar control on the raw path, and 1.95 times on the gravity-first path. Thus the small transverse changes have measured nonlinear benefit. The result cannot be dismissed as scalar damping alone, but it is a small absolute gain bought with an extra full coupled solve. This does not measure an entire Newton iteration or prove amortized speedup. The scalar replay records preparation and residuals; it is not an additional physical-model certification or quadrature study.

The next bounded question is whether damping or otherwise limiting the already computed correction can retain this transverse benefit while reducing its own nonlinear curvature, using additional preparations instead of additional full solves. Test that together with a work-aware attempt trigger; simply increasing the attempt budget can spend multiple outer-solve equivalents on rejected candidates. Keep normalization, equations, physical validation and true-defect gates fixed during that comparison. This follow-up is not implemented here.

Physical validation

Every available selected state is freshly prepared and evaluated at quadrature orders 14 and 18, using the existing field reconstruction, rotation, Piola transforms and physical diagnostics. This postprocessing is separate from timed trial selection and does not change acceptance.

At alpha = 1/256 and quadrature order 18:

Quantity	Raw + coupled	Gravity-first + coupled
Enthalpy-pressure energy virial error	4.05889e-6	4.06134e-6
Enthalpy-pressure force virial error	2.56495e-6	2.56840e-6
Absolute mass error	3.13178e-6	3.13420e-6
Angular momentum Jz	0	0
Minimum sampled density	-8.28793e-4	-8.24696e-4
Negative density samples	9600	9600
Negative-density mass fraction	2.89684e-5	2.89654e-5
Scaled EOS density mismatch RMS	1.28264e-4	1.28231e-4

Enthalpy-pressure virial errors are below 1e-5, with order-14 to order-18 differences near 1.83e-9. However, pressure inferred from the negative density is undefined for this EOS, so the corresponding density-pressure virials are undefined too. A finite enthalpy-pressure virial is not a substitute for internal admissibility or pointwise EOS agreement. Negative-density volume fractions also change with quadrature (about 2.83% to 1.87% on the gravity-first candidate). No internally correct, converged stellar model, continuum accuracy, or mesh-convergence claim follows from this trial.

Tests and reproduction

• The Release tests and plateau_diagnostic targets build successfully.
• Fifteen focused correction/transaction cases pass in serial (584 assertions) and on two MPI ranks (586 assertions per rank). These include analytic mixed curvature, a lying backend, bounded real FGMRES failure, norm caps, stale preparation, zero remainders, nonfinite/overflow handling, empty-rank collectives, budgets, rollback and exceptions.
• Thirty-five relevant stellar/Newton/gravity regression cases pass with 356,933 assertions, including typed polynomial composition, optional borders, original-report preservation and observer/rollback behavior.
• The independent checker passes 27 synthetic tests. The completed screen and uncontended repeat pass artifact, provenance and physical-replay consistency checks. Its observational mechanism statistics distinguish the checked linear defect from the correction's own nonlinear remainder. Their final audits pass 2,105 and 885 checks; the separate scalar-control audit passes 68 hash, projection, shared-base and replay checks.
• Full stellar MPI operation remains outside this validation; its existing interpolation restriction is unchanged.

Evidence root: /Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/coupled_trial_2026-09-18. tests_provenance.json records the test executable and log hashes. The campaign directories contain complete commands, source revisions, immutable executables and hashes of mesh, inputs and outputs. All three campaign executables are byte-identical (SHA256 begins 620d625636bc77b1).

Source checkpoints: implementation 0d87b8b; initial campaign/checker 2d0d32b; checker fix and raw timing repeat 5aa48dc; scalar-control generator 0c926f4. The original checkout and its unrelated quadrature formatting change are preserved.

From this worktree, reproduce the initial screen with experiments/run_plateau_campaign.py, the committed coupled_trial_screen_2026-09-18.json, the configured Release build, sandbox.smesh, and a new output directory. Use coupled_trial_raw_repeat_2026-09-18.json for the raw repeat. Run check_coupled_trial.py CAMPAIGN --output OUTSIDE_CAMPAIGN.json afterward. make_coupled_scalar_controls.py SCREEN FRESH_OUTPUT generates the two normalized scalar inputs and cases.json for the existing campaign runner. All runs use one FEM rank; the seed and outer direction inputs are saved from the preceding investigation and are hashed into the new evidence.