MeanField/experiments/COUPLED_TRIAL_CORRECTION_2026-09-18.md

# Bounded coupled second-order trial correction

Branch `research/coupled-trial-correction-20260918`, based on the completed
nonlinear obstruction investigation at `2944dd5`. The configured worktree
retains its existing name, `worktrees/preconditioner-improvement`, so that
build reuse does not disturb the preceding branch or immutable executables.

## Hypothesis

At the recorded n = 1.5, J = 0 endpoint, tenfold tighter linear accuracy did
not improve the useful trial range. The full directional derivative was
consistent with the prepared residual. Gravity-only correction reduced the
remainder norm but increased its adverse projection through hydrostatic,
EOS and surface balance. The next hypothesis is that correcting the entire
trial remainder can preserve the intended Newton response over larger steps.

For the captured base residual F0 and verified Newton direction p, preserve
L(alpha) = F0 + alpha J0 p. At a prepared candidate y, solve

    J(y) c = -(F(y) - L(alpha)),

then prepare and evaluate y+c. All equations, normalization factors and
ordinary full-residual acceptance thresholds remain unchanged. A separate
auxiliary true-defect check is required; the base Newton report is never
replaced by a correction report. The locally second-order argument requires
sufficient residual smoothness and a bounded inverse, neither of which is
assumed to hold uniformly in this model.

## Implementation contracts

- A third, defaulted compile-time trial-path policy composes with the
  established enthalpy and gravity policies. Existing two-policy recipes
  retain their types and behavior. The new policy is incompatible with the
  eliminated-gravity path until a corresponding reduced-manifold contract
  is designed; unsupported combinations fail at compile time.
- The auxiliary solve has its own backend, a finite iteration budget and an
  independent application of the complete candidate Jacobian. Its initial
  options are relative tolerance 0.03, absolute tolerance 1e-14, at most160
  iterations and restart160.
- Limit the correction norm to alpha||p|| by default. The initial work limit
  is one actual auxiliary solve per Newton iteration, and one solve per
  selected candidate. Runtime expert controls change these bounds explicitly.
- Select correction only after a prepared candidate fails the existing
  acceptance predicate. Reuse the exact configured metric for selecting
  between original and corrected candidates. An already acceptable step
  incurs no coupled solve; an unprepared candidate cannot invoke one.
- Recheck the same selected admissibility policy and actual geometry after
  correction. Retain the original candidate if the auxiliary solve fails,
  the norm cap rejects it, preparation/admissibility fails, or its metric
  worsens. Restoring a fallback includes fresh operator preparation, not
  merely copying state vectors.
- The exported numerical core and trial transaction are shared by production
  and frozen diagnostics. The physics-facing model API is unchanged.

## Initial screen

The committed `coupled_trial_screen_2026-09-18.json` records complete controls
and immutable input paths. All heavy builds and numerical cases run serially.
Evidence is written to `diagnostics/coupled_trial_2026-09-18/` in the task
workspace; individual campaign directories are immutable.

Use the previously saved late eta0.03 direction, independently reapplying the
full Jacobian at the restored base before using it. This avoids conflating a
new linear solve with the new trial map. Test both raw and gravity-corrected
starting candidates at alpha1/16,1/64,1/256,1/1024. The middle two previously
prepared but failed the merit test. The largest failed pressure preparation;
the smallest already decreased the residual and must not trigger correction.

The seed control uses its saved eta0.03 direction and alpha1 with the
gravity-corrected path. Its candidate already passes; it should preserve that
result without auxiliary coupled work. Each frozen alpha is an independent
trial with a fresh per-iteration budget, not a simulated live line search.

Record inputs/factors/layout, F0,p,J0p, the actual auxiliary RHS and J(y)c,
candidate and returned state/residual vectors, true defects, norm-cap gates,
selection/fallback decisions, restoration invariants and complete work.
Useful frozen progress is a prerequisite for a bounded nonlinear campaign.
Any accepted model must also be checked for virial balance and internal
admissibility; a lower residual alone is insufficient.

## Result and present recommendation

The opt-in implementation is complete and tested. It creates a newly
Armijo-acceptable frozen candidate at alpha = 1/256 for both trial paths,
but does not establish a production convergence or speed improvement. Do not
enable it by default on the strength of these measurements.

The nominal fourfold increase over the previously accepted alpha = 1/1024
overstates the realized displacement. The coupled correction largely reverses
the original Newton direction. Its own nonlinear curvature is much larger
than the checked auxiliary linear error, and the useful candidate remains
internally nonphysical despite passing enthalpy-based virial checks.

Each frozen alpha has a fresh one-attempt budget. In a live search that spends
the default budget at the unsuccessful 1/64 trial, the later 1/256 trial
cannot use this correction. Thus these independent samples are not a measured
live line-search speedup. The existing seed protocol projects radial profiles
and regenerates fields and shape; it cannot faithfully resume the arbitrary
saved late state. No diagnostic mutation of accepted state was used to bypass
that restriction, and no new initialization API was added.

## Numerical results

The common late state has ||F0|| = 1.3966991043973936e-5 and the saved outer
direction again passes the full-J gate with relative defect 0.02955918315.
All four distinct auxiliary solves pass the independent 0.03 relative gate
and the norm cap. No auxiliary failure or relaxed acceptance threshold
explains the outcomes below.

| Prepared path | alpha | Before correction: ||F||/||F0|| | Corrected: ||F||/||F0|| | Selection | Auxiliary iterations | Added transaction time |
|---|---:|---:|---:|---|---:|---:|
| Raw | 1/64 | 1.80726810 | 1.03360191 | Improved, still fails Armijo | 34 | 27.33 s |
| Raw | 1/256 | 1.00250966 | 0.99746464 | Passes Armijo | 52 | 37.59 s |
| Gravity-corrected | 1/64 | 1.08032296 | 1.09566246 | Worse; original restored | 55 | 47.60 s |
| Gravity-corrected | 1/256 | 1.00202482 | 0.99810801 | Passes Armijo | 62 | 43.53 s |

At 1/16, both raw preparations fail at the pressure-force thermodynamic-domain
check, before any coupled solve. At 1/1024, the original candidates already
pass Armijo and receive no coupled solve: their residual ratios are
0.9991695118 (raw) and 0.9993941386 (gravity-corrected).

The Lane–Emden seed's gravity-corrected full step likewise remains acceptable
without a coupled solve. All frozen probes preserve accepted buffers,
normalization factors and direction buffers; the final residual and Jp at
the restored base agree exactly with the initially captured values.

Raw timings above come from the uncontended repeat. An initially launched
artifact audit accidentally rebuilt a gravity-index set for every vector
entry and overlapped the first raw case. It was stopped, the checker was
made linear in state size, and the raw case was repeated. Its trial vectors
and residuals reproduce the first run. The original evidence is retained;
its raw timing is excluded. All 50 saved vector files are byte-identical in
the repeat. See `AUDIT_RUN_NOTES.md` in the evidence folder.

The added times include refresh, solve, independent verification, candidate
preparation, and any fallback restoration. They do not include the original
trial preparation or captured base action. At 1/256 the full candidate-path
costs are 43.86 s (raw) and 63.45 s (gravity-first), versus 6.30 s and 18.59 s
for their already acceptable 1/1024 controls. Outer Newton work, previous
rejected trials, base capture and physical postprocessing are separate.
Timings are single observations, not statistical performance estimates.

## Why a verified solve does not ensure useful correction

For the actual candidate y, let b = L - F(y), delta = J(y)c - b and
E = F(y+c) - F(y) - J(y)c. The exact saved-vector identity is

    F(y+c) - L = delta + E.

The auxiliary gate bounds delta. It does not bound E. At alpha = 1/256,
||E|| is about 11.28 times the verified defect for the raw path and 254 times
for the gravity-first path. For the two gravity-first attempts,
||E||/||c||^2 is 13.5756 and 13.5358; the raw 1/256 value is 13.5271.
This is evidence that the correction's own quadratic curvature, rather
than inaccurate solution of its linear equation, dominates its remaining
nonlinear error.

On the gravity-first path at 1/256, the affine-target remainder actually
grows from 8.87747e-8 to 6.23769e-7 (7.03 times), despite the improved merit.
The raw path reduces that remainder by about 3.165 times. Gravity completion
before the coupled solve therefore does not help this particular comparison.

The correction/direction cosines are -0.9999330 (raw) and -0.9999942
(gravity-first). The realized displacement norms at 1/256 are only 1.7596
and 1.2342 times their respective raw-direction 1/1024 displacement norms.
On the gravity-first path, ||c||/||b|| increases from 480.8 to 2418.1 when
alpha decreases fourfold. These samples do not demonstrate a stable small
second-order correction regime. They do not identify a null mode either.
Tightening the auxiliary tolerance is consequently a weak next target.

Near collinearity is insufficient to conclude that scalar damping is
equivalent: small transverse displacements can have large Jacobian action.
The final scalar controls below address that attribution directly.

## Matched scalar controls

For each selected 1/256 candidate z, compute
t = dot(z-x,p)/dot(p,p), then freshly prepare x+t*p through the existing
`--replay-candidate` path. The input generator validates and hashes the
source artifacts before writing normalized candidates. No new linear solve,
repair, or acceptance threshold is introduced.

| Coupled candidate being matched | Scalar t | Scalar ||F||/||F0|| | Coupled ||F||/||F0|| | Scalar residual decrease | Coupled residual decrease |
|---|---:|---:|---:|---:|---:|
| Raw + coupled | 0.00171818301767 | 0.9988437736 | 0.9974646400 | 0.115623% | 0.253536% |
| Gravity-first + coupled | 0.00120521815790 | 0.9990306561 | 0.9981080052 | 0.096934% | 0.189199% |

Both scalar controls prepare and pass the ordinary Armijo criterion at t.
Their normalized base and full base residual reproduce the coupled screen
byte for byte. The replay uses the independent preconditioner configuration
because that is the existing replay interface; no inverse or linear solve is
used in evaluating these candidates. The equation operator and normalization
are the same. The campaign executable is byte-identical to both screens.

The coupled candidate produces 2.19 times the residual decrease of the
matched scalar control on the raw path, and 1.95 times on the gravity-first
path. Thus the small transverse changes have measured nonlinear benefit.
The result cannot be dismissed as scalar damping alone, but it is a small
absolute gain bought with an extra full coupled solve. This does not measure
an entire Newton iteration or prove amortized speedup. The scalar replay
records preparation and residuals; it is not an additional physical-model
certification or quadrature study.

The next bounded question is whether damping or otherwise limiting the
already computed correction can retain this transverse benefit while
reducing its own nonlinear curvature, using additional preparations instead
of additional full solves. Test that together with a work-aware attempt
trigger; simply increasing the attempt budget can spend multiple outer-solve
equivalents on rejected candidates. Keep normalization, equations, physical
validation and true-defect gates fixed during that comparison. This follow-up
is not implemented here.

## Physical validation

Every available selected state is freshly prepared and evaluated at
quadrature orders 14 and 18, using the existing field reconstruction,
rotation, Piola transforms and physical diagnostics. This postprocessing is
separate from timed trial selection and does not change acceptance.

At alpha = 1/256 and quadrature order 18:

| Quantity | Raw + coupled | Gravity-first + coupled |
|---|---:|---:|
| Enthalpy-pressure energy virial error | 4.05889e-6 | 4.06134e-6 |
| Enthalpy-pressure force virial error | 2.56495e-6 | 2.56840e-6 |
| Absolute mass error | 3.13178e-6 | 3.13420e-6 |
| Angular momentum Jz | 0 | 0 |
| Minimum sampled density | -8.28793e-4 | -8.24696e-4 |
| Negative density samples | 9600 | 9600 |
| Negative-density mass fraction | 2.89684e-5 | 2.89654e-5 |
| Scaled EOS density mismatch RMS | 1.28264e-4 | 1.28231e-4 |

Enthalpy-pressure virial errors are below 1e-5, with order-14 to order-18
differences near 1.83e-9. However, pressure inferred from the negative density
is undefined for this EOS, so the corresponding density-pressure virials
are undefined too. A finite enthalpy-pressure virial is not a substitute for
internal admissibility or pointwise EOS agreement. Negative-density volume
fractions also change with quadrature (about 2.83% to 1.87% on the
gravity-first candidate). No internally correct, converged stellar model,
continuum accuracy, or mesh-convergence claim follows from this trial.

## Tests and reproduction

- The Release `tests` and `plateau_diagnostic` targets build successfully.
- Fifteen focused correction/transaction cases pass in serial (584
  assertions) and on two MPI ranks (586 assertions per rank). These include
  analytic mixed curvature, a lying backend, bounded real FGMRES failure,
  norm caps, stale preparation, zero remainders, nonfinite/overflow handling,
  empty-rank collectives, budgets, rollback and exceptions.
- Thirty-five relevant stellar/Newton/gravity regression cases pass with
  356,933 assertions, including typed polynomial composition, optional
  borders, original-report preservation and observer/rollback behavior.
- The independent checker passes 27 synthetic tests. The completed screen
  and uncontended repeat pass artifact, provenance and physical-replay
  consistency checks. Its observational mechanism statistics distinguish
  the checked linear defect from the correction's own nonlinear remainder.
  Their final audits pass 2,105 and 885 checks; the separate scalar-control
  audit passes 68 hash, projection, shared-base and replay checks.
- Full stellar MPI operation remains outside this validation; its existing
  interpolation restriction is unchanged.

Evidence root:
`/Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/coupled_trial_2026-09-18`.
`tests_provenance.json` records the test executable and log hashes. The
campaign directories contain complete commands, source revisions, immutable
executables and hashes of mesh, inputs and outputs. All three campaign
executables are byte-identical (SHA256 begins `620d625636bc77b1`).

Source checkpoints: implementation `0d87b8b`; initial campaign/checker
`2d0d32b`; checker fix and raw timing repeat `5aa48dc`; scalar-control
generator `0c926f4`. The original checkout and its unrelated quadrature
formatting change are preserved.

From this worktree, reproduce the initial screen with
`experiments/run_plateau_campaign.py`, the committed
`coupled_trial_screen_2026-09-18.json`, the configured Release build,
`sandbox.smesh`, and a new output directory. Use
`coupled_trial_raw_repeat_2026-09-18.json` for the raw repeat. Run
`check_coupled_trial.py CAMPAIGN --output OUTSIDE_CAMPAIGN.json` afterward.
`make_coupled_scalar_controls.py SCREEN FRESH_OUTPUT` generates the two
normalized scalar inputs and `cases.json` for the existing campaign runner.
All runs use one FEM rank; the seed and outer direction inputs are saved
from the preceding investigation and are hashed into the new evidence.