MeanField/experiments/DAMPED_COUPLED_TRIAL_2026-09-18.md

# Damped fully coupled trial correction

This investigation branches from `46249d0` onto
`research/damped-coupled-trial-20260918`. It tests two separate mechanisms:
bounded damping of one fully coupled correction, and reserving that correction
solve for a trial whose merit is reasonably close to its affine prediction.
The previous full-step policy remains available and the default solver is unchanged.


The combined opt-in policy gives useful measured progress, but does not yet
establish convergence. On the matched six-update n=1.5,J=0 trajectory it reaches
F=1.24583e-5 versus 1.49720e-5 for gravity correction alone: 16.79% lower residual
for 22.08% more Newton iteration time. After their identical first update, the
residual reductions are 22.85% versus 7.29%, giving 2.53 times greater decrease
per iteration-second over that bounded interval. Both runs hit the six-update
cap. Endpoint enthalpy-based virials remain below 1e-5, while negative density
and local EOS defects remain unresolved. The default solver remains unchanged.

## Algorithm and architecture

At accepted state x, capture F0=F(x), the actual Newton direction p and J0 p.
For outer step alpha and prepared candidate y (raw or gravity-corrected), set
L=F0+alpha J0 p and solve J(y)c=L-F(y). The unscaled c must satisfy both the
existing independent full-J defect check and the existing norm cap.

The optional search prepares y+beta c with beta=1,1/2,1/4,1/8, always starting
from y. One auxiliary solve supplies all four candidates. The first improving
candidate passing ordinary outer acceptance ends the search. Otherwise the
lowest-merit admissible improvement is retained; an earlier selected candidate
is freshly prepared, and complete failure restores y with its current inverse.
No accepted-state buffer is written by this transaction.

For the optional attempt selector, D=M(F0)-M(L) must be finite and positive.
The solve is eligible when (M(F(y))-M(F0))/D <= k, with default k=1. A deferral
spends no auxiliary budget. This is a bounded-work heuristic, not a convergence
guarantee. It requires a deterministic metric of the supplied residual with
fixed parameters; the default normalized residual metric satisfies this contract.

Algorithms are selected by types: `FullCoupledStep`/`BacktrackedCoupledStep`
and `EveryRejectedCoupledAttempt`/`NearAcceptanceCoupledAttempt`. Runtime
iteration counts, reduction factors and tolerances are validated collectively.
`BacktrackedCoupledCurvatureCorrection` composes damping and near-acceptance
selection in the existing typed `NewtonNumerics` recipe. The original
`CoupledCurvatureCorrection` remains the full-step/every-rejected control.
Neither changes the physics-facing model specification or central-density policy.

An auxiliary report describes c, not beta c. For a damped candidate the exact
remainder identity is

    F(y+beta c)-L = (1-beta)(F(y)-L) + beta (J(y)c-(L-F(y))) + E_beta,
    E_beta = F(y+beta c)-F(y)-beta J(y)c.

Scaling c therefore does not imply solving the original auxiliary RHS to its
original tolerance. The actual freshly prepared residual decides acceptance.

## Reproduction and evidence

The versioned `damped_coupled_screen_2026-09-18.json` compares none/full/damped/near
for the late n=1.5,J=0 snapshot on raw and gravity-corrected paths, plus a seed
control. No central constraint is imposed. Each sequence captures once, shares
one auxiliary-solve budget, tests every halving from alpha=1 through 1/4096 and
stops at its first accepted candidate. The prior independent-alpha screen did
not represent this shared budget and omitted some intermediate step lengths.

The campaign runner builds a clean committed revision, copies an immutable
executable and hashes mesh, inputs and outputs. The experiment calls the same
production transaction and production gravity correction as Newton. It records
every candidate evaluation, fresh selected-state replay and unchanged accepted
buffers/normalization. Only the accepted trial is replayed for quadrature-14 and
quadrature-18 physical observations, outside the timed selection phase.

Operation timings include raw preparation, gravity correction and the coupled transaction.
They exclude setup, ordinary initial/final metric calls, final restoration, accepted
commit refresh, artifact output and physical replay; search wall time
also includes artifact output. Callback vector copies remain in the measured
transaction. The no-correction control captures the same tangent for a comparable
screen, so capture cost is reported separately and is not ordinary Newton work
for that control. Builds, timed numerical cases and artifact audits do not overlap; numerical
cases run one at a time.

## Frozen sequential trial results

The independent artifact audit passed all nine cases in `screen_v1` (see
`screen_v1_audit.json` in the evidence directory below). This audits the recorded transaction algebra and true-defect certificates
from saved vectors, provenance, and recorded restoration evidence; it does not
certify a converged or physically admissible stellar model.

### What improved

The late-state screen uses the same saved state and Newton direction, a halving
line search beginning at alpha=1, one shared correction-solve budget, and the
first accepted trial. The initial full residual norm is 1.3966991044e-5.

| Path | Policy | Accepted alpha | Residual decrease | Search operation time |
| --- | --- | ---: | ---: | ---: |
| Raw | No coupled correction | 1/512 | 0.116541% | 49.04 s |
| Raw | Full correction, every rejection eligible | 1/512 | 0.116541% | 69.00 s |
| Raw | Damped correction, every rejection eligible | 1/512 | 0.116541% | 95.33 s |
| Raw | Damped correction, near-acceptance eligibility | 1/256 | 0.253536% | 80.11 s |
| Gravity-prepared | No coupled correction | 1/512 | 0.047127% | 116.06 s |
| Gravity-prepared | Full correction, every rejection eligible | 1/512 | 0.047127% | Excluded: clock anomaly |
| Gravity-prepared | Damped correction, every rejection eligible | 1/512 | 0.047127% | 168.80 s |
| Gravity-prepared | Damped correction, near-acceptance eligibility | 1/256 | 0.189199% | 128.75 s |

These are original-screen operation timings, excluding artifact writes,
initialization, physical replay, and the separately measured base capture.
The three timing repeats below use the identical diagnostic binary. The excluded
original gravity/full timing is retained as evidence, without rescaling.

The demonstrated accepted-step gain comes from **eligibility timing**. The near
gate defers the prepared rejections at 1/32, 1/64, and 1/128, preserving the sole
solve for 1/256. Both accepted corrections use beta=1 and stop after one
candidate. Damping is therefore not responsible for their accepted improvement.
Without this gate, the correction budget is spent at 1/32, and the eventual
accepted state equals the corresponding baseline. Gravity damping does improve
the rejected 1/32 candidate at beta=1/2, but it remains unacceptable.

The nominal accepted alpha improves **twofold, not fourfold**. The earlier
independent-alpha screen omitted 1/512 and did not reproduce a shared-budget
search. The present residual decreases and costs describe one frozen search;
they establish neither sustained convergence nor a nonlinear speedup. They also
do not establish that the chosen eligibility threshold is effective at other
states, polytropic indices, or rotation rates.

### Seed control

The seed's independently verified outer defect is 0.0281993. Its first trial,
alpha=1, already passes after existing gravity preparation: the residual falls
from 2.736592176e-4 to 1.614841549e-5, a 94.0991% decrease. The new policy performs
zero coupled solves, beta evaluations, or deferrals. This confirms nonintervention
at this sampled seed; it is not a complete trajectory or performance control.

### Physical interpretation

All selected snapshots replay exactly, with accepted buffers, direction,
normalization factors, residual, and Jacobian action restored. The q18
enthalpy-pressure energy virial errors are 4.059–4.068e-6 for the late candidates
and 4.3646e-6 for the seed candidate. Force virial errors are 2.565–2.572e-6 and
2.8455e-6, respectively. Raising quadrature from q14 to q18 changes either
enthalpy-based virial diagnostic by only 1.828–1.834e-9. Mass errors are
3.13–3.71e-6, Jz is zero, and sampled mapping determinants remain above 0.9858.

However, every selected snapshot retains negative density: q18 minima range
from -8.25e-4 to -8.64e-4. Density-derived pressure is not defined everywhere,
and its reported virial diagnostics are NaN. The q18 negative-density mass
fraction is about 2.90e-5, **29–30% larger** than its q14 estimate. The sampled
density-minimum magnitude increases 15–17%, and the maximum scaled EOS mismatch
increases about 17%; the RMS scaled EOS mismatch changes about 0.21%.

Thus the enthalpy-based global virial diagnostic meets 1e-5 and is stable under
this quadrature comparison, while local density/EOS defects remain unresolved.
This comparison is neither a mesh-convergence study nor a pointwise positivity
certificate. Frozen Armijo acceptance must not be presented as model convergence
or complete physical validation.

## Timing repeats

The first gravity/full case had inconsistent elapsed clocks: 531.747 s UTC
versus 236.576 s Python monotonic, with a 325.024 s C++ auxiliary solve. It is
excluded from performance comparisons. The runner now records child user/system
CPU time, UTC elapsed time and monotonic elapsed time independently. A temporary
idle-sleep guard covers each subsequent campaign; no phase time is rescaled.

| Repeat | Search operation time | Total elapsed / child CPU | Result |
| --- | ---: | ---: | --- |
| Gravity/full | 149.17 s | 227.30 / 226.58 s | Same baseline endpoint; 53 auxiliary iterations |
| Raw/none | 45.76 s | 123.67 / 123.57 s | Same alpha=1/512 endpoint |
| Raw/near | 74.90 s | 153.11 / 153.05 s | Same alpha=1/256, beta=1 endpoint |

UTC and monotonic times differ by at most 0.0053 s in these repeats. Every saved
vector file has the same hash as its original run (68, 55 and 54 files,
respectively), and the independent vector audit passes all three. The raw
progress per operation-second ratio is 1.332 initially and 1.329 in the paired
repeat. These support about 33% greater fractional residual decrease per measured
operation-second at this frozen state, despite an approximately 7% timing shift
between campaigns. The gravity near/none ratio is 3.619 in the
one original pair; it has not received an additional paired timing repeat.

These ratios use fractional residual decrease, not merit decrease. They compare
one fixed state/direction and exclude ordinary outer linear solve, commit
refresh, initialization and physical replay. They are not Newton speedup factors.
The correction also changes the actual displacement, so a factor-two nominal
alpha is not a factor-two displacement claim.

## Matched live Newton follow-up

The paired six-update runs use the same seed, mesh, n=1.5,J=0 specification,
no central constraint, gravity-Schur preconditioner, restart 160 and outer true
defect tolerance 0.03. Only the trial policy changes: `curvature` versus
`curvature-backtracked-coupled`. Initial physical observations and layout hashes
match exactly, as do the first accepted residual, alpha, linear diagnostics and
block residuals. First-step state vectors were not saved, so these are recorded
diagnostic matches rather than a separate bitwise state comparison.

| Accepted update | Control alpha | Control F | New alpha | New F | Cumulative Newton seconds, control / new |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 1 | 1.614842e-5 | 1 | 1.614842e-5 | 39.25 / 39.11 |
| 2 | 1/16 | 1.579569e-5 | 1/8 | 1.614552e-5 | 173.68 / 194.99 |
| 3 | 1/16 | 1.553144e-5 | 1/4 | 1.595479e-5 | 248.91 / 249.76 |
| 4 | 1/16 | 1.514564e-5 | 1/2 | 1.574591e-5 | 324.13 / 305.06 |
| 5 | 1/32 | 1.499075e-5 | 1/2 | 1.273240e-5 | 419.19 / 431.45 |
| 6 | 1/32 | 1.497197e-5 | 1/8 | 1.245830e-5 | 502.21 / 613.09 |

Both runs complete six accepted updates and report nonconvergence at the
iteration limit (exit 3). The original strict residual tolerance is unchanged.
The combined policy produces 16.79% lower final residual, at 22.08% more
iteration time. Excluding the identical first update, residual decreases from
that common value are 7.285% in 462.95 s (control) and 22.851% in 573.98 s (new),
a 2.530 ratio of residual decrease per second over this fixed interval. This is
a bounded progress-rate comparison, not a speedup to a converged model.

The new method initially lags. First recorded crossings of fixed residual
thresholds illustrate the tradeoff without interpolating between states:

| F threshold | Control Newton seconds | New Newton seconds |
| ---: | ---: | ---: |
| 1.60e-5 | 173.68 | 249.76 |
| 1.55e-5 | 324.13 | 431.45 |
| 1.50e-5 | 419.19 | 431.45 |
| 1.40e-5 | Not reached within six updates | 431.45 |

Total process elapsed/child CPU times are 561.92/561.22 s (control) and
671.98/671.86 s (new). UTC and monotonic elapsed times differ by 0.0014 s and
-0.0318 s, respectively. The total process includes initialization, endpoint
physical diagnostics and artifact output; the iteration totals exclude these.
There is one live timing pair, not a statistical performance distribution.

### What the live corrections actually did

The control uses 354 outer Krylov iterations; the new run uses 341 outer plus
163 auxiliary iterations across three correction solves. The new gate defers
five trials, and auxiliary preparation adds 11 preparations, including one
best-candidate replay. Auxiliary solve/preparation/verification/refresh counters
are 84.35/64.03/0.67/4.77 s; these are nested work already included in iteration
time and must not be added to it. Ten beta candidates are evaluated.

At update 2 (zero-based iteration 1), alpha=1/8 has an eligible excess ratio
0.35155. The unscaled solve passes its true defect gate (relative 0.02570) and
cap (correction/cap=0.63369). Beta=1 worsens the merit; beta=1/2 passes ordinary
acceptance. It reduces residual by only 0.01795%, versus 2.18426% from the
control's next ordinary backtrack at alpha=1/16. Thus an accepted rescue need
not have the best immediate progress. It realizes only 0.1533% of the nominal
affine predicted merit decrease, despite satisfying Armijo by a factor 28.7.

Unlike the successful frozen rescues, this accepted live correction genuinely
uses damping. It changes the following trajectory: updates 3 and 4 accept
ordinary alpha=1/4 and 1/2 trials, and update 5 achieves a large residual drop.
At updates 5 and 6 the auxiliary searches keep their best improvements at
beta=1/8 and 1/4, respectively, but both still fail outer acceptance. The next
ordinary trial is accepted instead. Therefore three `kept` diagnostic events
mean only **one committed coupled rescue**. No later rejected correction is
carried into the accepted state.

A minimum realized-versus-predicted decrease guard is a plausible separate
investigation, not an established improvement: it would reject the initially
weak rescue that precedes this better final trajectory. A beta-aware prediction
F(y)+beta J(y)c would distinguish a damped candidate from the full target L
without another FEM action. Any guard must control final candidate admission
and fallback, not just beta-loop stopping. Comparing with one ordinary further
backtrack is another useful control, at one extra preparation cost. Neither
change is implemented or claimed beneficial here.

### Live endpoint physics

| q18 observation | Control | New |
| --- | ---: | ---: |
| Enthalpy-pressure energy virial error | 4.22590e-6 | 3.78554e-6 |
| Enthalpy-pressure force virial error | 2.71367e-6 | 2.31894e-6 |
| Absolute mass error | 3.41685e-6 | 2.53899e-6 |
| Absolute J error | 0 | 0 |
| Minimum sampled mapping determinant | 0.989275 | 0.989068 |
| Minimum density | -7.73185e-4 | -7.85522e-4 |
| Negative-density mass fraction | 2.89683e-5 | 2.88557e-5 |
| Scaled RMS EOS density mismatch | 1.27948e-4 | 1.27721e-4 |
| Scaled maximum EOS density mismatch | 9.63952e-4 | 9.73858e-4 |

Both endpoints meet the primary enthalpy-based virial milestone at q14 and q18;
the quadrature change is about 1.83–1.84e-9. Density-derived pressure and its
virials remain undefined because of negative samples. The new endpoint improves
the global measures while its sampled density minimum and maximum EOS defect
are slightly worse. No positivity, mesh convergence, or complete local-physics
claim follows from the virial result. The separate conservative solver/defined-
virial checker returns its expected nonpassing result for both endpoints.

### Implication and next experiment

Keep the new policy opt-in. The implementation and its cost bounds are tested;
the combined policy now has evidence of beneficial nonlinear trajectory changes
as well as frozen-step improvement. It has not yet demonstrated sustained
convergence or any higher-n/rotating result. The next useful test is a longer
matched n=1.5 trajectory from the same seed, preserving the physical diagnostics
and work accounting, before expanding to higher n and J. Do not promote the
policy or tune a new quality threshold solely from the early marginal rescue.

## Validation

Production and experiment targets build successfully with the configured release
Clang/C++23 module build. Existing duplicate-library linker warnings remain.

- Core correction tests: 23 serial cases / 1,129 assertions; the same 23 cases
  on two MPI ranks pass 1,134 assertions per rank.
- Broader Newton, numerical-strategy, gravity and FGMRES regression tests:
  35 cases / 357,016 assertions. Together the serial groups cover 58 cases.
- Python artifact-checker tests: 25 frozen-search cases, 27 prior cases and
  4 live-checker regressions, all 56 passing.
- Actual frozen campaigns: nine original cases and three repeats, all passing
  the independent recorded-vector/provenance audit.
- Matched live pair: 1,083 scalar-decision/provenance checks pass. These check
  recorded true-defect observations rather than independently reapplying FEM.

Coverage includes smaller-beta acceptance after full-step rejection, preparation
failure followed by success, first-accepted stopping, best-candidate replay,
complete rejection, later-candidate/replay exceptions, restoration, near-gate
budget preservation, zero predicted decrease, already accepted originals, invalid
and rank-mismatched options, rank-local rejection, and typed recipe composition.
Real-FEM production tests cover the original and new policies on raw and gravity
paths, preservation of the outer linear report, and checkpoint restoration.

The checker reconstructs the affine prediction, auxiliary RHS, unscaled full-J
defect and cap, beta state updates and remainder identity from saved vectors.
It verifies shared budget use and first-accept stopping, checks recorded fresh
selected-state replay and restoration evidence, and checks accepted-state and
normalization invariants and input/output hashes. It does
not independently rerun FEM or physical quadrature.

## Evidence and reproduction

Evidence root (outside the source tree):

`/Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/damped_coupled_2026-09-18`

The original frozen screen binds to source `25816d4`; the repeats and matched
Newton cases bind to `1876229`. Their diagnostic binary hashes are recorded in
each campaign's `provenance.json`. The repeats use the same binary hash as the
screen. Test provenance and logs are in the evidence root. Tests were compiled
at `422c46b`; the subsequent source change `25816d4` only hardened the frozen
experiment's finite-metric guards.

Versioned cases:

- `damped_coupled_screen_2026-09-18.json`
- `damped_coupled_repeat_2026-09-18.json`
- `damped_coupled_newton_2026-09-18.json`

Use `run_plateau_campaign.py` with a clean committed source, its configured build,
`--mesh`, `--cases`, and a fresh `--output`. The build copies an immutable
executable before any case runs. Case manifests currently name the retained
absolute snapshot paths; relocate those explicitly when reproducing elsewhere.

After timed cases finish, run:

```sh
python3 experiments/check_damped_coupled_trial.py EVIDENCE/screen_v1 --output EVIDENCE/screen_v1_audit.json
python3 experiments/check_damped_coupled_trial.py EVIDENCE/repeat_v1 --output EVIDENCE/repeat_v1_audit.json
python3 experiments/check_damped_coupled_newton.py EVIDENCE/newton_v1 --output EVIDENCE/newton_v1_review.json
```

`TIMING_NOTE.md`, `repeat_comparison.json`, the per-case run records and all raw
CSVs/vectors preserve the timing exclusion and reproducibility evidence.