MeanField/experiments/DAMPED_COUPLED_TRIAL_2026-09-18.md

Damped fully coupled trial correction

This investigation branches from 46249d0 onto research/damped-coupled-trial-20260918. It tests two separate mechanisms: bounded damping of one fully coupled correction, and reserving that correction solve for a trial whose merit is reasonably close to its affine prediction. The previous full-step policy remains available and the default solver is unchanged.

The combined opt-in policy gives useful measured progress, but does not yet establish convergence. On the matched six-update n=1.5,J=0 trajectory it reaches F=1.24583e-5 versus 1.49720e-5 for gravity correction alone: 16.79% lower residual for 22.08% more Newton iteration time. After their identical first update, the residual reductions are 22.85% versus 7.29%, giving 2.53 times greater decrease per iteration-second over that bounded interval. Both runs hit the six-update cap. Endpoint enthalpy-based virials remain below 1e-5, while negative density and local EOS defects remain unresolved. The default solver remains unchanged.

Algorithm and architecture

At accepted state x, capture F0=F(x), the actual Newton direction p and J0 p. For outer step alpha and prepared candidate y (raw or gravity-corrected), set L=F0+alpha J0 p and solve J(y)c=L-F(y). The unscaled c must satisfy both the existing independent full-J defect check and the existing norm cap.

The optional search prepares y+beta c with beta=1,1/2,1/4,1/8, always starting from y. One auxiliary solve supplies all four candidates. The first improving candidate passing ordinary outer acceptance ends the search. Otherwise the lowest-merit admissible improvement is retained; an earlier selected candidate is freshly prepared, and complete failure restores y with its current inverse. No accepted-state buffer is written by this transaction.

For the optional attempt selector, D=M(F0)-M(L) must be finite and positive. The solve is eligible when (M(F(y))-M(F0))/D <= k, with default k=1. A deferral spends no auxiliary budget. This is a bounded-work heuristic, not a convergence guarantee. It requires a deterministic metric of the supplied residual with fixed parameters; the default normalized residual metric satisfies this contract.

Algorithms are selected by types: FullCoupledStep/BacktrackedCoupledStep and EveryRejectedCoupledAttempt/NearAcceptanceCoupledAttempt. Runtime iteration counts, reduction factors and tolerances are validated collectively. BacktrackedCoupledCurvatureCorrection composes damping and near-acceptance selection in the existing typed NewtonNumerics recipe. The original CoupledCurvatureCorrection remains the full-step/every-rejected control. Neither changes the physics-facing model specification or central-density policy.

An auxiliary report describes c, not beta c. For a damped candidate the exact remainder identity is

F(y+beta c)-L = (1-beta)(F(y)-L) + beta (J(y)c-(L-F(y))) + E_beta,
E_beta = F(y+beta c)-F(y)-beta J(y)c.

Scaling c therefore does not imply solving the original auxiliary RHS to its original tolerance. The actual freshly prepared residual decides acceptance.

Reproduction and evidence

The versioned damped_coupled_screen_2026-09-18.json compares none/full/damped/near for the late n=1.5,J=0 snapshot on raw and gravity-corrected paths, plus a seed control. No central constraint is imposed. Each sequence captures once, shares one auxiliary-solve budget, tests every halving from alpha=1 through 1/4096 and stops at its first accepted candidate. The prior independent-alpha screen did not represent this shared budget and omitted some intermediate step lengths.

The campaign runner builds a clean committed revision, copies an immutable executable and hashes mesh, inputs and outputs. The experiment calls the same production transaction and production gravity correction as Newton. It records every candidate evaluation, fresh selected-state replay and unchanged accepted buffers/normalization. Only the accepted trial is replayed for quadrature-14 and quadrature-18 physical observations, outside the timed selection phase.

Operation timings include raw preparation, gravity correction and the coupled transaction. They exclude setup, ordinary initial/final metric calls, final restoration, accepted commit refresh, artifact output and physical replay; search wall time also includes artifact output. Callback vector copies remain in the measured transaction. The no-correction control captures the same tangent for a comparable screen, so capture cost is reported separately and is not ordinary Newton work for that control. Builds, timed numerical cases and artifact audits do not overlap; numerical cases run one at a time.

Frozen sequential trial results

The independent artifact audit passed all nine cases in screen_v1 (see screen_v1_audit.json in the evidence directory below). This audits the recorded transaction algebra and true-defect certificates from saved vectors, provenance, and recorded restoration evidence; it does not certify a converged or physically admissible stellar model.

What improved

The late-state screen uses the same saved state and Newton direction, a halving line search beginning at alpha=1, one shared correction-solve budget, and the first accepted trial. The initial full residual norm is 1.3966991044e-5.

Path	Policy	Accepted alpha	Residual decrease	Search operation time
Raw	No coupled correction	1/512	0.116541%	49.04 s
Raw	Full correction, every rejection eligible	1/512	0.116541%	69.00 s
Raw	Damped correction, every rejection eligible	1/512	0.116541%	95.33 s
Raw	Damped correction, near-acceptance eligibility	1/256	0.253536%	80.11 s
Gravity-prepared	No coupled correction	1/512	0.047127%	116.06 s
Gravity-prepared	Full correction, every rejection eligible	1/512	0.047127%	Excluded: clock anomaly
Gravity-prepared	Damped correction, every rejection eligible	1/512	0.047127%	168.80 s
Gravity-prepared	Damped correction, near-acceptance eligibility	1/256	0.189199%	128.75 s

These are original-screen operation timings, excluding artifact writes, initialization, physical replay, and the separately measured base capture. The three timing repeats below use the identical diagnostic binary. The excluded original gravity/full timing is retained as evidence, without rescaling.

The demonstrated accepted-step gain comes from eligibility timing. The near gate defers the prepared rejections at 1/32, 1/64, and 1/128, preserving the sole solve for 1/256. Both accepted corrections use beta=1 and stop after one candidate. Damping is therefore not responsible for their accepted improvement. Without this gate, the correction budget is spent at 1/32, and the eventual accepted state equals the corresponding baseline. Gravity damping does improve the rejected 1/32 candidate at beta=1/2, but it remains unacceptable.

The nominal accepted alpha improves twofold, not fourfold. The earlier independent-alpha screen omitted 1/512 and did not reproduce a shared-budget search. The present residual decreases and costs describe one frozen search; they establish neither sustained convergence nor a nonlinear speedup. They also do not establish that the chosen eligibility threshold is effective at other states, polytropic indices, or rotation rates.

Seed control

The seed's independently verified outer defect is 0.0281993. Its first trial, alpha=1, already passes after existing gravity preparation: the residual falls from 2.736592176e-4 to 1.614841549e-5, a 94.0991% decrease. The new policy performs zero coupled solves, beta evaluations, or deferrals. This confirms nonintervention at this sampled seed; it is not a complete trajectory or performance control.

Physical interpretation

All selected snapshots replay exactly, with accepted buffers, direction, normalization factors, residual, and Jacobian action restored. The q18 enthalpy-pressure energy virial errors are 4.059–4.068e-6 for the late candidates and 4.3646e-6 for the seed candidate. Force virial errors are 2.565–2.572e-6 and 2.8455e-6, respectively. Raising quadrature from q14 to q18 changes either enthalpy-based virial diagnostic by only 1.828–1.834e-9. Mass errors are 3.13–3.71e-6, Jz is zero, and sampled mapping determinants remain above 0.9858.

However, every selected snapshot retains negative density: q18 minima range from -8.25e-4 to -8.64e-4. Density-derived pressure is not defined everywhere, and its reported virial diagnostics are NaN. The q18 negative-density mass fraction is about 2.90e-5, 29–30% larger than its q14 estimate. The sampled density-minimum magnitude increases 15–17%, and the maximum scaled EOS mismatch increases about 17%; the RMS scaled EOS mismatch changes about 0.21%.

Thus the enthalpy-based global virial diagnostic meets 1e-5 and is stable under this quadrature comparison, while local density/EOS defects remain unresolved. This comparison is neither a mesh-convergence study nor a pointwise positivity certificate. Frozen Armijo acceptance must not be presented as model convergence or complete physical validation.

Timing repeats

The first gravity/full case had inconsistent elapsed clocks: 531.747 s UTC versus 236.576 s Python monotonic, with a 325.024 s C++ auxiliary solve. It is excluded from performance comparisons. The runner now records child user/system CPU time, UTC elapsed time and monotonic elapsed time independently. A temporary idle-sleep guard covers each subsequent campaign; no phase time is rescaled.

Repeat	Search operation time	Total elapsed / child CPU	Result
Gravity/full	149.17 s	227.30 / 226.58 s	Same baseline endpoint; 53 auxiliary iterations
Raw/none	45.76 s	123.67 / 123.57 s	Same alpha=1/512 endpoint
Raw/near	74.90 s	153.11 / 153.05 s	Same alpha=1/256, beta=1 endpoint

UTC and monotonic times differ by at most 0.0053 s in these repeats. Every saved vector file has the same hash as its original run (68, 55 and 54 files, respectively), and the independent vector audit passes all three. The raw progress per operation-second ratio is 1.332 initially and 1.329 in the paired repeat. These support about 33% greater fractional residual decrease per measured operation-second at this frozen state, despite an approximately 7% timing shift between campaigns. The gravity near/none ratio is 3.619 in the one original pair; it has not received an additional paired timing repeat.

These ratios use fractional residual decrease, not merit decrease. They compare one fixed state/direction and exclude ordinary outer linear solve, commit refresh, initialization and physical replay. They are not Newton speedup factors. The correction also changes the actual displacement, so a factor-two nominal alpha is not a factor-two displacement claim.

Matched live Newton follow-up

The paired six-update runs use the same seed, mesh, n=1.5,J=0 specification, no central constraint, gravity-Schur preconditioner, restart 160 and outer true defect tolerance 0.03. Only the trial policy changes: curvature versus curvature-backtracked-coupled. Initial physical observations and layout hashes match exactly, as do the first accepted residual, alpha, linear diagnostics and block residuals. First-step state vectors were not saved, so these are recorded diagnostic matches rather than a separate bitwise state comparison.

Accepted update	Control alpha	Control F	New alpha	New F	Cumulative Newton seconds, control / new
1	1	1.614842e-5	1	1.614842e-5	39.25 / 39.11
2	1/16	1.579569e-5	1/8	1.614552e-5	173.68 / 194.99
3	1/16	1.553144e-5	1/4	1.595479e-5	248.91 / 249.76
4	1/16	1.514564e-5	1/2	1.574591e-5	324.13 / 305.06
5	1/32	1.499075e-5	1/2	1.273240e-5	419.19 / 431.45
6	1/32	1.497197e-5	1/8	1.245830e-5	502.21 / 613.09

Both runs complete six accepted updates and report nonconvergence at the iteration limit (exit 3). The original strict residual tolerance is unchanged. The combined policy produces 16.79% lower final residual, at 22.08% more iteration time. Excluding the identical first update, residual decreases from that common value are 7.285% in 462.95 s (control) and 22.851% in 573.98 s (new), a 2.530 ratio of residual decrease per second over this fixed interval. This is a bounded progress-rate comparison, not a speedup to a converged model.

The new method initially lags. First recorded crossings of fixed residual thresholds illustrate the tradeoff without interpolating between states:

F threshold	Control Newton seconds	New Newton seconds
1.60e-5	173.68	249.76
1.55e-5	324.13	431.45
1.50e-5	419.19	431.45
1.40e-5	Not reached within six updates	431.45

Total process elapsed/child CPU times are 561.92/561.22 s (control) and 671.98/671.86 s (new). UTC and monotonic elapsed times differ by 0.0014 s and -0.0318 s, respectively. The total process includes initialization, endpoint physical diagnostics and artifact output; the iteration totals exclude these. There is one live timing pair, not a statistical performance distribution.

What the live corrections actually did

The control uses 354 outer Krylov iterations; the new run uses 341 outer plus 163 auxiliary iterations across three correction solves. The new gate defers five trials, and auxiliary preparation adds 11 preparations, including one best-candidate replay. Auxiliary solve/preparation/verification/refresh counters are 84.35/64.03/0.67/4.77 s; these are nested work already included in iteration time and must not be added to it. Ten beta candidates are evaluated.

At update 2 (zero-based iteration 1), alpha=1/8 has an eligible excess ratio 0.35155. The unscaled solve passes its true defect gate (relative 0.02570) and cap (correction/cap=0.63369). Beta=1 worsens the merit; beta=1/2 passes ordinary acceptance. It reduces residual by only 0.01795%, versus 2.18426% from the control's next ordinary backtrack at alpha=1/16. Thus an accepted rescue need not have the best immediate progress. It realizes only 0.1533% of the nominal affine predicted merit decrease, despite satisfying Armijo by a factor 28.7.

Unlike the successful frozen rescues, this accepted live correction genuinely uses damping. It changes the following trajectory: updates 3 and 4 accept ordinary alpha=1/4 and 1/2 trials, and update 5 achieves a large residual drop. At updates 5 and 6 the auxiliary searches keep their best improvements at beta=1/8 and 1/4, respectively, but both still fail outer acceptance. The next ordinary trial is accepted instead. Therefore three kept diagnostic events mean only one committed coupled rescue. No later rejected correction is carried into the accepted state.

A minimum realized-versus-predicted decrease guard is a plausible separate investigation, not an established improvement: it would reject the initially weak rescue that precedes this better final trajectory. A beta-aware prediction F(y)+beta J(y)c would distinguish a damped candidate from the full target L without another FEM action. Any guard must control final candidate admission and fallback, not just beta-loop stopping. Comparing with one ordinary further backtrack is another useful control, at one extra preparation cost. Neither change is implemented or claimed beneficial here.

Live endpoint physics

q18 observation	Control	New
Enthalpy-pressure energy virial error	4.22590e-6	3.78554e-6
Enthalpy-pressure force virial error	2.71367e-6	2.31894e-6
Absolute mass error	3.41685e-6	2.53899e-6
Absolute J error	0	0
Minimum sampled mapping determinant	0.989275	0.989068
Minimum density	-7.73185e-4	-7.85522e-4
Negative-density mass fraction	2.89683e-5	2.88557e-5
Scaled RMS EOS density mismatch	1.27948e-4	1.27721e-4
Scaled maximum EOS density mismatch	9.63952e-4	9.73858e-4

Both endpoints meet the primary enthalpy-based virial milestone at q14 and q18; the quadrature change is about 1.83–1.84e-9. Density-derived pressure and its virials remain undefined because of negative samples. The new endpoint improves the global measures while its sampled density minimum and maximum EOS defect are slightly worse. No positivity, mesh convergence, or complete local-physics claim follows from the virial result. The separate conservative solver/defined- virial checker returns its expected nonpassing result for both endpoints.

Implication and next experiment

Keep the new policy opt-in. The implementation and its cost bounds are tested; the combined policy now has evidence of beneficial nonlinear trajectory changes as well as frozen-step improvement. It has not yet demonstrated sustained convergence or any higher-n/rotating result. The next useful test is a longer matched n=1.5 trajectory from the same seed, preserving the physical diagnostics and work accounting, before expanding to higher n and J. Do not promote the policy or tune a new quality threshold solely from the early marginal rescue.

Validation

Production and experiment targets build successfully with the configured release Clang/C++23 module build. Existing duplicate-library linker warnings remain.

• Core correction tests: 23 serial cases / 1,129 assertions; the same 23 cases on two MPI ranks pass 1,134 assertions per rank.
• Broader Newton, numerical-strategy, gravity and FGMRES regression tests: 35 cases / 357,016 assertions. Together the serial groups cover 58 cases.
• Python artifact-checker tests: 25 frozen-search cases, 27 prior cases and 4 live-checker regressions, all 56 passing.
• Actual frozen campaigns: nine original cases and three repeats, all passing the independent recorded-vector/provenance audit.
• Matched live pair: 1,083 scalar-decision/provenance checks pass. These check recorded true-defect observations rather than independently reapplying FEM.

Coverage includes smaller-beta acceptance after full-step rejection, preparation failure followed by success, first-accepted stopping, best-candidate replay, complete rejection, later-candidate/replay exceptions, restoration, near-gate budget preservation, zero predicted decrease, already accepted originals, invalid and rank-mismatched options, rank-local rejection, and typed recipe composition. Real-FEM production tests cover the original and new policies on raw and gravity paths, preservation of the outer linear report, and checkpoint restoration.

The checker reconstructs the affine prediction, auxiliary RHS, unscaled full-J defect and cap, beta state updates and remainder identity from saved vectors. It verifies shared budget use and first-accept stopping, checks recorded fresh selected-state replay and restoration evidence, and checks accepted-state and normalization invariants and input/output hashes. It does not independently rerun FEM or physical quadrature.

Evidence and reproduction

Evidence root (outside the source tree):

/Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/damped_coupled_2026-09-18

The original frozen screen binds to source 25816d4; the repeats and matched Newton cases bind to 1876229. Their diagnostic binary hashes are recorded in each campaign's provenance.json. The repeats use the same binary hash as the screen. Test provenance and logs are in the evidence root. Tests were compiled at 422c46b; the subsequent source change 25816d4 only hardened the frozen experiment's finite-metric guards.

Versioned cases:

• damped_coupled_screen_2026-09-18.json
• damped_coupled_repeat_2026-09-18.json
• damped_coupled_newton_2026-09-18.json

Use run_plateau_campaign.py with a clean committed source, its configured build, --mesh, --cases, and a fresh --output. The build copies an immutable executable before any case runs. Case manifests currently name the retained absolute snapshot paths; relocate those explicitly when reproducing elsewhere.

After timed cases finish, run:

python3 experiments/check_damped_coupled_trial.py EVIDENCE/screen_v1 --output EVIDENCE/screen_v1_audit.json
python3 experiments/check_damped_coupled_trial.py EVIDENCE/repeat_v1 --output EVIDENCE/repeat_v1_audit.json
python3 experiments/check_damped_coupled_newton.py EVIDENCE/newton_v1 --output EVIDENCE/newton_v1_review.json

TIMING_NOTE.md, repeat_comparison.json, the per-case run records and all raw CSVs/vectors preserve the timing exclusion and reproducibility evidence.