Files
MeanField/experiments/NUMERICAL_CAMPAIGN_2026-09-16.md

334 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Targeted numerical campaign — September 16, 2026
This campaign tests separate explanations for the Newton plateau while keeping
the public model API unchanged. The original optimized source is preserved in
`research/baseline-20260916` at `80cc3d8`; `main` is unchanged. Hypothesis branches
and complete numerical artifacts are retained, including unsuccessful changes.
## Decisions
| Hypothesis | Current decision | Evidence |
| --- | --- | --- |
| The extra central constraint distorts n=1's local equilibrium | Retain the example correction | Removing it reduces peak EOS error about 22-fold while preserving virial balance |
| Restart 40 loses useful Krylov information | Retain restart 160 in the example | The matched n=1 rotating model reaches strict convergence; restart 40 fails its late linear gate |
| One-way material-to-gravity completion is enough | Do not promote | Both windows still fail at n=3; correction grows and admissibility worsens |
| Rejecting a harmful reused initial guess helps | Retain as limited linear-solver robustness | All eight n=1.5 linear gates pass; the nonlinear plateau remains |
Normalization, the nonlinear stopping tolerances, and the default production
preconditioner have not been changed. The general FGMRES default remains intact;
the example selects its existing restart option. No new physics-facing options
or automatic n-dependent central constraint are introduced.
## Physical model and comparable controls
All runs use the same 1216-element `sandbox.smesh` and the release build's
unchanged FEM spaces. Mesh SHA-256 is
`ce11ae99e21e6c3bbfbee402acd2e191c1da0d8261d2227b4203f73f2f337a74`.
The diagnostic chooses a Lane–Emden reference with M=R=G=1 and derives K and
the seed density for each index. Its n=1 model has K=2/pi and seed density pi/4.
At n=1 and n=1.5 the new controls omit the central constraint; n=3 retains it.
That is an experiment choice, not an automatically imposed library policy.
Controls use nonlinear relative tolerance 1e-8, zero absolute tolerance,
linear relative tolerance 0.03, and a 200-iteration linear cap. A small virial
error and successful algebraic stopping are recorded separately. J=0.05 or
the resulting angular velocity is not a measured fraction of critical rotation.
The common diagnostic revision is `9bd4af6`. The later shared-tools revision
`0470e3b` adds a result checker, source-relative input hashing, and hashes for
saved output states/metadata. It does not alter the stellar solve.
## Removing the n=1 central constraint
The matched J=0.05, restart-40 controls isolate this model change. The initial
physical measurements agree exactly, and their first five accepted step
lengths and linear iteration counts are identical. Both then fail the sixth
linear gate at essentially the same true relative defect, about 0.505.
| Final measurement, quadrature 18 | Central constraint | No central constraint |
| --- | ---: | ---: |
| Peak absolute EOS density mismatch | 7.30451e-3 | 3.32568e-4 |
| Peak EOS mismatch / sampled density scale | 9.33951e-3 | 4.30594e-4 |
| Bernoulli sampled range | 1.45751e-2 | 1.30398e-3 |
| Residual with artificial border source removed | 4.54227e-6 | 6.03609e-9 |
| P(h) energy virial error | 3.01617e-7 | 3.06913e-7 |
Mass, J, and angular velocity are essentially unchanged. The formerly fixed
central enthalpy coefficient falls from 1 to 0.983383508. The peak and range
improvements are localized: volume RMS errors barely change. Quadrature 14
independently shows the same trend. These sampled extrema are not certified
pointwise bounds.
The example now supplies density through the existing Lane–Emden seed policy
instead of constraining the final n=1 model. The added context regression
checks that explicit seeding works without a central constraint. For n=3 the
constraint remains optional; omission is not promised to overcome homology.
## Larger Krylov window: live n=1 success
At J=0.05, changing only restart 40 to 160 gives:
| Quantity | Restart 40 | Restart 160 |
| --- | ---: | ---: |
| Accepted Newton steps | 5, all full | 7, all full |
| Final normalized residual | 6.03609e-9 | 3.38036e-11 |
| Strict configured convergence | No | Yes |
| Total linear iterations, including failed attempts | 370 | 292 |
| End-to-end seconds, including diagnostics | 290.80 | 271.95 |
The first four accepted steps coincide exactly. The fifth solve first crosses
the smaller restart boundary. The final strict threshold is about 2.15e-10.
These are single timing observations, not a general speedup benchmark. A larger
Krylov window also requires more storage.
All four final virial definitions, at both quadrature orders, are below
3.735e-7. The P(h) energy result at q18 is 3.07200e-7. Mass and J errors are
1.75e-13 and 1.54e-14. No sampled density or enthalpy is negative. Local EOS and
Bernoulli errors remain essentially those of the improved unphased model.
The center-of-mass offset falls from 7.66e-6 to 2.69e-8 during the final
corrections, after virial and local EOS metrics have mostly stabilized. This
motivates examining weak position-related directions; it does not identify a
specific null mode or establish a need for an additional gauge.
## Higher index: the larger window is not sufficient
For n=1.5, J=0, no central constraint, restart 160, seven linear solves meet
their true-residual tolerance. Nonetheless, Newton step lengths shrink to
1/2048 and F remains 1.75135e-4. The eighth linear solve fails at relative
defect 0.07910. The first severe damping occurs with a zero initial guess,
before a reused correction can explain the difficulty. Later reused guesses
start with defects about 1.5–2.0 times the RHS norm.
Recorded rejected trials fail the merit test; their geometry remains valid.
Smaller reported starting-step caps include line-search history and must not
be mistaken for geometric admissibility limits. Gravity-gradient and Poisson
rows hold about 99.69% of the squared normalized endpoint residual. Those rows
include both interior and exterior contributions, so this does not locate the
physically important interior error.
P(h) energy/force virials pass at approximately 2.23e-6/2.33e-6, but Bernoulli
and hydrostatic consistency worsen. P(rho) is undefined at some samples because
of small negative DG density values; negative mass fraction is about 2.89e-5.
The corresponding pressure sensitivity budget is about 3.76e-7 in virial units.
This endpoint is not classified as a validated equilibrium.
## Rejected one-way gravity completion
The tested candidate `8c1f823` changes only the structure factorization to
material-then-gravity triangular completion. It retains exactly the default
material component, Chebyshev order 5 gravity mass inverse and three-cycle
AMG potential inverse. It includes density and shape couplings and rebuilds
the scalar border. Its baseline is `9bd4af6`.
Both runs reconstruct the identical n=3 constrained seed. Snapshot, layout,
initial physical measurements, and frozen RHS block norms agree exactly.
Both cold Krylov windows are measured; each policy's restart-160 correction
is then tested without committing any nonlinear step.
| Frozen result | Baseline | Candidate |
| --- | ---: | ---: |
| Restart-40 true residual / RHS, 200 iterations | 0.0563080 | 0.0715826 |
| Restart-160 true residual / RHS, 200 iterations | 0.0402590 | 0.0381781 |
| Restart-160 solve seconds | 98.92 | 117.71 |
| Selected normalized correction norm | 0.75764 | 1.54083 |
| Sampled thermodynamic step bound | 0.15244 | 0.09135 |
Neither clears the 0.03 linear gate. The candidate's alpha=0.1 trial is
inadmissible; the baseline's is admissible and lowers F by 3.40%. The
candidate's best sampled admissible trial, alpha=0.01, lowers F by about 1.01%.
The unchanged seed virial is not evidence about the uncommitted trial states.
Completion contracts the central-border column's Poisson defect about 4.8-fold
but worsens the Bernoulli column's Poisson defect. Those columns have zero
shape component, so this mixed response already occurs for density-source
completion. The experiment does not rule out better gravity subsolves or
two-way material/gravity feedback.
There is also direct evidence of a weak normalized direction. Baseline
restart-40 and restart-160 corrections have cosine -0.985 despite the same
state and Jacobian. If e_i=Jp_i+F, then
J(p_2-p_1)=e_2-e_1, giving a response bound
||J(p_2-p_1)||/||p_2-p_1|| <= 2.77e-4. The candidate adds still more amplitude
in essentially the restart-160 direction. This is not a measured singular
value or an identification of homology, translation, or surface error.
## Warm-start safeguard
The tested source is `1911a26`. The guard checks
the current true defect of a nonzero supplied correction and falls back to
zero only if that defect norm is nonfinite or exceeds ||b||. The unchanged
input contract still rejects nonfinite coefficients in the supplied guess.
It retains finite supplied guesses whose current defect is no larger than that
of a zero guess and leaves the damped-Newton history policy in place. A smaller
initial defect does not guarantee faster subsequent Krylov convergence.
It adds no public options. A nonzero guess costs one extra measured operator
action; zero guesses do not.
Five focused tests pass on one rank (356284 assertions). The two pure-backend
tests also pass on two MPI ranks (41 assertions on each rank), including a bad
guess or overflow confined to one rank. These results establish behavior of
the safeguard. The matched n=1 run then reproduces the control byte-for-byte:
accepted state, physical CSVs, trial records and all non-timing Newton columns.
Its eight virial checks pass, and it reaches the identical strict convergence.
At the first shared n=1.5 damped state, the guard changes the effective initial
defect/RHS from 1.5097 to 1 and reduces linear iterations from 71 to 65. The
same alpha=1/64 is accepted, with a slightly smaller nonlinear residual. Later
comparisons follow different trajectories and are not same-matrix comparisons.
| n=1.5 capped-run result | Control | Guard |
| --- | ---: | ---: |
| Successful linear solves | 7, then an eighth failure | All 8 |
| Total linear iterations | 672 | 561 |
| Iterations in first seven attempts | 472 | 467 |
| Final normalized residual | 1.75135e-4 | 1.74946e-4 |
| Final accepted step | 1/2048 | 1/8192 |
| End-to-end seconds | 550.36 | 508.16 |
Most of the total iteration reduction comes from avoiding the final 200-step
failure. The guard reaches the nonlinear iteration cap instead. Its residual
is only 0.108% lower, so this is not a nonlinear-plateau cure or a claim of
mesh-independent solver performance. The timings are single observations.
P(h) energy/force virials remain near 2.24e-6/2.35e-6, changing only about
9.0e-9/2.0e-8. EOS and Bernoulli errors improve slightly and the negative-density
budget barely changes. The conservative solver-plus-eight-check screen remains
unsatisfied: strict convergence is absent and P(rho) cannot be evaluated. P(h)
meets the primary virial milestone; this does not validate the local solution.
The center-of-mass offset increases from 1.67e-6 to 4.57e-6, so it would
be incorrect to claim that all physical metrics are unchanged. Both endpoints
remain unconverged. These limits motivate retaining the guard as a modest
improvement to the inner solver while pursuing the operator/weak-direction work.
## Validation and reproducibility
`experiments/run_plateau_campaign.py` builds a committed source revision and
retains exact case commands, build settings, mesh and binary hashes, an
immutable executable, snapshots, CSV diagnostics, and observed exit codes.
Cases run serially. A campaign exit of zero means observations completed;
a model exit of three remains recorded as nonconvergence. Timing samples are
not repeated benchmarks, and dynamically loaded dependencies are not frozen.
`experiments/check_plateau_physics.py RESULT_FOLDER` gates strict recorded
convergence and all eight defined, finite virial errors at 1e-5: energy/force,
P(h)/P(rho), and quadrature 14/18. This conservative joint screen is stricter
than the primary physical milestone. In particular, the undefined P(rho) at
n=1.5 does not by itself prove physical virial failure; the defined P(h)
virials pass. The checker separately reports sign, conservation,
EOS, and Bernoulli measurements. It does not certify pointwise correctness or
assign arbitrary local-error tolerances. The checker distinguishes the
converged n=1 case, the n=1 algebraic failure despite good virials, undefined
P(rho) at n=1.5, and the failing n=3 seed virial.
The combined source at `b3a752b` builds the sandbox, tests and diagnostic.
Three context/root tests pass (61 assertions): explicit seeding without a
central constraint, the generated angular-velocity/composable border, and
propagation of that angular velocity into physical rows. Together with the
five safeguard/Newton tests this is eight distinct focused C++ tests, with
two backend tests additionally repeated on two MPI ranks. It is not a rerun
of the entire historical suite. Later combined commits change documentation
and case manifests only; the final live campaign records `c88baa1`.
## Final combined physical checks
The combined executable has the same SHA-256 as the guarded executable. The
nonrotating n=1 case is the physical counterpart of the sandbox model, using
the diagnostic stopping controls. It converges in six full Newton steps,
with F=1.65157e-13 and 307 total linear iterations. Its worst virial error
among all eight checks is 3.73092e-7; the largest q14/q18 change is 1.29e-13.
At q18 the absolute mass error is 8.88e-15, J error is zero, and no negative
density or enthalpy is sampled.
Local consistency errors remain: peak/RMS EOS mismatch divided by sampled
density scale is 4.23549e-4/8.56537e-5, Bernoulli RMS is 1.13917e-4, and
hydrostatic gradient RMS is 5.48097e-3. These are measurements, not newly
invented acceptance tolerances. The literal sandbox executable was compiled;
its physical counterpart was run through the diagnostic for these measurements.
The J=0.1 transfer case doubles the previous target angular momentum and allows
12 Newton attempts, while retaining the numerical implementation and tolerances.
It is a physical transfer test, not a one-variable solver-policy comparison.
It does **not** converge: all 12 linear solves pass (507 total iterations), but
F plateaus at 6.25389278e-3 and the last accepted step is 4.76837e-7. The process
stops at its nonlinear iteration cap. Its 527.62 seconds include diagnostics;
summed Newton-attempt time is 462.33 seconds.
This endpoint also fails the primary virial milestone. At q18, the P(h) energy
and force virial errors are 4.75186e-4 and 8.18118e-4; q14 agrees to within
4.54e-14. Absolute mass and J errors are 6.42266e-4 and 2.13748e-4. Actual angular
velocity is 0.3431369602 in reference units, not a critical-rotation fraction.
P(rho) is undefined at negative-density samples; the negative mass fraction is
1.07284e-5. Peak/RMS scaled EOS mismatch is 1.02932e-2/5.75843e-4, and raw
Bernoulli RMS is 9.92365e-4. This is an unresolved failed model.
All 30 rejected trials fail pressure-force preparation before residual
measurement, unlike the merit-only n=1.5 plateau. The final sampled minimum
enthalpy is 1.82998e-10, with no sampled negative enthalpy; the minimum sampled
mapping determinant is 0.870965. Finite negative enthalpy is the leading
inference for rejected trials: pressure preparation explicitly rejects h<0,
and geometry preflight includes the pressure quadrature rules. However, the
recorded rejection also permits a nonpositive determinant/mapped-weight cause,
and the CSV omits the exact rejection code. An accepted state's sampled minima
do not certify its rejected trials. The present evidence supports locating an
admissibility obstruction, not declaring its exact sample or cause established.
Later supplied guesses are retained with defect/RHS near 0.03, so discarding
bad warm starts does not address this final obstruction.
## Retained Git history
The original checkout is left on `research/combined-20260916`, with the adopted
changes committed. The starting optimized source is committed on the baseline
branch; historical untracked caches and data are preserved. Separate hypothesis
branches remain available for inspection or reruns.
| Branch | Recorded revision / purpose |
| --- | --- |
| `research/baseline-20260916` | `80cc3d8`: preserved incoming source |
| `research/experiment-harness-20260916` | `9bd4af6`: common driver; `0470e3b`: checker; `899cb18`: screening clarification |
| `research/central-constraint-20260916` | `5167744`: hypothesis evidence; `2e19c2c`: combined validation notes |
| `research/krylov-window-20260916` | `937d2f2`: example restart 160; `1f9184b`: combined validation notes |
| `research/gravity-coupling-20260916` | tested `8c1f823`, outcome `db0ddd3`: rejected completion retained here |
| `research/krylov-20260916` | tested `1911a26`, outcome `7127383`: guarded warm guesses |
| `research/combined-20260916` | C++ build/context tests `b3a752b`, final live cases `c88baa1` |
## Next discriminating work
1. At a frozen stalled state, split a correction into shape-only and remaining
field components. Measure full, pure-shape, pure-field and mixed Taylor
remainders at common admissible step lengths. Fixed-shape gravity is affine
in the fields, so its field-only remainder should be at roundoff.
2. Characterize the weak correction differences with physical moments and
controlled symmetry/shape tangents before adding a gauge or regularization.
3. Separate interior, exterior, and interface gravity error. Exterior accuracy
matters through its effect on the interior; current aggregate block norms
cannot establish where the plateau lives.
4. For the stronger-rotation pressure-preparation failures, record the specific
rejection reason and offending enthalpy/mapping sample. Geometry preflight
already includes pressure quadrature; negative h is the leading inference,
but the existing CSV cannot exclude a mapping/preflight discrepancy. Locate
the obstruction before selecting a positivity or surface treatment.
5. If residual normalization is changed, first rescore identical physical trial
states. This separates a changed merit from a changed Newton correction.
High-rotation continuation and a measured critical-rotation fraction remain
future validation work. These results do not establish the 50% or 99% targets.
## Artifact index
The complete campaign is retained at
`/Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16`.
Each of `controls`, `baseline_n3`, `gravity`, `guard`, and `combined` contains its copied
case manifest, source/build provenance, executable, run records and diagnostics.
Use its retained `cases.json` with `run_plateau_campaign.py` at the recorded
source revision and a new output directory to repeat the observation. The
runner invocation and build requirements are in `plateau_diagnostic_README.md`.
- [Detailed controls](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/control_analysis.md>)
- [Gravity-coupling comparison](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/coupling_comparison.md>)
- [Gravity-coupling interpretation](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/coupling_interpretation.md>)
- [Warm-start comparison](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/guard_analysis.md>)
- [Final combined physical validation](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/combined_validation.md>)
- [Combined context/root test manifest](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/combined_checks/manifest.json>)
- [Focused safeguard test manifest](</Users/tboudreaux/.codex/.chatgpt-projects/g-p-6a6b3f2194ec8191bebcd59075897f1d/diagnostics/numerical_campaign_2026-09-16/guard_checks/manifest.json>)