Record preconditioner conclusions and define next nonlinear obstruction investigation

experiments/NONLINEAR_STEP_INVESTIGATION_2026-09-18.md

@@ -0,0 +1,111 @@
+# Nonlinear step obstruction after gravity preconditioning
+
+## Recorded decision from the completed preconditioner investigation
+
+The completed report is `PRECONDITIONER_IMPROVEMENT_2026-09-18.md`, committed
+at `dc1da62` on `research/preconditioner-benchmarks-20260918`. Its immutable
+campaigns live under the task workspace's
+`diagnostics/preconditioner_improvement_2026-09-18/` directory.
+
+- Consistent gravity Schur refinement solves the saved n = 1.5, J = 0 linear
+  system in 132 iterations and 72.3727 s, versus 231 iterations and 100.9943 s
+  for the independent control. Both satisfy the same independently verified
+  relative full-J defect of 0.03: a measured 1.39548x speedup.
+- Combining this refinement with stellar LDU solves the saved rotating
+  n = 1, J = 0.1 system in 142 iterations and 96.8236 s. The independent and
+  gravity-only controls still fail after 600 iterations. Their finite
+  time-to-convergence is unknown, so no rotating speedup ratio is reported.
+- Prepared geometry reuse makes the existing stellar LDU application 8.0425x
+  cheaper without improving its convergence by itself. Retain this
+  implementation optimization independently of policy selection.
+- The tested nine-mode shape correction does not improve the n = 1.5 case.
+  It remains an experimental negative result, not a rejection of all coarse
+  spaces.
+- The easy n = 1, J = 0 model preserves all eight virials below 3.74e-7,
+  while the stronger inverse costs about 4% more. Defaults remain unchanged.
+- Neither hard nonlinear model is physically acceptable. Negative sampled
+  density makes density-pressure virials undefined. In rotation, a smaller
+  nonlinear residual accompanied a larger enthalpy-pressure virial error.
+  Residual decrease is not a substitute for virial and field-admissibility
+  validation. No angular momentum has been calibrated to critical rotation.
+
+## New investigation and controls
+
+Branch: `research/nonlinear-step-obstruction-20260918`, based on `dc1da62`.
+The configured worktree is reused to avoid an unrelated full rebuild; the
+preceding branch and copied campaign executables retain their exact source
+and binary identities. This branch adds experimental diagnostics only.
+
+Start with the n = 1.5, J = 0 endpoint of
+`newton_v1/n1p5_J0_curvature_gravity_schur`. All twelve outer linear solves and
+all 31 auxiliary gravity solves passed, but the final nonlinear residual was
+1.3966991e-5 against a target of 2.7365922e-12. The final accepted alpha was
+1/512. Every trial prepared successfully; all nineteen rejections were
+residual-decrease failures, and geometry never limited the step.
+
+The final squared-residual shares are approximately 42.42% hydrostatic,
+26.23% closure, 15.87% surface, 10.43% gravity, and 5.05% mass. This motivates
+examining retained-equation curvature after the gravity correction, rather
+than assuming the earlier gravity or pressure-preparation failure persists.
+
+Keep the complete physical equations, fixed normalization, optional
+central-density specification (absent here), mesh, and acceptance thresholds
+unchanged. Use one gravity Schur refinement at relaxation 0.5 and the same
+restart length 160. Fresh cold solves and independently applied full-J true
+defects determine direction quality. Trial probes never commit a model.
+
+## Questions and discriminating evidence
+
+1. **Does the prepared trial path have the expected tangent?** For frozen
+   residual F and computed direction p, measure Jp and actual prepared
+   differences. The curvature-corrected path should preserve that tangent
+   because its gravity correction is second order in alpha. A discrepancy
+   that remains first order above arithmetic noise would implicate the
+   linearization or preparation path.
+2. **Does the inexact linear direction cause the obstruction?** Compare
+   directions at the same saved state with forcing tolerances 0.03 and 0.003,
+   subject to a recorded finite iteration budget. A tighter requested
+   tolerance counts only if its independent true gate passes. Failed solves
+   are recorded and do not substantiate this comparison.
+3. **Which nonlinear terms oppose the linear prediction?** Split p into its
+   actual surface coordinates and all remaining coordinates. Compare raw
+   and gravity-corrected trials for full, shape-only and fields-only
+   directions. Form mixed remainders by vector subtraction. Signed
+   projections against the full linear prediction distinguish adverse
+   curvature from large but cancelling terms.
+4. **Is the behavior specific to the late state?** Use the same-polytrope
+   Lane–Emden seed as an early-state control, where the previous production
+   run accepted its first full Newton step. Record preparation and enthalpy
+   feasibility separately from the residual metric.
+
+For phi = ||F||^2/2 and ||Jp + F|| <= eta ||F||,
+
+    phi'(0) = <F,Jp> <= -(1-eta) ||F||^2.
+
+At eta = 0.03 the slope must be strongly negative if J differentiates the
+prepared path. The production sufficient-decrease test therefore ought to
+accept sufficiently small resolved steps. The investigation measures whether
+large adverse higher-order terms explain how small those steps become.
+
+Each block's vector remainder R = F_trial - F - alpha Jp must satisfy
+
+    ||F_trial||^2 = ||F + alpha Jp||^2
+                   + 2 <F + alpha Jp, R> + ||R||^2.
+
+For the corrected trial map, its mixed remainder includes interactions
+introduced by gravity correction; it is not identified solely with the raw
+physical shape Hessian. Below-floor differences are unresolved, not evidence
+of exact linearity.
+
+## Execution and validation
+
+Run a late-state probe first, then use its evidence to choose the matched
+tighter-direction and early-state controls. All model jobs and builds run
+serially. The campaign runner records clean source commits, exact commands,
+input/output and executable hashes, and refuses existing output directories.
+The diagnostic saves the frozen vectors, layout and normalization, checks
+restoration and accepted-buffer invariance, and reports failed preparations
+and inner solves without inventing values for missing trials.
+
+Results are pending. No numerical remedy or new convergence claim follows
+from the investigation design alone.