MeanField/experiments/SHAPE_FIELD_WEAK_RESPONSE_2026-09-18.md

Shape–field weak response investigation — 2026-09-18

Prespecified scope

Follow up the large, nearly cancelling shape/field Newton direction for n=1.5, J=0 without a central-density constraint. This study distinguishes within-element physical field transport from surface variation and tests the specific hypothesis that the Q3 displacement/Q4 reference geometry mismatch contributes to weak response. Cancellation by itself is not evidence of a gauge or bad element orders.

The baseline branch is research/shape-field-weak-response-20260918, based on db854c0f7cdf5137e377273499f0ac5b6053b752. The isolated order variant is research/shape-field-displacement-q4-20260918. Preserve original checkout, immutable executables, source revisions, mesh/input hashes and serial run records.

Observation and controls

The diagnostic prepares a frozen state with fixed normalization, independently verifies its full-J linear defect, saves every value-block Jacobian contribution, and checks restoration of accepted buffers and operator action. Three small raw Taylor trials test the same original operator. Physical observations use independent grid functions and the prepared displacement; no accepted state is modified.

At common quadrature orders 10 and 14, record physical scalar pullback changes, transport grad(f).v, and Eulerian changes delta f - grad(f).v. Density and potential gradients are broken/local: facet jumps and RT Piola variations are not included, so these observations cannot certify a pure relabelling nullspace. Surface radial parameters are unweighted nodal samples. An independent affine identity projection checks ambient displacement representation, separately from the actual r-squared radial extension. Small-span SVDs describe only the sampled block directions, never the full Jacobian rank.

Baseline frozen cases: the recorded n=1.5 seed and late directions at eta=0.03, and a fresh cold solve at the six-update coupled endpoint. Then a fresh six-update baseline with gravity-Schur inverse, gravity-curvature globalization, restart 160, maximum 200 Krylov iterations, eta=0.03, and a fresh final direction. Match this live run with the Q4 variant from the same analytic seed, including a Q4 seed probe. Cases, builds and expensive audits run serially. A failed linear gate or preparation remains a failure; do not silently relax controls.

Order intervention (conditional on baseline observation)

Only change Displacement::vectorOrder from 3 + uniform increment to 4 + uniform increment on the variant branch; keep uniform increment zero. The loaded reference geometry remains Q4, density/potential L2 Q2, gravity RT index 2, and enthalpy H1 Q3. The shared displacement collection means the scalar surface trace also becomes Q4. This introduces shape modes and changes derived quadrature orders and cost; it is not a pure interpolation-error intervention. Document exact source line, date, rationale, DOFs, and compiled quadrature changes. Use a separate build directory and new analytic seeds: existing coefficient snapshots cannot be transferred unchanged between orders.

Decision criteria

Compare verified linear accuracy, accepted step lengths, within-variant growth of weak-direction indicators, common physical norms, virials, positivity/EOS metrics and elapsed cost. Across orders, coefficient norms and discrete residuals are not physical error measures. Q10/Q14 agreement diagnoses integration sensitivity, not convergence certification. Virial <=1e-5 remains required but cannot replace internal pointwise checks. Six updates are a bounded mechanistic comparison, not evidence of high-n/high-rotation convergence.

Results

The baseline branch retains Q3 displacement. The separate Q4 branch is an isolated numerical experiment, not a recommendation to change the default.

Reproducible baseline

Evidence root: ../../diagnostics/shape_field_weak_response_2026-09-18 relative to this checkout. Frozen probes baseline_probes_v1 use source d79db94; the six-update control q3_live_v1 uses 2c4a5e0. Both include immutable binaries, mesh/input hashes, commands, clean-source provenance, and independent elapsed and CPU clocks. All full-J linear gates passed. Independent audits baseline_probes_audit_v2.json and q3_live_audit_v1.json pass; accepted buffers, factors and restored actions agree exactly. Q3 curved/tensor/geometry regression: 5 cases, 21,368 assertions passed.

The first checker version omitted the new cache-preservation invariant from its schema. Version 2 requires it. A small finite synthetic matrix reproduced NumPy/Accelerate matmul warnings; direct finite-checked contractions avoid that backend path, without changing tolerances. Raw FEM artifacts remain unchanged. The artifact checker passes 38 tiny algebra/corruption fixtures. The angular analyzer independently checks its degree-8 recurrence, orthogonality, cubic symmetries, rank, and file binding on analytic fixtures.

Observation	Seed	Old late state (12 updates)	Fresh control (6 updates)	Coupled endpoint (6 updates)
Normalized residual	2.73659e-4	1.39670e-5	1.49720e-5	1.24583e-5
Sampled block-span minimum singular response	5.80574e-2	1.43543e-4	8.16558e-4	6.94894e-4
Sampled surface radius span / nodal mean	3.42403e-5	1.24122e-2	1.09849e-2	1.11125e-2

The largest sampled singular response remains approximately 22.1–22.6. The SVD uses the six nonzero disjoint value-block pieces of each p, normalized by their coefficient norms. The direction-dependent subspace changes between observations; the comparison combines changes in the operator and the probe subspace. It is not a full-J spectrum or rank test. In particular, comparing the 12-step late state with six-step endpoints does not demonstrate a causal benefit of the coupled correction: the matched six-step control has a slightly stronger minimum response than the coupled endpoint.

Late q14 Eulerian/pullback change norm ratios are density 0.05769, enthalpy 0.18870, potential 0.01288, versus seed 0.3243, 0.3080, 0.1993. This is substantial transport compensation but not physically invisible motion. Broken DG gradients omit jumps. The maximum q10/q14 relative difference in the three original cases' Eulerian squared integrals is 1.69e-5, an integration sensitivity observation.

The radius observations expose a validation gap: the old late state has nodal radii 0.997674–1.010093, while its q18 P(h)-based virial error is 4.06909e-6. The fresh six-step control has q18 virial error 4.22590e-6 and reproduces the old six-step residual trajectory. Its sampled hydrostatic-gradient RMS increases from 0.00152810 to 0.0106811 despite the lower residual and virial error. Neither is converged or physically certified; negative density samples remain. Virial success can coexist with appreciable nonrotating surface asphericity at these unfinished states.

q3_surface_modes_v1.json contains unweighted fits, not surface-integral power. The 12-step late direction is 99.9995107% cubic-invariant in sampled squared norm, but degree <=8 spherical harmonics leave 0.893679 of its nodal norm. Degree <=2 leaves 0.981646. Thus most of the nodal direction is unresolved by degree <=8 fits and is nearly cubic-invariant. This is consistent with mesh/discretization coupling, without identifying its source: cubic symmetry also exists in the bulk and gravity discretizations. These are not surface-area spectra or evidence that the surface discretization alone causes the problem.

The affine control on the actual stellar mesh gives Q3 relative value error 9.01277e-5 and gradient error 1.23472e-3. This warrants the isolated Q4 comparison, but adding surface modes could also worsen weak response.

Structural interpretation and a rejected shortcut

At J=0, bulk Bernoulli rows are unweighted moments. Their direct shape derivative at fixed reference-field coefficients is integral(q * B * div(v)); the EOS shape column similarly contains its local closure imbalance. Thus pointwise near-equilibrium shape information increasingly relies on gravity and mechanical coupling. Small projected residuals alone do not prove small pointwise imbalance. Current shape rows are the prescribed lift's transpose applied to volume force. For regular surface hs, densitys^n and pressure~s^(n+1), weakening near-surface coupling as n increases is plausible. DG jumps and the independent RT field prevent concluding exact discrete redundancy from the continuum identities.

Relevant assembly: libmeanfield/impl/operators/prepared_hydrostatic_equilibrium.cpp (lines 864–911, 1354–1397), operators/kernels/barotropic_closure_kernels.cpp (line 655), and operators/prepared_stellar_equilibrium.cpp (lines 806–811).

A naive replacement of mechanical shape rows by surface Bernoulli moments is rank deficient at the desired pointwise B=0, J=0 root with the current trace spaces. The actual closed stellar surface has 96 quads, 192 edges, 98 vertices. Continuous surface Qp has 96pp+2 degrees of freedom; broken Q(p-1) potential traces supply only 96pp. Q3 surface therefore has 866 rows versus 864 DG Q2 stellar trace coefficients, after eliminating enforced h=0 trace variations. C adds no independent trace dimension; the direct shape measure term vanishes with B. Surface-only Q4 widens this hypothetical deficit to 1538-864=674. This concerns that proposed replacement, not a proven deficiency of the current mechanical Jacobian. Any boundary formulation requires a compatible trace/test space and explicit rank/stability analysis.

An invertible linear/tangent coordinate or preconditioner change preserves the exact mapped physical Newton correction at a fixed state. It can improve coordinate-dependent singular values and inexact linear solves. A nonlinear coordinate map or retraction can also change finite trial curvature; none of these observations alone removes weak physical response. Shape representation, step control, and operator formulation remain distinct hypotheses.

Q4 intervention

Applied 2026-09-18 at approximately 17:45 UTC on research/shape-field-displacement-q4-20260918, branched from baseline 211be17. Changed libmeanfield/interface/field/field_registry.cppm, Displacement::vectorOrder, from 3 + uniformPolynomialOrderIncrement to 4 + uniformPolynomialOrderIncrement; configured uniform increment stays zero. The dated source comment records the rationale. Eleven expectations and their comments in six test files change to enforce the new displacement degree and derived quadrature, while assertions for all other field orders stay fixed.

The purpose is to remove measured ambient affine representation error and test its effect on the weak direction. The shared scalar surface increases from 866 to 1538 parameters, so this also adds shape modes. Derived pressure-force order increases by one, gravity-force order by two, and geometry-inspection request from 18 to 24, including the geometry-admissibility sampling; common physical diagnostic quadratures remain 10/14 (weak-direction observations) and 14/18 (state metrics). The surface metric, normalization and preconditioner also change with their FE space. Equal relative linear tolerance does not imply equal physical error or an identical line-search norm. The default r-squared radial lift stays unchanged and is not made globally affine by this ambient enrichment.

Use the separate cmake-build-release-displacement-q4 directory with the same LLVM 22.1.8, Release/IPO, MPI and library configuration. Rebuild all affected modules and registered-space tests together. Q4 cases start from the analytic seed; no coefficient snapshot is transferred between orders. The fixed 200-iteration linear budget and eta=0.03 remain unchanged. A 3600-second process limit bounds each Q4 numerical case; no silent control relaxation is permitted.

Completed Q3/Q4 comparison

Q4 source is 4be1a23c3ec2650af6a2a49528a8f5248cb001c3. Its only production change is the compile-time displacement degree at libmeanfield/interface/field/field_registry.cppm:163; all instrumentation and analysis remain in experiments/. No physics-facing API changed. Build and dependency configuration values match the Q3 build. The Q4 regression campaign passed 22 cases and 380,856 assertions: registry/quadrature contracts, field spaces, curved geometry, surface/radial/domain Jacobians, Lane–Emden projection, and prepared pressure/gravity force Jacobians. Cases, builds and full artifact audits were serialized.

q4_v1 contains the completed seed and live campaigns. The independent audit q4_audit_v1.json passes 101,282 and 101,310 checks respectively, including raw vector algebra, runner hashes, physical-integral identities, and exact buffer, normalization, residual and Jacobian-action restoration. An artifact audit pass does not turn a failed solve into a successful solve. The endpoint direction is explicitly classified finite_direction_failed_true_gate.

The analytic seed has identical physical volume diagnostics in both variants. Q4 reduces the ambient affine relative value error from 9.01277e-5 to 2.11629e-16 and gradient error from 1.23472e-3 to 3.24701e-15. Nevertheless, the seed Newton direction's stellar displacement L2 grows from 0.00121967 to 0.0385749, a factor of 31.63. The seed linear solve grows from 26 to 67 iterations; both meet the same independently checked eta=0.03 criterion. All three raw small Q4 seed trials prepare and exhibit a quadratic residual remainder. This is not evidence of a missing first-order derivative in the tested direction.

The fixed six-iteration-budget comparison gives:

Measurement	Q3 displacement / surface	Q4 displacement / surface
Surface parameters	866	1538
Total root unknowns	178073	178745
Accepted updates	6	3
Termination	Nonlinear iteration limit	Fourth linear solve exhausted 200 iterations
First three accepted-step time	269.31 s	505.65 s
Residual after three updates / own initial residual	0.056755	0.906188
Final residual / own initial residual	0.054710	0.906188
All Newton-iteration time, including failed attempt	547.34 s	638.42 s
Outer Krylov iterations, including failed attempt	354	492
Endpoint probe time, excluded above	93.07 s	222.45 s
q18 P(h) potential-energy virial error	4.22590e-6	8.89666e-5
q18 P(h) force virial error	2.71367e-6	8.27467e-6
q18 mass error	3.41685e-6	1.96178e-5
q18 scaled EOS density mismatch RMS	1.27948e-4	1.61156e-4
q18 hydrostatic-gradient RMS	0.0106811	0.325413
q18 gravity versus broken potential-gradient RMS	0.0121494	0.326881
Negative density mass fraction	2.89683e-5	4.19384e-5

These are differing stopped endpoints, not equal-accuracy or converged solutions. The three-update comparison isolates the accepted-count difference in the table, but all the previously listed order/space/quadrature/scaling confounders remain. Iteration time excludes initialization, final physical measurements and the diagnostic probe; total live-process times were 705.76 s and 944.98 s. The stdlib helper compare_shape_order_runs.py binds the small comparison inputs to runner hashes and reproduces q3_q4_comparison_v1.json.

Q3 accepts steps 1, 1/16, 1/16, 1/16, 1/32, 1/32. Q4 accepts approximately 0.1770, 0.04425, 0.04425. Its fourth linear solve and independently repeated cold endpoint solve both stop after 200 iterations with relative full-J defect 0.0868564, above 0.03. The finite failed direction remains a directional observation only. It is not used as an equally accurate Newton comparison.

Q4 also changes the observed physical character of the seed direction: the q14 Eulerian/pullback norm ratios for density, enthalpy and potential become 13.0825, 13.4289 and 13.2393. Thus the added deformation is not largely cancelled by scalar transport, unlike the late Q3 direction. These are broken scalar observations, not a complete distributional or RT transport analysis. The largest q10/q14 relative change in Q4 endpoint Eulerian squared integrals is 4.35e-5; this is an integration sensitivity check, not discretization convergence.

q4_surface_modes_v1.json passes its independent angular analysis. At the stopped Q4 endpoint, sampled radii span 0.927235–1.049839, or 12.2559% of their nodal mean. The Q3 endpoint's observed span is 1.09849%. These are different nodal sample sets, not common-point or full-surface extrema. In particular, zero initial radius variation at Q4 nodes does not make the curved reference surface an exact sphere: the same physical mesh sampled at Q3 nodes has a small nonzero span. The Q4 seed direction's degree <=8 nodal fit leaves 0.982810 of its norm; its sampled squared norm is 99.9999989% cubic-invariant. The corresponding failed endpoint direction leaves 0.988475 of its norm. This remains fine angular structure, without identifying which discretization component causes it.

The saved figure surface_order_comparison_v2.png uses a common color scale and labels the different sample and accepted-update counts. Version 1 is retained as an unused layout draft; version 2 fixes the caption/colorbar overlap.

Conclusions and the next isolated hypotheses

Do not promote the Q4-only change. It repairs ambient affine representation but worsens useful nonlinear progress, linear cost, interior gradient checks, and potential-energy virial balance in this case. This rules out order matching alone as a sufficient repair under the tested configuration. It does not show that the Q3 representation error contributes nothing, nor that a compatible enrichment of the other fields could not help.

The leading hypothesis is an unfavorable combination of surface freedom and the resolving power of the coupled field equations. The observed directions have strong fine angular structure, and adding surface freedom produced much larger physical motion without useful residual reduction. Normalization and preconditioning remain possible contributors; full-J residual accuracy does not bound physical correction error for a weak operator. More Krylov work might recover a linear solution, but the first three gate-passing Q4 directions already gave poor accepted progress. No exact nullspace or discrete stability theorem has been established for the present mechanical operator.

The next direct experiment should separate independent surface parameters from ambient displacement interpolation. A fixed map q=q0+B*a with small smooth bases of increasing degree can test this hypothesis around the same frozen state. Inject physical variations and restrict the existing dual shape force with B^T; the reduced blocks are the corresponding Galerkin restrictions. Pull back the surface metric and preconditioning surrogate consistently, then apply state/residual normalization. Do not transpose an already normalized value basis and assume it is the physical dual restriction. A surface-mass-orthonormal basis is a controlled choice. The deformation contract already separates parameter count from boundary-displacement size; the default problem constructor, PhysicalRiesz surface metric, and material-surface preconditioners still contain nodal assumptions that must be changed together.

That reduced solve changes the approximation/test space. Its excluded original shape forces and full original residual must remain visible. A frozen q0 retains its existing surface distortion; success there would justify a later analytic-seed sequence with increasing surface resolution, not certify a smooth physical root. Suggested first frozen bases use spherical degrees 2/4/6/8 (9/25/49/81 parameters). High-rotation work must demonstrate adequate shape resolution rather than enforce a low-mode answer. This experiment has not been implemented here.

Existing experiments/README.md, gravity_completed_rigid_motion.cpp, and the surface-mode/reachability/frequency probes provide relevant mode construction and gravity-completion infrastructure to reuse. The proposed addition is the fully coupled restricted Newton solve with a consistent dual restriction and excluded-force checks. Historical CSVs in the original checkout were not rerun or incorporated into this controlled order comparison.

A separate extension test addresses the virial dilation mode. Q4 ambient fields can represent the reference position X, but the production logical-radius-squared extension E0 does not generate X from its surface trace b. At a retained logical radius 1/8 vertex, lifting b gives only one eighth of the required affine value; this remains true with Q4 interpolation. A fixed rank-one correction E(z)=E0(z)+(X-E0(b))*ell(z), with ell(b)=1, preserves the supplied trace and gives E(b)=X. Its exact transpose is E0^T*f + ell^T*((X-E0(b))^T*f); its Jacobian is constant. With an interior state displacement u=E(z), direction b+z then generates current physical dilation X+u. Reference-coordinate access, MPI weights, and adjoint virtual work require explicit tests. Keep the fixed-infinity vacuum policy unchanged.

This would place the discrete dilation test in the mechanical force space; it does not automatically enforce the potential-energy virial using the separate mixed potential. The Q4 endpoint demonstrates why both independent virials are needed. The rank-one lift also does not remove the observed fine angular modes, so it is a separate structural control, not a presumed cure. It remains unimplemented and must not be bundled with surface reduction in a causal test.

All reported states remain nonconverged and include negative density samples; P(rho)-based virials are undefined there. The required 1e-5 virial milestone and internal pointwise validation have not been achieved by this order intervention. No claim about higher-n or rapidly rotating convergence follows from this study.