MeanField/FIELD_DOF_MIGRATION_HANDOFF.md

MeanField FieldDof Migration Handoff

Repository: /Users/tboudreaux/Programming/4DSTAR/MeanField
Prepared: 2026-08-28
Primary objective: make registered field/domain support, rather than raw MFEM true-vector size, define the nonlinear and prepared-operator coordinate systems.

1. Scope and relationship to the earlier review

This document is the continuation handoff for the work performed after the earlier Rotating Polytrope Review summary. The receiving agent is expected to receive that earlier summary as well, so this document does not reconstruct every pre-migration investigation. It concentrates on:

• the final FieldDof architecture and its invariants;
• the gravity, barotrope, prepared-operator, and root-state migrations completed in this work;
• the canonical FieldDofMap/GridFunction adapter;
• the replacement and hardening of the domain-mapping system;
• dead-code and file cleanup;
• failures encountered during migration and what they taught us;
• final validation evidence;
• the small amount of cleanup that remains.

2. Executive status

The functional FieldDof migration is complete for the currently registered production fields and the coupled stellar-equilibrium operator stack.

The final solver-facing convention is:

registered field + DomainSchema
              |
              v
        FieldDofMap
              |
     reduced solver coordinates
              |
     prepared operator/context boundary
              |
       explicit scatter/gather
              |
       full MFEM true vectors
              |
        element kernels / MFEM

The complete Catch2 inventory was covered by three sharded runs:

Validation group	Test cases	Assertion executions	Result
[gravity]	58	3,746,701	Pass
[barotrope]	88	192,694	Pass
~[gravity]~[barotrope]	176	136,493	Pass
Total executions	322	4,075,888	Pass

There are 320 unique registered test cases. Two prepared stellar-equilibrium tests carry both [gravity] and [barotrope], so the three filters intentionally execute those twice:

• Prepared Stellar Equilibrium Has A Restoring Jacobian Around An N3 Polytrope
• Prepared Stellar Equilibrium Matches The Analytic N1 Lane Emden State Up To The Mixed Projection Floor

Therefore:

58 gravity + 88 barotrope + 176 remainder - 2 overlap = 320 unique tests

No MFEM abort, MPI abort, assertion failure, crash signature, or empty shard was present in the final 24 logs.

The migration is not yet packaged as a clean commit. The worktree is intentionally dirty relative to HEAD and contains the migration. Preserve it.

3. Git and validation provenance

Important commits:

• 0f3ca8050b78f0c5c1ef98d485b9fbc2ba06183c — the user-created mid-migration checkpoint.
• 177ae8b38aa34d88c07203376961145c5403ccaa (HEAD when this document was written) — feat(support): continued migration to new field dof support system.

The final migration, hardening, deletions, and mapper consolidation are uncommitted changes on top of 177ae8b.

Final validation logs are under:

${TMPDIR:-/tmp}/meanfield-migration-validation

On this machine that resolved to:

/var/folders/v3/xfq17d692dg_gzdwv4yzmrkw0000gp/T/meanfield-migration-validation

The authoritative files are:

gravity-final-shard-{0..7}.txt
barotrope-final-shard-{0..7}.txt
migration-remainder-shard-{0..7}.txt

Do not confuse these with older repository-root gravity-shard-*.txt files from August 24. Those older logs captured intermediate failures and were not evidence for the final state.

Post-validation rename caveat

After the 24-shard validation completed, the user renamed:

• DomainMapperStateless to DomainMapper;
• DomainMapperStatelessOptions to DomainMapperOptions.

The public interface and most implementation state were compiled after the rename, but at handoff time libmeanfield/impl/mapping/domain_mapper.cpp had a modification timestamp two minutes newer than the latest test executable. The source-level audit found no remaining DomainMapperStateless, DomainMapperStatelessOptions, domain_mapper_new, or PolytropicBarotrope type/file references, and git diff --check passed. Nevertheless, run one compile-only check on the exact final tree before committing.

4. Final field and domain model

4.1 Registered support is the source of truth

Field support is declared in libmeanfield/interface/field/field_registry.cppm:

• Density::Support = DomainSupport<Stellar>;
• Enthalpy::Support = DomainSupport<Stellar>;
• Gravity::Support = DomainSupport<All>;
• Displacement::Support = DomainSupport<All>;
• BarotropicConstant::Support = NonSpatialSupport.

This distinction is now operational. Density and enthalpy solver vectors omit vacuum-only true DOFs. Gravity and displacement currently map over all domains and ordinarily produce identity maps. Global scalars are not routed through spatial FieldDofMap factories.

The concrete production schema is CoreEnvelopeVacuumDomainSchema. Field support is resolved through the schema's material bindings rather than through hard-coded assumptions such as “stellar attributes are 1 and 2” or “vacuum is attribute 3.” Tests deliberately exercise alternate material bindings to protect this property.

4.2 Supported continuous DOFs use union semantics

The active set is the union of DOFs touched by supported elements. It is not computed by removing every DOF that also touches an unsupported element.

This matters for continuous H1 fields. An enthalpy DOF shared by a stellar element and a vacuum element remains part of the stellar field because it participates in supported stellar elements and supplies the physical interface trace.

4.3 Parallel support resolution

resolve_field_dof_support performs the production parallel resolution:

1. Build a local vector-DOF marker from supported elements.
2. Decode MFEM's signed/oriented DOF indices.
3. Synchronize the marker across shared parallel DOFs.
4. Convert locally owned active vector DOFs to true DOFs.
5. Produce active/inactive markers and sorted index lists.

The synchronization step ensures that a shared interface DOF is active on every rank carrying it when any neighboring supported element activates it.

The implementation deliberately rejects nonconforming ParFiniteElementSpace objects. Supporting them correctly would require an explicit constraint/conforming-DOF projection; silently treating local DOFs as ordinary true DOFs was rejected as unsafe.

5. FieldDofMap contract

FieldDofMap is the canonical immutable indexing relation between a full MFEM true vector and a dense reduced field vector.

It stores:

• full_size() — MFEM true-vector size;
• reduced_size() — supported field-vector size;
• reduced_to_true() — canonical forward indices;
• true_to_reduced() — inverse indices with -1 for inactive true DOFs.

Construction requires the reduced-to-true list to be sorted, unique, in range, and strictly increasing. This makes reduced ordering deterministic and consistent with MFEM marker-to-list ordering.

Important operations and semantics:

• gather(full) selects supported true DOFs into reduced order.
• scatter(reduced) creates a complete full true vector and zeros unsupported entries.
• scatter_into(reduced, full) overwrites supported entries while preserving existing unsupported entries.
• scatter_add(reduced, full, scale) accumulates into supported entries only.
• All overloads validate exact source and destination sizes.
• Destination-vector overloads do not resize their output, so views into mfem::BlockVector remain valid.
• is_identity() is exact because canonical ordering guarantees reduced_to_true[i] == i whenever all true DOFs are active.

The object intentionally contains no mesh, field, domain, or solver policy after construction. Those semantics belong to the factory step; consumers get a small indexing object.

6. Canonical FieldDofMap/GridFunction adapter

FieldDofGridFunctionAdapter was added as the only canonical bridge between reduced field vectors and MFEM GridFunction objects.

It binds a FieldDofMap to the exact FiniteElementSpace from which the map was constructed. Matching vector sizes are not enough: a grid function using another FE-space object is rejected. This prevents silent cross-space corruption when two spaces happen to have equal true-vector sizes.

Operations:

• gather(GridFunction) calls GetTrueDofs and gathers into reduced coordinates.
• scatter(reduced, GridFunction) applies projection semantics and calls SetFromTrueDofs.
• scatter_into(reduced, GridFunction) preserves unsupported existing true-DOF values.
• make_field_dof_grid_function_adapter<Field, Schema>(space) is the typed construction path.

The gravity initialization/solve now uses these adapters for density, displacement, gravity flux, and gravity potential. The returned GravitySolution remains an MFEM-facing pair of ParGridFunctions, while the solve itself is performed in reduced coordinates.

7. Domain mapping replacement and hardening

7.1 Architectural replacement

The old stateful mapping implementation was removed. The canonical DomainMapper now owns only mapping policy:

• dimension;
• vacuum material attribute;
• the exterior compactification map.

Element field values are supplied explicitly through ElementMappingData, and temporary storage is supplied through DomainMapper::Workspace. Point, volume, face, and first-variation evaluation are explicit operations. This eliminates hidden displacement state and cross-state cache contamination inside the mapper itself.

The implementation formerly stored in libmeanfield/impl/mapping/domain_mapper_new.cpp is now consolidated into:

libmeanfield/impl/mapping/domain_mapper.cpp

domain_mapper_new.cpp was deleted and removed from CMake.

7.2 GridFunctionMappingEvaluator

GridFunctionMappingEvaluator is the stateful convenience layer for consumers that naturally own MFEM grid functions. It references displacement and compactification grid functions, extracts element-local data, and delegates actual geometry evaluation to DomainMapper.

Hardening added in the final stage:

• constructor validation for attached spaces;
• same-mesh validation;
• displacement vector-dimension validation;
• scalar compactification validation;
• mesh/domain-mapper dimension validation;
• exact FE-space binding checks after construction;
• detection of FE-space sequence changes;
• explicit, idempotent InvalidateCache();
• eager Refresh() of a currently cached element;
• a validated no-op when Refresh() is called before any element is cached;
• lazy reload after space-sequence invalidation;
• correct application of MFEM DofTransformation::InvTransformPrimal to element displacement DOFs;
• support for both byNODES and byVDIM displacement ordering.

The evaluator owns mutable workspace/cache state and is explicitly not thread-safe. Consumers must call InvalidateCache() or Refresh() after mutating grid-function values. FE-space sequence changes are detected automatically, but arbitrary value changes are not observable without explicit invalidation.

Integrator consumers were updated to invalidate mapping caches at assembly entry. Analysis, physics, coefficient, utility, and integrator consumers now use GridFunctionMappingEvaluator rather than the removed stateful mapper API.

8. Gravity-system migration

The gravity migration covered kernels, prepared operators, contexts, coupled operators, the reduced solve, tests, and downstream consumers.

8.1 Boundary decision: kernels remain full-true-DOF code

Low-level element kernels deliberately continue to accept full MFEM true vectors. Their interface comments now state this boundary explicitly. Examples include:

• mapped H(div) mass and its geometry variation;
• mapped density source and its geometry variation;
• gravity displacement force residual and all Jacobian actions;
• hydrostatic and pressure/rotation kernels used by prepared consumers.

This was a deliberate choice. Element assembly naturally indexes MFEM spaces. Making kernels understand reduced field support would duplicate mapping logic and mix algebra with support policy.

Prepared operators and contexts own the maps and perform the only scatter/gather operations around those kernels.

8.2 Prepared gravity components

PreparedMappedGravitySourceOperator now owns density, potential, and displacement maps. Its public Mult and MultTranspose use reduced density/potential coordinates; it expands to full true vectors internally.

PreparedMappedHDivMassOperator owns flux and displacement maps. It accepts reduced flux/displacement vectors, maintains full internal work vectors, and exposes both:

• a reduced diagonal for the reduced mass block;
• a full true diagonal for the Schur construction.

Both stellar and compactified-vacuum H(div) contributions are included. Positivity is tested across the complete gravity domain.

8.3 Gravity contexts and revision routing

GravityFieldGeometryContext owns the displacement map and the geometry-dependent prepared mass/source/divergence operators.

GravityFieldLinearizationContext owns density, displacement, gravity-gradient, and gravity-potential maps, together with the full true work vectors required by kernels.

Typed revision stamps control selective invalidation for:

• discretization;
• displacement;
• density;
• gravity gradient;
• gravity potential.

Tests verify first preparation, no-op repeat preparation, selective rebuild/update behavior, and independent contexts without shared cache contamination.

8.4 Coupled gravity operator and Jacobian

GravityFieldOperator, GravityFieldJacobianOperator, and their state/residual offsets now use reduced_size() for every registered field block.

The coupled gravity state is:

[ density(reduced), displacement(reduced), gravity-gradient(reduced), gravity-potential(reduced) ]

The gravity residual is:

[ gradient-equation(reduced flux), Poisson-equation(reduced potential) ]

MFEM divergence operators remain full-space objects. Their operands/actions are explicitly scattered and gathered at the operator boundary.

8.5 Reduced solve and preconditioner

solve_gravity_field now:

1. validates that the supplied grid functions use the expected FEM spaces;
2. creates typed field/grid-function adapters;
3. gathers density and displacement;
4. builds the reduced block layout;
5. prepares the reduced gravity operator and geometry context;
6. solves the reduced mixed system with MINRES;
7. scatters the reduced flux and potential solution back to ParGridFunctions.

The reduced block preconditioner uses:

• OperatorJacobiSmoother on the reduced positive H(div) mass diagonal;
• a full-space approximate Schur complement B diag(M)^{-1} B^T;
• HypreBoomerAMG for the potential Schur block;
• explicit potential scatter/gather around the full-space AMG object.

This split is important. The first attempted reduced preconditioner encountered zero diagonal entries because inactive/full-space structure was being handed to a smoother that requires a positive diagonal. A later attempt produced immediate MINRES non-convergence. The final implementation makes the coordinate space of each preconditioner component explicit and is protected by the positive-diagonal and deformed-MINRES tests.

9. Barotrope, pressure, hydrostatic, displacement, and root migration

The FieldDof conversion was propagated beyond gravity so the coupled root operator has one consistent coordinate convention.

Map-aware contexts now include:

• BarotropicClosureLinearizationContext;
• PressureForceLinearizationContext;
• HydrostaticEquilibriumContext;
• RotationalDisplacementForceLinearizationContext;
• gravity geometry and linearization contexts.

Map-aware prepared operators include:

• barotropic closure;
• pressure force;
• hydrostatic equilibrium;
• gravity displacement force;
• rotation displacement force;
• composed displacement residual;
• mass normalization;
• mapped gravity source;
• mapped H(div) mass;
• complete stellar equilibrium.

Density and enthalpy are reduced at public prepared-operator boundaries. Displacement, gravity flux, and gravity potential also use FieldDofMap, even where their maps are identities today. This avoids hard-coding the current support choice into the coupled algebra.

PreparedStellarEquilibriumOperator constructs every block size from maps. Its root state is:

[ density, displacement, gravity-gradient, gravity-potential, enthalpy, Bernoulli constant ]

Spatial blocks are in registered FieldDof coordinates; the Bernoulli constant remains a one-entry nonspatial block. The operator extracts reduced views, packs the gravity substate, routes the exact child dependencies, and assembles the residual only when a dependency or child preparation changes.

This removed the earlier ambiguous mixture in which some children expected full MFEM true vectors while others expected reduced field vectors.

10. EOS migration

The duplicate physics::PolytropicBarotrope implementation was removed.

The canonical EOS is now:

mean_field::eos::Polytrope

It implements the EquationOfState interface and supplies density/pressure/enthalpy conversions and derivatives, including defined surface behavior at zero enthalpy/density and validation of material parameters and physical inputs.

Deleted legacy module:

libmeanfield/interface/physics/barotrope.cppm

The file was removed from CMake, and no PolytropicBarotrope reference remains in production code, tests, or experiments. Local variables named barotrope may still refer to an eos::Polytrope; those names are descriptive variables, not the deleted class.

11. Dead and legacy code removed

The following dead implementations/modules were deleted:

• libmeanfield/impl/mapping/domain_mapper_new.cpp — merged into canonical domain_mapper.cpp;
• libmeanfield/interface/physics/barotrope.cppm — duplicate EOS implementation;
• libmeanfield/interface/physics/context.cppm — obsolete GravityContext owning the old assembled gravity solve/preconditioner stack.

CMake was updated so only the canonical mapper, EOS, and gravity context/operator stack are compiled.

Legacy-comparison gravity tests tied to the removed assembled implementation were removed when they ceased to compare two supported production paths. Analytic, variational, operator-identity, projection-floor, and self-consistency tests now provide the regression protection instead.

12. File and symbol renaming

Completed:

• canonical implementation file is libmeanfield/impl/mapping/domain_mapper.cpp;
• _new mapper filename removed;
• DomainMapperStateless renamed to DomainMapper;
• DomainMapperStatelessOptions renamed to DomainMapperOptions.

Intentionally not yet completed:

• the fem::FEM member remains named domainMapperStateless;
• many local variables and diagnostics still say “stateless domain mapper”;
• mapper test titles still begin with Stateless Domain Mapper.

These are naming cleanup items, not two competing mapper implementations. The user planned to perform broad identifier renames with IDE tooling. A reasonable final name for the FEM member is domainMapper.

13. Test infrastructure and tag cleanup

Tests use the typed compile-time tag system in tests/test_helpers.cppm:

• make_tag creates a Catch2 tag;
• sub_tag and operator& compose tag sets;
• canonical leaf/composite constants prevent hand-written duplicated tag strings.

New/normalized families include:

• field_dof_unit and field_dof_integration;
• mapping_evaluator_unit;
• gravity unit, integration, operator, prepared, context, kernel, accuracy, analytic, and consistency families;
• barotrope prepared, EOS, context, hydrostatic, pressure/quadrature, and mass-normalization families;
• rotation prepared/context/kernel/analytic families.

Touched gravity and prepared tests were converted to these typed tags. The guiding rule is to compose leaf semantics exactly once so output does not repeat inherited [physics] or [solver] tags.

Tests added during the final uncommitted migration

Genuinely new adapter/evaluator/diagonal/schema coverage includes:

• Domain Schema Builds Exact MFEM Attribute Markers;
• Field DOF Grid Function Adapter Gathers Exactly The Supported True DOFs;
• Field DOF Grid Function Adapter Scatter Projects And Round Trips Reduced Fields;
• Field DOF Grid Function Adapter Scatter Into Preserves Unsupported True DOFs;
• Field DOF Grid Function Adapter Is Exact For Identity Vector Field Maps;
• Field DOF Grid Function Adapter Rejects Incompatible Maps Spaces And Vectors;
• Grid Function Mapping Evaluator Refreshes A Cached Displacement Element;
• Grid Function Mapping Evaluator Invalidates A Cached Compactification Element;
• Grid Function Mapping Evaluator Rejects Invalid Fields And Rebinding;
• Prepared Mapped Hdiv Mass Diagonal Is Positive Across Both Domains.

The gravity analytic tests formerly prefixed New ... were promoted to canonical names after removal of the obsolete comparison path. EOS test titles were similarly changed from Polytropic Barotrope ... to Polytropic EOS ....

Important gravity coverage now includes:

• kernel equality, linearity, vacuum exclusion, and centered geometry variations;
• prepared source/mass identities and preparation reuse;
• full-column and simultaneous Jacobian comparisons;
• context selective invalidation and isolation;
• reduced block dimensions and routing;
• deformed MINRES solve;
• uniform sphere, homogeneous/deformed ellipsoid, Ferrers, monopole, and exterior compactification comparisons;
• virial and volume-preserving-deformation self-consistency;
• separation of physical exterior error from the finite-element projection floor.

Important barotrope/root coverage includes:

• EOS identities, derivatives, surface behavior, and invalid inputs;
• reduced barotropic closure under multiple EOS/geometry conditions;
• pressure residual and Jacobian comparisons;
• hydrostatic algebraic/displacement/complete Jacobians;
• mass-normalization routing and differences;
• displacement residual composition and MFEM routing;
• six-by-six stellar-equilibrium layout;
• child invalidation and complete centered differences;
• analytic N=1 and N=3 stellar states.

14. Migration failures and lessons

Several failures were valuable because they identified mixed-coordinate boundaries.

Density-size abort in mapped-source variation

An early gravity shard aborted because a reduced density vector reached apply_mapped_source_variation, whose element indexing requires a full density true vector.

Resolution: retain the full-vector kernel contract and scatter in the prepared/context layer. Tests were migrated so directions and states use the correct representation at each boundary.

Zero diagonal in OperatorJacobiSmoother

The first reduced gravity preconditioner attempted to construct a Jacobi smoother from a diagonal containing unsupported/full-space zeros.

Resolution: assemble and smooth the reduced positive mass diagonal, while retaining a separately assembled full true diagonal for the full divergence-based Schur approximation.

Immediate MINRES non-convergence

After removing the zero-diagonal abort, an intermediate preconditioner arrangement stopped at iteration zero without convergence.

Resolution: use a consistent block preconditioner—reduced mass Jacobi plus full Schur AMG with explicit potential scatter/gather—and add a deformed reduced-system solve test.

Stale shard files

Repository-root gravity shard files from August 24 showed several old analytic failures and an MPI abort. They were initially easy to mistake for the new run.

Resolution: use labeled logs in the dedicated temporary validation directory and verify modification times before diagnosing failures.

Cached grid-function values

GridFunctionMappingEvaluator caches copies of element-local field DOFs. Mutating a referenced grid function does not automatically change an already cached element.

Resolution: expose/document InvalidateCache() and Refresh(), harden FE-space change/rebinding detection, update integrator entry points, and add direct mutation/cache tests.

15. Known limitations and remaining work

The primary FieldDof migration is complete. Remaining items are cleanup or explicitly bounded future work:

1. Compile the exact post-rename tree. libmeanfield/impl/mapping/domain_mapper.cpp is newer than the latest test executable.
2. Finish cosmetic mapper naming. Rename domainMapperStateless, related local variables/messages, and optionally mapper test titles.
3. Commit the migration. Review the large dirty worktree carefully and preserve unrelated user artifacts. Do not reset it.
4. Nonconforming meshes remain unsupported by FieldDof resolution. Add the correct conforming-constraint projection before lifting the explicit rejection.
5. GridFunctionMappingEvaluator is not thread-safe. Use one evaluator/workspace per thread if threaded assembly is introduced.
6. Value mutation requires cache invalidation. Keep the assembly-entry invalidations when refactoring integrators.
7. Quadrature Query.domain remains legacy metadata. Physical element selection in migrated prepared closure/pressure paths already comes from field support plus DomainSchema; the metadata can be migrated independently with the quadrature subsystem.
8. The legacy GravityMomentumIntegrator exact geometry-Jacobian mode still aborts intentionally. Its message states that exact geometry variation has not been wired into that legacy integrator. The prepared gravity-displacement path does have the tested exact variation. Do not mistake the old integrator limitation for an incomplete FieldDof conversion.
9. No monolithic one-process full-suite run was performed. The three filters cover every registered test case, but a full unsharded invocation would additionally probe process-global/order interaction. Given the suite's runtime, the user accepted complete partitioned coverage as sufficient.

16. Recommended immediate continuation

The next agent should:

1. Read this document and the earlier Rotating Polytrope Review summary.

2. Inspect git status --short; assume all existing changes belong to the user/migration.

3. Run a compile-only check of cmake-build-debug-homebrew after the final mapper rename.

4. Search again for:

   DomainMapperStateless
   DomainMapperStatelessOptions
   domain_mapper_new
   PolytropicBarotrope
   

   The first item will still appear as the lower-camel FEM member domainMapperStateless; the old PascalCase types and _new file should not appear.

5. If the user wants the cosmetic rename, use IDE-aware rename for the FEM member and related identifiers, then compile again. The change is broad but mechanical.

6. Review git diff --check, the three intended deletions, and CMake source lists.

7. Create a new checkpoint/final migration commit only after the user confirms the desired naming cleanup.

If a final regression rerun is desired after cosmetic renaming, the most efficient confidence gate is:

./cmake-build-debug-homebrew/tests \
  "Field DOF Grid Function Adapter*,Grid Function Mapping Evaluator*,Prepared Mapped Hdiv Mass Diagonal Is Positive Across Both Domains,Reduced Gravity Field Operator Solves Deformed Gravity System With MINRES,Prepared Stellar Equilibrium*"

The already completed 24-shard run remains the substantive migration validation.

17. Invariants future changes must preserve

• Solver/prepared public spatial states are in registered FieldDof coordinates.
• Low-level MFEM element kernels receive full true vectors.
• Crossing that boundary must be visible as a map or adapter operation.
• Block sizes come from FieldDofMap::reduced_size(), never raw GetTrueVSize() unless the object is explicitly full-space.
• Field support comes from the field registry plus a DomainSchema, not numeric attribute conventions.
• Shared continuous interface DOFs use union-of-supported-elements semantics.
• scatter means projection/zero outside support; scatter_into means preserve outside support.
• Grid-function adapters validate FE-space identity, not merely equal sizes.
• Mapping evaluator caches must be invalidated after value mutation.
• Prepared contexts own their linearization maps, revisions, and full-vector workspaces.
• Test reference kernels must be passed full true vectors; solver/operator calls must be passed reduced vectors.

These invariants are the durable result of the migration and should be treated as the design contract going forward.