699 lines
38 KiB
Markdown
699 lines
38 KiB
Markdown
# Final optimization batch: pressure, gravity source, preparation, and fixed divergence
|
||
|
||
Build the targets explicitly; sandbox is excluded from the default build:
|
||
|
||
```sh
|
||
cmake --build cmake-build-release-homebrew \
|
||
--target system_isolation_profile_harness sandbox tests --parallel
|
||
```
|
||
|
||
Run from the repository root with a new output directory:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system final-optimizations --check full --implementation both \
|
||
--samples 5 --iterations 10 --warmup 2 \
|
||
--prepare-samples 3 --prepare-iterations 1 \
|
||
--output final_optimizations_01
|
||
```
|
||
|
||
This group measures:
|
||
|
||
- `pressure-displacement`: tensor evaluation of displacement gradients and
|
||
tensor gradient transpose, compared with the previous optimized pointwise
|
||
application (`cached_pointwise_reference`). The general prepared action
|
||
and stateless kernel remain correctness references.
|
||
- `pressure-preparation`: batched displacement and enthalpy evaluation and
|
||
tensor assembly of the enthalpy derivative matrix, compared with the
|
||
previous blocked matrix assembly and pointwise displacement evaluation.
|
||
- `gravity-source-displacement`: tensor density/displacement evaluation and
|
||
potential-test transpose, compared with the previous pointwise application.
|
||
- `hydrostatic-preparation`: batched displacement and scalar evaluation and
|
||
tensor assembly of the algebraic matrices, compared with the previous
|
||
pointwise/dense preparation.
|
||
- `gravity-schur-preparation`: reuse of the fixed divergence matrix during
|
||
Schur assembly, compared with assembling divergence on every call. This
|
||
timing excludes mass-diagonal preparation and AMG setup, which are unchanged.
|
||
|
||
Preparation timings change state/revisions before each measured call and
|
||
verify that pressure/hydrostatic preparation actually rebuilds. Schur timings
|
||
cycle through mass diagonals prepared from three displacement states.
|
||
Cold hydrostatic preparation and cold divergence-cache assembly are reported
|
||
separately. These medians measure the incremental benefit of this batch;
|
||
they do not reuse the slower baselines from earlier batches.
|
||
|
||
Full harness checks compare prepared and stateless derivatives, residuals,
|
||
algebraic actions, enthalpy diagonals, multiple states and directions, zero
|
||
source directions, and finite differences. Hydrostatic checks also reject
|
||
folded geometry and recover afterward. Schur checks compare cached and
|
||
fresh assembly, change and restore the mass diagonal, check exact inverse
|
||
scaling, clear the cache, and invalidate it with `Mesh::NodesUpdated()`.
|
||
New unit coverage checks the scalar gradient transpose against dense
|
||
reference gradients and adjoint products for multiple H1/L2 orders/bases,
|
||
preparation-mode switches, rejection/recovery, and production gravity-block
|
||
cache reuse/invalidation.
|
||
|
||
The divergence cache belongs to each prepared gravity block. It retains one
|
||
assembled sparse matrix for the block's lifetime. Every refresh still forms
|
||
a fresh weighted transpose and Schur matrix using the current mass diagonal.
|
||
A discretization revision clears the cache; mesh, node, FE-space sequences,
|
||
and quadrature-policy identity are also checked collectively before reuse.
|
||
Raw computational-coordinate edits must call `Mesh::NodesUpdated()`.
|
||
A standalone cache must be cleared after changing a quadrature policy in
|
||
place. Physical displacement changes do not invalidate fixed divergence.
|
||
|
||
The tensor paths retain the existing mapper validation, quadrature, DOF
|
||
transformations, and fallbacks for unsupported elements/rules. Metadata
|
||
reports `pressure_tensor_elements`, `source_tensor_variation_elements`, and
|
||
`hydro_preparation_tensor_elements`, plus divergence assembly counts. A zero
|
||
tensor count indicates fallback, not measured tensor acceleration.
|
||
|
||
The new individual selectors are `gravity-source-displacement`,
|
||
`hydrostatic-preparation`, and `gravity-schur-preparation`.
|
||
`pressure-displacement` includes its preparation benchmark. This batch is
|
||
selected together with `final-optimizations`.
|
||
|
||
Return `summary.txt`, `metadata.txt`, `timings.csv`, and `checks.csv`.
|
||
After the checks pass, run the rebuilt sandbox with the same preflight
|
||
setting as the previous run, and capture its Newton timings and profiles.
|
||
Compare the residual, accepted step, trial count, determinant minimum, Krylov
|
||
iterations, and true linear residual along with the wall times. The previous
|
||
reference was 24.174 s for Newton 0, 8.00964 s trial preparation, 2.14648 s
|
||
accepted refresh, and 13.6571 s linear solve (26 J and 24 preconditioner calls).
|
||
|
||
These source and test changes have not been compiled or executed by the agent.
|
||
|
||
---
|
||
|
||
# Application optimization batch: closure, rotation, primal mass, and trial preparation
|
||
|
||
Build both `system_isolation_profile_harness` and `sandbox` explicitly in
|
||
`cmake-build-release-homebrew`; sandbox is excluded from the default build.
|
||
|
||
Run from the repository root:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system application-optimizations --check full --implementation both \
|
||
--samples 5 --iterations 10 --warmup 2 \
|
||
--prepare-samples 3 --prepare-iterations 1 \
|
||
--output application_optimizations_01
|
||
```
|
||
|
||
The group measures these changes against the implementations immediately before
|
||
this batch:
|
||
|
||
- `barotropic-closure`, `barotropic-enthalpy`, and `barotropic-displacement`:
|
||
tensor interpolation and transpose versus the cached pointwise path. The
|
||
dedicated enthalpy and displacement baselines retain their dedicated entry
|
||
points, so these ratios measure tensor acceleration without counting the
|
||
earlier removal of zero-block work again.
|
||
- `rotation-force`: tensor density/displacement interpolation and force
|
||
transpose versus the cached pointwise complete action.
|
||
- `hdiv-mass`: the new CPU application versus retained MFEM partial assembly
|
||
(`mfem_pa_reference`), with identical prepared coefficients, quadrature,
|
||
restrictions, orientation handling, and true-DOF transfers. The new integrator
|
||
contracts only its marked domain, uses scratch sized to the actual order,
|
||
and specializes the current RT2/nine-point-per-axis case. Other supported
|
||
CPU orders use the same algorithm with runtime dimensions. Other backends
|
||
retain MFEM's application. Mass-only runs use primal preparation.
|
||
- `geometry-inspection`: the production trial-geometry validator with cached
|
||
computational-mesh inverse Jacobians and tensor gradients, versus the original
|
||
`GridFunction::GetVectorGradient` loop. Every volume element, inspection
|
||
point, and determinant check is retained. The first cold inspection is
|
||
reported separately; paired medians measure reuse with changing displacement.
|
||
- `gravity-source-preparation`: reuses the mesh weight already computed during
|
||
mapping evaluation, versus recomputing `ElementTransformation::Weight()`.
|
||
Both paths rebuild on every measured call; the harness checks preparation
|
||
counters and compares forward, transpose, and displacement actions.
|
||
|
||
The scalar tensor plans now include nodal H1 and L2 hexes. Basis reconstruction,
|
||
native DOF permutations, and Cartesian quadrature ordering are verified before
|
||
a plan is enabled. Unsupported elements/rules retain the dense implementation.
|
||
Metadata reports `closure_tensor_elements`, `rotation_tensor_elements`,
|
||
`fast_mass_elements`, and `geometry_inspection_tensor_elements` so a fallback
|
||
cannot be mistaken for acceleration.
|
||
|
||
Full checks include general prepared and independent stateless comparisons,
|
||
existing finite differences, mass symmetry/linearity/positive energy/zero input,
|
||
changing geometries, fold rejection, and non-finite geometry rejection. The
|
||
unit-test additions also cover both global vector orderings, curved-mesh cache
|
||
invalidation, explicit cache clearing, tetrahedron fallback, and L2 basis
|
||
variants. These tests are supplied for the user to compile and run.
|
||
|
||
The inspector stores nine doubles (72 bytes) per eligible inspection point,
|
||
plus shared reference tables and reusable element scratch. It refreshes fixed
|
||
geometry when the mesh, nodes, or finite-element-space sequence changes.
|
||
Raw coordinate edits must call `Mesh::NodesUpdated()`, as with MFEM's geometry
|
||
cache, or clear a standalone inspector with `ClearCache()`. Displacement is
|
||
read anew for every inspection. Its mutable scratch is intended for sequential
|
||
calls, consistent with the prepared operators.
|
||
|
||
Use a new output directory and return `summary.txt`, `metadata.txt`,
|
||
`timings.csv`, and `checks.csv`. Application timings use `--samples` and
|
||
`--iterations`; inspection/source preparation timings use `--prepare-samples`
|
||
and `--prepare-iterations`. This group omits unrelated force-preparation
|
||
benchmarks. Individual systems remain selectable. After checks pass, rebuild
|
||
sandbox too and compare its printed Newton timings and convergence against the
|
||
previous run.
|
||
|
||
---
|
||
|
||
# Runtime optimization batch: tensor applications, geometry cache, normalization
|
||
|
||
Build both `system_isolation_profile_harness` and `sandbox` explicitly in
|
||
`cmake-build-release-homebrew`; sandbox is excluded from the default build.
|
||
|
||
Run from the repository root:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system runtime-optimizations --check full --implementation both \
|
||
--samples 5 --iterations 10 --warmup 2 \
|
||
--prepare-samples 3 --prepare-iterations 1 \
|
||
--output runtime_optimizations_01
|
||
```
|
||
|
||
This group checks and measures:
|
||
|
||
- `normalization-diagonals`: H(div), potential, density, enthalpy, and surface
|
||
reference-mass diagonals, comparing volume partial assembly and direct surface
|
||
diagonal integration to full true-DOF matrix assembly. Both diagonal norms
|
||
are recorded, and the surface diagonal must have positive entries. These use
|
||
the preparation sample/iteration counts.
|
||
- `geometry-preparation`: complete H(div) preparation with the fixed-mesh
|
||
geometry cache enabled versus disabled, changing displacement between paired
|
||
samples. Both use the same optimized operator. Mass and displacement actions
|
||
are checked outside the timer. The initial cold-cache preparation is reported
|
||
separately; median timings measure reuse. Cache hit/miss/point counts are saved.
|
||
- `hdiv-displacement`: tensor interpolation/transpose versus the **previous
|
||
optimized pointwise** implementation (`cached_pointwise_reference`).
|
||
- `gravity-force`: tensor application versus the **previous optimized pointwise**
|
||
application. General prepared paths and finite differences remain independent
|
||
checks. The older gravity preparation comparison is omitted from this group.
|
||
|
||
The metadata includes `tensor_variation_elements` and
|
||
`gravity_tensor_elements`. The current hex mesh should use tensor paths on all
|
||
eligible elements. General geometries and quadrature retain the existing paths;
|
||
these counts distinguish real acceleration from a correct fallback.
|
||
|
||
Individual systems remain selectable, including the two new setup systems.
|
||
Use a new output directory for every run. Upload `summary.txt`, `metadata.txt`,
|
||
`timings.csv`, and `checks.csv`. Preparation timings use `--prepare-samples`
|
||
and `--prepare-iterations`; application timings use `--samples` and
|
||
`--iterations`.
|
||
|
||
## Preflight option
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/sandbox --skip-preflight
|
||
```
|
||
|
||
Library option: `NewtonOptions.backtracking.geometryPreflight = false`.
|
||
The default remains `true`; sandbox also accepts `--preflight`. Skipping avoids
|
||
both the predictive geometry scan and its lazy rule construction. Trial mapping
|
||
validation, finite-value checks, backtracking, and accepted-state restoration
|
||
remain active. Skipped scans have zero reported time and an empty optional
|
||
geometry estimate; `geometryLimitedIterations` counts only predictive limits.
|
||
A skipped scan can trade its time for extra rejected trials on a large step.
|
||
|
||
## Geometry and normalization contracts
|
||
|
||
The mapper caches only fixed computational-mesh positions, Jacobians, inverse
|
||
Jacobians, and weights. It recomputes displacement and compactification mapping
|
||
data on each call. Cache keys use element and actual reference coordinates, so
|
||
copied or changed quadrature rules cannot reuse the wrong point; quadrature
|
||
weights are applied by each caller. Mesh and node sequence changes invalidate
|
||
the entries. After raw coordinate changes, call `Mesh::NodesUpdated()`, as for
|
||
MFEM's geometric-factor cache, or `DomainMapper::ClearGeometryCache()`. Clear
|
||
the cache before destroying a mesh that the mapper will outlive. Cache entries
|
||
consume memory proportional to the distinct element/quadrature points visited,
|
||
and the mapper cache, like the element transformations it uses, is not thread-safe.
|
||
|
||
Normalization retains full matrix assembly for nonconforming, variable-order,
|
||
and unsupported element spaces. In particular, it does not substitute the
|
||
approximate partial-assembly diagonal for `diag(P^T A P)` on hanging-node meshes.
|
||
Supported conforming hex volume spaces use MFEM partial assembly followed by
|
||
exact true-DOF diagonal assembly. The surface path integrates only diagonal
|
||
entries over marked boundary elements, using the same quadrature and physical
|
||
shape functions as full assembly. This includes the internal star/vacuum
|
||
interface, which MFEM's exterior boundary partial-assembly restriction omits.
|
||
The direct surface path supports conforming scalar H1 spaces without general
|
||
DOF transformations; other spaces retain full assembly.
|
||
|
||
The new tests in existing test sources cover scalar/RT tensor transposes and
|
||
adjoint products, integrated RT bases, retained-table lifetime, general fallback,
|
||
geometry cache invalidation, changed displacement/compactification, exact
|
||
normalization diagonals, nonconforming fallback, and Newton acceptance/fold
|
||
rejection with preflight skipped. Compilation and execution are left to the user.
|
||
|
||
---
|
||
|
||
# System isolation profile harness
|
||
|
||
This executable checks and times individual operators without constructing a
|
||
stellar model, preconditioner, or Newton solver. Available systems are:
|
||
|
||
- `hdiv-displacement`: optimized versus retained prepared H(div) derivative.
|
||
- `hdiv-mass`: new CPU primal application versus retained MFEM partial assembly.
|
||
- `mass-normalization`: cached scalar mass row versus retained quadrature action.
|
||
- `angular-momentum`: cached scalar angular-momentum row versus retained quadrature action.
|
||
- `hydrostatic-displacement`: tensor evaluation versus the previous cached pointwise derivative; checks also compare the general mapping variation.
|
||
- `stellar-derivatives`: mass normalization, angular momentum, and hydrostatic displacement.
|
||
- `barotropic-closure`: complete density/enthalpy/displacement closure action.
|
||
- `barotropic-enthalpy`: tensor versus cached dense dedicated enthalpy coupling.
|
||
- `barotropic-displacement`: tensor versus cached pointwise dedicated displacement coupling.
|
||
- `pressure-displacement`: direct pressure displacement contraction versus the old matrix products.
|
||
- `pressure-enthalpy`: local-matrix application after blocked versus original matrix preparation.
|
||
- `gravity-force`: complete density/displacement/gravity-gradient force derivative.
|
||
- `rotation-force`: complete density/displacement force derivative at fixed rotation.
|
||
- `linear-derivatives`: the seven closure/force comparisons above, plus pressure/gravity/rotation preparation timings.
|
||
- `preparation-optimizations`: pressure displacement and enthalpy, gravity force, rotation force, their preparation comparisons, then hydrostatic displacement.
|
||
- `geometry-inspection`: cached fixed geometry and tensor gradients versus the original geometry scan.
|
||
- `gravity-source-preparation`: cached mesh weights versus repeated weight evaluation.
|
||
- `application-optimizations`: the closure, rotation, primal mass, inspection, and source comparisons described above.
|
||
- `all`: all application and preparation comparisons, including geometry inspection and normalization diagonals.
|
||
A failure stops the affected group before its benchmarks.
|
||
|
||
The default remains `hdiv-displacement`. Scalar systems prepare a shared gravity
|
||
context to supply their frozen state, but do not solve for gravity. Hydrostatic
|
||
only runs do not construct that shared context.
|
||
|
||
## Preparation and hydrostatic optimization comparison
|
||
|
||
The current changes fuse gravity/rotation residual assembly with element-cache
|
||
preparation, block pressure enthalpy-matrix assembly, and use tensor contractions
|
||
for supported hydrostatic displacement elements. From the repository root,
|
||
after the user builds the release targets:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system preparation-optimizations --check full --implementation both \
|
||
--samples 5 --iterations 10 --warmup 2 \
|
||
--prepare-samples 3 --prepare-iterations 1 \
|
||
--output preparation_optimizations_01
|
||
```
|
||
|
||
Return the four output files. The new preparation medians are
|
||
`pressure-preparation.*`, `gravity-force-preparation.*`, and
|
||
`rotation-force-preparation.*`. Every timed pair gets fresh dependency revisions
|
||
with the same frozen physical state. Order alternates between samples.
|
||
Preparation counters must increase once per call; an accidental no-op is an error.
|
||
Shared gravity-context preparation and residual copying/norms are outside these
|
||
timers. These are warm re-preparation timings; they include geometry rebuilding
|
||
and do not measure first-construction cost or a complete Newton iteration.
|
||
Initial and unchanged-state preparation remain separately reported.
|
||
|
||
Preparation uses its own sample/iteration counts to avoid accidentally repeating
|
||
multi-second work with the application iteration count. Set `--prepare-samples 0`
|
||
to skip preparation timings. Both implementations remain available through
|
||
`--implementation`; correctness checks can still prepare references in an
|
||
optimized-only timing run. Independent reference operators require additional
|
||
memory in the harness, not in the production solver.
|
||
|
||
Checks compare prepared residuals and pressure enthalpy actions against the
|
||
original preparation at five states, as well as the existing derivative and
|
||
finite-difference checks. Hydrostatic timings use
|
||
`cached_pointwise_reference`, isolating this change from the earlier fast-path
|
||
optimization. Metadata reports `hydrostatic-displacement.tensor_elements`.
|
||
On the standard mesh this should equal the stellar element count.
|
||
|
||
Focused regression cases:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/tests '*Tensor reference*,*Pressure*Force*,*Gravity Displacement Force*,*Rotational Displacement Force*,*Prepared Hydrostatic*'
|
||
```
|
||
|
||
For a nonlinear EOS check, repeat with `--polytropic-index 3` and a new output
|
||
directory. No stellar seed or Newton convergence is required.
|
||
|
||
## First comparison of closure and force derivatives
|
||
|
||
Explicitly build the harness target: it is excluded from the default build.
|
||
From `/Users/tboudreaux/Programming/4DSTAR/MeanField`, use:
|
||
|
||
```sh
|
||
cmake --build cmake-build-release-homebrew \
|
||
--target system_isolation_profile_harness tests sandbox --parallel 2
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system linear-derivatives --check full --implementation both \
|
||
--samples 5 --iterations 10 --warmup 2 \
|
||
--output linear_derivatives_01
|
||
```
|
||
|
||
Return `summary.txt`, `checks.csv`, `metadata.txt`, and `timings.csv`.
|
||
Use a new output directory for every run. Individual system names above let you
|
||
isolate a failure or time one operator without constructing the others.
|
||
The group defaults to a polytrope with n=1 and K=1. To exercise a nonlinear EOS
|
||
without creating a stellar seed, add `--polytropic-index 3`; this option applies
|
||
only to the closure/force group and does not change the FEM mesh.
|
||
|
||
The complete closure comparison measures the contraction change. The two
|
||
coupling comparisons include savings from omitting absent columns, exactly as
|
||
the preconditioner now does. Their reference still executes the old complete
|
||
action with explicit zeros. Pressure measures both displacement and enthalpy derivatives.
|
||
Gravity and rotation measure their combined derivatives; their individual
|
||
stateless column methods remain available as independent checks.
|
||
|
||
Full validation covers undeformed and deformed geometry, a density/enthalpy-only
|
||
update, a gravity/rotation update, and a rotation-only update to zero angular
|
||
velocity. Nonzero rotation uses a tilted axis and an offset center. Each state
|
||
compares every requested direction and an exact zero direction. Column checks
|
||
compare old/new complete actions; gravity and rotation also compare their
|
||
independent stateless columns. Prepared gravity/rotation residuals are also
|
||
compared with independently assembled stateless residuals at every validation
|
||
state. Centered differences of primal residuals use two step sizes. Grouped runs check the complete closure FD;
|
||
individual coupling runs check that coupling's FD. Unchanged dependency
|
||
revisions must preserve preparation counts.
|
||
|
||
The scaled comparison uses the same unit-floor metric as the stellar group.
|
||
An additional relative comparison is reported for non-negligible outputs,
|
||
with tolerance 2e-10 for references and 2e-6 for finite differences. Full
|
||
validation may take several minutes because it prepares many states; its time
|
||
is reported separately and excluded from application samples.
|
||
`--check reference` checks one state plus zero input and unchanged revisions;
|
||
`--check none` skips correctness checks.
|
||
|
||
Initial, repeated, and unchanged preparation times are separate. The gravity
|
||
context is counted once, separately from gravity force preparation.
|
||
Application comparisons retain the earlier reference derivatives. The pressure
|
||
enthalpy comparison uses independently prepared local matrices, and the separate
|
||
preparation benchmarks compare original versus optimized preparation.
|
||
Gravity/rotation metadata reports fast-element counts; on the standard
|
||
3D mesh these should equal the stellar element count. Their fast paths require
|
||
VALUE/GRAD displacement with byNODES global ordering; gravity also requires an
|
||
H(div) reference table. Other configurations retain the previous implementation.
|
||
Closure and pressure use their direct contractions in 3D and retain their
|
||
previous general-dimensional calculations otherwise.
|
||
|
||
Focused regression cases, including material/surface coupling tests:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/tests '*Barotropic Closure*,*Pressure*Force*,*Gravity Displacement Force*,*Rotational Displacement Force*,*Material*Surface*'
|
||
```
|
||
|
||
## First comparison of the scalar rows and hydrostatic derivative
|
||
|
||
After compiling the updated release executable, run from the repository root:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system stellar-derivatives --check full --implementation both \
|
||
--samples 5 --iterations 20 --warmup 2 \
|
||
--output stellar_derivatives_01
|
||
```
|
||
|
||
Return `summary.txt`, `checks.csv`, `metadata.txt`, and `timings.csv`. To isolate
|
||
one system, replace `stellar-derivatives` with `mass-normalization`,
|
||
`angular-momentum`, or `hydrostatic-displacement`. The scalar benchmarks apply
|
||
the complete row (density/displacement and, for angular momentum, Omega).
|
||
The hydrostatic benchmark applies only its displacement derivative.
|
||
|
||
Full checks compare four states: undeformed, deformed, changed density, and
|
||
changed rotation including negative Omega. They use several directions, with
|
||
an element-scale perturbation in the second displacement direction. The
|
||
rotation axis is tilted and its center is displaced from the origin. Checks
|
||
cover separate scalar columns and zero inputs, the mass transpose identity,
|
||
lazy row construction, reuse on rotation/target-only updates, and invalidation
|
||
on geometry/density updates. Two centered differences perturb density,
|
||
displacement and Omega together for the scalar rows; the hydrostatic difference
|
||
holds the other fields and rotation fixed. Finite differences use the existing
|
||
primal residual preparation, which has not been replaced by the new row code.
|
||
|
||
For these three systems, errors are
|
||
`norm(actual-reference) / max(1, norm(actual), norm(reference))`; the unit floor
|
||
also makes zero scalar columns testable. Reference tolerance is 2e-11 and finite
|
||
difference tolerance is 2e-6. Lifecycle checks use exact counts. `--check reference`
|
||
checks one state and all requested directions; `--check none` performs no
|
||
correctness checks. These runs do not establish stellar accuracy or MPI correctness.
|
||
|
||
Preparation is reported separately for the shared gravity context and each
|
||
selected operator, on initial construction and on re-preparation. Scalar rows
|
||
are built lazily in production on first Jacobian use. The harness explicitly
|
||
times `PrepareJacobianRows()` before warmup, including in reference-only runs,
|
||
so that repeated-application medians do not hide that cost in one sample.
|
||
`explicit_row_prepare_seconds` records this first-use cost. When both paths are
|
||
timed, the summary also reports the break-even application count and the
|
||
reference/optimized ratio for 1, 10 and 25 calls including one row assembly.
|
||
A break-even value of -1 means no measured per-application saving. These ratios
|
||
exclude the common state/context preparation costs, which are reported separately.
|
||
Hydrostatic comparison uses the same augmented geometry cache for both paths;
|
||
it does not independently measure the previous preparation time.
|
||
|
||
Metadata reports scalar row eligibility and the hydrostatic fast-element count.
|
||
On the standard mesh both scalar flags should be 1 and the hydrostatic count
|
||
should equal the stellar element count. The optimized scalar rows retain two
|
||
local vectors per operator; hydrostatic adds nine doubles (72 bytes) per eligible
|
||
quadrature point. Reference tables are shared across operators with matching
|
||
elements and quadrature rules. Old mapping caches remain for reference/fallback.
|
||
|
||
The user-run `stellar_derivatives_01` comparison passed all checks and measured
|
||
885x and 1349x repeated scalar applications, and 1.94x hydrostatic displacement.
|
||
These exclude common preparation. The closure/force changes described below
|
||
have not been compiled or executed by the coding agent.
|
||
|
||
## H(div) comparison
|
||
|
||
From the repository root, run:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system hdiv-displacement \
|
||
--check full \
|
||
--implementation both \
|
||
--samples 5 --iterations 3 --warmup 1 \
|
||
--output hdiv_isolation_01
|
||
```
|
||
|
||
The output directory must not exist. This uses `sandbox.smesh`, the production
|
||
quadrature policy and registered element orders, and one MPI rank on the CPU.
|
||
It performs no equilibrium solve. Directions and gravity coefficients are
|
||
deterministic; the geometry is a small nonuniform polynomial displacement with
|
||
shear and cross terms. `--deformation 0` selects the undeformed timing state;
|
||
the default is `0.7`, multiplying displacements whose coefficients are of order
|
||
0.01–0.04. This argument is not a fractional stellar radius or rotation rate.
|
||
|
||
Full validation checks two geometries (zero and the selected deformation), all
|
||
configured directions against the retained prepared derivative, and the first
|
||
direction against the independent stateless kernel. It also checks the selected
|
||
geometry against centered differences of the primal mass operator at two step
|
||
sizes. Those differences hold the gravity coefficients fixed. A failed check
|
||
stops the run before timing and returns status 3. Execution errors return 2;
|
||
successful requested checks and timing return 0. With `--check none`, status 0
|
||
means execution completed, not that correctness was checked.
|
||
|
||
Return `summary.txt`, `checks.csv`, `metadata.txt`, and `timings.csv` for analysis.
|
||
There is no need to run the full stellar model for this first comparison.
|
||
|
||
## Timing and interpretation
|
||
|
||
- `metadata.txt`: mesh path, geometry parameter, build/compiler information,
|
||
problem sizes, domain element counts, quadrature count, and how many elements
|
||
use the new derivative path. For the standard 3D Kelvin case, the last count
|
||
should equal the total element count.
|
||
- `summary.txt`: finite-element setup, operator construction, initial and repeat
|
||
preparation, validation time, median application times and their ratio.
|
||
- `timings.csv`: every measured sample, application count, seconds per
|
||
application, and the last output norm. Warmup is excluded. A/B execution order
|
||
alternates between samples. Output norms and file writes are outside the
|
||
application timer. Each sample cycles through the same direction sequence.
|
||
- `checks.csv`: numerical errors, tolerances, and pass/fail status. The derivative
|
||
comparison tolerance is 2e-11; centered differences use 2e-6 to allow subtraction
|
||
error. A numerical failure is not permission to relax a tolerance.
|
||
|
||
For `hdiv-displacement`, `prepared_reference` is the previous prepared application
|
||
algorithm, retained in the same executable and using the same prepared state,
|
||
quadrature and field maps. This is the relevant application speedup comparison.
|
||
Both paths include true/local vector transfers and element assembly. Preparation
|
||
has already constructed the new cache in either case: this comparison does not
|
||
measure the old preparation cost or old peak memory use.
|
||
|
||
For `hdiv-mass`, `optimized` is the new CPU contraction and
|
||
`mfem_pa_reference` is the retained MFEM application using the same PA data.
|
||
This ratio measures the primal mass improvement from the latest batch. Full
|
||
checks still compare the independent stateless kernel, which reevaluates mapping.
|
||
Mass-only runs use primal preparation and report zero variation-cache elements.
|
||
|
||
The new tangent cache adds 12 doubles per eligible quadrature point (96 bytes),
|
||
plus shared reference displacement tables. Existing frozen mapping data remains
|
||
available for the reference/fallback implementation. Initial preparation may
|
||
therefore increase; compare preparation plus the application's actual number of
|
||
repeated calls when judging the total benefit. The earlier `hdiv_isolation_01`
|
||
user run measured a 3.13x median application speedup on the standard mesh; this
|
||
is separate from the new scalar/hydrostatic optimizations.
|
||
|
||
For a longer optimized-only profiling window after correctness passes:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/system_isolation_profile_harness \
|
||
--system hdiv-displacement --implementation optimized --check none \
|
||
--samples 10 --iterations 20 --warmup 3 \
|
||
--output hdiv_optimized_profile_01
|
||
```
|
||
|
||
Use a separate directory for each run. Avoid concurrent builds or other heavy
|
||
jobs while collecting comparisons. To inspect another resolution, use
|
||
`--refinements N`; to change the number of direction/flux pairs, use
|
||
`--directions N`. These are local operator tests, not demonstrations of final
|
||
stellar accuracy or distributed correctness.
|
||
|
||
## Derivative implementation
|
||
|
||
For computational-mesh coordinates X, the mapped H(div) mass tensor is
|
||
|
||
\[
|
||
M = J^T J / \det J, \qquad J = \partial x_{physical}/\partial X.
|
||
\]
|
||
|
||
The fast path contracts shared reference displacement gradients before applying
|
||
the cached mesh inverse. In the stellar domain, delta J is grad(delta u). For
|
||
Kelvin compactification, the existing Kelvin radial-factor calculation supplies
|
||
the fixed scale and its gradient:
|
||
|
||
\[
|
||
\delta J = a\,\nabla_X\delta u + \delta u (\nabla_X a)^T.
|
||
\]
|
||
|
||
Only `a * J_mesh^-1` and `grad_X(a)` are added to the point cache. The directional
|
||
mass action on a computational H(div) vector g is evaluated as
|
||
|
||
\[
|
||
\delta M g = \frac{\delta J^T(Jg) + J^T[\delta Jg -
|
||
\operatorname{tr}(J^{-1}\delta J)Jg]}{\det J}.
|
||
\]
|
||
|
||
This avoids computing unused inverse-mapping, inverse-element, physical-position,
|
||
and quadrature-weight derivatives, and avoids constructing the full mass-tensor
|
||
derivative. The fixed computational-mesh Piola factors, signed DOFs, primal/dual
|
||
DOF transformations, quadrature weights and true/local transfers are retained.
|
||
There is no change to equations, integration order, or solver tolerances.
|
||
|
||
The specialization requires three dimensions, an H(div) reference table, and
|
||
VALUE/GRAD displacement elements. Compactified elements additionally require
|
||
the concrete Kelvin map and a differentiable compactification element. Other
|
||
maps use the retained general implementation; merely declaring a map affine
|
||
does not select the Kelvin specialization.
|
||
|
||
## Compile and regression checks
|
||
|
||
Configure the release build with
|
||
`CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE=PRE_TEST`, so compiling test targets
|
||
does not execute them to enumerate cases. Discovery occurs when the user runs
|
||
CTest. Configure/build commands are:
|
||
|
||
```sh
|
||
cmake -S . -B cmake-build-release-homebrew \
|
||
-DCMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE=PRE_TEST
|
||
cmake --build cmake-build-release-homebrew \
|
||
--target system_isolation_profile_harness tests sandbox --parallel 2
|
||
```
|
||
|
||
Focused regression cases for the user to run:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/tests '*Prepared Hdiv*,*Prepared Mapped Hdiv*,Kelvin Prepared Jacobian Factors*'
|
||
```
|
||
|
||
Focused scalar/hydrostatic regression cases for the user to run after building:
|
||
|
||
```sh
|
||
./cmake-build-release-homebrew/tests '*Prepared Mass Normalization*,*Prepared Angular Momentum*,*Prepared Hydrostatic*'
|
||
```
|
||
|
||
These existing suites now also check retained-reference agreement, lazy row
|
||
construction, selective row invalidation, reuse across unrelated updates, and
|
||
angular-momentum rejection/recovery after rows have already been built. Existing
|
||
finite differences, transpose pairing and context lifecycle tests remain active.
|
||
|
||
## Closure and force implementation
|
||
|
||
All changes preserve quadrature, equations, field maps, signed element DOFs,
|
||
primal/dual transformations, and true/local transfers. There is no tolerance
|
||
based dropping of closure or vacuum terms.
|
||
|
||
- Closure couplings have supported-coordinate entry points for enthalpy alone
|
||
and displacement alone. The preconditioner calls these rather than executing
|
||
a complete action with known zero columns. The complete action still includes
|
||
every column. Its geometric scalar remains
|
||
`weightedResidual * tr(K * du/dxi)`, evaluated with a small direct contraction.
|
||
- Pressure uses the existing physical test gradients G and computes H=du/dx.
|
||
Its displacement contribution is `p*w*(G*H-tr(H)*G)`. This avoids constructing
|
||
the inverse-Jacobian variation and the matrix of all gradient variations.
|
||
- Gravity evaluates the existing expression
|
||
`w_ref * [F*(delta_rho*g + rho*delta_g) + rho*(du/dxi)*(J_mesh^-1*g)]`.
|
||
The last term contracts the reference gradients with a vector directly.
|
||
Density interpolation and test-basis transpose are done per element; H(div)
|
||
reference interpolation keeps the existing mesh Piola factor.
|
||
- Rotation uses shared reference value/gradient tables, bulk interpolation and
|
||
transpose application, and a cached 3x3 centrifugal acceleration derivative.
|
||
The force variation is
|
||
`w * [(delta_rho + rho*delta_log_w)*a + rho*delta_a]`.
|
||
The acceleration derivative is rebuilt with the existing preparation lifecycle.
|
||
|
||
The new paths add per-element scratch buffers reused between elements and
|
||
shared reference tables for rotation; they do not add large per-point dense
|
||
Jacobian blocks. Existing data and reference paths are retained. Reference
|
||
applications do not increment production column/complete application counters.
|
||
|
||
## Scalar and hydrostatic derivatives
|
||
|
||
For the supported stellar VALUE/GRAD displacement field, let `B_a` be the
|
||
reference basis, `K` the inverse deformed element Jacobian, and `w` the mapped
|
||
quadrature weight. An element displacement direction satisfies
|
||
|
||
```
|
||
delta x_c = sum_a B_a delta u_ac
|
||
delta log(w) = sum_acj K_jc (dB_a/dxi_j) delta u_ac
|
||
```
|
||
|
||
The mass row integrates `w delta rho + rho w delta log(w)`. The moment-of-inertia
|
||
row integrates `w r_perp^2 delta rho + rho w [r_perp^2 delta log(w) +
|
||
2 r_perp dot delta x]`. Angular momentum applies `Omega delta I + I delta Omega`.
|
||
The local coefficient vectors are assembled with signed element DOFs and dual
|
||
DOF transformations, then paired with prolonged true-DOF directions. The final
|
||
scalar retains the existing MPI sum. Keeping these as local duals avoids
|
||
double-counting shared true DOFs. Mass transpose uses the transpose prolongation.
|
||
|
||
Hydrostatic displacement contracts
|
||
`w [(h + Phi - Psi - C) delta log(w) - grad(Psi) dot delta x]`, then applies the
|
||
existing enthalpy test-basis transpose and assembly. It does not construct a
|
||
full mapping variation or repeatedly evaluate reference shapes. Only 3D interior
|
||
VALUE/GRAD elements use its fast path. Scalar row caching supports interior
|
||
VALUE/GRAD elements (mass in dimensions 1-3; angular momentum is already 3D).
|
||
Unsupported configurations retain the original general application. Density or
|
||
geometry changes invalidate scalar rows; Omega/target-only updates reuse them.
|
||
The equations, quadrature rules and solver tolerances are unchanged.
|
||
|
||
These include comparison with the stateless derivative, the general exterior-map
|
||
fallback, primal-to-linearization transitions, candidate rejection/recovery,
|
||
and Kelvin factor checks. The harness adds independent finite differences and
|
||
multiple direction pairs. Compilation alone does not validate their outcomes.
|
||
|
||
## Preparation and tensor implementation details
|
||
|
||
Gravity forms the residual quadrature force from the already computed
|
||
`rho * w_ref * F * g_ref`; rotation uses `rho * w * acceleration`.
|
||
Each then applies the cached displacement test-basis transpose. The optimized
|
||
preparation visits geometry once, retains element DOF transforms and true/local
|
||
assembly, and synchronizes candidate rejection before collective assembly.
|
||
`PrepareReference` retains the separate stateless residual pass. Switching
|
||
preparation implementations invalidates the relevant cached result even if
|
||
physical dependency revisions are unchanged.
|
||
|
||
Pressure retains the same local enthalpy matrix and its application. It packs
|
||
`-P'(h) * w * grad(test)` once per element and accumulates 32-row by 8-column
|
||
matrix tiles with contiguous row access. The original per-quadrature assembly
|
||
remains available through `PrepareReference`. EOS, mapping and finite-result
|
||
checks remain active; no quadrature points or matrix entries are dropped.
|
||
|
||
The scalar reference-table cache now owns small one-dimensional value/gradient
|
||
tables and a copied lexicographic-to-native DOF map for H1 hexahedra.
|
||
Cartesian quadrature ordering and the reconstruction of the existing full
|
||
values/gradients are checked once before publishing tensor support. Unsupported
|
||
elements/rules use the existing path. Evaluation contracts x, y and z in turn
|
||
and produces displacement values and reference gradients, then the hydrostatic
|
||
operator applies the same inverse-Jacobian contraction, physical coefficients,
|
||
test-basis transpose and assembly. Scratch is reused for one element; no dense
|
||
per-element displacement Jacobian is added. The immutable tables own all tensor
|
||
data, including when the FE, quadrature rule or cache that created them is gone.
|