Check frozen numerical strategies without conflating incomplete evidence with success
This commit is contained in:
72
experiments/check_numerical_strategies_README.md
Normal file
72
experiments/check_numerical_strategies_README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Checking numerical-strategy experiments
|
||||
|
||||
Run the checker on each completed frozen-probe directory:
|
||||
|
||||
```sh
|
||||
python3 experiments/check_numerical_strategies.py RESULT_FOLDER --require gravity
|
||||
python3 experiments/check_numerical_strategies.py RESULT_FOLDER --require polynomial
|
||||
python3 experiments/check_numerical_strategies.py LATE_ROTATING_FOLDER --require polynomial --expect-negative-base
|
||||
python3 experiments/test_check_numerical_strategies.py
|
||||
```
|
||||
|
||||
Both sections are discovered automatically when their CSV files exist. Repeating
|
||||
`--require` demands both sections. Exit 0 means the requested checks passed;
|
||||
3 means an invariant or numerical expectation failed; 2 means evidence is
|
||||
incomplete. An honestly reported failed inner solve, unavailable trial, or
|
||||
unresolved certificate must not turn into a passing verification. Expected
|
||||
negative-base detection and rejection are successful guard tests, not successful
|
||||
equilibrium solves.
|
||||
|
||||
Accepted state, residual, physical state, saved correction, and retained-coordinate
|
||||
changes must be exactly zero. Restored residual drift defaults to exactly zero;
|
||||
`--restoration-tolerance` relaxes only that comparison when explicitly justified.
|
||||
The JSON output records the chosen tolerance.
|
||||
|
||||
For successful curvature probes, the checker compares the fresh gravity remainder
|
||||
with the selected inner threshold plus a documented arithmetic scale, verifies
|
||||
that only gravity changed, and checks correction/alpha² arithmetic. It requires
|
||||
two small steps resolved above that scale to assess a second-order trend. An
|
||||
observed order of at least 1.7 passes the finite-range trend check; this does not
|
||||
prove an asymptotic rate. Higher order is allowed because a quadratic coefficient
|
||||
can vanish.
|
||||
|
||||
Reduced probes must satisfy the freshly prepared gravity completion tolerance,
|
||||
the lifted gravity tangent gate, and agreement of the Schur action with the
|
||||
retained part of the full lifted action. Central differences must reach relative
|
||||
error at most 1e-5 and show a resolved interval with order at least 1.5, or reach
|
||||
the recorded residual-subtraction scale. A plateau above that scale is incomplete
|
||||
evidence rather than an automatic claim of a bad Jacobian: the retained-residual
|
||||
amplification of the remaining gravity error has not been bounded. The recorded
|
||||
roundoff scales are heuristics, not rigorous forward-error bounds.
|
||||
|
||||
This script deliberately does not duplicate physical acceptance. For each
|
||||
end-to-end model, run `check_plateau_physics.py` on its endpoint. That separate
|
||||
gate requires recorded strict nonlinear convergence and all eight defined virial
|
||||
measurements at or below 1e-5. Its local EOS, hydrostatic, and sign measurements
|
||||
remain part of the physical assessment.
|
||||
|
||||
## Bounded validation matrix
|
||||
|
||||
These are proposed operational limits, not measured performance predictions.
|
||||
Use one numerical process at a time and preserve all capped failures as results.
|
||||
|
||||
| Layer | Cases | Bound / acceptance |
|
||||
|---|---|---|
|
||||
| Algebra and lifecycle | Gravity subsystem, Schur reduction, polynomial certificate tests; one and two MPI ranks | 2 minutes per test group; true-defect, stale-state, empty-local, rollback, type-contract coverage |
|
||||
| Python checker | Synthetic passing, false-convergence, mutation, missing-data, negative-base, and NaN records | 30 seconds; standard-library unit tests |
|
||||
| Positivity frozen states | n=1 J=0.1 captured iteration 2 and iteration 11; a certified seed/control | 10 minutes per probe; iteration 11 must be negative and reject repair with `invalid_base`; no claim that the earlier base is positive before checking it |
|
||||
| Gravity curvature | n=1.5 J=0 captured iteration 1 | 15 minutes; alpha = 1, 0.1, 0.01, 0.001; gate fresh remainder cancellation and small-step order |
|
||||
| Reduced gravity | Same n=1.5 state, plus constrained n=3 saved seed direction | 15 minutes per probe; completion and three central-difference pairs; n=3 direction remains an approximate diagnostic correction |
|
||||
| Regression equilibrium | n=1 J=0 and low-rotation J=0.05 baseline; opt-in strategies individually | Preserve existing nonlinear/linear caps, add a 20-minute wall limit per model; strict solver and all virial gates remain mandatory |
|
||||
| Failure-prone equilibrium | n=1 J=0.1 and n=1.5 J=0, strategies separately before combinations | 8 Newton steps / 20 minutes; report residual progress, auxiliary work, rejection causes, and physical endpoint without equating the cap with convergence |
|
||||
|
||||
At the supplied frozen-gravity defaults, there are at most 20 explicitly requested
|
||||
inner solves, each capped at 160 iterations, and at most 36 preparations in the
|
||||
normal completed path. The maximum covers four curvature corrections, seven
|
||||
gravity completions with up to two corrections each, and two reduced actions.
|
||||
Rejected preparations can reduce those counts. Root integration controls and
|
||||
end-to-end outer solves are separately bounded; their nested work is not covered
|
||||
by this frozen-probe count.
|
||||
|
||||
Treat outer reduced-solve time as inclusive of nested gravity solves. Report the
|
||||
inner counters separately, but do not add the two times as if they were disjoint.
|
||||
Reference in New Issue
Block a user