MeanField_SERiF/docs/static/dependencies.md

# Computational graphs

`serif::utils::dependencies` is a computational graph library: computations are
nodes, and their input keys declare incoming edges from sources or other nodes.
The existing namespace and API names are retained. The library is independent of
MFEM, equations, surfaces, and deformation. A context supplies external inputs;
a graph owns the computations and their execution order; a cache owns that graph,
source snapshots, and accepted results.

## A complete calculation

```cpp
#include <array>
#include "serif/utils/dependencies/dependencies.hpp"

/// Compute b = lambda * a, then reuse it while both inputs remain unchanged.
int main() {
    namespace dependencies = serif::utils::dependencies;
    namespace keys = dependencies::keys;
    using Vector = std::array<double, 3>;

    Vector a{1.0, 2.0, 3.0};
    double lambda = 2.0;
    const auto context = dependencies::make_dependency_context(
        dependencies::borrow<keys::Input>(a),
        dependencies::borrow<keys::Scale>(lambda)
    );
    const auto scale_vector = dependencies::make_computation<
        keys::Output, dependencies::InputList<keys::Input, keys::Scale>
    >(
        [](const Vector& input, double scale) -> Vector {
            return {scale * input[0], scale * input[1], scale * input[2]};
        }
    );
    auto cache = dependencies::make_computation_cache(context, scale_vector);

    auto report = cache.try_prepare(context);
    if (!report) {
        return 1;
    }
    const Vector b = cache.get<keys::Output>(); // <2, 4, 6>

    report = cache.try_prepare(context);
    if (!report) {
        return 1;
    }
    return report->status<keys::Output>() == dependencies::PreparationStatus::reused ? 0 : 1;
}
```

Creating the cache does not run the calculation or retain the context. The first
preparation computes the result. Further preparations compare input snapshots and
reuse the result while both `a` and `lambda` remain unchanged. Changing either one
requires recomputation. Keep the cache between calls so it can retain this state.
The [dependency sandbox](../../tests/sandbox/dependencies_sandbox.cpp) prints the values, per-node computation/reuse status, and elapsed time through those cases.

## Keys and ordered input lists

A key is a type identifying a role, independently of the type of value stored for
that role. `keys.hpp` offers `keys::Input`, `keys::Output`, `keys::Scale`,
`keys::Parameters`, and `keys::Coordinates` for small computations. These keys are
optional; custom types and existing discretization term descriptors work without
registration. A key need not be empty or constructible, because no key object is
created. Use distinct application keys when a graph has several different inputs
or outputs; giving two results the same convenience key is still a duplicate.

`InputList<A, B>` passes the values for `A` and `B` to a callable in that order.
`RequestedResults<X, Y>` selects which outputs to prepare. Both are aliases of the
same `KeyList` implementation, which also supports context selection and key
combination. `CombinedKeyList` removes duplicate roles in first-appearance order.

## Multiple computations and explicit graphs

The ordinary factory accepts any number of named computations:

```cpp
auto cache = dependencies::make_computation_cache(context, first, second);
```

Their supplied order is unrestricted. Dependency relationships establish the
evaluation order at compile time; each required computation runs at most once per
preparation. For a named or reusable collection, build an explicit graph:

```cpp
const auto graph = dependencies::make_computation_graph(first, second);
auto cache = dependencies::make_computation_cache(graph, context);
auto report = cache.try_prepare(context, dependencies::RequestedResults<FinalResult>{});
```

Named copyable computations and graphs are copied into their owner, so ordinary
calls need no `std::move`. Temporaries transfer ownership. A named computation or
graph owning noncopyable state requires an explicit transfer with `std::move`.
The factories reject unavailable copies at compile time.

## Observing execution and input consumption

Observers belong to the cache that executes a graph. Node observers run before and
after a calculation; edge observers inspect the inputs that calculation consumes.
The graph and its numerical computations do not need to know about observation.
Existing factory calls remain valid; an explicit graph accepts an optional observer
bundle as the third argument to `make_computation_cache`.

Continuing the vector-scaling example, include `<chrono>` and `<cstddef>`, and attach
a timer to the output node and a callback to its scale input:

```cpp
using Clock = std::chrono::steady_clock;
std::chrono::nanoseconds elapsed{};
std::size_t executions = 0;
double last_scale = 0.0;
dependencies::NodeOutcome last_outcome = dependencies::NodeOutcome::completed;

const auto observers = dependencies::make_observers(
    dependencies::observe_node<keys::Output>(
        []() { return Clock::now(); },
        [&](Clock::time_point& started, dependencies::NodeOutcome outcome) noexcept {
            elapsed += std::chrono::duration_cast<std::chrono::nanoseconds>(Clock::now() - started);
            ++executions;
            last_outcome = outcome;
        }
    ),
    dependencies::observe_edge<keys::Scale, keys::Output>(
        [&](const double& scale) { last_scale = scale; }
    )
);
const auto graph = dependencies::make_computation_graph(scale_vector);
auto observed_cache = dependencies::make_computation_cache(graph, context, observers);
const auto observed_report = observed_cache.try_prepare(context);
```

The cache owns its callbacks. Each before callback returns a movable object that
belongs to that one execution. The matching after callback receives this same
object by reference, which gives a timer or other paired operation local storage
without adding state to the numerical computation. Accumulated measurements belong
to the application, as the captures above illustrate. Captured references must
outlive every use of the cache; an owning capture may be used instead. Named,
noncopyable observer bundles must be transferred with `std::move`.

A node execution has the following order:

1. Matching before callbacks run in observer registration order.
2. Edge callbacks run in the node's declared `InputList` order, with callbacks for
   the same edge running in registration order.
3. The numerical callable runs.
4. Matching after callbacks run in reverse registration order.

An edge is identified by **input key, then consuming node key**. The input can be
an external source or another node's result. Each callback receives the underlying
value by const reference, including for owning computations whose numerical
callable receives shared handles. An unchanged prerequisite can therefore have an
edge callback when its consumer recomputes. Callbacks must not modify source or
result state, including objects reachable through pointers. Callbacks must not
prepare, invalidate, or move the cache whose execution they are observing. Numerical
transformations belong in declared computations.

No callbacks run for reused or unrequested nodes. Node keys, actual graph edges,
and callback signatures are checked at compile time. Multiple observers may attach
to the same node or edge. A node observer needs both before and after callbacks;
return a small empty object from before when no per-execution data is needed.

`NodeOutcome::completed` means the callable returned a value, `rejected` means it
returned an unsuccessful `std::expected`, and `exception` means execution exited
through an exception, including from an edge callback. These describe individual
execution, **not publication of the request**: a later node can still fail and
cause the whole preparation to be rejected. Inspect the final preparation report
when recording accepted requests.

After callbacks must return `void` and be `noexcept`, so they can safely finish
during exception propagation. Before and edge callbacks may propagate exceptions;
the cache retains the same transactional protection as numerical-callable failures.
If a before callback fails, its after callback does not run; already-started node
observers finish with `exception`. Observation side effects, such as recorded
measurements, are not rolled back. The sandbox records measurements in callbacks
and prints them afterward, keeping potentially throwing output out of after hooks.

## Operator caches

The MFEM operator backend uses these ordinary caches directly. Its volume
factories declare domain-specific calculations and return `ComputationCache` objects;
there is no second lifecycle owner around them. Input-validation helpers construct
current contexts before invoking the cache. Residual and Jacobian functions consume
the accepted data, and the MFEM adapter borrows the cache. The
[operator guide](operators.md#model-ownership-and-startup) shows a complete volume
example.

## Where things belong

| Component | Responsibility |
| --- | --- |
| `keys.hpp` | Optional general role names; applications may supply their own. |
| `binding.hpp`, `key_list.hpp` | Associate role keys with target types and express ordered inputs or requested results. |
| `key_list_operations.hpp` | Combine key lists without duplicating roles. |
| `resource.hpp`, `dependency_context.hpp` | Supply owned or borrowed objects and select views of their roles. |
| `snapshot.hpp` | Specify which source state determines whether cached work remains valid. |
| `computation.hpp` | Attach result and input keys to a callable. |
| `computation_graph.hpp` | Store calculations and establish their dependency order. |
| `computation_cache.hpp`, `preparation.hpp` | Prepare requested results, retain accepted work, and report status or rejection. |
| `observers.hpp` | Attach execution callbacks to node and edge keys, with per-execution node state. |
| `detail/` | Graph traversal, type resolution, and cache storage used by these interfaces. |

## Computation and lifetime contracts

- A callable can be a lambda, function, or function object. It must be callable
  through a const reference, using its inputs as const references or copies. Each
  result must be a movable object, returned by value.
- Every changing input that affects a result belongs in its `InputList`. Captures
  may hold fixed configuration. Changing state hidden behind captures, globals,
  or pointers cannot be discovered automatically.
- `own<Key>(value)` copies an lvalue or moves an rvalue. `borrow<Key>(object)`
  borrows an existing object; temporaries are rejected. The object must outlive
  the context's uses. Owning a view does not own its referenced data.
  `context.select(KeyList<...>{})` borrows its parent context and preserves the
  selected source identities.
- Results and captured snapshots must retain their relevant state independently
  of subsequent source edits. An owning container works; a view into mutable
  source storage does not freeze that storage. Treat cached results and their
  referenced objects as read-only.
- Preparation is serial. Sources and callable configuration must remain stable
  during each call. References obtained from `get` are valid until the next
  preparation, invalidation, or destruction of their owner.

`cache.share<Key>()` retains an immutable result across later preparations,
rejection, invalidation, and cache destruction. It owns the result itself; anything
borrowed inside that result must still outlive its use. Both `get` and `share`
require the result to belong to the latest successful request.

Use `make_owning_computation<Key, InputList<...>>(callable)` when a result needs
to retain its computed prerequisites. Its inputs are immutable shared handles,
so retained results keep the exact prerequisite generation that produced them.
Every input to an owning computation must be produced by the same graph; external
context sources cannot be retained through this interface.

## Tracking changes and reusing work

The default `ValueSnapshot` owns a copy and compares it using equality. It works
for numbers and value-owning standard containers. Objects with indirect state,
expensive copies, or no copy operation need an appropriate policy. A revision
counter can be used if its owner updates it for every relevant edit:

```cpp
auto source = dependencies::borrow<Samples>(samples,
    dependencies::track_by([](const auto& value) { return value.revision(); }));
```

This example assumes a source type providing `revision()`. For more control,
provide a policy with `capture(const Source&)` returning owned state and
`matches(const Captured&, const Source&)` returning a boolean. Snapshots also track
source identity: replacing a borrowed object forces recomputation even if its
tracked value is equal. A policy can override `identity(const Source&)` to return
`const void*` identifying the underlying object of a temporary view. Returning
`nullptr` deliberately opts into content-only comparison; the snapshot must then
include any identity relevant to correctness. Unused context objects need no
snapshot policy.

`try_prepare(context)` defaults to `PreparationMode::reuse_unchanged` and requests every result. `RequestedResults<...>{}` restricts
the request to those results and their prerequisites. Changed inputs invalidate
their dependents, including those requested in a later call. Other branches
retain their caches but are unavailable through `get` until requested again.
Accepted results stay frozen between preparations. When an input changes, the
computation and its required dependents run even if recomputation happens to produce
the same numerical output; result equality is not used to infer reuse.

The report's `status<Key>()` returns `PreparationStatus::not_required`, `reused`,
or `computed`. `computed` includes first evaluation and reevaluation; `not_required`
means outside the requested results and their prerequisites. The report also
provides `source_changed<Key>()`, `computed_count()`, and `reused_count()`.
An uninspected source is reported unchanged.

Pass `PreparationMode::rebuild` as the third argument to refresh the requested
results and their prerequisites, or call `invalidate()` to discard the whole
cache. Forced refreshes also mark inspected sources changed. `contains<Key>()`
checks result availability; `ready()` reports a successful current preparation.

`preparation_count()` counts successful requests, including requests that reused every
result and requests for only part of a graph. Rejected or throwing attempts do not
increment it. Invalidation retains the count. Moving the cache transfers the count
and accepted state to the destination and resets the source to an unprepared state
with count zero. This counter describes accepted requests, not numerical work;
use the report to distinguish computation from reuse.

## Rejection and compile-time checks

A computation can return `std::expected<Value, Error>` when it may reject an input.
Ordinary value-returning and fallible computations can share a graph. The default
error type is `std::string`; use `make_computation_cache<Error>(context, computations...)`
or `make_computation_cache<Error>(graph, context)` for another type. Each fallible
computation must return that exact error type, or an exact, unambiguous alternative
when the cache error is a `std::variant`. Implicit conversions between unrelated
error types are rejected. A failure contains the error
and the computation's index in the supplied graph; `ComputationGraph::index<Key>()`
gives the corresponding index for a known result key.

Preparation stages snapshots and results together. Success publishes all requested
results and their prerequisites. Rejection publishes none and makes results
unavailable until the next successful preparation. The previous accepted cache is
retained for possible reuse. Exceptions from callables or policies propagate with
the same cache protection. External side effects are not rolled back. A backend may
choose a stricter validation boundary: the MFEM `try_prepare_*` helpers invalidate
storage on rejection so that a later retry starts fresh. That policy is explicit
in the backend and does not change the generic cache's behavior.

Duplicate context roles, duplicate producers, and cycles fail at construction.
`ComputationGraphFor<Graph, Context>` checks input availability, source/result key
collisions, callable signatures, and result types. `CanPrepare<Graph, Context, Error>`
also checks snapshot policies and error compatibility. Both cache factories apply
these checks. Requests are constrained to result keys. Reading an unavailable
result throws `std::logic_error` because it indicates a caller sequencing error.

The general dependency module owns snapshots, accepted results, and rejection
handling; it contains no MFEM types.