#pragma once /** * @file dependencies.hpp * @author Emily M. Boudreaux * @brief This file is a convenience header which includes all of the headers in the dependencies module. Further, we include the detailed * documentation for the dependencies module here. * */ #include "serif/utils/dependencies/keys.hpp" #include "serif/utils/dependencies/binding.hpp" #include "serif/utils/dependencies/key_list.hpp" #include "serif/utils/dependencies/key_list_operations.hpp" #include "serif/utils/dependencies/dependency_context.hpp" #include "serif/utils/dependencies/computation.hpp" #include "serif/utils/dependencies/computation_graph.hpp" #include "serif/utils/dependencies/observers.hpp" #include "serif/utils/dependencies/computation_cache.hpp" #include "serif/utils/dependencies/snapshot.hpp" /** * *Computational Graphs* * * Demonstration code for how to use the SERiF computational graph system * The broad idea here is that there are many calculations which depend on each other but which may not necessarily always need to be recomputed just * because another part of the model has changed. We model this as a combination of an acyclic computational graph, defined at compile time, and * a runtime cache of values. The general workflow is that a developer defines the "context" of the computational graph, think of this * as all of the inputs to the graph, or maybe more specifically, anything that could change due to some external event (e.g. a time step, a newton step, etc...) * This context is probably the most CS intensive part of this code as the developer needs to think about lifetimes and ownership. * * **Context** * Contexts are made with the make_dependency_context function, this accepts a variadic pack of bindings between a type and a runtime value. * You can generate these yourself using the binding functions and types; however, we provide two convenience wrappers which generally developers should consider * using. * - borrow: Used to bind a type to a reference to a value. This is useful for values which are owned by the caller and which the caller will manage the lifetime of (takes an lvalue). * - own: Used to bind a type to a value which is owned by the context (copies an lvalue or moves an rvalue) * Each of these functions accepts a type as a template parameter and a runtime value as an argument. The type is used as an ID, this is key to understand, if we write * \code{.cpp} * using namespace serif::utils::dependencies; * * double x{1.0}; * auto context = make_dependency_context(borrow(x)); * \endcode * * we are not saying that x is of type keys::Input, this is clearly not the case. Rather, we are registering in the computational graph a binding * between the type keys::Input and the runtime value x. Think of this as naming the various inputs to the graph. * * **Computations** * After defining a context we can define a computation, each computation defines a node, while its ordered input keys define the incoming edges. * Consider the following example (assuming the context from above): * * \code{.cpp} * auto func = make_computation>( * [](const double _x) -> double { * return _x * _x; * } * ); * \endcode * * what we have defined here is a node which takes a single input tagged keys::Input and produces an output tagged as keys::Output. The input key * declares an edge from that source to this node. Other nodes can use keys::Output as an input, and a successfully prepared result can be read with get * or retained with share. These connections allow us to form a larger graph. Consider now the following code * * \code{.cpp} * using namespace serif::utils::dependencies; * * struct tags { * struct X{}; * struct Y{}; * * struct Output1{}; * struct Output2{}; * }; * * double x{1.0}; * double y{2.0}; * auto context = make_dependency_context(borrow(x), borrow(y)); * * auto func1 = make_computation>( * [](const double _x) -> double { * return _x * _x; * } * ); * * auto func2 = make_computation>( * [](const double _x, const double _y) -> double { * return _x + _y; * } * ); * \endcode * * we have now defined two computations, the first takes a single input, whichever input is tagged as tags::X and produces an output tagged as tags::Output1. * The second computation takes two inputs, the first is the output of the first computation (lookup by the tag) and the second is whichever input is tagged as tags::Y, * and produces an output tagged as tags::Output2. Note that the order of arguments to the lambda must be the same as the order of the tags in the InputList, the * dependencies module will automatically order arguments based on the InputList. There is type checking to ensure that the types match, but if you have two * inputs of the same type that are flipped there is no way for the system to inspect that and warn you. Further, depending on your compiler and flags * there may be implicit casting between convertable types (e.g. int <-> double). * * @todo We may want to remove the ability to have implicit casting between types, this is a bit of a footgun and may lead to subtle bugs. * * @note This is primary goal of this system, split computations into small pieces with the ability to cache intermediate results and reuse * as many of those intermediate results as we can can prove is safe to do. The whole reason this system is so complex is that very purpose. There is a fundamental * problem with high dimensional stellar models, they are always going to be slow, therefore we need to accept some degree of complexity in order to make them tractable. * * **Computation Cache** * Once we have the computations we need to connect them together, generally use the helper function make_computation_cache which * takes a context and a variadic pack of computations. There are *a lot* of compile time checks which run to ensure that the graph is valid, the context * describes what is needed to compute the graph, and that each computation is valid. Generally if a static analysis error or compile error pops up here * it probably means there is a mistake in the definition of the context or the computations. I've tried to make these errors as clear as possible; however, * there is only so much that you can do with the C++ compiler. I recommend first checking, probably with a drawn-out diagram, that all the * tags are correct and what you expect. * * A developer can also choose to use the make_computation_graph function to explicitly name a graph. This * is useful when naming or reusing a collection of computations (make_computation_cache * calls make_computation_graph and then fixes the source types from the supplied context without retaining the context). Now consider the following code assuming the context and computations from above: * * \code{.cpp} * auto graph_cache = make_computation_cache(context, func1, func2); * \endcode * * At this point we have a cache which we can access various parts of; however, no computations have been run yet. To run a computation we must call try_prepare. * * try_prepare returns std::expected: ordinary rejection is a value, and no candidate results or snapshots are published on failure. * Exceptions from a callable or snapshot policy propagate with the same cache protection. Previously retained results remain valid, * but get and share require a successful current preparation. Always check the report before reading results. * * try_prepare takes the context and evaluates the requested nodes and their prerequisites, reusing unchanged results. By default every result * is requested, so the first call evaluates the entire graph. RequestedResults can restrict evaluation to a branch. * * \code{.cpp} * auto report = graph_cache.try_prepare(context); * if (!report) { * return; // Handle report.error() before reading the cache. * } * * double func1_result = graph_cache.template get(); * double func2_result = graph_cache.template get(); * * auto func1_status = report->template status(); * auto func2_status = report->template status(); * * std::string_view func1_status_string = preparation_status_name_map[func1_status]; // note this is an inline map defined in serif::utils::dependencies * std::string_view func2_status_string = preparation_status_name_map[func2_status]; * * std::println("Func1 Result: {}, Func2 Result: {}, Func1 Status: {}, Func2 Status: {}", func1_result, func2_result, func1_status_string, func2_status_string); * \endcode * * Now what we could do is change only one part of the input context and call try_prepare again. Lets say we change only y, such that only func2 is stale. * * \code{.cpp} * y = 3.0; * report = graph_cache.try_prepare(context); * if (!report) { * return; // Handle report.error() before reading the cache. * } * * double func1_result = graph_cache.template get(); * double func2_result = graph_cache.template get(); * * auto func1_status = report->template status(); * auto func2_status = report->template status(); * * std::string_view func1_status_string = preparation_status_name_map[func1_status]; // note this is an inline map defined in serif::utils::dependencies * std::string_view func2_status_string = preparation_status_name_map[func2_status]; * * std::println("Func1 Result: {}, Func2 Result: {}, Func1 Status: {}, Func2 Status: {}", func1_result, func2_result, func1_status_string, func2_status_string); * \endcode */ namespace serif::utils::dependencies{};