.. _execution-model:

Execution Model
===============

Stencil operations are executed in a three dimensional :term:`Iteration space`. The first two dimensions of the
iteration space, usually referred to as `I` and `J` dimensions identify the horizontal dimension. There is no
prescription on in what order stencil operators in different points of the same `IJ` plane will be executed. Stencil
operators in the third dimension of the iteration space, usually referred as `K` or vertical dimension, can have
prescribed order of executions. There are three different ways of execution policies for the `K` dimension:

- `forward`: The computation at index `k` in the vertical dimension is executed after index `k - 1` for all points in
  the horizontal plane;
- `backward`: The computation at index `k` in the vertical dimension is executed after index `k + 1` for all points in
  the horizontal plane;
- `parallel`: No order is specified and execution can happen concurrently.

An execution strategy for a computation is indicated by ``execute::order()``,
where `order` is one of the `K` dimension execution orders: ``forward``, ``backward``, ``parallel``.

The parallel execution policy allows for tuning the degree of parallelism (in the CUDA backend), by selecting
``execute::parallel_block<vertical_block_size>``, where ``vertical_block_size`` describes the dimension of CUDA block
in the vertical (a too high value e.g. each k-level in a separate CUDA-thread might result in a too low instruction level
parallelism and therefore in non-optimal performance). The default vertical bock size can be set by defining the macro
``GT_DEFAULT_VERTICAL_BLOCK_SIZE``.

.. note::

   The option to specify the vertical block size will most likely be moved to the backend in the future,
   see :ref:`backend-selection`.

More concretely, a multistage is a list of stages (implemented with stencil operators) to be executed with a certain
execution policy.  A computation combines several multistages, and will execute one multistage after the other.

- For each `IJ` plane, the stages of a multistage will be executed strictly one after the other. This means, that a
  stage can assume, that the previous stage has been applied to the the whole `IJ` plane before it is executed. The user
  can explicitly create independent stages, that don't require this restriction.
- If the execution policy is `parallel`, a stage cannot impose any assumptions on which stages are applied before in
  another `IJ` plane.
- If the execution policy is `forward`, it is guaranteed, that if a stage is executed at index ``k``, then all stages of
  the multistage were already applied to the same column with smaller ``k``, and that no stages of the multistage are
  already applied to the indices in the same column with larger ``k``.
- If the execution policy is `backward`, it is guaranteed, that if a stage is executed at index ``k``, then all stages of
  the multistage were already applied to the same column with larger ``k``, and that no stages of the multistage are
  already applied to the indices in the same column with smaller ``k``.


-------------------
Access restrictions
-------------------

The execution model imposes restrictions on how accessors can be evaluated. The following restrictions apply:

 1. Accessors can be read at any offset at any time, if the fields bound to the accessor are read-only within
    a whole *multistage*

 2. A stage may write to a field through an accessor. In this stage and in all later stages in the same multistage,
    this field must not be read anymore with any non-zero offsets, except with k-offsets in case of
    non-parallel policy. For example, you must not access such a field with ``eval(field(-1, -1, -1))``.
