ViennaCL - The Vienna Computing Library: Memory Model

ViennaCL - The Vienna Computing Library 1.7.1 Free open-source GPU-accelerated linear algebra and solver library. |

All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Memory Model

With the support of multiple compute backends, memory buffers need to be managed differently depending on whether CUDA, OpenCL or a plain host-based buffer is in use. These different memory domains are abstracted in a class `viennacl::backend::mem_handle`, which is able to refer to a buffer in all three backends, possibly at the same time. Objects of type `mem_handle` are the building blocks of scalars, vectors and matrices in ViennaCL, see Basic Types.

The raw handles for each memory domain can be obtained via the member functions `cuda_handle()`, `opencl_handle()`, and `ram_handle()`. Note that the former two may not be available if no support for the respective backend is activated using the preprocessor constants `VIENNACL_WITH_CUDA` and `VIENNACL_WITH_OPENCL`, see Enabling OpenMP, OpenCL, or CUDA Backends

# Memory Handle Operations

Each supported backend is required to support the following functions (arguments omitted for brevity, see reference documentation in `doc/doxygen` for details):

- `memory_create()`: Create a memory buffer
- `memory_copy()`: Copy the (partial) contents of one buffer to another
- `memory_write()`: Write from a memory location in CPU RAM to the buffer
- `memory_read()`: Read from the buffer to a memory location in CPU RAM

A common interface layer in `viennacl::backend` dispatches into the respective routines in the backend for the currently active memory domain of the handle.

# Querying and Switching Active Memory Domains

The global default memory domain can be queried by

    viennacl::backend::default_memory_type();

If the global memory domain should be changed to a new default, pass the new memory domain as parameter to `default_memory_type()`. For example, to allocate buffers in main memory even though OpenCL or CUDA are enabled, call

    viennacl::backend::default_memory_type(viennacl::MAIN_MEMORY);

and all subsequent buffers are allocated in main memory unless specified otherwise.

A global change of the default memory domain is not thread-safe, hence one can specify the memory domain individually for each object holding one or more memory handles internally. Unless specified otherwise, such a `mem_handle` object creates its buffer for the default memory type. The current memory domain of a `mem_handle` can be queried using the member function `memory_domain()` and returns one of the values `MEMORY_NOT_INITIALIZED`, `MAIN_MEMORY`, `OPENCL_MEMORY`, or `CUDA_MEMORY` defined in the struct `viennacl::memory_types`.

The currently active memory handle can be switched from outside using the member function `switch_memory_domain()`. For example, to indicate that the memory referenced by a handle `h`, the line

    h.switch_active_handle_id(viennacl::MAIN_MEMORY);

is sufficient. However, no memory is created, copied, or manipulated when switching the currently active handle, because a `mem_handle` object does not know what the buffer content is referring to and is thus not able to convert data between different memory domains if required.

In order to copy the contents of a memory buffer in one memory domain to a memory buffer in another memory domain within the same `mem_handle`-object, the data type must be supplied. This is accomplished using the function `viennacl::backend::switch_memory_domain(mem_handle, viennacl::memory_types)`, which takes the data type as template argument. Thus, in order to make current data of type `float` availabe in CPU RAM for a handle `h`, the function

    viennacl::backend::switch_memory_domain<float>(h, viennacl::MAIN_MEMORY);

is sufficient.

If data should be transferred from one memory handle `h1` to another memory handle `h2`, the function `viennacl::backend::typesafe_memory_copy(h1, h2)` is provided. It takes the data type as template argument and ensures a data conversion between different memory domains if required (e.g. `cl_uint` to `unsigned int`).