/* StarPU --- Runtime system for heterogeneous multicore architectures.
 *
 * Copyright (C) 2009-2024  Université de Bordeaux, CNRS (LaBRI UMR 5800), Inria
 * Copyright (C) 2016       Uppsala University
 * Copyright (C) 2020,2021  Federal University of Rio Grande do Sul (UFRGS)
 *
 * StarPU is free software; you can redistribute it and/or modify
 * it under the terms of the GNU Lesser General Public License as published by
 * the Free Software Foundation; either version 2.1 of the License, or (at
 * your option) any later version.
 *
 * StarPU is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 *
 * See the GNU Lesser General Public License in COPYING.LGPL for more details.
 */

/*! \page ExecutionConfigurationThroughEnvironmentVariables Execution Configuration Through Environment Variables

The StarPU library and tools's behavior can be tuned using the following environment variables. To access these variables, you can use the provided functions.
<ul>
<li>
starpu_getenv() retrieves the value of an environment variable.
</li>
<li>
starpu_get_env_string_var_default() retrieves the value of an
environment variable as a string. If the variable is not set, you can
provide a default value.
</li>
<li>
starpu_get_env_size_default() retrieves the value of an environment
variable as a size in bytes, or a default value if the environment
variable is not set.
</li>
</ul>

These functions allow to fine-tune the behavior of StarPU
according to your preferences and requirements by leveraging
environment variables.

\section EnvConfiguringWorkers Configuring Workers

\subsection Basic General Configuration
<dl>
<dt>STARPU_WORKERS_NOBIND</dt>
<dd>
\anchor STARPU_WORKERS_NOBIND
\addindex __env__STARPU_WORKERS_NOBIND
Setting it to non-zero will prevent StarPU from binding its threads to
CPUs. This is for instance useful when running the test suite in parallel.
</dd>

<dt>STARPU_WORKERS_GETBIND</dt>
<dd>
\anchor STARPU_WORKERS_GETBIND
\addindex __env__STARPU_WORKERS_GETBIND
By default StarPU uses the OS-provided CPU binding to determine
how many and which CPU cores it should use. This is notably useful when running
several StarPU-MPI processes on the same host, to let the MPI launcher set the
CPUs to be used. Default value is 1.

If that binding is erroneous (e.g. because the job scheduler binds to just one
core of the allocated cores), you can set \ref STARPU_WORKERS_GETBIND to 0 to make
StarPU use all cores of the machine.
</dd>

<dt>STARPU_WORKERS_CPUID</dt>
<dd>
\anchor STARPU_WORKERS_CPUID
\addindex __env__STARPU_WORKERS_CPUID
Passing an array of integers in \ref STARPU_WORKERS_CPUID
specifies on which logical CPU the different workers should be
bound. For instance, if <c>STARPU_WORKERS_CPUID="0 1 4 5"</c>, the first
worker will be bound to logical CPU #0, the second CPU worker will be bound to
logical CPU #1 and so on.  Note that the logical ordering of the CPUs is either
determined by the OS, or provided by the library <c>hwloc</c> in case it is
available.
Ranges can be provided: for instance, <c>STARPU_WORKERS_CPUID="1-3
5"</c> will bind the first three workers on logical CPUs #1, #2, and #3, and the
fourth worker on logical CPU #5. Unbound ranges can also be provided:
<c>STARPU_WORKERS_CPUID="1-"</c> will bind the workers starting from logical
CPU #1 up to last CPU.

Note that the first workers correspond to the CUDA workers, then come the
OpenCL workers, and finally the CPU workers. For example, if
we have <c>STARPU_NCUDA=1</c>, <c>STARPU_NOPENCL=1</c>, <c>STARPU_NCPU=2</c>
and <c>STARPU_WORKERS_CPUID="0 2 1 3"</c>, the CUDA device will be controlled
by logical CPU #0, the OpenCL device will be controlled by logical CPU #2, and
the logical CPUs #1 and #3 will be used by the CPU workers.

If the number of workers is larger than the array given in
\ref STARPU_WORKERS_CPUID, the workers are bound to the logical CPUs in a
round-robin fashion: if <c>STARPU_WORKERS_CPUID="0 1"</c>, the first
and the third (resp. second and fourth) workers will be put on CPU #0
(resp. CPU #1).

This variable is ignored if the field
starpu_conf::use_explicit_workers_bindid passed to starpu_init() is
set.

Setting \ref STARPU_WORKERS_CPUID or \ref STARPU_WORKERS_COREID overrides the binding
provided by the job scheduler, as described for \ref STARPU_WORKERS_GETBIND.
</dd>

<dt>STARPU_WORKERS_COREID</dt>
<dd>
\anchor STARPU_WORKERS_COREID
\addindex __env__STARPU_WORKERS_COREID
Same as \ref STARPU_WORKERS_CPUID, but bind the workers to cores instead of PUs
(hyperthreads).
</dd>

<dt>STARPU_NTHREADS_PER_CORE</dt>
<dd>
\anchor STARPU_NTHREADS_PER_CORE
\addindex __env__STARPU_NTHREADS_PER_CORE
Specify how many threads StarPU should run on each core. The
default is 1 because kernels are usually already optimized for using a full
core. Setting this to e.g. 2 instead allows exploiting hyperthreading.
</dd>

<dt>STARPU_MAIN_THREAD_BIND</dt>
<dd>
\anchor STARPU_MAIN_THREAD_BIND
\addindex __env__STARPU_MAIN_THREAD_BIND
Tell StarPU to bind the thread that calls starpu_initialize() to
a reserved CPU, subtracted from the CPU workers.
</dd>

<dt>STARPU_MAIN_THREAD_CPUID</dt>
<dd>
\anchor STARPU_MAIN_THREAD_CPUID
\addindex __env__STARPU_MAIN_THREAD_CPUID
Tell StarPU to bind the thread that calls starpu_initialize() to
the given CPU ID (using logical numbering).
</dd>

<dt>STARPU_MAIN_THREAD_COREID</dt>
<dd>
\anchor STARPU_MAIN_THREAD_COREID
\addindex __env__STARPU_MAIN_THREAD_COREID
Same as \ref STARPU_MAIN_THREAD_CPUID, but bind the thread that calls
starpu_initialize() to the given core (using logical numbering), instead of the PU (hyperthread).
</dd>

<dt>STARPU_WORKER_TREE</dt>
<dd>
\anchor STARPU_WORKER_TREE
\addindex __env__STARPU_WORKER_TREE
Define to 1 to enable the tree iterator in schedulers.
</dd>

<dt>STARPU_SINGLE_COMBINED_WORKER</dt>
<dd>
\anchor STARPU_SINGLE_COMBINED_WORKER
\addindex __env__STARPU_SINGLE_COMBINED_WORKER
Tell StarPU to create several workers which won't be able to work
concurrently. It will by default create combined workers, which size goes from 1
to the total number of CPU workers in the system. \ref STARPU_MIN_WORKERSIZE
and \ref STARPU_MAX_WORKERSIZE can be used to change this default.
</dd>

<dt>STARPU_MIN_WORKERSIZE</dt>
<dd>
\anchor STARPU_MIN_WORKERSIZE
\addindex __env__STARPU_MIN_WORKERSIZE
Specify the minimum size of the combined workers. Default value is 2.
</dd>

<dt>STARPU_MAX_WORKERSIZE</dt>
<dd>
\anchor STARPU_MAX_WORKERSIZE
\addindex __env__STARPU_MAX_WORKERSIZE
Specify the minimum size of the combined workers. Default value is the
number of CPU workers in the system.
</dd>

<dt>STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER</dt>
<dd>
\anchor STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
\addindex __env__STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
Specify how many elements are allowed between combined workers
created from \c hwloc information. For instance, in the case of sockets with 6
cores without shared L2 caches, if \ref STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER is
set to 6, no combined worker will be synthesized beyond one for the socket
and one per core. If it is set to 3, 3 intermediate combined workers will be
synthesized, to divide the socket cores into 3 chunks of 2 cores. If it set to
2, 2 intermediate combined workers will be synthesized, to divide the socket
cores into 2 chunks of 3 cores, and then 3 additional combined workers will be
synthesized, to divide the former synthesized workers into a bunch of 2 cores,
and the remaining core (for which no combined worker is synthesized since there
is already a normal worker for it).

Default value is 2, thus makes StarPU tend to build binary trees of combined
workers.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_COPY
Disable asynchronous copies between CPU and GPU devices.
The AMD implementation of OpenCL is known to
fail when copying data asynchronously. When using this implementation,
it is therefore necessary to disable asynchronous data transfers. One can call starpu_asynchronous_copy_disabled() to check whether asynchronous data transfers between CPU and accelerators are disabled.

See also \ref STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY and \ref
STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY.
</dd>

<dt>STARPU_EXPECTED_TRANSFER_TIME_WRITEBACK</dt>
<dd>
\anchor STARPU_EXPECTED_TRANSFER_TIME_WRITEBACK
\addindex __env__STARPU_EXPECTED_TRANSFER_TIME_WRITEBACK
Set to 1 to make task transfer time estimations artificially include
the time that will be needed to write back data to the main memory.
</dd>

<dt>STARPU_DISABLE_PINNING</dt>
<dd>
\anchor STARPU_DISABLE_PINNING
\addindex __env__STARPU_DISABLE_PINNING
Disable (1) or Enable (0) pinning host memory allocated through starpu_malloc(), starpu_memory_pin()
and friends. Default value is Enable.
This permits to test the performance effect of memory pinning.
</dd>

<dt>STARPU_BACKOFF_MIN</dt>
<dd>
\anchor STARPU_BACKOFF_MIN
\addindex __env__STARPU_BACKOFF_MIN
Set minimum exponential backoff of number of cycles to pause when spinning. Default value is 1.
</dd>

<dt>STARPU_BACKOFF_MAX</dt>
<dd>
\anchor STARPU_BACKOFF_MAX
\addindex __env__STARPU_BACKOFF_MAX
Set maximum exponential backoff of number of cycles to pause when spinning. Default value is 32.
</dd>

<dt>STARPU_SINK</dt>
<dd>
\anchor STARPU_SINK
\addindex __env__STARPU_SINK
Defined internally by StarPU when running in master slave mode.
</dd>

<dt>STARPU_ENABLE_MAP</dt>
<dd>
\anchor STARPU_ENABLE_MAP
\addindex __env__STARPU_ENABLE_MAP
Disable (0) or Enable (1) support for memory mapping between memory nodes.  The
default is Disabled. One can call starpu_map_enabled() to check whether memory mapping support between memory nodes is enabled.
</dd>

<dt>STARPU_DATA_LOCALITY_ENFORCE</dt>
<dd>
\anchor STARPU_DATA_LOCALITY_ENFORCE
\addindex __env__STARPU_DATA_LOCALITY_ENFORCE
Enable (1) or Disable(0) data locality enforcement when picking up a worker to execute a task. Default value is Disable.
</dd>

</dl>

\subsection cpuWorkers CPU Workers
<dl>
<dt>STARPU_NCPU</dt>
<dd>
\anchor STARPU_NCPU
\addindex __env__STARPU_NCPU
Specify the number of CPU workers (thus not including workers
dedicated to control accelerators). Note that by default, StarPU will
not allocate more CPU workers than there are physical CPUs, and that
some CPUs are used to control the accelerators.
</dd>

<dt>STARPU_RESERVE_NCPU</dt>
<dd>
\anchor STARPU_RESERVE_NCPU
\addindex __env__STARPU_RESERVE_NCPU
Specify the number of CPU cores that should not be used by StarPU, so the
application can use starpu_get_next_bindid() and starpu_bind_thread_on() to bind
its own threads.

This option is ignored if \ref STARPU_NCPU or starpu_conf::ncpus is set.
</dd>

<dt>STARPU_NCPUS</dt>
<dd>
\anchor STARPU_NCPUS
\addindex __env__STARPU_NCPUS
Deprecated. You should use \ref STARPU_NCPU.
</dd>

</dl>

\subsection cudaWorkers CUDA Workers
<dl>
<dt>STARPU_NCUDA</dt>
<dd>
\anchor STARPU_NCUDA
\addindex __env__STARPU_NCUDA
Specify the number of CUDA devices that StarPU can use. If
\ref STARPU_NCUDA is lower than the number of physical devices, it is
possible to select which GPU devices should be used by the means of the
environment variable \ref STARPU_WORKERS_CUDAID. By default, StarPU will
create as many CUDA workers as there are GPU devices.
</dd>

<dt>STARPU_NWORKER_PER_CUDA</dt>
<dd>
\anchor STARPU_NWORKER_PER_CUDA
\addindex __env__STARPU_NWORKER_PER_CUDA
Specify the number of workers per CUDA device, and thus the number of kernels
which will be concurrently running on the devices, i.e. the number of CUDA
streams. Default value is 1.
</dd>

<dt>STARPU_CUDA_THREAD_PER_WORKER</dt>
<dd>
\anchor STARPU_CUDA_THREAD_PER_WORKER
\addindex __env__STARPU_CUDA_THREAD_PER_WORKER
Specify whether the cuda driver should use one thread per stream (1) or to use
a single thread to drive all the streams of the device or all devices (0), and
\ref STARPU_CUDA_THREAD_PER_DEV determines whether is it one thread per device or one
thread for all devices. Default value is 0. Setting it to 1 is contradictory
with setting \ref STARPU_CUDA_THREAD_PER_DEV.
</dd>

<dt>STARPU_CUDA_THREAD_PER_DEV</dt>
<dd>
\anchor STARPU_CUDA_THREAD_PER_DEV
\addindex __env__STARPU_CUDA_THREAD_PER_DEV
Specify whether the cuda driver should use one thread per device (1) or to use a
single thread to drive all the devices (0). Default value is 1. It does not
make sense to set this variable if \ref STARPU_CUDA_THREAD_PER_WORKER is set to to 1
(since \ref STARPU_CUDA_THREAD_PER_DEV is then meaningless).
</dd>

<dt>STARPU_CUDA_PIPELINE</dt>
<dd>
\anchor STARPU_CUDA_PIPELINE
\addindex __env__STARPU_CUDA_PIPELINE
Specify how many asynchronous tasks are submitted in advance on CUDA
devices. This for instance permits to overlap task management with the execution
of previous tasks, but it also allows concurrent execution on Fermi cards, which
otherwise bring spurious synchronizations. Default value is 2. Setting the value to 0 forces a synchronous
execution of all tasks.
</dd>

<dt>STARPU_WORKERS_CUDAID</dt>
<dd>
\anchor STARPU_WORKERS_CUDAID
\addindex __env__STARPU_WORKERS_CUDAID
Select which CUDA devices should be used to run CUDA workers (similarly to the \ref STARPU_WORKERS_CPUID environment variable).
On a machine equipped with 4 GPUs, setting <c>STARPU_WORKERS_CUDAID="1 3"</c> and
<c>STARPU_NCUDA=2</c> specifies that 2 CUDA workers should be created, and that
they should use CUDA devices #1 and #3 (the logical ordering of the devices is
the one reported by CUDA).

This variable is ignored if the field
starpu_conf::use_explicit_workers_cuda_gpuid passed to starpu_init()
is set.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
Disable asynchronous copies between CPU and CUDA devices. One can call starpu_asynchronous_cuda_copy_disabled() to check whether asynchronous data transfers between CPU and CUDA accelerators are disabled.

See also \ref STARPU_DISABLE_ASYNCHRONOUS_COPY and \ref
STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY.
</dd>

<dt>STARPU_ENABLE_CUDA_GPU_GPU_DIRECT</dt>
<dd>
\anchor STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
\addindex __env__STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
Enable (1) or Disable (0) direct CUDA transfers from GPU to GPU, without copying
through RAM. Default value is Enable.
This permits to test the performance effect of GPU-Direct.
</dd>

<dt>STARPU_CUDA_ONLY_FAST_ALLOC_OTHER_MEMNODES</dt>
<dd>
\anchor STARPU_CUDA_ONLY_FAST_ALLOC_OTHER_MEMNODES
\addindex __env__STARPU_CUDA_ONLY_FAST_ALLOC_OTHER_MEMNODES
Specify if CUDA workers should do only fast allocations
when running the datawizard progress of
other memory nodes. This will pass the internal value
_STARPU_DATAWIZARD_ONLY_FAST_ALLOC to allocation methods.
Default value is 0, allowing CUDA workers to do slow allocations.

This can also be specified with starpu_conf::cuda_only_fast_alloc_other_memnodes.
</dd>

</dl>

\subsection openclWorkers OpenCL Workers
<dl>
<dt>STARPU_NOPENCL</dt>
<dd>
\anchor STARPU_NOPENCL
\addindex __env__STARPU_NOPENCL
Specify the number of OpenCL devices that StarPU can use. If
\ref STARPU_NOPENCL is lower than the number of physical devices, it is
possible to select which GPU devices should be used by the means of the
environment variable \ref STARPU_WORKERS_OPENCLID. By default, StarPU will
create as many OpenCL workers as there are GPU devices.

Note that by default StarPU will launch CUDA workers on GPU devices.
You need to disable CUDA to allow the creation of OpenCL workers.
</dd>

<dt>STARPU_WORKERS_OPENCLID</dt>
<dd>
\anchor STARPU_WORKERS_OPENCLID
\addindex __env__STARPU_WORKERS_OPENCLID
Select which GPU devices should be used to run OpenCL workers (similarly to the \ref STARPU_WORKERS_CPUID environment variable)
On a machine equipped with 4 GPUs, setting <c>STARPU_WORKERS_OPENCLID="1 3"</c> and
<c>STARPU_NOPENCL=2</c> specifies that 2 OpenCL workers should be
created, and that they should use GPU devices #1 and #3.

This variable is ignored if the field
starpu_conf::use_explicit_workers_opencl_gpuid passed to starpu_init()
is set.
</dd>

<dt>STARPU_OPENCL_PIPELINE</dt>
<dd>
\anchor STARPU_OPENCL_PIPELINE
\addindex __env__STARPU_OPENCL_PIPELINE
Specify how many asynchronous tasks are submitted in advance on OpenCL
devices. This for instance permits to overlap task management with the execution
of previous tasks, but it also allows concurrent execution on Fermi cards, which
otherwise bring spurious synchronizations. Default value is 2. Setting the value to 0 forces a synchronous
execution of all tasks.
</dd>

<dt>STARPU_OPENCL_ON_CPUS</dt>
<dd>
\anchor STARPU_OPENCL_ON_CPUS
\addindex __env__STARPU_OPENCL_ON_CPUS
Specify that OpenCL workers can also be run on CPU devices. By default, the OpenCL driver only enables GPU devices.
</dd>

<dt>STARPU_OPENCL_ONLY_ON_CPUS</dt>
<dd>
\anchor STARPU_OPENCL_ONLY_ON_CPUS
\addindex __env__STARPU_OPENCL_ONLY_ON_CPUS
Specify that OpenCL workers can ONLY be run on CPU devices. By default, the OpenCL driver enables GPU devices.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
Disable asynchronous copies between CPU and OpenCL devices.
The AMD implementation of OpenCL is known to
fail when copying data asynchronously. When using this implementation,
it is therefore necessary to disable asynchronous data transfers. One can call starpu_asynchronous_opencl_copy_disabled() to check whether asynchronous data transfers between CPU and OpenCL accelerators are disabled.

See also \ref STARPU_DISABLE_ASYNCHRONOUS_COPY and \ref
STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY.
</dd>
</dl>

\subsection maxfpgaWorkers Maxeler FPGA Workers
<dl>
<dt>STARPU_NMAX_FPGA</dt>
<dd>
\anchor STARPU_NMAX_FPGA
\addindex __env__STARPU_NMAX_FPGA
Specify the number of Maxeler FPGA devices that StarPU can use. If
\ref STARPU_NMAX_FPGA is lower than the number of physical devices, it is
possible to select which Maxeler FPGA devices should be used by the means of the
environment variable \ref STARPU_WORKERS_MAX_FPGAID. By default, StarPU will
create as many Maxeler FPGA workers as there are GPU devices.
</dd>

<dt>STARPU_WORKERS_MAX_FPGAID</dt>
<dd>
\anchor STARPU_WORKERS_MAX_FPGAID
\addindex __env__STARPU_WORKERS_MAX_FPGAID
Select which Maxeler FPGA devices should be used to run Maxeler FPGA workers (similarly to the \ref STARPU_WORKERS_CPUID environment variable).
On a machine equipped with 4 Maxeler FPGAs, setting <c>STARPU_WORKERS_MAX_FPGAID="1 3"</c> and
<c>STARPU_NMAX_FPGA=2</c> specifies that 2 Maxeler FPGA workers should be created, and that
they should use Maxeler FPGA devices #1 and #3 (the logical ordering of the devices is
the one reported by the Maxeler stack).

<dt>STARPU_DISABLE_ASYNCHRONOUS_MAX_FPGA_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_MAX_FPGA_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MAX_FPGA_COPY
Disable asynchronous copies between CPU and Maxeler FPGA devices. One can call starpu_asynchronous_max_fpga_copy_disabled() to check whether asynchronous data transfers between CPU and Maxeler FPGA devices are disabled.
</dd>

\subsection mpimsWorkers MPI Master Slave Workers
<dl>
<dt>STARPU_NMPI_MS</dt>
<dd>
\anchor STARPU_NMPI_MS
\addindex __env__STARPU_NMPI_MS
Specify the number of MPI master slave devices that StarPU can use.
</dd>

<dt>STARPU_NMPIMSTHREADS</dt>
<dd>
\anchor STARPU_NMPIMSTHREADS
\addindex __env__STARPU_NMPIMSTHREADS
Specift the number of threads to use on the MPI Slave devices.
</dd>

<dt>STARPU_MPI_MS_MULTIPLE_THREAD</dt>
<dd>
\anchor STARPU_MPI_MS_MULTIPLE_THREAD
\addindex __env__STARPU_MPI_MS_MULTIPLE_THREAD
Specify whether the master should use one thread per slave, or one thread for
driver all slaves. Default value is 0.
</dd>

<dt>STARPU_MPI_MASTER_NODE</dt>
<dd>
\anchor STARPU_MPI_MASTER_NODE
\addindex __env__STARPU_MPI_MASTER_NODE
Specify the rank of the MPI process which will be the master. Default value is 0.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
Disable asynchronous copies between CPU and MPI Slave devices. One can call starpu_asynchronous_mpi_ms_copy_disabled() to check whether asynchronous data transfers between CPU and MPI Slave devices are disabled.
</dd>

</dl>

\subsection tcpipmsWorkers TCP/IP Master Slave Workers

<dl>

<dt>STARPU_NTCPIP_MS</dt>
<dd>
\anchor STARPU_NTCPIP_MS
\addindex __env__STARPU_NTCPIP_MS
Specify the number of TCP/IP master slave devices that StarPU can use.
</dd>

<dt>STARPU_TCPIP_MS_SLAVES</dt>
<dd>
\anchor STARPU_TCPIP_MS_SLAVES
\addindex __env__STARPU_TCPIP_MS_SLAVES
Specify the number of TCP/IP master slave processes that are expected to be
run. This should be provided both to the master and to the slaves.
</dd>

<dt>STARPU_TCPIP_MS_MASTER</dt>
<dd>
\anchor STARPU_TCPIP_MS_MASTER
\addindex __env__STARPU_TCPIP_MS_MASTER
Specify (for slaves) the IP address of the master so they can connect to
it. They will then automatically connect to each other.
</dd>

<dt>STARPU_TCPIP_MS_PORT</dt>
<dd>
\anchor STARPU_TCPIP_MS_PORT
\addindex __env__STARPU_TCPIP_MS_PORT
Specify the port of the master, for connexions between slaves and the master.
Default value is 1234.
</dd>

<dt>STARPU_NTCPIPMSTHREADS</dt>
<dd>
\anchor STARPU_NTCPIPMSTHREADS
\addindex __env__STARPU_NTCPIPMSTHREADS
Specify the number of threads to use on the TCP/IP Slave devices.
</dd>

<dt>STARPU_TCPIP_MS_MULTIPLE_THREAD</dt>
<dd>
\anchor STARPU_TCPIP_MS_MULTIPLE_THREAD
\addindex __env__STARPU_TCPIP_MS_MULTIPLE_THREAD
Specify whether the master should use one thread per slave, or one thread for
driver all slaves. Default value is 0.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_TCPIP_MS_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_TCPIP_MS_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_TCPIP_MS_COPY
Disable asynchronous copies between CPU and TCP/IP Slave devices. One can call starpu_asynchronous_tcpip_ms_copy_disabled() to check whether asynchronous data transfers between CPU and TCP/IP Slave devices are disabled.
</dd>

</dl>

\subsection hipWorkers HIP Workers

<dl>
<dt>STARPU_NHIP</dt>
<dd>
\anchor STARPU_NHIP
\addindex __env__STARPU_NHIP
Specify the number of HIP devices that StarPU can use. If
\ref STARPU_NHIP is lower than the number of physical devices, it is
possible to select which HIP devices should be used by the means of the
environment variable \ref STARPU_WORKERS_HIPID. By default, StarPU will
create as many HIP workers as there are HIP devices.
</dd>

<dt>STARPU_WORKERS_HPIID</dt>
<dd>
\anchor STARPU_WORKERS_HIPID
\addindex __env__STARPU_WORKERS_HIPID
Select which HIP devices should be used to run HIP workers (similarly to the \ref STARPU_WORKERS_HIPID environment variable).
On a machine equipped with 4 HIP devices, setting <c>STARPU_WORKERS_HIPID="1 3"</c> and
<c>STARPU_NHIP=2</c> specifies that 2 HIP workers should be created, and that
they should use HIP devices #1 and #3.

This variable is ignored if the field
starpu_conf::use_explicit_workers_hip_gpuid passed to starpu_init()
is set.
</dd>

<dt>STARPU_DISABLE_ASYNCHRONOUS_HIP_COPY</dt>
<dd>
\anchor STARPU_DISABLE_ASYNCHRONOUS_HIP_COPY
\addindex __env__STARPU_DISABLE_ASYNCHRONOUS_HIP_COPY
Disable asynchronous copies between CPU and HIP devices. One can call starpu_asynchronous_hip_copy_disabled() to check whether asynchronous data transfers between CPU and HIP accelerators are disabled.
</dd>

</dl>


\subsection mpiConf MPI Configuration
<dl>

<dt>STARPU_MPI_THREAD_CPUID</dt>
<dd>
\anchor STARPU_MPI_THREAD_CPUID
\addindex __env__STARPU_MPI_THREAD_CPUID
Tell StarPU to bind its MPI thread to the given CPU id,
subtracted from the CPU workers (unless \ref STARPU_NCPU is defined).

Default value is -1, it will let StarPU allocate a CPU.
</dd>

<dt>STARPU_MPI_THREAD_COREID</dt>
<dd>
\anchor STARPU_MPI_THREAD_COREID
\addindex __env__STARPU_MPI_THREAD_COREID
Same as \ref STARPU_MPI_THREAD_CPUID, but bind the MPI thread to the given core
ID, instead of the PU (hyperthread).
</dd>

<dt>STARPU_MPI_NOBIND</dt>
<dd>
\anchor STARPU_MPI_NOBIND
\addindex __env__STARPU_MPI_NOBIND
Setting it to non-zero will prevent StarPU from binding the MPI to
a separate core. This is for instance useful when running the testsuite on a single system.
</dd>

<dt>STARPU_MPI_GPUDIRECT</dt>
<dd>
\anchor STARPU_MPI_GPUDIRECT
\addindex __env__STARPU_MPI_GPUDIRECT
Enable (1) or disable (0) MPI GPUDirect support. Default value (-1) is to enable if available.
If \ref STARPU_MPI_GPUDIRECT is explicitly set to 1, StarPU-MPI will warn if MPI does not provide the GPUDirect support.
</dd>

<dt>STARPU_MPI_PSM2</dt>
<dd>
\anchor STARPU_MPI_PSM2
\addindex __env__STARPU_MPI_PSM2
This variable allows to supercede PSM2 detection when asking for MPI GPUDirect support.
This is helpful when using old intel compilers, for which PSM2 detection is always true.
The default (1) is to enable it. If PSM2 is detected whereas it should not be, this variable
can be set to 0.
</dd>

<dt>STARPU_MPI_REDUX_ARITY_THRESHOLD</dt>
<dd>
\anchor STARPU_MPI_REDUX_ARITY_THRESHOLD
\addindex __env__STARPU_MPI_REDUX_ARITY_THRESHOLD
The arity of the automatically-detected reduction trees follows the following
rule: when the data to be reduced is of small size a flat tree is unrolled
i.e. all the contributing nodes send their contribution to the root of
the reduction. When the data to be reduced is of big size, a binary tree is used instead.
The default threshold between flat and binary tree is 1024 bytes. By setting the environment
variable with a negative value, all the automatically detected reduction trees will use flat trees.
If this value is set to 0, then binary trees will always be selected. Otherwise,
the setup value replaces the default 1024.
</dd>

</dl>

\section ConfiguringTheSchedulingEngine Configuring The Scheduling Engine

<dl>

<dt>STARPU_SCHED</dt>
<dd>
\anchor STARPU_SCHED
\addindex __env__STARPU_SCHED
Select the scheduling policy from those proposed by StarPU: work
random, stealing, greedy, with performance models, etc.

Use <c>STARPU_SCHED=help</c> to get the list of available schedulers.
</dd>

<dt>STARPU_SCHED_LIB</dt>
<dd>
\anchor STARPU_SCHED_LIB
\addindex __env__STARPU_SCHED_LIB
Specify the location of a dynamic library to choose a
user-defined scheduling policy. See \ref UsingaNewSchedulingPolicy for
more information.
</dd>

<dt>STARPU_MIN_PRIO</dt>
<dd>
\anchor STARPU_MIN_PRIO_env
\addindex __env__STARPU_MIN_PRIO
Set the minimum priority used by priorities-aware schedulers.
The flag can also be set through the field starpu_conf::global_sched_ctx_min_priority.
</dd>

<dt>STARPU_MAX_PRIO</dt>
<dd>
\anchor STARPU_MAX_PRIO_env
\addindex __env__STARPU_MAX_PRIO
Set the maximum priority used by priorities-aware schedulers.
The flag can also be set through the field starpu_conf::global_sched_ctx_max_priority.
</dd>

<dt>STARPU_CALIBRATE</dt>
<dd>
\anchor STARPU_CALIBRATE
\addindex __env__STARPU_CALIBRATE
Set to 1 to calibrate the performance models during the execution.
Set to 2 to drop the previous values and restart the calibration from scratch.
Set to 0 to disable calibration, this is the default behaviour.

Note: this currently only applies to <c>dm</c> and <c>dmda</c> scheduling policies.
</dd>

<dt>STARPU_CALIBRATE_MINIMUM</dt>
<dd>
\anchor STARPU_CALIBRATE_MINIMUM
\addindex __env__STARPU_CALIBRATE_MINIMUM
Define the minimum number of calibration measurements that will be made
before considering that the performance model is calibrated.  Default value is 10.
</dd>

<dt>STARPU_BUS_CALIBRATE</dt>
<dd>
\anchor STARPU_BUS_CALIBRATE
\addindex __env__STARPU_BUS_CALIBRATE
Set to 1 to recalibrate the bus during initialization.
</dd>

<dt>STARPU_PREFETCH</dt>
<dd>
\anchor STARPU_PREFETCH
\addindex __env__STARPU_PREFETCH
Enable (1) or disable (0) data prefetching. Default value is Enable.

If prefetching is enabled, when a task is scheduled to be
executed e.g. on a GPU, StarPU will request an asynchronous transfer in
advance, so that data is already present on the GPU when the task starts. As a
result, computation and data transfers are overlapped.
</dd>

<dt>STARPU_SCHED_ALPHA</dt>
<dd>
\anchor STARPU_SCHED_ALPHA
\addindex __env__STARPU_SCHED_ALPHA
To estimate the cost of a task StarPU takes into account the estimated
computation time (obtained thanks to performance models). The alpha factor is
the coefficient to be applied to it before adding it to the communication part.
</dd>

<dt>STARPU_SCHED_BETA</dt>
<dd>
\anchor STARPU_SCHED_BETA
\addindex __env__STARPU_SCHED_BETA
To estimate the cost of a task StarPU takes into account the estimated
data transfer time (obtained thanks to performance models). The beta factor is
the coefficient to be applied to it before adding it to the computation part.
</dd>

<dt>STARPU_SCHED_GAMMA</dt>
<dd>
\anchor STARPU_SCHED_GAMMA
\addindex __env__STARPU_SCHED_GAMMA
Define the execution time penalty of a joule (\ref Energy-basedScheduling).
</dd>

<dt>STARPU_SCHED_READY</dt>
<dd>
\anchor STARPU_SCHED_READY
\addindex __env__STARPU_SCHED_READY
For a modular scheduler with sorted queues below the decision component, workers
pick up a task which has most of its data already available. Setting this to 0
disables this.
</dd>

<dt>STARPU_SCHED_SORTED_ABOVE</dt>
<dd>
\anchor STARPU_SCHED_SORTED_ABOVE
\addindex __env__STARPU_SCHED_SORTED_ABOVE
For a modular scheduler with queues above the decision component, it is
usually sorted by priority. Setting this to 0 disables this.
</dd>

<dt>STARPU_SCHED_SORTED_BELOW</dt>
<dd>
\anchor STARPU_SCHED_SORTED_BELOW
\addindex __env__STARPU_SCHED_SORTED_BELOW
For a modular scheduler with queues below the decision component, they are
usually sorted by priority. Setting this to 0 disables this.
</dd>

<dt>STARPU_IDLE_POWER</dt>
<dd>
\anchor STARPU_IDLE_POWER
\addindex __env__STARPU_IDLE_POWER
Define the idle power of the machine (\ref Energy-basedScheduling).
</dd>

<dt>STARPU_PROFILING</dt>
<dd>
\anchor STARPU_PROFILING
\addindex __env__STARPU_PROFILING
Enable on-line performance monitoring (\ref EnablingOn-linePerformanceMonitoring).
</dd>

<dt>STARPU_CODELET_PROFILING</dt>
<dd>
\anchor STARPU_CODELET_PROFILING
\addindex __env__STARPU_CODELET_PROFILING
Enable on-line performance monitoring of codelets (\ref Per-codeletFeedback).
(enabled by default)
</dd>

<dt>STARPU_ENERGY_PROFILING</dt>
<dd>
\anchor STARPU_ENERGY_PROFILING
\addindex __env__STARPU_ENERGY_PROFILING
Enable on-line energy monitoring of tasks (\ref Per-codeletFeedback).
(disabled by default)
</dd>

<dt>STARPU_PROF_PAPI_EVENTS</dt>
<dd>
\anchor STARPU_PROF_PAPI_EVENTS
\addindex __env__STARPU_PROF_PAPI_EVENTS
Specify which PAPI events should be recorded in the trace (\ref PapiCounters).
</dd>

</dl>

\section ConfiguringHeteroprio Configuring The Heteroprio Scheduler

\subsection ConfiguringLaHeteroprio Configuring LAHeteroprio
<dl>

<dt>STARPU_HETEROPRIO_USE_LA</dt>
<dd>
\anchor STARPU_HETEROPRIO_USE_LA
\addindex __env__STARPU_HETEROPRIO_USE_LA
Enable the locality aware mode of Heteroprio which guides the distribution of tasks to workers
in order to reduce the data transfers between memory nodes.
</dd>

<dt>STARPU_LAHETEROPRIO_PUSH</dt>
<dd>
\anchor STARPU_LAHETEROPRIO_PUSH
\addindex __env__STARPU_LAHETEROPRIO_PUSH
Choose between the different push strategies for locality aware Heteroprio:
\c WORKER, \c LcS, \c LS_SDH, \c LS_SDH2, \c LS_SDHB, \c LC_SMWB, \c AUTO (by default: AUTO). These are detailed in
\ref LAHeteroprio
</dd>

<dt>STARPU_LAHETEROPRIO_S_[ARCH]</dt>
<dd>
\anchor STARPU_LAHETEROPRIO_S_[ARCH]
\addindex __env__STARPU_LAHETEROPRIO_S_arch
Specify the number of memory nodes contained in an affinity group. An affinity
group will be composed of the closest memory nodes to a worker of a given architecture,
and this worker will look for tasks available inside these memory nodes, before
considering stealing tasks outside this group.
ARCH can be \c CPU, \c CUDA, \c OPENCL, \c SCC, \c MPI_MS, etc.
</dd>

<dt>STARPU_LAHETEROPRIO_PRIO_STEP_[ARCH]</dt>
<dd>
\anchor STARPU_LAHETEROPRIO_PRIO_STEP_[ARCH]
\addindex __env__STARPU_LAHETEROPRIO_PRIO_STEP_arch
Specify the number of buckets in the local memory node in which a worker will look for
available tasks, before this worker starts looking for tasks in other memory nodes' buckets.
ARCH indicates that this number is specific to a given arch which can be:
\c CPU, \c CUDA, \c OPENCL, \c SCC, \c MPI_MS, etc.
</dd>

</dl>

\subsection ConfiguringAutoHeteroprio Configuring AutoHeteroprio
<dl>

<dt>STARPU_HETEROPRIO_USE_AUTO_CALIBRATION</dt>
<dd>
\anchor STARPU_HETEROPRIO_USE_AUTO_CALIBRATION
\addindex __env__STARPU_HETEROPRIO_USE_AUTO_CALIBRATION
Enable the auto calibration mode of Heteroprio which assign priorities to tasks automatically
</dd>

<dt>STARPU_HETEROPRIO_DATA_DIR</dt>
<dd>
\anchor STARPU_HETEROPRIO_DATA_DIR
\addindex __env__STARPU_HETEROPRIO_DATA_DIR
Specify the path of the directory where Heteroprio stores data about program executions.
By default, these are stored in the same directory used by perfmodel.
</dd>

<dt>STARPU_HETEROPRIO_DATA_FILE</dt>
<dd>
\anchor STARPU_HETEROPRIO_DATA_FILE
\addindex __env__STARPU_HETEROPRIO_DATA_FILE
Specify the filename where Heteroprio will save data about the current program's execution.
</dd>

<dt>STARPU_HETEROPRIO_CODELET_GROUPING_STRATEGY</dt>
<dd>
\anchor STARPU_HETEROPRIO_CODELET_GROUPING_STRATEGY
\addindex __env__STARPU_HETEROPRIO_CODELET_GROUPING_STRATEGY
Choose how Heteroprio groups similar tasks. It can be <c>0</c> to group
the tasks with the same perfmodel or the same codelet's name if no perfmodel was assigned.
Or, it could be <c>1</c> to group the tasks only by codelet's name.
</dd>

<dt>STARPU_AUTOHETEROPRIO_PRINT_DATA_ON_UPDATE</dt>
<dd>
\anchor STARPU_AUTOHETEROPRIO_PRINT_DATA_ON_UPDATE
\addindex __env__STARPU_AUTOHETEROPRIO_PRINT_DATA_ON_UPDATE
Enable the printing of priorities' data every time they get updated.
</dd>

<dt>STARPU_AUTOHETEROPRIO_PRINT_AFTER_ORDERING</dt>
<dd>
\anchor STARPU_AUTOHETEROPRIO_PRINT_AFTER_ORDERING
\addindex __env__STARPU_AUTOHETEROPRIO_PRINT_AFTER_ORDERING
Enable the printing of priorities' order for each architecture every time there's a reordering.
</dd>

<dt>STARPU_AUTOHETEROPRIO_PRIORITY_ORDERING_POLICY</dt>
<dd>
\anchor STARPU_AUTOHETEROPRIO_PRIORITY_ORDERING_POLICY
\addindex __env__STARPU_AUTOHETEROPRIO_PRIORITY_ORDERING_POLICY
Specify the heuristic which will be used to assign priorities automatically.
It should be an integer between 0 and 27.
</dd>

<dt>STARPU_AUTOHETEROPRIO_ORDERING_INTERVAL</dt>
<dd>
\anchor STARPU_AUTOHETEROPRIO_ORDERING_INTERVAL
\addindex __env__STARPU_AUTOHETEROPRIO_ORDERING_INTERVAL
Specify the period (in number of tasks pushed), between priorities reordering operations.
</dd>

<dt>STARPU_AUTOHETEROPRIO_FREEZE_GATHERING</dt>
<dd>
\anchor STARPU_AUTOHETEROPRIO_FREEZE_GATHERING
\addindex __env__STARPU_AUTOHETEROPRIO_FREEZE_GATHERING
Disable data gathering from task executions.
</dd>

</dl>

\section Extensions Extensions

<dl>

<dt>SOCL_OCL_LIB_OPENCL</dt>
<dd>
\anchor SOCL_OCL_LIB_OPENCL
\addindex __env__SOCL_OCL_LIB_OPENCL
Set the location of the file <c>libOpenCL.so</c> of the OCL ICD implementation.
The SOCL test suite is only run when \ref SOCL_OCL_LIB_OPENCL is defined.
</dd>

<dt>OCL_ICD_VENDORS</dt>
<dd>
\anchor OCL_ICD_VENDORS
\addindex __env__OCL_ICD_VENDORS
Set the directory where ICD files are installed. This is useful when using SOCL
with OpenCL ICD (https://forge.imag.fr/projects/ocl-icd/). Default
directory is <c>/etc/OpenCL/vendors</c>. StarPU installs ICD
files in the directory <c>$prefix/share/starpu/opencl/vendors</c>.
</dd>

<dt>STARPU_COMM_STATS</dt>
<dd>
\anchor STARPU_COMM_STATS
\addindex __env__STARPU_COMM_STATS
Deprecated. You should use \ref STARPU_MPI_STATS.
</dd>

<dt>STARPU_MPI_STATS</dt>
<dd>
\anchor STARPU_MPI_STATS
\addindex __env__STARPU_MPI_STATS
Enable (!= 0) or Disable (0) communication statistics for starpumpi (\ref MPIDebug). Default value is Disable.
</dd>

<dt>STARPU_MPI_CACHE</dt>
<dd>
\anchor STARPU_MPI_CACHE
\addindex __env__STARPU_MPI_CACHE
Disable (0) or Enable (!= 0) communication cache for starpumpi (\ref MPISupport). Default value is Enable.
</dd>

<dt>STARPU_MPI_COMM</dt>
<dd>
\anchor STARPU_MPI_COMM
\addindex __env__STARPU_MPI_COMM
Enable (1) communication trace for starpumpi (\ref MPISupport). Also needs for StarPU to
have been configured with the option \ref enable-verbose "--enable-verbose".
</dd>

<dt>STARPU_MPI_CACHE_STATS</dt>
<dd>
\anchor STARPU_MPI_CACHE_STATS
\addindex __env__STARPU_MPI_CACHE_STATS
Enable (1) statistics for the communication cache (\ref MPISupport).
Messages are printed on the standard output when data are added or removed from the received
communication cache.
</dd>

<dt>STARPU_MPI_PRIORITIES</dt>
<dd>
\anchor STARPU_MPI_PRIORITIES
\addindex __env__STARPU_MPI_PRIORITIES
Disable (0) the use of priorities to order MPI communications (\ref MPISupport).
</dd>

<dt>STARPU_MPI_NDETACHED_SEND</dt>
<dd>
\anchor STARPU_MPI_NDETACHED_SEND
\addindex __env__STARPU_MPI_NDETACHED_SEND
Set the number of send requests that StarPU-MPI will emit concurrently.
Default value is 10. Setting it to 0 removes the limit of concurrent send
requests.
</dd>

<dt>STARPU_MPI_NREADY_PROCESS</dt>
<dd>
\anchor STARPU_MPI_NREADY_PROCESS
\addindex __env__STARPU_MPI_NREADY_PROCESS
Set the number of requests that StarPU-MPI will submit to MPI before
polling for termination of existing requests. Default value is 10. Setting it to
0 removes the limit: all requests to submit to MPI will be submitted before
polling for termination of existing ones.
</dd>

<dt>STARPU_MPI_FAKE_SIZE</dt>
<dd>
\anchor STARPU_MPI_FAKE_SIZE
\addindex __env__STARPU_MPI_FAKE_SIZE
Setting to a number makes StarPU believe that there are as many MPI nodes, even
if it was run on only one MPI node. This allows e.g. to simulate the execution
of one of the nodes of a big cluster without actually running the rest.
Of course, it does not provide computation results and timing.
</dd>

<dt>STARPU_MPI_FAKE_RANK</dt>
<dd>
\anchor STARPU_MPI_FAKE_RANK
\addindex __env__STARPU_MPI_FAKE_RANK
Setting to a number makes StarPU believe that it runs the given MPI node, even
if it was run on only one MPI node. This allows e.g. to simulate the execution
of one of the nodes of a big cluster without actually running the rest.
Of course, it does not provide computation results and timing.
</dd>

<dt>STARPU_MPI_COOP_SENDS</dt>
<dd>
\anchor STARPU_MPI_COOP_SENDS
\addindex __env__STARPU_MPI_COOP_SENDS
Disable (0) dynamic collective operations: grouping same requests to
different nodes until the data becomes available and then use a broadcast tree
to execute requests.<br>
By now, it is only supported with the NewMadeleine library (see \ref Nmad).
</dd>

<dt>STARPU_MPI_RECV_WAIT_FINALIZE</dt>
<dd>
\anchor STARPU_MPI_RECV_WAIT_FINALIZE
\addindex __env__STARPU_MPI_RECV_WAIT_FINALIZE
Disable (1) releasing the write acquire of receiving handles when
data is received but the communication library still needs the data. Set to 0
by default to unlock as soon as possible tasks which only require a read access
on the handle; write access will become possible for tasks when the
communication library will not need the data anymore.<br/>
By now, it is only supported with the NewMadeleine library (see \ref Nmad).
</dd>

<dt>STARPU_MPI_TRACE_SYNC_CLOCKS</dt>
<dd>
\anchor STARPU_MPI_TRACE_SYNC_CLOCKS
\addindex __env__STARPU_MPI_TRACE_SYNC_CLOCKS
When \c mpi_sync_clocks is available, this library will be used to have more
precise clock synchronization in traces coming from different nodes. However,
the clock synchronization process can take some time (several seconds) and can
be disabled by setting this variable to \c 0. In that case, a less precise but
faster synchronization will be used. See \ref TraceMpi for more details.
</dd>

<dt>STARPU_MPI_DRIVER_CALL_FREQUENCY</dt>
<dd>
\anchor STARPU_MPI_DRIVER_CALL_FREQUENCY
\addindex __env__STARPU_MPI_DRIVER_CALL_FREQUENCY
When set to a positive value, activates the interleaving of the execution of
tasks with the progression of MPI communications (\ref MPISupport). The
starpu_mpi_init_conf() function must have been called by the application
for that environment variable to be used. When set to 0, the MPI progression
thread does not use at all the driver given by users, and only focuses on
making MPI communications progress.
</dd>

<dt>STARPU_MPI_DRIVER_TASK_FREQUENCY</dt>
<dd>
\anchor STARPU_MPI_DRIVER_TASK_FREQUENCY
\addindex __env__STARPU_MPI_DRIVER_TASK_FREQUENCY
When set to a positive value, the interleaving of the execution of tasks with
the progression of MPI communications mechanism to execute several tasks before
checking communication requests again (\ref MPISupport). The
starpu_mpi_init_conf() function must have been called by the application
for that environment variable to be used, and the
\ref STARPU_MPI_DRIVER_CALL_FREQUENCY environment variable set to a positive value.
</dd>

<dt>STARPU_MPI_MEM_THROTTLE</dt>
<dd>
\anchor STARPU_MPI_MEM_THROTTLE
\addindex __env__STARPU_MPI_MEM_THROTTLE
When set to a positive value, this makes the starpu_mpi_*recv* functions
block when the memory allocation required for network reception overflows the
available main memory (as typically set by \ref STARPU_LIMIT_CPU_MEM)
</dd>

<dt>STARPU_MPI_EARLYDATA_ALLOCATE</dt>
<dd>
\anchor STARPU_MPI_EARLYDATA_ALLOCATE
\addindex __env__STARPU_MPI_EARLYDATA_ALLOCATE
When set to 1, the MPI Driver will immediately allocate the data for early
requests instead of issuing a data request and blocking. Default value is 0,
issuing a data request. Because it is an early request and we do not know its
real priority, the data request will assume \ref STARPU_DEFAULT_PRIO. In cases
where there are many data requests with priorities greater than
\ref STARPU_DEFAULT_PRIO the MPI drive could be blocked for long periods.
</dd>

<dt>STARPU_SIMGRID</dt>
<dd>
\anchor STARPU_SIMGRID
\addindex __env__STARPU_SIMGRID
When set to 1 (default value is 0), this makes StarPU check that it was really
build with simulation support. This is convenient in scripts to avoid using a
native version, that would try to update performance models...
</dd>

<dt>STARPU_SIMGRID_TRANSFER_COST</dt>
<dd>
\anchor STARPU_SIMGRID_TRANSFER_COST
\addindex __env__STARPU_SIMGRID_TRANSFER_COST
When set to 1 (which is the default value), data transfers (over PCI bus, typically) are taken into account
in SimGrid mode.
</dd>

<dt>STARPU_SIMGRID_CUDA_MALLOC_COST</dt>
<dd>
\anchor STARPU_SIMGRID_CUDA_MALLOC_COST
\addindex __env__STARPU_SIMGRID_CUDA_MALLOC_COST
When set to 1 (which is the default value), CUDA malloc costs are taken into account
in SimGrid mode.
</dd>

<dt>STARPU_SIMGRID_CUDA_QUEUE_COST</dt>
<dd>
\anchor STARPU_SIMGRID_CUDA_QUEUE_COST
\addindex __env__STARPU_SIMGRID_CUDA_QUEUE_COST
When set to 1 (which is the default value), CUDA task and transfer queueing costs are
taken into account in SimGrid mode.
</dd>

<dt>STARPU_PCI_FLAT</dt>
<dd>
\anchor STARPU_PCI_FLAT
\addindex __env__STARPU_PCI_FLAT
When unset or set to 0, the platform file created for SimGrid will
contain PCI bandwidths and routes.
</dd>

<dt>STARPU_SIMGRID_CUDA_QUEUE_COST</dt>
<dd>
\anchor STARPU_SIMGRID_CUDA_QUEUE_COST
\addindex __env__STARPU_SIMGRID_CUDA_QUEUE_COST
When unset or set to 1, simulate within SimGrid the GPU transfer queueing.
</dd>

<dt>STARPU_MALLOC_SIMULATION_FOLD</dt>
<dd>
\anchor STARPU_MALLOC_SIMULATION_FOLD
\addindex __env__STARPU_MALLOC_SIMULATION_FOLD
Define the size of the file used for folding virtual allocation, in
MiB. Default value is 1, thus allowing 64GiB virtual memory when Linux's
<c>sysctl vm.max_map_count</c> value is the default 65535.
</dd>

<dt>STARPU_SIMGRID_TASK_SUBMIT_COST</dt>
<dd>
\anchor STARPU_SIMGRID_TASK_SUBMIT_COST
\addindex __env__STARPU_SIMGRID_TASK_SUBMIT_COST
When set to 1 (which is the default value), task submission costs are taken into
account in SimGrid mode. This provides more accurate SimGrid predictions,
especially for the beginning of the execution.
</dd>

<dt>STARPU_SIMGRID_TASK_PUSH_COST</dt>
<dd>
\anchor STARPU_SIMGRID_TASK_PUSH_COST
\addindex __env__STARPU_SIMGRID_TASK_PUSH_COST
When set to 1 (which is the default value), task push costs are taken into
account in SimGrid mode. This provides more accurate SimGrid predictions,
especially with large dependency arities.
</dd>

<dt>STARPU_SIMGRID_FETCHING_INPUT_COST</dt>
<dd>
\anchor STARPU_SIMGRID_FETCHING_INPUT_COST
\addindex __env__STARPU_SIMGRID_FETCHING_INPUT_COST
When set to 1 (which is the default value), fetching input costs are taken into
account in SimGrid mode. This provides more accurate SimGrid predictions,
especially regarding data transfers.
</dd>

<dt>STARPU_SIMGRID_SCHED_COST</dt>
<dd>
\anchor STARPU_SIMGRID_SCHED_COST
\addindex __env__STARPU_SIMGRID_SCHED_COST
When set to 1 (0 is the default value), scheduling costs are taken into
account in SimGrid mode. This provides more accurate SimGrid predictions,
and allows studying scheduling overhead of the runtime system. However,
it also makes simulation non-deterministic.
</dd>

<dt>STARPUPY_MULTI_INTERPRETER</dt>
<dd>
\anchor STARPUPY_MULTI_INTERPRETER
\addindex __env__STARPUPY_MULTI_INTERPRETER
Enable (1) or disable (0) multi interpreters in the StarPU Python interface (\ref MultipleInterpreters). Default value is Disable.
</dd>

<dt>STARPUPY_OWN_GIL</dt>
<dd>
\anchor STARPUPY_OWN_GIL
\addindex __env__STARPUPY_OWN_GIL
Enable (1) or disable (0) using per-interpreter GIL (\ref
PythonParallelism). Default value is Disable for now, until python is fully
ready for this.
</dd>

</dl>

\section MiscellaneousAndDebug Miscellaneous And Debug

<dl>

<dt>STARPU_HOME</dt>
<dd>
\anchor STARPU_HOME
\addindex __env__STARPU_HOME
Specify the main directory in which StarPU stores its
configuration files. Default value is <c>$HOME</c> on Unix environments,
and <c>$USERPROFILE</c> on Windows environments.
</dd>

<dt>STARPU_PATH</dt>
<dd>
\anchor STARPU_PATH
\addindex __env__STARPU_PATH
Only used on  Windows environments.
Specify the main directory in which StarPU is installed
(\ref RunningABasicStarPUApplicationOnMicrosoft)
</dd>

<dt>STARPU_PERF_MODEL_DIR</dt>
<dd>
\anchor STARPU_PERF_MODEL_DIR
\addindex __env__STARPU_PERF_MODEL_DIR
Specify the main directory in which StarPU stores its
performance model files. Default value is <c>$STARPU_HOME/.starpu/sampling</c>.
See \ref Storing_Performance_Model_Files for more details.
</dd>

<dt>STARPU_PERF_MODEL_PATH</dt>
<dd>
\anchor STARPU_PERF_MODEL_PATH
\addindex __env__STARPU_PERF_MODEL_PATH
Specify a list of directories separated with ':' in which StarPU stores its
performance model files.
See \ref Storing_Performance_Model_Files for more details.
</dd>

<dt>STARPU_PERF_MODEL_HOMOGENEOUS_CPU</dt>
<dd>
\anchor STARPU_PERF_MODEL_HOMOGENEOUS_CPU
\addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CPU
When set to 0, StarPU will assume that CPU devices do not have the same
performance, and thus use different performance models for them, thus making
kernel calibration much longer, since measurements have to be made for each CPU
core.
</dd>

<dt>STARPU_PERF_MODEL_HOMOGENEOUS_CUDA</dt>
<dd>
\anchor STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
\addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
When set to 1, StarPU will assume that all CUDA devices have the same
performance, and thus share performance models for them, thus allowing kernel
calibration to be much faster, since measurements only have to be once for all
CUDA GPUs.
</dd>

<dt>STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL</dt>
<dd>
\anchor STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
\addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
When set to 1, StarPU will assume that all OpenCL devices have the same
performance, and thus share performance models for them, thus allowing kernel
calibration to be much faster, since measurements only have to be once for all
OpenCL GPUs.
</dd>

<dt>STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS</dt>
<dd>
\anchor STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
\addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
When set to 1, StarPU will assume that all MPI Slave devices have the same
performance, and thus share performance models for them, thus allowing kernel
calibration to be much faster, since measurements only have to be once for all
MPI Slaves.
</dd>

<dt>STARPU_HOSTNAME</dt>
<dd>
\anchor STARPU_HOSTNAME
\addindex __env__STARPU_HOSTNAME
When set, force the hostname to be used when managing performance model
files. Models are indexed by machine name. When running for example on
a homogenenous cluster, it is possible to share the models between
machines by setting <c>export STARPU_HOSTNAME=some_global_name</c>.
</dd>

<dt>STARPU_MPI_HOSTNAMES</dt>
<dd>
\anchor STARPU_MPI_HOSTNAMES
\addindex __env__STARPU_MPI_HOSTNAMES
Similar to \ref STARPU_HOSTNAME but to define multiple nodes on a
heterogeneous cluster. The variable is a list of hostnames that will be assigned
to each StarPU-MPI rank considering their position and the value of
starpu_mpi_world_rank() on each rank. When running, for example, on a
heterogeneous cluster, it is possible to set individual models for each machine
by setting <c>export STARPU_MPI_HOSTNAMES="name0 name1 name2"</c>. Where rank 0
will receive \c name0, rank1 will receive \c name1, and so on.
This variable has precedence over \ref STARPU_HOSTNAME.
</dd>

<dt>STARPU_OPENCL_PROGRAM_DIR</dt>
<dd>
\anchor STARPU_OPENCL_PROGRAM_DIR
\addindex __env__STARPU_OPENCL_PROGRAM_DIR
Specify the directory where the OpenCL codelet source files are
located. The function starpu_opencl_load_program_source() looks
for the codelet in the current directory, in the directory specified
by the environment variable \ref STARPU_OPENCL_PROGRAM_DIR, in the
directory <c>share/starpu/opencl</c> of the installation directory of
StarPU, and finally in the source directory of StarPU.
</dd>

<dt>STARPU_SILENT</dt>
<dd>
\anchor STARPU_SILENT
\addindex __env__STARPU_SILENT
Disable verbose mode at runtime when StarPU
has been configured with the option \ref enable-verbose "--enable-verbose". Also
disable the display of StarPU information and warning messages.
</dd>

<dt>STARPU_MPI_DEBUG_LEVEL_MIN</dt>
<dd>
\anchor STARPU_MPI_DEBUG_LEVEL_MIN
\addindex __env__STARPU_MPI_DEBUG_LEVEL_MIN
Set the minimum level of debug when StarPU
has been configured with the option \ref enable-mpi-verbose "--enable-mpi-verbose".
</dd>

<dt>STARPU_MPI_DEBUG_LEVEL_MAX</dt>
<dd>
\anchor STARPU_MPI_DEBUG_LEVEL_MAX
\addindex __env__STARPU_MPI_DEBUG_LEVEL_MAX
Set the maximum level of debug when StarPU
has been configured with the option \ref enable-mpi-verbose "--enable-mpi-verbose".
</dd>

<dt>STARPU_LOGFILENAME</dt>
<dd>
\anchor STARPU_LOGFILENAME
\addindex __env__STARPU_LOGFILENAME
Specify in which file the debugging output should be saved to.
</dd>

<dt>STARPU_FXT_PREFIX</dt>
<dd>
\anchor STARPU_FXT_PREFIX
\addindex __env__STARPU_FXT_PREFIX
Specify in which directory to save the generated trace if FxT is enabled.
</dd>

<dt>STARPU_FXT_SUFFIX</dt>
<dd>
\anchor STARPU_FXT_SUFFIX
\addindex __env__STARPU_FXT_SUFFIX
Specify in which file to save the generated trace if FxT is enabled.
</dd>

<dt>STARPU_FXT_TRACE</dt>
<dd>
\anchor STARPU_FXT_TRACE
\addindex __env__STARPU_FXT_TRACE
Enable (1) or disable (0) the FxT trace generation in \c /tmp/prof_file_XXX_YYY (the directory and file name can be changed with \ref STARPU_FXT_PREFIX and \ref STARPU_FXT_SUFFIX). Default value is Disable.
</dd>

<dt>STARPU_FXT_EVENTS</dt>
<dd>
\anchor STARPU_FXT_EVENTS
\addindex __env__STARPU_FXT_EVENTS
Specify which events will be recorded in traces. By default, all events (but
<c>VERBOSE_EXTRA</c> ones) are recorded. One can set this variable to a comma-
or pipe-separated list of the following categories, to record only events
belonging to the selected categories:
- <c>USER</c>
- <c>TASK</c>
- <c>TASK_VERBOSE</c>
- <c>TASK_VERBOSE_EXTRA</c>
- <c>DATA</c>
- <c>DATA_VERBOSE</c>
- <c>WORKER</c>
- <c>WORKER_VERBOSE</c>
- <c>DSM</c>
- <c>DSM_VERBOSE</c>
- <c>SCHED</c>
- <c>SCHED_VERBOSE</c>
- <c>LOCK</c>
- <c>LOCK_VERBOSE</c>
- <c>EVENT</c>
- <c>EVENT_VERBOSE</c>
- <c>MPI</c>
- <c>MPI_VERBOSE</c>
- <c>MPI_VERBOSE_EXTRA</c>
- <c>HYP</c>
- <c>HYP_VERBOSE</c>
</dd>

The choice of which categories have to be recorded is a tradeoff between
required information for offline analyzis and acceptable overhead introduced
by tracing. For instance, to inspect with ViTE which tasks workers execute, one
has to at least select the <c>TASK</c> category.

Events in <code>VERBOSE_EXTRA</code> are very costly to record and can have an
important impact on application performances. This is why there are disabled by
default, and one has to explicitly select their categories using this variable
to record them.

<dt>STARPU_LIMIT_CUDA_devid_MEM</dt>
<dd>
\anchor STARPU_LIMIT_CUDA_devid_MEM
\addindex __env__STARPU_LIMIT_CUDA_devid_MEM
Specify the maximum number of megabytes that should be
available to the application on the CUDA device with the identifier
<c>devid</c>. This variable is intended to be used for experimental
purposes as it emulates devices that have a limited amount of memory.
When defined, the variable overwrites the value of the variable
\ref STARPU_LIMIT_CUDA_MEM.
</dd>

<dt>STARPU_LIMIT_CUDA_MEM</dt>
<dd>
\anchor STARPU_LIMIT_CUDA_MEM
\addindex __env__STARPU_LIMIT_CUDA_MEM
Specify the maximum number of megabytes that should be
available to the application on each CUDA devices. This variable is
intended to be used for experimental purposes as it emulates devices
that have a limited amount of memory.
</dd>

<dt>STARPU_LIMIT_OPENCL_devid_MEM</dt>
<dd>
\anchor STARPU_LIMIT_OPENCL_devid_MEM
\addindex __env__STARPU_LIMIT_OPENCL_devid_MEM
Specify the maximum number of megabytes that should be
available to the application on the OpenCL device with the identifier
<c>devid</c>. This variable is intended to be used for experimental
purposes as it emulates devices that have a limited amount of memory.
When defined, the variable overwrites the value of the variable
\ref STARPU_LIMIT_OPENCL_MEM.
</dd>

<dt>STARPU_LIMIT_OPENCL_MEM</dt>
<dd>
\anchor STARPU_LIMIT_OPENCL_MEM
\addindex __env__STARPU_LIMIT_OPENCL_MEM
Specify the maximum number of megabytes that should be
available to the application on each OpenCL devices. This variable is
intended to be used for experimental purposes as it emulates devices
that have a limited amount of memory.
</dd>

<dt>STARPU_LIMIT_HIP_devid_MEM</dt>
<dd>
\anchor STARPU_LIMIT_HIP_devid_MEM
\addindex __env__STARPU_LIMIT_HIP_devid_MEM
Specify the maximum number of megabytes that should be
available to the application on the HIP device with the identifier
<c>devid</c>. This variable is intended to be used for experimental
purposes as it emulates devices that have a limited amount of memory.
When defined, the variable overwrites the value of the variable
\ref STARPU_LIMIT_HIP_MEM.
</dd>

<dt>STARPU_LIMIT_HIP_MEM</dt>
<dd>
\anchor STARPU_LIMIT_HIP_MEM
\addindex __env__STARPU_LIMIT_HIP_MEM
Specify the maximum number of megabytes that should be
available to the application on each HIP devices. This variable is
intended to be used for experimental purposes as it emulates devices
that have a limited amount of memory.
</dd>

<dt>STARPU_LIMIT_CPU_MEM</dt>
<dd>
\anchor STARPU_LIMIT_CPU_MEM
\addindex __env__STARPU_LIMIT_CPU_MEM
Specify the maximum number of megabytes that should be
available to the application in the main CPU memory. Setting it enables allocation
cache in main memory. Setting it to zero lets StarPU overflow memory.

Note: for now not all StarPU allocations get throttled by this
parameter. Notably MPI reception are not throttled unless \ref
STARPU_MPI_MEM_THROTTLE is set to 1.
</dd>

<dt>STARPU_LIMIT_CPU_NUMA_devid_MEM</dt>
<dd>
\anchor STARPU_LIMIT_CPU_NUMA_devid_MEM
\addindex __env__STARPU_LIMIT_CPU_NUMA_devid_MEM
Specify the maximum number of megabytes that should be available to the
application on the NUMA node with the OS identifier <c>devid</c>.  Setting it
overrides the value of \ref STARPU_LIMIT_CPU_MEM.
</dd>

<dt>STARPU_LIMIT_CPU_NUMA_MEM</dt>
<dd>
\anchor STARPU_LIMIT_CPU_NUMA_MEM
\addindex __env__STARPU_LIMIT_CPU_NUMA_MEM
Specify the maximum number of megabytes that should be available to the
application on each NUMA node. This is the same as specifying that same amount
with \ref STARPU_LIMIT_CPU_NUMA_devid_MEM for each NUMA node number. The total
memory available to StarPU will thus be this amount multiplied by the number of
NUMA nodes used by StarPU. Any \ref STARPU_LIMIT_CPU_NUMA_devid_MEM additionally
specified will take over \ref STARPU_LIMIT_CPU_NUMA_MEM.
</dd>

<dt>STARPU_LIMIT_BANDWIDTH</dt>
<dd>
\anchor STARPU_LIMIT_BANDWIDTH
\addindex __env__STARPU_LIMIT_BANDWIDTH
Specify the maximum available PCI bandwidth of the system in MB/s. This can only
be effective with simgrid simulation. This allows to easily override the
bandwidths stored in the platform file generated from measurements on the native
system. This can thus be used accelerate or slow down the system bandwidth.
</dd>

<dt>STARPU_SUBALLOCATOR</dt>
<dd>
\anchor STARPU_SUBALLOCATOR
\addindex __env__STARPU_SUBALLOCATOR
Enable (1) or disable (0) the StarPU suballocator. Default value is to
enable it to amortize the cost of GPU and pinned RAM allocations for small
allocations: StarPU allocate large chunks of memory at a time, and suballocates
the small buffers within them.
</dd>

<dt>STARPU_MINIMUM_AVAILABLE_MEM</dt>
<dd>
\anchor STARPU_MINIMUM_AVAILABLE_MEM
\addindex __env__STARPU_MINIMUM_AVAILABLE_MEM
Specify the minimum percentage of memory that should be available in GPUs, i.e.
not used at all by StarPU (or in main memory, when using out of core), below
which a eviction pass is performed. Default value is 0%.
</dd>

<dt>STARPU_TARGET_AVAILABLE_MEM</dt>
<dd>
\anchor STARPU_TARGET_AVAILABLE_MEM
\addindex __env__STARPU_TARGET_AVAILABLE_MEM
Specify the target percentage of memory that should be available in GPUs, i.e.
not used at all by StarPU (or in main memory, when using out of core), when
performing a periodic eviction pass. Default value is 0%.
</dd>

<dt>STARPU_MINIMUM_CLEAN_BUFFERS</dt>
<dd>
\anchor STARPU_MINIMUM_CLEAN_BUFFERS
\addindex __env__STARPU_MINIMUM_CLEAN_BUFFERS
Specify the minimum percentage of number of buffers that should be clean in GPUs
(or in main memory, when using out of core), i.e. used by StarPU, but for which
a copy is available in memory (or on disk, when using out of core), below which
asynchronous writebacks will be issued. Default value is 5%.
</dd>

<dt>STARPU_TARGET_CLEAN_BUFFERS</dt>
<dd>
\anchor STARPU_TARGET_CLEAN_BUFFERS
\addindex __env__STARPU_TARGET_CLEAN_BUFFERS
Specify the target percentage of number of buffers that should be reached in
GPUs (or in main memory, when using out of core), i.e. used by StarPU, but for
which a copy is available in memory (or on disk, when using out of core), when
performing an asynchronous writeback pass. Default value is 10%.
</dd>

<dt>STARPU_DISK_SWAP</dt>
<dd>
\anchor STARPU_DISK_SWAP
\addindex __env__STARPU_DISK_SWAP
Specify a path where StarPU can push data when the main memory is getting
full.
</dd>

<dt>STARPU_DISK_SWAP_BACKEND</dt>
<dd>
\anchor STARPU_DISK_SWAP_BACKEND
\addindex __env__STARPU_DISK_SWAP_BACKEND
Specify the backend to be used by StarPU to push data when the main
memory is getting full. Default value is \c unistd (i.e. using read/write functions),
other values are \c stdio (i.e. using fread/fwrite), \c unistd_o_direct (i.e. using
read/write with O_DIRECT), \c leveldb (i.e. using a leveldb database), and \c hdf5
(i.e. using HDF5 library).
</dd>

<dt>STARPU_DISK_SWAP_SIZE</dt>
<dd>
\anchor STARPU_DISK_SWAP_SIZE
\addindex __env__STARPU_DISK_SWAP_SIZE
Specify the maximum size in MiB to be used by StarPU to push data when the main
memory is getting full. Default value is unlimited.
</dd>

<dt>STARPU_LIMIT_MAX_SUBMITTED_TASKS</dt>
<dd>
\anchor STARPU_LIMIT_MAX_SUBMITTED_TASKS
\addindex __env__STARPU_LIMIT_MAX_SUBMITTED_TASKS
Allow users to control the task submission flow by specifying
to StarPU a maximum number of submitted tasks allowed at a given time, i.e. when
this limit is reached task submission becomes blocking until enough tasks have
completed, specified by \ref STARPU_LIMIT_MIN_SUBMITTED_TASKS.
Setting it enables allocation cache buffer reuse in main memory.
See \ref HowToReduceTheMemoryFootprintOfInternalDataStructures.
</dd>

<dt>STARPU_LIMIT_MIN_SUBMITTED_TASKS</dt>
<dd>
\anchor STARPU_LIMIT_MIN_SUBMITTED_TASKS
\addindex __env__STARPU_LIMIT_MIN_SUBMITTED_TASKS
Allow users to control the task submission flow by specifying
to StarPU a submitted task threshold to wait before unblocking task submission. This
variable has to be used in conjunction with \ref STARPU_LIMIT_MAX_SUBMITTED_TASKS
which puts the task submission thread to
sleep.  Setting it enables allocation cache buffer reuse in main memory.
See \ref HowToReduceTheMemoryFootprintOfInternalDataStructures.
</dd>

<dt>STARPU_TRACE_BUFFER_SIZE</dt>
<dd>
\anchor STARPU_TRACE_BUFFER_SIZE
\addindex __env__STARPU_TRACE_BUFFER_SIZE
Set the buffer size for recording trace events in MiB. Setting it to a big
size allows to avoid pauses in the trace while it is recorded on the disk. This
however also consumes memory, of course. Default value is 64.
</dd>

<dt>STARPU_GENERATE_TRACE</dt>
<dd>
\anchor STARPU_GENERATE_TRACE
\addindex __env__STARPU_GENERATE_TRACE
When set to <c>1</c>, indicate that StarPU should automatically
generate a Paje trace when starpu_shutdown() is called.
</dd>

<dt>STARPU_GENERATE_TRACE_OPTIONS</dt>
<dd>
\anchor STARPU_GENERATE_TRACE_OPTIONS
\addindex __env__STARPU_GENERATE_TRACE_OPTIONS
When the variable \ref STARPU_GENERATE_TRACE is set to <c>1</c> to
generate a Paje trace, this variable can be set to specify options (see
<c>starpu_fxt_tool --help</c>).
</dd>

<dt>STARPU_ENABLE_STATS</dt>
<dd>
\anchor STARPU_ENABLE_STATS
\addindex __env__STARPU_ENABLE_STATS
Enable gathering various data statistics (\ref DataStatistics).
</dd>

<dt>STARPU_MEMORY_STATS</dt>
<dd>
\anchor STARPU_MEMORY_STATS
\addindex __env__STARPU_MEMORY_STATS
When set to 0, disable the display of memory statistics on data which
have not been unregistered at the end of the execution (\ref MemoryFeedback).
</dd>

<dt>STARPU_MAX_MEMORY_USE</dt>
<dd>
\anchor STARPU_MAX_MEMORY_USE
\addindex __env__STARPU_MAX_MEMORY_USE
When set to 1, display at the end of the execution the maximum memory used by
StarPU for internal data structures during execution.
</dd>

<dt>STARPU_BUS_STATS</dt>
<dd>
\anchor STARPU_BUS_STATS
\addindex __env__STARPU_BUS_STATS
Enable the display of data transfers statistics when calling
starpu_shutdown() (\ref Profiling). By default, statistics are printed
on the standard error stream, use the environment variable \ref
STARPU_BUS_STATS_FILE to define another filename.
</dd>

<dt>STARPU_BUS_STATS_FILE</dt>
<dd>
\anchor STARPU_BUS_STATS_FILE
\addindex __env__STARPU_BUS_STATS_FILE
Define the name of the file where to display data transfers
statistics, see \ref STARPU_BUS_STATS.
</dd>

<dt>STARPU_WORKER_STATS</dt>
<dd>
\anchor STARPU_WORKER_STATS
\addindex __env__STARPU_WORKER_STATS
Enable the display of workers statistics when calling
starpu_shutdown() (\ref Profiling). When combined with the
environment variable \ref STARPU_PROFILING, it displays the energy
consumption (\ref Energy-basedScheduling).  By default, statistics are
printed on the standard error stream, use the environment variable
\ref STARPU_WORKER_STATS_FILE to define another filename.
</dd>

<dt>STARPU_WORKER_STATS_FILE</dt>
<dd>
\anchor STARPU_WORKER_STATS_FILE
\addindex __env__STARPU_WORKER_STATS_FILE
Define the name of the file where to display workers statistics, see
\ref STARPU_WORKER_STATS.
</dd>

<dt>STARPU_STATS</dt>
<dd>
\anchor STARPU_STATS
\addindex __env__STARPU_STATS
When set to 0, data statistics will not be displayed at the
end of the execution of an application (\ref DataStatistics).
</dd>

<dt>STARPU_WATCHDOG_TIMEOUT</dt>
<dd>
\anchor STARPU_WATCHDOG_TIMEOUT
\addindex __env__STARPU_WATCHDOG_TIMEOUT
When set to a value other than 0, allows to make StarPU print an error
message whenever StarPU does not terminate any task for the given time (in µs),
but lets the application continue normally. Should
be used in combination with \ref STARPU_WATCHDOG_CRASH
(see \ref DetectionStuckConditions).
</dd>

<dt>STARPU_WATCHDOG_CRASH</dt>
<dd>
\anchor STARPU_WATCHDOG_CRASH
\addindex __env__STARPU_WATCHDOG_CRASH
When set to a value other than 0, trigger a crash when the watch
dog is reached, thus allowing to catch the situation in gdb, etc
(see \ref DetectionStuckConditions)
</dd>

<dt>STARPU_WATCHDOG_DELAY</dt>
<dd>
\anchor STARPU_WATCHDOG_DELAY
\addindex __env__STARPU_WATCHDOG_DELAY
Delay the activation of the watchdog by the given time (in µs). This can
be convenient for letting the application initialize data etc. before starting
to look for idle time.
</dd>

<dt>STARPU_TASK_PROGRESS</dt>
<dd>
\anchor STARPU_TASK_PROGRESS
\addindex __env__STARPU_TASK_PROGRESS
Print the progression of tasks. This is convenient to determine whether a
program is making progress in task execution, or is just stuck.
</dd>

<dt>STARPU_TASK_BREAK_ON_PUSH</dt>
<dd>
\anchor STARPU_TASK_BREAK_ON_PUSH
\addindex __env__STARPU_TASK_BREAK_ON_PUSH
When this variable contains a job id, StarPU will raise \c SIGTRAP when the task
with that job id is being pushed to the scheduler, which will be nicely caught by debuggers
(see \ref DebuggingScheduling)
</dd>

<dt>STARPU_TASK_BREAK_ON_SCHED</dt>
<dd>
\anchor STARPU_TASK_BREAK_ON_SCHED
\addindex __env__STARPU_TASK_BREAK_ON_SCHED
When this variable contains a job id, StarPU will raise \c SIGTRAP when the task
with that job id is being scheduled by the scheduler (at a scheduler-specific
point), which will be nicely caught by debuggers.
This only works for schedulers which have such a scheduling point defined
(see \ref DebuggingScheduling)
</dd>

<dt>STARPU_TASK_BREAK_ON_POP</dt>
<dd>
\anchor STARPU_TASK_BREAK_ON_POP
\addindex __env__STARPU_TASK_BREAK_ON_POP
When this variable contains a job id, StarPU will raise \c SIGTRAP when the task
with that job id is being popped from the scheduler, which will be nicely caught by debuggers
(see \ref DebuggingScheduling)
</dd>

<dt>STARPU_TASK_BREAK_ON_EXEC</dt>
<dd>
\anchor STARPU_TASK_BREAK_ON_EXEC
\addindex __env__STARPU_TASK_BREAK_ON_EXEC
When this variable contains a job id, StarPU will raise \c SIGTRAP when the task
with that job id is being executed, which will be nicely caught by debuggers
(see \ref DebuggingScheduling)
</dd>

<dt>STARPU_DISABLE_KERNELS</dt>
<dd>
\anchor STARPU_DISABLE_KERNELS
\addindex __env__STARPU_DISABLE_KERNELS
When set to a value other than 1, it disables actually calling the kernel
functions, thus allowing to quickly check that the task scheme is working
properly, without performing the actual application-provided computation.
</dd>

<dt>STARPU_HISTORY_MAX_ERROR</dt>
<dd>
\anchor STARPU_HISTORY_MAX_ERROR
\addindex __env__STARPU_HISTORY_MAX_ERROR
History-based performance models will drop measurements which are really far
froom the measured average. This specifies the allowed variation. Default value is
50 (%), i.e. the measurement is allowed to be x1.5 faster or /1.5 slower than the
average.
</dd>

<dt>STARPU_RAND_SEED</dt>
<dd>
\anchor STARPU_RAND_SEED
\addindex __env__STARPU_RAND_SEED
The random scheduler and some examples use random numbers for their own
working. Depending on the examples, the seed is by default juste always 0 or
the current time() (unless SimGrid mode is enabled, in which case it is always
0). \ref STARPU_RAND_SEED allows to set the seed to a specific value.
</dd>

<dt>STARPU_GLOBAL_ARBITER</dt>
<dd>
\anchor STARPU_GLOBAL_ARBITER
\addindex __env__STARPU_GLOBAL_ARBITER
When set to a positive value, StarPU will create a arbiter, which
implements an advanced but centralized management of concurrent data
accesses (see \ref ConcurrentDataAccess).
</dd>

<dt>STARPU_USE_NUMA</dt>
<dd>
\anchor STARPU_USE_NUMA
\addindex __env__STARPU_USE_NUMA
When defined to 1, NUMA nodes are taking into account by StarPU, i.e. StarPU will
expose one StarPU memory node per NUMA node, and will thus schedule tasks
according to data locality, migrated data when appropriate, etc.

::STARPU_MAIN_RAM is then associated to the NUMA node associated to the
first CPU worker if it exists, the NUMA node associated to the first GPU discovered otherwise.
If StarPU doesn't find any NUMA node after these steps, ::STARPU_MAIN_RAM is the first NUMA node
discovered by StarPU.

Applications should thus rather pass a \c NULL pointer and a -1 memory node to
<c>starpu_data_*_register</c> functions, so that StarPU can manage memory as it
wishes.

If the application wants to control memory allocation on NUMA nodes for
some data, it can use starpu_malloc_on_node and pass the memory node to the
<c>starpu_data_*_register</c> functions to tell StarPU where the allocation was
made. starpu_memory_nodes_get_count_by_kind() and
starpu_memory_node_get_ids_by_type() can be used to get the memory nodes numbers
of the CPU memory nodes.

starpu_memory_nodes_numa_id_to_devid() and starpu_memory_nodes_numa_devid_to_id()
are also available to convert between OS NUMA id and StarPU memory node number.

If this variable is unset, or set to 0, CPU memory is considered as only one
memory node (::STARPU_MAIN_RAM) and it will be up to the OS to manage migration
etc. and the StarPU scheduler will not know about it.
</dd>

<dt>STARPU_IDLE_FILE</dt>
<dd>
\anchor STARPU_IDLE_FILE
\addindex __env__STARPU_IDLE_FILE
When defined, a file named after its contents will be created at the
end of the execution. This file will contain the sum of the idle times
of all the workers.
</dd>

<dt>STARPU_HWLOC_INPUT</dt>
<dd>
\anchor STARPU_HWLOC_INPUT
\addindex __env__STARPU_HWLOC_INPUT
When defined to the path of an XML file, \c hwloc will use this file
as input instead of detecting the current platform topology, which can
save significant initialization time.

To produce this XML file, use <c>lstopo file.xml</c>
</dd>

<dt>STARPU_CATCH_SIGNALS</dt>
<dd>
\anchor STARPU_CATCH_SIGNALS
\addindex __env__STARPU_CATCH_SIGNALS
By default, StarPU catch signals \c SIGINT, \c SIGSEGV and \c SIGTRAP to
perform final actions such as dumping FxT trace files even though the
application has crashed. Setting this variable to a value other than 1
will disable this behaviour. This should be done on JVM systems which
may use these signals for their own needs.
The flag can also be set through the field starpu_conf::catch_signals.
</dd>

<dt>STARPU_DISPLAY_BINDINGS</dt>
<dd>
\anchor STARPU_DISPLAY_BINDINGS
\addindex __env__STARPU_DISPLAY_BINDINGS
Display the binding of all processes and threads running on the machine. If MPI is enabled, display the binding of each node.<br>
Users can manually display the binding by calling starpu_display_bindings().
</dd>
</dl>

\section ConfiguringTheHypervisor Configuring The Hypervisor

<dl>

<dt>SC_HYPERVISOR_POLICY</dt>
<dd>
\anchor SC_HYPERVISOR_POLICY
\addindex __env__SC_HYPERVISOR_POLICY
Choose between the different resizing policies proposed by StarPU for the hypervisor:
\c idle, \c app_driven, \c feft_lp, \c teft_lp, \c ispeed_lp, \c throughput_lp etc.

Use <c>SC_HYPERVISOR_POLICY=help</c> to get the list of available policies for the hypervisor
</dd>

<dt>SC_HYPERVISOR_TRIGGER_RESIZE</dt>
<dd>
\anchor SC_HYPERVISOR_TRIGGER_RESIZE
\addindex __env__SC_HYPERVISOR_TRIGGER_RESIZE
Choose how should the hypervisor be triggered: <c>speed</c> if the resizing algorithm should
be called whenever the speed of the context does not correspond to an optimal precomputed value,
<c>idle</c> it the resizing algorithm should be called whenever the workers are idle for a period
longer than the value indicated when configuring the hypervisor.
</dd>

<dt>SC_HYPERVISOR_START_RESIZE</dt>
<dd>
\anchor SC_HYPERVISOR_START_RESIZE
\addindex __env__SC_HYPERVISOR_START_RESIZE
Indicate the moment when the resizing should be available. The value correspond to the percentage
of the total time of execution of the application. Default value is the resizing frame.
</dd>

<dt>SC_HYPERVISOR_MAX_SPEED_GAP</dt>
<dd>
\anchor SC_HYPERVISOR_MAX_SPEED_GAP
\addindex __env__SC_HYPERVISOR_MAX_SPEED_GAP
Indicate the ratio of speed difference between contexts that should trigger the hypervisor.
This situation may occur only when a theoretical speed could not be computed and the hypervisor
has no value to compare the speed to. Otherwise the resizing of a context is not influenced by the
the speed of the other contexts, but only by the the value that a context should have.
</dd>

<dt>SC_HYPERVISOR_STOP_PRINT</dt>
<dd>
\anchor SC_HYPERVISOR_STOP_PRINT
\addindex __env__SC_HYPERVISOR_STOP_PRINT
By default the values of the speed of the workers is printed during the execution
of the application. If the value 1 is given to this environment variable this printing
is not done.
</dd>

<dt>SC_HYPERVISOR_LAZY_RESIZE</dt>
<dd>
\anchor SC_HYPERVISOR_LAZY_RESIZE
\addindex __env__SC_HYPERVISOR_LAZY_RESIZE
By default the hypervisor resizes the contexts in a lazy way, that is workers are firstly added to a new context
before removing them from the previous one. Once this workers are clearly taken into account
into the new context (a task was popped there) we remove them from the previous one. However if the application
would like that the change in the distribution of workers should change right away this variable should be set to 0
</dd>

<dt>SC_HYPERVISOR_SAMPLE_CRITERIA</dt>
<dd>
\anchor SC_HYPERVISOR_SAMPLE_CRITERIA
\addindex __env__SC_HYPERVISOR_SAMPLE_CRITERIA
By default the hypervisor uses a sample of flops when computing the speed of the contexts and of the workers.
If this variable is set to <c>time</c> the hypervisor uses a sample of time (10% of an approximation of the total
execution time of the application)
</dd>

</dl>

*/
