Scientific.Threading.TaskManager.TaskManager

Package Scientific :: Package Threading :: Module TaskManager :: Class TaskManager |
[frames] | no frames]

# Class TaskManager

Parallel task manager for shared-memory multiprocessor machines

This class provides a rather simple way to profit from shared-memory multiprocessor machines by running several tasks in parallel. The calling program decides how many execution threads should run at any given time, and then feeds compute tasks to the task manager, who runs them as soon as possible without exceeding the maximum number of threads.

The major limitation of this approach lies in Python's Global Interpreter Lock. This effectively means that no more than one Python thread can run at the same time. Consequently, parallelization can only be achieved if the tasks to be parallelized spend significant time in C extension modules that release the Global Interpreter Lock.

Instance Methods
__init__(self, nthreads) |
runTask(self, function, args) Run a task as soon as processing capacity becomes available |
terminate(self) Wait until all tasks have finished |

Method Details

### __init__(self, nthreads) (Constructor) |
Parameters: `nthreads` (`int`) - the maximum number of compute threads that should run in parallel. Note: This does not include the main thread which generated and feeds the task manager!

### runTask(self, function, args) |
Run a task as soon as processing capacity becomes available Parameters: `function` (callable) - the function that will be executed as the body of the task `args` (`tuple`) - the arguments that will be passed to function when it is called. An additional argument will be added at the end: a lock object that the task can use to get temporarily exclusive access to data shared with other tasks.