Functions
enum (*sequential, **named) |
Enumeration values to serve as status tags passed |
split_ranks (N_ranks, N[, include_all]) |
Divide the ranks into chunks, attempting to have N ranks in each chunk. |
Classes
TaskManager (cpus_per_task[, comm, debug, …]) |
An MPI task manager that distributes tasks over a set of MPI processes, using a specified number of independent workers to compute each task. |
nbodykit.batch.
TaskManager
(cpus_per_task, comm=None, debug=False, use_all_cpus=False)[source]¶An MPI task manager that distributes tasks over a set of MPI processes, using a specified number of independent workers to compute each task.
Given the specified number of independent workers (which compute tasks in parallel), the total number of available CPUs will be divided evenly.
The main function is iterate
which iterates through a set of tasks,
distributing the tasks in parallel over the available ranks.
Parameters: |
|
---|
Methods
is_root () |
Is the current process the root process? |
is_worker () |
Is the current process a valid worker? |
iterate (tasks) |
A generator that iterates through a series of tasks in parallel. |
map (function, tasks) |
Like the built-in map() function, apply a function to all of the values in a list and return the list of results. |
__enter__
()[source]¶Split the base communicator such that each task gets allocated the specified number of cpus to perform the task with
__exit__
(exc_type, exc_value, exc_traceback)[source]¶Exit gracefully by closing and freeing the MPI-related variables
is_root
()[source]¶Is the current process the root process?
Root is responsible for distributing the tasks to the other available ranks
is_worker
()[source]¶Is the current process a valid worker?
Workers wait for instructions from the master
iterate
(tasks)[source]¶A generator that iterates through a series of tasks in parallel.
Notes
This is a collective operation and should be called by all ranks
Parameters: | tasks (iterable) – an iterable of task items that will be yielded in parallel across all ranks |
---|---|
Yields: | task – the individual items of tasks, iterated through in parallel |
map
(function, tasks)[source]¶Like the built-in map()
function, apply a function to all
of the values in a list and return the list of results.
If tasks
contains tuples, the arguments are passed to
function
using the *args
syntax
Notes
This is a collective operation and should be called by all ranks
Parameters: |
|
---|---|
Returns: | results – the list of the return values of |
Return type: | list |
nbodykit.batch.
enum
(*sequential, **named)[source]¶Enumeration values to serve as status tags passed between processeseee