In this section, we provide a brief overview of the major functionality of nbodykit, as well as an introduction to some of the technical jargon needed to get up and running quickly. We try to familiarize the user with the various aspects of nbodykit needed to take full advantage of nbodykit’s computing power. This section also serves as a nice outline of the documentation, with links to more detailed descriptions included throughout.
lab framework¶A core design goal of nbodykit is maintaining an interactive user
experience, allowing the user to quickly experiment and play around
with data sets and statistics, while still leveraging the power of
parallel processing when necessary. Motivated by the power of
Jupyter notebooks, we adopt a lab
framework for nbodykit, where all of the necessary data containers
and algorithms can be imported from a single module:
from nbodykit.lab import *
[insert cool science here]
With all of the necessary tools now in hand, the user can easily load a data set, compute statistics of that data via one of the built-in algorithms, and save the results in just a few lines. The end result is a reproducible scientific result, generated from clear and concise code that flows from step to step.
The nbodykit package is fully parallelized using the Python
bindings of the Message Passage Interface (MPI) available in mpi4py. While
we aim to hide most of the complexities of MPI from the top-level user
interface, it is helpful to know some basic aspects of the MPI framework
for understanding how nbodykit works to compute its results. If you are
unfamiliar with MPI, a good place to start is the documentation for
mpi4py. Briefly,
MPI allows nbodykit to use a specified number of CPUs, which work independently
to achieve a common goal and pass messages back and forth to coordinate their
work.
Note
It is important to keep in mind that memory is not shared across different CPUS when using MPI. This is particularly important when loading data in parallel using nbodykit, as the data is spread out evenly across all of the available CPUs. This allows nbodykit to load very large data sets quickly, given a necessary number of CPUs are available, which otherwise would not fit into the memory of a single CPU, given the prohibitively large size. However, a single CPU does not have access to the full dataset, but merely the portion stored in its memory (usually \(1/N\) of the full data set, where \(N\) is the number of CPUs).
nbodykit aims to provide a unified treatment of both simulation and observational datasets, allowing it to be used in the analysis of data from not only N-body simulations, but also from current and future large-scale structure surveys.