nbodykit is currently supported on macOS and Linux architectures. The
recommended installation method is to install nbodykit and its
dependencies as part of the Anaconda
Python distribution. For more advanced users, or those without an
Anaconda distribution, the software can also be installed using the
utility or directly from the source. In these latter cases, additional
difficulties may arise when trying to compile and install some of
The package is available for Python versions 2.7, 3.5, and 3.6.
Installing nbodykit with Anaconda¶
We have pre-built binaries for nbodykit and all of its dependencies available via Anaconda that are compatible with Linux and macOS platforms. To avoid dependency conflicts, we recommend installing nbodykit into a fresh environment. This can be achieved with the following commands:
$ conda create --name nbodykit-env python=3 # or use python=2 for python 2.7* $ source activate nbodykit-env $ conda install -c bccp nbodykit
Updating nbodykit and its dependencies¶
To keep nbodykit and its dependencies up to date, use
$ conda update -c bccp --all
Installing From Source for Conda-based Installs¶
Users can also install nbodykit from source from within a conda environment,
git is installed. First, clone the GitHub repository,
$ git clone http://github.com/bccp/nbodykit $ cd nbodykit
Then, install all of the required dependencies using
$ conda install -c bccp --file requirements.txt $ conda install -c bccp --file requirements-extras.txt
Now, the desired git branch can be installed easily. By default, the
branch is active, and the latest development state of nbodykit can be installed
$ pip install -e .
Note that the
-e flag installs nbodykit in “develop” mode, allowing the
installed version to reflect the latest changes in the source code. The latest
changes to the
master branch can be incorporated from GitHub via
$ git checkout master $ git pull origin master
Installing nbodykit with
The easiest and recommended method to install nbodykit and its dependencies is using the Anaconda package. See Installing nbodykit with Anaconda for more details.
To build nbodykit from source, you will need to make sure all of the dependencies are properly installed on your system. To start, the following dependencies should be installed first:
$ pip install numpy cython mpi4py
Next, we must compile the remaining dependencies, which depends on the user’s machine.
To install nbodykit as well as all of its external dependencies on a Linux machine into the default Python installation directory:
$ pip install nbodykit[extras]
A different installation directory can be specified via the
--root <dir> options of the
pip install command.
More care is required to properly build the dependencies on macOS machines.
autotools software is required, which can be installed using
the MacPorts package manager using:
$ sudo port install autoconf automake libtool
Using recent versions of MacPorts, we also need to tell
mpicc to use
rather than the default
clang compiler, which doesn’t compile
due to the lack of
openmp support. Additionally, the
environment variable must be explicitly set.
In bash, the installation command is:
$ export OMPI_CC=gcc $ export LDSHARED="mpicc -bundle -undefined dynamic_lookup -DOMPI_IMPORTS"; pip install nbodykit[extras]
This command will compile and install the dependencies of nbodykit and then
install nbodykit. Again, a different installation directory can be specified via
--root <dir> options of the
pip install command.
Building a Docker Image with example Notebooks¶
(Instructions contributed by Peter Nugent)
Here we setup a basic docker image packages with an example notebook (lognormal mock).
Write the following docker file.
FROM continuumio/miniconda3:4.7.10 MAINTAINER Your Name <firstname.lastname@example.org> # Install bccp software RUN conda install -c bccp nbodykit notebook matplotlib RUN mkdir -p /opt/notebooks COPY lognormal-mocks.ipynb /opt/notebooks
In the directory with the Dockerfile copied the lognormal-mocks.ipynb (from nbodykit-cookbook) into it and then build the image with:
There is an example of the docker file in the nbodykit-cookbook repository linked above.
docker build --tag bccp:1.0 .
The commands need access to the internet due to the conda command. If this is not enabled by default, (e.g. Fedora 32), add –network=host to the command line.
Once built, start the docker with:
docker run -it -p 8888:8888 bccp:1.0 /bin/bash
Then inside the image run:
/opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --allow-root --ip 0.0.0.0 --port 8888
It’ll point you to the location to pull up the notebook and voila, the lognormal notebook is there and works.
nbodykit on NERSC¶
This section covers using nbodykit on the computing nodes of NERSC. The development of nbodykit is supported by NERSC under the allocation m3035.
The computing nodes requires special care because they do not work with the simple MPI provided from Anaconda.
If instead you wish to use nbodykit on the login nodes of NERSC or the Jupyter Hub services (available at https://jupyter.nersc.gov and https://jupyter-dev.nersc.gov/), users should follow the Anaconda installation instructions to install nbodykit. The login nodes and JuptyerHub machines are very similar to standard computers. For more information on the JupyterHub services, see the official NERSC guide.
Development and testing of nbodykit was performed on the NERSC super-computing machines at Lawrence Berkeley National Laboratory. We maintain a daily build of the latest stable version of nbodykit on NERSC systems for Python versions 2.7, 3.6, and 3.7 and provide a tool to automatically load the appropriate environment when running jobs on either the Edison or Cori machines.
To load the latest stable version of nbodykit on NERSC, the following line should be added to the beginning of the user’s job script:
# load python 3.7 with latest stable nbodykit source /global/common/software/m3035/conda-activate.sh 3.7
If instead the user wishes to install the latest development version of nbodykit, the following lines should be added to the job script:
# first load python 3.7 with latest stable nbodykit source /global/common/software/m3035/conda-activate.sh 3.7 # overwrite nbodykit with the latest version from the tip of master bcast-pip https://github.com/bccp/nbodykit/archive/master.zip
In the nbodykit source directory, we include an example Python script and job script for users. To run this example on NERSC, first download the necessary files:
# download the example script $ wget https://raw.githubusercontent.com/bccp/nbodykit/master/nersc/example.py # download the job script $ wget https://raw.githubusercontent.com/bccp/nbodykit/master/nersc/example-job.slurm
and then if on the Cori machine, the job can be submitted using
$ sbatch -C haswell example-job.slurm
of if on the Edison machine, use
$ sbatch example-job.slurm
The example job script simply loads the nbodykit environment and executes the Python script in parallel, in this case, using 16 CPUs.
#!/bin/bash #SBATCH -p debug #SBATCH -o nbkit-example #SBATCH -n 16 # load nbodykit source /global/common/software/m3035/conda-activate.sh 3.7 # run the main nbodykit example srun -n 16 python example.py
If successful, this will save a file
nbkit_example_power.json to the
current working directory.