Dealing with Data on a Mesh

What is a MeshSource?

Often in large-scale structure data analysis, we wish to manipulate representations of continuous quantities on a discrete grid. The canonical example is the analysis of the cosmological density field, interpolated on to a 3D mesh from a discrete set of galaxies. To support such calculations, nbodykit provides the nbodykit.base.mesh.MeshSource object.

Fundamentally, the MeshSource object stores a (possibly weighted) density field on a three-dimensional mesh, with the Nmesh parameter determining the number of grid cells per side (such that there are \(\mathrm{Nmesh}^3\) mesh cells). nbodykit adds the functionality to analyze these fields in both configuration space (often referred to real space) and Fourier space through an interface to the RealField and ComplexField objects implemented by the pmesh package. These objects are paired classes, related through the operation of a 3D fast Fourier transform (FFT). The FFT operation implemented in pmesh relies on the pfft-python package, which is a Python binding of PFFT, a massively parallel FFT library.

Use Cases

The MeshSource is an abstract base class – it cannot be directly initialized. Instead, nbodykit includes several specialized subclasses of MeshSource in the nbodykit.source.mesh module. In general, these subclasses fall into three categories:

  1. Generating mesh data from a CatalogSource (see Converting a CatalogSource to a Mesh)

  2. Reading mesh data from disk (see Saving and Loading a Mesh)

  3. Generating mock fields directly on a mesh (see Gaussian Realizations)

Painting the Mesh

The MeshSource.paint() function produces the values of the field on the mesh, returning either a RealField or ComplexField. This function treats the mesh equally in either configuration space or Fourier space, internally performing the appropriate FFTs. By specifying the mode keyword to the paint() function, users can access either the field data in configuration space or the complex modes of the field in Fourier space.

The “painting” nomenclature derives from the most common use case. The process of interpolating a set of discrete objects on to the mesh evokes the imagery of “painting” the mesh. More generally, the paint() function is responsible for filling in the mesh with data, which could also involve reading data from disk or generating mock fields directly on the mesh.

For further details and examples of painting a catalog of discrete objects to a mesh, see Painting Catalogs to a Mesh.

Fields: RealField and ComplexField

The MeshSource class provides an interface to the pmesh.pm.RealField and pmesh.pm.ComplexField objects. These classes behave like numpy arrays and include functions to perform parallel forward and inverse FFTs. These field objects are initialized from a pmesh.pm.ParticleMesh, which sets the number of mesh cells and stores FFT-related grid quantities.

[1]:

from pmesh.pm import ParticleMesh, RealField, ComplexField

# a 8^3 mesh
pm = ParticleMesh(Nmesh=[8,8,8])

# initialize a RealField
rfield = RealField(pm)

# shape
print("shape = ", rfield.shape)

# set entire mesh to unity
rfield[...] = 1.0

# print the mean of the underlying array
print("mean = ", rfield.value.mean())

shape =  (8, 8, 8)
mean =  1.0

All MeshSource objects implement either the MeshSource.to_real_field() function or the MeshSource.to_complex_field() function. These functions are responsible for returning either a RealField or a ComplexField. The MeshSource.paint function calls these functions, providing the core functionality of the MeshSource class.

The c2r() and r2c() functions

Users can transform between RealField and ComplexField objects using the r2c() function for forward FFTs and the c2r() function for inverse FFTs. These operations take advantage of the fact that the field objects in configuration space store real-valued quantities to perform real-to-complex FFTs. This type of FFT uses the symmetry of real-valued quantities to store only half of the complex modes along the z axis.

[2]:

# perform the forward FFT
cfield = rfield.r2c()

# stores Nmesh/2+1 in z axis b/c of conjugate symmetry
print("shape = ", cfield.shape)

# k=0 mode is the mean value of configuration space field
print("mean of configuration space field from k=0 = ", cfield[0,0,0])

# perform the inverse FFT
rfield2 = cfield.c2r()

# print the mean of the underlying array
print("mean of real field = ", rfield2.value.mean())

shape =  (8, 8, 5)
mean of configuration space field from k=0 =  (1+0j)
mean of real field =  1.0

Storing Meta-data

For all MeshSource objects, the input parameters and additional meta-data are stored in the attrs dictionary attribute.

API

For more information about specific mesh sources, please see the API section.