nbodykit.base.catalogmesh module¶

class nbodykit.base.catalogmesh.CatalogMesh(source, BoxSize, Nmesh, dtype, weight, value, selection, position='Position')[source]¶

Bases: nbodykit.base.mesh.MeshSource, nbodykit.base.catalog.CatalogSource

A class to convert a CatalogSource to a MeshSource, by interpolating the position of the discrete particles on to a mesh.

Parameters:

Parameters:	source (CatalogSource) – the input catalog that we wish to interpolate to a mesh BoxSize – the size of the box Nmesh (int, 3-vector) – the number of cells per mesh side dtype (str) – the data type of the values stored on mesh weight (str) – column in `source` that specifies the weight value for each particle in the `source` to use when gridding value (str) – column in `source` that specifies the field value for each particle; the mesh stores a weighted average of this column selection (str) – column in `source` that selects the subset of particles to grid to the mesh position (str, optional) – column in `source` specifying the position coordinates; default is `Position`

source (CatalogSource) – the input catalog that we wish to interpolate to a mesh
BoxSize – the size of the box
Nmesh (int, 3-vector) – the number of cells per mesh side
dtype (str) – the data type of the values stored on mesh
weight (str) – column in source that specifies the weight value for each particle in the source to use when gridding
value (str) – column in source that specifies the field value for each particle; the mesh stores a weighted average of this column
selection (str) – column in source that selects the subset of particles to grid to the mesh
position (str, optional) – column in source specifying the position coordinates; default is Position

Attributes

`actions`	The actions to apply to the interpolated density field, optionally included the compensation correction.
`attrs`	A dictionary storing relevant meta-data about the MeshSource.
`columns`	All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.
`compensated`	Boolean flag to indicate whether to correct for the windowing kernel introduced when interpolating the discrete particles to a continuous field.
`csize`	The total, collective size of the CatalogSource, i.e., summed across all ranks.
`hardcolumns`	The names of the hard-coded columns in the source.
`interlaced`	Whether to use interlacing when interpolating the density field.
`size`	The number of local particles.
`use_cache`	If set to `True`, use the built-in caching features of `dask` to cache data in memory.
`window`	String specifying the name of the interpolation kernel when gridding the density field.

Methods

`CompensateCIC`(w, v)	Return the Fourier-space kernel that accounts for the convolution of
`CompensateCICAliasing`(w, v)	Return the Fourier-space kernel that accounts for the convolution of
`CompensateTSC`(w, v)	Return the Fourier-space kernel that accounts for the convolution of the gridded field with the TSC window function in configuration space.
`CompensateTSCAliasing`(w, v)	Return the Fourier-space kernel that accounts for the convolution of
`Selection`()	A boolean column that selects a subset slice of the CatalogSource.
`Value`()	When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.
`Weight`()	The column giving the weight to use for each particle on the mesh.
`apply`(func[, kind, mode])	Return a view of the mesh, with `actions` updated to
`compute`(args, *kwargs)	Our version of `dask.compute()` that computes multiple delayed dask collections at once.
`copy`()	Return a copy of the CatalogSource object
`get_hardcolumn`(col)	Return a hard-coded column
`make_column`(array)	Utility function to convert a numpy array to a `dask.array.Array`.
`paint`([mode, Nmesh])	Paint the density on the mesh and apply any transformation functions specified in `actions`.
`preview`([axes, Nmesh, root])	Gather the mesh into as a numpy array, with (reduced) resolution.
`read`(columns)	Return the requested columns as dask arrays.
`save`(output[, dataset, mode])	Save the mesh as a `BigFileMesh` on disk, either in real or complex space.
`to_complex_field`([out])	Convert the mesh source to the Fourier-space field, returning a `pmesh.pm.ComplexField` object.
`to_field`([mode, out])	Return the mesh as a `pmesh` Field object, either in Fourier space or configuration space, based on `mode`.
`to_mesh`([Nmesh, BoxSize, dtype, interlaced, …])	Convert the CatalogSource to a MeshSource, using the specified parameters.
`to_real_field`([out, normalize])	Paint the density field, by interpolating the position column on to the mesh.
`update_csize`()	Set the collective size, `csize`.
`view`()	Return a “view” of the MeshSource, in the spirit of numpy’s ndarray view.

static CompensateCIC(w, v)[source]¶

Return the Fourier-space kernel that accounts for the convolution of the gridded field with the CIC window function in configuration space

Note

see equation 18 (with p=2) of Jing et al 2005

Parameters:	w (list of arrays) – the list of “circular” coordinate arrays, ranging from \([-\pi, \pi)\). v (array_like) – the field array

static CompensateCICAliasing(w, v)[source]¶

Return the Fourier-space kernel that accounts for the convolution of the gridded field with the CIC window function in configuration space, as well as the approximate aliasing correction

Note

see equation 20 of Jing et al 2005

Parameters:	w (list of arrays) – the list of “circular” coordinate arrays, ranging from \([-\pi, \pi)\). v (array_like) – the field array

static CompensateTSC(w, v)[source]¶

Return the Fourier-space kernel that accounts for the convolution of the gridded field with the TSC window function in configuration space.

Note

see equation 18 (with p=3) of Jing et al 2005

Parameters:	w (list of arrays) – the list of “circular” coordinate arrays, ranging from \([-\pi, \pi)\). v (array_like) – the field array

static CompensateTSCAliasing(w, v)[source]¶

Return the Fourier-space kernel that accounts for the convolution of the gridded field with the TSC window function in configuration space, as well as the approximate aliasing correction

Note

see equation 20 of Jing et al 2005

Parameters:	w (list of arrays) – the list of “circular” coordinate arrays, ranging from \([-\pi, \pi)\). v (array_like) – the field array

Selection()¶

A boolean column that selects a subset slice of the CatalogSource.

By default, this column is set to True for all particles.

Value()¶

When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles.

Weight()¶

The column giving the weight to use for each particle on the mesh.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles.

__delitem__(col)¶: Delete a column; cannot delete a “hard-coded” column

__getitem__(sel)¶

The following types of indexing are supported:

strings specifying a column in the CatalogSource; returns a dask array holding the column data
boolean arrays specifying a slice of the CatalogSource; returns a CatalogCopy holding only the revelant slice
slice object specifying which particles to select
list of strings specifying column names; returns a CatalogCopy holding only the selected columnss

__len__()¶: Length of a mesh source is zero

__setitem__(col, value)¶: Add columns to the CatalogSource, overriding any existing columns with the name col.

actions¶: The actions to apply to the interpolated density field, optionally included the compensation correction.

apply(func, kind='wavenumber', mode='complex')¶

Return a view of the mesh, with actions updated to apply the specified function, either in Fourier space or configuration space, based on mode

Parameters:	func (callable) – func(x, y) where x is a list of `r` (`k`) values that broadcasts into a full array, when `mode` is ‘real’ (‘complex’); the value of x depends on `kind`. `y` is the value of the mesh field on the corresponding locations. kind (string, optional) – The kind of value in x. When `mode` is ‘complex’: ’wavenumber’ means wavenumber from [- 2 pi / L * N / 2, 2 pi / L * N / 2). ’circular’ means circular frequency from [- pi, pi). ’index’ means [0, Nmesh ) When `mode` is ‘real’: ’relative’ means distance from [-0.5 Boxsize, 0.5 BoxSize). ’index’ means [0, Nmesh ) mode ('complex' or 'real', optional) – whether to apply the function to the mesh in configuration space or Fourier space
Returns:	a view of the mesh object with the `actions` attribute updated to include the new action
Return type:	MeshSource

attrs¶: A dictionary storing relevant meta-data about the MeshSource.

columns¶: All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.

compensated¶

Boolean flag to indicate whether to correct for the windowing kernel introduced when interpolating the discrete particles to a continuous field.

See the documentation for further details.

compute(*args, **kwargs)¶

Our version of dask.compute() that computes multiple delayed dask collections at once.

This should be called on the return value of read() to converts any dask arrays to numpy arrays.

If use_cache is True, this internally caches data, using dask’s built-in cache features.

Parameters:	args (object) – Any number of objects. If the object is a dask collection, it’s computed and the result is returned. Otherwise it’s passed through unchanged.

Notes

The dask default optimizer induces too many (unnecesarry) IO calls – we turn this off feature off by default. Eventually we want our own optimizer probably.

copy()¶

Return a copy of the CatalogSource object

Returns:	the new CatalogSource object holding the copied data columns
Return type:	CatalogCopy

csize¶

The total, collective size of the CatalogSource, i.e., summed across all ranks.

It is the sum of size across all available ranks.

get_hardcolumn(col)[source]¶: Return a hard-coded column

hardcolumns¶: The names of the hard-coded columns in the source.

interlaced¶

Whether to use interlacing when interpolating the density field.

See the documentation for further details.

See also: Section 3.1 of Sefusatti et al. 2015

logger = <logging.Logger object>¶

make_column(array)¶: Utility function to convert a numpy array to a dask.array.Array.

paint(mode='real', Nmesh=None)¶

Paint the density on the mesh and apply any transformation functions specified in actions.

The return type of the pmesh Field object is specified by mode. This calls to_field() to convert the mesh to a Field.

See the documentation on painting for more details on painting catalogs to a mesh.

Parameters:	mode ('real' or 'complex') – the type of the returned Field object, either a RealField or ComplexField Nmesh (int or array_like, or None) – If given and different from the intrinsic Nmesh of the source, resample the mesh to the given resolution
Returns:	either a RealField of ComplexField, with the functions in `actions` applied to it
Return type:	`RealField`, `ComplexField`

preview(axes=None, Nmesh=None, root=0)¶

Gather the mesh into as a numpy array, with (reduced) resolution. The result is broadcast to all ranks, so this uses \(\mathrm{Nmesh}^3\) per rank.

Parameters:	Nmesh (int, array_like) – The desired Nmesh of the result. Be aware this function allocates memory to hold a full `Nmesh` on each rank. axes (int, array_like) – The axes to project the preview onto., e.g. (0, 1) root (int, optional) – the rank number to treat as root when gathering to a single rank
Returns:	out – An numpy array holding the real density field.
Return type:	array_like

read(columns)¶

Return the requested columns as dask arrays.

Parameters:	columns (list of str) – the names of the requested columns
Returns:	the list of column data, in the form of dask arrays
Return type:	list of `dask.array.Array`

save(output, dataset='Field', mode='real')¶

Save the mesh as a BigFileMesh on disk, either in real or complex space.

Parameters:	output (str) – name of the bigfile file dataset (str, optional) – name of the bigfile data set where the field is stored mode (str, optional) – real or complex; the form of the field to store

size¶: The number of local particles.

to_complex_field(out=None)¶

Convert the mesh source to the Fourier-space field, returning a pmesh.pm.ComplexField object.

Not implemented in the base class, unless object is a view.

to_field(mode='real', out=None)¶

Return the mesh as a pmesh Field object, either in Fourier space or configuration space, based on mode.

This will call to_real_field() or to_complex_field() based on mode.

Parameters:	mode ('real' or 'complex') – the return type of the field
Returns:	either a RealField of ComplexField, storing the value of the field on the mesh
Return type:	`RealField`, `ComplexField`

to_mesh(Nmesh=None, BoxSize=None, dtype='f4', interlaced=False, compensated=False, window='cic', weight='Weight', value='Value', selection='Selection', position='Position')¶

Convert the CatalogSource to a MeshSource, using the specified parameters.

Parameters:	Nmesh (int, optional) – the number of cells per side on the mesh; must be provided if not stored in `attrs` BoxSize (scalar, 3-vector, optional) – the size of the box; must be provided if not stored in `attrs` dtype (string, optional) – the data type of the mesh array interlaced (bool, optional) – use the interlacing technique of Sefusatti et al. 2015 to reduce the effects of aliasing on Fourier space quantities computed from the mesh compensated (bool, optional) – whether to correct for the window introduced by the grid interpolation scheme window (str, optional) – the string specifying which window interpolation scheme to use; see pmesh.window.methods weight (str, optional) – the name of the column specifying the weight for each particle value (str, optional) – the name of the column specifying the field value for each particle selection (str, optional) – the name of the column that specifies which (if any) slice of the CatalogSource to take position (str, optional) – the name of the column that specifies the position data of the objects in the catalog
Returns:	mesh – a mesh object that provides an interface for gridding particle data onto a specified mesh
Return type:	CatalogMesh

to_real_field(out=None, normalize=True)[source]¶

Paint the density field, by interpolating the position column on to the mesh.

This computes the following meta-data attributes in the process of painting, returned in the attrs attributes of the returned RealField object:

N : int

the (unweighted) total number of objects painted to the mesh
W : float

the weighted number of total objects, equal to the collective sum of the ‘weight’ column
shotnoise : float

the Poisson shot noise, equal to the volume divided by N
num_per_cell : float

the mean number of weighted objects per cell

Note

The density field on the mesh is normalized as \(1+\delta\), such that the collective mean of the field is unity.

See the documentation on painting for more details on painting catalogs to a mesh.

Returns:	real – the painted real field; this has a `attrs` dict storing meta-data
Return type:	`pmesh.pm.RealField`

update_csize()¶

Set the collective size, csize.

This function should be called in __init__() of a subclass, after size has been set to a valid value (not NotImplemented)

use_cache¶: If set to True, use the built-in caching features of dask to cache data in memory.

view()¶

Return a “view” of the MeshSource, in the spirit of numpy’s ndarray view.

This returns a new MeshSource whose memory is owned by self.

window¶

String specifying the name of the interpolation kernel when gridding the density field.

See the documentation for further details.

Note

Valid values must be in pmesh.window.methods