nbodykit.source.catalog.halos¶

Classes

class nbodykit.source.catalog.halos.HaloCatalog(source, cosmo, redshift, mdef='vir', mass='Mass', position='Position', velocity='Velocity')[source]¶

A CatalogSource of objects that represent halos, which can be populated using analytic models from halotools.

Parameters:

source (CatalogSource) – the source holding the particles to be interpreted as halos cosmo (Cosmology) – the cosmology instance; redshift (float) – the redshift of the halo catalog mdef (str, optional) – string specifying mass definition, used for computing default halo radii and concentration; should be ‘vir’ or ‘XXXc’ or ‘XXXm’ where ‘XXX’ is an int specifying the overdensity mass (str, optional) – the column name specifying the mass of each halo position (str, optional) – the column name specifying the position of each halo velocity (str, optional) – the column name specifying the velocity of each halo

Attributes:

Index The attribute giving the global index rank of each particle in the list. attrs A dictionary storing relevant meta-data about the CatalogSource. columns All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user. csize The total, collective size of the CatalogSource, i.e., summed across all ranks. hardcolumns A list of the hard-coded columns in the CatalogSource. size The number of objects in the CatalogSource on the local rank.

Methods

Concentration()	The halo concentration, computed using nbodykit.transform.HaloConcentration().
Mass()	The halo mass column, assumed to be in units of \(M_\odot/h\).
Position()	The halo position column, assumed to be in units of \(\mathrm{Mpc}/h\).
Radius()	The halo radius, computed using nbodykit.transform.HaloRadius().
Selection()	A boolean column that selects a subset slice of the CatalogSource.
Value()	When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.
Velocity()	The halo velocity column, assumed to be in units of km/s.
VelocityOffset()	The redshift-space distance offset due to the velocity in units of distance.
Weight()	The column giving the weight to use for each particle on the mesh.
compute(*args, **kwargs)	Our version of dask.compute() that computes multiple delayed dask collections at once.
copy()	Return a shallow copy of the object, where each column is a reference of the corresponding column in self.
get_hardcolumn(col)	Construct and return a hard-coded column.
gslice(start, stop[, end, redistribute])	Execute a global slice of a CatalogSource.
make_column(array)	Utility function to convert an array-like object to a dask.array.Array.
populate(model[, BoxSize, seed])	Populate the HaloCatalog using a halotools model.
read(columns)	Return the requested columns as dask arrays.
save(output, columns[, dataset, datasets, …])	Save the CatalogSource to a bigfile.BigFile.
sort(keys[, reverse, usecols])	Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.
to_halotools([BoxSize])	Return the HaloCatalog as a halotools.sim_manager.UserSuppliedHaloCatalog.
to_mesh([Nmesh, BoxSize, dtype, interlaced, …])	Convert the CatalogSource to a MeshSource, using the specified parameters.
to_subvolumes([domain, position, columns])	Domain Decompose a catalog, sending items to the ranks according to the supplied domain object.
view([type])	Return a “view” of the CatalogSource object, with the returned type set by type.

create_instance

Concentration()[source]¶

The halo concentration, computed using nbodykit.transform.HaloConcentration().

This uses the analytic formulas for concentration from Dutton and Maccio 2014.

Users can override this column to implement custom mass-concentration relations.

Index¶

The attribute giving the global index rank of each particle in the list. It is an integer from 0 to self.csize.

Note that slicing changes this index value.

Mass()[source]¶

The halo mass column, assumed to be in units of \(M_\odot/h\).

Position()[source]¶

The halo position column, assumed to be in units of \(\mathrm{Mpc}/h\).

Radius()[source]¶

The halo radius, computed using nbodykit.transform.HaloRadius().

Assumed units of \(\mathrm{Mpc}/h\).

Selection()¶

A boolean column that selects a subset slice of the CatalogSource.

By default, this column is set to True for all particles, and all CatalogSource objects will contain this column.

Value()¶

When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.

Velocity()[source]¶

The halo velocity column, assumed to be in units of km/s.

VelocityOffset()[source]¶

The redshift-space distance offset due to the velocity in units of distance. The assumed units are \(\mathrm{Mpc}/h\).

This multiplies Velocity by \(1 / (a 100 E(z)) = 1 / (a H(z)/h)\).

Weight()¶

The column giving the weight to use for each particle on the mesh.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.

__delitem__(col)¶

Delete a column; cannot delete a “hard-coded” column.

Note

If the base attribute is set, columns will be deleted from base instead of from self.

__finalize__(other)¶

Finalize the creation of a CatalogSource object by copying over any additional attributes from a second CatalogSource.

The idea here is to only copy over attributes that are similar to meta-data, so we do not copy some of the core attributes of the CatalogSource object.

Parameters:	other – the second object to copy over attributes from; it needs to be a subclass of CatalogSourcBase for attributes to be copied
Returns:	return self, with the added attributes
Return type:	CatalogSource

__getitem__(sel)¶

The following types of indexing are supported:

1. strings specifying a column in the CatalogSource; returns a dask array holding the column data
2. boolean arrays specifying a slice of the CatalogSource; returns a CatalogSource holding only the revelant slice
3. slice object specifying which particles to select
4. list of strings specifying column names; returns a CatalogSource holding only the selected columns

Notes

• Slicing is a collective operation
• If the base attribute is set, columns will be returned from base instead of from self.

__len__()¶

The local size of the CatalogSource on a given rank.

__setitem__(col, value)¶

Add columns to the CatalogSource, overriding any existing columns with the name col.

attrs¶

A dictionary storing relevant meta-data about the CatalogSource.

columns¶

All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.

Note

If the base attribute is set, the value of base.columns will be returned.

compute(*args, **kwargs)¶

Our version of dask.compute() that computes multiple delayed dask collections at once.

This should be called on the return value of read() to converts any dask arrays to numpy arrays.

. note::

If the base attribute is set, compute() will called using base instead of self.

Parameters:	args (object) – Any number of objects. If the object is a dask collection, it’s computed and the result is returned. Otherwise it’s passed through unchanged.

copy()¶

Return a shallow copy of the object, where each column is a reference of the corresponding column in self.

Note

No copy of data is made.

Note

This is different from view in that the attributes dictionary of the copy no longer related to self.

Returns:	a new CatalogSource that holds all of the data columns of self
Return type:	CatalogSource

csize¶

The total, collective size of the CatalogSource, i.e., summed across all ranks.

It is the sum of size across all available ranks.

If the base attribute is set, the base.csize attribute will be returned.

get_hardcolumn(col)¶

Construct and return a hard-coded column.

These are usually produced by calling member functions marked by the @column decorator.

Subclasses may override this method and the hardcolumns attribute to bypass the decorator logic.

Note

If the base attribute is set, get_hardcolumn() will called using base instead of self.

gslice(start, stop, end=1, redistribute=True)¶

Execute a global slice of a CatalogSource.

Note

After the global slice is performed, the data is scattered evenly across all ranks.

Note

The current algorithm generates an index on the root rank and does not scale well.

Parameters:	start (int) – the start index of the global slice stop (int) – the stop index of the global slice step (int, optional) – the default step size of the global size redistribute (bool, optional) – if True, evenly re-distribute the sliced data across all ranks, otherwise just return any local data part of the global slice

hardcolumns¶

A list of the hard-coded columns in the CatalogSource.

These columns are usually member functions marked by @column decorator. Subclasses may override this method and use get_hardcolumn() to bypass the decorator logic.

Note

If the base attribute is set, the value of base.hardcolumns will be returned.

static make_column(array)¶

Utility function to convert an array-like object to a dask.array.Array.

Note

The dask array chunk size is controlled via the dask_chunk_size global option. See set_options.

Parameters:	array (array_like) – an array-like object; can be a dask array, numpy array, ColumnAccessor, or other non-scalar array-like object
Returns:	a dask array initialized from array
Return type:	dask.array.Array

populate(model, BoxSize=None, seed=None, **params)[source]¶

Populate the HaloCatalog using a halotools model.

The model can be a built-in model from nbodykit.hod (which will be converted to a Halotools model) or directly a Halotools model instance.

This assumes that this is the first time this catalog has been populated with the input model. To re-populate using the same model (but different parameters), call the repopulate() function of the returned PopulatedHaloCatalog.

Parameters:	model (nbodykit.hod.HODModel or halotools model object) – the model instance to use to populate; model types from nbodykit.hod will automatically be converted BoxSize (float, 3-vector, optional) – the box size of the catalog; this must be supplied if ‘BoxSize’ is not in attrs seed (int, optional) – the random seed to use when populating the mock **params – key/value pairs specifying the model parameters to use
Returns:	cat – the catalog object storing information about the populated objects
Return type:	PopulatedHaloCatalog

Examples

Initialize a demo halo catalog:

>>> from nbodykit.tutorials import DemoHaloCatalog
>>> cat = DemoHaloCatalog('bolshoi', 'rockstar', 0.5)

Populate with the built-in Zheng07 model:

>>> from nbodykit.hod import Zheng07Model
>>> galcat = cat.populate(Zheng07Model, seed=42)

And then re-populate galaxy catalog with new parameters:

>>> galcat.repopulate(alpha=0.9, logMmin=13.5, seed=42)

read(columns)¶

Return the requested columns as dask arrays.

Parameters:	columns (list of str) – the names of the requested columns
Returns:	the list of column data, in the form of dask arrays
Return type:	list of dask.array.Array

save(output, columns, dataset=None, datasets=None, header='Header')¶

Save the CatalogSource to a bigfile.BigFile.

Only the selected columns are saved and attrs are saved in header. The attrs of columns are stored in the datasets.

Parameters:

output (str) – the name of the file to write to columns (list of str) – the names of the columns to save in the file dataset (str, optional) – dataset to store the columns under. datasets (list of str, optional) – names for the data set where each column is stored; defaults to the name of the column (deprecated) header (str, optional, or None) – the name of the data set holding the header information, where attrs is stored if header is None, do not save the header.

size¶

The number of objects in the CatalogSource on the local rank.

If the base attribute is set, the base.size attribute will be returned.

Important

This property must be defined for all subclasses.

sort(keys, reverse=False, usecols=None)¶

Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.

Sort columns must be floating or integer type.

Note

After the sort operation, the data is scattered evenly across all ranks.

Parameters:	keys (list, tuple) – the names of columns to sort by. If multiple columns are provided, the data is sorted consecutively in the order provided reverse (bool, optional) – if True, perform descending sort operations usecols (list, optional) – the name of the columns to include in the returned CatalogSource

to_halotools(BoxSize=None)[source]¶

Return the HaloCatalog as a halotools.sim_manager.UserSuppliedHaloCatalog.

The Halotools catalog only holds the local data, although halos are labeled via the halo_id column using the global index.

Parameters:	BoxSize (float, array_like, optional) – the size of the box; must be supplied if ‘BoxSize’ is not in the attrs dict
Returns:	cat – the Halotools halo catalog, storing the local halo data
Return type:	halotools.sim_manager.UserSuppliedHaloCatalog

to_mesh(Nmesh=None, BoxSize=None, dtype='f4', interlaced=False, compensated=False, resampler='cic', weight='Weight', value='Value', selection='Selection', position='Position', window=None)¶

Convert the CatalogSource to a MeshSource, using the specified parameters.

Parameters:	Nmesh (int, optional) – the number of cells per side on the mesh; must be provided if not stored in attrs BoxSize (scalar, 3-vector, optional) – the size of the box; must be provided if not stored in attrs dtype (string, optional) – the data type of the mesh array interlaced (bool, optional) – use the interlacing technique of Sefusatti et al. 2015 to reduce the effects of aliasing on Fourier space quantities computed from the mesh compensated (bool, optional) – whether to correct for the resampler window introduced by the grid interpolation scheme resampler (str, optional) – the string specifying which resampler interpolation scheme to use; see pmesh.resampler.methods weight (str, optional) – the name of the column specifying the weight for each particle value (str, optional) – the name of the column specifying the field value for each particle selection (str, optional) – the name of the column that specifies which (if any) slice of the CatalogSource to take position (str, optional) – the name of the column that specifies the position data of the objects in the catalog window (str, deprecated) – use resampler instead.
Returns:	mesh – a mesh object that provides an interface for gridding particle data onto a specified mesh
Return type:	CatalogMesh

to_subvolumes(domain=None, position='Position', columns=None)¶

Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. Using the position column as the Position.

This will read in the full position array and all of the requested columns.

Parameters:	domain (:pyclass:`pmesh.domain.GridND` object, or None) – The domain to distribute the catalog. If None, try to evenly divide spatially. An easiest way to find a domain object is to use pm.domain, where pm is a :pyclass:`pmesh.pm.ParticleMesh` object. position (string_like) – column to use to compute the position. columns (list of string_like) – columns to include in the new catalog, if not supplied, all catalogs will be exchanged.
Returns:	A decomposed catalog source, where each rank only contains objects belongs to the rank as claimed by the domain object. self.attrs are carried over as a shallow copy to the returned object.
Return type:	CatalogSource

view(type=None)¶

Return a “view” of the CatalogSource object, with the returned type set by type.

This initializes a new empty class of type type and attaches attributes to it via the __finalize__() mechanism.

Parameters:	type (Python type) – the desired class type of the returned object.

class nbodykit.source.catalog.halos.PopulatedHaloCatalog(data, model, cosmo, comm=None)[source]¶

A CatalogSource to represent a set of objects populated into a HaloCatalog.

Note

Users should not access this class directly, but rather, call HaloCatalog.populate() to generate a PopulatedHaloCatalog.

Parameters:

data (structured numpy.ndarray) – the data of the populated objects model – the Halotools model instance cosmo (nbodykit.cosmology.cosmology.Cosmology) – the cosmology instance

Attributes:

Index The attribute giving the global index rank of each particle in the list. attrs A dictionary storing relevant meta-data about the CatalogSource. columns All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user. csize The total, collective size of the CatalogSource, i.e., summed across all ranks. hardcolumns The union of the columns in the file and any transformed columns. size The number of objects in the CatalogSource on the local rank.

Methods

Selection()	A boolean column that selects a subset slice of the CatalogSource.
Value()	When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.
Weight()	The column giving the weight to use for each particle on the mesh.
compute(*args, **kwargs)	Our version of dask.compute() that computes multiple delayed dask collections at once.
copy()	Return a shallow copy of the object, where each column is a reference of the corresponding column in self.
get_hardcolumn(col)	Return a column from the underlying data array/dict.
gslice(start, stop[, end, redistribute])	Execute a global slice of a CatalogSource.
make_column(array)	Utility function to convert an array-like object to a dask.array.Array.
read(columns)	Return the requested columns as dask arrays.
repopulate([seed])	Re-populate the catalog in-place, using the specified seed or model parameters.
save(output, columns[, dataset, datasets, …])	Save the CatalogSource to a bigfile.BigFile.
sort(keys[, reverse, usecols])	Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.
to_mesh([Nmesh, BoxSize, dtype, interlaced, …])	Convert the CatalogSource to a MeshSource, using the specified parameters.
to_subvolumes([domain, position, columns])	Domain Decompose a catalog, sending items to the ranks according to the supplied domain object.
view([type])	Return a “view” of the CatalogSource object, with the returned type set by type.

create_instance

Index¶

The attribute giving the global index rank of each particle in the list. It is an integer from 0 to self.csize.

Note that slicing changes this index value.

Selection()¶

A boolean column that selects a subset slice of the CatalogSource.

By default, this column is set to True for all particles, and all CatalogSource objects will contain this column.

Value()¶

When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.

Weight()¶

The column giving the weight to use for each particle on the mesh.

The mesh field is a weighted average of Value, with the weights given by Weight.

By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.

__delitem__(col)¶

Delete a column; cannot delete a “hard-coded” column.

Note

If the base attribute is set, columns will be deleted from base instead of from self.

__finalize__(other)¶

Finalize the creation of a CatalogSource object by copying over any additional attributes from a second CatalogSource.

The idea here is to only copy over attributes that are similar to meta-data, so we do not copy some of the core attributes of the CatalogSource object.

Parameters:	other – the second object to copy over attributes from; it needs to be a subclass of CatalogSourcBase for attributes to be copied
Returns:	return self, with the added attributes
Return type:	CatalogSource

__getitem__(sel)¶

The following types of indexing are supported:

1. strings specifying a column in the CatalogSource; returns a dask array holding the column data
2. boolean arrays specifying a slice of the CatalogSource; returns a CatalogSource holding only the revelant slice
3. slice object specifying which particles to select
4. list of strings specifying column names; returns a CatalogSource holding only the selected columns

Notes

• Slicing is a collective operation
• If the base attribute is set, columns will be returned from base instead of from self.

__len__()¶

The local size of the CatalogSource on a given rank.

__setitem__(col, value)¶

Add columns to the CatalogSource, overriding any existing columns with the name col.

attrs¶

A dictionary storing relevant meta-data about the CatalogSource.

columns¶

All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.

Note

If the base attribute is set, the value of base.columns will be returned.

compute(*args, **kwargs)¶

Our version of dask.compute() that computes multiple delayed dask collections at once.

This should be called on the return value of read() to converts any dask arrays to numpy arrays.

. note::

If the base attribute is set, compute() will called using base instead of self.

Parameters:	args (object) – Any number of objects. If the object is a dask collection, it’s computed and the result is returned. Otherwise it’s passed through unchanged.

copy()¶

Return a shallow copy of the object, where each column is a reference of the corresponding column in self.

Note

No copy of data is made.

Note

This is different from view in that the attributes dictionary of the copy no longer related to self.

Returns:	a new CatalogSource that holds all of the data columns of self
Return type:	CatalogSource

csize¶

The total, collective size of the CatalogSource, i.e., summed across all ranks.

It is the sum of size across all available ranks.

If the base attribute is set, the base.csize attribute will be returned.

get_hardcolumn(col)¶

Return a column from the underlying data array/dict.

Columns are returned as dask arrays.

gslice(start, stop, end=1, redistribute=True)¶

Execute a global slice of a CatalogSource.

Note

After the global slice is performed, the data is scattered evenly across all ranks.

Note

The current algorithm generates an index on the root rank and does not scale well.

Parameters:	start (int) – the start index of the global slice stop (int) – the stop index of the global slice step (int, optional) – the default step size of the global size redistribute (bool, optional) – if True, evenly re-distribute the sliced data across all ranks, otherwise just return any local data part of the global slice

hardcolumns¶

The union of the columns in the file and any transformed columns.

static make_column(array)¶

Utility function to convert an array-like object to a dask.array.Array.

Note

The dask array chunk size is controlled via the dask_chunk_size global option. See set_options.

Parameters:	array (array_like) – an array-like object; can be a dask array, numpy array, ColumnAccessor, or other non-scalar array-like object
Returns:	a dask array initialized from array
Return type:	dask.array.Array

read(columns)¶

Return the requested columns as dask arrays.

Parameters:	columns (list of str) – the names of the requested columns
Returns:	the list of column data, in the form of dask arrays
Return type:	list of dask.array.Array

repopulate(seed=None, **params)[source]¶

Re-populate the catalog in-place, using the specified seed or model parameters.

This re-uses the model that was last used to create this catalog. It is faster than HaloCatalog.populate() as it avoids initialization steps. It is intended to be used when looping over different parameter sets, e.g., when performing parameter optimization.

Note

This operation is performed in-place.

Parameters:	seed (int, optional) – the random seed to use when populating the mock **params – key/value pairs specifying the model parameters to use

save(output, columns, dataset=None, datasets=None, header='Header')¶

Save the CatalogSource to a bigfile.BigFile.

Only the selected columns are saved and attrs are saved in header. The attrs of columns are stored in the datasets.

Parameters:

output (str) – the name of the file to write to columns (list of str) – the names of the columns to save in the file dataset (str, optional) – dataset to store the columns under. datasets (list of str, optional) – names for the data set where each column is stored; defaults to the name of the column (deprecated) header (str, optional, or None) – the name of the data set holding the header information, where attrs is stored if header is None, do not save the header.

size¶

The number of objects in the CatalogSource on the local rank.

If the base attribute is set, the base.size attribute will be returned.

Important

This property must be defined for all subclasses.

sort(keys, reverse=False, usecols=None)¶

Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.

Sort columns must be floating or integer type.

Note

After the sort operation, the data is scattered evenly across all ranks.

Parameters:	keys (list, tuple) – the names of columns to sort by. If multiple columns are provided, the data is sorted consecutively in the order provided reverse (bool, optional) – if True, perform descending sort operations usecols (list, optional) – the name of the columns to include in the returned CatalogSource

to_mesh(Nmesh=None, BoxSize=None, dtype='f4', interlaced=False, compensated=False, resampler='cic', weight='Weight', value='Value', selection='Selection', position='Position', window=None)¶

Convert the CatalogSource to a MeshSource, using the specified parameters.

Parameters:	Nmesh (int, optional) – the number of cells per side on the mesh; must be provided if not stored in attrs BoxSize (scalar, 3-vector, optional) – the size of the box; must be provided if not stored in attrs dtype (string, optional) – the data type of the mesh array interlaced (bool, optional) – use the interlacing technique of Sefusatti et al. 2015 to reduce the effects of aliasing on Fourier space quantities computed from the mesh compensated (bool, optional) – whether to correct for the resampler window introduced by the grid interpolation scheme resampler (str, optional) – the string specifying which resampler interpolation scheme to use; see pmesh.resampler.methods weight (str, optional) – the name of the column specifying the weight for each particle value (str, optional) – the name of the column specifying the field value for each particle selection (str, optional) – the name of the column that specifies which (if any) slice of the CatalogSource to take position (str, optional) – the name of the column that specifies the position data of the objects in the catalog window (str, deprecated) – use resampler instead.
Returns:	mesh – a mesh object that provides an interface for gridding particle data onto a specified mesh
Return type:	CatalogMesh

to_subvolumes(domain=None, position='Position', columns=None)¶

Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. Using the position column as the Position.

This will read in the full position array and all of the requested columns.

Parameters:	domain (:pyclass:`pmesh.domain.GridND` object, or None) – The domain to distribute the catalog. If None, try to evenly divide spatially. An easiest way to find a domain object is to use pm.domain, where pm is a :pyclass:`pmesh.pm.ParticleMesh` object. position (string_like) – column to use to compute the position. columns (list of string_like) – columns to include in the new catalog, if not supplied, all catalogs will be exchanged.
Returns:	A decomposed catalog source, where each rank only contains objects belongs to the rank as claimed by the domain object. self.attrs are carried over as a shallow copy to the returned object.
Return type:	CatalogSource

view(type=None)¶

Return a “view” of the CatalogSource object, with the returned type set by type.

This initializes a new empty class of type type and attaches attributes to it via the __finalize__() mechanism.

Parameters:	type (Python type) – the desired class type of the returned object.