Classes
RandomCatalog (csize[, seed, comm]) |
A CatalogSource that can have columns added via a collective random number generator. |
UniformCatalog (nbar, BoxSize[, seed, dtype, …]) |
A CatalogSource that has uniformly-distributed Position and Velocity columns. |
nbodykit.source.catalog.uniform.
RandomCatalog
(csize, seed=None, comm=None)[source]¶A CatalogSource that can have columns added via a collective random number generator.
The random number generator stored as rng
behaves
as numpy.random.RandomState
but generates random
numbers only on the local rank in a manner independent of
the number of ranks.
Parameters: | |
---|---|
Attributes: |
|
Methods
Selection () |
A boolean column that selects a subset slice of the CatalogSource. |
Value () |
When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell. |
Weight () |
The column giving the weight to use for each particle on the mesh. |
compute (*args, **kwargs) |
Our version of dask.compute() that computes multiple delayed dask collections at once. |
copy () |
Return a shallow copy of the object, where each column is a reference of the corresponding column in self . |
get_hardcolumn (col) |
Construct and return a hard-coded column. |
gslice (start, stop[, end, redistribute]) |
Execute a global slice of a CatalogSource. |
make_column (array) |
Utility function to convert an array-like object to a dask.array.Array . |
persist ([columns]) |
Return a CatalogSource, where the selected columns are computed and persist in memory. |
read (columns) |
Return the requested columns as dask arrays. |
save (output[, columns, dataset, datasets, …]) |
Save the CatalogSource to a bigfile.BigFile . |
sort (keys[, reverse, usecols]) |
Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys. |
to_mesh ([Nmesh, BoxSize, dtype, interlaced, …]) |
Convert the CatalogSource to a MeshSource, using the specified parameters. |
to_subvolumes ([domain, position, columns]) |
Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. |
view ([type]) |
Return a “view” of the CatalogSource object, with the returned type set by type . |
create_instance |
Index
¶The attribute giving the global index rank of each particle in the
list. It is an integer from 0 to self.csize
.
Note that slicing changes this index value.
Selection
()¶A boolean column that selects a subset slice of the CatalogSource.
By default, this column is set to True
for all particles, and
all CatalogSource objects will contain this column.
Value
()¶When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.
The mesh field is a weighted average of Value
, with the weights
given by Weight
.
By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.
Weight
()¶The column giving the weight to use for each particle on the mesh.
The mesh field is a weighted average of Value
, with the weights
given by Weight
.
By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.
__delitem__
(col)¶Delete a column; cannot delete a “hard-coded” column.
Note
If the base
attribute is set, columns will be deleted
from base
instead of from self
.
__finalize__
(other)¶Finalize the creation of a CatalogSource object by copying over any additional attributes from a second CatalogSource.
The idea here is to only copy over attributes that are similar
to meta-data, so we do not copy some of the core attributes of the
CatalogSource
object.
Parameters: | other – the second object to copy over attributes from; it needs to be a subclass of CatalogSourcBase for attributes to be copied |
---|---|
Returns: | return self , with the added attributes |
Return type: | CatalogSource |
__getitem__
(sel)¶The following types of indexing are supported:
Notes
base
attribute is set, columns will be returned
from base
instead of from self
.__len__
()¶The local size of the CatalogSource on a given rank.
__setitem__
(col, value)¶Add columns to the CatalogSource, overriding any existing columns
with the name col
.
attrs
¶A dictionary storing relevant meta-data about the CatalogSource.
columns
¶All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.
Note
If the base
attribute is set, the value of
base.columns
will be returned.
compute
(*args, **kwargs)¶Our version of dask.compute()
that computes
multiple delayed dask collections at once.
This should be called on the return value of read()
to converts any dask arrays to numpy arrays.
base
attribute is set, compute()
will called using base
instead of self
.Parameters: | args (object) – Any number of objects. If the object is a dask collection, it’s computed and the result is returned. Otherwise it’s passed through unchanged. |
---|
copy
()¶Return a shallow copy of the object, where each column is a reference
of the corresponding column in self
.
Note
No copy of data is made.
Note
This is different from view in that the attributes dictionary
of the copy no longer related to self
.
Returns: | a new CatalogSource that holds all of the data columns of self |
---|---|
Return type: | CatalogSource |
csize
¶The total, collective size of the CatalogSource, i.e., summed across all ranks.
It is the sum of size
across all available ranks.
If the base
attribute is set, the base.csize
attribute
will be returned.
get_hardcolumn
(col)¶Construct and return a hard-coded column.
These are usually produced by calling member functions marked by the
@column
decorator.
Subclasses may override this method and the hardcolumns attribute to bypass the decorator logic.
Note
If the base
attribute is set, get_hardcolumn()
will called using base
instead of self
.
gslice
(start, stop, end=1, redistribute=True)¶Execute a global slice of a CatalogSource.
Note
After the global slice is performed, the data is scattered evenly across all ranks.
Note
The current algorithm generates an index on the root rank and does not scale well.
Parameters: |
|
---|
hardcolumns
¶A list of the hard-coded columns in the CatalogSource.
These columns are usually member functions marked by @column
decorator. Subclasses may override this method and use
get_hardcolumn()
to bypass the decorator logic.
Note
If the base
attribute is set, the value of
base.hardcolumns
will be returned.
make_column
(array)¶Utility function to convert an array-like object to a
dask.array.Array
.
Note
The dask array chunk size is controlled via the dask_chunk_size
global option. See set_options
.
Parameters: | array (array_like) – an array-like object; can be a dask array, numpy array, ColumnAccessor, or other non-scalar array-like object |
---|---|
Returns: | a dask array initialized from array |
Return type: | dask.array.Array |
persist
(columns=None)¶Return a CatalogSource, where the selected columns are computed and persist in memory.
read
(columns)¶Return the requested columns as dask arrays.
Parameters: | columns (list of str) – the names of the requested columns |
---|---|
Returns: | the list of column data, in the form of dask arrays |
Return type: | list of dask.array.Array |
rng
¶A MPIRandomState
that behaves as
numpy.random.RandomState
but generates random
numbers in a manner independent of the number of ranks.
save
(output, columns=None, dataset=None, datasets=None, header='Header', compute=True)¶Save the CatalogSource to a bigfile.BigFile
.
Only the selected columns are saved and attrs
are saved in
header
. The attrs of columns are stored in the datasets.
Parameters: |
|
---|
size
¶The number of objects in the CatalogSource on the local rank.
If the base
attribute is set, the base.size
attribute
will be returned.
Important
This property must be defined for all subclasses.
sort
(keys, reverse=False, usecols=None)¶Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.
Sort columns must be floating or integer type.
Note
After the sort operation, the data is scattered evenly across all ranks.
Parameters: |
|
---|
to_mesh
(Nmesh=None, BoxSize=None, dtype='f4', interlaced=False, compensated=False, resampler='cic', weight='Weight', value='Value', selection='Selection', position='Position', window=None)¶Convert the CatalogSource to a MeshSource, using the specified parameters.
Parameters: |
|
---|---|
Returns: | mesh – a mesh object that provides an interface for gridding particle data onto a specified mesh |
Return type: |
to_subvolumes
(domain=None, position='Position', columns=None)¶Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. Using the position column as the Position.
This will read in the full position array and all of the requested columns.
Parameters: |
|
---|---|
Returns: | A decomposed catalog source, where each rank only contains objects belongs to the rank as claimed by the domain object. self.attrs are carried over as a shallow copy to the returned object. |
Return type: |
view
(type=None)¶Return a “view” of the CatalogSource object, with the returned
type set by type
.
This initializes a new empty class of type type
and attaches
attributes to it via the __finalize__()
mechanism.
Parameters: | type (Python type) – the desired class type of the returned object. |
---|
nbodykit.source.catalog.uniform.
UniformCatalog
(nbar, BoxSize, seed=None, dtype='f8', comm=None)[source]¶A CatalogSource that has uniformly-distributed Position
and Velocity
columns.
The random numbers generated do not depend on the number of available ranks.
Parameters: | |
---|---|
Attributes: |
|
Methods
Position () |
The position of particles, uniformly distributed in BoxSize |
Selection () |
A boolean column that selects a subset slice of the CatalogSource. |
Value () |
When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell. |
Velocity () |
The velocity of particles, uniformly distributed in 0.01 x BoxSize |
Weight () |
The column giving the weight to use for each particle on the mesh. |
compute (*args, **kwargs) |
Our version of dask.compute() that computes multiple delayed dask collections at once. |
copy () |
Return a shallow copy of the object, where each column is a reference of the corresponding column in self . |
get_hardcolumn (col) |
Construct and return a hard-coded column. |
gslice (start, stop[, end, redistribute]) |
Execute a global slice of a CatalogSource. |
make_column (array) |
Utility function to convert an array-like object to a dask.array.Array . |
persist ([columns]) |
Return a CatalogSource, where the selected columns are computed and persist in memory. |
read (columns) |
Return the requested columns as dask arrays. |
save (output[, columns, dataset, datasets, …]) |
Save the CatalogSource to a bigfile.BigFile . |
sort (keys[, reverse, usecols]) |
Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys. |
to_mesh ([Nmesh, BoxSize, dtype, interlaced, …]) |
Convert the CatalogSource to a MeshSource, using the specified parameters. |
to_subvolumes ([domain, position, columns]) |
Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. |
view ([type]) |
Return a “view” of the CatalogSource object, with the returned type set by type . |
create_instance |
Index
¶The attribute giving the global index rank of each particle in the
list. It is an integer from 0 to self.csize
.
Note that slicing changes this index value.
Selection
()¶A boolean column that selects a subset slice of the CatalogSource.
By default, this column is set to True
for all particles, and
all CatalogSource objects will contain this column.
Value
()¶When interpolating a CatalogSource on to a mesh, the value of this array is used as the Value that each particle contributes to a given mesh cell.
The mesh field is a weighted average of Value
, with the weights
given by Weight
.
By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.
Weight
()¶The column giving the weight to use for each particle on the mesh.
The mesh field is a weighted average of Value
, with the weights
given by Weight
.
By default, this array is set to unity for all particles, and all CatalogSource objects will contain this column.
__delitem__
(col)¶Delete a column; cannot delete a “hard-coded” column.
Note
If the base
attribute is set, columns will be deleted
from base
instead of from self
.
__finalize__
(other)¶Finalize the creation of a CatalogSource object by copying over any additional attributes from a second CatalogSource.
The idea here is to only copy over attributes that are similar
to meta-data, so we do not copy some of the core attributes of the
CatalogSource
object.
Parameters: | other – the second object to copy over attributes from; it needs to be a subclass of CatalogSourcBase for attributes to be copied |
---|---|
Returns: | return self , with the added attributes |
Return type: | CatalogSource |
__getitem__
(sel)¶The following types of indexing are supported:
Notes
base
attribute is set, columns will be returned
from base
instead of from self
.__len__
()¶The local size of the CatalogSource on a given rank.
__setitem__
(col, value)¶Add columns to the CatalogSource, overriding any existing columns
with the name col
.
attrs
¶A dictionary storing relevant meta-data about the CatalogSource.
columns
¶All columns in the CatalogSource, including those hard-coded into the class’s defintion and override columns provided by the user.
Note
If the base
attribute is set, the value of
base.columns
will be returned.
compute
(*args, **kwargs)¶Our version of dask.compute()
that computes
multiple delayed dask collections at once.
This should be called on the return value of read()
to converts any dask arrays to numpy arrays.
base
attribute is set, compute()
will called using base
instead of self
.Parameters: | args (object) – Any number of objects. If the object is a dask collection, it’s computed and the result is returned. Otherwise it’s passed through unchanged. |
---|
copy
()¶Return a shallow copy of the object, where each column is a reference
of the corresponding column in self
.
Note
No copy of data is made.
Note
This is different from view in that the attributes dictionary
of the copy no longer related to self
.
Returns: | a new CatalogSource that holds all of the data columns of self |
---|---|
Return type: | CatalogSource |
csize
¶The total, collective size of the CatalogSource, i.e., summed across all ranks.
It is the sum of size
across all available ranks.
If the base
attribute is set, the base.csize
attribute
will be returned.
get_hardcolumn
(col)¶Construct and return a hard-coded column.
These are usually produced by calling member functions marked by the
@column
decorator.
Subclasses may override this method and the hardcolumns attribute to bypass the decorator logic.
Note
If the base
attribute is set, get_hardcolumn()
will called using base
instead of self
.
gslice
(start, stop, end=1, redistribute=True)¶Execute a global slice of a CatalogSource.
Note
After the global slice is performed, the data is scattered evenly across all ranks.
Note
The current algorithm generates an index on the root rank and does not scale well.
Parameters: |
|
---|
hardcolumns
¶A list of the hard-coded columns in the CatalogSource.
These columns are usually member functions marked by @column
decorator. Subclasses may override this method and use
get_hardcolumn()
to bypass the decorator logic.
Note
If the base
attribute is set, the value of
base.hardcolumns
will be returned.
make_column
(array)¶Utility function to convert an array-like object to a
dask.array.Array
.
Note
The dask array chunk size is controlled via the dask_chunk_size
global option. See set_options
.
Parameters: | array (array_like) – an array-like object; can be a dask array, numpy array, ColumnAccessor, or other non-scalar array-like object |
---|---|
Returns: | a dask array initialized from array |
Return type: | dask.array.Array |
persist
(columns=None)¶Return a CatalogSource, where the selected columns are computed and persist in memory.
read
(columns)¶Return the requested columns as dask arrays.
Parameters: | columns (list of str) – the names of the requested columns |
---|---|
Returns: | the list of column data, in the form of dask arrays |
Return type: | list of dask.array.Array |
rng
¶A MPIRandomState
that behaves as
numpy.random.RandomState
but generates random
numbers in a manner independent of the number of ranks.
save
(output, columns=None, dataset=None, datasets=None, header='Header', compute=True)¶Save the CatalogSource to a bigfile.BigFile
.
Only the selected columns are saved and attrs
are saved in
header
. The attrs of columns are stored in the datasets.
Parameters: |
|
---|
size
¶The number of objects in the CatalogSource on the local rank.
If the base
attribute is set, the base.size
attribute
will be returned.
Important
This property must be defined for all subclasses.
sort
(keys, reverse=False, usecols=None)¶Return a CatalogSource, sorted globally across all MPI ranks in ascending order by the input keys.
Sort columns must be floating or integer type.
Note
After the sort operation, the data is scattered evenly across all ranks.
Parameters: |
|
---|
to_mesh
(Nmesh=None, BoxSize=None, dtype='f4', interlaced=False, compensated=False, resampler='cic', weight='Weight', value='Value', selection='Selection', position='Position', window=None)¶Convert the CatalogSource to a MeshSource, using the specified parameters.
Parameters: |
|
---|---|
Returns: | mesh – a mesh object that provides an interface for gridding particle data onto a specified mesh |
Return type: |
to_subvolumes
(domain=None, position='Position', columns=None)¶Domain Decompose a catalog, sending items to the ranks according to the supplied domain object. Using the position column as the Position.
This will read in the full position array and all of the requested columns.
Parameters: |
|
---|---|
Returns: | A decomposed catalog source, where each rank only contains objects belongs to the rank as claimed by the domain object. self.attrs are carried over as a shallow copy to the returned object. |
Return type: |
view
(type=None)¶Return a “view” of the CatalogSource object, with the returned
type set by type
.
This initializes a new empty class of type type
and attaches
attributes to it via the __finalize__()
mechanism.
Parameters: | type (Python type) – the desired class type of the returned object. |
---|