TOAST

The sotodlib.toast package contains toast operators for use in workflows for Simons Observatory. The first step is to make sure you have TOAST (and sotodlib) installed.

User Installation

If you are using a stable release, then you can install the toast package directly from either a python wheel or a conda package. Make sure to create a virtualenv or conda env first:

conda install toast

OR:

pip install toast

And follow that by installing so3g:

pip install so3g

and sotodlib:

pip install git+https://github.com/simonsobs/sotodlib.git@master

OR, if you have a local git checkout of sotodlib:

cd sotodlib
pip install .

Note

Until toast-3.0 is released, use a pre-release python wheel from PyPI by doing pip install --pre toast.

Developer Installation

If you will be installing TOAST and modifying the source, then the easiest way forward is to use a conda environment. This is also the path to take if you want to use the latest upstream code version.

To begin, make sure you already have a recent conda “base” environment. If you are starting from scratch, I recommend running the “miniforge” installer from conda-forge. Make sure your base conda environment is activated:

conda env list

Now decide on a name for your new development environment. In this example we use “simons”. Clone toast locally and checkout the toast3 branch:

git clone https://github.com/hpc4cmb/toast.git
cd toast
git checkout -b toast3 -t origin/toast3

Next we will run a helper script to create (or activate) our new conda env and install toast dependencies. Use the name you have chosen here:

cd toast
./platforms/conda_dev_setup.sh simons

This new environment should show up with:

conda env list

Now we can install the toast package itself after activating our environment:

cd toast
conda activate simons
./platforms/conda.sh

Whenever you pull new changes to your git checkout, you can re-run the platforms/conda.sh script to install an updated version of the toast package. Now install so3g:

pip install so3g

and sotodlib:

pip install git+https://github.com/simonsobs/sotodlib.git@master

OR, if you have a local git checkout of sotodlib:

cd sotodlib
pip install .

Interactive Use

Example notebooks will be placed in pwg-tutorials/toast. You can mix and match high-level “workflow” functions to do broad tasks (load data, simulate observing, mapmaking, etc) and other individual operators or your own custom code.

Batch Use

When running workflows in batch mode, we typically use a slightly different structure from a notebook. There is usually additional boilerplate code to ensure that any unhandled python exceptions correctly terminate an MPI job. There is often more extensive logging of the job configuration and also workflow functions will get their arguments exclusively from config files and commandline arguments.

There is a large-scale simulation and reduction workflow in sotodlib/toast/scripts/so_sim.py. This script has nearly every possible operator and focuses on a typical sequence of simulating different effects and multiple different analyses. Different reduction paths and mapmakers can be selectively enabled or disabled from the commandline or config files.

For processing data that already exists on disk, you can look at sotodlib/toast/scripts/so_map.py as an example workflow. This loads data in any of the supported formats and has multiple mapmaking / reduction code paths that can be enabled.

Of course not every script needs to include all of these simulation and reduction techniques. You can make small, focused workflows that do only the things you want.

Reference

This is an API reference for the software in sotodlib.toast.

Workflow Tools

For high-level operations (e.g. “load some data”, “make a map”, etc) it is often helpful to leverage the workflow functions that can use a mix of config files and user parameters to setup and run a pre-defined set of operations. The best examples are in the source files sotodlib/toast/workflows/so_*.py. The basic pattern is to select the high level operations you want to use, call the “setup” function for each, call the setup_job function to merge options from config files and other sources, and then run the operations in the desired sequence.

Here is a list of the supported high-level operations:

sotodlib.toast.workflows.setup_load_data_context(operators)[source]

Add commandline args and operators for loading from a context.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.load_data_context(job, otherargs, runargs, data)[source]

Load data from a Context.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_load_data_hdf5(operators)[source]

Add commandline args and operators for loading from HDF5 files.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.load_data_hdf5(job, otherargs, runargs, data)[source]

Load data from one or more HDF5 files.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_load_data_books(operators)[source]

Add commandline args and operators for directly loading L3 books.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.load_data_books(job, otherargs, runargs, data)[source]

Load data from one or more books.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_save_data_hdf5(operators)[source]

Add commandline args and operators for saving to HDF5 files.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.save_data_hdf5(job, otherargs, runargs, data)[source]

Save data to HDF5 files.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_save_data_books(operators)[source]

Add commandline args and operators for saving to L3 books.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.save_data_books(job, otherargs, runargs, data)[source]

Save data to books.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_pointing(operators)[source]

Add commandline args and operators for pointing.

This sets up operators different types of pointing matrix scenarios.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.select_pointing(job, otherargs, runargs, data)[source]

Select pointing scheme and check for consistency

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_demodulate(operators)[source]

Add commandline args and operators for demodulation.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.demodulate(job, otherargs, runargs, data)[source]

Run timestream demodulation.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The input data container.

Returns:

The new, demodulated data.

Return type:

(Data)

sotodlib.toast.workflows.setup_deconvolve_detector_timeconstant(operators)[source]

Add commandline args and operators for timeconstant convolution.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.deconvolve_detector_timeconstant(job, otherargs, runargs, data)[source]

Deconvolve the detector timeconstants.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filter_hwpss(operators)[source]

Add commandline args and operators for HWPSS filters.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filter_hwpss(job, otherargs, runargs, data)[source]

Filter HWP synchronous signal.

This filters the HWPSS, as opposed to solving for it in the mapmaking with a template.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filter_ground(operators)[source]

Add commandline args and operators for ground filters.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filter_ground(job, otherargs, runargs, data)[source]

Filter Azimuth synchronous signal.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filter_poly1d(operators)[source]

Add commandline args and operators for 1D polynomial filters.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filter_poly1d(job, otherargs, runargs, data)[source]

Filter scans with 1D polynomial.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filter_poly2d(operators)[source]

Add commandline args and operators for 2D polynomial filters.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filter_poly2d(job, otherargs, runargs, data)[source]

Filter data with a 2D polynomial.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filter_common_mode(operators)[source]

Add commandline args and operators for common mode filters.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filter_common_mode(job, otherargs, runargs, data)[source]

Filter data to remove common modes across the detectors.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_flag_sso(operators)[source]

Add commandline args and operators for SSO flagging.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.flag_sso(job, otherargs, runargs, data)[source]

Flag solarsystem objects.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_flag_noise_outliers(operators)[source]

Add commandline args and operators for flagging detectors with bad noise.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.flag_noise_outliers(job, otherargs, runargs, data)[source]

Flag detectors with extremely bad noise properties.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_mapmaker_filterbin(operators)[source]

Add commandline args, operators, and templates for TOAST FilterBin mapmaker.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.mapmaker_filterbin(job, otherargs, runargs, data)[source]

Run the TOAST FilterBin mapmaker.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_mapmaker_madam(operators)[source]

Add commandline args and operators for the MADAM mapmaker.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.mapmaker_madam(job, otherargs, runargs, data)[source]

Run the MADAM mapmaker.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_mapmaker_ml(operators)[source]

Add commandline args and operators for the S.O. maximum likelihood mapmaker.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.mapmaker_ml(job, otherargs, runargs, data)[source]

Run the S.O. maximum likelihood mapmaker.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_mapmaker(operators, templates)[source]

Add commandline args, operators, and templates for TOAST mapmaker.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.mapmaker(job, otherargs, runargs, data)[source]

Run the TOAST mapmaker.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_noise_estimation(operators)[source]

Add commandline args and operators for estimating detector noise.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.noise_estimation(job, otherargs, runargs, data)[source]

Run noise estimation and create a best-fit 1/f model.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_raw_statistics(operators)[source]

Add commandline args and operators for raw timestream statistics.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.raw_statistics(job, otherargs, runargs, data)[source]

Compute timestream statistics on the raw data.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_filtered_statistics(operators)[source]

Add commandline args and operators for filtered timestream statistics.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.filtered_statistics(job, otherargs, runargs, data)[source]

Compute timestream statistics on the filtered data.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_hn_map(operators)[source]

Add commandline args and operators for H_n map.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.hn_map(job, otherargs, runargs, data)[source]

Compute the H_n map.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_cadence_map(operators)[source]

Add commandline args and operators for the cadence map.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.cadence_map(job, otherargs, runargs, data)[source]

Compute the cadence map.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_crosslinking_map(operators)[source]

Add commandline args and operators for the crosslinking map.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.crosslinking_map(job, otherargs, runargs, data)[source]

Compute the crosslinking map.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_observing(parser, operators)[source]

Add commandline args and operators for simulated observing.

Parameters:

parser (ArgumentParser) – The parser to update.
operators (list) – The list of operators to extend.

Returns:

None

sotodlib.toast.workflows.simulate_observing(job, otherargs, runargs, comm)[source]

Simulate the observing motion of the selected detectors with the schedule.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
comm (MPI.Comm) – The MPI world communicator (or None).

Returns:

The simulated data.

Return type:

(Data)

sotodlib.toast.workflows.setup_simple_noise_models(operators)[source]

Add commandline args and operators for simple noise models.

These noise models use just nominal values from the focalplane and an elevation-weighting of the NET.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simple_noise_models(job, otherargs, runargs, data)[source]

Generate trivial noise models.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_sky_map_signal(operators)[source]

Add commandline args and operators for scanning from a map.

There are pointing matrix operators associated with each scan map operator and these are disabled by default. If not enabled, the scan map operator use the same pointing matrix as the mapmaking.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_sky_map_signal(job, otherargs, runargs, data)[source]

Scan a sky map into detector signals.

This uses detector pointing to sample from a map into detector timestreams. The maps should already be smoothed with any desired beam effects.

We scan the sky with the “final” pointing model in case that is different from the solver pointing model.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_conviqt_signal(operators)[source]

Add commandline args and operators for beam covolution with conviqt.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_conviqt_signal(job, otherargs, runargs, data)[source]

Use libconviqt to generate beam-convolved sky signal timestreams.

This uses detector pointing and input beam and sky a_lm expansions to generate detector timestreams.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_source_signal(operators)[source]

Add commandline args and operators for artifical sources.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_source_signal(job, otherargs, runargs, data)[source]

Simulate detector response from artificial sources.

This uses Az/El detector pointing to observe a terrestrial calibration source (e.g. a drone).

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_sso_signal(operators)[source]

Add commandline args and operators for solar system objects.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_sso_signal(job, otherargs, runargs, data)[source]

Simulate detector response from solar system objects.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_catalog_signal(operators)[source]

Add commandline args and operators for point source catalogs.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_catalog_signal(job, otherargs, runargs, data)[source]

Simulate detector response from a point source catalog.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_atmosphere_signal(operators)[source]

Add commandline args and operators for simulating atmosphere.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_atmosphere_signal(job, otherargs, runargs, data)[source]

Simulate atmosphere signal.

This creates or loads a realization of the atmosphere for each observing session and uses detector pointing to integrate the power seen by each detector at each sample.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_wiregrid_signal(operators)[source]

Add commandline args and operators for simulating wiregrid.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_wiregrid_signal(job, otherargs, runargs, data)[source]

Simulate wire grid signal.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_stimulator_signal(operators)[source]

Add commandline args and operators for simulating the stimulator.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_stimulator_signal(job, otherargs, runargs, data)[source]

Simulate stimulator signal.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_scan_synchronous_signal(operators)[source]

Add commandline args and operators for scan-synchronous signal.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_scan_synchronous_signal(job, otherargs, runargs, data)[source]

Simulate scan-synchronous signal.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_hwpss_signal(operators)[source]

Add commandline args and operators for HWP synchronous signal.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_hwpss_signal(job, otherargs, runargs, data)[source]

Simulate HWP synchronous signal.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_detector_timeconstant(operators)[source]

Add commandline args and operators for timeconstant convolution.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_detector_timeconstant(job, otherargs, runargs, data)[source]

Simulate the effects of detector timeconstants.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_detector_noise(operators)[source]

Add commandline args and operators for simulating detector noise.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_detector_noise(job, otherargs, runargs, data)[source]

Simulate the intrinsic detector and readout noise.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_readout_effects(operators)[source]

Add commandline args and operators for simulating readout effects.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_readout_effects(job, otherargs, runargs, data)[source]

Simulate various readout effects.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_detector_yield(operators)[source]

Add commandline args and operators for simulating detector yield.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_detector_yield(job, otherargs, runargs, data)[source]

Simulate detector yield.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_simulate_calibration_error(operators)[source]

Add commandline args and operators for simulating gain error.

Parameters:: operators (list) – The list of operators to extend.
Returns:: None

sotodlib.toast.workflows.simulate_calibration_error(job, otherargs, runargs, data)[source]

Simulate calibration errors.

Parameters:

job (namespace) – The configured operators and templates for this job.
otherargs (namespace) – Other commandline arguments.
runargs (namespace) – Job related runtime parameters.
data (Data) – The data container.

Returns:

None

sotodlib.toast.workflows.setup_job(parser=None, operators=[], templates=[], opts=None, config_files=None, argparse_opts=None, verbose=False)[source]

Generate configuration for a workflow and parse options.

Given a list of operators and templates to be used in a workflow, build the default configuration. Then parse any option overrides from the commandline OR from the combination of argparse option dictionary and config files. If an existing ArgumentParser is given, then it is used. Otherwise a new empty parser is created.

This function is designed to work from a script running in a batch job as well as interactively from a notebook. The final configuration can optionally be written out for reference.

For interactive use or ad-hoc scripts, the opts dictionary can be passed with options matching typical commandline syntax without the leading dashes. A list of config files to load can also be supplied with config_files.

For non-interactive scripts that need to parse commandline arguments, leave opts and config_files at None. Or the commandline can be replaced with a list of argparse-style options in argparse_opts.

Parameters:

parser (ArgumentParser) – The input parser to modify. If None, a new one is created.
operators (list) – The list of operator instances to configure.
templates (list) – The list of template instances to configure.
opts (dict) – The dictionary of interactive config options.
config_files (list) – A list of configuration files to load.
argparse_opts (list) – When running in batch mode, an argparse list of command line arguments to use instead of parsing sys.argv.
verbose (bool) – If True, dump out the final config to stdout.

Returns:

The (job namespace, config dictionary, other args, runtime args): from the result of parsing opts + config_files or command line and then configuring all operators and templates.

Return type:

(tuple)

Simulated Instrument Model

The TOAST instrument model consists of a Telescope instance which is associated with a Site and a Focalplane. These are specified for every observation- typically a single wafer and frequency for some constant elevation scan. When working with a “bootstrapped” instrument model that is simulated from nominal parameters, we have use a special Focalplane class which uses the synthetic hardware model to build the table of detector properties and boresight offsets:

class sotodlib.toast.SOFocalplane(*args: Any, **kwargs: Any)[source]

SO Focalplane class.

This can be constructed in several ways:

From a file while applying selection criteria.
From an existing (pre-selected) Hardware instance.
From a nominal sim on the fly with selections.

Note that to support serialization, the simulated hardware dictionary elements have values in standard units. When constructing our focalplane table we restore those units and build Quantities.

Parameters:

hw (Hardware) – If specified, construct from a Hardware object in memory.
hwfile (str) – If specified, load this hardware model from disk.
det_info_file (str) – If simulating a Hardware model, optionally specify a det_info format file to load
det_info_version (str) – The version of the det_info file format.
telescope (str) – If not None, select only detectors from this telescope.
sample_rate (Quantity) – Use this sample rate for all detectors.
bands (str) – Comma separated string of bands to use.
wafer_slots (str) – Comma separated string of wafers to use.
tube_slots (str) – Comma separated string of tubes to use.
thinfp (int) – The factor by which to reduce the number of detectors.
creation_time (float) – Optional timestamp to use when building readout_id.
comm (MPI.Comm) – Optional MPI communicator.

Instrument Model for Real Data

When starting from real data, we already have most properties available directly and associated with a readout ID. We use this to build our Focalplane detector table directly. If the detector to readout mapping is available, the detector name is also added to this table. Similarly for the detector focalplane offsets.

Simulation Operators

Here we document only the tools that are specific to Simons Observatory. There are many other generic things that are supplied in toast. The operators in sotodlib.toast.ops can be used in a workflow along with the standard operators in toast.

class sotodlib.toast.ops.CoRotator(*args: Any, **kwargs: Any)

Compute the LAT focalplane rotation.

The optical design configuration of the LAT projected on the sky has the “O6” cryo tube in the positive azimuth direction when the telescope is at 60 degrees elevation.

If corotate_lat is False, then the projected focalplane will rotate on the sky. The saved corotation angle will be zero. If corotate_lat is True, then the corotation angle is set to compensate for the rotation caused by the observing elevation.

The rotation of the projected focalplane is given by:

R = elevation - offset - corotator angle

This operator should be applied before expanding the detector pointing from boresight.

class sotodlib.toast.ops.SimSSO(*args: Any, **kwargs: Any): Operator that generates Solar System Object timestreams.

class sotodlib.toast.ops.SimCatalog(*args: Any, **kwargs: Any)

Operator that generates variable and static point source signal.

Signal is generated by sampling the provided beam map at appropriate locations and scaling the resulting signal to match the perceived intensity of the source.

Source SED is convolved with the detector bandpass recorded in the focalplane table.

Example catalog entries:

[example_static_source]
# Celestial coordinate are always given in degrees
ra_deg = 30
dec_deg = -30
# the SED can be specified using an arbitrary number of
# frequency bins.  The SED is interpolated in log-log space to
# convolve with the detector bandpass
# Use either `flux_density_mJy` or `flux_density_Jy` and adjust
# the values accordingly
freqs_ghz = [ 1.0, 1000.0,]
flux_density_mJy = [ 10.0, 1.0,]
# Omitting polarization fraction results in an
# unpolarized source
pol_frac = 0.1
pol_angle_deg = 0

[example_variable_source]
ra_deg = 30
dec_deg = -25
freqs_ghz = [ 1.0, 1000.0,]
# An arbitrary number of SED vectors can be provided but the
# location of the frequency bins is fixed.  Effective SED is
# interpolated between the specified epochs.
flux_density_Jy = [ [ 10.0, 1.0,], [ 30.0, 10.0,], [ 10.0, 1.0,],]
# Omitting the times_mjd entry resuls in a static source
times_mjd = [ 59000.0, 60000.0, 61000.0,]
# The polarization properties can also vary
pol_frac = [ 0.05, 0.15, 0.05,]
pol_angle_deg = [ 45, 45, 45,]

[example_transient_source]
ra_deg = 30
dec_deg = -20
freqs_ghz = [ 1.0, 1000.0,]
flux_density_Jy = [ [ 10.0, 1.0,], [ 30.0, 10.0,],]
# Difference between a variable and transient source is
# simply that the specified epochs do not cover the entire
# simulation time span.  The operator will not extrapolate
# outside the epochs.
times_mjd = [ 59410.0, 59411.0,]

class sotodlib.toast.ops.SimHWPSS(*args: Any, **kwargs: Any)

Simulate HWP synchronous signal.

NOTE: The HWPSS template is interpolated (with a 5th order spline interpolation). Interpolation errors produce spurious peaks in the frequency domain which are 7 to 8 orders of magnitude below the amplitude of the original signal.

class sotodlib.toast.ops.SimSource(*args: Any, **kwargs: Any): Operator that generates an Artificial Source timestreams.

class sotodlib.toast.ops.SimWireGrid(*args: Any, **kwargs: Any): Simulate wiregrid calibration signal

class sotodlib.toast.ops.SimStimulator(*args: Any, **kwargs: Any): Simulate stimulator calibration signal

class sotodlib.toast.ops.SimReadout(*args: Any, **kwargs: Any): Simulate various readout-related systematics

class sotodlib.toast.ops.SimMuMUXCrosstalk(*args: Any, **kwargs: Any)

Simulate nonlinear muMUX crosstalk

Simulate inductive and capacitive crosstalk in the readout:

phase_target = phase_target(true) + chi*sin(phase_source - phase_target)

Data Reduction Operators

These are the data reduction operators specific to Simons Observatory. Many other operators for different kinds of mapmaking, filtering, demodulation, etc are available in toast.

class sotodlib.toast.ops.Hn(*args: Any, **kwargs: Any): Evaluate geometrical h_n factors to support map-based simulations.

class sotodlib.toast.ops.MLMapmaker(*args: Any, **kwargs: Any): Operator which accumulates data to the Maximum Likelihood Mapmaker.

Data Load / Save Operators

The toast package has built-in operators for saving / loading data to / from a directory of HDF5 files. There are also several operators specific to Simons Observatory:

class sotodlib.toast.ops.SaveBooks(*args: Any, **kwargs: Any)

Export observations to Level-3 Book format.

Create one book per tube, per observing session. Each session consists of observations from the same telescope tube. Basic metadata is written to Observation frames at the start of each Primary File Group and the M_observation.yaml file.

class sotodlib.toast.ops.LoadBooks(*args: Any, **kwargs: Any)

Load Level-3 Books into observations.

If no focalplane directory is specified, a minimal focalplane is created using only the stream_id and readout names in the G3 files. Similarly for noise models. If the noise_dir is specified, noise models are loaded from there.

class sotodlib.toast.ops.LoadContext(*args: Any, **kwargs: Any)

Load one or more observations from a Context.

NOTE: each “observation” in the Context system maps to a toast “observing session”.

Given a context, load one or more observations. If specified, the context should exist on all processes in the group. The readout_ids, detsets, and bands traits are used to select subsets of detectors. Alternatively the context file can be specified, in which case this is passed to the context constructor.

The traits starting with an “ax_*” prefix indicate the axis manager names that should be mapped to the standard toast shared and detdata objects. For ax_flags tuples, a negative bit value indicates that the flag data should first be inverted before combined.

Additional nested axismanager data is grouped into detdata, shared, or metadata objects based on whether they use detector / sample axes. The nested structure is flattened into names built from the keys in the hierarchy.