HWP submodule

G3tHWP

G3tHWP is class to analyze HWP parameter in level 2 housekeeping g3 file

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]

Initialize

G3tHWP class has be instantiated with a config yaml file. The config file contains the HWP hardware information. We can include a time range + data path or a full-path file list of input g3 data.

hwp_angle_tool = hwp.G3tHWP('hwp_config.yaml')

Here’s an annotated example of yaml config file:

# path to L2 HK directory
data_dir: '/so/level2-daq/satp1/hk/'

# path to ManifestDb directory
output_dir: '/so/metadata/satp1/manifests/hwp_angles/'

# prefix of field name
# for 1st encoder
field_instance: 'satp1.hwp-bbb-e1.feeds.HWPEncoder'

# for 2nd encoder
field_instance_sub: 'satp1.hwp-bbb-e2.feeds.HWPEncoder'

# name list of HWP_encoder field variable
field_list:
- 'rising_edge_count'
- 'irig_time'
- 'counter'
- 'counter_index'
- 'irig_synch_pulse_clock_time'
- 'irig_synch_pulse_clock_counts'
- 'quad'

# HWP OCS packet size
packet_size: 120

# IRIG type
# 0: 1Hz IRIG (default), 1: 10Hz IRIG
irig_type: 0

# number of slots * 2
num_edges: 1140

# Number of slots representing the width of the reference slot
ref_edges: 2

# Threshoild for outlier data to calculate nominal slit width
slit_width_lim: 0.1

# Search range of reference slot
ref_range: 0.1

# force to quad value
# 0: use measured quad value (default)
# 1: positive rotation direction, -1: negative rotation direction
force_quad: 0

Data Loading

There are two functions to load L2 HK g3 file.

1) Load data for a given time range

HK data can be loaded by load_data function with time range and data file path.

data = hwp_angle_tool.load_data(start, end, data_dir)

We can also specify start, end and data_dir by config yaml file

# start and end time for `update_hwp_angle.load_data`
start: timestamp
end: timestamp

# HK data directory for `update_hwp_angle.load_data`
data_dir: '/path/to/L2/HK/data/'

If you input both, config file setting will be overwritten.

As optional, HWP field name instance can be overwritten from config file.

data = hwp_angle_tool.load_data(start, end, data_dir, instance)

2) Load data with given file name list

data = hwp_angle_tool.load_data(file_list)

We can also specify file_list by config yaml file

# input file name for `update_hwp_angle.load_file`
file_list: '/mnt/so1/data/ucsd-sat1/hk/16183/1618372860.g3'

If you input both, config file will be overwritten.

As optional, HWP field name instance can be overwritten from config file.

data = hwp_angle_tool.load_file(file_list, instance)

Analyze loaded data

We can simply get HWP angle and related info. by

solved = hwp_angle_tool.analyze(data)

where the return value is python dict of

{fast_time, angle, slow_time, stable, locked, hwp_rate}

See refelence section for the definition of each variable.

Write HWP g3 file

There is a fucntion to output analyzed data with HWP g3 format

hwp_angle_tool.write_solution(solved,'output.g3')

We can also specify output file path+name in config file.

# output g3 file path + name
output: './output.g3'

If you specify both, config file setting will be overwritten.

Write HWP metadata

There is a fucntion to output analyzed data with HDF5 metadata format. This is mainly used for make-hwp-solutions site pipeline element.

hwp_angle_tool.write_solution_h5(aman,h5_filename,h5_address)

Reference

Class and function references should be auto-generated here.

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]
load_data(start=None, end=None, data_dir=None, instance=None)[source]

Loads house keeping data for a given time range and returns HWP parameters in L2 HK .g3 file

Parameters:

  • start (timestamp or datetime) – start time for data, assumed to be in UTC unless specified

  • end (timestamp or datetime) – end time for data, assumed to be in UTC unless specified

  • data_dir (str or None) – path to HK g3 file, overwrite config file

  • instance (str or None) – instance of field list, overwrite config file ex.) lab data ‘observatory.HBA.feeds.HWPEncoder’ ex.) site data ‘satp1.hwp-bbb-e1.feeds.HWPEncoder’ ex.) site data ‘satp3.hwp-bbb-a1.feeds.HWPEncoder’

Returns:

{alias[i] : (time[i], data[i])}

Return type:

dict

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]
load_file(file_list=None, instance=None)[source]

Loads house keeping data with specified g3 files. Return HWP parameters from SO HK data.

Parameters:

  • file_list (str or list or None) – path and file name of input level 2 HK g3 file

  • instance (str or None) – instance of field list, overwrite config file ex.) lab data ‘observatory.HBA.feeds.HWPEncoder’ ex.) site data ‘satp1.hwp-bbb-e1.feeds.HWPEncoder’ ex.) site data ‘satp3.hwp-bbb-a1.feeds.HWPEncoder’

Returns:

{alias[i] : (time[i], data[i])}

Return type:

dict

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]
analyze(data, ratio=None, mod2pi=True, fast=True, suffix='_1')[source]

Analyze HWP angle solution to be checked by hardware that 0 is CW and 1 is CCW from (sky side) consistently for all SAT

Parameters:

  • data (dict) – HWP HK data from load_data

  • ratio (float, optional) – parameter for referelce slit threshold = 2 slit distances +/- ratio

  • mod2pi (bool, optional) – If True, return hwp angle % 2pi

  • fast (bool, optional) – If True, run fast fill_ref algorithm

Returns:

{fast_time, angle, slow_time, stable, locked, hwp_rate, template, filled_indexes}

Return type:

dict

Notes

  • fast_time: timestamp

    • IRIG synched timing (~2kHz)

  • angle (float): IRIG synched HWP angle in radian

  • slow_time: timestamp

    • time list of slow block

  • stable: bool

    • if non-zero, indicates the HWP spin state is known.

    • i.e. it is either spinning at a measurable rate, or stationary.

    • When this flag is non-zero, the hwp_rate field can be taken at face value.

  • locked: bool

    • if non-zero, indicates the HWP is spinning and the position solution is working.

    • In this case one should find the hwp_angle populated in the fast data block.

  • hwp_rate: float

    • the “approximate” HWP spin rate, with sign, in revs / second.

    • Use placeholder value of 0 for cases when not “stable”.

  • filled_indexes: boolean array

    • indexes where we filled due to packet drop etc.

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]
write_solution(solved, output=None)[source]

Output HWP angle + flags as SO HK g3 format

Parameters:

  • solved (dict) – dict data from analyze

  • output (str or None) – output path + file name, override config file

Notes

Output file format

  • Provider: ‘hwp’
    • Fast block

      • ‘hwp.hwp_angle’

    • Slow block

      • ‘hwp.stable’

      • ‘hwp.locked’

      • ‘hwp.hwp_rate’

  • fast_time: timestamp

    IRIG synched timing (~2kHz)

  • angle: float

    IRIG synched HWP angle in radian

  • slow_time: timestamp

    time list of slow block

  • stable: bool

    if non-zero, indicates the HWP spin state is known. i.e. it is either spinning at a measurable rate, or stationary. When this flag is non-zero, the hwp_rate field can be taken at face value.

  • locked: bool

    if non-zero, indicates the HWP is spinning and the position solution is working. In this case one should find the hwp_angle populated in the fast data block.

  • hwp_rate: float

    the “approximate” HWP spin rate, with sign, in revs / second. Use placeholder value of 0 for cases when not “locked”.

class sotodlib.hwp.g3thwp.G3tHWP(config_file=None)[source]
write_solution_h5(tod, output=None, h5_address=None, load_h5=True)[source]

Output HWP angle, flags, metadata as AxisManager format The results are stored in HDF5 files. Since HWP angle solution HDF5 files are large, we automatically split into the new output files. We save the copy of raw hwp encoder hk data into HDF5 file, to save time for re-calculating the hwp angle solutions.

Parameters:

  • tod (AxisManager)

  • output (str or None) – output path + file name, overwrite config file

  • load_h5 – If true, try to load raw encoder data from hdf5 file

Notes

Output file format

  • Primary output

    • timestamp: float (samps,)

      SMuRF synched timestamp

    • hwp_angle: float (samps,)

      The latest version of the SMuRF synched HWP angle in radian. * ver1: HWP angle calculated from the raw encoder signal. * ver2: HWP angle after the template subtraction. * ver3: HWP angle after the template and off-centering subtraction. The field ‘version’ indicates which version this hwp_angle is.

    • primaty encoder: int

      This field indicates which encoder is used for hwp_angle, 1 or 2

    • version: int

      This field indicates the version of the HWP angle in hwp_angle.

  • Supplementary output

    Suffix _1/2 indicates the encoder_1 or encoder_2

    • version_1/2: int

      This field indicates the version of the HWP angle of each encoder.

    • hwp_angle_ver1/2/3_1/2: float (samps,)

      This field stores the ver1/2/3 angle data.

    • stable_1/2: bool (samps,)

      If non-zero, indicates the HWP spin state is known. i.e. it is either spinning at a measurable rate, or stationary. When this flag is non-zero, the hwp_rate field can be taken at face value.

    • locked_1/2: bool (samp,)

      If non-zero, indicates the HWP is spinning and the position solution is working. In this case one should find the hwp_angle populated in the fast data block.

    • hwp_rate_1/2: float (samps,)

      The “approximate” HWP spin rate, with sign, in revs / second. Use placeholder value of 0 for cases when not “locked”.

    • logger_1/2: str

      Log message for angle calculation status ‘No HWP data’, ‘HWP data too short’, ‘Angle calculation failed’, ‘Angle calculation succeeded’

    • filled_flag_1/2: bool (samps,)

      Array to indicate the index of hwp angle filled due to packet drop, etc.

    • quad_1/2: int (quad,)

      0 or 1 or -1. 0 means no data

    • template_1/2: float (1140,)

      Template of the non uniformity of hwp encoder plate

    • offcenter: float (2,)

      • (average offcenter, std of offcenter) unit is (mm)

  • Rotation direction

    Rotation direction estimated by several methods. 0 or 1 or -1. 0 means no data.

    • quad_direction_1/2: int

      Estimation by median encoder quadrature for each encoder

    • pid_direction: int

      Estimation by median pid controller commanded direction

    • offcenter_direction: int

      Estimation by the offcentering measured by the time offset between two encoders. To be implemented.

    • template_direction: int

      Estimation by the template of encoder plate. To be implemented.

    • scan_direction: int

      Estimation by scan synchronous modulation of rotation speed.

  • Raw output

    x is a number different for each data and each observation

    • raw_rising_edge_count_1/2: int (2, x)

    • raw_irig_time_1/2: float (2, x)

    • raw_counter_1/2: float (2, x)

    • raw_counter_index_1/2: int (2, x)

    • raw_irig_synch_pulse_clock_time_1/2: float (2, x)

    • raw_irig_synch_pulse_clock_counts_1/2: int (2, x)

    • raw_quad_1/2: bool (2, x)

    • raw_pid_direction: bool (2, x)

HWPSS

sotodlib.hwp.hwp.get_hwpss(aman, signal_name=None, hwp_angle=None, bin_signal=True, bins=360, lin_reg=True, modes=[1, 2, 3, 4, 6, 8], apply_prefilt=True, prefilt_cfg=None, prefilt_detrend='linear', flags=None, merge_stats=True, hwpss_stats_name='hwpss_stats', merge_model=True, hwpss_model_name='hwpss_model')[source]

Extracts HWP synchronous signal (HWPSS) from a time-ordered data (TOD) using linear regression or curve-fitting. The curve-fitting or linear regression are either run on the full time ordered data vs hwp angle or the time ordered data binned in hwp_angle. If the curve-fitting option is used it must be performed on the binned data.

Parameters:

  • aman (AxisManager object) – The TOD to extract HWPSS from.

  • signal_name (str) – The field name in the axis manager to use for the TOD signal. If not provided, signal will be used.

  • hwp_angle (array-like, optional) – The HWP angle for each sample in aman. If not provided, aman.hwp_angle will be used.

  • bin_signal (bool, optional) – Whether to bin the TOD signal into HWP angle bins before extracting HWPSS. Default is True.

  • bins (int, optional) – The number of HWP angle bins to use if bin_signal is True. Default is 360.

  • lin_reg (bool, optional) – Whether to use linear regression to extract HWPSS from the binned signal. If False, curve-fitting will be used instead. Default is True.

  • modes (list of int, optional) – The HWPSS harmonic modes to extract. Default is [1, 2, 3, 4, 6, 8].

  • apply_prefilt (bool, optional) – Whether to apply a high-pass filter to signal before extracting HWPSS. Default is True. If run through preprocess and signal is not aman.signal then default to False.

  • prefilt_cfg (dict, optional) – The configuration of the high-pass filter, in Hz. Only used if apply_prefilt is True. Default is sine2 filter of with cutoff frequency of 1.0 Hz and trans_width of 1.0 Hz.

  • prefilt_detrend (str or None) – Method of detrending when you apply prefilter. Default is linear. If data is already detrended or you do not want to detrend, set it to None.

  • flags (RangesMatrix, optional) – Flags to be masked out before extracting HWPSS. If Default is None, and no mask will be applied.

  • merge_stats (bool, optional) – Whether to add the extracted HWPSS statistics to aman as new axes. Default is True.

  • hwpss_stats_name (str, optional) – The name to use for the new field containing the HWPSS statistics if merge_stats is True. Default is ‘hwpss_stats’.

  • merge_extract (bool, optional) – Whether to add the extracted HWPSS to aman as a new signal field. Default is True.

  • hwpss_extract_name (str, optional) – The name to use for the new signal field containing the extracted HWPSS if merge_extract is True. Default is ‘hwpss_extract’.

Returns:

hwpss_stats –

The extracted HWPSS and its statistics. The statistics include:

  • coeffs (n_dets x n_modes) : coefficients of the model

\[\sum_n \mathrm{coeffs}[2n]\sin{(\mathrm{modes}[n] \chi_{\mathrm{hwp}})} + \mathrm{coeffs}[2n+1]\cos{(\mathrm{modes}[n] \chi_{\mathrm{hwp}})}\]

where the sum on n range(len(modes)). Note: n_modes is 2*len(modes)

  • covars (n_dets x n_modes x n_modes) : variance covariance matrix of the fitted coefficients for each detector.

  • redchi2 (n_dets) : reduced chi^2 of the fit for each detector.

In the binned case the following are returned:

  • binned_angle (n_bins) : binned version of hwp_angle in range (0, 2pi] with number of bins set by bins argument.

  • binned_signal (n_dets x n_bins) : binned signal for each detector.

  • sigma_bin (n_dets) : average over all bins of the standard deviation of the signal within each bin.

In the non-binned case the following are returned:

  • sigma_tod (n_dets) : estimate of the standard deviation of the signal using function estimate_sigma_tod

Return type:

AxisManager object

sotodlib.hwp.sim_hwp.sim_hwpss(aman, name='hwpss_sim', hwp_freq=2.0, bandpass='090', loading=10.0, inc_ang=5.0, amp_2f_r=0.05, amp_4f_r=0.05, phi_2f_r=0.1, phi_4f_r=0.1)[source]

  • HWPSS simulation function using 2f and 4f amp and phase from I_to_P_param function.

  • The simulated HWSS is added to the input AxisManager as “name”.

Parameters:

  • aman (AxisManager) – target AxisManager

  • hwp_freq (float) – HWP rotation speed, 2 [Hz] (default)

  • bandpass (str) – an 3-digit (front zero padded) integer “XYZ” where XYZ = 030/040/090/150/220/280

  • loading (float) – intensity [Kcmb]

  • inc_ang (float) – incident angle [deg]

  • amp_2f_r (float, optional) – 2f HWPSS amplitude fluctuation

  • amp_4f_r (float, optional) – 4f HWPSS amplitude fluctuation

  • phi_2f_r (float, optional) – 2f HWPSS phase shift fluctuation

  • phi_4f_r (float, optional) – 4f HWPSS phase shift fluctuation

sotodlib.hwp.sim_hwp.sim_hwpss_2f4f(aman, name='hwpss_sim', hwp_freq=2.0, bandpass='090', amp_2f=300, amp_4f=30, phi_2f=0, phi_4f=0, amp_2f_r=0.05, amp_4f_r=0.05, phi_2f_r=0.1, phi_4f_r=0.1)[source]

  • HWPSS simulation function using given 2f and 4f amp and phase.

  • The simulated HWSS is added to the input AxisManager as “name”.

Parameters:

  • aman (AxisManager) – target AxisManager

  • hwp_freq (float) – 2 [Hz] (default)

  • bandpass (str) – an 3-digit (front zero padded) integer “fXYZ” where XYZ = 030/040/090/150/220/280

  • amp_2f (float) – 2f HWPSS amplitude [Kcmb]

  • amp_4f (float) – 4f HWPSS amplitude [Kcmb]

  • phi_2f (float) – 2f HWPSS phase shift [deg.]

  • phi_4f (float) – 4f HWPSS phase shift [deg.]

  • amp_2f_r (float) – 2f HWPSS amplitude fluctuation

  • amp_4f_r (float) – 4f HWPSS amplitude fluctuation

  • phi_2f_r (float) – 2f HWPSS phase shift fluctuation

  • phi_4f_r (float) – 4f HWPSS phase shift fluctuation

sotodlib.hwp.sim_hwp.I_to_P_param(bandpass='f090', loading=10, inc_ang=5)[source]

  • HWP I to P leakage simulation function

  • Mueller matirices of three layer achromatic HWP for each frequency and each incident angle are derived by RCWA simulation.

  • AR coating based on Mullite-Duroid coating for SAT1 at UCSD. Thickness are used fabricated values.

  • 2f and 4f amplitudes and phases are extracted by HWPSS fitting of MIQ and MIU.

  • Amplitudes and phases are fitted by polynominal functions w.r.t each incident angle.

  • Only support MF because UHF and LF are design/fabrication phase. We will add them after the fabrications.

Parameters:

  • bandpass (str) – an 3-digit (front zero padded) integer “fXYZ” where XYZ = 030/040/090/150/220/280

  • loading (float) – intensity [Kcmb]

  • inc_ang (float) – incident angle [deg]

Returns:

  • amp_2f (float) – 2f amplitude [mKcmb]

  • amp_4f (float) – 4f amplitude [mKcmb]

  • phi_2f (float) – 2f phase shift [rad]

  • phi_4f (float) – 4f phase shift [rad]

Demodulation

sotodlib.hwp.hwp.demod_tod(aman, signal_name='signal', demod_mode=4, bpf_cfg=None, lpf_cfg=None)[source]

Demodulate TOD based on HWP angle

Parameters:

  • aman (AxisManager) – The AxisManager object

  • signal_name (str, optional) – Axis name of the demodulated signal in aman. Default is ‘signal’.

  • demod_mode (int, optional) – Demodulation mode. Default is 4.

  • bpf_cfg (dict) – Configuration for Band-pass filter applied to the TOD data before demodulation. If not specified, a 4th-order Butterworth filter of (demod_mode * HWP speed) +/- 0.95*(HWP speed) is used. Example) bpf_cfg = {‘type’: ‘butter4’, ‘center’: 8.0, ‘width’: 3.8} See filters.get_bpf for details.

  • lpf_cfg (dict) – Configuration for Low-pass filter applied to the demodulated TOD data. If not specified, a 4th-order Butterworth filter with a cutoff frequency of 0.95*(HWP speed) is used. Example) lpf_cfg = {‘type’: ‘butter4’, ‘cutoff’: 1.9} See filters.get_lpf for details.

Returns:

The demodulated TOD data is added to the input aman container as new signals: ‘dsT’ for the original signal filtered with lpf, ‘demodQ’ for the demodulated signal real component filtered with lpf and multiplied by 2, and ‘demodU’ for the demodulated signal imaginary component filtered with lpf and multiplied by 2.

Return type:

None