ch_util.finder

Data Index Searcher for CHIME

Search routines for locating data withing the CHIME data index.

Data tables

  • DataFlag

  • DataFlagType

Exceptions

High Level Index Searcher

Routines

Functions

files_in_range(acq, start_time, end_time, ...)

Get files for a given acquisition within a time range.

Classes

BaseDataInterval([iterable])

A single data index search result.

CalibrationGainDataInterval([iterable])

Derived class from BaseDataInterval for calibration gain data.

CorrDataInterval([iterable])

Derived class from BaseDataInterval for correlator data.

DataInterval

alias of CorrDataInterval

DataIntervalList([iterable])

A list of data index search results.

DigitalGainDataInterval([iterable])

Derived class from BaseDataInterval for digital gain data.

Finder([acqs, node_spoof])

High level searching of the CHIME data index.

FlagInputDataInterval([iterable])

Derived class from BaseDataInterval for flag input data.

HKDataInterval([iterable])

Derived class from BaseDataInterval for housekeeping data.

WeatherDataInterval([iterable])

Derived class from BaseDataInterval for weather data.

Exceptions

DataFlagged

Raised when data is affected by a global flag.
class ch_util.finder.BaseDataInterval(iterable=(), /)[source]

Bases: tuple

A single data index search result.

Just a normal python tuple with some helper methods. Instances are created by calls to Finder.get_results().

A data interval as two elements: a list of filenames and a time range within those files.

You should generally only use the classes derived from this one (i.e., CorrDataInterval, etc.)

as_loaded_data(**kwargs)[source]

Load data interval to memory as an andata.AnData instance.

Parameters:

datasets (list of strings) – Passed on to andata.AnData.from_acq_h5()

Returns:

data – Data interval loaded into memory.

Return type:

andata.AnData

as_reader()[source]

Get data interval as an andata.Reader instance.

The andata.Reader is initialized with the filename list part of the data interval then the time range part of the data interval is used as an arguments to andata.Reader.select_time_range().

Returns:

reader

Return type:

andata.Reader

class ch_util.finder.CalibrationGainDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for calibration gain data.

class ch_util.finder.CorrDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for correlator data.

as_loaded_data(prod_sel=None, freq_sel=None, datasets=None)[source]

Load data interval to memory as an andata.CorrData instance

Parameters:

  • prod_sel (valid numpy index) – Passed on to andata.CorrData.from_acq_h5()

  • freq_sel (valid numpy index) – Passed on to andata.CorrData.from_acq_h5()

  • datasets (list of strings) – Passed on to andata.CorrData.from_acq_h5()

Returns:

data – Data interval loaded into memory.

Return type:

andata.CorrData

exception ch_util.finder.DataFlagged[source]

Bases: CHIMEdbError

Raised when data is affected by a global flag.

ch_util.finder.DataInterval

alias of CorrDataInterval

class ch_util.finder.DataIntervalList(iterable=(), /)[source]

Bases: list

A list of data index search results.

Just a normal python list of DataInterval-derived objects with some helper methods. Instances are created by calls to Finder.get_results().

iter_loaded_data(**kwargs)[source]

Iterate over data intervals loading as andata.AnData.

Parameters:

**kwargs (argument list) – Pass any parameters accepted by the BaseDataInverval-derived class that you are using.

Returns:

Iterator over data intervals loaded into memory as andata.BaseData-derived instances.

Return type:

loaded_data_iterator

Examples

Use this method to loop over data loaded into memory.

>>> for data in interval_list.iter_loaded_data():
...     pass

Data is loaded into memory on each iteration. To immediately load all data into memory, initialize a list using the iterator:

>>> loaded_data_list = list(interval_list.iter_loaded_data())
iter_reader()[source]

Iterate over data intervals converting to andata.Reader.

Returns:

Iterator over data intervals as andata.Reader instances.

Return type:

reader_iterator

class ch_util.finder.DigitalGainDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for digital gain data.

class ch_util.finder.Finder(acqs=(), node_spoof=None)[source]

Bases: object

High level searching of the CHIME data index.

This class gives a convenient way to search and filter data acquisitions as well as time ranges of data within acquisitions. Search results constitute a list of files within an acquisition as well as a time range for the data within these files. Convenient methods are provided for loading the precise time range of constituting a search result.

This is intended to make the most common types of searches of CHIME data as convenient as possible. However for very complex searches, it may be necessary to resort to the lower level interface.

Searching the index

There are four ways that a search can be modified which may be combined in any way.

  1. You can restrict the types of acquisition that are under consideration, using methods whose names begin with only_. In this way, one can consider only, say, housekeeping acquisitions.

  2. The second is to adjust the total time range under consideration. This is achieved by assigning to time_range or calling methods beginning with set_time_range_. The total time range affects acquisitions under consideration as well as the data time ranges within the acquisitions. Subsequent changes to the total time range under consideration may only become more restrictive.

  3. The data index may also be filtered by acquisition using methods whose names begin with filter_acqs. Again subsequent filtering are always combined to become more restrictive. The attribute acqs lists the acquisitions currently included in the search for convenience when searching interactively.

  4. Time intervals within acquisitions are added using methods with names beginning with include_. Time intervals are defined in the time_intervals attribute, and are inclusive (you can add as many as you want).

  5. Finally, upon calling :meth:get_results or :meth:get_results_acq, one can pass an arbitrary condition on individual files, thereby returning only a subset of files from each acquisition.

Getting results

Results of the search can be retrieved using methods whose names begin with get_results An individual search result is constituted of a list of file names and a time interval within these files. These can easily loaded into memory using helper functions (see BaseDataInterval and DataIntervalList).

Parameters:

  • acqs (list of chimedb.data_index.ArchiveAcq objects) – Acquisitions to initially include in data search. Default is to search all acquisitions.

  • node_spoof (dictionary) – Normally, the DB will be queried to find which nodes are mounted on your host. If you are on a machine that is cross-mounted, though, you can enter a dictionary of “node_name”: “mnt_root” pairs, specifying the nodes to search and where they are mounted on your host.

Examples

To find all the correlator data between two times.

>>> from ch_util import finder
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.only_corr()
>>> f.set_time_range(datetime(2014,02,24), datetime(2014,02,25))
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140219T145849Z_abbot_corr  |   378053.1  |    86400.0  |  25
   2  |  20140224T051212Z_stone_corr  |        0.0  |    67653.9  |  19
Total 154053.858720 seconds of data.

Search for transits of a given source.

>>> from ch_util import ephemeris
>>> f.include_transits(ephemeris.CasA, time_delta=3600)
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140219T145849Z_abbot_corr  |   452087.2  |     3600.0  |  2
   2  |  20140224T051212Z_stone_corr  |    55288.0  |     3600.0  |  2
Total 7200.000000 seconds of data.

To read the data,

>>> from ch_util import andata
>>> results_list = f.get_results()
>>> # Pick result number 1
>>> result = results_list[0]
>>> # Pick product number 0 (autocorrelation)
>>> data = result.as_loaded_data(prod_sel=0)
>>> print data.vis.shape
(1024, 1, 360)

More intricate filters on the acquisitions are possible.

>>> import chimedb.data_index as di
>>> f = finder.Finder()
>>> # Find ALL 10ms cadence data correlated by 'stone' with 8 channels.
>>> f.filter_acqs((di.CorrAcqInfo.integration < 0.011)
...               & (di.CorrAcqInfo.integration > 0.009)
...               & (di.CorrAcqInfo.nfreq == 1024)
...               & (di.CorrAcqInfo.nprod == 36)
...               & (di.ArchiveInst.name == 'stone'))
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140211T020307Z_stone_corr  |        0.0  |      391.8  |  108
   2  |  20140128T135105Z_stone_corr  |        0.0  |     4165.2  |  104
   3  |  20131208T070336Z_stone_corr  |        0.0  |     1429.8  |  377
   4  |  20140212T014603Z_stone_corr  |        0.0  |     2424.4  |  660
   5  |  20131210T060233Z_stone_corr  |        0.0  |     1875.3  |  511
   6  |  20140210T021023Z_stone_corr  |        0.0  |      874.1  |  240
Total 11160.663510 seconds of data.

Here is an example that uses node spoofing and also filters files within acquisitions to include only LNA housekeeping files:

>>> f = finder.Finder(node_spoof = {"gong" : "/mnt/gong/archive",
                                        "suzu" : "/mnt/suzu/hk_data"})
>>> f.only_hk()
>>> f.set_time_range(datetime(2014, 9, 1), datetime(2014, 10, 10))
>>> f.print_results_summary()
# | acquisition                          |start (s)| len (s) |files |     MB
0 | 20140830T005410Z_ben_hk              |  169549 |  419873 |   47 |   2093
1 | 20140905T203905Z_ben_hk              |       0 |   16969 |    2 |      0
2 | 20140908T153116Z_ben_hk              |       0 | 1116260 |   56 |      4
3 | 20141009T222415Z_ben_hk              |       0 |    5745 |    2 |      0
>>> res = f.get_results(file_condition = (di.HKFileInfo.atmel_name == "LNA"))
>>> for r in res:
...   print "No. files: %d" % (len(r[0]))
No. files: 8
No. files: 1
No. files: 19
No. files: 1
>>> data = res[0].as_loaded_data()
>>> for m in data.mux:
...   print "Mux %d: %s", (m, data.chan(m))
Mux 0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Mux 1: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Mux 2: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
>>> print "Here are the raw data for Mux 1, Channel 14:", data.tod(14, 1)
Here are the raw data for Mux 1, Channel 14: [ 1744.19091797  1766.34472656  1771.03356934 ...,  1928.61279297 1938.90075684  1945.53491211]

In the above example, the restriction to LNA housekeeping could also have been accomplished with the convenience method Finder.set_hk_input():

>>> f.set_hk_input("LNA")
>>> res = f.get_results()
accept_all_global_flags()[source]

Set global flag behaviour to accept all data.

property acqs

Acquisitions remaining in this search.

Returns:

acqs

Return type:

list of chimedb.data_index.ArchiveAcq objects

property data_flag_types

Types of DataFlag to exclude from results.

exclude_RA_interval(start_RA, end_RA)[source]

Add time intervals to exclude passings of given right RA intervals

Parameters:

  • start_RA (float) – Starting right ascension in degrees.

  • end_RA (float) – Ending right ascension in degrees.

Examples

Look under include_RA_interval for very similar example.

exclude_data_flag_type(flag_type)[source]

Exclude times that overlap with DataFlags of this type.

Parameters:

flag_type (string or list of string) – Name of DataFlagType(s) to exclude from results, e.g. “rain”.

exclude_daytime()[source]

Add time intervals to exclude all day time data.

exclude_global_flag(flag)[source]

Update time_intervals to exclude a global flag.

Parameters:

flag (integer or string) – Global flag ID or name, e.g. “run_pass1_a”, or 65.

See also

Look

Notes

Global flag ID numbers, names, and descriptions are listed at http://bao.phas.ubc.ca/layout/event.php?filt_event_type_id=7

exclude_nighttime()[source]

Add time intervals to exclude all night time data.

exclude_sun(time_delta=4000.0, time_delta_rise_set=4000.0)[source]

Add time intervals to exclude sunrise, sunset, and sun transit.

Parameters:

  • time_delta (float) – Total amount of time to exclude surrounding the sun transit in seconds. Default is to use 4000.0 seconds.

  • time_delta_rise_set (float) – Total amount of time to exclude after sunrise and before sunset in seconds. Default is to use 4000.0 seconds.

exclude_time_interval(start_time, end_time)[source]

Exclude a time interval.

Examples

>>> from ch_util import finder
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.set_time_range(datetime(2014,04,04), datetime(2014,04,14))
>>> # f.print_results_summary() will show all the files in this time range
>>> # Now want to exclude all data from 04, 10 to 04, 11
>>> f.exclude_time_interval(datetime(2014,04,10),datetime(2014,04,11))
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140330T102505Z_abbot_corr  |   394484.2  |   231900.8  |  65
   2  |  20140403T152314Z_blanchard_corr  |    30988.4  |   309649.3  |  86
   3  |  20140408T222844Z_abbot_corr  |        0.0  |    75589.3  |  21
   4  |  20140409T184530Z_blanchard_corr  |        0.0  |     3795.0  |  2
   5  |  20140409T165603Z_blanchard_corr  |        0.0  |     4952.7  |  2
   6  |  20140411T003404Z_blanchard_corr  |        0.0  |   161606.5  |  45
   7  |  20140411T000920Z_blanchard_corr  |        0.0  |     1080.4  |  36
   8  |  20140413T002319Z_blanchard_corr  |        0.0  |    84981.7  |  24
Total 873555.739000 seconds of data.
exclude_transits(body, time_delta)[source]

Add time intervals to exclude transits for given celestial body.

Parameters:

  • body (ephem.Body or float) – Transiting celestial body. If a float, interpret as a right ascension in degrees.

  • time_delta (float) – Total amount of time to include surrounding the transit in seconds. Default is to use twice the value of min_interval.

Examples

>>> from ch_util import finder
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.set_time_range(datetime(2014,02,20), datetime(2014,02,22))
>>> import ephem
>>> f.exclude_transits(ephem.Sun(), time_delta=43200)
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140219T145849Z_abbot_corr  |    32453.1  |    51128.4  |  15
   2  |  20140219T145849Z_abbot_corr  |   126781.5  |    43193.0  |  13
   3  |  20140219T145523Z_stone_corr  |    32662.5  |    18126.9  |  6
   4  |  20140220T213252Z_stone_corr  |    16740.8  |    43193.0  |  13
Total 155641.231275 seconds of data.
filter_acqs(condition)[source]

Filter the acquisitions included in this search.

Parameters:

condition (peewee comparison) – Condition on any on chimedb.data_index.ArchiveAcq or any class joined to chimedb.data_index.ArchiveAcq: using the syntax from the peewee module [1].

Examples

>>> from ch_util import finder
>>> import chimedb.data_index as di
>>> f = finder.Finder()
>>> f.filter_acqs(di.ArchiveInst.name == 'stone')
>>> f.filter_acqs((di.AcqType == 'corr') & (di.CorrAcqInfo.nprod == 36))

See also

Finder.filter_acqs_by_files()

References

filter_acqs_by_files(condition)[source]

Filter the acquisitions by the properties of its files.

Because each acquisition has many files, this filter should be significantly slower than Finder.filter_acqs().

Parameters:

condition (peewee comparison) – Condition on any on chimedb.data_index.ArchiveAcq, chimedb.data_index.ArchiveFile or any class joined to chimedb.data_index.ArchiveFile using the syntax from the peewee module [2].

See also

Finder.filter_acqs()

Examples

References

get_results(file_condition=None)[source]

Get all search results.

Parameters:

file_condition (peewee comparison) – Any additional condition for filtering the files within the acquisition. In general, this should be a filter on one of the file information tables, e.g., chimedb.data_index.CorrFileInfo.

Returns:

  • interval_list (DataIntervalList) – Search results.

  • cond (peewee comparison) – Any extra filters, particularly filters on individual files.

get_results_acq(acq_ind, file_condition=None)[source]

Get search results restricted to a given acquisition.

Parameters:

  • acq_ind (int) – Index of Finder.acqs for the desired acquisition.

  • file_condition (peewee comparison) – Any additional condition for filtering the files within the acquisition. In general, this should be a filter on one of the file information tables, e.g., CorrFileInfo.

Returns:

interval_list – Search results.

Return type:

DataIntervalList

property global_flag_mode

Global flag behaviour mode.

Defines how global flags are treated when finding data. There are three severities of global flag: comment, warning, and severe. There are four possible behaviours when a search result overlaps a global flag, represented by module constants:

GF_REJECT:

Reject any data overlapping flag silently.

GF_RAISE:

Raise an exception when retrieving data intervals.

GF_WARN:

Send a warning when retrieving data intervals but proceed.

GF_ACCEPT:

Accept the data silently, ignoring the flag.

The behaviour for all three severities is represented by a dictionary. If no mode is set, then the default behaviour is {‘comment’ : GF_ACCEPT, ‘warning’ : GF_WARN, ‘severe’ : GF_REJECT}.

This is modified using Finder.update_global_flag_mode().

Returns:

global_flag_mode – Specifies finder behaviour.

Return type:

dictionary with keys ‘comment’, ‘warning’, ‘severe’.

include_26m_obs(source, require_quality=True)[source]

Add time intervals to include 26m observations of a source.

Parameters:

  • source (string) – Source observed. Has to match name on database exactly.

  • require_quality (bool (default: True)) – Require the quality flag to be zero (ie that the 26 m pointing is trustworthy) or None

Examples

>>> from ch_util import (finder, ephemeris)
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.only_corr()
>>> f.set_time_range(datetime(2017,8,1,10), datetime(2017,8,2))
>>> f.filter_acqs((di.ArchiveInst.name == 'pathfinder'))
>>> f.include_26m_obs('CasA')
>>> f.print_results_summary()
   # | acquisition                          |start (s)| len (s) |files |     MB
   0 | 20170801T063349Z_pathfinder_corr     |   12337 |   11350 |    2 | 153499
   1 | 20170801T131035Z_pathfinder_corr     |       0 |    6922 |    1 |  75911
Total  18271 seconds, 229410 MB of data.
include_RA_interval(start_RA, end_RA)[source]

Add time intervals to include passings of given right RA intervals

Parameters:

  • start_RA (float) – Starting right ascension in degrees.

  • end_RA (float) – Ending right ascension in degrees.

Examples

>>> from ch_util import finder
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.set_time_range(datetime(2014,04,04), datetime(2014,04,14))
>>> f.include_RA_interval(90., 180.)
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140330T102505Z_abbot_corr  |   398689.9  |    21541.0  |  7
   2  |  20140330T102505Z_abbot_corr  |   484854.0  |    21541.0  |  7
   3  |  20140330T102505Z_abbot_corr  |   571018.1  |    21541.0  |  7
   4  |  20140403T152314Z_blanchard_corr  |    35194.1  |    21541.0  |  7
   5  |  20140403T152314Z_blanchard_corr  |   121358.2  |    21541.0  |  7
   6  |  20140403T152314Z_blanchard_corr  |   207522.3  |    21541.0  |  7
   7  |  20140403T152314Z_blanchard_corr  |   293686.4  |    21541.0  |  6
   8  |  20140408T222844Z_abbot_corr  |     8491.2  |    21541.0  |  7
   9  |  20140410T003326Z_blanchard_corr  |      754.5  |     1419.2  |  48
  10  |  20140410T031023Z_blanchard_corr  |        0.0  |     1376.5  |  46
  11  |  20140410T014136Z_blanchard_corr  |        0.0  |     2347.4  |  78
  12  |  20140411T003404Z_blanchard_corr  |      397.4  |    21541.0  |  7
  13  |  20140411T003404Z_blanchard_corr  |    86561.5  |    21541.0  |  7
  14  |  20140413T002319Z_blanchard_corr  |      664.1  |    21541.0  |  7
Total 242094.394565 seconds of data.
include_global_flag(flag)[source]

Update time_intervals to include a global flag.

Parameters:

flag (integer or string) – Global flag ID or name, e.g. “run_pass1_a”, or 11292.

Notes

Global flag ID numbers, names, and descriptions are listed at http://bao.phas.ubc.ca/layout/event.php?filt_event_type_id=7

include_time_interval(start_time, end_time)[source]

Include a time interval.

Examples

First a certain layout is chosen

>>> from ch_util import finder
>>> f = finder.Finder()
>>> f.set_time_range_layout(26)
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140311T192616Z_abbot_corr  |    16412.8  |      667.1  |  1
   2  |  20140312T001123Z_abbot_corr  |        0.0  |     1150.5  |  314
   3  |  20140312T003054Z_abbot_corr  |        0.0  |    79889.4  |  23
   4  |  20140312T224940Z_abbot_corr  |        0.0  |      591.0  |  4
   5  |  20140312T230108Z_abbot_corr  |        0.0  |   171909.0  |  48
   6  |  20140315T014330Z_abbot_corr  |        0.0  |    35119.7  |  10
   7  |  20140318T154959Z_abbot_corr  |        0.0  |    51739.6  |  15
   8  |  20140320T120437Z_abbot_corr  |        0.0  |   186688.6  |  52
   9  |  20140325T174231Z_abbot_corr  |        0.0  |    86019.3  |  24
  10  |  20140326T175440Z_abbot_corr  |        0.0  |   286487.7  |  80
  11  |  20140330T064125Z_abbot_corr  |        0.0  |     2998.6  |  1590
  12  |  20140330T102505Z_abbot_corr  |        0.0  |   626385.0  |  174
  13  |  20140403T000057Z_blanchard_corr  |        0.0  |    54912.3  |  16
  14  |  20140403T152314Z_blanchard_corr  |        0.0  |   340637.8  |  94
  15  |  20140408T222844Z_abbot_corr  |        0.0  |    75589.3  |  21
  16  |  20140409T184530Z_blanchard_corr  |        0.0  |     3795.0  |  2
  17  |  20140410T003326Z_blanchard_corr  |        0.0  |     2173.7  |  72
  18  |  20140409T165603Z_blanchard_corr  |        0.0  |     4952.7  |  2
Total 2011706.304970 seconds of data.

To find a specific day in that layout choose the functionality include_time_interval

>>> from datetime import datetime
>>> f.include_time_interval(datetime(2014,04,8), datetime(2014,04,9))
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140408T222844Z_abbot_corr  |        0.0  |     5465.1  |  2
Total 5465.059670 seconds of data.
include_transits(body, time_delta=None)[source]

Add time intervals to include transits for given celestial body.

Parameters:

  • body (ephem.Body or float) – Transiting celestial body. If a float, interpret as a right ascension in degrees.

  • time_delta (float) – Total amount of time to include surrounding the transit in seconds. Default is to use twice the value of min_interval.

Examples

>>> from ch_util import (finder, ephemeris)
>>> from datetime import datetime
>>> f = finder.Finder()
>>> f.set_time_range(datetime(2014,02,20), datetime(2014,02,22))
>>> f.include_transits(ephemeris.CasA, time_delta=3600)
>>> f.print_results_summary()
interval | acquisition | offset from start (s) | length (s) | N files
   1  |  20140219T145849Z_abbot_corr  |   107430.9  |     3600.0  |  2
   2  |  20140219T145849Z_abbot_corr  |   193595.0  |     3600.0  |  2
   3  |  20140220T213252Z_stone_corr  |        0.0  |      990.2  |  1
   4  |  20140220T213252Z_stone_corr  |    83554.3  |     3600.0  |  2
Total 11790.181012 seconds of data.
property min_interval

Minimum length of a block of data to be considered.

This can be set to any number. The default is 240 seconds.

Returns:

min_interval – Length of time in seconds.

Return type:

float

classmethod offline(acqs=())[source]

Initialize Finder when not working on a storage node.

Normally only data that is available on the present host is searched, and as such Finder can’t be used to browse the index when you don’t have access to the acctual data. Initializing using this method spoofs the ‘gong’ and ‘niedermayer’ storage nodes (which should have a full copy of the archive) such that the data index can be search the full archive.

only_chime_weather()[source]

Only include chime weather acquisitions in this search. This excludes the old format mingun-weather.

only_corr()[source]

Only include correlator acquisitions in this search.

only_digitalgain()[source]

Only include digital gain data in this search

only_flaginput()[source]

Only include input flag data in this search

only_gain()[source]

Only include calibration gain data in this search

only_hfb()[source]

Only include HFB acquisitions in this search.

only_hk()[source]

Only include housekeeping acquisitions in this search.

only_hkp()[source]

Only include Prometheus housekeeping data in this search

only_rawadc()[source]

Only include raw ADC acquisitions in this search.

only_weather()[source]

Only include weather acquisitions in this search.

print_acq_info()[source]

Print the acquisitions included in this search and thier properties.

This method is convenient when searching the data index interactively and you want to see what acquisitions remain after applying filters or restricting the time range.

See also

Finder.print_results_summary()

print_results_summary()[source]

Print a summary of the search results.

set_hk_input(name)[source]

Restrict files to only one HK input type.

This is a shortcut for specifying file_condition = (chimedb.data_index.HKFileInfo.atmel_name == name) in get_results_acq(). Instead, one can simply call this function with name as, e.g., “LNA”, “FLA”, and calls to get_results_acq() will be appropriately restricted.

Parameters:

name (str) – The name of the housekeeping input.

set_time_range(start_time=None, end_time=None)[source]

Restrict the time range of the search.

This method updates the time_range property and also excludes any acquisitions that do not overlap with the new range. This method always narrows the time range under consideration, never expands it.

Parameters:

  • start_time (float or datetime.datetime) – Unix/POSIX time or UTC start of desired time range. Optional.

  • end_time (float or datetime.datetime) – Unix/POSIX time or UTC end of desired time range. Optional.

set_time_range_global_flag(flag)[source]

Set time range to correspond to a global flag.

Parameters:

flag (integer or string) – Global flag ID or name, e.g. “run_pass1_a”, or 11292.

Notes

Global flag ID numbers, names, and descriptions are listed at http://bao.phas.ubc.ca/layout/event.php?filt_event_type_id=7

set_time_range_season(year=None, season=None)[source]

Set the time range by as specific part of a given year.

NOT YET IMPLEMENTED

Parameters:

  • year (integer) – Calender year

  • season (string) – Month name (3 letter abbreviations are acceptable) or one of ‘winter’, ‘spring’, ‘summer’, or ‘fall’.

property time_exclusions

Periods in time to be excluded.

Returns:

time_exclusions – Each entry is the Unix/POSIX beginning and end of the time interval to be excluded.

Return type:

list of pairs of floats

property time_intervals

Periods in time to be included.

Periods are combined with OR unless list is empty, in which case no filtering is performed.

Returns:

time_intervals – Each entry is the Unix/POSIX beginning and end of the time interval to be included.

Return type:

list of pairs of floats

property time_range

Time range to be included in search.

Data files and acquisitions that do not overlap with this range are excluded. Assigning to this is equivalent to calling set_time_range().

Returns:

time_range – Unix/POSIX beginning and end of the time range.

Return type:

tuple of 2 floats

update_global_flag_mode(comment=None, warning=None, severe=None)[source]

Update Finder.global_flag_mode, the global flag mode.

Parameters:

  • comment (One of GF_REJECT, GF_RAISE, GF_WARN, or GF_ACCEPT.)

  • warning (One of GF_REJECT, GF_RAISE, GF_WARN, or GF_ACCEPT.)

  • severe (One of GF_REJECT, GF_RAISE, GF_WARN, or GF_ACCEPT.)

class ch_util.finder.FlagInputDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for flag input data.

class ch_util.finder.HKDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for housekeeping data.

class ch_util.finder.WeatherDataInterval(iterable=(), /)[source]

Bases: BaseDataInterval

Derived class from BaseDataInterval for weather data.

ch_util.finder.files_in_range(acq, start_time, end_time, node_list, extra_cond=None, node_spoof=None)[source]

Get files for a given acquisition within a time range.

Parameters:

  • acq (string or int) – Which acquisition, by its name or id key.

  • start_time (float) – POSIX/Unix time for the start or time range.

  • end_time (float) – POSIX/Unix time for the end or time range.

  • node_list (list of chimedb.data_index.StorageNode objects) – Only return files residing on the given nodes.

  • extra_cond (peewee comparison) – Any additional expression for filtering files.

Returns:

file_names – List of filenames, including the full path.

Return type:

list of strings