Querying a Data Set¶

The potential size of ASDF data sets demands efficient and easy ways to loop over and work on certain subsets of it. This functionality is available in the pyasdf.asdf_data_set.ASDFDataSet.ifilter() method which is very powerful and explained in the rest of this section.

Quick Example¶

In this introductory example we will loop over a data set and extract all waveform data that has a network starting with T, a raw_recording tag, and a sampling rate of more than 10 Hz. The ifilter() method returns an iterator meaning one has to loop over it to cause it to extract the required information. It yields a FilteredWaveformAccessor which works the same as the normal waveform accessor object but with a potentially limited dataset. Have a look at the syntax and it should be pretty self-explanatory.

import pyasdf

ds = pyasdf.ASDFDataSet("example.h5")

for station in ds.ifilter(ds.q.network == "T*",
                          ds.q.tag == "raw_recording",
                          ds.q.sampling_rate > 10.0):
    # Extract the StationXML information in the same way as always. Might
    # be `None` if it does not exists.
    inv = station.StationXML
    if inv is None:
        continue

    # You can also get the coordinates.
    coords = station.coordinates

    # Waveform access also works the normal way by tag.
    st = station.raw_recording

    # Process it and do whatever you want.
    st.detrend("linear")
    st.taper(type="hann", max_percentage=0.05)
    st.attach_response(inv)
    st.remove_response()
    ...

More Example Queries¶

We’ll show some more example to give you a feel for how to construct queries and what they mean. The syntax is not as powerful as a full SQL implementation but for practical purposes it gets pretty close and it can express and resolve some fairly complex queries. You can chain an arbitrary number of constraints which must all be fulfilled at the same time, each constraint is based on the rich comparison operators from Python: ==, !=, <, <=, >, and >=.

Separate arguments are chained by a logical AND, so every comma can be interpreted as a logical AND . The following query will iterate over all waveforms with a network starting with T, but not the TA (USArray) network. Wildcards are resolved by the fnmatch module.

for station in ds.ifilter(ds.q.network == "T*",
                          ds.q.network != "TA"):
    ...

String attributes (network, station, channel, …) can also be queried by passing a list of potentially wildcarded strings. Each element in such a list is joined by a logical OR, e.g. the next query will select all stations starting with B, or the CA station, or any two letter station starting with T:

for station in ds.ifilter(ds.q.station == ["B*", "ALTM", "T?"]):
    ...

You can also search over geographical constraints based on the latitude, longitude, and elevation_in_m keys. As soon as one of these keys is given it will only return stations that have coordinate information in the form of StationXML.

for station in ds.ifilter(ds.q.latitude >= 10.0, ds.q.latitude <= 20,
                          ds.q.longitude <= -101.2,
                          ds.q.elevation_in_m > 200.0)
    ...

Get all vertical component channels from USArray with a sampling rate of at least 5 Hz that are not processed (raw_recording tag by convention) in January 2015:

for station in ds.ifilter(ds.q.network == "TA",
                          ds.q.channel == "*Z",
                          ds.q.sampling_rate >= 5,
                          ds.q.tag == "raw_recording",
                          ds.q.starttime >= obspy.UTCDateTime(2015, 1, 1),
                          ds.q.endtime <= obspy.UTCDateTime(2015, 2, 1))
    ...

Get waveforms for all stations that have no coordinates:

for station in ds.ifilter(ds.q.latitude == None):
    ...

Get all waveforms associated with a certain event:

event = obspy.read_events(...)[0]

for station in ds.ifilter(ds.q.event == event):
    ...

Get all waveforms if any label is either “Fun” or matches the pattern “A*C”:

for station in ds.ifilter(ds.q.labels == ["Fun", "A*C"]):
    ...

Query Types¶

The available query keys alongside other information regarding their usage are listed in the following table:

Name	Allowed Types	Description
String Parameters: These can be given as a single (wildcarded) string or as a list of (wildcarded) strings which will be connected via a logical `OR`.
`network`	`str` or list of `str`	The network code.
`station`	`str` or list of `str`	The station code.
`location`	`str` or list of `str`	The location code.
`channel`	`str` or list of `str`	The channel code.
`tag`	`str` or list of `str`	The hierarchical tag associated with the trace.
`labels`	`str` or list of `str`	Search over the labels of each waveforms.
Geographical Parameters: Search over geographical parameters stored in the StationXML files.
`longitude`	`float`	The longitude of the recording station.
`latitude`	`float`	The latitude of the recording station.
`elevation_in_m`	`float`	The elevation in meters of the recording station.
Temporal Parameters: Pass as a `UTCDateTime` object or any string or number that can be parsed to one.
`starttime`	`UTCDateTime`, `str`, or `float`	The start time of the waveform.
`endtime`	`UTCDateTime`, `str`, or `float`	The end time of the waveform.
Waveform Attribute Parameters: Evaluated per waveform trace. Don’t use `==` for floats but rather a combination of `>=` and `<=`.
`sampling_rate`	`float`	The sampling rate of the waveform in Hz.
`npts`	`int`	The number of samples of the waveform.
Event Relation Parameters: These are not wildcarded as `?` and `*` are perfectly valid URL components and most IDs are URLs.
`event`	`Event`, `ResourceIdentifier`, or `str`	The event associated with the waveform.
`magnitude`	`Magnitude`, `ResourceIdentifier`, or `str`	The magnitude associated with the waveform.
`origin`	`Origin`, `ResourceIdentifier`, or `str`	The origin associated with the waveform.
`focal_mechanism`	`FocalMechanism`, `ResourceIdentifier`, or `str`	The focal mechanism associated with the waveform.