With the creation of the MUSTANG automated data quality system, IRIS Data Services’ Quality Assurance (QA) team sought to create a new QA process that would leverage MUSTANG metrics in order to

• speed identification and reporting of data quality issues to network operators
• track the resolution of reported issues, and
• support each network’s existing operational agreements and reporting needs.

The Alaska Earthquake Center (AEC) staff agreed to help us develop, test and improve this new approach by

• providing Alaska Regional Network (AK) data as a test data set and
• giving us feedback on the value of the new process, as well as suggestions for its improvement.

First, IRIS and the AEC identified AK reporting needs in a Quality Assurance Cooperative Agreement that describes the channels, reporting period and issue status (resolved or ongoing) covered by the reports. It also listed project contacts at both IRIS and the AEC.

After agreeing on reporting needs, IRIS Quality Assurance staff began reviewing AK data. Analysis started by gathering MUSTANG metrics and time spans that could indicate potential data issues within the reporting period. For broadband instruments, the following “tests” are just a few examples of useful indicators (test name: diagnostic metric values):

noData: percent_availability = 0
noTime: clock_locked = 0
dead: dead_channel_exp < 0.3 && pct_below_nlnm > 20
lowAmp: dead_channel_exp >= 0.3 && pct_below_nlnm > 20
flat: sample_unique < 200
polarity: polarity_check <= -0.5
noise1: dead_channel_exp < 0.3 && pct_above_nhnm > 20
noise2: dead_channel_lin < 2 && pct_below_nlnm <= 20 && num_gaps < 10
hiAmp: sample_rms > 50000 && pct_above_nhnm > 30
gainRatio: ms_coherence >= 0.999 && (gain_ratio <= 0.95 || gain_ratio >= 1.05)
nonCoher: ms_coherence <= 0.990
dcOffsets: dc_offset > 10
badRESP: pct_above_nhnm > 90 || pct_below_nlnm > 90

The table below illustrates some of the metrics and time spans returned by automated MUSTANG queries for AK data for the month of January 2017.

Focusing attention on these channels and time spans, QA staff reviewed additional metrics, waveforms, and power spectral density (PSD)/probability density function (PDF) noise profiles to identify and describe the actual data problems.

For each confirmed data problem, a ticket was created in our tracking system that includes

• a description of the problem,
• the network, station, location code, channel and time span affected,
• the metrics or techniques used to diagnose the problem, and
• its resolution status.

As directed by the IRIS-AK Cooperative Agreement, issues that remained unresolved at the end of each report period were parsed into an HTML report and sent to AK network operators (see example below).

Natalia Rupert, seismic network manager of the AK network, says this about the new MUSTANG-based QA reports:

“Alaska Earthquake Center (AEC) operates over 100 remote broadband stations located across the state. Assessing state of health of field sites is a responsibility shared by the AEC staff. While AEC maintains its own set of tools and procedures for monitoring data acquisition continuity and data quality, we also make use of external resources such as MUSTANG tools. Monthly network data quality reports produced by the IRIS DMC analysts include additional information regarding data quality issues that are not easily detectable via standard tools available at our disposal. DMC data quality analyst Laura Keyson flagged several data quality issues at the remote sites that otherwise would have been left undiagnosed. This is valuable information that we will be able to use to plan our summer field maintenance visits.”

As we work with Natalia and the AEC staff, we look forward to optimizing these reports and eventually offering this analysis for additional seismic networks.