Tests for Quality Control

These are the tests available in CoTeDe. Most of the QC recommended guides follow simmilar procedures with small variations, as described below when relevant.

Valid Date

Check if there is a valid date and time associated with the measurement.

For Argo, the year also must be later than 1997.

For underwater gliders, the year must be later than 1998.

Valid Position

Check if there is a valid position associated with the measurement. It should have a latitude between -90 and 90, and a longitude between -180 and 360.

GTSPP restricts the longitude range to between -180 and 180 degrees.

Location at Sea

Check if the position is at sea by using a bathymetry database. If not specified, it is used ETOPO1, a bathymetry with resolution of 1 minute. This test implies approval on Valid Position.

If the position is determined at sea, GTSPP also evaluates the sounding, if present, but that is redundant to the Sounding Test, so this part is neglected in CoTeDe.

Global Range

This test evaluates if the measurement is a possible value in the ocean in normal conditions. The thresholds used are extreme values, wide enough to accommodate all possible values and do not discard uncommon, but possible, conditions.

Regional Range

This test is equivalent to the Global Range but with a horizontal domain where it is applicable. Good examples are the Mediterranean Sea and the Red Sea, where the feasible range is more restrictive than required for the Global Range.

Regional Range was first introduced in the GTSPP manual 1990.

Note

This test requires the Python package Shapely to read a polygon geometry and evaluate which positions are within that domain.

GTSPP also requires the maximum depth to be less than 5200m in the Mediterranean Sea, and less than 3500m in the Red Sea.

Profile Envelop

This test is equivalent to the Global Range but with a vertical domain where it is applicable, i.e. it defines the acceptable range per layer. For instance, GTSPP defines that deeper than 1100m up to 3000m the acceptable temperature range is from -1.5C to 18C.

Profile Envelop was first introduced in the GTSPP manual 1990.

Gradient

This test compares

X_i = \left| V_i - \frac{V_{i+1} + V_{i-1}}{2} \right|

Gradient Depth Conditional

Spike

X_i = \left| V_i - \frac{V_{i+1} + V_{i-1}}{2} \right| - \left| \frac{V_{i+1} - V_{i-1}}{2} \right|

Spike Depth Conditional

Tukey 53H

This method to detect spikes is based on the procedure initially proposed by Goring & Nikora 2002 for Acoustic Doppler Velocimeters, and similar to the one adopted by Morello 2011. It takes advantage of the robustness of the median to create a smoother data series, which is then compared with the observation. This difference is normalized by the standard deviation of the observed data series after removing the large–scale variability.

For one individual measurement x_i, where i is the position of the observation, it is evaluated as follows:

  1. x^{(1)} is the median of the five points from x_{i-2} to x_{i+2};

  2. x^{(2)} is the median of the three points from x^{(1)}_{i-1} to x^{(1)}_{i+1};

  3. x^{(3)} is the defined by the Hanning smoothing filter:

    \frac{1}{4}\left( x^{(2)}_{i-1} +2x^{(2)}_{i} +x^{(2)}_{i+1} \right)

  4. x_i is a spike if \frac{|x_i-x^{(3)}|}{\sigma} > k, where \sigma is the standard deviation of the lowpass filtered data.

The default behavior in CoTeDe is to flag 4 if the test yields values higher than k=1.5, and flag 1 if it is lower.

Climatology

X_i = \frac{V_{it} - <V_t>}{\sigma}

QARTOD climatological test is based on range

Rate of Change

For QARTOD, the delta change is normalized by the standard deviation.

Density Inversion

This test looks for density inversions in the water column, i.e. higher density above lower density.

Since density inversion is unstable it is not expected to be observed in nature in normal conditions. Note that weak inversions migth be observed near the surface under special conditions of sea surface heat fluxes. Sometimes a small negative threshold is used.

Density Inversion was first introduced in the GTSPP manual 1990.

Constant Cluster

For Argo …

This test evaluates a cluster of adjacent measurements that are identical or nearly-identical. This was implemented in CoTeDe as a generalization of the tests: Constant Profile, Stuck Value, Flat Line.

The Constant Profile tests was first introduced in the GTSPP manual 1990.

GTSPP call it Constant Profile test, and requires the full profile to be identical 3 or more measurements.

Deepest Pressure

Check for each measurement if the reference pressure (depth) is deeper than the operational limit for that sensor/platform. For instance, the Argo Solo-II operates up to 2000m while the Deep Solo goes up to 6000m. Measurements deeper than that suggest a bad vertical position.

Reference: Argo QC manual 2.9.1

Digit Rollover

Every sensor has a limit of bits available to store the sample value, with this limit planned to cover the possible range. A spurious value over the bit range would be recorded as the scale rollover, resulting in a misleading value inside the possible scale. This test identifies extreme jumps on consecutive measurements, that are wider than expected, suggesting a rollover error.

The difference on consecutive measurements must be smaller or equal to the threshold to be approved.

Monotonic Vertical Displacement

Evaluates the vertical movement of a profiling platform. If the vertical movement stops, like in a constant depth, all measurements without vertical displacement, except the first one, are flagged bad. If there is an inversion, all measurements in the inverted sequence are flagged bad.

The vertical movement is usually infered by the records of pressure or depth along the time.

The argument from this test might come from the assimetry of every sampling platform. A CTD in a rosette usually samples in the downcast so the CTD itself measures a water parcel less disturbed by the sampling bottles and the rest of the rosette. Spray underwater gliders have the CTD intake in the upper part of the hull, hence the ascencion phase of the dive leads to cleaner samples. Therefore, a change in the “flight” pattern wouldn’t mean a bad measurement per se, but an inconsistent measurement with he previous and following values.

GTSPP evaluates depth.

Argo evaluates pressure.