Quality Checks

It is important to archive data set of good quality, directly usable for scientific or operational purposes. Therefore a cruise is only archived at the end of a series of quality checks, according to the Intergovernmental Commission and MAST recomendations. These checks are a necessity for the historical data, but are also useful for recent data already validated by the scientific laboratories, bacause any data handling is a potential source of error.

Each data set (cruise) is submitted to a series of automatic, therefore objective tests, followed by manual tests by displaying the results of the automatic test on a screen. As a result, a quality flag is added to each numerical value, but the values themselves are not modified. The choosen flag scale is simple and already used in other international projects. The flags are than validated or corrected manually, taking into consideration the overall coherence of the data within the cruise, which is somehow subjective but not arbitrary. The automatic checks are made by using pre-existing knowledge on the region: extreme values for broad range checks (corresponding to high error level), and previous climatological profiles for narrow range checks; there is also some subjectivity in the running of these parameters.

Specific software have been developed to perform these tests on UNIX ( SCOOP) by SISMER.

It is expected not to get any flag different from 1, otherwise the originator will be contacted to see where is the problem and if the data point has to be corrected.


FLAG SCALE

The flag scale is the flag scale adopted by the IOC for the GTSPP International program of temperature exchange in real time. The flag values, if documented, is proportional to the level of error.

--> 0 : No QC --> 1 : Correct --> 2 : Inconsistent --> 3 : Dubious --> 4 : False --> 5 : Modified


QC-0 Check of the Format

This check include the completness of the documentation like ship name and code etc. Requested corrections or completion are made before any further control.


QC-1 Check of the Headers : date and location

The following tests are performed automatically and the erroneous results displayed on a screen to validate the flags. All these checks concern location and date and if any error is identified, the originator is contacted:

TEST FLAG VALUE
1.1 : Check for duplicates ELIMINATED
1.2 : Check the date --> 5
1.3 : Check the ship velocity --> 5
1.4 : Check the location/shoreline --> 5
1.5 : Check the bottom sounding (ETOPO5) --> 2
No outliers --> 1
1.6 : Visualisation and manual checks


QC-2 Check of the parameters

These checks do not modify the observation but add the quality flags. These checks are implemented regionally, according to the expertise of the data centres, and only for temperature and salinity vertical profiles. The measured parameters are controlled with references to the local characteristics of temperature and salinity: minimal and maximal values, spike threshold, climatological values.

The higher severity checks are performed first, because there is no reason to perform for example narrow range checks, if a value is already out of the regional broad range scale. Only the vertical stability check is performed at the end because it makes use of the results of the other checks and it is more difficult to implement (4 values are taken into account).

TEST FLAG VALUE
2.1 : Missing pressure --> 4
2.2 : Costant values on the vertical --> 4
2.3 : Impossible regional values (min. & max.) --> 4
2.4 : Check for increasing reference (pressure) --> 4
2.5 : Data point below the bottom depth --> 4
2.6 : Check for spikes --> 3
2.7 : Compare with the pre-existing statistics (MODB) --> 2
2.8 : Check the vertical stability --> 4
No outliers --> 1
2.9 : Visualisation and manual checks

The coherence and continuity of the observations within a cruise is only checked subjectively, and allow to make manual corrections of the flags especially :

If the flags are changed manually, a note is added in the " COMMENT " fields.


Global Parameter Flag

When a parameter is fully checked, a " global parameter flag " is assigned to each parameter, depending on the percentage of flagged values. It has been chosen not to attribute any quality index to profiles with less than 3 good levels in the vertical, first because this test can be automatically recomputed, also because the interest of such " gappy " profiles, depends on the potential further scientific analysis for example time series of coastal stations or deep sea geostrophic computations.

If at least 80% of the values are without outliers, the global parameter is flagged to 1 (good). If not, the global flag is assigned to the most frequent error flag (20%).


GLOBAL

Q Flags

1

Correct

2

Inconsistent with statistics

3

doubtful, questionable

4

bad, wrong, erroneous

9

Missing value

 

PROFILE

Few Outliers

 

no temperature recorded

- no pressure recorded (nor depth)

- constant profiles

not allowed

PARAMETER

> 80% values without outliers

> 20% values with Q flag=>2

> 20% values with Q flag=>3

> 20% values with Q flag=>4

 


Last update 24/07/1997

Return Back