- » Validation
Validation of land products derived from Earth observation
Validation is a critical step in the quality control of products derived from Earth observation. It may involve comparison with similar products derived by conventional means, in which case it is important that those products have themselves been validated and their accuracy and uncertainty established. The first problem encountered is the disparity between the size and type of objects measured on the ground and those measured by the remote sensing system. For example, measurements of chlorophyll content in the field may be based on the relative reflectance of green and red light by individual leaves, whereas a similar measurement from an aircraft or satellite sensor would be based on the spectral radiance of a pixel, which would include shadow, soil background and non-green plant parts as well as green leaves.
A more fundamental problem concerns the definition of the variable being measured. Whereas it is possible to measure Leaf Area Index directly in the field by harvesting the vegetation within a unit area, a remotely sensed determination of the same variable might be based on the amount of light reflected in different wavelengths, or on some sort of spectral index. Thus any comparison between a destructive estimate of LAI and one made remotely depends upon the modelled relationship between actual LAI and what may be termed 'spectral plant cover'. Validation, therefore, involves informed comparison between EO data products and ground measurements. The aim of this section of the website is to provide the information necessary to make such comparisons, and to document emergent best practice in this area. It aims to underpin international efforts in this area, such as those of the Committee of Earth Observing Satellites Working Group on Calibration and Validation (CEOS WGCV).
Although validation is very important to users of EO data, complete validation of any EO data product may be more of a goal than a realistic expectation, at least in the short- to medium-term. Therefore it is important to recognise stages along the way to complete validation, and understand the limitations that incomplete validation places upon the use of a product. For example, NASA recognises three stages of validation for MODIS land products:
|Stage 1||Product accuracy has been estimated using a small number of independent measurements obtained from selected locations at particular times. Some ground data collection involved.|
|Stage 2||Product accuracy has been assessed over a widely distributed set of locations and at a number of times via several ground data collection and validation efforts.|
|Stage 3||Product accuracy has been assessed and the uncertainties in the product well established via independent measurements in a systematic and statistically robust way representing global conditions.|
All EO data products should be provided with validation to stage 1 by the data provider. Validation to stage 2 should be a pre-requisite for any data product to be used for monitoring change. Validation to stage 3 is required for data products to be used in decision support systems.
Anyone planning a validation campaign or remote sensing field experiment will benefit from reading the following articles which contain a lot of practical information as well as warnings about the pitfalls:
Gamon et al. (2004)
Summary of the lessons learned in the Boreal Ecosystem Atmosphere Study (BOREAS), a major 10-year validation experiment in Canada.
Privette et al. (2000).
A description of a validation experiment at the Jornada test site in New Mexico.
The US National Academy of Sciences has published a useful book of case studies drawn from experiments across the environmental sciences, including the First International Satellite Land Surface Climatology Project (ISLSCP). The Executive Summary is available on-line at http://books.nap.edu/catalog/4896.html. The title of this book is 'Finding the Forest in the Trees : the challenge of combining diverse environmental data' and one of the interesting findings is that of ten 'keys to success' identified, only two were technical in origin, whereas eight resulted from human behaviour and the social, cultural and organisational context within which major scientific experiments take place.
Hall et al. (1992) summarises the outcomes of the First ISLSCP Field Experiment (FIFE) which took place in Kansas between 1987-89. The legacy website for this project can be found at http://www-eosdis.ornl.gov/FIFE/FIFE_About.html