Landcover Mapping Methods

A synopsis of the mapping methods for the region is described in the Data_Quality_Information section (Process_Step) of the FGDC Metadata for the land cover dataset.

Detailed descriptions of mapping methods are available for each state responsibility area and each mapping zone. These more detailed descriptions of methods provide the following information: summary of predictor layer information (imagery, etc.), description of training samples, description of modeling methods and validation results. For more information on the mapping legend, see the legend descriptions.

Mapping Zones

As a coordinated multi-institution endeavor each state institution was responsible for various tasks which were divided geographically. The five-state region was divided into ecologically and spectrally similar mapping zones. Each mapping zone provided a functional working area for project management, data collection and modeling. Each state was responsible for the mapping zones roughly corresponding to their state jurisdiction. Mapping zones were created at two levels. At the first level, 47 mapping zones were created for the five-state region. These were primarily used for collecting and organizing field training data. At the second level, 25 mapping zones were created for the five-state region. These were used for land cover modeling and validation.

• Download 25-zone dataset / Metadata
• Download 47-zone dataset / Metadata
• View the description of the map zone creation process

Map Quality Assessment

Objectives

Our primary objective is to provide map users with as much information as possible regarding the quality of the map product. Map quality is presented in two forms: 1) through a quantitative measure of how well the product agrees with land cover classes on the ground and, 2) through qualitative summaries of perceived map quality provided by people involved in the mapping process. Both approaches to map assessment provide information on individual land cover classes for specific eco-regional mapping zones. Error matrices and qualitative assessments for each land cover class are provided on a mapping zone by mapping zone basis rather than for the entire 5-state region.

Model Validation

Our approach involved randomly selecting 20% of all available samples stratified by land cover class for each given mapping zone, and withholding them from initial model generation. Following initial model/map generation reference samples were intersected through the resulting map and output formatted as an error matrix, with an accompanying kappa calculation. Since reference samples were polygons, the map was considered correct when the majority of map pixels agreed with the land cover label for a given reference sample polygon.

The final map was generated using 100% of the available sample data. In addition, for a minority of classes, post-modeling steps including conditional models, localized supervised clustering, or on-screen digitizing were used to refine the map. Thus, we emphasize that the assessment process described above is not considered an assessment of the map per se, but is an assessment of how well the modeling approach predicted most of the land cover classes. It should also be noted that for rare land cover classes, withholding 20% of available sample data provided very few reference samples. Despite these caveats, we believe that this approach provides the best quantitative measure of map quality possible, given the size and scope of our mapping effort. For more information on the modeling process please download methods descriptions using the map.

Fuzzy Set Assessment

Our approach to fuzzy set assessment is based on the work of Gopal and Woodcock (1994) and described by Congalton and Green (1999). The premise behind fuzzy set theory for thematic map accuracy assessment is that thematic mapping involves placing a continuum of land cover into (somewhat artificially) discrete land cover classes. In reality the landscape does not fit unambiguously into these discrete categories. The objective of using fuzzy sets for thematic map assessment is to provide map users with information about the frequency and magnitude of map error. In other words a reference site may have been mapped incorrectly, but how incorrect was it? An answer to this question can be provided by re-evaluating the error matrix within the context of recognized similarities among land cover classes.

Our methods focused on ecological similarities among land cover types (as opposed to spectral similarities, for example). Criteria were established for types of ecological similarity (Table 1) and their assignment of a relative similarity score (Table 2). Given these recognized similarity types, each error cell (off diagonal cells) in the original error matrix (e.g. Table 3) was evaluated (e.g. Table 4) and given a similarity score (e.g. Table 5).

The relative similarity scores from Table 5 were then used in conjunction with the original error matrix to produce iterative samples of “fuzzy set adjusted” error matrices (Table 6, Table 7, and Table 8). The fuzzy set adjusted error matrices demonstrate how both user’s and producer’s accuracy may be improved when the measure of agreement between the reference samples and the map incorporates a wider understanding of ecological similarity.

The adjusted error matrices present three levels of error that include: misclassification of very similar land cover classes, misclassification of very similar and moderately similar land cover classes, and misclassification of very similar, moderately similar and somewhat similar land cover classes. The relative difference in user’s and producer’s accuracy considering the three variations of fuzzy set adjusted similarity is available in Table 9 and Table 10 respectively, with their corresponding graphs in Graph 1 and Graph 2. A more detailed set of instructions for using Tables 1, 2, and 3 to produce Tables 4 – 10, can be found here.

Error matrices and graphs for SWReGAP mapping zones are available for download using the above map.

Qualitative Assessment

Recall that some land cover classes were not modeled with the decision tree classifier but were instead incorporated into the map as a post-modeling step. In addition, for some classes withholding 20% of the available samples resulted in very few reference samples. Due to these shortfalls with the quantitative assessment, and because the land cover teams involved in the mapping process believe there is value in a qualitative summary, we have included in this assessment summaries for each land cover class by mapping zone.

Land cover qualitative summaries are brief descriptions provided by the teams involved in the mapping process for a particular mapping zone. They are intended to provide an honest evaluation from the perspective of the land cover mapping analyst of how well the land cover class was mapped, taking into consideration the number of training and reference samples available for the cover class and the team’s knowledge or familiarity with the mapping area. Often the summary provides a narrative interpretation of the error matrix, identifying in qualitative terms where a particular land cover class is being misclassified and with which cover types it is being confused.

References

Congalton, R. G. and K. Green. 1999. Assessing the accuracy of remotely sensed data: Principles and practices. Lewis Publishers, New York.

Gopal, S. and C. Woodcock. 1994. Theory and methods for accuracy assessment of thematic maps using fuzzy sets. Photogrammetric Engineering and Remote Sensing. 60: 181-188.