From the Ground Up
What does accuracy as related to surveying and mapping deliverables mean to you? Merriam-Webster defines accuracy as: 1 freedom from mistake or error; 2 conformity to truth or to a standard or model; 3 degree of conformity of a measure to a standard or a true value. Geospatial data, whether taking the form of base mapping for a city's GIS, surveys for a new land development project, or inventory locations of utility features like manholes and fire hydrants, can be critically important as a base on which to build additional surveying information or engineering design. Moreover, clients are becoming much more sophisticated in the way they use geospatial data. Because of this increasing sophistication, the accuracy of the data, or the data's conformity to its true value, is critically important, and in turn, increasingly subject to accuracy assessments.
How do you define accuracy in your professional life? Clients often ask for a specific accuracy level during initial project discussions. For example, they may state, "We want the mapping accurate to within one half of a foot." Similarly they may comment, "Your peers have told us they can provide data accurate to within one quarter foot. I assume you can provide the same level of accuracy." But those statements by themselves are not complete and don't clearly define the accuracy expectations for a project. Does the client mean that their expectations are that all portions of the project fall within these defined limits? Is the intention of their accuracy statements based on a 90 percent confidence interval, a 95 percent confidence interval, or even some other expectation? Statements about accuracy all too often mean very different things to different people.
Because of the many different interpretations of "accuracy" in geospatial data, combined with the relative obsolescence of the National Map Accuracy Standards (NMAS) of 1947, a new standard was developed for assessing the accuracy of geospatial data in 1998. The National Standard for Spatial Data Accuracy (NSSDA) was developed by the Federal Geodetic Data Committee (FGDC) and is gaining popularity within the professional community. Many clients now request a formal assessment of accuracy under the NSSDA as a project deliverable in both mapping and traditional surveying projects. Therefore, it is critically important to have a fundamental understanding of accuracy in geospatial data today.
Accuracy StandardAccording to the FGDC, the "NSSDA implements a well-defined statistic and testing methodology for positional accuracy of maps and geospatial data derived from sources such as aerial photographs, satellite imagery, or maps." The intent of the FGDC was to create a new standard that would replace the older, outdated NMAS that were developed in 1947. Furthermore, "The applicability of NMAS is limited to graphic maps, as accuracy is defined by map scale. The NSSDA was developed to report accuracy of digital geospatial data that is not constrained by scale." Under NSSDA, the accuracy expectations are determined by the client independent of scale on a project-by-project basis.
The NSSDA defines accuracy at the statistical 95 percent confidence level and assumes errors taken on a normal distribution. Therefore, when a client states (under NSSDA) that they want their product accurate at the 1-foot level, they are telling us that we should be confident that 95 percent of the geospatial data we provide are accurate to within 1 foot of their actual values. This contrasts with the older NMAS, which defined accuracy at the 90 percent confidence level.
But the applications of the two standards also have similarities. In both cases, spatial data from the source to be tested (the work we do as surveyors and mappers) is compared against data from an "independent source of higher accuracy." In other words, values of well-defined features found within the new geospatial information are compared against other values for the same features that we hold to be "true," gained from methods in which we have a very high level of confidence.
Many times, GPS surveys are used to compute ground truth positions. The GPS methods used can vary significantly, from high-precision static or real-time kinematic (RTK) methods to sub-meter differential observations. The intended accuracy of the geospatial data being tested will drive the methods for determining the truth position, with higher accuracy data requiring higher accuracy surveying methods for truthing. The positional values of the same feature from the two sources are compared and statistics computed from these comparisons.
Application of the StandardsThe mathematical application of the NSSDA is very straightforward. The 95 percent confidence intervals for both the horizontal and vertical accuracies found in the NSSDA are based on the statistical Root Mean Square Error (RMSE). "The RMSE is the square root of the average of the set of squared differences between data set coordinate values and coordinate values from an independent source of higher accuracy for identical points." This can be shown mathematically for the vertical RMSE as:
RMSEz = sqrt [S(z data i - z check i)2/n]
z data i is the ith data point in the data set being checked
z check i is the ith check point from the independent source
n is the number of check points tested
The RMSE is similarly calculated for the x and y components when determining horizontal accuracy. However, because the NSSDA defines a confidence interval based on a radial error and not the individual x and y components of this radial error, one additional mathematical step has to take place to determine the radial error from these components. Mathematically, this can be shown as:
RMSEr = sqrt[RMSEx2 + RMSEy2]
A minimum of 20 well-defined check points should be selected from the data set for testing. Larger projects will generally require more check points. Their locations should be selected based on their geographic distribution within the data set being tested and the ease of gaining comparable positions to test against. For example, if GPS field methods are being used to determine the independent positions, the points should be located in areas that are both accessible to ground crews and suitable for GPS observations. And of course, the points must also be readily identifiable in both the data set and on the ground. The types of points selected will vary with the level of accuracy being tested, the output scale and the types of geospatial features found in the data set. Intersections of roads, railroads, or other major linear features may be suitable for lesser accuracy requirements, but the exact location of such features can be a bit ambiguous. For higher accuracy requirements, more exact features like sidewalk intersections, utility access covers or fence intersections represent more accurate locations.
Once the positions of all check points are determined, a simple spreadsheet can be created to compare the data points versus the check points, determine the individual RMSE values, and finally determine the accuracy of the data set, which in terms of the NSSDA is stated as the 95 percent confidence interval.
The relationship of the RMSE values and the 95 percent confidence intervals is, again, very straightforward. The statistical bases for these determinations are beyond this article, but they can be summed up as follows:
Vertical Accuracy = 1.9600 x RMSEz
Horizontal Accuracy = 1.7308 x RMSEr
RMSEz is the RMSE of the vertical differences
RMSEr is the RMSE of the radial (combination of x and y) differences
The multiples of the RMSE values are different because the vertical accuracy is stated in terms of a linear measurement, while the horizontal is stated in terms of a circular error.
A Summary of the NSSDAThe NSSDA is significantly broader than the older NMAS it was intended to replace. In fact, a quick glance at the names of each of the standards provides insight into the fundamental differences of the two. The older standard was developed to assess "map accuracy" while the new standard encompasses all "spatial data," whether presented as a map, in tabular form or as a layer in a GIS. As such, it extends well beyond mapping professionals to traditional surveying firms and GIS professionals.
The NSSDA places more formal requirements on surveying and mapping professionals in terms of accuracy requirements for geospatial data deliverables, but it also creates new and interesting market opportunities for traditional surveying professionals in terms of performing independent assessments of geospatial deliverables. These assessments might be performed directly for the end user of the data (client) or for the geospatial provider so they can include the independent assessment as part of their overall deliverables. Whether planning for or evaluating the accuracy of your own data, or providing professional services to assess the accuracy of the work of others, spend some time familiarizing yourself with the NSSDA. The investment of your time could yield dividends in the future.