LIDAR System CalibrationCalibration is performed with all remote-sensing systems in order to detect and correct systematic errors. The American Society for Photogrammetry and Remote Sensing (ASPRS) has not yet established standard LIDAR calibration procedures, but a committee is working this year to develop such procedures.
For the North Carolina floodplain mapping initiative, both LIDAR firms were required to submit plans for daily system calibration, on or near the airports where they are based. A typical airport control scheme for LIDAR system calibration is shown in Figure 1.
Both firms established a series of survey control points on flat terrain on or near the runways, plus other control points surveyed on rooftops of several buildings adjacent to the runways plus on the ground surrounding those buildings. The ground control points are used to determine the vertical accuracy of the LIDAR system, but such control on flat terrain does not confirm the horizontal accuracy of the system. Only when the elevations are correct on both the ground and rooftop points, resulting from multiple fly-overs in different directions, can one be assured that the system is operating correctly in both vertical and horizontal positioning.
Typically, for “hard targets” such as roofs and runways, the vertical root mean square error (RMSE) is 15 cm or better and the horizontal RMSE is 1 meter or better. The true challenge for LIDAR is in measurement of “soft targets,” especially forests with dense canopies.
Temporary Continuously Operating Reference Stations (CORS)As recommended by ESP Associates, subcontractor to Watershed Concepts, NCGS agreed to support the intense LIDAR data collection missions, which during peak times can operate 24 hours a day/7 days a week, by establishing and maintaining a network of 18 project-specific continuously operating reference stations (CORS) in four of the six river basins. In addition to supporting the LIDAR data collection, this network also supports the hydrologic and hydraulic (H&H) structure surveys as well as the independent, quality control survey firms that are ground truthing the LIDAR data.
Each of these project CORS is tied to the North Carolina High Accuracy Reference Network (HARN) and provides a 30 km radius of coverage for data acquisition. Each station is equipped with a dual frequency Global Positioning System (GPS) receiver, a geodetic quality antenna with ground plane, and a computer using specialized software to continuously log data at one-second intervals. In addition to the project CORS, two existing NCGS CORS sites (Raleigh and Washington) are also being used to support the LIDAR data collection missions.
Each of these project CORS was sited using detailed survey procedures, has an unobstructed view of the sky to 10 degrees above the horizon, and is located in a secure area within a public or a private facility. At each station, the horizontal and vertical position of both the GPS antenna phase center (APC) and antenna reference point (ARP) were obtained by:
1. Using a minimum of four HARN monuments;
2. Performing repeat static GPS sessions on different days and times between sessions to ensure different satellite geometry, as specified in NGS-58, “Guidelines for Establishing GPS-Derived Ellipsoid Heights (Standards: 2 cm and 5 cm);” and
3. Conducting additional GPS observations utilizing NGS CORS sites to independently verify the positional information at each of these project CORS.
At the completion of this phase of the project, the 18 project CORS will be moved to sites in the next five river basins (Catawba, Chowan, New, Roanoke and Yadkin), which are scheduled to have LIDAR data acquisition in the 2001-2002 leaf-off season.
Generation of Bare-Earth TINs and DEMsThe first step in generating bare-earth Triangulated Irregular Networks (TINs) is to reduce the raw LIDAR, GPS and inertial measurement unit (IMU) data into XYZ coordinates. Proprietary algorithms use electronic timing signals to compute ranges to ground targets. These ranges are combined with positional information from the airborne GPS and orientation information from the IMU in order to compute 3-D vectors in space and compute XYZ ground coordinates. System calibration data is integrated in the processing algorithms. Cross-flight lines, flown perpendicular to the normal flight lines, are also used to identify and correct other errors. The cross-flight lines enforce consistency between overlapping flight lines.
The second step is to use automated post-processing to filter and remove points that appear to be on rooftops or on vegetation canopy. Software parameters can be varied to control the filtering of data. This requires a great deal of experience. Filtering trees on steep terrain, for example, is difficult because a point on the top of a tree can be nearly the same elevation as an adjoining uphill point on the ground. Rooftops are more easily filtered since they normally have abrupt changes in elevations between the rooftop and surrounding ground elevations. Dikes and levees are problematic since they appear almost identical to hedgerows that need to be removed.
The third step is to use manual or interactive post-processing to remove artifacts that remain after automated post-processing. A shaded relief view of a TIN or Digital Elevation Model (DEM) is often used for this purpose, but this also can be misleading. It would be correct to remove a high point if it were caused by a tree, for example, but it would be wrong to remove the high point if it were caused by a rock outcrop; and both would appear identical when viewing a TIN or shaded relief model of the terrain. For this reason, it is preferred to utilize digital orthophotos produced from aerial photography that is reasonably current. For the North Carolina project, neither firm is acquiring metric aerial photography concurrently with the LIDAR data acquisition. This is because LIDAR is commonly flown at night when light conditions are unsuitable for photography. Also, the field of view and flying heights of LIDAR sensors and cameras are often incompatible unless cameras are specially designed for integration with the LIDAR sensor. Although some LIDAR systems already utilize digital cameras that were designed for compatibility with the LIDAR sensor, it is difficult to mosaic small images together for coverage of large areas with the desired horizontal accuracy.
To illustrate these bare-earth generation processes from a FEMA/U.S. Army Corps of Engineers demonstration in California, Figure 2a shows a last-return LIDAR dataset prior to post-processing. The higher elevations are mostly tops of buildings or trees not penetrated by the LIDAR.
Figure 3a shows the same last-return LIDAR dataset after post-processing. Here, most of the higher elevations (tops of buildings or trees not penetrated by the LIDAR) have been removed by the automated and manual post-processing procedures. Now, all the black points show areas where LIDAR points were removed by the post-processing procedures. Figures 2 and 3 were provided as a courtesy by the U.S. Army Topographic Engineering Center.
A TIN is the first bare-earth product that results from these processes. The TIN is the dataset that should be evaluated for accuracy. A uniformly spaced DEM may subsequently be produced by interpolation of TIN data to generate elevations at the computed horizontal coordinates, which are normally State Plane coordinates where Northings and Eastings are evenly divisible by 5 meters (in the case of North Carolina) or any other specified point spacing required.
Quality Control (QC)FEMA regulations require LIDAR TINs to be independently evaluated for accuracy prior to use in automated or semi-automated hydraulic modeling. For each county to be remapped, a minimum of 20 checkpoints must be surveyed in each of the three to six major land cover categories that are representative of the county’s floodplain. For North Carolina, the following land cover categories were selected:
1. Bare-earth and low grass (includes sand, rock, plowed fields, lawns and golf courses)
2. High grass, weeds and crops
3. Brush lands and low trees (evergreen, deciduous and mixed shrublands)
4. Forested areas
5. Urban areas
NCGS contracted with eight survey firms (not affiliated with the Watershed Concepts or Greenhorne & O’Mara teams) to perform the independent QC surveys, all tied to NCGS HARN stations. NCGS worked with the North Carolina Center for Geographic Information and Analysis (CGIA), using the state’s land cover database to select representative areas throughout each county within which the survey firms are to survey the checkpoints needed for the five land cover categories. All checkpoints must be on flat or uniformly sloping terrain within 5 meters of the checkpoints. This means that no checkpoint can be within 5 meters of a breakline where there would be a significant change in slope. This requirement enables TIN linear interpolation procedures to be used; i.e., to interpolate the LIDAR elevation from TIN points that surround the horizontal coordinates of the surveyed checkpoints.
Although consistent with the Federal Geographic Data Committee (FGDC) National Standard for Spatial Data Accuracy (NSSDA), this QC process has a built-in pitfall due to inherent limitations of computing the RMSE. Suppose one ground checkpoint happened to fall in a forested area where the LIDAR failed to penetrate the vegetation (10 meters high), and the LIDAR elevations for the 99 other checkpoints for that county all had vertical errors of only 10 cm. In the RMSE calculation, the squared errors for the 99 good points sum to 0.99 m2, and the squared error for the 1 bad point is 100 m2. The mean of the 100 squared errors would be 1.01 m2, and the square root (the vertical RMSE) would be approximately 1 meter, equivalent to a contour interval of 3.3 meters (3.2898 x vertical RMSE) or 10.85 feet. However, with the old National Map Accuracy Standard (NMAS), 10 percent of the “outliers” are discarded, regardless of size; the one large outlier in this example would be discarded per the NMAS, and the equivalent contour interval would be less than 20 cm (per NMAS procedures) instead of 3.3 meters (per NSSDA procedures).
The problem here is that the NSSDA assumes all errors are random errors that have a normal distribution thus “outliers” are not routinely discarded in computing the RMSE. When errors do not have a normal distribution (as when a LIDAR pulse fails to penetrate dense vegetation), all bets are off, and the RMSE calculation is misleading. LIDAR appears to be able to measure hard targets with a vertical RMSE of 15 cm or less, where all factors contributing to the error probably have a normal distribution. But in the case of soft targets, especially dense vegetation, the LIDAR’s laser pulses will not penetrate to the ground if a person standing on the ground cannot see the sky above. LIDAR technology should not be faulted for this anymore than photogrammetry is faulted by an inability to see the ground in stereo.
Photogrammetry has traditionally used “obscured terrain” and dashed contour lines to depict terrain where the analyst was unable to see the ground on stereo imagery for generation of spot heights or compilation of accurate contours. We need something comparable for LIDAR.
For these reasons, North Carolina has directed its LIDAR contractors to specify areas where they believe the LIDAR failed to penetrate the vegetation. In such areas, assuming they are not too extensive, vertical checkpoints may be discounted or discarded. The remote-sensing community as a whole needs to address this issue to prevent unfair comparison with competing technologies that have similar problems. Although LIDAR is better able than photogrammetry to see between the trees, or through the trees, it is not the panacea for all scenarios. We still have unavoidable rules of physics with which to contend.
For the North Carolina project, the worst that can happen is that more ground surveys may be required than originally intended, but this remains to be seen. This project is seen as the “litmus test” for reliance on LIDAR technology for semi-automated H&H modeling.
Conventional Field SurveysEven if LIDAR mapped the bare-earth terrain with perfect accuracy, conventional field surveys are still needed for the remapping of North Carolina floodplains. For example:
- Temporary bench marks are surveyed relative to NCGS HARN monuments so that local surveys will have network accuracy relative to the National Spatial Reference System (NSRS).
- Hydraulic structures (bridges and culverts) are surveyed to obtain the structure geometry and the natural ground at a bridge or culvert. Several cross sections are normally surveyed at each structure, one just upstream of the hydraulic structure, one along the roadway centerline, and possibly a third cross section just downstream of the structure. For culverts, the upstream and downstream inverts of each culvert barrel are surveyed, as well as the shape and dimensions of the entrance and exit headwalls and wingwalls.
- When there are long distances between hydraulic structures on a stream, it is common to survey intermediate cross sections to determine the channel geometry, especially if there is cause to believe that the channel geometry changes between the upstream and downstream bridges/culverts. (Note, as mentioned above, LIDAR is good at determining the topography in overbank areas, but is limited between stream banks, especially if the channel is filled or partly filled with water).
- Dams are surveyed by three cross sections: upstream (underwater) cross section of the lake; a cross section across the top of the dam; and a cross section downstream of the dam embankment. Spillways also receive special surveys.
- High-water marks from Hurricane Floyd and other peak events are surveyed to assist in calibration of the hydraulic models and for validation of predicted flood elevations.
Not all North Carolina streams will be surveyed by such detailed methods. Detailed methods will be determined by age of existing studies, future development and other factors. However, the new studies, including those with approximate rather than detailed procedures, will be considerably more accurate than the studies they replace. This project is destined to demonstrate how far we have come with LIDAR in recent years and challenges that remain. We are confident that improvements in flood risk assessment accuracy resulting from the LIDAR surveys will save many millions of dollars in future flood losses. This project is also expected to demonstrate how advanced technology can be used to cost-effectively improve Flood Insurance Studies nationwide so that FEMA’s limited budget and budgets of Cooperating Technical Partners can yield significantly greater returns for each dollar invested.
Learning More About the Latest DEM TechnologiesThe American Society for Photogrammetry and Remote Sensing (ASPRS) is currently developing a new reference book to be titled Digital Elevation Models: New Technologies and Applications. This book will discuss the capabilities and limitations of various remote-sensing technologies and it will have a user focus to help readers understand the relative advantages and disadvantages of diverse technologies. To be published in the fourth quarter of 2001, this book will address the following major topics:
- 3-D Surface Concepts
- Vertical Datums
- Accuracy Standards
- National Digital Elevation Program
- Interferometric Synthetic Aperture Radar (IFSAR)
- Topographic Light Detection and Ranging (LIDAR)
- Airborne LIDAR Bathymetry
- Enabling Technologies (e.g., airborne GPS and IMU technologies)
- DEM Quality Assessment
- DEM Applications
- Accuracy and Cost Comparisons
ASPRS and the Management Association of Private Photogrammetric Surveyors (MAPPS) are teaming to sponsor a DEM specialty conference in St. Petersburg, Florida, the week of October 29th through November 2nd, 2001. This conference will have the theme “Measuring the Earth – Digital Elevation Technologies and Applications.” See the ASPRS website at www.asprs.org for up-to-date details.
ASPRS also has a working group to develop guidelines for LIDAR surveys. The draft guidelines were presented at the annual ASPRS conference in St. Louis in April 2001.
For more information regarding this North Carolina floodplain mapping initiative, readers are encouraged to consult the project’s website at www.ncfloodmaps.com. This website includes detailed background information, news and status, public documents, links and resources.