- SPECIAL REPORTS
- THE MAGAZINE
The idea started simply enough. In 2008, Fay Rubin, director of the New Hampshire statewide GIS known as the GRANIT System, began exploring the possibility of collecting LiDAR data on the Piscataqua River and several other watersheds that cross the boundary between Maine and New Hampshire. Collaboration with other agencies is a key tenet of the NH GRANIT program, so Rubin reached out to Michael Smith, director of the Maine Office of GIS, to find out whether Maine might be interested in participating in a joint effort. The idea began to take root.
Then, in February 2009, the U.S. Congress passed the American Recovery and Reinvestment Act (ARRA) of 2009, making $275 billion available for federal contracts, grants and loans. The move eventually injected $17 million into the USGS National Geospatial Program (NGP), $14 million of which was allocated for LiDAR and elevation data. As the potential impact of this funding was absorbed by various state agencies, the idea that originated in New Hampshire and Maine began to grow. “I was talking with Dan Walters, the USGS geospatial liaison for Maine, and we just started brainstorming,” Smith says. “We knew the stimulus funding was coming down the pipe, and we knew there were states with more clout than Maine and New Hampshire. I thought if we teamed up with the other New England states, then we would be able to bring something to the table that none of the other states would have, which was regional cooperation and coordination.”
The GIS coordinators in the small states of New Hampshire and Rhode Island immediately saw the benefits of joining with the other states to request ARRA funding for LiDAR data. Connecticut and Vermont also indicated that they were onboard. Massachusetts, however, which has the highest population of the New England states, was understandably a bit skeptical. What was the likelihood that USGS would award the requested funding to the six-state collaborative? And even if the ARRA funding was awarded, a substantial amount of matching funds would have to come from the states. How would each state ensure a return equal to the amount of its investment? Would the states be better off pursuing individual grants?
After challenging Smith and the other state GIS coordinators with these and other critical questions, Massachusetts agreed to join the collaborative effort. But then the group hit another roadblock: The USGS announced it would prioritize projects that focused on the coastline. This meant Vermont couldn’t participate. The team quickly formulated an alternate plan. “We realized it would be politically expedient to include New York since the state has so many people in the coastal area,” Smith says, “and there was interest from New York in being part of the project.”
Within the USGS, the proposal generated a mix of excitement and trepidation. “Their goal of getting continuous LiDAR coverage from Maine to New York fit beautifully with our objective to obtain data along the coastal areas, and we believed the collaboration would lead to cost-effectiveness and consistent datasets across a large area,” says Teresa Dean, a geographer and ARRA coordinator for the USGS National Geospatial Program. “But working with a large group like this can be very challenging in terms of getting the group to agree on a data specification, and then working the financial end. We have had previous projects where the more partners that are involved, the harder it is because of all the complexities of it.”
Despite these concerns, USGS decided to move forward. “This project has so much value that we wanted to do it,” Dean says.
In January 2010, USGS awarded the requested $1.4 million in ARRA funding to the six-state collaborative to pursue the LiDAR for the North East project. It was the highest amount awarded to any individual project. Smith and the other state GIS coordinators were elated; however, the real work had only just begun.
In Massachusetts, for example, all of the state-contributed funding had to be committed by July 1, 2010. The clock was ticking.
As Smith and the other state GIS coordinators began studying the USGS Geospatial Products and Services Contract (GPSC, referred to as “gypsy”) under which the task order would be issued, they soon realized they had another problem: Their original cost estimates had been too low. Instead of the anticipated $200 per square mile, the project was going to cost nearly $280 per square mile. The agreement that the six-states had so carefully crafted would have to be reshaped. Further complicating the situation was that each state was contributing a different amount of of matching funds, and some states wanted the data collected at higher point densities (1-meter nominal post spacing vs. 2-meter spacing), increased vertical accuracies (9.25 cm vertical root mean square error [RMSEz] vs. 15 cm RMSEz), and tide coordinated LiDAR acquisition (predicted mean low tide +/- 90 minutes) compared to that of the USGS base LiDAR specification (version 12) being used for the rest of the project.
The final project footprint covered 8,000 square miles along the Northeast coastline, with Massachusetts and Rhode Island putting in more funding to obtain a higher point density and improved surface definition compared to the rest of the states.
In June 2010, just prior to the Massachusetts funding deadline, the USGS issued the first task order to Lexington, KY-based Photo Science, a GPSC contractor, to begin work on the ARRA-funded portion of the project. A second task order for all other funding portions was issued to Photo Science in September 2010. “We were running right up to the wire getting all the details squared away,” Smith says.
Once the contracts were all in place, the next step was planning the LiDAR data collection effort. The data collection flights would need to take place in leaf-off conditions without any snow or flooding. Additionally, the region would need to be free of clouds and fog between the aircraft sensors and the ground. Data voids were unacceptable unless caused by water absorption or low infrared reflectivity due to asphalt, composition roofing, or filling in another flight line swath. With these parameters in mind, the Photo Science team identified the fall of 2010 and winter of 2011 as the optimum collection window.
A storm was brewing in the state GIS offices, as well. Although everyone understood the unpredictability of the weather, promises had been made, and money had already been spent. “We had proposed and expected to collect at least 80 percent of the data in the autumn window of 2010,” Smith says. “In reality, only one area of Maine was collected, and nothing basically south of Bar Harbor was done in the fall except maybe some flights along the Hudson River in New York. I don’t think any part of New Hampshire, Massachusetts or Rhode Island was collected during that time. That created quite a bit of frustration among those partners. They were concerned that they had sold a bill of goods to their respective partners and weren’t going to be able to deliver on it.”
Ultimately, these efforts paid off. “It required a lot of understanding on everybody’s part--each state had to get through whatever issues they were looking at and come up with some alternative plans and expectations,” Smith says. “In the end we were able to diplomatically make it all work.”
Despite unfavorable weather patterns that persisted throughout the spring, Photo Science was able to resume data collection efforts in mid-March and finish by May 30, just two months behind the anticipated schedule. The firm allocated additional resources to the project to speed its completion; at one point, six aircraft were assigned to gather LiDAR data. This resulted in a flood of data that had to be analyzed and processed on the back end. “This was a very large project, but more than the physical size, it was a very challenging project in terms of its geographic and topographic diversity,” Shillenn says. “It went from Long Island, a major urban center in terms of ground features with one of the most tightly controlled airspaces in the country, to downeast Maine, which has mountains and a tremendous amount of relief, and just about everything in between. Those factors posed a bigger challenge to getting it all collected and processed than just the square mileage itself. And the weather only added to that.”
Drawing from experience gained through other challenging projects, such as the Humboldt Bay Ecosystem Mapping Project in California, the team was able to keep everything in perspective. “Our past experiences really helped us internally calibrate our own expectations as well as articulate and communicate with our clients what they should expect and give them a level of confidence that we had been down this path before,” Shillenn says. “Although this was probably the severest broad-area challenge we’d ever had in terms of the weather and snow conditions, I always believed that we would prevail. We had the right approach, the right attitude and the right resources.”
The team also developed a new classification--bare water (Class 19)--to address challenges in processing the coastal water data. Temporal and tide differences in the missions resulted in large differences in coastal water levels. Using routine hydro flattening in an effort to assign uniform elevation would have resulted in a loss of near-shore and island bare-earth data. “By using a bare water classification, the team is able to retain the actual water elevation surface while eliminating noise and outliers,” Shillenn explains. “The classification also enhances islands smaller than one acre, wave action and near-shore shallow water definition.”
The final deliverables will include all return data in the raw point cloud (delivered by flight swath in LAS v1.2), a fully classified point cloud, a georeferenced bare-earth digital elevation model (DEM) raster delivered in an ERDAS IMG format, and shapefiles of hydro breaklines, as well as project reports and metadata. Substantial portions of the data have already been delivered, and the remainder is expected to be delivered by the end of the year.
The resulting dataset will be the largest continuous piece of LiDAR data from a single project and will provide substantial taxpayer savings as the value of the data is extracted over the long term. “The beauty of the project is that we’ll have continuous coverage from Maine to New York,” Dean says.
Shillenn agrees. “Within the last year, there has been an explosion in the popularity, versatility and utility of LiDAR data to support a whole host of applications,” he says “The software manufacturers are working hard to catch up, and they’re building the tools to directly ingest, analyze and edit LiDAR data in its native format, the robust LAS format, which carries rich information about each point in the point cloud. Even Esri is involved--the latest version of ArcGIS has a very robust set of LiDAR tools that are going to really mainstream the use of LiDAR data.”
As the demand for that data increases, other states and other consumers will undoubtedly look to the trailblazers for ideas on collecting and applying that data as well as collaborating to fund LiDAR projects. For his part, Smith is excited to serve in that role. “I’m most proud of the fact that we got six states to work together,” he says. “It could have been a feeding frenzy where we were all competing against each other for the available USGS funding. Instead, the New England states banded together and were able as a unit to come up with a much better proposal and a much better model for funding geospatial assets. I think that was the most beautiful part of the whole project--even better than getting data for the entire Maine coast.”
Aircraft/Sensor Configuration• Cessna U-206s
• Four Leica ALS 50 II & ALS 60 with a max pulse rate of 200 KHz, multi-pulse
• Three Optech Gemini ALTM with a max pulse rate of 167 KHz, multi-pulse
• Applanix POS/AV DG position and orientation systems
• Trimble 7500 dual frequency GPS receivers (base stationing)
• Calibration – home base and local
• Horizontal: UTM Zones 18 and 19, NAD83, meters
• Vertical: NAVD88, meters (most recent Geoid model approved by the NGS)
Managing LiDAR Data
Shillenn says Photo Science is increasingly involved in data education and exploitation. “We are data developers--that’s our legacy and heritage. But clearly, it’s leveraging the intelligence and value, and extracting the derivative information from really rich datasets like this one with high-resolution imagery that allows us to support clients at all levels of government and do things like change detection, impervious surface analysis, land cover mapping, support environmental/green infrastructure development. A lot of good work can be done by going beyond simply developing data to helping our clients leverage those great datasets to solve problems. That’s really where the opportunity is and where there’s a lot of excitement at Photo Science.”
LiDAR post processing
Automated and manual filtering of the bare earth surface model during the post processing effort results in the classification of 90 percent of artifacts, 95 percent of outliers, 95 percent of vegetation and 98 percent of buildings. Photo Science uses Applied Imagery’s Quick Terrain Modeler (QT Modeler) to check the bare earth dataset at the tile level. GeoCue’s LP360EQC and Esri’s ArcGIS software are used to assess the quality of multiple tiles at the macro level. Two hundred fifteen supplemental GPS ground control points obtained by James W. Sewall & Co. survey crews are also being used to check and validate ground classification during the post-processing phase. (An additional 120 GPS quality assurance check points established in open terrain are being used by USGS quality assurance staff to validate fundamental vertical accuracies but are not being used by Photo Science in its workflow.)