Will Fisheye Lenses Work for 3D Mapping?
Fisheye lenses produce a panoramic image, useful for viewing wide areas in one picture. Due to the non-uniform resolution around the edges, fisheye imagery is not typically used for traditional mapping applications. However, as the quest for small light-weight payloads for unmanned aerial vehicles (UAV) continues, even cameras equipped with fisheye lenses are being considered.
As part of its mission to explore innovative technology related to UAV, Pix4D, a Swiss company focused on UAV software solutions, investigated the possibility that
|The DJI Phantom 2 Vision+ UAV carries a camera equipped with a fisheye lens.|
fisheye lenses are useful for more than taking wide pictures with fuzzy edges. Christoph Strecha, co-founder of Pix4D, presented the results of his research at the 2015 International LiDAR Mapping Forum (ILMF) in Denver. By performing rigorous testing and comparing the acquisition time and resulting accuracies between laser scanning, perspective imagery and fisheye imagery, Strecha was able to make a convincing argument that fisheye lenses may work quite well for some niche applications.
Research on fisheye lenses is a continuation of Pix4D’s efforts to develop UAV mapping software based on ten years of research in photogrammetry and computer vision. Pix4Dmapper, Pix4D’s software solution, is designed to automatically process terrestrial and aerial imagery, and offers viewing, editing and annotating features as well. Pix4D’s website says, “Use any camera and lens, from any angle … for your geo-referenced orthomosaics, DSMs, DTMs, point clouds, textured 3D models and simplified CAD models.”
Pix4D’s goal was to produce an accurate 3D model of a complex structure using fisheye imagery, and compare the results to a 3D model produced using more traditional methods, i.e., a laser scanner. Chillon Castle, a well-known landmark located on the shore of Lake Geneva, Switzerland, was selected as the subject. The interior and exterior of the castle was imaged using a variety of cameras—SONY Alpha 7R (fisheye), SONY NEX-7 (perspective), Canon 6D (fisheye), Canon 7D (perspective), GoPro Hero3 (fisheye handheld), DJI Phantom 2 Vision (fisheye UAV)—resulting in over 6,200 images and 95 million points. The accuracy of the model varies between 5mm and 20cm. A laser scan was completed of the west side of the castle with a Faro Focus 3D X130 scanner for comparison purposes. Without interruptions the scan would normally have taken about 2-3 hours to complete.
Highlights of the results show that there were differences in accuracy between the cameras and the laser scanner, but for some applications these deviations would not be a significant issue:
Phantom Vision 2 UAV, acquisition time ~15 minutes, deviation from laser -0.7 +/- 6.3 cm
GoPro Hero 3+ handheld, acquisition time ~10 minutes, deviation from laser -1.3 +/- 7.1 cm
SONY NEX-7 16mm perspective lens, on a tripod, acquisition time ~20 minutes, deviation from laser -0.2 +/- 4.5 cm
|Chillon Castle in Switzerland was selected as the subject of Pix4D’s 3D modeling research with fisheye lenses.|
Pix4D’s research was conducted with several consumer-grade cameras costing in the range of $500-$2,500 and weighing less than four pounds including the lenses. For certain applications, a fisheye lens could meet the resolution and accuracy requirements while offering some other advantages, such as faster acquisition time and fewer images to process. Particularly for projects with restricted space, such as interiors, 160-degree fisheye lenses are more practical and their accuracy at distances of 10-15 meters is comparable to a perspective lens.
Mapping professionals are understandably skeptical about using consumer-grade cameras and lenses for projects that have specific accuracy and resolution requirements, which makes Strecha’s research all the more interesting. If the software is capable of overcoming the deficiencies of the camera to produce an accurate usable model, it opens up opportunities for lowering the cost of data acquisition. And what is the relation between cost (time) of acquisition and processing and accuracy of the model? At some point the cost savings may be significant enough to make some tradeoff in accuracy worthwhile. If it is accurate enough for the application and costs half as much, customers will probably reconsider their choices.