Efficient Options for Geospatial Data Management
To maintain high performance levels when handling large volumes of geospatial data, many service providers are turning to the cloud. Innovations in cloud technology offer new ways to manage information efficiently. A hybrid storage approach combining onsite servers with cloud services offers maximum flexibility and other valuable benefits while processing, storing, and distributing high-resolution images, dense point clouds, hyperspectral data, SAR and more.
Who Should Move to the Cloud?
Every geospatial company is faced with increasing amounts of data, so serious consideration should be given to a move to the cloud to alleviate problems of processing, archiving, and sharing raw data, products and information. The change will not always reduce costs of data management; however, there are other significant advantages that will positively impact business operations.
Data transfer rates are a critical factor in the move to the cloud due to the enormous volume of geospatial data being handled. According to David McDaniel, director, Professional Services at Eplexity LLC, a Denver-based managed services provider, “The fastest download speed I’ve found in the U.S. today is advertised at 1 Gbps, while the average is just over 102 Mbps, and at upload speeds of 33 Mbps, it would take over three days to transfer 1TB of data.” For a time-sensitive business, taking advantage of faster cloud resources is an attractive option.
Also, a typical business has spikes in demand that can overwhelm its data pipeline and computing capabilities, and reduce productivity and customer satisfaction. This may occur while uploading raw data to a processing center or delivering finished products. One of the major advantages of using the cloud is dynamic capacity — meaning additional resources can be added at any time. Rather than adding hardware and more bandwidth onsite, which may not be fully utilized all the time, a cloud service is responsive to short-term needs. A cloud services customer pays for a certain data rate, a quantity of processing minutes, and an amount of storage capacity, and changes can be made as necessary without a long-term investment.
Other benefits of the cloud include dedicated cyber security staff, redundant data as a backup and lower facility costs. A decision to move to the cloud should take into consideration the fees paid to the cloud provider offset by reduced expenses at an onsite facility, as well as the related benefits such as greater customer satisfaction, faster product turnaround and enhanced security.
Broad Range of Choices
Cloud services offer a variety of options that fit different needs. Small businesses with variable data usage may opt for the cloud to avoid high upfront equipment and facility costs and ongoing maintenance expenses. Large companies might prefer establishing a hybrid arrangement with immediate access to some data stored onsite and backup data, and less crucial data stored on the cloud. Businesses that rely on very low latency to serve customers quickly might set up a geographically distributed group of servers called a content delivery network (CDN), while others pay extra to have a direct connection to the data center to enable large-volume data transfers.
“Amazon Web Services (AWS) is the leader in the cloud space, followed by Microsoft Azure, Google Cloud and a number of smaller platforms,” says McDaniel. “Thanks to ‘co-opetition’ and efforts to establish cloud standards, data moves between the cloud platforms seamlessly, which is good for everyone. The geospatial industry is well suited for the cloud because of the importance of a reliable and fast data distribution method.”
Although physical transportation of hard drives has not been eliminated from the process, devices like the AWS Snowball have a capacity for 50 TB of encrypted data and are delivered within three to five days, which is considerably faster than high-speed Internet for large-volume data transfers. Raw data on a Snowball can be sent to the AWS data center for $200 plus shipping, and products are shipped back to the commercial vendor for the same price. Time can also be saved on the front-end by pre-processing as data are being loaded, which reduces compute time later.
At an AWS data center, cloud customers have several data access choices. Low-cost “Glacier” storage is suitable for archiving and long-term backup, and offers three levels of retrieval options ranging from minutes to hours. The Simple Storage Service (S3) is appropriate for all types of data, including websites, mobile applications and IoT devices. Capacity and accessibility are scalable to each customer’s needs.
Mapping Products on the Cloud
To maximize the value of complex geospatial data, customers need to know what is available, who owns it and how to get a copy. Fortunately, the cloud makes a wide variety of datasets and value-added products searchable and more accessible. The ability to quickly and affordably display and distribute geospatial information via the cloud opens new markets and creates business opportunities.
There are numerous websites offering mapping products online, but one-stop-shopping can save time. Harris Geospatial Solutions’ Intelliearth offers an online catalog of geospatial products, including satellite and aerial imagery and elevation data. Custom mapping services like orthorectification and mosaicking with contrast and color balancing are also available.
To browse satellite imagery from Airbus, DigitalGlobe and Planet, and other providers, customers use the self-serve portal to identify an area of interest using latitude/longitude coordinates or a bounding box and review a list of all available data that meets the specifications. Information about each image — e.g., date collected, format, metadata, etc. — is accessible via an API to each commercial vendor’s cloud storage. This interface makes it simple for a customer to view the options in one place, select an image and receive the files over the internet.
“Intelliearth resells data from more than 40 different vendors who may use Amazon Web Services, Microsoft Azure or Google Cloud, but we are platform agnostic,” says Rebecca Lasica, director, Worldwide Commercial Business at Harris Geospatial Solutions. “Once you’re in the cloud environment, accessing and transferring data between platforms is incredibly fast, and the process is transparent to the customer.”
Making use of the cloud structure allows Intelliearth to be flexible and add new vendors without capacity constraints. By writing an API that interacts with the vendor’s cloud, Intelliearth transfers information about the available data rather than making a copy of the data. It is an efficient and secure way to manage geospatial products.
The cloud also makes it possible to expand the use of analytical tools. “In addition to reselling data, Intelliearth offers analytical services to deliver useful information directly to our customers,” Lasica explains. “This has expanded our market beyond traditional geospatial experts in academia who do their own analysis to include new markets such as finance, real estate, transportation and utilities.”
Cloud technology is proving beneficial to the geospatial industry by expanding access to new customers and improving the level of service provided. Transferring data and products is faster, and the costs of storage are going down while the volume is going up. The pay-for-what-you-use business model is attractive to businesses of all sizes.