As unmanned aircraft use expands for a variety of business purposes, including surveying and geospatial projects, drone manufacturers want to add more capability to the drones, principally in the form of additional cameras and sensors that can gather even more data during flight.

The catches are storage, bandwidth and data transmission latency. If you need to harness a growing tide of data in real time or near-real time, how can you obtain enough bandwidth to transmit such robust data streams? And if you need to transmit and store large files of data on the drone itself, where can these payloads be accommodated?

PrecisionHawk is a drone solutions provider whose drones most often collect LiDAR, visual and multispectral data. “We fly for many of the world’s top agriculture and seed firms that are continually developing new phenotypes and traits,” says Philip Ferguson, vice president of product for PrecisionHawk.

A phenotype is a set of observable characteristics of an individual element that results from the interaction of the genetic constitution of an element with the environment it is in. Phenotype research is undertaken by agricultural firms to understand which types of seeds and/or seed combinations grow best in certain environments.

“These companies use our multispectral sensors to evaluate phenotype performance at various growth stages throughout the year,” Ferguson says. “They focus on particular events such as frosts or specific flowering stages. The multispectral surveys they produce, along with the added value from our algorithm market place, give them the information they need to hone their phenotypes for production.”

At the same time, however, multispectral surveys use images and sensors and can rapidly become highly complex amalgamations of data that are aggregated from a variety of different sources. This begs the questions of how you can stay on top of data collection, processing and transmission of such large and variegated data payloads.

“Managing data is still a challenge for the industry” Ferguson says. “With every new band comes more and more data to handle and process. Our clients’ insatiable appetite for data is growing so quickly, we have to create innovative solutions to support the bandwidth.”

Ferguson is not alone in his concerns.

As early as 2013, U.S. Air Force Lt. General Larry D. James spoke about the challenges of data management on drones. James also acknowledged that managing the data that drones collect was a major challenge. He discussed upgrading network bandwidth, data storage and software to securely handle all of the data. “The future is going to be taking all sources of information and developing knowledge and intelligence from that,” James told Wired at the time. “The software tools will lead the way ... and it’s not just the military that’s worried about how you handle this big data. There’s lots of corporate and commercial interests out there in terms of video and imagery and what do I do with it and how can I track things and see them.”

Firms like PrecisionHawk are tackling the data management problem with internally developed software InFlight, which it uses for its Lancaster fixed wing drone. The company has also partnered with drone manufacturer DJI in the agricultural market. One of the solutions engaged in the effort is DataMapper, a cloud-based platform for storing and analyzing remote sensing data captured by drones, planes and satellites that was developed by PrecisionHawk. By offloading storage and analytics of the data collected by drones to the cloud, data storage and processing can be undertaken in cloud-resident facilities that can accommodate them.

However, there is still the issue of getting the raw data collected by the drone to the cloud.

“We have come a long way with data compression, which we offer to clients through our DataMapper software tool, but we need more intelligent ways to handle surveys that are 20  to 75 GB large,” Ferguson says.

Others also understand the hazards of bandwidth limitations and latency problems.

Of particular note are steps that major cloud providers like Microsoft (Azure) and Amazon (AWS) are taking to offer companies direct, dedicated and redundant (for failover) data communications lines with high bandwidth and reduced latency. Compressing data and also eliminating extraneous data that you don't need can also help with problem.

Finally, for companies that are managing their own drone fleets, steps can be taken to optimize company networks for quality of service (QoS), planning in advance for bandwidth by calculating the number of drones that will synchronously be flying and transmitting data over the networks. Companies can also   invest in their own infrastructures, using data speed-up protocols like multiprotocol label switching (MPLS), which facilitates high-performance communications by routing data over shorter transport paths than those used by traditional networks.

There are also companies that are optimizing their Internet-facing wide area networks (WANs) with WAN optimization tools that help make the most of available Internet bandwidth by  eradicating data transmissions that are redundant, compressing and prioritizing data, staging data in local storage caches and streamlining chatty protocols.