We encourage our team members to stretch their minds and explore new projects whenever possible. Jonathan Dandois, a GIS software engineer with Fearless, worked with drones throughout his dissertation and has wanted to find a better way to leverage the data a drone can collect.
For example: It’s easy for a drone to fly over a field full of crops and collect photos but Jonathan said analyzing those photos for dead crops, overgrown areas, or other specific markers can be time-consuming.
“Leveraging cloud computing resources or high-end local workstations to process 10’s of GBs of data can take several hours or days,” Dandois said, adding that more time is sunk in the transit of information.
Getting images to the cloud under poor internet connection scenarios or physically getting images to a workstation is time that can be better spent. So Jonathan wanted to put a little Fearless brain power behind the problem and see what the team could come up with
He quickly got to work with senior software developer Maurice Benson on a proposal for a software solution for better leveraging drone data.
Jonathan wrote some code that would help the software parse an image for specific data. For instance, you run a picture that has a basketball through the software and the code finds the basketball. Then the user can run other photos through the software and find the basketballs in each image. This can be personalized for each search focus.
Instead of basketballs, Jonathan tested his code by telling it to find trees without leaves (i.e., trees that are in a deciduous state) from an aerial image over a tropical rainforest.
“Finding image features is the starting block of analysis in computer vision and is often the first step in image recognition, object detection, matching,” Jonathan said. “Image features are like fingerprints of a group of pixels from an image that describe different properties, including average and variation of color, texture, pattern, size and more complicated traits.
“We divide the image into tiles and compute the likelihood that a tile is like the template,” said Jonathan. “The image below represents a tiled copy of the original image where brighter tiles are more likely to be like the template (more deciduous) and darker tiles are less likely to be like the target (more green):
The idea would be that you can turn the circles or points that the software is coded to find into GPS points. Then once the features are tagged in GPS coordinates they can be aggregated and plotted on a web map for user interaction.
This drone project is at the beginning stages but there are more plans to come. Jonathan used his Fearless technology budget to buy a small drone and will be attaching a Raspberry Pi to the drone. By extending running the code on a raspberry pi the pi is a ‘system on a chip’ that can be powered by a simple 5V USB power connections and runs a flavor of Linux that can run the computer vision scripts needed for tracking features in images. The goal would be to use the Pi to identify features of interest in images in real-time while the drone is still flying, reducing the time it takes to go from images to action to near-zero.
Image credit: The drone images over tropical forest were collected by Jonathan during his post-doctoral research with the Smithsonian Tropical Research Institute in Panama (2014-2015) under the supervision of Dr. Helene Muller-Landau and Dr. Stephanie Bohlman. Thanks to Marino Ramirez and Ryan Nolin for assistance in collecting images.