QI Researchers Help Scripps Scientists Document and Model Coral Reefs for 100 Island Challenge

In a study published recently in the journal Coral Reefs, scientists from Scripps Institution of Oceanography created and analyzed detailed photomosaics of the coral reef at Palmyra Atoll in the Pacific Ocean.  Led by Scripps coral-reef ecology Ph.D. student Clinton Edwards, the team canvassed more than 17,000 square feet of reef and 44,008 coral colonies, taking more than 39,000 images.

Rather than taking the conventional route of stitching together the collected images into high-resolution 2D image mosaics, Scripps researchers used cloud computing techniques developed by UC San Diego engineers and computer scientists to enable the creation of detailed models in 3D and even in 4D (i.e., adding the dimension of time).  The models can be navigated and studied fully interactively, thanks to a team of technologists from UC San Diego’s Cultural Heritage Engineering Initiative (CHEI), led by UC San Diego structural engineering professor and computer science faculty-affiliate Falko Kuester.

Creating the mosaics and wringing useful information out of them is a time-consuming process. Gathering the data, the scientists generally did three dives a day, and it took more than five full days of diving to collect the images for the 16 plots used in the study. Back in the lab, Edwards used a custom high-performance computing system to stitch together the 2,500-3,500 individual images that make up each mosaic. It takes the software several days to complete the rendering of the composite image, and around 100 hours to label and classify all the corals in each image. Then the final step is to extract the species information and analyze it, which takes another three days or so per image.

Digitization of the images is clearly the limiting step, said Edwards. But that may change soon, he added: “We have excellent collaborators in the Computer Science and Engineering department at UC San Diego, and are getting close to having a computer-assisted workflow that will dramatically accelerate this process.” The computer scientists and engineers are part of CHEI, based in the Qualcomm Institute.

The data and scans of coral reefs from Palmyra Atoll represent some of the first results from the 100 Island Challenge (100IC), a project to create a global perspective on how coral reefs are changing over time. Scripps professor Stuart Sandin leads the overall project as principal investigator, with technology support from technologists in CHEI. The CHEI team is developing the novel 3D imaging and visual analytics techniques that Scripps scientists are using to create the mosaics that capture every detail of coral reef structure and ecology. So far, Scripps and international partners in 100IC have visited 70 of the 100 islands, and they plan to resurvey each site every two years. The resulting images, 3D models and analysis will also become baseline data to help local agencies study their own reefs.

In addition to Prof. Kuester, the 100 Island Challenge technology team from CHEI includes a handful of researchers affiliated with the CSE department: Ph.D. student Vid Petrovic, who is developing the software that the Scripps scientists use to create, visualize and analyze the coral reefs through 3D models and photomosaics; graduate student Dominique Meyer, who completed his B.S. in Physics at UC San Diego in 2016; software engineer and Computer Science alumnus Chris McFarland (B.S. ’12) and Computer Engineering alumnus  Eric Lo (B.S. ’14), who is a staff robotics engineer in CHEI.

The CHEI team has also deployed the technology for the Bermuda 100 Challenge , in which CHEI and Bermuda’s Custodian of Historic Wrecks are creating digital replicas of shipwrecks and coral reefs surrounding Bermuda.

CSE’s Petrovic is now working on creating custom algorithms to accelerate the work on the coral-reef effort and other underwater mapping and computer modeling of the environment.

“More and more imagery is being collected across the field of marine sciences, and the pace and scale of the effort will only increase — but more data doesn’t automatically mean more, or better, science,” observed Petrovic. “It’s an honor and a joy to be working so closely with a group of marine ecologists to address this, developing collaboratively the tools and workflows that are needed to make productive use of the imagery, whether for monitoring reef health, or for advancing basic science.”

According to Petrovic’s advisor, Prof. Kuester, the team is demonstrating that extremely detailed models consisting of billions of 3D data points can now be created overnight literally, when using distributed computing strategies. For a recent study, 14,000 images were turned into a comprehensive 3D model in less than a day. A digital surrogate of sorts, the model helped researchers annotate, measure and study the target environment in ways that would be impossible to achieve in-situ. By lifting the image data from 2D to 3D models or even 4D (including time), the technology is adding diagnostic value. Traditional analysis techniques when working on photomosaics can be supported by exporting a mosaic directly from the model, using the equivalent of an ultra-resolution virtual camera with user-definable characteristics.

Petrovic says the team is making it possible for scientists to virtually explore reefs in the lab, allowing them to time-travel from year to year and track the growth and decline of individual colonies, and to study spatial and temporal relationships across the reef.

“We’re speeding up the digitization and annotation, and clearing a path to letting machine-learning techniques carry more of this burden,” Petrovic said. “This is all terribly exciting, and with much more to come. But the most rewarding aspect for me is the interdisciplinary collaboration that makes it possible in the first place, that lets us apply a decade of visualization research in support of vital ecological work.”

Beyond advancing the state of science in order to better safeguard coral reefs, the team’s research is literally giving coral reefs a voice. The 100 Island Challenge’s adoption of 3D digital tools allows the public at large to dive and experience these stunning ecosystems in virtual reality, to see what otherwise cannot be seen, and hopefully become stewards for coral reef preservation. “If a photo is worth a thousand words,” observed CHEI’s Kuester, “just imagine what a fully interactive 3D environment can accomplish for scientific storytelling.”