February 15, 2018
A collaborative group of researchers from the University of California San Diego traveled to Turin, Italy recently to digitally map an entire portion of the city—complete with historic architecture, expansive murals and stunning works of art.
Digital data will be used by students and researchers on campus to explore the site’s buildings and artifacts, ultimately recreating an interactive, virtual-reality experience. Through high-resolution images and 3D models, students can study all the pieces together without the difficulty of travel to the site, nor fear of losing objects to passing time.
“The idea is to create a model— a digital surrogate—of these structures that allows us to interact with them, to analyze them, to annotate them and to make measurements to
really understand their state of health,” said Dominique Rissolo, an archaeologist and assistant research scientist at the UC San Diego Qualcomm Institute. “We have a unique capability here on campus … that allows us to go one step beyond the model to actually create the digital surrogate.”
The work is being conducted as a project of the Cultural Heritage Engineering Initiative at the Qualcomm Institute, organized by researchers at the institute, Division of Arts and Humanitiesand Turin’s Polytechnic University.
Trip findings are an extension of Division of Arts and Humanities Dean Cristina Della Coletta’s research on the historical significance of the 1911 Turin International, the world’s fair that took place in the city’s Parco del Valentino. With a goal to visually recreate the 1911 fair, the team digitally captured a majority of the park, which includes a medieval village and the Castello del Valentino, or “Valentino Castle”—home to Polytechnic University’s architecture department.
“For the first time, we really have a wide array of expertise that is brought to bear on the project: we have engineers, working with architects, working with cultural historians,” Della Coletta said. “This is a winning combination.”
Led by Della Coletta, Rissolo, Falko Kuester and Vid Petrovic, the Turin fieldwork brings together engineering students and arts and humanities students for the betterment of both. Cross training these students, Kuester said, creates empowered and dynamic learning: just as the cultural preservation work is important, honing and testing new technical skills in the field and classroom is equally as important.
“The exciting part for us, as educators, is that we get to work with talented students from a broad range of disciplines—disciplines that historically do not really collaborate together as much,” said Kuester, the Kinsella Heritage Engineering Director at Qualcomm Institute and professor at the UC San Diego Jacobs School of Engineering.
“By doing this, and being able to put on these different lenses, our students get to learn to speak each other’s language [and] to communicate in ways we were never required to do before,” he said. “Our students, in the process, are becoming more complete human beings and innovators.”
Considered the initial steps in recreating the structures of the 1911 Turin World’s Fair, Rissolo said the team collected field data by using terrestrial laser scanners, structure-from-motion photogrammetry and stereo spherical giga-pixel imaging. Taking several scans from many different perspectives in the park allows the team to process the data quickly and with a high degree of certainty. They collected thousands of images, and use advanced software on campus to create digital models of the structures.
“The most important takeaway for our students is the ability to connect theory and practice, whether it is in the archives or in the digital lab,” said Della Coletta, who participated in the data gathering during fall quarter. “What makes the project meaningful is the ability of our students to learn by doing.”
With a first round of digital information collected, the researchers have been busy on campus recreating a 3D model of the buildings, both inside and out. Viewing the models at the university’s WAVE lab, Qualcomm Institute research and development engineer Eric Lo said the results were a good representation of what he saw on the ground in Italy, but some data was missing. Once the 3D models are created, gaps appear, giving the team an overview of what to record on future trips.
“Ultimately, it’s a map to the site, but a map that also tells us what else to map in order to first create the best possible digital surrogate, combining site geometry, building materials and overall state of health, as well as its art and history,” Kuester said.
The Turin team also included Department of Visual Arts alumnus Farshid Bazmandegan and students at Italy’s Polytechnic University, headed by geomatics professor Filiberto Chiabrando. The Polytechnic team continues to digitally document historic structures in the park.
“The approach that we will follow in this project is very interesting, since it’s connected to the cultural heritage, [and] it’s connected to the humanities as well. This is a fruitful collaboration that we have started,” Chiabrando said. “The most important thing is to merge different backgrounds, and different experiences. A multidisciplinary approach is the best way.”
The Cultural Heritage Engineering Initiative at UC San Diego brings the power of student-driven engineering to the study and preservation of historic structures, archaeological sites, art and other artifacts. There are multiple projects underway, from visualizing shipwrecks near Bermuda to the Hearing Seascapes initiative with Department of Music professor Lei Liang.
“It’s a phenomenal opportunity to have the Division of Arts and Humanities partner with the Cultural Heritage Engineering Initiative, the Jacobs School of Engineering and, on top of that, the Polytechnic University in Turin,” said Della Coletta. “What better place to engage in a project that connects engineering and technology with the humanities.”
San Diego, February 9, 2018 – PBS Nova has featured work by a team of researchers at the UC San Diego Qualcomm Institute in a recent article in its NOVA NEXT online publication.
In the article, titled “How VR Helped Archaeologists Excavate a Fossil-Rich Submerged Cave,” QI Research Scientist Dominique Rissolo describes his team’s work at the QI Cultural Heritage Engineering Initiative (CHEI) to visualize – in high-resolution 3D – ancient human and animal remains found in Hoyo Negro, an underwater cave on Mexico’s Yucatan peninsula.
The remains were discovered by lead diver Alberto Nava and his colleagues on the floor of a flooded pit 130 feet from the surface. Among them were parts of more than 30 animal skeletons, including the nearly intact skull and skeleton of a teenage girl of around 16 who had fallen to her death in the pit at the end of the last Ice Age some 13,000 years ago. Also found were fossils of Ice Age megafauna such as saber-tooth cats and huge Shasta ground sloths.
The researchers used the state-of-the art SunCAVE (Cave Automatic Virtual Environment) at QI, which allowed scientists associated with a NOVA documentary to interact with, map and measure the fossils, as well as plan future diving missions. QI and CHEI are an integral part of the Hoyo Negro Project – working with the technical dive time to develop optimal image acquisition strategies, creating the high-resolution digital models, and powering the visual analytics necessary to bring this remote site to the scientific community.
The QI effort is being led by Rissolo, an archaeologist who has been working in the Yucatan for 25 years, as well as cultural heritage engineering specialists Falko Kuester, Vid Petrovic and Eric Lo. Many of the researchers studying the site’s diverse Ice Age fauna will never have a chance to go there. Not only has the virtual “twin” of the site enabled paleontologists to study the bones remotely, but they are making discoveries in the data – bones and tell-tale features that have eluded detection by divers at the bottom of the deep dark pit.
Exploring an environment that only few have seen before, yet to be the first to see it as a whole and in its full beauty, combining site-scale context with the finest possible details captured by its digital twin for in-depth analysis, is truly transformative, says Falko Kuester, Professor for Visualization and Virtual Reality at CHEI.
“Not only is the virtual cave essential for a comprehensive fossil inventory,” writes NOVA Next writer Evan Hadingham, “it enables the team to take measurements and print accurate 3D replicas of specific bones, including Naia’s skull.” Explorers on the most recent National Geographic-funded mission to Hoyo Negro used the virtual SunCAVE to plan their excursion in detail, which ultimately allowed them to bring up parts of eleven ancient animals, some of them previously unknown.
To watch a documentary film about Hoyo Negro, visit http://www.pbs.org/wgbh/nova/evolution/first-face-america.html
And to see QI’s Hoyo Negro website, visit http://hoyonegro.ucsd.edu/index.php
San Diego, February 6, 2018 – “Lost Treasures of the Maya Snake Kings,” a new one-hour National Geographic special premiering today at 9/8 p.m. central, shows how LiDAR laser imaging technology is revolutionizing archaeology and features the WAVE data visualization technology created by researchers at the University of California San Diego Qualcomm Institute (QI). Albert Yu-Min Lin, an affiliate of QI, is the host of the program.
The documentary explores what’s being hailed as a “major breakthrough” in Maya archaeology: the identification of ruins of more than 60,000 houses, palaces, elevated highways, and other human-made features that have been hidden for centuries under the jungles of northern Guatemala. The work was conducted by researchers of the PACUNAM LiDAR initiative.
Using a powerful technology known as LiDAR (short for “Light Detection And Ranging”), scholars digitally removed the tree canopy from aerial images of the now-unpopulated landscape, revealing the ruins of a sprawling pre-Columbian civilization that was far more complex and interconnected than most Maya specialists had supposed.
“The LiDAR images make it clear that this entire region was a settlement system whose scale and population density had been grossly underestimated,” said Thomas Garrison, an Ithaca College archaeologist and National Geographic Explorer who specializes in using digital technology for archaeological research.
Garrison is part of a consortium of researchers who are participating in the project, which was spearheaded by the PACUNAM Foundation, a Guatemalan nonprofit that fosters scientific research, sustainable development, and cultural heritage preservation.
Working closely with National Geographic Explorers, QI’s Cultural Heritage Engineering Initiative (CHEI) was launched in 2007 and has since created a comprehensive toolbox and talent pool that brings the power of student-driven science and engineering to the study and preservation of archaeological sites, monuments, historic structures and other artifacts. QI researchers have a history of working on various ground based and drone based LiDAR imaging projects in Guatemala. Lin, along with his collaborators in the QI Engineers for Exploration program —co-directors Ryan Kastner and Curt Schurgers — collaborated with Garrison to lead teams of students on expeditions over the past four years to the jungles of Guatemala to test out various platforms for mapping and imaging. QI Staff Engineer Eric Lo and Ph.D. student Dominique Meyer were also instrumental in these field expeditions.
“Engineering and exploration go hand in hand — National Geographic was co-founded by the Alexander Graham Bell who invented the telephone,” said Lin. “The things we create allow us to go further, and with the exponential rate of innovation today this truly is the new golden age of exploration.”
The Qualcomm Institute is a leader in visualization technologies that make it possible to look at data at a massive scale. Researchers at CHEI — including Falko Kuester, Vid Petrovic, Eric Lo, Christopher McFarland, Jurgen Schulze, Greg Dawe, Joel Polizzi, Joe Keefe and Tom DeFanti — played a primary role in developing CHEI’s hardware and software toolbox, including the 70-megapixel Wide Angle Virtual Environment (WAVE), which is featured heavily in the documentary. The team at CHEI also developed the VisCore visual analytics engine that allows archaeologists to use virtual reality to literally walk into the arena of data-enabled scientific discovery, as featured in the “Lost Treasures of the Maya Snake Kings.”
“Turning big-data into insights and action is one of the truly transformative elements that our team enables,” says CHEI Director Kuester. “Lots of data is being acquired and simulated these days, but making sense of it all is a completely different story. The opportunity to work in highly interdisciplinary teams that change to state of knowledge is where it gets truly exciting.”
An advanced civilization
The PACUNAM project mapped more than 800 square miles (2,100 square kilometers) of the Maya Biosphere Reserve in the Petén region of Guatemala, producing the largest LiDAR data set ever obtained for archaeological research.
The results suggest that Central America supported an advanced civilization that was, at its peak some 1,200 years ago, more comparable to sophisticated cultures such as ancient Greece or China than to the scattered and sparsely populated city states that ground-based research had long suggested.
In addition to hundreds of previously unknown structures, the LiDAR images show raised highways connecting urban centers and quarries. Complex irrigation and terracing systems supported intensive agriculture capable of feeding masses of workers who dramatically reshaped the landscape.
The ancient Maya never used the wheel or beasts of burden, yet “this was a civilization that was literally moving mountains,” said Marcello Canuto, a Tulane University archaeologist and National Geographic Explorer who participated in the project.
“We’ve had this western conceit that complex civilizations can’t flourish in the tropics, that the tropics are where civilizations go to die,” said Canuto, who conducts archaeological research at a Guatemalan site known as La Corona. “But with the new LiDAR-based evidence from Central America and [Cambodia’s] Angkor Wat, we now have to consider that complex societies may have formed in the tropics and made their way outward from there.”
“LiDAR is revolutionizing archaeology the way the Hubble Space Telescope revolutionized astronomy,” said Francisco Estrada-Belli, a Tulane University archaeologist and National Geographic Explorer. “We’ll need 100 years to go through all [the data] and really understand what we’re seeing.”
Already, though, the survey has yielded surprising insights into settlement patterns, inter-urban connectivity, and militarization in the Maya Lowlands. At its peak in the Maya classic period (approximately A.D. 250–900), the civilization covered an area about twice the size of medieval England, but it was far more densely populated.
“Most people had been comfortable with population estimates of around 5 million,” said Estrada-Belli, who directs a multi-disciplinary archaeological project at Holmul, Guatemala. “With this new data it’s no longer unreasonable to think that there were 10 to 15 million people there—including many living in low-lying, swampy areas that many of us had thought uninhabitable.”
Virtually all the Maya cities were connected by causeways wide enough to suggest that they were heavily trafficked and used for trade and other forms of regional interaction. These highways were elevated to allow easy passage even during rainy seasons. In a part of the world where there is usually too much or too little precipitation, the flow of water was meticulously planned and controlled via canals, dikes, and reservoirs.
Among the most surprising findings was the ubiquity of defensive walls, ramparts, terraces, and fortresses. “Warfare wasn’t only happening toward the end of the civilization,” said Garrison. “It was large-scale and systematic, and it endured over many years.”
The survey also revealed thousands of pits dug by modern-day looters. “Many of these new sites are only new to us; they are not new to looters,” said Marianne Hernandez, president of the PACUNAM Foundation. (Read “Losing Maya Heritage to Looters.”)
Environmental degradation is another concern. Guatemala is losing more than 10 percent of its forests annually, and habitat loss has accelerated along its border with Mexico as trespassers burn and clear land for agriculture and human settlement.
“By identifying these sites and helping to understand who these ancient people were, we hope to raise awareness of the value of protecting these places,” Hernandez said.
The survey is the first phase of the PACUNAM LiDAR Initiative, a three-year project that will eventually map more than 5,000 square miles (14,000 square kilometers) of Guatemala’s lowlands, part of a pre-Columbian settlement system that extended north to the Gulf of Mexico.
“The ambition and the impact of this project is just incredible,” said Kathryn Reese-Taylor, a University of Calgary archaeologist and Maya specialist who was not associated with the PACUNAM survey. “After decades of combing through the forests, no archaeologists had stumbled across these sites. More importantly, we never had the big picture that this data set gives us. It really pulls back the veil and helps us see the civilization as the ancient Maya saw it.
San Diego, February 2, 2018,
Performances of Erasure and Hearing Seascapes set for 5-7pm on Feb. 8 at UC San Diego’s Qualcomm Institute
In spring 2017, UC San Diego music professor and former Qualcomm Institute (QI) composer in residence Lei Liang, and Falko Kuester, the institute’s professor of visualization and virtual reality, organized a unique seminar course on “Hearing Seascapes: A Collaborative Seminar on the Sonification of Coral Reefs.” It provided graduate students, primarily from the Music department, with an opportunity to develop multimedia projects to highlight the dangers facing coral reefs in many parts of the world.
Out of that seminar course emerged two performance-and-installation works accepted into the Qualcomm Institute’s Initiative for Digital Exploration of Arts & Sciences (IDEAS) 2017-2018 season. The two works will premiere simultaneously in QI’s Atkinson Hall on the UC San Diego campus. The immersive works include:
Hearing Seascapes will be staged in the SunCAVE virtual-reality (VR) room in the Immersive Visualization Lab, and Erasure in the Reconfigurable Media Lab, both on the first floor of QI’s Atkinson Hall. The works will premiere Thursday, February 8, 2018 from 5-7 p.m. in Atkinson Hall, followed by a public reception. The schedule:
“Over the last several decades, coral-reef ecosystems have suffered significant impacts from both local and global factors,” said Koh. “We aim to convey important messaging to the audience and illuminate data that show the declining health of ocean coral reefs.”
Through an interconnected network of three-dimensional (3D) photomosaic models of coral reefs and spatially as well as electronically processed percussion sounds, a metaphorical ecosystem forms and responds directly to human presence and the temporal history of that presence throughout the work’s existence.
Erasure responds negatively to human presence in the installation environment. As more and more people enter the space, the installation begins to break down: the sonic tapestry of percussion sounds contort and particulate, the synthetic biome of coral visualizations begin to morph into unnatural forms, and the entire system mutates – all with the presence of the audience – and members of that audience are aesthetically confronted with their impact on these remote and fragile ecosystems. The transformation, however, is neither irreversible nor unidirectional. “As viewers leave, the piece rebounds, but more slowly than the rate at which it broke down,” explained Ph.D. student Sandstrom. “The system bounces back from the immediate and long-term human impact, and it reflects the resilience of the reefs to withstand and adapt to global shifts in climate and the ecosystem.”
The visual component of Erasure consists of 3D photomosaic models of coral reefs taken from the 100 Island Challenge, based in the Scripps Institution of Oceanography. These digital reproductions were created by the Challenge’s technical visualization advisors from the Qualcomm Institute’s Cultural Heritage Engineering Initiative (CHEI), jointly with Scripps Oceanography. By rigorously photographing and collecting data from reef sites and rendering the data into 3D computer models using custom software developed by CHEI’s Falko Kuester and Vid Petrovic, students from the Music department were able to observe the reefs from various angles, light levels, and distances.
“These coral reef constructions ebb and flow between their natural state—meticulously constructed synthetic ecosystems—and transformative states: from granulations of the stony corals and polyps into whirling cascades of particles, to fissions of vast reef colonies into splintered slabs that recede in and out of focus,” said Music’s Anthony Vine. “Lingering traces of the piece in its untouched state float among the remains, and the metaphorical ecosystem appears to be dreaming and longing to return to an undisturbed state.”
The sound-space of Erasure is created from a reservoir of percussion improvisations that both reflect the sounds that might be found in and around a coral-reef environment, as well as poetic expansions that reach beyond the palette of oceanic utterances: scraped and struck limestone tiles, sweeping washes of hands streaked across a bass drum, and the murky drones of rolled bell plates. By manipulating these samples in a simple causal network, emergent behavior materializes to constitute a lush atmosphere of sound. In this way, the behavior of the sound world is not unlike the behavior found in reef ecology: masses of small units combining to create a complex and rich environment.
“Music can bring an image to life, and by giving a voice to the coral reefs, we can help the audience make an emotional connection to ecology and realize the fragility of these reefs,” said Lauren Jones. “The goal of this project is to create an immersive experience for the viewer that allows them to submerge themselves in the world of the coral.”
Viewers use a joystick to ‘dive’ and explore the reef, controlling the location, viewpoint, depth and speed of navigation. Audience members hear different sounds that represent different species in the data set. Each species has its own specific personality (represented by different sounds). “Coral reefs are living, breathing organisms that are vulnerable to small changes of the surrounding environment and climate,” explained Stella Koh. “We assign the coral reefs a distinct personality by examining certain characteristics such as texture, habit, origin and growth. To convey messages through music, we’ve recorded an underwater dialogue of voices.” The installation was designed to induce conversation between coral reefs and a fish maneuvering through the reefs, and the sound becomes louder or softer depending on how far away it is from the coral.
Jones and Koh set for themselves three goals with Hearing Seascapes: to display experiments with different aspects of sound and innovative graphic design to create an enjoyable environment for the audiences; to tell an effective and interactive story invoking concepts of adventure, imagination and humor to motivate people to recognize environment health; and to create an inviting seascape with a synergy of voices, images, synthesized sounds and human emotion.
“Based on the notion of acoustic ecology, we want to bring out the positive aspects of sound in the ocean environment,” they noted. “We hope to highlight the importance of engaging in the soundscapes of coral reefs in hopes that our musical voice can make scientific results more accessible to society.”
San Diego, CA, January 25, 2018 — One drone landed on a moving Roomba-like robot while being guided by arm gestures. Another streamed a live, high-definition, 360-degree video feed as it flew overhead. These were some of the demos last week that helped inaugurate the first open-air aerodrome for unmanned aerial vehicles here on campus.
The aerodrome’s inauguration was the first step in what engineers hope will be a new era for drone research at the University of California San Diego. One of their goals is to create a living laboratory for unmanned aerial vehicles by bringing together researchers from across campus, including computer scientists, structural, mechanical, aerospace, electrical and computer engineers and scientists at the Scripps Institution of Oceanography.
|(L-R) Contextual Robotics Institute Director Henrik Christensen and mechanical engineering graduate students Aamodh Suresh and Aaron Ma, from the lab of Sonia Martinez and Jorge Cortez, monitor a demo at the aerodrome opening.|
“We are bringing everyone together so that one plus one equals substantially more than two,” said Henrik I. Christensen, a professor of computer science at the Jacobs School of Engineering at UC San Diego and the director of the campus’ Contextual Robotics Institute.
The institute oversees the new drone testbed, which is located next to the UC San Diego Powell Structures Lab. It is made up of a 30-foot-tall mesh cage over a 2,500-square-foot outdoor area.
The facility will primarily be open to university research. Projects that could take flight in the new aerodrome include everything from quadcopters built by undergraduate students to balloons designed to monitor conditions inside hurricanes.
|A research drone from the DroneLab takes flight in the new drone testbed, which is located next to the UC San Diego Powell Structures Lab. It is made up of a 30-foot-tall mesh cage over a 2,500-square-foot outdoor area.|
“We expect to see new research results on control of swarms of small UAVs and on the coordination between humans and robots,” Christensen added. “We will also test entirely new vehicles for long endurance flights.”
The aerodrome is opening not only in response to new University of California regulations governing the use of drones in residential areas but also in response to demand from researchers on campus. The testbed will be equipped with motion capture devices, which will allow researchers to test the quality of their flight control algorithms and will provide ground truth data for all drones.
A second phase will expand the testbed to an indoor 100-foot-tall area, connected to the outdoor testing cage. The expansion, which should take place next year, will result in a unique facility in the United States.
“The opening of the aerodrome is exciting because it enables drone research in a controlled environment, supporting experiments with a multitude of autonomous platforms and payloads as well as control, navigation, guidance and communications strategies,” said Falko Kuester, a professor in the Department of Structural Engineering who leads the DroneLab at the Qualcomm Institute at UC San Diego.
Kuester and his team are using drones to assess damage after natural disasters—both real and simulated. For example, they developed drone-based imaging techniques to map full-scale buildings before, during and after earthquakes and fires. They subsequently tested these on the world’s largest outdoor shake table here at UC San Diego. The drones acquired images and videos of a six-story building to document damage as a simulated earthquake and then a staged fire unfolded. Powerful algorithms then turned the data the drones collected into detailed three-dimensional models that researchers could literally walk through in a Holodeck-type environment to look for damage.
|Dominique Meyer, a graduate student from Falko Kuester’s lab, prepares one of the team’s drones for a demonstration at the opening.|
“The ability to develop, test and refine drone-based imaging techniques is critical for our rapid response team,” said Kuester. The team deploys around the world to assist with post-disaster reconnaissance and data collection, as well as the study and preservation of sensitive world heritage sites and ecosystems. They traveled to Puerto Rico after Hurricane Maria and to Mexico after a series of earthquakes hit the country in September.
“Being able to develop and test indoor-outdoor flight transitions, under the broad range of scenarios that can be created within the aerodrome is a game changer for us and will ultimately help with saving lives, following disasters,” Kuester said.
Another research team, led by professors Jorge Cortes and Sonia Martinez from the Department of Mechanical and Aerospace Engineering, is working to develop algorithms that allow swarms of robots and drones to work together. The swarms could be used in a wide range of situations, including search and rescue missions, environmental monitoring, and pickup and delivery in transportation. Cortes and Martinez also develop algorithms that allow elements of the swarm to sense, estimate and plan. Robots and drones can then talk to one another to determine the most efficient way to deploy.
|(L-R) Contextual Robotics Institute project manager Timothy McConnell; Executive Vice President, Technology Qualcomm Technologies Matthew Grob; Contextual Robotics Institute Director Henrik Christensen; and UC San Diego facilities senior project manager Rita Johnson pose with a drone inside the new open-air aerodrome.|
The algorithms allow one person to control the swarm, acting as somewhat of an orchestra conductor. As a first step, researchers interfaced with the swarm with an app on a tablet. They are now working on controlling the swarm via gestures processed via a motion-tracking bracelet.
The new UAV aerodrome is in part supported by funds from Northrup Grumman and Qualcomm—building on a long-standing collaboration. Qualcomm and UC San Diego created the first FAA-approved sites for drone technology development and testing in early 2016 and Northup Grumman hosted its 2016 Quad Cup Innovation Challenge on the UC San Diego campus.
The campus continues to explore other industry partnerships. The associated research is currently supported by the Army Research Laboratory, Army Corps of Engineers, National Science Foundation (NSF), National Institutes of Standards and Technology (NIST) and Defense Advanced Projects Agency (DARPA).
“There is today a shortage of engineers that have experience with drone technologies,” Christensen said. “The aerodrome will allow us to fill that gap.”
In a study published recently in the journal Coral Reefs, scientists from Scripps Institution of Oceanography created and analyzed detailed photomosaics of the coral reef at Palmyra Atoll in the Pacific Ocean. Led by Scripps coral-reef ecology Ph.D. student Clinton Edwards, the team canvassed more than 17,000 square feet of reef and 44,008 coral colonies, taking more than 39,000 images.
Rather than taking the conventional route of stitching together the collected images into high-resolution 2D image mosaics, Scripps researchers used cloud computing techniques developed by UC San Diego engineers and computer scientists to enable the creation of detailed models in 3D and even in 4D (i.e., adding the dimension of time). The models can be navigated and studied fully interactively, thanks to a team of technologists from UC San Diego’s Cultural Heritage Engineering Initiative (CHEI), led by UC San Diego structural engineering professor and computer science faculty-affiliate Falko Kuester.
Creating the mosaics and wringing useful information out of them is a time-consuming process. Gathering the data, the scientists generally did three dives a day, and it took more than five full days of diving to collect the images for the 16 plots used in the study. Back in the lab, Edwards used a custom high-performance computing system to stitch together the 2,500-3,500 individual images that make up each mosaic. It takes the software several days to complete the rendering of the composite image, and around 100 hours to label and classify all the corals in each image. Then the final step is to extract the species information and analyze it, which takes another three days or so per image.
Digitization of the images is clearly the limiting step, said Edwards. But that may change soon, he added: “We have excellent collaborators in the Computer Science and Engineering department at UC San Diego, and are getting close to having a computer-assisted workflow that will dramatically accelerate this process.” The computer scientists and engineers are part of CHEI, based in the Qualcomm Institute.
The data and scans of coral reefs from Palmyra Atoll represent some of the first results from the 100 Island Challenge (100IC), a project to create a global perspective on how coral reefs are changing over time. Scripps professor Stuart Sandin leads the overall project as principal investigator, with technology support from technologists in CHEI. The CHEI team is developing the novel 3D imaging and visual analytics techniques that Scripps scientists are using to create the mosaics that capture every detail of coral reef structure and ecology. So far, Scripps and international partners in 100IC have visited 70 of the 100 islands, and they plan to resurvey each site every two years. The resulting images, 3D models and analysis will also become baseline data to help local agencies study their own reefs.
In addition to Prof. Kuester, the 100 Island Challenge technology team from CHEI includes a handful of researchers affiliated with the CSE department: Ph.D. student Vid Petrovic, who is developing the software that the Scripps scientists use to create, visualize and analyze the coral reefs through 3D models and photomosaics; graduate student Dominique Meyer, who completed his B.S. in Physics at UC San Diego in 2016; software engineer and Computer Science alumnus Chris McFarland (B.S. ’12) and Computer Engineering alumnus Eric Lo (B.S. ’14), who is a staff robotics engineer in CHEI.
The CHEI team has also deployed the technology for the Bermuda 100 Challenge , in which CHEI and Bermuda’s Custodian of Historic Wrecks are creating digital replicas of shipwrecks and coral reefs surrounding Bermuda.
CSE’s Petrovic is now working on creating custom algorithms to accelerate the work on the coral-reef effort and other underwater mapping and computer modeling of the environment.
“More and more imagery is being collected across the field of marine sciences, and the pace and scale of the effort will only increase — but more data doesn’t automatically mean more, or better, science,” observed Petrovic. “It’s an honor and a joy to be working so closely with a group of marine ecologists to address this, developing collaboratively the tools and workflows that are needed to make productive use of the imagery, whether for monitoring reef health, or for advancing basic science.”
According to Petrovic’s advisor, Prof. Kuester, the team is demonstrating that extremely detailed models consisting of billions of 3D data points can now be created overnight literally, when using distributed computing strategies. For a recent study, 14,000 images were turned into a comprehensive 3D model in less than a day. A digital surrogate of sorts, the model helped researchers annotate, measure and study the target environment in ways that would be impossible to achieve in-situ. By lifting the image data from 2D to 3D models or even 4D (including time), the technology is adding diagnostic value. Traditional analysis techniques when working on photomosaics can be supported by exporting a mosaic directly from the model, using the equivalent of an ultra-resolution virtual camera with user-definable characteristics.
Petrovic says the team is making it possible for scientists to virtually explore reefs in the lab, allowing them to time-travel from year to year and track the growth and decline of individual colonies, and to study spatial and temporal relationships across the reef.
“We’re speeding up the digitization and annotation, and clearing a path to letting machine-learning techniques carry more of this burden,” Petrovic said. “This is all terribly exciting, and with much more to come. But the most rewarding aspect for me is the interdisciplinary collaboration that makes it possible in the first place, that lets us apply a decade of visualization research in support of vital ecological work.”
Beyond advancing the state of science in order to better safeguard coral reefs, the team’s research is literally giving coral reefs a voice. The 100 Island Challenge’s adoption of 3D digital tools allows the public at large to dive and experience these stunning ecosystems in virtual reality, to see what otherwise cannot be seen, and hopefully become stewards for coral reef preservation. “If a photo is worth a thousand words,” observed CHEI’s Kuester, “just imagine what a fully interactive 3D environment can accomplish for scientific storytelling.”
September 18, 2017. A team of archaeologists from the University of California San Diego and two leading Israeli universities has wrapped up a three-week expedition to document two major sites in Israel using the latest in 3D scientific visualization technologies.
The expedition, led by UC San Diego Department of Anthropology professor Thomas E. Levy, involved a land-and-sea approach to understanding trade and exchange during Biblical times, at sites dating to the Iron Age, circa 1200 to 586 BC. Officially sanctioned by The Explorers Club, the expedition carried Explorers Club Flag #179 to the ancient mining region of Timna in southern Israel as well as to Tel Dor on the Mediterranean coast, the site of a submerged port facility.
“Between the two sites, we captured more than 24,000 high-definition photos for use in creating 3D computer models of these two very historic areas,” said Levy, who directs UC San Diego’s Center for Cyber-Archaeology and Sustainability (CCAS), based in the Qualcomm Institute, and co-directs the Scripps Center for Marine Archaeology (SCMA). “Back in San Diego, we’ll be processing the images to build large-scale virtual models to help archaeologists and students better understand the two historically important regions in Israel and how they were united in antiquity through the copper trade.”
Levy’s co-principal investigators on the expedition included University of Haifa professor Assaf Yasur-Landau in the Department of Maritime Civilizations, and UC San Diego alumnus Erez Ben-Yosef (Ph.D. ’10), who did his doctoral dissertation with Levy and is now a professor of archaeology at Tel Aviv University. Ben-Yosef organized the logistical infrastructure of the Timna imaging projects, while Yasur-Landau organized the 3D Tel Dor imaging project through his Laboratory for Coastal Archaeology and Underwater Survey, with support from former Scripps Institution of Oceanography postdoc Tali Treibitz from the University of Haifa’s Marine Imaging Laboratory. Treibitz facilitated the underwater data collection in the waters of Dor. Also helping to make the expedition a success were CHEI director Falko Kuester for student support and the allocation of processing resources and equipment to the project, as well as researcher Eric Lo for image processing.
The UC San Diego team included Christian McDonald, scientific diving safety officer at Scripps Institution of Oceanography, Department of Anthropology graduate student Anthony Tamberino, as well as Dominique Meyer, a Ph.D. student in the Department of Computer Science and Engineering and researcher in the Cultural Heritage Engineering Initiative (CHEI). Meyer and Tamberino were responsible for the provision of equipment and data collection of aerial and ground-based 3D images. Meyer also served as the primary drone operator and flight planner for the expedition, for which he operated Phantom 4 drones with standard 20-mm cameras as well as directing a heavy-lift drone octocopter for video and structure-from-motion (SfM) photography with 36-megapixel cameras operated by the Griffin company. The logistics of the project were greatly supported by the Israel Antiquities Authority whose own drone operator, Yitzhak Marmelstein, worked with the team for the duration of the project.
“Previously, copper ingots found in the waters north of Tel Dor were traced via lead isotope analysis to the Faynan region in Jordan, where extensive Iron Age mines and smelting factories have been found and excavated by our UC San Diego team,” said Levy. “It seemed most likely that Tel Dor was the main Iron Age port along the Israeli coast for the trade of copper, from the desert zones of Faynan in the northern Aravah valley and Timna at its southern end.”
The aerial drones carried out regional 3D scanning of more than 15 major ancient sites related to copper mining and smelting in the Timna mining region, including at Shiqmim in the northern Negev desert. In Shiqmim, Levy had previously excavated Copper Age ruins, including an ancient 24-acre village dating to the Chalcolithic period (ca. 4500 – 3600 BC), when copper was first smelted and cast in the southern Levant.
To capture the submerged port facility at Tel Dor, the team used scuba and high-precision underwater cameras to image remains of the ca. 9th century BC port.
The Israel 3D Land and Sea Project 2017 was generously funded by Marian Scheuer Sofaer and Abraham Sofaer of Palo Alto, CA..
In addition to the 3D imaging project at Tel Dor and Timna, the expedition laid the groundwork for the first UC San Diego-University of Haifa Marine Archaeology Field School, tentatively planned for July 2018.
San Diego, Calif., Monday, Sept. 11, 2017 — Cultural heritage around the globe is continually subject to a multitude of threats, from malicious acts during wartime to neglect, overexposure and natural disasters. Researchers at the University of California San Diego Qualcomm Institute – along with conservationists and researchers at other universities – increasingly turn to advanced digital tools and techniques to better document, preserve and share the world’s historic places.
This approach was discussed in detail at the 26th International Comité Internationale de la Photogrammétrie Architecturale (CIPA) Symposium, hosted by Carleton University in Ottawa, Canada. The symposium, titled Digital Workflows for Heritage Conservation, brought together archaeologists, engineers, architects and conservation specialists from universities, government agencies and private foundations around the world. They discussed best practices for digitizing and restoring heritage structures and monuments while exploring advances in virtual and augmented reality to engage more diverse audiences.
In attendance at the conference – along with delegates from more than 25 countries – were Assistant Research Scientist Dominique Rissolo and Structural Engineering Ph.D. student Michael Hess of the QI Cultural Heritage Engineering Initiative (CHEI) at UC San Diego. Rissolo and Hess were part of a research team that included CHEI Director Falko Kuester as well as Vid Petrovic, Dominique Meyer, and Eric Lo, who are students and staff at QI and the Jacobs School of Engineering.
Rissolo presented on digital preservation of ancient Maya cave architecture along the Caribbean coast of Mexico. With the rapid expansion of tourism development on the “Riviera Maya,” many cave sites – and unique altars and shrines that they have long concealed – are exposed to new threats.
Hess demonstrated new interactive software tools for classifying and analyzing construction materials via 3D point clouds. Novel capabilities adapted by Hess are unique to the point-based visual analytics software developed by Petrovic and are transforming the way specialists analyze data from heritage structures. The paperpresented by Hess was among the top-10 selected to appear in the journal Virtual Archaeology Review.
Other featured projects at the symposium ranged from monitoring rock-fall hazards at Petra in Jordan to digitally preserving war-ravaged mosques in Aleppo, Syria and earthquake-rattled structures in Katmandu, Nepal. Also discussed were novel fabrication techniques used to replace the lost or disappearing architectural elements of Canada’s treasured Parliament buildings.
Says Rissolo: “It was exciting to interface with such a diverse group – there were representatives from UNESCO, Google, the US National Park Service, CyArk, and many other entities that share a common passion for documenting and safeguarding our world cultural heritage. We were also impressed by the extraordinarily talented staff of the Carleton Immersive Media Studio and we look forward to building stronger ties with a broad group of domain experts, innovators and stake holders in pursuit of preserving world heritage.”
Rissolo and Hess were recently invited by the Department of Civil and Environmental Engineering at Princeton University to give a lecture on their work in November. Through CHEI and their CIPA partners, Rissolo and Hess say they hope to contribute to a stronger and more vibrant digital heritage community.
A team of researchers from across UC San Diego is developing a new approach for detecting damage to buildings during earthquakes and other extreme events.
They came together at the Geisel Library recently to use lasers and drones to create a digital record of the structure that will serve as a baseline health assessment. In the event that a sizeable earthquake hits nearby, the team will reconvene to retake the digital measurements and assess any damage to the building such as tilting or cracks. (View photo gallery.)
The information is intended to provide both researchers and emergency responders with more detailed information on how structures respond during the earthquakes—beyond the simple visual inspection of buildings currently in use—prior to allowing them to reopen.
According to researchers at the Scripps Institution of Oceanography and Jacobs School of Engineering at UC San Diego who are spearheading the project, the iconic library is the perfect location to begin what they hope will become an effort to digitize the entire campus.
“We are using this culturally significant building on campus as a reference model to help detect structural changes over time,” said Falko Kuester, a professor of structural engineering who serves as director of the Qualcomm Institute’s Cultural Heritage Engineering Initiative (CHEI) and DroneLab.
For Yehuda Bock, a distinguished researcher and director of the Orbit and Permanent Array Center at Scripps Oceanography, the primary motivation for the recent survey of Geisel Library was to integrate structural monitoring into his early-warning prototype system for earthquakes and tsunamis.
“Our system tracks ground motions at a millimeter level of accuracy,” said Bock. “This allows us to detect large earthquakes within the critical first minute before the shaking begins.”
Six months ago, Bock equipped the Geisel Library with sensors that continuously measure ground motion from the many faults that crisscross Southern California. The technique he helped pioneer, called seismogeodesy, relies on a combination of GPS receivers and accelerators to very rapidly pinpoint the location and magnitude of strong earthquakes—6.0 magnitude or greater—before the hazardous trembling begins.
The late-July project involved nearly two hours of drone flights led by CHEI researcher Eric Lo, capturing more than 1,000 high-resolution images of the Geisel Library that will be turned into a photorealistic model of the structure. Lo’s drone survey was accompanied by a several-hour ground survey by professional land surveyor Richard Maher using lidar (light detection and ranging), an instrument that sends pulsating laser lights at an object to provide a precise 3-D model. By combining these techniques, the team will create a geometrically as well as visually detailed and accurate final model.
Bock’s GPS sensors provide a precise 3-D reference to tie together the high-resolution drone and lidar imagery, allowing for the accurate detection of subtle permanent displacements of the structure’s outer shell as a measure of its integrity after an event.
Kuester currently leads research teams developing drone technology for crisis management and response, as well as applying the work to study and help preserve ancient Mayan structures in Mexico, Neanderthal caves in Italy, and shipwrecks and coral reefs in Bermuda. For him, this project is a first step to create a digital surrogate or as he calls it, a “cyber-twin” of the campus, before new buildings and bridges transform the campus’s physical appearance in the future.
Although Kuester often looks at ancient buildings and ruins, he points out that “it’s important to also document modern buildings before the perils of time or extreme events cause them to deteriorate or create an even less fortunate outcome.”
The digital record on the condition of buildings as they exist today provides a baseline for comparison in the future as a building ages, or in the case of a fire, earthquake or other natural hazard, actionable data, according to the researchers, to swiftly respond and mitigate risks.
Within a few minutes of completing the drone flights, Lo had a quick 360-degree rotating view of the geometrically shaped building from the images collected during the flights for view in 3-D.
Another important motivation for Bock and Kuester is to have students involved in real-world research projects, both on site and as classroom teaching tools.
“As an educator, it’s important that I expose my students to real-world conditions,” said Falko. “The contribution to science needs to be useful and usable.”
Kuester and Lo will also be turning the drone imagery into a virtual reality experience for those interested in flying themselves around the outside of the library.
Bock and Kuester are hoping the project will attract more interest and funding to seismically monitor and digital archive all buildings throughout the UC San Diego campus.
Meanwhile, at Bock’s lab at Scripps Oceanography, the seismic monitoring data is continuously streamed back in real-time. When the next powerful quake strikes, the system will alert him first of the primary signal, called a P-wave, which indicates an earthquake has taken place, and that the destructive S-wave, the one responsible for the strong earth-shaking, is seconds to minutes away. The GPS and seismic sensors on the Geisel Library will quickly indicate whether it has suffered significant shaking and displacement.
For Bock and Kuester, how these buildings and others respond to outside influences is an important component to how we better prepare as a society for extreme events in the future.
San Diego, July 27, 2017 — In a long feature article and companion video, CNN Travel has posted the feature story on “Charted Waters”. In it, writer Alexander Rosen and his co-producer Joseph Coleman follow Dr. Philippe Rouja, Bermuda’s Custodian of Historic Wrecks, offshore and underwater to visit shipwreck sites in Bermuda’s waters. The report notes that “Rouja documents the current state of Bermuda’s wrecks using photography to create 3-D maps and models in a partnership with the University of California San Diego for a project called the Bermuda 100 Challenge.”
CNN goes on to say that the computer models “provide a snapshot-in-time and comparing them tells a story about the evolution of both Bermuda’s wrecks and the environment.”
The article explains that the Bermuda 100 Challenge also allows anyone on the Internet to take a virtual dive and experience Bermuda’s cultural and environmental heritage through the web portal created by UC San Diego’s Qualcomm Institute. “Aside from its scientific importance, there’s a connectivity importance, that people can connect to a period of history, a shipwreck, the ocean,” Rouja tells CNN. “And that’s what shipwrecks do. They are actually a great segue for people into the marine environment that might otherwise not get there. So, when you’re out looking at a shipwreck, because you care about the history or you think shipwrecks are cool, suddenly you’re also learning about the rock fish that lives on that shipwreck or the particular spawning aggregation that’s next door. Giving shipwrecks that work to do is actually a big part of the work I do.”