San Diego, February 2, 2018,

Performances of Erasure and Hearing Seascapes set for 5-7pm on Feb. 8 at UC San Diego’s Qualcomm Institute

In spring 2017, UC San Diego music professor and former Qualcomm Institute (QI) composer in residence Lei Liang, and Falko Kuester, the institute’s professor of visualization and virtual reality, organized a unique seminar course on “Hearing Seascapes: A Collaborative Seminar on the Sonification of Coral Reefs.” It provided graduate students, primarily from the Music department, with an opportunity to develop multimedia projects to highlight the dangers facing coral reefs in many parts of the world.

Scene from Hearing Seascapes

Out of that seminar course emerged two performance-and-installation works accepted into the Qualcomm Institute’s Initiative for Digital Exploration of Arts & Sciences (IDEAS) 2017-2018 season. The two works will premiere simultaneously in QI’s Atkinson Hall on the UC San Diego campus. The immersive works include:

  • Erasure, an ambitious multimedia installation produced by a robust collaboration among three UC San Diego Music Ph.D. students: Jacob Sundstrom in Computer Music, Fiona Digney in Music Performance, and Anthony Vine in Musical Composition, together with Computer Science and Engineering Ph.D. student Vid Petrovic; and
  • Hearing Seascapes, which combines coral-reef imagery and audio data to generate sound based on the location and viewpoints of endangered coral reefs. The work was co-developed by Lauren Jones, a Music graduate student in Vocal Performance, and Music Ph.D. student Eunjeong Stella Koh, both at UC San Diego.

Hearing Seascapes will be staged in the SunCAVE virtual-reality (VR) room in the Immersive Visualization Lab, and Erasure in the Reconfigurable Media Lab, both on the first floor of QI’s Atkinson Hall. The works will premiere Thursday, February 8, 2018 from 5-7 p.m. in Atkinson Hall, followed by a public reception. The schedule:

  • 5:00pm Erasure and Hearing Seascapes installations open to visitors
  • 5:30pm Music Prof. Lei Liang introduces both teams of artists, followed by artist talks
  • 6:00pm Both installations reopen for viewing; public reception begins in entry hall in front of the Calit2 Theater.

“Over the last several decades, coral-reef ecosystems have suffered significant impacts from both local and global factors,” said Koh. “We aim to convey important messaging to the audience and illuminate data that show the declining health of ocean coral reefs.”


Through an interconnected network of three-dimensional (3D) photomosaic models of coral reefs and spatially as well as electronically processed percussion sounds, a metaphorical ecosystem forms and responds directly to human presence and the temporal history of that presence throughout the work’s existence.

Audio-visual cues label types of coral with unique sounds and painted overlays on top of 3D photomosaic models from the 100 Island Challenge.

Erasure responds negatively to human presence in the installation environment. As more and more people enter the space, the installation begins to break down: the sonic tapestry of percussion sounds contort and particulate, the synthetic biome of coral visualizations begin to morph into unnatural forms, and the entire system mutates – all with the presence of the audience – and members of that audience are aesthetically confronted with their impact on these remote and fragile ecosystems.  The transformation, however, is neither irreversible nor unidirectional. “As viewers leave, the piece rebounds, but more slowly than the rate at which it broke down,” explained Ph.D. student Sandstrom. “The system bounces back from the immediate and long-term human impact, and it reflects the resilience of the reefs to withstand and adapt to global shifts in climate and the ecosystem.”

The visual component of Erasure consists of 3D photomosaic models of coral reefs taken from the 100 Island Challenge, based in the Scripps Institution of Oceanography. These digital reproductions were created by the Challenge’s technical visualization advisors from the Qualcomm Institute’s Cultural Heritage Engineering Initiative (CHEI), jointly with Scripps Oceanography. By rigorously photographing and collecting data from reef sites and rendering the data into 3D computer models using custom software developed by CHEI’s Falko Kuester and Vid Petrovic, students from the Music department were able to observe the reefs from various angles, light levels, and distances.

“These coral reef constructions ebb and flow between their natural state—meticulously constructed synthetic ecosystems—and transformative states: from granulations of the stony corals and polyps into whirling cascades of particles, to fissions of vast reef colonies into splintered slabs that recede in and out of focus,” said  Music’s Anthony Vine. “Lingering traces of the piece in its untouched state float among the remains, and the metaphorical ecosystem appears to be dreaming and longing to return to an undisturbed state.”

The sound-space of Erasure is created from a reservoir of percussion improvisations that both reflect the sounds that might be found in and around a coral-reef environment, as well as poetic expansions that reach beyond the palette of oceanic utterances: scraped and struck limestone tiles, sweeping washes of hands streaked across a bass drum, and the murky drones of rolled bell plates. By manipulating these samples in a simple causal network, emergent behavior materializes to constitute a lush atmosphere of sound. In this way, the behavior of the sound world is not unlike the behavior found in reef ecology: masses of small units combining to create a complex and rich environment.


“Music can bring an image to life, and by giving a voice to the coral reefs, we can help the audience make an emotional connection to ecology and realize the fragility of these reefs,” said Lauren Jones. “The goal of this project is to create an immersive experience for the viewer that allows them to submerge themselves in the world of the coral.”

3D laser scan of coral reef featured in Hearing Seascapes.

Viewers use a joystick to ‘dive’ and explore the reef, controlling the location, viewpoint, depth and speed of navigation. Audience members hear different sounds that represent different species in the data set. Each species has its own specific personality (represented by different sounds). “Coral reefs are living, breathing organisms that are vulnerable to small changes of the surrounding environment and climate,” explained Stella Koh. “We assign the coral reefs a distinct personality by examining certain characteristics such as texture, habit, origin and growth. To convey messages through music, we’ve recorded an underwater dialogue of voices.” The installation was designed to induce conversation between coral reefs and a fish maneuvering through the reefs, and the sound becomes louder or softer depending on how far away it is from the coral.

Jones and Koh set for themselves three goals with Hearing Seascapes: to display experiments with different aspects of sound and innovative graphic design to create an enjoyable environment for the audiences; to tell an effective and interactive story invoking concepts of adventure, imagination and humor to motivate people to recognize environment health; and to create an inviting seascape with a synergy of voices, images, synthesized sounds and human emotion.

“Based on the notion of acoustic ecology, we want to bring out the positive aspects of sound in the ocean environment,” they noted. “We hope to highlight the importance of engaging in the soundscapes of coral reefs in hopes that our musical voice can make scientific results more accessible to society.”

In a study published recently in the journal Coral Reefs, scientists from Scripps Institution of Oceanography created and analyzed detailed photomosaics of the coral reef at Palmyra Atoll in the Pacific Ocean.  Led by Scripps coral-reef ecology Ph.D. student Clinton Edwards, the team canvassed more than 17,000 square feet of reef and 44,008 coral colonies, taking more than 39,000 images.

Coral reef researchers canvas a reef in Palmyra Atoll,
south of Hawaii. (Photo by Stefani Gordon)

Rather than taking the conventional route of stitching together the collected images into high-resolution 2D image mosaics, Scripps researchers used cloud computing techniques developed by UC San Diego engineers and computer scientists to enable the creation of detailed models in 3D and even in 4D (i.e., adding the dimension of time).  The models can be navigated and studied fully interactively, thanks to a team of technologists from UC San Diego’s Cultural Heritage Engineering Initiative (CHEI), led by UC San Diego structural engineering professor and computer science faculty-affiliate Falko Kuester.

Creating the mosaics and wringing useful information out of them is a time-consuming process. Gathering the data, the scientists generally did three dives a day, and it took more than five full days of diving to collect the images for the 16 plots used in the study. Back in the lab, Edwards used a custom high-performance computing system to stitch together the 2,500-3,500 individual images that make up each mosaic. It takes the software several days to complete the rendering of the composite image, and around 100 hours to label and classify all the corals in each image. Then the final step is to extract the species information and analyze it, which takes another three days or so per image.

GIF of 3D model of coral, achieved using algorithms and techniques developed in QI’s Cultural Heritage Engineering Initiative.

Digitization of the images is clearly the limiting step, said Edwards. But that may change soon, he added: “We have excellent collaborators in the Computer Science and Engineering department at UC San Diego, and are getting close to having a computer-assisted workflow that will dramatically accelerate this process.” The computer scientists and engineers are part of CHEI, based in the Qualcomm Institute.

The data and scans of coral reefs from Palmyra Atoll represent some of the first results from the 100 Island Challenge (100IC), a project to create a global perspective on how coral reefs are changing over time. Scripps professor Stuart Sandin leads the overall project as principal investigator, with technology support from technologists in CHEI. The CHEI team is developing the novel 3D imaging and visual analytics techniques that Scripps scientists are using to create the mosaics that capture every detail of coral reef structure and ecology. So far, Scripps and international partners in 100IC have visited 70 of the 100 islands, and they plan to resurvey each site every two years. The resulting images, 3D models and analysis will also become baseline data to help local agencies study their own reefs.

In addition to Prof. Kuester, the 100 Island Challenge technology team from CHEI includes a handful of researchers affiliated with the CSE department: Ph.D. student Vid Petrovic, who is developing the software that the Scripps scientists use to create, visualize and analyze the coral reefs through 3D models and photomosaics; graduate student Dominique Meyer, who completed his B.S. in Physics at UC San Diego in 2016; software engineer and Computer Science alumnus Chris McFarland (B.S. ’12) and Computer Engineering alumnus  Eric Lo (B.S. ’14), who is a staff robotics engineer in CHEI.

100 Island Challenge tech team includes six researchers affiliated with the Qualcomm Institute (l-r):Calit2 professor of scientific visualization and virtual reality Falko Kuester; CSE Ph.D. student Vid Petrovic; CSE graduate student Dominique Meyer; QI programmer-analyst and CSE alumnus Chris McFarland, QI staff engineer and Computer Engineering alumnus Eric Lo; and QI-CHEI staff Dominique Rissolo, a marine archaeologist.

The CHEI team has also deployed the technology for the Bermuda 100 Challenge , in which CHEI and Bermuda’s Custodian of Historic Wrecks are creating digital replicas of shipwrecks and coral reefs surrounding Bermuda.

CSE’s Petrovic is now working on creating custom algorithms to accelerate the work on the coral-reef effort and other underwater mapping and computer modeling of the environment.

Top photo at Palmyra Atoll contains 5,000 corals;
at bottom, identical photo after being annotated by
hand to label each coral species with different
color. 3D technology can dramatically reduce the
time to capture coral reef data for later analysis
in the lab.
Watch video of Palmyra Atoll coral
reef annotated to label species of coral in color
overlays .

“More and more imagery is being collected across the field of marine sciences, and the pace and scale of the effort will only increase — but more data doesn’t automatically mean more, or better, science,” observed Petrovic. “It’s an honor and a joy to be working so closely with a group of marine ecologists to address this, developing collaboratively the tools and workflows that are needed to make productive use of the imagery, whether for monitoring reef health, or for advancing basic science.”

According to Petrovic’s advisor, Prof. Kuester, the team is demonstrating that extremely detailed models consisting of billions of 3D data points can now be created overnight literally, when using distributed computing strategies. For a recent study, 14,000 images were turned into a comprehensive 3D model in less than a day. A digital surrogate of sorts, the model helped researchers annotate, measure and study the target environment in ways that would be impossible to achieve in-situ. By lifting the image data from 2D to 3D models or even 4D (including time), the technology is adding diagnostic value. Traditional analysis techniques when working on photomosaics can be supported by exporting a mosaic directly from the model, using the equivalent of an ultra-resolution virtual camera with user-definable characteristics.

Petrovic says the team is making it possible for scientists to virtually explore reefs in the lab, allowing them to time-travel from year to year and track the growth and decline of individual colonies, and to study spatial and temporal relationships across the reef.

“We’re speeding up the digitization and annotation, and clearing a path to letting machine-learning techniques carry more of this burden,” Petrovic said. “This is all terribly exciting, and with much more to come. But the most rewarding aspect for me is the interdisciplinary collaboration that makes it possible in the first place, that lets us apply a decade of visualization research in support of vital ecological work.”

Beyond advancing the state of science in order to better safeguard coral reefs, the team’s research is literally giving coral reefs a voice. The 100 Island Challenge’s adoption of 3D digital tools allows the public at large to dive and experience these stunning ecosystems in virtual reality, to see what otherwise cannot be seen, and hopefully become stewards for coral reef preservation. “If a photo is worth a thousand words,” observed CHEI’s Kuester, “just imagine what a fully interactive 3D environment can accomplish for scientific storytelling.”

September 18, 2017. A team of archaeologists from the University of California San Diego and two leading Israeli universities has wrapped up a three-week expedition to document two major sites in Israel using the latest in 3D scientific visualization technologies.

The expedition, led by UC San Diego Department of Anthropology professor Thomas E. Levy, involved a land-and-sea approach to understanding trade and exchange during Biblical times, at sites dating to the Iron Age, circa 1200 to 586 BC. Officially sanctioned by The Explorers Club, the expedition carried Explorers Club Flag #179 to the ancient mining region of Timna in southern Israel as well as to Tel Dor on the Mediterranean coast, the site of a submerged port facility.

“Between the two sites, we captured more than 24,000 high-definition photos for use in creating 3D computer models of these two very historic areas,” said Levy, who directs UC San Diego’s Center for Cyber-Archaeology and Sustainability (CCAS), based in the Qualcomm Institute, and co-directs the Scripps Center for Marine Archaeology (SCMA). “Back in San Diego, we’ll be processing the images to build large-scale virtual models to help archaeologists and students better understand the two historically important regions in Israel and how they were united in antiquity through the copper trade.”

Explorers Club Flag 179 at the submerged Iron Age port adjacent to Tel Dor, Israel.  Thomas Levy (left) and Assaf Yasur-Landau (right) display the flag in rough water. (Photo courtesy T.E. Levy, Scripps Center for Marine Archaeology)

Levy’s co-principal investigators on the expedition included University of Haifa professor Assaf Yasur-Landau in the Department of Maritime Civilizations, and UC San Diego alumnus Erez Ben-Yosef (Ph.D. ’10), who did his doctoral dissertation with Levy and is now a professor of archaeology at Tel Aviv University. Ben-Yosef organized the logistical infrastructure of the Timna imaging projects, while Yasur-Landau organized the 3D Tel Dor imaging project through his Laboratory for Coastal Archaeology and Underwater Survey, with support from former Scripps Institution of Oceanography postdoc Tali Treibitz from the University of Haifa’s Marine Imaging Laboratory. Treibitz facilitated the underwater data collection in the waters of Dor. Also helping to make the expedition a success were CHEI director Falko Kuester for student support and the allocation of processing resources and equipment to the project, as well as researcher Eric Lo for image processing.

Drone view of contact zone between Tel Dor and various Iron Age and Classical period building with the Mediterranean Sea shore (Photo courtesy Griffin Ltd., Scripps Center for Marine Archaeology, and Center for Cyber-Archaeology and Sustainability, UC San Diego)

The UC San Diego team included Christian McDonald, scientific diving safety officer at Scripps Institution of Oceanography, Department of Anthropology graduate student Anthony Tamberino, as well as Dominique Meyer, a Ph.D. student in the Department of Computer Science and Engineering and researcher in the Cultural Heritage Engineering Initiative (CHEI). Meyer and Tamberino were responsible for the provision of equipment and data collection of aerial and ground-based 3D images. Meyer also served as the primary drone operator and flight planner for the expedition, for which he operated Phantom 4 drones with standard 20-mm cameras as well as directing a heavy-lift drone octocopter for video and structure-from-motion (SfM) photography with 36-megapixel cameras operated by the Griffin company. The logistics of the project were greatly supported by the Israel Antiquities Authority whose own drone operator, Yitzhak Marmelstein, worked with the team for the duration of the project.

CSE Ph.D. student Dominique Meyer adjusting Phantom 4 drones before flights over Shiqmim and the Wadi Beersheva. (Photo courtesy T.E. Levy, Scripps Center for Marine Archaeology and Center for Cyber-Archaeology and Sustainability, UC San Diego).

“Previously, copper ingots found in the waters north of Tel Dor were traced via lead isotope analysis to the Faynan region in Jordan, where extensive Iron Age mines and smelting factories have been found and excavated by our UC San Diego team,” said Levy. “It seemed most likely that Tel Dor was the main Iron Age port along the Israeli coast for the trade of copper, from the desert zones of Faynan in the northern Aravah valley and Timna at its southern end.”

Levy (center) with former Scripps postdoc Tali Treiblitz (left) and Scripps’ diving safety officer Christian McDonald.

The aerial drones carried out regional 3D scanning of more than 15 major ancient sites related to copper mining and smelting in the Timna mining region, including at Shiqmim in the northern Negev desert. In Shiqmim, Levy had previously excavated Copper Age ruins, including an ancient 24-acre village dating to the Chalcolithic period (ca. 4500 – 3600 BC), when copper was first smelted and cast in the southern Levant.

To capture the submerged port facility at Tel Dor, the team used scuba and high-precision underwater cameras to image remains of the ca. 9th century BC port.

The Israel 3D Land and Sea Project 2017 was generously funded by Marian Scheuer Sofaer and Abraham Sofaer of Palo Alto, CA..

In addition to the 3D imaging project at Tel Dor and Timna, the expedition laid the groundwork for the first UC San Diego-University of Haifa Marine Archaeology Field School, tentatively planned for July 2018.

San Diego, July 18, 2017 — Technology deployed by the Qualcomm Institute to create a 3D digital database of more than 100 shipwrecks surrounding Bermuda is the focus of a TV report by the BBC in the United Kingdom and broadcast in countries around the world. The report is now also available for viewing on-demand on the BBC Travel Show’s YouTube channel (see below) worldwide.

The widely-watched BBC Travel Show’s latest episode leads with the Bermuda 100 Challenge, a partnership between UC San Diego’s Cultural Heritage Engineering Initiative (CHEI), the island’s Department of Environment and Natural Resources, as well as Look Bermuda and Nonsuch Expeditions.  The project is using photogrammetry to capture shipwrecks in Bermuda waters and turn the data into 3D computer models that can easily be navigated via a web browser. The Bermuda 100 web portal features information on dozens of shipwrecks, including four which are already viewable as 3D models that document each site in crisp detail.

The BBC program follows Bermuda’s lead expert on marine heritage and Custodian of Historic Wrecks,  Dr. Philippe Rouja, on a visit to one of the four shipwrecks already fully documented by the team. The Montana, which was used to run embargoed goods for the Confederacy during the Civil War, sank in 1783. “It was one of the first ships to be digitally recreated using this cutting-edge technology,” Rouja tells Travel Show host Ade Adepitan, before showing what the 70-meter-long vessel looks like on the Bermuda 100 web portal. “I have never really seen this shipwreck from end to end. You can see how big and long it is, so in one snorkel, you can’t visually take it all in.”

BBC Travel Show host Ade Adepitan (left) interviews Bermuda’s Custodian of Historic Wrecks, Philippe Rouja.

Adds Rouja: “The minute that it’s laid out in that 3D fashion, using photogrammetry, suddenly you can see it from the stern all the way to the bow and you just take in the full extent of the ship and it literally brings it back to life.”

According to the TV report, the team in Bermuda is working with the University of California [San Diego] to “record thousands of images and build a 3D digital replica of [each] vessel. It means that it will be preserved for generations.”

Ultimately, reports the BBC’s Adepitan, the Bermuda 100 web portal is bringing the excitement of discovery to people around the world who unlikely to see first-hand Bermuda’s 400 years of shipwrecks (and the coral reef ecosystems that caused most of those wrecks). “Diving is a luxury that is out of reach for many people,” he concludes. “So the plan is now to map at least 100 of the shipwrecks, meaning that anyone from anywhere in the world will be able to take a digital dive and experience centuries of maritime history.”

November 7, 2013 — An interdisciplinary team of Ph.D. students from the University of California, San Diego, recently spent time at archaeological sites in the southern Italian region of Calabria, and they came away with a newfound respect for the daily rou- tine of uncovering the past – and a better understanding of how to safeguard archaeological sites and artifacts for the future.


Some members of the UC San Diego and University of Calabria teams include (l-r) local artists and cultural experts Battista Favoino and Romina Giordano, CSE grad student David Vanoni, professor Falko Kuester, structural engineering grad student Michael Hess, and anthropological archaeology grad student Ashley Richter.

The Oct. 8-15 visit coincided with the signing of a Memorandum of Understanding (MOU) between UC San Diego’s Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3) and the Department of Archaeology and Art History of Italy’s University of Calabria (UNICAL).

Professor Giuseppe Roma, head of the Department of Archaeology and History at the University of Calabria, extended an invitation to the CISA3 team to visit and work in a handful of sites that represent civilizations going back at least to the Greeks.

“This is an important partnership because the two sides have interests that are closely aligned,” said CISA3 Director Falko Kuester about the document signed by UCSD Vice Chancellor for Research Sandra Brown. “Calabria is the location of some of the most important archaeological finds in Europe going back many civilizations, and UC San Diego is developing technologies and new methodologies that will enhance our ability to uncover the secrets of our past.”

Kuester, who is also Principal Investigator on UCSD’s National Science Foundation (NSF) IGERT graduate training grant in cultural heritage diagnostics, led CISA3’s Calabria expedition including UCSD IGERT Ph.D. students David Vanoni and Vid Petrovic (both Computer Science and Engineering); Michael Hess (Structural Engineering); Sa- mantha Stout (Materials Science), and Ashley M. Richter (Anthropological Archaeolo- gy). “One truly remarkable aspect of this expedition was the strong leadership role that our students took in planning, organizing and executing the expedition, demonstrating the unique talent set that these emerging innovators have at their disposal,” explained Kuester. “Beyond advancing research on their respective dissertations, they demonstrated what it means to do interdisciplinary work across historic domain boundaries, while catalyzing new international collaborations.”

The team stayed in the medieval town of Rocca Imperiale, where the University of Calabria has been excavating at numerous sites in and around the historic town.


Vid Petrovic and Mike Hess undertake digital imaging at Murgie di Santa Caterina.

Digitizing archaeological sites was a major focus as the 2013 excavation season neared its conclusion. Murgie di Santa Caterina is an early medieval monastery site built on top of several layers of previous occupation dating back to prehistory. The CISA3 team worked with members of the UNICAL team on how to collect data and brainstorm ways to improve data collection, processing, and analysis, as well as new cyber-archaeology tools. While CISA3 students focused on building a virtual model of the site using terrestrial laser scanning, UNICAL researchers created time-lapse data sets using Structure-from-Motion (SfM) modeling to document the process of excava- tion. They were also digitizing 3D models of excavated pottery for later analysis.

Overlooking the medieval city is the Castello Svevo di Rocca Imperiale. After a grass- roots movement, townspeople got the municipality to purchase the historic site and refurbish it. Ongoing excavations have turned up secret passageways and lower lev- els. After the architect in charge of the project gave the CISA3 team a tour of the site, the UC San Diego students undertook to digitally document the Castello Svevo using photogrammetry, laser scanning, Structure-from-Motion, and stereoscopic photogra- phy, as well as thermal imaging on both the interior and exterior of the structure for structural analysis. A major goal was to collect the reference record needed to study the castle’s state of health over time and ascertain how well the castle might withstand a sizeable earthquake.

The same technologies, plus X-ray fluorescence (XRF), were deployed in an effort to determine the structural health of the Santuario di Santa Regina di Anglona. The Sanctuary is an 11th century church decorated with a fresco depicting a complex mixture of symbols on the church’s walls and columns – reflecting a rare, brief period of peace between local Christian and Muslim communities in the region.

Professors Roma of the University of Calabria and Kuester of UC San Diego led the joint team, which determined that despite several restorations, many of the church’s walls are sloping badly. A probable cause is soil settlement, combined with past seismic activity in the region and continuous monitoring would be desir- able to capture further changes early. The XRF was used to analyze the material used for the wall frescoes, which could shine a light on their origins. “The Sanctu- ary is one of the most prominent monuments of the region,” said Kuester, who is also a professor of structural engineering at UC San Diego. “The new data we acquired will help us create a conservation plan for the highly-valued sanctuary– the only standing architecture that remains of the ancient city that once enveloped the hill around it.”

Another highlight of the expedition was a visit to the ancient city of Sybaris, a Greek colony that thrived some 3,000 years ago, earning so much wealth that the English adjective, sybaritic, derives from the excessive luxury for which the ancient city was known – a city that was lost to history until it was re-discovered and excavated starting in the 1960s.


David Vanoni (foreground) adjusts the thermal camera during a night shoot at the Castle Svevo.

The team from CISA3 also updated their Calabrian counterparts on efforts to adapt technologies to the needs of archaeologists in the field. The two sides also shared their latest thoughts on other issues facing archaeologists everywhere, including data management, visualization of multimedia for analysis of cultural artifacts, user interfaces, and technology for archaeological research.

“The University of Calabria and others are moving into the realm of digital archaeology with increasing speed,” said Ashley Richter, the Ph.D. student in ar- chaeology on the expedition. “However, they do not often have the equipment to do so themselves. With laser scanning for instance, data acquisition is frequently outsourced to firms with little to no archaeology experience, but all the right tech- nical proficiency. CISA3 is bridging that gap.”

“The key part is our integrative diagnostic imaging methodology, which creates spatially and temporally layered information in one sweep,” added Kuester. “This methodology allows different data modalities to inform each other, creating a means to really capture the big picture.”

As the CISA3 team’s stay in Calabria neared its end, the mayor of Rocca Imperiale, Ferdinando Di Leo, called a press conference to let the media know some of the highlights of the 2013 excavation season, including the participation of UC San Diego faculty and Ph.D. students. The mayor insisted that international col- laboration is critical to future archaeologists in the region.The press conference ended in particular Calabrian fashion – with a toast (and taste) of prosecco, an Italian sparkling wine.

After the press conference, Mayor Di Leo had a special request. The central church of Rocca Imperiale, Santa Maria in Cielo Assunta, has been continually destroyed and rebuilt since the 12th century. The mayor wanted to show the UC San Diego team a palimpsest of stonework on one of the outer walls of the church. The group brought along some of their equipment to capture and potentially analyze the palimpsest.

The final day ended with another surprise: at the bed-and-breakfast inn where the CISA3 team was staying, the mayor commandeered the kitchen and insisted on personally cooking his special “midnight spaghetti” for American and Italian collaborators.




The partnership between the Qualcomm Institute’s Albert Lin, TopCoder and NASA invites coders to participate in a three-week challenge to build a powerful machine-learning algorithm to identify ground features in satellite images of Mongolia.

September 13, 2013 — A crowdsourcing effort led by University of California, San Diego research scientist Albert Yu-Min Lin is central to a new challenge as programmers worldwide are invited to develop a machine-learning algorithm to match human perception in picking out interesting features in satellite imagery. While the images come from Lin’s search for the lost burial site of Genghis Khan, a new algorithm could help NASA scientists decipher images of distant planets.

TopCoder, Inc., the world’s largest open innovation platform and competitive community of digital creators, announced Sept. 12 the new challenge through the NASA Tournament Lab (NTL), in partnership with UC San Diego’s Lin, who works in Calit2’s Qualcomm Institute and the Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3) at UC San Diego. A National Geographic Emerg- ing Explorer, Lin is also a three-time alumnus of UCSD’s Jacobs School of Engineering in electrical engineering (BS ’04, MS ’05) and materials science (Ph.D. ’08).

The three-week Collective Minds & Machines Exploration Challenge asks TopCoder community members from around the world to study a massive data set of crowdsourced, human-generated analytics tags of satellite imagery of the uninhabited landscape of Northern Mongolia, a likely location of the lost tomb. They are challenged to develop an algorithm that learns from the crowd, and emulates the sensitivity of human perception when recognizing and categorizing subtle details and features in images (for example, to pick out topographical features that could indicate the presence of an ancient human-built structure). Once developed, the algorithm could integrate machines and crowds to accelerate discovery in a wide array of big data problems ranging from planetary to medical imaging exploration. The partnership between the Qualcomm Institute’s Albert Lin, TopCod- er and NASA invites coders to participate in a three-week challenge to build a powerful machine-learning algorithm to identify ground features in satellite images of Mongolia. Click here to watch a video about the challenge to the coding community.

Lin recently applied the concept of crowdsourcing to an ambitious satellite imagery labeling initiative. In the first-of-its-kind collective exploration experiment, tens of thousands of amateur analysts worldwide helped Lin’s team at UC San Diego identify sites of cul- tural and historical significance. “Here we turn towards the crowd not only to tackle the data size challenge of large-scale satellite remote sensing, but more importantly to pool human perception and intuition when sifting through the data for anything that looks ‘out of the ordinary,’” said Lin. His team then mounted a series of land expeditions to verify regions of high interest defined by the crowd.

On the heels of this successful crowdsourcing application, Lin turned to researchers at the Harvard NASA Tournament Lab and TopCoder to leverage open innovation. Their goal: to develop an algorithm that can amplify the power of crowdsourced human perception when presented with ill-defined patterns.


Better algorithms for detecting human-built structures from satellite images will enhance exploration in space and on Earth, by accelerating how NASA assesses planetary imagery. The goal is to identify, protect and preserve the integrity of undiscovered sites of cultural and historical significance worldwide.

“In science, data sets continue to get bigger and answers continue to get fuzzier,” noted Lin. “A collaboration between machine and crowd is a natural next step as we face the increasingly overwhelming data avalanches of our digital world.”

To respect the traditional beliefs of indigenous people, Lin and his team have used state-of-the-art technologies to conduct a noninvasive survey of Genghis Khan’s homeland. While Lin’s work enabled the online crowd of explorers to detect and label human-built structures in satellite imagery, TopCoder is challenging its community to develop an active, machine-learning algorithm that matches human capabil- ity. The successful algorithm will allow machines to learn as they compute and respond to the human input data from the crowdsourcing initiative. The long-term goal of this project is to create an algorithm that exceeds human capability.

“The quest to find Genghis Khan’s tomb has mystified scientists for hundreds of years,” said TopCoder President and COO Rob Hughes. “This powerful new algorithm will accelerate Albert’s research and realize the full potential of blending collective human intelligence with machine learning created through open innovation. The TopCoder community’s cutting-edge solutions are a natural fit for NTL and a technology-enabled explorer like Lin, whose mission is to uncover humankind’s best kept secrets.”

The Collective Minds & Machines Exploration Challenge was developed through the NTL, a collaboration between NASA, Harvard Business School in association with the Institute of Quantitative Social Science, and TopCoder. It allows the TopCoder com- munity to compete to create the most in- novative, efficient and optimized solutions for specific, real-world challenges. UC San Diego, where Lin co-directs the Engineers for Exploration Program, is experimenting with mechanisms to discern locations of ancient archeological places using satel- lite images. NASA and UC San Diego will work together to harness UC San Diego’s previous efforts and the resources of the NTL to produce code that will analyze open-source aircraft and satellite images by using amateur feedback for classifica- tion. NASA can then apply this same technology to imagery assessment of planetary objects.

Registration for the challenge is now open and contestants must be TopCoder members. TopCoder membership and regis- tration are free at The competition ends on October 3, 2013. For more information on this adventurous algorithmic challenge, visit

The Valley of the Khans Project

Albert Lin’s research has taken him into the most remote regions of the world, has led to industry- changing innovations and has expanded the role of media in science. The effort he is most known for, the Valley of the Khans Project, is a high-tech ground-, aerial- and satellite- based, remote sensing, non-invasive search for the tomb of Genghis Khan that was featured in a one-hour National Geographic Channel documentary film, “Forbidden Tomb of Genghis Khan,” which he also narrated. The effort earned him recognition as National Geographic Adventure Magazine’s “2010 Readers Choice Adventurer of the Year” and the 2011 Lowell Thomas Medal for Exploration from New York- based Explorers Club. Lin’s goal is to enable international protection of a sacred region of Mongolia. His team’s use of digital media to perform massive satellite data analytics through crowdsourcing (recognized with the 2011 United States Geospatial Intelligence Founda- tion Academic Achievement award) has been emulated for applications ranging from emergency response to humanitarian monitoring and led to the formation of California-based Tomnod Inc. where he served as co-founder and chairman of the Board of Directors until 2013, when the company was successfully acquired by the Colorado-based commercial satellite imagery provider, Digital Globe. He has been invited to present his work on massive-scale collaboration and crowd-based collective intelligence to organizations including the US Geospatial Intelligence Agency, Harvard Business School, and the Department of Defense.


TopCoder is the world’s largest Open Innovation Community of digital creators with over 500,000 members representing algorithmists, software developers and creative artists from over 200 countries. The TopCoder Open Innovation Community creates digital assets in- cluding analytics, software and creative designs and solutions for a wide-ranging client base through a competitive, rigorous, standards based methodology. Combined with our extremely talented community this groundbreaking methodology results in superior outcomes for our clients.

September 10, 2013 — A team of researchers from the University of California, San Diego, has received $3.4 million in grant and matching funds to design and build new 3D display technologies for visualizing big data from macroscopic to microscopic scales.

ucsd_chei_ falko_vroom

Falko Kuester, profesor of visualization and virtual reality at UC San Diego’s Qualcomm Institute, stands in front of the VROOM digital display while viewing data on a tablet computer. The new $3.4 million SCOPE project is the next step in the evolution of digital display technologies at the Qualcomm Institute.

The project, called SCOPE, is both “an acronym and a visual metaphor for a combined microscope and telescope, enabling users to explore data from the nano to micro to macro to mega scale,” said principal investigator Falko Kuester, a professor of visualization and virtual reality in UC San Diego’s Qualcomm Institute and an associate professor of structural engineering in the Jacobs School of Engineering. “SCOPE stands for Scalable Omnipresent Environment, which also captures the vision behind the large-scale, ‘walk-in’ collaborative visual computing instrument,” he added. “It’s a Holodeck of sorts that will connect spatially distributed research teams working on big-data challenges.”

The funds for the project include $2.4 million from the National Science Founda- tion (NSF) Major Research Instrumentation (MRI) program, and an additional $1.4 million in matching funds from UC San Diego, its Jacobs School, and the Qualcomm Institute (the UC San Diego division of the California Institute for Telecommunications and Information Technology, or Calit2), as well as project partners Jackson State University (JSU) and the University of Texas Medical Branch (UTMB).

Kuester says the goal of SCOPE is to create a highly interactive collaboration instrument equipped with a natural human-computer interface that supports advanced 3D modeling and rendering capabilities. The data will be rendered to scales suitable for analyzing large amounts of visual and numerical data pertain- ing to phenomena of wide dimensions and extreme time scales.

UC San Diego Research Scientist Thomas A. DeFanti, a pioneer in virtual-reality systems, calls SCOPE “the next major evolution in our series of super high- resolution data visualization displays — scores of times higher resolution than HDTV.” DeFanti is a co-PI on the project, along with Tajana S. Rosing, an As- sociate Professor of Computer Science and Engineering (CSE), and Qualcomm Institute Assistant Research Scientist Jürgen P. Schulze, who is also an Adjunct Assistant Professor in the CSE department.

Over the past 20 years, SCOPE researchers and their partners have designed many of the world’s most advanced visualization display systems, including VizClass, HIPerWall, HIPerSpace, the CAVE, StarCAVE, NexCAVE, TourCAVE, and VROOM. SCOPE will allow remote collaboration from anywhere in the world. Its 40-100 gigabit/second high-speed networking and embedded flash memory technology — originally developed at the San Diego Supercomputer Center — will make it possible to explore big data interactively and collaboratively, even when the volume of data on display reaches hundreds of terabytes or even petabytes. SCOPE will eventually connect researchers at UC San Diego with team members at JSU and UTMB. Several years of hands-on work- shops at Calit2 with Minority Serving Institutions (MSI)-CyberInfrastructure Empowerment Coalition (MSI-CIEC) led to the partnership with JSU (an MSI) and UTMB.


UCSD Research Scientist Thomas A. DeFanti and the NexCAVE, one of the digital display technologies developed at the Qualcomm Institute.

SCOPE will be used by researchers in disciplines such as computer science, structural engineering, marine science, anthropology, art, health, geo-sciences and bioengineering.

The SCOPE team encompasses 45 researchers from more than a dozen different departments that rely on fundamental computer science visualization research. This set of researchers will be further augmented by: (a) faculty recipients of Calit2 Strategic Research Opportuni- ties (CSRO) grants; (b) graduate students in the NSF-funded Integrative Graduate Education and Research Traineeship (IGERT-TEECH) program for cultural heritage diagnostics and preservation; and (c) undergraduate researchers supported by Research Experiences for Undergraduates (REU) supplements.

IGERT-TEECH, which is based at the Qualcomm Institute’s Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3) served as an inspiration for the SCOPE proposal, tracing its roots back to UCSD Chancellor’s Interdisciplinary Collaboratories Initiative. “It’s a win-win situation,” said Kuester. “This project allows us to develop a truly unique research instrument while building a team of highly interdisciplinary researchers driving innovation through collaboration.”


Calit2 researcher and canine friend testing camera trap outside UC San Diego’s Atkinson Hall.

April 1, 2013 — From three-dimensional imaging of underwater artifacts to thermal imaging-based tracking of animals in the wild, the Engineers for Exploration program at the University of California, San Diego is continually seeking new ways to break down barriers in the world of exploration with UC San Diego’s partners in the program, the National Geographic Society, Hubbs SeaWorld Research Institute, and San Diego Zoo Global. Now, with support from a Research Experiences for Undergraduates (REU) grant awarded by the National Science Foundation (NSF) to program co-directors Ryan Kastner, Albert Yu-Min Lin, and Curt Schurgers, a new crop of undergraduate students will spend their summer in San Diego tackling real-world engineering challenges in exploration and scientific discovery.

Kastner, a professor of computer science and engineering in the Jacobs School of Engineering, and Lin and Schurgers, research scientists in the UC San Diego division of the California Institute for Telecommunications and Information Technology (Calit2), were awarded $380,000 to launch a new Computer and Information Science and Engineering (CISE) REU site, with research activities set to start in June 2013. The Engineers for Exploration program and its newly-funded REU activities are managed through Calit2’s Center of Interdisciplinary Science for Art, Architecture, and Archaeology (CISA3).


Quad Copter aerial photography platform in flight.

This research initiative is the only such REU program on the UC San Diego campus funded by the NSF’s CISE directorate; it will build upon the structure of the existing Engineers for Exploration program, which matches upper-level UC San Diego undergraduates with a variety of high-tech research projects overseen by scientists, engineers, and explorers from the university and program partners. In the program’s first year, four REU slots will be filled by UC San Diego students, and an equal number of undergraduate researchers will be recruited for the summer from other institutions.

“This NSF funding will allow us to expand the Engineers for Exploration program to invite enthusiastic engineering and computer science majors from across the U.S. to come to San Diego to design and test their own research projects in the field,” said Kastner. “All of the undergrads will benefit from the guidance of existing undergraduate leaders in the program, graduate students, professors, and scientific mentors from our partner institutes.”

The 2013 Engineers for Exploration REU program is scheduled to run from June 20 to Aug. 29. Participants will be provided free room and board on-campus for the duration of the 10-week program, and will receive a $5,000 stipend for their work. The three-year program is expected to enroll additional participants each year beyond the initial eight undergraduates selected for the 2013 summer session.

Past Engineers for Exploration projects have tackled a wide range of exploration-related technologies. One team created a low-power camera trap using wireless vibration sensors and a thermal camera, and another developed an easy-to-use flying camera platform for aerial photography, surveying, and mapping (see the Engineers for Exploration website for more details). REU participants’ summer projects are expected to be similarly robust in terms of engineering rigor and opportunities for collaborating with explorers, scientists, and engineers from across the globe.

“The students selected for our REU program will develop and try out in the field a number of exploration-themed technologies of their own design,” noted Kastner. “Through this program, students will learn that research is not just shown on paper, but can be used to explore the world around them.”


Field-testing the Quad Copter at Scripps Pier.

As part of this initiative, participants will work with research mentors based at UC San Diego and at Engineers for Exploration partner organizations. On the UC San Diego campus, participants will make use of the robust technological resources available at Calit2, including the institute’s StarCAVE and NexCAVE virtual-reality environments, and large-scale visualization systems. The program can also make available access to research labs in participating departments, including computer science, electrical and mechanical engineering, oceanography, and nanoengineering. High-definition videoconferencing will be used to connect participants with colleagues at National Geographic and at partner research institutions outside of the San Diego area.

It is this collaboration with scientists from program partners in San Diego and around the world that Engineers for Exploration REU coordinator Saura Naderi considers of paramount importance, as such teamwork will help program participants gain novel insights into the scientific process and the ins-and-outs of working in a research environment. “It’s rare for undergraduates to get to spend their summer working closely with scientists from some of the world’s top research institutions, let alone to plan and test out projects that they’ve created themselves,” said Naderi.

“Our program is intended to introduce students who are passionate about science — especially those students who may not have easy access to scientific internships or research opportunities — to the world of exploration through the wide reach of UC San Diego’s scientific community, and to increase the pipeline to graduate school,” Naderi added. “What we’re doing here is exciting because students have the chance to work on something that is both academically rigorous and fun.”

November 21, 2012 — The number of Ph.D. students participating in the Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3) has risen in the past year from six to 19, thanks to support from private donors and from the National Science Foundation through its Integrative Graduate Education and Research Traineeship (IGERT) grant to a project for engi- neering in cultural heritage diagnostics. The two-year-old IGERT project has also produced a dozen alumni — including four who have gone on to become professors in their respective disciplines.

In a letter to friends of the center, CISA3 Director Falko Kuester also noted that 45 undergraduate students are now participating in the UCSD-National Geographic Engineers for Exploration Program, led by Calit2 research scientist Albert Yu-Min Lin.

“CISA3 has gone through a remarkable growth period, which truly came into focus during our five-year anniversary celebration earlier this year,” said Kuester, thanking donors and friends of the center for their “participation, passion and support towards creating a future for the past.”

Many of the graduate students funded as IGERT Trainees are participating in a “Cyber-Archaeology Field School” and expedition in Jordan this fall, led by CISA3 Associate Director Thomas E. Levy. They are collecting data and artifacts for their research, while deploying and testing new tech- nologies and methodologies in the wild – technologies they have been de- veloping over the past year. The team returns to San Diego in late Novem- ber with what Kuester describes as “a series of remarkable discoveries.” Further evidence of CISA3’s global reach can be seen in the number of faculty and Ph.D. students giving talks and co-authoring papers for conferences stretching from Honolulu to Cyprus:

Christine Wittich and Richard Wood traveled to Cyprus this month to present research papers at the International Conference on Cultural Heritage (EUROMED 2012). The papers co-authored with Tara Hutchinson and Falko Kuester included “Characterizing Cracks in the Frescoes of Sala degli Elementi within Florence’s Palazzo Vecchio,” and “A Methodology for Integrative Documentation and Characterization of Culturally Important Statues to Support Seismic Analysis.”

Samantha Stout, Tom Wypych and Kathleen Bennallack traveled to the Denver X-ray Conference (DXC) in Colorado to present their research co-authored with Maurizio Seracini, Falko Kuester, Tara Hutchinson, Tom

Levy, Vid Petrovic in the Section in Applications for Cultural Heritage Research. The papers included “X-ray Fluorescience (XRF) As- sisted, Multispectral Imaging of Historic Drawings” (by Stout, Petrovic, Kuester and Seracini); “Progressive Structural Decomposition Analysis Using In-Situ Digital Radiography” (Wypych, Hutchinson and Kuester); and “Applications of PXRF for Cultural Heritage Diagnostics: Rapid In-Field Analysis at Khirbat Faynan” (Bennallack, Levy and anthropology grad student Lauren Hahn).

Yuma Matsui traveled to Washington, D.C., to present “ArchaeoSTOR Map: Publishing Archaeological Geodata on the Web,” co-authored with Aaron Gidding, Tom Levy, Falko Kuester, and Tom DeFanti. The paper was delivered at the International Conference on Computing for Geospatial Research and Applications (COM.Geo’12) in July.

Matsui also traveled to Honolulu to present research at the IEEE International Conference on Cloud Computing (CLOUD 2012). The paper on “Portable Data Management Cloud for Field Science” was co-authored by Gidding, Levy, Kuester and DeFanti.

Andrew Huynh presented research co-authored with Albert Lin at the 10th International Workshop on Remote Sensing for Disaster Management in Sendai, Japan.

Traveling to Edinburgh, Scotland, CISA3’s Maurizio Seracini – who leads the center’s high-profile projects in Italy and is co-principal investigator on the NSF IGERT project in engineering for cultural heritage diagnostics – delivered a talk on “The Secret Lives of Paintings” at TEDGLOBAL 2012. The event is the most prestigious of international TED conferences, and Seracini’s talk is now available via streaming video on the TED website.

Also over the summer, Tom Levy spoke at the TEDxSonomaCounty event about CISA3’s cyber-archaeology vision and how it is advancing and transforming archaeology. His talk on “Cyber-Archaeology in a Holy Land” is now available on YouTube’s TEDx channel.


IGERT Trainee and Structural Engineering Ph.D. student Christine Wittich overlooking Florence, where she studied Renaissance sculptures — and how to protect them from earthquakes.

CISA3 Director Falko Kuester also highlighted the work of Christine Wittich, a Ph.D. student in Structural Engineering. The IGERT Trainee is trying to develop accurate models to predict how statues and sculptures will behave during seismic events. In November, she begins a series of experiments using sculpture models on a UCSD shake table. Her work builds on research she conducted in 2011, when she studied culturally significant sculptures in Florence.

“Christine’s experiments are just one example of the impactful research being done by our IGERT Trainees,” said Kuester. “We believe that our future lies in this next generation of innovators.”

Kuester also thanked donors for their support, and laid out some areas where funding can make a difference, particularly in the lives of UC San Diego students: for example, $1,000 provides one quarter of foreign language and art history instruction for grad students doing field work abroad; $3,000 would fund one field expedition for a student, staffer or faculty member; and fellowships ranging from $10,000 (for one quarter of field school and lab work for a student), to $50,000 (for one full year of support for a graduate student).