AI-Generated Iceberg Models Could Help Sailors Navigate Arctic Waters

Cy Keener could see only the tip of the iceberg.

For years the University of Maryland associate professor of art had voyaged to the Arctic to gather the raw material for his digital artwork: data about its melting sea ice and shifting icebergs, collected using drones and computer software. But his visual interpretations captured just the matter above water, not beneath.

Keener, whose background includes some engineering, longed to create a rendering from top to base. It would be more than eye-catching or thought-provoking; it could also help scientists predict the path of icebergs, and mariners navigate ice-choked waters more safely.

He dispatched underwater drones during two trips to Alaska last year to collect underwater imagery of ice. Both outings were disasters. The cameras couldn’t see through the turbid waters. 

“We’re taking all the pictures we can, and they didn’t add up to anything,” Keener recalled.

Now, however, Keener has reason to be hopeful, following an introduction to UMD computer scientist Christopher Metzler. An expert in penetrating thick barriers like tissue or fog to see hidden objects, the assistant professor had designed an underwater camera and sonar device that worked in the lab but hadn’t been field-tested. 

In June, Keener affixed Metzler’s sonar to a 12-foot data-gathering instrument he designed in an art lab to resemble a periscope. Equipped with waterproofed cameras, the refined device was fabricated with custom-made aluminum, laser-cut acrylic plates, 3D printing and computer-aided design software. After traveling to Greenland and navigating alongside an iceberg, the professor’s research team wired the pole to a laptop, hung it over the hull and plunged it into the dark waters.

Eventually on the computer screen, clear images began to emerge, which became sharper after more processing back home. The sonar beam measured the depth and shape of the iceberg up to 60 feet deep while the cameras added resolution.

Metzler’s team will use artificial intelligence (AI) to stitch together the images and sonar data in 3D. The process, called inverse differential rendering, will rely on an algorithm designed by Metzler and fifth-year computer science Ph.D. student Kevin Zhang that fills in visual gaps between images that were gathered in intervals, animating details as fine as a ripple in the water. 

Sensor fusion—pairing sound with light—"is one of the foundational problems in computer vision, and inverse differential rendering is a really promising technique for solving it,” Metzler said of the project, which is funded by a grant from the Artificial Intelligence Interdisciplinary Institute at Maryland (AIM) and co-developed by third-year computer science Ph.D. student Isabelle Rathbun and former postdoctoral researcher Russell Shomberg. 

Ultimately, the AI-generated 3D models could help oceanographers understand and predict iceberg drift, allowing seafarers to plot more efficient routes in hazardous waters. In a 2024 report, the Arctic Institute, a Washington think tank, warned that recent spikes in extreme weather events, wind speeds and marine traffic “can affect ship stability and maneuverability, which affects safety.”

[Professor Combines Art and Data to Capture Beauty, Diversity of Arctic Ice]

The same UMD instrument and modeling will also be applied to sea ice that juts upward and downward to form ridges, blocking transport routes above and below the waterline. While modern-day satellite mapping is effective in measuring blockades above the surface, it struggles to ascertain what’s below.

Sinéad Farrell, a UMD associate professor of geographical, atmospheric and oceanic science who specializes in remote sensing, is working with Keener and Metzler through the support of the AIM grant to collect data on underwater ridges.

“Merging the field and satellite data, we will gain new information about the location, size, formation and breakup of sea ice ridges,” she said. New maps will also improve the safety of those working and living in the coastal sea-ice zone while potentially enhancing weather forecasts and bolstering the tourism and fishing sectors, she added.

For Keener, who teaches art courses, electronics and digital fabrication and has long been fascinated by natural phenomena seen and unseen, the project helps tackle “a really important scientific problem and a really beautiful art problem,” said the professor, who is enthusiastic about using the complete data for a later exhibit.

Metzler, meanwhile, appreciated the chance to turn his makeshift sonar device—“hacked together with two-by-fours,” he said—into something elegant and useful.

“We had the algorithms for a while, and then, finally, we get this well-put-together instrument you could trust in the field,” he said. “It’s super exciting.”