Produced by the Office of Marketing and Communications
UMD Researchers Use AI and Robotics to Save Struggling Industry After Diseases (and Pirates) Plunder Cheasapeake’s Bounty
Photo by John T. Consoli
IT’S A ROUGH DAY FOR ROBOTS. The glare on the surface of the Choptank River, the green murk below and strong gusts stirring up the mile-and-a-half-wide channel all leave Keshav Rajasekaran Ph.D. ’22 struggling in an open boat to control a pair of underwater drones as they skitter along the bottom, or noisily emerge with propellers spraying saltwater.
The mechanical engineering student peels off his T-shirt. Not to tan or take a dip—he just needs to be able to see. He secures one end around his head with a ball cap and drapes the other over a laptop he’s using to monitor the microwave oven-size aquatic drones fitted with optical and acoustic sensors. In the makeshift shade, Rajasekaran spots an empty riverbed.
“Sand, sand, sand,” he mutters in frustration.
The goal of the University of Maryland-led project for which he’s gathering data today is a Chesapeake Bay that—instead of being barren—practically bursts with oysters. Researchers in the project funded by the U.S. Department of Agriculture (USDA) and the National Science Foundation (NSF) envision a future when the bivalve mollusks again approach numbers they did when explorer John Smith had to carefully navigate around jagged oyster reefs protruding from the water in the bay and tributaries like the Choptank 400 years ago. But that abundance—and many of the water-quality benefits these natural filters bring—has disappeared amid overharvesting, mismanagement and devastating diseases.
Central to this comeback, the researchers believe, is the adaptation of technologies revolutionizing land-based agriculture. So-called “precision farming” often uses robotic aerial drones mounted with sensors to scan fields for factors like topography and soil content, giving farmers data to plan seeding or harvesting schedules, or apply fertilizers in exact amounts where needed.
Overall project leader Miao Yu, a mechanical engineering professor whose specialty is robotic sensing, says Maryland’s archaic oyster trade needs modern methods to augment its traditions.
“The shellfish industry in the Chesapeake Bay is mostly using the same technology from 200 years ago, with most things done very laboriously by hand,” she says. “It has not evolved, not adapted like terrestrial farming.” Decisions as basic as where to plant oyster larvae and steer a boat to find fully grown ones are still based on intuition and experience, rather than objective data.
The need for such data is why her student, Rajasekaran, is out in a boat on this steamy June morning. He and his research partner for the day, Alan Williams, a master’s student studying with fisheries scientist and project co-leader Matthew Gray of the University of Maryland Center for Environmental Science, are trying to direct the robots to gather visual and sonar imagery of oysters.
Hunched over his laptop, Rajasekaran says, “The idea is: The sonar can see the oysters through the turbid water from a distance although with low resolution, while the camera has high resolution, but can only see the oysters close up—so we plan to use both,” he says. With all that and a type of artificial intelligence known as machine learning, Yu and collaborators are teaching a computer system to recognize sonar signatures of marketable oysters in the project funded by a $10 million USDA grant. One day, a drone—even a swarm of them—could zoom through an area and quickly provide an oyster farmer with a map showing a host of metrics, including where oysters are ready to harvest, where they’re immature, and empty zones.
The latter is what Rajasekaran and Williams have stumbled into—an inconvenience that illustrates the overall problem they hope to help solve. Richie Long, a staffer at the Horn Point Lab who’s captaining the boat today, steers it slowly around a several-acre oyster “lease” held by a Maryland waterman happy to provide an area for testing. At one point, the two students’ excitement rises.
“Oh my God! There!” Rajasekaran shouts. “I think it’s an oyster! Or is it a rock?”
VIEWED SOLELY THROUGH the lens of the present, oysters are an odd proposition: slick, silvery flesh in a shell with a taste that causes many people to reach for the Tabasco (while others extol their subtle flavor differences). On the half shell, they can command ludicrous prices, given that a raw bar chef simply pops them open and makes one quick cut before serving them as it’s been done on riverbanks and around campfires for thousands of years—still alive and slurped from the shell.
Most oysters, however, are more modest fare—sold with meat removed and packed for use in various dishes. A revival could make them economically akin to poultry, which was once too pricey to serve on every table, says Don Webster, a University of Maryland Extension principal agent focused on aquaculture, and one of the co-leaders of the project. “As I tell my growers, ‘I want to see Wendy’s, Popeye’s and Burger King arguing over who’s got the best oyster sandwich,’” he says.
If this sounds unlikely, rewind 200 years. In the late 18th and early 19th centuries, oysters were not a connoisseur’s item, but a national food staple consumed by people of all classes—so common that leftovers were sometimes spread on fields as fertilizer and shells used to build roads and even create new land in the bay. (It’s the foundation for parts of Crisfield and other cities.) In Maryland, they became part of the bedrock of the young state’s economy as local watermen plied the fertile oyster beds with hand tongs, emulating the technique used by Native tribes of the region.
But a trio of new technologies—refrigeration, railroads and canning—emerged in the early to mid-1800s, and oysters exploded in popularity as they became available beyond coastal areas. The bounty of the seemingly inexhaustible Chesapeake caught the notice of distant mariners who had stripped their own waters clean.
“Northerners, who had largely depleted their local oyster beds, looked at those down here like the gold fields of California, and they brought the towed dredge from New England,” Webster says. “That changed everything.”
Pulled behind Chesapeake sailing vessels, the teethed dredges efficiently tore through oyster beds and scooped big loads up to the surface at a rate far exceeding what watermen with tongs could accomplish. Soon the Chesapeake was undergoing a transformation.
As related in historian John R. Wennerstein’s “The Oyster Wars of the Chesapeake Bay,” Maryland passed laws in the mid-1800s against out-of-state oysterers—“plundering Yankee drudgers,” as they were called. And yet the rapacious dredging continued. In 1868, Maryland established a watergoing police force, nicknamed “The Oyster Navy,” to stop unlawful harvesting by oyster pirates.
Pitched gun battles erupted on the bay between the police and the pirates, Webster says. Some vessels carried cannons and rifles, killings were commonplace, and outraged oystermen plotted assassinations of rivals and officials.
Even some Maryland watermen complained of overzealous policing by the navy, which later became the Maryland Natural Resources Police. But with the nation’s hunger for oysters growing, nothing could stop the mayhem, or the enormous harvest.
By the late 1800s, some 15 million bushels of oysters a year were being pulled from the Maryland section of the bay, each holding perhaps 275 oysters. By comparison, this year’s Maryland oyster harvest—the biggest in 35 years, likely thanks to favorable weather—was about 540,000 bushels, or less than 4% of the historical high.
As zoologist and Johns Hopkins University Professor William K. Brooks declared in “The Oyster,” his 1891 natural history of the species written as oyster populations were approaching collapse, “(F)or many years we strove to hide even from ourselves, that our indifference and lack of foresight, and our blind trust in our natural advantages, have brought this grand inheritance to the verge of ruin.”
MORE ROBUST MANAGEMENT helped tame the crazed harvesting, and Maryland oystermen brought in 2 to 3 million bushels annually from 1930 until the 1980s, supporting dozens of shucking houses and packing plants along with thousands of families involved in the seafood trade in rural working waterfront communities.
As the 20th century began drawing to a close, a new disaster was creeping up the Chesapeake from warmer waters: two diseases, Haplosporidium nelsoni and Perkinsus marinus, which are popularly known as MSX and dermo, respectively. They are harmless to humans, but kill oysters in the shell before they reach marketable size.
“After I graduated from college and started working with my father in 1978, the landings were well over a million bushels a year,” says Casey Todd, owner and operator of MeTompkin Bay Oyster Company in Crisfield, Md. “Then before long, because of the disease, it dropped to 150,000 bushels.”
The company started by his father—the late Ira Todd ’39, who graduated with a UMD business degree—never stopped shucking oysters, even when live, edible ones were few and far between.
Todd has traveled the country to see how oyster farming works elsewhere, and he envies producers in the Pacific Northwest who can saunter out across exposed oyster beds when the tide drops, and even harvest that way. But Maryland oyster beds 10 to 20 feet down might as well be in a trench, they’re so hard to see.
Harvesting here means dredging with care—not with the devil-may-care destructiveness of 19th-century “drudgers.” But dragging the 3-and-a-half-feet-wide device where it doesn’t belong can still kill young oysters, he says, and sometimes there’s no way to avoid it.
“The water around here isn’t clear—you can’t see or judge the shape of your beds,” he says. “So if there’s a way to know what’s on my beds without going around taking a bite here and bite there with a pair of tongs, I’m interested.”
Oyster visibility is the project’s first order of business, but it could one day lead to a range of revolutionary practices, Yu says: underwater robots planting tiny oysters growing on shells, or “spat,” in perfect spots, or harvesting with delicate precision that takes only viable shellfish and leaves growing ones and the bay bottom undisturbed. “People think we’re dreaming if we mention all that,” she admits.
Another aspect of the project, led by bioengineering Professor Yang Tao, is examining how to plot perfect dredging paths to avoid immature oysters while using as little fuel as possible. Other collaborators hail from the University of Maryland Eastern Shore, Louisiana State University, Pacific Shellfish Institute, Virginia Polytechnic Institute and State University, Georgia Tech and the Greater College Park-based Fraunhofer Center for Experimental Software Engineering. Together, the technologies could be applied to other types of seafood, from mussels to crabs.
But the first step is hardly an easy one.
“If you take everything you use for vision in the air and you move it underwater, it doesn’t work there,” says Yiannis Aloimonos, a professor of computer science who specializes in computer vision. “There’s sediment moving around everywhere, the light is different. It’s a mess—and it raises a lot of very interesting questions.”
Aloimonos and students are working on a parallel path to Yu; rather than focusing on incorporating sonar to extend sensory capabilities, he’s incorporating AI to create a vision system able to spot live oysters while disregarding dead ones. His team too has struggled with the availability of research footage of oysters, so it created a workaround with AI-simulated imagery. “I can show you real oysters and simulated ones, and (the viewer) can’t tell the difference,” he says. “It gives us much more latitude to train our system.”
Aloimonos says his goal is to get the emerging technology into the hands of the watermen who’ve worked the bay for centuries—keeping a traditional trade alive with futuristic technology.
To Webster, advanced aquaculture technologies could completely reshape the Maryland oyster industry. His goal is 100,000 acres (compared to around 8,000 today) of well-managed oyster leases, supporting small and large producers alike.
“We’d be adding another agricultural county to Maryland, but underwater,” he says. “And from there, you could develop an industry that would produce 10 or 15 million bushels a year or higher than ever.”
BENNIE HORSEMAN, 33-year-old co-owner of Madison Bay Seafood, points to the hallway outside the business office of the facility he and his brother renovated and operate next to the Little Choptank River. “There was a tree growing right there when we took over the building,” he says. The historic oyster shucking house had been abandoned and overrun by foliage since the ’80s, when dermo and MSX began their rampage in Maryland.
Until recently, Horseman was focused on a family business restoring fishing boats—fixing engines, sheathing old wooden hulls with fiberglass. But his grandfather and generations before were watermen, making the business move feel natural. In April 2021, in an act of supreme faith in the future of the industry, his new seafood company incorporated, and today it ships truckloads of oysters, crabs, fish and other catches daily to restaurants and markets in the Baltimore-D.C. region and along the East Coast.
“I think I might be the youngest lunatic in this business,” he chuckles.
In addition to buying from watermen, the company operates a small fleet of boats and participates in a state-sponsored program to seed oyster spat in public oyster beds, as well as provide the service to those who lease oystering grounds.
On the last day of June, Horseman drives about 20 miles northwest to tiny Wittman, Md., located on a peninsula on the bay just west of the Eastern Shore tourist town of St. Michaels, to meet with his competitor and friend, Nick Hargrove. He’s another uncharacteristically young business operator in what’s increasingly an aging trade, managing his family’s Wittman Wharf Seafood company, which like Horseman’s, launched in recent years in a formerly shuttered facility.
During the summer, Maryland’s public oyster grounds are closed, but those like Hargrove with private leases can harvest oysters, so the two head down Harris Creek in a classic Chesapeake Bay fishing boat known as a deadrise, fitted with a crane and winch. Hargrove pulls a lever to drop his dredge off the starboard side, and soon the small boat is straining to pull the dredge across the rough bottom, gunwale dipping near the surface. He stops to reel in a load that he and Horseman dump onto a table on the deck.
The bottom here is far different from the one where Keshav Rajasekaren steered a robot a few weeks earlier. They sort through the contents of dredge— perhaps a dozen live oysters along with empty shells—and then chuck the lot back in.
Hargrove’s checking the growth in an area he’s previously seeded. It doesn’t make much sense that he must do so by scraping the bottom—potentially killing some of his crop—with destructive tools developed hundreds of years ago. If the UMD-led project can drag oyster farming into the 21st century through robotic-enabled aquaculture, producers like him and Horseman are on board.
“We’re young, we’re in this for the long haul, we want to succeed, we want the whole industry to succeed,” Hargrove says, rapid-fire. “So yeah, we need technology. We need modernization. We need to be able to see what we’ve got.”
Artificial Intelligence Bioengineering Computer Science Mechanical Engineering Robotics Research
A. James Clark School of Engineering College of Computer, Mathematical, and Natural Sciences
Maryland Today is produced by the Office of Marketing and Communications for the University of Maryland community on weekdays during the academic year, except for university holidays.
Faculty, staff and students receive the daily Maryland Today e-newsletter. To be added to the subscription list, sign up here:Subscribe