Within the Northeastern United States, the Gulf of Maine represents one of the biologically various marine ecosystems on the planet — residence to whales, sharks, jellyfish, herring, plankton, and a whole bunch of different species. However whilst this ecosystem helps wealthy biodiversity, it’s present process fast environmental change. The Gulf of Maine is warming sooner than 99 % of the world’s oceans, with penalties which can be nonetheless unfolding.
A brand new analysis initiative creating at MIT Sea Grant, referred to as LOBSTgER — quick for Studying Oceanic Bioecological Methods By way of Generative Representations — brings collectively synthetic intelligence and underwater images to doc the ocean life left weak to those modifications and share them with the general public in new visible methods. Co-led by underwater photographer and visiting artist at MIT Sea Grant Keith Ellenbogen and MIT mechanical engineering PhD scholar Andreas Mentzelopoulos, the venture explores how generative AI can increase scientific storytelling by constructing on field-based photographic knowledge.
Simply because the Nineteenth-century digital camera remodeled our skill to doc and reveal the pure world — capturing life with unprecedented element and bringing distant or hidden environments into view — generative AI marks a brand new frontier in visible storytelling. Like early images, AI opens a inventive and conceptual house, difficult how we outline authenticity and the way we talk scientific and inventive views.
Within the LOBSTgER venture, generative fashions are educated solely on a curated library of Ellenbogen’s authentic underwater images — every picture crafted with inventive intent, technical precision, correct species identification, and clear geographic context. By constructing a high-quality dataset grounded in real-world observations, the venture ensures that the ensuing imagery maintains each visible integrity and ecological relevance. As well as, LOBSTgER’s fashions are constructed utilizing customized code developed by Mentzelopoulos to guard the method and outputs from any potential biases from exterior knowledge or fashions. LOBSTgER’s generative AI builds upon actual images, increasing the researchers’ visible vocabulary to deepen the general public’s connection to the pure world.
This ocean sunfish (Mola mola) picture was generated by LOBSTgER’s unconditional fashions.
AI-generated picture: Keith Ellenbogen, Andreas Mentzelopoulos, and LOBSTgER.
At its coronary heart, LOBSTgER operates on the intersection of artwork, science, and expertise. The venture attracts from the visible language of images, the observational rigor of marine science, and the computational energy of generative AI. By uniting these disciplines, the group is just not solely creating new methods to visualise ocean life — they’re additionally reimagining how environmental tales will be instructed. This integrative strategy makes LOBSTgER each a analysis device and a inventive experiment — one which displays MIT’s long-standing custom of interdisciplinary innovation.
Underwater images in New England’s coastal waters is notoriously tough. Restricted visibility, swirling sediment, bubbles, and the unpredictable motion of marine life all pose fixed challenges. For the previous a number of years, Ellenbogen has navigated these challenges and is constructing a complete document of the area’s biodiversity by means of the venture, House to Sea: Visualizing New England’s Ocean Wilderness. This massive dataset of underwater photos gives the muse for coaching LOBSTgER’s generative AI fashions. The pictures span various angles, lighting situations, and animal behaviors, leading to a visible archive that’s each artistically putting and biologically correct.
Picture synthesis by way of reverse diffusion: This quick video exhibits the de-noising trajectory from Gaussian latent noise to photorealistic output utilizing LOBSTgER’s unconditional fashions. Iterative de-noising requires 1,000 ahead passes by means of the educated neural community.
Video: Keith Ellenbogen and Andreas Mentzelopoulos / MIT Sea Grant
LOBSTgER’s customized diffusion fashions are educated to copy not solely the biodiversity Ellenbogen paperwork, but in addition the inventive fashion he makes use of to seize it. By studying from 1000’s of actual underwater photos, the fashions internalize fine-grained particulars resembling pure lighting gradients, species-specific coloration, and even the atmospheric texture created by suspended particles and refracted daylight. The result’s imagery that not solely seems visually correct, but in addition feels immersive and transferring.
The fashions can each generate new, artificial, however scientifically correct photos unconditionally (i.e., requiring no person enter/steering), and improve actual images conditionally (i.e., image-to-image era). By integrating AI into the photographic workflow, Ellenbogen will have the ability to use these instruments to get well element in turbid water, alter lighting to emphasise key topics, and even simulate scenes that may be practically unattainable to seize within the area. The group additionally believes this strategy could profit different underwater photographers and picture editors dealing with comparable challenges. This hybrid methodology is designed to speed up the curation course of and allow storytellers to assemble a extra full and coherent visible narrative of life beneath the floor.
Left: Enhanced picture of an American lobster utilizing LOBSTgER’s image-to-image fashions. Proper: Authentic picture.
Left: AI genertated picture by Keith Ellenbogen, Andreas Mentzelopoulos, and LOBSTgER. Proper: Keith Ellenbogen
In a single key sequence, Ellenbogen captured high-resolution photos of lion’s mane jellyfish, blue sharks, American lobsters, and ocean sunfish (Mola mola) whereas free diving in coastal waters. “Getting a high-quality dataset is just not straightforward,” Ellenbogen says. “It requires a number of dives, missed alternatives, and unpredictable situations. However these challenges are a part of what makes underwater documentation each tough and rewarding.”
Mentzelopoulos has developed authentic code to coach a household of latent diffusion fashions for LOBSTgER grounded on Ellenbogen’s photos. Growing such fashions requires a excessive stage of technical experience, and coaching fashions from scratch is a fancy course of demanding a whole bunch of hours of computation and meticulous hyperparameter tuning.
The venture displays a parallel course of: area documentation by means of images and mannequin improvement by means of iterative coaching. Ellenbogen works within the area, capturing uncommon and fleeting encounters with marine animals; Mentzelopoulos works within the lab, translating these moments into machine-learning contexts that may prolong and reinterpret the visible language of the ocean.
“The aim isn’t to exchange images,” Mentzelopoulos says. “It’s to construct on and complement it — making the invisible seen, and serving to folks see environmental complexity in a manner that resonates each emotionally and intellectually. Our fashions goal to seize not simply organic realism, however the emotional cost that may drive real-world engagement and motion.”
LOBSTgER factors to a hybrid future that merges direct statement with technological interpretation. The group’s long-term aim is to develop a complete mannequin that may visualize a variety of species discovered within the Gulf of Maine and, ultimately, apply comparable strategies to marine ecosystems world wide.
The researchers counsel that images and generative AI type a continuum, somewhat than a battle. Pictures captures what’s — the feel, mild, and animal habits throughout precise encounters — whereas AI extends that imaginative and prescient past what’s seen, towards what might be understood, inferred, or imagined primarily based on scientific knowledge and inventive imaginative and prescient. Collectively, they provide a strong framework for speaking science by means of image-making.
In a area the place ecosystems are altering quickly, the act of visualizing turns into extra than simply documentation. It turns into a device for consciousness, engagement, and, in the end, conservation. LOBSTgER remains to be in its infancy, and the group seems to be ahead to sharing extra discoveries, photos, and insights because the venture evolves.
Reply from the lead picture: The left picture was generated utilizing utilizing LOBSTgER’s unconditional fashions and the precise picture is actual.
For extra data, contact Keith Ellenbogen and Andreas Mentzelopoulos.
Within the Northeastern United States, the Gulf of Maine represents one of the biologically various marine ecosystems on the planet — residence to whales, sharks, jellyfish, herring, plankton, and a whole bunch of different species. However whilst this ecosystem helps wealthy biodiversity, it’s present process fast environmental change. The Gulf of Maine is warming sooner than 99 % of the world’s oceans, with penalties which can be nonetheless unfolding.
A brand new analysis initiative creating at MIT Sea Grant, referred to as LOBSTgER — quick for Studying Oceanic Bioecological Methods By way of Generative Representations — brings collectively synthetic intelligence and underwater images to doc the ocean life left weak to those modifications and share them with the general public in new visible methods. Co-led by underwater photographer and visiting artist at MIT Sea Grant Keith Ellenbogen and MIT mechanical engineering PhD scholar Andreas Mentzelopoulos, the venture explores how generative AI can increase scientific storytelling by constructing on field-based photographic knowledge.
Simply because the Nineteenth-century digital camera remodeled our skill to doc and reveal the pure world — capturing life with unprecedented element and bringing distant or hidden environments into view — generative AI marks a brand new frontier in visible storytelling. Like early images, AI opens a inventive and conceptual house, difficult how we outline authenticity and the way we talk scientific and inventive views.
Within the LOBSTgER venture, generative fashions are educated solely on a curated library of Ellenbogen’s authentic underwater images — every picture crafted with inventive intent, technical precision, correct species identification, and clear geographic context. By constructing a high-quality dataset grounded in real-world observations, the venture ensures that the ensuing imagery maintains each visible integrity and ecological relevance. As well as, LOBSTgER’s fashions are constructed utilizing customized code developed by Mentzelopoulos to guard the method and outputs from any potential biases from exterior knowledge or fashions. LOBSTgER’s generative AI builds upon actual images, increasing the researchers’ visible vocabulary to deepen the general public’s connection to the pure world.
This ocean sunfish (Mola mola) picture was generated by LOBSTgER’s unconditional fashions.
AI-generated picture: Keith Ellenbogen, Andreas Mentzelopoulos, and LOBSTgER.
At its coronary heart, LOBSTgER operates on the intersection of artwork, science, and expertise. The venture attracts from the visible language of images, the observational rigor of marine science, and the computational energy of generative AI. By uniting these disciplines, the group is just not solely creating new methods to visualise ocean life — they’re additionally reimagining how environmental tales will be instructed. This integrative strategy makes LOBSTgER each a analysis device and a inventive experiment — one which displays MIT’s long-standing custom of interdisciplinary innovation.
Underwater images in New England’s coastal waters is notoriously tough. Restricted visibility, swirling sediment, bubbles, and the unpredictable motion of marine life all pose fixed challenges. For the previous a number of years, Ellenbogen has navigated these challenges and is constructing a complete document of the area’s biodiversity by means of the venture, House to Sea: Visualizing New England’s Ocean Wilderness. This massive dataset of underwater photos gives the muse for coaching LOBSTgER’s generative AI fashions. The pictures span various angles, lighting situations, and animal behaviors, leading to a visible archive that’s each artistically putting and biologically correct.
Picture synthesis by way of reverse diffusion: This quick video exhibits the de-noising trajectory from Gaussian latent noise to photorealistic output utilizing LOBSTgER’s unconditional fashions. Iterative de-noising requires 1,000 ahead passes by means of the educated neural community.
Video: Keith Ellenbogen and Andreas Mentzelopoulos / MIT Sea Grant
LOBSTgER’s customized diffusion fashions are educated to copy not solely the biodiversity Ellenbogen paperwork, but in addition the inventive fashion he makes use of to seize it. By studying from 1000’s of actual underwater photos, the fashions internalize fine-grained particulars resembling pure lighting gradients, species-specific coloration, and even the atmospheric texture created by suspended particles and refracted daylight. The result’s imagery that not solely seems visually correct, but in addition feels immersive and transferring.
The fashions can each generate new, artificial, however scientifically correct photos unconditionally (i.e., requiring no person enter/steering), and improve actual images conditionally (i.e., image-to-image era). By integrating AI into the photographic workflow, Ellenbogen will have the ability to use these instruments to get well element in turbid water, alter lighting to emphasise key topics, and even simulate scenes that may be practically unattainable to seize within the area. The group additionally believes this strategy could profit different underwater photographers and picture editors dealing with comparable challenges. This hybrid methodology is designed to speed up the curation course of and allow storytellers to assemble a extra full and coherent visible narrative of life beneath the floor.
Left: Enhanced picture of an American lobster utilizing LOBSTgER’s image-to-image fashions. Proper: Authentic picture.
Left: AI genertated picture by Keith Ellenbogen, Andreas Mentzelopoulos, and LOBSTgER. Proper: Keith Ellenbogen
In a single key sequence, Ellenbogen captured high-resolution photos of lion’s mane jellyfish, blue sharks, American lobsters, and ocean sunfish (Mola mola) whereas free diving in coastal waters. “Getting a high-quality dataset is just not straightforward,” Ellenbogen says. “It requires a number of dives, missed alternatives, and unpredictable situations. However these challenges are a part of what makes underwater documentation each tough and rewarding.”
Mentzelopoulos has developed authentic code to coach a household of latent diffusion fashions for LOBSTgER grounded on Ellenbogen’s photos. Growing such fashions requires a excessive stage of technical experience, and coaching fashions from scratch is a fancy course of demanding a whole bunch of hours of computation and meticulous hyperparameter tuning.
The venture displays a parallel course of: area documentation by means of images and mannequin improvement by means of iterative coaching. Ellenbogen works within the area, capturing uncommon and fleeting encounters with marine animals; Mentzelopoulos works within the lab, translating these moments into machine-learning contexts that may prolong and reinterpret the visible language of the ocean.
“The aim isn’t to exchange images,” Mentzelopoulos says. “It’s to construct on and complement it — making the invisible seen, and serving to folks see environmental complexity in a manner that resonates each emotionally and intellectually. Our fashions goal to seize not simply organic realism, however the emotional cost that may drive real-world engagement and motion.”
LOBSTgER factors to a hybrid future that merges direct statement with technological interpretation. The group’s long-term aim is to develop a complete mannequin that may visualize a variety of species discovered within the Gulf of Maine and, ultimately, apply comparable strategies to marine ecosystems world wide.
The researchers counsel that images and generative AI type a continuum, somewhat than a battle. Pictures captures what’s — the feel, mild, and animal habits throughout precise encounters — whereas AI extends that imaginative and prescient past what’s seen, towards what might be understood, inferred, or imagined primarily based on scientific knowledge and inventive imaginative and prescient. Collectively, they provide a strong framework for speaking science by means of image-making.
In a area the place ecosystems are altering quickly, the act of visualizing turns into extra than simply documentation. It turns into a device for consciousness, engagement, and, in the end, conservation. LOBSTgER remains to be in its infancy, and the group seems to be ahead to sharing extra discoveries, photos, and insights because the venture evolves.
Reply from the lead picture: The left picture was generated utilizing utilizing LOBSTgER’s unconditional fashions and the precise picture is actual.
For extra data, contact Keith Ellenbogen and Andreas Mentzelopoulos.
