How visual learning happens in the brain

The wiring and rewiring of the brain never ends. Neural pathways are constantly being reshaped as we interact with the world and learn new things. At MIT’s McGovern Institute for Brain Research and York University in Toronto, Ontario, scientists are combining detailed analysis of brain activity with computational modeling to better understand that change.

McGovern Institute postdoc Lynn Sörensen, McGovern investigator and MIT Professor James DiCarlo, and York University Assistant Professor Kohitij Kar, worked together to compare what happened when animals and an artificial neural network with brain-like architecture were trained to visually identify the same objects. As the model’s performance improved, it reorganized itself in ways that closely paralleled changes the team detected in the animal brains.

Their open-access work, reported July 8 in the journal Nature Communications, shows how changes in visual processing support animals’ ability to learn to discriminate new kinds of objects. By modeling these changes, the researchers hope to better predict how training reshapes perception, which could one day inform educational strategies for a wide range of learners.

Subtle changes

Learning about a new object calls on many parts of the brain. Visual-processing areas work together to make sense of information taken in through the eyes, then communicate with other brain areas to give the visual information meaning and guide behavior. Multiple parts of this system likely change during learning, and the research team wanted a clearer understanding of how that change is distributed.

Neuroscientists have debated how much change occurs in the brain’s visual-processing areas when an animal learns to recognize new objects. Some suspected that visual-processing pathways remain largely unchanged during learning to avoid broadly disrupting visual perception, but others have reported changes in activity within dedicated visual-processing areas with this kind of learning in humans and other primates.  

To take a closer look, the team focused on neural activity in a key component of the brain’s visual object-processing network, the inferior temporal (IT) cortex. By the time visual information reaches the IT cortex, key object features are clearly represented — so much so that it’s possible to “decode” what object the subject is seeing and even predict what errors it’s likely to make in identifying it, simply by analyzing patterns of neural activity there.

The team recorded neural activity in the IT cortex from animals as they looked at and identified images of objects. Some of the animals were untrained, so the images they saw had little meaning to them. Others had already learned to identify similar objects, so they could usually discriminate between elephants, chairs, and other select objects, even when those objects were presented at different sizes, from different angles, or against different backgrounds than the ones they had seen before.

The broad pattern of activity in the IT cortex was largely similar in trained and untrained animals, suggesting that learning had not dramatically rewritten this high-level visual representation. Still, the group found subtle but reliable differences in the way neurons in the IT cortex responded to images in animals that had learned to recognize the kinds of objects they were shown, compared to the untrained animals.

Modeling learning

The group turned to computational models to investigate how those modest changes might contribute to learning. Sörensen trained a suite of artificial neural networks whose internal components had been mapped to the IT cortex to identify the same categories of objects the animals had seen. The models were designed to learn using gradient descent, meaning they continually improved their accuracy by adjusting their parameters in response to errors.

Only some of the animal models showed learning behavior that matched that of the subjects. In those that did, the IT-like stage changed in ways that resembled the learning-related changes the researchers had observed in the IT cortex of trained animals.

While gradient descent is commonly used to train artificial intelligence, it is generally considered biologically implausible as a direct model of how the brain learns. The researchers say the strong match in learning effects between the animals and their model demonstrates that these kinds of artificial neural networks can offer insights into biological learning at a useful level of abstraction, even if the brain does not learn in the same way.

“This shows that you can actually build in silico versions of future experiments,” Sörensen says. “I think that gives us this playground of asking ‘what if’ questions — and potentially predicting new things that go beyond the experimenter’s intuition.”

Most of the changes that allowed for learning in the model occurred outside of the IT cortex. “This tells us that there is a lot between the area we recorded from and the final behavioral readout that needs to change during this process,” Kar says. He adds that the team’s model will be useful as researchers look more deeply into how downstream brain areas contribute to learning.

The researchers stress that their study allowed more granular measurements of brain activity than would be possible in humans, and because the animal brains are organized similarly to our own, their experiments have direct relevance to human learning. They say understanding the impact of plasticity in the subjects’ IT cortex could help researchers design new learning strategies for humans.

“Our prior conceptual working model of you learning new objects was that your brain makes changes to synaptic connections that are largely downstream of your visual system, so you don’t destroy your visual system,” says DiCarlo, who is also the Peter de Florez Professor of Brain and Cognitive Sciences and director of the MIT Siegel Family Quest for Intelligence. “You wouldn’t want your whole visual system to become an elephant detector [just because you’ve learned to identify an elephant]. But this study went beyond that to say actually, when you learn ‘elephant,’ your IT does change a little bit to make it a little more relevant to elephants.”

That likely has consequences for recognizing other visual features, too. Subtle changes in the IT cortex that support elephant recognition might also make you better at identifying things other than elephants, DiCarlo says. Likewise, the same changes might make it a little harder to identify something else.

These kinds of consequences may be difficult to predict intuitively, but become obvious with computational modeling. For instance, the team’s models revealed that after learning to recognize new objects, the IT cortex contained more information about objects’ locations. By providing insights like these, models could aid the design of more effective training strategies for visual tasks, including for people with altered sensory processing, who may learn from visual information in atypical ways.

de MIT News https://ift.tt/SLB4GaQ

Can AI build a jet engine? JARVIS Challenge tests role of AI copilots in tough-tech engineering

Artificial intelligence has rapidly transformed software engineering. Generative AI and large language models (LLMs) can create huge volumes of code and documentation; machine-learning algorithms can monitor performance and detect security vulnerabilities. But when the task is to conceive, design, and make a complex physical system such as a jet engine, are those AI tools equally transformative?

This past semester, the JARVIS Challenge (Jet-engine AI Research and Validation Intensive Sprint) set out to explore whether AI can compress the design-build-test cycle, asking MIT undergraduates to discover whether AI can help them to build faster and better. 

“The JARVIS challenge showed that AI can substantially accelerate safety-critical hardware engineering, but engineering judgment remains the decisive differentiator. An AI-native engineer is not defined by using AI, but by leading it — knowing when to trust it, when to challenge it, and how to translate AI outputs into working hardware. Manufacturing — not engineering design or analysis — remained the fundamental rate-limiting step,” says Professor Zolti Spakovszky, director of the MIT Gas Turbine Laboratory.

The teams, the tools, the task

The challenge gave undergraduates four weeks to design, fabricate, assemble, and test a small gas turbine aero engine, using AI as their primary engineering partner. The objective: build a “JARVIS-class” single-spool jet engine producing 50–100 pounds of thrust, running on Jet-A, and completing five 60-second runs. Teams had total freedom over design, materials, and fabrication. 

Representing nearly every department in the School of Engineering, 31 students organized into seven teams, ranging from all first-years to senior-heavy groups. Many of the competitors initially had little experience in turbomachinery, compressible flows, or, in the case of the younger students, even thermodynamics. Many had never seen the inside of a gas turbine before signing up to build one.  

At their disposal: MIT’s machine shops and manufacturing vendors; commercial software including Concepts NREC, SolidWorks, and ABAQUS; and various test rigs for characterizing and assembling individual components.

The teams also had access to MIT Parley, a newly launched platform that aggregates frontier large language models through a single interface. Through Parley, JARVIS leads could see directly how the students were using the AI tools, including their prompts, the cost per prompt, the specific LLMs being used, and other critical information. The JARVIS leads secured early access to Parley for all participants, and with financial support from MIT Lincoln Laboratory, the Department of Mechanical Engineering, and corporate sponsors Safran, Voyager Technologies, and Beehive Industries, students had access to essentially unlimited use of AI.

The sponsors were drawn by recruiting interest and genuine curiosity about how AI might reshape engineering workflows. 

“We see this as the future of engineering,” Ryan (Hal) Hefron of Voyager Technologies told the students. “You’re honing skills that are not just nice to have — they’re going to be the future baseline in the engineering workforce.”

Vincent Garnier, managing director of Safran Tech, watched the competition unfold with excitement. “JARVIS was a genuine experiment, a learning endeavor. We frankly didn’t know what to expect, from the students or from the AI models. What struck me coming from the students was: first, the enthusiasm to explore; then, as the project developed, they all came to the cool-headed realization of what AI could or could not help them with, and then almost instantly adapted for that,” he says. “It makes me confident that this generation of leading engineers will probably not fall prey to easy and shortsighted use of AI, and will do so by keeping ever more in contact with experiments — physical or thought experiments.”

The faculty leadership — professors Zachary Cordero, Zolti Spakovszky, Masha Folk, and Andreea Bobu of the Department of Aeronautics and Astronautics, along with Lincoln Laboratory engineers and a team of teaching assistants — were there to ensure safety. In weekly progress reviews, they would critically evaluate the student progress and assess how the students were using AI.

Spakovszky developed a careful technique for guiding teams in the right direction without giving away answers or providing help. After a team’s presentation, he might ask: “Do you know what a rabbet fit is? Take in the comment.”

Where AI helps and hurts

By the end of week 1, one team withdrew from the competition; the others had, with varying degrees of success, developed an initial design for their gas turbines. Different teams used AI to summarize textbooks, teach them to use design software, source vendors, create Excel sheets, answer specific questions, find references, and create comparative analysis between design decisions. One team created an agent in Parley and tasked it with serving as their project manager. 

By week 2, teams had to start working on detailed CAD designs, ordering parts, and prototyping their combustors. This is where the teams started to hit limitations in their use of AI. While Claude and ChatGPT were good at offering design alternatives and filling knowledge gaps, teams found that the hallucinations, sycophancy, and lack of physical understanding that have become notorious features of generative AI were undermining their confidence and slowing them down. 

“AI is a helpful tool, great at finding information, helping organize things, and can write well, but it can’t do design,” says Elizabeth Tupaj, a member of team 811 Crew. “The moment the engineer doesn’t know what is going on and the AI is in charge is the moment the design becomes unreliable, at least with AI at its present capabilities.”

Teaching assistant John Zhang notes, “seeing this firsthand with the students reminded me how much first impressions matter. If the students couldn’t get answers from the AI early on, they quickly grew frustrated and formed a lasting opinion that precluded them from using it later.” 

In the final weeks, the finalists hit another obstacle no AI could solve: working with vendors. “AI searches found vendors we had no rapport with, who had no interest in our tight timeline,” students reported. “The vendors who came through were the ones our team had personal relationships with.”

Of the three finalists, only Fast and Fractured achieved first-attempt ignition of their mini-combustor. The team had used AI heavily for trade studies and architecture comparisons, arriving at a viable design despite none of them having prior gas turbine experience.

“The JARVIS Challenge showed what’s possible when you combine AI-enabled design with motivated students and a culture of rapid experimentation,” says Masha Folk, the Charles Stark Draper Career Development Professor of Aeronautics and Astronautics. “The moment that stood out most was when the first student-designed combustor was installed on the test stand. It ignited flawlessly, ramped to full power, transitioned to dual-fuel operation, and then sustained stable combustion on 100 percent Jet-A fuel. This was proof that we can dramatically accelerate the cycle of design, build, and test while giving students hands-on experience with a real engineering challenge.”

At the vanguard of AI-native engineering

By the end of May, the two more senior teams – Fast and Fractured and 811 Crew – had completed full engine tests. Fast and Fractured, with their AI-assisted design, were delayed by vendor headaches week after week, but finally made it to test. Unfortunately, their hot fire was cut short when the rotor rubbed and seized against the stationary housing. Team 811 Crew, however, who had more exposure to turbomachinery and propulsion concepts going into the competition, emerged victorious. Their engine started, successfully transitioned to Jet-A, and generated net thrust. 

“As we stood there with the air-starter, hearing their engines spool up and watching them spit fire, it felt like my heart was racing out of my chest. There were so many ways it could go wrong! What these students accomplished in such a short time span is nothing short of amazing,” says PhD student Joe Chiapperi. 

The 811 team had been resistant to using AI throughout the competition, trusting instead to their fundamentals and teamwork. “We had people who were at least somewhat familiar with the design software, mechanical engineers who knew how to build anything, and aerospace engineers who had taken classes on the design of gas turbine engines specifically,” says Tupaj. 

From the start of the JARVIS Challenge, younger students used Parley more frequently and cleverly, while the juniors and seniors leveraged deeper experience. 

“JARVIS taught me that getting value from AI takes two things: enough expertise to judge what it tells you and catch it when it’s wrong, and enough curiosity to actually lean on it where it could help,” says Professor Andreea Bobu. “The team that moved fastest in the sprint was experienced and leaned heavily on AI to get there. The team that eventually won was more resistant to AI; they had the expertise, but that skepticism made them slower. The sweet spot seems to be knowing enough to stay in charge of the tool, and being eager enough to pick it up in the first place. To me, that’s the real opportunity ahead: training the next generation of engineers who have the judgment to direct these AI tools and the instinct to reach for them.”

The competition’s clearest finding: engineering experience is a multiplier, and the human factor remains a vital element. Mastering the first principles and fundamental concepts breeds good engineering judgment and the ability to navigate strings of tough decisions in the face of incomplete information. And when it comes to building safety-critical physical systems, nothing can replace human hands and human accountability. 

“JARVIS has shown that AI copilots can have a multiplicative effect on engineering productivity, with judgment and first-principles thinking serving as the key differentiators among teams,” adds teaching assistant Kyle Woody. 

But the implications of AI in aerospace are significant. If small teams using well-managed AI copilots can compress design-build-test cycles from years to weeks, the consequences for workforce structure, R&D timelines, and competitive dynamics could be substantial. The students who tackled the JARVIS Challenge are among the first engineers to grapple with those stakes not as a thought experiment, but in a machine shop, with a jet engine on the test stand.

“JARVIS highlighted the power of AI in the design of physical systems,” says Cordero, associate director of the MIT Gas Turbine Laboratory. “But it also showed that the key to unlocking that power is education, through coursework, internships, and hands-on extracurriculars like MIT Motorsports and Rocket Team. Performance in JARVIS correlated strongly with year in school. My main takeaway is that in the AI era, education is more valuable than ever.”

de MIT News https://ift.tt/qByusMw

MIT engineers find a precise way to grow artificial blood vessels

Tissue engineers are finding ways to grow living organs and tissues from cells, with the aim of replacing diseased and damaged counterparts in the body. Scientists have successfully grown artificial muscles, livers, kidneys, skin, and other tissues. But there’s been no reliable way to engineer precisely patterned networks of blood vessels, some of which can be finer than a human hair. 

Without a vascular network to deliver nutrients, any artificial tissues, no matter how life-like, can’t function. 

Now MIT engineers have found they can engineer and control the growth of blood vessels by mechanically stretching them. 

The team has built a human “blood vessel on a chip,” composed of a central artery made from human endothelial cells, that is embedded in a gel that also contains a small magnet. The researchers studied how the main artery responded as they jostled the gel back and forth using an external magnet to move the magnet embedded within the gel.

They found that the simple mechanical action of repeatedly jostling the artery stimulated the artery to sprout other, smaller capillaries. By changing the direction in which the artery is jostled or stretched, the researchers could redirect the growing new vessels. And stretching the artery by various degrees influenced how many more new vessels sprouted. 

Their results, reported in the Proceedings of the National Academy of Sciences, offer scientists a new way to engineer artificial blood vessels and program the patterns in which they grow. 

“Healthy tissues depend on organized blood vessel networks, but state-of-the-art protocols don't enable fabricating such networks within engineered tissues,” says Ritu Raman, associate professor of mechanical engineering at MIT and the study’s co-lead author. “The ability to program blood vessel growth with physical cues may enable reproducible and scalable fabrication of engineered tissues that can be implanted in the body to restore function after debilitating disease or injury.”

The study’s MIT co-authors include Sina Kheiri, Jessica Shah, Shashaank Venkatesh, and Roger Kamm, along with Peiyuan Chai and Ryan Flynn at Harvard University. 

“Moving is good”

Blood vessels are tricky to grow and control using conventional fabrication techniques. While 3D printers can produce vessels at the scale of major arteries and veins, the technology is not precise enough to print intricate networks of much finer, thread-like capillaries. Scientists have had some success with growing blood vessels from individual cells, by cultivating them in Petri dishes filled with nutrients and growth factors. But controlling how and where they grow remains a challenge. 

“You can try to pattern chemical cues, like growth factors, to direct where vessels grow, but you can’t do this very precisely,” Raman says. “We thus need other types of patternable cues that can help us build tissues with organized vessels.”

She and her students wondered whether they could grow and control new blood vessels using a protocol they previously developed to grow artificial muscles and nerves. In their previous works, the team engineered a small chip filled with a gel that they infused with nutrients and growth factors. They embedded a small magnet within the gel, and then carpeted the surface of the gel with live muscle or neuron cells. They then manipulated an external magnet to pull the embedded magnet, and the cell-covered gel, back and forth. This work revealed that mechanical “exercise,” pulling the cells back and forth, directly influenced how the cells grew. 

In their new work, the team used a similar setup to see if they could grow and control new blood vessels. 

The researchers built a “blood-vessel-on-a-chip,” smaller than a postage stamp, and filled it with a similar nutrient-rich gel containing a small magnet. They poked a thin tube lengthwise through the gel to create a hollow channel, and coated the channel with live endothelial cells, which naturally grow and fuse to form blood vessels in the body. Once the cells took on the channel’s shape, they started sprouting new, capillary-like vessels in the gel. 

Placing the device under a motorized stage fitted with small, suspended magnets, the researchers moved the magnets back and forth in different directions, and by various degrees, and observed whether and how blood vessels sprouted from the central artery in response. 

“The main takeaway is: Stretching the blood vessel back and forth seems to enhance the number of new capillaries that grow,” Raman says. 

If the main artery were simply left alone in the gel, it would grow some new vessels in random locations along its length. But when the artery was jostled, significantly more vessels sprouted. When the team used the magnets to stretch the gel back and forth, by 5 percent of the gel’s total width, many new vessels grew out from the main artery. When they stretched by 15 percent, fewer vessels sprouted, but those that did grew longer. And when the team changed the direction of stretching, the new vessels followed in response, taking turns and following the pattern of the team’s mechanical stimulation.

“We’re finding that moving is good, which is always the takeaway of everything we do in our lab,” Raman says. “Mechanical forces play an important role in our bodies. That means that if you want to grow more or less vessels, or shorter or longer vessels, or vessels in certain directions, we now know how to do that.”

A gatekeeping gene

The researchers went a step further to investigate why blood vessels grow in response to mechanical forces. To do so, they looked to gene editing, and the role of one particular gene: Piezo1. 

Raman had recently attended a talk by molecular biologist Ardem Patapoutian. In 2021, Patapoutian received the Nobel Prize in Physiology or Medicine for his discovery of ion channels in cell membranes that open and close in response to mechanical pressure. These channels, named PIEZO1 and PIEZO2, act as a cell’s gatekeepers, controlling what goes in and what comes out of a cell. Both types of channels, Patapoutian found, are regulated by their respective genes, also named PIEZO1 and PIEZO2. 

After his talk, Raman showed Patapoutian her group’s experimental results, which showed a connection between blood vessel growth and mechanical stimulation. Patapoutian in turn proposed that the explanation could be the PIEZO1 channel; by mechanically exercising the central artery, Raman may have been stimulating ion channels in the artery’s cells to open, triggering new blood vessels to grow. 

To test this hypothesis, Raman looked to knock down the PIEZO1 gene. If this gene were less active, and fewer blood vessels grew as a result, then it would mean that blood vessels do indeed grow in response to mechanical stimulation, and specifically, through the activation of PIEZO1 ion channels. 

The team repeated their experiments, this time with endothelial cells that were genetically edited to suppress the PIEZO1 gene. Sure enough, they observed that significantly fewer new blood vessels sprouted, even as they mechanically exercised the central artery.

Now that the team has found a way to grow and control blood vessel growth, they plan to apply the protocol to grow organized networks of vessels to supply artificial organs and tissues. “We are now investigating how precisely patterning blood vessel growth can help improve muscle function,” says co-author Jessica Shah.

This work was supported, in part, by the U.S. Department of War Army Research Office Early Career Program and PECASE Grant, and a Department of War DURIP Program Grant.

de MIT News https://ift.tt/mMBT8FK

AI agents create virtual playgrounds to help robots get crucial training data

Robots walking down the street, surrounded by astounded onlookers, is an increasingly common sight. But these machines aren’t yet the do-it-all assistants you’d want working in a kitchen or factory, and a major bottleneck is data. Much like humans, robots learn best by experience. The challenge is that it’s labor-intensive and time-consuming to physically teach these machines so many actions across different settings. 

“One natural idea is to use simulation as a training ground. While there has been significant progress over the last few years in the physics engines that power robotics simulators, one of the remaining challenges has been creating sufficiently rich and diverse simulation content to capture the complexity of the real world,” says Russ Tedrake, the Toyota Professor of Electrical Engineering and Computer Science (EECS), Aeronautics and Astronautics, and Mechanical Engineering at MIT, and a principal investigator at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).

It turns out that AI agents, or semi-autonomous programs that “think” and complete well-defined tasks, could help produce the lifelike virtual settings that robots need. The new “SceneSmith” system developed by researchers at MIT CSAIL and Toyota Research Institute uses three agents to piece together the objects, walls, and overall look of a 3D scene. Its recreations of indoor spaces such as restaurants, bedrooms, and hotels are more realistic and detailed than prior systems, helping robots practice skills and try out different ways of doing tasks before they’re powered on. In turn, engineers save time on real-world testing.

The agents have a sense of how everyday places are supposed to look because they each call on a multi-modal system called a vision-language model (VLM), specifically the state-of-the-art VLM GPT-5.2. It’s trained on lots of text and images from the internet to handle more visual prompts. This advanced model gives each agent a sort of spatial knowledge: First, a “designer” agent generates the elements of a scene, then a “critic” advises whether it looks realistic, and finally, an “orchestrator” manages their back-and-forth, deciding when the design is done. Once the three VLMs wrap up their creative collaboration, the scene is ready to load directly into physics simulation software.

“We’ve found that the system can construct 3D scenes the way a human designer would,” says MIT EECS PhD student Nicholas Pfaff, a CSAIL researcher and a lead author on a paper with Tedrake presenting the work. “We made over 1,300 scenes using a leading VLM that has internet-scale priors, and it made insanely creative and diverse arrangements. I hadn’t taught the system to do that in the prompts; it just improvised.”

Talk to my agent

Thanks to VLM agents, you can ask SceneSmith to do things like “generate a garage with a car, a workbench, tires stacked in the corner, and a ladder against the wall,” and get a virtual playground rich with objects a robot can tinker with. These rooms are decorated with up to six times more items per scene than prior methods, making them great for helping robots learn skills such as putting a cup in the sink, placing fruit on plates, and moving a soda can from a shelf to a table.

With so many rich virtual environments handy, you can evaluate whether your robot is ready for deployment without so much trial and error in the physical world. The researchers tested out different action plans (also called “policies”) in SceneSmith’s digital worlds, generating 100 unique spaces in the process. A VLM agent evaluated each attempt, and it found the robot’s plans were faulty, with the machine often failing at its chores. Humans agreed with the model’s verdicts over 99 percent of the time, which could help roboticists weed out flawed approaches in simulation before a robot moves in the real world.

But how realistic are these virtual worlds, really? It can be difficult to prove outright, so the researchers approached the question from several angles. The most telling test: they dropped a pretrained robot policy — an AI controller trained largely on real-world data, which had never seen a SceneSmith scene — into the generated environments. In one test, users told the system to “take the apple from the bowl and place it onto the cutting board,” and the simulated robot did exactly that. If the scenes didn’t closely resemble the real settings the policy had learned from, it simply wouldn’t have worked. 

The team also teleoperated robots through the virtual spaces, guiding them to open cabinets, put away bottles, and navigate between rooms. Their experiments revealed that the environments hold up under sustained physical interaction, expanding beyond visual inspection.

Behind the scenes

The agents that SceneSmith uses each have a well-defined role in the generative process, fleshing out scenes in stages. They essentially create a floor plan and bring it to life. 

Let’s say you wanted to create a scene similar to the first floor of a house. The “designer” VLM would start with a general layout, which the “critic” reviews, and then the “orchestrator” signs off. The agents repeat this approach for each step: adding furniture, placing objects on walls and then ceilings, and finally, dropping in objects that robots can manipulate. For example, the VLMs can add cabinets that the robots can open and close — an articulated item, which prior baselines didn’t often have.

At each stage, the second VLM ensures the scene is practical, advising that a bathtub is removed from a living room, for example. The third VLM ensures a high-quality scene is generated, even taking the design process a few turns back if the visuals aren’t up to par. Once the three VLMs wrap up their creative collaboration, the mechanics of the physical world are added via simulation software.

With a sound understanding of how rooms should look, where objects should be placed, and real-world physics, SceneSmith has a noticeable edge over prior methods. Compared to scene-generation baselines such as “HSM” and “Holodeck,” SceneSmith made environments with more objects, including a private office, a pottery store, and even a Minecraft-themed gaming room.

SceneSmith was also a favorite among over 200 users. They found the system’s visuals to be more realistic over 90 percent of the time. They also observed that, generally speaking, it followed prompts more closely than other approaches did. In other words, it was the best at generating the virtual playgrounds users actually wanted to see.

A system of many talents

Realism, diversity, and richness are all strong suits for SceneSmith, even when it comes to generating individual 3D objects. You can prompt it to create a rolling serving cart, and it’ll make a 2D image that it then turns into a detailed model with physical properties like mass, friction, and inertia.

Such a detailed process does come with a speed trade-off, though. It can take multiple hours to produce a single scene because the agents are creating and closely scrutinizing each object. With more computing power, the system could see dramatic increases in efficiency. CSAIL engineers are also hoping to expand to deformable objects (like sponges), should extensive 3D libraries become available.

“SceneSmith represents a significant advance in this regard by providing an agentic framework for generating simulation-ready indoor environments just from a simple text prompt,” says Jeremy Binagia, an applied scientist at Amazon Robotics who wasn’t involved in the research. “It advances the state of the art in several ways, including pushing the limits of the density of objects in the simulated environment, ensuring that all of the objects are physically accurate (versus just being visually realistic), and creating assets that are not constrained to a fixed library, since they can be generated via text-to-3D.”

Pfaff and Tedrake wrote the paper with Thomas Cohn SM ’24, an MIT PhD student and CSAIL researcher; and Toyota Research Institute roboticists Sergey Zakharov and Rick Cory SM ’08, PhD ’10. Their work was supported, in part, by Amazon, the U.S. Office of Naval Research, the Toyota Research Institute, and the U.S. National Science Foundation.

The team presented their findings as a spotlight at last week’s International Conference on Machine Learning. 

de MIT News https://ift.tt/xf30i69

Tiny infrared chip could improve detection of gases and heat

Infrared cameras can be used to spot useful information that our eyes can’t see, such as gases escaping from a pipeline, chemicals in the atmosphere, or heat leaking from a building. But sensing infrared light in sophisticated ways still requires expensive and bulky systems.

Now MIT researchers have created a chip-based optical device that can dynamically control incoming infrared light, to act as a tunable lens that gathers additional information for infrared cameras. Each microscopic pixel of the device’s lens can control infrared light independently, allowing it to change its focus and help cameras detect different signals without moving parts.

The system is described in a paper published in Nature Communications. The researchers also explain how they built a lab-scale demonstration using mostly conventional manufacturing processes in a semiconductor chip factory, suggesting the approach could be implemented at industrial scales.

The technology could enable compact, tunable infrared cameras for more dynamic thermal imaging, chemical sensing, pollution monitoring, and even new kinds of optical computing.

“This could give us more information as we study space, or help with environmental protections where you want to monitor for specific compounds in the atmosphere,” explains first author Cosmin-Constantin Popescu PhD ’25. “Thermal imaging is another application, and you can think of military applications where night vision goggles are currently being used. Basically, a lot of organic molecules absorb in the mid-infrared wavelength, and you could use this system to detect them.”

Joining Popescu on the paper are MIT PhD students Maarten Robbert Anton Peters and Khoi Phuong Dao; Dynasil company scientists Oleg Maksimov and Harish Bhandari; University of Central Florida PhD candidate Kathleen Richardson and scientist Rashi Sharma; University of Washington Professor Arka Majumdar; Korea Advanced Institute of Science and Technology Associate Professor Hyun Jung Kim; MIT postdoc Rui Chen; Luigi Ranno PhD ’25; Brian Mills ’20, PhD ’26; Draper Laboratory scientist Dennis Calahan; MIT principal investigator Tian Gu; and Juejun Hu, MIT’s John F. Elliott Professor of Materials Science and Engineering.

Chip-based lenses

In recent years, researchers have developed ways to dynamically control light by etching tiny, precise patterns on transparent materials known as “metasurfaces,” which could enable more compact, programmable cameras and other advanced optical devices.

Hu’s research group at MIT has experimented with a class of metasurfaces that shift from solid to liquid after heat is applied. The phase changes can be harnessed to control how the materials interact with light. In 2021, Hu and collaborators created a miniature lens that could adjust its focus to different depths through such phase changes.

The device worked reliably, but it could only adjust focus all at once across the entire material, which is how most metasurfaces work. For their new study the researchers wanted to build on that approach to control light independently at each microscopic pixel of the material.

“Most active metasurfaces trying to do single-pixel tuning need wires going to every pixel, and how you route the wires becomes a big issue,” Hu explains. “The best approach so far has been one-dimensional pixel control with a bunch of wires.”

The researchers also wanted to create a system that worked with the mid-infrared wavelength of light, which the human eye can’t see but is useful for detecting heat signatures and molecules including methane and propane. Mid-infrared detection devices are already used to detect gas leaks and study Earth’s atmosphere, and for a number of defense and aerospace applications.

To build their system, the researchers adapted an approach commonly used in displays in which two layers of neatly packed copper wires are placed on top of each other perpendicularly. Below the wires is a layer of doped silicon that generates heat at the cross points of the wires and sits on top of the phase-change material. The silicon’s heat is used to switch each pixel of the material between crystalline and amorphous structures, which changes how the material interacts with the infrared light coming in. The silicon also includes a diode selector, which helps prevent unintended currents from leaking through neighboring pixels.

“We did some calculations showing this architecture allows us to scale to potentially millions of pixels without having any issues with the [unintended] currents,” Hu says. “The key innovation is this crossbar architecture, which creates a scalable way to increase the pixel-level switching of metasurfaces. We didn’t invent this architecture — it’s used in displays — but it’s the first time anyone’s used it for active phase-change metasurfaces to show you can get pixel-level control. People have been working toward two-dimensional pixel-level control for a long time, and it’s the first time anyone’s implemented it.”

The researchers worked with equipment in MIT.nano and with a factory that manufactures semiconductor chips, ultimately creating a two-dimensional system that featured a 6-by-6 metasurface pixel array. They tested their system and found it could switch on and off reliably.

“We found this mesh architecture to be very resilient,” Popescu says. “You don’t want these materials to switch once and not work anymore. You want it to switch a large number of times: maybe tens of thousands of times or more.”

Scaling up

The researchers say integrating part of their system’s design into existing semiconductor manufacturing should help it move beyond a research prototype.

“As you want to scale up, you need something that’s part of a consistent process, and that’s why chip foundry manufacturing becomes so important,” Hu says. “Working with a semiconductor foundry with well-defined process control is very powerful. It also allows you to implement each of the components into a single efficient process.”

The researchers are working to add more pixels to their array and develop more robust versions of their system so that it can start capturing more infrared information.

“In lots of cases when you’re taking images, you have prior knowledge of what you’re looking for,” Hu says. “You might be looking for a human in a dark room, or some specific features in an image, like a tree, and that prior information can be useful because now you can configure this system to specifically highlight those features.”

Hu also notes that researchers have used metasurfaces to emulate computational neural networks that power AI systems, though he notes that applications could be farther away from taking hold.

“This could enable more effective optical computing, where metasurfaces are used to encode network weights in neural networks,” Hu explains. “When light passes through the material, it interacts with the metasurface, and that information gets encoded in such a way that you can infer computational results. Researchers have already used this approach to emulate very complex neural networks.”

The work was supported, in part, by the U.S. Air Force, the U.S. National Science Foundation, the National Research Foundation of Korea, and the Draper Scholar Program.

de MIT News https://ift.tt/I5RCWfv

Discovery could lead to brighter, more energy-efficient digital displays

A new study led by MIT researchers could drive the development of more energy-efficient digital displays — such as flat-screen TVs, augmented and virtual reality headsets, smartphone screens, medical imaging devices, and even large-area ambient lighting surfaces — that also generate richer, brighter colors.

The MIT scientists, in collaboration with researchers at Samsung, studied the microscopic changes that occur inside LEDs that utilize electrically excited quantum dots, which are precisely shaped nanoscale semiconductor particles that emit extremely pure colored light. 

Quantum dots are currently used in some of the computer and television displays with the best picture quality available. The efficiency of these displays could be further improved, and their manufacturing process further simplified, if the quantum dots could be electrically excited, as was first demonstrated in the quantum dot LED (QD-LED) structures over 20 years ago. 

But limitations on the operating lifespans of these QD-LEDs have prevented their widespread use in commercial applications.

The new study shows how encapsulating QD-LEDs in an acrylate-based resin can extend their lifespan by minimizing the physical degradation that would otherwise occur during QD-LED operation. 

The researchers demonstrated that encapsulating QD-LEDs with a resin layer using a simple, scalable process boosts stability and performance. In some devices, resin encapsulation enabled a 5,000-fold lifespan improvement. Importantly, their study reveals the fundamental reasons resin encapsulation is effective.

“The insights into how and why quantum dot LEDs get modified during their operation open the possibility of fixing everything that holds back commercialization of QD-LED displays. This technology can provide a light source like never before — pure in color, paper thin, and of large area, transforming how we produce both displays and general lighting,” says Vladimir Bulović, the Fariborz Maseeh (1990) Professor of Emerging Technology, principal investigator in the Research Laboratory of Electronics (RLE), director of MIT.nano, and senior author of this study.

He is joined on the paper by lead author Ruiqi Zhang, an electrical engineering and computer science graduate student; Moungi Bawendi, the Lester Wolfe Professor of Chemistry; and other colleagues at MIT and Samsung SAIT. The research appears today in Science Advances.

A blue bottleneck

This paper draws on foundational work by Bawendi, who shared the Nobel Prize in Chemistry in 2023 for discovering and synthesizing quantum dots, and engineering work by Bulović, who joined MIT in 2000, when he began collaborating with Bawendi to make efficient LED displays using quantum dots. 

Conventional LED displays utilize thousands of tiny lightbulbs that generate the red, green, and blue light needed to create the perception of any color on the visible spectrum. More advanced OLED screens, which Bulović was developing through his graduate work at Princeton University, utilize electrically excited, glowing organic molecules instead of light bulbs.

Bulović, Bawendi, and others at MIT sought to replace the organic molecules with quantum dots, which emit purer red, green, and blue light in a more energy-efficient manner.

“With quantum dots, the color quality of the screen would be more visually appealing and more optically flexible. One can mix and match those quantum dot colors more precisely to generate any color that is needed,” says Bulović.

Their collaboration generated a series of inventions on quantum dot LED technologies, leading to the launch of the startup QD Vision, which successfully commercialized the first-ever displays containing quantum dots. In 2016, QD Vision was acquired by Samsung, which incorporated a less efficient form of quantum dot technology into their “QLED” displays.

Although they are more energy-efficient, electrically excited QD-LEDs have still not been commercialized, particularly since the limited lifetime of the blue QD-LED does not meet the requirements of commercial displays.

“The blue quantum dot LEDs are 50 to 100 times less stable than their red and green counterparts. If you use them in an LED display, your TV might last for just a few months before it stops working. We wanted to understand what is different about the blue quantum dot LEDs,” Zhang says.

A nanoscale investigation

He and his collaborators developed a technique to slice a tiny QD-LED in nanoscale-thin slivers, revealing the device cross-section. They examined these cross-sections under extremely powerful microscopes at MIT.nano. This precise method allowed them to see what happens at the nanoscale to the ultrathin layers of materials stacked inside the QD-LED.

They explored the structural and chemical changes that occurred in each layer of red and blue QD-LEDs by comparing cross-sections of freshly made devices to cross-sections of devices that were operated on overdrive. The researchers found that during operation, the three core functional layers that enable blue QD-LEDs to glow are degraded, with modified morphology and reduced thickness. 

The distinct quantum dots also get merged together, losing their shape. This layer thinning and coarsening is caused, in part, by the release of extra hydrogen and oxygen during operation.

“We don’t yet know exactly where these extra elements are coming from — there are so many possibilities. But we definitely don’t want extra hydrogen and oxygen in the device,” Zhang says.

To prevent this degradation, they utilized a technique sometimes adopted by industry. They encapsulated the QD-LEDs with an acrylate-based resin.

They discovered that this encapsulation technique suppresses the release of the hydrogen and oxygen and inhibits some of the degradation that changes the morphology of the layers of the blue QD-LED. 

“For the first time, we have insights into the details of what happens inside these structures of many mixed and layered materials that form the QD-LED. No one knew this before,” Bulović says.

This encapsulation strategy, which is a cost-effective and scalable technique, led to an eightfold improvement in the lifetime of red QD-LEDs and more than a 5,000-fold lifetime improvement in blue QD-LEDs.

The researchers believe the resin prevents the formation of moisture in the cloud of gases that surrounds the quantum dot. That moisture likely causes the QD-LED to degrade. 

However, their experiments revealed that resin encapsulation does not eliminate all sources of degradation. 

The researchers are now exploring the addition of extra layers to QD-LEDs that could further improve efficiency and lifespan. They also plan to build on the lessons learned in this study to increase the stability of QD-LEDs for other applications. 

“This version of quantum dot LEDs would be better than anything that exists now — simpler to make, more efficient, and higher performing. This could open vistas into many more ways of thinking about this technology, not just for the sake of displays or lighting, but also for sensors, lasers, and so on,” says Bulović.

This work was funded by the Samsung Advanced Institute of Technology. The research was carried out, in part, using MIT.nano facilities.

de MIT News https://ift.tt/9thzAoW

Beyond the pitch: The founder’s journey

The path to launching and growing a startup can be full of twists and turns. For a budding entrepreneur, gaining perspective from those who have already experienced the journey can be incredibly valuable, and highly inspirational. 

“There are so many amazing entrepreneurial stories among our alumni. We want to bring those stories to our students and our community and build networks with our incredible alumni founders,” says John Hart, the Class of 1922 Professor and head of the Department of Mechanical Engineering (MechE). “Through the Founder’s Journey class and other new programs, we want to cultivate interest in entrepreneurship among our students and expand opportunities to bring MechE-born technologies to the world.” 

According to a 2015 report on MIT’s global entrepreneurial impact, there are more than 30,000 active companies founded by MIT alumni worldwide, employing some 4.6 million people. Marina Hatsopoulos SM ’93, founding CEO of Z Corp., an early market leader in 3D printing, said one of the aims of the course was to show students they don’t need to reinvent everything. “So much of this has been done before. I want them to understand that this is a well-trod path.” 

Class 2.S977/2.S979 (Founder’s Journey: Launching and Scaling Hardware Startups) explores real-life challenges of startups focused on building and scaling hardware technologies. First held in spring 2025, the inaugural class invited students to “find and activate their entrepreneurial energy” through the lens of challenges faced by founders and their teams at various stages in development of new hardware-focused companies — ranging from fundraising to supply chain development, and much more. 

Each week of the class was structured around a key challenge faced during the development and growth of a hardware startup, presented by the instructors and guest speaker. The speakers were founders of companies in robotics, energy, 3D printing, consumer products, and other frontier technologies. Students engaged through preparing questions for the speakers and participating in follow-on discussions and reflective exercises throughout the semester. 

Ken Zolot, senior lecturer at MIT, and Hatsopoulous co-led the class and developed it along with Hart. Hart, who was among the alumni speakers in the course’s first iteration, also spoke to the class about his experience as a co-founder of VulcanForms, which began through collaboration with fellow co-founder Martin Feldmann MEng ’14. 

The other alumni speakers included Mick Mountz (Kiva/Amazon); Jon Hirschtick (Solidworks/Onshape); Max Lobovsky (Formlabs); Elise Strobach (Aeroshield); Greg Mark (Markforged); Seemantini Nadkarni (Coalesenz); Eran Egozy (Harmonix); Renuka Babu (DOTS Technology); Davide Marini (Inkbit); Loewen Cavill (Amira); and Colin Angle (iRobot).

Colin Angle ’89, SM ’91, co-founder of iRobot

Colin Angle ’89, SM ’91, co-founder and former CEO of iRobot, now CEO and co-founder of Familiar Machines and Magic, identified a passion for building things early on. 

“This idea that you can create something from nothing, that you can have an idea and not just draw it, but build it and make it real, is something I’ve always loved,” he says. “MIT had such a strong, hands-on ethos, and that really, powerfully resonated.”

While living in the Alpha Delta Phi Fraternity house at MIT, Angle watched several companies get their start (by his count, five multimillion-dollar companies were started by his fraternity brothers during his time in the house). Seeing others do it helped to demystify the process. 

He started iRobot in his living room, beginning at first not with a product concept, but a grand vision. “We’re supposed to have robots. So, if not us, who? And if not now, when? It was a magical day.” 

iRobot may be best known for the Roomba, an autonomous robotic vacuum cleaner, but through the years the company also sent robots to Afghanistan (saving thousands of lives with the Pack Bot tactical mobile robot) and explored the Great Pyramid in Giza live on National Geographic. 

“The joy I have taken from my entrepreneurial journey has been the ability to build bigger things, from building teams to building a company capable of building something far beyond what I could have ever imagined doing myself … we created inventions that no one thought possible, simply because we believed we could.”

Elise Strobach SM ’17, PhD ’20, CEO and co-founder of AeroShield

Elise Strobach SM ’17, PhD ’20 is CEO and co-founder of AeroShield Materials. The company, co-founded with Kyle Wilke PhD ’19 and Aaron Baskerville-Bridges SM ’20, MBA ’20, develops super-insulating transparent window inserts with technology based on transparent silica aerogels developed by Strobach while she was completing her PhD in Professor Evelyn Wang’s lab.

“I wasn’t thinking of myself as an entrepreneur at that time, but looking back, that’s definitely where that seed was planted,” says Strobach. As entrepreneurs, she says, “We have the … freedom to find the best problem to solve and to continue to seek the best way to solve that problem.”

Aerogels, which were first invented almost 100 years ago and were first commercialized by NASA to insulate equipment in space, had a hazy blue tint that limited their use in certain applications. The aerogel material created by Strobach and her team is completely see-through, creating a variety of new everyday applications. The company recently achieved another milestone, with their work on display at the Smithsonian National Air and Space Museum in Washington.

“You don’t have to know everything to start. You just have to know that this is what you want to do and just get started.”

Maxim Lobovsky SM ’11, CEO and co-founder of FormLabs

Maxim Lobovsky SM ’11 was already working on 3D printers when he came to MIT to study at the MIT Media Lab. As he was finishing his master’s degree, he saw an opportunity to build something new.  

Lobovsky, with fellow Media Lab graduates David Cranor SM ’11 and Natan Linder SM ’11, founded Formlabs, a developer and manufacturer of 3D printing technology. The trio set out to build a professional-level 3D printer, but a significant cost reduction and one that would be easier to use than what was then available on the market. At a time when 3D printers could cost $100,000 or more, Formlabs’ product started around $3,000.

“We definitely built Formlabs in a classic, disruptive innovation path,” Lobovsky says. They achieved the cost reduction through several different ways, including replacing technology developed in the 1980s with modern consumer electronics components like the laser diodes that were developed for Blu-ray Disc players, and with “just a lot of clever engineering.” 

It was a long grind to raise the first round of funding, he says. The team participated in MIT’s 100K competition and pitched their idea to many potential investors (with limited success, initially). Their big break came in the form of an overheard conversation. 

“As someone who is naturally introverted, shy engineer … a really important lesson [was] that, sometimes, you can get lucky,” he says. “Sometimes talking loudly at a restaurant is actually a good way to get things going.” 

Lobovsky and one of his co-founders were having dinner with a potential investor at Legal Seafoods in Harvard Square. The pitch to the initial investor didn’t go well, but Mitch Kapor, the founder of Lotus Software and an early pioneer in the PC industry overheard the conversation, and he ended up leading Formlabs’ first round of funding. 

Today, Formlabs is the largest supplier of professional stereolithography and selective laser sintering 3D printers in the world. 

Jon Hirschtick ’83, SM ’83, co-founder of SolidWorks and Onshape

Jon Hirschtick ’83, SM ’83, co-founder of SolidWorks and Onshape, says the first time he can remember thinking about starting a company was when he was an undergraduate. 

“I had heard about startups, and it sounded like a lot of things that I was drawn to … a sense of being able to realize your vision, express yourself; a sense of excitement, of making money, and even the idea of a chaotic environment,” he says. 

Hirschtick has spent over four decades building computer-aided design (CAD) software, starting as an intern at MIT in 1981 and continuing that work today. “I thought, ‘hey, the world could use this software.’ It’ll be a better place with the software that I envisioned.”

He refers to CAD as a meta product design. “We’re designing a product that other people use to design products, and that’s just really cool to me.” 

“I think startups just fit me,” he says. “The excitement, the idea of trying to solve a lot of problems at the same time. MIT is a place of problem-solving ... and a startup is a place where there’s lots of problems to solve.” He adds that a lot of big companies are doing new things, but “startups are always doing things.”

He says most anything today that is a manufactured product is modeled in CAD first. “If you’re interested and excited by product development, then building a CAD system lets you get involved in the world’s product development.”

“Nobody knows for sure when they start a company whether it’s going to be successful or not. If it were, if there was a way of knowing for sure, then there wouldn’t be all these classes in entrepreneurship. They’d just tell you the secret. There’s always risk. Visions and hallucinations, they look and feel the same. You only find out which is which once you really try to realize them.”

A version of this story appears in the 2026 issue of MechE Connects, the Department of Mechanical Engineering’s magazine. 

de MIT News https://ift.tt/3uNUm9t