TEM Agent AI System Simplifies and Accelerates Electron Microscope Experiments

[Original Article]

Operating one of the world's most powerful electron microscopes requires weeks of training. At the Lawrence Berkeley National Laboratory, researchers can now describe experiments in simple English and let an AI agent carry out the operations. The system, named TEM Agent, was developed at the Molecular Foundry, a facility located at Berkeley Lab. It controls a transmission electron microscope, which can reveal the precise locations of atoms within materials, determining their strength, conductivity, energy storage capacity in batteries, and how drugs interact with cells.

The microscope at the Molecular Foundry can image structures about half the size of a hydrogen atom, attracting scientists from universities and companies who come specifically to use the equipment. The learning curve is quite steep; some visiting researchers, despite having taken a semester-long course on electron microscopy, still require weeks of practical training. Even experts encounter difficulties in data processing: a single detector can generate about 700GB of data in just 15 seconds, far exceeding manual processing capabilities. This creates a bottleneck. While the equipment continues to become more powerful, the number of people skilled in operating and analyzing it has not kept pace.

AI Enhances Electron Microscope Efficiency

The team already had automated routines to handle repetitive tasks, such as imaging large quantities of nanoparticles. However, as the lead scientist of the study, Peter Elues, noted, these routines function like factory assembly lines, executing predefined procedures and halting when unexpected situations arise, with no ability for non-robotic engineers to modify them. The team decided not to start over but to rewrite these routines using a model context protocol, an open standard that provides the AI with a fixed set of specific preset operations rather than open-ended control. With this toolkit, TEM Agent can string together steps, even for requests that have not been programmed.

Researchers describe experiments in simple English, and the agent adjusts magnification, tilts samples, refocuses beams, and links dozens of small steps as needed. The team tested this system in electron tomography, a technique that reconstructs a three-dimensional view of samples from tilted images, which typically takes hours. Initially, the agent confused the rotation of the sample with the rotation of the scanning direction, but a brief written description resolved the issue. Stefanie Rebet, a research scientist at the Molecular Foundry, subsequently pushed it into more experimental domains, including phase diffraction, a technique that reconstructs high-resolution images from diffraction patterns, which usually requires considerable expertise to set up.

Natural Language Interface Facilitates Communication

Connecting AI to scientific hardware is more challenging than linking it to literature or data pipelines, Rebet stated, while the protocol provides the team with a means to bridge both. The agent is also connected to the National Energy Research Scientific Computing Center, a supercomputing facility of the Department of Energy, as well as the data platform Crucible at the Molecular Foundry, which organizes the metadata of experiments. Raw data can be processed in seconds, and researchers can inquire about the settings they used on samples weeks earlier without having to sift through documents or notebooks.

Morgan Wall, the lead developer of Crucible, stated that users can communicate through a single interface in natural language instead of operating five scripts across five computers simultaneously. This approach can be extended to other instruments with open interfaces, and Wall's team is helping other nanoscale research centers adopt Crucible. The release states that the goal is not to replace expertise but to expand it, providing a faster pathway from ideas to experiments for both novices and experienced researchers. Elues anticipates that such assistants will democratize this work. The research is supported by the Department of Energy's Office of Science and its results are published in the journal npj Computational Materials.