Navigating the Future of Microscopy Image Analysis with AI
A written panel* with ZEISS technology experts
In the dynamic landscape of technological advancements, the surge of Artificial Intelligence (AI) in diverse industries is one of the hottest and most transformative topics of our era. As we navigate the intricate realms of digital innovation, AI has emerged as a beacon of efficiency and precision. Microscopy is not immune to its profound influence, bringing about substantial changes in processes such as image acquisition, sample detection, image processing, storage, and image analysis.
The exploration of AI for microscopy image analysis promises to unlock the full potential of visual data. We asked experts at ZEISS Microscopy to tell us how they think AI for microscopy and image analysis will unfold.
--
Challenges microscopists face today
Microscopists currently face a myriad of challenges that span the spectrum of research and product development in life and material science. Dr. Sreenivas Bhattiprolu (Sreeni) and Phillip Holroyd see the struggle with intricate image analysis tasks, complex system interfaces and the overwhelming surge of large-volume automated data from advanced microscopes such as the ZEISS Lattice Lightsheet 7, AxioScan 7 and CellDiscoverer 7.
Dr. Fahimeh Farhadifard speaks of the time-consuming manual correlation of data from different microscopes, while Dr. Alexander Freytag (Alex) additionally laments the lack of standardized protocols, insufficient data of the highest quality, and limited expert resources as roadblocks. Falko Löffler emphasizes the need for user-friendly software to tackle big data, diverse analysis tasks, and time constraints.
It becomes clear that researchers must cope with storing, extracting, analyzing, and sharing huge amounts of data. Dr. Sebastian Rhode (Sebi) agrees and sees multifaceted challenges, but he fears that AI, though promising, risks being oversold as a cure-all without a true grasp of real needs.
How can AI help microscopists?
Sreeni sees benefits in automating image analysis using AI models. Labeling images becomes a breeze for training them, and you can get good results with few annotations. Even if you initially invest some time in annotating your images, you save time in the long run because you can repeatedly use the model. For example, in High Content Analysis (HCA), where data is consistent, AI can accelerate the journey from image to insights.
Martin Kuttge points out that AI can make life easier for newbie microscopists, as they can use image segmentation models trained by seasoned professionals. Fahimeh agrees that automating image processing and documentation is a big help that simplifies the life of the average scientist.
Alex showcases AI's versatility throughout the microscopist's day. It tackles challenges across the entire microscopy workflow. It's not just about analyzing data but also about enhancing image quality. For example, unveiling hidden details through denoising and super-resolution algorithms. Sebi agrees; AI integrates into broader workflows, from smart sample finding to aligning complex hardware systems using Language Learning Models (LLMs) such as ChatGPT. Falko expands its contribution even further: LLMs are strong in deep diving into specific topical silos and summarizing information. That is why AI can serve to help with planning and prioritizing experiments or sifting through data from publications, scientific forums, and developer posts.
Phillip sees in AI the potential to be the ultimate multitasker, as it tackles pattern recognition, making everything smoother, faster, and user-friendly. It can help make the coding journey a tad smoother, reducing the learning curve for creating more specified workflows. According to Alex, “AI will probably serve as a smart personalized microscopist assistant. It will make our interactions with microscopy hardware and microscopy software more intuitive and efficient.”.
Yet, the journey isn't all sunshine; you need to ensure you have enough quality data and consider the time and cost of training models. Where there are good foundations, AI can do wonders. At ZEISS Microscopy AI isn't just a tool; it's the backbone of problem-solving and digitalization in the tech frontier. With this holistic approach, we are building hardware and software products that integrate to create a unified digital ecosystem that provides solutions for the entire microscopy workflow.
How does AI serve ZEISS customers today?
Sreeni has been driving the use of AI in ZEISS products for years. with a goal of making AI accessible to everyone. With the AI toolkit, available now in ZEISS arivis Pro, ZEISS arivis Cloud, and in ZEN, image segmentation has been significantly simplified. We work with established and advanced technologies such as Docker and Microsoft Azure to make AI-driven products.
Falko sees the AI toolkit in ZEISS software as the bridge between complex analysis and user-friendly simplicity. No more need for an image analysis specialist; AI makes complex analysis more accessible today without the need to code, and in the future, it will deliver the "one-click" solutions that users want. In the ZEISS arivis software platform, it's not just about accessibility; it's about making trained models a seamless part of complex image analysis pipelines. Reproducible and scalable results naturally follow.
Martin sees AI as the performance maximizer. It's the co-pilot that helps users squeeze every drop of brilliance from their microscopes. In the ZEN core team, Martin is on a mission to bring AI tools that seamlessly fit into materials analysis workflows, making “impossible” images a thing of the past.
AI-guided image acquisition, sample finding and multidimensional image analysis and classification for reliable results is already available today. According to Sreeni there's more coming up! The future holds AI suggesting optimal imaging settings and even personalized solutions based on users' quirks and history.
Fahimeh works to further develop ZEISS arivis Pro to create even more easy-to-use, preconfigured multidimensional analysis solutions, machine-learning trainings, and integrating pre-trained models, to help our customers resolve image data challenges.
Phillip says AI enhances analysis, handling complex images with ease to extract the smallest detail. Scientific questions get a green light with AI's accelerated time-to-result. Phillip’s goal is to keep increasing the use of AI in the ZEISS ZEN software suite and turning its benefits into part of the microscopy routine.
Alex says AI serves microscopists today by playing a role in crafting better analysis solutions with minimal effort. Smarter annotation, less training data, and quicker setup times are key. Cloud technology makes complex analysis even more accessible. For Sebi it's all about finding new deployment methods for AI models – think containers for cross-system compatibility and scalability. “AI helps researchers solve real-life problems. Seamless integration is the magic word, turning AI into the everyday hero for ZEISS users.”, he says.
What lies ahead?
What does the future hold for AI in microscopy? Sreeni paints a vivid picture: “Imagine AI revolutionizing disease diagnosis and drug discovery, shortening medicine development time by half. It is starting to happen today with diverse data mining solutions” he says. “Think of AI as the powerhouse speeding up processes, improving accuracy, and playing a role in every microscopy nook. From virtually staining samples to real-time guidance via AR, the possibilities are endless. Cloud technology joins the party, facilitating seamless AI model sharing and large-scale studies. AI has the potential to unlock novel imaging realms.”
Falko, on the other hand, sees processes that are a fusion of classical and AI approaches. In the research realm, where unpredictability reigns, AI serves as a trusty assistant. Cloud technologies enable the integration of AI services for a smoother collaboration. For High Content Analysis (HCA), AI becomes the linchpin, automating workflows and supporting decision-making based on trial outcomes.
From providing information to flagging hidden system issues, Martin agrees that AI is the ultimate guardian angel of efficient workflows. Phillip foresees AI taking center stage in research, shaping codes of conduct, and speeding up data extraction. Fahimeh dreams big, “AI will be integrated into all aspects of microscopy,” she says. “From microscopes to software, it will be seamlessly integrated, automating processes and delivering precise results in record time.”
Alex agrees and flips the script on the current perception of AI. It will no longer feel like a daunting tool – something we use but do not understand or which is too difficult to set up. AI will become the friendly companion in every microscopist’s toolkit.” He says.
Sebi reminds us not to underestimate AI's long-term impact. “It's not just a trend, it's woven into the fabric of microscopy workflows,” he says. Big models such as SAM make training tools less crucial, putting the spotlight on integration. The future to his mind is in data-driven model development – using imaging data to boost performance and incentivize users to provide more and better data. Additionally, more AI will be integrated directly into the image acquisition process (online processing) possible by utilizing edge computing devices.
Musing about the AI of tomorrow
All our experts agree that AI will continue to feature in our work to speed up R&D and tackle complexity challenges. AI can help spot trends and sniff out problems, helping us prioritize our efforts. AI can be a savvy decision-making buddy and a coding assistant. It can monitor experiments and drive innovation with the most cutting-edge technologies. Combining computer vision LLMs can ensure modularity and reusability.
Falko warns that AI needs a dose of cautious handling. We're hooked on it, but we must also question it. Result verification is the name of the game. Alex is excited about smarter microscopy assembly lines, and AI's the glue that puts it all together. In this AI for Industry 4.0 era, not only are microscopes getting brainier, but their manufacturing process is catching up. Ultimately, Alex dreams of a bright future, with increased amounts of more meaningful research findings that will help to make our world a better place, step, by step.
Sebi knows that users want AI to be rock-solid, but here's the twist – models need a regular workout with new data to remain reliable. It is the only way to avoid data drift and ensure model robustness. The challenge? Making retraining a breeze. It's like having a car that needs regular tuning for optimized performance. We're talking tools and visual aids, making sure everyone's on board from day one. Retraining isn't just a feature; it's the heart and soul of every AI software application and only if it becomes a natural part of the AI workflows the promise of robustness and reliability will come true.
Unleashing the potential of AI for microscopy image analysis
AI has already made huge strides in the realm of new imaging possibilities. The accuracy and scalability offered by AI-powered image analysis have not only streamlined traditional workflows but have opened doors to novel applications previously deemed unattainable.
Despite the gray areas and the lack of clear guidelines, microscopists should be jumping on the AI bandwagon to make the most of the benefits entailed in this technology. Why? Because it's the ticket to automated data analysis with a click of a button. The impact of AI in these domains ushers a new era revolutionizing the way we interpret and comprehend microscopic data. And this is just the beginning of the journey.
--
* The written panel participants were: Dr. Sreenivas Bhattiprolu (Sreeni), Head of Digital Solutions (ZEISS arivis). Falko Löffler, Senior Software Developer, Dr. Martin Kuttge, Product Line Manager ZEN core, Dr. Phillip Holroyd, Product Manager Light Microscopy, Dr. Fahimeh Farhadifard, Software Engineer and DL Group Lead, Dr. Alexander Freytag (Alex), AI Expert at ZEISS Research & Technology, and Dr. Sebastian Rhode (Sebi), Software Architect AI Solutions.
Comments collected and blog written by Ofra Kleinberger Riedrich, Content and Product Marketing (ZEISS arivis).
