How Large Language AI Models Can Transform with Drew Williamson

Dr. Drew Wiliamson, MD joined Emory University's Department of Pathology and Laboratory Medicine early this year as an assistant professor. During this time, he has become a member of the Cell and Molecular Biology Research Program at the Winship Cancer Institute.

We sat down for an interview with him regarding how his research in disease pathology intertwines with Artificial Intelligence technology.

Q: For someone unfamiliar with large language models, how would you explain them in simple terms?

Large language models are a type of AI designed to predict the next thing in a sequence. For example, given a sentence, the model predicts the next word. With enough data, these models can generate text indistinguishable from human writing. They learn relationships between words, sentences, and ideas, enabling them to assist in various applications, including medicine.

Q: Can you discuss your work with PathChat and explain the role AI played in its development?

PathChat is like ChatGPT, but for pathology. Basically, every part of it is built with AI. The idea is to have something like ChatGPT that a human could interact with but with pathology knowledge embedded in it. It uses advances in large language models and applies them specifically to pathology. 

Pathology is a very technical field that takes a long time to learn, both in terms of language and images. For instance, hematoxylin and eosin-stained tissue slides require expertise to interpret. PathChat was designed to act like a co-pilot for pathologists. It helps with cases by suggesting potential diagnoses, additional tests and other important insights.

Q: With PathChat combining vision encoders and pre-trained language models, what advantages do vision-language models offer in pathology? How do they differ from traditional image-based approaches in terms of performance and applications?

These models incorporate both images and text. Historically, AI in general and in pathology has focused on either images or text but not both. Only recently have we had enough data and computational power to train models that understand both modalities. For me as a pathologist, this makes sense because pathology is taught with both text and images. Pathology textbooks combine detailed descriptions with images to teach diagnosis, treatment implications and disease characteristics. Similarly, vision-language models do better when trained with both text and images compared to just one. The model can learn more from the added context, much like how humans understand an image better with a caption. This approach benefits pathology but also has broader implications beyond it. By providing more information, these models can offer deeper insights.

Q: What ethical considerations do you think are important as AI is integrated into clinical workflows, especially in your work?

There are many ethical considerations. A big one is ensuring that AI performs equally well for all patients. Currently, many models are trained on data from North America and Europe, which doesn’t represent the global population. This tendency can lead to models performing better for patients from affluent, white or academic backgrounds while underperforming for others. Ensuring that the training data is mixed and representative is important.

Another consideration is the impact on clinicians and jobs. AI models should provide value beyond what a human can do. If a model simply replaces a human without improving outcomes, we need to reconsider its deployment. There’s also concern about exporting AI to resource-limited areas where it might replace roles instead of creating opportunities. This could perpetuate resource drains from these regions, as the technology’s profits and control remain centralized in high-income countries.

Q: How do you think large language models will shape the future of pathology and laboratory medicine?

I see them as invaluable assistants. For instance, during my training, having a tool to answer questions or double-check my work would have been a game-changer. These models can democratize expertise, enabling every pathologist to practice at the highest level regardless of experience. For example, a seasoned expert’s 30 years of knowledge could be accessible to someone just starting. This backup system, powered by large language models, could ensure more consistent and accurate pathology practice.

Q: How important is interdisciplinary collaboration in your work?

Collaboration is important. Medicine and research are highly specialized, and delivering the best care requires expertise from various fields. A cancer patient, for instance, interacts with multiple specialists from diagnosis to treatment. No single person can understand the entire journey. Similarly, computer scientists or engineers may not fully grasp the nuances of clinical workflows. Interdisciplinary collaboration ensures we’re solving real-world problems. If we don’t involve experts from every relevant area, we risk developing solutions that are intellectually interesting but not practically useful for patients.

Q: What are your final thoughts on the future of AI and large language models?

These AI tools are revolutionary, much like the invention of the microscope. Before microscopes, pathology relied on what could be seen with the naked eye. The microscope revealed details that changed how we understood diseases. Similarly, AI tools allow us to analyze data in ways we couldn’t before. I’m excited to see how these tools help us uncover new insights in cancer biology, infectious diseases and more. They’re opening doors to discoveries we’ve never been able to make.