This event has already occurred.
“Cross-Modal Learning and Foundational Models in Medical AI: From Techniques to Applications in Placenta Analysis” - Yimu Pan, Penn State
Date & Time: March 26, 2025 from 03:00 PM - 04:00 PM
Location: E202 Westgate and Virtual via Zoom
This event is part of the Young Achievers Symposium series hosted by Penn State's Center for Socially Responsible Artificial Intelligence.
About the Talk
Cross-modal learning and foundational models represent cutting-edge advancements in medical artificial intelligence (AI), enabling the integration of diverse data modalities—such as medical images and clinical text—to address challenges in data scarcity, heterogeneity, and clinical interpretability. Foundational models, pre-trained on large-scale multimodal datasets, provide a versatile framework for harmonizing information across clinical domains, uncovering latent patterns, and enhancing diagnostic precision. This work explores methodologies for cross-modal alignment, including contrastive learning and attention-based fusion, which empower models to bridge imaging and textual data for comprehensive insights. We demonstrate their application in placenta analysis, a critical yet under-researched domain in prenatal care, where multimodal integration improves the prediction of placental complications.
Generative models, such as diffusion models, are further leveraged to augment scarce medical image datasets, mitigate domain shifts, and enhance model robustness in real-world clinical settings. By synthesizing realistic medical images or generating synthetic clinical narratives, these approaches address data limitations while preserving biological relevance. Our study highlights the synergy of cross-modal learning and foundational models in advancing placenta-centric AI tools, emphasizing their potential to transform prenatal diagnostics, risk stratification, and personalized care. We conclude with practical considerations for deploying such systems in clinical workflows, underscoring the importance of interpretability, ethical data practices, and collaboration with domain experts to ensure translational impact.
About the Speaker
Yimu Pan is a doctoral student in Informatics in the College of Information Sciences and Technology at Penn State, under the supervision of James Z. Wang. His research focuses on computer vision and multimodal learning, with applications in health care, emotion understanding, and AI-assisted clinical decision-making. In 2024, he worked as an applied scientist intern at Amazon, where he collaborated on projects related to last mile delivery optimization. He earned bachelor's degrees from Penn State in Computer Science (Summa Cum Laude) and Mathematics and Statistics (Magna Cum Laude).
About the Series
The CSRAI Young Achievers Symposium series highlights early career researchers in diverse fields of AI for social impact. The series seeks to focus on emerging research, stimulate discussions, and initiate collaborations that can advance research in artificial intelligence for societal benefit.
Our events and programs are open to all students regardless of sex, gender, sexual orientation, race, or any other protected class. The College of Information Sciences and Technology is committed to building a community of belonging for all.