2025 workshop program
Multimodality and Multimodal Models in Astrophysics
The work
Abstract
Astrophysics has always been a multimodal science, drawing insights from the integration of diverse data types, such as images, spectra, and time-series (photometry), both observed and simulated. With the growing volume and complexity of data coming from current and upcoming astronomical surveys, machine learning methods capable of handling and aligning multiple modalities are becoming increasingly important.
This talk serves as an early-stage review of the emerging role of multimodal AI in astrophysics. We clarify terminology - distinguishing between traditional, foundation, and generative multimodal models. We explain the core technical ideas behind modern approaches such as contrastive representation learning and generative vision-language models. We discuss how these models can be trained or fine-tuned for astrophysical use cases, and survey the relatively limited but growing body of work applying them to astronomical data.