8xav��������

Recent advances in machine learning (ML) and artificial intelligence (AI) have made it possible to develop new methods for understanding brain function based on medical imaging. One very promising approach, which revolutionized the field of natural language processing by enabling tools like ChatGPT, is to use embeddings, which are low-dimensional representations of high-dimensional data, for example mapping text to a vector of numbers. Embeddings can be used to capture the relationships between different data modalities and feature dimensions, may it be spatial patterns, temporal dynamics, or even semantic correlations. In this project, we will transfer the concept of embeddings to the domain of image analysis for studying brain function and cognition. By using embeddings, we create a lower-dimensional space which enables us to fuse different data modalities in this space, for examples MRI imaging of brains, blood-based markers such as proteins or cytokine levels, and results of psychological cognition testing – modalities that are originally quite “incompatible”. By projecting them to a common space using recent ML/AI techniques, novel joint analysis is enabled. To evaluate performance of the joint models and ensure that their “black box” approach is based on solid ground in the application domain, we will apply them first to the field of pain research. This audacious project will yield outcomes in two domains: In the methodological / computational domain, we will develop techniques to efficiently apply joint embedding methods, as established in language processing, to other modalities revolving around medical imaging and cognitive science data. On an applied level in cognitive science, we will generate novel insights in the identification of multi-modal markers for cognitive/clinical states, in particular for pain. We will, for the first time, integrate cognitive and behavioral data with MR imaging information and blood-based markers in a large, joined analysis. This specific combination of data modalities is also highly relevant in neurodegenerative disorders like Alzheimer’s or Parkinson’s disease. As such, our focus on pain research provides an ideal testbed for opening audacious perspectives in analyzing cognitive processes reaching far beyond the current context.