Mapping the Cognitive Model for Single-Word Processing onto Brain Dynamics

Background: The human brain can process single words during different tasks and via different modalities. We can hear the word ‘apple’ and repeat it out loud, or we see the word written on a page and read it, or we think of its concept, or see an image of it, and name it out loud. These three tasks (word repetition, reading and naming) require different computations at early stages, but share others at higher, amodal, levels. Decades of research in neuropsychology and cognitive sciences have produced a detailed description of the various information-processing stages during each of these tasks, based on examination of patients with brain damage, often highly localized. Each of these processing steps is considered a separate module, which can be selectively impaired following a focal brain lesion, which would lead to specific error patterns in one or more of these tasks (Friedmann & Coltheart, 2016). Neuroimaging studies have isolated cortical regions putatively associated with several of these processing stages, however, they often target a specific component of the model, in a specific modality (Kearns et al., 2019), and thus could not provide a comprehensive view on single-word processing in the human brain. A similar limitation persists in computational work on single-word processing. Most work addressed a single task, such as reading (Norris, 2013) or word repetition (Sajid et al., 2022), often using symbolic, discrete, models, without exploiting the advances made by modern deep neural models. In this project, we will develop a new experimental design to study the neural basis of these three tasks jointly, using fMRI and intracranial data. The results will provide a global map of the cortical regions involved in modality-specific and amodal processing of words, as well as of the various information-processing pathways. Throughout the project you will work closely with students who work on the development of deep neural models that simulate the three tasks jointly. Predictions from the models about neural mechanisms will be tested against fMRI data. Main Goal: Create a global map of the processing pathways in the brain for single-word processing and identify its amodal parts. The ultimate goal of the project is to provide a global view of single-word processing in the brain, delineating the various processing stages and, in particular, disentangling modality-specific and amodal processing of words. We will test the hypothesis that the unified single-word processing model (Dotan et al., 2015) can be mapped onto the human language network, such that each processing component (boxes and arrows in the model) can be mapped onto dedicated brain regions. A combination of fMRI experiments and deep learning modeling makes it possible today to entirely understand word processing, a phenomenon simple enough to be investigated fully, and yet at the root of language as we know it.