My main line of research investigates face perception, identity recognition, and emotion perception thorugh facial gestures.
My work investigates the behavioral and neural correlates of recognition of familiar faces with different degrees of familiarity, with a particular focus on the role of visual familiarity, and the role played by retrieval of person knowledge.
My work on face perception has highlighted how learning optimizes face processing at all levels, including those that precede awareness. Furthermore, her work has shown how processing of identity and emotion in face perception might follow distinct pathways.
Currently, one line of my research investigates individual differences in cognition and their neural correlates, with potential clinical applications for diagnosis and treatment.
LINES OF RESEARCH
1. Face Perception
This line of my research focuses on face perception with a particular interest in the mechanisms for recognition of personally familiar faces. The vast majority of past and ongoing work on face perception has focused on unfamiliar faces, and models tend to assume that the neural mechanisms for face processing are constructed for efficient processing of faces regardless of familiarity. By contrast, my hypothesis is that there is a clear distinction in the way the face system processes familiar and unfamiliar faces that has important implications for social cognition and learning. People are very skillful in determining that a face is unfamiliar and reading social cues from faces of strangers (e.g. facial expressions of emotion conveyed by unfamiliar faces). The topic of face processing becomes more complicated, though, when considering the recognition of unique and view-invariant identity. Data show that despite the subjective impression of high efficiency for recognition of unfamiliar face identity, performance is vastly superior for familiar than for unfamiliar faces (Bruce, 1994; White et al. 2015; Burton et al. 2015; Guntupalli & Gobbini, 2017; Visconti di Oleggio Castello et al, 2017; Ramon & Gobbini, 218). The illusion of efficient recognition of unfamiliar faces has real-world consequences, such as ill-placed faith in eyewitness identification or matching identity based on photo ID documents. Much in the same way that the human language system is adapted and optimized in each individual to process that individual’s native language, I hypothesize that the face perception system is finely tuned in each individual for interaction with the people that play an important role in that individual’s life. Therefore, to investigate the system at its full potential, personally familiar faces are the appropriate and necessary stimuli. The type of personal familiarity I am interested in is the one that develops with protracted, repeated personal interactions such as the ones we have with close friends, family or long-term colleagues.
The research I have conducted supports my model that personally familiar faces are processed in a facilitated way as compared to unfamiliar faces and the optimization for personally familiar faces occurs at multiple levels in the distributed system for face perception.
1. Optimization of early visual processes as reflected in facilitated detection and orientation to personally familiar faces, as well as facilitated detection of social cues in personally familiar faces. Optimization of processing of personally familiar faces might even precede the activation of a view-invariant representation and explicit recognition of identity (Gobbini et al., 2013; Visconti di Oleggio Castello, Guntupalli et al., 2014; Visconti di Oleggio Castello & Gobbini, 2015; Visconti di Oleggio Castello, Taylor, Cavanagh & Gobbini, 2018).
2. Optimization of later visual processes. The view-invariant representation of personally familiar faces is dramatically more robust and efficient, affording effortless recognition of identity over large variations in image quality (Gobbini et al., 2013; Visconti di Oleggio Castello et al., 2017).
3. Optimization of post-perceptual processes as reflected in spontaneous activation of person knowledge and an appropriate, individual-specific emotional response to personally familiar faces. These processes are an integral component of recognition of a familiar individual and involve activation of the extended system for face perception. Retrieval of person knowledge and the emotional response to familiar individuals in turn, further facilitate efficient recognition through top-down processes (Gobbini, 2010; Visconti di Oleggio Castello, Halchenko et al, 2017; Guntupalli & Gobbini, 2017; Ramon & Gobbini, 2018; Visconti di Oleggio Castello et al, 2021).
Understanding the mechanisms for personally familiar face detection and recognition can provide a window onto the mechanisms for learned optimized processing of salient visual stimuli and, more generally, on the acquisition of social knowledge. Furthermore, establishing the neural basis of familiar face recognition can help better understand the pathophysiology of neurological and psychiatric conditions such as prosopagnosia, schizophrenia, Capgras syndrome and social phobia.
2. Individual differences in cognitive function
In the past few years, the focus of my research has extended to leverage hyperalignment and the Individual Tuning (INT) model to investigate individual differences in cognitive functions, for example, in face perception. We have shown that hyperalignment makes it possible to project functional localizer fMRI data from other brains into a new brain and estimate idiosyncratic functional topographies, such as retinotopy and visual category selectivity, with high fidelity. This procedure allows us to define idiosyncratic topographies for a wide range of functions in each participant with only movie data (Jiahui et al., 2020). We have shown further that hyperalignment and INT individuate brains with surprisingly high reliability and these differences strongly predict psychological differences. I am in the process now of correlating behavioral data on face recognition performance with the face perception system (extent of the areas and connectivity among them) to investigate the neural basis of the differences in face perception capabilities.
3. Perception of Social Actions
Naturalistic stimuli, as compared to still images in more controlled experiments, evoke strong neural responses over a larger extent of cortex and those responses carry more information that can be leveraged with state-of-the-art analyses of fMRI data, such as hyperalignment (Haxby et al., 2011) and geometries of brain responses (Kriegeskorte et al.).
Paradigm shifts in science are often driven by technical developments, such as the introduction of naturalistic, dynamic stimuli in cognitive neuroscience. These developments not only change how experiments are conducted but reveal gaps in our collective knowledge and motivate fundamentally novel questions.
Nearly all movies depict people or other agents performing actions. Studies of brain activity evoked by observed actions in naturalistic, dynamic stimuli suggest that the representation of these actions plays a dominant role, overshadowing the representation of animate form.
Early functional neuroimaging work has shown that agentic action, known to evoke activity in motion-sensitive visual areas, also evokes strong responses in ventral temporal (VT) cortical areas. Some of these studies used dynamic stimuli that contain minimal form information, such as point-light displays of biological motion and animations showing social interactions depicted with moving simple geometric figures. Both of these types of minimalist dynamic stimuli evoke strong responses in ventral temporal cortex, as well as in lateral temporal areas for visual motion perception, despite the lack of animate forms in the stimuli. Other studies have used videos of moving robots and showed that inanimate robots performing meaningful actions evoke strong activity in ventral and lateral temporal areas associated with representation of animate form and action, even if the robots bear no resemblance to animate forms.
Even in the FFA (fusiform face area), the semantic qualities of animacy, social interaction, and mobility play a surprisingly important role in the response space (after controlling for motion energy; Çukur et al. 2013).
Nastase et al. (2017) used representational similarity analysis (RSA) to study the geometry of representations evoked by viewing naturalistic movie clips of behaving animals. They used separate predictors of representational geometry based on action categories (eating, fighting, running, swimming), independent of the species of animal performing the action, and on similarities of taxonomic categories (primates, ungulates, birds, reptiles, insects), independent of the action being performed.
The results showed that representational geometry was dominated by species-invariant behaviors in a broad expanse of extrastriate visual, somatosensory, motor, and premotor cortical fields. Surprisingly, the dominance of action representation was even evident in VT cortex. This contrasts with previous work based on experiments using still images that emphasized the role of VT cortex in the perception of form-related properties.
Naturalistic stimuli, as compared to still images in more controlled experiments, evoke strong neural responses over a larger extent of cortex and those responses carry more information that affords enhanced MVPC, functional hyperalignment and further understanding on how the action perception system might be organized.
4. Brain plasticity
The question motivating this line of research focuses on how perceptual experience shapes the brain during development. To investigate this question, we measured with fMRI responses to different types of stimuli in the congenitally blind. We found that congenitally blind individuals during non-visual object recognition show topographically-organized category-related patterns of neural response in the ventral “visual” pathway. Furthermore, we found that emotions conveyed by voices can be decoded in homologues of the core face system, areas that respond to faces in sighted participants. These data suggest that areas that have been typically thought of as ‘visual’ might have a broader role in representing semantic information about others beyond processing visual inputs.
5. Studies on patients
Normal cognitive functions require the efficient transfer of information across multiple brain areas and networks and alteration of transfer of information is the cause of cognitive disfunction.
fMRI is a common tool to measure information processing and brain connectivity.
A more recent approach in investigating brain connectivity is based on fine-grained response patterns (vertex by vertex) that allows to model individual differences in brain organization.
This line of research aims to characterized how brain connectivity changes during the development of symptoms in neurological conditions (e.g., MCI -mild cognitive impairment; Alzheimer) and psychiatric diseases (e.g. major depression).
6. The nature of stigma for medical conditions
Several medical conditions are subjected to stereotypes and misconceptions even by professionals in the health care system. As a result, the delivery of care toward patients who are affected by stigmatized conditions might be suboptimal. One example of stigmatized disease is mental illness, but also other conditions such as obesity, dermatological diseases are stigmatized.
Stigma often arises because of negative attitudes, beliefs, and prejudices towards those diseases but also from a lack of understanding and knowledge.
The purpose of this line of research, in collaboration with Dr. Edita Fino, is to evaluate the roots of stigma in medical students and how attitudes toward stigmatized medical conditions changes during the medical studies with a more thorough knowledge of the pathological details associated with diseases.
