The content of auditory feedback to human early visual cortex and its impact on visual perception

The goal of the current research project was to investigate auditory feedback to cortical regions at the early stages of visual processing (i.e., V1, V2 and V3), while also looking at the consequences this flux of information has on visual perception itself. First, I conducted a functional magnetic resonance imaging (fMRI) study on blindfolded participants, which aimed to understand the semantic content of auditory information that is fed back to early visual cortex and its degree of specificity in the absence of visual stimulation. I presented different types of natural sounds, categorised in a hierarchical fashion, and tested whether early visual cortex received sufficient information to differentiate between semantic sound categories. This was done through a classifier algorithm in a multivariate technique called Multi-Voxel Pattern Analysis (MVPA). I found that sounds belonging to superordinate categories such as animate and inanimate, and to more specific categories such as humans, animals, vehicles and objects, could be decoded from neural activity patterns in early visual cortex. Additionally, human sounds seemed to be better decoded than other sound categories. I then conducted a binocular rivalry experiment with the scope of investigating whether sounds with varying degrees of semantic congruency can offer an advantage towards the perception of ambiguous human visual stimuli. These novel findings suggest that semantically congruent sounds can boost perception of congruent visual stimuli, but only when these are fully congruent rather than merely related (e.g., a clapping sound will boost perception of clapping hands, but not the image of a baby). This effect was only observed in the case of human sounds and human images. Taken together, the findings from these studies contribute to the evidence that suggests early visual cortex might belong to the network of multisensory processing, particularly when the stimuli are of human nature.