NeuroTalk: Neural computations of audio-visual coherence

Online Symposium

 

Evidence for neural computations of task-irrelevant temporally coherent visual stimuli on the auditory selective attention

 

Presenter: Dr. Fei Peng

 

Auditory selective attention can be enhanced or impaired depending on whether task-irrelevant visual stimuli are temporally coherent with acoustic targets or with competing distractors. Audio-visual (AV) temporal coherence might facilitate the binding of the features of auditory and visual stimuli to enable the AV object formation. However, it is unclear how AV temporal coherence and auditory selective attention interact at the neurophysiological level. Here, we measured neural activity using electroencephalography (EEG) while participants performed an auditory selective attention task. In the task, the envelope of two competing auditory streams changed independently, while the radius of a visual disc was manipulated to control the AV temporal relationships. Participants detected auditory transient deviants presented in the continuous target speech and ignored the deviants presented in the masker speech. We examined how AV temporal coherence affects the neural signatures of sound processing. Using the forward and backward model to quantify the neural response to speech envelope, we found that temporal coherence enhanced the AV integration for both attended and unattended speech. For the deviants presented in the unattended speech stream, the event-related response (ERP) amplitude was also modulated by the visual coherence, suggesting the task-irrelevant temporal coherence alone facilities the binding of the AV features pre-attentively. The result from the unattended stream provides evidence of neural signatures of bottom-up effects in the AV object formation. However, attention also modulated the neural response to the speech envelope. While, in most EEG channels, attention enhanced the ERP to deviant sounds independent of AV coherence, we also identified a spatiotemporal component of the ERP, likely originating from the superior temporal gyrus and the frontoparietal network, in which both attention and coherence synergistically modulated ERP amplitude. These results provide evidence of the neural signatures of bottom-up (coherence) and top-down (attention) effects in the AV object formation.

 

Bio

 

Dr. Fei Peng obtained her Ph.D. in Biomedical Engineering at Chongqing University, Chongqing, in 2019. She is currently a postdoctoral fellow in the City University of Hong Kong. Her research interests include neural correlates of auditory scene analysis and neural coding mechanisms of natural sounds, such as voice pitch contours of Mandarin tones and statistical features in the sound textures.

 

Moderator: Mr. Owen, Yuwen HE

(Ph.D. student, FHS/ICI-CCBS, University of Macau)

 

Time: 7:00 PM, Sep 3^rd  

Zoom ID: 959 6270 3011

Passcode: ccbs

Acknowledgements

NeuroTalk x Brain Technology