This study is motivated by the increased need to understand human response to video-links， 3G telephony and avatars。 We focus on response of participants to audiovisual presentations of talking heads， and examine the effect of noise and temporal misalignment of channels。 We show that misalignment of audio and visual channels not only cause strange perception phenomena - the McGurk effect but also cause participants to apply extra mental effort， which is detectable from the physiological data collected。 These data allow inferences to be drawn about the impact of the McGurk effect， thus providing indication of stress levels。 This illuminates both to the mental and physical aspects of users interacting with multi-modal interfaces
multi-modal interfaces
Azra N. Ali、Philip H. Marsden
展开 >
University of Huddersfield, Huddersfield, England
International conference on Intelligent user interfaces
Miami, FL(US)
Proceedings of the 8th international conference on Intelligent user interfaces
NETL
