It's not what you said, it's how you said it: discriminative perception of speech as a multichannel communication system

Abstract

People convey information extremely effectively through spoken interaction using multiple channels of information transmission - the lexical channel of what is said, and the non-lexical channel of how it is said. We propose studying human perception of spoken communication as a means to better understand how information is encoded across these channels, focusing on the question What characteristics of communicative context affect listener’s expectations of speech?

To investigate this, we present a novel behavioural task testing whether listeners can discriminate between the true utterance in a dialogue and utterances sampled from other contexts with the same lexical content. We characterize how perception – and subsequent discriminative capability–is affected by different degrees of additional contextual information across both the lexical and non-lexical channel of speech.

Results demonstrate that people can effectively discriminate between different prosodic realisations, that non-lexical context is informative, and that this channel provides more salient information than the lexical channel, highlighting the importance of the non-lexical channel in spoken interaction.