Nigel Ward: Research Projects

Towards Persuasive Dialog Systems

Time-Based Language Modeling

Responsiveness in Spoken Tutorial Dialogs

Cross-Cultural Misperceptions of Dialog Behaviors

Automating the Discovery of Dialog Cue Patterns

Speaking Rate in Dialog

Speech recognizers all include a component for predicting, based on the past context, what words are likely to appear next. Today these components, known as language models, operate at the symbol level, abstracted away from the details of how and when the words are spoken. Spoken language, however, is not just a symbolic or mathematical object, but is produced and understood by human brains, with specific processing constraints, and these can directly affect what happens when in dialog.

This project is developing language models and ``dialog models'' that explicitly use the information in the timings of words. Inspired by psychological research suggesting that dialog and language behaviors are the result of multiple simultaneously active cognitive processes, the working assumption is that the words likely to be spoken at a given time depend, probabilistically, on the elapsed time since various reference points: for example since the speaker began talking, since the speaker's last disfluency, since the listener's last back-channel, etc. Statistical analyses of large corpora of human-human spoken dialogs, with machine learning methods, are revealing patterns and regularities which are being used to build language models with improved predictive power.

These language models implicitly represent some aspects of dialog dynamics, with the potential to lead to an integrated understanding of the nature of dialog as a human ability. These improved language models are also likely to improve speech recognition accuracy, enabling the development of spoken language systems that are more accurate, more efficient, and more useful.

(with Alejandro Vega, Shreyas Karkhedkar, Ben Walker, Nisha Kiran, and Shubhra Datta)

The rules governing real-time interpersonal interaction are today not well understood. With only a few exceptions, there are no quantitative, predictive rules explaining how to respond in real-time, in the sub-second range, in order to be an effective communicator in a given culture. This can be a problem in intercultural interactions; if an American knows only the words of a foreign language, not the rules of interaction, he can easily appear uninterested, ill-informed, thoughtless, discourteous, passive, indecisive, untrusting, dull, pushy, or worse. Short of long-term cultural exposure, there are today no reliable ways to train speakers to understand and follow such rules and attain mastery of interaction at the sub-second level. The purpose of this research is to increase our knowledge and know-how in this area.

The pacing of today's dialog systems tends to be rigid. In addition to the problems of turn-taking, the speaking rate itself is generally fixed. For example, the automatic number-giving that comes at the end of directory assistance calls is at a fixed rate: too slow for some people and too fast for others. We have found that the speaking rate can be adapted automatically for these dialogs, based just on the user's speaking rate and response latency. However simple correlations break down for more complex types of dialog; there the speaking rate appears to depend more on dialog act and dialog state. (with S. Kumar Mamidipally, continuing work by Satoshi Nakagawa)

See also:	Publications
	Research Themes in Dialog System Usability
	Earlier Projects
	Interactive Systems Group Projects Page