A Framework for Conversational System Modeling

The conversation theory introduced by Paul Grice posits that effective communication between two people involves a desire for cooperation. This cooperative mindset drives individuals to be informative, truthful, relevant, and clear in their interactions. Modern AI systems capable of voice interaction, such as Alexa and Siri, do not aim to be cooperative in a human sense. However, they can predict conversational events to simulate human-like conversations.

FinVolution, a Chinese fintech firm, organized a competition called FinVolution Teach AI When to Speak, focusing on modeling conversational speech. The competition, which closed in July, provided base training and inference logic, allowing participants to develop their models using Chinese-language conversations. The goal is to provide an understanding of the problem setup so that readers can create their own training datasets and build their own models.

In a two-participant conversation, five distinct conversational events can be defined: Continuation (C), Turn Change (T), Backchannel (BC), Interruption (I), and Silence (NA). Continuation refers to a participant speaking and continuing their idea. Turn Change occurs when the other participant becomes the current speaker, often signaled by a question from the previous speaker.

Backchannel involves the non-current speaker making a short utterance without taking over the turn, such as using filler words. Interruption happens when both participants speak simultaneously, with one attempting to take over the turn. Silence represents short windows where neither participant is speaking.

By focusing on audio events, the simplification allows for easier modeling and aligns with FinVolution's conversational model used in the competition. However, this simplification can be overcome by incorporating other sensory cues, such as visual or cultural behaviors.