From meeting speech to task dispatch: wiring on-device ASR into an AI collaboration workflow

From speech to task assignment: integrating on-device ASR into AI workflow for collaborative work

The integration of Octic, an on-device AI recorder, with Octo, an open-source collaboration platform, was aimed at creating a seamless workflow where a verbal action in a meeting instantly results in an assigned task. This article outlines the pipeline from audio capture to task execution and back to feedback.

The meeting environment generates 40% of agreed tasks that never make it into any tracking system. To address this issue, an end-to-end pipeline was devised, consisting of five stages:

1. On-device ASR via Octic: Audio is captured and transcribed locally, ensuring no data leaves the meeting room. Speaker diarization attributes each segment to a specific speaker, resulting in a timestamped, speaker-attributed JSON transcript.

2. NLU and intent extraction: The raw transcript is run through a lightweight NLU model, extracting action items, decisions, open questions, and other relevant information. Action items are further broken down into task descriptions, responsible persons, deadlines, and acceptance criteria.

3. Mapping to Octo Loops: Extracted action items are converted into Loops in Octo, the collaboration platform. Loops serve as work units that progress from conversation to delivery, with owners, deliverables, and acceptance criteria. Each action item creates a Loop with the responsible person's Agent, complete with source context for traceability.

4. Agent execution: Octo Agents, digital workforce clones, receive the assigned Loops. They acquire the necessary permissions, preferences, and prior experience to autonomously execute tasks. The agents communicate with human supervisors, who review and accept or reject the completed work. Feedback triggers iterative revisions, creating a trackable workflow.

5. Feedback loop and learning: Rejected deliverables are returned to agents with specific feedback, allowing for revisions and resubmissions. Every step of the process is tracked, enabling future review of the entire meeting, task creation, agent output, feedback, and final acceptance.

Key takeaways include the importance of reliable speaker diarization, the effectiveness of post-meeting batching for intent extraction, and the value of digital Agents that learn and adapt over time, streamlining the workflow while maintaining a transparent record of every action taken.