We Let an AI Product Owner Triage Our Idea Backlog

For products that let users submit ideas, a common issue arises: submission is easy, but review is costly. This leads to a growing backlog of open ideas. In one instance, over a hundred-and-forty ideas were open, some of which were eighteen months old and duplicates submitted by multiple people. Reading an idea properly, checking for duplicates, deciding its value, and providing a reason for rejection takes around ten minutes per idea.

Therefore, reviewing a hundred-and-forty ideas would take about three days. To address this, FlowBoard was built, integrating an AI assistant like Claude or Cursor to directly interact with the idea board via its API through a Message Control Protocol (MCP) server. This allowed the AI to perform four review actions - accept, reject, rework, or place - as well as scoring and following a prompt that transforms the process into a workflow.

The user would simply state "review my idea backlog," and the AI would take it from there, reading each idea individually, searching for duplicates, presenting a recommendation - accept, reject, or rework - along with reasoning and the exact text it would post, and waiting for user confirmation before moving on to the next idea.

The key is that the AI stops at the user's confirmation, preventing it from dismissing ideas or making incorrect rejections. The AI's recommendations are based on a rubric controlled by the user, which includes factors like prioritized customer segments, what the product team decided not to build and why, the definition of "too vague to action," and how the team defines a "reasonable" idea.

Over time, the AI's recommendations became more aligned with the user's product strategy as the team refined the rubric based on disagreements and new insights. The system was found to be particularly good at duplicate detection, which was a surprising outcome. It also wrote rejection reasons that were specific, kind, and concise, taking away the mental effort typically required for writing rejections.

The system was also capable of identifying stale context - ideas that were reasonable when submitted but are now outdated - and proposing implementation costs based on an engineering judgment. However, the AI was observed to systematically undervalue ideas from key customers and to occasionally call similar ideas duplicates when they were not.

Human oversight with a confirmation step mitigated these issues. Every change made by the AI is recorded, allowing for an audit trail to see what was changed, what it was before, and that an AI performed the action. This system thus balances AI efficiency with human judgment, preserving the unique context and understanding that only humans can provide.