Diagnosing sentiment in my typo-reporting workflow
This is more of a comment than a question. A few months ago I added a “Report a Typo” feature to Six Colors. The goal was to give people a simple place to go — sixcolors.com/typo …
The author created a "Report a Typo" feature on their website, Six Colors, to allow readers to easily report mistakes. They set up a Google form that inputs the information into a Google Sheet, which triggers a script to send notifications via Make.com, Discord, and Pushover. However, the introduction of this feature led to an influx of non-typo complaints, such as style preferences, political opinions, and personal criticisms.
The author admits that dealing with this type of feedback is frustrating, as they want to quickly fix actual errors but don't want to be inundated with subjective opinions. To address this issue, they turned to sentiment analysis using a large language model (LLM) plug-in within Make.com. This solution uses a simple model with a low API cost.
The author developed a prompt for the LLM to classify incoming comments as either "ALLOW" or "REJECT" based on predetermined criteria. The ALLOW classification requires the comment to contain a specific, factual, technical, or materially consequential error in the article. REJECT classifications encompass subjective opinions, political disagreements, tone/style critiques, speculative motives, emotional reactions, and off-topic or spam content.
The author finds this LLM-based approach surprisingly effective in handling the flood of unwanted feedback while still promptly notifying them of actual typos.
Written by urgent.news from Six Colors's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
