Rate limits are not quality gates: the guardrail stack behind an AI agent that posts publicly every day
Our AI agent posts publicly every day — social posts, replies to strangers, comments on other people's articles — with no human reviewing individual messages before they go out. That sentence should make you nervous. It makes us nervous, and we built the thing. Rate limits alone don't fix it. An agent that sends 20 polite, on-topic messages is fine; an agent that sends 20 copies of the same…
Our AI agent shares messages daily on various platforms, including social posts and replies to strangers, without any human intervention. While rate limits on their own may not prevent excessive spamming, they are a crucial component of the system's overall security. The system employs a multi-layered approach to ensure that the messages sent are both high-quality and appropriate.
The first layer consists of hard caps enforced in the code, which include a global daily cap (60 messages) and a per-batch reply cap (20 messages). When the cap is reached, the send function refuses to proceed, ensuring that the model does not have any decision-making power in this regard. This design rule prevents any attempts to manipulate the system by altering prompts or session data.
Layer two introduces sameness detectors, which target repetition as a key indicator of spam. The system uses a blocklist of common marketing phrases, along with a 3-gram Jaccard similarity algorithm to compare queued posts with the last 60 days of sent history. If the similarity score exceeds 0.4, the batch is rejected. This prevents the AI from sending identical or highly similar messages repeatedly.
Layer three involves per-target judgment, where each outbound text is scored by a separate model instance against a written rubric. This rubric evaluates the reply's usefulness, tone, and potential for condescension or unwanted pitching. If the reply fails to meet the standards, it either undergoes revision or is blocked entirely. This step ensures that the AI's interactions remain genuine and valuable, rather than robotic and impersonal.
The final layer involves maintaining an audit trail of every send, which is committed to a git repository. This includes the target, text, review verdict, and timestamps. By keeping a record of all interactions, the system can quickly identify any discrepancies or unusual activity, ensuring transparency and accountability. This approach acknowledges that autonomy without proper oversight is not a trustworthy system.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
