2. Earlyhunt: Launch your AI product early, get discovered weekly. Before I invite the next early-access batch, I rerun every input the last batch broke.

During early access for an AI tool, fixing one issue in the prompt can cause another previously working feature to deteriorate silently. Users often don't notice because they continue testing with the same three inputs. It's the batch two users who uncover these problems by entering unexpected inputs, receive bad answers, and leave without explaining the issue.

With this in mind, I now follow one rule before inviting the next batch: I rerun every input from the last failed batch, along with a few that succeeded. Each real user input is copied exactly as it was typed, including any typos. The rule states that if the core job gets worse, the invites wait. The row includes the input, the batch it came from, and a count of how many times it was fixed or broke.

I run each input three times, because a single lucky run can appear to be a fix. The rows only pass if all three runs succeed. These rows come from support emails, thumbs down button clicks, my own testing, and suggestions from others. When a tool appears on platforms like EarlyHunt, early adopters often test the edges on purpose, providing the best inputs for this process.

I ask for permission before including any data and remove any personal information. The invite rule is strict on purpose, and fixing 5 while breaking 1 still means waiting if the broken input is the core job. I keep batches small, around 10 to 15 people, to ensure I can read every output within the first 48 hours and turn any bad results into rows before the next group.

Deciding who gets into early access and the full sheet detailing the grading options and run log are covered in how to build an evaluation set for your AI product before users find bugs.