Bug blindness

For a long time, the author of this piece wondered why they seemed to notice more bugs than others. They quickly discovered that many people don't notice bugs, especially non-programmers. However, by pointing out bugs to friends and acquaintances, the author realized that many people could start noticing them too. This observation led to opportunities in various jobs where directors, VPs, and executives sought their opinion on issues they suspected might be problematic.

Sometimes the issues the author finds are minor, but other times they are severe enough that a normal user wouldn't be able to use the software. The author has witnessed many products launch with glaring flaws, only for users to encounter the same issues they had. They believe that if a product seems severely flawed when used, it likely is. With the help of LLMs, the author can simulate normal user behavior and demonstrate that the issues are consistent across various scenarios.

The author provides several examples to illustrate the point. They found that major search engines like Google, Bing, and Kagi were returning poor results, full of low-quality SEO spam and scams. While most people agreed with their characterization of Google and Bing, many insisted Kagi was better, even though the author found the same issues with it.

The author also cites the example of Volvo, whose reliability has been questioned by various sources. Despite the data showing Volvo as mediocre to poor, forum users insist that Volvos are among the most reliable cars. The author also mentions Blackboard, a course management software that was widely disliked by students and professors.

In 2011, a survey showed that 93% of respondents hated the company. The author shares a memory of awkwardly expressing their thoughts on working on a disliked software, only to have the person they spoke to apologize for the software's flaws. Overall, the author argues that if a product seems severely flawed when used, it probably is.