AI Made Me Look Foolish. Then I Learned

In February, the author stood before their collaborators to present a major update to VIDE, an algorithm they had developed more than a decade before. Voids, the vast empty regions between galaxies, are difficult to extract from surveys, which are often messy and full of gaps, masks, and ragged boundaries. The new version of VIDE was 10 times faster and could handle surveys a hundred times larger than before, with a more sophisticated approach to dealing with the complexities of real-world data.

The author had spent weeks refining the code, and felt proud of their work. They had also been working with an AI at their side, which had been instrumental in rewriting the core of the algorithm and generating the plots and commentary for their presentation. When a collaborator raised a concern about the edge handling of a survey, the author confidently answered, only to realize after the fact that they had made a critical mistake. Despite 20 years of training, the author had presented flawed information with complete confidence.

The author reflects on the seductive power of AI, likening it to a modern-day philosopher's stone that can transform machines into minds. The ancient alchemists spent centuries searching for a legendary substance that would turn lead into gold and make its owner wealthy beyond counting. Today, they acknowledge that we pour in text and electricity and wait for thinking to emerge on the other side, a process that feels magical rather than merely handy.

After the presentation, the author's humbled. Alchemy may have seemed like an embarrassment to science, but it was actually a necessary step towards scientific progress. The author realized that the alchemists' work was not an embarrassment, but rather a crucial component of the scientific journey. The AI system they were using, a large language model (LLM), sounded like it understood, but it was simply a sophisticated next-word predictor.

The correctness of the AI's answers is a side effect of the frequency of true statements in the human-written text it is trained on, and the fluency of the AI's responses is the result of intentional breeding. The author had been relying on an LLM to partner with them in their research, and while these tools can be incredibly useful, the author warns against trusting them completely.

The superpowers of AI are real, but they are volatile and tuned to please us. The author stresses the importance of maintaining a healthy skepticism towards AI and not allowing it to replace critical thinking and human judgment.