Want to get more from AI? Treat every prompt like an experiment

In late 2024 and early 2025, artificial intelligence systems demonstrated proficiency akin to that of PhD-level scientists in scientific reasoning assessments. To capitalize on these advances, I created an AI module to be used by incoming biomedical PhD students at Vanderbilt University. The course surveyed students and revealed that 81% had already employed AI tools for scientific purposes, but only 5% were proficient at crafting effective AI prompts. After completing the module, that number grew to 48%.

Recognizing the importance of proper AI prompt construction, I subsequently taught classes on AI-assisted grant writing, integrated AI into the scientific training of my research lab, and shared preprints outlining AI use and teaching strategies. Through this experience, I found that one doesn't require a computational background to utilize AI effectively.

My research focuses on the biochemistry of DNA replication and repair, but AI remains a versatile tool for various fields when applied with the scientific rigor already possessed by practitioners.

A crucial strategy I developed is treating each AI prompt as an experiment, with the model's output serving as the result. To conduct these experiments effectively, follow these ten tips:

1. AI language models can only reason about information they have access to, so ensure comprehensive knowledge by employing web search, particularly for biomedical work. The Model Context Protocol (MCP) facilitates connecting AI models to external systems, such as literature databases like PubMed, Scite, and Elicit.

2. Structure your prompts for better processing. Unstructured text is more challenging for models to parse than structured text. Markdown formatting is universally compatible and often sufficient, while XML and JSON offer more complexity and rigor when needed.

3. Begin AI experiments with standardized protocols, akin to scientific methods. The Model itself provides a built-in framework for crafting initial prompts. Describe your desired outcome in plain language and request the model to generate an optimized prompt, then run that prompt in a new conversation (meta-prompting).

4. Leverage existing validated prompts to avoid reinventing the wheel. Vetting prompts from others saves time and effort. I have compiled a set of prompts for common scholarly tasks on Zenodo and maintain an online library for free use and adaptation.

5. Be aware of conversational limitations. Models' performance declines as conversations lengthen and they cannot forget previously stated information. To address this, create a task within a conversation, have the model condense the results into a single prompt, and then execute that prompt in a fresh conversation.

6. Understand the probabilistic nature of AI model outputs. Repeating the same prompt in separate conversations reveals robustness and helps improve the completeness of answers. Run prompts multiple times to ensure the best possible results.

7. Never assume that a single run yields complete information. Continue running prompts until repeat runs do not yield new answers, ensuring a thorough investigation.