How To Report A Bug So It Actually Gets Fixed

Writing a bug report that actually gets fixed is an important skill for software engineers. Watching a video from Kovarex, the founder of Factorio, inspired me to create a step-by-step guide on how to report a bug effectively. Here is my account of the process:

1. Always assume you are wrong. When faced with a problem, it's essential to acknowledge that it may be our code causing the issue. In my case, I investigated whether the thousands of errors in my production service logs were due to a memory leak. After ruling out my code, I realized the importance of this initial step, as it often leads us to a faster resolution.

2. Make it deterministic. Random issues are usually the result of incomplete understanding. In the case of Netty, a networking library used in the Azure OpenAI SDK, the leak appeared sporadically due to garbage collection. To reproduce the issue, I created a reproduction that ran a hundred concurrent requests and called System.gc() every time a request failed. This approach helped me pinpoint the exact moment the issue occurred.

3. Bisect versions. Git bisect is a powerful tool that allows us to quickly identify the commit responsible for a bug by jumping between versions. In my case, I observed that reactor-netty-http version 1.1.23 was working correctly, while version 1.1.24 caused the leak. This process narrowed down the problematic version.

4. Shrink it into a public repository. Sharing a minimal, reproducible example is crucial for bug triage. I created a Gradle project with a single test case, precisely pinning the versions that exhibited the issue. This allowed maintainers to verify the bug without relying on my assumptions. Having a public repository also enables them to check whether the issue persists after the fix is deployed.

5. File the report at the right place. After identifying the source of the issue, I filed the bug report with the appropriate team. In this case, it was reactor-netty, as the bisecting between reactor-netty versions revealed the origin of the problem. When the report was initially closed, I engaged with the Azure SDK for Java maintainers to understand the connection between the two issues and the subsequent fix.

6. Do the archaeology. This optional step involves digging deeper into the project's history to gain context about the bug. In my investigation, I discovered a thread from five months earlier related to the same issue and the 1.1.24 connection lifecycle change. Through the efforts of reactor-netty maintainer violetagg, I found a GitHub thread that provided valuable insights into the bug's history. This information helped me understand the root cause and contributed to a quicker resolution.

7. Write the report you'd want to receive. A well-crafted bug report can significantly streamline the triage process. Include all relevant information, such as the steps to reproduce the issue, the environment details, and the observed behavior. Avoid leaving out crucial details that might hinder the maintainers' ability to assist you effectively. By presenting your report in a clear and comprehensive manner, you increase the likelihood of a swift and satisfactory resolution.