300問の問題集をAIで自動生成！ 250問目でAPIエラー、75分が水の泡に。チェックポイント機構で“途中再開”を実装した話
最近、ある資格試験の勉強をしてて、公式の問題集だけじゃ足りないなと思ってた。じゃあ作るか、と。GenAIを使えば、類題なんていくらでも作れる時代だし。 早速、試験のシラバスを食わせて、章ごとに問題と解説を300問分生成するスクリプトを書いた。 python generate_questions.py を叩いて、あとは待つだけ。プログレスバーがぐんぐん伸びていくのを見ながら、「いやー、便利な時代になったもんだ」なんて思ってた。 実行開始から75分後。プログレスバーは8割を超えたあたり。そろそろ終わるかな、とコンソールを覗き込んだら、見たくない赤い文字が表示されてた。 requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by…
An aspiring test-taker sought to generate practice questions for an exam using AI. They leveraged a GenAI API to create 300 questions and their explanations. A script, `generate_questions.py`, was written to execute this task. After running the script for 75 minutes, the progress bar indicated almost completion, but then a critical error occurred.
The script produced no output, leaving 250 questions ungenerated. The cause was the designer's naivety in processing everything at once. The script created a list of all questions, appending each new question to a list as it was generated. If the process was interrupted, all questions in the list would be lost, as the memory couldn't retain the data.
The solution was to implement a checkpoint mechanism, saving progress at regular intervals. The script was modified to check for a checkpoint file upon startup, read the file contents to determine which chapters were completed, and skip processing of completed chapters. Intermediate results were saved to a JSONL file after each chapter was processed, ensuring that the script could resume from the last successful completion point in case of interruption.
This change made the script robust, allowing it to recover gracefully from interruptions and continue from where it left off. The lesson learned was the importance of designing for interruptions, especially in long-running batch processes that involve external APIs or unreliable external services. This approach, known as checkpointing, not only improves reliability but also makes individual system failures less disruptive, as the system can recover without needing a full restart.
The speaker shared their own open-source tool, `rag-faq-api`, which is a provider-agnostic Retrieval-Augmented Generation Q&A API, MIT-licensed on GitHub.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
