Ten fine-tuning mistakes I see students make (and made myself)

Many students make common mistakes when fine-tuning models, mistakes I have also made. These errors prevent good results and wasted effort. The first mistake is not having a baseline. Without comparing the fine-tuned model to the original unmodified version, it is impossible to know if the fine-tuning improved performance. Always start by fine-tuning a model on the same evaluation set first.

Next, train on validated data, not unvalidated data. Remove malformed records, duplicates, and template mismatches before fine-tuning. Do not leak the evaluation metric into the training process. The train set and test set must be separate. Do not chase the training loss. It declines regardless of model improvement. Instead, monitor the evaluation loss and read the outputs to see true progress.

Use too few epochs on too little data. A few hundred examples for ten epochs is essentially memorization without real progress. Choose a big model to fine-tune, not the biggest model that fits your hardware. A 1–3B model fine-tuned well outperforms a 7B model poorly fine-tuned. Avoid fine-tuning for facts; retrieval (RAG) is better for fact-based queries.

Ignore the chat template if you're training for chat. Training with one format and serving with another leads to a model that forgets everything. Do not skip saving checkpoints. The best model may be at step 600, but you only kept step 1200. Have a deployment plan before you start fine-tuning. An adapter on a laptop does not help anyone else.

Decide how the fine-tuned model will be served before beginning. Write the evaluation before training. Define the held-out set, rubric, and baseline numbers in advance. Everything else becomes an experiment against a fixed target instead of an open-ended experiment. This habit, more than any hyperparameter, is what prevents most mistakes.

Learn from these mistakes on a small model once, on a weekend. Then never repeat them. Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA. He ships production RAG, fine-tuning, and agentic AI systems, mentors 200+ students, and judges student hackathons across India.