NTA paper-setting process: From question bank to sealed sets, how exam papers are prepared

The National Testing Agency (NTA) employs a multi-stage system to create examination papers, starting with a vast question bank and concluding with the printing, distribution, and secure storage of multiple question sets. This procedure has been under evaluation following the NTA's decision to re-conduct the UGC-NET exams in English, Commerce, and Sociology due to "numerous errors" in the tests administered between June 22 and June 30.

The agency contends that these "defects" render the papers "inadequate for fairness and error-free assessments," necessitating a re-examination on September 9. English will be administered from 9am to noon, while Commerce will be held from 3pm to 6pm.

The NTA's question-paper preparation involves several key stages. Initially, subject experts from prestigious academic institutions generate questions and multiple-choice answer options, unaware of the eventual utilization of their work. These questions are then evaluated by an additional team of subject experts, who choose appropriate questions for various paper sets, ensuring they align with the syllabus, possess correct answers, and hold generally consistent difficulty levels.

This process is structured so that no individual has comprehensive access to the complete examination paper. The following stage is vetting, where examiners independently solve the selected question sets and compare their answers with those proposed by the moderators. Any inconsistencies between the solutions are addressed by the examination director through a meeting between the moderators and vetters.

The government's affidavit highlights that experts primarily handle individual stages, with multiple sets finalized "entirely independently" and without any shared questions or experts.

Following the finalization of the sets, they are translated into the languages used for the examination. Subject experts perform the translations, which are subsequently back-translated by other experts to verify accuracy and prevent contextual loss. The NTA conducts tests in 13 languages, and a secure AI-assisted translation environment has been established.

This environment prevents internet access, use of personal devices, external storage media, or cloud connectivity for translators. The time allotted to each translator is also limited. Panels of senior faculty from various institutions participate in the translation process for approximately a week, surrendering their mobile phones and other digital devices upon entry.

Each panel consists of a convenor and four subject specialists, who work individually and are unaware of the questions being prepared by other panels. The panels create extensive pools of questions rather than the final examination paper. The final sets are generated through algorithm-based randomization, ensuring that the composition of the paper remains unknown until the final generation stage.

Despite the meticulous processes in place, errors such as spelling mistakes and repeated questions can occur. A central university faculty member involved in the UGC-NET question-setting attributes these issues to typos, oversight during proofreading, and translation. The errors stem primarily from contractual staff responsible for these tasks.

Additionally, repetition can occur if portions of an earlier question set are reused and subsequently mixed with questions from a new set. Finally, once the question sets are translated and finalized, they undergo printing, packaging, transportation, and secure storage concurrently, as per the government's affidavit. This parallel process also allows NTA to substitute a potentially compromised set if information about possible compromise reaches the agency before the examination.

The final decision on which set will be administered is made on the morning of the examination, with only one set being used, while the others remain sealed.