Assessing measurement uncertainty at laboratory network scale and its impact on diagnostic performance in the absence of a gold standard: application to ELISA tests for Coxiella burnetii in ruminants

Measurement uncertainty can influence whether an individual is classified as positive or negative, thereby affecting the diagnostic performance of a test. Due to the lack of a gold standard, it is challenging to apply existing methods to assess measurement uncertainty and its impact on diagnostic performance. In this study, a new method was developed that can be applied to any quantitative diagnostic test lacking a gold standard, and it was utilized to evaluate ELISA tests for Coxiella burnetii serology in ruminants.

The researchers employed a mixed-effects model to analyze data obtained from an inter-laboratory proficiency testing, allowing them to estimate measurement uncertainty. They then calculated sensitivities and specificities while considering the measurement uncertainty, taking into account each individual's true serostatus and the probability of a positive result when retested in another laboratory.

Additionally, the study estimated the sensitivity and specificity of each laboratory and batch, considering their respective biases. The findings indicate that the ELISA test's sensitivities and specificities were only slightly impacted by the measurement uncertainty, but they varied among laboratories and batches. Consequently, it is essential to standardize analytical practices across laboratories and calibrate batches.

Furthermore, the extent to which measurement uncertainty influences diagnostic performance depends not only on the inter- and intra-laboratory standard deviations but also on the cut-off position relative to the distribution of test values within the population. Therefore, the cut-off position, in addition to the analytical performance of a test, is a crucial factor to consider when evaluating diagnostic tests.