Information regimes in population-graph neuroimaging: cohort context and implementation-dependent evaluation sensitivity
Population-graph models aggregate a query subject with an unlabeled cohort, so their reported performance can depend on the evaluation regime rather than on predictive information alone. Evaluation context has two separable axes: the visibility of the cohort during training, and the context available at inference. We study the training-time axis under a protocol fixed before analysis, on an…
Population-graph models, which aggregate a subject with an unlabeled cohort, can show varying performance based on the evaluation regime instead of solely on predictive information. Two key factors influencing this regime are the visibility of the cohort during training and the context available at inference. The study focused on the training-time axis by conducting experiments on an ABIDE-I cohort consisting of 871 subjects from 20 acquisition sites.
Two different implementations of population-graphs were compared: a learned/adaptive graph (EV-GCN) and a static weighted graph (Parisot-GCN). Throughout the study, the cohort, connectivity features, outer splits, and training selection rules remained constant, allowing for the variation of only the information available to the graph.
The primary estimand analyzed was the interaction between the implementation and the training context, specifically the difference in Area Under the Curve (AUC) between the two implementations. Across three preselected outer splits (666, 1024, and 2024), the direction of the interaction was consistently positive, with interaction ranges between 0.06285 and 0.13090.
To assess uncertainty, paired subject and site-cluster bootstrap resampling was performed within each split, and the results were reported per split without pooling or averaging across splits. The findings emphasize the importance of explicitly reporting evaluation information regimes, as they tie reported population-graph performance to the context in which the cohort is visible.
The research also introduces a two-axis taxonomy of information regimes, provides preselected cross-split evidence linking sensitivity to cohort context to implementation, and offers a reusable reporting checklist.
Rewritten by urgent.news; may contain errors — check the original.