Continuous Integration and Software Quality in Scientific Software: A Large-Scale Empirical Analysis of GitHub Repositories

Bioinformatics software is indispensable in biological research; however, a significant number of academic projects lack adequate testing and continuous integration (CI) practices. This shortfall can lead to issues with reproducibility, accessibility, and long-term maintainability of the software. To investigate the potential impact of CI, software testing, and overall software quality on the academic software landscape, a team of researchers performed an empirical analysis of GitHub repositories focused on bioinformatics and other scientific software domains.

They utilized GitHub's Application Programming Interface (API) for domain-specific search terms to discover repositories. Subsequently, they employed a multi-layered data extraction pipeline, incorporating a Large Language Model (LLM) validation step, to connect repositories with published works, institutions, authors, funding information, and citation counts.

By combining repository activity, community engagement, and backlog health into a composite Repository Quality Index (RQI), the researchers discovered a significant association between the presence of CI and software tests and higher repository quality. Furthermore, they observed that repositories with higher academic citation counts tend to group within higher RQI tiers, implying that software quality, which is bolstered by proper testing and integration, is linked to academic impact and funding status.

In summary, the study underscores the fact that incorporating industry-standard development practices, such as CI and software testing, can be a practical route to enhance the sustainability and impact of academic software, as well as open up more funding opportunities.