Growing Pains with Five Repositories: Gitlinks and Dual CI
Usually, infrastructure stories start with the words: "Our project got too big, and we decided to split it into parts." But with us, everything was different. From the very first minute of the project, we consciously chose an architecture consisting of a scatter of independent repositories. The Pipeline , engine , design-docs , and user-docs projects were created almost simultaneously, and the…
The project began with the decision to create a scattered architecture of independent repositories from the very first moment. This choice aimed to provide code cleanliness, CI isolation, independent histories, and separation of access rights. Git Submodules formed the foundation of the system from day one, with a Gitlink trap lurking beneath the surface.
When editing code within the submodule folder, the workflow required two pushes: one from the submodule itself and another to update the pointer (gitlink) at the root of the parent repository. This discipline was easy to forget, leading to issues when the local commits were never pushed to the developer's laptop. Simultaneously, the engine repository had two personalities: a standalone repository and a submodule inside Pipeline.
The clash between these two personalities resulted in missing dependencies during standalone checks and hard AssertionError errors when tests attempted to find parent artifacts. To address these issues, a marker hack was introduced, with tests checking for the presence of a .workspace_config directory at the ascended level. If the directory exists, it signifies a nested context, and tests proceed.
Otherwise, tests are skipped using pytest.mark.skipif, preventing failures and maintaining a clean status.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
