I benchmarked 5 graph databases. The first four hours measured the Indian Ocean.
A reproducible comparison of CognoDB Cloud, Neo4j AuraDB, Memgraph, FalkorDB and ArangoDB, and the two things that nearly made it lie. I was asked to benchmark CognoDB Cloud against four other managed graph databases. Same data, same workloads, honest numbers. I figured the hard part would be writing fair queries across five different engines. It wasn't. The hard part was realising that my first…
Five graph databases were benchmarked in a reproducible comparison: CognoDB Cloud, Neo4j AuraDB, Memgraph, FalkorDB and ArangoDB. The benchmark aimed to measure performance of these managed graph databases using the same data and workloads. However, the results were not as straightforward as initially expected. While FalkorDB showed impressive speed at indexed lookups, it was found that CognoDB was simply running on a distant server in Northern Virginia, making the latency results meaningless.
Confusion arose when the first clean-looking results table was mostly a map of undersea internet cables. The benchmark revealed that the databases were located in different regions, which affected the comparison. The benchmarking process also uncovered that CognoDB builds indexes in the background, which can cause issues during data loading.
A critical bug was discovered where the database returned no data during the index-building phase, resulting in zero relationships being created. This issue was only resolved by polling the server for updates after the indexes were ready, ensuring accurate data representation. The scaling tests also showed unexpected results, revealing that trusting the count of rows submitted was not enough, as the database's own counters should be trusted for accurate information.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
