The Four Ghosts in Your Data Warehouse (And How to Evict Them)
Forgotten copies, full price. Pull up a chair, grab some chai, and let me tell you about a quiet disaster waiting in your cloud bill. $9,847. In exactly 22 seconds. That is what running a single bad query three times cost on Google BigQuery (Google’s managed cloud data warehouse—basically a massive analytical database built to crunch terabytes of reporting data in seconds). This was documented…
The headline reads: The Four Ghosts in Your Data Warehouse (And How to Evict Them).
The source material explains that a single bad query on Google BigQuery cost $9,847 in just 22 seconds. This happened because BigQuery bills based on the volume of data scanned, not the output shown. The engineer thought a LIMIT clause would save money, but it didn't.
The source identifies four main culprits for high warehouse bills: forgotten tables, copy sprawl, zombie snapshots, and temporary tables that never expire. Forgotten tables are one-off data pulls, abandoned machine learning experiments, and leftovers from pipelines that were migrated. Copy sprawl involves multiple tables with similar names, created as safety nets during updates.
Zombie snapshots are backup table snapshots left untouched indefinitely. Temporary tables that never expire are staging tables created by ETL pipelines without cleanup steps.
The source provides an SQL query to identify tables that haven't been queried in the last 180 days, helping to find the four ghosts. These forgotten tables, copy sprawl, zombie snapshots, and temporary tables that never expire are all expensive in the long run, even though storage costs are cheap. The source stresses the importance of finding what nobody queries to clean up unused tables and reduce costs.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
