{
  "id": 8813904,
  "title": "How Databricks Serverless Compute Cost My Team $14k in One Weekend",
  "url": "https://urgent.news/2026/09/21/how-databricks-serverless-compute-cost-my-team-14k-in-one-weekend",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-21T01:20:58.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/aniketsoni/how-databricks-serverless-compute-cost-my-team-14k-in-one-weekend-1f9g"
  },
  "original_language": "en",
  "account": "On a Sunday at 2:14 AM, a PagerDuty alert woke the reporter with a jarring frequency. Instead of a standard usage notification, the alert read: \"you've hit 80% of your monthly cloud spend in 48 hours.\" The reporter opened the Databricks console and saw that the sql_warehouse_prod_v2 was burning DBU (Databricks Units) like a crypto-mining operation. The team had recently shipped a new pipeline on Friday, and while everything looked fine, the bill was skyrocketing. The dashboard showed a flat line for three months, followed by a steep spike. The reporter first assumed a runaway loop in a Python job, but found no evidence. Checking the spark_query_history yielded no unusual patterns. The issue lay in the Auto-stop setting: 10 minutes. The reporter realized the warehouse wasn't idling but being kept alive by a ghost. The heartbeat pings from a BI tool, configured with broad CAN USE permissions, were causing the warehouse to remain active. The Auto-stop only triggered when the warehouse was truly idle. With the heartbeat hitting the warehouse every 8 minutes, the 10-minute timer never reset. The warehouse was being held hostage by a silent, low-latency heartbeat. The team fixed the issue by killing the connection, reducing the warehouse size, updating the connection string, and implementing a Tag policy. They also introduced a Budget Alarm to monitor daily burn rates.",
  "summary": "It’s Sunday, 2:14 AM. The PagerDuty alert hits my phone with that specific, jarring frequency that makes your stomach drop before you’ve even opened your eyes. My Databricks billing alert wasn’t a standard \"usage threshold reached\" notification; it was the \"you’ve hit 80% of your monthly cloud spend in 48 hours\" panic text. I sat up, opened the Databricks console, and stared at the Billing page.…",
  "key_points": [
    "PagerDuty alert revealed 80% of monthly cloud spend in 48 hours",
    "Auto-stop setting at 10 minutes failed to stop warehouse due to heartbeat",
    "Issue resolved by killing connection, updating settings, and implementing Budget Alarm"
  ],
  "editors_take": "The incident shows that a subtle misconfiguration, combined with a loosely permissioned tool, can quietly drive up costs, highlighting the need for tighter controls and monitoring in cloud-based services.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}