Urgent.News

What's breaking now, across thousands of outlets.

Tech

Production Logged Zero Errors. Every Call Was Refused First.

On the morning of November 13, nine calls to our server failed in a row. Our logs for that morning had zero errors in them. No 5xx all day. Both of those things were true at once, and it took us most of that day to understand how. I build Since, which tracks the packages, AI models, APIs and runtimes your code depends on, tells you and your coding agent what changed and when it breaks, and opens…

On November 13, nine consecutive calls to the server failed without any logged errors. The monitoring system seemed fine, with no 5xx errors and all endpoints responding. However, the root cause was hidden in the status codes: every call was refused first, then accepted. The logs showed a pattern of 400 errors before successful 200 responses.

The monitoring system missed this pattern, as 400 errors were logged at the info level and not treated as errors. The issue stemmed from the connector using a newer protocol revision that the server didn't support. The server refused the call with a 400 error, and the client fell back to an older revision, which worked upon retry.

Additionally, public scans running on the server's main thread caused health checks to fail due to time-outs, leading to dropped connections. The incident highlighted the importance of proper logging and monitoring of status codes, as well as the need to align protocol revisions between the server and client.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Terraform Import: Import Blocks, Commands and ID Formats

Terraform import does 1 thing: it ties a resource that already exists to an address in your Terraform state. Bottom line Use an import block.

  • Terraform import connects existing resources to state, introduced in 1.5 (June 2023)
  • Use import block with resource type and ID for known resources
  • Import process writes state entry, doesn't change cloud resources

The 10DLC Onboarding Process, Step by Step

Application-to-person (A2P) SMS over the major US carrier networks now requires brand and campaign registration via The Campaign Registry (TCR). The process — known industry-wide as 10DLC, for "10-digit long code" — has been mandatory in some form since 2021 and has progressively tightened.

More from Tuesday 17 November →