Urgent.News

What's breaking now, across thousands of outlets.

Tech

Chaos engineering with Dynatrace: measure before you inject a failure

Before scheduling a chaos engineering with Dynatrace experiment in production, one question matters: « if I slow down or cut this component, how will I know within ten minutes that the experiment worked, or that it must stop? » Without a numeric answer, a deliberate failure is just a failure. This article starts from that operations decision. It does not describe a fault-injection tool: in the…

In a recent article, Dynatrace shares their approach to chaos engineering, which involves measuring before injecting failures in production. The primary question to consider is whether one can determine within ten minutes if an experiment was successful or if it should be stopped. The article emphasizes that a chaos experiment should be treated as a hypothesis test, with a clear contract outlining what will be broken, what is expected, and the threshold that will trigger an immediate stop.

To run a successful experiment, start by establishing a baseline over a normal period, such as seven days, ensuring the traffic level is comparable to the experiment. Record key metrics, including failure rates and response times, as well as the number of open problems and their usual duration. If the baseline is already poor, it is advisable to postpone the experiment.

Next, create two platform building blocks to turn the hypothesis into checkable objectives: a Service Level Objective (SLO) and an Service Reliability Guardian (SRG) guardian. The SLO measures an SLI, which is a series normalised between 0 and 100% computed using a DQL query, compared against a target over an evaluation period. The error budget, the gap between the status and the target, helps determine how fast the error budget is being spent.

The SRG guardian groups objectives, each fed by a DQL query and judged against warning and failure thresholds, serving as the referee for the experiment.

Once the objectives are set, automate the verdict using a workflow. This workflow can call the guardian action after the injection window, using an event trigger or an on-demand call. The result will expose a validation status, such as pass, warning, fail, or error, along with an identifier and a link to the validation. Decision-making, such as widening the blast radius or stopping the experiment, is based on this validation status.

In conclusion, chaos engineering with Dynatrace begins with measurement—setting up a baseline, stop thresholds, and an automated verdict. Only then can the failure be introduced.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

How to handle refund requests in a subscription app

This guide was originally published on the Hailbell blog . Short answer: iOS: you can't refund App Store purchases. Apple decides.

  • Refund handling depends on store (App Store, Google Play, or own website)
  • Verify store, plan status, and charge details before responding
  • Prepare policy statement for consistent refund communication

BugMeNot Calls It Quits

BugMeNot, the password-sharing service launched in 2003 to help users bypass website registration requirements, has shut down without explanation after more than two decades.

Designing a Mobile Sales CRM Dashboard for Fast Decisions

Enterprise sales representatives use their phones between meetings, while full desktop CRMs hold more data than a small screen can comfortably display.

  • Mobile CRM dashboard answers quota progress, imminent closing deals, and overdue tasks
  • Streamlined design focuses on clarity, high-level metrics over detailed records
  • Visual hierarchy, color, spacing, typography, and accessibility ensure usability

More from Tuesday 17 November →