{
  "id": 5302503,
  "title": "Five Times Pixbu’s Tests Measured the Wrong Thing",
  "url": "https://urgent.news/2026/09/03/five-times-pixbus-tests-measured-the-wrong-thing",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-03T10:00:02.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/five-times-pixbus-tests-measured-the-wrong-thing?source=rss"
  },
  "original_language": "en",
  "account": "1. During early development of the self-care app Pixbu, the author discovered that measuring the wrong dimension caused the app's creature to appear to shrink during evolution. The initial tests showed progress, but upon counting the actual pixels, it was revealed that the creature had shrunken by 14% during the first evolution stage. The tests remained green because they were measuring the stage sizes from a metadata file, not the actual art, which still needed to be measured after the update.\n\n2. Another issue arose when trying to ensure a critical function was called within the app. The developer had a guard test in place to verify this, which passed for weeks. However, when the call was wrapped in an if (false) statement, the test still remained green, indicating that the test was only checking if the call existed in a file, not if it was actually executed. The developer now ensures that their mutation tests fail when they should, preventing false positives and wasting CI time.\n\n3. The developer initially believed that there was no way to check if a store update was approved. They wrote in their notes about the impossibility of testing this, and for a while, their assertion held true. However, after an hour of further investigation, they found that they had simply been reading the store page's status, while the update approval information was available in the HTML of the store page itself. To avoid this mistake, the developer now writes down the exact date, command used, and its output for any such impossibility claims, ensuring they remain valid and relevant.\n\n4. Lastly, the developer encountered a situation where a revenue dashboard showed fake money due to being in the sandbox mode. They saw $1,573 in revenue and 136 customers, but after toggling the sandbox data, the actual revenue was $0 and there were only 22 installs. The developer learned to always question the source of their information and double-check the context of the figures. By asking themselves if they were measuring the right thing, if the check could fail, and if the measurement or the memory was being recorded, they were able to avoid making false claims and wasted time in their testing process.",
  "summary": "Pixbu’s tests passed while its creature shrank, its guards asserted nothing, and its revenue was sandbox data. Better measurements changed the launch.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}