Urgent.News

What's breaking now, across thousands of outlets.

Tech

Time-Series Cardinality: Why One More Indexed Column Costs More Than a Million More Rows

Learn how high-cardinality time-series data affects PostgreSQL index size, query planning, and ingest performance, and how schema normalization can help.

Time-Series Cardinality: Why One More Indexed Column Costs More Than a Million More Rows

Time-series cardinality refers to the number of distinct values a dimension can take. The number of distinct series a database tracks is determined by the product of distinct counts across all indexed dimensions. For example, 10,000 assets, 100 sensors each, 10 firmware revisions, and 100 sites result in roughly a billion combinations.

This definition splits growth into two axes: linear and multiplicative. Adding values to a dimension you already index leads to a linear increase in series count, while adding a new indexed dimension results in a multiplicative increase. The cost of one more indexed column in Postgres is proportional to the existing number of rows and the width of the new column.

This cost scales with every row you already have, which can be true or false depending on the baseline. Adding new indexed dimensions can cause the planner's row estimates to collapse due to correlated selectivities, leading to a sequential scan instead of an index scan. The root cause of this issue is the same across different engines, with tag-indexed engines scaling memory and startup time with series count, while wide Postgres schemas experience index bloat, longer autovacuum times, and planner statistics drift.

The cost of the new column becomes more apparent as the table grows, and the planner's estimates become less accurate due to correlated predicates.

Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at hackernoon.com →

More in Tech

How i learn to how to learn?!😵

I learned about HTML, CSS, JS. Mostly I start learning using tutorials. Yeah, it's hell. At the end of the video, I learned about nothing, just a few pieces of memorized knowledge.

  • Learner acquired HTML, CSS, JavaScript skills through tutorials.
  • Dev challenge taught problem-solving without AI assistance.
  • Advice: Study others' code, utilize resources like w3schools and MDN.

A Fly’s Brain Is Running in a Computer… Human Brain Is Next? 💀

Scientists have mapped a fruit fly's brain in incredible detail and are using that information to build computational models of its neural network.

  • Scientists mapped fruit fly brain with high precision
  • Potential future: AI training human brains for knowledge
  • Human brain simulation and consciousness upload not yet possible

More from Wednesday 16 September →