{
  "id": 13970978,
  "title": "Parquet vs CSV: what you actually save, and why",
  "url": "https://urgent.news/2026/10/12/parquet-vs-csv-what-you-actually-save-and-why",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-12T14:00:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mohammed_arshadansari_f2/parquet-vs-csv-what-you-actually-save-and-why-2h97"
  },
  "original_language": "en",
  "account": "The debate between using Parquet and CSV for data storage continues to be a topic of discussion in the data engineering community. While many recommend using Parquet, the reasons behind this choice are often not fully explained. However, there are three mechanical reasons why Parquet is considered superior, and understanding them can help determine when it may not be the best choice.\n\nFirstly, Parquet only reads the columns that are requested. In contrast, CSV files read every column, row by row, which can be inefficient when only a subset of columns is needed for analysis. This is because CSV stores data row by row, so to retrieve information from a specific column, the entire file must be read and parsed. On the other hand, Parquet stores data column by column, so requesting a specific column requires only accessing that column, resulting in significantly faster read times. This becomes particularly advantageous when dealing with wide tables, where the majority of columns are not needed for the analytical query.\n\nSecondly, Parquet columns that share similar data types and structures are stored adjacent to each other, allowing for more effective compression. Compression is a technique that reduces the size of data by identifying and eliminating redundancy. CSV, on the other hand, stores data as a single string, making it difficult to compress. In contrast, Parquet encodes data at the column level, using different encoding methods for each column. For example, run-length encoding is used for data with repetitive values, while dictionary encoding is used for low-cardinality strings, such as city names. Delta encoding is applied to timestamp data, and general compression is applied to the resulting encoded data. This column-level compression results in significantly smaller file sizes compared to CSV files, especially when dealing with tables containing a mix of data types.\n\nThe compression ratio achieved with Parquet depends on the specific data stored in each column. For instance, a table containing boolean values can be compressed by up to 100x, while timestamps can be compressed by up to 50x. Low-cardinality strings, such as city names or user IDs, can be compressed by up to 30x, while high-cardinality strings, like free-form text, may only achieve compression ratios of 2-5x. Float and measurement data typically see compression ratios between 2-4x, while transactional data and sensor data may achieve compression ratios of 4-8x. It is essential to understand that the compression ratio depends on the schema and the data itself, and measuring your own data can provide a more accurate understanding of the potential savings.\n\nFinally, the potential savings with Parquet can be substantial. For example, a table containing synthetic user data compressed from approximately 635 MB in CSV format to about 29 MB in Parquet, representing a compression ratio of roughly 22x. Similarly, ten million NYC taxi rides can be compressed from 2.1 GB in CSV format to 180 MB in Parquet, achieving a compression ratio of around 12x. Event logs, which usually contain a mix of data types and structures, often land at compression ratios of 15 to 25x, while transactional data can be compressed by 10 to 15x. Float-heavy sensor data, on the other hand, may achieve compression ratios of 4 to 8x.\n\nIn conclusion, while CSV files have been widely used for data storage, Parquet offers several mechanical advantages that make it a more efficient choice for many use cases. By only reading the columns needed, storing similar data types adjacent to each other, and applying column-level compression, Parquet can significantly reduce file sizes and improve data processing times. However, it is crucial to consider the specific nature of your data and measure your own compression ratios to determine whether Parquet is the best choice for your particular use case.",
  "summary": "\"Just use Parquet\" is the most repeated advice in data engineering and the least explained. It is good advice. But if you do not know why it wins, you cannot tell when it will not, and you end up either converting files that should have stayed CSV or being surprised when your ratio is 3x instead of the 20x you read about. There are three mechanical reasons, and none of them need a paper. 1. It…",
  "key_points": [
    "Parquet reads only requested columns, unlike CSV which reads all columns.",
    "Parquet's columnar storage and compression achieve smaller file sizes.",
    "Compression ratios vary by data type, with potential savings up to 100x."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}