Urgent.News

One page, thousands of outlets. See who else covered it.

Editions

More in Tech

Why Go's encoding/csv Burns 540MB on 5M Rows (and How I Fixed It)

TL;DR: Go's built-in encoding/csv.Read() allocates string slices and strings for every single row, generating 10M+ allocations and burning 540 MB on a 5M-row file.

  • Go's encoding/csv allocates memory for []string slice and heap strings per row
  • This causes significant memory usage, up to 540MB for 5M-row file
  • go-zerocsv fixes this with in-place typed scanning and compacting buffer

More from Tuesday 18 August →