Single log line is 49KB+ (ext4) / 110KB+ (btrfs) of systemd-journald disk writes

The issue of systemd-journald generating excessive disk writes was reported in version 257.9 of the systemd component. The problem was observed on a Debian 13 distribution using Linux kernel 6.12.57. The unexpected behavior was that the virtual machine experienced around 50 input/output operations per second (IOPS) while writing just two lines of logs per second. This issue mirrors a similar problem reported on GitHub (https://github.com/systemd/systemd/issues/15292) that was closed without a satisfactory resolution.

To reproduce the issue, the steps were as follows: first, use journald in a mode that writes to the hard drive, preferably on an XFS file system. Next, ensure a constant stream of log entries is generated on a virtual machine. Finally, observe the virtual machine's IO traffic. The problem was illustrated by using a virtual machine, as the original complaint on GitHub suggested that iotop is not accurate due to it being before any OS write coalescing.

However, the observed traffic after each kernel mechanism was used confirmed that the issue was not due to the kernel making excessive IOPS. The problem lay in the inefficient format used by systemd-journald, which resulted in files being significantly larger than the actual content written. This inefficiency also led to potential data corruption issues following an unclean reboot, making the format less resilient.