Parsing WhatsApp chat exports: what real files taught me

WhatsApp chat export files appear simple at first glance, featuring a date, name, and text per line. However, after analyzing 28 real exports from public open-source repositories and generated files across numerous locale and platform profiles, several complexities emerged.

Firstly, the ordering of dates in the file is not explicitly stated. The date format depends on the phone's settings at export time, making it crucial to consider this factor when interpreting timestamps. For instance, if a date field exceeds 12, it follows a day-first format; conversely, exceeding the second field suggests a month-first format.

If the first field is a four-digit number, it indicates a year-month-day format. However, a chat ranging from the first 12 days of a month may remain ambiguous, and in such cases, it's best to directly inquire with the person involved.

Another key observation is that times in the export file lack time zone information. This means that if two individuals from different time zones export the same chat, the recorded timestamps will differ. Thus, the user should be responsible for confirming the time zone, and this information should be clearly indicated in the output.

Furthermore, exports do not record the user's name associated with the exported chat. Instead, they contain display names. In a one-to-one chat, the file name (e.g., "WhatsApp Chat with Anna.txt") may provide a hint about the other participant. However, the file name should be presented as a suggestion and left for the user to determine.

The text within the export file may also contain invisible characters like the left-to-right mark (U+200E). This can lead to potential parsing issues, such as interpreting a notice about end-to-end encryption as a message from a non-existent person. Similarly, bidirectional override characters (U+202D and U+202E) can alter the text display order, necessitating a visible representation of these characters for accurate representation.

Moreover, WhatsApp encrypts both messages and calls end-to-end, which must be preserved in the output. It's important to handle U+202D and U+202E characters carefully to avoid misinterpretation of the displayed text. The encoding and line endings may also vary, including formats like CRLF, BOM, UTF-16, and even invalid UTF-8. A defensive decoding approach should be adopted, with proper reporting of unreadable portions.

The phrase "Message deleted" appears in various languages, including phrases like "This message was deleted" and "edited" in the phone's native language. While a phrase table can assist in handling these instances, it may never be entirely comprehensive. In such cases, keeping the original text without labeling it as "deleted" or "edited" is the best course of action.

Attachments within the export file pose their own challenges. The text associated with an attachment may name a file, and the accompanying zip file may contain multiple files with the same name in different folders. Matching the exact file name first and treating ambiguous cases as unavailable with appropriate notes is essential. Additionally, the zip file should not be blindly trusted, and entry count, total size, and compression ratio should be capped when receiving exports from external sources.

It's also important to note that the exported file is plain text and can be edited by anyone using a text editor. Therefore, it's not advisable to treat it like a database read. Instead, record the SHA-256 hash of the exported file, and clearly state in the output that it originated from an export. Lastly, it's worth mentioning that a local iPhone backup, located at ChatStorage.sqlite within the domain AppDomainGroup-group.net.whatsapp.WhatsApp.shared, offers an alternative source for chat data.

However, this should not be considered legal advice, and for court-related matters, it's crucial to consult with a legal professional or court clerk.