Urgent.News

What's breaking now, across hundreds of outlets.

Tech

A 10-email thread holds 487 tokens of actual text. With quoted replies the model reads 5,068.

You build a small tool that summarizes support tickets. Each ticket is an email thread, and your help desk stores every message the way it arrived. You join the messages, send them to a model, and get a good summary back. The token count in the response is about ten times what you expected from looking at the conversation. Nothing went wrong. The model read exactly what you sent, and most of it…

A support ticket summary tool built by the reporter reveals an unexpected reality about email threads. Each thread is stored as a separate unit in the help desk system, with every message retained in its entirety. When sent to a language model for summarization, the response is significantly larger than anticipated.

The model reads and processes the entire thread, copying the content without modification. This results in the stored thread being about ten times larger than the original, with most of the extra tokens coming from quoted replies. The quoted replies are the previous message pasted under the current answer, starting with indents and line breaks. Each level of quoting adds additional tokens, costing more to the model.

The reporter created a synthetic thread with ten messages to analyze the issue. The messages ranged from 26 to 65 tokens in length, with the last message alone containing over 1,000 tokens. Even the larger messages were multiple times larger than the original content. The quoted replies accounted for a significant portion of the token count, with the markers and signatures adding extra tokens to every line.

The reporter found that the primary cause of the inflated token count is the quoted replies. Each level of quoting adds an additional set of characters to the start of every line, which the tokenizer counts as separate tokens. Even if the quoting was minimal, the signatures and signature-like lines also contributed to the overall token count.

To optimize costs, the reporter suggested stripping out the quoted blocks before sending the thread to the model for summarization. This reduces the number of tokens the model processes, lowering the cost per thread. However, the reporter acknowledged that handling HTML mail formats and localized "wrote:" lines would require additional parsing and patterns for each specific use case. The savings from this approach varied depending on the specific implementation and the proportion of input versus output tokens.

The reporter concluded that the issue is more complex than simply copying and pasting content. The quoted replies, indents, and signature-like lines all contribute to the increased token count, making it essential to understand and account for these factors when working with email thread data and language models.

Rewritten by urgent.news; may contain errors — check the original.

Source: dev.to

Read the original at dev.to →

Report a problem with this story

More in Tech

Why your "free" dynamic QR code stopped working, and how to design one that won't

A QR code is a picture of some text. When that text is a URL, phones open it. That is the whole trick, and it explains most of the problems people run into with QR codes printed on menus, posters and…

  • "Free" dynamic QR codes stop working due to hidden billing rules.
  • QR codes have two types: static (final URL) and dynamic (short link redirect).
  • To avoid issues, separate billing from redirect path in dynamic codes.

GPO WMI Filters: 15 Queries for Windows 11 and Server 2025

GPO WMI filters let a Group Policy Object apply only to computers that match a WMI query, such as Windows 11 workstations, laptops, virtual machines or member servers, even when they sit in the same…

  • WMI filters apply GPOs to specific computers based on WMI queries.
  • Queries check for Windows 11 workstations or servers during policy processing.
  • Guide includes 15 ready-made queries for Windows 11 and Server 2025.

Mastering TypeScript Utility Types: Write Cleaner, Smarter Code

Mastering TypeScript Utility Types: Write Cleaner, Smarter Code TypeScript utility types are built-in tools provided by the TypeScript language that allow developers to transform and manipulate…

  • TypeScript utility types create new data types from existing blueprints
  • Utility types modify properties like making fields optional or removing elements
  • Omit and Partial utility types streamline data model management in APIs

More from Sunday 22 November →