Token Service Isolation: Containing Customer Support Chat Reconnect and Backfill Failures

Isolating token issuance from the chat data path is crucial for containing failures in customer support chats, such as reconnect and backfill issues. To achieve this, each token should be scoped to a specific tenant, conversation, actor, and capability. Moreover, reconnect mechanisms should depend on a durable cursor rather than relying on any information stored within the token service.

This architectural approach helps separate two key questions: whether a leasing agent can join a resident conversation, and which messages the user has already received. Authorization addresses the first question, while a monotonically increasing message cursor handles the second. If both questions are answered by a single service, a token refresh could inadvertently become a history-recovery mechanism, leading to duplicate messages, missing context, or cross-workspace disclosures.

The token service should only issue short-lived, audience-bound authority, while the conversation service manages membership, and the durable log keeps track of the backfill position. The real-time edge should validate authority without making repeated calls to the token issuer for every message. The exact duration of token validity depends on the specific deployment scenario, including factors like revocation urgency, reconnect frequency, and mobile suspension patterns.

Conducting a replay test with production-like disconnects can help determine the optimal token lifetime.

When designing the token service isolation system, focus on invariants rather than specific components. For instance, a token accepted for Building A should never authorize a socket, media session, or backfill read for Building B. Similarly, a resident token must not acquire an agent's capability simply because both users belong to the same conversation.

A refreshed token can extend authority but should not alter the client's acknowledged message cursor. A reconnect should be idempotent, ensuring that repeating the same resume request yields the same ordered suffix, excluding any messages added after the request began.

The real-time edge should reject any mismatches before subscribing the connection to a channel, while the backfill reader should repeat the same tenant and conversation checks instead of relying on the edge's previous validation. Defense-in-depth measures are essential since the edge and history paths may fail differently. The token service should have enough information to issue constrained authority, but it should not become the system of record for chat progress.

Instead, avoid storing last_seen_message_id within a refresh-token session and treat it as the resume cursor. This prevents scenarios where two tabs acknowledge different messages, a suspended phone returns after the desktop has advanced, or an agent opens the same case on multiple devices.

The ideal request sequence involves authenticating the user, verifying current conversation membership, issuing a narrowly scoped token, validating it at connection admission, streaming new events, persistently storing acknowledgements independently, and, upon disconnect, reading from the durable log after the last acknowledged cursor. Token renewal and log replay can occur close together in time but should not alter each other's state.

Understanding potential failure modes is vital for implementing effective token service isolation. Scope confusion occurs when a valid signature is accepted without considering tenant, conversation, audience, actor, and capability simultaneously. Refresh drift happens when renewal changes subscription state or advances the cursor, linking authorization to delivery.

Replay gaps occur when the edge retains only an ephemeral buffer, making it impossible to reconstruct the ordered suffix once that buffer is gone. Duplicate application occurs when the client retries backfill and applies the same message twice, as message identity is not used as an idempotency key. Lastly, split admission occurs when a connection is admitted under one claim set but a history request is authorized using a broader session cookie.

To illustrate these concepts, consider a scenario where an agent handles a resident's maintenance request and switches from a laptop to a phone while leaving the office. The laptop has durably acknowledged cursor m_1842, while the phone was suspended at m_1817. Additionally, the agent's access token has expired. In this case, the phone must obtain fresh authority for the same tenant and conversation and then request events after m_1817 from the log.

The response might include messages that the laptop has already displayed, which is acceptable since device-local application is keyed by message ID, and any shared "read" state is a separate domain event.

If the token renewal process copies the account-wide cursor m_1842 into the phone session, messages m_1818 through m_1842 would disappear from that device's reconstructed view. This demonstrates the importance of maintaining clear boundaries between authentication, membership verification, token issuance, and cursor management. Observability should also reflect these boundaries by recording reason codes for admission denial, token renewal, replay start, replay count, duplicate suppression, and cursor advancement, while excluding token bodies and resident message content.