Choosing an Agent Memory Tool: Test Duplicate Writes Before You Commit

When assessing an AI agent memory tool, it's crucial to conduct a duplicate-write exercise. This involves asking three specific questions: what occurs when a save is retried, when the same fact is saved again intentionally, and when a similar statement serves as a correction. Before drawing a conclusion on the tool's deduplication capabilities, gather evidence for each scenario.

Distinguish between HTTP idempotency and content similarity. HTTP idempotency implies that repeating an identical request yields the same server effect as a single request. However, it doesn't necessitate identical responses or prevent separate request logs. RFC 9110, section 9.2.2, advises against automatically retrying non-idempotent requests without a client mechanism to confirm the appropriateness of the retry.

Organize your trial by separating these three scenarios: a normal save, a same-operation retry, and a new operation with identical content. Use a test setup with fictional information, recording the tool version, storage backend, identity, and integration settings without sharing credentials. Begin with the following statements: "Project Cedar uses the release-note heading Change summary," "Use Change summary as Cedar's release-note heading," and "Cedar now uses Release overview instead of Change summary."

For each scenario, examine the stored records and returned identifiers before asking a fresh session to recall the heading. During a normal save, record the outcome, stored identifier, content, and scope. In the case of a same-operation retry, repeat the documented retry mechanism and note the logical-operation key, returned identifier, and record count.

When saving a paraphrase, confirm whether a new record, no-op, or update is reported. Similarly, for a correction, assess the current status of the old and new records, retrieved context, and the final answer.

When carrying out these exercises, maintain a clear distinction between retry safety and content similarity. If the tool doesn't document an idempotency option, refrain from inventing one. Additionally, verify whether the tool distinguishes between exact content-hash duplicates and similar statements, as some tools treat exact duplicates as no-ops and similar statements as writes accompanied by near-duplicate information for review.