What if the problem with AI writing novels isn't context length, but how we're using memory?

The common argument surrounding AI writing novels is that it cannot create long-form fiction because the context becomes too large, causing details to be lost and the model to contradict itself. However, the problem may not be an LLM issue but rather an architecture problem. To understand this, consider how software engineering handles large codebases.

Developers don't need to keep the entire 200,000-line codebase in their active working context to make changes. They use files, folders, documentation, dependencies, state, tests, Git history, configuration, and architecture decisions. They pull the necessary information, make changes, test them, and move on. The entire codebase exists, but they don't need it all in their active working context.

Why does AI-assisted writing often work differently? Perhaps we need to stop treating a novel as one giant conversation. Instead, think of it as a structured project with chapters representing the actual story. The key distinction is that the model doesn't need to read everything every time. It requires the right information for the current task.

For instance, when writing Chapter 48, Claude may not need to have Chapters 1–47 in its context. Instead, it might need Chapter 47, Sarah's current character state, John's current character state, the active betrayal plot, the current timeline, and information about the letter introduced in Chapter 12. Relevant worldbuilding and established writing style would also be necessary.

Each piece of information is stored separately and retrieved when needed, a fundamentally different approach to the problem.

Now, consider how ideas could be treated differently from established facts. Imagine halfway through a novel, you think, "Wait... what if John's brother is actually working for the antagonist?" This idea isn't canon yet. So, the system should treat it differently from an established fact. Maybe it starts as a raw idea, and later, you develop it further.

At that point, the system should update the relevant character relationships, plot threads, and timeline. This is akin to version control for story ideas. Current AI writing workflows might be missing this crucial aspect.

Not every piece of information should have the same status. A writing system should distinguish between ideas, developing ideas, proposed canon, revised canon, and deprecated information. This distinction helps the AI distinguish between possibilities and facts, making it more aware of what's merely brainstorming and what's established canon. For example, "Maybe Sarah has a sister" versus "Sarah has a sister named Emily."

Character memory could also work differently. Instead of just stating that "Sarah is a detective," a persistent profile could include details like Sarah's wants, fears, beliefs about John, secrets she's hiding, what she knows that the reader doesn't, her changes since Chapter 1, and how she would react in different situations. This character memory shouldn't be static either.

Sarah in Chapter 5 might be different from Sarah in Chapter 50. The system shouldn't try to permanently remember Sarah; instead, it should maintain her current state and provide the relevant version when needed.

Continuity problems are another issue that AI-assisted fiction could address. Before committing a chapter, the system could run a test suite to check for continuity errors. For example, if Sarah says she's never been to Paris in Chapter 5 but lived in Paris for three years in Chapter 14, there's a conflict. Similarly, if John knows about a murder in the current plot state but didn't know about it in earlier chapters, there's a character-state conflict.

These checks can be likened to a novel compiler, ensuring the model doesn't commit continuity errors.

The context, memory, and source of truth are different concepts that are often mixed together. Context is the information the model currently sees, while memory contains information about the project. The problem might not be whether the AI has access to the information but rather its ability to identify which information is relevant at any given moment. Context, memory, and source of truth are distinct concepts that need to be addressed separately.