{
  "id": 176082,
  "title": "LLMs Have Learned All Open Code on GitHub - But Missed the Most Important Part.",
  "url": "https://urgent.news/2026/08/05/llms-have-learned-all-open-code-on-github-but-missed-the-most",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-05T13:33:14.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/kostya_m_17bb892f188e6872/llms-have-learned-all-open-code-on-github-but-missed-the-most-important-part-1odg"
  },
  "original_language": "en",
  "account": "LLMs have learned an immense amount of open code from GitHub, but they are missing a crucial aspect of programming: the thought process behind the code. The problem is that LLMs are trained on the final, polished versions of code, rather than the entire development history. This lack of context leaves them with a superficial understanding of code and a limited vision of the project as a whole. They can mimic good style, but they often only focus on local tasks and need human supervision to make architectural decisions.\n\nThe solution is to provide LLMs with the full history of a project's creation, including the thousands of commits, commit messages, and the evolution of code over time. By training models on this comprehensive history, they can learn more than just syntax and patterns; they can understand the reasons behind specific decisions and see how code evolved to solve problems. This would enable them to make informed architectural decisions tailored to the project's needs rather than relying solely on generic solutions.\n\nOne potential drawback is the enormous amount of data involved. However, the authors argue that we don't need to use all of it. Instead, we can focus on high-quality codebases like Linux, PostgreSQL, Rust, the Go compiler, and Redis, which serve as textbooks written by top engineers. By harnessing the full history of these projects, LLMs can become true experts in their domains, moving beyond mere imitation and becoming genuine engineers.\n\nThe authors emphasize that programming is unique in its complete and recorded history of a complex system's evolution. Every commit is a footprint, and every message a developer's thought. By training LLMs on this data, we can unlock a new level of expertise and understanding in the field of AI coding. This approach has the potential to transform AI coders from imitators to genuine, knowledgeable professionals, equipping them with the ability to tackle complex problems and make informed architectural choices.",
  "summary": "Or why AI coders are just imitators, not real engineers. We are used to thinking that modern large language models (LLMs) know everything about programming. They are trained on billions of lines of code from open GitHub repositories. They can write even a big app from scratch. But there is one problem. We gave them millions of ready-made answers. But we never gave them the main thing - the…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}