{
  "id": 5035400,
  "title": "Linear Mapping have a Memory",
  "url": "https://urgent.news/2026/09/02/linear-mapping-have-a-memory",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-02T07:00:52.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/sean_oconnor_4bd481930b6/linear-mapping-have-a-memory-3ckg"
  },
  "original_language": "en",
  "account": "Linear Mappings Have a Memory\n\nThe author has been examining the behavior of linear mappings when treated as linear associative memories. One striking finding is that when a weighted sum is trained beyond capacity, new examples do not simply overwrite old ones. Instead, the weight vector is continually perturbed, leading to the gradual addition of noise to previously stored associations. While more recent examples are recalled more effectively, older examples can persist statistically for an unexpectedly long period.\n\nInterestingly, below capacity, removing a training example may have no effect on the weight vector altogether. However, once weight decay is introduced, the mapping can shift to a different, lower-norm solution while still preserving the remaining associations. These basic observations have significant implications for initialization, weight decay, the dynamics of stochastic gradient descent (SGD), and ultimately for CCSLM (Continuous Compressive Sensing Learning Model), where the local experts can be considered as factorized associative memories.\n\nThe author has detailed the technical aspects and potential implications of these findings in the article titled \"Linear Mappings have a Memory.\"",
  "summary": "Linear Mappings Have a Memory I've been looking at something surprisingly simple: what happens when a linear mapping is treated explicitly as a linear associative memory. One particularly interesting observation is that when a weighted sum is trained beyond capacity, new examples don't simply erase old ones. Instead, the weight vector is continually perturbed, gradually adding noise to previously…",
  "key_points": [
    "Linear mappings exhibit memory-like behavior when treated as associative memories.",
    "Weighted sums trained beyond capacity add noise to stored associations.",
    "Weight decay allows mappings to shift to lower-norm solutions while preserving associations."
  ],
  "editors_take": "Training linear mappings beyond capacity introduces noise to old associations, while below capacity, removing examples may have no effect, impacting CCSLM and stochastic gradient descent dynamics.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}