What ChatGPT is actually doing when it answers you
It doesn't "understand" you. It does one thing, over and over: guess the next token, append it, guess again. Once that clicks, the weird behavior stops being mysterious: Tokens, not letters — that's why it can't count the r's in "strawberry". Meaning is geometry — words become vectors; "king" sits near "queen". Attention decides which earlier words matter for the next one. Pretraining makes a…
We haven't written up this one. Dev.to has the full story — the link below goes straight to it.
