Urgent.News

What's breaking now, across thousands of outlets.

More in Tech

Was TinyStories the Domain or the Vocabulary?

I have been running a long series of experiments on whether small models can be made to behave usefully, and one obvious thing to borrow is whatever TinyStories did.

  • Varying vocabulary while keeping domain constant
  • Smaller vocabulary led to 4-point score decrease
  • Higher vocabulary caps improved text coherence

More from Thursday 10 September →