Signal City Archive Desk · "You shall know a word by the company it keeps" (J.R. Firth, 1957) · back to the desk
This machine knows nothing except which word tends to come after which, in the text below. Press Babble and it writes by picking each next word from the company the last word kept. Change the text and it learns something else. Type a word to see what it thinks comes next.
This is the smallest version of the idea that won the argument in natural language processing. Real language models do something far bigger: they look at thousands of earlier words at once, not just one, and learn from more text than a person could read in many lifetimes. That's why they sound fluent where this one sounds like a dream. But the root is the same: predicting the next word from the company words keep. Hear the episode: History No. 13, The Company Words Keep. Everything runs in your browser.