Mathematicians’ n-gram next-word models inspire ChatGPT

youtube.com7 min read

The author explores the historical evolution of language models, tracing the lineage from early manual counting methods by Andrey Markov and Claude Shannon to n-gram models and modern AI. The transcript outlines how early machines tackled text classification, translation, and next-word prediction while running into persistent limitations involving context length and task flexibility. Open article →