Large Language Models (LLMs) are the engine behind ChatGPT, Claude, Gemini, and most AI tools you've heard of. But what are they, really?
The short version
An LLM is a system trained to predict the next word (technically, "token") in a sequence. That's it. Everything else — the responses, the reasoning, the code it writes — emerges from doing that one thing at massive scale, on most of the text humanity has ever written.
Tokens, not words
LLMs don't read words. They read tokens — chunks of characters that could be a full word, part of a word, or punctuation. "starOS" might be one token. "unbelievable" might be three. This matters because models have a context window — a maximum number of tokens they can hold in memory at once. Think of it like working memory: what's in the window is what the model can see and reason about.
How it learns
Training works like this: show the model billions of examples of text, and for each one, ask it to predict what comes next. When it's wrong, adjust the internal weights. Do this trillions of times. The result is a model with billions of parameters that encode statistical patterns across human language.
It doesn't "understand" text the way you do. It compresses the patterns of how words relate to each other, at a scale that produces outputs that *look* like understanding.
Why it hallucinates
Because it's a prediction machine, not a fact database. When the model doesn't know something, it doesn't say "I don't know" by default — it predicts what a plausible-sounding answer would look like. Sometimes that's accurate. Sometimes it's confident nonsense. This is called hallucination, and it's a structural property of how LLMs work, not a bug that will just get patched away.
What this means for you
- LLMs are great at: summarizing, reformatting, brainstorming, writing drafts, explaining concepts, generating code
- LLMs are unreliable at: exact facts, math, anything requiring real-time data or memory across sessions
- The key skill: knowing when to trust the output and when to verify it
That's the foundation. Everything else in AI builds on top of this.