How AI Really Writes: A Look Inside the LLM's 'Brain'

Have you ever stopped to think about what’s really happening when you type a prompt into an AI chatbot? It feels like magic, right? You ask a question, and a perfectly formed answer just appears.

But here’s the fascinating part: the AI isn’t thinking up the whole response at once. It’s actually building it piece by piece, word by word (or, more accurately, "token" by "token"). At every single step, it’s playing a probability game, looking at the sentence so far and calculating which word is most likely to come next.

Knowing the probabilities is one thing, but actually choosing the next word is another. This is where the strategy comes in. The method an LLM uses to pick that next word can completely change the tone and quality of the final text. Some strategies make the AI more focused and precise, while others crank up the creativity dial.

So, let's pull back the curtain and look at four of the most popular strategies that power the AI you use every day. Think of these as the different "personalities" we can give an AI.

Greedy Search: The Straight-and-Narrow Path

Let’s start with the simplest one: Greedy Search.

Imagine you're driving and your GPS has one simple rule: at every intersection, take the turn that gets you closest to your destination right now, without thinking about traffic or road closures further ahead. That’s Greedy Search in a nutshell.

At each step in building a sentence, the AI simply picks the single word with the absolute highest probability. It's fast, it's straightforward, and it's easy to implement.

But you can probably see the problem. Just like that GPS might lead you straight into a dead-end or a massive traffic jam, Greedy Search can be incredibly short-sighted. By always making the "best" local choice, it often misses out on a much better overall sentence. This can lead to text that feels repetitive, generic, and just… dull. It’s not great for tasks where you need a bit of flair or creativity.

Beam Search: Keeping a Few Options Open

Okay, so if Greedy Search is a bit too simple, what’s the next step up? That would be Beam Search.

Instead of just following one path like our single-minded GPS, Beam Search is like having a few cars all trying to find the best route. It keeps a handful of the most promising sentence fragments—called "beams"—alive at each step.

Let's say we set our "beam width" to three. The AI will generate the top three most likely next words. Then, for each of those three paths, it will find the top three next words again, and so on. It constantly evaluates which of these parallel paths is producing the most probable sentence overall.

This allows the AI to explore a bit more. It might choose a slightly less probable word at the beginning if it opens up a path to a much better sentence down the road.

Here’s a simple example:

Imagine the AI starts with "The..."

• Greedy Search sees that "slow" (60% probability) is more likely than "fast" (40%). It commits to "The slow..." and ends up with "The slow dog barks," a sentence with a decent, but not great, overall probability.
• Beam Search (with 2 beams) keeps both options alive. It explores the "The slow..." path and the "The fast..." path. After a few more words, it might realize that the path starting with "fast" actually leads to "The fast cat purrs," a sentence that, when all the probabilities are multiplied, is a stronger overall choice.

Beam Search is fantastic for more structured tasks like machine translation, where finding the most accurate and probable translation is key. But for open-ended creative writing, it can still fall into a trap of being a bit too predictable and repetitive, often overusing common words because they always have a high probability.

Nucleus Sampling: The "Good Enough" Club

This is where things get really interesting. Both Greedy and Beam search are all about finding the "best" or "most likely" path. But what if that’s not what makes writing sound human? Humans are a bit random, a little unpredictable.

Enter Nucleus Sampling (also known as Top-p Sampling).

Instead of picking the top word or top few words, Nucleus Sampling creates a small, exclusive club of "good enough" words to choose from. It works like this: you set a probability threshold, say 70% (or p=0.7). The AI then looks at its list of possible next words and starts adding them to a pool, starting with the most probable, until their combined probability hits that 70% threshold.

This pool of words is the "nucleus." The AI then randomly picks a word from only that group.

Think of it this way:

• If the AI is really sure about the next word (e.g., "apple" has a 95% chance of being next), the nucleus will be tiny—maybe just that one word. The output is focused.
• But if the AI is uncertain and there are ten different words with similar, low probabilities, the nucleus will be much bigger. This gives the AI more creative freedom to pick a less obvious word, making the text feel more varied and natural.

This dynamic approach is a huge reason why modern AI text feels so much less robotic. It balances coherence with a healthy dose of diversity.

Temperature Sampling: The AI's Creativity Dial

Finally, we have my personal favorite: Temperature Sampling. If you’ve ever played with AI art or text generation tools, you’ve probably seen a "temperature" or "creativity" slider. This is what it controls.

Think of temperature as a dial that controls the AI’s willingness to take risks.

• Low Temperature (e.g., 0.2): This is like turning the dial down. The AI becomes very conservative and safe. It will almost always pick the most likely, predictable words. The output will be focused and coherent, but potentially boring. This is great for factual summaries or technical explanations.

• Normal Temperature (1.0): This is the default setting. The AI just uses the raw probabilities it calculated, with no extra influence.

• High Temperature (e.g., 1.5): Now we’re turning the dial up! This makes the AI more adventurous and chaotic. It "flattens" the probabilities, meaning it gives more weight to less likely words. It might pick a weird, unexpected word just to see what happens. This can lead to incredible creativity and surprising results, but if you turn it up too high, the text can become nonsensical and lose all coherence.

The right temperature totally depends on your goal. Writing a poem? Crank it up. Writing a legal document? You’ll want to keep it very, very low.

So, the next time you get a response from an AI, remember what’s happening behind the scenes. It's not just one single process; it's a delicate dance between calculating probabilities and then using a specific strategy—a personality, really—to choose its next step. Understanding these dials and knobs is the key to getting exactly what you want out of these amazing tools.