What is an LLM and Why Does ChatGPT Feel Intelligent

ChatGPT, Gemini, Claude — seem to be everywhere today. These AI systems can do things that felt impossible just a few years ago.

How are these systems doing so many things? They can write code, explain Android architecture, summarize PDFs, draft emails, and answer complex questions. It often feels intelligent, almost as if it’s thinking. But what is actually happening behind the scenes? In this article, we’ll explore what an LLM is, how it works, and why it appears so intelligent despite fundamentally being a next-token prediction system.

But before we take a deep dive into how LLMs work, let’s first understand a few terms which are used interchangeably and are often confusing to a beginner. Understanding the relationship between these terms will make it easier to understand what an LLM actually is.

AI vs ML vs Deep Learning vs Generative AI vs LLMs

Artificial Intelligence refers to creating artificial systems which can do tasks that will normally require human intelligence.

How do these systems become intelligent? Is it because they have a massive amount of data and millions of lines of code? Not really. That’s where Machine learning (ML) comes into the picture.

Machine Learning is a subset of AI where systems learn patterns from data instead of relying on programmed rules and then make predictions based on those patterns.

This is how banks predict fraudulent transactions by checking your spending patterns.

Traditional machine learning works for many prediction and classification problems but complex problems like image search on google photos or Netflix’s recommendation system requires deep learning.

Deep Learning is a type of ML which uses multi-layered neural networks to learn complex patterns from large amounts of data.

Most Machine Learning systems either predict or classify. But this is not just what we want, we want our AI systems to even generate new content. And that’s where generative AI come in.

Generative AI is an AI system which can create new content like text, code and images by learning patterns from massive amounts of data.

Example: generate a birthday invitation for my daughter’s 3rd birthday with mermaid theme.

That brings us to our actual topic: LLM

What is LLM?

Large Language Models (LLMs) are a category of Generative AI that specializes in understanding and generating human language. They achieve this by repeatedly predicting the next token in sequence. An LLM is the technology that powers applications like ChatGPT, Gemini, Claude, and many other AI assistants.

So if you just go on any of the AI chat agents like ChatGPT, Gemini, Claude, Llama etc. and just write Johny Johny, it will automatically give you the complete poem.
How did it do that? Did it have the poem written somewhere in its database?

NO

It actually had a lot of data on which it was trained initially. When we wrote Johny Johny, the model uses the patterns and relationships it learned during training to determine the next token. Then it appends that token and repeats the process of figuring out the next one, and this goes for multiple cycles until it generates the whole poem.

This is what happens in simple words but let’s now understand a bit of technicality here. How does it actually find the next word? Is it a guess?

The Surprising Truth: LLMs Predict the Next Token

It might seem like models are simply guessing the next token. But if that were true, the responses would have been completely random. And if the model was using a fixed set of rules, the response would have also been the same. But the reality lies somewhere in between.

When we write a half sentence or a poem, the model predicts the next token. Not a word, a token.

But what is a Token?

Just like for us, a sentence is made up of words, an LLM sees a sentence as a sequence of tokens. A token can be a word, part of a word, a punctuation mark, or even a number. In simple terms, tokens are the basic building blocks that an LLM reads and generates.

When you type:

“The capital of India is”

the model doesn’t directly search for an answer in a database. Instead, it calculates the probability of thousands of possible next tokens.

For example:

Delhi → Very High Probability
Mumbai → Lower Probability
Kolkata → Lower Probability
London → Very Low Probability

Based on these probabilities, the model selects a token, appends it to the sentence, and repeats the process again.

This raises an interesting question.

If LLMs are simply predicting the next token over and over again, why do they feel so intelligent?

Why do LLMs seem intelligent?

The answer lies in how the model was trained.

During training, the model was exposed to massive amounts of text from books, articles, websites, documentation, and other sources. As it processed this data, it didn’t memorize every sentence. Instead, it learned patterns and relationships between concepts.

For example, it learned relationships such as:

France → Paris
Android → Kotlin
Kotlin → JetBrains
Doctor → Hospital

These relationships are not stored as rows in a database. They are encoded in billions of parameters (often called weights) inside the neural network. This is why an LLM can answer questions it has never seen before. Instead of performing a search in a database, it uses the patterns and relationships learned during training to generate a response.

For example if we ask a LLM:

Which company created the language most commonly used for Android development?

Even if the model has never seen this exact question, it can connect the relationships:

Android → Kotlin → JetBrains

and generate the correct answer.

This ability to combine learned relationships is one of the main reasons LLMs appear intelligent.

Why Do LLMs Hallucinate?

So far, we’ve seen that LLMs learn patterns and relationships from massive amounts of data and use them to predict the next token. This allows them to generate surprisingly intelligent responses. But if they’re so intelligent, why do they sometimes produce answers that are completely wrong?

That’s because an LLM’s job is to generate the next plausible token and not verify if it is factually correct or not. Hallucinations happen because the model generates the most likely continuation even when it lacks enough information to determine the correct answer. So in such cases, it might generate an answer which is plausible but not factually correct. And that’s what we call hallucinations.

Let’s understand this with some examples:

Example 1

What is my great-grandmother’s name?

It has no information about this, so it might respond with “I don’t know.” In this case, that would be the correct answer.

Example 2

What is 5*5?

The model is less likely to hallucinate in this case because it has learned strong arithmetic patterns and can reliably apply them to arrive at the correct answer.

Example 3

What was my favourite toy when I was 4 years old?

It might say Barbie doll because it knows I am a girl and 4 year old girls like dolls. It is plausible but it might be incorrect. The probability of it saying pharmaceuticals or android development here is very low because such tokens have very low probability of having any pattern or relation with a 4 year old girl. This is an example of a hallucination.

Hallucinations are not bugs. They are a natural consequence of how LLMs work. Since an LLM is designed to generate the most probable continuation rather than verify facts, it may sometimes produce answers that sound convincing but are not actually correct.

Conclusion

Let’s quickly recap everything now:

What is LLM? An LLM is a Generative AI model trained on massive amounts of text that generates human-like language by repeatedly predicting the next token in a sequence.

Why do LLMs feel intelligent? LLMs feel intelligent because they learn patterns and relationships from massive amounts of training data. Instead of looking up answers in a database, they use these learned patterns to generate responses to questions they may never have seen before.

Why do LLMs hallucinate? LLMs hallucinate because their objective is to generate the most likely continuation, not to verify facts. When they lack enough information to determine the correct answer, they may generate a response that sounds plausible but is factually incorrect.

Understanding these concepts helps us see LLMs for what they really are: powerful pattern-learning systems capable of remarkable results, but also limited by the information and relationships they have learned during training.