Skip to main content

How Generative AI Works (Without the Jargon)

Premium

You don’t need a PhD in machine learning to explain how a large language model works.

But you do need to explain it in a way that shows conceptual depth. You need to show that you understand what’s really happening under the hood, without getting lost in buzzwords.

This lesson gives you the mental model that interviewers expect non-ML candidates (like PMs, designers, analysts, or engineers outside ML) to have.

The core idea

Generative AI models like ChatGPT or Claude are prediction engines.

They don’t "think" or "reason" the same way a human brain does. They predict the next token based on patterns learned from their training data.

For example, if you give the model this prompt:

"Roses are red, violets are…"

It predicts the next most likely token might be "blue", then continues generating each following word, one token at a time. That’s the core mechanism: predicting what comes next in a sequence that makes sense.

What it learns from

To make these predictions, the model undergoes multiple training phases:

  • Pretraining: learning patterns in massive datasets (books, websites, code, articles, etc.) to understand general language usage.
  • Instruction tuning / alignment: guided training with human feedback or supervised data to improve reliability, adherence to instructions, and factual grounding.

Even with these steps, the model can sound confident but still be wrong. It imitates how humans write and respond based on patterns, rather than verifying facts independently.

From prompt to output

Here’s the simplified flow of what happens when you type a prompt:

  1. Tokenization. Your text is broken into tokens (pieces of words).
  2. Embedding. Each token is turned into numbers that represent its meaning and relationship to others.
  3. Attention. The model looks at all the tokens so far and decides which parts of the input are most relevant to predicting the next one.
  4. Prediction. It calculates probabilities for the next token and picks one (or samples randomly based on temperature settings).
  5. Repeat. It adds that token to the text and predicts again, over and over.

You can think of this like autocomplete on steroids. The model keeps finishing your sentence until it forms a complete response.

You’ll often hear that LLMs are built on a Transformer architecture.

That term simply means: A model that "transforms" how each token relates to every other token in context.

Before transformers, models could only look at nearby words (like reading five words at a time).

Transformers can see the whole sentence, paragraph, or conversation. That’s what makes modern LLMs like ChatGPT so coherent.

Model lifecycle

1. Pretraining: Learning language patterns

  • The base model is trained on massive text datasets.
  • It learns how language works e.g. grammar, facts, reasoning patterns, relationships between concepts.
  • This is extremely costly and usually done only by AI companies (e.g., OpenAI, Anthropic, Google).
  • Output at this stage is powerful but raw. It has no alignment or domain knowledge.

The datasets used for pretraining are often raw and messy, containing errors, inconsistencies, or duplicated content. While some cleaning and filtering is performed, large-scale pretraining typically tolerates a significant amount of noise. The model learns patterns despite imperfections in the data.

2. Post-training: Shaping behavior

Post-training is the broader phase that comes after the base model is pretrained on general data. It includes any additional training or adaptation. There are several ways this can happen:

  • Fine-tuning: The model is further trained on specific datasets (e.g., legal, medical, enterprise data) to make it perform better in a domain.
  • Instruction tuning: The model learns to follow natural language instructions.
  • RLHF (Reinforcement Learning with Human Feedback): Teaches the model to give answers people prefer.

You may also hear the term "alignment". Alignment is part of post-training and specifically refers to making the model’s behavior safe, helpful, and consistent with human values or user intent

3. Inference: Using the model

  • This is when you send a prompt.
  • The model doesn’t learn anything new here. It uses what it already knows.
  • It can, however, behave very differently depending on prompting and context you provide (e.g., via system messages or RAG).
  • Every token generated is based on probabilities from its training + the immediate context.

Why this matters for interviews

When interviewers ask:

  • "How can we make the model better for a specific use case?"
  • "Why doesn’t the model just learn from the conversation?"
  • "How can we adapt a model without retraining it?"

A good answer reflects this understanding:

  • Pretraining gives the model general capabilities.
  • Fine-tuning and alignment shape its behavior.
  • Contextualization lets us adapt it without retraining.
  • Inference is just applying all of that to a specific prompt.

When asked, "Can you explain how ChatGPT works?"

A strong, non-technical answer might sound like this:

"At its core, ChatGPT is a giant prediction engine. It was trained on huge amounts of text to learn patterns in how language is used.

When you prompt it, it doesn’t think or recall facts, it predicts the next most likely word based on the context.

Because of that, it’s powerful for generating natural language but prone to errors when it tries to fill gaps with plausible-sounding text."

That’s all you need: simple, accurate, and professional.

Next Lesson