What is Generative AI?

Generative AI is transforming how we create and interact with content. To speak confidently about it in interviews, you need to understand both how traditional AI worked and how foundation models reshaped the field.

Traditional AI models

Before foundation models, most AI was narrow and task-specific:

• Models were trained for a single purpose (e.g., spam detection, translation, image recognition).
• Each model required large labeled datasets for its specific task.
• For every new task, you typically had to train a separate model from scratch: collecting data, designing the model, and running full training cycles.
• This made adaptation slow, costly, and resource-intensive.

Foundation models

Foundation models changed this paradigm. Coined by Stanford’s Institute for Human-Centered AI, the term refers to large, pre-trained models with three defining features:

• Large-scale and pre-trained on massive datasets
• Adaptable to many tasks with minimal additional training
• Prompt-driven and multimodal: guided by natural language as well as other inputs like images, audio, or video

Instead of building a new model for each task, you can now use a single versatile engine that adapts to different tasks often with nothing more than a prompt.

Generative AI

Generative AI is the broader category of AI focused on creating new content (text, images, music, video, code). It’s the umbrella concept: AI systems that generate rather than just classify or analyze.

Foundation models are a specific type of generative AI. They serve as the powerful backbones behind many generative AI applications, such as:

• Large language models (LLMs) like GPT and Claude for text
• Image generators like DALL-E and MidJourney
• Code assistants like GitHub Copilot

All foundation models are generative AI, but not all generative AI systems are foundation models. Some are smaller, task-specific generative models trained for one kind of content.

Types of generative AI

Limitations

Generative AI is powerful, but it isn’t limitless. To use it responsibly and to speak confidently in interviews, you need to understand its key constraints:

• Knowledge boundaries: Models trained on public data can’t access proprietary or private information.
• Recency limits: Unless connected to the internet or given updated context, they don’t automatically know about recent events.
• Domain expertise gaps: Even when a model has read data in areas like medicine, law, or finance, it doesn’t reason like a trained professional. It lacks the judgment, ethical guardrails, and accountability that real experts provide.
• Hallucinations and interpretability: Models can generate false or misleading outputs, and it’s often difficult to trace why or how those errors occur.
• Bias and safety: Models can reproduce or amplify biases present in their training data, raising fairness and safety concerns in real-world applications.
• Scalability and efficiency: Training and deploying large models require significant compute resources, leading to high costs and slower inference times.
• Execution errors: Even with correct prompts, outputs may be incomplete, inaccurate, or misaligned with the task.

Understanding these limitations shows maturity in how you talk about generative AI. Interviewers expect you to know both its strengths and its boundaries.

Generative AI in companies

Most companies do not train foundation models from scratch. Building one requires enormous datasets, massive compute resources, and deep expertise—resources only a handful of organizations (such as OpenAI, Anthropic, Google DeepMind, and Meta) can afford.

Instead, the vast majority of companies adapt existing models to their particular business need by:

• Using APIs from providers like OpenAI, Anthropic, or Google.
• Leveraging open-weight models (e.g., Meta’s LLaMA, Mistral) that can be run and customized in-house.

Once companies have access to a model, they can adapt it using techniques such as:

• Prompt engineering: Crafting effective prompts to guide outputs.
• Prompt tuning: Learning small, task-specific parameters (soft prompts) to steer a model without changing its core weights i.e. shapes how a model interprets prompts.
• Fine-tuning: Training the model further on domain-specific data to improve performance.
• RAG (Retrieval-Augmented Generation): Integrating external data (often using embeddings for semantic search) so the model can reference up-to-date or proprietary information when generating outputs.

While most companies adapt existing models, a few have the resources to train foundation models from scratch. These include:

• OpenAI: creators of ChatGPT, focusing on large-scale language models for text, image (DALL-E), and video (Sora) generation.
• Anthropic: building AI models with safety and alignment in mind, such as Claude.
• Google DeepMind / Google Brain: developing large-scale models like PaLM and Gemini for multi-modal applications.
• Meta: working on LLaMA and other foundation models for research and commercial use.