Generative AI Core Concepts – Complete Guide for the AIF-C01 Exam
Why Are Generative AI Core Concepts Important?
Generative AI core concepts form the foundational layer of the AWS Certified AI Practitioner (AIF-C01) exam. Without a solid grasp of these fundamentals, it becomes extremely difficult to answer questions about model selection, prompt engineering, responsible AI, and AWS AI services. AWS expects candidates to understand what generative AI is, how it differs from traditional AI/ML, and why it matters in real-world applications. Approximately 20–30% of exam questions touch on these foundational ideas either directly or indirectly, making this one of the most high-yield topics to study.
What Is Generative AI?
Generative AI refers to a category of artificial intelligence systems that can create new content — including text, images, code, audio, and video — based on patterns learned from vast amounts of training data. Unlike discriminative models that classify or predict labels for input data, generative models learn the underlying distribution of the data and produce novel outputs that resemble the training data.
Key Terminology You Must Know:
• Foundation Models (FMs): Large-scale, pre-trained models (such as GPT, Claude, Titan, LLaMA, Stable Diffusion) that are trained on broad datasets and can be adapted to a wide range of downstream tasks. They are the backbone of modern generative AI.
• Large Language Models (LLMs): A subset of foundation models specifically designed to understand and generate human language. Examples include Amazon Titan Text, Anthropic Claude, and Meta LLaMA.
• Tokens: The fundamental units that LLMs process. Text is broken into tokens (words, subwords, or characters). Understanding tokenization is important because model pricing, context window limits, and performance are all measured in tokens.
• Context Window: The maximum number of tokens a model can process in a single input-output cycle. A larger context window allows the model to consider more information at once.
• Parameters: The learned weights within a neural network. More parameters generally means a more capable model, but also higher compute costs and latency.
• Training Data: The massive corpus of text, images, or other data used to train the model. The quality, diversity, and size of this data directly impact model performance.
• Inference: The process of using a trained model to generate outputs from new inputs. This is what happens when you send a prompt to an LLM and receive a response.
• Prompt: The input text or instruction provided to a generative AI model to elicit a desired output.
• Hallucination: When a generative AI model produces output that sounds plausible but is factually incorrect or fabricated. This is one of the most critical challenges in generative AI.
• Temperature: A parameter that controls the randomness of model outputs. A lower temperature (e.g., 0.1) produces more deterministic, focused responses; a higher temperature (e.g., 0.9) produces more creative, varied responses.
• Top-p (Nucleus Sampling): Another parameter controlling output diversity by limiting token selection to a cumulative probability threshold.
• Embedding: A numerical (vector) representation of data (text, images, etc.) that captures semantic meaning. Embeddings are used for search, similarity comparisons, and retrieval-augmented generation.
How Does Generative AI Work?
Generative AI works through a multi-stage process:
1. Pre-Training:
Foundation models are trained on enormous datasets using self-supervised learning. For LLMs, this typically involves predicting the next token in a sequence (autoregressive training). The model learns grammar, facts, reasoning patterns, and even some world knowledge from this process. Pre-training requires massive compute resources (thousands of GPUs over weeks or months).
2. Fine-Tuning (Optional):
After pre-training, models can be further trained on domain-specific or task-specific datasets to improve performance for particular use cases. Fine-tuning adjusts the model's weights to specialize its behavior.
3. Reinforcement Learning from Human Feedback (RLHF):
Many modern LLMs go through an additional alignment step where human evaluators rank model outputs and a reward model is trained to guide the LLM toward producing more helpful, harmless, and honest responses.
4. Inference:
When a user provides a prompt, the model processes the input tokens, uses its learned patterns to predict the most likely sequence of output tokens, and generates a response. Parameters like temperature and top-p influence how the output is sampled from the probability distribution.
Key Architectures to Know:
• Transformer Architecture: The foundational architecture behind virtually all modern generative AI models. Introduced in the 2017 paper Attention Is All You Need, it uses self-attention mechanisms to process input sequences in parallel, making it far more efficient than previous RNN/LSTM approaches.
• Encoder-Decoder Models: Process input through an encoder and generate output through a decoder (e.g., T5, BART). Good for translation and summarization.
• Decoder-Only Models: Generate text autoregressively, one token at a time (e.g., GPT family, Claude, Amazon Titan Text). Most common for general-purpose text generation.
• Diffusion Models: Used primarily for image generation (e.g., Stable Diffusion, Amazon Titan Image Generator). They work by learning to reverse a noise-adding process, gradually transforming random noise into coherent images.
• GANs (Generative Adversarial Networks): An older architecture using a generator and discriminator in competition. Less common now for text but still relevant for some image tasks.
• VAEs (Variational Autoencoders): Encode data into a latent space and then decode it to generate new samples. Useful for understanding the concept of latent representations.
Generative AI vs. Traditional AI/ML:
• Traditional ML: Typically task-specific. A model is trained for one purpose (e.g., fraud detection, demand forecasting). Requires labeled data and feature engineering.
• Generative AI: General-purpose by design. A single foundation model can perform text generation, summarization, translation, code generation, question answering, and more — often with zero or few examples (zero-shot or few-shot learning).
• Key Difference: Traditional ML models classify or predict. Generative AI models create new content.
Important Concepts for the AIF-C01 Exam:
• Zero-Shot Learning: The model performs a task without any task-specific examples in the prompt. It relies entirely on its pre-trained knowledge.
• Few-Shot Learning: The model is given a small number of examples in the prompt to guide its behavior for a specific task.
• Prompt Engineering: The practice of designing and optimizing prompts to get the best possible output from a generative AI model. This is a key skill and a separate exam domain.
• Retrieval-Augmented Generation (RAG): A technique that enhances LLM responses by first retrieving relevant information from an external knowledge base and then including that information in the prompt context. This helps reduce hallucinations and keeps responses grounded in factual data.
• Model Customization Spectrum: From least to most effort — prompt engineering → few-shot learning → fine-tuning → continued pre-training → training from scratch. The exam expects you to know when each approach is appropriate.
• Multimodal Models: Models that can process and generate multiple types of data (text, images, audio, video). Examples include Amazon Titan Multimodal Embeddings and models available through Amazon Bedrock.
AWS Services Related to Generative AI Core Concepts:
• Amazon Bedrock: A fully managed service that provides access to multiple foundation models from various providers (Anthropic Claude, Meta LLaMA, Stability AI, Cohere, Amazon Titan) through a unified API. This is the primary AWS service for generative AI.
• Amazon Titan: AWS's own family of foundation models, including Titan Text, Titan Image Generator, and Titan Embeddings.
• Amazon SageMaker JumpStart: Provides access to pre-trained foundation models that can be deployed and fine-tuned within SageMaker.
• Amazon Q: An AI-powered assistant for business and developer use cases built on generative AI technology.
• Amazon CodeWhisperer (now Amazon Q Developer): An AI-powered code generation assistant.
Common Challenges and Limitations of Generative AI:
• Hallucinations: Models can generate confident but incorrect information.
• Bias: Models can reflect and amplify biases present in training data.
• Data Privacy: Sensitive data used in prompts could potentially be exposed or retained.
• Cost: Large models are expensive to train and run inference on.
• Latency: Larger models may have higher response times.
• Lack of Real-Time Knowledge: Models have a training data cutoff and do not inherently know about recent events (unless augmented with RAG or similar techniques).
• Non-Determinism: The same prompt can produce different outputs each time (controlled by temperature and sampling settings).
Exam Tips: Answering Questions on Generative AI Core Concepts
• Tip 1 — Know the Vocabulary: Many exam questions test whether you understand key terms like foundation model, LLM, token, inference, hallucination, embedding, and context window. Memorize precise definitions.
• Tip 2 — Distinguish Generative from Discriminative: If a question describes a task that involves creating new content (text, images, code), it is a generative AI task. If it involves classifying or predicting, it is likely traditional ML. The exam tests this distinction frequently.
• Tip 3 — Understand the Customization Spectrum: Questions may present a scenario and ask for the most appropriate approach. Remember: start with prompt engineering (cheapest, fastest), escalate to fine-tuning only when necessary, and train from scratch only when no existing FM meets your needs.
• Tip 4 — Hallucination Mitigation: When a question asks how to reduce hallucinations, look for answers involving RAG, grounding with external data sources, lowering temperature, or adding validation steps. RAG is almost always the best answer for factual accuracy.
• Tip 5 — Temperature and Sampling: Lower temperature = more predictable, factual responses. Higher temperature = more creative, diverse responses. The exam loves testing this.
• Tip 6 — Foundation Models Are General-Purpose: If a question asks about using a single model for multiple tasks (summarization, Q&A, translation), the answer likely involves a foundation model with different prompts, not multiple specialized models.
• Tip 7 — Know When to Use Amazon Bedrock vs. SageMaker: Bedrock is the managed, serverless option for accessing FMs without managing infrastructure. SageMaker is for when you need more control over training, fine-tuning, or deployment. For most generative AI questions on the exam, Bedrock is the preferred answer.
• Tip 8 — Watch for Distractor Answers: Exam answers may include plausible-sounding but incorrect options. For example, a question about reducing hallucinations might include "increase temperature" as a distractor — this would actually make hallucinations worse.
• Tip 9 — Understand Tokens and Cost: Questions about cost optimization often relate to token usage. Shorter prompts, smaller context windows, and choosing appropriately sized models all reduce cost.
• Tip 10 — Process of Elimination: If you are unsure, eliminate answers that contradict core principles. For example, any answer suggesting that generative AI models are always 100% accurate should be immediately eliminated — hallucination is a well-known limitation.
By mastering these core concepts, you will have the foundational knowledge needed to confidently tackle a significant portion of the AIF-C01 exam. These concepts also serve as building blocks for more advanced topics like prompt engineering, responsible AI, and AWS service-specific configurations.
