Back to the library
artificial intelligence
LLM Fundamentals and Applications
What a language model is actually doing, built up from a miniature one you train yourself: tokens, attention, pre-training, alignment, and where the failure modes come from.
Opens by training a character-level model small enough to watch, so every later idea has something concrete to attach to.
Works up through tokenization and byte-pair encoding, self-attention and why sequence length costs what it does, then pre-training, scaling and the knowledge cutoff.
Ends where it matters in practice: alignment and RLHF, where hallucinations come from, and the prompt strategies that follow from the mechanism rather than from folklore.
Who it's for
- Engineers using LLM APIs who want the mechanism, not the metaphor
- Anyone who can read Python and wants to train a small model to understand a large one
- People who need to reason about why a model failed rather than guess
Chapter outline
Part 1: Training a Miniature Language ModelAfter this part you will be able to train a character-level language model from scratch on a tiny corpus, sample output at varying temperatures, and add your first vendor scorecard evaluation row.3 Lessons · ~2h
Part 2: Tokens, Byte-Pair Encoding, and Token BudgetsAfter this part you will be able to predict token counts for arbitrary text, explain subword tokenization, evaluate token cost and context window risks, and test a prompt's token behavior.3 Lessons · ~1.4h
- Deconstructing Text into Subwords with Byte-Pair Encoding31m
- Calculating Token Budgets and Context Window Limits10m
- Evaluating Token Costs and Context Window Risks43m
Part 3: Computing Self-Attention and Overcoming Sequential LimitsAfter this part you will be able to explain how self-attention computes relationships across token pairs simultaneously, analyze long-range context handling, and evaluate vendor context window limits.4 Lessons · ~2.8h
- Computing Self-Attention Across Token Pairs32m
- Why Sequential Models Choked on Long Text31m
- Analyzing Transformer Parallel Processing and Scaling Limits1h
- Context Windows and Vendor Evaluation Scorecard Checkpoint43m
Part 4: Pre-Training Pipelines, Scaling Laws, and Knowledge CutoffsAfter this part you will be able to describe the web-scale pre-training pipeline, explain how models store statistical patterns rather than facts, and evaluate vendor training recency and domain coverage.3 Lessons · ~1.8h
- Tracing the Web-Scale Pre-Training Data Pipeline32m
- How Next-Token Prediction at Scale Builds Statistical World Knowledge34m
- Scaling Laws, Knowledge Cutoffs, and Vendor Evaluation Scorecard Row43m
Part 5: Alignment, RLHF, and Instruction TuningAfter this part you will be able to distinguish base model behavior from RLHF-aligned outputs, explain how human preference tuning reshapes probability distributions, and evaluate vendor alignment strategies.3 Lessons · ~1.4h
- Distinguishing Base Model Completions from Aligned Outputs30m
- Mechanics of Reward Modeling and Policy Optimization in RLHF16m
- Evaluating Vendor Alignment Strategies and Scorecard Integration36m
Part 6: Capabilities, Hallucinations, and Failure ModesAfter this part you will be able to predict high-hallucination-risk input types, explain why hallucination stems from next-token prediction, and evaluate vendor hallucination surface areas.4 Lessons · ~2.5h
- Mapping Core LLM Capabilities and Enterprise Use Cases24m
- Unpacking Hallucination as a Consequence of Next-Token Prediction31m
- Assessing High-Risk Inputs and Vendor Hallucination Surface Area55m
- Evaluating Mitigation Strategies, RAG, and Vendor Guardrails38m
Part 7: Prompt Engineering, Mitigation Strategies, and Complete Evaluation ScorecardAfter this part you will be able to construct few-shot and chain-of-thought prompts, apply hallucination mitigation strategies like RAG and parameter tuning, and finalize your complete vendor evaluation scorecard.3 Lessons · ~1.6h
- Crafting Few-Shot and Chain-of-Thought Prompts29m
- Applying Hallucination Mitigations and RAG26m
- Finalizing the AI Vendor Evaluation Scorecard39m