EMBEDDED AI FIELD GUIDE
From Neural Networks to a Shippable Edge AI System
Built for engineers who already know ESP32, USB, and networking stacks. Each day follows one layer of the path—model → representation → runtime → kernel → memory and bandwidth → hardware → product—then connects its history, an illustrated analogy, and an interactive data-flow walkthrough.
SAVED ON THIS DEVICE ONLY
Your learning progress
Progress, mistakes, and lab notes stay in this browser and are never uploaded.
0 / 15chapters completed
0day learning streak
0questions to review
Stage badges
Neural foundationsDays 1–3
Edge deploymentDays 4–6
LLM foundationsDays 7–10
LLM engineeringDays 11–15
Three-day streak3 days
Learning archive and privacy
Export an archive to carry progress, mistakes, and lab notes between browsers. Restoring replaces the learning state in this browser.
Neural Network Foundations
Build neural-network intuition from tensors, layers, and loss functions
Tensor A multidimensional numeric block with a shape and data type—the common representation for model inputs, weights, and intermediate results. Shape The length and meaning of each dimension, such as batch, channel, height, and width in NCHW. Activation function A nonlinearity applied after a linear transform that lets multilayer networks express complex boundaries. MAC One multiply–accumulate operation, commonly used for a rough estimate of compute. Understand how a neural network turns input tensors into optimizable predictions, and learn to follow one forward and backward pass. DAY 02Training and Inference
Freeze learnable state into a portable, verifiable model artifact
eval mode A mode that makes Dropout and BatchNorm use inference semantics. Optimizer An algorithm—and its state—that updates parameters from gradients. Computation graph A set of nodes and edges describing how tensors pass through operators to produce outputs. ONNX opset The version of operator semantics declared by a model. Understand the difference between training and inference state, then turn a checkpoint into a deployment artifact with graph, weights, versions, and preprocessing contracts. DAY 03Model Quantization
Trade fewer bits for lower storage and bandwidth
Scale The proportional factor between an integer code and its real value. Zero-point The integer offset that represents real zero in the mapping. Calibration set Representative input samples used to estimate activation ranges. Per-channel Using separate quantization parameters for different weight channels. Master symmetric and asymmetric quantization, scale and zero-point, and learn to identify sources of error. DAY 04Operators and Kernels
Move from mathematical definitions to layouts, fusion, and hardware execution
Kernel Low-level code that implements an operator for specific hardware. Layout The arrangement of tensor dimensions in memory, such as NCHW or NHWC. Fusion Combining several consecutive operators into one execution. Tiling Partitioning a large computation to fit registers or cache. Understand the relationship among operator semantics, memory layouts, kernel selection, and operator fusion. DAY 05Edge Model Deployment
Put non-LLM vision, audio, and sensor models onto a real device
Representative data A dataset covering real device operating conditions. Tensor arena A memory region preallocated for MCU inference inputs, outputs, and intermediate tensors. Preprocessing The steps that transform raw sensor data into model input. Confusion matrix A table counting predicted classes against true classes. Complete the loop from data, training, export, and quantization to deployment and validation, with an emphasis on small models commonly used on MCUs. DAY 06Edge Vision Pipelines
Turn camera frames into stable events under memory, bandwidth, and real-time constraints
Peak live memory The total size of all buffers that must be retained at the same instant. PSRAM High-capacity RAM connected through an external interface. Stride The memory distance between adjacent pixel data on the same row. Letterbox A resize method that preserves aspect ratio and pads the borders. Budget peak memory and bandwidth for MCU/NPU vision, define frame-buffer, geometry, postprocessing, and ownership contracts, and accept the system with steady-state evidence. DAY 07LLM Foundations
Map the origins, historical path, mechanics, and essential terminology of language models
Language model A model that estimates sequence probability or the conditional probability of the next token. Parameter A trained weight reused unchanged during inference—the model's long-lived statistical state. Pretraining / alignment Pretraining learns broad patterns; alignment uses instructions, preferences, or rules to shape usable behavior. Hallucination Fluent model output that lacks factual support or conflicts with reality. Understand the path from statistical and neural language models to Transformers, pretraining, and instruction alignment, and accurately explain tokens, parameters, context, pretraining, inference, and hallucination. DAY 08Tokenizer
Understand how text becomes an integer stream a model can consume
Vocabulary A mapping from tokens to integer IDs. BPE An algorithm that constructs tokens by merging frequent substrings. Special token A token carrying control meaning such as beginning, end, or role. Chat template A template that formats a message list into the string expected during model training. Explain vocabularies, merges, special tokens, padding, and token budgets, and locate the cost of tokenization on edge systems. DAY 09Transformer Architecture
Understand the whole model before opening its Attention dataflow
Transformer block A repeated sequence unit composed of sublayers such as Attention, an FFN, residual paths, and normalization. Decoder-only A stack of decoder-style blocks with causal visibility, trained to generate the next token from one unified sequence. FFN / MLP A nonlinear feed-forward sublayer applied independently at each sequence position with shared parameters. Residual + Norm Short paths that preserve representations together with controls on numerical scale through deep stacks. Understand how a Transformer organizes embeddings, positions, Attention, FFNs, residual connections, and normalization into encoder, decoder, and decoder-only architectures, then reconstruct the complete token-to-logits path from a model configuration. DAY 10Attention Mechanisms
Turn sequence dependencies into an explicit, parallelizable dataflow
Attention Information exchange that matches Query against Key and uses the scores to aggregate Value. Multi-head Attention projected into several subspaces, run in parallel, and concatenated at the output. Causal mask An upper-triangular visibility constraint that prevents reading future tokens. Positional encoding Information about token order or relative distance injected into a non-recurrent attention model. Start from Attention's alignment problem, understand scaled dot products, Query, Key, Value, multiple heads, causal masks, and positional interaction, then explain its parallel advantage, quadratic cost, and I/O boundary. DAY 11KV Cache
Trade session memory for reuse of historical projections during Decode
Prefill The stage that processes an existing prompt in parallel and establishes initial per-layer K/V state. Decode The stage that appends one position and generates the next token after the previous output is known. KV cache Session state containing Key and Value projections produced by historical tokens at every layer. GQA / MQA Structures in which several Query heads share fewer K/V heads, reducing cache width per token. Understand the workload split between Prefill and Decode, why K/V can be cached, cache layout and capacity slopes, and the distinct effects of MHA, GQA, MQA, paging, windows, and low precision. DAY 12LLM Quantization and GGUF
Package low-bit weights and metadata into a deployable model artifact
GGUF A model-container format used by GGML-family executors. Block quantization Low-bit encoding in which values in a block share parameters such as a scale. Weight-only A strategy that quantizes weights while keeping activations at higher precision. mmap Mapping a file into virtual memory for on-demand access. Understand block quantization, mixed precision, weight metadata, and their relationship to the GGUF container. DAY 13LLM Runtime
Organize model packages, generation loops, and heterogeneous backends into an observable execution path
Context The tokens and KV state held by one inference session. Cancellation A mechanism that aborts an obsolete request and releases its resources. Backend The implementation layer that maps runtime requests onto specific hardware. Offload Assigning some layers or subgraphs to an accelerator. Understand how a runtime loads models, manages sessions, schedules Prefill and Decode, samples, streams, and cancels, then compare CPU, GPU, and NPU backends, compilation modes, subgraph partitions, and fallback with a layered acceptance matrix. DAY 14LLM Performance and Infrastructure
Trace single-machine TTFT and ITL into cluster data movement, topology, and SLOs
TTFT The time from request submission to the first returned token. p95 The tail-latency percentile that 95% of requests do not exceed. Goodput Useful work completed within correctness and SLO constraints, rather than raw peak throughput. PD disaggregation Placing prefill and decode in worker pools that can be scheduled and scaled independently. Use reproducible experiments to localize device inference bottlenecks, then understand how training parallelism, KV scheduling, Prefill/Decode disaggregation, and serving SLOs extend the same ledger to clusters. DAY 15Shipping Edge LLMs
Close models, heterogeneous hardware, MCU/host boundaries, and operational evidence into a shippable system
Safety boundary Permission and range constraints that a deterministic component must enforce. Delegate An interface that compiles and runs a partitioned subgraph on a CPU, GPU, NPU, or other backend. Peak RSS The maximum resident physical memory reached by a process during measurement, exposing load or prefill peaks. Explicit fallback Moving to another backend or the cloud only under defined permission, privacy, deadline, and failure policy. Complete the three-tier responsibility boundary, model-package contract, heterogeneous execution, capacity and energy budget, degradation behavior, version release, and acceptance report for an edge LLM.