Summary

This post is an AI basics summary with practice questions I made myself. Based on the AI study posts on this blog, it covers the key ideas and easy-to-confuse comparisons for 15 topics, with 1–2 self-made multiple-choice questions per topic (28 in total). Open the toggle right below each question to see the answer and explanation.

  • ★ marks topics that are easy to mix up.
  • Where general knowledge fills a gap the blog doesn’t cover, it is marked (supplementary). Numbers such as (191) at the end of an explanation are the blog post numbers used as the source.
  • Starting post: donggrri.tistory.com/200 — the AI study posts that follow from it are the sources (written in Korean).

How do Temperature and Top-p differ? ★

Key ideas

  • The model outputs a probability distribution for the next token, and a separate rule (decoding strategy) decides which one to pick.
  • Greedy always picks the top-ranked token. Sampling draws randomly according to probabilities, so the answer can change even for the same question.
  • Temperature divides the scores (logits) by T to reshape the distribution. Lowering T sharpens the distribution around the top token (consistent), while raising it flattens the distribution to make answers more diverse (and increases off-the-wall answers).
  • Top-k keeps only the top k tokens, while top-p keeps candidates only until their cumulative probability reaches p.
  • Set them low for code or factual answers where accuracy matters, and high when you need diversity like brainstorming ideas. Usually, you only adjust either temperature or top-p, not both.

Easy to mix up

  • Temperature controls the shape (sharp/flat) of the distribution, while top-k and top-p control the range of candidates.
  • Top-k keeps a fixed number of candidates, whereas top-p changes the number of candidates depending on the distribution’s shape—fewer when it is sharp, and more when it is flat.
  • If you want consistent answers, you must lower the temperature. Raising it makes the answers more diverse.

Practice questions

Q1. You want to get nearly the same answer every time for the same prompt. Which setting is most appropriate?

  1. A. Raise temperature to 2.0
  2. B. Raise top-p to 1.0
  3. C. Lower temperature close to 0 (or use greedy)
  4. D. Greatly raise both temperature and top-p
Show answer and explanation

Answer: C

Lowering the temperature close to 0 or using greedy concentrates probability on the top-1 token, making the answers nearly identical. Raising it makes answers more diverse. (191)

Q2. Which statement about top-k and top-p is correct?

  1. A. Top-p always leaves the same number of candidates
  2. B. Top-k leaves candidates until the cumulative probability reaches k
  3. C. For tasks where accuracy is important, like coding, top-p should be set high
  4. D. With top-p, there are fewer candidates when the distribution is sharp, and more candidates when it is flat
Show answer and explanation

Answer: D

Because top-p is based on cumulative probability, the number of candidates varies with the shape of the distribution. Top-k is the one with a fixed number of candidates. When accuracy is important, set it low. (191)

Sources: donggrri.tistory.com/191

Why do Prefill and Decode have different bottlenecks? (KV Cache · HBM) ★

Key ideas

  • Prefill is the stage that reads the entire prompt all at once (in parallel). It fully utilizes GPU cores, making it compute-bound.
  • Decode is the stage that generates the response one token at a time. Because weights and the previous tokens’ K and V must be read from memory again for every single token generated, it is memory-bound (memory bandwidth-bound).
  • KV Cache stores and reuses the K and V of previous tokens. A new token only computes its own Q, K, and V, and appends the new K and V to the cache. Q is created anew every time.
  • As the context gets longer, the KV Cache grows, increasing both memory usage and the amount of data to read per token. That is why high-bandwidth memory like HBM is important.
  • GPUs and NPUs perform parallel computations with thousands of cores and read data quickly with HBM. Reducing the KV Cache or making memory faster is a key challenge for AI chips.

Easy to mix up

  • Compute is the bottleneck for Prefill, and memory is the bottleneck for Decode. Be careful not to memorize them the other way around.
  • Only K and V are reused; Q is not reused.
  • The K and V of the system prompt are computed only once, but since they must be read for every new token, the cache size and read burden remain the same (188).

Practice questions

Q3. What is the most appropriate reason why the LLM Decode stage is called memory-bound?

  1. A. Because model weights and previous tokens’ K and V must be read from memory again every time a single token is generated
  2. B. Because hundreds of prompt tokens are computed all at once, filling up the GPU cores
  3. C. Because Softmax computation is much heavier than matrix multiplication
  4. D. Because Q must be fetched from the cache
Show answer and explanation

Answer: A

Decode has to re-read weights and K·V from memory for every token. B describes Prefill. (186)

Q4. Which statement about KV Cache is incorrect?

  1. A. It computes the K and V of previous tokens once, stores them, and reuses them
  2. B. A new token computes its own Q, K, and V, and adds the new K and V to the cache
  3. C. As the context gets longer, the cache grows, increasing memory usage and the amount read per token
  4. D. Q is also reused from previous tokens, so there is no need to recreate Q for each new token
Show answer and explanation

Answer: D

Q represents “what the current token is looking for,” so it is created anew every time. Only K and V are reused. (186)

Sources: donggrri.tistory.com/186 , donggrri.tistory.com/189

How do we distinguish supervised, unsupervised, and reinforcement learning? ★

Key ideas

  • Supervised learning learns the relationship between inputs and outputs using data with answers (labels).
  • Unsupervised learning finds patterns or clusters within data without labels.
  • Reinforcement learning learns actions that maximize reward through trial and error within an environment.
  • (supplementary) Supervised learning is further divided into two: predicting a number is regression (e.g., tomorrow’s temperature, house price), and predicting a category is classification (e.g., spam or not).
  • (supplementary) Typical applications: customer segmentation (clustering) for unsupervised learning, and games, robot control, and RLHF (193) for reinforcement learning.

Easy to mix up

  • Without labels, it is clustering (unsupervised); with labels and predicting a number, it is regression (supervised). It is easy to mix up the presence of labels and the prediction target (number vs. category).
  • Pre-training (193) is called self-supervised learning because the text itself serves as the answer. It differs from supervised learning where humans attach labels.

Practice questions

Q5. Customer purchase records have no answer labels. If you want to group similar customers together, which method is appropriate?

  1. A. Supervised learning – Regression
  2. B. Unsupervised learning – Clustering
  3. C. Supervised learning – Classification
  4. D. Reinforcement learning
Show answer and explanation

Answer: B

Grouping similar items together without labels is unsupervised learning, specifically clustering. (214)

Q6. (based on supplementary) You want to predict tomorrow’s temperature as a number using past temperature records (with answers). Which is appropriate?

  1. A. Unsupervised learning – Clustering
  2. B. Supervised learning – Classification
  3. C. Supervised learning – Regression
  4. D. Reinforcement learning
Show answer and explanation

Answer: C

Since there are answers and you are predicting a number, it is supervised learning regression. (supplementary)

Sources: donggrri.tistory.com/214 (distinction between regression and classification is supplementary)

How do overfitting and data leakage differ? ★

Key ideas

  • Overfitting is when a model memorizes the training data too closely and performs poorly on new data.
  • Data leakage is when information from the evaluation set or the target answers leaks into training beforehand, making scores appear better than they actually are.
  • Example of data leakage: it can happen if you preprocess the entire dataset first and then split it into training and evaluation sets.
  • Sparse models like MoE were more vulnerable to overfitting, making fine-tuning trickier (197).
  • (supplementary) To reduce overfitting, you can increase data, use regularization, or stop early using validation data (early stopping).

Easy to mix up

  • Overfitting means a high training score but a low score on new data. Data leakage means the evaluation score itself comes out falsely high.
  • Overfitting is a problem of the model memorizing too much, whereas data leakage is a mistake in the process of splitting and handling data.

Practice questions

Q7. A model achieves 99% accuracy on training data, but 60% on unseen data. What is this phenomenon?

  1. A. Data leakage
  2. B. Hallucination
  3. C. Overfitting
  4. D. Quantization loss
Show answer and explanation

Answer: C

This is overfitting, where memorizing the training data too closely causes performance to drop on new data. (214)

Q8. In which case can data leakage occur?

  1. A. Applying preprocessing to the entire dataset first, then splitting it into training and evaluation sets
  2. B. Splitting into training and evaluation sets first, and not using evaluation data for training
  3. C. Collecting more training data
  4. D. Using the evaluation data only once at the very end
Show answer and explanation

Answer: A

If you preprocess first and split afterward, information from the evaluation data mixes into training, causing data leakage. (214)

Sources: donggrri.tistory.com/214 , donggrri.tistory.com/197 (solutions are supplementary)

How do zero-shot, few-shot, CoT, fine-tuning, and LoRA differ? ★

Key ideas

  • Prompt engineering is the technique of designing only the input while leaving the model unchanged to get the desired output.
  • Zero-shot gives only instructions without examples. It is fast, but results can be inconsistent on difficult tasks.
  • Few-shot shows a pattern with two or three examples. It is strong at classification, formatting, and setting the tone.
  • CoT (Chain-of-Thought) makes the model think step by step. It improves accuracy on math and complex problems.
  • Fine-tuning (SFT) further trains the weights using instruction–response data. LoRA freezes the original weights and trains only small adapters, saving resources.

Easy to mix up

  • Zero-shot, few-shot, and CoT leave the weights unchanged and only change the input. Fine-tuning and LoRA change the weights (LoRA changes only the small added weights).
  • Few-shot only “shows” examples and does not train on them. Using examples to train the weights is fine-tuning.
  • Use prompting to try things quickly without changing the model, and use SFT to bake a specific format or domain behavior into the model. If resources are limited, use LoRA.

Practice questions

Q9. What do we call the method of putting 2–3 similar examples in the prompt so the model matches the output format?

  1. A. Zero-shot
  2. B. Chain-of-Thought
  3. C. Fine-tuning
  4. D. Few-shot
Show answer and explanation

Answer: D

Showing a pattern with a few examples is few-shot. (188)

Q10. Which of the following statements is correct?

  1. A. Few-shot prompting changes the model weights
  2. B. Fine-tuning (SFT) further trains weights with instruction–response data, while prompting leaves the weights unchanged
  3. C. Zero-shot gets better the more examples you include
  4. D. LoRA retrains all of the original weights
Show answer and explanation

Answer: B

SFT further trains weights, while prompting leaves weights unchanged. Few-shot does not change weights, and LoRA freezes the original weights. (193, 188)

Sources: donggrri.tistory.com/188 , donggrri.tistory.com/193

Why do hallucinations happen and how can we reduce them?

Key ideas

  • Hallucination is when AI states false information plausibly and confidently—such as making up non-existent papers or asserting wrong dates.
  • Cause: An LLM does not look up facts; it probabilistically predicts a plausible next token. Saying “I don’t know” is not its default behavior.
  • Ways to reduce it: RAG (putting reliable documents into the Context), tools (web search, up-to-date info), instructing “say you don’t know if you don’t know” (not perfect), and human verification (checking sources, running code, cross-checking).
  • Garbage in, garbage out: if the documents fed via RAG are wrong or outdated, you will get wrong answers.
  • The 4 major limitations of AI are hallucination, lack of up-to-date information, bad Context, and Context limits.

Easy to mix up

  • Instructing the model to “say you don’t know” helps, but it is not a complete solution, because the model does not always know what it doesn’t know.
  • Just because code builds or runs does not mean the result is right. You must also test edge cases and unusual inputs.

Practice questions

Q11. What is the most fundamental reason hallucinations occur in LLMs?

  1. A. Because they probabilistically predict a plausible next token rather than looking up facts
  2. B. Because real-time internet searches frequently fail
  3. C. Because GPU memory is insufficient
  4. D. Because the tokenizer cannot read Korean
Show answer and explanation

Answer: A

LLMs predict plausible next tokens by probability, so they can make things up even when they don’t know. (199)

Q12. Which of the following is least related to reducing hallucinations?

  1. A. Finding reliable documents and putting them into the Context (RAG)
  2. B. Fetching up-to-date information with a web search tool
  3. C. Checking sources and cross-checking with other materials
  4. D. Greatly raising the temperature to get more diverse answers
Show answer and explanation

Answer: D

Raising the temperature can increase off-the-wall answers. A–C are methods for reducing hallucinations covered in the blog. (199, 191)

Sources: donggrri.tistory.com/199 (basis for why D is wrong: 191, raising temperature increases off-the-wall answers)

How do we distinguish Embedding, Vector DB, RAG, and LLM?

Key ideas

  • Embedding: turning sentences or words into numerical vectors that capture meaning (the values are determined through training).
  • Vector DB: a storage system that holds those vectors and quickly finds similar ones.
  • RAG: the overall approach of finding documents related to a question and putting them into the LLM’s Context. The flow is: Document → Chunk → Embedding → Vector DB → Question Embedding → Search → Context → Answer.
  • If a Chunk is too small, context is broken; if it is too large, irrelevant content gets mixed in. So chunks are often cut with a slight overlap.
  • Cosine similarity compares the directions of two vectors. It is 1 if they point in the same direction, and 0 if they are perpendicular. One-hot vectors always have 0 between different words, so they carry no similarity information.

Easy to mix up

  • Saying “Vector DB equals RAG” is wrong. A Vector DB is just one component; retrieving via keyword search or web search is also RAG.
  • One-hot vectors are long, sparse, and set by rules. Embeddings are short, dense, and learned through training.

Practice questions

Q13. Which statement about RAG is correct?

  1. A. RAG must always use a Vector DB
  2. B. A Vector DB is the entire RAG method itself
  3. C. Finding documents via keyword search or web search and putting them into the Context is also RAG
  4. D. RAG is a method of further training model weights
Show answer and explanation

Answer: C

RAG is the overall approach of retrieving information and putting it into the Context, so a Vector DB is not strictly required. Keyword search or web search also counts as RAG. (199)

Q14. What is the cosine similarity between “king” and “queen” using one-hot encoding?

  1. A. 1
  2. B. 0
  3. C. 0.96
  4. D. −1
Show answer and explanation

Answer: B

One-hot vectors for different words are always orthogonal (perpendicular), so their cosine similarity is 0. (202)

Sources: donggrri.tistory.com/199 , donggrri.tistory.com/202

How do LLM, Workflow, and Agent differ?

Key ideas

  • LLM answers right away when given a question. It has no tools.
  • Workflow operates in a sequence predetermined by a human. Example: every day at 9 AM, search 3 news articles → summarize → send email.
  • Agent has the LLM assess the situation and decide what to do next on its own. It repeats choosing a tool, observing the result, and judging again.
  • Risks of Agents: they can go in unintended directions, repeatedly call tools and drive up costs, build next actions on top of intermediate hallucinations, and perform irreversible actions like deleting files or sending emails.
  • Countermeasures are limiting permissions, requiring human approval before important actions, and verifying results.

Easy to mix up

  • The criterion that separates the two is not whether tools are used, but who decides the next action. If a human predetermines it, it is a Workflow; if the LLM decides it, it is an Agent.

Practice questions

Q15. What do we call a system with a predetermined sequence: “Every day at 9 AM, search 3 news articles, summarize them, and send them by email”?

  1. A. Agent
  2. B. LLM
  3. C. RAG
  4. D. Workflow
Show answer and explanation

Answer: D

Because a human predetermined the sequence, it is a Workflow. (199)

Q16. What is the key criterion that distinguishes a Workflow from an Agent?

  1. A. Whether tools are used
  2. B. Model parameter size
  3. C. Who decides the next action (whether a human predetermined it or the LLM decides on its own)
  4. D. Context window length
Show answer and explanation

Answer: C

The criterion is who decides the next action, not whether tools are used. (199)

Sources: donggrri.tistory.com/199

What roles do Q, K, and V play in Self-Attention?

Key ideas

  • Self-Attention is the process of weighting how much the current token should reference other tokens.
  • Using a library analogy, Q is the search query (current token), K is the book label (features of other tokens), and V is the actual content of the book (information of other tokens).
  • The flow is: Embedding → Q·K·V → Q·K score → Softmax (sums to 1) → weighted sum of V → new representation.
  • It is not the process of producing the final answer; it is the process of creating a richer token representation that reflects context.

Easy to mix up

  • The score is made by comparing Q and K, while the information actually blended together is V.
  • Why Q, K, and V are separated: if a single Embedding tried to play all three roles, information would get mixed up and it would be hard to find relationships.

Practice questions

Q17. Which correctly matches the library analogy for Q, K, and V in Self-Attention?

  1. A. Q = search query, K = book label, V = actual content of the book
  2. B. Q = book label, K = actual content of the book, V = search query
  3. C. Q = actual content of the book, K = search query, V = book label
  4. D. Q, K, and V all play the exact same role
Show answer and explanation

Answer: A

Q is the search query, K is the label, and V is the actual content. (183)

Q18. After Softmax, the weights came out to 0.7 for token B and 0.3 for token C. How is the new representation created?

  1. A. 0.7·K(B) + 0.3·K(C)
  2. B. Only V(B) is used
  3. C. 0.7·Q(B) + 0.3·Q(C)
  4. D. 0.7·V(B) + 0.3·V(C)
Show answer and explanation

Answer: D

You take the weighted sum of V using the weights: 0.7·V(B) + 0.3·V(C). (183)

Sources: donggrri.tistory.com/183

What does a Transformer block look like?

Key ideas

  • The input is word Embedding + position information (Positional Encoding).
  • One block consists of Self-Attention (tokens exchanging information with each other) → FFN (each token processing on its own), with a Residual Connection and LayerNorm attached at each step. Multiple such blocks are stacked in layers.
  • Lower layers capture simple word relationships, while higher layers capture the deep meaning of the whole sentence.
  • RNNs process one word at a time sequentially, so they are slow and forget the beginning of long sentences. Transformers process in parallel, but computation and memory grow significantly as sentences get longer.

Easy to mix up

  • Self-Attention is a meeting done together, whereas FFN is individual work done alone.
  • Why position information is needed: Attention looks at all tokens at once, so it does not know the order. It needs to distinguish “a dog bit a person” from “a person bit a dog.”

Practice questions

Q19. Why does a Transformer add position information (Positional Encoding) to word Embeddings?

  1. A. To reduce the number of words
  2. B. Because Attention looks at all tokens at once, so the order must be provided separately
  3. C. To reduce KV Cache capacity
  4. D. To turn tokens into numbers
Show answer and explanation

Answer: B

Because Attention does not know the order, position information is added to it. (184)

Q20. Which statement about Self-Attention and FFN is correct?

  1. A. Self-Attention exchanges information between tokens, while FFN processes each token on its own without looking at other tokens
  2. B. FFN exchanges information between tokens
  3. C. Self-Attention does not look at other tokens at all
  4. D. Both process one word at a time sequentially
Show answer and explanation

Answer: A

Self-Attention is a meeting (exchanging information), and FFN is individual work (processing alone). (184)

Sources: donggrri.tistory.com/184

How much memory does Quantization save?

Key ideas

  • Quantization approximates and stores weights using fewer bits. It reduces memory and data to read, but precision drops, which can lower accuracy.
  • Per parameter, FP16 takes 2 bytes, INT8 takes 1 byte, and INT4 takes 0.5 bytes.
  • For a 7B model, weights alone take 14GB in FP16, 7GB in INT8, and 3.5GB in INT4. These numbers exclude KV Cache and overhead.
  • It is highly effective because Decode is memory-bound—it reduces the bytes read from HBM per token.
  • PTQ quantizes after training is finished, while QAT accounts for quantization during training. The KV Cache can also be quantized.

Easy to mix up

  • Reducing bits does not always make it faster. The hardware and kernels must support it.
  • Compared to FP16, memory is 1/2 for INT8 and 1/4 for INT4. The accuracy risk is highest with INT4.

Practice questions

Q21. If a 7-billion-parameter (7B) model is quantized to INT4, roughly how much weight memory does it take? (calculating weights only)

  1. A. 14GB
  2. B. 7GB
  3. C. 28GB
  4. D. 3.5GB
Show answer and explanation

Answer: D

0.5 bytes × 7B = 3.5GB. (189)

Q22. Why is quantization especially effective during the LLM Decode stage?

  1. A. Because Decode is compute-bound, so reducing computation makes it faster
  2. B. Because Decode is memory-bound, reading weights from memory for every token, so reducing the data to read relieves the bottleneck
  3. C. Because quantization eliminates the need for KV Cache
  4. D. Because quantization always increases accuracy
Show answer and explanation

Answer: B

Because Decode is memory-bound, reducing the bytes to read relieves the bandwidth bottleneck. (189, 186)

Sources: donggrri.tistory.com/189

In what order do Pre-training, SFT, and RLHF proceed?

Key ideas

  • ① Pre-training is the stage that builds a Base model by predicting the next token on massive text. Because the text itself is the answer, it is self-supervised learning.
  • ② SFT (fine-tuning) is the stage that builds an instruction-following model through additional training on instruction–response examples.
  • ③ RLHF is the stage where humans rank responses by preference, a reward model is trained on those rankings, and the LLM is tuned via reinforcement learning (PPO). That turns it into a conversational model.
  • The goal is alignment—giving helpful, harmless, and honest answers. DPO is an alternative (training directly on preference data without a reward model).
  • Training updates weights, whereas inference generates tokens with weights frozen.

Easy to mix up

  • When given a question, a Base model might not answer it and instead just continue writing similar questions. What makes it conversational is SFT and RLHF.
  • In RLHF, humans do not write the ground-truth answers; they rank multiple answers.

Practice questions

Q23. What is the standard order in which an LLM is built?

  1. A. RLHF → Pre-training → SFT
  2. B. SFT → Pre-training → RLHF
  3. C. Pre-training → SFT → RLHF
  4. D. Pre-training → RLHF → SFT
Show answer and explanation

Answer: C

The order is Pre-training → SFT → RLHF. (193)

Q24. Which best describes the human role in RLHF?

  1. A. Ranking multiple responses generated by the model by preference
  2. B. Attaching next-token answer labels to every training sentence
  3. C. Directly modifying the model weights
  4. D. Building the tokenizer vocabulary
Show answer and explanation

Answer: A

Humans rank multiple responses by preference, and a reward model is trained with that data. (193)

Sources: donggrri.tistory.com/193

What are tokens and the context window?

Key ideas

  • A token is the unit an LLM uses to process text. It can be a character, part of a word, a whole word, or punctuation.
  • In English, 1 token is roughly 4 characters or 3/4 of a word. Korean can take more tokens for the same meaning (depending on the encoding).
  • red, Red, and red (with a leading space) are different tokens. BPE builds a vocabulary by merging frequently adjacent character pairs.
  • The context window is the limit on the total number of tokens visible in a single request. If exceeded, you handle it by trimming, deduplicating, splitting, or summarizing. In reasoning models, invisible reasoning tokens also take up output space.
  • Inside the context window, the system prompt goes at the very beginning, conversation history in the middle, and the user message at the end (188).

Easy to mix up

  • The number of tokens is not equal to the number of characters or words.
  • As the context gets longer, the KV Cache memory burden also grows with it (connected to 186).

Practice questions

Q25. Which statement about tokens and the context window is incorrect?

  1. A. Even sentences with the same meaning can have different token counts depending on the language and encoding
  2. B. The number of tokens is always equal to the number of characters
  3. C. In reasoning models, reasoning tokens invisible on screen are also included in output usage
  4. D. If you exceed the context window, you trim, split, or summarize the content before sending
Show answer and explanation

Answer: B

The number of tokens is not equal to the number of characters or words. (196)

Sources: donggrri.tistory.com/196 , donggrri.tistory.com/188

Why does MoE (Mixture of Experts) reduce computation while keeping memory the same?

Key ideas

  • MoE is an architecture that places multiple expert FFNs instead of a single FFN, and a router selects only a subset (top-k) of experts per token for computation.
  • Only a subset is computed per token, but all experts must be kept in memory, resulting in a heavy memory burden. That is why HBM capacity and bandwidth are important.
  • Mixtral 8x7B has about 47B (46.7B) total parameters, but with top-2 routing, it actually uses about 13B (12.9B) per token.
  • A load balancing auxiliary loss is used so tokens do not crowd into specific experts. Distributing experts across multiple devices is called expert parallelism.
  • Sparse models are more vulnerable to overfitting. Nor do dedicated language-specific experts emerge cleanly.

Easy to mix up

  • Total parameters and active parameters are different. In a Dense model they are the same, whereas in MoE active parameters are much fewer.
  • Computation decreases, but memory does not decrease.

Practice questions

Q26. Which statement about MoE is correct?

  1. A. Every expert participates in computation for every token
  2. B. Unused experts do not need to be loaded into memory
  3. C. It is less vulnerable to overfitting than a Dense model
  4. D. Computation per token uses only a subset of experts, but the weights of all experts must be kept in memory
Show answer and explanation

Answer: D

MoE computes with only a subset of experts, but all of them must be loaded in memory. And it is more vulnerable to overfitting. (197)

Sources: donggrri.tistory.com/197

How do Prompt, Context, and Harness Engineering differ?

Key ideas

  • Prompt engineering is writing and structuring instructions given to an LLM (Anthropic’s definition).
  • Context engineering is selecting and managing all the information the model sees, including tool definitions, search results, conversation history, and memory outside the instructions. The principle is “the smallest possible set of high-signal tokens.”
  • Harness engineering is designing the execution environment surrounding the model (tools, loops, verification, permissions, state). It is often expressed as “Agent = Model + Harness.” Graph engineering fixes the flow with nodes, edges, and shared state (LangGraph). Both are recent terms without a standard definition.
  • A system prompt is an instruction planted before a session starts and persists throughout the session. A user prompt changes every turn. If the system prompt is long, less room remains for conversation.
  • Start with the simplest one: move up in the order of Prompt → Context → Harness → Graph, and verify effectiveness with evaluations (evals). Practical tips: specify the role, specify the output format, state what not to do, do one thing at a time, and adjust one line at a time.

Easy to mix up

  • The prompt is part of the context, and managing context is one of the jobs of the harness. You can think of the scope expanding as Prompt ⊂ Context ⊂ Harness.
  • If context is “what to show,” the harness is “what to let it do and how to verify it.”
  • Signs of context pollution: repeating the same question, forgetting what was decided earlier, and giving contradictory answers. When this happens, opening a fresh session is safer.

Practice questions

Q27. Which of the following is the best example of “context engineering”?

  1. A. Adding a single line to the prompt specifying a table as the output format
  2. B. Having an agent hold only file paths and read files when needed instead of pasting the entire file every time
  3. C. Training the model weights with LoRA
  4. D. Replacing the GPU with one that has larger HBM
Show answer and explanation

Answer: B

Selecting and managing the information the model sees is context engineering (just-in-time). A is prompt engineering. (208)

Q28. Which statement about the system prompt is correct?

  1. A. It is an instruction placed before the session starts, persists throughout the session, and reduces the space available for conversation as it gets longer
  2. B. It is a question newly entered by the user every turn
  3. C. It can be added without limit regardless of the context window
  4. D. Once computed, it never needs to be read from memory during subsequent token generation
Show answer and explanation

Answer: A

The system prompt persists throughout the session and takes up space in the context. Even if K and V are computed only once, they must be read for every new token, so D is wrong. (188)

Sources: donggrri.tistory.com/208 , donggrri.tistory.com/188

Should we also know concepts not covered in the blog? (supplementary)

  • Recall vs. Precision (supplementary): Precision is the proportion of true positives among what the model predicted as “positive” (when you want to reduce false alarms). Recall is the proportion of actual true positives that the model caught (when you must not miss cases, e.g., patient diagnosis, defect detection). When data is imbalanced, you should not look at accuracy alone; you need to look at recall or precision together.
  • Knowledge Distillation (supplementary): A method where a smaller model (student) learns to mimic the outputs of a larger model (teacher), creating a small and fast model. Quantization (189) reduces the number of bits per number, whereas distillation retrains a smaller model itself, so they are different.
  • 4D: Delegation · Description · Discernment · Diligence (supplementary): Delegation is deciding what to entrust to AI and what humans will do. Description is clearly communicating goals, context, and format. Discernment is verifying and judging AI results. Diligence is upholding responsibility, ethics, and transparency for the outcome. Similar ideas in the blog include: “AI is the hands and I am the brain” (Delegation) and “checking whether its own statements match using a verification prompt” (Discernment) in post 200, and common rules · conditions · verification · format (Description · Discernment) in post 214. However, the blog did not use the name 4D.

Which posts are the sources?

Part 2: AI Basics Practice Quiz Part 2: 28 Multiple-Choice Questions — Can You Answer the Same Concepts From a Different Angle?