Study notes preparing for the SK hynix AI Hackathon 2026 1st qualifying round (AI fundamentals and practical skills). A continuation of the Day4 Prompt Engineering post.

  1. What is hallucination? When an AI states something untrue in a plausible, confident way. Examples: inventing a paper title that doesn’t exist, or confidently giving a wrong date.

  2. Why does it happen? An LLM doesn’t look up facts; it predicts “the most plausible next token” by probability. Saying “I don’t know” when it doesn’t know is not its default behavior, so you can get an answer that sounds natural but is wrong.

    • Analogy: asked for three of King Sejong’s inventions while knowing only two, it makes one up because it doesn’t want to stop the conversation.
    • Exam one-liner: because an LLM predicts the plausible next token by probability instead of looking up facts, it can naturally make up things it doesn’t know.
  3. Ways to reduce hallucination

    • RAG: find trustworthy documents and put them in the Context so the model answers based on evidence.
    • Tools (web search): also solve the problem of up-to-date information after the training cutoff.
    • Prompting “if you don’t know, say you don’t know”: it helps, but it isn’t perfect, because the model doesn’t always know that it doesn’t know.
    • Human verification: check sources, actually run the code, cross-check.
  4. Bad Context (garbage in, garbage out): if the documents fed in through RAG are outdated or wrong, the AI trusts them and gives a wrong answer. RAG requires keeping documents current and managing sources.

  5. Four typical limitations of AI

    • Hallucination: it makes up plausible content even when it doesn’t know.
    • Up-to-date information: it doesn’t know what happened after its training cutoff.
    • Bad Context: put in wrong documents and you get wrong answers.
    • Context limits: not everything fits in the context window, and when it gets long the model loses track of earlier content.

Four typical AI limits (hallucination, up-to-date information, bad Context, context limits) and ways to reduce them (RAG, web search tools, keeping documents current and managing sources, a 'say you don't know' prompt), plus human verification (check sources, run the code, cross-check)

  1. Verifying code written by AI

    • Build and run it yourself.
    • Building and running doesn’t mean the result is right. Test with normal input, boundary values and odd input, and check that you get the expected result.
    • The compiler catches syntax errors, so the human focuses on whether the logic meets the requirements.
  2. Day5 conclusion: AI output is a draft, and verification is the human’s job. Check sources, test, cross-check.

Part 2. Embedding and RAG Architecture

  1. Telling the four apart (a frequent exam topic)

    • Embedding: turning a sentence into a numeric vector that carries its meaning
    • Vector DB: a store that keeps those vectors and quickly finds similar ones
    • RAG: the overall method of finding documents relevant to the question and putting them into the LLM’s Context
    • LLM: the model that reads that Context and produces the final answer
  2. RAG flow: documents → chunks (cut into small pieces) → embedding → store in Vector DB → embed the question too → search for similar documents → put them into the LLM Context → answer

RAG flow: ahead of time, documents → chunks → embedding → store in Vector DB; at question time, embed the question → search similar documents → put into the LLM Context → answer

  1. Why cut documents into chunks

    • Storing them in the units you need makes similar-document search more efficient and accurate.
    • Context limits: putting a whole document in eats up the context window, and unrelated content gets mixed in and blurs the answer.
    • Caution: too small and the context breaks; too big and irrelevant content creeps in. That’s why chunks are sometimes cut to overlap slightly.
  2. “A Vector DB is RAG” is wrong: a Vector DB is just one part of RAG; RAG is the whole method of finding, putting into Context, and having the LLM answer. RAG doesn’t have to use a Vector DB either. Retrieving with keyword search or web search is still RAG.

    • Exam one-liner: Embedding is conversion, a Vector DB is storage and search, RAG is the whole method of searching and putting results into Context, and the LLM is the model that produces the answer.

Part 3. AI Agents and Tool Calling

  1. What is an Agent? A plain LLM just answers right away when it gets a question. With an Agent, the LLM looks at the problem, judges for itself, picks a tool, tries it, looks at the result and decides the next action. It repeats this to reach the goal.
    • Flow: user question → LLM decides → picks a tool (search, Python, DB, API, files) → execution result → LLM decides again → … → final answer

Agent flow: user question → LLM decides → picks a tool (search, Python, DB, API, files) → execution result → LLM decides again → repeat → final answer. The criterion is who decides the next action, with permission limits, human approval and result verification as safeguards

  1. Example: “Tell me today’s weather in Seoul”

    • Plain LLM: it has no real-time information, so it either makes something up or says it doesn’t know. Either way, it can’t tell you the actual weather.
    • Agent: check the location → weather API or web search → look at the result and answer.
  2. LLM vs Workflow vs Agent (a frequent exam topic)

    • LLM: answers right away when asked. It has no tools.
    • Workflow: moves in an order a person set in advance. Example: every day at 9:00, search 3 news items → summarize → send an email.
    • Agent: the LLM looks at the situation and decides for itself what to do next. Example: given “Summarize recent issues in the semiconductor industry,” it chooses what to search for, searches again if that isn’t enough, and makes a table if needed.
    • The criterion is not whether tools are used but who decides the next action.
  3. Risks of Agents

    • The work can head in a direction the user didn’t want.
    • Calling tools repeatedly increases time and cost.
    • If a hallucination slips in midway, the next actions are built on a wrong result.
    • It can take irreversible actions such as deleting files or sending emails.
    • That’s why you put in permission limits, human approval before important actions, and result verification.

Next post

Machine learning basics, solving problems with AI, and comprehensive practice problems