Why Run ripgrep Before Agent Codebase Search

If you only ask an agent “where is this symbol used?”, the tools may loop through broad indexes, semantic search, and file opens until the context fills up. Run rg (ripgrep) first to cut candidate paths and symbols; then the agent reads, edits, and verifies inside an already-narrowed set.

This post covers three axes only: why rg first? · how does that differ from agent codebase search? · how do you put results into the prompt? No pricing, plans, token limits, affiliates, or invented reviews.

Grounded in the ripgrep GUIDE and rg(1) man page (ignore, glob, type), plus public product-UI descriptions of agent codebase search. We do not invent search quotas or fees.

Why run rg first?

One-line answer: rg is a reproducible CLI filter. It respects .gitignore / .ignore / .rgignore by default, lets you control the file set with -g / -t, and produces stdout you can paste into a ticket or prompt.

Relying on “search the whole repo” in the agent alone tends to:

  1. Drift scope per chat — which files open can change with tool choice and remaining context.
  2. Burn context on noise — vendor trees, build outputs, and unrelated extensions sneak in (see agent-deny-paths for deny lists).
  3. Leave weak evidence — for PRs and issues, command + output is the cheapest audit trail.

What ripgrep skips by default (per GUIDE):

FilterDefault
gitignore familyHonors .gitignore, .ignore, .rgignore
hiddenSkips .-prefixed files and dirs
binarySkips files with NUL (etc.)
symlinksDoes not follow unless -L

Common scope cuts (public flags only):

# Type + globs for first-party sources
rg 'handleSubmit' -t ts -g '!**/*.test.*' -g '!dist/**'

# Narrow with PATH args
rg 'TODO|FIXME' packages/api/src -n -C 2

# Paths only (hand a file list to the agent)
rg -l 'FeatureFlag' -t py apps/

-u / -uu / -uuu progressively disable ignore → hidden → binary. Do not jump to -uuu when a hit is missing—start from the intended source tree.

Avoid:

  • Giving only the repo root and repeating “search somehow.”
  • Trusting semantic hits alone to define the patch set.
  • Pasting rg dumps that include node_modules after disabling ignore.

One-line answer: rg is deterministic, local, filter-first; agent codebase search is a product UI tool (index / semantic / multi-file exploration). Use them as preprocess → agent, not as substitutes.

Axisrg (CLI)Agent codebase search (product UI)
InputRegex/literal + glob/type + PATHNatural language / symbols / tool-chosen queries
ReproducibilitySame command → same output (same tree)Can vary with chat, tools, and context
FilteringHuman-owned .gitignore + -g/-tProduct ignore/index rules (as documented in UI)
Artifactstdout paths/text — easy to pasteIn-chat summaries/open files — export separately for evidence
RoleCut the candidate setRead and edit inside that set

Codebase search as UI is strong for “find similar code / open related files.” We do not assert numeric limits, index sizes, or prices—those depend on plan and time. Practical rule of thumb:

  1. If the string is known, run rg first (symbol names, error text, feature-flag keys).
  2. If the name is fuzzy, use rg to cut directories, then restrict agent search/reads to those paths.
  3. When you need semantic help, still pass a cwd/mention scope such as packages/foo (monorepo-agent-root).
# Human locks scope
rg -n 'AuthProvider' -t ts packages/web/src

# Hand only those paths to the agent (conceptual)
# "Only these paths: … Edit login redirect. Do not search outside packages/web/src."

One-line principle: codebase search is an exploration UI; rg is a scope contract. Write the contract first, then keep the UI inside it.

How do you put results into the prompt?

One-line answer: Keep path, line, and short context from rg, and paste a fixed block: goal + allowed paths + denials + verify command. Do not dump the whole tree.

Recommended pipeline:

  1. Search — record pattern, -g/-t, and PATH.
  2. Shrink — -l for file lists; -n -C 2 for a few files only.
  3. Prompt block — use the template below.
  4. Agent — forbid search/edit outside allowed paths in Rules or the message.
  5. Verify — re-run the same rg (or tests) before/after.

Prompt fragment:

## Scope (from rg — do not widen)
Command: rg -n 'FeatureFlag\.BETA' -t ts -g '!**/*.test.*' packages/
Hits (path:line):
  packages/api/src/flags.ts:42
  packages/web/src/hooks/useFlags.ts:18

## Task
Change BETA default to false in api; update web hook consumer only.

## Constraints
- Do not search or edit outside the paths above.
- Do not open node_modules/, dist/, or .env*.
- After edit, re-run the same rg and summarize remaining hits.

## Done when
- rg shows only the intended call sites
- unit tests for flags still pass

What to include vs drop:

IncludeDrop
path:line + short -C snippetsHits under node_modules, locks, binaries
The exact rg command“Just search everywhere”
Allow/deny path listsScope explained only by screenshots
The same command for re-checkEntire prior chat history
# File list for the prompt
rg -l 'FeatureFlag\.BETA' -t ts packages/ | head -40

# Review context (cut PATH further if long)
rg -n -C 2 'FeatureFlag\.BETA' -t ts packages/api packages/web

Tip: Pin one line in AGENTS.md / Cursor Rules:
Prefer: run rg with -g/-t to narrow paths before codebase search. Paste path:line hits into the task brief.

One-line wrap-up: Cut candidates with rg → paste path:line as a prompt contract → let the agent edit only that scope.

FAQ

Q. Must it be rg instead of grep -r?
A. No. ripgrep’s ignore/glob/type defaults fit agent preprocess well. If the team standardizes on git grep, the same preprocess pattern still applies.

Q. Can the agent run rg in the shell itself?
A. Yes. Still have a human run it once to lock scope, then tell the agent “this command and these paths only” for reproducibility.

Q. When is -uuu appropriate?
A. When you intentionally disable ignore/hidden/binary filters. That can mix in secrets and vendor noise—do not paste that dump straight into an agent prompt.

Q. When is semantic search alone not enough?
A. When you have a literal—exact strings, flag keys, error codes. Then rg is cheaper and leaves evidence.

Sources