Skip to main content

Explainer

Context Window vs Codebase Indexing: Why the Token Count on the Box Misleads You

What context windows and codebase indexing actually mean for AI coding tools, why a 1M-token spec does not mean the model understands your repo, and what to look at instead.

Updated November 18, 2026 · Based on 12 sources · How we research

If you are picking an AI coding assistant from the spec sheet, the number most vendors lead with is context window: 200K tokens, 1M tokens, sometimes 2M. This guide is for developers and engineering leads who want to understand what that number actually does, where it stops, and why some tools with smaller windows answer better on real repositories. The one thing most buyers get wrong: context window and codebase indexing are not the same thing. A bigger window describes how much the model can look at in one turn. Indexing is how the tool decides what to put in that window in the first place. You can have a huge window and still miss the file that matters, or a modest window and get the right answer because retrieval did its job. Our AI coding assistants ranking covers 17 tools that make very different choices on this axis, and this page explains how to read those choices.

What a context window actually is

A context window is the maximum number of tokens an LLM can process in a single request. Everything the model sees shares that budget: the system prompt, your chat message, any files the tool quoted into the prompt, prior turns, tool call results, and the answer the model writes back. When the Claude platform docs describe the window, they explicitly include conversation history and generated output in the count, not just the user's message (Anthropic).

Tokens are chunks of text, not words. One token is about 0.75 English words for prose, but code is denser because of punctuation, identifiers, and whitespace. Google states that its 1M-token Gemini window fits about 30,000 lines of code in practice (Gemini long context). Any file you load competes for the same budget as the model's reasoning space and its output.

Does a bigger context window mean the AI knows my whole codebase?

Usually no. A mature product codebase is bigger than you think. A mid-sized Django or Rails app runs 150,000 to 300,000 lines once you count tests and migrations. A monorepo at a scaleup is comfortably over a million. Even Gemini's 1M window, the largest in mainstream use, holds roughly 30,000 lines of code per Google's own estimate, so the arithmetic does not work on anything larger than a small service (Gemini long context).

More importantly, no assistant actually loads your entire repo per turn, even when the window technically allows it. Loading 1M tokens on every request is slow (seconds of prefill), expensive (Anthropic lists Claude Sonnet input at $3 per million tokens, so a full-window request is $3 before you get an answer), and, as benchmarks below show, hurts accuracy. The spec sheet is an upper limit, not a workflow. What the tool does to decide which few thousand tokens out of your 500,000-token repo to put in the window on any given turn is the actual interesting question.

Even when the code fits, the model does not read it evenly

Long contexts leak accuracy in ways the marketing numbers hide. Liu et al. at Stanford (with coauthors from UC Berkeley and others) showed in 2023 that models attend well to the start and end of a prompt and poorly to the middle, producing a U-shaped accuracy curve across the window (Lost in the Middle). This matters most for coding, where the function definition you need is as likely to sit in the middle of a big concatenated context as at the edges.

LongCodeBench, a benchmark that tests coding LLMs at context lengths from 32K up to 1M, found striking drops on the bug-fixing task (LongSWE-Bench). Claude 3.5 Sonnet's resolution rate fell from 29% at 32K tokens to 15% at 128K and 3% at 256K, a near-total collapse inside its own advertised window. On the comprehension task (LongCodeQA, multiple choice), Gemini 2 Flash held up better, staying in a 63% to 70% band across the full sweep, which is why Gemini gets singled out when teams talk about long-context reliability for retrieval, but it too dropped into single digits on bug fixing (LongCodeBench).

A 2026 industry analysis of effective context found that the usable share of the advertised window is typically 50% to 65%, and much less for some models: the same write-up pegs GPT-5.5 at 50% of its 1M, Claude Opus 4.6 at 49%, and Gemini 3.1 Pro at just 13% on the MRCR v2 multi-needle benchmark (CodingFleet, Enterprise DNA).

The practical reading: "1M tokens" on the box means the request will not be rejected, not that the model will reason about all of it.

What codebase indexing actually is, and why it is different

Codebase indexing is a separate layer that sits in front of the model. Instead of loading your whole repo into every prompt, the tool builds a persistent, searchable representation of the repo and retrieves only the chunks it thinks are relevant. The model then sees a prompt stuffed with the most relevant files, not the whole tree.

Cursor is the canonical example. Cursor uses a Merkle tree to detect which files have changed without rescanning everything, splits changed files into syntactic chunks, embeds each chunk with a code-specific model, and stores the vectors in Turbopuffer under a namespace per codebase. When you type @codebase or run Composer, your query is embedded with the same model and matched by semantic similarity; the matching chunks get injected into the prompt. Cursor reports this retrieval layer lifts response accuracy by about 12.5% on average (Cursor blog, Bito analysis).

Sourcegraph Cody takes a stricter approach. Instead of relying only on vector similarity, Cody uses SCIP-based precise code intelligence to build a full code graph of the repo, with real "go to definition" and "find all references" edges, then layers embeddings on top. This is why Cody is pitched for monorepos and cross-service codebases: the graph resolves symbols accurately across repo boundaries, which chops down hallucinated function names and imaginary types (Sourcegraph).

GitHub Copilot Enterprise runs cross-repository context through Copilot Spaces, which bundle whole repos, files, issues, and notes into a curated context set and keep the GitHub sources synced as projects change (GitHub Docs). The Pro tier does not get that. On Pro, context is limited to open files plus recent edit history, which is one reason the Pro plan is cheaper and one reason it performs worse on cross-file questions in a large repo.

Indexing is therefore not a replacement for the context window. It is the step that happens before the window gets filled. Good indexing plus a modest window often beats bad indexing plus a huge window, because the model sees better material.

How today's tools actually combine the two

Different products make very different trade-offs between these two levers. The table below groups the approaches in plain terms. The point is not which approach is correct; it is that "context window size" alone cannot distinguish them.

Note the fourth row. Claude Code deliberately does not build a persistent index. Its creator Boris Cherny has written that early versions used RAG plus a local vector database, but the team found "agentic search generally works better" for its workflow, so the shipped product gives the model a short tool hierarchy (Glob for file patterns, Grep for regex content search, and Read for pulling file contents) and lets the model drive exploration step by step (vadim.blog). Fresh reads every session, no stale embeddings, no vector store to secure. The trade-off is that very broad questions on a large repo cost more tokens, because the model has to grep its way to the answer each time.

ApproachExample toolsWhat it does
Giant context windowGemini Code Assist (1M), Claude Code (200K)Fits more into one prompt; still limited by lost-in-the-middle
Semantic index plus embeddingsCursor Pro, Copilot EnterpriseRetrieves relevant chunks on demand via vector search
Precise code graphSourcegraph Cody EnterpriseUses SCIP indexing for symbol-accurate retrieval across repos
Agentic search, no indexClaude CodeModel issues grep and read calls live; freshness over speed
IDE project indexJetBrains JunieReuses the IDE's own type graph and dependency index

Why does my AI assistant lose track of things mid-session?

Even if every individual turn fits inside the window, a long agentic session does not. Every tool call, every file the agent reads, every chat reply you send stacks tokens onto the running context. A two-hour Composer or Claude Code session can accumulate hundreds of thousands of tokens of history, tool output, and partial code before anything gets evicted.

Once the budget is close to full, the tool has to do something. Most tools summarize older turns, drop tool outputs, or evict file contents that were loaded earlier. Morph's analysis of "context rot" catalogs three mechanisms that compound this: the U-shaped lost-in-the-middle curve, attention dilution as tokens accumulate, and distractor interference where semantically similar but irrelevant content misleads the model. Morph also cites agent-run data showing that success rates fall sharply the longer a session goes, with failure rate rising faster than task duration (Morph LLM context rot). That is what "the model forgot what we discussed" usually means in practice. It did not forget; the harness evicted it, or attention drifted off the right span.

This is why tools with aggressive retrieval, like Cursor and Cody, hold up better on long sessions than tools that depend on keeping everything in-window. Retrieval re-fetches the file when needed, so eviction is survivable.

Why do AI coding tools still miss things even when I have a large context?

Four reasons, in order of how often they bite real teams.

First, retrieval quality. A 200K window is useless if the indexer picked the wrong files to put in it. Cursor's proprietary embedding model, trained on agent session data, is tuned specifically for code similarity; a generic text embedding will often miss the right file (Bito). Sourcegraph's SCIP graph trades some generality for precise symbol lookups (Sourcegraph). On Copilot Pro, retrieval is limited to open files and recent edits, so questions about code you have not opened recently simply do not land in the prompt.

Second, index freshness. An embedding written last Tuesday does not know your rename from Wednesday. Merkle-tree-based indexers like Cursor re-embed only changed chunks on each sync, which keeps staleness low. Tools without incremental sync will show you yesterday's API.

Third, attention inside the window. The lost-in-the-middle effect is real and model-specific. Claude 3.5 Sonnet degraded hard past 128K on bug fixing; Gemini models held comprehension accuracy better across lengths but still dropped to single digits on bug-fixing at long contexts (LongCodeBench). The "best" model at 32K is not always the best model at 512K.

Fourth, tool loop design. Agentic systems that can iterate, re-read, and verify tend to recover from bad first retrievals. Pure one-shot chat does not. This is why Claude Code's terminal-first agent pattern, which re-greps liberally, produces reliable answers on large repos despite not pre-indexing anything.

Practical reading: what to look at on the spec sheet

When you are comparing tools, treat the context window number as one of several inputs, not the headline. The questions that actually predict quality on a large repo are: does the tool build a persistent index of my repo, how fresh is that index kept, does the retrieval use a code-trained embedding or a general one, and does the agent loop allow re-reads when the first guess misses. Our head-to-head of Cursor, Claude Code and Copilot Pro walks through how three very different choices on this axis land in day-to-day use.

A quick decision frame: if you work in a single-project startup repo, a tool with strong local indexing and a strong model pays off, which is why Cursor Pro is the default pick for many engineers. If you work in a large monorepo or across many services, cross-repo code-graph retrieval matters more than any single window size, which is Cody Enterprise's pitch. If you prefer terminal-first workflows with fresh reads each time, Claude Code's no-index model fits. If your codebase sits inside Google Cloud and you want the largest window on the market as a safety margin, Gemini Code Assist's 1M tokens is a genuine edge, backed by comparatively stable long-context comprehension in benchmarks.

The short version

Context window is how much the model can look at in one turn. Codebase indexing is how the tool decides what goes in that turn. They are complementary, not interchangeable, and benchmarks show that effective context is roughly half the advertised number on most models, with sharp accuracy drops past the halfway mark. If you are choosing an assistant for a real codebase, weigh retrieval quality, index freshness, and agent loop design at least as heavily as the headline token count. For small-to-mid single-repo work, Cursor Pro's indexed-RAG approach is a safe default; for large enterprise monorepos, Sourcegraph Cody's code-graph retrieval is the stronger shape; for terminal-first developers who trust the model to drive its own search, Claude Code's no-index agentic approach holds up surprisingly well.

See Best AI Coding Assistant 2026

Questions people ask

What does context window mean for an AI coding assistant?+

It is the maximum number of tokens the model can process in a single request, counting the system prompt, your message, any files loaded by the tool, prior conversation, tool call output, and the response the model writes. Everything shares that budget. One token is roughly 0.75 English words or about 4 characters for prose, and code is denser than prose, so a token holds less code than the equivalent English.

Does a bigger context window mean the AI knows my whole codebase?+

No. Google's own figures put a 1M-token Gemini window at around 30,000 lines of code, which is smaller than most production repos. Tools almost never load the whole repo per turn because it is slow, expensive, and hurts accuracy. What actually determines whether the AI understands your repo is how the tool decides which files to put in the window, not how big the window is.

What is the difference between a context window and codebase indexing?+

The context window is a model property, the maximum prompt size the model accepts. Codebase indexing is a tool feature that sits in front of the model, building a searchable representation of your repo so the tool can retrieve the most relevant files at query time. Indexing decides what goes into the window; the window limits how much can go in. Cursor and Sourcegraph Cody rely heavily on indexing; Claude Code deliberately does not index and uses live grep and read instead.

Why does my AI assistant lose track of things mid-session?+

Long agentic sessions accumulate tokens from every tool call and reply. Once the running total nears the context window limit, the tool starts summarizing older turns or evicting file contents. A file the agent read early in a session can be gone later on. Benchmarks on context rot and the lost-in-the-middle effect confirm this: information in the middle of a long context is where models fall off hardest, and attention dilutes as the token count climbs.

Why does Cursor use codebase indexing instead of just a huge context window?+

Because indexing is cheaper, faster, and more accurate. Cursor builds a Merkle tree of your repo, chunks changed files, embeds them with a proprietary code-tuned model, and stores the vectors in Turbopuffer. At query time it retrieves the top semantically matching chunks and puts those in the model's prompt. Cursor reports this lifts response accuracy by about 12.5% on average, and it avoids paying for a million input tokens on every request.

What does 200K tokens mean in practice for coding?+

If you extrapolate from Google's figure that 1M tokens holds about 30,000 lines of code, 200K is on the order of 6,000 lines of source. In reality much of the 200K goes to the system prompt, conversation history, tool output, and the response, so usable file space is smaller. Benchmarks also show that models are less accurate at retrieving from the middle of a 200K context than from a 32K one, so the practical sweet spot is often well below the stated limit.

Will an AI with a 1 million token context understand my entire project?+

Unlikely, for two reasons. First, 1M tokens is about 30,000 lines of code per Google's own number, smaller than most enterprise repos. Second, independent benchmarks like LongCodeBench and 2026 effective-context analyses show the usable portion of a 1M window is typically 50% to 65% of the nominal number, with some models (Gemini 3.1 Pro, Claude Opus 4.7) coming in much lower on multi-needle tests. Claude 3.5 Sonnet's bug-fixing rate dropped from 29% at 32K to 3% at 256K. Big windows help, but they do not deliver automatic whole-repo understanding.

Why do AI coding tools still miss things even when I have a large context?+

Four common reasons: the indexer retrieved the wrong files, the index is stale from a recent rename or refactor, the model's attention is weak in the middle of a long context, or the agent loop cannot re-read on failure. In most real cases where an assistant misses a reference, it is a retrieval or eviction problem, not a model-capacity problem. A tool that can grep and re-fetch by symbol is more useful than one that depends on everything already being in-window.

Sources

What this guide is based on. We research from published specifications, independent reviews and owner reports; we did not test these products ourselves.

  1. 1.Securely indexing large codebases, Cursor
  2. 2.How Cursor's codebase indexing works (2026 Guide), Bito
  3. 3.Context windows, Anthropic
  4. 4.Claude Code Doesn't Index Your Codebase. Here's What It Does Instead., vadim.blog
  5. 5.The anatomy of an AI coding assistant, Sourcegraph
  6. 6.About GitHub Copilot Spaces, GitHub
  7. 7.LongCodeBench: Evaluating Coding LLMs at 1M Context Windows, arXiv
  8. 8.Lost in the Middle: How Language Models Use Long Contexts, arXiv (Liu et al.)
  9. 9.The Context Window Lie: How Well AI Models Actually Use 1M Tokens in 2026, CodingFleet
  10. 10.Context Rot: Why LLMs Degrade as Context Grows, Morph LLM
  11. 11.Gemini in Pro and long context, Google
  12. 12.LLM Context Limits: What Engineers Actually Found, Enterprise DNA

Products mentioned

More on ai coding assistants