CodeIR

The problem isn't just volume - it's about what kind of information fills the context window. Consider what a transformer actually processes when it sees a Python function:

def get_user_permissions(user):
    if user.is_superuser:
        return Permission.objects.all()

To the model, this becomes roughly 30 tokens: def, get, _, user, _, permissions... and so on. But the actual architectural information is:

get_user_permissions → checks user.is_superuser → queries Permission model

Three facts. Everything else is syntax noise. In a typical Python file, 80% or more of tokens carry no architectural signal. Source code is a representation optimized for human editors and machine compilers. It was never designed for machine reasoning.

+ Deep Dive

Scale of the problem across real codebases:

Repository	Entities	Raw Python (est.)	CodeIR Index	Compression
Flask	1,629	~148k	19k	8:1
Tryton	20,457	~2.8M	214k	13:1
SQLAlchemy	38,672	~5.0M	467k	11:1
Django	41,819	~4.7M	475k	10:1

Token estimates use 1 token ≈ 4 characters. Actual counts may be slightly lower.

At Index level, CodeIR fits roughly 18,000 entities in a 200k context window. The same window holds less than 2,000 entities as raw source and only if you could perfectly select which files to load, which in practice you can't.

The bug was a subtle filter gap in finalize_flush_changes, the method that marks objects clean after a flush. The delete set correctly filters out list-only states with and not listonly. But the leftover set (other = states.difference(isdel)) doesn't apply the same filter, so cascade-pulled objects that were never modified flow into _register_persistent() and silently pollute the identity map. Each file reads fine on its own. The problem only surfaces when you trace the data flow across the set arithmetic and the downstream call.

Without CodeIR, Claude recognized it needed to orient before reasoning about the problem. It spawned a subagent that made 45 API calls, reading source files across the ORM layer to build a working understanding of the flush machinery. The main agent then used that context to read targeted sections and identify the bug. Total: 54 API calls, 2.06 million input tokens.

With CodeIR, that orientation step was already done. The agent searched the IR for flush-related entities, inspected one behavior summary to see the call graph, and expanded the 20 lines of source that mattered. Total: 14 API calls, 368k input tokens.

82% fewer input tokens. 74% fewer API calls. Same diagnosis, same fix.

+ Deep Dive

I introduced a single-line bug into SQLAlchemy's unit-of-work system and asked Claude Opus 4.5 to find it twice: once without CodeIR and once with it.

SQLAlchemy is 663 files and 38,672 entities. The bug was in finalize_flush_changes, the method that marks objects as clean or deleted after a flush. The delete set correctly filters list-only states: if isdelete and not listonly. But the non-delete set is computed as other = states.difference(isdel), which captures everything not being deleted, including list-only states that were pulled in by cascade but never modified. These flow into _register_persistent(), which replaces their identity map entries and resets their committed-state snapshots even though no SQL was emitted. Downstream queries then return stale in-memory data instead of re-loading from the database.

The prompt described symptoms, not locations:

"After a flush that involves relationship cascades, objects that were only pulled in as part of the cascade end up polluting the identity map as if they'd been persisted. Downstream queries return stale data. It only breaks with a mix of dirty objects and their unmodified related objects."

Results

Both runs found and correctly diagnosed the bug.

	API calls	Tool calls	Input tokens	Output tokens
Baseline (no CodeIR)	54	53	2,061,000	9,770
CodeIR	14	14	368,000	2,251
Reduction	74%	74%	82%	77%

What Happened

Without CodeIR, Claude spawned a subagent to orient itself. That subagent made 45 API calls and consumed 1.8 million tokens reading source files across the ORM layer, building the context needed to reason about the flush machinery. The main agent then read targeted sections of unitofwork.py, session.py, and identity.py to confirm the diagnosis. The orientation work was necessary but expensive.

With CodeIR, the orientation was already compiled. The agent ran codeir bearings to see the project structure, searched for flush-related entities, inspected finalize_flush_changes at the Behavior level to see its call graph without reading source, then expanded only the 20 lines of the method itself. It spotted the states.difference(isdel) gap, confirmed that other parts of the codebase correctly filter listonly, and proposed the fix.

Takeaway

Claude already knows it needs to understand a system before reasoning about it. Without CodeIR, it builds that understanding by reading source files, which is expensive and scales with repo size. CodeIR front-loads that work at index time so the agent starts with the architectural picture and drops to source only where it matters.

Setup

pip install codeir
codeir index ./your-repo

This creates a .codeir/ directory containing the index. CodeIR also generates files your agent reads automatically:

• .claude/rules/codeir.md - tool usage instructions (tells the agent how to use the CodeIR commands)
• .claude/bearings.md - the full architectural map of your codebase
• .claude/bearings/{category}.md - per-category detail for large codebases

The workflow changes from "read files and hope" to a structured discovery process:

The net effect: Claude Code stops reading files it doesn't need. It spends its context window on understanding the system instead of re-reading syntax.

+ Deep Dive

Available CLI Commands

codeir index <path>                              # Index or re-index a repository
codeir search <query> [--category <cat>]         # Multi-term OR search with ranking
codeir grep <pattern> [--path <dir>] [-i] [-C N] [-v]  # Regex search, grouped by entity
codeir show <entity_id> [--level Index|Behavior] # Show entity IR at specified level
codeir expand <entity_id>                        # Expand to full source
codeir callers <entity_id>                       # Three-tier caller resolution
codeir impact <entity_id> [--depth N]            # Reverse dependency analysis (BFS)
codeir scope <entity_id>                         # Minimal context for safe modification
codeir bearings                                  # Summary + menu with token estimates
codeir bearings [category]                       # Specific category (e.g., core_logic)
codeir bearings --full                           # Full module map

CodeIR is globally installed via symlink at /usr/local/bin/codeir. The .claude/rules/ directory is read automatically by Claude Code, so the integration requires zero manual configuration after indexing.

---

What bearings.md Looks Like

Each line describes one module: its ID, filename, category, entity count, dependencies, and churn. Grouped by category, ordered by entity count.

MD INVC invoice.py | cat:core_logic | entities:221 | deps:- | churn:-
MD SESS sessions.py | cat:core_logic | entities:18  | deps:auth,hooks,utils | churn:-
MD AUTH auth.py     | cat:core_logic | entities:12  | deps:sessions,models | churn:-
MD HOOK hooks.py    | cat:extension  | entities:7   | deps:sessions | churn:-
...

The entire architectural map of a codebase like Flask (83 files, 1,629 entities) fits in a few hundred tokens. Django (2,894 files, 41,819 entities) fits in a few thousand. This is the orientation layer - the agent reads it once and immediately knows the shape of the system.

For large codebases like Tryton, per-category bearings files break the map into manageable pieces. The agent loads .claude/bearings/core_logic.md when working on business logic, .claude/bearings/tests.md when writing tests - never the whole thing at once if it doesn't need to.

---

Entity IR at Each Level

Index - orientation (selection-level)

FN RDTKN #HTTP #CORE

Behavior - what it does and calls (reasoning-level)

MT WBHK.02 C=webhook_charge_captured,webhook_charge_dispute_closed,webhook_charge_dispute_created,webhook_charge_expired,webhook_charge_failed,webhook_charge_pending F=IR A=2 #CORE

Behavior fields: C= calls made, F= flags (R=returns, E=raises, I=conditionals, L=loops, T=try/except, W=with), A= assignment count, B= base class.

Flag/Field	Meaning	Why the LLM cares
C= (Calls)	Outgoing dependencies	Traces the "blast radius." If MT.A calls MT.B, the model knows a change in B might require an update in A.
F=IR (Flags)	Internal Logic (e.g., If, Return)	Indicates complexity. F=R is a simple pipe; F=ITLW (If, Try, Loop, With) is a high-logic function that needs careful handling.
A=2 (Assignments)	State changes	Tells the model if the function is "pure" or if it's juggling internal state. High assignment counts signal a "heavy lifter" function.
B= (Bases)	Class inheritance	Immediately establishes the "Rules of the Road" (e.g., knowing a class inherits from Model or Singleton without reading the imports).

Source expansion (verification-level)

[FN RDTKN @auth/tokens.py:47]
async def read_token(token: str, session: Session) -> TokenData:
    response = session.get(f"/tokens/{token}")
    ...

The agent navigates this hierarchy the same way a developer navigates a system: broad orientation first, then targeted deep dives only where needed.

---

Example Workflow (Tryton codebase)

1. codeir search "flush"               → no relevant results
2. codeir grep "def flush" --path orm/ → finds entity FLSH.04 in orm/session.py
3. codeir show FLSH.04                → see Behavior IR: what it calls, flags, assignments
4. codeir callers FLSH.04             → see what depends on it
5. codeir impact FLSH.04 --depth 2    → understand blast radius before changing
6. codeir scope FLSH.04               → get callers, callees, siblings for safe modification
7. codeir expand FLSH.04              → read source only for the entity you need to modify

The compilation process is one-time per repository: AST parsing extracts every entity (functions, classes, methods, constants), a classifier assigns each module to a category, and a multi-pass indexer generates all IR levels with stable, deterministic entity IDs. Re-indexing is incremental — only changed files get reprocessed.

+ Deep Dive

The Compilation Pipeline in Detail

Step 1: AST Extraction

CodeIR walks every Python file with an AST visitor, extracting functions, async functions, classes, methods, and module-level constants. Each entity gets boundary markers (start line, end line) and import analysis (what this entity imports and from where).

Step 2: Module Classification

Each file is classified into categories: core_logic, data_model, api_endpoint, auth, config, utility, extension, test, migration, cli, docs, constants, exceptions, init, router. Classification uses structural signals — import patterns, decorator presence, naming conventions — not LLM inference.

Step 3: Stable ID Generation

Every entity gets a deterministic ID that remains stable across re-indexing as long as the entity's name and location don't change. Format: TYPE STEM.SUFFIX (e.g., FN RDTKN.03, MT WBHK.02). The ID scheme uses vowel-stripped abbreviations with numeric suffixes for disambiguation.

Type	Meaning
FN	Function
CLS	Class
MT	Method
AMT	Async method

Stable IDs act as pointers for the LLM's memory, allowing it to maintain a constant "mental map" even as the underlying code changes.

Step 4: Multi-level IR Generation

Each entity is compressed at all levels simultaneously:

• Index strips everything except type, ID, and domain/category tags
• Behavior adds: calls made (C=), flags (F= with single-letter codes: R returns, E raises, I conditionals, L loops, T try/except, W with-blocks), assignment count (A=), base class (B=)
• Source wraps the original code with entity metadata headers ([TYPE ID @filepath:line])

Empty fields are omitted at Behavior level (no C=-, F=-, A=0). Trivial entities below a token threshold skip compression and store as source at all levels.

Step 5: Bearings Generation

Once classification and entity counting are complete, the index generates the module-level map. This contains no entity-level IR — it's a directory of modules grouped by category with dependency and churn information. For large codebases, per-category bearings files are generated in .claude/bearings/{category}.md.

Storage

SQLite with WAL mode, two-database design (entities + mappings). Created automatically in .codeir/ on first indexing. Incremental re-indexing uses change detection — only modified files trigger reprocessing.

---

A Note on What Was Evaluated and Removed

An intermediate level between Index and Behavior (type signatures only) was tested and cut. It was too compressed to be safe — agents would confidently reason from type information without seeing behavioral flags like exception handling, leading to plans that missed critical error conditions. The three-level stack is cleaner and each level has a clear, non-overlapping purpose.

What the baseline missed

The baseline read the reporting queries, found the aggregation problem, and fixed it: group by tax.rate, split the rows. Clean, minimal, correct… as long as nobody edits a Tax record after invoices have been posted against it.

But TaxLine doesn't store the rate that was used. It stores a foreign key to the Tax record: a live configuration object. The reporting fix joins back to that table at query time, so the report is only as reliable as the assumption that the Tax record still reflects what was true when the transaction happened.

The baseline didn't catch this because the vulnerability isn't visible in any single file. The reporting code looks fine. The Tax model looks fine. The TaxLine model looks fine. The problem lives in the relationship between them: a mutable record standing in for an immutable fact.

What CodeIR found

The CodeIR session traced the data lifecycle: how tax lines are created, what they store, and what they reference. That upstream exploration exposed the dependency on a mutable record and led to a different fix: capture the effective rate directly on TaxLine at write time.

It's a small schema change: one new field populated when the invoice is created. But it moves the guarantee from the report logic into the data model. Once a transaction is posted its tax rate becomes a recorded fact, not a live lookup. The reporting fix still applies: you still group by rate. But now you're grouping by a value that can't be silently rewritten.

CodeIR didn't stop at the reporting layer. It followed the data upstream and found a missing boundary between what the system configures and what it records.

+ Deep Dive

Same prompt, same repo (2,375 files, 20,457 entities). Once without CodeIR, once with it.

Run	Turns	Total tokens
Baseline	20	696,106
CodeIR	33	981,790

Baseline Plan

Core strategy: Modify aggregation queries to GROUP BY the effective tax rate, then report separate subtotals per rate within each period.

1. Phase 1: Add get_rate_periods() helper to detect rate boundaries within a period
2. Phase 2: Refactor AEAT303.get_context() to compute separate subtotals per rate-effective sub-period
3. Phase 3: Update ESVATList.table_query() and ESVATBook.table_query() GROUP BY to include tax.rate

CodeIR Plan

Core strategy: Capture the effective rate on TaxLine at write time. Report against the stored value.

Phase 1 — Schema: Add effective_rate field to TaxLine. Migration adds the column with NULL for historical records.

Phase 2 — Capture: Populate effective_rate at every TaxLine creation point:

• InvoiceTax.get_move_lines
• InvoiceLine._compute_taxes
• POSSale.get_tax_move_lines

Phase 3 — Reporting: Add tax_line.effective_rate to SELECT and GROUP BY in ESVATList.table_query().

Key files

File	Change
modules/account/tax.py:1337	Add effective_rate field to TaxLine
modules/account_invoice/invoice.py:3326	Store rate in InvoiceTax.get_move_lines
modules/account_invoice/invoice.py:3023	Store rate in InvoiceLine._compute_taxes
modules/account_es/reporting_tax.py:428	Group by rate in table_query
modules/account_es/reporting_tax.py:392	Add rate field to ESVATList model

The task: add per-blueprint session interfaces to Flask. The core challenge is timing: AppContext._get_session() runs before match_request(), so the session opens before the framework knows which blueprint will handle the request. Both runs identified this, proposed the same five entities that need changes, and surfaced the same design tradeoff: deferred session loading vs per-blueprint save only.

The difference was how they got there.

The baseline built its understanding by reading source files directly: sessions.py (386 lines), ctx.py (541 lines), and blueprints.py (693 lines). About 1,600 lines in total, most of which wasn't relevant to the final answer.

CodeIR took a different path. It navigated the system structurally, searching for session and blueprint entities, inspecting behavior summaries to understand call relationships, tracing callers of open_session and save_session, and expanding only the specific methods involved. It never read a full file.

Both runs used a similar number of tokens but the baseline spent them reading broadly and filtering down, while CodeIR navigated directly to the relevant parts of the system and read only what it needed.

+ Deep Dive

I asked Claude Opus 4.5 to design per-blueprint session overrides for Flask (83 files, 1,629 entities). Each blueprint should be able to specify its own session interface instead of using the app-wide one. I ran it twice: once without CodeIR and once with it.

Both runs arrived at the same diagnosis and the same design. The core constraint is timing: AppContext._get_session() fires before match_request() resolves which blueprint will handle the request. You can't override the session interface per-blueprint if the session is already open before the blueprint is known. Both runs identified the same five entities that need modification and surfaced the same tradeoff between deferred session loading and per-blueprint save-only.

The prompt:

"We're extending Flask's Blueprint system to support per-blueprint session overrides. Each blueprint should be able to specify its own session interface instead of using the app-wide one. Where would this need to be wired in, and what entities would need to change?"

Results

	API calls	Tool calls	Input tokens	Output tokens
Baseline (no CodeIR)	8	7	192,354	2,065
CodeIR	9	16	265,238	3,747

On a codebase this size, both approaches cost about the same in tokens. This is a small codebase. Flask is 83 files in total and the baseline's brute-force approach of reading three files cover-to-cover is cheap when the files are only a few hundred lines each.

The difference is in what Claude read and how it got there.

What Happened

Without CodeIR, Claude globbed the directory, grepped for class Blueprint and session_interface, then read three files in full: sessions.py (386 lines), ctx.py (541 lines), and sansio/blueprints.py (693 lines). That's 1,620 lines of source, most of which was irrelevant to the task. The relevant methods are maybe 80 lines across all three files.

With CodeIR, Claude searched for session and blueprint entities in the IR, inspected their Behavior summaries to understand call relationships without reading source, used callers to trace what calls open_session and save_session, then expanded only the specific methods involved. It made more tool calls (16 vs 7) but each one was targeted: a behavior inspection or a 20-line expansion rather than a 693-line file read. It never opened a full file.

Takeaway

Flask is the case where CodeIR's advantage is smallest. On a codebase this size, reading whole files is fast and cheap. The navigation advantage matters more as repos grow, which is what the SQLAlchemy and Tryton case studies show. But even here, the difference in approach is visible: one run reads everything and filters afterward, the other navigates to what matters and reads only that.