CodeIR

AMT ATHNTCT.02 #AUTH #CORE

Entity type (AMT = async method), stable ID (ATHNTCT.02), domain tags (#AUTH #CORE). At this level the full codebase fits in a single context window. The agent scans thousands of these to identify which entities are relevant to a task. Token cost: ~10 tokens per entity.

AMT ATHNTCT.02 C=get_by_email,hash,update,verify_and_update F=AIRT A=2 #AUTH #CORE

Same entity with outgoing calls (C=), behavioral flags (F=: A=assignments/augmented assigns, I=conditionals, R=returns, T=try/except), assignment count (A=), and domain tags. The agent can reason about dependencies, complexity, and blast radius without reading source. This is the level where most architectural decisions get made — the agent knows what this function touches and how complex it is. Token cost: ~25 tokens per entity.

async def authenticate(
    self, credentials: OAuth2PasswordRequestForm
) -> Optional[models.UP]:
    """
    Authenticate and return a user following an email and a password.

    Will automatically upgrade password hash if necessary.

    :param credentials: The user credentials.
    """
    try:
        user = await self.get_by_email(credentials.username)
    except exceptions.UserNotExists:
        # Run the hasher to mitigate timing attack
        # Inspired from Django: https://code.djangoproject.com/ticket/20760
        self.password_helper.hash(credentials.password)
        return None

    verified, updated_password_hash = self.password_helper.verify_and_update(
        credentials.password, user.hashed_password
    )
    if not verified:
        return None
    # Update password hash to a more robust one if needed
    if updated_password_hash is not None:
        await self.user_db.update(user, {"hashed_password": updated_password_hash})

    return user

This is what the model reads without CodeIR — every time, in full, whether it needs the implementation details or not. ~120 tokens for one entity. Multiply across thousands of entities and the context window fills with syntax, comments, and formatting before the model has oriented itself. With CodeIR, this level is reached only when the agent needs to verify implementation or make a change.

The problem isn't just volume - it's about what kind of information fills the context window. Consider what a transformer actually processes when it sees a Python function:

def get_user_permissions(user):
    if user.is_superuser:
        return Permission.objects.all()

To the model, this becomes roughly 30 tokens: def, get, _, user, _, permissions... and so on. But the actual architectural information is:

get_user_permissions → checks user.is_superuser → queries Permission model

Three facts. Everything else is syntax noise. In a typical Python file, 80% or more of tokens carry no architectural signal. Source code is a representation optimized for human editors and machine compilers. It was never designed for machine reasoning.

+ Deep Dive

Scale of the problem across real codebases:

Repository	Entities	Raw Python (est.)	CodeIR Index	Compression
Flask	1,629	~148k	19k	8:1
Tryton	20,457	~2.8M	214k	13:1
SQLAlchemy	38,672	~5.0M	467k	11:1
Django	41,819	~4.7M	475k	10:1

Token estimates use 1 token ≈ 4 characters. Actual counts may be slightly lower.

At Index level, CodeIR fits roughly 18,000 entities in a 200k context window. The same window holds less than 2,000 entities as raw source and only if you could perfectly select which files to load, which in practice you can't.

Setup

pip install codeir-tools
codeir index ./your-repo

This creates a .codeir/ directory containing the index. CodeIR also generates files your agent reads automatically:

• .claude/rules/codeir.md - tool usage instructions (tells the agent how to use the CodeIR commands)
• .claude/bearings.md - the full architectural map of your codebase
• .claude/bearings/{category}.md - per-category detail for large codebases

The workflow changes from "read files and hope" to a structured discovery process:

The net effect: Claude Code stops reading files it doesn't need. It spends its context window on understanding the system instead of re-reading syntax.

+ Deep Dive

Available CLI Commands

codeir index <path>                              # Index or re-index a repository
codeir search <query> [--category <cat>]         # Multi-term OR search with ranking
codeir grep <pattern> [--path <dir>] [-i] [-C N] [-v]  # Regex search, grouped by entity
codeir show <entity_id> [--level Index|Behavior] # Show entity IR at specified level
codeir expand <entity_id>                        # Expand to full source
codeir callers <entity_id>                       # Three-tier caller resolution
codeir impact <entity_id> [--depth N]            # Reverse dependency analysis (BFS)
codeir scope <entity_id>                         # Minimal context for safe modification
codeir bearings                                  # Summary + menu with token estimates
codeir bearings [category]                       # Specific category (e.g., core_logic)
codeir bearings --full                           # Full module map

CodeIR is globally installed via symlink at /usr/local/bin/codeir. The .claude/rules/ directory is read automatically by Claude Code, so the integration requires zero manual configuration after indexing.

---

What bearings.md Looks Like

Each line describes one module: its ID, filename, category, entity count, dependencies, and churn. Grouped by category, ordered by entity count.

MD INVC invoice.py | cat:core_logic | entities:221 | deps:- | churn:-
MD SESS sessions.py | cat:core_logic | entities:18  | deps:auth,hooks,utils | churn:-
MD AUTH auth.py     | cat:core_logic | entities:12  | deps:sessions,models | churn:-
MD HOOK hooks.py    | cat:extension  | entities:7   | deps:sessions | churn:-
...

The entire architectural map of a codebase like Flask (83 files, 1,629 entities) fits in a few hundred tokens. Django (2,894 files, 41,819 entities) fits in a few thousand. This is the orientation layer - the agent reads it once and immediately knows the shape of the system.

For large codebases like Tryton, per-category bearings files break the map into manageable pieces. The agent loads .claude/bearings/core_logic.md when working on business logic, .claude/bearings/tests.md when writing tests - never the whole thing at once if it doesn't need to.

---

Entity IR at Each Level

Index - orientation (selection-level)

MT FLSH.05 #HTTP #CORE

Behavior - what it does and calls (reasoning-level)

MT FLSH.05 C=UOWTransaction,UnmappedInstanceError,CATBGNT,_begin,_commit_all_states,_expunge_states F=EILRTW A=22 #HTTP #CORE

Behavior fields: C= calls made, F= flags (R=returns, E=raises, I=conditionals, L=loops, T=try/except, W=with), A= assignment count, B= base class.

Flag/Field	Meaning	Why the LLM cares
C= (Calls)	Outgoing dependencies	Traces the "blast radius." If MT.A calls MT.B, the model knows a change in B might require an update in A.
F=IR (Flags)	Internal Logic (e.g., If, Return)	Indicates complexity. F=R is a simple pipe; F=ITLW (If, Try, Loop, With) is a high-logic function that needs careful handling.
A=2 (Assignments)	State changes	Tells the model if the function is "pure" or if it's juggling internal state. High assignment counts signal a "heavy lifter" function.
B= (Bases)	Class inheritance	Immediately establishes the "Rules of the Road" (e.g., knowing a class inherits from Model or Singleton without reading the imports).

Source expansion (verification-level)

[FN RDTKN @auth/tokens.py:47]
async def read_token(token: str, session: Session) -> TokenData:
    response = session.get(f"/tokens/{token}")
    ...

The agent navigates this hierarchy the same way a developer navigates a system: broad orientation first, then targeted deep dives only where needed.

---

Example Workflow (Tryton codebase)

1. codeir search "flush"               → no relevant results
2. codeir grep "def flush" --path orm/ → finds entity FLSH.04 in orm/session.py
3. codeir show FLSH.04                → see Behavior IR: what it calls, flags, assignments
4. codeir callers FLSH.04             → see what depends on it
5. codeir impact FLSH.04 --depth 2    → understand blast radius before changing
6. codeir scope FLSH.04               → get callers, callees, siblings for safe modification
7. codeir expand FLSH.04              → read source only for the entity you need to modify

The compilation process is one-time per repository: AST parsing extracts every entity (functions, classes, methods, constants), a classifier assigns each module to a category, and a multi-pass indexer generates all IR levels with stable, deterministic entity IDs. Re-indexing is incremental — only changed files get reprocessed.

+ Deep Dive

The Compilation Pipeline in Detail

Step 1: AST Extraction

CodeIR walks every Python file with an AST visitor, extracting functions, async functions, classes, methods, and module-level constants. Each entity gets boundary markers (start line, end line) and import analysis (what this entity imports and from where).

Step 2: Module Classification

Each file is classified into categories: core_logic, data_model, api_endpoint, auth, config, utility, extension, test, migration, cli, docs, constants, exceptions, init, router. Classification uses structural signals — import patterns, decorator presence, naming conventions — not LLM inference.

Step 3: Stable ID Generation

Every entity gets a deterministic ID that remains stable across re-indexing as long as the entity's name and location don't change. Format: TYPE STEM.SUFFIX (e.g., FN RDTKN.03, MT WBHK.02). The ID scheme uses vowel-stripped abbreviations with numeric suffixes for disambiguation.

Type	Meaning
FN	Function
CLS	Class
MT	Method
AMT	Async method

Stable IDs act as pointers for the LLM's memory, allowing it to maintain a constant "mental map" even as the underlying code changes.

Step 4: Multi-level IR Generation

Each entity is compressed at all levels simultaneously:

• Index strips everything except type, ID, and domain/category tags
• Behavior adds: calls made (C=), flags (F= with single-letter codes: R returns, E raises, I conditionals, L loops, T try/except, W with-blocks), assignment count (A=), base class (B=)
• Source wraps the original code with entity metadata headers ([TYPE ID @filepath:line])

Empty fields are omitted at Behavior level (no C=-, F=-, A=0). Trivial entities below a token threshold skip compression and store as source at all levels.

Step 5: Bearings Generation

Once classification and entity counting are complete, the index generates the module-level map. This contains no entity-level IR — it's a directory of modules grouped by category with dependency and churn information. For large codebases, per-category bearings files are generated in .claude/bearings/{category}.md.

Storage

SQLite with WAL mode, two-database design (entities + mappings). Created automatically in .codeir/ on first indexing. Incremental re-indexing uses change detection — only modified files trigger reprocessing.

---

A Note on What Was Evaluated and Removed

An intermediate level between Index and Behavior (type signatures only) was tested and cut. It was too compressed to be safe — agents would confidently reason from type information without seeing behavioral flags like exception handling, leading to plans that missed critical error conditions. The three-level stack is cleaner and each level has a clear, non-overlapping purpose.

"What's in this codebase?" → codeir bearings returns the full architectural map: every module, its category, entity count, and dependencies. The agent reads this once and knows where to look before it looks anywhere.

"Which entities match this task?" → codeir search "auth token validate" does multi-term OR search with ranking across the full index. --category core_logic filters out tests and utilities.

"Where does this pattern appear in source?" → codeir grep "session_id" --path auth/ does regex search grouped by entity, with -v for full IR context per match.

"What does this function actually do?" → codeir show ATHNTCT.02 --level Behavior returns the entity's calls, flags, assignment count, and tags — enough to reason about it without reading source.

"What calls this function?" → codeir callers ATHNTCT.02 returns three tiers: import-level, local, and fuzzy matches (marked ~ for probable but uncertain).

"What breaks if I change this?" → codeir impact ATHNTCT.02 --depth 2 traces reverse dependencies via BFS, grouped by distance, with the full dependency chain.

"What's the minimum context to safely edit this?" → codeir scope ATHNTCT.02 returns callers, callees, and sibling methods — everything the agent needs to understand blast radius before making a change.

"Show me the actual code." → codeir expand ATHNTCT.02 returns the raw source with entity boundary markers. This is the last step, not the first.