Vision

PrefaceCopy Anchor Link

Over the last years, agents stepped from toy-like experiments to parallelizeable, production-scale systems. In the software development lifecycle that shift ran through roughly these stages:

• "smart" autocompletion
• per-file or per-function code generation
• multi-file generation, sometimes one-shot
• single-agent codebase reads and edits, with a high level of autonomous reasoning
• multi-agent development, where one agent writes the code and another reviews it

The quality of LLMs is supposedly on a constant improvement track, and various LLM providers have entered the market. Yet, for the particularly complex task of software development (one that requires not just generating text, but showing actual, in-depth contextual understanding of the task and the available ways to resolve it), LLM knowledge alone wasn't enough. This led to the blooming popularity of various ways to bring more determinism into how code should be written, read, and understood by agents - specifically, harnesses and skills were the approaches used to improve the quality of the results.

Harnesses and skills seem to be a great idea in general to contextualize knowledge, yet the problem remains the same - agents working with a codebase quite frequently miss a lot of context that can be clearly and deterministically extracted from it. The closest the tooling comes to addressing this is through syntax facts (primarily tree-sitter-based), with some LSP additions used sporadically.

Memory layers (sometimes quite uncurated) become the place to persist partial bits of substantial knowledge, yet it's saved there without clearly contextualizing where the memory fact happens to be in the codebase. Agents themselves continue to frequently fall back to using grep-like utilities, which sometimes either burn the context window or miss important facts that can be structurally and deterministically extracted from the source code.

So, after criticising every single available tool and approach, it's time to answer the question: "What does Rift do differently?"

The main idea of Rift is to bring as much deterministic, fact-based (or fact-derived by a set of deterministic algorithms) introspection into source code as possible. We must therefore keep in mind that this information is split between the local index, which covers the project actively edited by agents, and the global index, which covers collected OSS packages. Both indexes use the same layers of intelligence outlined below.

ConceptCopy Anchor Link

On a surface level, Rift is an MCP server that follows the standard MCP specification. It comes in two modes:

1. Planned Global server at https://api.volar.sh/rift/mcp/. An agent can connect to it directly to search and inspect symbols across collected OSS packages.
2. Local server launched in place with your project. It provides the same layers of intelligence for project source and can reach the global index for facts about the packages that project uses. Yet it's also understandable that some facts might be in languages that are not yet supported or covered by Rift's capabilities - for those, we use a textual approach as a fallback.

The information provided by both of these servers is inherently deterministic because it is based on source code facts and derived from them by logically aligned algorithms.

Layers of intelligenceCopy Anchor Link

The title of this chapter might sound a bit like marketing, but practically it represents a logical structure that we keep in mind when we're talking about efficient information retrieval from the codebase. We consider it a ladder that codebase information climbs through to deliver as much context as possible.

TextCopy Anchor Link

Text practically consists of raw, bare reads of the codebase itself. This is, in fact, how most LLMs perceive it. Yet LLMs are indeed trained to assume that certain text is more likely to be code than something else.

However, we can still slightly improve this perception by simply putting a search around this information, which already increases the retrieval capabilities of the local server and avoids unnecessary context pollution. We primarily do this to ensure that if a language is not covered by Rift, it can still contribute (yet without any additional structural knowledge) to the signal that the agent might be looking for in the codebase.

Agents already write regular expressions for grep and ripgrep, so the local server answers them too. A trigram index over the text the local index stores selects the files a pattern's literals can occur in, and the pattern then checks those files, so a regex search answers every match from the index rather than from a walk of the disk. Each match also names the declaration holding it, which is where the text layer meets syntax.

SyntaxCopy Anchor Link

The syntax level is one of the simplest, yet it brings a lot of useful information about the codebase without requiring a sidecar such as an LSP process. Syntax is quite an interesting representation because it introduces the logical construction of the codebase and specifics like "this is a class, this is a function," but brings no semantic meaning to it. Consider the following Pythonic example:

task = spawn(some_function())

The syntax facts will be that task is an assignment target whose value is the result of a call written as spawn(some_function()). The outer call has spawn as its callee, and its argument is another call whose callee is written some_function. This set of facts alone might be enough for some simpler cases, but it doesn't bring any of the necessary semantic meaning that might be so greatly required in the codebase. For instance, we won't be able to say what the expected type of task is, nor will we know where the spawn function comes from.

SemanticsCopy Anchor Link

Syntax tells us how code is constructed, but it doesn't tell us what names refer to or what a language engine concludes about them. That missing meaning belongs to semantics. In Rift today, semantic facts come through language engines.

A language engine runs the language's own analysis and returns facts the parse does not contain. Rift currently asks language engines for references, call hierarchy, and diagnostics through LSP.

For relationship traversal, Rift asks the engine for references to the requested declaration, or for the calls it makes, and uses syntax to map the returned source locations back to indexed declarations. These engine facts belong to that read; they do not enter the index.

In some cases, Rift embeds a language engine when it is available as a Rust library. Other configured language engines run as separate LSP processes.

Relationship traversalCopy Anchor Link

For reads over the local index, Rift supports incoming traversal within the current project. A language engine returns the source locations that reference the requested declaration, and syntax identifies the declaration containing each location. Together they establish a relationship from each caller to the requested declaration for that read.

At the next depth, each caller becomes the declaration whose references Rift requests. This gives Rift bounded traversal within the project without indexing every relationship in advance.

Since v0.0.46 · Rust, TypeScript, Python Outgoing traversal walks the other way, from a declaration to the declarations it calls. Rift asks the engine for the declaration's call hierarchy, and syntax identifies the declaration that holds each callee's name.

A callee in a standard library or an OSS package has no declaration in the local index, so the walk stops there and the answer counts the callees it dropped. At the next depth, each callee in the project becomes the declaration whose calls Rift requests.

Planned The global index will hold relationships within and between OSS packages, allowing similar traversal.

DocumentationCopy Anchor Link

Documentation is another layer derived from the same source. As Rift runs syntax and semantic analysis, it can extract doc comments and documentation files together with the code, then associate that text with the declarations and packages it describes.

The documentation carries the same revision and origin as the source it accompanies. Search and symbol lookup can return it together with the code, while the global index can run the same extraction once for each published package version.

Revision and originCopy Anchor Link

Rift records revision and source ownership alongside the intelligence it collects. This means a symbol found through syntax, semantics, or documentation can be traced back through its history and to the source it came from.

RevisionCopy Anchor Link

Git supplies committed source along a file's first-parent history. Rift parses past revisions with the same syntax provider it uses for the working tree, allowing a declaration in the current code to be traced through its earlier states.

Since v0.0.46 Walking git on every read gets slower as history grows, so the local server analyzes past revisions ahead of time, in the background, into a history store. There is one store per repository, and every worktree of that repository reads it. The [providers.history] strategy decides which revisions the store holds:

• everything - the newest first-parent commits of every worktree, 100 by default
• selective - the releases you name by tag, such as v1.*, in version order

The store fills at a bounded share of one core, and no request waits for it: while it fills, a read answers from what the store already holds.

Symbol historyCopy Anchor Link

A symbol lookup with history follows a declaration through the revisions the history store holds, newest first. It shows changes to the declaration's body, signature, or decorators, as well as its introduction, removal, or move, and each version names the commit's author. A file renamed without a changed byte keeps its timeline, and so does a declaration moved unchanged into another file; a move that also edits the declaration ends it.

A change comparison compares a committed revision with another revision, or with the working tree and its uncommitted edits. Rift parses the files on each side and names the declarations that changed, where git diff names changed lines.

The history store also keeps each commit's message, so the agent can search commit messages for the commit that introduced a change. History reads describe these operations in more detail.

OriginCopy Anchor Link

Rift also records whether source belongs to the current project or an OSS package version. This keeps each declaration attached to the source it came from, including when a repository contains several projects or languages. The local and global indexes use the same origin model.

Codebase intelligenceCopy Anchor Link

Rift connects a declaration to the source that defines it, the references that use it, the declarations it calls, and its committed history. An agent reaches those facts through search and symbol lookup, whether they come from the local or global index.

Languages and providersCopy Anchor Link

Rift selects a syntax provider by file type. Tree-sitter parses the source into declarations and syntax nodes; for project files, byte ranges identify the text each read returns. Language engines add references and calls that require name or type resolution.

Engine selection belongs to each language identity. Several identities can share one named process. Available facts depend on the syntax provider and engine selected for that language.

IndexCopy Anchor Link

Rift uses two indexes: local and global. Both collect the same kinds of facts across the layers outlined above. They differ in which source they cover and where that analysis runs.

LocalCopy Anchor Link

The local server builds a local index for one project. It reads the project's text, syntax, semantics, documentation, and revision history on the user's machine. Project source does not enter the global index.

The server also reads the project's dependency list, or accepts one through configuration. It uses that list to identify which OSS packages are relevant. When a read needs facts about those packages, the local server requests them from the global index through an API, carrying the context of the current project. The local server analyzes no package source itself. When the global index is unavailable or holds no release of a dependency, the read still answers from project source, and a warning in the answer names the package facts it lacks.

Global PlannedCopy Anchor Link

The global index is a cached continuation of the local index. Rift populates it by applying the same layers of intelligence to collected OSS package versions. It holds facts about those packages and links between packages, so each local project does not have to reproduce the same analysis.

The global server exposes that index through two entrypoints. An agent connected directly over MCP can search and inspect the collected OSS packages. A local Rift server reaches it through an API, and the global server answers using the package context supplied by that project.

SearchCopy Anchor Link

The local server processes and indexes project source on the user's machine. The global service does not process or index that source. Planned Global search uses a separate index of collected OSS package versions.

Local rankingCopy Anchor Link

The local baseline combines identifier-aware matching with a SQLite FTS5 index over declaration names and the text of the indexed files, which the index stores once. FTS5 ranks matching units with BM25.

Local search can add optional vector search to this baseline. The reviewed local model families are:

• Model2Vec retrieval models, including the default minishlab/potion-retrieval-32M
• BGE retrieval models that use Rift's supported BERT configuration and query contract, including BAAI/bge-small-en-v1.5

Rift can download supported model weights from Hugging Face or read them from a configured workspace directory. Both paths execute the model inside the local server, so neither path sends project text to an embedding service.

The full-text index covers more of a declaration than its name and its source: the name, the qualified name, the identifier words those two split into, the rendered signature, the attached documentation, the declaration's own source, and the text of a selected file, each weighted so a name outranks a body. An answer states which of them carried the term.

Identifier matching, full-text matching, and vector similarity now rank one query together, rather than answering as lanes in turn. They score in units nothing converts between, so Rift reads the position each puts a result in and combines those by weighted reciprocal rank. [search.ranking] carries identifier_weight, lexical_weight, vector_weight, and fusion_k. Only the rankings that answered take part, so a vector index still building changes no order.

An OpenAI-compatible embedding endpoint adds optional remote embedding execution, configured through [search.vector.embedding] with the credential read from the environment variable api_key_env names. Endpoint execution is explicit because it sends declaration text while building vectors and query text while searching, which is the one path that takes project text off the machine.

Rift supports these local model families and OpenAI-compatible embedding endpoints. Hugging Face repository support does not mean that Rift can execute every model in that repository. Other model families, including Qwen3 embeddings, require a separate reviewed local runtime.

Global rankingCopy Anchor Link

The global server ranks facts from collected OSS package versions. A local server supplies the resolved package versions for its project, which keeps the remote answer within the dependencies that project can use.

The local and global rankings do not need the same embedding model. Each index ranks its own corpus, then Rift merges their ranks rather than comparing their model-specific scores.

Query routingCopy Anchor Link

The request's scope selects its corpus:

• local, the default, searches the primary project
• local@name searches one extra project accepted by the connection
• Since v0.0.72 local:name searches one profile defined in the root rift.toml, and local@name:profile one profile defined in an extra project's own rift.toml
• global searches collected package and runtime facts selected by the dependency context
• all searches the primary project, profiles, accepted extra projects with their profiles, and global facts and merges the ranked results

A read can also name package versions of its own through packages, such as an upgrade target or a package the project does not use yet. The global index then answers for those versions in place of, or beside, the ones the project resolves.