Implementation

This page goes into how Rift is built. It's developer-oriented - if you just want to use Rift, check the main documentation.

The rift binary

Since v0.0.11

Rift is delivered as one binary, installed as the Install section describes. The CLI is implemented with clap, and Rift's two main capabilities are implemented as rift mcp and rift server.

rift mcp is the MCP stdio server an agent launches. Under the hood it takes input from the agent over the stdio transport and forwards each request to the workspace's rift server over Streamable HTTP; the process holds no state of its own - state lives in the rift server. To reach the workspace's server it checks the server lock file - ServerLock at .rift/server.json - and starts a server when no live one stands behind it. Before it adopts a live server, the proxy compares package version, executable SHA-256, and canonical MCP schema SHA-256 from the lock with its own identity. A mismatch starts a matching server rather than forwarding across different binaries or tool documents.

rift server

The server is an Axum application on the Tokio runtime. MCP clients use its /api/mcp route, which serves Streamable HTTP through rmcp's service layer; /api/stop serves rift server stop, and the rest of the routes are reserved for other surfaces, the SCIP export for example.

Startup runs one cycle:

Parse rift.toml and accept the workspace; an invalid file keeps the server running but refuses every request until fixed.
Take the exclusive file lock on .rift/server.lock, which picks one starter among racing rift mcp processes.
Bind the first free loopback port of the accepted [server] selection - the pinned port or the configured port_range - mint the bearer token, and write both into .rift/server.json.
Start the native filesystem watcher.
Schedule the initial scan and parse on a Tokio-managed blocking task, so traversal and parsing never occupy an async MCP worker.
Serve until no request remains active and the idle_timeout span passes after the last response completes, then stop.

rift server start runs that cycle detached and waits for .rift/server.json to appear; detached startup and rift mcp each allow 30 seconds for publication. rift server stop sends the authorized stop request and waits for the election to release; rift server restart chains the two. rift server status reports the workspace's current state - a serving server with its port, pid, and version, a stale .rift/server.json with the reason it went stale, or no server - and changes nothing. --foreground keeps the cycle in the calling process, where Ctrl-C stops it.

Blocking tasks run on a Tokio-controlled worker pool. A request for blocking work lands in a queue, and a worker picks it up when one is free. This behavior is configurable via the num_workers and worker_queue_timeout keys in the [server] section.

The project resolver skips symlinks and the .git, .rift, and target directories.

The symbol index stays in memory as one immutable value: a rebuild constructs the next index and accepted configuration outside the async lock, then swaps the complete value under its write lock. Each request clones the published value under a read lock and retains that capture through its result, so reads never combine state from two index revisions.

Since v0.0.25

Watcher events and Rift-applied changes advance one workspace change counter and enter the same serialized rebuild lane. Each observation carries the paths it saw, and the rebuild reads only those files, sharing every other file with the previous index. A watch failure, a written .gitignore, or a directory appearing or disappearing reads the whole workspace instead. A written rift.toml reads the whole workspace when [source], a language file selection, or [search.text] inclusion or chunk bound changed. Every other accepted configuration change reuses the published tree.

Before each current-tree read, the server captures an exact digest of every included workspace file, then waits until the published index counter matches its observation. That capture already read the tree, so a read that finds the index behind names the files that moved rather than asking for a whole rebuild.

Rift FS

Planned

Rift FS mounts each served tree through the host's native client: NFSv3 loopback via nfsserve on macOS - an unprivileged mount, no kernel extension - and FUSE via fuser on Linux, unprivileged through fusermount3. A host where neither mounts falls back to a copy-backed tree with the same contract.

The overlay holds a pinned lower layer and a per-tree upper store. The first write to a path detaches it into the store; a deletion leaves a whiteout; .rift is never served.

Lower-layer reads go through a foyer hybrid cache - hot 256 KiB segments in memory, warm segments on disk. Removal force-unmounts without waiting on processes still inside.

rift db

Since v0.0.9 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, Python

One SQLite database at .rift/db holds the lexical index, semantic vectors, and server logs. The server lock, projection directories, and execution copies stay separate entries under .rift; the database carries stores that are rows, not trees. Access goes through Toasty models; the full-text surface is raw SQL, because Toasty 0.10 has no typed virtual-table or MATCH API, and the raw boundary stays confined to the FTS table and its rows.

One transaction writes the units and the stamp together, so the store is never a partial mix of two trees. A rebuild that read the whole workspace replaces the whole unit set; a rebuild that named the files it read deletes those paths' rows and inserts their new ones, which costs one delete and one insert batch per changed path instead of a rewrite of every row.

The transaction commits before the rebuilt index becomes current. One process-wide write turn serializes index commits, vector batches, and log batches, and each transaction starts with BEGIN IMMEDIATE.

The database uses WAL with synchronous=NORMAL. Read-only connections use committed WAL snapshots. search.busy_timeout bounds a lock wait caused by another process.

A commit that refuses publishes nothing and leaves the previous rows and stamp intact.

The database tables:

lexical_units - one row per indexed unit: a code symbol, or a text file split into chunks past the [search.text] chunk bound.
lexical_index_state - stamps the set with the tree revision that published it. One read transaction reads that stamp and runs the query, so a commit landing between the two cannot slip another tree's rows into an answer. A store holding another tree returns no rows at all: the request captured a publication that has since been superseded, and it recaptures rather than ranking rows it cannot place.
lexical_units_fts - the FTS5 virtual table over name and content, with identity riding unindexed as the join key; ranking is FTS5's bm25 with per-column weights, comparable only within one search.
semantic_vectors - one row per model and declaration digest.
log_records - bounded server diagnostics read through rift://logs and rift server logs.

A rebuild that named its files deletes and reinserts exactly those paths, which is why lexical_units carries an index on path. Every named path is replaced, added ones included: two rebuilds captured from one publication both write what they read, and the second leaves what the first left.

Search is a layered configuration: the default is this full-text store alone, and [search.semantic] adds the vector store below, whose ranking the server fuses with this one by reciprocal rank.

Semantic search

Since v0.0.16 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, Python

rift-search owns the tier. It runs the encoder in the server's own process, so answering a query touches neither the network nor a subprocess.

The model's own config.json decides which of two encoders reads it. A model_type of model2vec is a static model: the weights are one table of vectors, and the encoder embeds a declaration by gathering the row each of its tokens addresses and averaging them. Anything else is read as a BERT checkpoint through candle, pooled from the CLS position, and a retrieval query carries the prefix that checkpoint was trained with. A static model runs no forward pass, so neither the prefix nor the pooling applies to it.

The shipped default is static. A workspace is embedded once before its first search can rank semantically, and a forward pass over every declaration costs minutes on a laptop CPU where a row gather costs seconds.

Weights are read through candle's safetensors loader rather than its memory-mapped constructor: mapping is an unsafe call, and this workspace forbids authored unsafe. The cost is that a load holds the file's bytes once while the tensors are built.

Every encoder call runs on a blocking thread. Candle's forward pass would otherwise hold a runtime worker for the length of a batch, and a request arriving meanwhile would wait behind it. Passes run one at a time, bounded by batch_declarations and max_tokens. A BERT checkpoint's attention memory grows with the square of the token window, and candle's kernels already spread one pass across the thread pool, so running several at once buys nothing and costs the machine its memory.

The vectors go in semantic_vectors, in the same .rift/db. A row is addressed by the model that produced it and the digest of the text it came from, so a declaration that moves or is renamed resolves to the row already stored and a refresh embeds only what changed.

A pass publishes its vectors, the map that places each of them on a unit, and the tree revision they were described for as one value. A query reads all three or none: a corpus described for a tree other than the one the request captured ranks nothing, so a workspace published moments ago is answered by the full-text tier alone until the pass for that tree lands. Values are little-endian single precision with no header, so a row is a slice; a row whose stored width differs from the loaded model's is skipped, which invalidates a checkpoint that changed shape rather than reading it as noise.

Two sweeps keep the table bounded: one drops what the previous model wrote when model changes, the other drops digests no live declaration addresses. max_vectors bounds what the table may hold at all.

The vector table joins the lexical index's migration set rather than opening a second database handle with its own. Toasty records applied migrations in one __toasty_migrations table per file, so two independent sets on one file would interleave their ids; one set covers both tiers, and each store applies the same idempotent set whatever order the two open in.

Language providers

Since v0.0.23

A provider composition is a typed pipeline of named stages - source resolvers, fact providers, and transforms - connected through declared input and output flows, with keyed joins for revision-keyed history. The Rust builder inserts, replaces, branches, merges, and nests stages. Rift validates types, capabilities, and cycles at build, then owns scheduling, caching, and invalidation. Composition controls which providers run and how data reaches them; it cannot establish Contribution equivalence.

Each provider publishes one immutable, revisioned Contribution collection. Normalization selects Contributions applicable to a captured index revision, validates identity evidence, and builds workspace-local SymbolRecord values. Reads assemble a Symbol from one record and its Contributions. Provider order selects presentation fields only after normalization establishes identity.

Syntax providers wrap tree-sitter grammars behind a Rift-owned adapter: grammar node kinds resolve to numeric ids once, and raw query and cursor APIs never leave the syntax crate.

Since v0.0.25

An LSP-backed language engine is a child process the server spawns in stdio mode, rooted at the targeted tree's directory. The server starts one process per accepted inline or named LSP definition per targeted tree, and every exact language identity that selects that definition shares it. Semantic facts and edit plans return through the provider contract.

After a change, the server sends one classified path batch to each affected engine before it opens final document bytes and pulls diagnostics. That batch invalidates readiness from earlier work. Fresh progress tokens, diagnostic refresh requests, and repeated full reports decide when those answers are settled.

Since v0.0.27

The binding provider is the rift-binding crate, run in process by the index build. A syntax provider may extract binding facts - scopes, definitions, references, imports - in one walk over the tree its parse produced and attach them to its document. The Rust provider does, and supplies the ModuleLayout implementation the build resolves module paths through, fetched once per language from the provider registry. The registry itself assembles from one shipped definition list in rift-syntax: each entry names a language's identity, the file extensions it claims, and the provider parsing it, so adding a language is one definition plus one list entry.

The build assembles every document's facts into one graph, links modules across units, resolves each reference on a bounded work queue with a per-step precedence rank, and publishes the resolutions beside the syntax publication. An item definition carries the same declaration binding as its syntax Contribution, the equivalence evidence normalization uses to join the two records.

Hooks and configuration

Since v0.0.5

rift.toml at the workspace root is accepted at startup and re-accepted on change; its schema is exported beside the MCP surface as rift.schema.json, so editors validate the file as it is written. A quantity value carries its unit: timeout = "120s".

A hook is a configured command the server runs inside the changed tree after direct edits land, with a declared environment, bounded output, and a timeout. A transform may retain successful source writes in its declared scope; Rift recomputes the result from original bytes to final bytes. Validation runs afterwards, may retain no source write, and attaches GuaranteeEvidence or a diagnostic without refusing the landed change. A hook that fails or does not finish still gates publication in a projection. The configuration page states the keys and defaults.

Since v0.0.25

A hook declares its program in one command key: a string for a bare program, or a list holding the program and its literal arguments. The server starts that command directly and runs no shell.

The server selects a change's hooks from its initially changed paths, through each hook's include and exclude, and fixes that list before the first hook runs.

Implementation

On this page