rift
Developer

Implementation

This page goes into how Rift is built. It's developer-oriented - if you just want to use Rift, check the main documentation.

The rift binary

Since v0.0.11

Rift is delivered as one binary, installed as the Install section describes. The CLI is implemented with clap, and Rift's two main capabilities are implemented as rift mcp and rift server.

rift mcp

rift mcp is the MCP stdio server an agent launches. Under the hood it takes input from the agent over the stdio transport and forwards each request to the workspace's rift server over Streamable HTTP; the process holds no state of its own - state lives in the rift server. To reach the workspace's server it checks the server lock file - ServerLock at .rift/server.json - and starts a server when no live one stands behind it.

A server that rift mcp or rift server start starts exits when its claim on .rift/server.lock meets any other lock. When that leaves no process holding the election - on Windows a probe's shared lock can outlive the probe for a moment - the starter starts another within the same 30-second window.

When the window closes on a server that holds the election but has not published .rift/server.json, the proxy weighs the index and metrics databases. A server writes .rift/index while it builds its first index and records its diagnostics into .rift/metrics, so when either file or its write-ahead log changed during the window, the request gets a temporarily_unavailable refusal saying the server is still building, and the agent resends it. A holder that wrote nothing keeps the refusal that names an operator action.

Each forwarded request waits at most the proxy's forward budget: the [server] readiness_timeout and worker_queue_timeout the workspace accepts, plus an allowance for the answer. When the budget ends, the proxy cancels the request, which closes its Streamable HTTP response stream, and the caller gets a temporarily_unavailable refusal naming the budget.

Since v0.0.62

rift mcp takes one --output choice: text, the default, or all. The proxy keeps the choice for its lifetime, across upstream reconnects, so the tool listing it advertises stays stable. With text, the proxy removes structuredContent from each completed tool result, outputSchema from every tool on every tools/list page, and the application/json contents of a completed resource read that holds another content, all after the upstream answer arrives; all forwards everything unchanged.

Each rift mcp process owns its choice, so two proxies on one workspace can differ.

Before it adopts a live server, the proxy compares the product identity from the lock with its own: the version and the canonical MCP schema SHA-256. The version is the package version with the git commit the binary was built from as semantic-versioning build metadata, such as 0.0.45+b006b8433ba06679f06a3c7f0743d65634d32c34. rift --version prints the package version alone.

A build script in the rift crate asks git for the commit at build time, and rift hands it to rift-mcp when it serves, so no start reads the executable's bytes. A build from a working tree with uncommitted changes adds .dirty. with the executable's size in bytes and modification time in nanoseconds, read from file metadata, so a rebuilt development binary never matches a server the earlier build started. A build with no git carries the package version alone.

When the identities differ, the proxy orders the two versions by semantic-versioning precedence, which leaves build metadata aside, so two builds of 0.0.45 stand level whatever their commits:

  • A server whose version strictly precedes the proxy's receives the authorized stop request rift server stop sends. The proxy waits for that server to release its election and then starts one from its own binary; a request the server already holds finishes first.
  • A server whose version equals the proxy's from another build, or follows it, is refused: the caller receives a JSON-RPC internal error naming both identities and asking for rift server stop.

The stop request carries the token the lock records, and every server mints its own, so it stops the one server the proxy read and never a server another proxy started after it. A proxy stops at most one server per connect: when another older server takes the election after the first one left, the proxy refuses rather than race the process that started it. A stop request that fails in transport decides nothing, since a server another proxy already stopped closes the connection as it exits. The proxy reads the election again, as after an accepted request, and refuses only when its 30-second wait for publication ends with that same server still serving.

rift server

The server is an Axum application on the Tokio runtime. MCP clients use its /api/mcp route, which serves Streamable HTTP through rmcp's service layer; /api/stop serves rift server stop.

Since v0.0.62

When the server builds a tool result, it renders compact text from the typed answer into content and serializes the same answer into structuredContent. Typed layout code renders every tool answer, every failure, and the workspace resource as sections: a title at column 0 that counts its entries, such as 2 results, 1 warning, or 2 errors, with the entries indented under it. The text writer enforces a byte limit on each text and refuses one that crosses it.

A workspace resource read carries the compact text as text/plain, then the JSON body as application/json. A map read carries one application/json content under both output choices.

Only rift mcp applies the output choice, so direct HTTP access returns both tool and workspace representations.

Where the server converts a tool method's result, it classifies a Rift operating failure on the tool path as a completed tool result with isError set and the failure's compact text in content: one N errors section whose entries are the failure, then its causes. Protocol failures stay JSON-RPC errors.

Startup runs one cycle:

  1. Take the exclusive file lock on .rift/server.lock, which picks one starter among racing rift mcp processes, and hold it until the process exits.
  2. Parse rift.toml and accept the workspace; an invalid file keeps the server running but refuses every request until fixed.
  3. Start the native filesystem watcher.
  4. Publish an empty index carrying local_index_preparing, then start background discovery and preparation on the bounded worker pool.
  5. Bind the first free loopback port of the accepted [server] selection - the pinned port or the configured port_range - mint the bearer token, and write both into .rift/server.json.
  6. Serve until no request remains active and the idle_timeout span passes after the last response completes, then stop.

rift server start runs that cycle detached, with the child's standard error on .rift/server.stderr, and waits for .rift/server.json to appear; detached startup and rift mcp each allow 30 seconds for publication. A start whose child is still indexing when that window closes reports the server as starting rather than failing, and a start that finds the election held by a server still indexing spawns nothing and waits the same window for that server's file. rift server stop sends the authorized stop request and waits for that process to exit or release its election. A proxy can start a replacement during this wait; the replacement does not extend the original stop. rift server restart chains stop and start. rift server status reports the workspace's current state - a serving server with its port, pid, and version, a server still indexing, a stale .rift/server.json with the reason it went stale, or no server - and changes nothing. A probe counts a server as serving only when its election is held and its recorded port accepts a connect. --foreground keeps the cycle in the calling process, where Ctrl-C stops it.

Since v0.0.33

--auth skip serves every request that clears the loopback boundary without the bearer check. The MCP specification's own conformance runner forced it: the runner addresses a server by URL alone and sends no Authorization header, so a checked server refuses it at the door. The CLI accepts the flag only together with --foreground, so no detached server ever serves unchecked; the server still mints its token and writes it into .rift/server.json, so rift mcp reaches it unchanged; and the serving process records one WARN naming the run.

Since v0.0.47

Filesystem and parser work runs through one named Rayon pool. Admission bounds concurrent operations before a Tokio blocking task enters that pool, and cancellation is checked between files. This behavior is configurable via the num_workers and worker_queue_timeout keys in the [server] section.

Freeing a large index can take longer than closing the listener. After the index supervisor stops, the blocking pool releases its retained index while the foreground process finishes the log drain and releases the election.

Since v0.0.62

A foreground stop runs its stages under one deadline, SERVER_STOP_DEADLINE (4 s) from where the stop begins, and each stage takes only what the one before it left:

  1. The serving task drains, and the engines and index supervisor shut down, by the deadline minus 1.5 s, the bound step 2 shares, so a stage that runs out its bound still leaves steps 3 to 5 their reserves.
  2. The index database, and the vectors database when it opened, close by the deadline minus 1.5 s: SERVER_EXPORT_STOP_RESERVE, SERVER_LOG_FLUSH_RESERVE, and SERVER_DATABASE_STOP_RESERVE, 500 ms each. A close failure becomes a warn record with operation database.close, so a later stage writes it.
  3. The otlp export stage sends the final spans, log records, and metric points and shuts the export down by the deadline minus 1 s, and at most SERVER_EXPORT_STOP_RESERVE after it starts. A collector that is down or stalled costs the stop that reserve and never fails it.
  4. The log drain's final flush ends by the deadline minus SERVER_DATABASE_STOP_RESERVE. A drain that outlasts it is aborted, and its warning carries unwritten: the records the drain accepted and never wrote.
  5. The metrics database closes by the deadline, so it outlives every record.
  6. The election guard drops, right before the process exits.
  7. The tracing shutdown stage flushes what the export still holds, within OTLP_SHUTDOWN_TIMEOUT (500 ms), and a process exits record with stage = "process exit" and the exit status is the last line on stderr before the process exits.

On Windows, a process does not end while one of its threads has pending I/O: "The terminated process cannot exit until all pending I/O has been completed or canceled" (TerminateProcess). A checkpoint syncs the database file, and on a slow disk that flush took up to 3.3 s against the 4 s stop, so no stop closes a database with a checkpoint: a flush that started would hold the exit past the bound. The checkpoint runs at the next open instead, where no stop deadline runs.

A foreground repository server opens no metrics database of its own; its log drain routes each record to the consumer of the workspace the record names. Each workspace stops inside the serving stage repository workspaces shutdown, every workspace at once. Within a workspace, the engines, the index supervisor, then the index and vectors databases stop by the workspace deadline minus WORKSPACE_LOG_FLUSH_RESERVE and WORKSPACE_DATABASE_STOP_RESERVE; its consumer's final flush ends by the deadline minus WORKSPACE_DATABASE_STOP_RESERVE; its metrics database closes last. The server's serving stages end by the deadline minus SERVER_EXPORT_STOP_RESERVE and SERVER_LOG_FLUSH_RESERVE, its otlp export stage by the deadline minus SERVER_LOG_FLUSH_RESERVE, and its log drain stage by the deadline.

Each stage runs inside a server.stop span and ends with a stop stage ended record carrying stage, remaining, and outcome, plus error and causes for a failure. The outcome is ok for a stage that ended inside the deadline, error for a failure, and timeout for a stage that ended without a failure once its bound had passed. Only a failure makes the stop exit with status 1.

The otlp export stage fails nothing, even with the outcome error, such as a refused connection: it records the outcome at warn, because the export carries diagnostics only. A process that exports nothing records the stage ok at once.

The stage runs before the log drain stops, so a drain the stop abandons counts its unwritten records in log.queue.dropped after the last export: the drain's warn record carries that count, and the collector never receives it. The log record export stops with the stage too: the stop stage ended record of otlp export and every record of the later stages reach stderr and, before the drain stops, the metrics database, and the collector receives none of them.

A stop that arrives while the index preparation runs a dependency version probe reaches the probe too. The probe reads the stop once per 25 ms poll, then kills its child and reaps it, and its close records outcome=cancelled with error.type=cancelled. A cancelled preparation publishes nothing.

After the kill the probe waits at most 200 ms for the child's output pipes to close. On Windows the child runs inside a Job object of its own, so the kill ends every process the child started. On Unix a process the child started can keep the pipes open, and the probe stops waiting on it.

A database close runs no checkpoint and syncs no file; the write-ahead log keeps every committed transaction for the next open. A close whose thread outlasts its bound ends its stage with timeout: the thread keeps running until it finishes or the process exits. A metrics close ends with timeout whatever stage its bound passed in, a close still queued behind an earlier write included. An index or vectors close that started past its bound ends with timeout too when the worker it finds still running outlasts it; a transaction that worker left open is rolled back at the next open. An index supervisor still running at its bound is aborted the same way: its stage ends with timeout and the table of operations in flight, and blocking work it started ends with the process.

Since v0.0.62

A server records each step of its life at INFO, so the store keeps them under the default [logs] capture. A record that opens a step carries phase = "start", one that ends it carries outcome = "ok", and one that fails it carries outcome = "error"; a printed line marks them →, ✓, and ✗:

RecordOperationFields
workspace election claimedserver.startstate_directory
another rift server already serves this workspace; this process exitsserver.start
MCP server readytransport, port, outcome
index preparation startedindex.buildtotal, phase
index preparation progressedindex.buildprepared, total
index preparation supersededindex.buildepoch, observed_epoch, trigger
index snapshot publishedindex.publishtrigger, epoch
lexical commit handedsearch.committree_revision
lexical commit committingsearch.committree_revision, phase
lexical commit settledsearch.committree_revision, outcome
history fill startedhistory.fillpending, phase
history fill finishedhistory.fillwritten, outcome
vector population startedsearch.populatetree_revision, phase
vector population finishedsearch.populatetree_revision, outcome
Git index lock wait endedindex.lockpath, elapsed_ms, outcome
index watch stoppedindex.supervisor
stop stage endedserver.stopstage, remaining, outcome
workspace election releasedserver.stopstate_directory

A lexical commit the store refused is owed instead of settled, and its the lexical commit failed record carries the error. A history fill with no commit pending records neither its start nor its end. A vector population that fails ends with its warning, carrying outcome = "error", in place of vector population finished.

A wait for the Git index.lock records Git index lock wait ended only when it found the lock present, with the outcome ok, timeout, or cancelled. The election guard drops after the metrics database closes, so workspace election released reaches stderr and not the store. A database thread whose close ended timeout still holds a clone of the election guard, so the stop retires server.json itself before the process exits, records a database thread still holds the workspace election at warn, and the election is released by the process exit.

A watch failure records once per step: index watch backend reported failure under watch.receive at WARN, or index watch failed under watch.observe at ERROR, each with error and error.type, the registered identity. Repeats of the same identity record nothing until the supervisor's next turn, or the watch's stop, records one line carrying their count.

rift mcp and rift server start record their side of a start on their own stderr. Each detached server they spawn records detached server spawned with its pid and where stdin, stdout, and stderr go, and a start that sees the child exit records spawned server exited with its pid and, when the platform reports one, its exit_code. The proxy's start window ends with start window closed with a server that answers, or start window closed without a server that answers, both carrying rounds, elapsed_ms, and presence.

A stopped server leaves its process as soon as serving has ended and the log drain has stopped, through the process exit rather than by returning from main. Dropping the runtime would wait for every blocking task still running, and a lexical transaction or a dependency pass that started behind a publication can run for minutes with nothing able to cancel it.

The project resolver skips symlinks and the .git, .rift, and target directories.

Since v0.0.47

Discovery and event filtering also skip nested linked Git worktrees. Submodules and independent nested repositories retain the existing source visibility rules.

The symbol index stays in memory as one immutable value: a rebuild constructs the next index and accepted configuration outside the async lock, then swaps the complete value under its write lock. Each request clones the published value under a read lock and retains that capture through its result, so reads never combine state from two index revisions.

Since v0.0.25

Watcher events advance one workspace change counter and enter the same serialized rebuild lane. Each observation carries the paths it saw, and the rebuild reads only those files, sharing every other file with the previous index. A loss event, a written .gitignore, or a directory appearing or disappearing reads the whole workspace instead: the rebuild walks the tree and reads and digests every visible file under the ignore files as they stand now, and parses only the files whose bytes the previous index did not already parse. A written rift.toml reads and parses the whole workspace when [source], a language file selection, [search.text] inclusion or chunk bound, [documentation], [dependencies], or [providers.syntax] changed. Every other accepted configuration change reuses the published tree.

Before each settled current-tree tool read, the server validates the digest of every included workspace file, then waits until the published index counter matches its observation. That capture already walked the tree, so a read that finds the index behind names the files that moved rather than asking for a whole rebuild. A file that appeared or disappeared because a .gitignore changed is the exception: the published index still applies the ignore files it was built under, so rebuilding that one file changes nothing, and the read asks for the whole workspace without waiting for the watcher to report the .gitignore write.

The watcher can report that .gitignore write after the read's capture already found it. The report asks for the whole workspace, so it supersedes the rebuild the read asked for, and the read captures again: a capture that matches its previous one waits for the next rebuild, and only a capture that differs, a tree still moving, answers from the published index with stale_index.

A rebuild that is still running asks the same question publication asks, between files and at each phase boundary: whether a later observation asks for the whole workspace, or names a path whose bytes differ from what the rebuild read there. Such a rebuild stops, returns its paths to the next rebuild, and wakes the waiting reads.

A later observation of the bytes the rebuild already read, such as a read's own request, leaves it running, and its index publishes for that observation. The comparison reads at most 64 observed paths over one rebuild; a larger observation is decided at publication.

Since v0.0.47

The capture records each file's stat beside its digests: length and modification time, and on Unix status change time, device, inode, and mode. It reads a timestamp from a temporary file on the workspace filesystem before capturing bytes. An unchanged stat permits digest reuse only when both recorded change times precede the boundary that validated those bytes.

This follows Git's racily-clean check, with a status-change time check for rewrites that restore modification time. Reused entries retain their original boundary, so copying an entry into a later capture does not make a same-tick write safe to skip.

Stat reuse applies to APFS and HFS+ on macOS, and ext4 and tmpfs on Linux. Windows and other filesystem types read every visible file again. A failed boundary, changed root, different device, or backwards timestamp also selects full reads under the same request budget.

Every complete pass still walks the source policy to discover new and removed paths. The watcher's loss flag requests this recovery before path filtering, including a loss event with no path. The observer and publisher share the publication lane, where the candidate's epoch and source policy are checked before the complete value changes.

Background validation uses that same source discovery and stat comparison even without watcher events. Its deadline follows [server] validation_interval, and requests and filesystem events do not move it. An unchanged check without a recorded failure queues no rebuild; changed paths use the existing incremental comparison, while a changed configuration or ignore policy requires a complete pass.

A due check runs on the supervisor's next turn, before the rebuild that turn owes. The turn after a superseded rebuild is the exception: the owed rebuild runs first, because the reads waiting on it already waited for a rebuild that published nothing, and the check takes the following turn. A check that is one interval late runs before the rebuild, so a check starts at most one interval after its deadline, plus the rebuild running when that interval ends.

A recorded rebuild failure is retried even after the tree returns to the publication's exact bytes. An empty incremental recovery reuses the read service and clears the failure without deriving source again.

The watcher also observes the worktree's Git directory, including a linked worktree's directory outside its source root. HEAD and index.lock events request validation before the source floor; other .git paths retain that floor.

Before a queued rebuild, the supervisor waits for index.lock to disappear within [server] version_control_timeout. The first observed lock fixes the deadline, so repeated events cannot renew the wait. An expired lock is named in the log and passed until its absence is observed; shutdown cancels the wait.

Repository serving

Since v0.0.47

Repository serving requires an explicit foreground start:

rift server start --foreground --repository
rift server status --repository
rift server stop --repository

The repository process accepts server settings from the main worktree and records them at startup. A proxy adopts it only when the workspace and main worktree accept those same settings; otherwise the workspace starts its own process, as the [server] configuration describes.

The process holds its election under <common git directory>/.rift/<product identity digest>/. The digest is SHA-256 of the canonical product identity, and the directory contains server.lock and server.json. Workspace routing remains inside the proxy and HTTP transport.

Each admitted workspace holds an exclusive lease on its .rift/server.lock and keeps its own index, vectors, and metrics databases. A competing serving process is refused before opening that workspace's databases until the lease is released. rift server logs still reads the metrics database, whose reads take no lock. The repository process shares its bounded worker pool and content cache; workspace source, configuration, publications, and language processes retain their own owners.

Each database's thread retains the workspace lease until it exits, including after a shutdown wait expires. A workspace's stop closes the index and vectors databases, then its metrics database, within WORKSPACE_STOP_BOUND, and releases the lease after a clean close. When a repository process stops, it stops every workspace at once, each by the shared deadline, so one workspace's slow close spends none of another's bound; each workspace's records sit under a server.stop span carrying its root. History retains the existing common Git store and its fill and live locks.

Local index preparation

Since v0.0.47

The initial publication answers reads before discovery finishes. The supervisor discovers visible paths, publishes their map counts, and then captures and analyzes bounded batches. Each publication owns its map, source catalog, configuration, and preparation counts.

search and get_symbol use the declarations already prepared. Their local_index_preparing warning remains until their selected local scope is complete, including an exact symbol read that already found a declaration. A nodes read can prepare its supplied path within the request budget without changing published progress.

The rift://workspace resource projects only the requested page from published visible file digests. It performs no whole-tree capture per page; the completed catalog includes unclassified and disabled-language files as well as files whose syntax was refused.

The process-owned content cache holds weak references to captured bytes and path-independent syntax facts. Its key includes the full content digest, language provider, and parse limits; path-dependent declarations remain with their workspace placement. Dropping the last publication that owns content releases it, while live publications remain readable.

A read that meets a recorded rebuild failure at or past the published counter waits for nothing: it answers from the published index and carries stale_index, whose detail names the counter the failed rebuild answered for, the counter the tree is at, and the failure's causes. The next observation retries the rebuild.

Since v0.0.34

With debug capture enabled, index.build records its start and epoch before reading a candidate tree. A get_symbol history read records its start and retains a span until the history request ends. The corpus shutdown tests use these records to observe work in progress before issuing rift server stop; starting a client request alone does not prove that work has started.

Databases

Since v0.0.9 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, Python

Three SQLite files under .rift hold what a server persists for its workspace:

FileHoldsCrateThread
.rift/indexlexical, trigram, and documentation tablesrift-indexrift-db-index
.rift/vectorssemantic_vectorsrift-indexrift-db-vectors
.rift/metricslog_recordsrift-tracingrift-db-metrics

The index and vectors databases go through Toasty models. The full-text surface is raw SQL, because Toasty 0.10 has no typed virtual-table or MATCH API, and the raw boundary stays confined to the FTS table and its rows. The metrics database goes through rusqlite directly, as the history store does.

The server never reads, migrates, or removes a .rift/db file left in the state directory; delete it and its -wal, -shm, and .lock files by hand.

Since v0.0.62

An elected server opens the metrics database first, then the index database, so the index database's open lands as a record. The vectors database opens through LazyDatabase, a handle that opens the file at the first vector operation, at most once, under the same election owner; an open that fails leaves the handle unopened, and the next vector operation tries again.

Each open stands alone. A refused index database emits a database.open warning and the server starts without it, while logs keep recording and rift server logs prints that warning.

Since v0.0.47

Each of the index and vectors databases is one WorkspaceDatabase: one owned SQLite worker thread executes driver operations for its file. Async callers send operations through a bounded queue and await their replies, so SQLite lock waits and statement execution occupy that thread. Each queue uses the accepted [search] pool_slots and busy_timeout bounds.

Every process that opens .rift/index or .rift/vectors holds an exclusive lock on .rift/index.lock or .rift/vectors.lock while it switches the file to WAL and applies that database's migration set. SQLite refuses a second process's WAL switch at once, without waiting for the first, and Toasty applies every migration it read as missing, so two processes preparing one new file side by side leave one of them without the database. A process that meets the lock waits for it up to [search] busy_timeout.

Since v0.0.62

LogStore::open in rift-tracing spawns rift-db-metrics, which opens the one write connection to .rift/metrics and keeps it for its life. One thread owns the only write connection, so writers in the process never compete for the file, and the metrics database has no write turn and no lock file.

rift server logs opens a query_only connection of its own through LogReader, which never creates the file or a table and never switches the journal mode. The file carries its schema version in PRAGMA user_version: a writer that finds another version drops log_records and creates it again, and a reader refuses.

The metrics database's bounds are compiled: METRICS_BUSY_TIMEOUT_MS, 1 s, and a 4 MiB write-ahead log limit. A log read therefore never inherits a search wait, and rift server logs reads while rift.toml is invalid. The writer connection waits through a busy handler of its own that sleeps SQLite's own busy timeout steps, 1 s in all, and counts each call in sqlite.busy.retries.

Documentation extraction lives in rift-analysis. Workspace and package adapters pass selected bytes, exact source identities, origins, and existing chunk ranges into that shared crate. Its collector feature enables syntax parsers; readers can validate stored metadata and project ranked identities without linking parsers or storage code.

Documentation adapters record source warnings when selected input is malformed or exceeds the selected documentation.max_file or remaining documentation.max_total bytes. Their defaults are 4 MiB per source and 512 MiB per collection; other accepted sources remain available.

documentation.max_sources counts content owners, including notebook cells and omitted sources. The default is 100,000, and a selection above the configured bound is refused before publication.

The index database stores the documentation metadata beside the lexical rows, one row per source: the source's facts, blocks, links, references, and unresolved spellings, under the digest of those bytes. A manifest row carries what describes the whole collection, and each exact reference also gets a reverse row filed under its source. Blocks retain byte ranges and digests; excerpts come from the captured source after pagination. The transaction that stamps a lexical write's tree revision writes its documentation metadata.

The collector keeps parser facts for each selected source in memory. Incremental rebuilds reuse facts when source metadata, chunk mappings, extractor revision, and accepted attached symbols match. Link and declaration changes invalidate affected blocks; unchanged notebook bytes reuse decoded cells. A restart rebuilds parser facts from source.

A write compares each source's digest with the one its row carries and rewrites only the sources that moved, removing a source's row and its reverse references together, so an edit to one documented file writes that file's row rather than the whole collection.

The store records, beside each file's rows, the digest of the bytes they were derived from, and its stamp names what else derived them: the analyzer manifest, the corpus shape, and the index-owned configuration. Every write compares before it writes. A rebuild that named the files it read deletes those paths' rows and inserts their new ones. A rebuild that read the whole workspace, startup included, compares every file it holds with the recorded digests and writes only the files whose bytes moved or that one side lacks, so a restart over an unchanged tree writes the stamp and nothing else. A store stamped by another analyzer revision or configuration keeps nothing: the write clears it first. An unrelated binary change keeps compatible rows.

The write runs behind the publication: the rebuilt index becomes current first, and the lexical lane runs the writes it was handed in publication order, one at a time. The lane holds one write behind the running one, and a write handed while one waits merges into it: the union of the files both named, derived from the newer publication, leaves the store exactly as running both would. A burst of changes therefore costs one write per lane turn. The index database's write turn serializes index commits and trigram batches, and the vectors database's turn its vector batches; each transaction starts with BEGIN IMMEDIATE. Log batches run on the metrics writer thread.

A write commits in transactions of at most [search.lexical] transaction_units units and transaction_size content bytes, one file's rows always in one transaction. Every transaction but the last stamps no tree, so a reader meets no publication rather than a mix of two, and only the last names the tree the write answers for. A server that stops during a write aborts one transaction: the ones already committed recorded their files' digests with their rows, so the next start's comparison writes only the files the stopped write did not reach.

Between writes, the lane fills the trigram index behind regex search. Each batch indexes the oldest rows the index lacks, under the same transaction_units and transaction_size bounds, and a write handed meanwhile runs before the next batch. A search never waits for a batch: it reads the rows the index lacks beside the index.

Each database uses WAL with synchronous=NORMAL. Read-only connections use committed WAL snapshots. In the index and vectors databases, search.busy_timeout bounds a lock wait caused by another process, and a wait for a pooled connection when every one of the search.pool_slots is checked out.

Past that wait, a search ranks without the store: it answers from identifier matching and warns lexical_ranking_unavailable, with a detail naming the slot wait. A regex search verifies every file its paths select in place of the rows the trigram index would select, and warns nothing. Every other checkout refuses, a write among them, with index.database_failed, which names the database and its path, as a failed open does.

Every checkout sets its pragmas again. mmap_size comes from [search.lexical] mmap_size on the index database and is zero on the vectors database, and journal_size_limit comes from [search] journal_size_limit on both.

Each close is the operation database.close, component storage, naming its database: it records operation opened when it starts, sits in the table of operations in flight while it runs, and its close record carries elapsed_ms and the outcome.

A close runs no checkpoint. SQLite's last connection checkpoints when it closes, unless SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE is set on it, and the index worker's connections live inside the Toasty driver, which takes no configuration. So WorkspaceDatabase::shutdown first opens one connection of its own on the file, on the rift-db-close thread, with that flag set, and reads the log's frames with PRAGMA wal_checkpoint(NOOP), which syncs nothing. A WAL connection keeps a SHARED lock on the database file while it is open, so no worker connection gets the EXCLUSIVE lock its close checkpoint needs; the close's own connection closes last, without a checkpoint. The close then records database closed; the write-ahead log stays for the next open with the frames the log held as log, the frames an earlier checkpoint had moved as checkpointed, and elapsed_ms. A worker whose stop outlasts the deadline is a warn record with operation database.close and fails nothing. The metrics writer thread sets the same flag on its connection and closes it after reading the frames, and the stop records the same message, or, past the deadline, database close outlasted the shutdown deadline; the write-ahead log stays for the next open at warn, naming the close stage, queued or close, and its elapsed_ms.

Every open runs PRAGMA wal_checkpoint(TRUNCATE) after its migrations, where no stop deadline runs: SQLite reads the log the last process left ("the WAL file might be retained on disk after all connections to the database have been closed"), and the checkpoint moves it into the database and empties the file. The index and vectors open records database checkpointed its write-ahead log with operation database.open, journal_size_limit in bytes or -1 for none, and wal_autocheckpoint, the log pages at which a commit checkpoints. Between opens the log stays bounded by SQLite's automatic checkpoint, "when the WAL file reaches a threshold size of 1000 pages", and is cut back to journal_size_limit ([search] for the index and vectors databases, METRICS_JOURNAL_SIZE_LIMIT_BYTES, 4 MiB, for the metrics database) at the commit that restarts it. Under synchronous = NORMAL "no sync operations occur during most transactions", so a commit the close leaves in the log survives a process exit and can roll back after a power loss, as any commit since the last checkpoint can.

A commit that refuses leaves the previous rows and stamp intact and records its cause in the server log; the next write compares its whole publication with the recorded digests.

The index database's tables:

  • lexical_documents - one row per indexed document: a declaration, carrying its names, signature, and documentation, or an indexed file's text, the one copy of it the store keeps, split into chunks past the [search.text] chunk bound. The row carries an integer id, the document's identity, path, kind, and content digest beside the searchable fields, and a chunk row its byte_offset in the file, so a position in the chunk maps back to the file without chunking it again.
  • lexical_files - one row per file the published index holds, the files it left out included, carrying the SHA-256 of the bytes that file's rows were derived from. A change writes a path's rows and its digest in one transaction.
  • lexical_index_state - stamps the set with the tree revision that published it, the corpus revision it was built under, and a digest of what besides the files' bytes derived the rows, which a later write compares before it keeps any row. One read transaction reads that stamp and runs the query, so a commit landing between the two cannot slip another tree's rows into an answer. A store holding another tree returns no rows at all: either the lane has not committed the captured publication yet, and search warns naming that revision, or the publication has since been superseded, and the request recaptures rather than ranking rows it cannot place. A store holding another corpus revision returns no rows either: its columns, derivation, tokenizer, or weights are not this build's. A stamp that names no tree answers the same way as an empty store.
  • lexical_documents_fts - the FTS5 virtual table over the searchable columns. It is an external-content index over lexical_documents: it holds the token index alone and reads column values from the row whose id is its rowid, so no field is stored twice. FTS5 leaves the two in step to the writer, so a path's index entries are removed with the 'delete' command while its rows still hold the indexed values, and a document is indexed only after its row is written. Ranking is FTS5's bm25 with the per-column weights the architecture page lists, comparable only within one search. The same statement asks bm25 once per column with every other weight zeroed, which is how an answer states that a term landed in documentation rather than in a name. The tokenizer is unicode61 remove_diacritics 0: keeping diacritics is what lets Rift split the same text the same way outside SQLite without carrying a folding table of its own.
  • lexical_documents_vocabulary - an fts5vocab table over the word index's columns. Its per-column document counts are the document frequencies body matches score by.
  • lexical_documents_trigram - a second external-content FTS5 index over file_content alone, tokenized trigram with detail=none: it records which rows hold each run of three characters, case-folded, and no positions, which is all a regex prefilter reads. Declaration rows carry no text and stay out of it, and columnsize=0 keeps FTS5's integrity-check from counting them as missing. The lane's trigram batches fill it, and a write that removes a path deletes the entries of the path's rows the index holds alone: a 'delete' naming a row the index never took leaves it failing integrity-check.
  • lexical_trigram_pending - the id of every file row the trigram index doesn't hold yet. A write files its new file rows here in the transaction that stores them, a trigram batch takes them off as it indexes them, and a regex search reads it beside the trigram index, so a row stored since the last batch still reaches the answer. The vectors database holds semantic_vectors, one row per embedding space and declaration digest. The metrics database holds log_records, the bounded server diagnostics rift server logs reads.

The index and vectors databases each apply a Toasty migration set numbered from 1 and record it in the file's own __toasty_migrations table.

A rebuild that named its files deletes and reinserts exactly those paths, which is why lexical_documents carries an index on path: the index finds a path's rows and their index entries without reading the full-text table. Every named path is replaced, added ones included: two rebuilds captured from one publication both write what they read, and the second leaves what the first left.

Search is a layered configuration: the default is this full-text store beside identifier matching, and [search.vector] adds the vector store below. All three fuse by weighted reciprocal rank, under the shares [search.ranking] sets.

History store

Since v0.0.46

The history store is a SQLite database apart from the workspace's three, so lexical and log writes never wait on a fill. It sits in the .rift/ folder of the common git directory, one file per derivation revision: the analyzer manifest, corpus revision, index-owned tables, and the [providers.history] strategy and releases. Compatible builds reuse that file, and a server deletes a file no live server holds.

rift-history-store opens the file through rusqlite in WAL mode, where a writer never blocks a reader. One filler writes a store at a time: the history task holds an exclusive std file lock beside it, and every server holds a shared lock on the revision it reads. Each batch commits in one BEGIN IMMEDIATE transaction after its parses finish.

Commit messages sit in an FTS5 table with external content over the commit rows, under the corpus tokenizer the lexical store uses. Deleting a commit issues FTS5's 'delete' command while the row still holds the message the entry was indexed from.

The history task runs on the runtime's blocking threads, outside the [server] num_workers pool. Its spans time each wait for an idle server (history.idle_wait), each batch (history.batch, history.analyze, history.write), each plan (history.plan), and the rename and move pairing (history.renames, history.moves). A comparison's spans time the changed-path listing and each side's index (search.change_paths, search.change_base, search.change_head).

Server logs

The server records diagnostics in the metrics database at .rift/metrics. rift server logs prints them oldest first, with or without a running server:

rift server logs

The flags narrow what prints:

  • -f, --follow keeps printing records as the server writes them
  • -n, --tail prints only the newest records
  • --since prints records recorded at or after an age such as 10m, or an RFC 3339 timestamp such as 2026-10-04T20:42:58Z
  • --until Since v0.0.62 prints records recorded before a bound, in the same two forms; with --since it selects one window
  • --level selects one severity
  • --component selects records from one component

The [logs] table bounds how many the server keeps.

rift-tracing holds a record from its capture to its row. The LogSink layer publishes each event and span close that [logs] capture admits to the persistence queue and independent subscriptions. A full persistence queue drops the newest record and counts it in log.queue.dropped with error.type queue_full.

The log drain writes the queue in batches. A batch the metrics database refuses stays with the drain, which retries it with no attempt count; meanwhile the queue fills and then drops, and the next batch written carries a the log queue was full and dropped records record with the dropped count. The stop's deadline bounds the retry: a drain aborted there reports what it held as unwritten.

runtime.logs().subscribe() returns Result<LogSubscription, RiftError> and registers an independent subscription. Disabled or closed capture returns tracing.log_stream_unavailable; registration at 16 admitted subscriptions, including persistence and closed subscriptions with retained publications, returns tracing.log_subscription_limit.

Registration and publication use one order, so subscriptions receive records from registration onward, with no replay. recv() reads one LogPublication; recv_batch() reads a bounded batch.

Each publication carries its sequence and an immutable record shared across subscriptions; record() borrows that record. The sequence preserves dropped positions, so consumers can compare it with their loss count.

Each subscription shares an 8 MiB byte budget between its queue and publications its consumer retains; a full record or byte budget drops the incoming record and increments that subscription's dropped() count. Dropping a subscription unregisters it and releases its queue; already returned publications release their bytes when their owner drops them. A closed or dropped subscription keeps its admission while buffered or returned publications retain bytes. Its place becomes available after the last publication drops. The 16 admitted budgets retain at most 128 MiB of records; explicit copies made by a caller sit outside those budgets.

The byte accounting includes the publication envelope, record, shared ownership header, and string lengths. Bounded channel slots and allocator overhead sit outside that accounting.

The [logs] keys queue_records, flush_interval, and retry_interval configure queue capacity, partial-batch flushes, and retained retries. Under repository serving, a routing drain sends records to one bounded consumer per admitted workspace, selected from the request span's workspace. Each consumer holds at most min(queue_records, 256) queued records and the same number in a batch, sharing a 1 MiB byte budget. At most 64 consumer budgets remain retained, including replaced consumers and their queued records or writer batches. Admission at that bound refuses the new consumer before replacing the current route. These budgets sit outside the 128 MiB subscription budgets.

Since v0.0.62

An event record names the spans it was emitted in, so the records of one request group by its request_id even while a background task logs between them. When a span opens, an unfiltered layer ahead of stderr and the LogSink writes its name and fields as one JSON object into the span's extensions, and writes it again when the span records a field later. An event inside the span copies that object into its fields as root_span for the outermost span, and as nearest_span for the span it was emitted in when that span is not the outermost. A span that closes inside another carries the outermost one as root_span on its close record. The layer keeps every span, so a record names a span its own filter left out, and stderr and the store write the same record. A dependency's span with neither component nor operation, such as rmcp's serve_inner, takes no part: the Rift span inside it is the root.

rift mcp opens mcp.forward with request_id empty and records the forwarded request's id once rmcp sends it, so the proxy's lines join the server's request group; records before the send carry none. The span also records the caller's original JSON-RPC id as caller_request_id.

traced!, #[rift_tracing::timed], info_span!, debug_span!, and the event macros record code.function.name, the function that opened the span or emitted the event, read from the type_name of an item declared in its body; the format of that text is unspecified, so no filter or query selects by it. A tool call's mcp.request span carries mcp.method.name, gen_ai.tool.name, and workspace, the workspace root, beside request_id and tool. A close record carries busy_ns and idle_ns, the nanoseconds the span was entered and was not, elapsed_ms, the time from the span's opening to the end of its traced! operation, the reading the duration histogram records, and status.code: Ok, or Error with error.type panic for a span closed while its thread unwinds a panic and cancelled for an awaited traced! operation dropped after its first poll. The worker admission started record of a search request carries:

{
  "code.function.name": "rift_mcp::server::BlockingExecutor::admitted",
  "work": "workspace fingerprint",
  "queue_timeout_ms": "30000",
  "root_span": {
    "name": "mcp.request",
    "fields": {"component": "mcp", "operation": "tools/call", "request_id": "2", "tool": "search",
               "mcp.method.name": "tools/call", "gen_ai.tool.name": "search", "workspace": "/work/project",
               "code.function.name": "<rift_mcp::server::RiftMcp as rmcp::handler::server::ServerHandler>::call_tool"}
  },
  "nearest_span": {
    "name": "worker.queue",
    "fields": {"component": "worker", "operation": "worker.queue", "work": "workspace fingerprint"}
  }
}

A span keeps at most 1 KiB of field members: a member that does not fit is left out whole and counted in fields_left_out, so the object stays valid JSON and both objects fit the record's 8 KiB fields bound. A record's own members keep the order they were recorded in, written through serde_json maps; the 8 KiB fields bound, LOG_FIELDS_BYTES_MAX, keeps each own member whole or leaves it out, and the count of members left out follows as fields_left_out. A task spawned without .instrument runs outside the span that spawned it, so its events carry neither member. A field the code itself names root_span, nearest_span, or fields_left_out is not recorded.

The CLI starts tracing through TracingRuntime::builder in rift-tracing, which installs one subscriber with a filter per layer:

  • stderr under RUST_LOG, or rift=info,rift_mcp=info,rift_server=info,rift_index=warn when that variable is unset or does not parse, with only the stall reports of rift_tracing::flight
  • the LogSink under [logs] capture, installed for a foreground server alone
  • the OTLP span export under RIFT_OTLP_FILTER, installed when OTEL_EXPORTER_OTLP_ENDPOINT is set
  • the OTLP log record export under [logs] capture, or under the default targets in a process without a capture, installed when OTEL_EXPORTER_OTLP_LOGS_ENDPOINT or OTEL_EXPORTER_OTLP_ENDPOINT is set

Neither OTLP export layer hands on a span or event of the crates the export runs through: the OpenTelemetry API, SDK, and OTLP exporter, and the HTTP client stack (EXPORT_OWN_TARGETS in crates/rift-tracing/src/otlp.rs). A filter that admits them, such as a bare debug, would otherwise export each batch's own records in the next one. Stderr keeps them under its own filter.

A foreground server whose stderr is not a terminal writes at most [logs] stderr_limit there, 1 MiB by default. The write that crosses the bound is followed by one notice, standard error reached its byte bound, and every later write is discarded and counted in log.stderr.discarded; the records the LogSink captured are unaffected. The log drain's stop records standard error bytes discarded with stderr_limit and the bytes discarded, at warn when the count is not zero. A second TracingRuntimeBuilder::install in one process returns InstallError and leaves the installation in place untouched.

Stderr, rift server logs, and a failure window print a record through one renderer, LogLines in rift-tracing. A message or a field value holding · or : prints as a JSON string. A line holds the time in UTC with milliseconds, the level, the function, three spaces, the context, two spaces, the nested operation, and the message:

  • The function is the code.function.name of the outermost span around the record; outside every span, the record's own. A record without one prints its target.
  • The context is the outermost span's fields: component, operation, then the rest sorted by key, request_id printed as req. A record outside every span prints its own component, operation, and fields there.
  • The nested operation is ↳ with the name and fields of the span the record belongs to, when that span is not the outermost.
  • A span close prints close, ✓ or ✗, the reason it did not complete (error.type=…, panicked, or cancelled), then busy and idle with three significant digits. ✗ follows a status.code of Error, any error.type, or an outcome other than ok or acquired that the span recorded itself, as a failed index.build records error. The close record states the same status.code, and so does traces.span.metrics.calls for a traced! operation, with the recorded error.type when it is panic, cancelled, timeout, refused, or a registered error identity, and _OTHER otherwise. A lifecycle record prints its mark before its message: → for phase=start, ✓ for outcome=ok, ✗ for any other outcome. Any other record prints its message, then its own fields sorted by key.
2026-10-05 11:57:37.429Z DEBUG rift_mcp::validation::run_index_supervisor_with   component=index epoch=1 trigger=filesystem  ↳ worker.run component=worker operation=worker.run work=filesystem index rebuild → index capture started component=index operation=index.build epoch=1 phase=start
2026-10-05 11:57:37.444Z INFO  rift_mcp::validation::run_index_supervisor_with   component=index epoch=1 trigger=filesystem  ↳ index.build component=index changed_count=1 files_count=3 outcome=ok tree_revision=8f107d85 close ✓ busy=7.95ms idle=19.3µs

A group is a run of records of one request; without a request, of one root span; without a span, of one function. A blank line separates two groups. rift server logs pads each column to the widest value of its group; stderr and rift server logs --follow, which cannot know a group ahead, pad to fixed widths. Every surface prints UTC; stderr colors the level on a terminal alone. Every character a terminal or a line reader acts on prints escaped as Rust's char::escape_debug writes it (\u{1b}, \n, \u{202e}): C0 and C1 controls, DEL, the line and paragraph separators, and the bidirectional overrides and isolates, so a value read from a workspace or a request cannot move the cursor, end its line, or reorder what a reader sees.

Every workspace crate other than rift-tracing emits through it: spans through traced!, info_span!, and Span, events through its trace! to error!, elapsed time a caller acts on through measure_elapsed!, whole functions through the #[rift_tracing::timed] attribute, and test capture through ScopedRecorder. The architecture check refuses a tracing, tracing-subscriber, log, OpenTelemetry, or sysinfo dependency or import in any other crate, rift-tracing-macros included, and a normal dependency that enables rift-tracing's fixtures feature, which compiles the test recorder into a release build.

#[rift_tracing::timed("package.analyze", component = "dependency")] rewrites a function's body into one traced! call: the block form for a function, the async form for an async fn. It records no argument unless a field names it. The attribute lives in rift-tracing-macros, which rift-tracing re-exports; it finds the consuming crate's name for rift-tracing through proc-macro-crate, so a renamed dependency resolves too.

Every instrument is a static declaration over an OpenTelemetry instrument: a Counter, Gauge, Histogram, or ObservableUpDownCounter with the InstrumentScope of the declaring crate, its name, unit, and label keys, whose label values are &'static str literals. Its points carry that crate's CARGO_PKG_NAME and CARGO_PKG_VERSION as their instrumentation scope, from one meter per scope, at most SCOPES_MAX, 7; a scope past it records nothing, and one WARN record, meter refused an instrumentation scope past its bound, names it. The traces.span.metrics.* points of a traced! operation take the scope of the crate whose traced! opened it. Synchronous recordings go directly to the OpenTelemetry SDK, which aggregates and exports them. Observable reads sum identical labels in a bounded map for one collection, then publish to the SDK and release that map. A process that installed no meter records nothing, at the cost of two atomic reads per recording.

A view on the meter provider bounds every instrument at [logs] cardinality_limit label sets, and the SDK records a label set past that into one series carrying otel.metric.overflow. A test reads what the SDK exported through ScopedRecorder::metrics, each series naming its scope, printed as otel.scope.name and otel.scope.version. Every traced! operation records traces.span.metrics.calls and traces.span.metrics.duration, in seconds, labeled with its span.name, its span.kind, Internal, and its outcome as status.code, with error.type panic or cancelled when it did not finish. Work whose value is Err(RiftError), the value of a block or the output of a future, records that error's registered identity as error.type. An inline ? converts the error to the enclosing function's error type before recording its registered identity; an inline return records the returned error. Nested functions, closures, and async blocks retain their own return behavior. Only ? and return written directly in the inline work are inspected; exits produced by another macro expansion require explicit error.type recording.

The metrics follow the operation, not its span: the duration is read once, at the operation's end, and the close record's elapsed_ms states the same reading, so a span clone held elsewhere lengthens neither, and the span's filters do not select it.

Since v0.0.62

Every request rift server answers records mcp.server.operation.duration, and every request rift mcp forwards to it records mcp.client.operation.duration, in seconds, bucketed at the boundaries the MCP semantic conventions advise, from 10 ms to 300 s. A series names mcp.method.name: tools/call, tools/list, resources/list, resources/templates/list, or resources/read, and on the server also initialize and ping. A request the server answers also names mcp.protocol.version, the version rmcp knows or _OTHER, and network.transport tcp. A tool call also names gen_ai.tool.name, when the tool is get_symbol, nodes, or search. A tool answered with isError records error.type tool_error; a JSON-RPC error records its code as both error.type and rpc.response.status_code, such as -32602, or error.type _OTHER for a code outside the JSON-RPC, MCP, and Rift codes. rift server serves MCP statelessly through NeverSessionManager and records no session duration.

Each .rift/index and .rift/vectors operation records db.client.operation.duration with db.system.name sqlite, db.namespace, and db.operation.name; a failed one adds error.type _OTHER and the SQLite primary result code as db.response.status_code, such as 5 for busy. A raw statement that starts with PRAGMA busy_timeout, PRAGMA wal_checkpoint(NOOP), or PRAGMA wal_checkpoint(TRUNCATE) records that text as its db.operation.name, the open's checkpoint among them, and each connection the worker closes at its stop records close. A transaction records sqlite.transaction.duration by sqlite.transaction.result, commit or rollback, and sqlite.transaction.statement.count. Each append to .rift/metrics records sqlite.queue.wait.duration, sqlite.write_lock.wait.duration, sqlite.transaction.duration, sqlite.commit.duration, and sqlite.transaction.statement.count under db.namespace metrics, and its inserts and its trim as insert and trim in db.client.operation.duration; each read connection records connect and each page or count it reads query. The history store records connect, insert, trim, query, and sqlite.transaction.statement.count under db.namespace history. The metrics database's open records PRAGMA wal_checkpoint(TRUNCATE) and its close PRAGMA wal_checkpoint(NOOP) and close in db.client.operation.duration.

The index supervisor counts index.rebuilds by index.rebuild.trigger: startup for the preparation at start, filesystem for a rebuild that reads the paths its observation names, rescan for one that reads every visible file again. A failed rebuild adds error.type. Each native event counts watch.events by watch.event.kind and watch.event.route. An observation whose signal the full invalidation channel refused counts index.invalidation.dropped under event and writes no record.

Since v0.0.62

Every build carries the OTLP export, and a process exports nothing until an operator sets an endpoint variable. With OTEL_EXPORTER_OTLP_METRICS_ENDPOINT or OTEL_EXPORTER_OTLP_ENDPOINT set, the process installs a meter whose reader exports over OTLP/HTTP protobuf every OTEL_METRIC_EXPORT_INTERVAL, 60 s by default; spans export when OTEL_EXPORTER_OTLP_ENDPOINT is set; log records export to /v1/logs when it or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT is set, batched every OTEL_BLRP_SCHEDULE_DELAY, 1 s by default. With none of them, the process builds no exporter, so it never dials OTLP's default http://localhost:4318. OTEL_SDK_DISABLED=true, in any case, turns every export off whatever endpoint is set, as the OpenTelemetry specification defines it. The rift-dev test runner sets it in each test process, so a test that installs tracing in its own process exports nothing, and the test harness removes it from every rift process it spawns.

An exported log record is the record the LogSink stores, for each event and each span close: its level as the severity, its target as the instrumentation scope name, its message as the body, and component, operation, and each member of its fields as an attribute, root_span and nearest_span as maps. A record written inside a span carries that span's trace and span identifiers, and a span close carries the closing span's own, when the span export admitted the span.

One export takes at most OTEL_METRIC_EXPORT_TIMEOUT for metrics, OTEL_BSP_EXPORT_TIMEOUT for spans, and OTEL_BLRP_EXPORT_TIMEOUT for log records, 2 s each when unset, and the SDK reports a failed export on stderr. A refused or slow collector never grows the span or log record queue past OTEL_BSP_MAX_QUEUE_SIZE or OTEL_BLRP_MAX_QUEUE_SIZE: a span or record the full queue refuses is dropped. The SDK reports the first drop on stderr, and at shutdown the total, as dropped_spans of BatchSpanProcessor.Shutdown and dropped_logs_count of BatchLogProcessor.LogsDropped; no metric carries it. Every exported span, metric point, and log record carries the resource attributes service.name (rift), service.instance.id, a random UUID per process, process.pid, and service.version, the server's version, so a collector that receives from several servers tells them apart.

Readings are observable instruments: the SDK runs their reads on its reader's task each time it collects, so no task of Rift's ticks for them, and a process that installed no meter reads nothing. The process readings go through sysinfo:

  • process.memory.usage and process.memory.virtual, in bytes
  • process.cpu.time, the CPU time consumed, in seconds, with no cpu.mode, because sysinfo reports one total
  • process.disk.io by disk.io.direction, read or write
  • the open file count: process.unix.file_descriptor.count, or process.windows.handle.count on Windows

process.cpu.time and process.disk.io stay cumulative, and a collector derives rates from them. A value the platform does not report stays absent and is never recorded as zero.

Retained cache entries report cache.entry.count by cache.name: WorkspaceContentCache, symbol_documents, and prepared_resolution, plus the global client's CachedCapabilities, CachedResolution, and CachedFailure. Each observation follows its owner, so clones share one reading and the last owner removes it. A global client slot retains at most one entry.

Independent owners with identical labels contribute to one sum per collection. Expiry stops reuse but leaves the retained entry counted until replacement, clearing, or drop.

The Tokio readings load the counters of Tokio's stable RuntimeMetrics for the runtime that installed the subscriber: tokio.runtime.worker.count, tokio.runtime.task.count, tokio.runtime.global_queue.length, and the cumulative tokio.runtime.worker.busy.time and tokio.runtime.worker.parks, summed over the workers. A busy time that grows by the whole export interval while the global queue grows names a blocked runtime worker.

The owners of a quantity read it the same way. Each open .rift/index, .rift/vectors, and .rift/metrics database reports sqlite.file.size for the database file and its write-ahead log, from file metadata, as the history store's filler does under db.namespace history while it holds the fill lock, sqlite.queue.length, the commands waiting for its worker, and sqlite.page.count by sqlite.page.state, used or free, from the page and freelist counts in the file header, which change at a checkpoint. The index and vectors databases also report sqlite.transaction.active, the transactions begun on the worker's connections and not ended, and count sqlite.connection.refusals, the connection requests refused at the connection bound, and sqlite.connection.reaped, the connections removed after their driver connection dropped. The blocking pool reports worker_pool.permit.count by worker_pool.permit.state, available or used. One instrument runs at most OBSERVATIONS_MAX live reads, three databases for each of the 64 workspaces a repository process admits; a read past that is refused, and the first refusal per instrument is a WARN record naming it.

Since v0.0.62

Work that has not finished is evidence too. The runtime keeps a table of operations in flight: every span carrying an operation field, every lock wait, and every held lock, from its opening to its close, with its name, kind (operation, wait, or held), component, lock name, parent operation, and start. A field recorded on the span after it opened, such as the pid of a child the operation started, joins the entry's fields with its latest value. The table holds at most 1,024 entries and counts each one refused past that bound as untracked.

rift_tracing::publish_in_flight(reason) writes the table as one operations in flight record, and the panic hook writes it at WARN with the reason panic, right after its the server panicked record. The stall report, a task of its own, reads the table every quarter of [logs] stall_delay, and at least every 5 s, and writes one operations in flight past the stall delay record, at WARN, for the entries open longer than stall_delay, each entry once. The table also feeds operation.active, the entries open at each collection by span.name, and operation.untracked, the entries it refused.

A hold its holder keeps for as long as it runs, such as a server's shared history.live lock and its history.fill lock, is declared with .lifelong(): the stall record leaves it out, and every other publication lists it with "lifelong": true. Both records list the open entries oldest first, with age_ms, as the JSON array in operations, keep it within 4 KiB, and count what did not fit in left_out:

[
  {"operation": "lexical.commit", "kind": "operation", "age_ms": 12043, "component": "lexical"},
  {"operation": "lock.wait", "kind": "wait", "age_ms": 11980, "lock.name": "index.write",
   "parent": "lexical.write_turn"},
  {"operation": "lock.held", "kind": "held", "age_ms": 30211, "lock.name": "index.write",
   "parent": "documentation.store"}
]

traced!(component = .., operation = .., open = true, ..) also records its opening: one operation opened event inside the operation's span, so the store holds the operation while it runs and a killed server leaves it behind. lexical.commit opens this way.

Since v0.0.62

rift_tracing::lock(name).acquire(future) wraps the caller's own acquisition future, such as mutex.lock(), and polls it as the caller would, so the lock's queue order and cancellation stay its own. A wait that does not acquire at its first poll opens a lock.wait span naming lock.name, lock.mode, the waiter operation, and the holder operation, and closes it with outcome acquired, refused, timeout, or cancelled. An acquisition that can fail, such as the migration lock's attempts within the busy-wait budget, goes through acquire_fallible, whose future answers the guard or a Refusal: Refusal::Refused ends the wait refused, Refusal::Timeout ends it timeout, and neither opens a hold. The returned guard dereferences to the wrapped guard and keeps a lock.held span open until it drops. Both feed lock.wait.duration and lock.held.duration. The write turn of .rift/index and .rift/vectors is recorded as index.write and vectors.write, so a lexical commit that waits for a documentation write shows as a lock.wait entry under lexical.commit while the holder's lock.held entry names the operation running its statements.

The published index is recorded as the lock index.snapshot: a publication holds it exclusive, and a read holds it shared. Tokio's RwLock lets a queued write go first, so a read that waits behind a held or queued publication shows as a lock.wait entry naming that publication's operation.

The claim of .rift/server.lock is recorded as the lock server.election, exclusive and lifelong: a refused claim ends its wait refused and records the holder's pid when server.json names one. The hold closes when the last clone of the election guard drops; a process that exits with a database thread still holding a clone ends it unrecorded. A probe that finds the election held, or whose attempt the platform fails, writes no record and counts in lock.wait.duration under lock.name server.election, lock.mode shared, and error.type refused. A probe that finds the election free holds it shared until it releases the lock.

Shared ranking

rift-ranking states the retrieval decisions every index makes: the document shape and its searchable fields, the corpus revision that says what those fields mean, bounded query construction with its precise and broad renderers, identifier splitting and match classing, and weighted reciprocal rank fusion over ordered identities. It also carries the corpus tokenizer's split, so text read outside SQLite splits into the terms the store indexes.

Nothing in it opens a database, loads a model, or resolves a project path. The architecture check refuses SQLite, candle, tokenizers, reqwest, rig-core, and Toasty in that crate's dependency closure. A PostgreSQL or REST adapter therefore consumes the same decisions and the same ordered identities rather than copying the algorithms; resolving an identity to a declaration needs the store that published it, and deciding which identities win does not.

Error registry

crates/rift-error/errors.toml defines concrete Rift failures and evidence fields. rift-error-codegen validates those definitions and writes the errors module of rift-error: crates/rift-error/src/errors.rs holds the registry constants and one pub mod line per namespace, and crates/rift-error/src/errors/<namespace>.rs holds one __rift_error_definition! declaration per error in that namespace, carrying only its name, slug, message, action, and fields. A namespace is the slug segment after rift., such as analysis or cli. Builder types, bounds, conversions, typestate, and setter names are derived by the macros, not written into the file. The committed_registry_files_stay_within_generated_line_bound test in rift-error-codegen holds each generated file under a fixed line bound.

__rift_error_definition!(
    context7_malformed,
    slug = "rift.analysis.context7_malformed",
    message = "context7.json has an invalid JSON shape",
    action = "correct context7.json shape and retry",
    fields = {
        file: required(path),
        key: optional(string),
        source: optional(source),
    },
);

Each field reads name: presence(kind, flags...). Presence is required or optional. The kind is the field's type in errors.toml (string, bool, integer, unsigned, pid, port, path, duration), or its role (source, cause). The flags are hidden for display = false and sensitive for sensitive = true.

The macros in crates/rift-error/src/definition expand each declaration into the builder. The kind table maps each kind to the Rust bound a setter accepts and the ErrorValue conversion it applies. Each required field adds one builder type parameter that starts at Unset and becomes Set when its setter runs; .error(), .fail(), and the IntoRiftError implementation exist only once every parameter is Set. Each optional field adds .maybe_<field>(), which takes Option<T> and leaves the builder type unchanged. The maybe_<field> and <Field>State names come from rift-error-macros, which depends on proc_macro alone. A crate that reaches rift-error by path, such as the generated-code compile test in rift-error-codegen, therefore builds it with --offline and no downloaded crates.

A source field retains an underlying Rust error and exposes it through std::error::Error::source. A cause field retains another RiftError in that chain. The runtime also converts owned and borrowed RiftError values, including shared Arc values, without dropping their source chains. Sensitive evidence is redacted in rendered output.

The registry does not contain transport codes. The MCP adapter maps each slug to its wire code and retry directive, then supplies the request phase with .tool_error(phase). Calling .fail() on that representation returns its JSON-RPC error. cargo xtask errors generate updates generated Rust. The generator writes the declarations as text in the layout above, which rustfmt leaves unchanged, and parses the result before returning it. cargo xtask errors generate also removes any file under crates/rift-error/src/errors/ that the registry no longer generates. cargo xtask errors check compares generated output without writing it and fails on a differing, missing, or unexpected file; CI runs that check through just generate-check.

Since v0.0.16 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, Python

rift-search owns the tier. The default runs the encoder in the server's own process, so answering a query touches neither the network nor a subprocess.

Every model reaches the tier through one contract: rig-core's EmbeddingModel, pinned at =0.42.0. One enum holds the concrete pairs the server can run, and each method matches once and hands the concrete model to a generic helper, so no boxed model callback and no untyped provider map exists. Adding a family changes the loader and its adapter tests; search, fusion, SQLite, and MCP keep calling the same method.

A local model is two handles over one loaded encoder: documents embed through one and queries through the other. A BERT retrieval checkpoint prefixes the query with its own instruction and leaves documents alone, while a static model treats the two alike, so the difference lives in the handles rather than in a flag every caller has to remember. Acquiring the files, reading them, and detecting the family all finish before a handle exists, because the contract's own constructor cannot fail.

The openai_compatible arm builds on the same contract. Rig's client carries the credential the api_key_env variable named, the configured base URL, and an HTTP client holding the configured request timeout; Rift issues the /embeddings request and decodes the response itself. Rig 0.42.0 parses each response index and then zips the response array with the input array by position, so a service answering out of order would attach every vector to the wrong declaration. Rift's contract makes array order meaningless: the declared index decides, and a duplicate index, a missing one, or one past the request's own length is refused rather than mis-associated. The mock-endpoint suite serves reversed and malformed responses to prove it, and makes no external request.

Batching is Rift's, not the library's. Rig's own batch builder fixes its chunk size to the model's MAX_DOCUMENTS and derives its own concurrency, so it cannot honor both batch_inputs and max_in_flight. The scheduler here cuts each batch to the lower of batch_inputs and what the model accepts, runs one local batch at a time, allows at most max_in_flight remote requests, and returns every completed batch to the input slots it came from.

The model's own config.json decides which of two encoders reads it. A model_type of model2vec is a static model: the weights are one table of vectors, and the encoder embeds a declaration by gathering the row each of its tokens addresses and averaging them. Anything else is read as a BERT checkpoint through candle, pooled from the CLS position, and a retrieval query carries the prefix that checkpoint was trained with. A static model runs no forward pass, so neither the prefix nor the pooling applies to it.

The shipped default is static. A workspace is embedded once before its first search can rank by vector, and a forward pass over every declaration costs minutes on a laptop CPU where a row gather costs seconds.

Weights are read through candle's safetensors loader rather than its memory-mapped constructor: mapping is an unsafe call, and this workspace forbids authored unsafe. The cost is that a load holds the file's bytes once while the tensors are built.

Every encoder call runs on a blocking thread. Candle's forward pass would otherwise hold a runtime worker for the length of a batch, and a request arriving meanwhile would wait behind it. Passes run one at a time, bounded by batch_inputs and max_tokens. A BERT checkpoint's attention memory grows with the square of the token window, and candle's kernels already spread one pass across the thread pool, so running several at once buys nothing and costs the machine its memory.

The vectors go in semantic_vectors, in the vectors database at .rift/vectors. A row is addressed by the embedding space that produced it and the digest of the text it came from, so a declaration that moves or is renamed resolves to the row already stored and a refresh embeds only what changed. The space folds in the source kind, the endpoint origin for a remote service, the model and its resolved revision, the width, and the document and query transformations, and nothing else: a credential never reaches it. Changing any of those mints another identity, and the rows the previous one wrote are dropped rather than scored against.

Rig returns double-precision coordinates. The stored format stays single precision, which is what the local index already holds and what a later global store would carry, so the conversion happens once at the boundary and a coordinate the narrower format cannot hold is refused there.

A pass publishes its vectors, the map that places each of them on a unit, and the tree revision they were described for as one value. A query reads all three or none: a corpus described for a tree other than the one the request captured ranks nothing, so a workspace published moments ago is answered by the full-text tier alone until the pass for that tree lands. Values are little-endian single precision with no header, so a row is a slice; a row whose stored width differs from the loaded model's is skipped, which invalidates a checkpoint that changed shape rather than reading it as noise.

Two sweeps keep the table bounded: one drops what the previous model wrote when model changes, the other drops digests no live declaration addresses. max_vectors bounds what the table may hold at all.

The vector table joins the lexical index's migration set rather than opening a second database handle with its own. Toasty records applied migrations in one __toasty_migrations table per file, so two independent sets on one file would interleave their ids; one set covers both tiers, and each store applies the same idempotent set whatever order the two open in.

Language providers

Since v0.0.23

A provider composition is a typed pipeline of named stages - source resolvers, fact providers, and transforms - connected through declared input and output flows, with keyed joins for revision-keyed history. The Rust builder inserts, replaces, branches, merges, and nests stages. Rift validates types, capabilities, and cycles at build, then owns scheduling, caching, and invalidation. Composition controls which providers run and how data reaches them; it cannot establish Contribution equivalence.

Each provider publishes one immutable, revisioned Contribution collection. Normalization selects Contributions applicable to a captured index revision, validates identity evidence, and builds workspace-local SymbolRecord values. Reads assemble a Symbol from one record and its Contributions. Provider order selects presentation fields only after normalization establishes identity.

Syntax providers wrap tree-sitter grammars behind a Rift-owned adapter: grammar node kinds resolve to numeric ids once, and raw query and cursor APIs never leave the syntax crate.

The registry itself assembles from one shipped definition list in rift-syntax: each entry names a language's identity, the file extensions it claims, and the provider parsing it, so adding a language is one definition plus one list entry.

Since v0.0.66 · HTML, CSS, Vue, Svelte, Angular, C, C++, Cython, JSONC

HTML preserves elements, attributes, and source references as syntax nodes. CSS publishes selectors, properties, custom properties, and at-rules as declarations; import and URL syntax retain their authored ranges. Vue and Svelte keep component structure and template expressions, including Vue directives and Svelte control-flow blocks.

HTML, Vue, Svelte, and Angular providers parse supported script and style blocks through Tree-sitter included ranges. A TypeScript script retains its TypeScript signature identity, while the containing document keeps its HTML or component identity. Included parses share the original source bytes and one aggregate node and depth bound, so a script symbol's range still addresses its containing file.

Angular template syntax uses the html:angular identity. Component ownership comes from a class decorator whose Component binding is imported from @angular/core, including named aliases and namespace imports. Literal template properties retain their original TypeScript byte ranges. A parameter or local variable or function binding with the imported name refuses ownership inside its scope; type annotations and default-value references do not create bindings. Literal templateUrl properties retain their authored paths. Dynamic and escaped template values remain unresolved expressions. The TypeScript class uses the existing provider.

An Angular template publishes structural nodes for bindings, interpolation, pipes, and control flow. A template with no script or style declarations publishes no symbols. Script and style blocks publish declarations through their included providers.

Each provider exposes parser errors through has_errors and node error flags, including missing nodes and grammar recovery. Authored source references retain their syntax ranges. Compiler and framework semantics remain outside these providers.

Framework context follows the nearest captured package.json. A runtime Component import owns literal inline templates and external templates inside that component's package; an unrelated HTML file keeps ordinary HTML syntax. Dynamic or escaped templates, missing templates, and template sources outside the component package produce framework_context_unresolved warnings.

Tailwind context requires an exact supported package version or an npm lockfile version, plus applicable configuration or stylesheet evidence. A captured npm lockfile pin takes precedence over the declared version; an invalid or unsupported pin leaves context unresolved. Version 3 uses tailwind.config.* or captured @tailwind syntax; version 4 uses a captured CSS import of tailwindcss. Explicit providers.syntax.frameworks entries supply context when captured evidence leaves it unresolved.

Under that context, static HTML, JSX, Angular, Vue, and Svelte classes publish authored utility and variant names. CSS publishes @apply names, version 3 custom utilities inside @layer utilities, and version 4 @utility, @custom-variant, and @theme names. Configuration, stylesheet, and theme references retain their authored ranges. Dynamic class expressions keep their original syntax and produce warnings; Rift does not choose a conditional branch or construct names from an expression. These facts do not establish whether a compiler generates a utility.

Context applies after raw parsing. Raw cached facts keep their parser identity and source digest; changed package, component, configuration, or stylesheet evidence rebuilds framework facts over the current bytes. SyntaxFacts records SyntaxOrigin::Provider for raw facts and SyntaxOrigin::Framework for contextual facts. SyntaxFactsParts retains that origin through checked restoration; package raw reuse accepts only provider origin, including raw embedded HTML, Vue, and Svelte facts. Included template parsing and added declarations share the source, node, and depth bounds. A contextual HTML template uses html:angular; TypeScript, Vue, Svelte, CSS, and JSX files keep their existing identities.

C and C++ publish functions, types, variables, macros, and include syntax. Function prototypes publish function; function pointer and array variables publish variable. The provider follows the declarator modifier nearest the declared name, so a function returning a pointer remains a function. Several names in one declaration retain separate symbols and name ranges; each source range covers the complete declaration. C++ also publishes namespaces and method prototypes. The native walk records each declaration's additional declarator IDs once per parse, bounded by the captured nodes; repeated declaration hooks reuse those IDs. Cython publishes Python and Cython declarations under the cython identity for .pyx, .pxd, and .pxi files. JSONC has its own jsonc identity and accepts comments; the existing JSON provider retains its behavior.

Stored facts validate each node kind against the host grammar or its supported included grammars. Restoration retains the host identity, original source digest, embedded signature identity, and configured parsing bounds.

Since v0.0.46 · JavaScript, TypeScript

The JavaScript and TypeScript providers publish a declaration's attached JSDoc as its documentation. The walk starts at the statement carrying the declaration - an export, or the const, let, or var statement it is the first declarator of - and the declaration's range starts there too, so the JSDoc sits inside the range get_symbol answers. Several blocks join with a line break, @ tags stay as written, and the /**, */, and leading * markers are stripped; a // or /* */ comment or a blank line ends the walk.

Dependency context

Since v0.0.46

rift-dependency holds the resolvers and the standard library pass. The resolvers read through a reader that offers a file read and a directory listing, so a resolver cannot start a process.

Besides entries, the context records where each pinned package is installed on this machine: a Cargo package's <name>-<version> folder minted from the lockfile, an npm package's node_modules/<name> folder, nested copies included, the import folders a Python distribution's .dist-info/RECORD lists, and the Rust standard library below rustc --print sysroot. Those folders never reach the global API.

Since v0.0.47

Global resolution retains one latest dependency context and its checked request bytes. The cache key compares the published dependency context, requested packages, and advertised entry bound; repeated reads reuse the bounded context even when the global API accepts fewer entries. The client compares a full request digest and analyzer and corpus revisions before reusing its answer.

The standard library pass runs rustup --version, rustc --version, rustc --print sysroot, and node --version through the server's bounded command runner, under [dependencies] resolution = "auto" alone. Every probe runs with RUSTUP_AUTO_INSTALL=0 and MISE_AUTO_INSTALL=0 laid over its environment and RUSTUP_TOOLCHAIN removed from it. A rustup proxy or a mise shim installs the toolchain a pin names when it runs in that pin's folder, and RUSTUP_TOOLCHAIN beats the project's toolchain file.

rustup honors RUSTUP_AUTO_INSTALL from 1.28.1, so rustup --version runs first from the filesystem root, where no toolchain file applies. Below 1.28.1, or when its first line names no version, rustc does not run and the entry keeps its static reading; the output decides, because rustup --version exits 0 whether or not it reached the active toolchain. Where no rustup starts, as beside a distribution's rustc, rustc --version runs directly.

Package analysis

Since v0.0.39

One package's source reaches the global index through package analysis in rift-analysis, which holds no I/O: it takes bytes a caller already read, parses them with the shipped syntax providers, places each file under rift://source/<manager>/<name>@<version>/<path>, assembles the declarations, and renders one PackagePublication. That publication is the logical package index global ingestion consumes. The local server runs no package analysis; its package facts come from the global index.

ExactPackageLimits::with_declarations sets the declaration bound for semantic assembly; stub and implementation declarations both count. ExactPackageLimits::from_configuration validates the configuration and uses the file, aggregate byte, declaration, and relationship bounds from [source].

The adapter also applies per-file parsing bounds from [providers.syntax], documentation collection bounds from [documentation], and publication bounds from [package]. Callers accept the configuration through rift_core::acceptance::accept_configuration to apply the same environment overrides as the local server, such as RIFT_SOURCE_DECLARATIONS.

ExactPackageLimits::new keeps the provider's 100,000-declaration default unless the caller sets an override. The configuration adapter leaves file selection to the caller.

package.retained_source bounds each retained source string; package.retained_total, when set, bounds their aggregate bytes. Units, symbols, and search documents each count their own source copy, and the analyzer checks the aggregate before allocation.

ArchiveLimits::from_configuration applies [package.archive] when the caller decodes an archive before constructing package input. Archive admission and package input each enforce their selected bounds; increasing compressed bytes alone does not raise expanded bytes or declaration capacity.

Since v0.0.46 · Rust, JavaScript, TypeScript, Python

File selection decides which files of a package the analysis reads, from package-relative paths alone. It keeps:

  • the source files the package language's definitions claim: .rs for Rust, .py and .pyi for Python, and every JavaScript and TypeScript extension for a TypeScript package, whose builds ship .js, .mjs, and .cjs beside their declaration files
  • the documentation files the workspace's own documentation selection keeps, narrowed by a package's context7.json

It skips node_modules, __pycache__, target, and .git, and the files a caller's [source] exclude patterns match, through the matcher the workspace's [source] table uses. Tests, examples, and benchmarks stay, since they show how a package is used.

Since v0.0.66 · HTML, CSS, Vue, Svelte, Angular, C, C++, Cython, JSONC

Package selection also accepts HTML, Angular, CSS, Vue, Svelte, C, C++, Cython, and JSONC. TypeScript packages include HTML, CSS, Vue, and Svelte sources beside JavaScript and TypeScript; Python packages include Cython sources. A C++ package selects .h as C++, and an Angular package selects .html and .htm as html:angular; ordinary workspace selection keeps .h as C and .html as HTML.

ExactPackageInput::with_framework_context adds captured metadata and explicit framework selections. Selected sources already belong to context; additional metadata paths must be distinct. The combined sources share package file and aggregate byte bounds. Metadata informs analysis without becoming a published source unit. PackageAnalysis::warnings reports unresolved framework context and dynamic expressions. Fresh, supplied, and checked restored raw syntax produce the same contextual publication; files with framework context parse current source before applying framework facts.

The stub and module join pairs a stub with the module it declares: mod.pyi with mod.py, index.d.ts with index.js, .d.mts with .mjs, and .d.cts with .cjs. A .d.ts beside no .js pairs with its .mjs, then its .cjs, so an ESM build claims a stub before a bundled one. Each name both files declare answers one symbol, addressed at the module: its source, range, and documentation come from the module, and its signatures and types from the stub, with the stub's overload forms gathered into one signatures list.

A declaration counts as public by its language's rule:

  • Rust - pub declarations, and the items of a pub trait and the variants of a pub enum
  • Python - names without a leading underscore
  • TypeScript - every member but private and protected ones
  • JavaScript - the declarations an export wraps or names, including module.exports and exports.name assignments, and the members of an exported class other than a # private element

In a paired module the stub defines the public set, and a stub declaration with no implementation answers from the stub alone.

Two publications are comparable by their own fields. format_revision states the shape; analyzer_revision is the digest of the analyzer manifest, which names each shipped grammar's version and checksum and the content digest of every source file that decides what a publication holds: all Rust source in rift-core, rift-analysis, rift-index, rift-provider, rift-protocol, rift-ranking, and rift-syntax, plus the history repository, MCP server and validation, and server read implementations. Every record - a source unit, a declaration, a search document - carries a digest over its own content, and the whole document renders as RFC 8785 canonical JSON, so two runs over the same bytes under the same analyzer revision produce byte-identical output, and an adapter can compute added, replaced, and removed records by identity and digest alone. A publication addresses source by package-relative path and source unit, so no machine's directory layout reaches a consumer.

The analyzer manifest is generated beside the served schemas and committed at crates/rift-analysis/src/analyzer-manifest.json, so a change to any of those files that ships without a regenerated manifest fails just generate-check. The publication's own schema is exported to package-index.schema.json.

Language engines

The rift-lsp crate owns LSP sessions. A session opens source documents, requests references and call hierarchy, and pulls diagnostics when the language server advertises that capability. It also retains published diagnostics under per-document and document-count bounds.

The reference request includes the declaration's own occurrence. An empty answer can therefore mean the engine has not loaded the file; it cannot prove the declaration has no references.

A call hierarchy answer carries no such evidence, so a walk reads the session's readiness alone. That readiness leaves out rust-analyzer's cargo check progress, which resolves nothing a walk reads, while a diagnostics read still waits for it. An embedded engine answers from its own in-memory state and declares itself ready.

The server records each publication's changed files per engine and sends them to a running session as one workspace/didChangeWatchedFiles batch before its next request. When more changed files are owed than the record holds, the server replaces the session at its next request instead, and the new session reads every file from disk.

Each process has configured startup and request timeouts. The [lsp] configuration also bounds retries and process replacements.

LSP startup records engine child spawned with the child's pid and its piped stdin, stdout, and stderr. Shutdown records engine.child.wait while waiting for exit; a timeout or failed wait records its outcome and enters engine.child.kill to kill and reap the child. engine child exited retains the pid observed before the wait or kill.

The bounded command runner records child spawned, an open process.wait operation, and child exited, with pid, stream destinations, and the observed exit_code when available. The wait records ok, refused, timeout, or cancelled, so a stalled dependency probe remains visible in the operations in flight.

Configuration

rift.toml at the workspace root is accepted at startup and re-accepted on change. Its schema is exported beside the MCP surface as rift.schema.json. A quantity value carries its unit: timeout = "120s". The configuration page states the keys and defaults.

On this page