Implementation
This page goes into how Rift is built. It's developer-oriented - if you just want to use Rift, check the main documentation.
The rift binary
Since v0.0.11Rift is delivered as one binary, installed as the Install section describes.
The CLI is implemented with clap, and Rift's two main capabilities are
implemented as rift mcp and rift server.
rift mcp
rift mcp is the MCP stdio server an agent launches. Under the hood it takes input from the
agent over the stdio transport and forwards each request to the workspace's rift server over
Streamable HTTP; the process holds no state of its own - state lives in the rift server. To
reach the workspace's server it checks the server lock file -
ServerLock at .rift/server.json - and starts a
server when no live one stands behind it.
A server that rift mcp or rift server start starts exits when its claim on .rift/server.lock
meets any other lock. When that leaves no process holding the election - on Windows a probe's
shared lock can outlive the probe for a moment - the starter starts another within the same
30-second window.
When the window closes on a server that holds the election but has not published
.rift/server.json, the proxy weighs the index and metrics databases. A server writes .rift/index
while it builds its first index and records its diagnostics into .rift/metrics, so when either file
or its write-ahead log changed during the window, the request gets a temporarily_unavailable refusal saying the server is still
building, and the agent resends it. A holder that wrote nothing keeps the refusal that names an
operator action.
Each forwarded request waits at most the proxy's forward budget: the
[server] readiness_timeout and worker_queue_timeout
the workspace accepts, plus an allowance for the answer. When the budget ends, the proxy cancels
the request, which closes its Streamable HTTP response stream, and the caller gets a
temporarily_unavailable refusal naming the budget.
rift mcp takes one --output choice: text, the default, or all. The proxy keeps the choice
for its lifetime, across upstream reconnects, so the tool listing it advertises stays stable. With
text, the proxy removes structuredContent from each completed tool result, outputSchema
from every tool on every tools/list page, and the application/json contents of a completed
resource read that holds another content, all after the upstream answer arrives; all forwards
everything unchanged.
Each rift mcp process owns its choice, so two proxies on one workspace can differ.
Before it adopts a live server, the proxy compares the product identity from the lock with its
own: the version and the canonical MCP schema SHA-256. The version is the package version with the
git commit the binary was built from as semantic-versioning build metadata, such as
0.0.45+b006b8433ba06679f06a3c7f0743d65634d32c34. rift --version prints the package version
alone.
A build script in the rift crate asks git for the commit at build time, and rift hands it to
rift-mcp when it serves, so no start reads the executable's bytes. A build from a working tree
with uncommitted changes adds .dirty. with the executable's size in bytes and modification time
in nanoseconds, read from file metadata, so a rebuilt development binary never matches a server
the earlier build started. A build with no git carries the package version alone.
When the identities differ, the proxy orders the two versions by semantic-versioning precedence,
which leaves build metadata aside, so two builds of 0.0.45 stand level whatever their commits:
- A server whose version strictly precedes the proxy's receives the authorized stop request
rift server stopsends. The proxy waits for that server to release its election and then starts one from its own binary; a request the server already holds finishes first. - A server whose version equals the proxy's from another build, or follows it, is refused: the
caller receives a JSON-RPC internal error naming both identities and asking for
rift server stop.
The stop request carries the token the lock records, and every server mints its own, so it stops the one server the proxy read and never a server another proxy started after it. A proxy stops at most one server per connect: when another older server takes the election after the first one left, the proxy refuses rather than race the process that started it. A stop request that fails in transport decides nothing, since a server another proxy already stopped closes the connection as it exits. The proxy reads the election again, as after an accepted request, and refuses only when its 30-second wait for publication ends with that same server still serving.
rift server
The server is an Axum application on the Tokio
runtime. MCP clients use its /api/mcp route, which serves Streamable HTTP through
rmcp's service layer; /api/stop serves
rift server stop.
When the server builds a tool result, it renders compact text from the typed answer into
content and serializes the same answer into structuredContent. Typed layout code renders every
tool answer, every failure, and the workspace resource as sections: a title at column 0 that counts its
entries, such as 2 results, 1 warning, or 2 errors, with the entries indented under it. The
text writer enforces a byte limit on each text and refuses one that crosses it.
A workspace resource read carries the compact text as text/plain, then the JSON body as
application/json. A map read carries one application/json content under both output choices.
Only rift mcp applies the output choice, so direct HTTP access returns both tool and workspace
representations.
Where the server converts a tool method's result, it classifies a Rift operating failure on the
tool path as a completed tool result with isError set and the failure's compact text in
content: one N errors section whose entries are the failure, then its causes. Protocol
failures stay JSON-RPC errors.
Startup runs one cycle:
- Take the exclusive file lock on
.rift/server.lock, which picks one starter among racingrift mcpprocesses, and hold it until the process exits. - Parse
rift.tomland accept the workspace; an invalid file keeps the server running but refuses every request until fixed. - Start the native filesystem watcher.
- Publish an empty index carrying
local_index_preparing, then start background discovery and preparation on the bounded worker pool. - Bind the first free loopback port of the accepted
[server]selection - the pinnedportor the configuredport_range- mint the bearer token, and write both into.rift/server.json. - Serve until no request remains active and the
idle_timeoutspan passes after the last response completes, then stop.
rift server start runs that cycle detached, with the child's standard error on
.rift/server.stderr, and waits for .rift/server.json to appear; detached startup and
rift mcp each allow 30 seconds for publication. A start whose child is still indexing when
that window closes reports the server as starting rather than failing, and a start that finds
the election held by a server still indexing spawns nothing and waits the same window for that
server's file. rift server stop sends the authorized stop request and waits for that process
to exit or release its election. A proxy
can start a replacement during this wait; the replacement does not extend the original stop.
rift server restart chains stop and start. rift server status reports the workspace's
current state - a serving server with its port, pid, and version, a server still indexing,
a stale .rift/server.json with the reason it went stale, or no server - and changes nothing.
A probe counts a server as serving
only when its election is held and its recorded port accepts a connect. --foreground keeps
the cycle in the calling process, where Ctrl-C stops it.
--auth skip serves every request that clears the loopback boundary without the bearer check.
The MCP specification's own conformance runner forced it: the runner addresses a server by URL
alone and sends no Authorization header, so a checked server refuses it at the door. The CLI
accepts the flag only together with --foreground, so no detached server ever serves
unchecked; the server still mints its token and writes it into .rift/server.json, so
rift mcp reaches it unchanged; and the serving process records one WARN naming the run.
Filesystem and parser work runs through one named Rayon pool. Admission bounds concurrent
operations before a Tokio blocking task enters that pool, and cancellation is checked
between files. This behavior is configurable via the
num_workers and worker_queue_timeout keys in the
[server] section.
Freeing a large index can take longer than closing the listener. After the index supervisor stops, the blocking pool releases its retained index while the foreground process finishes the log drain and releases the election.
Since v0.0.62A foreground stop runs its stages under one deadline, SERVER_STOP_DEADLINE (4 s) from where the
stop begins, and each stage takes only what the one before it left:
- The serving task drains, and the engines and index supervisor shut down, by the deadline minus 1.5 s, the bound step 2 shares, so a stage that runs out its bound still leaves steps 3 to 5 their reserves.
- The index database, and the vectors database when it opened, close by the deadline minus
1.5 s:
SERVER_EXPORT_STOP_RESERVE,SERVER_LOG_FLUSH_RESERVE, andSERVER_DATABASE_STOP_RESERVE, 500 ms each. A close failure becomes awarnrecord with operationdatabase.close, so a later stage writes it. - The
otlp exportstage sends the final spans, log records, and metric points and shuts the export down by the deadline minus 1 s, and at mostSERVER_EXPORT_STOP_RESERVEafter it starts. A collector that is down or stalled costs the stop that reserve and never fails it. - The log drain's final flush ends by the deadline minus
SERVER_DATABASE_STOP_RESERVE. A drain that outlasts it is aborted, and its warning carriesunwritten: the records the drain accepted and never wrote. - The metrics database closes by the deadline, so it outlives every record.
- The election guard drops, right before the process exits.
- The
tracing shutdownstage flushes what the export still holds, withinOTLP_SHUTDOWN_TIMEOUT(500 ms), and aprocess exitsrecord withstage = "process exit"and the exitstatusis the last line on stderr before the process exits.
On Windows, a process does not end while one of its threads has pending I/O: "The terminated
process cannot exit until all pending I/O has been completed or canceled" (TerminateProcess).
A checkpoint syncs the database file, and on a slow disk that flush took up to 3.3 s against the
4 s stop, so no stop closes a database with a checkpoint: a flush that started would hold the exit
past the bound. The checkpoint runs at the next open instead, where no stop deadline runs.
A foreground repository server opens no metrics database of its own; its log drain routes each
record to the consumer of the workspace the record names. Each workspace stops inside the serving
stage repository workspaces shutdown, every workspace at once. Within a workspace, the engines,
the index supervisor, then the index and vectors databases stop by the workspace deadline minus
WORKSPACE_LOG_FLUSH_RESERVE and WORKSPACE_DATABASE_STOP_RESERVE; its consumer's final flush
ends by the deadline minus WORKSPACE_DATABASE_STOP_RESERVE; its metrics database closes last.
The server's serving stages end by the deadline minus SERVER_EXPORT_STOP_RESERVE and
SERVER_LOG_FLUSH_RESERVE, its otlp export stage by the deadline minus
SERVER_LOG_FLUSH_RESERVE, and its log drain stage by the deadline.
Each stage runs inside a server.stop span and ends with a stop stage ended record carrying
stage, remaining, and outcome, plus error and causes for a failure. The outcome is ok
for a stage that ended inside the deadline, error for a failure, and timeout for a stage that
ended without a failure once its bound had passed. Only a failure makes the stop exit with status 1.
The otlp export stage fails nothing, even with the outcome error, such as a refused connection:
it records the outcome at warn, because the export carries diagnostics only. A process that
exports nothing records the stage ok at once.
The stage runs before the log drain stops, so a drain the stop abandons counts its unwritten
records in log.queue.dropped after the last export: the drain's warn record carries that
count, and the collector never receives it. The log record export stops with the stage too: the
stop stage ended record of otlp export and every record of the later stages reach stderr and,
before the drain stops, the metrics database, and the collector receives none of them.
A stop that arrives while the index preparation runs a dependency version probe reaches the probe
too. The probe reads the stop once per 25 ms poll, then kills its child and reaps it, and its close
records outcome=cancelled with error.type=cancelled. A cancelled preparation publishes nothing.
After the kill the probe waits at most 200 ms for the child's output pipes to close. On Windows the child runs inside a Job object of its own, so the kill ends every process the child started. On Unix a process the child started can keep the pipes open, and the probe stops waiting on it.
A database close runs no checkpoint and syncs no file; the write-ahead log keeps every committed
transaction for the next open. A close whose thread outlasts its bound ends its stage with
timeout: the thread keeps running until it finishes or the process exits. A metrics close ends with timeout whatever stage its bound passed in, a close
still queued behind an earlier write included. An index or vectors close that started past its
bound ends with timeout too when the worker it finds still running outlasts it; a transaction
that worker left open is rolled back at the next open. An index supervisor still running at its
bound is aborted the same way: its stage ends with timeout and the table of operations in
flight, and blocking work it started ends with the process.
A server records each step of its life at INFO, so the store keeps them under the default
[logs] capture. A record that opens a step carries phase = "start", one that ends it
carries outcome = "ok", and one that fails it carries outcome = "error"; a printed line
marks them →, ✓, and ✗:
| Record | Operation | Fields |
|---|---|---|
workspace election claimed | server.start | state_directory |
another rift server already serves this workspace; this process exits | server.start | |
MCP server ready | transport, port, outcome | |
index preparation started | index.build | total, phase |
index preparation progressed | index.build | prepared, total |
index preparation superseded | index.build | epoch, observed_epoch, trigger |
index snapshot published | index.publish | trigger, epoch |
lexical commit handed | search.commit | tree_revision |
lexical commit committing | search.commit | tree_revision, phase |
lexical commit settled | search.commit | tree_revision, outcome |
history fill started | history.fill | pending, phase |
history fill finished | history.fill | written, outcome |
vector population started | search.populate | tree_revision, phase |
vector population finished | search.populate | tree_revision, outcome |
Git index lock wait ended | index.lock | path, elapsed_ms, outcome |
index watch stopped | index.supervisor | |
stop stage ended | server.stop | stage, remaining, outcome |
workspace election released | server.stop | state_directory |
A lexical commit the store refused is owed instead of settled, and its the lexical commit failed record carries the error. A history fill with no commit pending records neither its
start nor its end. A vector population that fails ends with its warning, carrying
outcome = "error", in place of vector population finished.
A wait for the Git index.lock records Git index lock wait ended only when it found the lock
present, with the outcome ok, timeout, or cancelled. The election guard drops after the
metrics database closes, so workspace election released reaches stderr and not the store. A
database thread whose close ended timeout still holds a clone of the election guard, so the
stop retires server.json itself before the process exits, records a database thread still holds the workspace election at warn, and the election is released by the process exit.
A watch failure records once per step: index watch backend reported failure under
watch.receive at WARN, or index watch failed under watch.observe at ERROR, each with
error and error.type, the registered identity. Repeats of the same identity record nothing
until the supervisor's next turn, or the watch's stop, records one line carrying their count.
rift mcp and rift server start record their side of a start on their own stderr. Each
detached server they spawn records detached server spawned with its pid and where stdin,
stdout, and stderr go, and a start that sees the child exit records spawned server exited
with its pid and, when the platform reports one, its exit_code. The proxy's start window ends
with start window closed with a server that answers, or start window closed without a server that answers, both carrying rounds, elapsed_ms, and presence.
A stopped server leaves its process as soon as serving has ended and the log drain has stopped,
through the process exit rather than by returning from main. Dropping the runtime would wait
for every blocking task still running, and a lexical transaction or a dependency pass that
started behind a publication can run for minutes with nothing able to cancel it.
The project resolver skips symlinks and the .git, .rift, and target directories.
Discovery and event filtering also skip nested linked Git worktrees. Submodules and independent nested repositories retain the existing source visibility rules.
The symbol index stays in memory as one immutable value: a rebuild constructs the next index and accepted configuration outside the async lock, then swaps the complete value under its write lock. Each request clones the published value under a read lock and retains that capture through its result, so reads never combine state from two index revisions.
Since v0.0.25Watcher events advance one workspace change counter and enter the same
serialized rebuild lane. Each observation carries the paths it saw, and the rebuild reads only
those files, sharing every other file with the previous index. A loss event, a written
.gitignore, or a directory appearing or disappearing reads the whole workspace instead: the
rebuild walks the tree and reads and digests every visible file under the ignore files as they
stand now, and parses only the files whose bytes the previous index did not already parse. A
written rift.toml reads and parses the whole workspace when [source], a language file
selection, [search.text] inclusion or chunk bound, [documentation], [dependencies], or
[providers.syntax] changed. Every other accepted configuration change reuses the published tree.
Before each settled current-tree tool read, the server validates the digest of every included
workspace file, then waits until the published index counter matches its observation. That
capture already walked the tree, so a read that finds the index behind names the files that moved rather than
asking for a whole rebuild. A file that appeared or disappeared because a .gitignore changed
is the exception: the published index still applies the ignore files it was built under, so
rebuilding that one file changes nothing, and the read asks for the whole workspace without
waiting for the watcher to report the .gitignore write.
The watcher can report that .gitignore write after the read's capture already found it. The
report asks for the whole workspace, so it supersedes the rebuild the read asked for, and the
read captures again: a capture that matches its previous one waits for the next rebuild, and only
a capture that differs, a tree still moving, answers from the published index with stale_index.
A rebuild that is still running asks the same question publication asks, between files and at each phase boundary: whether a later observation asks for the whole workspace, or names a path whose bytes differ from what the rebuild read there. Such a rebuild stops, returns its paths to the next rebuild, and wakes the waiting reads.
A later observation of the bytes the rebuild already read, such as a read's own request, leaves it running, and its index publishes for that observation. The comparison reads at most 64 observed paths over one rebuild; a larger observation is decided at publication.
Since v0.0.47The capture records each file's stat beside its digests: length and modification time, and on Unix status change time, device, inode, and mode. It reads a timestamp from a temporary file on the workspace filesystem before capturing bytes. An unchanged stat permits digest reuse only when both recorded change times precede the boundary that validated those bytes.
This follows Git's racily-clean check, with a status-change time check for rewrites that restore modification time. Reused entries retain their original boundary, so copying an entry into a later capture does not make a same-tick write safe to skip.
Stat reuse applies to APFS and HFS+ on macOS, and ext4 and tmpfs on Linux. Windows and other filesystem types read every visible file again. A failed boundary, changed root, different device, or backwards timestamp also selects full reads under the same request budget.
Every complete pass still walks the source policy to discover new and removed paths. The watcher's loss flag requests this recovery before path filtering, including a loss event with no path. The observer and publisher share the publication lane, where the candidate's epoch and source policy are checked before the complete value changes.
Background validation uses that same source discovery and stat comparison even without watcher
events. Its deadline follows [server] validation_interval, and requests and filesystem events
do not move it. An unchanged check without a recorded failure queues no rebuild; changed paths use the existing
incremental comparison, while a changed configuration or ignore policy requires a complete pass.
A due check runs on the supervisor's next turn, before the rebuild that turn owes. The turn after a superseded rebuild is the exception: the owed rebuild runs first, because the reads waiting on it already waited for a rebuild that published nothing, and the check takes the following turn. A check that is one interval late runs before the rebuild, so a check starts at most one interval after its deadline, plus the rebuild running when that interval ends.
A recorded rebuild failure is retried even after the tree returns to the publication's exact bytes. An empty incremental recovery reuses the read service and clears the failure without deriving source again.
The watcher also observes the worktree's Git directory, including a linked worktree's directory
outside its source root. HEAD and index.lock events request validation before the source
floor; other .git paths retain that floor.
Before a queued rebuild, the supervisor waits for index.lock to disappear within
[server] version_control_timeout. The first observed lock fixes the deadline, so repeated events
cannot renew the wait. An expired lock is named in the log and passed until its absence is observed;
shutdown cancels the wait.
Repository serving
Since v0.0.47Repository serving requires an explicit foreground start:
rift server start --foreground --repository
rift server status --repository
rift server stop --repositoryThe repository process accepts server settings from the main worktree and records them at
startup. A proxy adopts it only when the workspace and main worktree accept those same settings;
otherwise the workspace starts its own process, as the
[server] configuration describes.
The process holds its election under <common git directory>/.rift/<product identity digest>/.
The digest is SHA-256 of the canonical product identity, and the directory contains server.lock
and server.json. Workspace routing remains inside the proxy and HTTP transport.
Each admitted workspace holds an exclusive lease on its .rift/server.lock and keeps its own
index, vectors, and metrics databases. A competing serving process is refused before opening that
workspace's databases until the lease is released. rift server logs still reads the metrics
database, whose reads take no lock. The repository process shares its bounded worker pool and content cache;
workspace source, configuration, publications, and language processes retain their own owners.
Each database's thread retains the workspace lease until it exits, including after a shutdown
wait expires. A workspace's stop closes the index and vectors databases, then its metrics
database, within WORKSPACE_STOP_BOUND, and releases the lease after a clean close. When a repository
process stops, it stops every workspace at once, each by the shared deadline, so one workspace's
slow close spends none of another's bound; each workspace's records sit under a server.stop span
carrying its root. History retains the existing common Git store and its fill and live locks.
Local index preparation
Since v0.0.47The initial publication answers reads before discovery finishes. The supervisor discovers visible paths, publishes their map counts, and then captures and analyzes bounded batches. Each publication owns its map, source catalog, configuration, and preparation counts.
search and get_symbol use the declarations already prepared. Their
local_index_preparing warning remains until their selected
local scope is complete, including an exact symbol read that already found a declaration. A nodes read can
prepare its supplied path within the request budget without changing published progress.
The rift://workspace resource projects only
the requested page from published visible file digests. It performs no whole-tree capture per
page; the completed catalog includes unclassified and disabled-language files as well as files
whose syntax was refused.
The process-owned content cache holds weak references to captured bytes and path-independent syntax facts. Its key includes the full content digest, language provider, and parse limits; path-dependent declarations remain with their workspace placement. Dropping the last publication that owns content releases it, while live publications remain readable.
A read that meets a recorded rebuild failure at or past the published counter waits for nothing:
it answers from the published index and carries stale_index, whose detail names the counter
the failed rebuild answered for, the counter the tree is at, and the failure's causes. The next observation retries the rebuild.
With debug capture enabled, index.build records its start and epoch before reading a candidate
tree. A get_symbol history read records its start and retains a span until the history request
ends. The corpus shutdown tests use these records to observe work in progress before issuing
rift server stop; starting a client request alone does not prove that work has started.
Databases
Since v0.0.9 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, PythonThree SQLite files under .rift hold what a server persists for its
workspace:
| File | Holds | Crate | Thread |
|---|---|---|---|
.rift/index | lexical, trigram, and documentation tables | rift-index | rift-db-index |
.rift/vectors | semantic_vectors | rift-index | rift-db-vectors |
.rift/metrics | log_records | rift-tracing | rift-db-metrics |
The index and vectors databases go through Toasty models.
The full-text surface is raw SQL, because Toasty 0.10 has no typed virtual-table or MATCH API,
and the raw boundary stays confined to the FTS table and its rows. The metrics database goes
through rusqlite directly, as the history store does.
The server never reads, migrates, or removes a .rift/db file left in the state directory;
delete it and its -wal, -shm, and .lock files by hand.
An elected server opens the metrics database first, then the index database, so the index
database's open lands as a record. The vectors database opens through LazyDatabase, a handle
that opens the file at the first vector operation, at most once, under the same election owner;
an open that fails leaves the handle unopened, and the next vector operation tries again.
Each open stands alone. A refused index database emits a database.open warning and the server
starts without it, while logs keep recording and rift server logs prints that warning.
Each of the index and vectors databases is one WorkspaceDatabase: one owned SQLite worker
thread executes driver operations for its file. Async callers send operations through a bounded
queue and await their replies, so SQLite lock waits and statement execution occupy that thread.
Each queue uses the accepted [search] pool_slots and
busy_timeout bounds.
Every process that opens .rift/index or .rift/vectors holds an exclusive lock on
.rift/index.lock or .rift/vectors.lock while it switches the file to WAL and applies that
database's migration set. SQLite refuses a second process's WAL switch at once, without waiting
for the first, and Toasty applies every migration it read as missing, so two processes preparing
one new file side by side leave one of them without the database. A process that meets the lock
waits for it up to [search] busy_timeout.
LogStore::open in rift-tracing spawns rift-db-metrics, which opens the one write connection
to .rift/metrics and keeps it for its life. One thread owns the only write connection, so
writers in the process never compete for the file, and the metrics database has no write turn
and no lock file.
rift server logs opens a query_only connection of
its own through LogReader, which never creates the file or a table and never switches the
journal mode. The file carries its schema version in PRAGMA user_version: a writer that finds another version drops
log_records and creates it again, and a reader refuses.
The metrics database's bounds are compiled: METRICS_BUSY_TIMEOUT_MS, 1 s, and a 4 MiB
write-ahead log limit. A log read therefore never inherits a search wait, and rift server logs
reads while rift.toml is invalid. The writer connection waits through a busy handler of its own
that sleeps SQLite's own busy timeout steps, 1 s in all, and counts each call in
sqlite.busy.retries.
Documentation extraction lives in rift-analysis. Workspace and package adapters pass selected
bytes, exact source identities, origins, and existing chunk ranges into that shared crate. Its
collector feature enables syntax parsers; readers can validate stored metadata and project ranked
identities without linking parsers or storage code.
Documentation adapters record source warnings when selected input is malformed or exceeds the
selected documentation.max_file or remaining documentation.max_total bytes. Their defaults
are 4 MiB per source and 512 MiB per collection; other accepted sources remain available.
documentation.max_sources counts content owners, including notebook cells and omitted sources.
The default is 100,000, and a selection above the configured bound is refused before publication.
The index database stores the documentation metadata beside the lexical rows, one row per source: the source's facts, blocks, links, references, and unresolved spellings, under the digest of those bytes. A manifest row carries what describes the whole collection, and each exact reference also gets a reverse row filed under its source. Blocks retain byte ranges and digests; excerpts come from the captured source after pagination. The transaction that stamps a lexical write's tree revision writes its documentation metadata.
The collector keeps parser facts for each selected source in memory. Incremental rebuilds reuse facts when source metadata, chunk mappings, extractor revision, and accepted attached symbols match. Link and declaration changes invalidate affected blocks; unchanged notebook bytes reuse decoded cells. A restart rebuilds parser facts from source.
A write compares each source's digest with the one its row carries and rewrites only the sources that moved, removing a source's row and its reverse references together, so an edit to one documented file writes that file's row rather than the whole collection.
The store records, beside each file's rows, the digest of the bytes they were derived from, and its stamp names what else derived them: the analyzer manifest, the corpus shape, and the index-owned configuration. Every write compares before it writes. A rebuild that named the files it read deletes those paths' rows and inserts their new ones. A rebuild that read the whole workspace, startup included, compares every file it holds with the recorded digests and writes only the files whose bytes moved or that one side lacks, so a restart over an unchanged tree writes the stamp and nothing else. A store stamped by another analyzer revision or configuration keeps nothing: the write clears it first. An unrelated binary change keeps compatible rows.
The write runs behind the publication: the rebuilt index becomes current first, and the lexical
lane runs the writes it was handed in publication order, one at a time. The lane holds one write
behind the running one, and a write handed while one waits merges into it: the union of the files
both named, derived from the newer publication, leaves the store exactly as running both would. A
burst of changes therefore costs one write per lane turn. The index database's write turn
serializes index commits and trigram batches, and the vectors database's turn its vector batches;
each transaction starts with BEGIN IMMEDIATE. Log batches run on the metrics writer thread.
A write commits in transactions of at most
[search.lexical] transaction_units units and
transaction_size content bytes, one file's rows always in one transaction. Every transaction
but the last stamps no tree, so a reader meets no publication rather than a mix of two, and only
the last names the tree the write answers for. A server that stops during a write aborts one
transaction: the ones already committed recorded their files' digests with their rows, so the next
start's comparison writes only the files the stopped write did not reach.
Between writes, the lane fills the trigram index behind regex search. Each batch indexes the
oldest rows the index lacks, under the same transaction_units and transaction_size bounds, and
a write handed meanwhile runs before the next batch. A search never waits for a batch: it reads
the rows the index lacks beside the index.
Each database uses WAL with synchronous=NORMAL. Read-only connections use committed WAL
snapshots. In the index and vectors databases, search.busy_timeout bounds a lock wait caused by
another process, and a wait for a pooled connection when every one of the search.pool_slots is
checked out.
Past that wait, a search ranks without the store: it answers from identifier matching and warns
lexical_ranking_unavailable, with a detail naming the slot wait. A regex search verifies every
file its paths select in place of the rows the trigram index would select, and warns nothing.
Every other checkout refuses, a write among them, with index.database_failed, which names the
database and its path, as a failed open does.
Every checkout sets its pragmas again. mmap_size comes from
[search.lexical] mmap_size on the index database and
is zero on the vectors database, and journal_size_limit comes from
[search] journal_size_limit on both.
Each close is the operation database.close, component storage, naming its database: it
records operation opened when it starts, sits in the table of operations in flight while it
runs, and its close record carries elapsed_ms and the outcome.
A close runs no checkpoint. SQLite's last connection checkpoints when it closes, unless
SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE is set on it, and the index worker's connections live inside
the Toasty driver, which takes no configuration. So WorkspaceDatabase::shutdown first opens one
connection of its own on the file, on the rift-db-close thread, with that flag set, and reads
the log's frames with PRAGMA wal_checkpoint(NOOP), which syncs nothing. A WAL connection keeps
a SHARED lock on the database file while it is open, so no worker connection gets the
EXCLUSIVE lock its close checkpoint needs; the close's own connection closes last, without a
checkpoint. The close then records database closed; the write-ahead log stays for the next open with the frames the log held as log, the frames an earlier checkpoint had moved as
checkpointed, and elapsed_ms. A worker whose stop outlasts the deadline is a warn record
with operation database.close and fails nothing. The metrics writer thread sets the same flag on
its connection and closes it after reading the frames, and the stop records the same message, or,
past the deadline, database close outlasted the shutdown deadline; the write-ahead log stays for the next open at warn, naming the close stage, queued or close, and its elapsed_ms.
Every open runs PRAGMA wal_checkpoint(TRUNCATE) after its migrations, where no stop deadline
runs: SQLite reads the log the last process left ("the WAL file might be retained on disk after
all connections to the database have been closed"), and the checkpoint moves it into the
database and empties the file. The index and vectors open records database checkpointed its write-ahead log with operation database.open, journal_size_limit in bytes or -1 for none,
and wal_autocheckpoint, the log pages at which a commit checkpoints. Between opens the log stays bounded by SQLite's
automatic checkpoint, "when the WAL file reaches a threshold size of 1000 pages", and is cut back
to journal_size_limit ([search] for the index and vectors databases,
METRICS_JOURNAL_SIZE_LIMIT_BYTES, 4 MiB, for the metrics database) at the commit that restarts
it. Under synchronous = NORMAL "no sync operations occur during most transactions", so a
commit the close leaves in the log survives a process exit and can roll back after a power loss,
as any commit since the last checkpoint can.
A commit that refuses leaves the previous rows and stamp intact and records its cause in the server log; the next write compares its whole publication with the recorded digests.
The index database's tables:
lexical_documents- one row per indexed document: a declaration, carrying its names, signature, and documentation, or an indexed file's text, the one copy of it the store keeps, split into chunks past the[search.text]chunk bound. The row carries an integerid, the document's identity, path, kind, and content digest beside the searchable fields, and a chunk row itsbyte_offsetin the file, so a position in the chunk maps back to the file without chunking it again.lexical_files- one row per file the published index holds, the files it left out included, carrying the SHA-256 of the bytes that file's rows were derived from. A change writes a path's rows and its digest in one transaction.lexical_index_state- stamps the set with the tree revision that published it, the corpus revision it was built under, and a digest of what besides the files' bytes derived the rows, which a later write compares before it keeps any row. One read transaction reads that stamp and runs the query, so a commit landing between the two cannot slip another tree's rows into an answer. A store holding another tree returns no rows at all: either the lane has not committed the captured publication yet, andsearchwarns naming that revision, or the publication has since been superseded, and the request recaptures rather than ranking rows it cannot place. A store holding another corpus revision returns no rows either: its columns, derivation, tokenizer, or weights are not this build's. A stamp that names no tree answers the same way as an empty store.lexical_documents_fts- the FTS5 virtual table over the searchable columns. It is an external-content index overlexical_documents: it holds the token index alone and reads column values from the row whoseidis its rowid, so no field is stored twice. FTS5 leaves the two in step to the writer, so a path's index entries are removed with the'delete'command while its rows still hold the indexed values, and a document is indexed only after its row is written. Ranking is FTS5'sbm25with the per-column weights the architecture page lists, comparable only within one search. The same statement asksbm25once per column with every other weight zeroed, which is how an answer states that a term landed in documentation rather than in a name. The tokenizer isunicode61 remove_diacritics 0: keeping diacritics is what lets Rift split the same text the same way outside SQLite without carrying a folding table of its own.lexical_documents_vocabulary- anfts5vocabtable over the word index's columns. Its per-column document counts are the document frequencies body matches score by.lexical_documents_trigram- a second external-content FTS5 index overfile_contentalone, tokenizedtrigramwithdetail=none: it records which rows hold each run of three characters, case-folded, and no positions, which is all a regex prefilter reads. Declaration rows carry no text and stay out of it, andcolumnsize=0keeps FTS5'sintegrity-checkfrom counting them as missing. The lane's trigram batches fill it, and a write that removes a path deletes the entries of the path's rows the index holds alone: a'delete'naming a row the index never took leaves it failingintegrity-check.lexical_trigram_pending- theidof every file row the trigram index doesn't hold yet. A write files its new file rows here in the transaction that stores them, a trigram batch takes them off as it indexes them, and a regex search reads it beside the trigram index, so a row stored since the last batch still reaches the answer. The vectors database holdssemantic_vectors, one row per embedding space and declaration digest. The metrics database holdslog_records, the bounded server diagnosticsrift server logsreads.
The index and vectors databases each apply a Toasty migration set numbered from 1 and record it
in the file's own __toasty_migrations table.
A rebuild that named its files deletes and reinserts exactly those paths, which is why
lexical_documents carries an index on path: the index finds a path's rows and their index
entries without reading the full-text table. Every named path is replaced, added ones
included: two rebuilds captured from one publication both write what they read, and the second
leaves what the first left.
Search is a layered configuration: the default is this full-text store beside identifier
matching, and [search.vector] adds the vector store
below. All three fuse by weighted reciprocal rank, under the shares
[search.ranking] sets.
History store
Since v0.0.46The history store is a SQLite database apart from the workspace's three, so lexical and log
writes never wait on a fill. It sits in the .rift/ folder of the common git directory, one file per
derivation revision: the analyzer manifest, corpus revision, index-owned tables, and the
[providers.history] strategy and releases. Compatible builds reuse that file, and a server
deletes a file no live server holds.
rift-history-store opens the file through rusqlite in
WAL mode, where a writer never blocks a reader. One filler writes a store at a time: the history
task holds an exclusive std file lock beside it, and every server holds a shared lock on the
revision it reads. Each batch commits in one BEGIN IMMEDIATE transaction after its parses finish.
Commit messages sit in an FTS5 table with external content over the commit rows, under the corpus
tokenizer the lexical store uses. Deleting a commit issues FTS5's 'delete' command while the row
still holds the message the entry was indexed from.
The history task runs on the runtime's blocking threads, outside the [server] num_workers pool.
Its spans time each wait for an idle server (history.idle_wait), each
batch (history.batch, history.analyze, history.write), each plan (history.plan), and the
rename and move pairing (history.renames, history.moves). A comparison's spans time the
changed-path listing and each side's index (search.change_paths, search.change_base,
search.change_head).
Server logs
The server records diagnostics in the metrics database at .rift/metrics. rift server logs
prints them oldest first, with or without a running server:
rift server logsThe flags narrow what prints:
-f,--followkeeps printing records as the server writes them-n,--tailprints only the newest records--sinceprints records recorded at or after an age such as10m, or an RFC 3339 timestamp such as2026-10-04T20:42:58Z--untilSince v0.0.62 prints records recorded before a bound, in the same two forms; with--sinceit selects one window--levelselects one severity--componentselects records from one component
The [logs] table bounds how many the server keeps.
rift-tracing holds a record from its capture to its row. The LogSink layer publishes each
event and span close that [logs] capture admits to the persistence queue and independent
subscriptions. A full persistence queue drops the newest record and counts it in
log.queue.dropped with error.type queue_full.
The log drain writes the queue in batches.
A batch the metrics database refuses stays with the drain, which retries it with no attempt
count; meanwhile the queue fills and then drops, and the next batch written carries a
the log queue was full and dropped records record with the dropped count. The stop's
deadline bounds the retry: a drain aborted there reports what it held as unwritten.
runtime.logs().subscribe() returns Result<LogSubscription, RiftError> and registers an
independent subscription. Disabled or closed capture returns tracing.log_stream_unavailable;
registration at 16 admitted subscriptions, including persistence and closed subscriptions
with retained publications, returns tracing.log_subscription_limit.
Registration and publication use one order, so subscriptions receive records from registration
onward, with no replay. recv() reads one LogPublication; recv_batch() reads a bounded batch.
Each publication carries its sequence and an immutable record shared across subscriptions;
record() borrows that record.
The sequence preserves dropped positions, so consumers can compare it with their loss count.
Each subscription shares an 8 MiB byte budget between its queue and publications its consumer
retains; a full record or byte budget drops the incoming record and increments that
subscription's dropped() count. Dropping a subscription unregisters it and releases its queue;
already returned publications release their bytes when their owner drops them. A closed or
dropped subscription keeps its admission while buffered or returned publications retain bytes.
Its place becomes available after the last publication drops. The 16 admitted budgets retain
at most 128 MiB of records; explicit copies made by a caller sit outside those budgets.
The byte accounting includes the publication envelope, record, shared ownership header, and string lengths. Bounded channel slots and allocator overhead sit outside that accounting.
The [logs] keys queue_records, flush_interval,
and retry_interval configure queue capacity, partial-batch flushes, and retained retries. Under repository serving, a routing drain sends records
to one bounded consumer per admitted workspace, selected from the request span's workspace.
Each consumer holds at most min(queue_records, 256) queued records and the same number in a
batch, sharing a 1 MiB byte budget. At most 64 consumer budgets remain retained, including
replaced consumers and their queued records or writer batches. Admission at that bound refuses
the new consumer before replacing the current route. These budgets sit outside the 128 MiB
subscription budgets.
An event record names the spans it was emitted in, so the records of one request group by its
request_id even while a background task logs between them. When a span opens, an unfiltered
layer ahead of stderr and the LogSink writes its name and fields as one JSON object into the
span's extensions, and writes it again when the span records a field later. An event inside
the span copies that object into its fields as root_span for the outermost span, and as
nearest_span for the span it was emitted in when that span is not the outermost. A span that
closes inside another carries the outermost one as root_span on its close record. The layer
keeps every span, so a record names a span its own filter left out, and stderr and the store
write the same record. A dependency's span with neither component nor operation, such as
rmcp's serve_inner, takes no part: the Rift span inside it is the root.
rift mcp opens mcp.forward with request_id empty and records the forwarded request's id
once rmcp sends it, so the proxy's lines join the server's request group; records before the
send carry none. The span also records the caller's original JSON-RPC id as caller_request_id.
traced!, #[rift_tracing::timed], info_span!, debug_span!, and the event macros record
code.function.name, the function that opened the span or emitted the event, read from the
type_name of an item declared in its body; the format of that text is unspecified, so no
filter or query selects by it. A tool call's mcp.request span carries mcp.method.name,
gen_ai.tool.name, and workspace, the workspace root, beside request_id and tool. A close record carries busy_ns and idle_ns,
the nanoseconds the span was entered and was not, elapsed_ms, the time from the span's opening
to the end of its traced! operation, the reading the duration histogram records, and
status.code: Ok, or
Error with error.type panic for a span closed while its thread unwinds a panic and
cancelled for an awaited traced! operation dropped after its first poll. The
worker admission started record of a search request carries:
{
"code.function.name": "rift_mcp::server::BlockingExecutor::admitted",
"work": "workspace fingerprint",
"queue_timeout_ms": "30000",
"root_span": {
"name": "mcp.request",
"fields": {"component": "mcp", "operation": "tools/call", "request_id": "2", "tool": "search",
"mcp.method.name": "tools/call", "gen_ai.tool.name": "search", "workspace": "/work/project",
"code.function.name": "<rift_mcp::server::RiftMcp as rmcp::handler::server::ServerHandler>::call_tool"}
},
"nearest_span": {
"name": "worker.queue",
"fields": {"component": "worker", "operation": "worker.queue", "work": "workspace fingerprint"}
}
}A span keeps at most 1 KiB of field members: a member that does not fit is left out whole and
counted in fields_left_out, so the object stays valid JSON and both objects fit the record's
8 KiB fields bound. A record's own members keep the order they were recorded in, written
through serde_json maps; the 8 KiB fields bound, LOG_FIELDS_BYTES_MAX, keeps each own member
whole or leaves it out, and the count of members left out follows as fields_left_out. A task
spawned without .instrument runs outside the span that spawned it,
so its events carry neither member. A field the code itself names root_span, nearest_span,
or fields_left_out is not recorded.
The CLI starts tracing through TracingRuntime::builder in rift-tracing, which installs one
subscriber with a filter per layer:
- stderr under
RUST_LOG, orrift=info,rift_mcp=info,rift_server=info,rift_index=warnwhen that variable is unset or does not parse, with only the stall reports ofrift_tracing::flight - the
LogSinkunder[logs]capture, installed for a foreground server alone - the OTLP span export under
RIFT_OTLP_FILTER, installed whenOTEL_EXPORTER_OTLP_ENDPOINTis set - the OTLP log record export under
[logs]capture, or under the default targets in a process without a capture, installed whenOTEL_EXPORTER_OTLP_LOGS_ENDPOINTorOTEL_EXPORTER_OTLP_ENDPOINTis set
Neither OTLP export layer hands on a span or event of the crates the export runs through: the
OpenTelemetry API, SDK, and OTLP exporter, and the HTTP client stack (EXPORT_OWN_TARGETS in
crates/rift-tracing/src/otlp.rs). A filter that admits them, such as a bare debug, would
otherwise export each batch's own records in the next one. Stderr keeps them under its own filter.
A foreground server whose stderr is not a terminal writes at most [logs] stderr_limit
there, 1 MiB by default. The write that crosses the bound is followed by one notice,
standard error reached its byte bound, and every later write is discarded and counted in
log.stderr.discarded; the records the LogSink captured are unaffected. The log drain's
stop records standard error bytes discarded with stderr_limit and the bytes discarded,
at warn when the count is not zero. A second TracingRuntimeBuilder::install in one process
returns InstallError and leaves the installation in place untouched.
Stderr, rift server logs, and a failure window print a record
through one renderer, LogLines in rift-tracing. A message or a field value holding · or
: prints as a JSON string. A line holds the time in UTC with milliseconds, the level, the
function, three spaces, the context, two spaces, the nested operation, and the message:
- The function is the
code.function.nameof the outermost span around the record; outside every span, the record's own. A record without one prints its target. - The context is the outermost span's fields:
component,operation, then the rest sorted by key,request_idprinted asreq. A record outside every span prints its owncomponent,operation, and fields there. - The nested operation is
↳with the name and fields of the span the record belongs to, when that span is not the outermost. - A span close prints
close,✓or✗, the reason it did not complete (error.type=…,panicked, orcancelled), thenbusyandidlewith three significant digits.✗follows astatus.codeofError, anyerror.type, or anoutcomeother thanokoracquiredthat the span recorded itself, as a failedindex.buildrecordserror. The close record states the samestatus.code, and so doestraces.span.metrics.callsfor atraced!operation, with the recordederror.typewhen it ispanic,cancelled,timeout,refused, or a registered error identity, and_OTHERotherwise. A lifecycle record prints its mark before its message:→forphase=start,✓foroutcome=ok,✗for any otheroutcome. Any other record prints its message, then its own fields sorted by key.
2026-10-05 11:57:37.429Z DEBUG rift_mcp::validation::run_index_supervisor_with component=index epoch=1 trigger=filesystem ↳ worker.run component=worker operation=worker.run work=filesystem index rebuild → index capture started component=index operation=index.build epoch=1 phase=start
2026-10-05 11:57:37.444Z INFO rift_mcp::validation::run_index_supervisor_with component=index epoch=1 trigger=filesystem ↳ index.build component=index changed_count=1 files_count=3 outcome=ok tree_revision=8f107d85 close ✓ busy=7.95ms idle=19.3µsA group is a run of records of one request; without a request, of one root span; without a
span, of one function. A blank line separates two groups. rift server logs pads each column
to the widest value of its group; stderr and rift server logs --follow, which cannot know a
group ahead, pad to fixed widths. Every surface prints UTC; stderr colors the level on a
terminal alone. Every character a terminal or a line reader acts on prints escaped as Rust's
char::escape_debug writes it (\u{1b}, \n, \u{202e}): C0 and C1 controls, DEL, the
line and paragraph separators, and the bidirectional overrides and isolates, so a value read
from a workspace or a request cannot move the cursor, end its line, or reorder what a reader
sees.
Every workspace crate other than rift-tracing emits through it: spans through traced!,
info_span!, and Span, events through its trace! to error!, elapsed time a caller acts on
through measure_elapsed!, whole functions through the #[rift_tracing::timed] attribute, and
test capture through ScopedRecorder.
The architecture check refuses a tracing, tracing-subscriber, log, OpenTelemetry, or
sysinfo dependency or import in any other crate, rift-tracing-macros included, and a normal
dependency that enables rift-tracing's fixtures feature, which compiles the test recorder
into a release build.
#[rift_tracing::timed("package.analyze", component = "dependency")] rewrites a function's body
into one traced! call: the block form for a function, the async form for an async fn. It
records no argument unless a field names it. The attribute lives in rift-tracing-macros, which
rift-tracing re-exports; it finds the consuming crate's name for rift-tracing through
proc-macro-crate, so a renamed dependency resolves too.
Every instrument is a static declaration over an OpenTelemetry instrument: a Counter,
Gauge, Histogram, or ObservableUpDownCounter with the InstrumentScope of the declaring
crate, its name, unit, and label keys, whose label values are &'static str literals. Its points
carry that crate's CARGO_PKG_NAME and CARGO_PKG_VERSION as their instrumentation scope, from
one meter per scope, at most SCOPES_MAX, 7; a scope past it records nothing, and one WARN
record, meter refused an instrumentation scope past its bound, names it. The
traces.span.metrics.* points of a traced! operation take the scope of the crate whose
traced! opened it. Synchronous recordings go directly to the OpenTelemetry SDK, which
aggregates and exports them. Observable reads sum identical labels in a bounded map for one
collection, then publish to the SDK and release that map. A process that installed no meter
records nothing, at the cost of two atomic reads per recording.
A view on the meter provider bounds every instrument at
[logs] cardinality_limit label sets,
and the SDK records a label set past that into one series carrying otel.metric.overflow. A test reads what the SDK exported through
ScopedRecorder::metrics, each series naming its scope, printed as otel.scope.name and
otel.scope.version. Every traced! operation records traces.span.metrics.calls and
traces.span.metrics.duration, in seconds, labeled with its span.name, its span.kind,
Internal, and its outcome as status.code, with error.type panic or cancelled when it did not finish. Work
whose value is Err(RiftError), the value of a block or the output of a future, records that
error's registered identity as error.type. An inline ? converts the error to the enclosing
function's error type before recording its registered identity; an inline return records the
returned error. Nested functions, closures, and async blocks retain their own return behavior.
Only ? and return written directly in the inline work are inspected; exits produced by
another macro expansion require explicit error.type recording.
The metrics follow
the operation, not its span: the duration is read once, at the operation's end, and the close
record's elapsed_ms states the same reading, so a span clone held elsewhere lengthens neither,
and the span's filters do not select it.
Every request rift server answers records mcp.server.operation.duration, and every request
rift mcp forwards to it records mcp.client.operation.duration, in seconds, bucketed at the
boundaries the MCP semantic conventions advise, from 10 ms to 300 s. A series names
mcp.method.name: tools/call, tools/list, resources/list, resources/templates/list, or
resources/read, and on the server also initialize and ping. A request the server answers
also names mcp.protocol.version, the version rmcp knows or _OTHER, and network.transport
tcp. A tool call also names gen_ai.tool.name, when the tool is get_symbol,
nodes, or search. A tool answered with isError records error.type tool_error; a
JSON-RPC error records its code as both error.type and rpc.response.status_code, such as
-32602, or error.type _OTHER for a code outside the JSON-RPC, MCP, and Rift codes.
rift server serves MCP statelessly through NeverSessionManager and records no session
duration.
Each .rift/index and .rift/vectors operation records db.client.operation.duration with
db.system.name sqlite, db.namespace, and db.operation.name; a failed one adds
error.type _OTHER and the SQLite primary result code as db.response.status_code, such as
5 for busy. A raw statement that starts with PRAGMA busy_timeout, PRAGMA wal_checkpoint(NOOP), or PRAGMA wal_checkpoint(TRUNCATE) records that text as its
db.operation.name, the open's checkpoint among them, and each connection the worker closes at
its stop records close. A
transaction records sqlite.transaction.duration by sqlite.transaction.result,
commit or rollback, and sqlite.transaction.statement.count. Each append to .rift/metrics
records sqlite.queue.wait.duration, sqlite.write_lock.wait.duration,
sqlite.transaction.duration, sqlite.commit.duration, and
sqlite.transaction.statement.count under db.namespace metrics, and its inserts and its trim
as insert and trim in db.client.operation.duration; each read connection records connect
and each page or count it reads query. The history store records connect, insert, trim,
query, and sqlite.transaction.statement.count under db.namespace history. The metrics
database's open records PRAGMA wal_checkpoint(TRUNCATE) and its close PRAGMA wal_checkpoint(NOOP) and
close in db.client.operation.duration.
The index supervisor counts index.rebuilds by index.rebuild.trigger: startup for the
preparation at start, filesystem for a rebuild that reads the paths its observation names,
rescan for one that reads every visible file again. A failed rebuild adds error.type. Each
native event counts watch.events by watch.event.kind and watch.event.route. An observation
whose signal the full invalidation channel refused counts index.invalidation.dropped under
event and writes no record.
Every build carries the OTLP export, and a process exports nothing until an operator sets an
endpoint variable. With OTEL_EXPORTER_OTLP_METRICS_ENDPOINT or OTEL_EXPORTER_OTLP_ENDPOINT
set, the process installs a meter whose reader exports over OTLP/HTTP protobuf every
OTEL_METRIC_EXPORT_INTERVAL, 60 s by default; spans export when OTEL_EXPORTER_OTLP_ENDPOINT is
set; log records export to /v1/logs when it or OTEL_EXPORTER_OTLP_LOGS_ENDPOINT is set,
batched every OTEL_BLRP_SCHEDULE_DELAY, 1 s by default. With none of them, the process builds no
exporter, so it never dials OTLP's default http://localhost:4318. OTEL_SDK_DISABLED=true, in any case, turns
every export off whatever endpoint is set, as the OpenTelemetry specification defines it. The
rift-dev test runner sets it in each test process, so a test that installs tracing in its own
process exports nothing, and the test harness removes it from every rift process it spawns.
An exported log record is the record the LogSink stores, for each event and each span close:
its level as the severity, its target as the instrumentation scope name, its message as the body,
and component, operation, and each member of its fields as an attribute, root_span and
nearest_span as maps. A record written inside a span carries that span's trace and span
identifiers, and a span close carries the closing span's own, when the span export admitted the
span.
One export takes at most OTEL_METRIC_EXPORT_TIMEOUT for metrics, OTEL_BSP_EXPORT_TIMEOUT for
spans, and OTEL_BLRP_EXPORT_TIMEOUT for log records, 2 s each when unset, and the SDK reports a
failed export on stderr. A refused or slow collector never grows the span or log record queue past
OTEL_BSP_MAX_QUEUE_SIZE or OTEL_BLRP_MAX_QUEUE_SIZE: a span or record the full queue refuses is
dropped. The SDK reports the first drop on stderr, and at shutdown the total, as dropped_spans of
BatchSpanProcessor.Shutdown and dropped_logs_count of BatchLogProcessor.LogsDropped; no
metric carries it. Every exported span, metric point, and log record carries the resource
attributes service.name (rift), service.instance.id,
a random UUID per process, process.pid, and service.version, the server's version, so a collector that receives from several servers
tells them apart.
Readings are observable instruments: the SDK runs their reads on its reader's task each time it
collects, so no task of Rift's ticks for them, and a process that installed no meter reads
nothing. The process readings go through sysinfo:
process.memory.usageandprocess.memory.virtual, in bytesprocess.cpu.time, the CPU time consumed, in seconds, with nocpu.mode, becausesysinforeports one totalprocess.disk.iobydisk.io.direction,readorwrite- the open file count:
process.unix.file_descriptor.count, orprocess.windows.handle.counton Windows
process.cpu.time and process.disk.io stay cumulative, and a collector derives rates from them.
A value the platform does not report stays absent and is never recorded as zero.
Retained cache entries report cache.entry.count by cache.name: WorkspaceContentCache,
symbol_documents, and prepared_resolution, plus the global client's CachedCapabilities,
CachedResolution, and CachedFailure. Each observation follows its owner, so clones share
one reading and the last owner removes it. A global client slot retains at most one entry.
Independent owners with identical labels contribute to one sum per collection. Expiry stops reuse but leaves the retained entry counted until replacement, clearing, or drop.
The Tokio readings load the counters of Tokio's stable RuntimeMetrics for the runtime that
installed the subscriber: tokio.runtime.worker.count, tokio.runtime.task.count,
tokio.runtime.global_queue.length, and the cumulative tokio.runtime.worker.busy.time and
tokio.runtime.worker.parks, summed over the workers. A busy time that grows by the whole
export interval while the global queue grows names a blocked runtime worker.
The owners of a quantity read it the same way. Each open .rift/index, .rift/vectors, and
.rift/metrics database reports sqlite.file.size for the database file and its write-ahead
log, from file metadata, as the history store's filler does under db.namespace history while
it holds the fill lock, sqlite.queue.length, the commands waiting for its worker, and
sqlite.page.count by sqlite.page.state, used or free, from the page and freelist counts in
the file header, which change at a checkpoint. The index and vectors databases also report
sqlite.transaction.active, the transactions begun on the worker's connections and not ended,
and count sqlite.connection.refusals, the connection requests refused at the connection bound,
and sqlite.connection.reaped, the connections removed after their driver connection dropped. The
blocking pool reports worker_pool.permit.count by worker_pool.permit.state, available or
used. One instrument runs at most OBSERVATIONS_MAX live reads, three databases for each of
the 64 workspaces a repository process admits; a read past that is refused, and the first refusal
per instrument is a WARN record naming it.
Work that has not finished is evidence too. The runtime keeps a table of operations in flight:
every span carrying an operation field, every lock wait, and every held lock, from its opening
to its close, with its name, kind (operation, wait, or held), component, lock name, parent
operation, and start. A field recorded on the span after it opened, such as the pid of a
child the operation started, joins the entry's fields with its latest value. The table holds at
most 1,024 entries and counts each one refused past
that bound as untracked.
rift_tracing::publish_in_flight(reason) writes the table as one operations in flight record,
and the panic hook writes it at WARN with the reason panic, right after its the server panicked record. The stall report, a task of its own, reads the table every quarter of
[logs] stall_delay, and at least every 5 s, and writes
one operations in flight past the stall delay record, at WARN, for the entries open longer
than stall_delay, each entry once. The table also feeds operation.active, the entries open at
each collection by span.name, and operation.untracked, the entries it refused.
A hold its holder keeps for as long as it runs, such as a server's shared
history.live lock and its history.fill lock, is declared with .lifelong(): the stall
record leaves it out, and every other publication lists it with "lifelong": true. Both records
list the open entries oldest first, with age_ms, as the JSON array in operations, keep it
within 4 KiB, and count what did not fit in left_out:
[
{"operation": "lexical.commit", "kind": "operation", "age_ms": 12043, "component": "lexical"},
{"operation": "lock.wait", "kind": "wait", "age_ms": 11980, "lock.name": "index.write",
"parent": "lexical.write_turn"},
{"operation": "lock.held", "kind": "held", "age_ms": 30211, "lock.name": "index.write",
"parent": "documentation.store"}
]traced!(component = .., operation = .., open = true, ..) also records its opening: one
operation opened event inside the operation's span, so the store holds the operation while it
runs and a killed server leaves it behind. lexical.commit opens this way.
rift_tracing::lock(name).acquire(future) wraps the caller's own acquisition future, such as
mutex.lock(), and polls it as the caller would, so the lock's queue order and cancellation
stay its own. A wait that does not acquire at its first poll opens a lock.wait span naming
lock.name, lock.mode, the waiter operation, and the holder operation, and closes it with
outcome acquired, refused, timeout, or cancelled. An acquisition that can fail, such as
the migration lock's attempts within the busy-wait budget, goes through acquire_fallible, whose
future answers the guard or a Refusal: Refusal::Refused ends the wait refused,
Refusal::Timeout ends it timeout, and neither opens a hold. The returned guard dereferences to
the wrapped guard and keeps a lock.held span open until it drops. Both feed lock.wait.duration
and lock.held.duration. The write turn of .rift/index and .rift/vectors is recorded as
index.write and vectors.write, so a lexical commit that waits for a documentation write shows
as a lock.wait entry under lexical.commit while the holder's lock.held entry names the
operation running its statements.
The published index is recorded as the lock index.snapshot: a publication holds it
exclusive, and a read holds it shared. Tokio's RwLock lets a queued write go first, so a
read that waits behind a held or queued publication shows as a lock.wait entry naming that
publication's operation.
The claim of .rift/server.lock is recorded as the lock server.election, exclusive and
lifelong: a refused claim ends its wait refused and records the holder's pid when
server.json names one. The hold closes when the last clone of the election guard drops; a
process that exits with a database thread still holding a clone ends it unrecorded. A probe that
finds the election held, or whose attempt the platform fails, writes no record and counts in
lock.wait.duration under lock.name server.election, lock.mode shared, and error.type
refused. A probe that finds the election free holds it shared until it releases the lock.
Shared ranking
rift-ranking states the retrieval decisions every index makes: the document shape and its
searchable fields, the corpus revision that says what those fields mean, bounded query
construction with its precise and broad renderers, identifier splitting and match classing, and
weighted reciprocal rank fusion over ordered identities. It also carries the corpus tokenizer's
split, so text read outside SQLite splits into the terms the store indexes.
Nothing in it opens a database, loads a model, or resolves a project path. The architecture
check refuses SQLite, candle, tokenizers, reqwest, rig-core, and Toasty in that crate's
dependency closure.
A PostgreSQL or REST adapter therefore consumes the same decisions and the same ordered
identities rather than copying the algorithms; resolving an identity to a declaration needs the
store that published it, and deciding which identities win does not.
Error registry
crates/rift-error/errors.toml defines concrete Rift failures and evidence fields. rift-error-codegen validates those
definitions and writes the errors module of rift-error: crates/rift-error/src/errors.rs holds the registry
constants and one pub mod line per namespace, and crates/rift-error/src/errors/<namespace>.rs holds one
__rift_error_definition! declaration per error in that namespace, carrying only its name, slug, message, action, and
fields. A namespace is the slug segment after rift., such as analysis or cli. Builder types, bounds, conversions, typestate, and setter
names are derived by the macros, not written into the file. The committed_registry_files_stay_within_generated_line_bound
test in rift-error-codegen holds each generated file under a fixed line bound.
__rift_error_definition!(
context7_malformed,
slug = "rift.analysis.context7_malformed",
message = "context7.json has an invalid JSON shape",
action = "correct context7.json shape and retry",
fields = {
file: required(path),
key: optional(string),
source: optional(source),
},
);Each field reads name: presence(kind, flags...). Presence is required or optional. The kind is the field's
type in errors.toml (string, bool, integer, unsigned, pid, port, path, duration), or its role
(source, cause). The flags are hidden for display = false and sensitive for sensitive = true.
The macros in crates/rift-error/src/definition expand each declaration into the builder. The kind table maps each
kind to the Rust bound a setter accepts and the ErrorValue conversion it applies. Each required field adds one builder
type parameter that starts at Unset and becomes Set when its setter runs; .error(), .fail(), and the
IntoRiftError implementation exist only once every parameter is Set. Each optional field adds .maybe_<field>(),
which takes Option<T> and leaves the builder type unchanged. The maybe_<field> and <Field>State names come from
rift-error-macros, which depends on proc_macro alone. A crate that reaches rift-error by path, such as the
generated-code compile test in rift-error-codegen, therefore builds it with --offline and no downloaded crates.
A source field retains an underlying Rust error and exposes it through std::error::Error::source.
A cause field retains another RiftError in that chain. The runtime also converts owned and
borrowed RiftError values, including shared Arc values, without dropping their source chains.
Sensitive evidence is redacted in rendered output.
The registry does not contain transport codes. The MCP adapter maps each slug to its wire code and
retry directive, then supplies the request phase with .tool_error(phase). Calling .fail() on
that representation returns its JSON-RPC error. cargo xtask errors generate updates generated
Rust. The generator writes the declarations as text in the layout above, which rustfmt leaves
unchanged, and parses the result before returning it. cargo xtask errors generate also removes any file under crates/rift-error/src/errors/ that the registry no longer
generates. cargo xtask errors check compares generated output without writing it and fails on a differing, missing,
or unexpected file; CI runs that check through just generate-check.
Vector search
Since v0.0.16 · Rust, JavaScript, TypeScript, Markdown, JSON, YAML, TOML, Pythonrift-search owns the tier. The default runs the encoder in the server's own process, so
answering a query touches neither the network nor a subprocess.
Every model reaches the tier through one contract:
rig-core's EmbeddingModel, pinned at =0.42.0. One
enum holds the concrete pairs the server can run, and each method matches once and hands the
concrete model to a generic helper, so no boxed model callback and no untyped provider map
exists. Adding a family changes the loader and its adapter tests; search, fusion, SQLite,
and MCP keep calling the same method.
A local model is two handles over one loaded encoder: documents embed through one and queries through the other. A BERT retrieval checkpoint prefixes the query with its own instruction and leaves documents alone, while a static model treats the two alike, so the difference lives in the handles rather than in a flag every caller has to remember. Acquiring the files, reading them, and detecting the family all finish before a handle exists, because the contract's own constructor cannot fail.
The openai_compatible arm builds on the same contract. Rig's client carries the credential
the api_key_env variable named, the configured base URL, and an HTTP client holding the
configured request timeout; Rift issues the /embeddings request and decodes the response
itself. Rig 0.42.0 parses each response index and then zips the response array with the
input array by position, so a service answering out of order would attach every vector to the
wrong declaration. Rift's contract makes array order meaningless: the declared index decides,
and a duplicate index, a missing one, or one past the request's own length is refused rather
than mis-associated. The mock-endpoint suite serves reversed and malformed responses to prove
it, and makes no external request.
Batching is Rift's, not the library's. Rig's own batch builder fixes its chunk size to the
model's MAX_DOCUMENTS and derives its own concurrency, so it cannot honor both batch_inputs
and max_in_flight. The scheduler here cuts each batch to the lower of batch_inputs and what
the model accepts, runs one local batch at a time, allows at most max_in_flight remote
requests, and returns every completed batch to the input slots it came from.
The model's own config.json decides which of two encoders reads it. A model_type of
model2vec is a static model: the weights are one table of vectors, and the encoder embeds
a declaration by gathering the row each of its tokens addresses and averaging them. Anything
else is read as a BERT checkpoint through
candle, pooled from the CLS position, and a
retrieval query carries the prefix that checkpoint was trained with. A static model runs no
forward pass, so neither the prefix nor the pooling applies to it.
The shipped default is static. A workspace is embedded once before its first search can rank by vector, and a forward pass over every declaration costs minutes on a laptop CPU where a row gather costs seconds.
Weights are read through candle's safetensors loader rather than its memory-mapped
constructor: mapping is an unsafe call, and this workspace forbids authored unsafe. The cost
is that a load holds the file's bytes once while the tensors are built.
Every encoder call runs on a blocking thread. Candle's forward pass would otherwise hold a
runtime worker for the length of a batch, and a request arriving meanwhile would wait behind
it. Passes run one at a time, bounded by
batch_inputs and max_tokens. A BERT checkpoint's
attention memory grows with the square of the token window, and candle's kernels already
spread one pass across the thread pool, so running several at once buys nothing and costs the
machine its memory.
The vectors go in semantic_vectors, in the vectors database at .rift/vectors. A row is addressed by the
embedding space that produced it and the digest of the text it came from, so a declaration that
moves or is renamed resolves to the row already stored and a refresh embeds only what changed.
The space folds in the source kind, the endpoint origin for a remote service, the model and its
resolved revision, the width, and the document and query transformations, and nothing else: a
credential never reaches it. Changing any of those mints another identity, and the rows the
previous one wrote are dropped rather than scored against.
Rig returns double-precision coordinates. The stored format stays single precision, which is what the local index already holds and what a later global store would carry, so the conversion happens once at the boundary and a coordinate the narrower format cannot hold is refused there.
A pass publishes its vectors, the map that places each of them on a unit, and the tree revision they were described for as one value. A query reads all three or none: a corpus described for a tree other than the one the request captured ranks nothing, so a workspace published moments ago is answered by the full-text tier alone until the pass for that tree lands. Values are little-endian single precision with no header, so a row is a slice; a row whose stored width differs from the loaded model's is skipped, which invalidates a checkpoint that changed shape rather than reading it as noise.
Two sweeps keep the table bounded: one drops what the previous model wrote when
model changes, the other drops digests no live declaration
addresses. max_vectors bounds what the table may hold at all.
The vector table joins the lexical index's migration set rather than opening a second database
handle with its own. Toasty records applied migrations in one __toasty_migrations table per
file, so two independent sets on one file would interleave their ids; one set covers both
tiers, and each store applies the same idempotent set whatever order the two open in.
Language providers
Since v0.0.23A provider composition is a typed pipeline of named stages - source resolvers, fact providers, and transforms - connected through declared input and output flows, with keyed joins for revision-keyed history. The Rust builder inserts, replaces, branches, merges, and nests stages. Rift validates types, capabilities, and cycles at build, then owns scheduling, caching, and invalidation. Composition controls which providers run and how data reaches them; it cannot establish Contribution equivalence.
Each provider publishes one immutable, revisioned Contribution collection. Normalization selects
Contributions applicable to a captured index revision, validates identity evidence, and builds
workspace-local SymbolRecord values. Reads assemble a Symbol from one record and its
Contributions. Provider order selects presentation fields only after normalization establishes
identity.
Syntax providers wrap tree-sitter grammars behind a Rift-owned adapter: grammar node kinds resolve to numeric ids once, and raw query and cursor APIs never leave the syntax crate.
The registry itself assembles from one shipped definition list in rift-syntax: each entry
names a language's identity, the file extensions it claims, and the provider parsing it, so
adding a language is one definition plus one list entry.
HTML preserves elements, attributes, and source references as syntax nodes. CSS publishes selectors, properties, custom properties, and at-rules as declarations; import and URL syntax retain their authored ranges. Vue and Svelte keep component structure and template expressions, including Vue directives and Svelte control-flow blocks.
HTML, Vue, Svelte, and Angular providers parse supported script and style blocks through Tree-sitter included ranges. A TypeScript script retains its TypeScript signature identity, while the containing document keeps its HTML or component identity. Included parses share the original source bytes and one aggregate node and depth bound, so a script symbol's range still addresses its containing file.
Angular template syntax uses the html:angular identity. Component ownership comes from a
class decorator whose Component binding is imported from @angular/core, including named
aliases and namespace imports. Literal template properties retain their original TypeScript
byte ranges. A parameter or local variable or function binding with the imported name refuses
ownership inside its scope; type annotations and default-value references do not create bindings. Literal
templateUrl properties retain their authored paths. Dynamic and escaped
template values remain unresolved expressions. The TypeScript class uses the existing provider.
An Angular template publishes structural nodes for bindings, interpolation, pipes, and control flow. A template with no script or style declarations publishes no symbols. Script and style blocks publish declarations through their included providers.
Each provider exposes parser errors through has_errors and node error flags, including
missing nodes and grammar recovery. Authored source references retain their syntax ranges.
Compiler and framework semantics remain outside these providers.
Framework context follows the nearest captured package.json. A runtime Component import
owns literal inline templates and external templates inside that component's package; an
unrelated HTML file keeps ordinary HTML syntax. Dynamic or escaped templates, missing templates,
and template sources outside the component package produce framework_context_unresolved warnings.
Tailwind context requires an exact supported package version or an npm lockfile version, plus
applicable configuration or stylesheet evidence. A captured npm lockfile pin takes precedence
over the declared version; an invalid or unsupported pin leaves context unresolved. Version 3
uses tailwind.config.* or captured
@tailwind syntax; version 4 uses a captured CSS import of tailwindcss. Explicit
providers.syntax.frameworks entries supply context
when captured evidence leaves it unresolved.
Under that context, static HTML, JSX, Angular, Vue, and Svelte classes publish authored utility
and variant names. CSS publishes @apply names, version 3 custom utilities inside
@layer utilities, and version 4 @utility, @custom-variant, and @theme names. Configuration,
stylesheet, and theme references retain their authored ranges. Dynamic class expressions keep
their original syntax and produce warnings; Rift does not choose a conditional branch or
construct names from an expression. These facts do not establish whether a compiler generates
a utility.
Context applies after raw parsing. Raw cached facts keep their parser identity and source
digest; changed package, component, configuration, or stylesheet evidence rebuilds framework
facts over the current bytes. SyntaxFacts records SyntaxOrigin::Provider for raw facts and
SyntaxOrigin::Framework for contextual facts. SyntaxFactsParts retains that origin through
checked restoration; package raw reuse accepts only provider origin, including raw embedded
HTML, Vue, and Svelte facts. Included template parsing and added declarations share the
source, node, and depth bounds. A contextual HTML template uses html:angular; TypeScript,
Vue, Svelte, CSS, and JSX files keep their existing identities.
C and C++ publish functions, types, variables, macros, and include syntax. Function prototypes
publish function; function pointer and array variables publish variable. The provider follows the
declarator modifier nearest the declared name, so a function returning a pointer remains a
function. Several names in one declaration retain separate symbols and name ranges; each source
range covers the complete declaration. C++ also publishes namespaces and method prototypes.
The native walk records each declaration's additional declarator IDs once per parse, bounded by
the captured nodes; repeated declaration hooks reuse those IDs.
Cython publishes Python and Cython declarations under the cython identity for
.pyx, .pxd, and .pxi files. JSONC has its own jsonc identity and accepts comments;
the existing JSON provider retains its behavior.
Stored facts validate each node kind against the host grammar or its supported included grammars. Restoration retains the host identity, original source digest, embedded signature identity, and configured parsing bounds.
Since v0.0.46 · JavaScript, TypeScriptThe JavaScript and TypeScript providers publish a declaration's attached JSDoc as its
documentation. The walk starts at the statement carrying the declaration - an export, or the
const, let, or var statement it is the first declarator of - and the declaration's range
starts there too, so the JSDoc sits inside the range get_symbol answers. Several blocks join
with a line break, @ tags stay as written, and the /**, */, and leading * markers are
stripped; a // or /* */ comment or a blank line ends the walk.
Dependency context
Since v0.0.46rift-dependency holds the resolvers and the standard library pass. The resolvers read through a
reader that offers a file read and a directory listing, so a resolver cannot start a process.
Besides entries, the context records where each pinned package is installed on this machine: a
Cargo package's <name>-<version> folder minted from the lockfile, an npm package's
node_modules/<name> folder, nested copies included, the import folders a Python distribution's
.dist-info/RECORD lists, and the Rust standard library below rustc --print sysroot. Those
folders never reach the global API.
Global resolution retains one latest dependency context and its checked request bytes. The cache key compares the published dependency context, requested packages, and advertised entry bound; repeated reads reuse the bounded context even when the global API accepts fewer entries. The client compares a full request digest and analyzer and corpus revisions before reusing its answer.
The standard library pass runs rustup --version, rustc --version, rustc --print sysroot,
and node --version through the server's bounded command runner, under
[dependencies] resolution = "auto" alone. Every
probe runs with RUSTUP_AUTO_INSTALL=0 and MISE_AUTO_INSTALL=0 laid over its environment and
RUSTUP_TOOLCHAIN removed from it. A rustup proxy or a mise shim installs the toolchain a pin
names when it runs in that pin's folder, and RUSTUP_TOOLCHAIN beats the project's toolchain
file.
rustup honors RUSTUP_AUTO_INSTALL from 1.28.1, so rustup --version runs first from the
filesystem root, where no toolchain file applies. Below 1.28.1, or when its first line names no
version, rustc does not run and the entry keeps its static reading; the output decides, because
rustup --version exits 0 whether or not it reached the active toolchain. Where no rustup
starts, as beside a distribution's rustc, rustc --version runs directly.
Package analysis
Since v0.0.39One package's source reaches the global index through package analysis in rift-analysis, which
holds no I/O: it takes bytes a caller already read, parses them with the shipped syntax
providers, places each file under rift://source/<manager>/<name>@<version>/<path>, assembles
the declarations, and renders one PackagePublication. That publication is the logical package
index global ingestion consumes. The local server runs no package analysis; its package facts
come from the global index.
ExactPackageLimits::with_declarations sets the declaration bound for semantic assembly;
stub and implementation declarations both count.
ExactPackageLimits::from_configuration validates the configuration and uses the file, aggregate
byte, declaration, and relationship bounds from [source].
The adapter also applies per-file parsing bounds from
[providers.syntax], documentation collection
bounds from [documentation], and publication
bounds from [package].
Callers accept the configuration through rift_core::acceptance::accept_configuration to apply
the same environment overrides as the local server, such as RIFT_SOURCE_DECLARATIONS.
ExactPackageLimits::new keeps the provider's 100,000-declaration default unless the caller
sets an override. The configuration adapter leaves file selection to the caller.
package.retained_source bounds each retained source string; package.retained_total, when set,
bounds their aggregate bytes. Units, symbols, and search documents each count their own source
copy, and the analyzer checks the aggregate before allocation.
ArchiveLimits::from_configuration applies [package.archive] when the caller decodes an archive
before constructing package input. Archive admission and package input each enforce their selected
bounds; increasing compressed bytes alone does not raise expanded bytes or declaration capacity.
File selection decides which files of a package the analysis reads, from package-relative paths alone. It keeps:
- the source files the package language's definitions claim:
.rsfor Rust,.pyand.pyifor Python, and every JavaScript and TypeScript extension for a TypeScript package, whose builds ship.js,.mjs, and.cjsbeside their declaration files - the documentation files the workspace's own documentation selection keeps, narrowed by a
package's
context7.json
It skips node_modules, __pycache__, target, and .git, and the files a caller's
[source] exclude patterns match, through the matcher
the workspace's [source] table uses. Tests, examples, and benchmarks stay, since they show how a
package is used.
Package selection also accepts HTML, Angular, CSS, Vue, Svelte, C, C++, Cython, and JSONC.
TypeScript packages include HTML, CSS, Vue, and Svelte sources beside JavaScript and TypeScript;
Python packages include Cython sources. A C++ package selects .h as C++, and an Angular package
selects .html and .htm as html:angular; ordinary workspace selection keeps .h as C and
.html as HTML.
ExactPackageInput::with_framework_context adds captured metadata and explicit framework
selections. Selected sources already belong to context; additional metadata paths must be
distinct. The combined sources share package file and aggregate byte bounds. Metadata informs
analysis without becoming a published source unit. PackageAnalysis::warnings reports unresolved
framework context and dynamic expressions. Fresh, supplied, and checked restored raw syntax
produce the same contextual publication; files with framework context parse current source before applying framework facts.
The stub and module join pairs a stub with the module it declares: mod.pyi with mod.py,
index.d.ts with index.js, .d.mts with .mjs, and .d.cts with .cjs. A .d.ts beside no
.js pairs with its .mjs, then its .cjs, so an ESM build claims a stub before a bundled one.
Each name both files declare answers one symbol, addressed at the module: its source, range, and
documentation come from the module, and its signatures and types from the stub, with the stub's
overload forms gathered into one signatures list.
A declaration counts as public by its language's rule:
- Rust -
pubdeclarations, and the items of apubtrait and the variants of apubenum - Python - names without a leading underscore
- TypeScript - every member but
privateandprotectedones - JavaScript - the declarations an
exportwraps or names, includingmodule.exportsandexports.nameassignments, and the members of an exported class other than a#private element
In a paired module the stub defines the public set, and a stub declaration with no implementation answers from the stub alone.
Two publications are comparable by their own fields. format_revision states the shape;
analyzer_revision is the digest of the analyzer manifest, which names each shipped grammar's
version and checksum and the content digest of every source file that decides what a
publication holds: all Rust source in rift-core, rift-analysis, rift-index, rift-provider,
rift-protocol, rift-ranking, and rift-syntax, plus the history repository, MCP server and
validation, and server read implementations.
Every record - a source unit, a declaration, a search document - carries a digest over its own
content, and the whole document renders as
RFC 8785 canonical JSON, so two runs over the same
bytes under the same analyzer revision produce byte-identical output, and an adapter can compute
added, replaced, and removed records by identity and digest alone. A publication addresses source
by package-relative path and source unit, so no machine's directory layout reaches a consumer.
The analyzer manifest is generated beside the served schemas and committed at
crates/rift-analysis/src/analyzer-manifest.json, so a change to any of those files
that ships without a regenerated manifest fails just generate-check. The publication's own schema is
exported to package-index.schema.json.
Language engines
The rift-lsp crate owns LSP sessions. A session opens source documents, requests references
and call hierarchy, and pulls diagnostics when the language server advertises that capability.
It also retains published diagnostics under per-document and document-count bounds.
The reference request includes the declaration's own occurrence. An empty answer can therefore mean the engine has not loaded the file; it cannot prove the declaration has no references.
A call hierarchy answer carries no such evidence, so a walk reads the session's readiness alone.
That readiness leaves out rust-analyzer's cargo check progress, which resolves nothing a walk
reads, while a diagnostics read still waits for it. An embedded engine answers from its own
in-memory state and declares itself ready.
The server records each publication's changed files per engine and sends them to a running
session as one workspace/didChangeWatchedFiles batch before its next request. When more
changed files are owed than the record holds, the server replaces the session at its next request
instead, and the new session reads every file from disk.
Each process has configured startup and request timeouts. The
[lsp] configuration also bounds retries
and process replacements.
LSP startup records engine child spawned with the child's pid and its piped stdin,
stdout, and stderr. Shutdown records engine.child.wait while waiting for exit;
a timeout or failed wait records its outcome and enters engine.child.kill to kill and reap
the child. engine child exited retains the pid observed before the wait or kill.
The bounded command runner records child spawned, an open process.wait operation, and
child exited, with pid, stream destinations, and the observed exit_code when available.
The wait records ok, refused, timeout, or cancelled, so a stalled dependency probe
remains visible in the operations in flight.
Configuration
rift.toml at the workspace root is accepted at startup and re-accepted on change. Its schema
is exported beside the MCP surface as rift.schema.json.
A quantity value carries its unit: timeout = "120s".
The configuration page states the keys and defaults.