Nexus

Nexus is the ecosystem’s knowledge base. It answers questions about your organization’s knowledge (documents, policies, configs) and its operational reality (OpenTelemetry traces) only from evidence it can cite. Every answer carries a confidence score and a link back to the source chunk or trace it came from.

Ordinary RAG retrieves text and lets the model improvise, so it returns a plausible answer whether or not it has grounds. Nexus works the other way around: the system decides what can be retrieved and whether an answer is supported, and the model only writes over evidence that already exists. If nothing can be cited, Nexus returns no answer.

In short: enterprise retrieval for grounded answers. It’s the context layer for code-review and troubleshooting agents, so they work from real documents and observed telemetry rather than guesses.

Core concepts

Hybrid search. Two retrievers run in parallel and fuse with RRF (Reciprocal Rank Fusion, k=60): BM25 over Korean morphology (mecab-ko, so particles and endings are stripped correctly) and vector search over pgvector. Fused results are then capped per document so one file cannot flood the answer.
Graph is context, not ranking. A 2-hop entity traversal runs on graph-enabled routes, but it happens after fusion and never contributes to a hit’s score. Its edges are returned alongside the answer, not blended into the ranking.
Citations are verified, not trusted. The model is asked to cite, and then the citations are checked in code against the evidence packet it was actually given. Anything that does not resolve is reported separately as unverified rather than being passed off as a source. Numbers in the answer are checked the same way.
Evidence-driven edges. No relationship (edge) exists without evidence. Every edge is bound to a source chunk or a trace query reference.
Dual knowledge layer — Designed vs. Observed. Relationships extracted from design documents (CALLS, PUBLISHES) live alongside relationships observed in real traces (CALLS_OBSERVED, with call counts, error rates, latency).
Design-Observation diff. Nexus flags doc_only (documented but never observed — dead docs), observed_only (observed but undocumented — shadow dependencies), and conflict (both present but mismatched).
Default-deny security. PII/secrets (Korean SSN, card numbers, AWS keys, JWTs) are quarantined on detection and never indexed; every query is filtered by classification (PUBLIC < INTERNAL < RESTRICTED).
Provenance tier. A chunk written by a person and a chunk a model read out of a screenshot are not the same kind of evidence. Machine-read text is labelled as such, and the label travels all the way into the prompt, the API response, and the web UI — so an answer never quietly launders an extraction into an authored policy.
Declared index generations. The embedding model, its dimension, and the vector column move together as one generation, declared append-only in the database. Ingestion is refused before a single document is collected if the running configuration does not match a declared generation — the failure mode this closes is a documented command silently writing into a column nothing searches. The repository default is nomic-embed-text (768-d); a KURE-v1 (1024-d) generation is available and selected per deployment.
Section fill — retrieval picks the document, then fills in the rest of it. Retrieval decides two things: which document, and which passage inside it. Measurement showed the first is reliable and the second is not — a question and the passage that answers it can share no vocabulary at all, in which case no ranking reaches that passage. So when one document saturates the per-document cap, the remaining sections of that same document are added to the evidence, not to the ranking: recall and top-1 are byte-identical, only the packet grows. On by default in the shipped configuration, off in the code default so that a caller without configuration does not touch the database.
A correction is fetched with the thing it corrects. A document that supersedes another often ranks below it: the older one is longer, better matched, and says the thing being asked about at length. So after the first pass, the names the evidence used are searched again with change vocabulary attached, and any correcting document is pulled in by identity rather than by rank. On a treatment set this moved 0/5 to 5/5 with no control label harmed. The cost is stated rather than hidden: it is a second retrieval, and it roughly triples search latency — so it runs only on the answer path, never on a search-only route where a user would watch it. The code default is off; the shipped configuration turns it on.
A design and its implementation plan arrive together. Two documents describe one piece of work — the design holds why and what, the plan holds which files and in what order — so a question spanning both is answered by half the evidence if only one is retrieved. The pairing is not inferred: it is already in the filenames, and matches 27 pairs in the live corpus with no model call and no judgement. Measured at 0/5 to 4/5 on the treatment set, three control labels unchanged, +3 ms, and evidence volume up 26–134 % — that growth is the point and also the risk. It does nothing at all in a corpus without such pairs, and the controls were checked at this scale only.
The code’s current value is shown beside the document’s. Where a claim about a value is bound to a symbol, the code is read at answer time and both numbers are put in front of the reader — a document saying 30 characters next to a server that enforces @Size(max = 100). ⛔ The system does not decide which is right. A claim attaches only when every one of its concepts appears in the question, which is deliberately narrow: failing to attach leaves the answer as it is today, while attaching wrongly makes it false. Widening that rule was proposed and then refused on measurement — the near-misses turned out not to be spelling variants but questions about a different attribute, and the low attach rate traces to how few claims exist, not to the rule. Only the file a claim already names is read; walking the tree costs 50 seconds in this deployment.
Evidence fit — the system knows when it found nothing good. Rank fusion scores by 1/(k + rank), which carries rank and discards magnitude, so a well-matched query and an off-corpus one came out indistinguishable. Both retrieval paths already compute magnitude to sort by; it is now kept. When both are weak the narration contract changes — the answer says up front that the question is outside what the evidence covers, and stays short. It does not block: a wrong threshold costs brevity, not a withheld answer.
Document debt travels with the evidence. If a cited document has been superseded, or shares its title with another active document — which makes a [source: title] citation ambiguous — the answer is told. Only what is deterministically recorded qualifies. Semantic contradiction between documents is deliberately not judged here: doing it needs a judge model, and the evidence for that path is poor enough that the system describes conflicts without vouching for them.
Code anchors — the names a document calls, checked against the code as it is now. Ingestion binds back-ticked symbols in each chunk to the code index. At answer time the packet reports how many of the names a cited paragraph uses still exist, and which do not, resolved in a single set query rather than one lookup per anchor. A document that says FooService after FooService was deleted stops being quietly authoritative.
Follow-up questions are rewritten conservatively, or not at all. “So when does that start?” has no content words. With conversation history present, the query is rewritten — but only four edits are permitted (fill in a pronoun, drop a formatting request, carry a fact the user supplied, drop a source restriction), and the original query is always kept as its own retrieval channel so a bad rewrite has a floor under it. With no history the model is not called at all.
Staleness is shown, not enforced. Answers carry a per-snippet age warning against a per-type TTL. It is a label for the reader; it deliberately does not re-rank or exclude anything.
Deterministic abstention. If nothing can be cited, Nexus does not call the model at all — it returns a fixed statement that it has no evidence, and says so in the response payload.
Index, not storage. Originals stay in Git and in Tempo. Nexus stores only derived data — chunks, embeddings, graph edges.

What happens between the ranked list and the answer you read

Ranking is the visible half. The half that decides whether an answer is trustworthy happens after it: one assembly point attaches everything the reader needs to judge the evidence, and one verification pass checks the model’s output against the evidence it was actually handed.

Two properties of this stage are worth stating plainly, because both were bought with defects:

Everything attaches at one point. Four surfaces consume the packet — two HTTP endpoints, the agent protocol, and the CLI. When something was attached per-surface instead, one of them silently missed it and a person and an agent got different answers to the same question.
Weak evidence changes the narration, it does not block it. The fit check fires only when both retrieval paths are weak, and its effect is a shorter answer that states its own scope. A blocking design would make a wrong threshold cost a withheld answer; this way it costs brevity. See the engineering log for the measurement behind the thresholds and their known limits.

Quickstart

Nexus runs as a Docker Compose stack. By default only the core containers start (PostgreSQL + Ollama + the FastAPI app); the OTel observability pipeline is opt-in. The task one-liners below wrap the underlying docker compose commands — each step shows the raw equivalent too.

Prerequisites

Docker Desktop
(Optional) go-task for the task shortcuts
(Optional) an Anthropic API key, for LLM grounded-answer generation

1. Clone & configure

git clone https://github.com/LivingLikeKrillin/khala.git
cd khala
cp nexus/.env.example nexus/.env
# (optional) set ANTHROPIC_API_KEY in nexus/.env for LLM answer generation

2. Start — one line

task up

task up starts the containers (waiting for health), applies DB migrations, and pulls the embedding model automatically. No Task? Run those three yourself from nexus/:

docker compose up -d --wait                                  # containers + model auto-pull
docker compose exec -T nexus-app python -m scripts.migrate   # ← skip this and the source console / document management break

Starts PostgreSQL 16 + pgvector (5432), Ollama (11434), and the FastAPI app on 8000. The OTel collector + Tempo are opt-in — add them only for trace aggregation: docker compose --profile observability up -d.

4. Index documents & search

Open http://localhost:8000/ and ask in the chat — or use the CLI:

docker compose exec nexus-app nexus ingest ./docs
docker compose exec nexus-app nexus query "payment service dependencies"

The Web UI is served directly from FastAPI at http://localhost:8000/ — no build step. New to it? See Using the Nexus web app.

Update / stop

git pull && task update    # rebuild image, restart, apply DB migrations
task down                  # stop (or: docker compose down)

How-to

Get a grounded answer (with evidence)

curl -X POST http://localhost:8000/search/answer \
  -H "Content-Type: application/json" \
  -d '{"query": "which services does the payment service call?", "clearance": "INTERNAL"}'

The response includes evidence snippets with source URIs and provenance. Use /search/answer/stream for SSE streaming in the chat UI.

Explore the knowledge graph

docker compose exec nexus-app nexus graph payment-service        # 1-hop
docker compose exec nexus-app nexus graph payment-service -h 2   # 2-hop

Or via the API: GET /graph/{entity} resolves by name or rid.

Find design-vs-observation drift

docker compose exec nexus-app nexus otel-aggregate   # roll traces up into CALLS_OBSERVED
docker compose exec nexus-app nexus diff             # report doc_only / observed_only / conflict

The same report is available at GET /diff.

Reference

Source repo README: github.com/LivingLikeKrillin/khala (README.md)
API contract, pipeline, MCP server, Slack bot, and UI integration docs live under that repo’s docs/ (API_CONTRACT.md, PIPELINE_SPEC.md, MCP_SERVER.md, SLACK_BOT.md, UI_INTEGRATION.md).
MCP server exposes nine tools — nexus_search, nexus_answer, nexus_graph, nexus_suggest, nexus_diff, nexus_status, nexus_supersede, archon_claim_value, archon_grade_authority — via python -m nexus.mcp. Set NEXUS_MCP_TOKEN, or every call returns 401 (auth defaults to enforced).