RAG Knowledge Assistant

01What does it search — and do documents arrive with permissions attached?

Connectors ingest wikis, tickets, and contracts, preserving source, owner, version, and the ACL — a document with no permission metadata cannot be answered from safely, so permissions are captured at ingestion, not reconstructed later.

02How does it find the right passage — meaning, keywords, or both?

Both, always: semantic search catches paraphrase, the keyword index catches exact terms (error codes, contract numbers, acronyms), and metadata filters narrow by source or date — merged candidates pass a reranker before anything reaches the model.

03Can the assistant just answer from what the model already knows?

No: every claim in an answer cites the retrieved chunk it came from; when retrieval comes back weak, the assistant says so and shows the closest sources instead of improvising — no citation, no ship.

04Two employees ask the same question — same answer?

Not necessarily: retrieval carries each user’s permissions inside the index query, so an engineer and a lawyer asking about the same deal get answers built only from the documents each one can open.

05Someone edits — or deletes — a document. When do answers reflect it?

Within minutes: content hashes detect the change, only the affected chunks re-embed, and deletions tombstone across every index immediately — a deleted contract stops appearing in answers on a compliance clock, not at the next crawl.

06What happens when the assistant gets it wrong?

Users flag bad answers and stale sources in one click; flagged cases replay into the eval set, and repeated flags on one document trigger a reindex or an owner ping — feedback feeds the pipeline, not a dashboard nobody reads.

01What is the worst failure — a wrong answer, or a leaked one?

A leaked one: a wrong answer wastes minutes, a restricted contract surfacing to the wrong employee is a security incident — so the ACL filter runs inside the index query and is tested like an invariant, not a feature.

02How fast should an answer feel?

First streamed token in about 2 seconds: roughly 300 ms retrieval, 200 ms rerank, 1.5 s to first model token — each stage owns its budget, so "the assistant is slow" is never the whole diagnosis.

03How stale is acceptable?

Edits visible in answers within minutes and deletions guaranteed gone within the compliance window — freshness is a promise per source tier, not a best effort.

04An answer is challenged a month later — can we reconstruct it?

Yes: every answer stores its trace — query, retrieved chunks with scores, rerank order, prompt, model version, citations — so a dispute replays exactly, and the eval pipeline mines the same traces.

05How do we know quality moved — up or down?

A standing eval suite scores retrieval recall, groundedness, and citation accuracy separately on every pipeline change — a chunking tweak that silently drops recall@10 must fail a release gate, not a customer.