Open to AI engineering rolesRun the live demo →
//02Engineering

how Provenance earns permission to act.

A support ticket moves through eight explicit stages. Each stage narrows what the system is allowed to do, and every decision remains visible after the run.

Permission modelEight guarded decisions
Live system
System signal8 stagesinspectable at every step
  1. 01Screen
  2. 02Classify
  3. 03Retrieve
  4. 04Generate
  5. 05Verify
  6. 06Route
  7. 07Notify
  8. 08Audit
01
Control

Policy ingestion + versioning

Approved Markdown is split by heading, chunked, embedded, and upserted into Postgres. Each run receives a corpus version that follows every citation.

INGEST PIPELINERUN 12FPOLICY.MDPOSTGRESCORPUS V.1252 passages indexedstable IDsREADY
Implementation note +

Approved Markdown is split by heading, chunked, embedded, and upserted into Postgres. Each run receives a corpus version that follows every citation. This behavior is exercised by the guided demo and recorded in its audit history.

02
Control

Input security before spend

A fast deny-list and a small classifier separate genuine questions from off-topic or manipulative requests before retrieval, generation, or a costly database write.

Incoming ticketDENY-LISTCLASSIFY✓Safe to continue✕Blocked pre-spend
Implementation note +

A fast deny-list and a small classifier separate genuine questions from off-topic or manipulative requests before retrieval, generation, or a costly database write. This behavior is exercised by the guided demo and recorded in its audit history.

03
Reasoning

Transparent retrieval

The ticket is embedded and compared with every indexed passage. Ranked passages return with similarity scores and stable IDs; retrieval alone never decides that a question is answered.

Q.81.74.69.6152 INDEXED PASSAGES
Implementation note +

The ticket is embedded and compared with every indexed passage. Ranked passages return with similarity scores and stable IDs; retrieval alone never decides that a question is answered. This behavior is exercised by the guided demo and recorded in its audit history.

04
Reasoning

Evidence-bound generation

The model drafts only from retrieved passages, leaves the answer empty when support is missing, and is explicitly warned not to conflate adjacent policy concepts.

GENERATEEMPTY
Implementation note +

The model drafts only from retrieved passages, leaves the answer empty when support is missing, and is explicitly warned not to conflate adjacent policy concepts. This behavior is exercised by the guided demo and recorded in its audit history.

05
Reasoning

Claim-level verification

The draft is decomposed into factual claims. Each claim receives an entailment score plus a lexical sanity check so one weak statement cannot hide inside an otherwise grounded answer.

✓✓✓✓0.96MEAN ≥ 0.70
Implementation note +

The draft is decomposed into factual claims. Each claim receives an entailment score plus a lexical sanity check so one weak statement cannot hide inside an otherwise grounded answer. This behavior is exercised by the guided demo and recorded in its audit history.

06
Operation

Three responsible routes

Verified drafts become cited answers. Missing or ambiguous evidence goes to human review. Unsafe instructions remain blocked before the answer pipeline begins.

?Answer with citationsHuman reviewBlocked safely
Implementation note +

Verified drafts become cited answers. Missing or ambiguous evidence goes to human review. Unsafe instructions remain blocked before the answer pipeline begins. This behavior is exercised by the guided demo and recorded in its audit history.

07
Operation

Ticketing + human handoff

Customer sending is simulated, while the inbox and optional Slack notification carry evidence, reason, and approve or reject controls into the operator workflow.

INBOXNeeds reviewApproveReject
Implementation note +

Customer sending is simulated, while the inbox and optional Slack notification carry evidence, reason, and approve or reject controls into the operator workflow. This behavior is exercised by the guided demo and recorded in its audit history.

08
Operation

Audit + evaluation

Every stage writes a persisted event. A committed 45-case development suite runs the same pipeline and reports route accuracy, false refusal, and fabrication by category.

screenretrieveverifyrouteDEV SET45/45PASSING
Implementation note +

Every stage writes a persisted event. A committed 45-case development suite runs the same pipeline and reports route accuracy, false refusal, and fabrication by category. This behavior is exercised by the guided demo and recorded in its audit history.

09 / Decision gates

thresholds turn model confidence into an operational route.

GateFailure conditionRoute
Input screeningUnsafe or manipulative intentBlock before generation
Evidence sufficiencyNo direct support in retrieved passagesHuman review
Mean groundednessAggregate claim score misses thresholdHuman review
Minimum groundednessAny single claim misses its floorHuman review
All gates passEvery material claim is supportedAnswer with citations
Bug caught by evals

Related policy language is not the same as supporting evidence.

The verifier initially allowed a liability passage to support an insurance-coverage claim. The development suite exposed the concept conflation, leading to stricter generation and entailment instructions.

Live decision preview

See it decide something live.

Submit a support question and watch the system screen, retrieve, verify, and select a responsible route.

Run the demo
Decision trace Ready
Incoming questionDoes the policy support this answer?
Screen→Retrieve→Verify
AnswerReviewBlock
0.96Groundedness score3 citations attached