Direct Answer (TL;DR)
Yes. Brilo AI can limit access to knowledge sources so the Brilo AI voice agent only uses approved content when answering callers. Access is controlled at the connector and workflow level: you choose which knowledge base (KB), document store, or webhook endpoints the agent may query, and Brilo AI respects those boundaries during live calls and in session context. Administrators can also enforce source-level access control, confidence thresholds, and fallback rules so the agent refuses or hands off when a requested item is out of scope. This prevents unauthorized or unvetted documents from being surfaced during calls.
Can I restrict which KBs the Brilo AI agent reads? — Yes. You can configure allowed knowledge sources per agent or workflow so only designated KBs or document stores are queried.
How do I stop the agent from using external documents? — Configure source-level access control and remove connectors or API keys for any external document stores you do not want the agent to access.
Can I make the agent ask for human approval before using sensitive documents? — Yes. You can require human authorization or an intermediate review workflow for high-risk documents or topics.
Why This Question Comes Up (problem context)
Buyers ask whether access to knowledge sources can be limited because enterprises need predictable, auditable behavior from voice agents. Regulated sectors such as healthcare and banking often have strict data-handling rules, internal approval processes, and separate knowledge silos. Teams want to avoid accidental disclosure from unvetted documents, reduce regulatory risk, and make it clear which sources are authoritative for caller-facing answers. Limiting knowledge access also simplifies troubleshooting and traceability for audit and quality teams.
How It Works (High-Level)
Brilo AI enforces knowledge access limits through connector configuration and workflow scoping. At deployment you assign which knowledge sources the Brilo AI voice agent may query (for example, your internal KB, a document store, or an approved webhook). When a call arrives, the agent evaluates intent and then queries only the configured sources for grounding before producing an answer. If configured, Brilo AI will combine the retrieved text with the model output and tag the answer with source references for auditability.
In Brilo AI, a knowledge source is any configured content endpoint (for example, a knowledge base or document store) that the agent may query for factual grounding.
In Brilo AI, grounding is the process of retrieving and attaching source documents to the agent’s response so statements can be traced back to an approved source.
In Brilo AI, source-level access control is the connector and workflow setting that determines which endpoints the agent is allowed to query.
For implementation details on answer quality and grounding controls, see the Brilo AI knowledge base answer-quality controls guide: Brilo AI knowledge base answer-quality controls.
Guardrails & Boundaries
Limitations and safety settings you should configure for knowledge access:
Define allowed topics and map them to approved knowledge sources so the Brilo AI voice agent never queries out-of-scope content.
Require grounding for account-specific or regulated facts; if a groundable source is not available, configure the agent to refuse or escalate.
Enforce confidence thresholds (for intent detection and retrieved evidence) that trigger a human handoff when thresholds are not met.
Disable connectors or revoke API keys for any document store you do not want the agent to access.
In Brilo AI, a confidence threshold is a rule that forces clarification or escalation when model or retrieval confidence falls below a configured level.
For behavior when the agent is unsure and recommended escalation patterns, see the Brilo AI escalation and uncertainty guide: Brilo AI handling uncertainty and escalation.
Applied Examples
Healthcare: A hospital configures the Brilo AI voice agent to read only from the approved clinical FAQ KB and scheduling system. The agent refuses to answer any clinical-treatment queries that are not grounded in the approved KB and routes those calls to triage nurses.
Banking: A retail bank limits the Brilo AI voice agent to the customer-facing product knowledge base and the CRM for account metadata. Price sheets, internal policy documents, and product roadmaps are excluded so callers never receive answers from unapproved internal documents.
Insurance: An insurer restricts the Brilo AI voice agent to policy documents and the claims knowledge base for claim-status inquiries while routing all legal or underwriting questions to a human under defined escalation rules.
Human Handoff & Escalation
When the Brilo AI voice agent hits a knowledge access limit or a guardrail (for example, no allowed source returns evidence, confidence threshold not met, or the requested topic is out-of-scope), configured workflows can:
Offer a conservative fallback response that explains the limitation and ask to transfer the caller.
Place the call in a warm transfer queue to a human agent with a transcript and the attempted source lookups attached.
Create a ticket or callback request in your CRM and route the case to a specialist for manual handling.
These handoffs are driven by workflow rules you configure (e.g., intent + missing grounding -> escalate) and include contextual artifacts—transcript snippets, retrieved citations, and confidence tags—to speed resolution.
Setup Requirements
To restrict knowledge source access for a Brilo AI voice agent, provide these items and follow the steps below:
Prepare: Identify the allowed knowledge sources (your KB, approved document stores, or webhook endpoints) and the sources that must be blocked.
Configure: Add or remove connectors in the Brilo AI admin console so only approved endpoints are connected to the agent.
Scope: Assign allowed sources to the specific agent or workflow so policies are applied per use case.
Set rules: Define confidence thresholds, fallback text, and escalation triggers for missing grounding or low-confidence retrievals.
Test: Run calls that request in-scope and out-of-scope information to verify the agent refuses or escalates as expected.
Monitor: Review logs and tagged interactions to confirm only approved documents are ever cited in live responses.
For capacity and provisioning considerations required while configuring sources and connectors, see the Brilo AI performance and provisioning guidance: Brilo AI performance and provisioning guidance.
Business Outcomes
When you limit knowledge source access for Brilo AI, you get more predictable, auditable answers and reduced operational risk. Constraining the agent to approved KBs simplifies governance, reduces the chance of incorrect disclosures, and shortens review cycles for quality assurance. These controls also make it easier for compliance and audit teams to validate the agent’s behavior and provide clear escalation paths for sensitive or ambiguous calls.
FAQs
Can I restrict access per agent or per workflow?
Yes. Brilo AI supports configuring allowed knowledge sources at the agent and workflow level so different agents can have different access policies.
What happens if the agent can’t find an approved source for a caller question?
If configured, the agent will use a conservative fallback that refuses to answer the question, ask clarifying questions, or escalate to a human based on your escalation rules and confidence thresholds.
Can I audit which documents the agent used to answer a question?
Yes. Brilo AI can include source references and retrieval metadata with responses so reviewers can trace answers back to the original documents and connectors.
Does limiting knowledge sources affect model performance or latency?
Restricting sources can reduce retrieval overhead and improve predictability, but overall performance depends on connector latency and session configuration.
How do I revoke access to a document store immediately?
Remove or disable the connector/API key in the Brilo AI admin console for immediate effect; ensure workflows are tested after changes to confirm expected fallback behavior.
Next Step
If you need help mapping allowed sources to workflows or want a configuration review, contact your Brilo AI customer success representative or book a technical onboarding session.