Skip to main content

How can I customize prompts and knowledge for an AI phone agent?

Y
Written by Yatheendra Brahmadevera
Updated over a week ago

Direct Answer (TL;DR)

You can customize prompts and knowledge for an AI phone agent in Brilo AI by editing the agent’s core instruction (the prompt), uploading or linking structured knowledge (FAQ articles and KB content), and tuning fallback and confidence thresholds that control when the agent clarifies or escalates. These settings live in the Brilo AI Agents console and are applied per voice agent or phone flow; changes should be tested with live calls before deployment. Use the knowledge import and prompt editor to keep answers aligned with regulated scripts and required disclosures. Changes to voice, SSML, or custom phonetics may require a support request.

How do I edit the agent’s prompt? — Open the Brilo AI Agents console and update the agent’s core instruction under Prompt/Conversation settings; save and deploy to apply changes to live calls.

How do I add knowledge to the voice agent? — Upload structured FAQ documents or link your knowledge base so the Brilo AI voice agent can retrieve answers during calls.

How do I control when the AI escalates? — Set confidence score thresholds and fallback prompt behavior so the agent clarifies, retries, or transfers to a human when required.

Why This Question Comes Up (problem context)

Enterprises ask how to customize prompts and knowledge for an AI phone agent because phone interactions must reflect legal scripts, brand voice, and regulated disclosures while avoiding inconsistent or unsafe answers. Buyers in healthcare, banking, and insurance need predictable call flows, auditable prompt changes, and the ability to update knowledge rapidly when policies or rates change. Teams also want guardrails so the Brilo AI voice agent hands off to humans for sensitive topics or regulatory triggers.

How It Works (High-Level)

Brilo AI applies prompt and knowledge changes at the agent level. The agent’s core instruction (prompt) guides response style, allowed actions, and required disclosures. The agent’s knowledge sources supply factual answers at runtime. When a call arrives, Brilo AI performs intent recognition and looks up matching knowledge content; if the confidence score is high, the agent answers directly, otherwise it follows the fallback prompt flow or escalates. Prompt versioning is managed per agent so you can test changes in staging before deploying to production.

In Brilo AI, the prompt defines tone, mandatory script text, and fallback behavior. The knowledge base is the set of structured documents, FAQ entries, and indexed content the voice agent queries to produce factual answers.

For guidance on maintaining consistent prompts across calls, see the Brilo AI consistency guide: Brilo AI consistency guide: How the AI stays consistent across calls.

Relevant technical terms: prompt, fallback prompt, knowledge base, intent recognition, confidence score, SSML, warm transfer.

Guardrails & Boundaries

Brilo AI enforces several safety boundaries to prevent unsupported actions and to preserve compliance. Configure explicit guardrails in the agent prompt to require mandatory disclosures, to never provide medical or legal advice, and to transfer calls when predefined phrases or low confidence states occur. Avoid embedding private keys, account passwords, or unapproved PII in prompts or knowledge documents.

In Brilo AI, the confidence threshold is the numeric limit that determines when the agent should ask a clarifying question, use a fallback prompt, or escalate to a human. Do not rely on the agent for regulated decision-making unless you have documented workflows and human review; instead, use the fallback and transfer rules to route sensitive calls immediately. For best practices on uncertain-call handling and escalation behavior, see Brilo AI’s uncertain-call behavior guidance: Brilo AI guide: What happens when the AI is unsure?

Applied Examples

  • Healthcare: A Brilo AI voice agent uses an approved prompt that includes a privacy disclaimer and the clinic’s triage questions. If the caller mentions emergency keywords or the confidence score drops, the agent follows the fallback prompt and immediately triggers a warm transfer to an on-call nurse. The knowledge base contains approved FAQ answers and clinician-curated guidance snippets (do not use the agent as clinical advice).

  • Banking: The Brilo AI voice agent reads a required disclosure from the prompt when a caller asks about account freezes, pulls account-safe answers from a locked knowledge base, and requires additional verification steps before providing balance details. If intent recognition is ambiguous or regulatory language is triggered, the agent escalates to a human agent with context.

  • Insurance: The Brilo AI voice agent answers coverage questions using company policy documents uploaded to the knowledge base and appends the exact policy excerpt as a transcript note. For complex claims or low-confidence matches, the agent follows the fallback prompt and queues a human claims specialist with the call summary.

Human Handoff & Escalation

Brilo AI supports warm transfers, cold transfers, and callback scheduling when configured. Handoff is rule-driven: triggers include caller request for a person, low confidence, detected sentiment issues, or explicit regulatory topics. During transfer, Brilo AI passes recent transcript snippets, caller intent, and the active prompt context to the receiving agent to prevent repetition and speed resolution.

If the destination agent is unavailable, Brilo AI can capture structured caller information and create a callback task or voicemail according to your configured retry policy.

Setup Requirements

  1. Grant admin access to the Brilo AI console so your team can edit agents and knowledge.

  2. Prepare the agent prompt: draft the core instruction, required legal disclosures, and fallback prompt text for clarifications.

  3. Gather knowledge content: export FAQs, policy documents, and approved script text as structured files or a URL to your knowledge system.

  4. Configure intent recognition and confidence thresholds in the agent’s Conversation settings.

  5. Test the agent with a staging phone number and representative call scripts; iterate on prompts and knowledge entries.

  6. Enable call transfer rules and map destination phonebook entries for warm transfers or callback handling.

For details on tuning naturalness and agent settings, see: Brilo AI article: Does the AI sound natural or robotic? and for language/voice configuration steps see: Brilo AI article: How does the AI handle accents and speech variations?

Business Outcomes

Customizing prompts and knowledge for a Brilo AI voice agent improves answer accuracy, reduces unnecessary transfers, and ensures regulated scripts are read consistently. Well-structured knowledge and clear fallback rules lower human agent load for routine queries and accelerate resolution for complex calls by preserving context during handoffs. These operational improvements support better customer experience in healthcare, banking, and insurance while keeping enterprise controls in place.

FAQs

How do I version or roll back a prompt change?

Brilo AI stores prompt edits at the agent level; follow your change management workflow by testing in a staging agent, then deploy. If a prompt causes problems, revert to the previous version in the Agents console and redeploy.

Can the voice agent use our internal FAQ system as a knowledge source?

Yes. Brilo AI can query structured FAQ content or an indexed knowledge base. Provide export files or a connector endpoint (your webhook or knowledge feed) and validate content formatting before deployment.

What happens if the agent returns an incorrect answer on a regulated topic?

The agent should be configured to escalate on low confidence or match specific guarded keywords. Use fallback prompts and immediate human handoff rules for regulated topics; keep an audit trail of prompt versions and knowledge updates for review.

Are changes to voice, SSML, or phonetic lexicons supported out of the box?

Brilo AI supports SSML and voice selection in the agent settings; in some cases, custom prosody or phonetic lexicon updates may require a support request—follow the steps in the Agents console and consult support for custom voice work.

Next Step

If you need help mapping prompts to compliance requirements or designing escalation flows for healthcare or financial services, open a support request from the Brilo AI console so an implementation specialist can review your prompt and knowledge strategy.

Did this answer your question?