AssistantFleet API

Build voice assistants programmatically. Create an assistant, point a phone number at it, and your callers talk to the selected OpenAI or Gemini realtime provider. Every call is logged, costed, and (optionally) forwarded to your backend via signed webhooks.

Quick start

1. Sign in at assistant.voicefleet.ai with Google or an email sign-in link. Your tenant is provisioned on first login.
2. Create an API key in the dashboard under API keys. Copy it once — it starts with af_live_.
3. Create an assistant:

   curl -X POST https://assistant.voicefleet.ai/assistants \
     -H "Authorization: Bearer af_live_YOUR_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Front desk",
       "instructions": "You are a friendly receptionist for Acme Clinic. Answer questions about hours and bookings.",
       "first_message": "Thanks for calling Acme, how can I help?",
       "realtime_provider": "openai",
       "model": "gpt-realtime-2.1",
       "voice": "marin"
     }'

4. Bind a phone number on a supported provider (Telnyx, Twilio, Voximplant, VoIPCloud). E.164, no spaces:

   curl -X PUT https://assistant.voicefleet.ai/numbers/+13125550100 \
     -H "Authorization: Bearer af_live_YOUR_KEY" \
     -H "Content-Type: application/json" \
     -d '{"assistant_id": "<assistant-id-from-step-3>", "provider": "telnyx"}'

   provider defaults to telnyx. See Telecom providers for the per-provider webhook URL and setup.

5. Point your provider at us. Configure your number's voice webhook URL — depending on provider:
   • Telnyx → https://assistant.voicefleet.ai/telnyx/route
   • Twilio → https://assistant.voicefleet.ai/twilio/route
   • Voximplant → https://assistant.voicefleet.ai/voximplant/route (queried from a VoxEngine scenario)
   • VoIPCloud → https://assistant.voicefleet.ai/voipcloud/route (PBX queries for forward target)
   Call the number — you should reach the assistant.

Don't want to set up Telnyx yet? Use the Test in browser button in the dashboard to talk to your assistant from the browser with your microphone — the call is logged the same way as a phone call.

Authentication

Every request to a tenant-scoped endpoint must include a Bearer token:

Authorization: Bearer af_live_…

Keys are shown once at creation and stored hashed (SHA-256). You can revoke them anytime from the dashboard. Three invalid credentials from the same IP within 5 minutes triggers a temporary 429.

The dashboard itself uses a session cookie set after Google or email-link sign-in — you don't need an API key for the UI.

Base URL & format

• Base URL: https://assistant.voicefleet.ai
• Request bodies: JSON (Content-Type: application/json)
• Responses: JSON, UTF-8
• Timestamps: ISO 8601 (UTC)
• Phone numbers: E.164 (+13125550100) — required on bind

Telecom providers

A phone number is bound to one telecom provider via the provider field on the binding. The provider determines which webhook URL to configure on the provider side, and how the live media is bridged.

Telnyx setup

1. Create a TeXML application; set Inbound voice URL to https://assistant.voicefleet.ai/telnyx/route.
2. Assign your number to that application.
3. (Optional) Set TELNYX_ROUTE_SECRET in your tenant env and add header x-telnyx-route-secret on the app.
4. For outbound + transfer + DTMF, also set TELNYX_API_KEY and TELNYX_CONNECTION_ID server-side.

Twilio setup

1. In the Twilio console, open the phone number → Voice → A call comes in.
2. Set webhook to https://assistant.voicefleet.ai/twilio/route (HTTP POST).
3. Bind the number with "provider": "twilio" via PUT /numbers/:number.
4. For outbound + transfer + DTMF set TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN server-side.

Voximplant setup

Voximplant scenarios run JavaScript in their cloud — bind a minimal scenario like this:

// VoxEngine scenario — fetch our routing decision then bridge raw audio.
require(Modules.WebSocket);

VoxEngine.addEventListener(AppEvents.CallAlerting, async (e) => {
  const call = e.call;
  call.answer();

  const r = await fetch('https://assistant.voicefleet.ai/voximplant/route', {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: 'To=' + encodeURIComponent(call.number()) + '&From=' + encodeURIComponent(call.callerid()),
  });
  const action = await r.json();
  if (action.action !== 'assistantfleet') { call.hangup(); return; }

  const ws = VoxEngine.createWebSocket(action.websocket_url);
  ws.addEventListener(WebSocketEvents.OPEN, () => {
    call.sendMediaTo(ws, { encoding: 'pcm', sampleRate: 16000 });
    ws.sendMediaTo(call);
  });
  ws.addEventListener(WebSocketEvents.CLOSE, () => call.hangup());
});

Audio format must be slin16 (16-bit linear PCM, mono, 16 kHz) — that's what /media/voximplant expects. The scenario is the customer's responsibility; we don't run code in Voximplant.

For outbound, configure a second scenario (or extend this one) that reads VoxEngine.customData(), calls VoxEngine.callPSTN(), and bridges to our WS. See the template under Outbound calls. Set VOXIMPLANT_RULE_ID to the Rule that points at that scenario, plus VOXIMPLANT_ACCOUNT_ID and VOXIMPLANT_API_KEY server-side.

VoIPCloud setup

VoIPCloud is a PBX / SIP-trunk provider operating in five regions:

• uk → https://uk.voipcloud.online
• ie → https://ie.voipcloud.online
• nz → https://nz.voipcloud.online
• us → https://us.voipcloud.online
• ca → https://ca.voipcloud.online

Each binding carries its own region, so a single AssistantFleet deployment can serve VoIPCloud numbers across regions. We don't terminate SIP. There are two paths:

Inbound (PBX → AI):

1. Bind your VoIPCloud number with "provider": "voipcloud", "region": "uk" (or your region).
2. Also bind a Telnyx or Twilio number to the same assistant — that's where calls actually bridge.
3. Configure your VoIPCloud PBX to query /voipcloud/route for the forward target; we respond with the sibling Telnyx/Twilio number.
4. The PBX SIP-forwards the call there; the streaming provider picks it up.

Outbound (click-to-call) — verified against VoIPCloud's /api/integration/v2/call-to-number:

1. Put VOIPCLOUD_API_KEY in the server env. The base URL is derived from each binding's region; you can override with VOIPCLOUD_BASE_URL for single-region installs or testing.
2. Have a VoIPCloud user/extension whose dial behavior forwards to a sibling Telnyx/Twilio AI number — that puts the AI on the calling leg after the bridge.
3. POST /calls/outbound with user_number set to that VoIPCloud extension. VoIPCloud rings it first, then dials to and bridges.

The Vapi-style AI is the calling leg directly via SIP pattern (used by their warm-transfer-experimental + sipVerb: dial) requires a SIP UAC in the AI stack — Vapi has one, AssistantFleet doesn't. If that's what you need, use Telnyx/Twilio for outbound or peer their BYO-SIP trunk through your VoIPCloud trunk on the provider side.

Assistants

Create

POST /assistants

Returns 201 with the created assistant (includes id, revision, created_at) and an ETag containing the revision. Responses also include a credential-free effective_runtime_config snapshot with inherited server defaults resolved.

Cascading pipeline (STT → LLM → TTS)

By default an assistant runs on a single OpenAI Realtime model that listens, reasons and speaks over one connection; realtime_provider: "gemini" selects Gemini Live on the same one-connection pipeline. Setting pipeline_mode: "cascading" splits those three jobs across separate vendors instead. The reason to do it is voice coverage and deterministic provider controls. Gemini Live can be prompted to use an Irish accent, but it does not expose an en-IE locale or numeric accent-strength parameter; cascading voices can provide an explicitly regional voice. Per-minute cost is roughly comparable — text-to-speech is billed by characters or tokens and is the expensive leg — and latency is somewhat higher, since a cascading turn pays for endpointing plus a model round-trip before speech begins.

At Google's published paid-tier rates, Gemini 3.1 Live audio is approximately $0.005/min input plus $0.018/min output. Text, thinking, enabled transcriptions, and conversation context are additional token charges; AssistantFleet enables a sliding context window to keep long-call growth bounded.

{
  "name": "Dublin native audio",
  "pipeline_mode": "realtime",
  "realtime_provider": "gemini",
  "model": "gemini-3.1-flash-live-preview",
  "voice": "Aoede",
  "instructions": "Speak English throughout with a strong, natural Irish accent."
}

Tools, knowledge, webhooks, recordings, transcripts and post-call analytics behave identically in both modes. Call logs gain cost_stt and cost_tts alongside cost_openai, which keeps meaning "the model leg".

curl -X POST https://assistant.voicefleet.ai/assistants \
  -H "Authorization: Bearer af_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Dublin front desk",
    "pipeline_mode": "cascading",
    "stt_provider": "deepgram",
    "stt_model": "nova-3",
    "stt_language": "multi",
    "llm_model": "gpt-4.1-mini",
    "tts_provider": "soniox",
    "tts_voice": "Adrian",
    "tts_model": "tts-rt-v2",
    "tts_settings": { "audio_tags": true }
  }'

A deployment with the private Piper sidecar can instead select an Irish VCTK speaker: {"tts_provider":"piper","tts_model":"en_GB-vctk-medium","tts_voice":"p266","delivery_style":"expressive"}. Piper needs no tenant API credential; PIPER_TTS_BASE_URL points AF to the sidecar on the deployment's private network. Minimal, Professional, and Expressive map to conservative Piper variability presets; optional tts_settings overrides can tune noise_scale and noise_w_scale for a tested speaker.

Gemini native TTS is available with {"tts_provider":"gemini","tts_model":"gemini-3.1-flash-tts-preview","tts_voice":"Kore"}. Gemini 3.1 streams audio for lower latency; the 2.5 Flash and Pro preview models return each submitted phrase in one response. All three use the same 30 prebuilt voices and support controllable delivery and pacing.

Gemini 3.5 Transcribe Live is available with {"stt_provider":"gemini","stt_model":"gemini-3.5-transcribe-live","stt_language":"multi"}. AF converts phone and browser input to Gemini's PCM16 16 kHz stream, forwards speech keyterms as custom vocabulary, and uses Gemini's finalized transcript as the caller turn. The same gemini credential also authorizes TTS.

Credentials for each leg are set per tenant through PUT /credentials/:provider using the provider names deepgram, openai, azure_speech, polly, elevenlabs, cartesia, telnyx, and soniox, and gemini. Piper is server-scoped and takes no tenant credential. Anything a tenant has not configured falls back to the server's own keys.

List, get, update, delete

GET    /assistants
GET    /assistants/:id
PATCH  /assistants/:id    # any subset of the create fields
DELETE /assistants/:id    # 204 on success; unbinds its numbers

GET /assistants/:id and successful create/update responses include a monotonic revision and matching ETag. Send that value in If-Match: "<revision>" when updating or deleting. A stale write returns 409 ASSISTANT_REVISION_CONFLICT with current_revision.

Example: update an assistant's prompt

curl -X PATCH https://assistant.voicefleet.ai/assistants/abc123 \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -H 'If-Match: "7"' \
  -d '{"instructions": "You are a senior support agent. Stay polite under pressure."}'

Phone numbers

A phone number is bound to exactly one assistant at a time, and a number is globally unique — you cannot bind a number that another tenant has already claimed.

Bind

PUT /numbers/:number — body {"assistant_id": "<id>", "provider": "telnyx"}

The path parameter must be E.164. provider defaults to telnyx — see Telecom providers for the supported values. When provider is voipcloud, also pass region: one of uk | ie | nz | us | ca (defaults to uk). The region selects the regional VoIPCloud portal (https://{region}.voipcloud.online) used for the Integration API.

List

GET /numbers
# → [{ "phone_number": "13125550100", "assistant_id": "abc", "provider": "telnyx", "region": null, "created_at": "…" }, …]

Unbind

DELETE /numbers/+13125550100
# → 204 No Content

Call logs

GET /calls?page=1&limit=25&sort=started_at&order=desc

Returns a tenant-scoped page of calls. Aggregate stats reflect the active filters.

Filters: search, assistant_id, source, model, voice, outcome, sentiment, date_from, and date_to. Sort by started_at, source, duration_seconds, from_number, did, assistant_name, model, voice, outcome, ended_reason, sentiment, or cost_total; order is asc or desc. The maximum page size is 1000.

{
  "stats": { "calls": 42, "totalSeconds": 3120, "totalCost": 5.18 },
  "calls": [
    {
      "id": "…",
      "started_at": "2026-05-24T12:01:33Z",
      "ended_at": "2026-05-24T12:03:01Z",
      "duration_seconds": 88,
      "source": "telnyx",          // or "web"
      "did": "13125550100",
      "from_number": "13644448442",
      "assistant_id": "abc",
      "assistant_name": "Front desk",
      "model": "gpt-realtime-2.1",
      "voice": "marin",
      "ended_reason": "assistant_ended_call",
      "recording_available": true,
      "recording_url": "/calls/…/recording",
      "waveform_url": "/calls/…/waveform",
      "cost_total": 0.1432,
      "cost_openai": 0.1378,
      "cost_telnyx": 0.0054,
      "transcript": [
        { "role": "assistant", "text": "Thanks for calling…", "t": "2026-05-24T12:01:34Z" },
        { "role": "user", "text": "Hi, I'd like to book…", "t": "2026-05-24T12:01:38Z" }
      ],
      "tool_events": [
        {
          "type": "tool",
          "call_id": "call_…",
          "name": "check_availability",
          "arguments": { "date": "2026-05-24" },
          "output": { "available": true },
          "status": "completed",
          "started_at": "2026-05-24T12:01:42Z",
          "completed_at": "2026-05-24T12:01:42.240Z",
          "duration_ms": 240
        }
      ]
    }
  ],
  "pagination": {
    "total": 42,
    "page": 1,
    "limit": 25,
    "total_pages": 2,
    "sort": "started_at",
    "order": "desc"
  }
}

tool_events are ordered into the dashboard transcript timeline by started_at. Stored payloads include bounded model arguments and tool responses, but never configured tool URLs, headers, or signing secrets.

Play a recording

GET /calls/:id/recording

Returns the mixed call audio as audio/wav. The endpoint uses the same tenant authentication as the rest of the management API and supports browser range requests. Recordings are available for calls made after recording support is deployed; older rows cannot be reconstructed from their transcripts.

GET /calls/:id/waveform

Returns compact 0–255 peak arrays used by the dashboard's interactive waveform. New recordings include separate caller and assistant channels; recordings created before waveform support are analyzed lazily and returned as a single mixed channel. This endpoint is authenticated and tenant-scoped too.

Recording is enabled by default. Set ASSISTANTFLEET_RECORDINGS_ENABLED=0 to disable it or ASSISTANTFLEET_RECORDINGS_DIR to change storage. Make sure your call flow complies with the recording-consent laws that apply to your callers.

Post-call analytics

Every call gets a cheap LLM pass after it ends — one structured-JSON call to gpt-4o-mini on the transcript. The result is stored on the call_logs row and visible in the Call logs view + included in GET /calls responses.

Fire-and-forget — analytics never blocks call teardown or webhook delivery. If the LLM call fails or the transcript is empty, the row stays null. Disable with OPENAI_ANALYTICS_ENABLED=0; tune model with OPENAI_ANALYTICS_MODEL.

Metrics & latency

GET /metrics returns a JSON snapshot of the in-memory metrics registry (resets on process restart — not Prometheus-persistent). Admin sees aggregate; tenants see their own slice. Use it to sanity-check capacity and turn-latency on real calls.

curl https://assistant.voicefleet.ai/metrics \
  -H "Authorization: Bearer af_live_YOUR_KEY"

{
  "uptime_seconds": 12483,
  "active_calls": 2,
  "calls_started": 145,
  "calls_ended": 143,
  "calls_by_provider": { "telnyx": 138, "twilio": 7 },
  "latencies": {
    "setup_ms":         { "n": 143, "min": 220, "p50": 410, "p95": 720, "p99": 980, "max": 1200 },
    "greeting_ms":      { "n": 142, "min": 320, "p50": 560, "p95": 940, "p99": 1300, "max": 1600 },
    "turn_ms":          { "n": 412, "min": 380, "p50": 720, "p95": 1180, "p99": 1450, "max": 2100 },
    "queue_depth_peak": { "n": 143, "min": 3,   "p50": 22,  "p95": 58,   "p99": 91,   "max": 110  }
  }
}

What each latency means:

• setup_ms — WS open → session.updated. The OpenAI Realtime connection cost.
• greeting_ms — session.updated → first audio delta. Time to "Thanks for calling…".
• turn_ms — caller speech-start → first response audio delta. What callers actually feel.
• queue_depth_peak — max outbound-frame queue size during the call. High values mean we're producing audio faster than the provider can drain it.

Samples are rolling (cap 1000 per metric); old values are dropped. Per-call summary lines also appear in the process logs: setup=420ms greeting=560ms turns=4 p50=680ms p95=910ms qpeak=22.

Tenant credentials

Each tenant can bring their own provider accounts — Telnyx, Twilio, Voximplant, VoIPCloud — and they're stored encrypted at rest (AES-256-GCM with the server's ASSISTANTFLEET_ENCRYPTION_KEY). When a dialer fires it resolves credentials tenant-first, falling back to the server env if the tenant hasn't configured that provider.

Save credentials

curl -X PUT https://assistant.voicefleet.ai/credentials/twilio \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"account_sid": "AC...", "auth_token": "..."}'

Provider-specific shapes:

List configured providers

GET /credentials
# → [{ "provider": "twilio", "configured": true, "created_at": "…", "updated_at": "…" }, …]

Secret values are never returned by GET — only the fact that a provider is configured and when it was last updated.

Clear credentials

DELETE /credentials/twilio
# → 204

After clearing, the server-env defaults (if any) are used again.

Outbound calls

Dial out from one of your bound Telnyx numbers and bridge the call straight to an assistant — for appointment reminders, follow-ups, surveys, etc.

Start a call

curl -X POST https://assistant.voicefleet.ai/calls/outbound \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "assistant_id": "<assistant-id>",
    "from": "+13644448442",
    "to":   "+13125550100"
  }'

Returns 201 with Telnyx call ids:

{
  "status": "initiated",
  "call_sid": "…",
  "call_control_id": "v3:…",
  "from": "+13644448442",
  "to":   "+13125550100",
  "assistant_id": "…"
}

Rules

• from must be E.164 and bound to your tenant (use PUT /numbers/:number). The dial provider is whatever that number is bound on.
• to must be E.164.
• Telnyx-from: requires TELNYX_API_KEY + TELNYX_CONNECTION_ID server-side, else 503. AI is the calling leg.
• Twilio-from: requires TWILIO_ACCOUNT_SID + TWILIO_AUTH_TOKEN, else 503. AI is the calling leg.
• Voximplant-from: requires VOXIMPLANT_ACCOUNT_ID + VOXIMPLANT_API_KEY + VOXIMPLANT_RULE_ID (the rule must point at a scenario that reads VoxEngine.customData() and dials via callPSTN — template below). AI is the calling leg.
• VoIPCloud-from: requires VOIPCLOUD_API_KEY + VOIPCLOUD_BASE_URL, plus a user_number field in your POST body (your VoIPCloud user/extension that will be rung first). This is a click-to-call flow — VoIPCloud rings your user, then bridges to to. Have your user extension forward to a sibling Telnyx/Twilio number bound to the assistant so the AI ends up on that leg. The AI-as-calling-leg path requires SIP origination through VoIPCloud's BYO SIP trunk — we don't ship a SIP UAC; configure your AI provider's BYO SIP credential for that pattern.

Voximplant outbound scenario template

Configure a VoxEngine scenario tied to the rule you set in VOXIMPLANT_RULE_ID:

require(Modules.WebSocket);

VoxEngine.addEventListener(AppEvents.Started, async () => {
  const data = JSON.parse(VoxEngine.customData() || '{}');
  if (!data.to || !data.websocket_url) { VoxEngine.terminate(); return; }

  const call = VoxEngine.callPSTN(data.to, data.from);
  call.addEventListener(CallEvents.Connected, () => {
    const ws = VoxEngine.createWebSocket(data.websocket_url);
    ws.addEventListener(WebSocketEvents.OPEN, () => {
      call.sendMediaTo(ws, { encoding: 'pcm', sampleRate: 16000 });
      ws.sendMediaTo(call);
    });
    ws.addEventListener(WebSocketEvents.CLOSE, () => call.hangup());
  });
  call.addEventListener(CallEvents.Disconnected, () => VoxEngine.terminate());
  call.addEventListener(CallEvents.Failed, () => VoxEngine.terminate());
});

VoIPCloud outbound example

curl -X POST https://assistant.voicefleet.ai/calls/outbound \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "assistant_id": "<assistant-id>",
    "from":         "+441216306195",     // VoIPCloud DID (caller_id)
    "to":           "+447418328008",     // destination
    "user_number":  "+44XXXXXXXXXX"      // your VoIPCloud user/extension to ring first
  }'

Outbound calls fire the same call.started / call.ended webhooks as inbound, and the resulting call_logs row has direction: "outbound" so you can filter.

Tools

Tools let the assistant call your HTTP endpoint mid-conversation (Vapi-style function calling). You define a tool once, attach it to one or more assistants, and we forward it to the active OpenAI pipeline as a function. When the model decides to call it, we POST signed JSON to your URL and feed the response back so the model can speak the result.

Create a tool

curl -X POST https://assistant.voicefleet.ai/tools \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "lookup_customer",
    "description": "Look up a customer by phone number.",
    "progress_message": "I'll check that now.",
    "url": "https://your-app.com/tools/lookup",
    "method": "POST",
    "parameters": {
      "type": "object",
      "properties": { "phone": { "type": "string", "description": "E.164 phone" } },
      "required": ["phone"]
    }
  }'

The response includes a secret shown once — store it; you'll use it to verify our request signature.

progress_message is optional and limited to 160 characters. A cascading assistant speaks it only when the model calls that custom tool without first giving its own update. This removes dead air during slow lookups without duplicating a model-generated preamble. Pass null in a PATCH to clear it.

Attach to an assistant

curl -X PUT https://assistant.voicefleet.ai/assistants/<assistant-id>/tools \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tool_ids": ["<tool-id-1>", "<tool-id-2>"]}'

This replaces the assistant's tool set — pass an empty array to detach all.

What your endpoint receives

POST https://your-app.com/tools/lookup
Content-Type: application/json
X-AssistantFleet-Tool: lookup_customer
X-AssistantFleet-Call: <openai call_id>
X-AssistantFleet-Signature: sha256=<hmac of raw body>

{
  "call_id": "call_…",
  "name": "lookup_customer",
  "arguments": { "phone": "+13125550100" },
  "context": { "assistant_id": "…", "did": "13644448442", "from": "15551112222", "source": "telnyx" },
  "created_at": "2026-05-24T13:42:01Z"
}

What you return

Respond with JSON — anything serializable. The whole body is fed back to the model as the function output, which the model uses to compose its next spoken turn.

{
  "found": true,
  "customer": { "name": "Mariano", "tier": "gold", "last_visit": "2026-04-12" }
}

Timeouts (default 8s, configurable per tool 0.5–30s), 5xx, and network errors are returned to the model as a structured {"error": "..."} envelope so it can recover gracefully ("I'm having trouble looking that up — could you try again in a moment?"). Keep responses short: long payloads cost more tokens and make the model verbose.

Verifying signatures (Node.js)

app.post('/tools/lookup', express.raw({ type: '*/*' }), (req, res) => {
  const sig = req.header('X-AssistantFleet-Signature') || '';
  const expected = 'sha256=' + crypto
    .createHmac('sha256', process.env.AF_TOOL_SECRET)
    .update(req.body).digest('hex');
  if (!crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected))) return res.sendStatus(401);
  const { arguments: args, context } = JSON.parse(req.body.toString());
  // ... your lookup logic ...
  res.json({ found: true, customer: { name: 'Mariano' } });
});

Manage tools

GET    /tools
GET    /tools/:id
PATCH  /tools/:id     # any subset of the create fields
DELETE /tools/:id     # cascades the assistant attachment

Tips

• The model uses the name + description + parameter schema to decide when to call the tool. Be explicit and concrete in the description.
• Function names follow OpenAI's rule: ^[A-Za-z_][A-Za-z0-9_]{0,63}$.
• GET tools receive the arguments as query string instead of a body.
• Tool URLs must be HTTPS (same SSRF guard as webhooks).

Built-in tools

Runtime tools are baked in — you don't define them as custom HTTP tools. Some are always available and others are enabled by assistant settings.

wait_for_user

Always available. When VAD produces a turn containing only silence, background noise, or no intelligible request, the assistant invokes this no-op and waits without speaking. This avoids awkward filler responses to false turns.

end_call

Lets the assistant hang up cleanly when the conversation is done. Enabled by default on every new assistant; toggle off via:

curl -X PATCH https://assistant.voicefleet.ai/assistants/<id> \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"end_call_enabled": false}'

transfer_call

Lets the assistant transfer the live call to a human at one of a whitelist of phone numbers (Telnyx Call Control transfer under the hood). Only exposed when you set the list:

curl -X PATCH https://assistant.voicefleet.ai/assistants/<id> \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"transfer_phone_numbers": ["+13125550100", "+13125550199"]}'

The destinations are echoed to the model in the tool description, so it knows the options. The model is instructed to announce the transfer before invoking the tool.

DTMF (keypad)

Touch-tone digits work in both directions on real phone calls (not in web tests).

Caller → assistant

When a caller presses a key, Telnyx forwards a dtmf media event. We surface it to the model as a user text turn ("[Caller pressed keypad: 1]") and trigger a response, so prompts like "Press 1 for sales, 2 for support" just work if the model is instructed to treat keypad input as a choice.

Assistant → call (e.g. navigating an outbound IVR)

Enable on the assistant — exposes a send_dtmf built-in tool:

curl -X PATCH https://assistant.voicefleet.ai/assistants/<id> \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dtmf_enabled": true}'

The model can then call send_dtmf({digits: "1234#"}); we relay through Telnyx Call Control send_dtmf. Digits accepted: 0-9 * # w W (w/W = half-second pause).

Voicemail handling (outbound)

For outbound calls only, the assistant can be instructed how to react when it lands on a voicemail / answering machine instead of a human. Three modes:

• continue (default) — talk to the machine the same as a human.
• hang_up — stay silent and end the call as soon as voicemail cues are detected.
• leave_message — wait for the beep, say voicemail_message verbatim, then hang up.

curl -X PATCH https://assistant.voicefleet.ai/assistants/<id> \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "voicemail_action": "leave_message",
    "voicemail_message": "Hi, this is an automated reminder about your appointment tomorrow at 3 PM. Please call us back."
  }'

Detection is currently model-based (the model identifies voicemail cues from the greeting audio and a prompt block injected only on outbound calls). It's effective for most mainstream voicemail greetings, but not as deterministic as carrier AMD. If you need stricter handling at scale, we can wire Telnyx AMD later.

Knowledge base

Plain-text documents (FAQs, business info, policies) that get injected into the assistant's system prompt at the start of every call. No embeddings — the documents are sent verbatim, so the model has everything in context. Best for small, stable corpora (hours, prices, scripts).

Limits

• 50 KB per document.
• 200 KB total injected per assistant per call (excess is truncated).
• Token cost: the docs ride the cacheable prefix of the prompt, so repeated calls hit the cache (~98% discount on cached input tokens).

Create

curl -X POST https://assistant.voicefleet.ai/knowledge \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "business-hours",
    "content": "We are open Monday through Friday 9 AM to 6 PM Central, Saturday 10–4. Closed Sundays and federal holidays."
  }'

Attach to an assistant

curl -X PUT https://assistant.voicefleet.ai/assistants/<id>/knowledge \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"knowledge_ids": ["<doc-id-1>", "<doc-id-2>"]}'

This replaces the attached set — pass an empty array to detach all.

Manage

GET    /knowledge
GET    /knowledge/:id
PATCH  /knowledge/:id     # any subset of name/content
DELETE /knowledge/:id

Simulations

Test assistants before (and after) every prompt change. A persona is a mock caller — an LLM playing a character. A scenario gives that caller a goal and defines the checks that decide pass/fail. A simulation pairs an assistant with one persona and one scenario; a run executes it live and grades the transcript. Runs are asynchronous: creating one returns status: "queued" immediately, and each run also produces a call-log row (source: "simulation") with the full transcript, tool events, cost and — for voice runs — the recording.

Modes

• chat — the assistant's real pipeline driven as text (same instructions, tools, model; the cascading pipeline runs its actual LLM leg, the realtime pipeline runs a real Realtime session with text output). Fast and cheap; good for CI.
• voice — full audio: the persona speaks TTS audio through the same media path a browser test call uses, with real turn-taking, and the run gets a WAV recording.

Personas

curl -X POST https://assistant.voicefleet.ai/simulations/personas \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Impatient Dublin caller",
    "system_prompt": "You are Dara, 34, calling from a noisy office. You are blunt, skip pleasantries, and give details only when asked twice.",
    "tts_provider": "deepgram",
    "tts_voice": "flux-jack-en"
  }'

llm_model overrides the caller model (default SIMULATION_CALLER_MODEL); tts_provider/tts_voice/tts_model/voice_speed shape the voice-mode voice and fall back to the server TTS defaults when null.

Scenarios & evaluations

curl -X POST https://assistant.voicefleet.ai/simulations/scenarios \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Book a table for four",
    "instructions": "Book a table for 4 people this Friday at 7pm under the name Aoife Byrne, callback 087 123 4567. Do not accept Saturday.",
    "max_turns": 10,
    "tool_mocks": { "book_table": { "ok": true, "confirmation_id": "BK-1042" } },
    "evaluations": [
      { "id": "tool-booked", "kind": "tool_call", "tool": "book_table",
        "status": "completed", "path": "arguments.party_size", "equals": 4 },
      { "id": "phone-ok", "kind": "tool_call", "tool": "book_table",
        "path": "arguments.phone", "equals": "0871234567", "digits_only": true },
      { "id": "no-ai-talk", "kind": "transcript", "role": "assistant",
        "excludes": "as an ai", "required": false },
      { "id": "clean-end", "kind": "call", "path": "ended_reason",
        "equals": "assistant_ended_call" },
      { "id": "date-agreed", "kind": "ai",
        "question": "Which day of the week did they finally agree on? Answer with one lowercase day name, or \"none\".",
        "expected": "friday", "comparator": "=" }
    ]
  }'

Four check kinds, all with optional id, name and required (default true):

• tool_call — the assistant called a tool. tool (required), status (completed|error), occurrence (0 = first match), path (dot-path into the event, e.g. arguments.party_size), equals (scalar or array of accepted values).
• transcript — the conversation text. role filters to one side; includes / excludes are substring checks.
• call — the finished call row. path + equals (e.g. ended_reason).
• ai — a judge model (SIMULATION_JUDGE_MODEL, default gpt-4o-mini) answers question from the transcript; the answer is compared against expected with comparator (= != > < >= <=).

String comparisons are case-insensitive with collapsed whitespace by default; digits_only strips everything but digits first (phone numbers). Tool mocks map tool name → canned JSON: during a simulation the assistant's matching HTTP tools return the mock instead of calling your endpoint, and the tool event is flagged mocked: true. Built-ins (end_call etc.) are never mocked.

Simulations & runs

# pair assistant x persona x scenario
curl -X POST https://assistant.voicefleet.ai/simulations \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"assistant_id": "<id>", "persona_id": "<id>", "scenario_id": "<id>", "name": "Booking x impatient"}'

# start one run (async — poll it)
curl -X POST https://assistant.voicefleet.ai/simulations/<id>/runs \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"mode": "chat"}'          # or "voice"

# run every simulation of one assistant under a single batch id
curl -X POST https://assistant.voicefleet.ai/simulations/runs/batch \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"assistant_id": "<id>", "mode": "chat"}'

GET    /simulations                 # ?assistant_id= filter
GET    /simulations/runs            # ?simulation_id=&assistant_id=&batch_id=&status=&page=&limit=
GET    /simulations/runs/:id        # includes the linked call (transcript, recording_url, ...)
DELETE /simulations/runs/:id
GET/POST/PATCH/DELETE /simulations/personas[/:id], /simulations/scenarios[/:id], /simulations[/:id]

Run result

{
  "id": "…", "status": "completed", "passed": true, "mode": "chat",
  "batch_id": null, "call_id": "…",
  "results": {
    "checks": [
      { "id": "tool-booked", "kind": "tool_call", "required": true,
        "passed": true, "expected": 4, "actual": 4, "reason": null },
      { "id": "date-agreed", "kind": "ai", "comparator": "=",
        "passed": true, "expected": "friday", "actual": "friday", "reason": null }
    ],
    "required_checks": 4, "passed_checks": 4,
    "judge": { "model": "gpt-4o-mini", "skipped": false, "error": null },
    "caller": { "model": "gpt-4.1-mini", "turns": 5, "hung_up": true, "timed_out": false },
    "duration_ms": 14211
  }
}

passed is true when every check with required: true passed. Lifecycle: queued → running → completed | error (error means infrastructure failed — the caller LLM, a vendor socket — not that a check failed). A simulation.completed webhook fires for every finished run, carrying the run row above. CI can gate deploys on it, or on polling GET /simulations/runs?batch_id=….

Config

SIMULATIONS_ENABLED=1               # default on; 0 disables the run endpoints (503)
SIMULATION_JUDGE_MODEL=gpt-4o-mini
SIMULATION_CALLER_MODEL=gpt-4.1-mini
SIMULATION_MAX_TURNS=12             # default caller-turn cap (per-scenario max_turns overrides)
SIMULATION_MAX_CONCURRENT=2
SIMULATION_TIMEOUT_MS=180000        # whole run
SIMULATION_TURN_TIMEOUT_MS=30000    # one assistant turn (chat mode)

Webhooks

We POST signed JSON to your URL when calls start and end, when a simulation run finishes (simulation.completed), and—when subscribed—when an assistant is created, updated, or deleted. Create a webhook in the dashboard or via API:

curl -X POST https://assistant.voicefleet.ai/webhooks \
  -H "Authorization: Bearer af_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://your-app.com/hooks/assistantfleet", "events": ["call.started", "call.ended", "simulation.completed", "assistant.created", "assistant.updated", "assistant.deleted"]}'

The response includes the signing secret — shown once.

Assistant lifecycle payloads use the same shape as GET /assistants/:id, including revision. A delete payload contains id, the last revision, and updated_at. Lifecycle events are opt-in so existing webhook registrations retain their previous event set.

Event payload

POST https://your-app.com/hooks/assistantfleet
Content-Type: application/json
X-AssistantFleet-Event: call.ended
X-AssistantFleet-Delivery: 9f8a-…
X-AssistantFleet-Signature: sha256=<hex hmac of raw body>

{
  "event": "call.ended",
  "delivery": "9f8a-…",
  "created_at": "2026-05-24T12:03:02Z",
  "data": { /* same shape as the GET /calls row */ }
}

Verifying signatures (Node.js)

const crypto = require('crypto');

app.post('/hooks/assistantfleet', express.raw({ type: '*/*' }), (req, res) => {
  const sig = req.header('X-AssistantFleet-Signature') || '';
  const expected = 'sha256=' + crypto
    .createHmac('sha256', process.env.AF_WEBHOOK_SECRET)
    .update(req.body)            // raw Buffer
    .digest('hex');
  if (!crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected))) {
    return res.status(401).end();
  }
  const event = JSON.parse(req.body.toString());
  // handle call, simulation, or assistant lifecycle events
  res.status(204).end();
});

Retries

Call and simulation webhooks retry 5xx and 429 with exponential backoff (default up to 4 retries). Assistant lifecycle events first enter a durable SQLite outbox, survive process restarts and deploys, and retry with capped exponential backoff for 24 hours by default. Other 4xx responses are permanent and are not retried. Replies are expected within 5 seconds — return early and process asynchronously.

Manage webhooks

GET    /webhooks
PATCH  /webhooks/:id   # body { enabled: false } to pause
DELETE /webhooks/:id

Models & voices

Models

AssistantFleet offers current general-purpose, bidirectional OpenAI Realtime models and Gemini Live native audio. Translation-only and transcription-only models are not assistant replacements and are not shown.

• gpt-live-1 — continuous voice with a separately configurable delegated model for business reasoning and tools.
• gpt-realtime-2.1 — latest flagship reasoning voice model with tool use; current AssistantFleet default.
• gpt-realtime-2.1-mini — latest faster, lower-cost reasoning voice model.
• gpt-realtime-2 — previous flagship generation.
• gpt-realtime-1.5 — flagship audio model for voice agents and customer support.
• gpt-realtime
• gpt-realtime-mini — cheapest, good for simple flows.
• gemini-3.1-flash-live-preview — Google's low-latency native audio-to-audio preview with synchronous function calling and thinking levels.

Existing assistants keep any previously stored model identifier, including older models that are no longer offered for new selections. Availability still depends on the models enabled for your OpenAI account.

GPT-Live configuration

Select pipeline_mode: "realtime", realtime_provider: "openai" and model: "gpt-live-1". GPT-Live speaks and listens; the delegated model uses your Business Instructions, knowledge and attached tools. The starting configuration is GPT-5.6 Terra, low reasoning and Standard processing. Tool calls run sequentially through the existing booking confirmation checks. Higher reasoning and Fast processing should be tested for task success, latency and cost on your workload.

{
  "model": "gpt-live-1",
  "voice": "willow",
  "live_settings": {
    "backend_model": "gpt-5.6-terra",
    "reasoning_effort": "low",
    "service_tier": "default",
    "max_output_tokens": 4096,
    "voice_instructions": "Speak naturally with an Irish accent; keep booking details clear."
  }
}

Integrations with an accent picker can set live_settings.accent_instructions to the selected language/accent prompt while preserving the other settings. It goes directly to the speaking model and takes precedence over voice-style accent instructions. A Natural selection should specify the voice’s natural accent; an empty value restores the fallback Irish instruction for Willow and Stone.

Use willow (Irish feminine) or stone (Irish masculine), or the default marin. Audition with Test in browser. Realtime speed, VAD, transcription-model and response-token controls do not apply to Live. Use audio/voice simulations; chat-only simulations are unsupported. Original timed transcript fragments are preserved, including overlapping speech.

Live voice costs $0.05/minute, billed per second, plus delegated-model usage. The call log records both costs and traces the delegated model. Shutdown waits for final usage before saving the record; an estimated flag in the usage trace identifies unconfirmed totals after connection loss. See OpenAI delegation documentation.

Input transcription models

This setting controls the separate asynchronous caller transcript used in the dashboard and call logs. The Realtime assistant consumes the original audio directly, so its understanding can differ from the displayed transcript.

• gpt-4o-transcribe — best accuracy; recommended when transcript quality matters most.
• gpt-4o-mini-transcribe — lower-cost default.
• whisper-1 — legacy compatibility.

Gemini transcription models

• gemini-3.5-transcribe-live — low-latency WebSocket transcription for cascading calls.
• gemini-3.5-transcribe — unary/file transcription through POST /stt/transcribe.

POST /stt/transcribe
Authorization: Bearer af_live_...
Content-Type: application/json

{
  "provider": "gemini",
  "model": "gemini-3.5-transcribe",
  "audio_base64": "...",
  "mime_type": "audio/wav",
  "language_codes": ["en-IE"],
  "custom_vocabulary": ["Aoife", "Rathmines"],
  "mode": "verbatim"
}

For larger inputs, upload through the Gemini Files API and send its uri instead of audio_base64. Smart mode is also accepted; verbatim is the default.

Voices

OpenAI Realtime

alloy, ash, ballad, cedar, coral, echo, marin, sage, shimmer, verse. OpenAI recommends marin and cedar for best quality.

Gemini Live

Zephyr, Puck, Charon, Kore, Fenrir, Leda, Orus, Aoede, Callirrhoe, Autonoe, Enceladus, Iapetus, Umbriel, Algieba, Despina, Erinome, Algenib, Rasalgethi, Laomedeia, Achernar, Alnilam, Schedar, Gacrux, Pulcherrima, Achird, Zubenelgenubi, Vindemiatrix, Sadachbia, Sadaltager, Sulafat. Voice selects timbre; accent is prompt-steered in instructions and is not guaranteed by a locale setting.

Authenticated integrations can fetch GET /voice-presets for stable, curated voice choices. Each item includes an assistant_config that can represent either a Realtime voice or a tested cascading provider/voice combination; clients should persist the preset id.

Errors

All errors come back as JSON with the same shape:

{ "error": { "message": "name is required" } }