Production settings & security

Enterprise controls for AI Agents live at the workspace level (Settings) and per agent (Audit tab, Guardrails, Deploy).

Workspace Settings

Bring your own model (BYOK & custom SLM endpoints)

By default the whole workspace runs on Convoship's Anthropic (Claude) models. Workspace admins can override this in Settings → AI Agent model, and the override applies workspace-wide: playground, evals, red-team runs, and every public channel.

• Anthropic (BYOK) — use your own Anthropic API key and choose the Claude model.
• Custom model endpoint (SLM) — point the workspace at any OpenAI-compatible chat-completions endpoint, e.g. a small language model fine-tuned for your support domain served by vLLM, Ollama, or TGI. Host it yourself or have Convoship host it for you.
• Custom endpoints need a base URL and model name; the API key is optional for private deployments. The base URL must be publicly reachable over HTTP(S) — private and internal addresses are rejected, and all calls go through Convoship's SSRF-pinned HTTP client.
• Use 'Test connection' after saving — it fires a one-token completion and reports latency or the exact endpoint error.
• Workspace → Models (admins): create a fine-tuned model end to end — build a training dataset from your resolved conversations (explicit data-use confirmation required; transcripts are PII-redacted and re-scrubbed), start a LoRA fine-tune on an open-weight base, then deploy Convoship-hosted or download a self-host bundle and attach it to the workspace in one click.
• Run evals against any live deployment: your authored agent evals (prompt + expected outcome) are replayed against the model's endpoint and the pass rate is stored on the model — the same gate-on-evals philosophy as publishing.
• Cold start is covered two ways: Convoship vertical catalog models (pre-trained per vertical, deployable by any workspace on day one) and synthetic dataset generation (agent capabilities expanded into varied customer messages and answered by the live agent, tool calls included; stub replies are discarded).
• Tiered fallback (opt-in, off by default): if the custom endpoint fails mid-conversation, retry the turn on Convoship's default Claude model. Leave off when conversation content must never leave your endpoint — data-residency workspaces stay on the deterministic fallback instead.
• Guardrails, task routing, tool allowlists, budgets, and redaction are enforced by the Convoship runtime, not the model — they apply identically on custom models.
• If the configured endpoint errors mid-conversation, the turn degrades to the deterministic fallback instead of failing the channel.

Per-agent rate limits

check_agent_rpm enforces requests per minute per agent ID on public embed and playground. Uses Redis INCR when Redis is available; falls back to in-process counters on single-node dev.

Eval CI gate on publish

When eval_pass_threshold > 0, publish runs every AiAgentEval against the draft config, judges pass/fail, and stores results on the new AiAgentVersion. Publish returns 422 if pass rate is below threshold.

Audit trail

• AiAgentAuditEvent table — dedicated rows for agent admin actions.
• Dual-write to workspace audit_log for compliance search.
• Studio: /app/agents/{slug}/audit — last 100 events.
• API: GET /v1/ai-agents/{slug}/audit-events?limit=50
• Tool handler updates store before_state / after_state on handler_kind and config.

PII, drafts, and RLS

• PII — REDACT guardrails also redact transcript and tool_args at persist.
• RLS — Postgres policies on all ai_agent_* tables; public embed sets app.ai_agent_public_token GUC.

Encryption & KMS

Workspace secrets use Fernet (workspace-scoped key). Optional AWS KMS envelope encryption is available for BYOK API keys and secrets. Contact your account team for KMS setup.

Default guardrails at create

New agents receive five seeded guardrails (AUTH, LIMIT, REDACT, ESCALATE, BUDGET) via ai_agent_defaults.py — no GET side effects. Customize on the Guardrails tab.