@pouchy_ai/companion-sdk
@pouchy_ai/companion-sdk
Embed the Pouchy companion — chat, voice, tools, memory, live world-state, Instant UI, and agent-to-agent messaging — in any app, game, or site.
All six companion capabilities are reachable over the SDK (memory · reasoning ·
skills · A2A/social · wallet · instant UI), and each rides a platform-neutral
contract — so the same companion runs on web, native iOS/Android, or a CLI. See
companion-capabilities.
Install
npm i @pouchy_ai/companion-sdk
No bundler? Pull it from a CDN via an import map instead — see
companion-integration-faq.
The package ships ESM-only ("type": "module", no CJS build). Every modern
bundler and Node ≥ 18 consumes it as-is; on Node ≥ 20.17 even require() of the
ESM entry works natively.
Changes per release are tracked in CHANGELOG.md; the
pinnable CDN releases (immutable URL + SRI per version) are indexed at
pouchy.ai/sdk/versions.
React? @pouchy_ai/react
wraps this SDK in a <CompanionProvider> + useCompanion / useMessages /
useTyping / useCall hooks so you don't wire the handshake, stream, or
teardown yourself.
Svelte / Vue? This package ships thin adapters as subpath exports —
@pouchy_ai/companion-sdk/svelte and
@pouchy_ai/companion-sdk/vue (0.31.0).
Quickstart
Two steps. 1) Your backend exchanges the project Secret Key (create one
in the dashboard → Keys) for a per-user
session token — first-seen external_user_ids are provisioned automatically:
POST https://pouchy.ai/v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }
→ { "session_token": "pchy_…", "expires_in": 3600, "instance": { … } }
2) The client connects with the session token (never ship the Secret Key):
import { createCompanion } from '@pouchy_ai/companion-sdk';
const companion = createCompanion({ baseUrl: 'https://pouchy.ai', token: sessionToken });
await companion.connect();
companion.onMessage((t) => console.log(t));
companion.start();
await companion.sendText('hey');
await companion.connectCall(); // live voice (browser)
companion.sendWorldState({ type: 'game.player.hp', data: { hp: 12 }, retained: true });
(Personal/single-user integrations can pass a server-minted Personal Access
Token as token instead — same API. See the docs' authentication section.)
- Guide:
companion-quickstart - Spec:
companion-sdk-protocol - API reference:
companion-api-reference - Capabilities map:
companion-capabilities - Instant UI renderer contract:
companion-instant-ui
Framework adapters (Svelte / Vue)
Thin, logic-free bindings over one shared view controller
(createCompanionView, also exported from the root for any other framework).
The view tracks { streamState, transcript, draft, typing, pendingConfirms }
as immutable snapshots, wraps sendText (optimistic user turn — rolled back
if the send rejects, 0.32.1, except on queued_offline / reply_timeout /
needs_event_stream, where the text did not get lost; a rejected send also
clears its dead draft and typing, 0.63.1) and confirmAction (confirm-card bookkeeping;
a confirm_resolved/confirm_not_found rejection drops the stale card,
since a confirm can be resolved elsewhere — hosted page or an IM keyword
reply), and can restore recent history with { restore: N }. You own the
client lifecycle — create/connect/start it once, hand it to the adapter,
close it when done. React hosts: use the standalone
@pouchy_ai/react package
instead (it owns the client lifecycle and adds a voice-call hook; this view owns
the richer render-state).
<script>
import { companionStore } from '@pouchy_ai/companion-sdk/svelte';
const companion = companionStore(client, { restore: 20 });
</script>
{#each $companion.transcript as turn}<p>{turn.role}: {turn.text}</p>{/each}
{#if $companion.typing}<p>…</p>{/if}
<button onclick={() => companion.sendText('hey')}>send</button>
// Vue 3
import { useCompanion } from '@pouchy_ai/companion-sdk/vue';
const { snapshot, sendText } = useCompanion(client, { restore: 20 });
// snapshot.value.transcript / .draft / .streamState — auto-disposes with scope
Constructor options (beyond baseUrl + token)
createCompanion({ … }) also takes: surface (one resumable session per
surface), modalities / handles / contextKinds / tools /
appContext (the capability handshake — since 0.77.0 connect()'s ack
carries droppedTools: [{ index, reason }] for any tools entry it refused:
missing_name / duplicate_name / over_cap), visitor (representative mode,
below), onAuthError (401 → return a fresh token and the client retries
transparently), stream: 'sse' | 'websocket' (receive transport —
'websocket' opts into the lower-latency WS plane when the deployment serves
one — pouchy.ai does not (serverless); an optional self-hosted WebSocket
gateway worker does, for a host that can hold sockets — and falls back to
SSE automatically — when the socket cannot open, and since 0.73.1 when the
gateway refuses it with a 4xxx close (4401 unauthorized, 4502 upstream
failure) — observable as streamState === 'degraded_sse'), streamBaseUrl (0.59.0 — the origin the
WS plane is served from, i.e. your self-hosted gateway; only the reply socket
goes there, every REST call and the SSE fallback keep using baseUrl; unset
derives the socket URL from baseUrl), debug (0.32.0 — true logs structured
instrumentation events via console.debug; a function receives every
CompanionDebugEvent: HTTP request/response with method/path/status/ms, each
delivered envelope post-dedup, stream-state transitions, synthesized errors —
never the token, headers, or bodies), queueOffline (0.34.0 — opt-in offline
send queue, below) with outboxStore (custom persistence for it; default
localStorage), requestTimeoutMs (0.35.0 — deadline for any request to
produce response HEADERS, default 30 000 ms, 0 disables; expiry rejects
code: 'request_timeout'; never bounds reading a streaming body; since
0.40.0/0.40.1 the long-work POSTs — whose headers only arrive when the server
work completes — default to that work's own ceiling instead of 30 s: buffered
/input sends, sendToolResult and the two knowledge ingests get 310 s,
confirmAction 65 s, so a long-running turn/ingest/action is no longer
declared dead while it succeeds server-side;
an explicit requestTimeoutMs still bounds every request at your value),
replayPendingToolCalls (0.37.0, default true — when connect() resumes a
session paused on YOUR tools, re-emit each outstanding call to onToolCall
with replayed: true; see "Resuming a paused turn" below), and two
injection points for Node/tests: fetch (custom fetch implementation) and
webSocketImpl (WebSocket constructor when globalThis.WebSocket is absent).
Events
Subscribe with on(type, fn) (or '*'), or these typed convenience helpers.
Since 0.30.0 on() narrows the envelope payload per event name
(OutboundPayloadMap) — e.g. on('companion.typing', (env) => env.payload.active)
type-checks without a cast; '*' keeps the untyped envelope:
| Helper | Event | Use |
|---|---|---|
onMessage(fn) |
companion.message |
the assistant reply (fires exactly once per turn, streaming or not). When the turn PRODUCED files (create_spreadsheet / create_document on the companion side), envelope.payload.attachments (0.54.0) carries them structurally — MessageAttachment[]: { fileId, fileName, mimeType, url }, url a 7-day capability download URL — so render a download affordance from the field, not by parsing the prose |
onDelta(fn) |
(POST-response SSE) | token streaming — registering makes sendText stream the reply as it is generated: fn(chunk, {reset?}) fires per text chunk (reset = clear the partial — that text was tool-call deliberation); onMessage then fires once with the authoritative final text (replay-deduplicated). Force per call with sendText(text, { stream: true | false }) |
onToolCall(fn) |
companion.tool_call |
the companion asks your app to run a declared tool → sendToolResult. The call carries argsJson — args pre-parsed as JSON (undefined on empty/malformed args). replayed: true (0.37.0) marks a re-delivery of a still-outstanding call after a mid-pause reconnect — treat id as an idempotency key for side-effecting tools (see "Resuming a paused turn") |
onRender(fn) |
companion.ui_action |
Instant UI — draw payload.interface (platform-neutral genui schema) with your own renderer (web / iOS / Android / CLI). Needs ui.render |
onInterfaceUpdate(fn) |
companion.ui_update |
live {key,value} update to an already-rendered panel (no rebuild) |
onSocialMessage(fn) |
companion.social_message |
inbound A2A friend message, delivered cross-app. Needs social.message |
onConfirmRequest(fn) |
companion.confirm_request |
a sensitive-op approval request. Platform session tokens (your end users): show your own confirm card and resolve it with confirmAction — this is how confirm-gated custom skills (POST / credentialed) run. First-party user tokens: observe-only — approval stays first-party (where the stepUp:true biometric gate lives). Payload { confirmId, scope, summary, summaryKey?, summaryVars?, stepUp? } — render summaryKey ? localize(summaryKey, summaryVars) : summary to show the approval in the user's language (0.43.0) |
onAudio(fn) |
companion.audio |
TTS clip of the reply, non-call modality (0.58.0): { url }, a capability URL of an mp3. Emitted after a text turn when the agent opted in (dashboard → reply cues → audio), the session negotiated voice and the token holds the voice scope. |
onExpression(fn) |
companion.expression |
avatar cue (0.58.0): { expression }, a VRM 1.0 preset (happy / sad / angry / surprised / relaxed / neutral) read from the reply's tone. Emitted after a text turn when the agent opted in (dashboard → reply cues → expression). viseme / gesture stay unused. |
onVoiceInject(fn) |
companion.voice_inject |
fn({text, speak}) — a voiceRelevant world-state line to say aloud during a live call; route text to your voice session when speak |
onTyping(fn) |
companion.typing |
activity indicator — fn({active}) fires true when a turn starts working and false when it finishes / pauses, spanning the tool-loop / thinking phase before the first text delta. Drive a "typing…" state |
on('companion.data_activity', fn) |
companion.data_activity |
metadata-only Data-plane activity (0.44.0) — the agent read a View or settled an Action: { kind, capability, outcome, ms, actionId? }. Never rows or intent values; an action outcome: "unknown" means pending verification — never render it as success or failure. Requires the data.activity scope (granted from the agent's Data flag) |
on('companion.skill_card', fn) |
companion.skill_card |
a skill's own card, rendered (0.51.0) — the companion ran a tool of an installed skill whose author declared a card_template: { skill, tool, markdown }. The shape comes from the skill's manifest and its install review, not from the model, so this is a declared shape delivered to you rather than prose to parse. markdown is TEXT, not HTML; unfilled placeholders are already stripped. Requires the skills.execute scope; a handler that never fires means the skill ships no template, not that the call failed |
on('control.call_ready', fn) |
control.call_ready |
a voice call is ready — the stream echo of startCall's accept. Deliberately secret-free ({ provider, agentId/model, voice, … } — the actual WebRTC credentials only ride the startCall HTTP response); useful for UI state on surfaces that didn't initiate the call |
onUsage(fn) |
control.usage |
per-turn metering echo, emitted after an input turn that billed tokens: fn(payload, env) where payload carries { promptTokens, completionTokens, cachedTokens, reasoningTokens?, model? } (whole-turn totals; reasoningTokens is a subset of completion) |
onError(fn) |
control.error |
agent / stream errors — agent_error (server-side turn failed after accept; safe to re-send), call_mint_failed (voice-credential mint failed — retry later or fall back to text) or the SDK-synthesized stream_unauthorized (stream 401 exhausted the onAuthError refresh retries, or the stream got a 403 — immediately terminal, commonly a token missing the events.subscribe scope; refresh/fix the token, start() again). Since 0.30.0 the vocabulary is EXPORTED TYPED: CONTROL_ERROR_CODES (runtime list, drift-tested against the server) + the ControlErrorCodeValue union fn's err.code now carries — switch on it with autocomplete |
Instant UI
import { renderInterface } from './instant-ui'; // your renderer (web example in examples/web-instant-ui)
let panel: ReturnType<typeof renderInterface> | null = null;
companion.onRender(({ interface: ui }) => { panel = renderInterface(ui, mountEl); });
companion.onInterfaceUpdate(({ update }) => panel?.applyUpdate(update));
Reference renderers ship in the repo under examples/: web-instant-ui (vanilla),
native-instant-ui (SwiftUI + Jetpack Compose), and cli-skills (terminal +
local/device skills).
Already have an A2UI-catalog renderer?
(0.42.0) toA2UI(payload.interface) projects any panel losslessly to Google's
A2UI v0.9 flat wire format (standard component/field names; Pouchy-only
concepts ride a _pouchy extension A2UI renderers ignore), and fromA2UI
ingests a surface back. INSTANT_UI_NODE_TYPES lists the atom catalog as data
for capability checks. Embedded Button / table-row actions are opaque
EmbeddedAction { type } values — hand them back to the companion, never
interpret them.
import { toA2UI } from '@pouchy_ai/companion-sdk';
companion.onRender(({ interface: ui }) => myA2uiRenderer.render(toA2UI(ui)));
Client methods (beyond the quickstart)
| Method | What it does |
|---|---|
sendText(text, { awaitReply: true, replyTimeoutMs? }) |
Request/response mode: the promise resolves with the completed turn's reply { seq, text, envelope, usage? } (SendTextReply) — no onMessage wiring or hand-rolled turn gate. usage (a TurnUsage: { promptTokens, completionTokens, cachedTokens, reasoningTokens?, model? }) is the turn's token totals for cost accounting, present when the server reported them. Works without start() when the turn completes in text (the reply rides the same streaming request). A turn that pauses on YOUR declared tools resolves via the event stream, so it DOES need start() — without it the promise rejects immediately with code: 'needs_event_stream'. replyTimeoutMs (default 60s) bounds the whole operation (0.28.0); on expiry it aborts the in-flight stream and rejects with code: 'reply_timeout'. Each turn carries a minted turnId the server echoes as replyTo on the reply (API ≥ 1.1), so the fallback resolves on THE reply — proactive messages can't steal it. Retrying safely (0.72.0): every result — plain or awaitReply — carries the turnId it went out under, and sendText(text, { turnId }) sends your own (1–256 chars since 0.73.2, 64 before; anything else rejects with code: 'invalid_option' before sending, as the server refuses it). After an AMBIGUOUS failure (reply_timeout, request_timeout, a dropped stream, a 5xx) retry under the SAME turnId: the server dedupes completed turns (60 min, last 32 per session) and replays the recorded one with duplicate: true — no model call, no bill — where a retry without it runs and bills a second turn. An awaitReply retry resolves with the recorded reply on either leg (the stream: false replay carries it since 0.73.0); if the server no longer holds that reply it rejects code: 'reply_unavailable' at once rather than waiting out replyTimeoutMs. instructions (0.63.0) carries developer-authored rules for THIS turn: joined to the prompt after the history and immediately before the user's message, and never scored as user speech by inbound moderation. Needs the chat.instructions scope (403 missing_scope without it), 32,000 chars max (413, never a silent truncation), and it is NOT persisted — it governs the turn it rides and nothing after it. Before it existed, an app with per-turn rules had to prepend them to text, where the classifier read the developer's own prose as the user's (issue #3379). A turn refused by that gate now says so: blocked / blockedBy (`'classifier' |
recall({ query?, limit? }) / remember({ content, … }) / forget(id) |
Read / write / forget in the token-scoped memory namespace. query runs SEMANTIC search over the facts server-side (embedding-based, ranked fallback); omit it for the plain importance/recency ranking. remember takes ONE fact of at most 2,000 characters — longer content throws a payload_too_large CompanionError rather than being stored truncated, so split it into separate facts (or ingest it as knowledge, which chunks document-length text for you). forget(id) (0.53.0) takes the id a recall row carries and sets the user's authoritative "stop telling me this" — the fact stops recalling and stays forgotten; needs the namespace's write scope (a 403 missing_scope CompanionError without it), answers one uniform 404 for anything the token cannot see, and is idempotent. remember resolves { cloudId, namespace, dedup } (0.60.0): dedup.matched: true means the write MERGED into a row the user already had — cloudId then names that existing row (the handle forget(id) takes), not a new one — and dedup.revived says the merged-into row had been decay-archived and is live again; dedup.skipped says why no match ran. |
ingestKnowledge({ text, name, … }) / ingestFile(file) |
Push documents into the user's knowledge base ("My materials"); needs memory.write:core. Both return truncated (0.49.0) — true means the source overran the ingest caps and the remainder was dropped, so split it and ingest the rest. |
history({ limit? }) |
Fetch the session transcript — restore chat on reload. |
setModalities([...]) |
Switch active I/O mid-session (within the token's grant). |
ping() |
Keepalive for long-idle embeds. |
setToken(token) |
Swap the bearer for every subsequent request + stream reconnect — session tokens expire (1h default), so call this when your backend re-mints one. Prefer the onAuthError constructor option to refresh on demand instead. |
pendingConfirms() / confirmAction(id, approve) |
List / resolve pending sensitive-action confirms (platform session tokens). A listed row carries status (0.61.0) — pending is one the user has never answered, exec_failed one they ALREADY approved whose run flaked, so render them differently; execUnsettled says that flake was the DEADLINE and the action may still complete on its own; stepUp is the same advisory the live confirm_request carries, so a card rebuilt after a reload can announce the passkey gate instead of meeting it as a 401 step_up_required. Switch on the response's outcomeClass (0.55.0) — success / error / unknown / rejected / denied / unrecognized — to tell an action that ran from one that failed or was refused; status: 'approved' alone only says the approval was accepted, and outcome is prose for a person. On { status: 'exec_failed', retryable: true } you MAY re-call the same confirmId — idempotent actions only. mandate (0.57.0) — present only after a passkey step-up on the first-party surface — is the receipt: { intentDigest, credentialId, signedAt }, a signature bound to WHAT was approved. |
getMandate(confirmId) |
The full user-signed approval document behind a confirm's mandate receipt (0.59.0; design #27 Phase 2): `{ confirmId, status, intent, intentDigest, mandate, credential: { credentialId, publicKey, format } |
endSession({ transcript?, callGen? }) |
Distill the session into durable memory now; returns { ok, facts?, extracted?, filtered?, merged?, revived?, writeError?, skipped? } — skipped: 'no_session' | 'no_content' | 'throttled' | 'no_visitor_scope' (0.56.0: a representative session whose token lacks represent:remember); filtered (0.56.0) = proposed facts the identifier filter refused, so extracted − facts does not read as a refused write; merged (0.62.0) is how many of facts folded into a memory the store ALREADY held instead of minting a new one (write-time dedup), so facts − merged is the count of NEW memories — a repeated endSession() re-reads the same recent turns and routinely re-extracts what it already stored, which facts alone reported as growth (revived is the subset whose target decay had archived and the merge brought back); extracted (0.48.0) = facts the extraction proposed vs facts STORED, so extracted > 0 with facts: 0 means the store refused them and writeError says why (e.g. missing memory-write scope). Also closes the server's 20-min call-active window — generation-fenced since 0.41.0: it sends the callGen of the last call THIS client minted (null if none), so a text client's teardown never kills a live call's window on the same session, and a stale call handle's late /end can't clear a newer call's window. Self-plumbed calls: pass startCall's creds.callGen here at call end. |
close() |
One-call teardown: stop() + endSession() — the text-session mirror of the call handle's close(). |
lastEndSessionError |
Read-only accessor (0.77.0): when the last endSession() / close() returned null because the SERVER refused it, { status, code? } (e.g. 401 invalid_token, 404 session_not_found, 429 rate_limited); null after a call that got a diagnostic or never reached the server (no session / fetch rejected). endSession() still returns null on a refusal — read this to tell "token expired, nothing consolidated" from "nothing to do" without wiring debug. |
startCall(opts) / connectCall(opts) |
Voice plane: raw credentials / fully-wired call handle (call.close() ends + folds the transcript into memory). audioTags: true (0.64.0) asks for performed voice cues — see below. record: true (0.65.0) records the whole call — see below; fetch it with getCallRecording(callGen) / delete it with deleteCallRecording(callGen). The mint stamps a 20-min server-side call-active window (routes voiceRelevant world-state into the live call) and returns its callGen fencing handle (0.41.0, see endSession). On the EL path it also returns voiceToolToken (0.46.0) — the in-call memory tool's credential, which connectCall passes for you and a self-plumbed host must pass under BOTH names (0.47.0): dynamicVariables: { [VOICE_TOOL_VARIABLE]: creds.voiceToolToken, [VOICE_TOOL_VARIABLE_PLAIN]: creds.voiceToolToken } — ElevenLabs ignores a CLIENT-supplied secret__ variable, so the plain twin is the one that arrives; omitting it throws nothing and silently disables recall (see below). If connectCall throws AFTER a successful mint (mic denied, missing @elevenlabs/client, SDP failure — 0.40.1), it tears down like an instant hang-up first: /end closes the window, then the original error is rethrown. |
streamState / onStreamStateChange(fn) |
Receive-stream lifecycle (0.30.0): 'idle' | 'connecting' | 'connected' | 'reconnecting' | 'degraded_sse' | 'stopped' (CompanionStreamState). fn(state, prev) fires on change only — drive a connection indicator; degraded_sse = the WS transport errored and delivery continues on the SSE fallback; stopped = stop()/close() or a permanent stream_unauthorized failure. |
sessionId |
Read-only accessor: the session id of the last successful connect() (HelloAck.session), or null before the first. Not cleared by endSession()/close()/setToken() — an identifier for correlating client-side logs with server-side records (recall audit, transcripts), not a liveness check. It changes when a re-handshake mints a NEW session (idle past the server's ~12h resume window, or after endSession()); since 0.54.2 an open event stream follows it — connect() restarts the stream on the new id, where before it stayed bound to the old session and every reply for the new one went undelivered. |
getAvatar() / brandIconUrl(size?) |
The user's avatar (VRM/portrait) and the Pouchy brand icon for your UI. Standalone (no client/token): pouchyBrandIconUrl(baseUrl, size?) derives the same URL before/without connecting. |
getWallet() |
Read the instance's own wallet — { balances: [{ currency, amount }], totalUsd, currency }. Read-only + receive-only; needs the wallet.read scope. |
flushOutbox() / pendingOutbox() / onOutboxChange(fn) |
Offline send queue (0.34.0, opt-in queueOffline: true). A sendText whose fetch fails at the NETWORK level (or navigator.onLine === false) resolves { seq: null, queued: true, turnId } instead of rejecting; with awaitReply: true it rejects with code: 'queued_offline' (the item is still queued). HTTP errors (4xx/5xx incl. 429) are never queued — the server saw those; a 2xx whose body is not JSON (code: 'bad_response', 0.77.0 — a captive portal's HTML page) is queued as an ambiguous send. The queue replays FIFO — automatically after connect() and whenever the stream reconnects, or manually via flushOutbox() → { sent, remaining } — each item a buffered POST reusing its ORIGINAL turnId, which the server dedupes (completed turnIds, 60 min / last 32 per session). Replay is double-run-safe past that window too: a provably-unsent item (QueuedSend.unsent, navigator.onLine was false) replays at any age, an ambiguous item is auto-replayed only under OUTBOX_AMBIGUOUS_MAX_AGE_MS (30 min) and dropped past it, and a transient 409/408/423/425 mid-flush is kept (only a terminal 4xx drops the item). Since 0.50.0 the keep-arm is BOUNDED for the one class that carries no status at all: after OUTBOX_MAX_REPLAY_ATTEMPTS (5) attempts that rejected at the network level while navigator.onLine !== false, an item older than the 30-min window is dropped instead of blocking the queue behind it forever — the shape a host-refused oversize body produces (downscale images before sending). pendingOutbox() lists the queued sends (QueuedSend[]); onOutboxChange(fn) fires on every enqueue/flush/drop (drive a "N pending" badge). Persistence: localStorage by default (in-memory page-lifetime fallback), or bring your own via the outboxStore option; the key is pouchy-outbox:<surface>, and on a representative session (visitor) pouchy-outbox-v:<surface>:<visitorId> since 0.50.1 — a visitor-scoped session gets a visitor-scoped queue, so one visitor's queued sends can never replay into another's thread on a shared browser. |
Tool calling beyond your own declared tools — voice calls only: on
connectCall() sessions the SDK automatically offers HOST_CONTROL_TOOLS
(universal verbs — invoke_action, set_feature, set_value, navigate,
highlight) once you declare tools/handles, and AVATAR_VISUAL_TOOLS
(play_gesture / play_expression) let avatar-rendering embeds receive
expression cues — declare them or they no-op silently. Text turns expose
exactly the tools[] you declared (nothing is auto-added), so declare the
verbs there yourself if you want them outside calls. Full contract:
companion-host-control.
Cancellation (0.29.0, completed in 0.29.1): every request-performing method
accepts an optional AbortSignal — including sendToolResult(callId, result, { signal }) and startCall({ signal }) since 0.29.1. Methods with an options
object take a signal? field (sendText(text, { signal }),
recall({ signal }), …); the rest take a trailing opts
(getAvatar({ signal }), ping({ signal }), …). Aborting rejects with
CompanionError code: 'aborted', and for sendText({ awaitReply }) it also
tears down the in-flight event stream.
const ac = new AbortController();
const p = companion.sendText('summarize this', { awaitReply: true, signal: ac.signal });
// …user navigated away
ac.abort(); // p rejects with code: 'aborted'
Offline send queue (0.34.0, opt-in): queueOffline: true turns a
network-level send failure (fetch rejected / navigator.onLine === false)
into a queued send instead of a lost one — replayed FIFO on reconnect with
its original turnId (server-deduped, so a retry can never double-run a
turn; see the methods table above for the full per-item semantics). Since
0.39.0 the replay is double-run-safe even after a long offline gap: a send
known to have never reached the server (navigator.onLine was false) replays
at any age, while an AMBIGUOUS one (the turn may have run server-side) is
replayed only while younger than OUTBOX_AMBIGUOUS_MAX_AGE_MS (30 min) and
dropped past that rather than risk a duplicate; a transient 409 mid-flush
keeps the item instead of discarding it:
const companion = createCompanion({ baseUrl, token, queueOffline: true });
await companion.connect();
companion.onOutboxChange((items) => render(items.length)); // "N pending" badge
const r = await companion.sendText('hey');
if (r.queued) render('queued — sends on reconnect', r.turnId);
Resuming a paused turn (0.37.0): a turn that pauses on YOUR declared tools
used to be unrecoverable if the embed lost its state (tab reload / crash
mid-pause) — the only unblock was abandoning it via endSession() (/end).
Now connect() on a session resumed mid-pause returns the outstanding calls as
HelloAck.pendingToolCalls (PendingToolCall[] — { id, name, args, turnId?, pausedAt? }; always present, empty when none), and — unless you pass
replayPendingToolCalls: false — re-emits each to your onToolCall handler
with replayed: true (one tick after connect() resolves, at most once per
client instance), so your existing tool loop completes the turn instead of
losing it. Register onToolCall before connect()/start(). Because the
app may have already PERFORMED a replayed call and died before posting the
result, side-effecting tools should treat id as an idempotency key when
replayed is set (the server's result apply is idempotent per id too — a
double sendToolResult for the same id is safe); use pausedAt to judge
staleness before re-running something expensive:
declare function runTool(name: string, args: unknown): Promise<string>; // yours
const performed = new Set<string>(); // your durable "already ran" ledger
companion.onToolCall(async ({ id, name, argsJson, replayed }) => {
if (replayed && performed.has(id)) {
await companion.sendToolResult(id, { ok: true, result: 'already applied' });
return;
}
performed.add(id);
const result = await runTool(name, argsJson);
await companion.sendToolResult(id, { ok: true, result });
});
const ack = await companion.connect(); // resumed mid-pause → ack.pendingToolCalls
companion.start();
Scopes (typed)
The full capability-scope vocabulary ships as typed constants — no more copying scope strings from prose:
import {
COMPANION_SCOPES, SENSITIVE_SCOPES, DEFAULT_SCOPES,
hasScope, grantsScope, isSensitiveScope, type CompanionScope
} from '@pouchy_ai/companion-sdk';
const ack = await c.connect();
if (!hasScope(ack.grantedScopes, 'skills.execute')) {
// hide the "run a skill" affordance instead of eating a 403
}
// Gate a READ-ONLY affordance with grantsScope, not hasScope: a key granted
// only `wallet.spend` can still read the balance, so hasScope(g,'wallet.read')
// would wrongly hide it. grantsScope honours that subsumption.
if (grantsScope(ack.grantedScopes, 'wallet.read')) {
// safe to show the wallet balance (getWallet() will succeed)
}
COMPANION_SCOPES lists every scope a key can carry; SENSITIVE_SCOPES are
the ones that are never on by default (money / skills / social / core memory /
the representative plane); DEFAULT_SCOPES is what a fresh key gets.
hasScope(granted, required) is an exact-grant check; grantsScope(granted, required) additionally honours scope subsumption (wallet.spend ⊇
wallet.read) — use it to gate whether an API call will succeed. Also
exported: COMPANION_MODALITIES, REPRESENT_SCOPES, DEFAULT_MODALITIES,
isRepresentScope(). The list mirrors the server vocabulary and is guarded by
the SDK's drift test.
The wire-protocol runtime constants are exported too — PROTOCOL_VERSION,
INBOUND_TYPES, OUTBOUND_TYPES, and COMPANION_ERROR_CODES (the append-only
HTTP code vocabulary behind CompanionError.code, e.g. missing_scope /
session_not_found / turn_pending) — so you can assert the version, validate
the event vocabulary, or switch on error codes without hard-coding strings.
Every helper populates CompanionError.code when the server names a cause
(0.24.0 — previously only the POST-backed calls did). 0.27.0 adds
rate_limited for the runtime's 429 cost gates (turn burst ceiling, demo
daily budget); on those failures CompanionError.retryAfter carries the
server's Retry-After in seconds, so back off with
if (e.code === 'rate_limited') await sleep((e.retryAfter ?? 1) * 1000)
(retryAfter is undefined when the server names no window). The SDK also synthesizes
a few client-side codes outside that HTTP vocabulary: reply_timeout
(awaitReply fallback gave up), stream_unauthorized (event-stream 401/403 —
a 401 exhausted the token-refresh retries; a 403 is immediately terminal,
commonly a missing events.subscribe scope), and — 0.26.0 — the voice connect-step codes
call_unsupported (no WebRTC/mic in this environment), call_connect_failed
(mic timeout / SDP exchange failed) and call_dependency_missing
(@elevenlabs/client not installed). 0.28.0 adds the confirm/step-up codes
(confirm_not_found, confirm_resolved, step_up_required, step_up_failed),
the client-side needs_event_stream, and types CompanionError.code as the
CompanionErrorCodeValue union (autocomplete in your switch; still open for
newer servers). 0.34.0 adds queued_offline (a send parked in the offline
outbox for replay) and 0.35.0 adds request_timeout (the requestTimeoutMs
headers deadline fired). 0.72.0 adds invalid_option (an option the SDK
refused before sending, e.g. a turnId outside 1–256 chars) and 0.73.0
reply_unavailable (an awaitReply retry of a completed turnId whose
reply the server no longer holds). 0.45.0 adds the server code quota_exhausted (402 —
the account's monthly allowance is spent, credits or realtime voice minutes,
e.g. startCall past the plan's voice ceiling): not retryable, unlike
rate_limited — upgrade the plan or wait for the 1st (UTC) reset. The authoritative reference is the exported
CompanionErrorCodeValue union on CompanionError.code (the internal
SDK_SYNTHESIZED_ERROR_CODES list is drift-tested but deliberately not part
of the public index surface). (stream_unauthorized above is a
control.error/onError code, not a thrown CompanionError.code.) Every
outbound payload has a named type; ToolCallPayload ({ id, name, args, replayed? }) types the companion.tool_call event, MessagePayload
({ text, replyTo?, attachments? }) types companion.message (attachments:
MessageAttachment[] — files the turn produced, 0.54.0), PendingToolCall (0.37.0)
types the HelloAck.pendingToolCalls entries, and onToolCall's callback
receives ToolCallEvent — the payload plus argsJson (pre-parsed args).
On a persisted server fault (a real 5xx) CompanionError.errorId (0.56.0)
carries the server's err_… reference — quote it to support; it is the id the
server's own error row is keyed by. Absent on 4xx and transport failures.
For advanced voice hosts, the lower-level
openCompanionCall(creds, options?, bridge?) primitive is exported too — it
consumes the CallCredentials that startCall() mints (bring-your-own call
lifecycle; see companion-integration-faq).
In-call memory recall (voiceToolToken, 0.46.0)
On the ElevenLabs path the agent carries a server-side webhook tool that
reads the session's memory mid-call. It authenticates with the call-scoped
creds.voiceToolToken, which you hand to EL as a dynamic variable at session
start:
import { VOICE_TOOL_VARIABLE, VOICE_TOOL_VARIABLE_PLAIN } from '@pouchy_ai/companion-sdk';
const creds = await companion.startCall();
if (creds.provider === 'elevenlabs-convai') {
// Conversation.startSession({
// conversationToken: creds.token, connectionType: 'webrtc',
// dynamicVariables: {
// [VOICE_TOOL_VARIABLE]: creds.voiceToolToken,
// [VOICE_TOOL_VARIABLE_PLAIN]: creds.voiceToolToken // 0.47.0 — the one EL actually substitutes
// }
// })
}
EL substitutes it into that tool's Authorization header and never sends the
secret__ class of variable to the LLM, so the credential rides a live
conversation without entering a prompt.
connectCall() handles this automatically. If you plumb the call yourself, this
is the one omission that surfaces no error: EL substitutes the agent's empty
default, the tool call 401s, and the companion simply says it doesn't recall
anything — a call that sounds entirely normal with memory switched off. The
field is absent (not empty) when the session holds no memory grant; pass it
through as-is and omit the key when it is missing.
Representative mode (代聊)
Pass a visitor and the session flips from owner-facing to representative:
the companion answers that visitor on the owner's behalf (代聊 /
customer-service) using only screened owner context — never the owner's private
memory, system prompt, or PII. Works over text and voice.
const c = createCompanion({
baseUrl: 'https://pouchy.ai',
token: OWNER_PAT, // must hold the `represent` scope
surface: 'support-widget',
appContext: { name: 'AcmeShop', description: 'Order support' },
visitor: { id: stableVisitorId, displayName: 'Sam' } // stable per end-user
});
const ack = await c.connect(); // → { representative: true, visitorPaired, … }
await c.sendText('do you ship to Canada?');
const call = await c.connectCall(); // same, over voice
Scopes (granted by the owner's "Let it represent you to visitors" key toggle):
| Scope | Adds |
|---|---|
represent |
required — open a visitor-facing session |
expose:knowledge |
answer from the owner's knowledge base ("My materials") |
expose:facts |
share a wider set of screened facts |
represent:remember |
durable per-visitor notes across visits (isolated store) |
represent:pair |
c.pairVisitor(visitorPAT) — pair a visitor who's also a Pouchy user → unlock A2A. A new pairing returns request.approveUrl for the visitor to approve on pouchy.ai; poll c.getPairRequest(id) (0.74.0) |
Without represent, supplying a visitor is rejected (403). First-party tools
(wallet / skills / social) are withheld; your own declared tools still work. Full
walkthrough: API reference §3.3–3.4.
Voice (optional dependency)
Zero-code alternative: if you just want a chat box, skip the SDK entirely —
<iframe src="https://pouchy.ai/embed?token=…&theme=dark&accent=%23ff6b81"> is a
hosted drop-in widget with a postMessage control plane
(docs).
connectCall({ bargeIn: true }) opts into full-duplex voice: the mic stays
open while the companion speaks so the user can interrupt mid-utterance (the
providers' native interruption takes over). The default is half-duplex — on
phone speakers echo cancellation can't fully remove the companion's own voice
and it would interrupt itself. Enable it only where AEC holds (headphones,
desktop, or devices you've verified).
connectCall({ audioTags: true }) (0.64.0) lets the companion perform
short voice cues on an ElevenLabs call: it laughs, sighs, whispers and speaks
softly instead of only saying words (Eleven v4). Opt-in, because the provider's
transcript then contains the cues ([laughs] 你也太可爱了吧) — connectCall
strips them before onTranscript and before the transcript reaches memory. The
server may decline (OpenAI Realtime fallback, or the operator switched it off);
startCall credentials carry audioTags: true only when granted. Plumbing the
ElevenLabs session yourself? Run every agent transcript line through
stripAudioTags(text) before you show or store it.
To see what was performed (0.70.0): on a granted call every companion
onTranscript line carries audioTags: string[] (['sighs', 'laughs'], or
[]); the field is absent on user lines and on calls without the grant.
Self-plumbed: extractAudioTags(line) gives the same list.
onEnd(end) (0.71.0) says why a call ended, once per call, on every path:
end.reason is 'user' (your close()), 'agent' (the companion hung up
with ElevenLabs' end_call tool), 'remote' (the provider closed it cleanly
without saying why) or 'error' (end.error; onError got it first). Passing
onEnd moves a clean far-side end out of onError, which then means failure
only; without it nothing changes. 'agent' needs the ElevenLabs agent to send
its tool responses to the client (otherwise it reads 'remote'), and
openai-realtime has no clean remote end. connectCall sends the call to
/end on every end, the companion's hang-up included.
connectCall({ record: true }) (0.65.0) records the whole call — both
voices, kept by the voice provider, so a host that cannot capture audio on
the device (React Native's native WebRTC has no WebAudio / MediaRecorder) gets
the same recording as a browser. Turn it on per agent in the dashboard (the
agent's recording switch, which grants call.record); it runs on ElevenLabs
only, and you own telling readers their calls are recorded. The handle says what happened
before the call connects: call.recording is true, or false with
call.recordingUnavailable ('not_enabled' | 'not_configured' | 'no_conversation_id' | 'store_failed') — tell the reader the truth either way.
After the call, getCallRecording(call.callGen) answers { state: 'ready', audio, contentType } (an MP3 Blob), { state: 'pending', retryAfterSec }
while the provider is still processing, or { state: 'none' | 'expired' | 'deleted' }. Recordings are kept for 7 days: copy the file to your own
storage. deleteCallRecording(callGen) removes the provider's copy (audio and
transcript) — use it when a reader deletes their account. Self-plumbed:
startCall({ record: true }) returns the same recording /
recordingUnavailable / callGen on the credentials.
connectCall({ avatar: { faceId } }) (0.75.0) puts a talking-face video
avatar on the call. Pouchy runs it on Simli and holds the Simli key: the
handle carries call.video — { provider: 'simli', faceId, sessionToken, iceServers, maxSessionLength, model } — which you open with
simli-client and feed the
companion's remote audio track; the voice is unchanged, the face lip-syncs to
it. You build the faces yourself, with your project's admin key: admin-sdk
createVideoFace(image) (or POST /v1/admin/video-faces) builds one from an
image of your character, and listVideoFaces() reports it ready once the
build finishes (minutes to hours; billed per face built). faceId must be one
of your project's ready faces, and the session needs the call.video scope,
which it gets once the project has one — withhold it per reader with
withhold_scopes to keep the voice and drop the face. A call that asked and got
no face still connects, with call.videoUnavailable ('not_enabled' | 'not_configured' | 'face_not_allowed' | 'face_not_ready' | 'mint_failed';
face_not_ready since 0.75.1: the face is yours but still building). Billed per minute on
top of the voice. Self-plumbed: startCall({ avatar }) returns the same fields
on the credentials.
avatar.model (0.76.0) picks Simli's animation model: 'fasttalk' (Simli's
default, used when you omit it) or 'artalk'. call.video.model says which
one the session was minted with. Same per-minute rate either way — try both on
your own face; how expressive it reads depends on the face image too.
connectCall({ brief }) (0.67.0) tells the companion what this call is
about: your application's note ("she just got the acceptance letter", "you
are calling from the station"). It opens on it and keeps to it, and never reads
it out. Up to 1,000 characters. It needs the chat.instructions scope (the
agent's per-turn instructions switch), because it joins the prompt as your
words and is not moderated as user speech; send it from a token your backend
minted.
A call made with a Pouchy World role's session token carries where the
story is: the scene, the story clock, flags, the facts already revealed, and
the role's own last beats. PUBLIC state only — never the role's secrets,
notes, goals or unrevealed facts — because a call's instructions reach the
reader's device (the ElevenLabs web SDK takes its prompt as a client-side
override); keep anything the reader must not see out of the persona and
brief too. call.worldBrief (and
creds.worldBrief) says 'included', 'no_story' or 'unavailable' (the
call went ahead without it). call.privateStory (0.68.0) says whether the role
can reach its private layer on this call: 'tool' (the in-call
recall_private_story tool, served ElevenLabs ↔ Pouchy, never through your
page) or 'none' (the role keeps it vague). call.trimmed (0.68.0) reports the
characters the server cut to keep the instructions within the realtime cap
({ story?, context? }; your brief is never cut) and is absent when nothing
was. Nothing said on the call changes the story: after
the call, fold it back in as a beat on the world turn door with
trigger.context: "this was a phone call", the gist as text, and any
proposedPatches the call should leave behind.
Mute and typing (0.69.0). call.setMuted(true) mutes the reader's
microphone until setMuted(false); it holds across the companion's lines (the
half-duplex re-open never undoes it), and connectCall({ muted: true }) opens
the call muted. Show a mute control only when call.canMute is true.
call.sendText(text) sends a line the reader TYPED as their turn: the companion
answers it as something they said, it reaches onTranscript as
role: 'user', and it is in the memory consolidated at hang-up. It cuts an
utterance in progress, like speaking would. Nothing is sent before you call it,
so run any check of your own first. injectEvent is different: a context
line, never reported as the reader's words.
call.interrupt() (0.36.0) is the silent cut: stop the utterance currently
being spoken WITHOUT generating new speech — for real-time commentary going
stale at a beat boundary (next hand starts while last hand's remark is still
playing). Idempotent; check call.provider for the tier: openai-realtime is
guaranteed (response.cancel + output-buffer clear — playback stops within ~an
audio frame, the session continues); elevenlabs-convai is best-effort (the EL
client exposes no public stop-playback API — we signal user_activity and
attempt the client's internal buffer cut; worst case a no-op). For guaranteed
preemption on EL, injectEvent(text, true) — which does speak new content —
remains the reliable lever.
connectCall() for the ElevenLabs Convai provider needs the optional peer
dependency:
npm i @elevenlabs/client
OpenAI Realtime needs no extra dependency (self-contained WebRTC).
Build & publish
The source lives in the app repo at src/lib/companion-sdk and is decoupled from
any $lib import, so this package builds with plain tsc:
cd packages/companion-sdk
npm run build # tsc → dist/ (ESM + .d.ts + source/declaration maps) + src/ (shipped sources)
npm run build:cdn # cdn/companion-sdk{,.min}.js + versions/<v>/ + integrity.json (SRI)
npm publish # public scope (publishConfig.access: "public"); runs build via prepublishOnly
The CDN artifacts are committed and served from pouchy.ai:
/sdk/companion-sdk{,.min}.js (floating, 1h cache) and
/sdk/v<version>/companion-sdk{,.min}.js (pinned, immutable) — pin the
versioned URL together with the sha384 hash from cdn/integrity.json (also
published at pouchy.ai/sdk) to opt out of silent
upgrades.
The wire contract (protocol.ts) is a self-contained mirror of the server's;
protocol.drift.test.ts (run by the app's vitest) fails CI if they diverge.
Full release steps (version bump, CHANGELOG.md, drift test, tag) live in
PUBLISHING.md.
License
Proprietary — © 2026 Pouchy.ai. The published package may be installed and used
to build integrations with Pouchy's official services; all other rights are
reserved. See LICENSE.
Related Packages
A collection of Open Source agent and audio components that you can customize an...
New Schools SDK - Multi-framework components and modules for integrating New Sch...
Official Vue 3 composable and components for PurpleCallio, a developer-focused r...
Version History
| Version | Released | License |
|---|---|---|
| 0.77.0 | Custom | |
| 0.76.0 | Custom | |
| 0.75.1 | Custom | |
| 0.74.0 | Custom | |
| 0.73.2 | Custom | |
| 0.73.1 | Custom | |
| 0.73.0 | Custom | |
| 0.72.0 | Custom | |
| 0.71.1 | Custom | |
| 0.71.0 | Custom | |
| 0.70.0 | Custom | |
| 0.69.0 | Custom | |
| 0.68.0 | Custom | |
| 0.67.0 | Custom | |
| 0.66.0 | Custom | |
| 0.65.0 | Custom | |
| 0.64.0 | Custom | |
| 0.63.1 | Custom | |
| 0.63.0 | Custom | |
| 0.62.0 | Custom | |
| 0.61.0 | Custom | |
| 0.60.0 | Custom | |
| 0.59.1 | Custom | |
| 0.59.0 | Custom | |
| 0.58.0 | Custom | |
| 0.56.0 | Custom | |
| 0.55.0 | Custom | |
| 0.54.2 | Custom | |
| 0.54.1 | Custom | |
| 0.54.0 | Custom |
Showing the latest 30 of 90. See every release on npm