@pouchy_ai/companion-sdk

Embed the Pouchy companion — chat, voice, tools, memory, live world-state, instant UI, and agent-to-agent messaging — in any app, game, or site.
14,543
Install
npm install @pouchy_ai/companion-sdk
Latest Version:0.77.0
Vue:3
License:Custom
Last Updated:Oct 22, 2026
Links: npm
Maintainer: oviswang

@pouchy_ai/companion-sdk

Embed the Pouchy companion — chat, voice, tools, memory, live world-state, Instant UI, and agent-to-agent messaging — in any app, game, or site.

All six companion capabilities are reachable over the SDK (memory · reasoning · skills · A2A/social · wallet · instant UI), and each rides a platform-neutral contract — so the same companion runs on web, native iOS/Android, or a CLI. See companion-capabilities.

Install

npm i @pouchy_ai/companion-sdk

No bundler? Pull it from a CDN via an import map instead — see companion-integration-faq.

The package ships ESM-only ("type": "module", no CJS build). Every modern bundler and Node ≥ 18 consumes it as-is; on Node ≥ 20.17 even require() of the ESM entry works natively. Changes per release are tracked in CHANGELOG.md; the pinnable CDN releases (immutable URL + SRI per version) are indexed at pouchy.ai/sdk/versions.

React? @pouchy_ai/react wraps this SDK in a <CompanionProvider> + useCompanion / useMessages / useTyping / useCall hooks so you don't wire the handshake, stream, or teardown yourself.

Svelte / Vue? This package ships thin adapters as subpath exports — @pouchy_ai/companion-sdk/svelte and @pouchy_ai/companion-sdk/vue (0.31.0).

Quickstart

Two steps. 1) Your backend exchanges the project Secret Key (create one in the dashboard → Keys) for a per-user session token — first-seen external_user_ids are provisioned automatically:

POST https://pouchy.ai/v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }

→ { "session_token": "pchy_…", "expires_in": 3600, "instance": { … } }

2) The client connects with the session token (never ship the Secret Key):

import { createCompanion } from '@pouchy_ai/companion-sdk';

const companion = createCompanion({ baseUrl: 'https://pouchy.ai', token: sessionToken });
await companion.connect();
companion.onMessage((t) => console.log(t));
companion.start();
await companion.sendText('hey');
await companion.connectCall();              // live voice (browser)
companion.sendWorldState({ type: 'game.player.hp', data: { hp: 12 }, retained: true });

(Personal/single-user integrations can pass a server-minted Personal Access Token as token instead — same API. See the docs' authentication section.)

Framework adapters (Svelte / Vue)

Thin, logic-free bindings over one shared view controller (createCompanionView, also exported from the root for any other framework). The view tracks { streamState, transcript, draft, typing, pendingConfirms } as immutable snapshots, wraps sendText (optimistic user turn — rolled back if the send rejects, 0.32.1, except on queued_offline / reply_timeout / needs_event_stream, where the text did not get lost; a rejected send also clears its dead draft and typing, 0.63.1) and confirmAction (confirm-card bookkeeping; a confirm_resolved/confirm_not_found rejection drops the stale card, since a confirm can be resolved elsewhere — hosted page or an IM keyword reply), and can restore recent history with { restore: N }. You own the client lifecycle — create/connect/start it once, hand it to the adapter, close it when done. React hosts: use the standalone @pouchy_ai/react package instead (it owns the client lifecycle and adds a voice-call hook; this view owns the richer render-state).

<script>
  import { companionStore } from '@pouchy_ai/companion-sdk/svelte';
  const companion = companionStore(client, { restore: 20 });
</script>
{#each $companion.transcript as turn}<p>{turn.role}: {turn.text}</p>{/each}
{#if $companion.typing}<p>…</p>{/if}
<button onclick={() => companion.sendText('hey')}>send</button>
// Vue 3
import { useCompanion } from '@pouchy_ai/companion-sdk/vue';
const { snapshot, sendText } = useCompanion(client, { restore: 20 });
// snapshot.value.transcript / .draft / .streamState — auto-disposes with scope

Constructor options (beyond baseUrl + token)

createCompanion({ … }) also takes: surface (one resumable session per surface), modalities / handles / contextKinds / tools / appContext (the capability handshake — since 0.77.0 connect()'s ack carries droppedTools: [{ index, reason }] for any tools entry it refused: missing_name / duplicate_name / over_cap), visitor (representative mode, below), onAuthError (401 → return a fresh token and the client retries transparently), stream: 'sse' | 'websocket' (receive transport — 'websocket' opts into the lower-latency WS plane when the deployment serves one — pouchy.ai does not (serverless); an optional self-hosted WebSocket gateway worker does, for a host that can hold sockets — and falls back to SSE automatically — when the socket cannot open, and since 0.73.1 when the gateway refuses it with a 4xxx close (4401 unauthorized, 4502 upstream failure) — observable as streamState === 'degraded_sse'), streamBaseUrl (0.59.0 — the origin the WS plane is served from, i.e. your self-hosted gateway; only the reply socket goes there, every REST call and the SSE fallback keep using baseUrl; unset derives the socket URL from baseUrl), debug (0.32.0 — true logs structured instrumentation events via console.debug; a function receives every CompanionDebugEvent: HTTP request/response with method/path/status/ms, each delivered envelope post-dedup, stream-state transitions, synthesized errors — never the token, headers, or bodies), queueOffline (0.34.0 — opt-in offline send queue, below) with outboxStore (custom persistence for it; default localStorage), requestTimeoutMs (0.35.0 — deadline for any request to produce response HEADERS, default 30 000 ms, 0 disables; expiry rejects code: 'request_timeout'; never bounds reading a streaming body; since 0.40.0/0.40.1 the long-work POSTs — whose headers only arrive when the server work completes — default to that work's own ceiling instead of 30 s: buffered /input sends, sendToolResult and the two knowledge ingests get 310 s, confirmAction 65 s, so a long-running turn/ingest/action is no longer declared dead while it succeeds server-side; an explicit requestTimeoutMs still bounds every request at your value), replayPendingToolCalls (0.37.0, default true — when connect() resumes a session paused on YOUR tools, re-emit each outstanding call to onToolCall with replayed: true; see "Resuming a paused turn" below), and two injection points for Node/tests: fetch (custom fetch implementation) and webSocketImpl (WebSocket constructor when globalThis.WebSocket is absent).

Events

Subscribe with on(type, fn) (or '*'), or these typed convenience helpers. Since 0.30.0 on() narrows the envelope payload per event name (OutboundPayloadMap) — e.g. on('companion.typing', (env) => env.payload.active) type-checks without a cast; '*' keeps the untyped envelope:

Helper Event Use
onMessage(fn) companion.message the assistant reply (fires exactly once per turn, streaming or not). When the turn PRODUCED files (create_spreadsheet / create_document on the companion side), envelope.payload.attachments (0.54.0) carries them structurally — MessageAttachment[]: { fileId, fileName, mimeType, url }, url a 7-day capability download URL — so render a download affordance from the field, not by parsing the prose
onDelta(fn) (POST-response SSE) token streaming — registering makes sendText stream the reply as it is generated: fn(chunk, {reset?}) fires per text chunk (reset = clear the partial — that text was tool-call deliberation); onMessage then fires once with the authoritative final text (replay-deduplicated). Force per call with sendText(text, { stream: true | false })
onToolCall(fn) companion.tool_call the companion asks your app to run a declared tool → sendToolResult. The call carries argsJson — args pre-parsed as JSON (undefined on empty/malformed args). replayed: true (0.37.0) marks a re-delivery of a still-outstanding call after a mid-pause reconnect — treat id as an idempotency key for side-effecting tools (see "Resuming a paused turn")
onRender(fn) companion.ui_action Instant UI — draw payload.interface (platform-neutral genui schema) with your own renderer (web / iOS / Android / CLI). Needs ui.render
onInterfaceUpdate(fn) companion.ui_update live {key,value} update to an already-rendered panel (no rebuild)
onSocialMessage(fn) companion.social_message inbound A2A friend message, delivered cross-app. Needs social.message
onConfirmRequest(fn) companion.confirm_request a sensitive-op approval request. Platform session tokens (your end users): show your own confirm card and resolve it with confirmAction — this is how confirm-gated custom skills (POST / credentialed) run. First-party user tokens: observe-only — approval stays first-party (where the stepUp:true biometric gate lives). Payload { confirmId, scope, summary, summaryKey?, summaryVars?, stepUp? } — render summaryKey ? localize(summaryKey, summaryVars) : summary to show the approval in the user's language (0.43.0)
onAudio(fn) companion.audio TTS clip of the reply, non-call modality (0.58.0): { url }, a capability URL of an mp3. Emitted after a text turn when the agent opted in (dashboard → reply cues → audio), the session negotiated voice and the token holds the voice scope.
onExpression(fn) companion.expression avatar cue (0.58.0): { expression }, a VRM 1.0 preset (happy / sad / angry / surprised / relaxed / neutral) read from the reply's tone. Emitted after a text turn when the agent opted in (dashboard → reply cues → expression). viseme / gesture stay unused.
onVoiceInject(fn) companion.voice_inject fn({text, speak}) — a voiceRelevant world-state line to say aloud during a live call; route text to your voice session when speak
onTyping(fn) companion.typing activity indicator — fn({active}) fires true when a turn starts working and false when it finishes / pauses, spanning the tool-loop / thinking phase before the first text delta. Drive a "typing…" state
on('companion.data_activity', fn) companion.data_activity metadata-only Data-plane activity (0.44.0) — the agent read a View or settled an Action: { kind, capability, outcome, ms, actionId? }. Never rows or intent values; an action outcome: "unknown" means pending verification — never render it as success or failure. Requires the data.activity scope (granted from the agent's Data flag)
on('companion.skill_card', fn) companion.skill_card a skill's own card, rendered (0.51.0) — the companion ran a tool of an installed skill whose author declared a card_template: { skill, tool, markdown }. The shape comes from the skill's manifest and its install review, not from the model, so this is a declared shape delivered to you rather than prose to parse. markdown is TEXT, not HTML; unfilled placeholders are already stripped. Requires the skills.execute scope; a handler that never fires means the skill ships no template, not that the call failed
on('control.call_ready', fn) control.call_ready a voice call is ready — the stream echo of startCall's accept. Deliberately secret-free ({ provider, agentId/model, voice, … } — the actual WebRTC credentials only ride the startCall HTTP response); useful for UI state on surfaces that didn't initiate the call
onUsage(fn) control.usage per-turn metering echo, emitted after an input turn that billed tokens: fn(payload, env) where payload carries { promptTokens, completionTokens, cachedTokens, reasoningTokens?, model? } (whole-turn totals; reasoningTokens is a subset of completion)
onError(fn) control.error agent / stream errors — agent_error (server-side turn failed after accept; safe to re-send), call_mint_failed (voice-credential mint failed — retry later or fall back to text) or the SDK-synthesized stream_unauthorized (stream 401 exhausted the onAuthError refresh retries, or the stream got a 403 — immediately terminal, commonly a token missing the events.subscribe scope; refresh/fix the token, start() again). Since 0.30.0 the vocabulary is EXPORTED TYPED: CONTROL_ERROR_CODES (runtime list, drift-tested against the server) + the ControlErrorCodeValue union fn's err.code now carries — switch on it with autocomplete

Instant UI

import { renderInterface } from './instant-ui'; // your renderer (web example in examples/web-instant-ui)
let panel: ReturnType<typeof renderInterface> | null = null;
companion.onRender(({ interface: ui }) => { panel = renderInterface(ui, mountEl); });
companion.onInterfaceUpdate(({ update }) => panel?.applyUpdate(update));

Reference renderers ship in the repo under examples/: web-instant-ui (vanilla), native-instant-ui (SwiftUI + Jetpack Compose), and cli-skills (terminal + local/device skills).

Already have an A2UI-catalog renderer? (0.42.0) toA2UI(payload.interface) projects any panel losslessly to Google's A2UI v0.9 flat wire format (standard component/field names; Pouchy-only concepts ride a _pouchy extension A2UI renderers ignore), and fromA2UI ingests a surface back. INSTANT_UI_NODE_TYPES lists the atom catalog as data for capability checks. Embedded Button / table-row actions are opaque EmbeddedAction { type } values — hand them back to the companion, never interpret them.

import { toA2UI } from '@pouchy_ai/companion-sdk';
companion.onRender(({ interface: ui }) => myA2uiRenderer.render(toA2UI(ui)));

Client methods (beyond the quickstart)

Method What it does
sendText(text, { awaitReply: true, replyTimeoutMs? }) Request/response mode: the promise resolves with the completed turn's reply { seq, text, envelope, usage? } (SendTextReply) — no onMessage wiring or hand-rolled turn gate. usage (a TurnUsage: { promptTokens, completionTokens, cachedTokens, reasoningTokens?, model? }) is the turn's token totals for cost accounting, present when the server reported them. Works without start() when the turn completes in text (the reply rides the same streaming request). A turn that pauses on YOUR declared tools resolves via the event stream, so it DOES need start() — without it the promise rejects immediately with code: 'needs_event_stream'. replyTimeoutMs (default 60s) bounds the whole operation (0.28.0); on expiry it aborts the in-flight stream and rejects with code: 'reply_timeout'. Each turn carries a minted turnId the server echoes as replyTo on the reply (API ≥ 1.1), so the fallback resolves on THE reply — proactive messages can't steal it. Retrying safely (0.72.0): every result — plain or awaitReply — carries the turnId it went out under, and sendText(text, { turnId }) sends your own (1–256 chars since 0.73.2, 64 before; anything else rejects with code: 'invalid_option' before sending, as the server refuses it). After an AMBIGUOUS failure (reply_timeout, request_timeout, a dropped stream, a 5xx) retry under the SAME turnId: the server dedupes completed turns (60 min, last 32 per session) and replays the recorded one with duplicate: true — no model call, no bill — where a retry without it runs and bills a second turn. An awaitReply retry resolves with the recorded reply on either leg (the stream: false replay carries it since 0.73.0); if the server no longer holds that reply it rejects code: 'reply_unavailable' at once rather than waiting out replyTimeoutMs. instructions (0.63.0) carries developer-authored rules for THIS turn: joined to the prompt after the history and immediately before the user's message, and never scored as user speech by inbound moderation. Needs the chat.instructions scope (403 missing_scope without it), 32,000 chars max (413, never a silent truncation), and it is NOT persisted — it governs the turn it rides and nothing after it. Before it existed, an app with per-turn rules had to prepend them to text, where the classifier read the developer's own prose as the user's (issue #3379). A turn refused by that gate now says so: blocked / blockedBy (`'classifier'
recall({ query?, limit? }) / remember({ content, … }) / forget(id) Read / write / forget in the token-scoped memory namespace. query runs SEMANTIC search over the facts server-side (embedding-based, ranked fallback); omit it for the plain importance/recency ranking. remember takes ONE fact of at most 2,000 characters — longer content throws a payload_too_large CompanionError rather than being stored truncated, so split it into separate facts (or ingest it as knowledge, which chunks document-length text for you). forget(id) (0.53.0) takes the id a recall row carries and sets the user's authoritative "stop telling me this" — the fact stops recalling and stays forgotten; needs the namespace's write scope (a 403 missing_scope CompanionError without it), answers one uniform 404 for anything the token cannot see, and is idempotent. remember resolves { cloudId, namespace, dedup } (0.60.0): dedup.matched: true means the write MERGED into a row the user already had — cloudId then names that existing row (the handle forget(id) takes), not a new one — and dedup.revived says the merged-into row had been decay-archived and is live again; dedup.skipped says why no match ran.
ingestKnowledge({ text, name, … }) / ingestFile(file) Push documents into the user's knowledge base ("My materials"); needs memory.write:core. Both return truncated (0.49.0) — true means the source overran the ingest caps and the remainder was dropped, so split it and ingest the rest.
history({ limit? }) Fetch the session transcript — restore chat on reload.
setModalities([...]) Switch active I/O mid-session (within the token's grant).
ping() Keepalive for long-idle embeds.
setToken(token) Swap the bearer for every subsequent request + stream reconnect — session tokens expire (1h default), so call this when your backend re-mints one. Prefer the onAuthError constructor option to refresh on demand instead.
pendingConfirms() / confirmAction(id, approve) List / resolve pending sensitive-action confirms (platform session tokens). A listed row carries status (0.61.0) — pending is one the user has never answered, exec_failed one they ALREADY approved whose run flaked, so render them differently; execUnsettled says that flake was the DEADLINE and the action may still complete on its own; stepUp is the same advisory the live confirm_request carries, so a card rebuilt after a reload can announce the passkey gate instead of meeting it as a 401 step_up_required. Switch on the response's outcomeClass (0.55.0) — success / error / unknown / rejected / denied / unrecognized — to tell an action that ran from one that failed or was refused; status: 'approved' alone only says the approval was accepted, and outcome is prose for a person. On { status: 'exec_failed', retryable: true } you MAY re-call the same confirmId — idempotent actions only. mandate (0.57.0) — present only after a passkey step-up on the first-party surface — is the receipt: { intentDigest, credentialId, signedAt }, a signature bound to WHAT was approved.
getMandate(confirmId) The full user-signed approval document behind a confirm's mandate receipt (0.59.0; design #27 Phase 2): `{ confirmId, status, intent, intentDigest, mandate, credential: { credentialId, publicKey, format }
endSession({ transcript?, callGen? }) Distill the session into durable memory now; returns { ok, facts?, extracted?, filtered?, merged?, revived?, writeError?, skipped? } — skipped: 'no_session' | 'no_content' | 'throttled' | 'no_visitor_scope' (0.56.0: a representative session whose token lacks represent:remember); filtered (0.56.0) = proposed facts the identifier filter refused, so extracted − facts does not read as a refused write; merged (0.62.0) is how many of facts folded into a memory the store ALREADY held instead of minting a new one (write-time dedup), so facts − merged is the count of NEW memories — a repeated endSession() re-reads the same recent turns and routinely re-extracts what it already stored, which facts alone reported as growth (revived is the subset whose target decay had archived and the merge brought back); extracted (0.48.0) = facts the extraction proposed vs facts STORED, so extracted > 0 with facts: 0 means the store refused them and writeError says why (e.g. missing memory-write scope). Also closes the server's 20-min call-active window — generation-fenced since 0.41.0: it sends the callGen of the last call THIS client minted (null if none), so a text client's teardown never kills a live call's window on the same session, and a stale call handle's late /end can't clear a newer call's window. Self-plumbed calls: pass startCall's creds.callGen here at call end.
close() One-call teardown: stop() + endSession() — the text-session mirror of the call handle's close().
lastEndSessionError Read-only accessor (0.77.0): when the last endSession() / close() returned null because the SERVER refused it, { status, code? } (e.g. 401 invalid_token, 404 session_not_found, 429 rate_limited); null after a call that got a diagnostic or never reached the server (no session / fetch rejected). endSession() still returns null on a refusal — read this to tell "token expired, nothing consolidated" from "nothing to do" without wiring debug.
startCall(opts) / connectCall(opts) Voice plane: raw credentials / fully-wired call handle (call.close() ends + folds the transcript into memory). audioTags: true (0.64.0) asks for performed voice cues — see below. record: true (0.65.0) records the whole call — see below; fetch it with getCallRecording(callGen) / delete it with deleteCallRecording(callGen). The mint stamps a 20-min server-side call-active window (routes voiceRelevant world-state into the live call) and returns its callGen fencing handle (0.41.0, see endSession). On the EL path it also returns voiceToolToken (0.46.0) — the in-call memory tool's credential, which connectCall passes for you and a self-plumbed host must pass under BOTH names (0.47.0): dynamicVariables: { [VOICE_TOOL_VARIABLE]: creds.voiceToolToken, [VOICE_TOOL_VARIABLE_PLAIN]: creds.voiceToolToken } — ElevenLabs ignores a CLIENT-supplied secret__ variable, so the plain twin is the one that arrives; omitting it throws nothing and silently disables recall (see below). If connectCall throws AFTER a successful mint (mic denied, missing @elevenlabs/client, SDP failure — 0.40.1), it tears down like an instant hang-up first: /end closes the window, then the original error is rethrown.
streamState / onStreamStateChange(fn) Receive-stream lifecycle (0.30.0): 'idle' | 'connecting' | 'connected' | 'reconnecting' | 'degraded_sse' | 'stopped' (CompanionStreamState). fn(state, prev) fires on change only — drive a connection indicator; degraded_sse = the WS transport errored and delivery continues on the SSE fallback; stopped = stop()/close() or a permanent stream_unauthorized failure.
sessionId Read-only accessor: the session id of the last successful connect() (HelloAck.session), or null before the first. Not cleared by endSession()/close()/setToken() — an identifier for correlating client-side logs with server-side records (recall audit, transcripts), not a liveness check. It changes when a re-handshake mints a NEW session (idle past the server's ~12h resume window, or after endSession()); since 0.54.2 an open event stream follows it — connect() restarts the stream on the new id, where before it stayed bound to the old session and every reply for the new one went undelivered.
getAvatar() / brandIconUrl(size?) The user's avatar (VRM/portrait) and the Pouchy brand icon for your UI. Standalone (no client/token): pouchyBrandIconUrl(baseUrl, size?) derives the same URL before/without connecting.
getWallet() Read the instance's own wallet — { balances: [{ currency, amount }], totalUsd, currency }. Read-only + receive-only; needs the wallet.read scope.
flushOutbox() / pendingOutbox() / onOutboxChange(fn) Offline send queue (0.34.0, opt-in queueOffline: true). A sendText whose fetch fails at the NETWORK level (or navigator.onLine === false) resolves { seq: null, queued: true, turnId } instead of rejecting; with awaitReply: true it rejects with code: 'queued_offline' (the item is still queued). HTTP errors (4xx/5xx incl. 429) are never queued — the server saw those; a 2xx whose body is not JSON (code: 'bad_response', 0.77.0 — a captive portal's HTML page) is queued as an ambiguous send. The queue replays FIFO — automatically after connect() and whenever the stream reconnects, or manually via flushOutbox() → { sent, remaining } — each item a buffered POST reusing its ORIGINAL turnId, which the server dedupes (completed turnIds, 60 min / last 32 per session). Replay is double-run-safe past that window too: a provably-unsent item (QueuedSend.unsent, navigator.onLine was false) replays at any age, an ambiguous item is auto-replayed only under OUTBOX_AMBIGUOUS_MAX_AGE_MS (30 min) and dropped past it, and a transient 409/408/423/425 mid-flush is kept (only a terminal 4xx drops the item). Since 0.50.0 the keep-arm is BOUNDED for the one class that carries no status at all: after OUTBOX_MAX_REPLAY_ATTEMPTS (5) attempts that rejected at the network level while navigator.onLine !== false, an item older than the 30-min window is dropped instead of blocking the queue behind it forever — the shape a host-refused oversize body produces (downscale images before sending). pendingOutbox() lists the queued sends (QueuedSend[]); onOutboxChange(fn) fires on every enqueue/flush/drop (drive a "N pending" badge). Persistence: localStorage by default (in-memory page-lifetime fallback), or bring your own via the outboxStore option; the key is pouchy-outbox:<surface>, and on a representative session (visitor) pouchy-outbox-v:<surface>:<visitorId> since 0.50.1 — a visitor-scoped session gets a visitor-scoped queue, so one visitor's queued sends can never replay into another's thread on a shared browser.

Tool calling beyond your own declared tools — voice calls only: on connectCall() sessions the SDK automatically offers HOST_CONTROL_TOOLS (universal verbs — invoke_action, set_feature, set_value, navigate, highlight) once you declare tools/handles, and AVATAR_VISUAL_TOOLS (play_gesture / play_expression) let avatar-rendering embeds receive expression cues — declare them or they no-op silently. Text turns expose exactly the tools[] you declared (nothing is auto-added), so declare the verbs there yourself if you want them outside calls. Full contract: companion-host-control.

Cancellation (0.29.0, completed in 0.29.1): every request-performing method accepts an optional AbortSignal — including sendToolResult(callId, result, { signal }) and startCall({ signal }) since 0.29.1. Methods with an options object take a signal? field (sendText(text, { signal }), recall({ signal }), …); the rest take a trailing opts (getAvatar({ signal }), ping({ signal }), …). Aborting rejects with CompanionError code: 'aborted', and for sendText({ awaitReply }) it also tears down the in-flight event stream.

const ac = new AbortController();
const p = companion.sendText('summarize this', { awaitReply: true, signal: ac.signal });
// …user navigated away
ac.abort(); // p rejects with code: 'aborted'

Offline send queue (0.34.0, opt-in): queueOffline: true turns a network-level send failure (fetch rejected / navigator.onLine === false) into a queued send instead of a lost one — replayed FIFO on reconnect with its original turnId (server-deduped, so a retry can never double-run a turn; see the methods table above for the full per-item semantics). Since 0.39.0 the replay is double-run-safe even after a long offline gap: a send known to have never reached the server (navigator.onLine was false) replays at any age, while an AMBIGUOUS one (the turn may have run server-side) is replayed only while younger than OUTBOX_AMBIGUOUS_MAX_AGE_MS (30 min) and dropped past that rather than risk a duplicate; a transient 409 mid-flush keeps the item instead of discarding it:

const companion = createCompanion({ baseUrl, token, queueOffline: true });
await companion.connect();
companion.onOutboxChange((items) => render(items.length)); // "N pending" badge
const r = await companion.sendText('hey');
if (r.queued) render('queued — sends on reconnect', r.turnId);

Resuming a paused turn (0.37.0): a turn that pauses on YOUR declared tools used to be unrecoverable if the embed lost its state (tab reload / crash mid-pause) — the only unblock was abandoning it via endSession() (/end). Now connect() on a session resumed mid-pause returns the outstanding calls as HelloAck.pendingToolCalls (PendingToolCall[] — { id, name, args, turnId?, pausedAt? }; always present, empty when none), and — unless you pass replayPendingToolCalls: false — re-emits each to your onToolCall handler with replayed: true (one tick after connect() resolves, at most once per client instance), so your existing tool loop completes the turn instead of losing it. Register onToolCall before connect()/start(). Because the app may have already PERFORMED a replayed call and died before posting the result, side-effecting tools should treat id as an idempotency key when replayed is set (the server's result apply is idempotent per id too — a double sendToolResult for the same id is safe); use pausedAt to judge staleness before re-running something expensive:

declare function runTool(name: string, args: unknown): Promise<string>; // yours

const performed = new Set<string>(); // your durable "already ran" ledger
companion.onToolCall(async ({ id, name, argsJson, replayed }) => {
  if (replayed && performed.has(id)) {
    await companion.sendToolResult(id, { ok: true, result: 'already applied' });
    return;
  }
  performed.add(id);
  const result = await runTool(name, argsJson);
  await companion.sendToolResult(id, { ok: true, result });
});
const ack = await companion.connect(); // resumed mid-pause → ack.pendingToolCalls
companion.start();

Scopes (typed)

The full capability-scope vocabulary ships as typed constants — no more copying scope strings from prose:

import {
  COMPANION_SCOPES, SENSITIVE_SCOPES, DEFAULT_SCOPES,
  hasScope, grantsScope, isSensitiveScope, type CompanionScope
} from '@pouchy_ai/companion-sdk';

const ack = await c.connect();
if (!hasScope(ack.grantedScopes, 'skills.execute')) {
  // hide the "run a skill" affordance instead of eating a 403
}
// Gate a READ-ONLY affordance with grantsScope, not hasScope: a key granted
// only `wallet.spend` can still read the balance, so hasScope(g,'wallet.read')
// would wrongly hide it. grantsScope honours that subsumption.
if (grantsScope(ack.grantedScopes, 'wallet.read')) {
  // safe to show the wallet balance (getWallet() will succeed)
}

COMPANION_SCOPES lists every scope a key can carry; SENSITIVE_SCOPES are the ones that are never on by default (money / skills / social / core memory / the representative plane); DEFAULT_SCOPES is what a fresh key gets. hasScope(granted, required) is an exact-grant check; grantsScope(granted, required) additionally honours scope subsumption (wallet.spend ⊇ wallet.read) — use it to gate whether an API call will succeed. Also exported: COMPANION_MODALITIES, REPRESENT_SCOPES, DEFAULT_MODALITIES, isRepresentScope(). The list mirrors the server vocabulary and is guarded by the SDK's drift test.

The wire-protocol runtime constants are exported too — PROTOCOL_VERSION, INBOUND_TYPES, OUTBOUND_TYPES, and COMPANION_ERROR_CODES (the append-only HTTP code vocabulary behind CompanionError.code, e.g. missing_scope / session_not_found / turn_pending) — so you can assert the version, validate the event vocabulary, or switch on error codes without hard-coding strings. Every helper populates CompanionError.code when the server names a cause (0.24.0 — previously only the POST-backed calls did). 0.27.0 adds rate_limited for the runtime's 429 cost gates (turn burst ceiling, demo daily budget); on those failures CompanionError.retryAfter carries the server's Retry-After in seconds, so back off with if (e.code === 'rate_limited') await sleep((e.retryAfter ?? 1) * 1000) (retryAfter is undefined when the server names no window). The SDK also synthesizes a few client-side codes outside that HTTP vocabulary: reply_timeout (awaitReply fallback gave up), stream_unauthorized (event-stream 401/403 — a 401 exhausted the token-refresh retries; a 403 is immediately terminal, commonly a missing events.subscribe scope), and — 0.26.0 — the voice connect-step codes call_unsupported (no WebRTC/mic in this environment), call_connect_failed (mic timeout / SDP exchange failed) and call_dependency_missing (@elevenlabs/client not installed). 0.28.0 adds the confirm/step-up codes (confirm_not_found, confirm_resolved, step_up_required, step_up_failed), the client-side needs_event_stream, and types CompanionError.code as the CompanionErrorCodeValue union (autocomplete in your switch; still open for newer servers). 0.34.0 adds queued_offline (a send parked in the offline outbox for replay) and 0.35.0 adds request_timeout (the requestTimeoutMs headers deadline fired). 0.72.0 adds invalid_option (an option the SDK refused before sending, e.g. a turnId outside 1–256 chars) and 0.73.0 reply_unavailable (an awaitReply retry of a completed turnId whose reply the server no longer holds). 0.45.0 adds the server code quota_exhausted (402 — the account's monthly allowance is spent, credits or realtime voice minutes, e.g. startCall past the plan's voice ceiling): not retryable, unlike rate_limited — upgrade the plan or wait for the 1st (UTC) reset. The authoritative reference is the exported CompanionErrorCodeValue union on CompanionError.code (the internal SDK_SYNTHESIZED_ERROR_CODES list is drift-tested but deliberately not part of the public index surface). (stream_unauthorized above is a control.error/onError code, not a thrown CompanionError.code.) Every outbound payload has a named type; ToolCallPayload ({ id, name, args, replayed? }) types the companion.tool_call event, MessagePayload ({ text, replyTo?, attachments? }) types companion.message (attachments: MessageAttachment[] — files the turn produced, 0.54.0), PendingToolCall (0.37.0) types the HelloAck.pendingToolCalls entries, and onToolCall's callback receives ToolCallEvent — the payload plus argsJson (pre-parsed args). On a persisted server fault (a real 5xx) CompanionError.errorId (0.56.0) carries the server's err_… reference — quote it to support; it is the id the server's own error row is keyed by. Absent on 4xx and transport failures.

For advanced voice hosts, the lower-level openCompanionCall(creds, options?, bridge?) primitive is exported too — it consumes the CallCredentials that startCall() mints (bring-your-own call lifecycle; see companion-integration-faq).

In-call memory recall (voiceToolToken, 0.46.0)

On the ElevenLabs path the agent carries a server-side webhook tool that reads the session's memory mid-call. It authenticates with the call-scoped creds.voiceToolToken, which you hand to EL as a dynamic variable at session start:

import { VOICE_TOOL_VARIABLE, VOICE_TOOL_VARIABLE_PLAIN } from '@pouchy_ai/companion-sdk';

const creds = await companion.startCall();
if (creds.provider === 'elevenlabs-convai') {
  // Conversation.startSession({
  //   conversationToken: creds.token, connectionType: 'webrtc',
  //   dynamicVariables: {
  //     [VOICE_TOOL_VARIABLE]: creds.voiceToolToken,
  //     [VOICE_TOOL_VARIABLE_PLAIN]: creds.voiceToolToken   // 0.47.0 — the one EL actually substitutes
  //   }
  // })
}

EL substitutes it into that tool's Authorization header and never sends the secret__ class of variable to the LLM, so the credential rides a live conversation without entering a prompt.

connectCall() handles this automatically. If you plumb the call yourself, this is the one omission that surfaces no error: EL substitutes the agent's empty default, the tool call 401s, and the companion simply says it doesn't recall anything — a call that sounds entirely normal with memory switched off. The field is absent (not empty) when the session holds no memory grant; pass it through as-is and omit the key when it is missing.

Representative mode (代聊)

Pass a visitor and the session flips from owner-facing to representative: the companion answers that visitor on the owner's behalf (代聊 / customer-service) using only screened owner context — never the owner's private memory, system prompt, or PII. Works over text and voice.

const c = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: OWNER_PAT,                 // must hold the `represent` scope
  surface: 'support-widget',
  appContext: { name: 'AcmeShop', description: 'Order support' },
  visitor: { id: stableVisitorId, displayName: 'Sam' }  // stable per end-user
});
const ack = await c.connect();      // → { representative: true, visitorPaired, … }
await c.sendText('do you ship to Canada?');
const call = await c.connectCall(); // same, over voice

Scopes (granted by the owner's "Let it represent you to visitors" key toggle):

Scope Adds
represent required — open a visitor-facing session
expose:knowledge answer from the owner's knowledge base ("My materials")
expose:facts share a wider set of screened facts
represent:remember durable per-visitor notes across visits (isolated store)
represent:pair c.pairVisitor(visitorPAT) — pair a visitor who's also a Pouchy user → unlock A2A. A new pairing returns request.approveUrl for the visitor to approve on pouchy.ai; poll c.getPairRequest(id) (0.74.0)

Without represent, supplying a visitor is rejected (403). First-party tools (wallet / skills / social) are withheld; your own declared tools still work. Full walkthrough: API reference §3.3–3.4.

Voice (optional dependency)

Zero-code alternative: if you just want a chat box, skip the SDK entirely — <iframe src="https://pouchy.ai/embed?token=…&theme=dark&accent=%23ff6b81"> is a hosted drop-in widget with a postMessage control plane (docs).

connectCall({ bargeIn: true }) opts into full-duplex voice: the mic stays open while the companion speaks so the user can interrupt mid-utterance (the providers' native interruption takes over). The default is half-duplex — on phone speakers echo cancellation can't fully remove the companion's own voice and it would interrupt itself. Enable it only where AEC holds (headphones, desktop, or devices you've verified).

connectCall({ audioTags: true }) (0.64.0) lets the companion perform short voice cues on an ElevenLabs call: it laughs, sighs, whispers and speaks softly instead of only saying words (Eleven v4). Opt-in, because the provider's transcript then contains the cues ([laughs] 你也太可爱了吧) — connectCall strips them before onTranscript and before the transcript reaches memory. The server may decline (OpenAI Realtime fallback, or the operator switched it off); startCall credentials carry audioTags: true only when granted. Plumbing the ElevenLabs session yourself? Run every agent transcript line through stripAudioTags(text) before you show or store it.

To see what was performed (0.70.0): on a granted call every companion onTranscript line carries audioTags: string[] (['sighs', 'laughs'], or []); the field is absent on user lines and on calls without the grant. Self-plumbed: extractAudioTags(line) gives the same list.

onEnd(end) (0.71.0) says why a call ended, once per call, on every path: end.reason is 'user' (your close()), 'agent' (the companion hung up with ElevenLabs' end_call tool), 'remote' (the provider closed it cleanly without saying why) or 'error' (end.error; onError got it first). Passing onEnd moves a clean far-side end out of onError, which then means failure only; without it nothing changes. 'agent' needs the ElevenLabs agent to send its tool responses to the client (otherwise it reads 'remote'), and openai-realtime has no clean remote end. connectCall sends the call to /end on every end, the companion's hang-up included.

connectCall({ record: true }) (0.65.0) records the whole call — both voices, kept by the voice provider, so a host that cannot capture audio on the device (React Native's native WebRTC has no WebAudio / MediaRecorder) gets the same recording as a browser. Turn it on per agent in the dashboard (the agent's recording switch, which grants call.record); it runs on ElevenLabs only, and you own telling readers their calls are recorded. The handle says what happened before the call connects: call.recording is true, or false with call.recordingUnavailable ('not_enabled' | 'not_configured' | 'no_conversation_id' | 'store_failed') — tell the reader the truth either way. After the call, getCallRecording(call.callGen) answers { state: 'ready', audio, contentType } (an MP3 Blob), { state: 'pending', retryAfterSec } while the provider is still processing, or { state: 'none' | 'expired' | 'deleted' }. Recordings are kept for 7 days: copy the file to your own storage. deleteCallRecording(callGen) removes the provider's copy (audio and transcript) — use it when a reader deletes their account. Self-plumbed: startCall({ record: true }) returns the same recording / recordingUnavailable / callGen on the credentials.

connectCall({ avatar: { faceId } }) (0.75.0) puts a talking-face video avatar on the call. Pouchy runs it on Simli and holds the Simli key: the handle carries call.video — { provider: 'simli', faceId, sessionToken, iceServers, maxSessionLength, model } — which you open with simli-client and feed the companion's remote audio track; the voice is unchanged, the face lip-syncs to it. You build the faces yourself, with your project's admin key: admin-sdk createVideoFace(image) (or POST /v1/admin/video-faces) builds one from an image of your character, and listVideoFaces() reports it ready once the build finishes (minutes to hours; billed per face built). faceId must be one of your project's ready faces, and the session needs the call.video scope, which it gets once the project has one — withhold it per reader with withhold_scopes to keep the voice and drop the face. A call that asked and got no face still connects, with call.videoUnavailable ('not_enabled' | 'not_configured' | 'face_not_allowed' | 'face_not_ready' | 'mint_failed'; face_not_ready since 0.75.1: the face is yours but still building). Billed per minute on top of the voice. Self-plumbed: startCall({ avatar }) returns the same fields on the credentials.

avatar.model (0.76.0) picks Simli's animation model: 'fasttalk' (Simli's default, used when you omit it) or 'artalk'. call.video.model says which one the session was minted with. Same per-minute rate either way — try both on your own face; how expressive it reads depends on the face image too.

connectCall({ brief }) (0.67.0) tells the companion what this call is about: your application's note ("she just got the acceptance letter", "you are calling from the station"). It opens on it and keeps to it, and never reads it out. Up to 1,000 characters. It needs the chat.instructions scope (the agent's per-turn instructions switch), because it joins the prompt as your words and is not moderated as user speech; send it from a token your backend minted.

A call made with a Pouchy World role's session token carries where the story is: the scene, the story clock, flags, the facts already revealed, and the role's own last beats. PUBLIC state only — never the role's secrets, notes, goals or unrevealed facts — because a call's instructions reach the reader's device (the ElevenLabs web SDK takes its prompt as a client-side override); keep anything the reader must not see out of the persona and brief too. call.worldBrief (and creds.worldBrief) says 'included', 'no_story' or 'unavailable' (the call went ahead without it). call.privateStory (0.68.0) says whether the role can reach its private layer on this call: 'tool' (the in-call recall_private_story tool, served ElevenLabs ↔ Pouchy, never through your page) or 'none' (the role keeps it vague). call.trimmed (0.68.0) reports the characters the server cut to keep the instructions within the realtime cap ({ story?, context? }; your brief is never cut) and is absent when nothing was. Nothing said on the call changes the story: after the call, fold it back in as a beat on the world turn door with trigger.context: "this was a phone call", the gist as text, and any proposedPatches the call should leave behind.

Mute and typing (0.69.0). call.setMuted(true) mutes the reader's microphone until setMuted(false); it holds across the companion's lines (the half-duplex re-open never undoes it), and connectCall({ muted: true }) opens the call muted. Show a mute control only when call.canMute is true. call.sendText(text) sends a line the reader TYPED as their turn: the companion answers it as something they said, it reaches onTranscript as role: 'user', and it is in the memory consolidated at hang-up. It cuts an utterance in progress, like speaking would. Nothing is sent before you call it, so run any check of your own first. injectEvent is different: a context line, never reported as the reader's words.

call.interrupt() (0.36.0) is the silent cut: stop the utterance currently being spoken WITHOUT generating new speech — for real-time commentary going stale at a beat boundary (next hand starts while last hand's remark is still playing). Idempotent; check call.provider for the tier: openai-realtime is guaranteed (response.cancel + output-buffer clear — playback stops within ~an audio frame, the session continues); elevenlabs-convai is best-effort (the EL client exposes no public stop-playback API — we signal user_activity and attempt the client's internal buffer cut; worst case a no-op). For guaranteed preemption on EL, injectEvent(text, true) — which does speak new content — remains the reliable lever.

connectCall() for the ElevenLabs Convai provider needs the optional peer dependency:

npm i @elevenlabs/client

OpenAI Realtime needs no extra dependency (self-contained WebRTC).

Build & publish

The source lives in the app repo at src/lib/companion-sdk and is decoupled from any $lib import, so this package builds with plain tsc:

cd packages/companion-sdk
npm run build      # tsc → dist/ (ESM + .d.ts + source/declaration maps) + src/ (shipped sources)
npm run build:cdn  # cdn/companion-sdk{,.min}.js + versions/<v>/ + integrity.json (SRI)
npm publish        # public scope (publishConfig.access: "public"); runs build via prepublishOnly

The CDN artifacts are committed and served from pouchy.ai: /sdk/companion-sdk{,.min}.js (floating, 1h cache) and /sdk/v<version>/companion-sdk{,.min}.js (pinned, immutable) — pin the versioned URL together with the sha384 hash from cdn/integrity.json (also published at pouchy.ai/sdk) to opt out of silent upgrades.

The wire contract (protocol.ts) is a self-contained mirror of the server's; protocol.drift.test.ts (run by the app's vitest) fails CI if they diverge.

Full release steps (version bump, CHANGELOG.md, drift test, tag) live in PUBLISHING.md.

License

Proprietary — © 2026 Pouchy.ai. The published package may be installed and used to build integrations with Pouchy's official services; all other rights are reserved. See LICENSE.

Related Packages

artifactuse

The Artifact SDK for AI Agents - Turn AI outputs into interactive experiences

7,162 2
xxai-agent

AI Agent JS SDK

2,483 0
elevenlabs-ui-vue

A collection of Open Source agent and audio components that you can customize an...

6,972 199
@newschools/sdk

New Schools SDK - Multi-framework components and modules for integrating New Sch...

2,312
@purplecallio/vue

Official Vue 3 composable and components for PurpleCallio, a developer-focused r...

206

Version History

Version Released License
0.77.0 Custom
0.76.0 Custom
0.75.1 Custom
0.74.0 Custom
0.73.2 Custom