rest-pipeline-js

Rest Pipeline JS

Flexible, modular pipeline orchestrator for REST APIs — sequential and parallel stages, retry with backoff, response caching, rate limiting, auth provider, stream stages (SSE / AsyncIterable), plugin system, and Vue / React integrations — all with a single dependency (axios).

Features

• createRestClient() — full-featured HTTP client built on top of axios: retry with exponential backoff and Retry-After support, response caching with a pluggable CacheStore backend (incl. targeted invalidateCache()), rate limiting (concurrency + req/interval) with a pluggable distributed RateLimiterStore, circuit breaker with a pluggable distributed CircuitBreakerStore, auth provider with automatic 401 refresh and optional token caching, request cancellation by key, custom HTTP adapters
• Request tracing — W3C traceparent header generation plus a TracingProvider hook (duck-typed against OpenTelemetry's Span API) for wiring in a real tracing backend
• Idempotency keys — Idempotency-Key header on mutating requests, manual or auto-generated per logical request across retry attempts
• Offline queue — queue mutating requests made while offline and replay them (same idempotency key on every attempt) once back online; pluggable persistence, isOnline/onOnlineChange for non-browser environments
• PipelineOrchestrator — sequential and parallel stage execution; each stage has condition, before, request, after, validateInput, validateOutput, errorHandler hooks (all receive the pipeline's AbortSignal); sharedData pool shared across all stages
• Error recovery — errorHandler can return recoverStep(data) to turn a failed stage back into a successful one and keep the pipeline going, instead of only transforming the error
• Global middleware — beforeEach / afterEach / onError hooks that apply to every stage without modifying individual configs
• Parallel groups — multiple stages run concurrently via Promise.all, or through a bounded pool via concurrency; single failure stops the group
• Pause / Resume / Abort — pause() waits after the current stage; resume() continues; abort() cancels the current HTTP request and propagates its AbortSignal into every stage hook so custom request/before/after functions can cancel their own work too
• Export / Import state — serialize stageResults + logs to a plain object; restore on the next page load
• Stream stages — stream: async function* for SSE / any AsyncIterable; onChunk callback in real time; abort-aware
• WebSocket stages — a stage over a persistent connection (onOpen/onMessage/onClose/onError, closeOn); pluggable createWebSocket (defaults to globalThis.WebSocket) for Node <22/edge runtimes
• Pipeline metrics & run correlation — onPipelineStart, onPipelineEnd, onStepDuration callbacks, plus a runId (also on getRunId(), log entries, and step events) shared by every callback/event from the same run
• createPipeline() / pipe() builder — short factory and fluent builder API for common patterns; in TypeScript, pipe().step() chains infer prev's type from the previous step automatically
• validatePipelineConfig() — catch duplicate keys, empty keys, type errors before runtime
• Plugin system — install reusable behavior (logging, analytics, etc.); cleanup via destroy()
• Persist adapter — pluggable save/load interface; auto-save after each stage
• Log sanitization — mask sensitive headers (authorization, x-api-key, cookie, …) in metrics callbacks, on by default
• Vue integration — usePipelineRunVue, usePipelineProgressVue, and more (import from rest-pipeline-js/vue)
• React integration — usePipelineRunReact, usePipelineProgressReact, and more (import from rest-pipeline-js/react)
• paginate() / paginateAll() / flattenPages() — iterate a paginated API (cursor- or offset/limit-based) as an AsyncGenerator<T[]>, standalone or as a StreamStageConfig source
• createMockAdapter() (separate rest-pipeline-js/testing entry point) — route-based HttpAdapter for testing code that uses this package without a real backend, with call history and sequenced responses for exercising retry
• Tree-shakeable — sideEffects: false; Vue and React entry points are code-split

When you'd reach for this

A plain fetch call with error handling works fine until your project needs retries, caching, rate limits, and token refresh all at once — rest-pipeline-js pulls all of that into one configurable pipeline instead of scattered ad-hoc code for every case.

• A checkout shouldn't double-charge — A shopper taps "Pay" twice because the network stalls — idempotency keys guarantee the retry doesn't create a second order, and an offline queue delivers the operation once the connection comes back if it drops entirely.
• A dashboard is assembled from dependent requests — One request returns a list of objects, the next pulls metrics for each one, a third compares them to the previous period, and each step's result shapes the parameters of the next. The whole chain is described as a single pipeline, where each step automatically receives what it needs from the one before it, instead of requests nested inside each other and data passed around by hand.
• Booking a service means several linked checks — Confirming a reservation means checking slot availability, pricing in a discount, reserving the spot, and sending a notification — if one step fails, you need to know exactly what already happened and retry just the stuck step instead of starting the whole flow over.
• A server response fails validation — A validation schema (Zod, Yup, or Valibot) catches a malformed response before it reaches the rest of your code, and the process itself gets a chance to fall back to a default value instead of aborting entirely.

Installation

The core package has no required peer dependencies — only axios (a regular dependency). Vue and React are optional peers, needed only if you import from the matching entry point (rest-pipeline-js/vue / rest-pipeline-js/react):

npm install rest-pipeline-js

Peer dependencies for framework integrations:

# Vue
npm install vue@>=3.3

# React
npm install react@>=18 react-dom@>=18

A CDN build is also available — see installation on the full docs for the <script> tag setup.

Quick start

import { createRestClient, PipelineOrchestrator } from 'rest-pipeline-js'

// 1. Create a REST client
const client = createRestClient({
  baseURL: 'https://api.example.com',
  retry: { attempts: 2, delayMs: 500, backoffMultiplier: 2 },
  cache: { enabled: true, ttlMs: 60000 },
  auth: {
    getToken: async () => localStorage.getItem('token') ?? '',
    onUnauthorized: async () => {
      /* refresh token */
    },
  },
})

const res = await client.get('/users/1')

// 2. Run a pipeline
const orchestrator = new PipelineOrchestrator({
  config: {
    stages: [
      {
        key: 'fetchUser',
        request: async ({ sharedData }) => client.get(`/users/${sharedData.userId}`),
      },
      {
        key: 'processData',
        request: async ({ prev }) => ({ ...prev.data, processed: true }),
      },
    ],
  },
  sharedData: { userId: 42 },
})

const result = await orchestrator.run()
console.log(result.success, result.stageResults)

More examples

Vanilla JS

A circuit breaker for a failing backend

After a run of consecutive failures, the client stops hitting the dead API and rejects requests locally right away — then probes on its own to see if it's back.

import { createRestClient, CircuitOpenError } from 'rest-pipeline-js'

const client = createRestClient({
  baseURL: 'https://api.example.com',
  circuitBreaker: {
    failureThreshold: 5, // open after 5 consecutive failures
    openMs: 30_000, // stay open for 30s before probing again
    isFailure: (error) => error.status === undefined || error.status >= 500, // ignore 4xx
  },
})

try {
  await client.get('/flaky-endpoint')
} catch (err) {
  if (err instanceof CircuitOpenError) {
    // rejected locally — no network call was made
  }
}

Vue

Reactive pipeline run in Vue

usePipelineRunVue and usePipelineProgressVue return the run state and current stage as reactive refs — the component re-renders on its own as the pipeline runs, no manual event subscriptions.

<script setup>
import { PipelineOrchestrator, usePipelineProgressVue, usePipelineRunVue } from 'rest-pipeline-js/vue'

const orchestrator = new PipelineOrchestrator({
  config: {
    stages: [
      /* ... */
    ],
  },
})
const progress = usePipelineProgressVue(orchestrator)
const { run, running, result, error, abort, pause, resume, rerunStep } = usePipelineRunVue(orchestrator)
</script>

<template>
  <div>
    <div>Current stage: {{ progress.currentStage }}</div>
    <button @click="run()" :disabled="running">Start</button>
    <button @click="abort()" :disabled="!running">Abort</button>
    <button @click="pause()">Pause</button>
    <button @click="resume()">Resume</button>
    <div v-if="result">Done: {{ result }}</div>
    <div v-if="error">Error: {{ error.message }}</div>
  </div>
</template>

React

The same pipeline as a React hook

usePipelineRunReact and usePipelineProgressReact — the same run and progress, as plain React state and a function.

import { useState } from 'react'
import { PipelineOrchestrator, usePipelineProgressReact, usePipelineRunReact } from 'rest-pipeline-js/react'

export function PipelineComponent() {
  // Lazy initializer — created once per component instance, not shared
  // across every <PipelineComponent> on the page like a module-scope
  // orchestrator would be.
  const [orchestrator] = useState(
    () =>
      new PipelineOrchestrator({
        config: {
          stages: [
            /* ... */
          ],
        },
      }),
  )
  const progress = usePipelineProgressReact(orchestrator)
  const [run, { running, result, error, abort, pause, resume, rerunStep, clearStageResults }] =
    usePipelineRunReact(orchestrator)

  return (
    <div>
      <div>Current stage: {progress.currentStage}</div>
      <button onClick={() => run()} disabled={running}>
        Start
      </button>
      <button onClick={() => abort()} disabled={!running}>
        Abort
      </button>
      <button onClick={() => pause()}>Pause</button>
      <button onClick={() => resume()}>Resume</button>
      {result && <div>Done: {JSON.stringify(result)}</div>}
      {error && <div>Error: {error.message}</div>}
    </div>
  )
}

Documentation & links

• 📖 Full documentation: npm.vuecraft.ru/en/packages/rest-pipeline-js
• 🌐 VueCraft: vuecraft.ru/en
• 👤 Author: macrulez.ru/en
• 💻 GitHub: macrulezru/pipeline-js
• 📦 NPM: rest-pipeline-js
• 🐛 Issues: github.com/macrulezru/pipeline-js/issues

License

MIT

💖 Support the project

Open source takes time and effort. If this library saves you time or brings value, consider supporting further development.

Thank you for being part of this journey. ❤️