@icjia/pdf-search-index

@icjia/pdf-search-index

Apache Solr for client-side apps — without Solr. Build-time text extraction from PDF, DOCX, PPTX, XLSX that turns every document on your site into a searchable row and ships the index as a static JSON file. No JVM, no Tika service, no search server, no native deps — Node at build time, JSON at runtime.

Multi-format added in 1.1. PDF support is bundled (unpdf); DOCX/PPTX/XLSX are unlocked by installing the optional officeparser peer dep. The package emits a uniform IndexedDocument row with a format discriminator, so downstream search engines (Fuse.js, MiniSearch, FlexSearch, …) handle all four formats identically.

@icjia/pdf-search-index is the core library: it fetches documents, dispatches to the appropriate extractor (unpdf for PDF, officeparser for Office formats), and returns plain IndexedDocument[] rows. The output is fully framework-agnostic — first-party integrations ship for Astro 5 and Nuxt 4 (see Adapter packages below), and the same core works just as well from a prebuild script in Next.js, SvelteKit, Remix, Eleventy, Vite/Vue, or vanilla HTML.

Fuse.js is recommended but optional. The plain-JSON output drops into Fuse.js, MiniSearch, Orama, Lunr, FlexSearch, Pagefind, Typesense, MeiliSearch, or Algolia — your call. The /fuse and /snippet entry points are conveniences for Fuse callers, not gatekeepers; the core indexPdfs / extractPdfText / extractPdfsFromBody functions don't require fuse.js at all.

Why this replaces Solr for static / Jamstack apps: the typical Solr+Tika deployment is a JVM service, a schema, a managed index, and a network round-trip per query — enormous overhead when your corpus is the 50–500 PDFs your CMS or public/ folder already publishes. This package collapses Solr's build-time stage (the Tika part) into a pnpm build hook and lets the framework you already use serve the JSON result. Zero ops, zero servers, zero JVM tuning.

ESM only. Node 20 LTS / 22 LTS. MIT.

Install

npm install @icjia/pdf-search-index
# or
pnpm add @icjia/pdf-search-index
# or
yarn add @icjia/pdf-search-index

Optional peer dependency — fuse.js@^7 — only when you import the /fuse or /snippet subpaths. The package's peerDependencies range is "^7.0.0 || >=7.4.0-beta.0" so both stable Fuse 7.x and the Fuse 7.4 beta channel resolve. The v1.0.3 examples in this monorepo pin to 7.4.0-beta.6 to demo the newest beta surface; consumers who prefer stability can pin to a stable ~7.3.0 and the peer still resolves.

Security

Status as of v1.3.0 (last audited 2026-05-17): Every Critical and Important finding from the original audit against the core package is either remediated and verified in a shipped release, or has a documented active mitigation while the structural fix lands in a future release. Zero unaddressed exploitable issues against the documented usage envelope. Six independent audit passes confirm this: initial 1.0.1 + 1.0.3 delta + 1.0.5 verification (2026-05-16); 1.1.0 multi-format + 1.2.0 perf/security-extension + 1.3.0 search-engine-entries (2026-05-17). The v1.2 release closed I6 (maxUrls cap) and the inflate-bomb deferral; v1.3 ships two new search-engine adapter entries (/flexsearch, /pagefind) plus officeparser-as-direct-dep (supply-chain hardening). Remaining tracked items: C2 SSRF allowlist (v1.4), I2 cache-key normalization (v2.0), I5 CLI sitemap hardening (v1.4), full officeparser source vendoring (v1.4).

Remediation scorecard (core-relevant items)

(C5 is an Astro-adapter finding; see the Astro package README. I2 + I5 are deferred at the monorepo level — see the top-level scorecard for the full picture.)

Per-finding remediation detail

For the complete picture across the monorepo (including the Astro-adapter finding C5, the I2 / I5 deferred items, and the v1.0.5 verification pass report), read the top-level README's Security section and the Security considerations & audit history. 115 tests pass at v1.0.5; every ✅ row above is pinned by a named test in test/security.test.ts.

The 30-second integration

import { indexPdfs } from '@icjia/pdf-search-index';

const pdfRows = await indexPdfs([
  'https://example.com/annual-report-2024.pdf',
  'https://example.com/faqs.pdf',
]);

const allRows = [...yourPageRows, ...pdfRows];
const fuse = new Fuse(allRows, { keys: ['title', 'text'], includeMatches: true });

That's it. Each row is { id, url, title, text, pages?, extractedAt? }. Failed extractions return rows with text: '' (the build doesn't fail unless you pass --strict to the CLI).

For highlighted snippets in results:

import { snippetHTMLFor } from '@icjia/pdf-search-index/snippet';

for (const r of fuse.search('stigma')) {
  console.log(r.item.title, snippetHTMLFor(r));
  // → "Stigma PDF For Posting" "…recovery from substance use disorder is hampered by <mark>stigma</mark>…"
}

snippetHTMLFor accepts { contextChars?, matchKey?, collapseWhitespace?, maxSnippets?, separator? }. The maxSnippets option (added 1.0.3, default 1 for backward compatibility) renders up to N non-overlapping highlighted spans per result, joined by separator (default ' … '). Useful for surfacing several passages from a long PDF that matches the query in multiple regions.

Core API

Full option tables, semantics, and edge-case behavior are in the top-level README's Core API section.

Entry points

The IndexedPdf row shape

interface IndexedPdf {
  id: string; // 'pdf-' + first 12 hex chars of SHA-256(url) — stable across rebuilds
  url: string;
  title: string; // markdown link text > pdf.js info-dict Title > humanized filename
  text: string; // empty string on extraction failure (not an error)
  pages?: number;
  extractedAt?: string; // ISO timestamp; OMITTED on cache hits so JSON is byte-stable
}

pages and extractedAt are optional. extractedAt is omitted on cache hits so the JSON stays byte-stable across rebuilds — diffs stay reviewable and CDN caching works.

Options summary

indexPdfs accepts IndexPdfsOptions = ExtractOptions & { concurrency? }:

CLI quick reference

# One-shot: index URLs to JSON on stdout
npx @icjia/pdf-search-index https://...pdf https://...pdf

# From a file (one URL per line, # comments allowed)
npx @icjia/pdf-search-index --from urls.txt

# From a sitemap (scans pages for PDF links, indexes them)
npx @icjia/pdf-search-index --from-sitemap https://example.com/sitemap.xml

# Write to a file instead of stdout
npx @icjia/pdf-search-index --out public/searchIndex.json https://...pdf

# Force re-extraction
npx @icjia/pdf-search-index --refresh https://...pdf
npx @icjia/pdf-search-index --refresh-all https://...pdf

# Sanity check / search / cache management
npx @icjia/pdf-search-index verify https://...pdf
npx @icjia/pdf-search-index search index.json "drug testing"
npx @icjia/pdf-search-index cache ls
npx @icjia/pdf-search-index cache rm <url>
npx @icjia/pdf-search-index cache clear

Exit code is 0 even when individual PDFs fail (the index stays valid; failed rows have text: ''). Pass --strict to flip to exit 1 for CI where a broken upload pipeline should fail the build.

Full CLI option table and output formats: README CLI section.

MCP server

For LLM workflows where the model needs to search inside PDFs during a conversation:

npx -p @icjia/pdf-search-index@latest pdf-search-index-mcp

Always use @latest when wiring into Claude Desktop / Cursor / any MCP-aware client so the client picks up security patches and bug fixes automatically. Sample config:

{
  "servers": {
    "pdf-search": {
      "command": "npx",
      "args": ["-p", "@icjia/pdf-search-index@latest", "pdf-search-index-mcp"]
    }
  }
}

Tools: extract_pdf, index_pdfs, get_pdf_index, search_pdfs, clear_cache, get_status. Since v1.0.2, every tool's cacheDir argument is jailed under <os.tmpdir>/pdf-search-index-mcp/ — LLM-supplied paths can't escape the safe base. Full MCP details: README MCP section.

Where to learn more

The top-level README is the source of truth. Key sections:

• Where your PDFs can live — static /public/, external CMS (Strapi v3/v4/v5, Sanity, Contentful, Drupal), external CDN (S3, R2), local file://. Includes Strapi quirks: relative URLs, token-gated media, structured media relations.
• Using a search engine other than Fuse.js — recipes for MiniSearch, Orama, Lunr, FlexSearch, Pagefind, Typesense, MeiliSearch, Algolia.
• Security considerations — trust model, v1.0.2 hardening details, embedding the index into HTML.
• Examples — seven runnable example sites covering every integration pattern.

Adapter packages

For frameworks without an adapter (Vite, Next.js, Eleventy, SvelteKit, Remix, plain Node, etc.), use this package directly with a prebuild script — see the AGENTS.md "Path A" recipe.

Versioning

Currently at 1.0.3 (documentation + ecosystem release, additive snippetHTMLFor maxSnippets option, default DEFAULT_FUSE_OPTIONS.threshold lowered to 0.2, fuse.js dev/example pin moved to 7.4.0-beta.6, second adversarial red/blue team audit pass on the v1.0.3 deltas on 2026-05-16 — see CHANGELOG.md and the top-level audit history). All three packages in this monorepo move in lockstep.

License

MIT