Local-First Document RAG System with Hybrid Search & Multi-Format Support

We built a complete local-first RAG system with multi-format document ingestion, structure-aware chunking, local embedding generation, a hybrid search pipeline (semantic + full-text + recency), cross-encoder reranking, and a web-based UI — all running entirely on the user's machine.

Architecture

• Document Loaders: Format-specific parsers for PDF, DOCX, XLSX, CSV, HTML, Markdown, and plain text
• Chunker: Structure-aware splitting that preserves page, sheet, and heading boundaries
• Embeddings: Local embedding model via Transformers.js (no external API calls)
• Vector Database: LanceDB (serverless, file-based) for embedding storage and similarity search
• Full-Text Search: Trigram-based indexing for lexical matching
• Reranker: Cross-encoder model for context-aware result scoring
• Query Analyzer: Intent detection routing between semantic and structured queries
• Web Server: Express.js API with project management and search endpoints
• Frontend: Web-based UI for document upload, management, and interactive search

Document Processing Pipeline

Multi-Format Loaders

A registry pattern auto-detects file type and routes to the appropriate parser:

• PDF — Text extraction with page-level segmentation
• Word (.docx/.doc) — Heading-aware parsing preserving document hierarchy
• Excel/CSV — Sheet-by-sheet parsing with header detection and row-level content
• HTML — Tag-aware extraction with structure preservation
• Markdown — Heading-based section parsing
• Plain Text — Line-based segmentation

Each loader extracts metadata (title, author, creation date, page/sheet count, word count) alongside the content, producing structured sections with source references.

Structure-Aware Chunking

Unlike naive text splitting, the chunker respects document boundaries:

• Preserves page breaks (PDFs), sheet boundaries (spreadsheets), and heading hierarchy (Word/Markdown)
• Token-based sizing with configurable chunk size and overlap
• Hierarchical fallback: splits by sections first, then paragraphs, then sentences
• Each chunk retains source metadata (page number, sheet name, heading) for attribution

Embedding & Indexing

Local Embedding Model

• Runs entirely locally via Transformers.js — no data leaves the machine
• Quantized model for performance optimization
• Batch embedding for efficient bulk processing
• Automatic truncation at word boundaries with L2 normalization

Vector Storage

LanceDB provides serverless vector storage:

• File-based (no separate database server needed)
• Per-project isolation with independent indices
• SHA256-based cache keys for deduplication
• Metadata stored alongside vectors for filtered retrieval

Hybrid Search Pipeline

The retrieval pipeline combines three ranking signals for better results than any single approach:

Signal 1: Embedding Search (Semantic)

Vector similarity search finds chunks with related meaning even when different words are used. Handles paraphrasing, synonyms, and conceptual queries.

Signal 2: Full-Text Search (Lexical)

Trigram-based indexing with Jaccard similarity catches exact keyword matches that embedding search might miss — important for technical terms, names, and identifiers.

Signal 3: Recency Boost

Exponential decay weighting favors recently accessed or modified documents, ensuring up-to-date information surfaces first.

Score Combination

Signals are combined with configurable weights (default: 50% semantic, 25% lexical, 25% recency), normalized, and filtered by a minimum score threshold.

Cross-Encoder Reranking

After initial retrieval, a cross-encoder model re-scores the top candidates:

• Context-aware scoring considers query-document pairs together (not independently)
• Keyword boost calculation for term overlap
• Blended scoring (cross-encoder + keyword signals)
• Produces a final ranked list with higher precision than first-pass retrieval alone

Structured Data Support

For spreadsheet content, the system provides additional capabilities:

• Auto-detection of column types (numeric, date, boolean, string)
• Natural language filtering (e.g., "employees in engineering with salary above threshold")
• Aggregation support (count, sum, average, min, max)
• Query analyzer routes structured queries to a dedicated engine rather than embedding search

Web Interface

• Project Management — Create, update, and delete knowledge base projects
• Document Upload — Drag-and-drop file upload with format auto-detection
• Document Creation — Create documents from text directly in the UI
• Interactive Search — Natural language query interface with ranked results
• Statistics — Index size, document count, and format distribution per project

Key Features

1. Fully Local — All processing on-device; no external API calls for embeddings or search
2. 9 Input Formats — PDF, DOCX, DOC, XLSX, XLS, CSV, HTML, Markdown, plain text
3. Structure-Aware Chunking — Preserves pages, sheets, and headings as chunk boundaries
4. Hybrid Search — Combines semantic, lexical, and recency signals for better retrieval
5. Cross-Encoder Reranking — Second-pass scoring for higher precision results
6. Structured Queries — Natural language filtering and aggregation on spreadsheet data
7. Serverless Vector DB — LanceDB file-based storage with no infrastructure overhead
8. Document Writing — Export capabilities for PDF, DOCX, and XLSX creation
9. Project Isolation — Independent knowledge bases with separate indices
10. Web UI — Complete interface for document management and interactive search