Memorybook
The Problem
Tools like NotebookLM are useful for asking questions over your own documents, but they are closed products, no self-hosted option, no visibility into how retrieval or memory decisions actually get made.
Architecture & Topology
A full-stack hybrid-RAG app: an Express + TypeScript API with Drizzle ORM over ParadeDB (Postgres with BM25 full-text search and pgvector for embeddings), a self-hosted Inngest pipeline for durable background jobs (ingestion, knowledge-graph extraction, memory extraction), Neo4j for a knowledge-graph layer, and a Next.js App Router client.
Engineering Challenges
Plain vector search misses exact keyword matches and plain keyword search misses paraphrased questions, and any real ingestion pipeline (parsing a PDF, a website, a YouTube transcript) is slow enough that it cannot run inside the request that triggered it.
Applied Solutions
Fused BM25 lexical search with pgvector semantic search using Reciprocal Rank Fusion, so exact terms and paraphrased queries both surface relevant chunks. Moved ingestion, graph extraction, and memory extraction into a self-hosted Inngest pipeline, durable, automatically retried, content-hash deduplicated, and off the request thread entirely.
Results & Benchmarks
A working hybrid-RAG pipeline with multi-layer memory, conversation history, user profile/semantic/episodic/procedural memory, a Neo4j knowledge graph, and document retrieval, coordinated per chat turn against a token budget so context never silently overflows the model.
Performance Tuning
Async ingestion via Inngest keeps upload/parse/embed work off the request thread; RRF fusion weights were tuned so hybrid retrieval does not just default to whichever method returns more results.
Lessons Learned
A knowledge graph and four kinds of memory only help if something enforces a token budget across all of them, otherwise you have just moved the context-overflow problem one layer deeper.