Problem
Document Q&A systems often use flat chunking that fragments narrative context, or persist all user uploads to shared storage — creating privacy risks and data collisions in multi-user deployments. Scanned PDFs and missing metadata make source traceability unreliable.
Approach
- — Parent-child split: small child chunks embedded in Qdrant for precise ANN search; parent ID lookup expands to full section context at query time.
- — Hybrid search combining all-MiniLM-L6-v2 dense embeddings with FastEmbed sparse vectors.
- — Pydantic-validated Groq LLM extracts document metadata (title, author, type, date) from page 1 on upload.
- — Session-scoped InMemoryStore and :memory: Qdrant collection per upload — zero cross-session data collision.
- — Chainlit UI with async token streaming and expandable citation sidebar (title, author, page, excerpt).
Outcome
- Parent context preserved at generation time while child chunks enable precise vector retrieval.
- Multiple simultaneous users upload and query different documents without shared state.
- Citations surface document metadata and source excerpts automatically on every answer.
- Real-time streaming responses via async LangChain generators and Chainlit.
Metrics
- Retrieval
- Top-K 6
- Child chunks fetched per query (default)
- Embeddings
- 384-dim
- all-MiniLM-L6-v2 + FastEmbed hybrid
- Chunk sizes
- 1500 / 200
- Parent / child characters
- Isolation
- Per-session
- UUID-scoped in-memory Qdrant
Architecture
Trade-offs
Storage
Chosen
In-memory per session
Alternative
Persistent disk-backed index
Session isolation guarantees privacy and multi-user safety. Documents are cleared on restart — acceptable for ephemeral Q&A workflows.
Chunking
Chosen
Parent-child with ID lookup
Alternative
Flat fixed-size chunks
Child chunks enable precise ANN search; parent expansion preserves narrative continuity the LLM needs for faithful answers.
Ingestion
Chosen
PDF-only via PyPDFLoader
Alternative
Multi-format with OCR pipeline
Focused scope for the MVP. Scanned or image-based PDFs without embedded text are a known limitation.
Next project
0%