Verification-Guided RAG
A 0.5B model that beats a 70B baseline on physics correctness — by verifying its own equations.
A neuro-symbolic retrieval-augmented generation pipeline for semiconductor device physics. A fine-tuned 0.5B LLM (LoRA) answers from hybrid retrieval, and a three-stage SymPy verification layer — symbolic parsing, dimensional analysis, numerical plausibility — checks every candidate, boosting physics-correctness by 135% over a 70B baseline (p = 0.002) while running fully offline on an 8GB GPU.
General-purpose LLMs give fluent but physically wrong answers in technical domains — wrong units, broken equations, implausible numbers — and the usual fix, a larger model judging a smaller one, is expensive and just as unverifiable.
Hybrid retrieval (FAISS dense + BM25 sparse with a custom physics tokenizer, fused via RRF and cross-encoder reranking) grounds a LoRA-tuned 0.5B model. It generates several candidates, and a physics-score-driven Best-of-N step keeps the one that passes deterministic SymPy checks, replacing LLM-as-a-judge evaluation. Statistical tests and ablation studies validate each stage.
- 1Verifying physics deterministically — symbolic, dimensional, and numerical checks instead of an LLM judge.
- 2Retrieving the right equations: dense + sparse retrieval with a physics-aware tokenizer.
- 3Fitting the whole pipeline on an 8GB GPU, fully offline.
- Three-stage SymPy verification: symbolic parsing, dimensional analysis, numerical plausibility.
- Hybrid FAISS + BM25 retrieval with a custom physics tokenizer, RRF fusion, and reranking.
- Deterministic Best-of-N selection replaces LLM-as-a-judge evaluation.
- 116× less VRAM and 7.1× lower latency than the 70B baseline — fully offline.