The $0 runtime API cost is an interesting angle. A lot of AI products I've seen are fine with the initial demo, but the economics get very different once you have real usage.
I'd be interested to see how the quality compares with a traditional RAG setup on less predictable queries. The cost reduction is obviously attractive if the quality stays close.
I built RAGless to remove the LLM from the critical query path. Traditional RAG is powerful, but introduces non-deterministic behavior (hallucinations) and recurring API costs at runtime.
RAGless shifts the LLM entirely to build/ingestion time. It processes your documents offline to generate a deterministic artifact of Question-Answer pairs, strictly grounded in source quotes.
At runtime, there is no LLM in the loop. It relies solely on semantic search (Q-Q matching) between the user's query and the pre-computed questions.
$0 LLM API costs per query.
Deterministic outputs: it only serves pre-validated answers.
Low latency: zero generation wait time.
The hardest bottleneck in this "compile-time" architecture is validating the generated artifact before deployment. Currently, I use strict JSON schema enforcement and an optional --judge pass. The judge acts purely as a tripwire/veto to discard ungrounded blocks, keeping the ultimate trust on the deterministic layer.
I'd love to hear your thoughts on this pattern and how you tackle artifact validation.
Im not deep into the space but surely zero LLM cost would be default for search???
The $0 runtime API cost is an interesting angle. A lot of AI products I've seen are fine with the initial demo, but the economics get very different once you have real usage. I'd be interested to see how the quality compares with a traditional RAG setup on less predictable queries. The cost reduction is obviously attractive if the quality stays close.
This is interesting, especially if the data doesn't change much.
thanks :)
Hi HN,
I built RAGless to remove the LLM from the critical query path. Traditional RAG is powerful, but introduces non-deterministic behavior (hallucinations) and recurring API costs at runtime.
RAGless shifts the LLM entirely to build/ingestion time. It processes your documents offline to generate a deterministic artifact of Question-Answer pairs, strictly grounded in source quotes.
At runtime, there is no LLM in the loop. It relies solely on semantic search (Q-Q matching) between the user's query and the pre-computed questions.
$0 LLM API costs per query.
Deterministic outputs: it only serves pre-validated answers.
Low latency: zero generation wait time.
The hardest bottleneck in this "compile-time" architecture is validating the generated artifact before deployment. Currently, I use strict JSON schema enforcement and an optional --judge pass. The judge acts purely as a tripwire/veto to discard ungrounded blocks, keeping the ultimate trust on the deterministic layer.
I'd love to hear your thoughts on this pattern and how you tackle artifact validation.