A few years ago, when terminal agents were still somewhat new, indexing was thought to be the holy grail for coding agents. But in most cases, it did us more harm than good. More often than not, it confused the model instead of letting it directly grep for context and work its way around the codebase. Not to mention that it also increased input token costs and added extra round trips.
Similarity search is rarely that relevant when working on codebases. Our biggest learning actually came from an accidental technical bug where indexing had been returning zero hits for quite some time, and we barely noticed any performance degradation. In fact, the agent seemed to be working better than ever.
After discovering the bug, we did some rigorous A/B testing and realized it was best to drop indexing altogether. There could still be a case for it in extremely large codebases with a lot of docs e.g. excel or several text files.
A few years ago, when terminal agents were still somewhat new, indexing was thought to be the holy grail for coding agents. But in most cases, it did us more harm than good. More often than not, it confused the model instead of letting it directly grep for context and work its way around the codebase. Not to mention that it also increased input token costs and added extra round trips.
Similarity search is rarely that relevant when working on codebases. Our biggest learning actually came from an accidental technical bug where indexing had been returning zero hits for quite some time, and we barely noticed any performance degradation. In fact, the agent seemed to be working better than ever.
After discovering the bug, we did some rigorous A/B testing and realized it was best to drop indexing altogether. There could still be a case for it in extremely large codebases with a lot of docs e.g. excel or several text files.