I'm curious about the "don't use vector search" suggestion. I've done this recently for tool discovery in my own tool service, and it seems to work well, but obviously I'd like to know why that was a bad idea.
Agree. I think it is a little reductive. Regex / free text search can obviously miss things (as can vector stores). Combining both of them is fairly typical (along with a re-ranker, potentially). You are basically trading quality for latency and cost.
I think the problem with vector search is that it’s unbounded - is the 3rd result relevant at all? What about the 4th?
On top of that, I believe it’s clear with how Cursor and Claude converged on regex search after using vectors that asking LLMs to come up with similar terms to regex is more effective than searching for the user’s original term over a vector db.
I'm curious about the "don't use vector search" suggestion. I've done this recently for tool discovery in my own tool service, and it seems to work well, but obviously I'd like to know why that was a bad idea.
Agree. I think it is a little reductive. Regex / free text search can obviously miss things (as can vector stores). Combining both of them is fairly typical (along with a re-ranker, potentially). You are basically trading quality for latency and cost.
I think the problem with vector search is that it’s unbounded - is the 3rd result relevant at all? What about the 4th?
On top of that, I believe it’s clear with how Cursor and Claude converged on regex search after using vectors that asking LLMs to come up with similar terms to regex is more effective than searching for the user’s original term over a vector db.