This looks great, actually using streaming for responses is a nice touch.
Having built a few front end ai chat features, one of the key things is making sure the llm provider is swappable, and that chat history is stored in your own database.
The architecture I typically use is chat frontend (usually react) -> custom llm wrapper api -> openai/anthropic api | my own db for chat history .
Would this be able to support calling my custom llm wrapper out of the box?
This looks great, actually using streaming for responses is a nice touch.
Having built a few front end ai chat features, one of the key things is making sure the llm provider is swappable, and that chat history is stored in your own database.
The architecture I typically use is chat frontend (usually react) -> custom llm wrapper api -> openai/anthropic api | my own db for chat history .
Would this be able to support calling my custom llm wrapper out of the box?