So what's supposed to make this service better than OpenRouter (https://openrouter.ai), which has an extensive list of available models with transparent pricing for all of them?
> You can authenticate users through any identity provider, like Google or Apple, and set per-user request or spending quotas. You can also define an allowlist for endpoints and models. To monetize, you can apply different quotas for various user tiers.
So, I'll be honest, I don't understand this market. I get that one can be profitable selling shovels during the gold rush, sure. But I have trouble understanding who is knowledgeable/dedicated enough to try to get their AI app going, but would pay to abstract/outsource this part of the chain.
(I suppose, relately, I have trouble understanding why anyone would just sort of presume OpenAI would be forever the best backend here as well?)
Here's my hypothesis: since we are still in the gold rush phase of AI, going to market early and pivoting quickly is imperative. Therefore, it is worthwhile to focus on your differentiating factor and outsource everything else. Plus we use the OpenAI API format, so there isn't vendor lock-in - though we hope we provide enough value for you to stay!
And you are right on the money that OpenAI may not be the best backend forever. That's why we also support Anthropic, Groq, and Mistral.
A friend of mine works for a very large company and he was telling me about an AI startup they bought for (he thinks) in the ballpark of 8 digits. After they were on boarded he was able to dig into what it was. It's literally a GPT backend with system prompts and RAG. If you have a good sales team you can print money for doing almost nothing right now.
Thanks! Despite my best Google-fu, I didn't find Puter in my market research :) It is a very interesting project for sure!
Looking at Puter's offering, at the end of the day, we serve the same goal of making developer's life easier. ServerlessAI is more narrowly focused on AI use cases, while Puter is providing a more generic app runtime. Both have their best use cases!
To potentially save you some headache, take a look at serverless.com and weigh the likelihood they come after you about that name if you are planning on making this a business.
(And yes, I hate their name too. I don't honestly know how defendable an entire technology term actually is. It also results in terrible Googling.)
IANAL, but since "Serverless, Inc." chose a generic term for their trademark, usually this means only very specific instances of that mark are enforceable, e.g. "Serverless Framework", "serverless.com" and "Serverless, Inc." But if a company were to say "We make it really easy to use serverless technologies in the cloud!" (note lower case s), that shouldn't be infringing because it's just using serverless as a generic, descriptive term.
"ServerlessAI" and "serverlessai.dev" shouldn't be infringing because they incorporate the generic term in their mark, not the Serverless Framework-specific term. Of course, this means that ServerlessAI would have the same issues you point out - a less-defensible mark and poor Google results.
Also, in the US pretty much anyone can sue for anything, so even if you're in the right it can be an expensive headache to have to defend yourself.
Thank you both! In a way this could be a nice problem to have - attracting the attention of BigCo means we are big enough right ;) But yes, we will definitely be prepared for that possibility!
Thank you! We do believe app developers will find this valuable, and we are also working on IAP integration. Sadly I am not an app developer, so if anyone have any suggestions on how I can serve this community better, I'm eager to hear from you! My email is in my HN profile.
Will look into Firebase Marketplace! That is a great suggestion!
Hah! Nice idea! I built something with a similar mindset but instead of calling cloud AI providers my aim is to provide a self-hostable complete AI platform: https://github.com/singulatron/singulatron
I know that might sound like putting the server back to serverless. But I would say it's being your own serverless provider - once you have the platform installed on your servers you can build frontend-only AI apps on top.
Hope you don't mind the self-plug. Your approach definitely a ton of advantages when starting out (no infra to manage etc).
Great idea! I like the ergonomics of this for the developer-side, it's easy to add and puts the onus on the developer to have a robust auth system that avoids users creating 1000s of accounts so they can get unlimited LLM access.
One challenge on frontend-only apps is if the prompt is proprietary then this will be exposed unless you will then offer prompt templating or prompt mapping on your side i.e. the frontend says prompt: Template_123 and then this maps to the actual prompt somehow. Prompting is important still and maybe for a while so having the internals externally available could be sensitive.
I disagree. These sorts of pricing structures rule out consumer usage quickly due to high numbers of users. Have flat usage fees and get rid of these per user limits. Both doesn’t make sense unless you want to turn away any app that has many users.
I’d also recommend they clean up the copy of what they offer (expand on the why).
Thanks for the feedback! Pricing is something we are iterating on. Our intention is that we only make money if our customers make money - which is why only authenticated users count towards the limit.
If pricing is preventing anyone from using our product, please shoot me an email (in my HN profile) and we'd love to hear about your use case!
Good call on the marketing copy! We will do some revisions!
Thanks for the feedback! We are building prompt templates right now. Besides the security benefits you mentioned, this can also enable developers to tweak prompts without redeployment, run A/B tests, and evaluate different models. It's an incredibly powerful tool!
So what's supposed to make this service better than OpenRouter (https://openrouter.ai), which has an extensive list of available models with transparent pricing for all of them?
Perhaps:
> You can authenticate users through any identity provider, like Google or Apple, and set per-user request or spending quotas. You can also define an allowlist for endpoints and models. To monetize, you can apply different quotas for various user tiers.
So, I'll be honest, I don't understand this market. I get that one can be profitable selling shovels during the gold rush, sure. But I have trouble understanding who is knowledgeable/dedicated enough to try to get their AI app going, but would pay to abstract/outsource this part of the chain.
(I suppose, relately, I have trouble understanding why anyone would just sort of presume OpenAI would be forever the best backend here as well?)
Here's my hypothesis: since we are still in the gold rush phase of AI, going to market early and pivoting quickly is imperative. Therefore, it is worthwhile to focus on your differentiating factor and outsource everything else. Plus we use the OpenAI API format, so there isn't vendor lock-in - though we hope we provide enough value for you to stay!
And you are right on the money that OpenAI may not be the best backend forever. That's why we also support Anthropic, Groq, and Mistral.
Doesn't everyone ask this question about Okta, like what am I doing farming out my user login?
I've seen an AI yearbook photos styled app print money that was serverless.
There's lots of novelty apps atm that, as the other commented stated, just want to get to market as soon as possible to validate an idea
A friend of mine works for a very large company and he was telling me about an AI startup they bought for (he thinks) in the ballpark of 8 digits. After they were on boarded he was able to dig into what it was. It's literally a GPT backend with system prompts and RAG. If you have a good sales team you can print money for doing almost nothing right now.
This is cool, congrats on launching!
How is it different from Puter AI, which offers auth + free inference?
https://docs.puter.com/AI/chat/
Thanks! Despite my best Google-fu, I didn't find Puter in my market research :) It is a very interesting project for sure!
Looking at Puter's offering, at the end of the day, we serve the same goal of making developer's life easier. ServerlessAI is more narrowly focused on AI use cases, while Puter is providing a more generic app runtime. Both have their best use cases!
Founder of Puter here. Thank you very much for mentioning us!
To potentially save you some headache, take a look at serverless.com and weigh the likelihood they come after you about that name if you are planning on making this a business.
(And yes, I hate their name too. I don't honestly know how defendable an entire technology term actually is. It also results in terrible Googling.)
IANAL, but since "Serverless, Inc." chose a generic term for their trademark, usually this means only very specific instances of that mark are enforceable, e.g. "Serverless Framework", "serverless.com" and "Serverless, Inc." But if a company were to say "We make it really easy to use serverless technologies in the cloud!" (note lower case s), that shouldn't be infringing because it's just using serverless as a generic, descriptive term.
"ServerlessAI" and "serverlessai.dev" shouldn't be infringing because they incorporate the generic term in their mark, not the Serverless Framework-specific term. Of course, this means that ServerlessAI would have the same issues you point out - a less-defensible mark and poor Google results.
Also, in the US pretty much anyone can sue for anything, so even if you're in the right it can be an expensive headache to have to defend yourself.
Thank you both! In a way this could be a nice problem to have - attracting the attention of BigCo means we are big enough right ;) But yes, we will definitely be prepared for that possibility!
This is a great idea. You should market to app devs as well.
I would also build this on top of firebase marketplace: https://extensions.dev
Thank you! We do believe app developers will find this valuable, and we are also working on IAP integration. Sadly I am not an app developer, so if anyone have any suggestions on how I can serve this community better, I'm eager to hear from you! My email is in my HN profile.
Will look into Firebase Marketplace! That is a great suggestion!
If you don't want to pay for this service, keytrustee.org does this for free.
Hah! Nice idea! I built something with a similar mindset but instead of calling cloud AI providers my aim is to provide a self-hostable complete AI platform: https://github.com/singulatron/singulatron
I know that might sound like putting the server back to serverless. But I would say it's being your own serverless provider - once you have the platform installed on your servers you can build frontend-only AI apps on top.
Hope you don't mind the self-plug. Your approach definitely a ton of advantages when starting out (no infra to manage etc).
Please hire a real artist those graphics on the home page are disturbing.
Great idea! I like the ergonomics of this for the developer-side, it's easy to add and puts the onus on the developer to have a robust auth system that avoids users creating 1000s of accounts so they can get unlimited LLM access.
One challenge on frontend-only apps is if the prompt is proprietary then this will be exposed unless you will then offer prompt templating or prompt mapping on your side i.e. the frontend says prompt: Template_123 and then this maps to the actual prompt somehow. Prompting is important still and maybe for a while so having the internals externally available could be sensitive.
I disagree. These sorts of pricing structures rule out consumer usage quickly due to high numbers of users. Have flat usage fees and get rid of these per user limits. Both doesn’t make sense unless you want to turn away any app that has many users.
I’d also recommend they clean up the copy of what they offer (expand on the why).
Other than that looks cool
Thanks for the feedback! Pricing is something we are iterating on. Our intention is that we only make money if our customers make money - which is why only authenticated users count towards the limit.
If pricing is preventing anyone from using our product, please shoot me an email (in my HN profile) and we'd love to hear about your use case!
Good call on the marketing copy! We will do some revisions!
Thanks for the feedback! We are building prompt templates right now. Besides the security benefits you mentioned, this can also enable developers to tweak prompts without redeployment, run A/B tests, and evaluate different models. It's an incredibly powerful tool!