We really need to band together to fund / sponsor targeted inducement prizes (a la Nobel laureate Michael Kremer) for open models.
Every 6-12 months, give out $200K to the first model to hit a min threshold on a set of ~5-10 hard benchmarks (+ perhaps one secret benchmark) using a total of 16GB / 32GB / 64GB / 128GB of VRAM (at a min context length of 200K), then move the threshold up. Quantization etc. is dealers choice, it just needs to nail the benchmark on a reference machine by using exactly that much VRAM (no mapping to RAM / disk etc.)
You could crowdsource the funding, and cross subsidize by adding targeted prizes focused on corporate needs (the classic one is PDF processing benchmarks), and say that 25% of each corporate prize funding also flows into the general prize pool.
For a lot of these open-source model companies, it's less about the $s (though $200K is nothing to sneeze at), it's the clear recognition that helps their model efforts stand out, gain usage etc.
I think the Korean government did have a competition like this, I remember last year we got a bunch of models released at the same time to make the cut for the next stage. The models weren't anything to write home about, IMO.
Having it with clear hw requirements tiers is a nice differentiator. The only issue is that the benchmarks would 100% need to be closed, no other way around it. And then you have the issue of creating and curating good evals for every "stage" of the project. That's a hard task even for "honest" lab-internal evals. And you'd have to publish those evals after each round (for trust purposes), and start over for the next round. Doable, but it would cost a lot (probably more than the prize pools) and you'd have to keep doing this.
It was a competition organized by the Korean government but the directive wasn't for the same cause as the writer. It was more for constructing Sovereign AI for the country. Also, all models except Exaone had some weights copied from Chinese models, and from the corporate point of view, developing from-scratch model is not cheap despite the financial support from the government.
Yes, I hope the open model communities will someday be able to run current frontier models which will be able to handle autonomous tasks and the hardware to run it will be served at consumer level; however, like how recycling isn't profitable, no companies will fully commit to the movement. Don't get it twisted, I don't have a solution but maybe a global scale movement to liberate knowledge-library could suffice.
This seems like a good idea but also just fun. I can’t train a frontier model but maybe I could compete in the 16 GB tier. I would suspect there are a ton of optimizations out there for the taking that aren’t being considered because frontier models are way above these weight classes
They already invest in open-source AI, but nothing is truly free. Commercial AI will usually dominate because devs are paid to make it their primary effort. Goodwill and part-time contributions cannot reliably compete with livelihood and profit incentives.
That's what people said about operating systems, and databases, and compilers, and so many other big complicated categories of software that over time became increasingly dominated by OSS
I think the main problem in LLM models is that you can not make a PR to an open source project to tweak some training parameters, prove it is an improvement and merge it.
If you can not run the training yourself you can not contribute. So open source contribution model does not work. All examples you gave have a fairly low threshold of capital expenditure required to be a contributor (basically a laptop).
Even back in the 90s a person could get a standard, but powerful, PC to do these things. The one exception was 3d graphics which took quite some time to become affordable and even there it was a single one-time expenditure (a workstation) per contributor.
For normal OSS the only competition between contributors was for attention of maintainers to review and accept patches.
In an open-source LLM model contributors would compete with each other for computing resources for model tweaks and changes. The alternative model is that the contributor pays for the compute, but that increases the bar really high for contributions.
OSS only dominates for software that is commoditized and the published computer science research for that software domain is close to the frontier.
OSS struggles at being relevant when software is non-commodity e.g. office suites. In software domains like databases where the state-of-the-art computer science research is often unpublished, OSS struggles to be relevant at the higher end of the market on technical merits.
When deciding what should be OSS, it is useful to consider the preconditions that have made it successful.
I personally expect token production to commoditized like mobile data. It's already happening.
See open weights gaining adoption, OpenAi talking about how 5.6 is cheaper than Fable, people are taking multiple approaches to reduce their token spend, expectations for progress in hardware and algos, and certain Ai leaders talking about how token prices should be 10-100x lower than they are.
OSS does not necessarily mean the contributions are from "goodwill or part-time contributions". In fact, I would wager the most widely used OSS software is largely written by contributors paid to do so by corporations. At least for Linux, about 80% - 85% of contributions are from developers paid to do it (https://newsletter.pragmaticengineer.com/p/how-linux-is-buil...)
Corporations have had many reasons to invest their money in open source software -- custom requirements, marketing / developer mindshare, commoditizing complements -- but as cutting edge LLMs get more and more expensive to train, you'd be hard-pressed to find corporations who will put in that kind of money if they cannot recoup their investments.
on the small chance that the four billionaires who currently have near-exclusive control of closed sota models, (that is altman, amodei, zuckerberg and musk), are not fleecing their investors and actually build AGI, closed source leaves a choice of powerful government or powerful oligopoly/monarchy.
further explanation of this list:
musk - structural command
zuckerberg - structural command
altman - de facto command after purging rivals and privatisation, loyalty of personnel
amodei - influential, could potentially overthrow current governance
Just because a software is closed-source doesn't mean the knowledge can't be shared. You don't need to see the underlying code to explain to someone architectural patterns or best practices.
The library analogy in the scenario would hold true if LLM providers refused to answer any questions about RL or Transformers.
I am a big proponent of open-source open-weight models, but mostly because I think it's just a better product. We've seen that they are much cheaper to train and operate. Frontier intelligence might not be needed for most tasks. Just let the market decide. My bet is that LLMs will become analogous to programming languages, and big labs will make their money by fine-tuning models for very specific use cases or by deploying them for customers.
The library analogy works (for me), but the uncomfortable part is that most "open" models are closer to receiving a compiled binary than receiving the library
No, I want govt to tax themmore. So far frontier AI companies produce negative value to near everyone (by sheer power cost increase it adds essentially tax to every other business) but themselves economy wise.
Yeah, wooho, new model found a bunch of bugs, now the bad guys can do it too so security expenses spiked! It's only good for shovel sellers.
> Even if the current approaches will continue to scale, this would be as if in the early days of computing, perhaps someone invented a bubble sort for sorting numbers (an n-squared algorithm), and the tech companies at the time decided they were going to build vast data centers to sort numbers and not bother to figure out that there's an n-log-n way of doing it <laughs>
...to which I have to say: yes, definitely! And he's right about open-source AI too.
We really need to band together to fund / sponsor targeted inducement prizes (a la Nobel laureate Michael Kremer) for open models.
Every 6-12 months, give out $200K to the first model to hit a min threshold on a set of ~5-10 hard benchmarks (+ perhaps one secret benchmark) using a total of 16GB / 32GB / 64GB / 128GB of VRAM (at a min context length of 200K), then move the threshold up. Quantization etc. is dealers choice, it just needs to nail the benchmark on a reference machine by using exactly that much VRAM (no mapping to RAM / disk etc.)
You could crowdsource the funding, and cross subsidize by adding targeted prizes focused on corporate needs (the classic one is PDF processing benchmarks), and say that 25% of each corporate prize funding also flows into the general prize pool.
For a lot of these open-source model companies, it's less about the $s (though $200K is nothing to sneeze at), it's the clear recognition that helps their model efforts stand out, gain usage etc.
I think the Korean government did have a competition like this, I remember last year we got a bunch of models released at the same time to make the cut for the next stage. The models weren't anything to write home about, IMO.
Having it with clear hw requirements tiers is a nice differentiator. The only issue is that the benchmarks would 100% need to be closed, no other way around it. And then you have the issue of creating and curating good evals for every "stage" of the project. That's a hard task even for "honest" lab-internal evals. And you'd have to publish those evals after each round (for trust purposes), and start over for the next round. Doable, but it would cost a lot (probably more than the prize pools) and you'd have to keep doing this.
It was a competition organized by the Korean government but the directive wasn't for the same cause as the writer. It was more for constructing Sovereign AI for the country. Also, all models except Exaone had some weights copied from Chinese models, and from the corporate point of view, developing from-scratch model is not cheap despite the financial support from the government.
Yes, I hope the open model communities will someday be able to run current frontier models which will be able to handle autonomous tasks and the hardware to run it will be served at consumer level; however, like how recycling isn't profitable, no companies will fully commit to the movement. Don't get it twisted, I don't have a solution but maybe a global scale movement to liberate knowledge-library could suffice.
Yeah, I suspect the $200K check would be the cheapest line item in the whole thing
This seems like a good idea but also just fun. I can’t train a frontier model but maybe I could compete in the 16 GB tier. I would suspect there are a ton of optimizations out there for the taking that aren’t being considered because frontier models are way above these weight classes
I would just add a reproducibility requirement and avoid keeping the exact same benchmarks for too long
Not sure that would even cover power for training
They already invest in open-source AI, but nothing is truly free. Commercial AI will usually dominate because devs are paid to make it their primary effort. Goodwill and part-time contributions cannot reliably compete with livelihood and profit incentives.
That's what people said about operating systems, and databases, and compilers, and so many other big complicated categories of software that over time became increasingly dominated by OSS
I think the main problem in LLM models is that you can not make a PR to an open source project to tweak some training parameters, prove it is an improvement and merge it.
If you can not run the training yourself you can not contribute. So open source contribution model does not work. All examples you gave have a fairly low threshold of capital expenditure required to be a contributor (basically a laptop).
Even back in the 90s a person could get a standard, but powerful, PC to do these things. The one exception was 3d graphics which took quite some time to become affordable and even there it was a single one-time expenditure (a workstation) per contributor.
For normal OSS the only competition between contributors was for attention of maintainers to review and accept patches.
In an open-source LLM model contributors would compete with each other for computing resources for model tweaks and changes. The alternative model is that the contributor pays for the compute, but that increases the bar really high for contributions.
OSS only dominates for software that is commoditized and the published computer science research for that software domain is close to the frontier.
OSS struggles at being relevant when software is non-commodity e.g. office suites. In software domains like databases where the state-of-the-art computer science research is often unpublished, OSS struggles to be relevant at the higher end of the market on technical merits.
When deciding what should be OSS, it is useful to consider the preconditions that have made it successful.
I personally expect token production to commoditized like mobile data. It's already happening.
See open weights gaining adoption, OpenAi talking about how 5.6 is cheaper than Fable, people are taking multiple approaches to reduce their token spend, expectations for progress in hardware and algos, and certain Ai leaders talking about how token prices should be 10-100x lower than they are.
OSS does not necessarily mean the contributions are from "goodwill or part-time contributions". In fact, I would wager the most widely used OSS software is largely written by contributors paid to do so by corporations. At least for Linux, about 80% - 85% of contributions are from developers paid to do it (https://newsletter.pragmaticengineer.com/p/how-linux-is-buil...)
Corporations have had many reasons to invest their money in open source software -- custom requirements, marketing / developer mindshare, commoditizing complements -- but as cutting edge LLMs get more and more expensive to train, you'd be hard-pressed to find corporations who will put in that kind of money if they cannot recoup their investments.
all those have either a consulting company around it or few big corporate contributors
AGI is not software
on the small chance that the four billionaires who currently have near-exclusive control of closed sota models, (that is altman, amodei, zuckerberg and musk), are not fleecing their investors and actually build AGI, closed source leaves a choice of powerful government or powerful oligopoly/monarchy.
further explanation of this list:
musk - structural command
zuckerberg - structural command
altman - de facto command after purging rivals and privatisation, loyalty of personnel
amodei - influential, could potentially overthrow current governance
Just because a software is closed-source doesn't mean the knowledge can't be shared. You don't need to see the underlying code to explain to someone architectural patterns or best practices.
The library analogy in the scenario would hold true if LLM providers refused to answer any questions about RL or Transformers.
I am a big proponent of open-source open-weight models, but mostly because I think it's just a better product. We've seen that they are much cheaper to train and operate. Frontier intelligence might not be needed for most tasks. Just let the market decide. My bet is that LLMs will become analogous to programming languages, and big labs will make their money by fine-tuning models for very specific use cases or by deploying them for customers.
The library analogy works (for me), but the uncomfortable part is that most "open" models are closer to receiving a compiled binary than receiving the library
I wonder if some sort of member owned cooperative would be the way forward if we the people want to retain any control.
No, I want govt to tax themmore. So far frontier AI companies produce negative value to near everyone (by sheer power cost increase it adds essentially tax to every other business) but themselves economy wise.
Yeah, wooho, new model found a bunch of bugs, now the bad guys can do it too so security expenses spiked! It's only good for shovel sellers.
Ah, this from the same David Siegel who said almost 2 yrs ago (in a talk found here: https://youtu.be/0z60xUDo-NI?si=PTDe11-sn2P53qo5&t=420) that the AI data center buildout was premature because:
> Even if the current approaches will continue to scale, this would be as if in the early days of computing, perhaps someone invented a bubble sort for sorting numbers (an n-squared algorithm), and the tech companies at the time decided they were going to build vast data centers to sort numbers and not bother to figure out that there's an n-log-n way of doing it <laughs>
...to which I have to say: yes, definitely! And he's right about open-source AI too.
> AI data center buildout was premature
Ask Amodei how he feels about going to spaceman bad for compute that he couldn't find anywhere else in the market.
Title was: I argued with the father of open source for 2 years. Now the AI fight is the same — only bigger
Op-ed alt link: https://fortune.com/2026/07/03/open-source-ai-same-fight-as-...
ABSLOUTELY NOT.
this is like saying "gov should invest in pyramid schem, because everyone is doing it". or btc. or web3 pictures of monkeys.
what i expect the gov to do is to add a 999% tax or tarif on top of GPUs bougth for AI, after the first 100mi that company spends on it each year.