500 points if your "AI agent" is a ChatGPT wrapper that reads a CSV and sends a Slack message but your pitch deck says "autonomous multi-agent orchestration platform"
- report benchmark that conveniently omits well known SOTAs, 20 points
- conveniently omit well known benchmarks because not SOTA, 30 points
- change one tiny term in GRPO and call it a completely different acronym, 50 points
- try to slide in a systems hack and but title your paper as though it is a model improvement, 100 points (prizes to the first replier who figures out which recent paper i am subtweeting here)
A: We’ve developed the next generation of Artificial Intelligence.
B: So this model/product takes more human-like actions or has a more fully-realized world model?
A: no AI is a term of art synonymous to large language models. We just made an LLM that uses 10% less RAM.
In this case the short version would be “it’s not general intelligence, it’s an LLM”. The reason it’s a motte and bailey is because you have a 100 slide deck implying your product can write the next great American novel—implying the bailey exists.
I thought the author was talking about market positioning [0] E.g. Apple calling Vision Pro a spatial computing headset rather than VR. That didn't quite mesh with the hedging/motte-and-bailey reference. So I suppose they are really trying to say that you overpromise and underdeliver, using weasel wording to cover the difference? I'm not sure I would call even that a motte-and-bailey, but at least it makes more sense than my original interpretation.
The list avoids many of the real sins and has plenty of mis-analysis, for instance,
20 points for doing the usual motte-and-bailey or hedging in the form of “It is not X. It is Y.”
I mean, that language pattern is often appropriate but for people who are paying attention today it is a sign of... something.
I mean, I am tired of Copilot giving answers like "You're not a fur, you're a therianthrope" It is really a tracer, I think, for someone for whom the lights are on and nobody is home, like they want to be a top blogger about AI but they haven't caught on that the "It's not X, it's Y" pattern is a tell.
500 points if your "AI agent" is a ChatGPT wrapper that reads a CSV and sends a Slack message but your pitch deck says "autonomous multi-agent orchestration platform"
1000 points if your "proprietary model" is GPT-4 with a system prompt and you call it your "research lab"
Research is Science and Science doens't care what you believe!
- report benchmark that conveniently omits well known SOTAs, 20 points
- conveniently omit well known benchmarks because not SOTA, 30 points
- change one tiny term in GRPO and call it a completely different acronym, 50 points
- try to slide in a systems hack and but title your paper as though it is a model improvement, 100 points (prizes to the first replier who figures out which recent paper i am subtweeting here)
- forbes 30 under 30, 100 points
fine tune an oss model and call it a groundbreaking innovation -- 20 points
> forbes 30 under 30, 100 points
too far, at this point you go to jail
Just for fun. I asked Claude to build an online rater using the rubrics https://yujqiao.github.io/bs-index/
-10 points if you actually read your marketing yourself before copy and pasting it on the website
The author has gone so far as to link to Motte-and-Bailey fallacy (doctrine?) [0], but after reading it I see no relation to "It's not X, it's Y"?
[0] https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
Charitably I think it would go like this:
A: We’ve developed the next generation of Artificial Intelligence.
B: So this model/product takes more human-like actions or has a more fully-realized world model?
A: no AI is a term of art synonymous to large language models. We just made an LLM that uses 10% less RAM.
In this case the short version would be “it’s not general intelligence, it’s an LLM”. The reason it’s a motte and bailey is because you have a 100 slide deck implying your product can write the next great American novel—implying the bailey exists.
I thought the author was talking about market positioning [0] E.g. Apple calling Vision Pro a spatial computing headset rather than VR. That didn't quite mesh with the hedging/motte-and-bailey reference. So I suppose they are really trying to say that you overpromise and underdeliver, using weasel wording to cover the difference? I'm not sure I would call even that a motte-and-bailey, but at least it makes more sense than my original interpretation.
[0] https://www.aprildunford.com/post/a-product-positioning-exer...
I submit that doing (4) earns 40 points, rather than 20.
I call upon this body to draw up a spreadsheet of some popular AI marketing and their scores.
A similar BS is also every CEO claiming AI made their company 10x faster yet GDP trends not really budging
Why would you expect to see speed of software development reflected in the current GDP?
If every software company was _actually_ 4-5x or more productive (like CEOs are saying) that should be easily detectable in economic data
Because, allegedly, everyone is writing bespoke software that solves their every need?
Could not care less about AI, but this font is amazing.
https://practicaltypography.com/equity.html
it looks kind of messy on my 1200p monitor, maybe its better on high res
agree, its visually interesting but not easy to read
Highlights from the Crackpot Index [1] that inspired TFA:
1 point for every statement that is widely agreed on to be false.
2 points for every statement that is clearly vacuous.
3 points for every statement that is logically inconsistent.
5 points for each such statement that is adhered to despite careful correction.
5 points for using a thought experiment that contradicts the results of a widely accepted real experiment.
5 points for each word in all capital letters (except for those with defective keyboards).
10 points for pointing out that you have gone to school, as if this were evidence of sanity.
10 points for each new term you invent and use without properly defining it.
10 points for arguing that a current well-established theory is "only a theory", as if this were somehow a point against it.
20 points for suggesting that you deserve a Nobel prize.
20 points for defending yourself by bringing up (real or imagined) ridicule accorded to your past theories.
20 points for naming something after yourself. (E.g., talking about the "The Evans Field Equation" when your name happens to be Evans.)
20 points for talking about how great your theory is, but never actually explaining it.
40 points for comparing those who argue against your ideas to Nazis, stormtroopers, or brownshirts.
[1] _ https://math.ucr.edu/home/baez/crackpot.html
[dead]
The list avoids many of the real sins and has plenty of mis-analysis, for instance,
I mean, that language pattern is often appropriate but for people who are paying attention today it is a sign of... something.I mean, I am tired of Copilot giving answers like "You're not a fur, you're a therianthrope" It is really a tracer, I think, for someone for whom the lights are on and nobody is home, like they want to be a top blogger about AI but they haven't caught on that the "It's not X, it's Y" pattern is a tell.