To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture.
I'm now working on a new, not yet published version of pgrust that incorporates a lot of techniques. Currently the new version:
- Passes 100% of Postgres regression suite
- Implements a thread per connection model instead of the process per connection model Postgres does
- Is 50% faster than Postgres on transaction workloads
- Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
If you have any questions, I'm happy to answer them.
I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all.
Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all my tests passed with flying colors the rewrite itself was broken even on basic functionality or missed a ton of details. It was in effect useless when I dived into it. Initial tests also showed massive gain in performance, and I know people who were involved aren't really dumb so something smelled funny. Turns out all those things left out and honestly... moments were the key ingredients.
What I did learn from those beginning explorations though was that one-shotting, grand architecture or source up-front, master plans up-front.. all these do not yield good results - YET. Who know what we'll see in few years though. What I did found that works (FOR ME, nota bene) is to keep the design and checklists for myself, written by myself and then do a small piece by piece.. as if you would if you were coding alone or if you would waterfalling a small team of talented juniors. Then, suddenly super happy results come out, but then it's mostly you driving all the way where llm writes code and offers advice (which for the most part you ignore). It's a happy place for myself at least. It's then truly unlocking yourself to the mythical 10x.
Rewriting a large proven system with decades of ultra expertise behind it, which I don't have, is guaranteed not to end up the same 1:1 replacement. If you found a recipe for that - please do share.
I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?
That's the million dollar question. Do you/we/us have tests that cover everything which covers QA as well? If such a mythical beast exists, maybe from remnants of ye olde TDD past and hasn't been modified as such.. then maybe this would be possible to do as such.
what model did you try it with? I agree and also push back a bit: How will we know when the LLMs reach the point of handling it if no one takes the leap? I applaud more people sludging through the slop and hauling their slop buckets around.
An example is Fable being released. I felt like the most complex thing I was willing to sludge through was having it clone llama-server's web UI with my own opinions (I really like the original, kudos to them). And the initial skeleton was working so well I felt like I had sunk the tokens and committed to getting it the rest of the way: https://inkcap.click
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
That’s not true for Postgres however: due to its usage of a shared memory pool, whenever a subprocess is terminated unexpectedly, Postgres will kill all other processes and enter recovery mode, replaying the WAL, during which time it will not accept connection requests.
It does this because it can’t possibly know whether the dying process did bad things to the shared memory pool.
You are correct! TIL, and thank you. Connection processes get a SIGQUIT, shared buffers cleared, and WAL replayed, but postmaster stays alive. It's effectively an online restart.
I’d be very curious to hear what an online restart approach for sketchy extensions crashing a shared thread per connection process might look like. This is more a question for the author of the project but I’m curious if they have plans in that direction.
What's online about this restart? Just that the tcp port keeps listening? I assume all running transaction will have to be aborted, right? And connections dropped?
The postmaster is effectively your supervisor. It could see the child segfault and abort the whole DB, but that would be deferring supervision to your init system. Not ideal.
A mixture of threads and processes that can be used to match processors, disk I/O, and network interfaces.
A very long time ago, there was once a feature called "Data Blades" which tanked a commercial database vendor. A badly behaving blade could bring down the entire database. Most anyone who has been working on databases for a few decades remembers this and makes a point of either not introducing these sorts of features or making use of processes over threads.
I have not looked at the code referenced in the mentioned project, but thus far I haven't seen a model that could craft a complete SQL parser on its own.
There are a number of problems, and design decisions, that a developer decides on when writing a database that I don't see any current models… just because you have the ingredients does not mean that the stew is edible.
Informix. Michael Stonebraker, to his credit, learned a lot along the way and revised his thinking about database technology and capability after believing that a single engine could be good at everything.
I performed both searches on Kagi and didn't see anything about IBM. I do see results for "DataBlade" and "Data blade database" but I didn't try those specific variations after my first two attempts returned nothing of interest. That's too much effort to decipher a HN post.
It would have been less effort than it took you to write this comment. Perhaps next time that you can’t be bothered, just ignore the comment and move on to another thread that meets your required spoonfeeding levels.
Yes, Kagi was sufficiently confused as to what "Data blades" are that it actually thought I was looking for replacement woodworking blades. "DataBlade" finds the IBM Informix product.
While PG's behavior doesn't guarantee a lack of data corruption, "an extension crashed, all bets are off, tear everything down" is going to give you a much better fighting chance against data corruption vs the alternative.
An extension written for a single threaded host system might not work in a multi-threaded context. For example if has global or shared state that isn't protected with locks or similar (which is unfortunately fairly common in c code)
Threads does not offer any major performance advantage, performance of processes vs threads is virtually the same. The reason the PostgreSQL project is moving towards threads is to make development easier.
> Threads does not offer any major performance advantage
This is very not true. When it comes to parallel queries, a process model adds a ton of overhead. You can't pass pointers between processes because the address space is different. This adds a ton of overhead in a bunch of different places. For example when doing a parallel hash join, Postgres will have each worker build a local hash table. Then it will take all the tuples out of the local hash table and copy them through shared memory to the leader who will then construct a new hash table. This duplicates a lot of work as you have to hash the tuples multiple times.
A lot of getting to Clickhouse level performance was making better use of parallelism.
Sorry, what? Passing a pointer is a matter of wrapping the value into the CPU register. OTOH passing an offset into a shared memory is a write to main memory so several magnitudes slower.
Ok ... you know PostgreSQL supports hash tables in shared memory, right? PostgreSQL could in theory share those if we wanted to. The issue is just that coding anything which uses shared memory is a lot of work.
Additionally the reasons PostgreSQL does not offer Clickhouse performance has very little to do with parallelism. PostgreSQL plans to move to threading but the efforts around imporving OLAP performance are almost entirely unrelated.
> The issue is just that coding anything which uses shared memory is a lot of work.
Doesn’t that kind of prove the parent’s point though? In theory shared memory can do anything that threads can do. But if in practice some feature doesn’t get implemented in the multi-process design (because shared memory is hard), when it likely would have been implemented in a threaded design, then that’s still an advantage for threads.
There's nothing special about threads vs processes in Linux. mmap works the same, the challenge is to map the same file. You can share a path, pass a file descriptor via fork or unix domain socket, among other techniques.
As long as we're pedantic ... the subject is shared memory. Unless you specify the same, non-null, target address in the call to mmap (and the kernel happens to grant you that mapping on all calling sites), the addresses will be different; the address space is not shared (each mapping might also have different access permissions).
That distinction is important as pointers generally cannot be shared (a problem which can of course be solved with one more indirection ;-) .
MSSQL can handle 32k open connections no need to run a pooler in front of it, can PG do 32k connections and a process for each?
MSSQL shares cached query plans between connections including jitted code, PG cannot do that and the changes needed to make the plans cross process portable would be extensive while sharing between threads is just normal code sharing between threads.
Some, but not that much. Switching PostgreSQL to a threaded model will not magically make spawning connections fast. PostgreSQL connections are quite heavyweight.
The reason to use threads is almost entirely about ease of development, not about performance. If you use shrared memory like PostgreSQL does you need to write your own allocators, etc. So much you get for free if you use threads.
Doesn't postgres (rightly) have a cow if a process has a disorderly shutdown (at least while in a write transaction) because there's shared memory between the processes?
Some see a 30 year old system and think "outdated", I see a 30 year old system and think "time tested."
Clearly a process per connection is more stable and that's what I'm using.
It's unclear what problem such optimizations are solving anyway, with the old way you could only support a million concurrent users with a single server? Are we missing out on supporting ten million concurrent users with 2 servers instead of 10? Ostensibly reducing the minimum db hardware opex for a 10B$ company from 10k$/month to 2k$/month?
A million users? Hell, I'd bet 99.999% of live postgres databases in existence serve less than 5 users on average. Even among products that actually make a profit, I bet 99.9% of them serve less than 100 customers a day. We hooligans on hacker news manage the 0.1% of databases, and in my newfound consulting life, I'm hoping to never support one of those again.
Even if those processes share most of their memory, and are written in a notoriously memory-unsafe language?
A significant performance improvement can well be the difference between being able to run the entire database on one beefy server, and having to shard. And that has a huge cost in terms of complexity and thus reliability and development time.
I'm not informed of the Postgres's internals, but, maybe, that can be solved by grouping threads into different processes depending on which set of extensions they request.
An OS thread per connection can be fine for performance if you don't have to scale your connections, but if you don't need to scale connections why have connections at all? Databases are even more performant when you eliminate connection overhead entirely.
Ouf. I don't know. I don't want to call you out without evidence -- I myself make benchmark claims all the time -- but 50% improvement in OLTP seems suspicious. I get that you used a standard benchmark, and I don't even know what it entails, but my spidey sense is going off. Perhaps, some trade off somewhere that won't make it to prod because it breaks MVCC -- and yes, I saw that it passes regression tests.
Just checking, is fsync on? :) Regression tests don't catch bad IO patterns afaik.
Yeah, claims that it can be faster than CH are very suspicious. CH guys are very good at their craft, they spend hundreds of hours optimizing one single small detail.
Yeah. I don't think PG could even come close. Column-oriented is fundamentally different, and pairs well with all the SIMD acceleration ClickHouse is also doing. There's just no comparison. If a Postgres rewrite came close to that, it must've sacrificed something else.
Remember when databases were faster to run in virtualbox rather than bare metal? (because virtual box was completely ignoring all the instructions to flush the data on the disks)
It'd be very unfortunate if Postgres didn't have regression tests for data loss due to bad io patterns. Should be possible to do some checks against those in an appropriate test harness. Which might mean "have qemu run something we can kill off and examine the results".
If those don't exist, I hope folks recognize how useful they are and add them.
What's your actual background and expertise with Postgres and databases more broadly? Basically, do you actually know what you're doing, or is there likely a massive footgun you don't know or haven't shared with us?
I spent a couple years managing a Postgres cluster with a petabyte of data. I wrote a couple blog posts from my work then[0][1]. I also wrote dozens of posts on the Postgres internals[2]. I've also given talks on how to generate fractals with SQL[3] and how to write a lisp interpreter in SQL[4].
It doesn't sound like you were trying to launch a product, but doing an experiment and someone threw you under the HN-spotlight-bus :) Is this a "see what I can achieve with LLM coding" or is this "build this and see how much of the coding can be accepted from LLMs"?
> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?
In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax
Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.
Yeah, spent a lot of time on parallelism, vectorizing, pipelining, filter push-downs, bloom filters, all the tricks out there. It's really fun to make pretty steady progress on this.
pg_mooncake (now effectively abandoned due to being acquired by Databricks, but still up at https://github.com/Mooncake-Labs/pg_mooncake) pulled the DuckDB engine into Postgres wholesale, if I remember right.
pg_lake also uses DuckDB but keeps it external, routing through Postgres and managing Iceberg tables (but not the data itself) there (https://github.com/Snowflake-Labs/pg_lake).
Both of these were neck and neck with ClickHouse last time I tried them.
Actually δx is faster than the "duckdb embedded in postgres" options: https://benchmark.clickhouse.com/#system=+_etx|_b|_i)|dula|pnc&type=-&machine=-6t|ca2|6ax|g4e|6ale|3al&cluster_size=-&opensource=-&hardware=+c&tuned=+n&metric=combined&queries=-
Plus all the normal Postgres features work as expected: physical/logical replication, crash recovery, pg_dump/pg_restore, etc.
That isn't what the data shows, but we don't need to discuss it further. My reply was to the person interested in learning from other Postgres OLAP designs. This stuff is all pretty immature though and I wouldn't actually build on it outside of some very narrow, well-understood workloads.
What was your methodology and structure in making the prompts for the rewrite? Did you let the LLM roam in all of the codebase and tests from the beginning, or revealed things to it gradually in some way?
Are you fixing the heap and table management ?. Postgres does not use an undo log and manages all table updates directly in table storage which slows MVCC.
That's something I eventually want to fix. The challenge is the storage format is so integral to Postgres that it's going to be a huge PITA to come up with a novel design.
Right now OrioleDB is in beta. Once that becomes production ready, I'll evaluate incorporating it into pgrust.
While a thread-per-connection seems like an improvement, do you have any plans to allow query multiplexing over a single connection? That would be a huge improvement IMO.
Can you elaborate on the use case for query multiplexing? Is it so your client would only need to establish one connection with Postgres and then could run as many queries as it wanted?
Microsoft SQL Server has had a similar feature for a while -- Multiple Active Result Sets aka MARS. I don't have a good read on whether it actually helps any workloads. I've seen adapters that don't support it because of the extra complication.
Multiplexing would have a number of benefits. As you say, each client would only need a single connection regardless of the number of queries being sent. Resulting in:
On the client side, there is usually a local connection pool. When a burst of traffic comes in, the client needs to either wait for the pool to free up or establish a new connection, which adds latency. This latency hit wouldn’t occur with multiplexing.
With multiplexing, systems like pgbouncer would be unnecessary.
Also, even with a thread-per-connection, you can still quickly exhaust the servers resources when you have lots of connections because threads have a lot of overhead. Reducing the number of connections needed would greatly increase the number of clients that a database can serve.
"Is 50% faster than Postgres on transaction workloads" - That is a very big claim! 50% faster on everything? Is it a strict improvement across the board or are there tradeoffs that make some workloads slower?
The 50% is specifically on percona-tpcc[0]. I got there through a mix of batching (postgres processes a row at a time), prefetching, and several handful of other optimizations.
Rust actually made the change pretty simple. The main changes are:
- Use thread local variables
- Move everything from shared memory to process memory
- Use threads instead of processes
I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not something I have to do.
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust.
This naturally lended itself to a suite of skills to describe how to rewrite a crate from unsafe rust to idiomatic rust. The main three skills I had were 1) a skill for identifying the next crates to port 2) a skill for rewriting a crate and 3) a skill for auditing a crate and making sure there weren't any outstanding issues.
My exact approach for managing subagents changed throughout the project. Initially I was doing parallel coding sessions with Conductor. After dynamic workflows came out, I used that as it was really easy to spin up dozens of parallel subagents and manage it from a single orchestrator. Over time I switched from using dynamic workflows to manually spinning up subagents from a central agent. The issue with dynamic workflows is they waterfall. Each step needs to finish before the next one starts. By manually spinning up subagents, I could have claude start porting a new crate as soon as a prior subagent finished.
Actually the inverse. I initially gave claude an outline of what I wanted, had it do some research into how to write idiomatic rust, and then had it draft a series of skills to do the work. I would then try out the skills, audit the results, and then give claude feedback based on what I was seeing. Once I started getting runs where the results were working, I would start to scale things up and audit things with an exponential backoff.
It's not used in production. I've been using different benchmarks to compare the performance vs other systems. Namely sysbench-tpcc[0] and clickbench[1]
How much of the performance gain is from using Rust, compared to using optimizations that are not done in the original PostgreSQL code (like using threads instead of processes, etc.)?
I am simply curious what the benefits of using Rust are in this instance.
I am super curious how you went about the port using LLMs. At $WORK we are looking to port code, preferably with LLMs, and it seems daunting, even with a test suite. Do you have an approach that works well for you?
Just a couple of ideas if you run out of backlog:)
- proper versionnumber (64bit)
- native json streaming. It would be awesome to get to the point where i could somehow redirect the sql output to the browser directly, but piping will do for now. The idea is to be able to stream rows to client without caching and building json along the way.
It's a completely new era of software production (I will no longer call it development)
LLMs give us unlimited manpower, and the language give us constraints to make more modern and safer softwares.
Love to see this rewrite in Rust, and expecting much much more in next few month.
Nothing major yet. Once I wrap up the performance work I'm doing I'll start looking at the best way to go about testing. I suspect there's a lot of novel things you can do with agents.
when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward?
use both synthetic and real data to sample the internals of the original software to duplicate.
locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.
This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).
Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.
I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.
Awesome work. I'd love to see you add something like kusto query language or pql. The autocomple on kusto, (which can be embedded into web apps) is really amazing.
- typically they are behind a single person. That’s usually bad because of spf
- typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
- anyone that wants to contribute to the project needs to pay. Needs to pay tokens because it’s increasingly difficult to maintain these projects without AI
So, who wants to put something like this in production? Doesn’t make much sense
I'd be interested to hear the author's answer to your question, but I see it as an interesting proof-of-concept. It's testing the viability of not only rewriting PostgreSQL in Rust (and their choice of deps) but also in switching the threading model and other architectural changes. LLMs shine at pumping out prototypes insanely fast, and a working prototype can put an end to a lot of speculation.
I likely wouldn't use a rewrite of such a huge project if it doesn't have the backing of the original team (or a significant fraction thereof) and a believable story for having matched/exceeded the original code quality and maintenance. I also think in general using an LLM for license-laundering is legally and morally hard to defend, although this case is different in that they chose a more restrictive license. Not a lawyer, but my understanding is that you can just download PostgreSQL, do s/MIT/AGPL/ and release it, legally. (The original MIT-licensed version still exists, so no reason anyone would prefer yours until you make another release with some compelling new feature.)
The gp feels like less of a targeted individual criticism & more of a general musing about a trend.
Nobody is saying this author hasn't demonstrated discipline & won't maintain this project, but the statistical averages across most projects fitting this trend make it likely enough to question their worth in aggregate.
The author is free to ignore any and all complaints they consider unfounded. It’s not even like the author is recieving any complaints personally; they have to come here to see any. And if they come here, they will get to read the viewpoint visible from here.
Because chances are that it's never going to be production ready, never trustworthy enough to be used by anything serious, and the project will eventually be archived and become at best a (extremely inefficient) learning experience for the author, or at worst a total waste of tokens. I could be completely wrong about this specific project, but most of these projects are like that, and statistics don't lie.
But this is also just silly. The maintainer here did nothing (just a little chat here and there), but is sure that only optimizations are missing for this to be production-ready. How so, based on what exactly?
It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations.
No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contributed by a ton of brilliant programmers, no matter how you look at it.
And if we take this as the truth, then logically there's still an unanswered question - why? Lets take an undeniable fact - rewriting an existing project give you full control over the new implementation. You can do whatever you want with it, you can't be sued. The only thing now you have to hope for is for your implementation to gather a good enough user base and from then on you can practically hijack the original project.
And this is me speculating - rewriting stuff in Rust isn't about the greater good for that magical "memory safety" argument, but at the end it's an attempt to hijack popular software projects.
> it's an attempt to hijack popular software projects
Exactly. And this is why those projects are typically called "XXX in Rust" or "XXX-rs". Because the creators get to do their favorite thing - coding in their loved language - while skipping all the hardships of designing, accepting real feedback, involving users and getting traction - all while simultaneously hijacking the existing brand.
As of now I know of a single project that changed their name after being called and the project surprisingly got some traction.
I think we're a year or two away from the same argument being used by languages which enforce proof of correctness. The economics seem to be shifting in that direction.
Finding exploits is getting exponentially cheaper, and the cost of producing proofs is rapidly going down. For a lot of software correctness is rapidly becoming non-optional.
It's not just a rewrite ; it has improvements. I did the same thing for fun for the same reason; I wanted to see if I can improvements on some of the legacy design stuff and, especially, the stuff PG people have told us that it cannot be done differently. It can. I would not put it in production, but it thought me a lot about the internals of databases. To keep my brain happy in the age of LLMs, I implement database things on our (also old but many times refactored/rewritten) production db without an LLM. I'm sweating through Flexible Paxos now; probably we will just keep using raft as it's old and stable and simple but it's interesting anyway.
I'm not sure I see the value in "showing the team it's possible". I would presume the team are intelligent enough to be well aware that it's possible, given tradeoffs. And as it's clear those tradeoffs have been "traded" in this case, it doesn't seem like having the knowledge confirmed is particularly surprising.
I guess it is cool to have it around but it comes off as a popular useful case for AI which it really isn't, unless you have a very good test suite to throw the agent against, it is practically useless for rewrites of decades old badly documented code riddled with technical debt. That is where the real value and challenge would be. I see it as marketing and social clout stunt
Are there modular databases with clearly differentiated components? I wouldn't be surprised if there aren't, as the trade-off with modularity is usually performance.
1. Nobody, not even the Googles or NSAs of the world do that. No single entity has the expertise nor resources necessary to maintain a fork for every open source project they use -- forking and maintaining Linux alone takes teams of people. And no, going full psychosis mode with LLMs is not going to save you.
Forking and maintaining Linux does not take teams of people. I've been at it for > 10 years and maintained support for my 15 or so various SBCs/mobile devices, writing drivers, debugging, cleaning things up. It's a weekend project every 3 months + whatever you want to put in for development.
And there are others doing it for projects like Armbian, openwrt, etc.
The companies I have worked for before all have used open source software like postgres, mysql, go, python, k8s, etc. 99% of the time we relied on free work; never contributed to these projects nor forked them for our own needs. I don’t think this behaviour is the unusual path tbh
>typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
Lindy effect! The longer something has been around, the longer it probably will be.
Not the same but it is much faster and easier to re-create a 3d model of an existing set of drawings than from scratch. This is because a lot of the decisions have already been made.
I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.
How would one go about reviewing a piece of code like this?
One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you much anyway.
ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
I think the focus for projects like this is going to shift to reviewing the testing/fuzzing process instead of reviewing each commit (going much further than what the postgres regression/isolation/crash tests do).
Some of this post reminds me of a story I heard long ago from someone who had worked at a HW/SW company. They’d transferred an engineer from the ASIC design team to the OS kernel team, though he’d never been on a software team before. After a while the manager called him in for the following conversation:
Manager: You’re doing amazing work — zero bugs in production! I’d like you to mentor the other SWEs on how to get their bug count down too.
Funny story but in my experience hardware engineers produce some of the worst software of the industry. Of course there must be some hardware engineers out there who do hood software but generally what they build are disasters.
Honestly, i do not blame them for that though. The whole eco-system of in C- procedurally bitbang on some registers and read on others, until some circuit that might be there is coerced into doing work - often with faulty prototypes you have to rewire yourself, whos documentation is - non-existant-complete while having deadlines within deadlines. And the project-culture is just "aggregate" a layer, wrap the problem like a pearl in shellacks as the main abstraction.
I bet it's being organized by project rather than product. Conway's law ensures such an org will create code around projects, not products, and that always ends horribly.
Very carefully - and I mean extremely carefully - word it so that it describes what it does, while implying what it should do without confirming that it actually does that.
Also helps if you fix the bug or change the behavior, the docs are still technically correct. I'm only partially kidding, I swear I've seen this a million times in documentation I read.
For large projects like this I think a hierarchical division of labor also helps.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
Though beyond testing, I think there will be increasing focus on proofs of correctness. (Testing can only show the presence of bugs, not the absence. —Dijkstra)
At any rate, it's never been this cheap to produce the proof of correctness of a program, or on the other hand, to produce an exploit for an incorrect program.
> One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project
I could not care less about any of this. Truth is code, as it is now. I don't care when (and certainly not by who) a bug got introduced, it's here, shut up and fix it.
> But with LLMs generating 7101 commits in less than a month that isn't feasible.
I don't think trying to understand LLM generated code is feasible for anything other than very small projects. IMO it's a big problem with using LLMs for coding. Sure, they can generate a bunch of stuff, but in some ways that just makes the real problems of software development even harder.
In general (I’m not saying this is the case with this project) if you don’t have their prompt history and you can’t re-run the LLM “compilation” yourself, is it open source? It feels a bit more like those “source available” projects where you can read the code but don’t have access to the build system.
On the other hand, aside from the commit messages, one didn’t ever have access to the underlying thought process of human developers either, so maybe it’s not equivalent to say that secret prompts mean closed-source.
What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.
The lock format is a multi-line TOML, with a varying number of lines per dep due to redundantly listing deps-of-deps, so a naive line count massively overstates the number.
Cargo.lock contains many unused dependencies, because it's a superset of all combinations of all optional/disabled features of all transitive deps across all possible platforms (so that the deps don't reshuffle even if you enable/disable feature flags or compile on another platform). But that means Cargo.lock is going to have 3 async runtimes even if you use one. It's going to have syscall definitions for RedoxOS and wrappers for WASM, because some dep of dep is compatible with those platforms. But these deps won't even be downloaded if you don't build for these platforms.
The number you got presented is not representing the unit you're insinuating. A crate in Rust is a compilation unit. It's common for projects to ship as a collection of many crates. It's a smaller unit than what C counts as one dependency, and slightly coarser than an .o file. I don't see people freaking out by how many .o files their projects have, including all transitive ones from deps like openssl or curl.
Do you pick your databases based on how quickly they compile and how many dependencies they have? I normally chose based on factors like performance and reputation for reliability
You can also depend on a lot of libraries that have potentially high quality rather then writing a lot yourself. The defense against compromised dependency can't be 'Ill write everything myself and do it with the same quality as the ecosystem'.
True, but internal dependencies aren’t upstream. I care very little how a project is laid out on disk before it’s built unless I’m going to work on the source.
In a typical Rust project you organize your project into many crates. I haven't checked, but I'd guess that the vast majority of those dependencies are internal dependencies. After all, this project started by running an automated C to Rust converter, which the authors claimed produced over a thousand crates.
Disaster. Well, I read stories about Rust and how there isn't much in the stdlib, but this is just too much. How many dependencies are there, on average, in other projects? I guess I am spoiled with Go.
These rewrites are just test-driven development taken to the absolute extreme. Created under the hope that the existing tests are exhaustive and cover every relevant use case, such that if they all pass, the rewrite must be at least as good as the original. So just go with the vibes and burn tokens until they pass, and your job is done.
In practice, this is never true for any codebase above a certain level of complexity, especially not one as mature and widely used as Postgres. But reality doesn't seem to be an obstacle for vibe coders.
The challenge is that more and more people are producing project like this - 1,000s of commits and > 200k lines of code - and saying it was carefully created using agent based workflows and not vibe coded.
Quite amusing we have decades of human written code much of it sub standard and yet no one demanded proof till now of Open Source projects having to ‘demonstrate’ anything.
I get what you’re saying and agree with the last sentence. Just wanted to touch on the “why” part.
In the world of exclusively human written software the existence of the artefact itself (code, documentation) served as the proof that there’s someone with half a brain behind it. Now that’s not the case anymore.
The conclusion stays though - it’s OSS, authors/maintainers have no obligation to anyone to do anything. Like it, use it, don’t like it, don’t use it.
As for me, I’ve found that the community and activity proxies are still good.
It will be interesting to see how the project activity is unfolds? Are people using it in production. How many errors do they find. What do those fixes entail. What happens with the docs over time. Etc.
I haven't had a change to look in depth, but based on a quick glance I'd say that the activity on the project seems like the tempo you'd expect of a similar open source project.
One of the projects Im working on and off is a tamper-proof audit log, based on some PoC code I created almost 10 years go; unit and integration testing are good at preventing defects and regressions, but they will not guarantee your software will work. However, with the power of LLMs, one can easily use model checking (in my case with Quint) and/or other formal proof approaches to ensure the software conforms as specified. The result (in my opinion) is an implementation guided by a single human that is actually more trustworthy than manual human-made software using the traditional approach.
> How would one go about reviewing a piece of code like this?
That's a wrong question. The right question is "why would one go about rewriting a piece of code in X". Once and if you find a good answer to that question, you will see the answer to your's.
2664 "unsafe {", 1835 "unsafe fn". This is completely unsafe. It doesn't look like a rewrite that understands what's actually going on or how the architecture should be redesigned to take advantage of Rust strengths. Instead, it looks like an AI generated transpilation with extensive use of raw pointers.
Note that most of the unsafes are confined to the parser which was generated by running c2rust over the Postgres parser. The Postgres parser is itself generated from yacc/bison, so I decided to port it over mechanically rather than idiomatically.
If there's particular unsafes that you think are egregious, let me know.
Counterpoint: All of the current Postgres codebase is already wrapped in an invisible unsafe{}.
The difference with a Rust codebase like this is that all of the unsafe code has been neatly isolated and clearly marked. The outside code is safe — at least according to the definition of what Rust considers safe, which is a high bar indeed and objectively superior to the unsafe mess that is C — and the unsafe code is naturally fenced in, which means that it can be seen by developers and tackled by incrementally.
In some cases unsafe is unavoidable, but it is possible for a human to verify that it is, in fact, acceptably safe even if inside an unsafe block.
This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1].
I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably more ethical by choosing the AGPL - I just wonder, safer with no change at all?
You seem to have the restrictiveness backwards? The MIT license (uutils coreutils) is less restrictive than the GPL (GNU coreutils), and the AGPL is more restrictive than the PostgreSQL license.
And it doesn't violate the PostgreSQL license to license the rewrite more restrictively. That's part of what makes MIT-style licenses less restrictive than the GPL or AGPL: they allow for more-restrictive relicensing.
I think licensing will have to be re-invented, because now it's trivial to just say: claude, do a "clean-room implementation", so that noone can accuse me of stealing code.
That's what LLM companies have done for years anyway. Inline completions are trained on licensed code. And nobody cares! ;)
If you don’t like the license just let an LLM spend a few days “porting” it and give that port any license you like because that is apparently what we do now.
Making a movie that has basically the same plot as a previous movie with slight changes is a common occurrence.
The thing you have to be wary of with movies is trademark law. Your Star Wars copy can't use the word "Darth Vader", that's trademarked. It can't use Darth Vader's mask, Darth Vader's suit or Darth Vader's breathing either, all trademarked. And with trademark law the bar to pass is basically "would a reasonable but uninformed consumer be at risk of confusing your product for the trademark". LLMs can't launder that for you. You have to make actual changes, like Spaceballs did
Spaceballs is a parody, which is specifically an exception to the rules; it’s called “fair use”. If Spaceballs was not a comedy, it would not be permitted to exist.
The alternative is that you are not easily replaceable, which means that moi2389 is the one who doesn't get to do the work. Which is good for moi2388, perhaps, but what about moi2389? Either way someone is going to be left out in the cold.
The PostgreSQL License is a variant of the BSD license and is therefore compatible with the (A)GPL.
Comprehend it this way: You create a blank (A)GPL project and incorporate the upstream BSD codebase into it. While those original upstream files remain under their original permissive license, the project as a whole is governed by the (A)GPL (plus the attribution requirements of the upstream license, which the GPL permits). From there, you can add your own code under the AGPL and distribute the combined work under the AGPL.
If someone takes your code and uses only your portion, they can use it under the AGPL alone. However, if they also include the upstream source code, then the attribution requirements of the upstream license must still be met.
Yes, BSD licenses are compatible with AGPL meaning BSD licensed code can be combined with AGPL licensed code while complying with both licenses. However, it does not give you permission to relicense the BSD code (or derivative works) as AGPL. The author is free to license any new code they write as AGPL, however the license for the machine translated code is another question. If it is considered a derivative work (which I think it should be) then it must remain under the Postgres license.
If it is not a derivative work, then for copyright to apply at all then it must be an "original work" which has "at least a modicum" of creativity applied by malisper in the translation. If this is satisfied then malisper could choose any license for the translated code they want, compatible with Postgres or not. If it isn't satisfied then no license applies, because it isn't eligible for copyright - essentially it is public domain.
The safe and polite thing to do is to keep the same license when performing machine translation.
IANAL, but calling this "relicensing" is technically inaccurate. It is more precise to describe it as adding constraints. When you combine your work with upstream code, you are layering additional requirements (like copyleft) onto the existing attribution requirements. The original limitations remain in effect. Therefore, it is not a shift from A to B, but rather from A to A ∪ B.
This practice is entirely compatible with the PostgreSQL License, but it is often prohibited by GPL variants. You typically cannot combine GPL code with code under most other copyleft licenses, such as the Eclipse Public License.
Regarding copyright status, AI-assisted work is increasingly recognized as copyrightable in many jurisdictions, provided the process involves a sufficient level of human creative input (though the specific threshold varies by jurisdiction). Only work generated purely by AI, with no human involvement, is arguably public domain. In a case like this, which is akin to "pair programming," the output is almost certainly copyrightable.
IAAL (not legal advice) and I’m not sure the issue is settled.
The BSD license only explicitly permits the author “to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement.”
By default, the owner of a protected work retains all rights not conveyed to someone else. Changing the license isn’t one of the enumerated activities, and so I think there’s a case to be made that it’s not permitted.
Now if the author wants to claim it’s a new work, as opposed to a modification (which opens up a big bag of issues by itself because this was AI-authored), then the author can license it however they see fit.
You are not really changing the license of the original work though. You are not impacting what the author or anyone else can do. You are distributing a copy under different, more restrictive terms.
A bit like me buying a comic book, then offering to sell it to you under the condition that you never let my brother read it and that you make any future owner agree to the same terms. That's perfectly legal, and there is no reason I would need permission from the author (or publisher) to do that
A comic book is a physical object. This analogy doesn’t hold. When you give a comic book to someone, you’re only transferring the copy and its implied license that carries with it. You can set the terms of the physical object, but you can’t change the license of the content within it.
Suppose you get a license to view a copy of a work (streaming or a paid subscription to a newspaper site). Your permission to consume the media begins and ends with the license terms, which allow you to view the work (and, since it’s necessary, to make a transient copy) during the period of the subscription. You don’t get to relicense it to someone else under those terms.
Your redistribution license only applies to the part you own copyright on. Everything else is still under the original license. You can add AGPL but that doesn't take away the BSD license from the parts you didn't create.
Being created by a "mechanical process" from an existing creative work doesn't mean it's not a derivative work.
For instance, if I take a copy of $BIG_BUDGET_MOVIE, and resample the video frames from 1080p to 720p through a purely mechanical transformation, that doesn't make the output public domain.
You might be right, if it is a derivative work and not a new one. There’s some evidence that the authors consider it a new one because they’re attempting to change its license.
The BSD license doesn’t explicitly convey the right to create derivative works but it does convey the right to “modify” the software. (They seem similar but “modify” is a narrower verb.) So is this a modification? A derivative work? A new work entirely? If it shares no code with the work from which it was derived, things are more complex than it may seem (and it no longer fits the “compressed video” analogy very well).
I would agree with you but for the author’s attempt to publish it under a new license. I think they can either claim it’s a new work (in which case it’s public domain) or claim it’s a derivative work (in which case I don’t think they can change the license).
I imagine a court would call it a derivative work if tested.
Things get released as GPL or AGPL that were originally BSD or MIT license all the time. The terms of the copyleft license include all the terms of the attribution licenses. Whether this is a valid thing legally I’m not sure but it doesn’t seem anyone’s challenged it. BSD code is often found in closed-source proprietary products as long as the required attributions are met and the original contributors are understood not to be held to any responsibility or warranty.
The licenses tend to say unmodified or modified copies can be redistributed in source or binary form “provided that the following conditions are met”. Relicensing the code in a way that guarantees those conditions are met has been the accepted thing to do in the community for years - whether that’s GPL/LGPL/AGPL or a proprietary license.
If you can do a Rust rewrite with AI, I can create one as well. What makes yours better than mine? Your decade long expertise in database or Rust language? Your reputation? Your proven track record to manage large, complex projects? Your time committed to the project? I don't see any of that.
I don't know why anyone would choose this over the actively (community) maintained proper Postgres project.
His project is cool. Students could use it as an example of porting code. Companies could switch to it, if it works. There are hundreds of reasons why people may want it, so it's awesome he's publishing one. Something needs to be done to get interest and then adoption.
The project is not cool. This is not a new idea, and there is nothing special.
Students won't use it as an example of porting code. I am not aware "porting code" is part of standard software engineering or computer science curriculum. That's not the kind of thing being taught in schools.
Companies won't switch to it unless their CTOs are either insane or incompetent.
Just keep using the proper Postgres project if it makes more sense to you. But notice that this isn't just a strict translation, it actually makes changes (moving from process to thread based design, for instance). Time will tell if people find these changes beneficial or if the original remains preferable.
It's also worth noting, that while you are able to use LLM's to produce your own translation, he's actually done it. There's value in actually putting in the work.
This isn't a unique situation at all. Many Postgres extensions are developed and maintained by a single person, and may therefore be avoided by more conservative users, even if they offer some technical advantages. To each, their own.
There’s no claim being made that your rewrite cannot be better.
He has provided benchmark results which provide a dimension amongst which to measure your rewrite. If you can do better by all means post your rewrite.
Finally, these kind of projects can eventually over time become projects that are actively used. Postgres is not some entity that existed before the universe was created; it was also created by someone and then eventually adoption picked up over time.
In terms of the software industry, Postgres actually is kind of that entity. It predates the entire commercial DB (and, er software) industry and is one of the first implementations of a viable relational database server, back into the 70s.
It has a 40ish year continuous history. (Which is also why it has technical/design warts like process-per-connection that we probably wouldn't do in 2026, but that's another story.)
Even with that I remember getting funny looks 20-ish years ago when I would advocate for using it instead of MySQL.
Came here to the same thing. I did something similar (but bolder, it doesn't slavishly copy Postgres and is based on current DB research papers and the like, and other bits I've been exposed to over the years). It has a full TPC-C-esque benchmark suite, replication, embedded v8/JavaScript relations/stored procedures, a giant suite of regression tests, it kicks the crap out of a lot of existing OLTP DB stuff out there. And I personally do have a background in commercial DB development.
But I choose not to publish it or promote it for many of the reasons you mention above (and more)
... For one, if "I" can do it, so can a hundred other people. And all the bold claims behind it would need to be backed up and supported and it promoted, etc, which is a whole pile of time that doesn't involve writing code.
It's the organization around a project that matters, not the code. It's not the 90s anymore w/ people piling into MySQL because it was the only option. People aren't going to be trusting your software with their data, if they can't trust you.
And unless someone is going to dump a pile of money or something on [me|them], I don't have the ability to build that organization ... as I need to feed my family... Nor am I willing to put my personal reputation on the line by putting up a huge quickly written application and then someone finding something in it I can't explain.
So like probably 500 other projects I have it sitting in a private repo.
It's a very weird time right now. "Technical" excellence isn't the important part. Organizational excellence is. This was always the case but it's more so now.
... In the meantime, if anybody has angel investment to burn, I have something potentially better/more-exciting than this guy's project but... see above...
Author of this project is not that different than you. The only difference is that he is willing to take the risk. If you have an invention and sit on it, nothing will happen.
For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.
`mostly` is doing a lot lifting here. The Go rewrite uses plenty of copilot. The reason you trust it is because you trust the people doing the rewrite.
AI is a great use for this kind of boring, rote translation where precision is important. Humans are quite bad at it and tend to make mistakes. In either case the focus should be on improving testing, not trying to manually verify if the translation was correct by eye.
I really wonder where all of these people who believe that tests perfectly encapsulate the behaviour of software come from. Maybe it's because LLMs happen to work better when you give them acceptance criteria and people struggle to distinguish between "better" and "good"?
The real test is years in production. Over time your test suite grows when bugs are found and fixed, but not every bugfix necessarily gets a test, and it's very rare that a bugfix is exhaustively tested. Relying on the test suite as a directional indicator that your vibecoded rewrite functions something like the original is probably sensible. But it isn't "done" until you've run it in production for at least as long as the original. And that's where it all falls apart, because maintenance will be a nightmare. Nobody knows how the new thing works.
Not sure it’s so simple. I think close to 100% of new ambitious projects are going to leverage AI at least to some degree. I know a couple that have strict no-AI policies (e.g. Zig), but it’s a tiny minority i think.
So how much AI usage does it make it an “AI rewrite”?
Dunno. I got rather the impression that it's ambitious single-developer projects with no intention of maintenance which leverage those 'AI' code generators the most.
Who wants to contribute to an unmaintainable code base?
Is there any measurable difference in quality between the two, or are you just going on "vibes"? Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Such crude takes only cause unnecessary friction. If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary. Most of the code on the internet is already a black box to you. What percentage of code running on your machines have you vetted by who wrote it and code quality?
AI coding isn't going anywhere and will likely end up generating most code going forward so instead of rejecting it outright or arbitrarily categorizing it we need to focus on solid quantitative and qualitative measures of code and functionality regardless of who wrote it.
I have read up on it again, and while it was entirely dysfunctional at the very early stages, it quickly came up to par or beyond, with the LLM especially helped by the huge test suite written in Typescript, different from both Zig and Rust.
However, Jarred still describes a lot of unsafe, and usage of Miri in continuous integration.
Funnily enough, RAII is cited as a major benefit of rewriting from Zig to Rust, while C++ already has RAII. I wonder if C++ and Rust are more suited to larger programs than Zig, unless the architecture in Zig is handled carefully.
> Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
If the dev doesn't vet the code, it doesn't matter how good quality a dev they would be if they wrote the code - they didn't. Sure, the dev would probably drive the initial architecture discussion better and some people are using AI in small batches with tests and vetting everything, but some previously great devs are throwing in PRs that touch hundreds of files at once with one commit.
A lot of people I previously considered great developers have become people I would not recommend for a job in the past 2-3 years.
> If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary.
Sure, but this is just begging the question. If nobody could tell, the term 'slop' wouldn't have become so popular.
You must be replying to a different comment. Seems completely unrelated to what I wrote. I never claimed that there wasn't AI slop. My point is that there are different levels of code coming out of AI, both due to the quality of the model and harness, and the quality of the engineer that is driving it. Thus you can't just bucket all AI developed code the same.
100% there is slop created by humans and really solid code bases generated by AI driven by a meticulous developer. You are making the exact error I was addressing, which is bucketing all AI code as the same.
I quote-replied to your comment, so I doubt it was unrelated.
> I never claimed that there wasn't AI slop
No, but you implied that a top tier dev doesn't produce slop when using AI.
> If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box
My point was that "if" is doing a lot of heavy lifting here and you're coming very close to begging the question.
> bucketing all AI code as the same.
Most people are not "top tier devs" and over time this will probably become more true. Even if I accepted your premise that "top tier devs" only generate solid code bases with AI, the ease of entry and the ease of spitting out thousands of lines of code means the ratio of bad AI to good AI will not go in a good direction unless it becomes too expensive for non "top tier devs" to use. Given this, I think it's fair to assume AI code is low quality until proven otherwise.
Yes most people are not top tier devs and most code is slop whether written by AI or not. I've probably dug through tens of thousands of code bases in my over 30 year career as a software engineer and most are slop.
I also did not claim that all "top tier devs" would always produce better code with AI, but the qualification for a "top tier dev" in this case would be someone who verifies code multiple ways to make sure it is correct. I've seen amazing code come from bad interns that was reviewed mercilessly by season devs, and there's absolutely no reason it would not be the same with AI generated code.
You do realize that you can review the entire architecture and code line for line even if it's AI generated right? My black box comment did not mean you couldn't see the code, it meant you don't know whether a machine wrote it or not.
You've dug through tens of thousands of code bases? 30 years would give you ~10,950 days, so you'd have to be digging into 2 code bases per day, every single day without any breaks for 30 years straight, to get to "tens of thousands".
When I read things like this it makes it very hard to give any credence to the rest of your pro-AI arguments, because it just seems incredibly likely that you're a bullshitter.
> Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Aren't you making a strawman argument ? AFAIK this project is not made by an official PostgreSQL core developer, so the entire premise of your argument is invalid.
I phrased that improperly which made you and probably others misunderstand. What I meant is, is the quality of AI generated code correlated with the developer? The answer is yes, a bad dev will absolutely produce worse code using AI than a good developer - the point being that there isn't just one level of quality of code coming out of AI, even with the same model and harness.
I agree but I think from Bun we learned that a project with really good tests and enough tokens can be converted from one language to another quite good!
It is more and more the future. No human would want to rewrite one technology to another because it is too marginal a gain. AI on the other hand does not give a shit.
I dont think Opus 4.8 is an average coder, with my own experience (I have coded 20 + years before even llms existed) it is anything but average. I don't think training data alone determines the success of these models, there are lots of reinforncement learning principles and fine tuning takes place, a crappy code in the dataset doesnt hold those llms scoring high in benchmarks, I dont think an average programmer can score 70% (opus 4.8) in SWE Bench Pro, which is a good one.
I would say it's an average coder when it comes to writing functions because it keeps using regex. It might pass a benchmark but doesn't pass the smell test.
Well, it’s up to the user or post-trainer of the LLM what they believe to be above average. Then they can design around that.
In the case of real world LLMs and post-training, what is above average is defined roughly as: labeled good by expert humans, and scoring high on RL environments related to coding like debugging, passing tests, or running efficiently and verifiably correctly.
Porting perfectly working C code that has been in production for decades (and has a great track record wrt to security fixes) to rust is more or less an obfuscation challenge.
Given that you reuse Postgres' tests and LLM have been clearly trained on Postgres' three decades of contributions, this may constitute a license violation. Unless you include the original license of course. From a human level I also understand how you ended up with a less permissive license.
I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.
> not mostly the test, but the real world production scars
Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some guarantee of equivalent diligence.)
Sure, but behaviors that never have a bug or regression don't get a test. Software of this kind of complexity has all kinds of behavior that has never been broken, and doesn't have a specific test written for it.
Getting an extensive test suite passing is certainly orders of magnitude better than having no test suite at all, but it still doesn't tell you as much as you need to know. I would absolutely never trust an LLM Postgres rewrite (in any language) in production based on "only" Postgres's test suite passing.
I've also seen situations where a customer reports a bug, the fix breaks some regression, and the updated behavior to work around the fix breaking the regressions turns into an undocumented feature.
How do you break a regression? A regression is breakage. Are you one of the people who use "regression" to mean "regression test"? Did Codex learn this from you? I hate it.
The same basically holds for proofs in the absence of coherent global correctness criteria like, say, confluence and normalization for a lambda calculus, or soundness and completeness for a logic.
Fable's napkin estimate of the effort required to produce a passable reference semantics for Postgres, which would involve novel discoveries in denotational semantics of concurrent transactions and so on, might be in the ballpark of 30–60 years of PhD level work.
So realistically I think the only way to validate a Postgres implementation involves differential testing, fuzzing, acceptance test suites, etc. And still you'll have bugs that need to be hammered out the good old fashioned way.
Or even a human rewrite merely because some language is the current fad. A rewrite in a different language should be done for very good reasons, to solve problems that are bigger than the costs of all the bugs that will be introduced.
Perhaps before embarking on one of these rewrites the first step should be a heavy round of mutation testing and property based testing. Contribute any new testing code from this back to the original project. And *then* embark on the rewrite.
If that's your concern, then your argument becomes "software should never change". Why dare patch any bug ever? It might be load-bearing in some unknown, undocumented, unsupported workflow somewhere in the world. No test imaginable can catch that apart from the scream test.
There are reasonable arguments against language ports, but this is not one. You're making an argument against code changing at all ever.
The maintainers that wrote those tests will have experience you won't get out of a rewrite.
I think this is also where the real work is. A rewrite is one thing, that you can show off with a flashy blogpost. The maintenance, for years to come, won't be of that nature yet it still requires as much work.
This feels like the image of the plane that returns from battle with bullet holes, and the engineer being asked to path up where the holes to make it stronger. Only to be told to patch where there weren't holes as those planes didn't make it home.
While not an exact fit of an analogy, those tests patch what was a problem with Postgres in the wild. What it doesn't cover are the things that worked in Postgres without tests, but may fail in port and go undetected.
I don't necessarily disagree, but two other points to consider:
1. Every test that is written is another use case that wasn't tested before. 100% test coverage is often impractical, but the more tests you have the more of the code you can be confident about.
2. Every test you add is another regression that can't happen in the future; if you test the index rebuilding code and validate the output then you know that you aren't going to make a change that breaks the index rebuilding code. If you have a legitimate change you update the tests, but if you're not expecting the change then you know there's a bug somewhere.
Anything that doesn't have tests is unspecified behaviour. While it is true that a port may differ in behaviour where the behaviour is unspecified, "fail" is not the right framing as there is no definition of what it should do.
> Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
If you can be 100% guaranteed that there indeed is a test for every occurred bug. Sometimes maintainers are not so strict about it.
And some programmers are so good that some issues are self-explanatory and they write good code to note a thing but don't write a test, because implementing the test is more expensive.
> And some programmers are so good that some issues are self-explanatory and they write good code to note a thing but don't write a test, because implementing the test is more expensive.
You don't write a test (just) to verify that your change fixed the issue, but to ensure it doesn't regress in the future after an unrelated refactor.
very naive. the runtime behavior of a rewrite should be significantly different in all kinds of unpredictable ways nobody see coming or might expect. It is a combination of language semantics, compiler behavior, operating system behavior, file system behavior, driver behavior, ..
yea, I found LLMs poor at reasoning through system-wide behaviors. Good luck to the vibers when they try to rootcause a rare data-corruption problem. They will hand it off to fable which will attempt trial-and-error bug fixes which will eventually muck up the code. What happens then ?
So you get other bugs when rewriting in another language without existing tests, got it. This is why I hate all the announcements of "it is rewritten in rust so it is obviously better than the original since it passes all the tests". Edit: and it's an LLM rewrite. Add that to the pile of over hyped messaging.
Unfortunately, too many people are getting captured by marketing and are divorcing themselves from reality. A rewrite can be an improvement, even if in the same or any other language.
But, there are also levels, in terms of quality and human code review, when dealing with rewrites. New bugs can be introduced or there can be style issues, that can take time to fully reveal themselves, and particularly if the person or people involved are not familiar with the other language.
So many comments here talking about the downsides. The only reason to do a rewrite is because there are massive upsides. Maybe the implicit point is that the upside (memory safety must be the biggest), isn't worth the downside (lots of bugs to be figured out before you trust it).
I find a lot of HN discussions quickly turn into thought experiments and philosophical debates that largely forget the original topic. For the most part, I find this idiosyncrasy charming and entertaining, but it does frequently result in forests being missed for the trees.
I agree. I also agree with the sibling reply that -
> every time you have a bug or a regression, you write a test that confirms correct behaviour.
What I fail to see in these rewrites however is - what about new bugs introduced by virtue of this rewrite? I mean it'll have to go through its own challenges in real-world scenarios, right?
> I start to see a lot of these re-writes that depend on tests to state that its working.
There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now
Not sure if there's tooling for that though. If there is, just give it to Claude so they will incorporate it in their development loop
Indeed. This approach is an improvement / augmentation over tests, not a panacea.
Neither postgres nor pgrust have their behavior specified using formal methods. (pgrust could write some contracts using something like kani or creusot, but having upstream postgres also write contracts is a tougher sell). If they had, one could write a giant proof that said the two software essentially do the same thing (at least in a subset of environments and some simplifying assumptions)
I can recommend proptest. What you're describing is a common pattern in property-based testing which basically boils down to "comparing against an oracle". In this case, postgres would be the oracle, pgrust is the system under test, and the idea is to generate strategies comprised of sequences of valid (and invalid) SQL statements and ensure the system under test behaves the same as the oracle in every case.
Not weighing in on this specific rewrite but tests are how you specify that your software works correctly. If a behavior isn’t covered by an automated test in some form you can’t assert that any given change doesn’t break it.
I think it is completely reasonable to use a preexisting unmodified test suite to state that something is working. The larger the project the more true this becomes. Real world production scars are documented and guarded against in the test suite otherwise those lessons get lost.
Also SQLite is legendary for its massive test suite and extensive fuzzing. They have 590x the amount of test code and scripts than normal code. Source: https://sqlite.org/testing.html
So, we should make it easier to feed that reliability back upstream.
Probably the most useful thing you can do with these LLM-transpilations for now: If the transpiled version passes all original tests, I can run my application test suite against it and use it to discover test coverage deficiencies in the original!
If it crashes or otherwise observably misbehaves, I know the real project was missing regression tests for something. We could make upstream so much more resilient against accidentally breaking stuff in future updates, if only it becomes safe (offline + no side effects) and easy (if it crashes/locks, it is not from some memory safety bug from 25k transactions earlier) to run these transpiled projects as one row in our everyday integration matrix.
As sibling mentioned - bugs and regressions are the thing that are (in a perfect world) usually covered.
The problem however is non-covered success cases. A visualisation of the problem: let's say universe of interaction for DB consists of 10.000 SQL queries. Over 10 years various regressions were found and 2.000 SQL queries are guarded by tests. In reference implementation remaining 8.000 never surfaced over this time and it's unclear if they will work.
And, thinking of how many various SQL queries PostgreSQL users around the world are using vs the test cases covered it's obvious that feature space isn't covered in 1% of the success ratio cases.
Now the new, test-based implementation, has to prove it can handle remaining 99%.
And also the amount of people running it in thousands of scenarios. Not sure if these areas can be even tested for, but I guess time will tell (can observe Bun if it breaks somewhere as that’s afaik the first big AI rewrite which got into prod for masses).
A lot of the signal (github, forums, mailing lists, discord, etc.) can be turned into signal. Right now it's easy enough to collect. In future it will be easy enough to cluster and generate preferences, experience, etc.
Every bug report, code change as a result, PR / commit message, PR comment that steers preferences, etc. is solid signal to generate future tests.
That's precisely what a regression test suite is for. There is a bug, you fix the bug, you add a regression test. So if the test suite is well maintained these real world production scars are reflected in the tests.
Software like a Database should have an extensive test bench with concurrency tests, all corner cases etc.
I'm not here running the new version on production to tell the maintainer/devs that my 'production unit tests failed'.
What is this even for logic?
I mean there is balance when i write tests for my production software, but my software is used by me. If i would have a library, i would test everything.
And there was some blog post about another database system were they even virtualized the File access to test cases like when the disk controller stops working.
The test suite is the result of these years of years of running in production. Every time you fix a bug, you add a non-regression test to ensure you don’t break it again.
In a project like PostgreSQL, those scars are reflected in unit tests demonstrating that they’re fixed. It’d be hard to pass its test suite and not be as robust as the original.
> It’d be hard to pass its test suite and not be as robust as the original.
This is not true, even in principle, even for Postgres itself. You'd be right to say that it'd be hard to pass the test suite and not be robust at all to some extent. But even in Postgres, I bet that you can quite easily introduce a change that will pass the whole test suite but reduce robustness compared to the latest release (for a somewhat silly example, add a call to `exit()` on a timer that's longer than the longest duration test in the suite - that will significantly reduce robustness while still passing the entire test suite).
Sure but these scars/tests are from the original implementation. Just because it doesn't have issues there doesn't mean it didn't bring its own set of issues
This is all well and good in theory, but the number of times I've seen tests that don't actually test what they say they're testing is hard to count. Yes even when you encourage the developers to ensure the test fails first and do TDD. Tests help you ship with confidence but there's usually at least a few that are just passing by pure luck.
So no, I wouldn't judge a rewrite as being equal just because it passes the tests. That said, I don't think that means you shouldn't do it. You just have to be pragmatic about it.
I dunno...I can envision something vibecoded prioritizing passing test suites producing something that does that, but isn't even functional in real-world production. Sort of like in the pre-AI world, where someone claims 'standards compliance' by way of passing compliance test suites, but can't actually interoperate well with other implementations of the standard. YMMV.
Unit tests aren't useful for rewrites, only integration tests are. So there may be missing coverage. Also many things are simply difficult to test (eg performance under very specific conditions)
I think it's rather unfortunate that writing regression tests is seen as a heavy burden. Compared to determining the cause of a bug and the right strategy to fixing it in a maintainable way, writing such a test should be simple, no?
I'm not at ease regarding LLM generated code changes to a project with a (hopefully) long expected life time, but LLM generated regression tests should be less contentious. I wouldn't expect them to be maintained much; rather, if they don't perform as intended against a future build, just have them recreated.
If you examine what you're saying here and slippery slope it a little, examining past effort for misses and correcting them at scale is not worth doing? There are many ways that the second part of your perspective are incorrect (burden / lack of point). I bet that a coding agent could in an automated fasion find at least reasonable additions that you'd say add value and reduce potential for error long term that you'd find valuable. They've probably already done so (I don't know postgres dev at all, so just supposition here. They will 100% do so in the future.
That's not relevant though. All concerns are secondary to security and Rust is the only language with security GUARANTEES. No other language is as secure. Therefore, even the worst Rust rewrite is automatically better than the best work in any other language, because it is the only one with guaranteed security.
If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation, then you are simply not giving security the priority it both demands and deserves, and that makes you disastrously insecure. This is a serious issue that should be given all priority. There is no room for debate. Your only policies should be security before all else and compliance with those policies must be absolute and without deviation, or all is lost.
Also, there is more to security than memory errors. SQL injection, authentication, and access rules matter. It doesn’t matter if Rust database is secure to bad data if it lets anyone in to do anything. Or if it is crashing all the time or corrupting your data.
> If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation
This is silly.
Rust is awesome, and it's hard to argue against in many domains. However, software is more than the language it is written in or the runtime serving it. Is the Rust rewrite fully compatible? Is it supported by a strong community? Is it likely to continue to be supported? Is its release cadence sensible? Is its licence compatible with your intended usage?
There are many questions needing to be answered before making rash decisions based purely on tech.
None of those concerns approach the level of priority that must be assigned to security. On defense, your security must be perfect forever or you are absolutely defeated. None of us are on the red team. It's not a rash decision, it's the only decision that logic allows. If you are not secure, you are NOTHING.
I strongly disagree. The easiest way to shut down your business is to insist on being 100% secure because the only way to have perfect security is to do nothing.
1. Piggybacking established brand names (Postgres + Rust)
2. … without practicality nor advancement (e.g. this solves no extra problems)
3. … without trust (i.e. LLM-driven rewrite, with no capabilities to thoroughly review it)
I think people get easily upset when the title has high-signal names like Postgres, and the title touts it somehow, yet it’s obviously impractical for obvious reasons (short-/long-term practicality, social trust & network effect, etc)
But that's the thing, without the decades of work, it wouldn't BE trivial.
Everyone is standing on the shoulders of those which came before. If LLMs allow us to combine the incredible decades of effort and knowledge and experiences that's gone into building something as great as Postgres, and take that and combine the experience and philosophy that has led to the creation of a language that potentially provides tangible benefits, and for far less human time and effort that it would have otherwise taken...surely something that should be celebrated as absolutely incredible?
There are new power tools for our craft. People are experimenting and having fun with said power tools, and have interesting results that may be transferrable to $YOUR_PROJECT.
Doing things just because we can is a great reason for hacking around.
Kudos for the author for answering questions and keeping up resilience - HN crowd is not what it used to be (shakes fist at a different cloud).
I'd love to be proven wrong, but chances are that nobody will use this in production, people will completely forget about the project in 6 months, and the project will be archived not long after that.
I mean you can learn a lot. You can learn what's possible with less effort to build a proof of concept. It's kind of like you had another engineer do it for you, you don't completely learn how to do it yourself but you can still learn a lot with much less effort
You would learn way more by asking an LLM how postgres works. Nobody is even reading this Rust code. There are 7000 commits over 2 weeks. What is being learned from that?
You learn that you can build a version of Postgres that passes the tests and improves on these benchmarks in rust. You gain access to a codebase that does it which you can learn a lot from too. Definitely not zero like someone else said
exactly zero knowledge gained. all former expert understanding is buried down in slop so it is inaccessible and will be completely inaccessible once slop-machine stops
That's how Ai generated code is. I am almost convinced that Models are intentionally taught to write obtuse code because AI companies don't want us to write code at all
More that I got confused by the C function returning bool, not as an error value, but as a result, which is my fault for skimming it quickly.
I have taken a closer look at the code, and it seems superficially a somewhat faithful rewrite, not quite idiomatic Rust, but closer than I anticipated at first. I know there are non-LLM rewriting tools for C to Rust, and with a test suite to help, a rewrite to Rust might be greatly helped. The new Rust code does have some drawbacks in some ways, and there are topics I am curious about.
People feel threatened by LLMs doing things well that they feel should require their skills and talent.
That's understandable but it's still a bit of a negative emotion that probably isn't very productive. Or very rational. This thread is full of people trying to argue that this can't be any good, shouldn't be any good, and is clearly going to end in tears. And obviously this thing passing tens of thousands of carefully curated tests that accumulated over decades suggests otherwise. It's hard to argue against that.
This probably is going to have some new issues. But it's an impressive achievement.
I don't really understand how "written by AI" and "for learning purposes" can ever be compatible. What exactly does one learn from typing "Rewrite this in Rust, make no mistakes" into a terminal?
Regression tests start to play a different role with LLMs.
On one hand, they give an LLM a short feedback loop to correct itself, and iterate fast when writing code. A human also uses it as a feedback loop, but we don't iterate as fast and don't handle big walls of conditions, so its effect is not as big.
On the other hand, LLM's ability to handle a big wall of if-conditions can backfire if it starts taking shortcuts and taking the tests-as-a-spec too literally, overfitting the solution, overly focusing on the given datapoints (conditions checked by tests) and missing the overall behavior shape that the tests intend to pin down. For humans, this is less of a concern because we are bad at big walls of if-conditions, and we'd rather try to see the original shape that the tests are pinning down than monkey-patch the solution to fit the individual points.
It's interesting to see how one balanced these two. In this case particularly. Maybe you could play around with separating the data you give an LLM into "training set" and "validation set", training set can be seen fully, but validation set is hidden and is only queried when the solution is deemed ready. Say, training set = original source code + half of the tests; LLM uses that for quick feedback loop. And validation set = the remaining half of the tests; test code is not shown to the LLM and run only when the LLM says it's done to catch potential overfitting of the resulting solution over training set.
To me, the credibility of a solution like that would depend on what methodology the authors used. If they just let the LLM see all tests, I'd be skeptical (albeit unable to point out specific bugs due to the volume of work and LLM's ability to make bad things look trustworthy). The good thing is, real-life use will add new, unseen before datapoints for testing — so validation set will build up with time. Really curious to see how it will work.
Rust and its ecosystem needs to become more original. There are so many new problems that needs software solutions. Existing solutions that already work don't have to be rewritten in Rust.
A lot of it is actually GPL-washing and rust is the excuse.
I'm on the rewrite it in rust bandwagon, but I secretly want to rewrite things in rust so they can be refactored and made easier to maintain and add features. So "rewrite it in rust" is just like "rewriting it in anything that I'm currently enamored with," and doing it with an LLM (defactoring?) would miss the point for me.
rust being safe(r) just makes the rewrite less risky.
A lot of the software being rewritten in Rust is not GPL to begin with, like PostgreSQL here which has its own BSD/MIT-like license: https://www.postgresql.org/about/licence/
(I'm working with malisper on this), we are now focusing on improving many things about postgres! Some we have written about before [0], and we have much more in mind too. Malis wrote another comment about analytical workloads being 300x faster now than postgres for a version we're working on right now
Aiming for postgres compatible database with a 2026 architecture
> Aiming for postgres compatible database with a 2026 architecture
Except you didn't improve the architecture, did you? You just asked an LLM to copy what was already there. Making real improvements to the database architecture requires understanding the database architecture, not just asking a calculator to do the work for you.
Better benchmark performance means nothing if the underlying guarantees break, and a 300x improvement sure makes me suspicious. I would look at something like this if it passes a Jepsen test, otherwise you simply will not be able to convince me that it's worth my time.
The version we have live right now is pre architecture changes, we wanted to make sure we could hit this milestone first. And agree about proving the underlying guarantees. It will be pretty exciting when we do
You are now at 0.1%... now submit upstream in sensible chunks (function or maybe file/module), waiting for people to review (a few per week, maybe) and approve/merge.
There is big appetite for PostgreSQL in business cases. But there's also a lot of problems in PG and people want to solve them. But there's ~10 people in the core of PostgreSQL who contribute to it and know how to change the core. If you have a business use case that would require changing the core, doing it in safer, less error prone technology would be way better. That's why you have other products that had to be created that talk PG protocol, but aren't using official build.
Because Rust is what's cool these days. Don't you wanna be cool? Also Rust has memory safety things that C++ doesn't have, so there's a class of bugs that can't happen in the Rust version. That doesn't mean the Rust version is 100% bug free, but just that it's not vulnerable to that class of bugs. So it's a good thing for security reasons if you're running a database server somewhere that attackers could get at it. There might be performance benefits down the road if they choose to focus on that.
Congrats for launching this project. I think it's awesome. I did two similar projects, just to learn how to work with LLMs and it felt good--reminded me times debugging code with GDB and going through stuff in semi-manual mode.
I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.
SQLite already does this but hasn’t stopped the rust /go clones from popping up .
These are toy projects with no serious interest in maintaining the port long term. and even with things like bun where the port is merged it remains to be seen on maintainability over time.
Why should a developer use this for anything beyond a pet project? Just because it is written in Rust?
All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
As someone who loves Rust the language and tool set: This class of projects [LLM rewrite of a reliable piece of software honed over decades) is embarrassing.
If you are watching this and haven't used rust: Please don't judge the language by this part of its users.
Well, this approach is more similar to imposing a dogma thank engineering.
Is managing memory safely important? YES
Is managing memory safely the solution to most of the problems? Absolutely not.
Advocating the language ignoring everything else (having as first and only argument that the code was rewritten in rust fully qualify for this case) is dogma and not engineering.
I actually had a lot of problems with software cult followers of influencer gurus like ThePrimeagen, Lex Fridman, Theo, etc... Those are so worst. You can't resonate with them.
I have some familiarity with the bank situation, and while a lot of them are on some very old systems (maybe COBOL, maybe something else, either way they want off it) the cost of actually re-writing the code is far from the most significant issue.
Consider: You have a big mainframe running your tier 1 bank. Assume that you can see all the code on it, and you can feed all that to an LLM if you like. Getting it to spit out a Rust version is not what you actually want - you now have a modern language but it's still a singleton instance, so where do you run it? Most hardware doesn't give you enough uptime for what you need here, because what you actually needed was a re-architecture for distribution / failover / whatever, and while you could ask your LLM to do that you aren't going to run your bank on the result.
> while you could ask your LLM to do that you aren't going to run your bank on the result.
Why not?
I feel like we're entering a new era of prejudice against not a category of humans, but against non-human intelligences.
The design patterns for distributed and fault-tolerant systems are well-known and established in the industry. Both humans and AIs are familiar with them!
So if you sketch a design for the AI to follow, establish the rules in AGENTS.md, have a robust test suite, use a frontier model dialed up to eleven, etc... why not rely on the LLM output?
At the end of the day, humans are not without fault either.
I've been wading through some legacy "pre-AI" code recently and it has more bugs than a rainforest! Static fields used incorrectly, causing data races. Floating point types used for money amounts. JavaScript and SQL injection up the wazoo. Wildly unsafe password handling. So on, and so forth. This is the norm for most human-written software, not the exception.
As a proof-of-concept, I tried an AI rewrite of one such legacy app[1], and it is not bug free, but it notably has fewer bugs than the original. Different bugs, sure, and I'll have to iron them out after a round or two of UAT, but I'm honestly more confident with what I got from the chatbot than the code inherited from humans.
[1] Deals with money, but admittedly at a much lower level of risk and consequence than a banking app running on a mainframe.
Because you know that the current one works. If you have a bank running on COBOL (or whatever), you've had that for 30+ years now, so while it might have bugs, you know what they are. You don't know what the LLM output is. Hence back to my original point: writing the code is not the hard bit. Making yourself (and your CEO etc) comfortable to put that into production is one of the hard bits.
> but it's still a singleton instance, so where do you run it? Most hardware doesn't give you enough uptime for what you need here, because what you actually needed was a re-architecture for distribution / failover / whatever, and while you could ask your LLM to do that you aren't going to run your bank on the result.
If only we had a way to solve these issues with tools capable of running Rust programs in that way. I guess every company that needs distribution / failover has a mainframe sitting in their office nowadays huh?
You could run one of these things on a mainframe, because it's a zero-downtime machine - you can swap out parts of them as they run. But fundamentally it's a singleton. It is deeply naive to believe you can trivially translate that to something running on Kubernetes just "because Rust".
Of course most companies that need distribution do manage to do that, and eventually the banks will get there too. But it isn't feasible to do that by translating their existing non-distributed COBOL code, they need a fundamental re-architecture, and that is much harder.
It's not enough to do a rewrite. Someone has to maintain it. Such a huge codebase with literally zero experts is unmaintainable. There is no one who knows how the internals work.
Sure you could keep vibe coding it but I wouldn't bet my data on that. A database needs to be rock solid.
This seems to be the issue with using LLMs for any code generation. Even with my own code bases that I've written entirely by hand over years, if I use AI to implement anything, I don't go through the mental model of architecting it, so I don't know how it works. I can only imagine this to be far, far worse for large code bases maintained by a team of people who are all using AI.
That depends on the language you are using. Some language communities had already rejected "architecture astronauts" before LLMs were born, so the training data was highly consistent, which has lead to LLMs being highly consistent in their output. You know how it works because the LLM spits out the same as what you would have written yourself. It's almost eerie that they can do that.
Unfortunately that doesn't apply to all languages. LLMs are especially bad at producing code for the languages that were historically known as beginner-friendly as the training data was full of code by beginners doing what beginners do. All bets are off if you get stuck there. (Although maybe you could use an LLM to translate your code to a language that LLMs are good at!)
One problem there is that even if the code is consistent, idiomatic Rust is not the same as idiomatic C.
So either the translation produced Rust code in the "shape" of C code or the code is quite different from the C implementation.
Add to that that you need to have people proficient in Rust and also Postgres as well as the very much unknown codebase as a whole you get a recipe for pain.
> Cue some story here on a bank or airline somewhere still relying on cobol backend servers.
There's existing money and expertise in those environments to rewrite the whole thing, yet they don't. You may loan them free engineers/experts and they might still not rewrite anything.
The existing system works. Yes, it costs a lot to maintain, and you could definitely reduce that if you moved to a more modern system. So now you're talking payback periods. Cost of development / maintenance cost savings per year = number of years before you pay back the project.
Problem is, that the cost of the development is often unclear, and the maintenance cost savings, while definitely above zero, and often unclear, and approximated the numbers usually come to a payback period in decades.
And that's without the usual tech caveats; We can't promise there won't be bugs. We can't promise deadlines will be met. We can't promise the project will succeed at all. We can't promise existing functionality will be faithfully reproduced in the new system. The normal risks around any software dev project.
All in all, it looks really expensive and really risky compared to just doing nothing and running the same old system for another five years.
Source: I helped do some of the maths on this for a Y2K project.
> the biggest blocker on moving to a new programming language, is the cost of re-writing everything
In 2026, not sure if it was satire. Do some people truly believe that all their software stack has to be single tech, from device drivers to end user apps? Does that extend to remotely accessed services?
At the same time that was ever the only reason for moving to a new programming language: abandoning all the bad ideas and craft that had accumulated in the previous language ecosystem. Needing to rewrite everything meant starting from a clean slate, allowing the new systems to be designed for the new age, making everything in that new language feel sleek and modern and thus appealing. Of course, as time progresses even the new language starts to accumulate bad ideas and cruft, historically necessitating yet another language to offer the clean slate again.
If the code is going to be translated forward instead of abandoned and then rewritten, as is now completely viable via LLM, there is no reason to move to a new language at all.
> significant part of the rust community consists of software talibans
I seriously don't get it though. Rust is a nice language, but so is X. However we don't see X people brigading existing projects with constant bombardment with "rewritten in X". What is that about Rust that prompts this behavior?
what do you mean by that? were there people brigarding postgres to rewrite to rust? otherwise relative to popularity i do also constantly see posts on here about Project X rewritten in Go, Zig, C etc...
Rust attracts zealots because of the various kinds of safety guarantees. The speed means it can replace more or less anything.
People see the safety as a moral superiority so it attracts obnoxious zealots.
Other languages' features and syntax aren't nearly so easy for zealots to form behind. The perception of absolute safety it puts in some people makes them crazy.
People were told for years they can't use Rust for their new projects because it hasn't been "proven" in industry yet. So the option was to sit back and wait (chicken and egg) or move to rewrite a bunch of projects so that it could actually be "proven". Not saying this is the only reason why it happens (every language has its Zealots) but it certainly makes more sense.
Due to the explosion of new programming languages over the past few decades your options are to either aggressively expand wherever possible or die out because you're not "proven".
> All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
Nailed it ! There are some folks who behave holier-than-thou just because they happen to use some language. Language missionaries if you will, and they are insufferable.
I think this shouldn't be taken too seriously, from what I understand it's an exploration of what's possible with today's LLMs.
You're right to talk about the trend though, because what it shows is how the cost of re-writing well covered project has completely crashed, so that in itself is a learning.
I have no issues recognizing that I had memory-related problems in production (I program embedded systems in C).
But most of my issues were related to concurrency and data sanification, especially when the other end of communication fails with unexpected behavior. These bugs are nastier than memory.
So, I have pointers, and I am not afraid to use them.
> Why should a developer use this for anything beyond a pet project?
If it _is_ 50% faster, then that's the reason
Obviously like any new database it's very risky to use so probably only used for niche use cases at first, but if it turns out to be just as reliable as postgres and faster then why not?
These days there's little chance for a new DB to build its community using network effect. If you want this to catch on, switch to manually grinding community building ASAP gold plating the experience for a specific niche (AI can guide your priorities but will hinder your comms). Otherwise, have fun building!
I mean if it's actually just another Postgres, you don't have to worry about network effects as much because anyone could use it in place of Postgres. But sure, the more people you know using this, the safer it would feel to use it
Code is code. It either does the job or doesn't. If there is nothing that could convince you that AI generated code is trustworthy (for whatever definition of trust) then this is an article of faith, not a rational position.
> Code is code. It either does the job or doesn't.
Surely that is not the only dimension that matters when evaluating software. Maintainability and readability, for example, are crucial for any long-lived project.
In the future, code readability will not really matter, the concept of maintainability will also change: Is it maintainable by AI, if yes, then it's maintainable, we will approach the day where we have virtually ZERO code written by human, so all the tools must be built for AIs, not humans anymore.
Perhaps in the future my grandmother will have wheels… But today radiologists still do radiology (despite AI being better at reading scans), taxi drivers still have work, and truck drivers are still able to put food on the table. It’s still 2026, time traveler, so touch grass…
“Does the job” means all of those things. Figure out what’s important, find a way to measure it (could be also be qualitative.) How does the AI generated code do? If you simply say its no good because of the way it was created, it’s not a rational decision process.
That's a straw man. My argument isn't about 'the way it was created'; it's about the point I clarified in my previous comment. If you want to discuss that, great, but calling my take irrational based on an argument I didn't make doesn't make sense (when your position is “YOLO to AI”). To reiterate — Trust is the core of my argument, and there’s no trust without human ownership and proof of work/understanding yet (possibly ever).
Devils advocate: Are they? Why? If LLM's are capable of taking absolute crap and iterating on it to achieve a purpose, then does readability really matter?
Disagree, it’s already a leap of faith to trust even human code (which is why we try to prove code, or at the very minimum create guardrails with tests). Human reasoning itself is under doubt. It’s even more of a leap of faith (>>) to trust code generated by AI reasoning (which I can only currently call “homunculus reasoning,” it’s not inferior but it’s just got the probabilistic aspect of reasoning). People who YOLO their work to Al are practicing a different form of faith. I like knowing what my code does, and making correct predictions about it. A mental model is very important. My position is very rational, yours is faith-based. This is a technology that has many cool uses, but this is not one of those uses (in 2026). Unless the author can convince me they have as good of a mental model of this rewrite as the makers of the original source code. Maybe they do, I’m sure they would run circles around me, but I need the confidence that this isn’t going to wake me up at 3am because of some bug. And the only way I know that about PG is because I trust the creators.
Is the 300x performance boost attributable to the threading model vs process model?
Was the code for the threading model written by hand or was it translated from the WIP threading model the human PG team is busy with as part of the 2028 roadmap?
I'd like to know on what machine and on what testbench the supposed 300x boost is achieved so that we can independently verify it. My assembler-written fork of postgres is achieving 600x boost on SELECT 1;
Setting aside the “why” that everyone is so focused on, I want to know, how? I can’t get Claude to do anything even a fraction of this complexity. How are people setting up their agents, Claude.md, etc to do such big projects? There are a lot of lessons to be learned.
> I can’t get Claude to do anything even a fraction of this complexity.
I remember writing a postgresql compatible DB with Opus 4.5, that used S3 as storage and local caching to make it speedy.
Ironically, Opus 4.5 is by todays standards is antique. If you have some knowledge about Databases, it goes a long way.
But you need to do it step by step. Getting the core to work, getting a Pratt parsing going. The whole pgwire protocol ... the data format ...
Step by step ...
With todays Models, your can probably get away with using /goal and telling it to make a postgresql compatible database, while having it run a few days.
Now, making a fun database test project and having it production ready! Big difference!
Let's hope it did not special case (overfit) for the specific tests. One of the failiure modes that in my experience takes the most effort to mitigate for.
I'm starting to get a bit of fatigue for these projects that boil down to just "I asked Claude to re-write this code into a new language that's in vogue right now!"
I really don't understand why this is needed outside of an opportunity to show how impressive LLMs can be when working within large codebases, but even then people in the comments are finding bizarre implementation choices that a human developer wouldn't make. I'll stick with Postgres and its - gasp - C implementation for now, thanks.
In this case it's justified because Rust allows safe implementation of threaded code. Current Postgres is per-process. Switching to threading yields performance improvements.
If you’re going to make a confident blanket claim, be ready to back it up - and asking for clarification is not trolling, by the way. You should be ready to engage in technical conversations if you want to make technical claims.
I'm not interested in a technical conversation with you; I was answering someone else. The top voted comment thread of this page, which dominates discussion, already covers everything you'd want to know.
There is an expectation that threads are more performant than processes all else being equal. Obviously there is some work involved to make "all else be equal", like maybe threads have less isolation from each other etc but if you can mechanically convert code to use threads where it used to use processes I can see the claim standing up.
Quite a lot of projects are trying this "rewrite to a new language using LLM", both internally, or externally (like is here). For me, they confirm some (slightly controversial) takes.
1. human code reviews are dead. We don't yet know what's next. Two reasons they are dead: too much code to review, and code reviewing sucks (who wants to spend their days reviewing code?)
2. Not knowing how to review LLM code is a big barrier to adoption, but bigger regression test suites (testability/evals) is almost certainly the direction.
3. There are a lot of projects that haven't moved to more modern infra because it was too hard. Now it's much easier. Sure stuff will go wrong. Sure it all has to be tested. What's new here?
4. Programming languages for LLMs are coming.
5. Projects that don't allow AI coding will be forced to come around or fade.
Separately, bit off topic:
New projects will often have LLMs built in, so non-determinism will be inherent in the project. No amount of code review will be able to eliminate that.
I just want to quibble that the 100% Postgres regression tests do not test the threaded aspect of this project, and that is a pretty fundamental architecture change.
I think we will actually see some successful projects coming out of this. There are definitely people who want x old project in this new/better programming language and who are willing to put effort into maintaining it not just doing one off port.
I started my programming career by porting code from one model of TI calculator to another. It was code I could not have written from scratch myself at the time. I learned a lot about the two different versions of TI Basic that the calculators used, but I didn't learn how the program really worked. I can't even remember now if it was Tetris or the tank game that I ported. Maybe it was both? That was a boring English class...
I totally understand why porting code is fun. It's kind of like when I checked out drawing books from the library as a kid and just traced the pictures because my own attempts at drawing were so bad. It gives you a feeling of accomplishment, even though you didn't actually do anything that difficult. And you do learn some things along the way.
Doing the same with an LLM probably gives you that similar feeling of accomplishment, even though you didn't actually do that much (sorry, hate to say it that way). I wonder if you learn even less in the process. Maybe you just learn different things.
Now that I think about it, even writing some code from scratch with an LLM is not much different than doing a porting project. Someone else did the hard work of creating the original programs that the LLM was trained on, and now you (the LLM really) are just porting/restating what someone else did. I hadn't thought of that before
How is the performance compared to regular PostgreSQL?
I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?
I also suspect this will die very shortly, which is a real shame, not because it will be beneficial but because of the time and tokens needlessly spent on something that will be thrown out.
This seems to be a multi-phased project. First phase (completed) was the re-write in Rust. There doesn't seem to be a performance gain and no significant one should be expected. In a 2nd phase a new architecture is implemented which malisper claims to perform much better.
I wished the two phases would have been tackled in reverse order.
> I wished the two phases would have been tackled in reverse order.
Well, tackling them in reverse order would require the humans behind this to develop an actual understanding of the existing code and architecture before starting the project, instead of just asking claude to do it. So, here we are.
The version in the GitHub repo is ~8x slower than Postgres. I have a new unpublished version that is 50% faster than Postgres on transactional workloads and ~300x faster on analytical workloads.
Thank you! I'm a big fan of your writing. I wanted to make sure there was something people could try out so I could show pgrust was real and not vaporware.
I think the cool thing about these projects is that even if test parity reaches 100%, some bugs are going to surface on the new project that don't exist on the original project.
This is usually a good example of a test case that the upstream project is not covering and can be contributed back.
Parity should be bidirectional, so definitely it is possible for both parties to benefit from it.
It’s interesting to see how llms have turned the concept of rewrite it in rust, from an impossibility for some projects (code is too large and complicated, it will take too much time) to a real possibility for even large projects.
> rewrite of already popular technology in a different language
> look at commit history
> "Claude xxx committed yyy ago"
I'm sorry, but what is this need to just vibe code a port of an existing technology to a different language/framework/etc.? If it's just a personal challenge then sure I guess, but this surely can't be used as a real product?
What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...
I would expect Postgres to be heavily tested with things like Valgrind and various sanitizers. I'd be surprised if there were low-hanging fruit. But also, if there is code that does something fishy with pointers, wouldn't the AI likely paper over it by adding an unsafe block in the Rust version, preserving the same fishiness? It's hard to know how hard it would try to prove that the original is broken.
> The goal is to make Postgres easier to change from the inside
uh-huh, sure.
you want to show off "look what the LLM can do / look what I burned a bunch of tokens on"?
you want to brag about how your LLM-generated slop is somehow more maintainable than the original because blah blah blah Rust?
here [0] is the version history of Postgres. pick a version from the past. let's say 14.x because it's the most current that's still under active support.
have your LLM implement version parity with 14.x. show off how it passes all the tests blah blah blah.
then have it upgrade your codebase to parity with 15.x, implementing whatever new features and bugfixes that includes.
and have it generate an automated test that demonstrates upgrading an actual database from LLM-14.x to LLM-15.x and verifying there's no data loss or corruption. maybe even multiple such tests, if you're feeling fancy.
then lather, rinse and repeat with 16, 17, and 18.
and show off the diffs of each version. does the LLM rewrite a huge pile of already-working code in the process of each version upgrade? does it introduce new latent bugs in the process - the kind of things the existing test suite didn't think to explicitly test for?
"I took a static snapshot of code and converted it to another static snapshot of code" is meaningless. all you're doing is bragging about having more money than good sense.
the stability and trustworthiness of software like Postgres does not come from a one-time snapshot showing tests passing. it comes from the engineering process that produces the software and its test suite.
oh, and for shits and giggles, because this same test was so illuminating with the Bun "rewrite" into Rust, here is the file with the most unsafe blocks in the codebase:
It looks as if it's building structs out of information in (mutable pointers) to other structs without an Rc in sight. Which makes sense for a C parser: you've got a table with data, so you just link to it. It's fast, and when you know you're not going to touch it, it's safe. But this doesn't make the Rust code any better than the C code.
Note that the code I believe you are referring to is from the parser which was generated with c2rust. The Postgres parser is generated from yacc/bison so rather than try to rewrite it idiomatically, I did so mechanically.
This is actually a great point I have not seen articulated. A motivation for clear architecture and good design is being able to evolve the system over time. Does LLM code have that capacity? Are LLMs themselves able to increment iteratively?
Rust feels like the just right language to produce all those slops of AI and the language itself. Great to promote yourself as a productive engineer, but at the end of the day you’re just reinforcing the statement that AI and the language itself are great, not you.
Like most modern languages, Rust has its own build system and package manager, Cargo. Everything you're referring to relates to that, and has nothing to do with LLM coding.
Edit: saw the clarification in another comment. But, in that case the essential point seems to be "I'm not familiar with something, therefore it's suspect."
The project itself seems to be structued around 1.4k micro-crates[0] which I admit is a bit weird. Rust's compilation unit is the crate unlike C's per-file compilation unit, so if this was a 1:1 AI-assisted translation from the original Postgres source this might be an artifact of the translation.
This is correct. We first used c2rust to translate the C into unsafe Rust. The generated Rust code had one crate per C compilation unit. We then took the unsafe unidiomatic Rust and one crate at a time converted it to safe Rust.
Did similar with S3 and that too (eventually) did well against tests (the ceph s3 ones).
...but haven't dared use it for anything meaningful yet. Still feels like there is a real world gap in confidence when it comes to vibecoded rewrites.
Been wondering whether the answer is to insert a proxy...something that effectively splits traffic to a known S3 and the rewrite and compares outcomes over time. Do that for a couple different workloads for a month or so and if it's all identical then it's probably fine...
it's a neat experiment, obviously inspired by the bun guy.
but given the author/maintainer is essentially unknown, I highly doubt this will reach it's target audience. But one thing I do agree with is that postgres is long overdue for a re-write into a memory safe language.
also any real swe with more than a few years of experience knows "100% of regression suite passing" doesn't mean anything other than a neat checkmark for C-level executives.
Woah! AGPL? That's interesting. I think Postgres has shown an open source SQL server didn't need a copy-left license to develop sustainably, so I'm not entirely aure about that, but I do like the license in general.
When the software consists entirely of ~$1000 worth of Claude credits and ~40 hours of developer time prompting and curating it, literally what does it matter what license the resulting 100k LoC artifact is provided under?
Copyleft and the whole software licensing ecosystem only matter when producing that software actually requires serious human effort and dedication.
For my machine translation of SQLite to Go I added this to the README as to licencing:
Most of the code here is machine translated using wasm2go. As such, the original authors retain copyright and the original licenses remain in effect. Everything else is licensed under MIT-0.
The translator (wasm2go) has a licence chosen by, and a copyright notice from, me. Makes no sense for the translated code.
I do the same for translated code. It's not creative work which is a prerequisite for being copyrightable.
And avoid relying on direct LLM output for actual work to make sure I don't accidentally include some regurgitated snippet from an incompatible license.
It helps that LLMs struggle to write good, idiomatic code in my language of choice.
> Copyleft and the whole software licensing ecosystem are only applicable when producing software that actually required human effort.
Fixed that for you. Code generated by an LLM is not copyrightable (because copyright only protects human effort), so the codebase is automatically public domain and cannot be licensed at all.
They could theoretically copyright the prompts that they used, but as that's not part of the output, and the output doesn't deterministically arise from those prompts, they'd struggle to use that to back a copyright claim.
It's silly, nearby all Rust projects are rewrites of existing projects and now as you don't need to learn Rust to do rewrites, people just let the AI rewrite projects in Rust.
It's so great. One thing that I'd like to be changed in PostgreSQL, which may be done in this rewrite, is resigning from the "one connection = one process" design choice and instead handle the connections using threads/tasks within the main process.
What is the future of this? Code is not the same as a viable open-source project with a community, contributors, advocates, users and funding, even if it's perfect code.
Even though I'm sure it won't be easy to convince the Postgres project to switch to Rust, I do think that trying would be time better spent.
It is theoretically possible to have a Rust port of Postgres support extensions. If you make all the relevant functions and structures ABI compatible with Postgres, extensions should work. The issue is the moment you're dealing with C pointers and C strings, pretty much all the code you have to write is unsafe.
The return type in the rewrite is both some sort of Error tagged union that supports the Try machinery in Rust; but, it also contains a boolean that apparently must be checked; or something. It seems labyrinthical and possibly broken and terrible.
I make no claim as to whether the change makes sense given that I didn't look at the callers of this function, but Result<bool> is an entirely reasonable pattern in Rust. If you want the callers to be able to distinguish between "has the subclass", "doesn't have the subclass", and "something went wrong" this is idiomatic Rust.
I wrongly guessed that the boolean in the original C code was for error handling when I skimmed it, but instead it is just a result value, while elog() and related macros/functions are used for general error handling in the C version. I agree that it makes sense in Rust and other languages with tagged unions.
Though often when applicable, a simple tagged union is used instead when that would document the intention better. Like, the Rust version of search_pg_class_full_form::call() returns a Some for cache hit and None for cache miss as far as I can skim, and that group of methods returning that could arguably have returned a basic enum instead with CacheHit(value) and CacheMiss. Though this is a nitpick on my part.
It is a feature in Rust, not a bug :-) (I know you didn't say it is a bug.)
The error-tagged union is PgResult<bool> - which means it contains bool as the result if things go well. (The other part in the union is of course the error.)
In the original function also, it is returning a boolean: "bool has_subclass".
So anyway you have to check for the boolean as part of the logic. That is what it is doing.
Yes, but the original boolean seems to have been used for error handling, and the tagged union is also used for error handling. Why have both simultaneously in the same function instead of just one of the two?
Edit: Looking at the code again, perhaps I was mistaken, since the boolean might not have been for error handling, just the result of the function, and C's limitations regarding error handling led it to using something like elog(), apparently a macro defined in https://github.com/postgres/postgres/blob/master/src/include... .
And they'll reproduce virtually the same problems, because code was never a primary issue for output. Products reflect organizational problems, which AI can't solve.
From what I skimmed manually, not that many, but the code itself seems labyrinthical. Like, why have both Rust Try-supporting Error-like tagged union, but also booleans, for error handling, in the same function?
Now that I have taken a closer look, the code looks significantly better than it seemed at first glance, though there are still peculiarities, and some drawbacks.
An unfortunate aspect is that the code has become a bit more bloated in some regards due to usage of Result, instead of an implicit elog() macro and similar. Passing Result around, in some ways as an alternative to an unwinding exception, is cleaner in some ways, but it also bloats the code somewhat.
The rewrite also could have simpler code in some cases, like
I see a lot of MemoryContext. I am not sure how much that bloats the code (though the C code is bloated due to C's issues and problems, like re-using collections and such). Does it incur an overhead?
> The rewrite also could have simpler code in some cases
The Rust code is a literal translation of the Postgres code which returns the value at the end instead of an early return.
> I see a lot of MemoryContext
MemoryContext in C is used for multiple reasons: 1) performance 2) keeping track of how much memory has been allocated and where and 3) preventing memory leaks.
Reasons 1 and 2 are still relevant for Rust. The challenge is in C memory contexts are stored in a global variable. Global variables don't work well with the rust borrow checker so I opted for passing memory contexts as function arguments instead.
Sorry, I wrongly assumed in the C code when I skimmed it that the boolean was for error handling, not the result value. The elog() macro is used for error handling.
Rewrites in Rust are kinda impressive. This language with its move semantics and close ownership tracking is very different from every other language. To create a rewrite in it, you have to rearchitect the code. There is not as much freedom there when it comes to where to keep what and where you can pass what as it is in other languages.
I have privately wondered for years, pre-AI, why Apple hadn’t paid some engineers to go off and write some comprehensive test suites and then port these to Swift. It would shut down entire swaths of memory safety bugs they have been coping with for literally decades. SO MANY of the zeroclick iOS exploits can be traced to a few fragile and vulnerable foss libraries, xkcd 2347 style.
We had one for SQLite (which is SQL-ite btw, not SQ-Lite which doesn't make any sense) via Turso, no wonder we see the same for Postgres. Personally I do want to see libraries be in as much memory safe languages as possible.
How do you know it's not SQL-lite with the single L serving a double role?
Common pronunciations allow you to stay perfectly ambiguous about where the L goes, which aligns quite well with the name as spelled. If you do it right, nobody can tell if you're saying sequel-ite or sequel-lite or seque-lite on the one hand, or S-Q-L-ite or S-Q-L-lite or S-Q-lite on the other.
AFAIK there is no official word on how the name is intended to be read or said.
Interesting, thanks for mentioning that. I've always wondered about the origins of the name, never found anything, but now with your mention of "mineral" I was able to find this:
> (Hipp) How do I pronounce the name of the product? I say S-Q-L-ite, like a mineral.
> But I also hear a lot people say, "Sequel lite and SQL lite." You know, I don't care. Whatever comes off of your tongue easily is fine with me.
Hey author here. Wasn't expecting to see this up.
To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture.
I'm now working on a new, not yet published version of pgrust that incorporates a lot of techniques. Currently the new version:
If you have any questions, I'm happy to answer them.I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all.
Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all my tests passed with flying colors the rewrite itself was broken even on basic functionality or missed a ton of details. It was in effect useless when I dived into it. Initial tests also showed massive gain in performance, and I know people who were involved aren't really dumb so something smelled funny. Turns out all those things left out and honestly... moments were the key ingredients.
What I did learn from those beginning explorations though was that one-shotting, grand architecture or source up-front, master plans up-front.. all these do not yield good results - YET. Who know what we'll see in few years though. What I did found that works (FOR ME, nota bene) is to keep the design and checklists for myself, written by myself and then do a small piece by piece.. as if you would if you were coding alone or if you would waterfalling a small team of talented juniors. Then, suddenly super happy results come out, but then it's mostly you driving all the way where llm writes code and offers advice (which for the most part you ignore). It's a happy place for myself at least. It's then truly unlocking yourself to the mythical 10x.
Rewriting a large proven system with decades of ultra expertise behind it, which I don't have, is guaranteed not to end up the same 1:1 replacement. If you found a recipe for that - please do share.
I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?
That's the million dollar question. Do you/we/us have tests that cover everything which covers QA as well? If such a mythical beast exists, maybe from remnants of ye olde TDD past and hasn't been modified as such.. then maybe this would be possible to do as such.
what model did you try it with? I agree and also push back a bit: How will we know when the LLMs reach the point of handling it if no one takes the leap? I applaud more people sludging through the slop and hauling their slop buckets around.
An example is Fable being released. I felt like the most complex thing I was willing to sludge through was having it clone llama-server's web UI with my own opinions (I really like the original, kudos to them). And the initial skeleton was working so well I felt like I had sunk the tokens and committed to getting it the rest of the way: https://inkcap.click
Last time it was Opus 4.8 to no avail. Fable is too expen..precious to try that yet :)
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
That’s not true for Postgres however: due to its usage of a shared memory pool, whenever a subprocess is terminated unexpectedly, Postgres will kill all other processes and enter recovery mode, replaying the WAL, during which time it will not accept connection requests.
It does this because it can’t possibly know whether the dying process did bad things to the shared memory pool.
You are correct! TIL, and thank you. Connection processes get a SIGQUIT, shared buffers cleared, and WAL replayed, but postmaster stays alive. It's effectively an online restart.
I’d be very curious to hear what an online restart approach for sketchy extensions crashing a shared thread per connection process might look like. This is more a question for the author of the project but I’m curious if they have plans in that direction.
systems, upstart or even simple primitive script wrapper.
Built-in is also possible: just fork once after start, and you have parent as watchdog and restarter.
What's online about this restart? Just that the tcp port keeps listening? I assume all running transaction will have to be aborted, right? And connections dropped?
The postmaster is effectively your supervisor. It could see the child segfault and abort the whole DB, but that would be deferring supervision to your init system. Not ideal.
If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.
A mixture of threads and processes that can be used to match processors, disk I/O, and network interfaces.
A very long time ago, there was once a feature called "Data Blades" which tanked a commercial database vendor. A badly behaving blade could bring down the entire database. Most anyone who has been working on databases for a few decades remembers this and makes a point of either not introducing these sorts of features or making use of processes over threads.
I have not looked at the code referenced in the mentioned project, but thus far I haven't seen a model that could craft a complete SQL parser on its own.
There are a number of problems, and design decisions, that a developer decides on when writing a database that I don't see any current models… just because you have the ingredients does not mean that the stew is edible.
Informix. Michael Stonebraker, to his credit, learned a lot along the way and revised his thinking about database technology and capability after believing that a single engine could be good at everything.
> A very long time ago, there was once a feature called "Data Blades" which tanked a commercial database vendor.
I have no idea what this is and a web search turned up Harbor Freight woodworking tools.
My first web search (data blades) turned up harbor freight woodworking tools.
Then I made a second web search: data blades database.
That turned up some ibm database software module technology which I assume is what's being discussed.
I performed both searches on Kagi and didn't see anything about IBM. I do see results for "DataBlade" and "Data blade database" but I didn't try those specific variations after my first two attempts returned nothing of interest. That's too much effort to decipher a HN post.
> That's too much effort to decipher a HN post.
It would have been less effort than it took you to write this comment. Perhaps next time that you can’t be bothered, just ignore the comment and move on to another thread that meets your required spoonfeeding levels.
Yeah you're absolutely correct. Too late to edit/delete.
ignoring the part where you tell others how to live, I believe the effort estimates you used suggest you may struggle typing and project it on others.
Doing a few searches and reviewing their results is effort.
Google doesn't really say what went wrong if it's about Informix.
people complain when not spoonfed; however there seems to be no issues about spending effort to complain.
I think it's this: https://www.ibm.com/docs/en/informix-servers/15.0.x?topic=co...
And what you found seems to be "Dado", not "Data".
Yes, Kagi was sufficiently confused as to what "Data blades" are that it actually thought I was looking for replacement woodworking blades. "DataBlade" finds the IBM Informix product.
Especially since all those sketchy extensions can be rewritten in rust over a weekend and have their bugs fixed as well.
While PG's behavior doesn't guarantee a lack of data corruption, "an extension crashed, all bets are off, tear everything down" is going to give you a much better fighting chance against data corruption vs the alternative.
In the age of vibe-generated code, I promise you're gonna want the safety.
What about just not installing sketchy extensions?
I know right? Postgres is not firefox, we don't operate with 25 extensions on all the time. At most we have 1 and normally we have 0.
So load them up in read replicas
Or use webassembly to sandbox them
You have to have a write path. You're gonna want what to be non-sketchy and performant.
You could fix probably the sketchy extension issue with WASM.
Limit, not fix.
Do you mean that you don't run sketchy extensions and therefore this doesn't affect you, or that you're ok with data loss due to extension failures?
What about extensions that are not sketchy? Lots of good ones out there.
People are assuming the extensions can't also be rewritten to be good.
Presumably they'd be fine running in a threaded context.
An extension written for a single threaded host system might not work in a multi-threaded context. For example if has global or shared state that isn't protected with locks or similar (which is unfortunately fairly common in c code)
Extensions like pgvector, TimescaleDB would probably need to be ported tho, not sure how much but there are footguns.
Not needed in many (most?) cases.
But it isn't.
Threads does not offer any major performance advantage, performance of processes vs threads is virtually the same. The reason the PostgreSQL project is moving towards threads is to make development easier.
> Threads does not offer any major performance advantage
This is very not true. When it comes to parallel queries, a process model adds a ton of overhead. You can't pass pointers between processes because the address space is different. This adds a ton of overhead in a bunch of different places. For example when doing a parallel hash join, Postgres will have each worker build a local hash table. Then it will take all the tuples out of the local hash table and copy them through shared memory to the leader who will then construct a new hash table. This duplicates a lot of work as you have to hash the tuples multiple times.
A lot of getting to Clickhouse level performance was making better use of parallelism.
Passing pointers is not significantly faster than passing offsets into a shared memory pool.
Sorry, what? Passing a pointer is a matter of wrapping the value into the CPU register. OTOH passing an offset into a shared memory is a write to main memory so several magnitudes slower.
Ok ... you know PostgreSQL supports hash tables in shared memory, right? PostgreSQL could in theory share those if we wanted to. The issue is just that coding anything which uses shared memory is a lot of work.
Additionally the reasons PostgreSQL does not offer Clickhouse performance has very little to do with parallelism. PostgreSQL plans to move to threading but the efforts around imporving OLAP performance are almost entirely unrelated.
> The issue is just that coding anything which uses shared memory is a lot of work.
Doesn’t that kind of prove the parent’s point though? In theory shared memory can do anything that threads can do. But if in practice some feature doesn’t get implemented in the multi-process design (because shared memory is hard), when it likely would have been implemented in a threaded design, then that’s still an advantage for threads.
Apart from being a lot of work are you really gaining much at that point? Memory corruption can still take down both sides...
i may be missing context, but shared memory across processes, without ipc?
There's nothing special about threads vs processes in Linux. mmap works the same, the challenge is to map the same file. You can share a path, pass a file descriptor via fork or unix domain socket, among other techniques.
That induces disk I/O overhead (even if it somehow doesn't impact IPC performance)
It doesnt. Processes can share memory
The file doesn’t have to be disk-backed.
Don't you need something mounted for that?
No, you can use a memfd.
And `/dev/shm/` (which postgres uses by default on most Unix platforms)
Being really pedantic here, shared memory is considered IPC, but not the kind you're thinking of. Shared address space, no overhead.
As long as we're pedantic ... the subject is shared memory. Unless you specify the same, non-null, target address in the call to mmap (and the kernel happens to grant you that mapping on all calling sites), the addresses will be different; the address space is not shared (each mapping might also have different access permissions).
That distinction is important as pointers generally cannot be shared (a problem which can of course be solved with one more indirection ;-) .
MSSQL can handle 32k open connections no need to run a pooler in front of it, can PG do 32k connections and a process for each?
MSSQL shares cached query plans between connections including jitted code, PG cannot do that and the changes needed to make the plans cross process portable would be extensive while sharing between threads is just normal code sharing between threads.
unless you're spawning them for new connections.
Some, but not that much. Switching PostgreSQL to a threaded model will not magically make spawning connections fast. PostgreSQL connections are quite heavyweight.
The reason to use threads is almost entirely about ease of development, not about performance. If you use shrared memory like PostgreSQL does you need to write your own allocators, etc. So much you get for free if you use threads.
Doesn't postgres (rightly) have a cow if a process has a disorderly shutdown (at least while in a write transaction) because there's shared memory between the processes?
Some see a 30 year old system and think "outdated", I see a 30 year old system and think "time tested."
Clearly a process per connection is more stable and that's what I'm using.
It's unclear what problem such optimizations are solving anyway, with the old way you could only support a million concurrent users with a single server? Are we missing out on supporting ten million concurrent users with 2 servers instead of 10? Ostensibly reducing the minimum db hardware opex for a 10B$ company from 10k$/month to 2k$/month?
A million users? Hell, I'd bet 99.999% of live postgres databases in existence serve less than 5 users on average. Even among products that actually make a profit, I bet 99.9% of them serve less than 100 customers a day. We hooligans on hacker news manage the 0.1% of databases, and in my newfound consulting life, I'm hoping to never support one of those again.
> Clearly a process per connection is more stable
Even if those processes share most of their memory, and are written in a notoriously memory-unsafe language?
A significant performance improvement can well be the difference between being able to run the entire database on one beefy server, and having to shard. And that has a huge cost in terms of complexity and thus reliability and development time.
Is that a serious issue? Wouldn't it just restart the split second later? Or does it take a long time to start?
(Or I guess it would get stuck in a doom loop or something?)
I'm not informed of the Postgres's internals, but, maybe, that can be solved by grouping threads into different processes depending on which set of extensions they request.
A more modern way to do this might be to support WebAssembly plugins.
The extensions might need to be rewritten, but hey, we have AI for that now, so why not :-)
An OS thread per connection can be fine for performance if you don't have to scale your connections, but if you don't need to scale connections why have connections at all? Databases are even more performant when you eliminate connection overhead entirely.
Thread pools
Ouf. I don't know. I don't want to call you out without evidence -- I myself make benchmark claims all the time -- but 50% improvement in OLTP seems suspicious. I get that you used a standard benchmark, and I don't even know what it entails, but my spidey sense is going off. Perhaps, some trade off somewhere that won't make it to prod because it breaks MVCC -- and yes, I saw that it passes regression tests.
Just checking, is fsync on? :) Regression tests don't catch bad IO patterns afaik.
Anyway... sounds like a fun project to work on!
Yeah, claims that it can be faster than CH are very suspicious. CH guys are very good at their craft, they spend hundreds of hours optimizing one single small detail.
Yeah. I don't think PG could even come close. Column-oriented is fundamentally different, and pairs well with all the SIMD acceleration ClickHouse is also doing. There's just no comparison. If a Postgres rewrite came close to that, it must've sacrificed something else.
Remember when databases were faster to run in virtualbox rather than bare metal? (because virtual box was completely ignoring all the instructions to flush the data on the disks)
It'd be very unfortunate if Postgres didn't have regression tests for data loss due to bad io patterns. Should be possible to do some checks against those in an appropriate test harness. Which might mean "have qemu run something we can kill off and examine the results".
If those don't exist, I hope folks recognize how useful they are and add them.
What's your actual background and expertise with Postgres and databases more broadly? Basically, do you actually know what you're doing, or is there likely a massive footgun you don't know or haven't shared with us?
I spent a couple years managing a Postgres cluster with a petabyte of data. I wrote a couple blog posts from my work then[0][1]. I also wrote dozens of posts on the Postgres internals[2]. I've also given talks on how to generate fractals with SQL[3] and how to write a lisp interpreter in SQL[4].
[0] https://www.heap.io/blog/testing-database-changes-right-way
[1] https://www.heap.io/blog/analyzing-performance-millions-sql-...
[2] https://malisper.me/table-of-contents/
[3] https://www.youtube.com/watch?v=xKoYIvMFnoQ
[4] https://www.youtube.com/watch?v=MPSMH8w7nfw
It doesn't sound like you were trying to launch a product, but doing an experiment and someone threw you under the HN-spotlight-bus :) Is this a "see what I can achieve with LLM coding" or is this "build this and see how much of the coding can be accepted from LLMs"?
> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?
In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax
It is currently about 30-40% slower than ClickHouse (single node, ofc). The PR to add it to clickbench was just accepted, so you can see the comparison here: https://benchmark.clickhouse.com/#system=+liH|_etx|gQ|saB&ty...
Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.
Yeah, spent a lot of time on parallelism, vectorizing, pipelining, filter push-downs, bloom filters, all the tricks out there. It's really fun to make pretty steady progress on this.
pg_mooncake (now effectively abandoned due to being acquired by Databricks, but still up at https://github.com/Mooncake-Labs/pg_mooncake) pulled the DuckDB engine into Postgres wholesale, if I remember right.
pg_lake also uses DuckDB but keeps it external, routing through Postgres and managing Iceberg tables (but not the data itself) there (https://github.com/Snowflake-Labs/pg_lake).
Both of these were neck and neck with ClickHouse last time I tried them.
Actually δx is faster than the "duckdb embedded in postgres" options: https://benchmark.clickhouse.com/#system=+_etx|_b|_i)|dula|pnc&type=-&machine=-6t|ca2|6ax|g4e|6ale|3al&cluster_size=-&opensource=-&hardware=+c&tuned=+n&metric=combined&queries=-
Plus all the normal Postgres features work as expected: physical/logical replication, crash recovery, pg_dump/pg_restore, etc.
That isn't what the data shows, but we don't need to discuss it further. My reply was to the person interested in learning from other Postgres OLAP designs. This stuff is all pretty immature though and I wouldn't actually build on it outside of some very narrow, well-understood workloads.
What was your methodology and structure in making the prompts for the rewrite? Did you let the LLM roam in all of the codebase and tests from the beginning, or revealed things to it gradually in some way?
> build a better version of Postgres
Faster is quantifiable. How do you measure better?
Are you fixing the heap and table management ?. Postgres does not use an undo log and manages all table updates directly in table storage which slows MVCC.
also have you told Ben Dicken ? https://x.com/BenjDicken/status/2074326407795417435
That's something I eventually want to fix. The challenge is the storage format is so integral to Postgres that it's going to be a huge PITA to come up with a novel design.
Right now OrioleDB is in beta. Once that becomes production ready, I'll evaluate incorporating it into pgrust.
For Ben Dicken, he has seen the project: https://x.com/BenjDicken/status/2074512043462603236. We're still working on all the novel features so I don't think it meets his bar quite yet.
best of luck to you, it will be great to see a rearchitected postgres.
While a thread-per-connection seems like an improvement, do you have any plans to allow query multiplexing over a single connection? That would be a huge improvement IMO.
Can you elaborate on the use case for query multiplexing? Is it so your client would only need to establish one connection with Postgres and then could run as many queries as it wanted?
Microsoft SQL Server has had a similar feature for a while -- Multiple Active Result Sets aka MARS. I don't have a good read on whether it actually helps any workloads. I've seen adapters that don't support it because of the extra complication.
https://learn.microsoft.com/en-us/sql/relational-databases/n...
Multiplexing would have a number of benefits. As you say, each client would only need a single connection regardless of the number of queries being sent. Resulting in:
On the client side, there is usually a local connection pool. When a burst of traffic comes in, the client needs to either wait for the pool to free up or establish a new connection, which adds latency. This latency hit wouldn’t occur with multiplexing.
With multiplexing, systems like pgbouncer would be unnecessary.
Also, even with a thread-per-connection, you can still quickly exhaust the servers resources when you have lots of connections because threads have a lot of overhead. Reducing the number of connections needed would greatly increase the number of clients that a database can serve.
At that point does it make sense to use http/3 to avoid head-of-line blocking on the network?
Not the OP, but yes - just imagine a web server talking to DB over one connection without any connection pooler
"Is 50% faster than Postgres on transaction workloads" - That is a very big claim! 50% faster on everything? Is it a strict improvement across the board or are there tradeoffs that make some workloads slower?
The 50% is specifically on percona-tpcc[0]. I got there through a mix of batching (postgres processes a row at a time), prefetching, and several handful of other optimizations.
This is great! Those analytical workloads numbers are mad - I'd love to see the benches, and I'm happy to contribute to some of the profiling.
How does your thread-per-connection model compare to Heikki's proposal[0][1] from back in 2023?
[0]: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5... [1]: https://www.youtube.com/watch?v=xLLakMmVtbY
Rust actually made the change pretty simple. The main changes are:
I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not something I have to do.Super impressive! Is it possible for you to share your methodology of using LLMs?
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust.
This naturally lended itself to a suite of skills to describe how to rewrite a crate from unsafe rust to idiomatic rust. The main three skills I had were 1) a skill for identifying the next crates to port 2) a skill for rewriting a crate and 3) a skill for auditing a crate and making sure there weren't any outstanding issues.
My exact approach for managing subagents changed throughout the project. Initially I was doing parallel coding sessions with Conductor. After dynamic workflows came out, I used that as it was really easy to spin up dozens of parallel subagents and manage it from a single orchestrator. Over time I switched from using dynamic workflows to manually spinning up subagents from a central agent. The issue with dynamic workflows is they waterfall. Each step needs to finish before the next one starts. By manually spinning up subagents, I could have claude start porting a new crate as soon as a prior subagent finished.
I think I would be horrified looking at your Claude API bill.
Thanks for the insights, are the skills available anywhere?
Did you hand write the skills or did you have an agent audit your work and infer patterns?
Actually the inverse. I initially gave claude an outline of what I wanted, had it do some research into how to write idiomatic rust, and then had it draft a series of skills to do the work. I would then try out the skills, audit the results, and then give claude feedback based on what I was seeing. Once I started getting runs where the results were working, I would start to scale things up and audit things with an exponential backoff.
Thanks for sharing.
Is it being used in production anywhere, even if only a toy app?
I know you say it's not production ready and not optimized yet, but in the same breath - in your comment here - you say it's already faster.
It's not used in production. I've been using different benchmarks to compare the performance vs other systems. Namely sysbench-tpcc[0] and clickbench[1]
[0] https://github.com/Percona-Lab/sysbench-tpcc
[1] https://github.com/ClickHouse/ClickBench
How much of the performance gain is from using Rust, compared to using optimizations that are not done in the original PostgreSQL code (like using threads instead of processes, etc.)?
I am simply curious what the benefits of using Rust are in this instance.
I am super curious how you went about the port using LLMs. At $WORK we are looking to port code, preferably with LLMs, and it seems daunting, even with a test suite. Do you have an approach that works well for you?
Bun has an interesting blog post about how it was ported. It did cost a lot, much more than hiring people would cost outside USA.
Just a couple of ideas if you run out of backlog:) - proper versionnumber (64bit) - native json streaming. It would be awesome to get to the point where i could somehow redirect the sql output to the browser directly, but piping will do for now. The idea is to be able to stream rows to client without caching and building json along the way.
PG Wire proto 3 is my largest source of frustrations.
I'm playing with a POC for a better wire protocol here: https://github.com/solidcoredata/pgwire4
404 Not Found
Now public.
Would you like to submit to ClickBench?
I can also do it if you would prefer...
It's a completely new era of software production (I will no longer call it development) LLMs give us unlimited manpower, and the language give us constraints to make more modern and safer softwares. Love to see this rewrite in Rust, and expecting much much more in next few month.
Do you have anything in the regression test suite like jepsen etc?
Nothing major yet. Once I wrap up the performance work I'm doing I'll start looking at the best way to go about testing. I suspect there's a lot of novel things you can do with agents.
when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward?
use both synthetic and real data to sample the internals of the original software to duplicate.
locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.
That forces you not only in not the intentional good decisions of the past but also copies too many bad ones.
i highly doubt you can make it faster than clickhouse, but happy to see it.
That…is really impressive. Well done!
Is it your first rewrite/migration (with or without llm) ?
good luck nonetheless
300x is mostly a marketing term, especially without the test description.
BTW, showing no respect to what it is trying to copy looks uncomfortable.
This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).
Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.
I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.
Keep doing it legend.
Awesome work. I'd love to see you add something like kusto query language or pql. The autocomple on kusto, (which can be embedded into web apps) is really amazing.
Don’t understand these rewrites.
- typically they are behind a single person. That’s usually bad because of spf
- typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
- anyone that wants to contribute to the project needs to pay. Needs to pay tokens because it’s increasingly difficult to maintain these projects without AI
So, who wants to put something like this in production? Doesn’t make much sense
I'd be interested to hear the author's answer to your question, but I see it as an interesting proof-of-concept. It's testing the viability of not only rewriting PostgreSQL in Rust (and their choice of deps) but also in switching the threading model and other architectural changes. LLMs shine at pumping out prototypes insanely fast, and a working prototype can put an end to a lot of speculation.
I likely wouldn't use a rewrite of such a huge project if it doesn't have the backing of the original team (or a significant fraction thereof) and a believable story for having matched/exceeded the original code quality and maintenance. I also think in general using an LLM for license-laundering is legally and morally hard to defend, although this case is different in that they chose a more restrictive license. Not a lawyer, but my understanding is that you can just download PostgreSQL, do s/MIT/AGPL/ and release it, legally. (The original MIT-licensed version still exists, so no reason anyone would prefer yours until you make another release with some compelling new feature.)
This is very critical of an open source project that the maintainer didn't even post here?
"Status:
pgrust is not production-ready yet. It is not performance optimized yet."
The maintainer is not suggesting you use this for anything yourself. So why do you care about spf or (lol) his "discipline in creating the project"?
The gp feels like less of a targeted individual criticism & more of a general musing about a trend.
Nobody is saying this author hasn't demonstrated discipline & won't maintain this project, but the statistical averages across most projects fitting this trend make it likely enough to question their worth in aggregate.
The author is free to ignore any and all complaints they consider unfounded. It’s not even like the author is recieving any complaints personally; they have to come here to see any. And if they come here, they will get to read the viewpoint visible from here.
(Repost of <https://news.ycombinator.com/item?id=45253509>)
Because chances are that it's never going to be production ready, never trustworthy enough to be used by anything serious, and the project will eventually be archived and become at best a (extremely inefficient) learning experience for the author, or at worst a total waste of tokens. I could be completely wrong about this specific project, but most of these projects are like that, and statistics don't lie.
100%
People on HN love complaining at any given moment.
I’d wager most of these people don’t really produce much and are constantly bikeshedding.
But this is also just silly. The maintainer here did nothing (just a little chat here and there), but is sure that only optimizations are missing for this to be production-ready. How so, based on what exactly?
It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations. No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contributed by a ton of brilliant programmers, no matter how you look at it. And if we take this as the truth, then logically there's still an unanswered question - why? Lets take an undeniable fact - rewriting an existing project give you full control over the new implementation. You can do whatever you want with it, you can't be sued. The only thing now you have to hope for is for your implementation to gather a good enough user base and from then on you can practically hijack the original project. And this is me speculating - rewriting stuff in Rust isn't about the greater good for that magical "memory safety" argument, but at the end it's an attempt to hijack popular software projects.
> it's an attempt to hijack popular software projects
Exactly. And this is why those projects are typically called "XXX in Rust" or "XXX-rs". Because the creators get to do their favorite thing - coding in their loved language - while skipping all the hardships of designing, accepting real feedback, involving users and getting traction - all while simultaneously hijacking the existing brand.
As of now I know of a single project that changed their name after being called and the project surprisingly got some traction.
I think we're a year or two away from the same argument being used by languages which enforce proof of correctness. The economics seem to be shifting in that direction.
Finding exploits is getting exponentially cheaper, and the cost of producing proofs is rapidly going down. For a lot of software correctness is rapidly becoming non-optional.
> impossible to replace years of experience
People with no experience don't have enough experience to realize what exactly they are missing here.
It's not just a rewrite ; it has improvements. I did the same thing for fun for the same reason; I wanted to see if I can improvements on some of the legacy design stuff and, especially, the stuff PG people have told us that it cannot be done differently. It can. I would not put it in production, but it thought me a lot about the internals of databases. To keep my brain happy in the age of LLMs, I implement database things on our (also old but many times refactored/rewritten) production db without an LLM. I'm sweating through Flexible Paxos now; probably we will just keep using raft as it's old and stable and simple but it's interesting anyway.
> I would not put it in production
Nobody else would either.
If this is meant for personal learning, do it that way and make it clear that others should not even consider using this project.
In fact, even for personal learning it's wasteful. You can learn so much about database without a rewrite like this.
I'm really happy to see these rewrites. It shows what's possible, especially as the projects get more and more complex.
And maybe Remacs will get reactivated https://github.com/remacs/remacs/wiki/Progress
Maybe even I'll be able to do it when I wait for the bus!
It's useful to show the actual team that it's possible. From there, they can make the decision of whether to go the bun route with more information.
I'm not sure I see the value in "showing the team it's possible". I would presume the team are intelligent enough to be well aware that it's possible, given tradeoffs. And as it's clear those tradeoffs have been "traded" in this case, it doesn't seem like having the knowledge confirmed is particularly surprising.
This is true but these projects are rarely presented or interpreted as a proof of concept.
The bun route == spending 100k for an LLM rewrite as an advertisement for the LLM company you sold your soul to.
I guess it is cool to have it around but it comes off as a popular useful case for AI which it really isn't, unless you have a very good test suite to throw the agent against, it is practically useless for rewrites of decades old badly documented code riddled with technical debt. That is where the real value and challenge would be. I see it as marketing and social clout stunt
> it’s increasingly difficult to maintain these projects without AI
It's pretty much impossible in a project of this size. IIRC Postgres has over 1M loc.
Are there modular databases with clearly differentiated components? I wouldn't be surprised if there aren't, as the trade-off with modularity is usually performance.
It’s open source. You take ownership of it for your deployment and stop relying on continued free work.
You can use llm to pull in updates as they are released. It’s not gpl, so you don’t need to publish your port
1. Nobody, not even the Googles or NSAs of the world do that. No single entity has the expertise nor resources necessary to maintain a fork for every open source project they use -- forking and maintaining Linux alone takes teams of people. And no, going full psychosis mode with LLMs is not going to save you.
2. This project is AGPLv3.
> going full psychosis mode with LLMs is not going to save you
People in AI psychosis don't know that.
Forking and maintaining Linux does not take teams of people. I've been at it for > 10 years and maintained support for my 15 or so various SBCs/mobile devices, writing drivers, debugging, cleaning things up. It's a weekend project every 3 months + whatever you want to put in for development.
And there are others doing it for projects like Armbian, openwrt, etc.
Except Amazon (Linux, Elastic, RDS..)
The companies I have worked for before all have used open source software like postgres, mysql, go, python, k8s, etc. 99% of the time we relied on free work; never contributed to these projects nor forked them for our own needs. I don’t think this behaviour is the unusual path tbh
You don't even need to publish your port if it's GPLv3 as long as you don't publish the binary.
> So, who wants to put something like this in production?
I don't think anyone suggested deploying this to production - the author is quite explicit that it's an experiment.
>typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
Lindy effect! The longer something has been around, the longer it probably will be.
https://en.wikipedia.org/wiki/Lindy_effect
Not the same but it is much faster and easier to re-create a 3d model of an existing set of drawings than from scratch. This is because a lot of the decisions have already been made.
Everything has to start somewhere
How is that going for Claude's C compiler [0], or Cursor's web browser? [1]
[0] https://github.com/anthropics/claudes-c-compiler
[1] https://github.com/wilsonzlin/fastrender
Terribly, but just because any two people suck at basketball doesn't mean a third person must suck too.
malisper is the LeBron James of porting Postgres to Rust!
Maybe! Too early in the LLM rewrite era to tell whether this requires a LeBron or just someone who takes the local pickup game more seriously.
This seems to be functional software, though.
spf meaning...
single point of failure
not sun protection factor?!
Sender Policy Framework
SPoF :)
This year, not me.
In five? Everybody but me.
If you were a token burning company a project like this seems like a solution to this: https://xkcd.com/2347/
that hits your metrics without the problem that your contributions are not welcome.
I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.
Can you control the timing of queries across two db instances well enough to expect the tables to be identical?
Good question, I guess you can run two identical Postgress instances and check if they have a diff
How would one go about reviewing a piece of code like this?
One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you much anyway.
ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
[1] - https://github.com/malisper/pgrust/commits/main/?since=2026-...
(I'm working with malisper on pgrust),
I think the focus for projects like this is going to shift to reviewing the testing/fuzzing process instead of reviewing each commit (going much further than what the postgres regression/isolation/crash tests do).
related post from danluu: https://danluu.com/ai-coding/
Some of this post reminds me of a story I heard long ago from someone who had worked at a HW/SW company. They’d transferred an engineer from the ASIC design team to the OS kernel team, though he’d never been on a software team before. After a while the manager called him in for the following conversation:
Manager: You’re doing amazing work — zero bugs in production! I’d like you to mentor the other SWEs on how to get their bug count down too.
Engineer: We’re allowed to have bugs?
Funny story but in my experience hardware engineers produce some of the worst software of the industry. Of course there must be some hardware engineers out there who do hood software but generally what they build are disasters.
Honestly, i do not blame them for that though. The whole eco-system of in C- procedurally bitbang on some registers and read on others, until some circuit that might be there is coerced into doing work - often with faulty prototypes you have to rewire yourself, whos documentation is - non-existant-complete while having deadlines within deadlines. And the project-culture is just "aggregate" a layer, wrap the problem like a pearl in shellacks as the main abstraction.
I bet it's being organized by project rather than product. Conway's law ensures such an org will create code around projects, not products, and that always ends horribly.
I know it’s a typo, but I love the idea of “hood software” lol
Hardware engineers call them errata ;-)
How many engineers does it take to fix a bug?
Hardware Engineers: "None. We'll fix it in firmware."
Firmware Engineers: "None. We'll fix it in software."
Software Engineers: "None. We'll document it in the manual."
Technical Writers: "None. The user can figure it out." etc.
While I get the joke, as a technical writer, you might be surprised how often I've found myself as a defacto QA engineer:
Me: This is what you said it does, and this is what it actually seems to do. Which one is right?
Engineer: Shit.
Very carefully - and I mean extremely carefully - word it so that it describes what it does, while implying what it should do without confirming that it actually does that.
Also helps if you fix the bug or change the behavior, the docs are still technically correct. I'm only partially kidding, I swear I've seen this a million times in documentation I read.
The joke is awesome.
Sadly, monospaced fonts kill sarcasm.
For large projects like this I think a hierarchical division of labor also helps.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
Oh, I just posted a similar comment elsewhere in the thread.
https://news.ycombinator.com/item?id=48856535
Though beyond testing, I think there will be increasing focus on proofs of correctness. (Testing can only show the presence of bugs, not the absence. —Dijkstra)
At any rate, it's never been this cheap to produce the proof of correctness of a program, or on the other hand, to produce an exploit for an incorrect program.
And what happens when your "tests" are also vibecoded. Right now all of these houses of cards reset on human-written tests. What happens without them?
> reviewing the testing/fuzzing process
I've got insanely good at designing testing oracles over the last year for exactly this reason.
I've ported some extremely finicky software between languages that it would have been borderline abusive to have a human do.
Codex 5.3 and later for those interested.
The github cli has a command to query commits with a sorting asc/desc flag
https://cli.github.com/manual/gh_search_commits
here's the docs with more syntax using the "before x date"
https://docs.github.com/en/search-github/searching-on-github...
there's also an advanced search page, but it does not support commits when filtering with dates
https://github.com/search/advanced
or you can bisect the date in the search widget, this is the first day with a commit
https://github.com/malisper/pgrust/commits/main/?since=2026-...
first commit:
https://github.com/malisper/pgrust/commit/22113dc36b02973060...
Thanks for all the info you've provided!
Maybe I'm just being a little grumpy. If I really need to look into a repository, I clone it and use vanilla git command line tools to have a look.
It's just annoying that the modern web UI from GitHub takes >1s second to load a page with 34 commits
> How do you easily get to the first commit in a repo on GitHub?
You can use the syntax github.com/user/repo/commits/?after=last_commit_hash+number_of_commits-2 (-1 for the latest and -1 for the last)
ex : https://github.com/malisper/pgrust/commits/?after=3646a73515...
> One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project
I could not care less about any of this. Truth is code, as it is now. I don't care when (and certainly not by who) a bug got introduced, it's here, shut up and fix it.
So your solution is to just read through all 1M+ lines of code?
And yours is to read through all 37000 commit messages??
don't be obtuse. Nobody needs to do either.
This was already addressed:
> But with LLMs generating 7101 commits in less than a month that isn't feasible.
I don't think trying to understand LLM generated code is feasible for anything other than very small projects. IMO it's a big problem with using LLMs for coding. Sure, they can generate a bunch of stuff, but in some ways that just makes the real problems of software development even harder.
In general (I’m not saying this is the case with this project) if you don’t have their prompt history and you can’t re-run the LLM “compilation” yourself, is it open source? It feels a bit more like those “source available” projects where you can read the code but don’t have access to the build system.
On the other hand, aside from the commit messages, one didn’t ever have access to the underlying thought process of human developers either, so maybe it’s not equivalent to say that secret prompts mean closed-source.
What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.
I started by looking at the dependencies.
Then I lost count, so I ran wc -l Cargo.lock
Easily over a thousand dependencies. And "rewritten in Rust" is supposed to be a good thing? I bet this doesn't even compile faster than the original.Use cargo tree to understand Rust/Cargo deps.
The lock format is a multi-line TOML, with a varying number of lines per dep due to redundantly listing deps-of-deps, so a naive line count massively overstates the number.
Cargo.lock contains many unused dependencies, because it's a superset of all combinations of all optional/disabled features of all transitive deps across all possible platforms (so that the deps don't reshuffle even if you enable/disable feature flags or compile on another platform). But that means Cargo.lock is going to have 3 async runtimes even if you use one. It's going to have syscall definitions for RedoxOS and wrappers for WASM, because some dep of dep is compatible with those platforms. But these deps won't even be downloaded if you don't build for these platforms.
The number you got presented is not representing the unit you're insinuating. A crate in Rust is a compilation unit. It's common for projects to ship as a collection of many crates. It's a smaller unit than what C counts as one dependency, and slightly coarser than an .o file. I don't see people freaking out by how many .o files their projects have, including all transitive ones from deps like openssl or curl.
Do you pick your databases based on how quickly they compile and how many dependencies they have? I normally chose based on factors like performance and reputation for reliability
If a dependency gets compromised, that’s a problem. If you have thousands, you increase the odds vs if you have one.
You can also depend on a lot of libraries that have potentially high quality rather then writing a lot yourself. The defense against compromised dependency can't be 'Ill write everything myself and do it with the same quality as the ecosystem'.
Reputation for reliability can be directly impacted by thousands of upstream dependencies, though.
True, but internal dependencies aren’t upstream. I care very little how a project is laid out on disk before it’s built unless I’m going to work on the source.
The naysaying here is insane. Should a project like this not exist?
In a typical Rust project you organize your project into many crates. I haven't checked, but I'd guess that the vast majority of those dependencies are internal dependencies. After all, this project started by running an automated C to Rust converter, which the authors claimed produced over a thousand crates.
Disaster. Well, I read stories about Rust and how there isn't much in the stdlib, but this is just too much. How many dependencies are there, on average, in other projects? I guess I am spoiled with Go.
Vibe code was never meant to be reviewed.
These rewrites are just test-driven development taken to the absolute extreme. Created under the hope that the existing tests are exhaustive and cover every relevant use case, such that if they all pass, the rewrite must be at least as good as the original. So just go with the vibes and burn tokens until they pass, and your job is done.
In practice, this is never true for any codebase above a certain level of complexity, especially not one as mature and widely used as Postgres. But reality doesn't seem to be an obstacle for vibe coders.
The challenge is that more and more people are producing project like this - 1,000s of commits and > 200k lines of code - and saying it was carefully created using agent based workflows and not vibe coded.
In that case they need to document the process and workflow, and demonstrate the care that was taken.
Quite amusing we have decades of human written code much of it sub standard and yet no one demanded proof till now of Open Source projects having to ‘demonstrate’ anything.
If ya don’t wanna use it, don’t. Simple.
I get what you’re saying and agree with the last sentence. Just wanted to touch on the “why” part.
In the world of exclusively human written software the existence of the artefact itself (code, documentation) served as the proof that there’s someone with half a brain behind it. Now that’s not the case anymore.
The conclusion stays though - it’s OSS, authors/maintainers have no obligation to anyone to do anything. Like it, use it, don’t like it, don’t use it.
As for me, I’ve found that the community and activity proxies are still good.
> As for me, I’ve found that the community and activity proxies are still good.
Definitely still something to look into. A project I'm checking in on from time to time is https://github.com/emdash-cms/emdash/.
It will be interesting to see how the project activity is unfolds? Are people using it in production. How many errors do they find. What do those fixes entail. What happens with the docs over time. Etc.
I haven't had a change to look in depth, but based on a quick glance I'd say that the activity on the project seems like the tempo you'd expect of a similar open source project.
Maybe the principal maintainer can be trusted, but pull requests could do with some of that evidence.
spec-driven development is pretty good at this
> The challenge is that
Why is that a challenge? As long as they are open about this, all is OK.
> Vibe code was never meant to be reviewed.
It was also never meant to hit production.
> reality doesn't seem to be an obstacle for vibe
Went straight into my vault of brilliant quotes!
One of the projects Im working on and off is a tamper-proof audit log, based on some PoC code I created almost 10 years go; unit and integration testing are good at preventing defects and regressions, but they will not guarantee your software will work. However, with the power of LLMs, one can easily use model checking (in my case with Quint) and/or other formal proof approaches to ensure the software conforms as specified. The result (in my opinion) is an implementation guided by a single human that is actually more trustworthy than manual human-made software using the traditional approach.
And run them in test setups to try to find bugs.
If you find some, fix them.
You don’t. You trust that passing the regression tests means you are totally compatible with the original version.
> ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
I usually check the history of a file not easily changed like .gitignore.
The first commit seems to be this one
https://github.com/malisper/pgrust/commit/22113dc36b02973060...
Very smart. I like it.
> How would one go about reviewing a piece of code like this?
That's a wrong question. The right question is "why would one go about rewriting a piece of code in X". Once and if you find a good answer to that question, you will see the answer to your's.
2664 "unsafe {", 1835 "unsafe fn". This is completely unsafe. It doesn't look like a rewrite that understands what's actually going on or how the architecture should be redesigned to take advantage of Rust strengths. Instead, it looks like an AI generated transpilation with extensive use of raw pointers.
Note that most of the unsafes are confined to the parser which was generated by running c2rust over the Postgres parser. The Postgres parser is itself generated from yacc/bison, so I decided to port it over mechanically rather than idiomatically.
If there's particular unsafes that you think are egregious, let me know.
Just wanted to say: I'm thoroughly impressed with how far in the weeds you're replying in this comment section. I'm learning a lot from the threads.
Same. Sorry to see so much hate here.
Thank you!
I don't know if converting the code could be an issue with copyright, but might be contrived as plagiarism
oh no.. will they get grounded?
Counterpoint: All of the current Postgres codebase is already wrapped in an invisible unsafe{}.
The difference with a Rust codebase like this is that all of the unsafe code has been neatly isolated and clearly marked. The outside code is safe — at least according to the definition of what Rust considers safe, which is a high bar indeed and objectively superior to the unsafe mess that is C — and the unsafe code is naturally fenced in, which means that it can be seen by developers and tackled by incrementally.
In some cases unsafe is unavoidable, but it is possible for a human to verify that it is, in fact, acceptably safe even if inside an unsafe block.
Valid point, but unsafe in Rust is more dangerous than unsafe in C, bcs of aliasing. For example PG is compiled with -fno-strict-aliasing.
I set all my Rust LLM written projects to 'unsafe=deny'. Not sure why not everyone is anticipating your comment.
Let me just copy this review comment into my prompt.
a few hours later
Fixed!
Why even use rust...
Because in C everything is unsafe, by definition.
what a nonsense
Care to explain?
This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1].
I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably more ethical by choosing the AGPL - I just wonder, safer with no change at all?
[0] https://www.postgresql.org/about/licence/
[1] https://github.com/malisper/pgrust?tab=AGPL-3.0-1-ov-file
You seem to have the restrictiveness backwards? The MIT license (uutils coreutils) is less restrictive than the GPL (GNU coreutils), and the AGPL is more restrictive than the PostgreSQL license.
And it doesn't violate the PostgreSQL license to license the rewrite more restrictively. That's part of what makes MIT-style licenses less restrictive than the GPL or AGPL: they allow for more-restrictive relicensing.
Perhaps ‘restrictive’ was a poor choice of words. ‘Less free’ in the free-software sense.
I’m still putting together my thoughts on various open source licenses especially as we see automated AI rewriting.
I think licensing will have to be re-invented, because now it's trivial to just say: claude, do a "clean-room implementation", so that noone can accuse me of stealing code.
That's what LLM companies have done for years anyway. Inline completions are trained on licensed code. And nobody cares! ;)
If you don’t like the license just let an LLM spend a few days “porting” it and give that port any license you like because that is apparently what we do now.
Yeah. I'm gonna have an LLM rewrite Star Wars and then film it. Should be fine, right?
Making a movie that has basically the same plot as a previous movie with slight changes is a common occurrence.
The thing you have to be wary of with movies is trademark law. Your Star Wars copy can't use the word "Darth Vader", that's trademarked. It can't use Darth Vader's mask, Darth Vader's suit or Darth Vader's breathing either, all trademarked. And with trademark law the bar to pass is basically "would a reasonable but uninformed consumer be at risk of confusing your product for the trademark". LLMs can't launder that for you. You have to make actual changes, like Spaceballs did
Spaceballs is a parody, which is specifically an exception to the rules; it’s called “fair use”. If Spaceballs was not a comedy, it would not be permitted to exist.
LGTM. copyright laws don't matter anymore.
Only for the billion $ LLM companies that pay dear leader
> LGTM. copyright laws don't matter anymore.
hear hear, and good riddance too.
The only good thing to come out of AI.
How is this good? Copyleft licenses are also being ignored.
Patents and copyright stifles innovation. Knowledge should be communal and free.
> Knowledge should be communal and free
I agree, which is why I license my work under copyleft licenses such as AGPLv3 so that anyone has to make their modifications public too.
I think LLMs (especially closed-source models) will make this worse.
Free and communal knowledge means you're easily replaceble by someone else.
I wonder what communal answer you have for someone who would say, "moi2388, your work will be now done by moi2389. Thanks and good bye."
The alternative is that you are not easily replaceable, which means that moi2389 is the one who doesn't get to do the work. Which is good for moi2388, perhaps, but what about moi2389? Either way someone is going to be left out in the cold.
moi2389 should research the market and create a new value, not replace someone else by providing the same value.
Star Wars itself is a rewrite of King Arthur stuff so go right the fuck ahead.
The PostgreSQL License is a variant of the BSD license and is therefore compatible with the (A)GPL.
Comprehend it this way: You create a blank (A)GPL project and incorporate the upstream BSD codebase into it. While those original upstream files remain under their original permissive license, the project as a whole is governed by the (A)GPL (plus the attribution requirements of the upstream license, which the GPL permits). From there, you can add your own code under the AGPL and distribute the combined work under the AGPL.
If someone takes your code and uses only your portion, they can use it under the AGPL alone. However, if they also include the upstream source code, then the attribution requirements of the upstream license must still be met.
Yes, BSD licenses are compatible with AGPL meaning BSD licensed code can be combined with AGPL licensed code while complying with both licenses. However, it does not give you permission to relicense the BSD code (or derivative works) as AGPL. The author is free to license any new code they write as AGPL, however the license for the machine translated code is another question. If it is considered a derivative work (which I think it should be) then it must remain under the Postgres license.
If it is not a derivative work, then for copyright to apply at all then it must be an "original work" which has "at least a modicum" of creativity applied by malisper in the translation. If this is satisfied then malisper could choose any license for the translated code they want, compatible with Postgres or not. If it isn't satisfied then no license applies, because it isn't eligible for copyright - essentially it is public domain.
The safe and polite thing to do is to keep the same license when performing machine translation.
IANAL, but calling this "relicensing" is technically inaccurate. It is more precise to describe it as adding constraints. When you combine your work with upstream code, you are layering additional requirements (like copyleft) onto the existing attribution requirements. The original limitations remain in effect. Therefore, it is not a shift from A to B, but rather from A to A ∪ B.
This practice is entirely compatible with the PostgreSQL License, but it is often prohibited by GPL variants. You typically cannot combine GPL code with code under most other copyleft licenses, such as the Eclipse Public License.
Regarding copyright status, AI-assisted work is increasingly recognized as copyrightable in many jurisdictions, provided the process involves a sufficient level of human creative input (though the specific threshold varies by jurisdiction). Only work generated purely by AI, with no human involvement, is arguably public domain. In a case like this, which is akin to "pair programming," the output is almost certainly copyrightable.
IAAL (not legal advice) and I’m not sure the issue is settled.
The BSD license only explicitly permits the author “to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement.”
By default, the owner of a protected work retains all rights not conveyed to someone else. Changing the license isn’t one of the enumerated activities, and so I think there’s a case to be made that it’s not permitted.
Now if the author wants to claim it’s a new work, as opposed to a modification (which opens up a big bag of issues by itself because this was AI-authored), then the author can license it however they see fit.
You are not really changing the license of the original work though. You are not impacting what the author or anyone else can do. You are distributing a copy under different, more restrictive terms.
A bit like me buying a comic book, then offering to sell it to you under the condition that you never let my brother read it and that you make any future owner agree to the same terms. That's perfectly legal, and there is no reason I would need permission from the author (or publisher) to do that
A comic book is a physical object. This analogy doesn’t hold. When you give a comic book to someone, you’re only transferring the copy and its implied license that carries with it. You can set the terms of the physical object, but you can’t change the license of the content within it.
Suppose you get a license to view a copy of a work (streaming or a paid subscription to a newspaper site). Your permission to consume the media begins and ends with the license terms, which allow you to view the work (and, since it’s necessary, to make a transient copy) during the period of the subscription. You don’t get to relicense it to someone else under those terms.
Your redistribution license only applies to the part you own copyright on. Everything else is still under the original license. You can add AGPL but that doesn't take away the BSD license from the parts you didn't create.
I’m not sure it’s so easily severable in this case.
The Postgres license is already fully compatible with the AGPL. BSD/MIT is more permissive.
Wow, yeah, really hate this.
If this software was written by a mechanical process, the license is a nullity. It’s public domain.
Being created by a "mechanical process" from an existing creative work doesn't mean it's not a derivative work.
For instance, if I take a copy of $BIG_BUDGET_MOVIE, and resample the video frames from 1080p to 720p through a purely mechanical transformation, that doesn't make the output public domain.
You might be right, if it is a derivative work and not a new one. There’s some evidence that the authors consider it a new one because they’re attempting to change its license.
The BSD license doesn’t explicitly convey the right to create derivative works but it does convey the right to “modify” the software. (They seem similar but “modify” is a narrower verb.) So is this a modification? A derivative work? A new work entirely? If it shares no code with the work from which it was derived, things are more complex than it may seem (and it no longer fits the “compressed video” analogy very well).
The law and courts are not really settled here from what I can tell. I expect some day hundreds of millions $$$ will be spent on this
It wasn’t written by a mechanical process, though. It was ported (translated) from an existing creative work into a new language.
Surely you wouldn’t say a Spanish translation of a Harry Potter work is in the public domain while the original work is under copyright?
I would agree with you but for the author’s attempt to publish it under a new license. I think they can either claim it’s a new work (in which case it’s public domain) or claim it’s a derivative work (in which case I don’t think they can change the license).
I imagine a court would call it a derivative work if tested.
Things get released as GPL or AGPL that were originally BSD or MIT license all the time. The terms of the copyleft license include all the terms of the attribution licenses. Whether this is a valid thing legally I’m not sure but it doesn’t seem anyone’s challenged it. BSD code is often found in closed-source proprietary products as long as the required attributions are met and the original contributors are understood not to be held to any responsibility or warranty.
The licenses tend to say unmodified or modified copies can be redistributed in source or binary form “provided that the following conditions are met”. Relicensing the code in a way that guarantees those conditions are met has been the accepted thing to do in the community for years - whether that’s GPL/LGPL/AGPL or a proprietary license.
If you can do a Rust rewrite with AI, I can create one as well. What makes yours better than mine? Your decade long expertise in database or Rust language? Your reputation? Your proven track record to manage large, complex projects? Your time committed to the project? I don't see any of that.
I don't know why anyone would choose this over the actively (community) maintained proper Postgres project.
His project is cool. Students could use it as an example of porting code. Companies could switch to it, if it works. There are hundreds of reasons why people may want it, so it's awesome he's publishing one. Something needs to be done to get interest and then adoption.
Let me just point out:
The project is not cool. This is not a new idea, and there is nothing special.
Students won't use it as an example of porting code. I am not aware "porting code" is part of standard software engineering or computer science curriculum. That's not the kind of thing being taught in schools.
Companies won't switch to it unless their CTOs are either insane or incompetent.
Just keep using the proper Postgres project if it makes more sense to you. But notice that this isn't just a strict translation, it actually makes changes (moving from process to thread based design, for instance). Time will tell if people find these changes beneficial or if the original remains preferable.
It's also worth noting, that while you are able to use LLM's to produce your own translation, he's actually done it. There's value in actually putting in the work.
This isn't a unique situation at all. Many Postgres extensions are developed and maintained by a single person, and may therefore be avoided by more conservative users, even if they offer some technical advantages. To each, their own.
There’s no claim being made that your rewrite cannot be better.
He has provided benchmark results which provide a dimension amongst which to measure your rewrite. If you can do better by all means post your rewrite.
Finally, these kind of projects can eventually over time become projects that are actively used. Postgres is not some entity that existed before the universe was created; it was also created by someone and then eventually adoption picked up over time.
In terms of the software industry, Postgres actually is kind of that entity. It predates the entire commercial DB (and, er software) industry and is one of the first implementations of a viable relational database server, back into the 70s.
It has a 40ish year continuous history. (Which is also why it has technical/design warts like process-per-connection that we probably wouldn't do in 2026, but that's another story.)
Even with that I remember getting funny looks 20-ish years ago when I would advocate for using it instead of MySQL.
With databases, reputation is everything.
Came here to the same thing. I did something similar (but bolder, it doesn't slavishly copy Postgres and is based on current DB research papers and the like, and other bits I've been exposed to over the years). It has a full TPC-C-esque benchmark suite, replication, embedded v8/JavaScript relations/stored procedures, a giant suite of regression tests, it kicks the crap out of a lot of existing OLTP DB stuff out there. And I personally do have a background in commercial DB development.
But I choose not to publish it or promote it for many of the reasons you mention above (and more)
... For one, if "I" can do it, so can a hundred other people. And all the bold claims behind it would need to be backed up and supported and it promoted, etc, which is a whole pile of time that doesn't involve writing code.
It's the organization around a project that matters, not the code. It's not the 90s anymore w/ people piling into MySQL because it was the only option. People aren't going to be trusting your software with their data, if they can't trust you.
And unless someone is going to dump a pile of money or something on [me|them], I don't have the ability to build that organization ... as I need to feed my family... Nor am I willing to put my personal reputation on the line by putting up a huge quickly written application and then someone finding something in it I can't explain.
So like probably 500 other projects I have it sitting in a private repo.
It's a very weird time right now. "Technical" excellence isn't the important part. Organizational excellence is. This was always the case but it's more so now.
... In the meantime, if anybody has angel investment to burn, I have something potentially better/more-exciting than this guy's project but... see above...
Author of this project is not that different than you. The only difference is that he is willing to take the risk. If you have an invention and sit on it, nothing will happen.
That's not it works anymore. Inventing is cheap.
> If you can do a Rust rewrite with AI, I can create one as well.
Translations are a lot more likely to be error-free and robust, as the original data structures and algorithms have been battle tested.
I feel like we need to heavily differentiate between a rewrite and an AI rewrite.
For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.
> mostly by humans
`mostly` is doing a lot lifting here. The Go rewrite uses plenty of copilot. The reason you trust it is because you trust the people doing the rewrite.
What about human written software makes it more reliable than LLM written software?
is it the craftsmanship, or the deliberate decision making of industry veterans?
It's the notably poor quality of LLM-generated code
As opposed to the incredible code that humans are known to write...
If attention is all we need then what is better
Hours of human attention
or a few seconds of AI attention?
I am not just talking about writing the code but the brainstorming that goes into it.
Many projects that were done by humans and took a year can certainly not be trusted.
AI is a great use for this kind of boring, rote translation where precision is important. Humans are quite bad at it and tend to make mistakes. In either case the focus should be on improving testing, not trying to manually verify if the translation was correct by eye.
I have an issue with the precision of generated code.
LLMs sometimes confidently leave things out or they will overbuild.
I use them all the time but mistakes happen. It's not exactly a scalpel, more like a sledge hammer.
With programs large enough tests aren't going to ever be enough. Formal verification might work, but then who checks the specification for bugs?
I really wonder where all of these people who believe that tests perfectly encapsulate the behaviour of software come from. Maybe it's because LLMs happen to work better when you give them acceptance criteria and people struggle to distinguish between "better" and "good"?
The real test is years in production. Over time your test suite grows when bugs are found and fixed, but not every bugfix necessarily gets a test, and it's very rare that a bugfix is exhaustively tested. Relying on the test suite as a directional indicator that your vibecoded rewrite functions something like the original is probably sensible. But it isn't "done" until you've run it in production for at least as long as the original. And that's where it all falls apart, because maintenance will be a nightmare. Nobody knows how the new thing works.
In the case of rewrites, the specification is the original behavior, no? bugs and all.
Deciding if two programs do the same thing is provably impossible in the strict mathematical sense.
There is often no spec, just the old code, copied with the old bugs and with new ones sprinkled on top.
If precision is important then non deterministic AI is simply not a good tool.
Not sure it’s so simple. I think close to 100% of new ambitious projects are going to leverage AI at least to some degree. I know a couple that have strict no-AI policies (e.g. Zig), but it’s a tiny minority i think.
So how much AI usage does it make it an “AI rewrite”?
Dunno. I got rather the impression that it's ambitious single-developer projects with no intention of maintenance which leverage those 'AI' code generators the most.
Who wants to contribute to an unmaintainable code base?
> I think close to 100% of new ambitious projects are going to leverage AI at least to some degree.
Once the free money dries up that number will rapidly tend towards 0%.
> So how much AI usage does it make it an “AI rewrite”?
Any amount.
When the majority of the code is written by AI, it is more than 50%.
A human rewrite without maintenance is just a hobby project. An AI rewrite is just wasting tokens for god knows what?
rewrites feel like an area where LLMs are better suited than humans imo
It’s mostly grunt work and LLMs are well suited for translation tasks (iirc transformers arch was originally invented for translation)
It’s just a build step now.
nope. any rewrite will be an AI rewrite soon.
Is there any measurable difference in quality between the two, or are you just going on "vibes"? Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Such crude takes only cause unnecessary friction. If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary. Most of the code on the internet is already a black box to you. What percentage of code running on your machines have you vetted by who wrote it and code quality?
AI coding isn't going anywhere and will likely end up generating most code going forward so instead of rejecting it outright or arbitrarily categorizing it we need to focus on solid quantitative and qualitative measures of code and functionality regardless of who wrote it.
Didn't the initial rewrite of Bun into Rust have an ocean of "unsafe" in it, and wasn't it entirely dysfunctional?
There's still no release of rust-bun so then it might just not exist (until it proves itself).
Yesterday we learned that it’s been shipping with Claude Code since mid June, so it has a lot of active users already.
Also, the unsafe footprint seems reasonable — the bulk of it in FFI wrappers.
Yes, that was the point. It made unsafe behaviour visible in a way that could be addressed. I hadn't heard any reports of it being dysfunctional.
I have read up on it again, and while it was entirely dysfunctional at the very early stages, it quickly came up to par or beyond, with the LLM especially helped by the huge test suite written in Typescript, different from both Zig and Rust.
However, Jarred still describes a lot of unsafe, and usage of Miri in continuous integration.
Funnily enough, RAII is cited as a major benefit of rewriting from Zig to Rust, while C++ already has RAII. I wonder if C++ and Rust are more suited to larger programs than Zig, unless the architecture in Zig is handled carefully.
The LLMs may have seen larger codebases in Rust, helping them to cope better.
> Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
If the dev doesn't vet the code, it doesn't matter how good quality a dev they would be if they wrote the code - they didn't. Sure, the dev would probably drive the initial architecture discussion better and some people are using AI in small batches with tests and vetting everything, but some previously great devs are throwing in PRs that touch hundreds of files at once with one commit.
A lot of people I previously considered great developers have become people I would not recommend for a job in the past 2-3 years.
> If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary.
Sure, but this is just begging the question. If nobody could tell, the term 'slop' wouldn't have become so popular.
You must be replying to a different comment. Seems completely unrelated to what I wrote. I never claimed that there wasn't AI slop. My point is that there are different levels of code coming out of AI, both due to the quality of the model and harness, and the quality of the engineer that is driving it. Thus you can't just bucket all AI developed code the same.
100% there is slop created by humans and really solid code bases generated by AI driven by a meticulous developer. You are making the exact error I was addressing, which is bucketing all AI code as the same.
I quote-replied to your comment, so I doubt it was unrelated.
> I never claimed that there wasn't AI slop
No, but you implied that a top tier dev doesn't produce slop when using AI.
> If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box
My point was that "if" is doing a lot of heavy lifting here and you're coming very close to begging the question.
> bucketing all AI code as the same.
Most people are not "top tier devs" and over time this will probably become more true. Even if I accepted your premise that "top tier devs" only generate solid code bases with AI, the ease of entry and the ease of spitting out thousands of lines of code means the ratio of bad AI to good AI will not go in a good direction unless it becomes too expensive for non "top tier devs" to use. Given this, I think it's fair to assume AI code is low quality until proven otherwise.
Yes most people are not top tier devs and most code is slop whether written by AI or not. I've probably dug through tens of thousands of code bases in my over 30 year career as a software engineer and most are slop.
I also did not claim that all "top tier devs" would always produce better code with AI, but the qualification for a "top tier dev" in this case would be someone who verifies code multiple ways to make sure it is correct. I've seen amazing code come from bad interns that was reviewed mercilessly by season devs, and there's absolutely no reason it would not be the same with AI generated code.
You do realize that you can review the entire architecture and code line for line even if it's AI generated right? My black box comment did not mean you couldn't see the code, it meant you don't know whether a machine wrote it or not.
You've dug through tens of thousands of code bases? 30 years would give you ~10,950 days, so you'd have to be digging into 2 code bases per day, every single day without any breaks for 30 years straight, to get to "tens of thousands".
When I read things like this it makes it very hard to give any credence to the rest of your pro-AI arguments, because it just seems incredibly likely that you're a bullshitter.
> Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Aren't you making a strawman argument ? AFAIK this project is not made by an official PostgreSQL core developer, so the entire premise of your argument is invalid.
I phrased that improperly which made you and probably others misunderstand. What I meant is, is the quality of AI generated code correlated with the developer? The answer is yes, a bad dev will absolutely produce worse code using AI than a good developer - the point being that there isn't just one level of quality of code coming out of AI, even with the same model and harness.
It's not that... It's a rewrite by project maintainers vs a fork.
We already have a well established term for AI rewrites.
I agree but I think from Bun we learned that a project with really good tests and enough tokens can be converted from one language to another quite good!
It is more and more the future. No human would want to rewrite one technology to another because it is too marginal a gain. AI on the other hand does not give a shit.
You underestimate what people are willing to do just for fun.
Yeah like what do they think the people porting doom to everything possible are thinking?
> No human would want to rewrite one technology to another
Except for when they do, like the new TypeScript...
That was before good end to end models though, they started it in 2024 where it was in 2025 that models were capable of long term continuous work.
I'd %100 prefer an opus 4.8 rewrite over %99 of the time. Unless Fabrice Bellard is rewriting the stuff I need, I'd prefer AI over a human coder.
Or, you know, you can use Postgres. It's right there for you.
why? if a rewrite is better/faster/secure, why not? (I'm not saying PGrust is better, I didnt even install it, my perspective is in general)
Of course, if the rewrite is all of those things, then it seems like a good option. The whole point is how do we know if it actually is better.
Because it is not trustworthy, which is really the most important thing.
AI is an average coder.
It was trained on all code the code that could be found.
Not just code written by genius programmers like Carmack and Bellard.
Given that it's average, I'd prefer a human coder above average :)
I dont think Opus 4.8 is an average coder, with my own experience (I have coded 20 + years before even llms existed) it is anything but average. I don't think training data alone determines the success of these models, there are lots of reinforncement learning principles and fine tuning takes place, a crappy code in the dataset doesnt hold those llms scoring high in benchmarks, I dont think an average programmer can score 70% (opus 4.8) in SWE Bench Pro, which is a good one.
I think Opus 8.4 is a below average developer, but maybe I have just worked with good developers and have a skewed perspective of what the average is.
I would say it's an average coder when it comes to writing functions because it keeps using regex. It might pass a benchmark but doesn't pass the smell test.
LLMs learn a distribution during pre-training, not only an average.
Then, by giving them context or by post-training, you can make them sample non-average parts of the distribution they learned.
> Then, by giving them context or by post-training, you can make them sample non-average parts of the distribution they learned.
How do you derive that something is "below average" or "average" or "above average"?
Well, it’s up to the user or post-trainer of the LLM what they believe to be above average. Then they can design around that.
In the case of real world LLMs and post-training, what is above average is defined roughly as: labeled good by expert humans, and scoring high on RL environments related to coding like debugging, passing tests, or running efficiently and verifiably correctly.
> How do you derive that something is "below average" or "average" or "above average"?
One technique is RLHF: have an human expert assess it.
Mhm, I just wonder how many samples they get and how much time they have to come to the conclusion.
Like a short example is easier to grade, but not in the same ballpark as a whole codebase.
>How do you derive that something is "below average" or "average" or "above average"?
How do you? I mean, that was your point basis.
Which you will necessarily have if they’ve completed a Rust rewrite.
That is not how it works. IF it was condemned to be average models wouldn't be constantly improving, given that humans aren't getting better.
You haven't been using AI extensively I presume...
I've been programming a long time and considered myself among the top in my domain and AI agents using like GPT 5.5 etc. are much better than me.
> You haven't been using AI extensively I presume...
Ex falso quodlibet
> I've been programming a long time and considered myself among the top in my domain
I am not trying to attack you, but you considered yourself that... I don't know whether you actually were and frankly I don't care.
Porting perfectly working C code that has been in production for decades (and has a great track record wrt to security fixes) to rust is more or less an obfuscation challenge.
An academic exercise for sure. The Postgres team won't use this and take it forward, hence it will go stale and rot within months.
Or someone who likes it could run codex-get -y upgrade on it once every two weeks and it'll be fine for as long as you can afford the tokens.
Yet nobody is going to use it in production.
So what's the point?
As I said, it's an academic purpose I guess. Unless the Postgres team takes this and "owns" it accountability-wise, it's dead.
Chuckled at the anti-climactic sentence ending! lol
Given that you reuse Postgres' tests and LLM have been clearly trained on Postgres' three decades of contributions, this may constitute a license violation. Unless you include the original license of course. From a human level I also understand how you ended up with a less permissive license.
License for test might be subject to debate, because afaik the project is merely running them.
but yes the test files should be presented under their original licence.
I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.
> not mostly the test, but the real world production scars
Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some guarantee of equivalent diligence.)
And yes, years and years of running.
Sure, but behaviors that never have a bug or regression don't get a test. Software of this kind of complexity has all kinds of behavior that has never been broken, and doesn't have a specific test written for it.
Getting an extensive test suite passing is certainly orders of magnitude better than having no test suite at all, but it still doesn't tell you as much as you need to know. I would absolutely never trust an LLM Postgres rewrite (in any language) in production based on "only" Postgres's test suite passing.
> Software of this kind of complexity has all kinds of behavior that has never been broken
This space of things is astronomically larger than the space of things expressly covered by any test suite.
"Program testing can be used to show the presence of bugs, but never to show their absence." -Edsger W. Dijkstra
I've also seen situations where a customer reports a bug, the fix breaks some regression, and the updated behavior to work around the fix breaking the regressions turns into an undocumented feature.
How do you break a regression? A regression is breakage. Are you one of the people who use "regression" to mean "regression test"? Did Codex learn this from you? I hate it.
Yeah, sorry, it was a codebase with _only_ regression tests dating to the 80s, so we called everything a "regression".
The same basically holds for proofs in the absence of coherent global correctness criteria like, say, confluence and normalization for a lambda calculus, or soundness and completeness for a logic.
Fable's napkin estimate of the effort required to produce a passable reference semantics for Postgres, which would involve novel discoveries in denotational semantics of concurrent transactions and so on, might be in the ballpark of 30–60 years of PhD level work.
So realistically I think the only way to validate a Postgres implementation involves differential testing, fuzzing, acceptance test suites, etc. And still you'll have bugs that need to be hammered out the good old fashioned way.
Or even a human rewrite merely because some language is the current fad. A rewrite in a different language should be done for very good reasons, to solve problems that are bigger than the costs of all the bugs that will be introduced.
Perhaps before embarking on one of these rewrites the first step should be a heavy round of mutation testing and property based testing. Contribute any new testing code from this back to the original project. And *then* embark on the rewrite.
If that's your concern, then your argument becomes "software should never change". Why dare patch any bug ever? It might be load-bearing in some unknown, undocumented, unsupported workflow somewhere in the world. No test imaginable can catch that apart from the scream test.
There are reasonable arguments against language ports, but this is not one. You're making an argument against code changing at all ever.
Agreed.And a rewrite in another language creates a high probability of a change in behaviour
The maintainers that wrote those tests will have experience you won't get out of a rewrite.
I think this is also where the real work is. A rewrite is one thing, that you can show off with a flashy blogpost. The maintenance, for years to come, won't be of that nature yet it still requires as much work.
This feels like the image of the plane that returns from battle with bullet holes, and the engineer being asked to path up where the holes to make it stronger. Only to be told to patch where there weren't holes as those planes didn't make it home.
While not an exact fit of an analogy, those tests patch what was a problem with Postgres in the wild. What it doesn't cover are the things that worked in Postgres without tests, but may fail in port and go undetected.
I don't necessarily disagree, but two other points to consider:
1. Every test that is written is another use case that wasn't tested before. 100% test coverage is often impractical, but the more tests you have the more of the code you can be confident about.
2. Every test you add is another regression that can't happen in the future; if you test the index rebuilding code and validate the output then you know that you aren't going to make a change that breaks the index rebuilding code. If you have a legitimate change you update the tests, but if you're not expecting the change then you know there's a bug somewhere.
Anything that doesn't have tests is unspecified behaviour. While it is true that a port may differ in behaviour where the behaviour is unspecified, "fail" is not the right framing as there is no definition of what it should do.
sqlite is the pathological case though; it has ~590 times more SLOC in the test suite than in the actual sqlite project.
https://sqlite.org/testing.html
> Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
If you can be 100% guaranteed that there indeed is a test for every occurred bug. Sometimes maintainers are not so strict about it.
And some programmers are so good that some issues are self-explanatory and they write good code to note a thing but don't write a test, because implementing the test is more expensive.
> And some programmers are so good that some issues are self-explanatory and they write good code to note a thing but don't write a test, because implementing the test is more expensive.
You don't write a test (just) to verify that your change fixed the issue, but to ensure it doesn't regress in the future after an unrelated refactor.
a code written to pass a test can surface unintended new bugs.
very naive. the runtime behavior of a rewrite should be significantly different in all kinds of unpredictable ways nobody see coming or might expect. It is a combination of language semantics, compiler behavior, operating system behavior, file system behavior, driver behavior, ..
yea, I found LLMs poor at reasoning through system-wide behaviors. Good luck to the vibers when they try to rootcause a rare data-corruption problem. They will hand it off to fable which will attempt trial-and-error bug fixes which will eventually muck up the code. What happens then ?
One issue is those are the bugs you get when you write it in C++.
They aren't the bugs you get when you write it in Rust.
The kind of bugs you get are usually a function of the problem, language, implementation approach.
So you get other bugs when rewriting in another language without existing tests, got it. This is why I hate all the announcements of "it is rewritten in rust so it is obviously better than the original since it passes all the tests". Edit: and it's an LLM rewrite. Add that to the pile of over hyped messaging.
Unfortunately, too many people are getting captured by marketing and are divorcing themselves from reality. A rewrite can be an improvement, even if in the same or any other language.
But, there are also levels, in terms of quality and human code review, when dealing with rewrites. New bugs can be introduced or there can be style issues, that can take time to fully reveal themselves, and particularly if the person or people involved are not familiar with the other language.
having a large test suite does not equal to testing every potential edge scenario that has never broken before in production.
So many comments here talking about the downsides. The only reason to do a rewrite is because there are massive upsides. Maybe the implicit point is that the upside (memory safety must be the biggest), isn't worth the downside (lots of bugs to be figured out before you trust it).
I find a lot of HN discussions quickly turn into thought experiments and philosophical debates that largely forget the original topic. For the most part, I find this idiosyncrasy charming and entertaining, but it does frequently result in forests being missed for the trees.
I agree. I also agree with the sibling reply that -
> every time you have a bug or a regression, you write a test that confirms correct behaviour.
What I fail to see in these rewrites however is - what about new bugs introduced by virtue of this rewrite? I mean it'll have to go through its own challenges in real-world scenarios, right?
> I start to see a lot of these re-writes that depend on tests to state that its working.
There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now
Not sure if there's tooling for that though. If there is, just give it to Claude so they will incorporate it in their development loop
> Just run both pgrust and postgres and compare the output.
The space of inputs and outputs is infinite. You can't prove programs are the same by "just" testing a bunch inputs.
Indeed. This approach is an improvement / augmentation over tests, not a panacea.
Neither postgres nor pgrust have their behavior specified using formal methods. (pgrust could write some contracts using something like kani or creusot, but having upstream postgres also write contracts is a tougher sell). If they had, one could write a giant proof that said the two software essentially do the same thing (at least in a subset of environments and some simplifying assumptions)
> Not sure if there's tooling for that though.
I can recommend proptest. What you're describing is a common pattern in property-based testing which basically boils down to "comparing against an oracle". In this case, postgres would be the oracle, pgrust is the system under test, and the idea is to generate strategies comprised of sequences of valid (and invalid) SQL statements and ensure the system under test behaves the same as the oracle in every case.
(I'm working with malisper on this) we built this too and are using it for a new version we're working on right now
Oh, that's cool!
Not weighing in on this specific rewrite but tests are how you specify that your software works correctly. If a behavior isn’t covered by an automated test in some form you can’t assert that any given change doesn’t break it.
I think it is completely reasonable to use a preexisting unmodified test suite to state that something is working. The larger the project the more true this becomes. Real world production scars are documented and guarded against in the test suite otherwise those lessons get lost.
Also SQLite is legendary for its massive test suite and extensive fuzzing. They have 590x the amount of test code and scripts than normal code. Source: https://sqlite.org/testing.html
But they aren't all open, so your llm rewrite/copyright eraser won't be taking advantage of them all.
Great point. Stated another way:
"Mom, can I have battle-tested, reliable software"
"We have battle-tested, reliable software at home"
Battle-tested, reliable software at home: (Pic of green text from `cargo test`)
> That's where the reliability comes from
So, we should make it easier to feed that reliability back upstream.
Probably the most useful thing you can do with these LLM-transpilations for now: If the transpiled version passes all original tests, I can run my application test suite against it and use it to discover test coverage deficiencies in the original!
If it crashes or otherwise observably misbehaves, I know the real project was missing regression tests for something. We could make upstream so much more resilient against accidentally breaking stuff in future updates, if only it becomes safe (offline + no side effects) and easy (if it crashes/locks, it is not from some memory safety bug from 25k transactions earlier) to run these transpiled projects as one row in our everyday integration matrix.
As sibling mentioned - bugs and regressions are the thing that are (in a perfect world) usually covered.
The problem however is non-covered success cases. A visualisation of the problem: let's say universe of interaction for DB consists of 10.000 SQL queries. Over 10 years various regressions were found and 2.000 SQL queries are guarded by tests. In reference implementation remaining 8.000 never surfaced over this time and it's unclear if they will work.
And, thinking of how many various SQL queries PostgreSQL users around the world are using vs the test cases covered it's obvious that feature space isn't covered in 1% of the success ratio cases.
Now the new, test-based implementation, has to prove it can handle remaining 99%.
Completely agree with this.
The biggest lie of software engineering is that everything can be testable with tests. That a 100% test coverage is an indicator of quality software.
And also the amount of people running it in thousands of scenarios. Not sure if these areas can be even tested for, but I guess time will tell (can observe Bun if it breaks somewhere as that’s afaik the first big AI rewrite which got into prod for masses).
A lot of the signal (github, forums, mailing lists, discord, etc.) can be turned into signal. Right now it's easy enough to collect. In future it will be easy enough to cluster and generate preferences, experience, etc.
Every bug report, code change as a result, PR / commit message, PR comment that steers preferences, etc. is solid signal to generate future tests.
That's precisely what a regression test suite is for. There is a bug, you fix the bug, you add a regression test. So if the test suite is well maintained these real world production scars are reflected in the tests.
I hope you are not true at all.
Software like a Database should have an extensive test bench with concurrency tests, all corner cases etc.
I'm not here running the new version on production to tell the maintainer/devs that my 'production unit tests failed'.
What is this even for logic?
I mean there is balance when i write tests for my production software, but my software is used by me. If i would have a library, i would test everything.
And there was some blog post about another database system were they even virtualized the File access to test cases like when the disk controller stops working.
"Everybody has a production system. The lucky ones also have a test system."
Wait - does the AI rewrite the tests too? If so, lol.
The test suite is the result of these years of years of running in production. Every time you fix a bug, you add a non-regression test to ensure you don’t break it again.
In a project like PostgreSQL, those scars are reflected in unit tests demonstrating that they’re fixed. It’d be hard to pass its test suite and not be as robust as the original.
> It’d be hard to pass its test suite and not be as robust as the original.
This is not true, even in principle, even for Postgres itself. You'd be right to say that it'd be hard to pass the test suite and not be robust at all to some extent. But even in Postgres, I bet that you can quite easily introduce a change that will pass the whole test suite but reduce robustness compared to the latest release (for a somewhat silly example, add a call to `exit()` on a timer that's longer than the longest duration test in the suite - that will significantly reduce robustness while still passing the entire test suite).
Sure but these scars/tests are from the original implementation. Just because it doesn't have issues there doesn't mean it didn't bring its own set of issues
This is all well and good in theory, but the number of times I've seen tests that don't actually test what they say they're testing is hard to count. Yes even when you encourage the developers to ensure the test fails first and do TDD. Tests help you ship with confidence but there's usually at least a few that are just passing by pure luck.
So no, I wouldn't judge a rewrite as being equal just because it passes the tests. That said, I don't think that means you shouldn't do it. You just have to be pragmatic about it.
Passing a regression test suite only proves that those particular regressions aren't present. It proves nothing about robustness beyond that.
I dunno...I can envision something vibecoded prioritizing passing test suites producing something that does that, but isn't even functional in real-world production. Sort of like in the pre-AI world, where someone claims 'standards compliance' by way of passing compliance test suites, but can't actually interoperate well with other implementations of the standard. YMMV.
Unit tests aren't useful for rewrites, only integration tests are. So there may be missing coverage. Also many things are simply difficult to test (eg performance under very specific conditions)
PostgreSQL's tests are mostly integration tests.
They ought to, but are they? In https://wiki.postgresql.org/wiki/Developer_FAQ I don't see a requirement to provide a regression test for a bug fix.
It would be reasonably easy to audit and automate this...
It is not a requirement the PostgreSQL project wants to have. It would be a heavy burden and mostly pointless.
I think it's rather unfortunate that writing regression tests is seen as a heavy burden. Compared to determining the cause of a bug and the right strategy to fixing it in a maintainable way, writing such a test should be simple, no?
I'm not at ease regarding LLM generated code changes to a project with a (hopefully) long expected life time, but LLM generated regression tests should be less contentious. I wouldn't expect them to be maintained much; rather, if they don't perform as intended against a future build, just have them recreated.
If you examine what you're saying here and slippery slope it a little, examining past effort for misses and correcting them at scale is not worth doing? There are many ways that the second part of your perspective are incorrect (burden / lack of point). I bet that a coding agent could in an automated fasion find at least reasonable additions that you'd say add value and reduce potential for error long term that you'd find valuable. They've probably already done so (I don't know postgres dev at all, so just supposition here. They will 100% do so in the future.
You immply that a testcase exists for every weird edge case. Especially filesystem and concurrency is things you can barely build test cases for.
Even a 100% test coversge is far away from verifying all behaviour.
Edsger W. Dijkstra:
"Program testing can be used to show the presence of bugs, but never to show their absence!"
That's not relevant though. All concerns are secondary to security and Rust is the only language with security GUARANTEES. No other language is as secure. Therefore, even the worst Rust rewrite is automatically better than the best work in any other language, because it is the only one with guaranteed security.
If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation, then you are simply not giving security the priority it both demands and deserves, and that makes you disastrously insecure. This is a serious issue that should be given all priority. There is no room for debate. Your only policies should be security before all else and compliance with those policies must be absolute and without deviation, or all is lost.
Pro: Database is perfectly secure.
Con: The database no longer exists.
Also, there is more to security than memory errors. SQL injection, authentication, and access rules matter. It doesn’t matter if Rust database is secure to bad data if it lets anyone in to do anything. Or if it is crashing all the time or corrupting your data.
What if the rust rewrite uses "unsafe" on every line?
> If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation
This is silly.
Rust is awesome, and it's hard to argue against in many domains. However, software is more than the language it is written in or the runtime serving it. Is the Rust rewrite fully compatible? Is it supported by a strong community? Is it likely to continue to be supported? Is its release cadence sensible? Is its licence compatible with your intended usage?
There are many questions needing to be answered before making rash decisions based purely on tech.
None of those concerns approach the level of priority that must be assigned to security. On defense, your security must be perfect forever or you are absolutely defeated. None of us are on the red team. It's not a rash decision, it's the only decision that logic allows. If you are not secure, you are NOTHING.
I strongly disagree. The easiest way to shut down your business is to insist on being 100% secure because the only way to have perfect security is to do nothing.
Security is always about tradeoffs.
Why so much negativity? I find these projects interesting for learning purposes and exploring new ways. What’s wrong with that?
Possibly:
1. Piggybacking established brand names (Postgres + Rust)
2. … without practicality nor advancement (e.g. this solves no extra problems)
3. … without trust (i.e. LLM-driven rewrite, with no capabilities to thoroughly review it)
I think people get easily upset when the title has high-signal names like Postgres, and the title touts it somehow, yet it’s obviously impractical for obvious reasons (short-/long-term practicality, social trust & network effect, etc)
Because it’s uncomfortable to see decades of work copied so trivially.
But that's the thing, without the decades of work, it wouldn't BE trivial.
Everyone is standing on the shoulders of those which came before. If LLMs allow us to combine the incredible decades of effort and knowledge and experiences that's gone into building something as great as Postgres, and take that and combine the experience and philosophy that has led to the creation of a language that potentially provides tangible benefits, and for far less human time and effort that it would have otherwise taken...surely something that should be celebrated as absolutely incredible?
But who is getting celebrated? The people who spent a lot of time on the original thing, or the AI rewrite that everyone now uses?
The people benefit from all human knowledge, future human beings.
if only we lived in such a world
I mean, all of it? Maybe someone born post rewrite will think wow AI made a cool thing but nobody is going whoa Postgres is made purely by AI?
I can trivially copy any code even without an LLM though with a simple tool called rsync!
They have the source right in front of them.
Imagine the feelings of a dude who used to code in assembly and then some punk writes in c++ and uses gcc... decades of work wasted.
That is a completely different thing than LLM generated code.
They typically respect software licenses and thus comply with the original software's wishes.
LLM companies steal the original work and LLM users dont even know where the replicated code was lifted from or how it was licensed.
Agreed, the negativity here is quite wild.
There are new power tools for our craft. People are experimenting and having fun with said power tools, and have interesting results that may be transferrable to $YOUR_PROJECT.
Doing things just because we can is a great reason for hacking around.
Kudos for the author for answering questions and keeping up resilience - HN crowd is not what it used to be (shakes fist at a different cloud).
Because it's a waste of token.
I'd love to be proven wrong, but chances are that nobody will use this in production, people will completely forget about the project in 6 months, and the project will be archived not long after that.
This is not the first one of similar projects.
can you enlighten me, what exactly do you learn from asking a llm to do a rewrite?
well, it accelerates icebergs meltdown. so it helps...
I mean you can learn a lot. You can learn what's possible with less effort to build a proof of concept. It's kind of like you had another engineer do it for you, you don't completely learn how to do it yourself but you can still learn a lot with much less effort
You would learn way more by asking an LLM how postgres works. Nobody is even reading this Rust code. There are 7000 commits over 2 weeks. What is being learned from that?
You learn that you can build a version of Postgres that passes the tests and improves on these benchmarks in rust. You gain access to a codebase that does it which you can learn a lot from too. Definitely not zero like someone else said
exactly zero knowledge gained. all former expert understanding is buried down in slop so it is inaccessible and will be completely inaccessible once slop-machine stops
No, I can’t. The way you frame your question tells me you’re not seeking enlightenment.
It's a very fair and generous way to frame the question, considering you like seeing these rewrites for "learning".
I am concerned about the quality. Even a cursory skim of the code makes the code appear asinine. Unless the genius aspects of the code elude me.
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
This is a (slightly more typesafe) transliteration of the C code.
https://github.com/postgres/postgres/blob/2e6578292a9184dcaa...
Yeah same. The structure makes no real sense and when digging into the code it reads like I'm the first human to look at it.
That's how Ai generated code is. I am almost convinced that Models are intentionally taught to write obtuse code because AI companies don't want us to write code at all
I'm too young but I imagine assembly programmers were feeling the same when automatic code generation by compilers took over. Very weird.
More that I got confused by the C function returning bool, not as an error value, but as a result, which is my fault for skimming it quickly.
I have taken a closer look at the code, and it seems superficially a somewhat faithful rewrite, not quite idiomatic Rust, but closer than I anticipated at first. I know there are non-LLM rewriting tools for C to Rust, and with a test suite to help, a rewrite to Rust might be greatly helped. The new Rust code does have some drawbacks in some ways, and there are topics I am curious about.
Wow, I would love to read an interview series based on this!!
I guess there also were macro-assemblers before C, so it was a bit more natural.
People feel threatened by LLMs doing things well that they feel should require their skills and talent.
That's understandable but it's still a bit of a negative emotion that probably isn't very productive. Or very rational. This thread is full of people trying to argue that this can't be any good, shouldn't be any good, and is clearly going to end in tears. And obviously this thing passing tens of thousands of carefully curated tests that accumulated over decades suggests otherwise. It's hard to argue against that.
This probably is going to have some new issues. But it's an impressive achievement.
Irrational fear… that's why we all collectively ditched gcc and moved to that llm rewrite made in rust right?
The only one using feelings rather than reason here is you.
I can't see the achievement here.
100% of the tests passing, on track to be faster and more scalable. That's not a trivial achievement.
I don't really understand how "written by AI" and "for learning purposes" can ever be compatible. What exactly does one learn from typing "Rewrite this in Rust, make no mistakes" into a terminal?
How much token this would burn is of interest I suppose
Regression tests start to play a different role with LLMs.
On one hand, they give an LLM a short feedback loop to correct itself, and iterate fast when writing code. A human also uses it as a feedback loop, but we don't iterate as fast and don't handle big walls of conditions, so its effect is not as big.
On the other hand, LLM's ability to handle a big wall of if-conditions can backfire if it starts taking shortcuts and taking the tests-as-a-spec too literally, overfitting the solution, overly focusing on the given datapoints (conditions checked by tests) and missing the overall behavior shape that the tests intend to pin down. For humans, this is less of a concern because we are bad at big walls of if-conditions, and we'd rather try to see the original shape that the tests are pinning down than monkey-patch the solution to fit the individual points.
It's interesting to see how one balanced these two. In this case particularly. Maybe you could play around with separating the data you give an LLM into "training set" and "validation set", training set can be seen fully, but validation set is hidden and is only queried when the solution is deemed ready. Say, training set = original source code + half of the tests; LLM uses that for quick feedback loop. And validation set = the remaining half of the tests; test code is not shown to the LLM and run only when the LLM says it's done to catch potential overfitting of the resulting solution over training set.
To me, the credibility of a solution like that would depend on what methodology the authors used. If they just let the LLM see all tests, I'd be skeptical (albeit unable to point out specific bugs due to the volume of work and LLM's ability to make bad things look trustworthy). The good thing is, real-life use will add new, unseen before datapoints for testing — so validation set will build up with time. Really curious to see how it will work.
Property testing and deterministic simulation seem like good alternatives.
Rust and its ecosystem needs to become more original. There are so many new problems that needs software solutions. Existing solutions that already work don't have to be rewritten in Rust.
A lot of it is actually GPL-washing and rust is the excuse.
I'm on the rewrite it in rust bandwagon, but I secretly want to rewrite things in rust so they can be refactored and made easier to maintain and add features. So "rewrite it in rust" is just like "rewriting it in anything that I'm currently enamored with," and doing it with an LLM (defactoring?) would miss the point for me.
rust being safe(r) just makes the rewrite less risky.
A lot of the software being rewritten in Rust is not GPL to begin with, like PostgreSQL here which has its own BSD/MIT-like license: https://www.postgresql.org/about/licence/
I am not trolling, but I have a simple question: Why? Why do I use this instead of the official build? What is the business case?
(I'm working with malisper on this), we are now focusing on improving many things about postgres! Some we have written about before [0], and we have much more in mind too. Malis wrote another comment about analytical workloads being 300x faster now than postgres for a version we're working on right now
Aiming for postgres compatible database with a 2026 architecture
[0] https://malisper.me/the-four-horsemen-behind-thousands-of-po...
> Aiming for postgres compatible database with a 2026 architecture
Except you didn't improve the architecture, did you? You just asked an LLM to copy what was already there. Making real improvements to the database architecture requires understanding the database architecture, not just asking a calculator to do the work for you.
Better benchmark performance means nothing if the underlying guarantees break, and a 300x improvement sure makes me suspicious. I would look at something like this if it passes a Jepsen test, otherwise you simply will not be able to convince me that it's worth my time.
The version we have live right now is pre architecture changes, we wanted to make sure we could hit this milestone first. And agree about proving the underlying guarantees. It will be pretty exciting when we do
Wondering if its possible to backport performance improvements into upstream. Should be a big deal.
I'd say it would be most likely impossible.
I think a business case for a "look I let an LLM rewrite a large codebase" does not exist.
You are now at 0.1%... now submit upstream in sensible chunks (function or maybe file/module), waiting for people to review (a few per week, maybe) and approve/merge.
There is big appetite for PostgreSQL in business cases. But there's also a lot of problems in PG and people want to solve them. But there's ~10 people in the core of PostgreSQL who contribute to it and know how to change the core. If you have a business use case that would require changing the core, doing it in safer, less error prone technology would be way better. That's why you have other products that had to be created that talk PG protocol, but aren't using official build.
Why not? The author have their own reasons to do it. Did they ask you to use it instead of the official build? It's a github repo.
Why does there need to be a business case? They aren't selling it.
Software raidership?
It's pure virtue signaling.
"Look mum, no brains!"
Because Rust is what's cool these days. Don't you wanna be cool? Also Rust has memory safety things that C++ doesn't have, so there's a class of bugs that can't happen in the Rust version. That doesn't mean the Rust version is 100% bug free, but just that it's not vulnerable to that class of bugs. So it's a good thing for security reasons if you're running a database server somewhere that attackers could get at it. There might be performance benefits down the road if they choose to focus on that.
Well, I will give 7/10 as an FYP
Rust doesn't have ACID and I'm sure this doesn't either.
I'd like to know if the "authors" know what I'm talking about.
You think that's bad, MongoDB doesn't even have schemas. Schemas!
Oh we used it in my first job. It was really slow and the data in there was all buggy.
We really should have used postgres instead, but it was the "big data" period so you had to use mongodb.
Congrats for launching this project. I think it's awesome. I did two similar projects, just to learn how to work with LLMs and it felt good--reminded me times debugging code with GDB and going through stuff in semi-manual mode.
I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.
I suspect the future of open source will be to never publish your tests. Or someone will just pump them into an LLM like this.
SQLite already does this but hasn’t stopped the rust /go clones from popping up .
These are toy projects with no serious interest in maintaining the port long term. and even with things like bun where the port is merged it remains to be seen on maintainability over time.
I'm glad there's a go clone for SQLite, it' easier to integrate into a Go project, especially when you have to support multiple platforms.
> These are toy projects with no serious interest in maintaining the port long term.
source?
Why should a developer use this for anything beyond a pet project? Just because it is written in Rust?
All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
As someone who loves Rust the language and tool set: This class of projects [LLM rewrite of a reliable piece of software honed over decades) is embarrassing.
If you are watching this and haven't used rust: Please don't judge the language by this part of its users.
> software talibans
I will note that, very funny
Well, this approach is more similar to imposing a dogma thank engineering.
Is managing memory safely important? YES
Is managing memory safely the solution to most of the problems? Absolutely not.
Advocating the language ignoring everything else (having as first and only argument that the code was rewritten in rust fully qualify for this case) is dogma and not engineering.
Yeah I'm using that one.
We have a problem with software religious fundamentalists in our organisation and it's an apt description.
I actually had a lot of problems with software cult followers of influencer gurus like ThePrimeagen, Lex Fridman, Theo, etc... Those are so worst. You can't resonate with them.
Trick is to ignore people who follow the cult.
We went down the earlier Udi Dahan and DDD crap.
what does it mean ?
Pushy fundamentalists, I suppose.
Yes
Often the biggest blocker on moving to a new programming language, is the cost of re-writing everything.
Cue some story here on a bank or airline somewhere still relying on cobol backend servers.
These LLM conversions really seem to make modernization of large parts software layers possible!
I have some familiarity with the bank situation, and while a lot of them are on some very old systems (maybe COBOL, maybe something else, either way they want off it) the cost of actually re-writing the code is far from the most significant issue.
Consider: You have a big mainframe running your tier 1 bank. Assume that you can see all the code on it, and you can feed all that to an LLM if you like. Getting it to spit out a Rust version is not what you actually want - you now have a modern language but it's still a singleton instance, so where do you run it? Most hardware doesn't give you enough uptime for what you need here, because what you actually needed was a re-architecture for distribution / failover / whatever, and while you could ask your LLM to do that you aren't going to run your bank on the result.
> while you could ask your LLM to do that you aren't going to run your bank on the result.
Why not?
I feel like we're entering a new era of prejudice against not a category of humans, but against non-human intelligences.
The design patterns for distributed and fault-tolerant systems are well-known and established in the industry. Both humans and AIs are familiar with them!
So if you sketch a design for the AI to follow, establish the rules in AGENTS.md, have a robust test suite, use a frontier model dialed up to eleven, etc... why not rely on the LLM output?
At the end of the day, humans are not without fault either.
I've been wading through some legacy "pre-AI" code recently and it has more bugs than a rainforest! Static fields used incorrectly, causing data races. Floating point types used for money amounts. JavaScript and SQL injection up the wazoo. Wildly unsafe password handling. So on, and so forth. This is the norm for most human-written software, not the exception.
As a proof-of-concept, I tried an AI rewrite of one such legacy app[1], and it is not bug free, but it notably has fewer bugs than the original. Different bugs, sure, and I'll have to iron them out after a round or two of UAT, but I'm honestly more confident with what I got from the chatbot than the code inherited from humans.
[1] Deals with money, but admittedly at a much lower level of risk and consequence than a banking app running on a mainframe.
Because you know that the current one works. If you have a bank running on COBOL (or whatever), you've had that for 30+ years now, so while it might have bugs, you know what they are. You don't know what the LLM output is. Hence back to my original point: writing the code is not the hard bit. Making yourself (and your CEO etc) comfortable to put that into production is one of the hard bits.
How did you conduct this rewrite? Did you hand the AI some specs, some tests, the existing code?
I feel like AI has dramatically changed how complete rewrites can be considered, especially for long-lived, legacy projects.
> but it's still a singleton instance, so where do you run it? Most hardware doesn't give you enough uptime for what you need here, because what you actually needed was a re-architecture for distribution / failover / whatever, and while you could ask your LLM to do that you aren't going to run your bank on the result.
If only we had a way to solve these issues with tools capable of running Rust programs in that way. I guess every company that needs distribution / failover has a mainframe sitting in their office nowadays huh?
https://k3s.io/
https://kubernetes.io/
https://aws.amazon.com/
https://www.erlang.org/
etc.
You misunderstand.
You could run one of these things on a mainframe, because it's a zero-downtime machine - you can swap out parts of them as they run. But fundamentally it's a singleton. It is deeply naive to believe you can trivially translate that to something running on Kubernetes just "because Rust".
Of course most companies that need distribution do manage to do that, and eventually the banks will get there too. But it isn't feasible to do that by translating their existing non-distributed COBOL code, they need a fundamental re-architecture, and that is much harder.
It's not enough to do a rewrite. Someone has to maintain it. Such a huge codebase with literally zero experts is unmaintainable. There is no one who knows how the internals work.
Sure you could keep vibe coding it but I wouldn't bet my data on that. A database needs to be rock solid.
This seems to be the issue with using LLMs for any code generation. Even with my own code bases that I've written entirely by hand over years, if I use AI to implement anything, I don't go through the mental model of architecting it, so I don't know how it works. I can only imagine this to be far, far worse for large code bases maintained by a team of people who are all using AI.
That depends on the language you are using. Some language communities had already rejected "architecture astronauts" before LLMs were born, so the training data was highly consistent, which has lead to LLMs being highly consistent in their output. You know how it works because the LLM spits out the same as what you would have written yourself. It's almost eerie that they can do that.
Unfortunately that doesn't apply to all languages. LLMs are especially bad at producing code for the languages that were historically known as beginner-friendly as the training data was full of code by beginners doing what beginners do. All bets are off if you get stuck there. (Although maybe you could use an LLM to translate your code to a language that LLMs are good at!)
One problem there is that even if the code is consistent, idiomatic Rust is not the same as idiomatic C.
So either the translation produced Rust code in the "shape" of C code or the code is quite different from the C implementation.
Add to that that you need to have people proficient in Rust and also Postgres as well as the very much unknown codebase as a whole you get a recipe for pain.
> Cue some story here on a bank or airline somewhere still relying on cobol backend servers.
There's existing money and expertise in those environments to rewrite the whole thing, yet they don't. You may loan them free engineers/experts and they might still not rewrite anything.
It's a clean-cut financial decision.
The existing system works. Yes, it costs a lot to maintain, and you could definitely reduce that if you moved to a more modern system. So now you're talking payback periods. Cost of development / maintenance cost savings per year = number of years before you pay back the project.
Problem is, that the cost of the development is often unclear, and the maintenance cost savings, while definitely above zero, and often unclear, and approximated the numbers usually come to a payback period in decades.
And that's without the usual tech caveats; We can't promise there won't be bugs. We can't promise deadlines will be met. We can't promise the project will succeed at all. We can't promise existing functionality will be faithfully reproduced in the new system. The normal risks around any software dev project.
All in all, it looks really expensive and really risky compared to just doing nothing and running the same old system for another five years.
Source: I helped do some of the maths on this for a Y2K project.
OK but, Postgres is not one of those clunky "we have to replace this" systems.
> the biggest blocker on moving to a new programming language, is the cost of re-writing everything
In 2026, not sure if it was satire. Do some people truly believe that all their software stack has to be single tech, from device drivers to end user apps? Does that extend to remotely accessed services?
At the same time that was ever the only reason for moving to a new programming language: abandoning all the bad ideas and craft that had accumulated in the previous language ecosystem. Needing to rewrite everything meant starting from a clean slate, allowing the new systems to be designed for the new age, making everything in that new language feel sleek and modern and thus appealing. Of course, as time progresses even the new language starts to accumulate bad ideas and cruft, historically necessitating yet another language to offer the clean slate again.
If the code is going to be translated forward instead of abandoned and then rewritten, as is now completely viable via LLM, there is no reason to move to a new language at all.
> significant part of the rust community consists of software talibans
I seriously don't get it though. Rust is a nice language, but so is X. However we don't see X people brigading existing projects with constant bombardment with "rewritten in X". What is that about Rust that prompts this behavior?
what do you mean by that? were there people brigarding postgres to rewrite to rust? otherwise relative to popularity i do also constantly see posts on here about Project X rewritten in Go, Zig, C etc...
It's pretty ergonomic to agents. Like typescript.
Rust attracts zealots because of the various kinds of safety guarantees. The speed means it can replace more or less anything.
People see the safety as a moral superiority so it attracts obnoxious zealots.
Other languages' features and syntax aren't nearly so easy for zealots to form behind. The perception of absolute safety it puts in some people makes them crazy.
This is a good point, actually. Might well be the reason.
A more plausible explanation:
People were told for years they can't use Rust for their new projects because it hasn't been "proven" in industry yet. So the option was to sit back and wait (chicken and egg) or move to rewrite a bunch of projects so that it could actually be "proven". Not saying this is the only reason why it happens (every language has its Zealots) but it certainly makes more sense.
Due to the explosion of new programming languages over the past few decades your options are to either aggressively expand wherever possible or die out because you're not "proven".
> All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
Nailed it ! There are some folks who behave holier-than-thou just because they happen to use some language. Language missionaries if you will, and they are insufferable.
How exactly are rewriting something the equivalent of being the taliban?
Because they are blowing up old monuments as part of an attempt to enforce a hardline but nonsensical purity on other people.
what is the metaphorical blowing up here?
Except for the monument still being there, and being the main thing everyone uses still.
I think this shouldn't be taken too seriously, from what I understand it's an exploration of what's possible with today's LLMs.
You're right to talk about the trend though, because what it shows is how the cost of re-writing well covered project has completely crashed, so that in itself is a learning.
The cost of surface level rewrites has crashed. Which will probably cover 80% of cases. Caveat emptor on which side your project falls.
I have no issues recognizing that I had memory-related problems in production (I program embedded systems in C).
But most of my issues were related to concurrency and data sanification, especially when the other end of communication fails with unexpected behavior. These bugs are nastier than memory.
So, I have pointers, and I am not afraid to use them.
> Why should a developer use this for anything beyond a pet project?
If it _is_ 50% faster, then that's the reason
Obviously like any new database it's very risky to use so probably only used for niche use cases at first, but if it turns out to be just as reliable as postgres and faster then why not?
It probably is 50% faster, probably by skipping flushing data to disk.
I'll make it 100% faster, skip saving any data to disk.
Jepsen or GTFO.
These days there's little chance for a new DB to build its community using network effect. If you want this to catch on, switch to manually grinding community building ASAP gold plating the experience for a specific niche (AI can guide your priorities but will hinder your comms). Otherwise, have fun building!
I mean if it's actually just another Postgres, you don't have to worry about network effects as much because anyone could use it in place of Postgres. But sure, the more people you know using this, the safer it would feel to use it
I don’t trust AI rewrites, definitely not in 2026 — possibly never.
Code is code. It either does the job or doesn't. If there is nothing that could convince you that AI generated code is trustworthy (for whatever definition of trust) then this is an article of faith, not a rational position.
> Code is code. It either does the job or doesn't.
Surely that is not the only dimension that matters when evaluating software. Maintainability and readability, for example, are crucial for any long-lived project.
In the future, code readability will not really matter, the concept of maintainability will also change: Is it maintainable by AI, if yes, then it's maintainable, we will approach the day where we have virtually ZERO code written by human, so all the tools must be built for AIs, not humans anymore.
Perhaps in the future my grandmother will have wheels… But today radiologists still do radiology (despite AI being better at reading scans), taxi drivers still have work, and truck drivers are still able to put food on the table. It’s still 2026, time traveler, so touch grass…
“Does the job” means all of those things. Figure out what’s important, find a way to measure it (could be also be qualitative.) How does the AI generated code do? If you simply say its no good because of the way it was created, it’s not a rational decision process.
That's a straw man. My argument isn't about 'the way it was created'; it's about the point I clarified in my previous comment. If you want to discuss that, great, but calling my take irrational based on an argument I didn't make doesn't make sense (when your position is “YOLO to AI”). To reiterate — Trust is the core of my argument, and there’s no trust without human ownership and proof of work/understanding yet (possibly ever).
Devils advocate: Are they? Why? If LLM's are capable of taking absolute crap and iterating on it to achieve a purpose, then does readability really matter?
Disagree, it’s already a leap of faith to trust even human code (which is why we try to prove code, or at the very minimum create guardrails with tests). Human reasoning itself is under doubt. It’s even more of a leap of faith (>>) to trust code generated by AI reasoning (which I can only currently call “homunculus reasoning,” it’s not inferior but it’s just got the probabilistic aspect of reasoning). People who YOLO their work to Al are practicing a different form of faith. I like knowing what my code does, and making correct predictions about it. A mental model is very important. My position is very rational, yours is faith-based. This is a technology that has many cool uses, but this is not one of those uses (in 2026). Unless the author can convince me they have as good of a mental model of this rewrite as the makers of the original source code. Maybe they do, I’m sure they would run circles around me, but I need the confidence that this isn’t going to wake me up at 3am because of some bug. And the only way I know that about PG is because I trust the creators.
Neat as a pet project, but anyone thinking of using this is production is insane.
Rewriten in Rust is becoming a meme now.
Been a meme for a while now. I avoid that noise on principle.
You got me with the rewrite, you lost me with AI.
Is the 300x performance boost attributable to the threading model vs process model?
Was the code for the threading model written by hand or was it translated from the WIP threading model the human PG team is busy with as part of the 2028 roadmap?
I'd like to know on what machine and on what testbench the supposed 300x boost is achieved so that we can independently verify it. My assembler-written fork of postgres is achieving 600x boost on SELECT 1;
Setting aside the “why” that everyone is so focused on, I want to know, how? I can’t get Claude to do anything even a fraction of this complexity. How are people setting up their agents, Claude.md, etc to do such big projects? There are a lot of lessons to be learned.
> I can’t get Claude to do anything even a fraction of this complexity.
I remember writing a postgresql compatible DB with Opus 4.5, that used S3 as storage and local caching to make it speedy.
Ironically, Opus 4.5 is by todays standards is antique. If you have some knowledge about Databases, it goes a long way.
But you need to do it step by step. Getting the core to work, getting a Pratt parsing going. The whole pgwire protocol ... the data format ...
Step by step ...
With todays Models, your can probably get away with using /goal and telling it to make a postgresql compatible database, while having it run a few days.
Now, making a fun database test project and having it production ready! Big difference!
Let's hope it did not special case (overfit) for the specific tests. One of the failiure modes that in my experience takes the most effort to mitigate for.
> <something> rewrite to rust using AI sound like meme now.
https://news.ycombinator.com/item?id=48474313
X written in Rust seems like the new Hello World for LLM coding agents now.
I'm starting to get a bit of fatigue for these projects that boil down to just "I asked Claude to re-write this code into a new language that's in vogue right now!"
I really don't understand why this is needed outside of an opportunity to show how impressive LLMs can be when working within large codebases, but even then people in the comments are finding bizarre implementation choices that a human developer wouldn't make. I'll stick with Postgres and its - gasp - C implementation for now, thanks.
In this case it's justified because Rust allows safe implementation of threaded code. Current Postgres is per-process. Switching to threading yields performance improvements.
> Current Postgres is per-process. Switching to threading yields performance improvements.
Please describe in detail what you believe this means and the mechanism by which switching from processes to threads improves performance.
There are hundreds of comment chains about this already, go troll somewhere else.
If you’re going to make a confident blanket claim, be ready to back it up - and asking for clarification is not trolling, by the way. You should be ready to engage in technical conversations if you want to make technical claims.
I'm not interested in a technical conversation with you; I was answering someone else. The top voted comment thread of this page, which dominates discussion, already covers everything you'd want to know.
There is an expectation that threads are more performant than processes all else being equal. Obviously there is some work involved to make "all else be equal", like maybe threads have less isolation from each other etc but if you can mechanically convert code to use threads where it used to use processes I can see the claim standing up.
The code is so weird, people will have a hard time reading it when they need to check for correctness.
There are much better ways to write it in Rust: https://github.com/malisper/pgrust/blob/14ffab7d31a31e5ab667...
I mean, you've linked to a function performing md5 hashing. That's pretty much exactly what I would expect such code to look like.
Case in point: https://github.com/postgres/postgres/blob/master/src/common/...
It's not idiomatic implementation in Rust, just translation from C. A few Rust macros simplify implementation and cut the code in half.
This is a stunt for freshpaint.io that sells AI. No one will use this plagiarized monstrosity.
Quite a lot of projects are trying this "rewrite to a new language using LLM", both internally, or externally (like is here). For me, they confirm some (slightly controversial) takes.
1. human code reviews are dead. We don't yet know what's next. Two reasons they are dead: too much code to review, and code reviewing sucks (who wants to spend their days reviewing code?) 2. Not knowing how to review LLM code is a big barrier to adoption, but bigger regression test suites (testability/evals) is almost certainly the direction. 3. There are a lot of projects that haven't moved to more modern infra because it was too hard. Now it's much easier. Sure stuff will go wrong. Sure it all has to be tested. What's new here? 4. Programming languages for LLMs are coming. 5. Projects that don't allow AI coding will be forced to come around or fade.
Separately, bit off topic:
New projects will often have LLMs built in, so non-determinism will be inherent in the project. No amount of code review will be able to eliminate that.
I wonder how long this will be maintained for...
As long as tokens are cheap
I just want to quibble that the 100% Postgres regression tests do not test the threaded aspect of this project, and that is a pretty fundamental architecture change.
I think we will actually see some successful projects coming out of this. There are definitely people who want x old project in this new/better programming language and who are willing to put effort into maintaining it not just doing one off port.
I started my programming career by porting code from one model of TI calculator to another. It was code I could not have written from scratch myself at the time. I learned a lot about the two different versions of TI Basic that the calculators used, but I didn't learn how the program really worked. I can't even remember now if it was Tetris or the tank game that I ported. Maybe it was both? That was a boring English class...
I totally understand why porting code is fun. It's kind of like when I checked out drawing books from the library as a kid and just traced the pictures because my own attempts at drawing were so bad. It gives you a feeling of accomplishment, even though you didn't actually do anything that difficult. And you do learn some things along the way.
Doing the same with an LLM probably gives you that similar feeling of accomplishment, even though you didn't actually do that much (sorry, hate to say it that way). I wonder if you learn even less in the process. Maybe you just learn different things.
Now that I think about it, even writing some code from scratch with an LLM is not much different than doing a porting project. Someone else did the hard work of creating the original programs that the LLM was trained on, and now you (the LLM really) are just porting/restating what someone else did. I hadn't thought of that before
How is the performance compared to regular PostgreSQL?
I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?
I also suspect this will die very shortly, which is a real shame, not because it will be beneficial but because of the time and tokens needlessly spent on something that will be thrown out.
I am not considering it at all. I am simply curious if switching to Rust has any significant performance benefits or not.
As is every slop generated project. It’s the Toy Story meme irl.
https://knowyourmeme.com/memes/i-dont-want-to-play-with-you-...
Maybe it will, but having a performance comparison will be very interesting nontheless.
This seems to be a multi-phased project. First phase (completed) was the re-write in Rust. There doesn't seem to be a performance gain and no significant one should be expected. In a 2nd phase a new architecture is implemented which malisper claims to perform much better.
I wished the two phases would have been tackled in reverse order.
> I wished the two phases would have been tackled in reverse order.
Well, tackling them in reverse order would require the humans behind this to develop an actual understanding of the existing code and architecture before starting the project, instead of just asking claude to do it. So, here we are.
The version in the GitHub repo is ~8x slower than Postgres. I have a new unpublished version that is 50% faster than Postgres on transactional workloads and ~300x faster on analytical workloads.
I don’t think that’s true.
The WebAssembly demo that runs in your browser is a really neat touch: https://pgrust.com
Thank you! I'm a big fan of your writing. I wanted to make sure there was something people could try out so I could show pgrust was real and not vaporware.
Thanks! Should work well on Mobile too :)
codebase full of code that wouldn't fly in production.
I ain't no Rustacean - but 'unsafe' calls all over.
I think the cool thing about these projects is that even if test parity reaches 100%, some bugs are going to surface on the new project that don't exist on the original project.
This is usually a good example of a test case that the upstream project is not covering and can be contributed back.
Parity should be bidirectional, so definitely it is possible for both parties to benefit from it.
finally, a database that will refuse to compile your schema migration because you used an unsafe join. progress.
It is very bad metric to measure a rewrite using the original tests. It is like to say you clone a cat by mimic 600 cat images.
It’s interesting to see how llms have turned the concept of rewrite it in rust, from an impossibility for some projects (code is too large and complicated, it will take too much time) to a real possibility for even large projects.
Database & AGPL bad combination
> rewrite of already popular technology in a different language > look at commit history > "Claude xxx committed yyy ago"
I'm sorry, but what is this need to just vibe code a port of an existing technology to a different language/framework/etc.? If it's just a personal challenge then sure I guess, but this surely can't be used as a real product?
Where will this leave the c based driver which I believe is the basis for many others?
What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...
I would expect Postgres to be heavily tested with things like Valgrind and various sanitizers. I'd be surprised if there were low-hanging fruit. But also, if there is code that does something fishy with pointers, wouldn't the AI likely paper over it by adding an unsafe block in the Rust version, preserving the same fishiness? It's hard to know how hard it would try to prove that the original is broken.
C programmer have learned how to deal with memory problems and have whole suites of tools for finding them.
Is it cheaper to find them at compile time rather than runtime? Yes.
But it’s not an unsolved problem. Memory bugs are a known unknown.
except that lately we've had a ton of CVE related to memory, so in practice it's not exactly right.
Rust itself has CVEs. Which projects are you referring to?
> What would be interesting is if they found a memory unsafe bug
They will ask relevant Claude skill.md
You are exactly right. There is no freaking way there was no unsafe behavior in a code case of the size of Postgres.
In fact from a porting effort this is the first blog post I would expect. Not that the hey we successfully did it.
> The goal is to make Postgres easier to change from the inside
uh-huh, sure.
you want to show off "look what the LLM can do / look what I burned a bunch of tokens on"?
you want to brag about how your LLM-generated slop is somehow more maintainable than the original because blah blah blah Rust?
here [0] is the version history of Postgres. pick a version from the past. let's say 14.x because it's the most current that's still under active support.
have your LLM implement version parity with 14.x. show off how it passes all the tests blah blah blah.
then have it upgrade your codebase to parity with 15.x, implementing whatever new features and bugfixes that includes.
and have it generate an automated test that demonstrates upgrading an actual database from LLM-14.x to LLM-15.x and verifying there's no data loss or corruption. maybe even multiple such tests, if you're feeling fancy.
then lather, rinse and repeat with 16, 17, and 18.
and show off the diffs of each version. does the LLM rewrite a huge pile of already-working code in the process of each version upgrade? does it introduce new latent bugs in the process - the kind of things the existing test suite didn't think to explicitly test for?
"I took a static snapshot of code and converted it to another static snapshot of code" is meaningless. all you're doing is bragging about having more money than good sense.
the stability and trustworthiness of software like Postgres does not come from a one-time snapshot showing tests passing. it comes from the engineering process that produces the software and its test suite.
oh, and for shits and giggles, because this same test was so illuminating with the Bun "rewrite" into Rust, here is the file with the most unsafe blocks in the codebase:
why does a single 2000-line file have over 100 unsafe blocks?why is the parser unsafe at all?!?
0: https://en.wikipedia.org/wiki/PostgreSQL#Release_history
It's not just unsafe, it's this:
It looks as if it's building structs out of information in (mutable pointers) to other structs without an Rc in sight. Which makes sense for a C parser: you've got a table with data, so you just link to it. It's fast, and when you know you're not going to touch it, it's safe. But this doesn't make the Rust code any better than the C code.Note that the code I believe you are referring to is from the parser which was generated with c2rust. The Postgres parser is generated from yacc/bison so rather than try to rewrite it idiomatically, I did so mechanically.
Why not just link to the C code then?
Genius! Rewrite Postgres in Rust by rewriting Rust in C!
This is actually a great point I have not seen articulated. A motivation for clear architecture and good design is being able to evolve the system over time. Does LLM code have that capacity? Are LLMs themselves able to increment iteratively?
> why is the parser unsafe at all?!?
The parser was generated by c2rust. The Postgres parser is generated from yacc/bison itself so I didn't bother making it idiomatic.
Rust feels like the just right language to produce all those slops of AI and the language itself. Great to promote yourself as a productive engineer, but at the end of the day you’re just reinforcing the statement that AI and the language itself are great, not you.
I love llm coding. I don't know what I am looking at here
https://github.com/malisper/pgrust/blob/main/Cargo.lock
What is happening.
No PRs? No Make files? I understand running tests and debugging is the workflow, but where do you log things? How do you orchestrate builds? Etc.
Like most modern languages, Rust has its own build system and package manager, Cargo. Everything you're referring to relates to that, and has nothing to do with LLM coding.
Edit: saw the clarification in another comment. But, in that case the essential point seems to be "I'm not familiar with something, therefore it's suspect."
`Cargo.lock` is a lockfile machine generated by Cargo. It's similar to package-lock.json. In any case, its machine generated the old fashioned way.
I am fully aware what Cargo.lock is. What I am surprised at is how many dependencies there are.
> I love llm coding. I don't know what I am looking at here
There might be some correlation here.
I am fully aware what Cargo.lock is. What I am surprised at is how many dependencies there are.
Sorry, the wording confused me.
The project itself seems to be structued around 1.4k micro-crates[0] which I admit is a bit weird. Rust's compilation unit is the crate unlike C's per-file compilation unit, so if this was a 1:1 AI-assisted translation from the original Postgres source this might be an artifact of the translation.
[0] https://github.com/malisper/pgrust/blob/main/Cargo.toml
This is correct. We first used c2rust to translate the C into unsafe Rust. The generated Rust code had one crate per C compilation unit. We then took the unsafe unidiomatic Rust and one crate at a time converted it to safe Rust.
it's like node_modules but cooler
Did similar with S3 and that too (eventually) did well against tests (the ceph s3 ones).
...but haven't dared use it for anything meaningful yet. Still feels like there is a real world gap in confidence when it comes to vibecoded rewrites.
Been wondering whether the answer is to insert a proxy...something that effectively splits traffic to a known S3 and the rewrite and compares outcomes over time. Do that for a couple different workloads for a month or so and if it's all identical then it's probably fine...
it's a neat experiment, obviously inspired by the bun guy.
but given the author/maintainer is essentially unknown, I highly doubt this will reach it's target audience. But one thing I do agree with is that postgres is long overdue for a re-write into a memory safe language.
also any real swe with more than a few years of experience knows "100% of regression suite passing" doesn't mean anything other than a neat checkmark for C-level executives.
>use Rust plus AI-assisted programming
>pgrust is licensed under AGPL-3.0
pgrust isn't licensed at all. AI generated code isn't copyrightable. Thanks for spending the tokens I guess.
If the underlying code ends up being a completely unreadable blob that no human will ever read why not directly do a port to assembly instead of rust?
Super impressive! Remember talking with Michael about his experience with Citus at Heap and reading his blog posts on Postgres
Super cool to see him working on this now, almost 10 years later
Woah! AGPL? That's interesting. I think Postgres has shown an open source SQL server didn't need a copy-left license to develop sustainably, so I'm not entirely aure about that, but I do like the license in general.
When the software consists entirely of ~$1000 worth of Claude credits and ~40 hours of developer time prompting and curating it, literally what does it matter what license the resulting 100k LoC artifact is provided under?
Copyleft and the whole software licensing ecosystem only matter when producing that software actually requires serious human effort and dedication.
Also can the code even be copyrighted?
For my machine translation of SQLite to Go I added this to the README as to licencing:
Most of the code here is machine translated using wasm2go. As such, the original authors retain copyright and the original licenses remain in effect. Everything else is licensed under MIT-0.
The translator (wasm2go) has a licence chosen by, and a copyright notice from, me. Makes no sense for the translated code.
I do the same for translated code. It's not creative work which is a prerequisite for being copyrightable.
And avoid relying on direct LLM output for actual work to make sure I don't accidentally include some regurgitated snippet from an incompatible license.
It helps that LLMs struggle to write good, idiomatic code in my language of choice.
> Copyleft and the whole software licensing ecosystem are only applicable when producing software that actually required human effort.
Fixed that for you. Code generated by an LLM is not copyrightable (because copyright only protects human effort), so the codebase is automatically public domain and cannot be licensed at all.
They could theoretically copyright the prompts that they used, but as that's not part of the output, and the output doesn't deterministically arise from those prompts, they'd struggle to use that to back a copyright claim.
It's silly, nearby all Rust projects are rewrites of existing projects and now as you don't need to learn Rust to do rewrites, people just let the AI rewrite projects in Rust.
It's so great. One thing that I'd like to be changed in PostgreSQL, which may be done in this rewrite, is resigning from the "one connection = one process" design choice and instead handle the connections using threads/tasks within the main process.
`seams` is the new emdash
Great! Now ask it to rewrite it in CSS!
That's actually interesting - gives C developers motivation to improve postgresql. after all people could say "look, Rust makes this easier".
What is the future of this? Code is not the same as a viable open-source project with a community, contributors, advocates, users and funding, even if it's perfect code.
Even though I'm sure it won't be easy to convince the Postgres project to switch to Rust, I do think that trying would be time better spent.
Does it support the extension ecosystem? Or would extensions need to be rewritten as well?
It is theoretically possible to have a Rust port of Postgres support extensions. If you make all the relevant functions and structures ABI compatible with Postgres, extensions should work. The issue is the moment you're dealing with C pointers and C strings, pretty much all the code you have to write is unsafe.
Perhaps pure pgrx extensions would make sense as a first target?
They would need to be rewritten as there is no formal extension API. Extensions can call into almost any part of PostgreSQL.
They would need rewriting (a few are included)
What a peculiar kind of rewrite.
Rust:
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
Original:
https://github.com/postgres/postgres/blob/df293aed46e3133df3...
Usage:
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
The return type in the rewrite is both some sort of Error tagged union that supports the Try machinery in Rust; but, it also contains a boolean that apparently must be checked; or something. It seems labyrinthical and possibly broken and terrible.
I make no claim as to whether the change makes sense given that I didn't look at the callers of this function, but Result<bool> is an entirely reasonable pattern in Rust. If you want the callers to be able to distinguish between "has the subclass", "doesn't have the subclass", and "something went wrong" this is idiomatic Rust.
I wrongly guessed that the boolean in the original C code was for error handling when I skimmed it, but instead it is just a result value, while elog() and related macros/functions are used for general error handling in the C version. I agree that it makes sense in Rust and other languages with tagged unions.
Though often when applicable, a simple tagged union is used instead when that would document the intention better. Like, the Rust version of search_pg_class_full_form::call() returns a Some for cache hit and None for cache miss as far as I can skim, and that group of methods returning that could arguably have returned a basic enum instead with CacheHit(value) and CacheMiss. Though this is a nitpick on my part.
It is a feature in Rust, not a bug :-) (I know you didn't say it is a bug.)
The error-tagged union is PgResult<bool> - which means it contains bool as the result if things go well. (The other part in the union is of course the error.)
In the original function also, it is returning a boolean: "bool has_subclass".
So anyway you have to check for the boolean as part of the logic. That is what it is doing.
Yes, but the original boolean seems to have been used for error handling, and the tagged union is also used for error handling. Why have both simultaneously in the same function instead of just one of the two?
Edit: Looking at the code again, perhaps I was mistaken, since the boolean might not have been for error handling, just the result of the function, and C's limitations regarding error handling led it to using something like elog(), apparently a macro defined in https://github.com/postgres/postgres/blob/master/src/include... .
First bun, now it's PG's turn again, although this isn't official.
I have a feeling that AI is rewriting everything!
And they'll reproduce virtually the same problems, because code was never a primary issue for output. Products reflect organizational problems, which AI can't solve.
Till its used in prod for few years and polished, I wont touch that.
Too many things tests wont catch.
Is this another llm-driven rewrite?
I wonder how many "unsafe" blocks are in there...
From what I skimmed manually, not that many, but the code itself seems labyrinthical. Like, why have both Rust Try-supporting Error-like tagged union, but also booleans, for error handling, in the same function?
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
I'm not sure what you mean? The rust code you're showing mimics the Postgres code: https://github.com/postgres/postgres/blob/2e6578292a9184dcaa...
The boolean being returned is the return value of the function. It's not used to return an error.
Now that I have taken a closer look, the code looks significantly better than it seemed at first glance, though there are still peculiarities, and some drawbacks.
An unfortunate aspect is that the code has become a bit more bloated in some regards due to usage of Result, instead of an implicit elog() macro and similar. Passing Result around, in some ways as an alternative to an unwinding exception, is cleaner in some ways, but it also bloats the code somewhat.
The rewrite also could have simpler code in some cases, like
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
could perhaps just be
match syscache_seams::search_pg_class_full_form::call(ctx.mcx(), relationId)? {
but that is a smaller thing.I see a lot of MemoryContext. I am not sure how much that bloats the code (though the C code is bloated due to C's issues and problems, like re-using collections and such). Does it incur an overhead?
> The rewrite also could have simpler code in some cases
The Rust code is a literal translation of the Postgres code which returns the value at the end instead of an early return.
> I see a lot of MemoryContext
MemoryContext in C is used for multiple reasons: 1) performance 2) keeping track of how much memory has been allocated and where and 3) preventing memory leaks.
Reasons 1 and 2 are still relevant for Rust. The challenge is in C memory contexts are stored in a global variable. Global variables don't work well with the rust borrow checker so I opted for passing memory contexts as function arguments instead.
Sorry, I wrongly assumed in the C code when I skimmed it that the boolean was for error handling, not the result value. The elog() macro is used for error handling.
Now which one is safer? A new Postgres written in Rust, or the original real world tested Postgres?
Also, are they calling it Postgrust?
That made me laugh.
This thread is enumerating all the same talking points of both sides.
Rust is a stripper
related? https://www.youtube.com/watch?v=e35AQK014tI
but did they change the process-per-connection model? if not, wtf??
Yes, see top comment.
Rewrites in Rust are kinda impressive. This language with its move semantics and close ownership tracking is very different from every other language. To create a rewrite in it, you have to rearchitect the code. There is not as much freedom there when it comes to where to keep what and where you can pass what as it is in other languages.
I don’t understand these rewrites, honestly, what is the point? Who have had any C++ related issues while working with Postgres?
Now do Freetype and libtiff/libpng/etc.
I have privately wondered for years, pre-AI, why Apple hadn’t paid some engineers to go off and write some comprehensive test suites and then port these to Swift. It would shut down entire swaths of memory safety bugs they have been coping with for literally decades. SO MANY of the zeroclick iOS exploits can be traced to a few fragile and vulnerable foss libraries, xkcd 2347 style.
The great Jarred Sumner pulled it off with bun, whether it can be pulled of with Postgres is an open question..
DST systems such as Antithesis can definitely help.
We had one for SQLite (which is SQL-ite btw, not SQ-Lite which doesn't make any sense) via Turso, no wonder we see the same for Postgres. Personally I do want to see libraries be in as much memory safe languages as possible.
How do you know it's not SQL-lite with the single L serving a double role?
Common pronunciations allow you to stay perfectly ambiguous about where the L goes, which aligns quite well with the name as spelled. If you do it right, nobody can tell if you're saying sequel-ite or sequel-lite or seque-lite on the one hand, or S-Q-L-ite or S-Q-L-lite or S-Q-lite on the other.
AFAIK there is no official word on how the name is intended to be read or said.
Richard Hipp says he doesn't care how anybody pronunces it. That said, he pronounces it "S-Q-L-ite".
Because the creator himself said it's an -ite suffix similar to minerals like bauxite, not -lite.
Interesting, thanks for mentioning that. I've always wondered about the origins of the name, never found anything, but now with your mention of "mineral" I was able to find this:
> (Hipp) How do I pronounce the name of the product? I say S-Q-L-ite, like a mineral.
> But I also hear a lot people say, "Sequel lite and SQL lite." You know, I don't care. Whatever comes off of your tongue easily is fine with me.
> (Q) But the official correct way is S-Q-L-ite?
> (Hipp) Yes, like a mineral.
https://www.listennotes.com/podcasts/the-changelog/why-sqlit...
So, he means SQL-ite, but doesn't want to proscribe this as the only way people should say it. I like all of that.
Maybe we should follow his example.