Show HN: Zedis – A Redis clone I'm writing in Zig

(github.com)

154 points | by barddoo 3 days ago ago

109 comments

  • destroycom 2 days ago ago

    > Written in Zig for optimal performance and memory safety

    Ironic to see this, as even a cursory glance over the code shows a lot of memory bugs, such as dangling pointers, data races, some potential bugs like double free, etc.

    It is good that the OP is learning things, but I would caution against relying on LLMs and taking on a bigger project like this, before the basics are well understood.

    Personally, this is one of the reasons I dislike the LLM hype, people are enabled to produce much more code, code that they aren't qualified to support or even understand.

    While the project linked is clearly designated as strictly for "learning purposes", the applications we will get in the large will be of no better quality.

    The difference being, before LLMs, those, who didn't have qualification, wouldn't even approach problems like this, now they can vibecode something that works on a lucky run, but is otherwise completely inadequate.

    • glouwbug a day ago ago

      I think this is the most apt summary of LLMs I’ve seen to date

  • k_bx 3 days ago ago

    Sorry if somewhat off-topic, but might help with getting more users. Redis is actually good enough and easy to install for most projects, but it has downside that it's "memory-only" (needs to fit in ram), and if you're in a tight container/vm – you'd better not rely on it solely.

    So, there's another project – called kvrocks https://github.com/apache/kvrocks , which is for people which don't need in-memory perf of redis but would like to use its protocol and be still very performant. However, its devs never packaged a release in deb and other formats.

    So, if you were to implement something similar ^ which will implement the protocol, have ability to grow on disk, and be available in distro repos (ideally) – would definitely hold a niche of many little projects which benefit from redis-like thing I've done.

    • Szpadel 3 days ago ago

      Redis is for high frequency cache, disk latencies would slow it too much. Maybe putting something that would normally be evicted to disk could be valid compromise.

      another major drawbacks of Redis is that It is single threaded and sync. you need expensive high frequency CPU if you need more performance. something that could benefit from multi core system and if you would want to use disk cache async is a must. single threading also affects scripts running on Redis side, when they execute whole Redis is waiting and is unable to fulfil other requests.

      another place for improvement would be proper easier to use HA, and maybe also auto scaling.

      • k_bx 3 days ago ago

        Most projects need to start with something and know that they will "be good". Many will hit RAM limitations earlier than kvrocks latencies or single-threaded CPU limitations (if they will – usually author doesn't need advice). Most would benefit from "moving fast" (in terms of development).

        Redis-compatible protocol guarantees that you can think about the things you've mentioned later. If project is successful – you'll find money to pay for RAM and move to pure Redis. If it's CPU-bound – you'll find multi-threaded alternatives.

        So this is a pretty nice niche IMO actually.

        • Karrot_Kream 3 days ago ago

          Why not use a managed Redis or a large memory instance and run Redis yourself then? They aren't that much more expensive if you need to move fast.

          • k_bx 2 days ago ago

            Yes, but why pay for it if `sudo apt install zedis` works? Many use-cases for redis growing out of memory would actually be ok if it stores old values on disk for you and keeps hot items in cache (and does compression).

            • barddoo 2 days ago ago

              It’s not feasible to do that in a single threaded system. The main drawback of disk persistence is that you need a log of memory to keep the file buffer open and pointers. If the key you’re looking for is not on disk, you’ll get awful results, the rdb format will give you worst then Postgres performance. You have to use another file format to optimize for a larger cache

      • progbits 2 days ago ago

        The Redis HA story is horrendous. I wonder if Sentinel is intentionally just crap enough to get your application locked in but then force you to pay them for hosted option.

        It regularly gets itself into messed up state and even with graceful shutdown for node updates the official client libraries throw errors for up to a minute.

    • jasonjmcghee 3 days ago ago
      • lyu07282 3 days ago ago

        That just makes it durable, the entire dataset still has to fit in memory.

        • barddoo 2 days ago ago

          The format was made for taking snapshots, it’s terrible for lookup.

    • nine_k 2 days ago ago

      If you're in a tight VM and are running Redis inside it, as opposed to a separate VM / cluster, you need an embedded DB, not Redis.

      You can use the ancient BerkeleyDB which is supported everywhere for last like 30 years. Or a more recent LevelDB. Or even SQLite which is plenty fast on simple queries.

      You'll have to tolerate the high latency of safely writing tings to a block device, compared to RAM. If your load is mostly reads, it should be OK.

    • ncruces 2 days ago ago

      Redka may be interesting to you: https://github.com/nalgeon/redka

      • barddoo 2 days ago ago

        I didn’t know that, that’s pretty cool

    • badLiveware 2 days ago ago

      Redis compatible projects with larger-than-memory support do exists, e.g. https://microsoft.github.io/garnet/docs/welcome/features#tie...

      • k_bx 16 hours ago ago

        No apt repo, C#.. kinda worse than kvrocks in that sense

        • mrsmrtss 15 hours ago ago

          Why C# makes it worse?

  • jasonjmcghee 3 days ago ago

    I'm not really sure what it costs these days - i know certain projects not entirely free like they were a few years ago, but there's a pretty good "build your own redis" among other things.

    It has little step-by-step tasks with automated tests. There are some other good ones like git and docker. It's pretty cool.

    https://app.codecrafters.io/courses/redis/overview

    • nine_k 3 days ago ago

      It's easy to write a KV store, even a KV store with key expiration. But Redis is quite a bit more than that. Consider sharding and replication at least. Then lists, sets, zsets, bitfields, streams, geospatial indexes, hyperloglog / bloom filters, time series.

      Something like LevelDB is relatively easy to write. Then you can build the rest of Redis on top of it.

      • johnisgood 2 days ago ago

        > Then lists, sets, zsets, bitfields, streams, geospatial indexes, hyperloglog / bloom filters, time series.

        I mean, I have my own C libraries for most of these!

        • dns_snek 2 days ago ago

          In Redis those operations are performed server-side, on a shared dataset, and concurrently by many clients.

    • barddoo 3 days ago ago

      Sounds pretty interesting.

  • johnisgood 3 days ago ago

    Seems like LLMs are getting good at Zig (with some help, I presume).

    • mtlynch 3 days ago ago

      Is there anything about this project that seems LLM-generated?

      I've found that LLMs are particularly bad at writing Zig because the language evolves quickly, so LLMs that are trained on Zig code from two years ago will write code that no longer compiles on modern Zig.

      • jasonjmcghee 3 days ago ago

        I skimmed, for me it was this: https://github.com/barddoo/zedis/blob/87321b04224b2e2e857b67...

        There seems to be a fair amount of stigma around using llms. And many people that use them are uncomfortable talking about it.

        It's a weird world. Depending on who is at the wheel, whether an llm is used _can_ make no difference.

        But the problem is, you can have no idea what you're doing and make something that feels like it was carefully hand-crafted by someone - a really great project - but there are hidden things or outright lies about functionality, often to the surprise of the author. Like, they weren't trying to mislead, just didn't take them time to see if it did all of what the LLM said it did.

        • barddoo 3 days ago ago

          Agree. I used it mostly for getting ideas, the memory management for example, Gemini listed so many different ways of managing memory I didn’t even know existed. I know I wanted to pre allocate memory like tigerbeetle does, so the hybrid approach was perfect. Essentially it has 3 different allocators, a huge one for the cache, a arena allocator for context, intermediate state like pub/sub and temp one, for requests. It was 100% Gemini’s idea.

        • boredemployee 3 days ago ago

          3 months ago I was vibe coding an idea and for some reason (and luck) I went to check a less important part of the code and saw that the LLM changed the env variable of an API key and hard coded the key explictly in the code. That was scary. I'm glad I saw it before PR and shit like that.

        • johnisgood 3 days ago ago

          I generally do not think it is a bad thing. I use LLMs too and I know what I am doing, so I do not know if it could be qualified as vibe coding.

          I think it is not inherently a bad thing to use LLMs, only if you have absolutely no clue about what you are doing, but even then, if the project is usable and as-is advertised, why not? shrugs

          As for the link, that is exactly the same code that caught my eye, besides the README.md itself. The LRU eviction thing is what GPT (and possibly other LLMs) always comes up with according to my experiences, and he could have just had it properly implemented then. :D

          Edit: I am glad author confirmed the use of an LLM. :P

      • rmonvfer 3 days ago ago

        As an avid Claude Code user, I can tell you with 99% probability, that README is LLM-generated. This exactly the same structure and wording used by Claude (of course, it has some human modification because otherwise I’d filled with emojis)

        In my experience, when you work with something like agentic development tools, you describe your goals and give it some constraints like “use modern zig”, “always run tests”… and when you ask it to write a README, it will usually reproduce those constraints more or less verbatim.

        The same thing happens with the features section, it reads like instructions for an LLM.

        I might be wrong but I spend way too much time using Claude, Gemini, Codex… and IMHO it’s pretty obvious.

        But hey, I don’t think it’s a problem! I write a lot of code using LLMs, mostly for learning (and… ejem, some of it might end up in production) and I’ve always found them great tools for learning (supposing you use the appropriate context engineering and make sure the agent has access to updated docs and all of that). For example, I wanted to learn Rust so I half-vibed a GPUI-based chat client for LLMs that works just fine and surprisingly enough, I actually learned and even had some fun.

        • samiv 2 days ago ago

          But wait.. if the chatbot is writing then code then it is not you who is writing the code, but the chatbot.

          • hnlmorg 2 days ago ago

            That would be true if you were vibe coding. However “normal” agentic development is a little more like “pair programming” in my opinion.

        • johnisgood 3 days ago ago

          > But hey, I don’t think it’s a problem! I write a lot of code using LLMs, mostly for learning (and… ejem, some of it might end up in production) and I’ve always found them great tools for learning (supposing you use the appropriate context engineering and make sure the agent has access to updated docs and all of that). For example, I wanted to learn Rust so I half-vibed a GPUI-based chat client for LLMs that works just fine and surprisingly enough, I actually learned and even had some fun.

          As I wrote in another comment, it is not inherently a bad thing. I use LLMs too (while knowing what I am doing), and if the project works, why not?

        • adastra22 3 days ago ago

          I don't know why you're being downvoted. This follows the LLM-generated-README template perfectly. And yeah, it usually ends up being a dumping ground for the constraints you gave it, almost verbatim.

      • johnisgood 3 days ago ago

        Fair enough, but I have made a couple of projects in Odin using Claude, and Odin is evolving too, and it is much more obscure than Zig. I made these projects successfully by feeding the LLM the documentation of Odin and gave it example projects... thus, if I could create Odin projects (with some reiterations and hand-holding, true), then Zig should be even better.

      • 5- 3 days ago ago

        https://github.com/barddoo/zedis/blob/87321b04224b2e2e857b67...

        these seem to occur only in college assignment projects, and in the output of text generators trained on those.

        • nine_k 3 days ago ago

          Has been doing this for years, even before LLMs were a thing. No, not in college assignments; by the time emoji appeared, I had long since walked out of my PhD program and went to the industry.

          I put such emojis at the beginning of big headings, because my eyes detect compact shapes and colors faster than entire words and sentences. This helps me (and hopefully others) locate the right section easier.

          In Slack, I put large emojis at the beginning of messages that need to stand out. These are few, and emojis work well in this capacity.

          (Disclaimer: I may contain a large language model of some kind, but very definitely I cannot be reduced to it in any area of my activity.)

          • adastra22 3 days ago ago

            FWIW it is really confusing to me and others. What is this emoji supposed to mean? Heck if I know.

            But the telltale signs are far more than just that. The whole document is exactly the kind of README produced by Claude.

          • MangoToupe 3 days ago ago

            I had assumed they were referring to stuff like "Type-safe operations with compile-time guarantees". What a weird detail to add to a readme. And the whole section is like that. I wonder if that's part of a prompt leaking through.

        • WD-42 3 days ago ago

          I will never place emojis in any of my readmes ever again.

          • tayo42 3 days ago ago

            People were doing this before llms, otherwise how did they learn it?

            • WD-42 3 days ago ago

              Sure, but llms absolutely love to do it for some reason.

              • tayo42 3 days ago ago

                I just took a look at a Readme I had cursor write a couple months ago and there's no emojis

                • johnisgood 2 days ago ago

                  You can always have it in your custom prompt: "Do not use emojis, ever".

              • pdntspa 2 days ago ago

                I had a manager who did this in his slack messages

            • adastra22 3 days ago ago

              That's why he said "never again"

          • chucky_z 3 days ago ago

            spell out 'development' with hammer emojis. bring ascii art back as emoji art.

            (i actually do this in slack messages and folks find it funny and annoying, but more funny)

    • barddoo 3 days ago ago

      Oh that’s nice of you. I take that as a compliment.

      • johnisgood 3 days ago ago

        It was not intended to be a criticism. I care about the end result, not the process. I use LLMs myself, too.

  • maleldil 3 days ago ago

    From the README:

    > Memory safety with RAII patterns

    I'm curious to see how they achieve RAII in Zig, which doesn't have destructors. If they mean defer + deinit methods, that's not the same thing.

    • teo_zero 3 days ago ago

      > If they mean defer + deinit methods, that's not the same thing.

      No? It's the first thing that came to my mind when I've read RAII.

      • rokob 2 days ago ago

        You have to call defer after initialization so you can’t technically call it RAII. As long as you remember to call it then it is effectively the same. Personally defer makes it so very similar to be a difference without a distinction but the “is initialization” part can’t be true if you need to do a second step.

    • justinhj 3 days ago ago

      Zig is all about making things explicit. Destructors are hidden code. I presume there will be an application level tracking of lifetime.

      • maleldil 2 days ago ago

        Yes, which is why Zig can't have RAII. Zig initialisation and destruction are fundamentally different to how RAII works in other languages. That's not a negative point; it's just how Zig is. But it's not RAII.

        • justinhj 2 days ago ago

          You are right. It seems the developer agrees and has changed the README

        • cakealert 2 days ago ago

          It's explicit opt in RAII, you opt in by using a defer.

          • lenkite 2 days ago ago

            RAII, which stands for Resource Acquisition Is Initialization, is a programming technique in which a resource's lifetime is tied to an object's lifespan. An explicit defer does not fit this paradigm.

          • ioasuncvinvaer 2 days ago ago

            free is an explicit opt-in garbage collector.

  • esafak 3 days ago ago

    Consider adding support for vector search and sketching (https://www.sketchingbigdata.org/).

    • barddoo 3 days ago ago

      I'll take a look. Thanks for the suggestion!

  • reenorap 3 days ago ago

    This is an interesting project but a poorly chosen name because it can easily be considered trademark infringement for being confusingly-similar to "Redis", especially since it's a self-admitted clone. I would change it now before you are forced to change your name and you lose time confusing people and wasting time renaming everything.

    • saghm 3 days ago ago

      I don't think it's particularly confusing. I wouldn't expect something official to have a name that changes the first letter. Would you honestly be confused about whether a search engine called Boogle, a laptop called a Yacbook, or a browser called Tirefox was an official product by Google, Apple, or Mozilla?

    • jiehong 2 days ago ago

      I’d see it in swift and then being called "Swedish" for giggles.

      Naming is famously difficult in this world!

    • thewisenerd 3 days ago ago

      i think “$proj - implements the redis protocol” are going to be as ubiquitous as the things which claim to be “S3 compatible”

      idk if redis labs (the organization aka big bad) are going to patent the “protocol” to disallow clones (or they’d be going after valkey first)

      • barddoo 2 days ago ago

        That’s a good point, I should had renamed it. You can’t trust these big corp

  • znpy 3 days ago ago

    OT: Reminds me of the late 2000s when python was having its boom and it was a lot of pythis and pythat

    • barddoo 3 days ago ago

      I just made it to learn Zig

      • znpy 2 days ago ago

        Oh yeah absolutely, i wasn’t bashing you in any way! Well done!

  • pdntspa 3 days ago ago

    Is zig lang stable enough now to start basing real projects on it?

    • pyrolistical 3 days ago ago

      Depends on your definition of stable. I would suspect most people would consider it unstable as they are still breaking the API for async. Wait for 1.0 if you really need true stability. But zig is just so good not to use now

    • barddoo 3 days ago ago

      The standard library is changing too much to be honest. During the development of the library Zig 0.15 launched and changed the whole Io library.

    • tonyhart7 3 days ago ago

      "Is zig lang stable enough now to start basing real projects on it?"

      only if you want to refactor/rewrite a lot

    • justinhj 3 days ago ago

      Check out Tigerbeetle and Ghostty

      • jrpelkonen 3 days ago ago

        Also bun

      • fishmicrowaver 3 days ago ago

        I mean no offense but a billionaires vanity terminal and a database with an anime bug mascot are a bit different than a redis alternative

        • rvrb 3 days ago ago

          Good point, the database written in Zig with a bug mascot tells us nothing about writing a database in Zig without a bug mascot

          • throwawaymaths 3 days ago ago

            im not sure i trust any hot takes from a person that doesn't know anything about anime

        • justinhj 3 days ago ago

          If all you know about Tigerbeetle is the mascot you should probably take another look before judging. Some very interesting db tech.

          • sroerick 3 days ago ago

            Tigerbeetle is one of the most interesting projects I've seen in recent memory

          • barddoo 3 days ago ago

            100% agree. Their algorithm for distributed consensus is pretty different than anything else in the market.

        • dpatterbee 3 days ago ago

          Is software not real software if it's written by a billionaire? What is it about the bug mascot that detracts from the database's legitimacy?

          • garbagepatch 3 days ago ago

            What's the problem with the mascot? PostgreSQL has an elephant and MySQL a dolphin. Is the bug too detailed for database software?

            • nickmonad 2 days ago ago

              Why would we want a literal BUG as a mascot?? /s

        • aschobel 3 days ago ago

          bun is real and spectacular. super loving using it

    • nitishr 3 days ago ago

      it hasn't reached version 1.0

  • era37 3 days ago ago

    Looks like a cool project; does the project aim to fix anything existing about Redis or is this just a fun side project? Thanks

    • barddoo 2 days ago ago

      Redis dynamically allocates memory, the idea of the project is to pre allocate everything, so it’s by principle faster than Redis.

  • mawadev 3 days ago ago

    This is a very cool project. I wish it could do the same stuff as RedisJson at some point!

    • barddoo 2 days ago ago

      I definitely plan to do that in the future

  • nitishr 3 days ago ago

    Are you keeping it single threaded?

    • barddoo 3 days ago ago

      No, I wasn’t able to get it working with async io using io_uring/kqueue.

  • gkorland 2 days ago ago

    Did you try to run redis-benchmark? (Compared to Redis)

  • montyanderson 2 days ago ago
  • didip 3 days ago ago

    What about calling it Zigdisk and make it save to disk?

  • BiraIgnacio 3 days ago ago

    cool stuff, keep at it!

    • barddoo 3 days ago ago

      Thanks! There are so many things to do still, authentication, glob-style keys search, json parsing.

  • tcfhgj 3 days ago ago

    So riiR is now riiZ? ;)

    • nine_k 3 days ago ago

      "Rewrite it away from C (or C++)". Whether you choose Rust, Zig, Ada, D, Nim, even Pascal, it's likely going to become more secure. It will be supported on fewer platforms though, but still should run fine under Linux, macOS, Windows, *BSD, on x64, Arm64, and likely RISC-V, too.

      • maleldil 3 days ago ago

        Zig isn't memory safe, though. It's safer than C or C++, but not much.

        • ashikns 3 days ago ago

          This is what I have struggled to understand about Zig. It seems pretty much like C in a mental model aspect - you are responsible for everything. It's slightly better than C, but C already runs on everything on the planet and can be made secure even if painfully so. So what niche is Zig aiming to fill?

          • throwawaymaths 3 days ago ago

            no, null pointers are enforced safe at the type level in zig, as are array bounds, this eliminates huge classes of errors, so you are not "responsible for everything". unlike c, you often (unless highly tuned performance is needed) do not have to resort to opaque void pointers, and the compiler gives you typesafety on that, another major footgun in c.

            also operators and integer types are unambiguous, and there is no UB in safe compilation modes.

            It's arguably much better than C, not "slightly better than C"

            • uecker 3 days ago ago

              If you write a modern style of C, you can have bounds checked code and do not need to use void pointers. I usually find that people overestimate the advantages of newer languages compared to using C by comparing to old and badly written C.

              • throwawaymaths 2 days ago ago

                GP: "you are responsible for everything"

                AKA You are responsible for opting into "modern C". In order to be unsafe in zig (in the dimensions I mentioned) you must opt out.

                • uecker 2 days ago ago

                  You would need to opt-in into zig first, which is more effort than opting into modern C when you come from C.

                  • throwawaymaths a day ago ago

                    i suggest you show this thread to a neutral third party and ask them if your line of argument makes sense.

          • barddoo 3 days ago ago

            Zig detects memory leaks pretty well when you build it using -Doptimize=Debug.

        • throwawaymaths 3 days ago ago

          checked array bounds is memory safety.

          • lyu07282 3 days ago ago

            Runtime bounds checking, but only in Debug/ReleaseSafe builds, besides there are many other types of memory safety issues left unaddressed.

            • throwawaymaths 2 days ago ago

              > Runtime bounds checking

              Compile time slice bounds checking is generally not a thing, since the length is not known until runtime. Its still safe.

              > but only in Debug/ReleaseSafe builds

              So, the default build modes?

              Istg zig should just eliminate releasefast and releasesmall and force you to opt in on a function by function basis.