Arch-delta Saves 80% Of Bandwidth On Upgrades

(djugei.github.io)

35 points | by birdculture 2 days ago ago

4 comments

  • phiresky a day ago ago

    This is great! Especially the DB sync part, because that happens before a user interaction, so you actually have to wait for it (the update itself can run in the background).

    It always felt like such a waste to me how the DB always downloads tens of megabytes of data when likely only 1kB has changed. I mean I also really appreciate the beauty of how simple it is. But I'd bet even a delta against a monthly baseline file would reduce the data by >90%.

    Also, it would be interesting to see how zstd --patch-from compares to the used delta library. That is very fast (as fast as normal zstd) and the code is already there within pacman.

    For the recompression issue, there is some hard to find libraries that can do byte-exact reproducible decompression https://github.com/microsoft/preflate-rs but I don't know of any that work for zstd.

  • estimator7292 7 hours ago ago

    I wish there were a simple and easy way to P2P my pacman updates. I have several Arch machines/VMs that all need to download a gigabyte of updates at the same time.

    This wastes not only my time and bandwidth, but adds extra load and bandwidth on the mirror. Why can't I have pacman sync with one of my other computers? Even if there's a different set of packages, the core OS is the same and can share packages with other machines.

    I did look briefly into hosting a proper pacman mirror locally, but it seemed like way too much effort to save 10 minutes a month.

    • yndoendo 5 hours ago ago

      Arch Wiki to the rescue. https://wiki.archlinux.org/title/Package_proxy_cache#

      Currently using the dkarhttpd route with rsync. Everything is stored in a custom directory so that older version can be retained. Pacoloco looks interesting.

    • dTal 7 hours ago ago

      Just update one then rsync the package cache across? Or mount it over NFS for a hands-off solution. Or hack something up with inotify.