If you have an Adafruit Fruit Jam, you can run the demo on that board [1]. It can stream 640x480 video off an SD card at 60fps, as well as display a static image from flash.
Also possibly of interest to HN, the decoder[2] is written in a new low level language called symbolasm[3], which seeks to combine the advantages of assembly language (it's possible to reason about and optimize performance with few limits) with some of the affordances of C, especially named variables and struct fields.
Don’t you get this from most macro assemblers? I know I’ve seen some very old and crusty ASM that nevertheless does structured programming and even limited OOP through macro-ASM facilities.
(That being said, symbolasm looks 1000x more ergonomic and “also communicates developer intent”-laden than any macro ASM ever did; and also is probably something you could learn once, while a project’s macros are specific to said project.)
I don't believe macro assemblers do a flow-based analysis to verify that the register allocation for variables is consistent; from what I understand the most you get is just a scoped alias for the name of the register. Of course, I could be missing something. If you know of anything beyond the relevant work I cite, please let me know.
I had some similar ideas for an assembler targeting MIPS, I wanted to use symbolic names, reserved within called scopes, and have the assembler find an allocation or error (requiring an explicit decision of what to spill), didn't find relevant examples at the time (circa 2018). Ultimately I decided I was wasting too much time on the assembler and went with global allocations for the most part.
Both the image compression and the symbolasm language started out being entirely hand coded, but I have used Claude to move it forward. I have complicated feelings about this. Basically I would have gotten these results even without the LLM, but it would have taken a lot longer.
I feel like one of the main tasks in front of us as an industry right now is to figure out how to use LLMs well, how to create quality and resist slop. As part of that, I'm taking care to write the prose myself.
I totally understand how any LLM usage can be a turnoff, though. I've felt that myself.
1) IMO pointless to put an emphasis on extreme decompression speed but not show numbers against JPEG even
2) BCx essentially does the same codebooking/paletting but more efficiently by allowing you to interpolate between two palette entries for an even larger compression
Why would a benchmark that shows "JPEG decoding - 20000ms, Nibnib decoding - 13ms" on some microcontroller be pointless? That'd be a great benchmark, if it was provided
If the issue is memory required for decoding the author could offer a benchmark on the smallest hardware that can still do both, with memory figures provided. Or the size of the largest image that can be decoded on an RP2350 with each algorithm
I wouldn't be surprised if you told me that JPEG decoding was memory bound or what the performance delta would be even if it isn't overall. The number of instructions isn't usually good performance indicator on its own.
I understand this is fast, okay. But at what trade-off?
Sure - muuuuuch less than Jpeg, but how many would you assume for JPEG on average?
How low could you go by misusing jpeg?
Typically, you would plot graphs with Bitrate / quality / speed. Maybe even with default jpeg settings and getting just a single data point for your new algorithm.
As a free anecdatum, I have no idea how long image decompression meaningfully takes! As far as I'm concerned, JPEGs decompress instantly but that's likely only because I've never dealt with scenarios where processing power or bandwidth is that limited. But if I somewhen were in such a situation, seeing some numbers would likely help contextualize the differences here.
A similar scenario happened to me some years back, when I had to pick an appropriate compression algorithm for some hundreds of gigabytes of parquet files because AWS Athena was being slow. The comparison table on zstd's GitHub was immense useful and provided that level 0 to level 1 of understanding on these matters.
Experienced people still have blind spots, and I believe it's good to explain the obvious if understanding it is essential to the matter. :)
This seems like straining the definition of "compression". We have a delta-coding filter and then palettization? That's it?
Edit: I should say, I find it surprising (but cool) that it apparently works so well. I would have expected errors to accumulate since the decoder has to un-delta-code. I saw mention of Floyd-Steinberg dithering and presumably that's what it's for; seems hard to get it working across multiple colour axes simultaneously, but the complexity is all in the encoder, so.
Floyd-Steinberg dithering preserves 100% of the quantization error by shifting it onto neighboring pixels (unlike eg Atkinson dithering which only preserves 75%[0]), so the lack of drift I guess isn't too surprising really.
As for tracking error across multiple color axes, I would naively assume it's a 3-vector (maybe in a linearized perceptual color space, or maybe RGB) and the error term the encoder minimizes is just the vector magnitude?
Very cool. I think it's best understood as a coding exercise for the embedded CPU vs a 'real' video format, but no less impressive IMO
I guess the key is that the encoder knows exactly what the decoder will reconstruct at each step, so it can compensate for previous errors when choosing the next palette entry
This reminds me a bit of Amiga HAM. Similar horizontal fringing.
I guess you could also call Amiga's HAM graphics mode a compression method. Getting 12-bit (4096 colors) with just 6 input bits, with the original Amiga chipset, often called as "OCS".
Or later 262144 colors with 8 input bits, with the AGA chipset.
Worth noting that something perf-optimized for RP2350 is almost certainly not going to be optimal on a modern desktop-class core (with deep pipelines and wide ILP/SIMD).
I'm impressed. The imbalance between encoding and decoding speeds seems a useful strategy when you are building special purpose hardware and software applications.
It seems if you're making a central source of data that is able to tolerate a much slower write process versus a quick read process that this approach can be advantageous.
Maybe using this core method in a problem space which maps to the radio/signal broadcast process of many receivers but a central source of truth.
I can imagine if you had an information base that changed slowly or which was pre-processed in a batch fashion, that it could use this method.
It doesn't even need for the central data to be decoded into a graphical image.
I think this radio model could map the problem space for a centralized system that maintained a list of items and their prices which needed to be broadcast to the retail outlets on a regular basis.
Or a common cache storage to make some complex data manipulation calculations centrally performed available to a large number of satellites who just need the end results but do not have the need to do local calculation.
There are no private messages on HN. This, and your comments in the last 3 months look (AI) generated, which is prohibited in the guidelines[0] but also loathed by the community. Can only speculate, though.
Dave is a brave chap, and posts his personal phone number in his profile. You might be able to contact him that way, but, personally, I wouldn't. I don't think it's really appropriate. I'd say that your comment is sufficient, and not inflammatory.
But I do agree, the comment smells "AIsh." It may not be, however, if this is his standard writing style. I have been accused of being a "bot," for stuff I wrote ten years ago. It's my observation, that people tend to accuse any lengthy comments of being LLM-generated.
It's a bit rambling, but I'm not sure I'd call it word salad. I promise you that in television spending a lot more resources on compression in a central location than on decompression at each cable headend, broadcast affiliate, smart TV, or customer streaming box is a desirable trait. For live programming, you'd need fast encoding but could use a heavyweight system with far more resources than your targets at the edge. For on-demand programming you could afford to encode slowly before putting the content on the CDN.
If you have an Adafruit Fruit Jam, you can run the demo on that board [1]. It can stream 640x480 video off an SD card at 60fps, as well as display a static image from flash.
Also possibly of interest to HN, the decoder[2] is written in a new low level language called symbolasm[3], which seeks to combine the advantages of assembly language (it's possible to reason about and optimize performance with few limits) with some of the affordances of C, especially named variables and struct fields.
[1]: https://github.com/raphlinus/vic-2350
[2]: https://github.com/raphlinus/vic-2350/blob/main/image.sy
[3]: https://github.com/raphlinus/symbolasm
> especially named variables and struct fields
Don’t you get this from most macro assemblers? I know I’ve seen some very old and crusty ASM that nevertheless does structured programming and even limited OOP through macro-ASM facilities.
(That being said, symbolasm looks 1000x more ergonomic and “also communicates developer intent”-laden than any macro ASM ever did; and also is probably something you could learn once, while a project’s macros are specific to said project.)
I don't believe macro assemblers do a flow-based analysis to verify that the register allocation for variables is consistent; from what I understand the most you get is just a scoped alias for the name of the register. Of course, I could be missing something. If you know of anything beyond the relevant work I cite, please let me know.
I had some similar ideas for an assembler targeting MIPS, I wanted to use symbolic names, reserved within called scopes, and have the assembler find an allocation or error (requiring an explicit decision of what to spill), didn't find relevant examples at the time (circa 2018). Ultimately I decided I was wasting too much time on the assembler and went with global allocations for the most part.
Cool projects! symbolasm looks awesome. Brings back memories of playing around with fasm on MenuetOS and trying to wrap my head around assembler.
[flagged]
At least some of your projects are vibe-coded, right? You mention using LLMs in both some of the repositories and some of your Hacker News comments.
Both the image compression and the symbolasm language started out being entirely hand coded, but I have used Claude to move it forward. I have complicated feelings about this. Basically I would have gotten these results even without the LLM, but it would have taken a lot longer.
I feel like one of the main tasks in front of us as an industry right now is to figure out how to use LLMs well, how to create quality and resist slop. As part of that, I'm taking care to write the prose myself.
I totally understand how any LLM usage can be a turnoff, though. I've felt that myself.
1) IMO pointless to put an emphasis on extreme decompression speed but not show numbers against JPEG even
2) BCx essentially does the same codebooking/paletting but more efficiently by allowing you to interpolate between two palette entries for an even larger compression
1) it may seem pointless until you realize it may allow targeting platforms which can't decode JPEG real time under certain situation
Why would a benchmark that shows "JPEG decoding - 20000ms, Nibnib decoding - 13ms" on some microcontroller be pointless? That'd be a great benchmark, if it was provided
If the issue is memory required for decoding the author could offer a benchmark on the smallest hardware that can still do both, with memory figures provided. Or the size of the largest image that can be decoded on an RP2350 with each algorithm
Which would have been clear if numbers were provided for a JPEG implementation in the repo
Why though? I thought it was pretty clear that you're not gonna decode JPEG pixels in 6 instructions.
I wouldn't be surprised if you told me that JPEG decoding was memory bound or what the performance delta would be even if it isn't overall. The number of instructions isn't usually good performance indicator on its own.
I understand this is fast, okay. But at what trade-off?
Sure - muuuuuch less than Jpeg, but how many would you assume for JPEG on average? How low could you go by misusing jpeg?
Typically, you would plot graphs with Bitrate / quality / speed. Maybe even with default jpeg settings and getting just a single data point for your new algorithm.
Mandatory https://xkcd.com/2501/
As a free anecdatum, I have no idea how long image decompression meaningfully takes! As far as I'm concerned, JPEGs decompress instantly but that's likely only because I've never dealt with scenarios where processing power or bandwidth is that limited. But if I somewhen were in such a situation, seeing some numbers would likely help contextualize the differences here.
A similar scenario happened to me some years back, when I had to pick an appropriate compression algorithm for some hundreds of gigabytes of parquet files because AWS Athena was being slow. The comparison table on zstd's GitHub was immense useful and provided that level 0 to level 1 of understanding on these matters.
Experienced people still have blind spots, and I believe it's good to explain the obvious if understanding it is essential to the matter. :)
And predictable decoding time matters too
>pointless to put an emphasis on extreme decompression speed but not show numbers against JPEG even
this decodes at HDMI line speed, 6 opcodes per pixel
[flagged]
This seems like straining the definition of "compression". We have a delta-coding filter and then palettization? That's it?
Edit: I should say, I find it surprising (but cool) that it apparently works so well. I would have expected errors to accumulate since the decoder has to un-delta-code. I saw mention of Floyd-Steinberg dithering and presumably that's what it's for; seems hard to get it working across multiple colour axes simultaneously, but the complexity is all in the encoder, so.
Floyd-Steinberg dithering preserves 100% of the quantization error by shifting it onto neighboring pixels (unlike eg Atkinson dithering which only preserves 75%[0]), so the lack of drift I guess isn't too surprising really.
As for tracking error across multiple color axes, I would naively assume it's a 3-vector (maybe in a linearized perceptual color space, or maybe RGB) and the error term the encoder minimizes is just the vector magnitude?
Very cool. I think it's best understood as a coding exercise for the embedded CPU vs a 'real' video format, but no less impressive IMO
[0] https://beyondloom.com/blog/dither.html
I guess the key is that the encoder knows exactly what the decoder will reconstruct at each step, so it can compensate for previous errors when choosing the next palette entry
This reminds me a bit of Amiga HAM. Similar horizontal fringing.
I guess you could also call Amiga's HAM graphics mode a compression method. Getting 12-bit (4096 colors) with just 6 input bits, with the original Amiga chipset, often called as "OCS".
Or later 262144 colors with 8 input bits, with the AGA chipset.
Worth noting that something perf-optimized for RP2350 is almost certainly not going to be optimal on a modern desktop-class core (with deep pipelines and wide ILP/SIMD).
Absolutely. This is optimized for highly resource constrained devices, not modern machines. I can tweak the README if that's at all confusing.
they were pretty clear it's for embedded image decoding directly to the video output port
Fun. Vaguely reminiscent of the Amiga HAM video mode.
Reminds me slightly of the old Hold And Modify idea. Big palette values are like loading new colours; small ones are like HAM.
Will you do the same to MJPG?
I'm impressed. The imbalance between encoding and decoding speeds seems a useful strategy when you are building special purpose hardware and software applications. It seems if you're making a central source of data that is able to tolerate a much slower write process versus a quick read process that this approach can be advantageous. Maybe using this core method in a problem space which maps to the radio/signal broadcast process of many receivers but a central source of truth. I can imagine if you had an information base that changed slowly or which was pre-processed in a batch fashion, that it could use this method. It doesn't even need for the central data to be decoded into a graphical image.
I think this radio model could map the problem space for a centralized system that maintained a list of items and their prices which needed to be broadcast to the retail outlets on a regular basis.
Or a common cache storage to make some complex data manipulation calculations centrally performed available to a large number of satellites who just need the end results but do not have the need to do local calculation.
There are no private messages on HN. This, and your comments in the last 3 months look (AI) generated, which is prohibited in the guidelines[0] but also loathed by the community. Can only speculate, though.
https://news.ycombinator.com/newsguidelines.html
Dave is a brave chap, and posts his personal phone number in his profile. You might be able to contact him that way, but, personally, I wouldn't. I don't think it's really appropriate. I'd say that your comment is sufficient, and not inflammatory.
But I do agree, the comment smells "AIsh." It may not be, however, if this is his standard writing style. I have been accused of being a "bot," for stuff I wrote ten years ago. It's my observation, that people tend to accuse any lengthy comments of being LLM-generated.
I'm trying to learn why I have been downvoted. Please reply or send me a private message so I can learn.
probably because what you wrote doesn't make any sense - it's word salad
It's a bit rambling, but I'm not sure I'd call it word salad. I promise you that in television spending a lot more resources on compression in a central location than on decompression at each cable headend, broadcast affiliate, smart TV, or customer streaming box is a desirable trait. For live programming, you'd need fast encoding but could use a heavyweight system with far more resources than your targets at the edge. For on-demand programming you could afford to encode slowly before putting the content on the CDN.