Beyond Floating Point: Next-Generation Computer Arithmetic

(web.stanford.edu)

30 points | by Bluestein a day ago ago

11 comments

  • AlotOfReading a day ago ago

    Gustafson's arguments haven't held up well to further development, in my opinion. As originally designed they aren't faster, they don't "solve" numerical analysis, and a lot of the issues people have with floats don't go away.

    They get a few things right like eliminating NaNs, but the reality is just so underwhelming compared to Gustafson's grandiose claims.

    • nextaccountic 21 hours ago ago

      > further development,

      Can you cite them?

      • AlotOfReading 20 hours ago ago

        Sure, here's a review article from last year [0]

            Regarding delay, area, and power, under the same precision the posit arithmetic units are not competitive against their floating-point counterparts.
        
        That paper was followed by a paper [1] published this year finally showing a posit variant implementation somewhat competitive with a mediocre float implementation, hurray. In order to achieve that, it had to give up most of the properties that Gustafson liked about posits and it only maintains a better precision than floats up to 2^10, at the cost of absolute relative precision.

        We even have systems that improve upon posits at the things they're good for, like takums [2]. And just for fun, a couple of papers comparing floats and posits:

            On the other hand, 32-bit posit are not exhibiting spectacular improvements in terms of efficiency over FP32. While they achieve the same or higher accuracy and can speedup the execution when the program has multiplications and divisions, they need around 30% more FPGA resources and use 6% more power compared to FP32. [3]
        
            The results confirm posit arithmetic’s strong performance at low precision, with takum following closely behind. Posits show stability issues at higher precisions, while OFP8 formats are unsuitable and bfloat16 underperforms compared to float16 and takum. [4]
        
        [0] https://dl.acm.org/doi/pdf/10.1145/3772284

        [1] https://arxiv.org/pdf/2603.01615

        [2] https://arxiv.org/pdf/2404.18603

        [3] https://arxiv.org/pdf/2109.08225

        [4] https://arxiv.org/pdf/2504.21197

        • Nevermark 12 hours ago ago

          When performance is measured, as far as I can tell (I didn't read those references in too much detail, because my brain is flatlining), its just directly against IEEE floating point. Is that right?

          Because posit delivers additional functionality, i.e. a reliable (in terms of closure) interval.

          Would it be fair to assume Posit as is, in hardware, would significantly outperform IEEE floating point [min, max] tuple arithmetic?

          Given all the extra instructions required to maintain intervals if there is no hardware interval arithmetic help.

          • AlotOfReading 11 hours ago ago

            Posits also require [min, max] interval arithmetic, so I'm not sure there's as large a difference as you're imagining. Are you thinking of type I and II unums instead?

            But yeah, none of these papers discuss interval arithmetic and frankly I've only rarely met anyone using it. Kahan has a blog post somewhere about how unums encounter most of the same issues with monsters, and it came up during the Kahan/gustafson debate iirc.

  • 0xcoffee a day ago ago

    Somewhat related, Anger Fog has a document about improving the IEEE 754 standard: https://www.agner.org/optimize/nan_propagation.pdf

  • Panzerschrek 20 hours ago ago

    For any bit width, they have a larger dynamic range, higher accuracy

    Isn't it a contradiction? One can't increase range without sacrificing accuracy.

    • entrope 19 hours ago ago

      It's not an automatic contradiction. IEEE floats have a lot of ways to represent NaN. (An exponent with all bits set encodes an infinity if the mantissa bits are all zero, and NaN if at least one mantissa bit is non-zero.)

      • Panzerschrek 18 hours ago ago

        Let's assume I can eliminate NaNs and use bit-patterns reserved for them for storing more numbers. For a 32-bit float it's roughly 8 millions of new values available, but it's only +0.4% increase. For 64-bit double it's even less (roughly +0.05%).

        • entrope 12 hours ago ago

          The slide deck isn't very long; you can see some of the examples of its claim. It also looks at very small formats, such as E4M3 eight-bit floats. Some (of what I think are meant to be) improved-accuracy examples involve catastrophic cancellation.

    • Boscolabs 9 hours ago ago

      [flagged]