B200 Attention Kernel from Scratch to Near-SOTA in 60 Diagrams

(iaroslavelistratov.github.io)

23 points | by magoghm a day ago ago

3 comments

  • xiphias2 a day ago ago

    Pretty cool, it's missing how these concepts map to ThunderKittens code (my favorite CUDA toolkit)

  • peter_d_sherman 18 hours ago ago

    One of the best AI GPU Attention Kernel how-to-implement articles I've read in a long time!

    Tremendous work, tremendous effort was placed into writing this article -- and it shows!

    Well done!

  • erichocean a day ago ago

    Really interesting project, appreciated.