3 points | by HardikVala 4 days ago ago
1 comments
Hey, cool initiative!
Worth mentioning in the title that it's CPU-only: >1200 tokens/s on a single thread is impressive.
Have you considered doing optimization iterations like nanogpt-speedrun? Would be interesting to see how far you can push the performance.
Hey, cool initiative!
Worth mentioning in the title that it's CPU-only: >1200 tokens/s on a single thread is impressive.
Have you considered doing optimization iterations like nanogpt-speedrun? Would be interesting to see how far you can push the performance.