SketchAgentBench – evaluate LLMs on 3D spatial reasoning

(sketchagent.dev)

9 points | by wsxiaoys 19 hours ago ago

5 comments

  • anilgulecha 12 hours ago ago

    It's only benchmarks where I hear about Google's models being frontier. Never really get to use it because they don't have a usable subscription model on par with Claude/Openai ones in terms of usage.

  • gorpy7 12 hours ago ago

    i had astra route some ducting thru joists and beams and other crap and i was so surprised and rejoicing because it knocked it out of the park! this is sketchup pro. i’m not especially good at sketchup. i also had it do some plumbing and again it was good. i wanted proper slope on all the joints and pipes and if i needed to adjust a route- boy was that aggravating! but then asked astra and just sipped my pop.

  • vivzkestrel 12 hours ago ago

    - can we get a benchmark on game decompilation / figuring out?

  • 19 hours ago ago
    [deleted]
  • 19 hours ago ago
    [deleted]