GoBench: Evaluating LLMs on the game of Go [R]
![GoBench: Evaluating LLMs on the game of Go [R]](/_next/image?url=https%3A%2F%2Fpreview.redd.it%2Ftbl17cn8ixph1.png%3Fwidth%3D140%26height%3D70%26auto%3Dwebp%26s%3Ddd4991d9d3d9dd4d98f8e7afc93bc826e38c523b&w=3840&q=75)
![GoBench: Evaluating LLMs on the game of Go [R]](https://preview.redd.it/tbl17cn8ixph1.png?width=140&height=70&auto=webp&s=dd4991d9d3d9dd4d98f8e7afc93bc826e38c523b "GoBench: Evaluating LLMs on the game of Go [R]") | GoBench evaluates LLMs on 9x9 Go games against a ladder of KataGo opponents, from random to superhuman. It measures general reasoning ability, strongly correlates with ARC-AGI 2 (r=0.83 correlation), and remains highly unsaturated. GPT-6 Astra max achieves 2500 Elo, much lower than the best KataGo, which achieves 4400 Elo. With coding tools and two hours of preparation before evaluation, Codex with Astra achieves 3560 Elo. I will keep the leaderboard updated as long as it is not saturated. leaderboard: https://rolandgao.com/blog/gobench/ code: https://github.com/RolandGao/gobench paper: https://github.com/RolandGao/gobench/blob/main/paper/gobench2.pdf x: https://x.com/Roland65821498/status/2100253388562723298?s=20 [link] [comments] |
Want to read more?
Check out the full article on the original site