SoTALab Blog · 4 articles
Benchmarks, evaluation methodology, and operating notes from the SoTALab team.
Eight LLM agents and eight human players took on The Password Game. Neither side finished — and the ways they lost reveal why long-horizon tasks are the real frontier for today's AI agents.
Expert data is the next unavoidable frontier for China's AI labs.
Answers are easy to verify; reasoning is hard to constrain—five advanced math examples show why process-oriented training data matters more than a question bank.
Tasks, data, scoring, and runtime—what benchmark numbers can tell you, what they cannot, and how to read a leaderboard.