ByteDance’s Seed team just released a new benchmark called EdgeBench to test whether AI agents can learn from their environment during long-horizon tasks lasting 12 to 72 hours. After analyzing more than 38,000 hours of agent logs, the researchers found that learning curves are highly predictable — performance improves steadily with interaction time and fits a log-sigmoid curve almost perfectly. They compare the process to graph search theory, where an agent’s progress expands outward like a frontier on a skill map. Seed has open-sourced the benchmark framework and the first 51 tasks.
But on the same day, research organization Epoch AI published results from its own EBR-bench evaluation that tell a very different story. In board game environments — even when given strategy guides — the AI showed almost no learning or self-improvement after repeated attempts.
The divergence suggests that an AI’s ability to learn autonomously depends heavily on the task environment and feedback design. In settings like systems engineering or scientific exploration where continuous feedback loops exist, agents can build cumulative advantages. But for discrete rule-based games with complex strategies, current online learning still struggles.