AI data startup Idler raises $9M to clone real companies

AI data startup Idler has closed a $9 million seed round led by Paradigm, with Y Combinator and Long Journey Ventures participating. It builds evals, benchmarks, and reinforcement learning environments for top model labs, letting AI systems run tasks in near-real work settings, get feedback, and keep training.

Idler previously focused on coding. Over the past year, it has produced thousands of programming evals and training environments for a major AI lab. YC's public records also show Idler signed a multi-million-dollar contract with a leading foundation model lab.

Now it's expanding into real enterprise work. Its first public benchmark, ShelfLife, turns a currently operating retail company into a digital twin. The site currently offers 200 tasks drawn from nearly a decade of that company's real business records. Claude Opus 5 has the highest pass rate at 69.0%; GPT-5.6 Sol sits at 55.7%.

Idler also used 41 of the ShelfLife tasks to post-train NVIDIA Nemotron 3 Nano, lifting its Finance Agent Benchmark score from 30.0% to 41.8%.

Beyond retail, Idler is already building environments for real work like corporate law. The goal is to push model training from “doing problem sets” toward “doing the job.”