Introducing UniClawBench: A New Benchmark for Evaluating Proactive Agents in Real-World Tasks

Researchers have developed UniClawBench, a pioneering benchmark specifically designed to evaluate proactive agents in dynamic, real-world environments. Traditional benchmarks often fall short by relying on sandboxed conditions and single-turn evaluations, which hinder a comprehensive understanding of agent capabilities. UniClawBench addresses these issues by focusing on five core model capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross-Platform Coordination. It features 400 bilingual real-world tasks and utilizes live Docker containers for assessment, incorporating a closed-loop evaluation strategy to simulate realistic human feedback. The benchmark aims to disentangle the effects of model capabilities from agent framework designs, facilitating further research in the field. The benchmark and its code are publicly available at https://github.com/HKU-MMLab/UniClawBench.