VLMs Transition to Teaching Role to Enhance Video Reasoning through Adaptive Test-Time Optimization

The study introduces a novel approach in video reasoning by leveraging Vision-Language Models (VLMs) as teachers rather than solvers. The research highlights the limitations of current Video Generation Models (VGMs) in adhering to task-specific rules and executing complex instructions. By utilizing VLMs to extract task-specific rules and formulate differentiable rewards, the proposed method enables a VGM Reasoner to undergo test-time online optimization via a lightweight LoRA module. This transition significantly enhances the reasoning capabilities of VGMs, yielding an average performance improvement of 16.7 points on video reasoning benchmarks compared to previous paradigms. The findings suggest that integrating VLMs as teachers offers a promising direction for advancing generalizable video reasoning.