![[2602.18856] Issues with Measuring Task Complexity via Random Policies in Robotic Tasks](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18856] Issues with Measuring Task Complexity via Random Policies in Robotic Tasks
Summary
This paper evaluates the effectiveness of measuring task complexity in robotic tasks using random policies, revealing contradictions in established metrics and suggesting the need for improved methods.
Why It Matters
Understanding task complexity is crucial for developing effective reinforcement learning benchmarks and curricula in robotics. This study challenges existing metrics, highlighting their limitations and the need for new approaches to accurately assess complexity in non-tabular settings.
Key Takeaways
- Current metrics like PIC and POIC may misrepresent task complexity in robotic settings.
- Empirical results show contradictions between established understanding and metric outcomes.
- The study advocates for the development of more reliable metrics beyond RWG-based methods.
Computer Science > Machine Learning arXiv:2602.18856 (cs) [Submitted on 21 Feb 2026] Title:Issues with Measuring Task Complexity via Random Policies in Robotic Tasks Authors:Reabetswe M. Nkhumise, Mohamed S. Talamali, Aditya Gilra View a PDF of the paper titled Issues with Measuring Task Complexity via Random Policies in Robotic Tasks, by Reabetswe M. Nkhumise and 2 other authors View PDF Abstract:Reinforcement learning (RL) has enabled major advances in fields such as robotics and natural language processing. A key challenge in RL is measuring task complexity, which is essential for creating meaningful benchmarks and designing effective curricula. While there are numerous well-established metrics for assessing task complexity in tabular settings, relatively few exist in non-tabular domains. These include (i) Statistical analysis of the performance of random policies via Random Weight Guessing (RWG), and (ii) information-theoretic metrics Policy Information Capacity (PIC) and Policy-Optimal Information Capacity (POIC), which are reliant on RWG. In this paper, we evaluate these methods using progressively difficult robotic manipulation setups, with known relative complexity, with both dense and sparse reward formulations. Our empirical results reveal that measuring complexity is still nuanced. Specifically, under the same reward formulation, PIC suggests that a two-link robotic arm setup is easier than a single-link setup - which contradicts the robotic control and empirica...