Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies

Reinforcement learning (RL) policies are prone to high-frequency oscillations, especially undesirable when deploying to hardware in the real-world. In this paper, we identify, categorize, and compare methods from the literature that aim to mitigate high-frequency oscillations in deep RL. We define two broad classes: loss regularization and architectural methods. At their core, these methods incentivize learning a smooth mapping, such that nearby states in the input space produce nearby actions in the output space. We present benchmarks in terms of policy performance and control smoothness on traditional RL environments from the Gymnasium and a complex manipulation task, as well as three robotics locomotion tasks that include deployment and evaluation with real-world hardware. Finally, we also propose hybrid methods that combine elements from both loss regularization and architectural methods. We find that the best-performing hybrid outperforms other methods, and improves control smoothness by 26.8% over the baseline, with a worst-case performance degradation of just 2.8%.

Autonomous Ground Navigation in Highly Constrained Spaces: Lessons learned from The 2nd BARN Challenge at ICRA 2023

The 2nd BARN (Benchmark Autonomous Robot Navigation) Challenge took place at the 2023 IEEE International Conference on Robotics and Automation (ICRA 2023) in London, UK and continued to evaluate the performance of state-of-the-art autonomous ground navigation systems in highly constrained environments. Compared to The 1st BARN Challenge at ICRA 2022 in Philadelphia, the competition has grown significantly in size, doubling the numbers of participants in both the simulation qualifier and physical finals: Ten teams from all over the world participated in the qualifying simulation competition, six of which were invited to compete with each other in three physical obstacle courses at the conference center in London, and three teams won the challenge by navigating a Clearpath Jackal robot from a predefined start to a goal with the shortest amount of time without colliding with any obstacle. The competition results, compared to last year, suggest that the teams are making progress toward more robust and efficient ground navigation systems that work out-of-the-box in many obstacle environments. However, a significant amount of fine-tuning is still needed onsite to cater to different difficult navigation scenarios. Furthermore, challenges still remain for many teams when facing extremely cluttered obstacles and increasing navigation speed. In this article, we discuss the challenge, the approaches used by the three winning teams, and lessons learned to direct future research.

The BARN Challenge 2023 -- Autonomous Navigation in Highly Constrained Spaces -- Inventec Team

Navigation in the real-world is hard and filled with complex scenarios. The Benchmark Autonomous Robot Navigation (BARN) Challenge is a competition that focuses on highly constrained spaces. Teams compete using a standard platform in a simulation and a real-world stage, with scenarios ranging from easy to challenging. This technical report presents the system and methods employed by the Inventec Team during the BARN Challenge 2023 (https://cs.gmu.edu/~xiao/Research/BARN_Challenge/BARN_Challenge23.html). At its core, our method uses the baseline learning-based controller LfLH. We developed extensions using a finite state machine to trigger recovery behaviors, and introduced two alternatives for forward safety collision checks, based on footprint inflation and model-predictive control. Moreover, we also present a backtrack safety check based on costmap region-of-interest. Compared to the original baseline, we managed a significant increase in the navigation score, from 0.2334 to 0.2445 (4.76%). Overall, our team ranked second place both in simulation and in the real-world stage. Our code is publicly available at: (https://github.com/inventec-ai-center/inventec-team-barn-challenge-2023.git)