Radar Meets Vision: Robustifying Monocular Metric Depth Prediction for Mobile Robotics

Mobile robots require accurate and robust depth measurements to understand and interact with the environment. While existing sensing modalities address this problem to some extent, recent research on monocular depth estimation has leveraged the information richness, yet low cost and simplicity of monocular cameras. These works have shown significant generalization capabilities, mainly in automotive and indoor settings. However, robots often operate in environments with limited scale cues, self-similar appearances, and low texture. In this work, we encode measurements from a low-cost mmWave radar into the input space of a state-of-the-art monocular depth estimation model. Despite the radar's extreme point cloud sparsity, our method demonstrates generalization and robustness across industrial and outdoor experiments. Our approach reduces the absolute relative error of depth predictions by 9-64% across a range of unseen, real-world validation datasets. Importantly, we maintain consistency of all performance metrics across all experiments and scene depths where current vision-only approaches fail. We further address the present deficit of training data in mobile robotics environments by introducing a novel methodology for synthesizing rendered, realistic learning datasets based on photogrammetric data that simulate the radar sensor observations for training. Our code, datasets, and pre-trained networks are made available at https://github.com/ethz-asl/radarmeetsvision.

WindSeer: Real-time volumetric wind prediction over complex terrain aboard a small UAV

Real-time high-resolution wind predictions are beneficial for various applications including safe manned and unmanned aviation. Current weather models require too much compute and lack the necessary predictive capabilities as they are valid only at the scale of multiple kilometers and hours - much lower spatial and temporal resolutions than these applications require. Our work, for the first time, demonstrates the ability to predict low-altitude wind in real-time on limited-compute devices, from only sparse measurement data. We train a neural network, WindSeer, using only synthetic data from computational fluid dynamics simulations and show that it can successfully predict real wind fields over terrain with known topography from just a few noisy and spatially clustered wind measurements. WindSeer can generate accurate predictions at different resolutions and domain sizes on previously unseen topography without retraining. We demonstrate that the model successfully predicts historical wind data collected by weather stations and wind measured onboard drones.

Soliro -- a hybrid dynamic tilt-wing aerial manipulator with minimal actuators

The ability to enter in contact with and manipulate physical objects with a flying robot enables many novel applications, such as contact inspection, painting, drilling, and sample collection. Generally, these aerial robots need more degrees of freedom than a standard quadrotor. While there is active research of over-actuated, omnidirectional MAVs and aerial manipulators as well as VTOL and hybrid platforms, the two concepts have not been combined. We address the problem of conceptualization, characterization, control, and testing of a 5DOF rotary-/fixed-wing hybrid, tilt-rotor, split tilt-wing, nearly omnidirectional aerial robot. We present an elegant solution with a minimal set of actuators and that does not need any classical control surfaces or flaps. The concept is validated in a wind tunnel study and in multiple flights with forward and backward transitions. Fixed-wing flight speeds up to 10 m/s were reached, with a power reduction of 30% as compared to rotary wing flight.

Long-Duration Fully Autonomous Operation of Rotorcraft Unmanned Aerial Systems for Remote-Sensing Data Acquisition

Recent applications of unmanned aerial systems (UAS) to precision agriculture have shown increased ease and efficiency in data collection at precise remote locations. However, further enhancement of the field requires operation over long periods of time, e.g. days or weeks. This has so far been impractical due to the limited flight times of such platforms and the requirement of humans in the loop for operation. To overcome these limitations, we propose a fully autonomous rotorcraft UAS that is capable of performing repeated flights for long-term observation missions without any human intervention. We address two key technologies that are critical for such a system: full platform autonomy to enable mission execution independently from human operators and the ability of vision-based precision landing on a recharging station for automated energy replenishment. High-level autonomous decision making is implemented as a hierarchy of master and slave state machines. Vision-based precision landing is enabled by estimating the landing pad's pose using a bundle of AprilTag fiducials configured for detection from a wide range of altitudes. We provide an extensive evaluation of the landing pad pose estimation accuracy as a function of the bundle's geometry. The functionality of the complete system is demonstrated through two indoor experiments with a duration of 11 and 10.6 hours, and one outdoor experiment with a duration of 4 hours. The UAS executed 16, 48 and 22 flights respectively during these experiments. In the outdoor experiment, the ratio between flying to collect data and charging was 1 to 10, which is similar to past work in this domain. All flights were fully autonomous with no human in the loop. To our best knowledge this is the first research publication about the long-term outdoor operation of a quadrotor system with no human interaction.

Disturbance Estimation and Rejection for High-Precision Multirotor Position Control

Many multirotor Unmanned Aerial Systems applications have a critical need for precise position control in environments with strong dynamic external disturbances such as wind gusts or ground and wall effects. Moreover, to maximize flight time, small multirotor platforms have to operate within strict constraints on payload and thus computational performance. In this paper, we present the design and experimental comparison of Model Predictive and PID multirotor position controllers augmented with a disturbance estimator to reject strong wind gusts up to 12 m/s and ground effect. For disturbance estimation, we compare Extended and Unscented Kalman filtering. In extensive in- and outdoor flight tests, we evaluate the suitability of the developed control and estimation algorithms to run on a computationally constrained platform. This allows to draw a conclusion on whether potential performance improvements justify the increased computational complexity of MPC for multirotor position control and UKF for disturbance estimation.

On Flying Backwards: Preventing Run-away of Small, Low-speed, Fixed-wing UAVs in Strong Winds

Small, low-speed fixed-wing Unmanned Aerial Vehicles (UAVs) operating autonomously, beyond-visual-line-of-sight (BVLOS) will inevitably encounter winds rising to levels near or exceeding the vehicles' nominal airspeed. In this paper, we develop a nonlinear lateral-directional path following guidance law with explicit consideration of online wind estimates. Energy efficient airspeed reference compensation logic is developed for excess wind scenarios (i.e. when the wind speed rises above the airspeed), enabling either mitigation, prevention, or over-powering of excess wind induced run-away from a given path. The developed guidance law is demonstrated on a representative small, low-speed test UAV in two flight experiments conducted in mountainous regions of Switzerland with strong, turbulent wind conditions, gusts reaching up to 13 meters per second. We demonstrate track-keeping errors of less than 1 meter consistently maintained during a representative duration of gusting, excess winds and a mean ground speed undershoot of 0.5 meters per second from the commanded minimum forward ground speed demonstrated in over 5 minutes of the showcased flight results.

Attitude- and Cruise Control of a VTOL Tiltwing UAV

This paper presents the mathematical modeling, controller design, and flight-testing of an over-actuated Vertical Take-off and Landing (VTOL) tiltwing Unmanned Aerial Vehicle (UAV). Based on simplified aerodynamics and first-principles, a dynamical model of the UAV is developed which captures key aerodynamic effects including propeller slipstream on the wing and post-stall characteristics of the airfoils. The model-based steady-state flight envelope and the corresponding trim-actuation is analyzed and the overactuation of the UAV solved by optimizing for, e.g., power-optimal trims. The developed control system is composed of two controllers: First, a low-level attitude controller based on dynamic inversion and a daisy-chaining approach to handle allocation of redundant actuators. Secondly, a higher-level cruise controller to track a desired vertical velocity. It is based on a linearization of the system and look-up tables to determine the strong and nonlinear variation of the trims throughout the flight-envelope. We demonstrate the performance of the control-system for all flight phases (hover, transition, cruise) in extensive flight-tests.

L1 guidance logic extension for small UAVs: handling high winds and small loiter radii

L1 guidance logic is one of the most widely used path following controllers for small fixed-wing unmanned aerial vehicles (UAVs), primarily due to its simplicity (low-cost implementation on embedded on-board processors, e.g. micro-controllers) and ability to track both circles and lines, which make up the vast majority of a typical fixed-wing vehicle's flight plan. The logic was later extended for speed independent dynamic similarity with an adaptive L1 distance, a formulation currently used on common open-source autopilot platforms. Two primary drawbacks remain, specific to small, slow flying fixed-wing UAVs; namely, 1) the combination of low operator defined gains and high ground speeds may violate the bounds of the algorithms convergence region for the case of loiter circles with small radii and 2) L1 logic breaks down when wind speeds exceed the vehicle's airspeed, another common predicament for small, slow-flying UAVs. This brief presents simple extensions to this extensively field tested algorithm, allowing legacy operators to keep existing controller tunings while taking advantage of the enhanced performance and safety features developed within.

Nonlinear Model Predictive Guidance for Fixed-wing UAVs Using Identified Control Augmented Dynamics

As off-the-shelf (OTS) autopilots become more widely available and user-friendly and the drone market expands, safer, more efficient, and more complex motion planning and control will become necessary for fixed-wing aerial robotic platforms. Considering typical low-level attitude stabilization available on OTS flight controllers, this paper first develops an approach for modeling and identification of the control augmented dynamics for a small fixed-wing Unmanned Aerial Vehicle (UAV). A high-level Nonlinear Model Predictive Controller (NMPC) is subsequently formulated for simultaneous airspeed stabilization, path following, and soft constraint handling, using the identified model for horizon propagation. The approach is explored in several exemplary flight experiments including path following of helix and connected Dubins Aircraft segments in high winds as well as a motor failure scenario. The cost function, insights on its weighting, and additional soft constraints used throughout the experimentation are discussed.

Free LSD: Prior-Free Visual Landing Site Detection for Autonomous Planes

Full autonomy for fixed-wing unmanned aerial vehicles (UAVs) requires the capability to autonomously detect potential landing sites in unknown and unstructured terrain, allowing for self-governed mission completion or handling of emergency situations. In this work, we propose a perception system addressing this challenge by detecting landing sites based on their texture and geometric shape without using any prior knowledge about the environment. The proposed method considers hazards within the landing region such as terrain roughness and slope, surrounding obstacles that obscure the landing approach path, and the local wind field that is estimated by the on-board EKF. The latter enables applicability of the proposed method on small-scale autonomous planes without landing gear. A safe approach path is computed based on the UAV dynamics, expected state estimation and actuator uncertainty, and the on-board computed elevation map. The proposed framework has been successfully tested on photo-realistic synthetic datasets and in challenging real-world environments.