OptiGrasp: Optimized Grasp Pose Detection Using RGB Images for Warehouse Picking Robots

In warehouse environments, robots require robust picking capabilities to manage a wide variety of objects. Effective deployment demands minimal hardware, strong generalization to new products, and resilience in diverse settings. Current methods often rely on depth sensors for structural information, which suffer from high costs, complex setups, and technical limitations. Inspired by recent advancements in computer vision, we propose an innovative approach that leverages foundation models to enhance suction grasping using only RGB images. Trained solely on a synthetic dataset, our method generalizes its grasp prediction capabilities to real-world robots and a diverse range of novel objects not included in the training set. Our network achieves an 82.3\% success rate in real-world applications. The project website with code and data will be available at http://optigrasp.github.io.

Hierarchical Control Strategy for Moving A Robot Manipulator Between Small Containers

In this paper, we study the implementation of a model predictive controller (MPC) for the task of object manipulation in a highly uncertain environment (e.g., picking objects from a semi-flexible array of densely packed bins). As a real-time perception-driven feedback controller, MPC is robust to the uncertainties in this environment. However, our experiment shows MPC cannot control a robot to complete a sequence of motions in a heavily occluded environment due to its myopic nature. It will benefit from adding a high-level policy that adaptively adjusts the optimization problem for MPC.

NeuriCam: Video Super-Resolution and Colorization Using Key Frames

We present NeuriCam, a key-frame video super-resolution and colorization based system, to achieve low-power video capture from dual-mode IOT cameras. Our idea is to design a dual-mode camera system where the first mode is low power (1.1~mW) but only outputs gray-scale, low resolution and noisy video and the second mode consumes much higher power (100~mW) but outputs color and higher resolution images. To reduce total energy consumption, we heavily duty cycle the high power mode to output an image only once every second. The data from this camera system is then wirelessly streamed to a nearby plugged-in gateway, where we run our real-time neural network decoder to reconstruct a higher resolution color video. To achieve this, we introduce an attention feature filter mechanism that assigns different weights to different features, based on the correlation between the feature map and contents of the input frame at each spatial location. We design a wireless hardware prototype using off-the-shelf cameras and address practical issues including packet loss and perspective mismatch. Our evaluation shows that our dual-camera hardware reduces camera energy consumption while achieving an average gray-scale PSNR gain of 3.7~dB over prior video super resolution methods and 5.6~dB RGB gain over existing color propagation methods. Open-source code: https://github.com/vb000/NeuriCam.

Electrostatic Brakes Enable Individual Joint Control of Underactuated, Highly Articulated Robots

Highly articulated organisms serve as blueprints for incredibly dexterous mechanisms, but building similarly capable robotic counterparts has been hindered by the difficulties of developing electromechanical actuators with both the high strength and compactness of biological muscle. We develop a stackable electrostatic brake that has comparable specific tension and weight to that of muscles and integrate it into a robotic joint. Compared to electromechanical motors, our brake-equipped joint is four times lighter and one thousand times more power efficient while exerting similar holding torques. Our joint design enables a ten degree-of-freedom robot equipped with only one motor to manipulate multiple objects simultaneously. We also show that the use of brakes allows a two-fingered robot to perform in-hand re-positioning of an object 45% more quickly and with 53% lower positioning error than without brakes. Relative to fully actuated robots, our findings suggest that robots equipped with such electrostatic brakes will have lower weight, volume, and power consumption yet retain the ability to reach arbitrary joint configurations.

Iterative Semi-parametric Dynamics Model Learning For Autonomous Racing

Accurately modeling robot dynamics is crucial to safe and efficient motion control. In this paper, we develop and apply an iterative learning semi-parametric model, with a neural network, to the task of autonomous racing with a Model Predictive Controller (MPC). We present a novel non-linear semi-parametric dynamics model where we represent the known dynamics with a parametric model, and a neural network captures the unknown dynamics. We show that our model can learn more accurately than a purely parametric model and generalize better than a purely non-parametric model, making it ideal for real-world applications where collecting data from the full state space is not feasible. We present a system where the model is bootstrapped on pre-recorded data and then updated iteratively at run time. Then we apply our iterative learning approach to the simulated problem of autonomous racing and show that it can safely adapt to modified dynamics online and even achieve better performance than models trained on data from manual driving.

Online Dynamic Trajectory Optimization and Control for a Quadruped Robot

Legged robot locomotion requires the planning of stable reference trajectories, especially while traversing uneven terrain. The proposed trajectory optimization framework is capable of generating dynamically stable base and footstep trajectories for multiple steps. The locomotion task can be defined with contact locations, base motion or both, making the algorithm suitable for multiple scenarios (e.g., presence of moving obstacles). The planner uses a simplified momentum-based task space model for the robot dynamics, allowing computation times that are fast enough for online replanning.This fast planning capabilitiy also enables the quadruped to accommodate for drift and environmental changes. The algorithm is tested on simulation and a real robot across multiple scenarios, which includes uneven terrain, stairs and moving obstacles. The results show that the planner is capable of generating stable trajectories in the real robot even when a box of 15 cm height is placed in front of its path at the last moment.

Variable Autonomy of Whole-body Control for Inspection and Intervention in Industrial Environments using Legged Robots

The deployment of robots in industrial and civil scenarios is a viable solution to protect operators from danger and hazards. Shared autonomy is paramount to enable remote control of complex systems such as legged robots, allowing the operator to focus on the essential tasks instead of overly detailed execution. To realize this, we proposed a comprehensive control framework for inspection and intervention using a legged robot and validated the integration of multiple loco-manipulation algorithms optimised for improving the remote operation. The proposed control offers 3 operation modes: fully automated, semi-autonomous, and the haptic interface receiving onsite physical interaction for assisting teleoperation. Our contribution is the design of a QP-based semi-analytical whole-body control, which is the key to the various task completion subject to internal and external constraints. We demonstrated the versatility of the whole-body control in terms of decoupling tasks, singularity toleration and constraint satisfaction. We deployed our solution in field trials and evaluated in an emergency setting by an E-stop while the robot was clearing road barrier and traversing difficult terrains.

Adversarial Generation of Informative Trajectories for Dynamics System Identification

Dynamic System Identification approaches usually heavily rely on the evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system identification. We propose to tackle this issue by using multiple shorter cyclic trajectories, which can be generated in parallel, and subsequently combined together to achieve the same effect as a longer trajectory. Crucially, we show how to scale this approach even further by increasing the generation speed and quality of the dataset through the use of generative adversarial network (GAN) based architectures to produce a large databases of valid and diverse excitation trajectories. To the best of our knowledge, this is the first robotics work to explore system identification with multiple cyclic trajectories and to develop GAN-based techniques for scaleably producing excitation trajectories that are diverse in both control parameter and inertial parameter spaces. We show that our approach dramatically accelerates trajectory optimisation, while simultaneously providing more accurate system identification than the conventional approach.

Bounded haptic teleoperation of a quadruped robot's foot posture for sensing and manipulation

This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian impedance controllers cascaded by a null space projector. The contact forces acting on supporting feet are optimized by quadratic programming (QP). The foot's Cartesian impedance controller may also estimate contact forces from trajectory tracking errors, and relay the force-feedback to the operator. A 7D haptic joystick, Sigma.7, transmits motion commands to the quadruped robot ANYmal, and renders the force feedback. Furthermore, the joystick's motion is bounded by mapping the foot's feasible force polytope constrained by the friction cones and torque limits in order to prevent the operator from driving the robot to slipping or falling over. Experimental results demonstrate the efficiency of the proposed framework.

Fractal Impedance for Passive Controllers

During the last decade, there has been increasing interest in new control frameworks to move robots from their industrial cages to unstructured environments where they may coexist with humans. Despite significant improvement in some specific applications (e.g., medical robotics), there is still the need of a general control framework to improve the robots' dynamics interaction performance without limiting system safety. The passive control framework has shown promising results in this direction; however, it relies on virtual energy tanks that can guarantee passivity as long as they do not run out of energy. In this paper, a fractal attractor is proposed to implement a variable impedance controller that is able to retain passivity without relying on the energy tank framework. The results prove that the proposed method can accurately track trajectories and apply forces at the end-effector. Furthermore, it can automatically deal with the extra energy introduced by changes in interaction conditions, null-space controller and environment. Therefore, These properties make the controller ideal for applications where the dynamic interaction at the end-effector is difficult to be characterized in advance, such as human-robot interaction and unknown dynamics.