Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach

This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network, in which we propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV and also leverage the MUAV as a recharging power source. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network. Depending on whether there is charging between the two UAVs, two optimization problems are formulated. To solve them, two multi-agent deep reinforcement learning (DRL) approaches are proposed, which are centralized training multi-agent deep deterministic policy gradient (CT-MADDPG) and multi-agent deep deterministic policy option critic (MADDPOC). It is shown that the CT-MADDPG can greatly reduce the requirement on the computing capability of the UAV hardware, and the proposed MADDPOC is able to support low-level multi-agent cooperative learning in the continuous action domains, which has great advantages over the existing option-based hierarchical DRL that only support single-agent learning and discrete actions.

Planning a Sequence of Base Positions for a Mobile Manipulator to Perform Multiple Pick-and-Place Tasks

In this paper, we present a planner that plans a sequence of base positions for a mobile manipulator to efficiently and robustly collect objects stored in distinct trays. We achieve high efficiency by exploring the common areas where a mobile manipulator can grasp objects stored in multiple trays simultaneously and move the mobile manipulator to the common areas to reduce the time needed for moving the mobile base. We ensure robustness by optimizing the base position with the best clearance to positioning uncertainty so that a mobile manipulator can complete the task even if there is a certain deviation from the planned base positions. Besides, considering different styles of object placement in the tray, we analyze feasible schemes for dynamically updating the base positions based on either the remaining objects or the target objects to be picked in one round of the tasks. In the experiment part, we examine our planner on various scenarios, including different object placement: (1) Regularly placed toy objects; (2) Randomly placed industrial parts; and different schemes for online execution: (1) Apply globally static base positions; (2) Dynamically update the base positions. The experiment results demonstrate the efficiency, robustness and feasibility of the proposed method.

Selecting and Designing Grippers for an Assembly Task in a Structured Approach

In this paper, we present a structured approach of selecting and designing a set of grippers for an assembly task. Compared to current experience-based gripper design method, our approach accelerates the design process by automatically generating a set of initial design options on gripper type and parameters according to the CAD models of assembly components. We use mesh segmentation techniques to segment the assembly components and fit the segmented parts with shape primitives, according to the predefined correspondence between primitive shape and gripper type, suitable gripper types and parameters can be selected and extracted from the fitted shape primitives. Then considering the assembly constraints, applicable gripper types and parameters can be filtered from the initial options. Among the applicable gripper configurations, we further minimize the required number of grippers for performing the assembly task, by exploring the gripper that is able to handle multiple assembly components during the assembly. Finally, the feasibility of the designed grippers are experimentally verified by assembling a part of an industrial product.

Planning an Efficient and Robust Base Sequence for a Mobile Manipulator Performing Multiple Pick-and-place Tasks

In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a sequence of base positions that reduce the number of necessary base movements for collecting the target objects, the base sequence is robust in that the mobile manipulator is able to complete the part-supply task even there is certain deviation from the planned base positions. Our experiments demonstrate both the efficiency compared to regular base sequence and the feasibility in real world applications.