Structure from Motion-based Motion Estimation and 3D Reconstruction of Unknown Shaped Space Debris

With the boost in the number of spacecraft launches in the current decades, the space debris problem is daily becoming significantly crucial. For sustainable space utilization, the continuous removal of space debris is the most severe problem for humanity. To maximize the reliability of the debris capture mission in orbit, accurate motion estimation of the target is essential. Space debris has lost its attitude and orbit control capabilities, and its shape is unknown due to the break. This paper proposes the Structure from Motion-based algorithm to perform unknown shaped space debris motion estimation with limited resources, where only 2D images are required as input. The method then outputs the reconstructed shape of the unknown object and the relative pose trajectory between the target and the camera simultaneously, which are exploited to estimate the target's motion. The method is quantitatively validated with the realistic image dataset generated by the microgravity experiment in a 2D air-floating testbed and 3D kinematic simulation.

Space Debris Reliable Capturing by a Dual-Arm Orbital Robot: Detumbling and Caging

A chaser satellite equipped with robotic arms can capture space debris and manipulate it for use in more advanced missions such as refueling and deorbiting. To facilitate capturing, a caging-based strategy has been proposed to simplify the control system. Caging involves geometrically constraining the motion of the target debris, and is achieved via position control. However, if the target is spinning at a high speed, direct caging may result in unsuccessful constraints or hardware destruction; therefore, the target should be de-tumbled before capture. To address this problem, this study proposes a repeated contact-based method that uses impedance control to mitigate the momentum of the target. In this study, we analyzed the proposed detumbling technique from the perspective of impedance parameters. We investigated their effects through a parametric analysis and demonstrated the successful detumbling and caging sequence of a microsatellite as representative of space debris. The contact forces decreased during the detumbling sequence compared with direct caging. Further, the proposed detumbling and caging sequence was validated through simulations and experiments using a dual-arm air-floating robot in two-dimensional microgravity emulating testbed.

Multi-modal Affect Analysis using standardized data within subjects in the Wild

Human affective recognition is an important factor in human-computer interaction. However, the method development with in-the-wild data is not yet accurate enough for practical usage. In this paper, we introduce the affective recognition method focusing on facial expression (EXP) and valence-arousal calculation that was submitted to the Affective Behavior Analysis in-the-wild (ABAW) 2021 Contest. When annotating facial expressions from a video, we thought that it would be judged not only from the features common to all people, but also from the relative changes in the time series of individuals. Therefore, after learning the common features for each frame, we constructed a facial expression estimation model and valence-arousal model using time-series data after combining the common features and the standardized features for each video. Furthermore, the above features were learned using multi-modal data such as image features, AU, Head pose, and Gaze. In the validation set, our model achieved a facial expression score of 0.546. These verification results reveal that our proposed framework can improve estimation accuracy and robustness effectively.

Action Units Recognition Using Improved Pairwise Deep Architecture

Facial Action Units (AUs) represent a set of facial muscular activities and various combinations of AUs can represent a wide range of emotions. AU recognition is often used in many applications, including marketing, healthcare, education, and so forth. Although a lot of studies have developed various methods to improve recognition accuracy, it still remains a major challenge for AU recognition. In the Affective Behavior Analysis in-the-wild (ABAW) 2020 competition, we proposed a new automatic Action Units (AUs) recognition method using a pairwise deep architecture to derive the Pseudo-Intensities of each AU and then convert them into predicted intensities. This year, we introduced a new technique to last year's framework to further reduce AU recognition errors due to temporary face occlusion such as hands on face or large face orientation. We obtained a score of 0.65 in the validation data set for this year's competition.

Action Units Recognition by Pairwise Deep Architecture

In this paper, we propose a new automatic Action Units (AUs) recognition method used in a competition, Affective Behavior Analysis in-the-wild (ABAW). Our method tackles a problem of AUs label inconsistency among subjects by using pairwise deep architecture. While the baseline score is 0.31, our method achieved 0.67 in validation dataset of the competition.