Planning Goals for Exploration

Dropped into an unknown environment, what should an agent do to quickly learn about the environment and how to accomplish diverse tasks within it? We address this question within the goal-conditioned reinforcement learning paradigm, by identifying how the agent should set its goals at training time to maximize exploration. We propose "Planning Exploratory Goals" (PEG), a method that sets goals for each training episode to directly optimize an intrinsic exploration reward. PEG first chooses goal commands such that the agent's goal-conditioned policy, at its current level of training, will end up in states with high exploration potential. It then launches an exploration policy starting at those promising states. To enable this direct optimization, PEG learns world models and adapts sampling-based planning algorithms to "plan goal commands". In challenging simulated robotics environments including a multi-legged ant robot in a maze, and a robot arm on a cluttered tabletop, PEG exploration enables more efficient and effective training of goal-conditioned policies relative to baselines and ablations. Our ant successfully navigates a long maze, and the robot arm successfully builds a stack of three blocks upon command. Website: https://penn-pal-lab.github.io/peg/

DA-CIL: Towards Domain Adaptive Class-Incremental 3D Object Detection

Deep learning has achieved notable success in 3D object detection with the advent of large-scale point cloud datasets. However, severe performance degradation in the past trained classes, i.e., catastrophic forgetting, still remains a critical issue for real-world deployment when the number of classes is unknown or may vary. Moreover, existing 3D class-incremental detection methods are developed for the single-domain scenario, which fail when encountering domain shift caused by different datasets, varying environments, etc. In this paper, we identify the unexplored yet valuable scenario, i.e., class-incremental learning under domain shift, and propose a novel 3D domain adaptive class-incremental object detection framework, DA-CIL, in which we design a novel dual-domain copy-paste augmentation method to construct multiple augmented domains for diversifying training distributions, thereby facilitating gradual domain adaptation. Then, multi-level consistency is explored to facilitate dual-teacher knowledge distillation from different domains for domain adaptive class-incremental learning. Extensive experiments on various datasets demonstrate the effectiveness of the proposed method over baselines in the domain adaptive class-incremental learning scenario.

Machine-learning based methodologies for 3d x-ray measurement, characterization and optimization for buried structures in advanced ic packages

For over 40 years lithographic silicon scaling has driven circuit integration and performance improvement in the semiconductor industry. As silicon scaling slows down, the industry is increasingly dependent on IC package technologies to contribute to further circuit integration and performance improvements. This is a paradigm shift and requires the IC package industry to reduce the size and increase the density of internal interconnects on a scale which has never been done before. Traditional package characterization and process optimization relies on destructive techniques such as physical cross-sections and delayering to extract data from internal package features. These destructive techniques are not practical with today's advanced packages. In this paper we will demonstrate how data acquired non-destructively with a 3D X-ray microscope can be enhanced and optimized using machine learning, and can then be used to measure, characterize and optimize the design and production of buried interconnects in advanced IC packages. Test vehicles replicating 2.5D and HBM construction were designed and fabricated, and digital data was extracted from these test vehicles using 3D X-ray and machine learning techniques. The extracted digital data was used to characterize and optimize the design and production of the interconnects and demonstrates a superior alternative to destructive physical analysis. We report an mAP of 0.96 for 3D object detection, a dice score of 0.92 for 3D segmentation, and an average of 2.1um error for 3D metrology on the test dataset. This paper is the first part of a multi-part report.