WorldBench: Disambiguating Physics for Diagnostic Evaluation of World Models

Recent advances in generative foundational models, often termed "world models," have propelled interest in applying them to critical tasks like robotic planning and autonomous system training. For reliable deployment, these models must exhibit high physical fidelity, accurately simulating real-world dynamics. Existing physics-based video benchmarks, however, suffer from entanglement, where a single test simultaneously evaluates multiple physical laws and concepts, fundamentally limiting their diagnostic capability. We introduce WorldBench, a novel video-based benchmark specifically designed for concept-specific, disentangled evaluation, allowing us to rigorously isolate and assess understanding of a single physical concept or law at a time. To make WorldBench comprehensive, we design benchmarks at two different levels: 1) an evaluation of intuitive physical understanding with concepts such as object permanence or scale/perspective, and 2) an evaluation of low-level physical constants and material properties such as friction coefficients or fluid viscosity. When SOTA video-based world models are evaluated on WorldBench, we find specific patterns of failure in particular physics concepts, with all tested models lacking the physical consistency required to generate reliable real-world interactions. Through its concept-specific evaluation, WorldBench offers a more nuanced and scalable framework for rigorously evaluating the physical reasoning capabilities of video generation and world models, paving the way for more robust and generalizable world-model-driven learning.

2-Factor Retrieval for Improved Human-AI Decision Making in Radiology

Human-machine teaming in medical AI requires us to understand to what degree a trained clinician should weigh AI predictions. While previous work has shown the potential of AI assistance at improving clinical predictions, existing clinical decision support systems either provide no explainability of their predictions or use techniques like saliency and Shapley values, which do not allow for physician-based verification. To address this gap, this study compares previously used explainable AI techniques with a newly proposed technique termed '2-factor retrieval (2FR)', which is a combination of interface design and search retrieval that returns similarly labeled data without processing this data. This results in a 2-factor security blanket where: (a) correct images need to be retrieved by the AI; and (b) humans should associate the retrieved images with the current pathology under test. We find that when tested on chest X-ray diagnoses, 2FR leads to increases in clinician accuracy, with particular improvements when clinicians are radiologists and have low confidence in their decision. Our results highlight the importance of understanding how different modes of human-AI decision making may impact clinician accuracy in clinical decision support systems.

AtOMICS: A neural network-based Automated Optomechanical Intelligent Coupling System for testing and characterization of silicon photonics chiplets

Recent advances in silicon photonics promise to revolutionize modern technology by improving performance of everyday devices in multiple fields. However, as the industry moves into a mass fabrication phase, the problem of effective testing of integrated silicon photonics devices remains to be solved. A cost-efficient manner that reduces schedule risk needs to involve automated testing of multiple devices that share common characteristics such as input-output coupling mechanisms, but at the same time needs to be generalizable to multiple types of devices and scenarios. In this paper we present a neural network-based automated system designed for in-plane fiber-chip-fiber testing, characterization, and active alignment of silicon photonic devices that use process-design-kit library edge couplers. The presented approach combines state-of-the-art computer vision techniques with time-series analysis, in order to control a testing setup that can process multiple devices and can be easily tuned to incorporate additional hardware. The system can operate at vacuum or atmospheric pressures and maintains stability for fairly long time periods in excess of a month.