How Many Bytes Can You Take Out Of Brain-To-Text Decoding?

Brain-computer interfaces have promising medical and scientific applications for aiding speech and studying the brain. In this work, we propose an information-based evaluation metric for brain-to-text decoders. Using this metric, we examine two methods to augment existing state-of-the-art continuous text decoders. We show that these methods, in concert, can improve brain decoding performance by upwards of 40% when compared to a baseline model. We further examine the informatic properties of brain-to-text decoders and show empirically that they have Zipfian power law dynamics. Finally, we provide an estimate for the idealized performance of an fMRI-based text decoder. We compare this idealized model to our current model, and use our information-based metric to quantify the main sources of decoding error. We conclude that a practical brain-to-text decoder is likely possible given further algorithmic improvements.

Automated Layout Design and Control of Robust Cooperative Grasped-Load Aerial Transportation Systems

We present a novel approach to cooperative aerial transportation through a team of drones, using optimal control theory and a hierarchical control strategy. We assume the drones are connected to the payload through rigid attachments, essentially transforming the whole system into a larger flying object with "thrust modules" at the attachment locations of the drones. We investigate the optimal arrangement of the thrust modules around the payload, so that the resulting system is robust to disturbances. We choose the $\mathcal{H}_2$ norm as a measure of robustness, and propose an iterative optimization routine to compute the optimal layout of the vehicles around the object. We experimentally validate our approach using four drones and comparing the disturbance rejection performances achieved by two different layouts (the optimal one and a sub-optimal one), and observe that the results match our predictions.

Brain encoding models based on multimodal transformers can transfer across language and vision

Encoding models have been used to assess how the human brain represents concepts in language and vision. While language and vision rely on similar concept representations, current encoding models are typically trained and tested on brain responses to each modality in isolation. Recent advances in multimodal pretraining have produced transformers that can extract aligned representations of concepts in language and vision. In this work, we used representations from multimodal transformers to train encoding models that can transfer across fMRI responses to stories and movies. We found that encoding models trained on brain responses to one modality can successfully predict brain responses to the other modality, particularly in cortical regions that represent conceptual meaning. Further analysis of these encoding models revealed shared semantic dimensions that underlie concept representations in language and vision. Comparing encoding models trained using representations from multimodal and unimodal transformers, we found that multimodal transformers learn more aligned representations of concepts in language and vision. Our results demonstrate how multimodal transformers can provide insights into the brain's capacity for multimodal processing.

Design and Control of a Midair Reconfigurable Quadcopter using Unactuated Hinges

This paper presents the design and control of a novel quadcopter capable of changing shape mid-flight, allowing for operation in four configurations with the capability of sustained hover in three. The normally rigid connections between the arms of the quadcopter and the central body are replaced by free-rotating hinges that allow the arms to fold downward; no additional actuators beyond the four motors that drive the propellers are used. Configuration transitions are accomplished by either reducing or reversing the thrust forces produced by specific propellers during flight. Constraints placed on the control inputs of the vehicle prevent the arms from folding or unfolding unexpectedly, allowing for the use of existing quadcopter controllers and trajectory generation algorithms. For our experimental vehicle at hover, we find that these constraints result in a 36% reduction of the maximum yaw torque the vehicle can produce, but do not result in a reduction of the maximum thrust or roll and pitch torques. Furthermore, the ability to change configurations is shown to enable the vehicle to traverse small passages, perch on hanging wires, and perform simple grasping tasks.

Staging energy sources to extend flight time of a multirotor UAV

Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is presented to show the endurance benefit for multirotors on staging. We consider two specific problems in discrete staging -- optimal order of staging given a certain number of energy sources, and optimal partitioning of a given energy storage mass budget into a given number of stages. We then derive results for two continuously staged cases -- an internal combustion engine driving propellers, and a rocket engine. A fundamental flight time limit is seen for the internal combustion engine case, but not for the rocket engine, suggesting that rocket engines could be a better choice in certain scenarios. Lastly, we conduct flight experiments on a custom two-stage battery-powered quadcopter. This quadcopter can eject a battery stage after consumption in-flight using a custom-designed mechanism, and continue hovering using the next stage. The experimental flight times are compared with those predicted from the analysis for our vehicle. The values match well which validates the presented staging analysis.