Pheromone-based Learning of Optimal Reasoning Paths

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities through chain-of-thought prompting, yet discovering effective reasoning methods for complex problems remains challenging due to the vast space of possible intermediate steps. We introduce Ant Colony Optimization-guided Tree of Thought (ACO-ToT), a novel algorithm that combines ACO with LLMs to discover optimal reasoning paths for complex problems efficiently. Drawing inspiration from Hebbian learning in neurological systems, our method employs a collection of distinctly fine-tuned LLM "ants" to traverse and lay pheromone trails through a centralized tree of thought, with each ant's movement governed by a weighted combination of existing pheromone trails and its own specialized expertise. The algorithm evaluates complete reasoning paths using a mixture-of-experts-based scoring function, with pheromones reinforcing productive reasoning paths across iterations. Experiments on three challenging reasoning tasks (GSM8K, ARC-Challenge, and MATH) demonstrate that ACO-ToT performs significantly better than existing chain-of-thought optimization approaches, suggesting that incorporating biologically inspired collective search mechanisms into LLM inference can substantially enhance reasoning capabilities.

MINDSTORES: Memory-Informed Neural Decision Synthesis for Task-Oriented Reinforcement in Embodied Systems

While large language models (LLMs) have shown promising capabilities as zero-shot planners for embodied agents, their inability to learn from experience and build persistent mental models limits their robustness in complex open-world environments like Minecraft. We introduce MINDSTORES, an experience-augmented planning framework that enables embodied agents to build and leverage mental models through natural interaction with their environment. Drawing inspiration from how humans construct and refine cognitive mental models, our approach extends existing zero-shot LLM planning by maintaining a database of past experiences that informs future planning iterations. The key innovation is representing accumulated experiences as natural language embeddings of (state, task, plan, outcome) tuples, which can then be efficiently retrieved and reasoned over by an LLM planner to generate insights and guide plan refinement for novel states and tasks. Through extensive experiments in the MineDojo environment, a simulation environment for agents in Minecraft that provides low-level controls for Minecraft, we find that MINDSTORES learns and applies its knowledge significantly better than existing memory-based LLM planners while maintaining the flexibility and generalization benefits of zero-shot approaches, representing an important step toward more capable embodied AI systems that can learn continuously through natural experience.

Deep Learning Models for Flapping Fin Unmanned Underwater Vehicle Control System Gait Optimization

The last few decades have led to the rise of research focused on propulsion and control systems for bio-inspired unmanned underwater vehicles (UUVs), which provide more maneuverable alternatives to traditional UUVs in underwater missions. Recent work has explored the use of time-series neural network surrogate models to predict thrust and power from vehicle design and fin kinematics. We develop a search-based inverse model that leverages kinematics-to-thrust and kinematics-to-power neural network models for control system design. Our inverse model finds a set of fin kinematics with the multi-objective goal of reaching a target thrust under power constraints while creating a smooth kinematics transition between flapping cycles. We demonstrate how a control system integrating this inverse model can make online, cycle-to-cycle adjustments to prioritize different system objectives, with improvements in increasing thrust generation or reducing power consumption of any given movement upwards of 0.5 N and 3.0 W in a range of 2.2 N and 9.0 W. As propulsive efficiency is of utmost importance for flapping-fin UUVs in order to extend their range and endurance for essential operations but lacks prior research, we develop a non-dimensional figure of merit (FOM), derived from measures of propulsive efficiency, that is able to evaluate different fin designs and kinematics, and allow for comparison with other bio-inspired platforms. We use the developed FOM to analyze optimal gaits and compare the performance between different fin materials, providing a better understanding of how fin materials affect thrust generation and propulsive efficiency and allowing us to inform control systems and weight for efficiency on the developed inverse gait-selector model.

Computational Approaches for Modeling Power Consumption on an Underwater Flapping Fin Propulsion System

The last few decades have led to the rise of research focused on propulsion and control systems for bio-inspired unmanned underwater vehicles (UUVs), which provide more maneuverable alternatives to traditional UUVs in underwater missions. Propulsive efficiency is of utmost importance for flapping-fin UUVs in order to extend their range and endurance for essential operations. To optimize for different gait performance metrics, we develop a non-dimensional figure of merit (FOM), derived from measures of propulsive efficiency, that is able to evaluate different fin designs and kinematics, and allow for comparison with other bio-inspired platforms. We create and train computational models using experimental data, and use these models to predict thrust and power under different fin operating states, providing efficiency profiles. We then use the developed FOM to analyze optimal gaits and compare the performance between different fin materials. These comparisons provide a better understanding of how fin materials affect our thrust generation and propulsive efficiency, allowing us to inform control systems and weight for efficiency on an inverse gait-selector model.

Data-Driven Machine Learning Models for a Multi-Objective Flapping Fin Unmanned Underwater Vehicle Control System

Flapping-fin unmanned underwater vehicle (UUV) propulsion systems provide high maneuverability for naval tasks such as surveillance and terrain exploration. Recent work has explored the use of time-series neural network surrogate models to predict thrust from vehicle design and fin kinematics. We develop a search-based inverse model that leverages a kinematics-to-thrust neural network model for control system design. Our inverse model finds a set of fin kinematics with the multi-objective goal of reaching a target thrust and creating a smooth kinematic transition between flapping cycles. We demonstrate how a control system integrating this inverse model can make online, cycle-to-cycle adjustments to prioritize different system objectives.