E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions

As vector representations have been pivotal in advancing natural language processing (NLP), some prior research has concentrated on creating embedding techniques for mathematical expressions by leveraging mathematically equivalent expressions. While effective, these methods are limited by the training data. In this work, we propose augmenting prior algorithms with larger synthetic dataset, using a novel e-graph-based generation scheme. This new mathematical dataset generation scheme, E-Gen, improves upon prior dataset-generation schemes that are limited in size and operator types. We use this dataset to compare embedding models trained with two methods: (1) training the model to generate mathematically equivalent expressions, and (2) training the model using contrastive learning to group mathematically equivalent expressions explicitly. We evaluate the embeddings generated by these methods against prior work on both in-distribution and out-of-distribution language processing tasks. Finally, we compare the performance of our embedding scheme against state-of-the-art large language models and demonstrate that embedding-based language processing methods perform better than LLMs on several tasks, demonstrating the necessity of optimizing embedding methods for the mathematical data modality.

Highlighting Named Entities in Input for Auto-Formulation of Optimization Problems

Operations research deals with modeling and solving real-world problems as mathematical optimization problems. While solving mathematical systems is accomplished by analytical software, formulating a problem as a set of mathematical operations has been typically done manually by domain experts. However, recent machine learning models have shown promise in converting textual problem descriptions to corresponding mathematical formulations. In this paper, we present an approach that converts linear programming word problems into meaning representations that are structured and can be used by optimization solvers. Our approach uses the named entity-based enrichment to augment the input and achieves state-of-the-art accuracy, winning the second task of the NL4Opt competition (https://nl4opt.github.io).