Expert Router: Orchestrating Efficient Language Model Inference through Prompt Classification

Large Language Models (LLMs) have experienced widespread adoption across scientific and industrial domains due to their versatility and utility for diverse tasks. Nevertheless, deploying and serving these models at scale with optimal throughput and latency remains a significant challenge, primarily because of the high computational and memory demands associated with LLMs. To tackle this limitation, we introduce Expert Router, a system designed to orchestrate multiple expert models efficiently, thereby enhancing scalability. Expert Router is a parallel inference system with a central routing gateway that distributes incoming requests using a clustering method. This approach effectively partitions incoming requests among available LLMs, maximizing overall throughput. Our extensive evaluations encompassed up to 1,000 concurrent users, providing comprehensive insights into the system's behavior from user and infrastructure perspectives. The results demonstrate Expert Router's effectiveness in handling high-load scenarios and achieving higher throughput rates, particularly under many concurrent users.

Evaluation of Parameterized Quantum Circuits: on the design, and the relation between classification accuracy, expressibility and entangling capability

Quantum computers promise improvements in terms of both computational speedup and increased accuracy. Relevant areas are optimization, chemistry and machine learning, of which we will focus on the latter. Much of the prior art focuses on determining computational speedup, but how do we know if a particular quantum circuit shows promise for achieving high classification accuracy? Previous work by Sim et al. proposed descriptors to characterize and compare Parameterized Quantum Circuits. In this work, we will investigate any potential relation between the classification accuracy and two of these descriptors, being expressibility and entangling capability. We will first investigate different types of gates in quantum circuits and the changes they incur on the decision boundary. From this, we will propose design criteria for constructing circuits. We will also numerically compare the classifications performance of various quantum circuits and their quantified measure of expressibility and entangling capability, as derived in previous work. From this, we conclude that the common approach to layer combinations of rotational gates and conditional rotational gates provides the best accuracy. We also show that, for our experiments on a limited number of circuits, a coarse-grained relationship exists between entangling capability and classification accuracy, as well as a more fine-grained correlation between expressibility and classification accuracy. Future research will need to be performed to quantify this relation.

Integration and Evaluation of Quantum Accelerators for Data-Driven User Functions

Quantum computers hold great promise for accelerating computationally challenging algorithms on noisy intermediate-scale quantum (NISQ) devices in the upcoming years. Much attention of the current research is directed to algorithmic research on artificial data that is disconnected from live systems, such as optimization of systems or training of learning algorithms. In this paper we investigate the integration of quantum systems into industry-grade system architectures. In this work we propose a system architecture for the integration of quantum accelerators. In order to evaluate our proposed system architecture we implemented various algorithms including a classical system, a gate-based quantum accelerator and a quantum annealer. This algorithm automates user habits using data-driven functions trained on real-world data. This also includes an evaluation of the quantum enhanced kernel, that previously was only evaluated on artificial data. In our evaluation, we showed that the quantum-enhanced kernel performs at least equally well to a classical state-of-the-art kernel. We also showed a low reduction in accuracy and latency numbers within acceptable bounds when running on the gate-based IBM quantum accelerator. We, therefore, conclude it is feasible to integrate NISQ-era devices in industry-grade system architecture in preparation for future hardware improvements.