TMIQ: Quantifying Test and Measurement Domain Intelligence in Large Language Models

The Test and Measurement domain, known for its strict requirements for accuracy and efficiency, is increasingly adopting Generative AI technologies to enhance the performance of data analysis, automation, and decision-making processes. Among these, Large Language Models (LLMs) show significant promise for advancing automation and precision in testing. However, the evaluation of LLMs in this specialized area remains insufficiently explored. To address this gap, we introduce the Test and Measurement Intelligence Quotient (TMIQ), a benchmark designed to quantitatively assess LLMs across a wide range of electronic engineering tasks. TMIQ offers a comprehensive set of scenarios and metrics for detailed evaluation, including SCPI command matching accuracy, ranked response evaluation, Chain-of-Thought Reasoning (CoT), and the impact of output formatting variations required by LLMs on performance. In testing various LLMs, our findings indicate varying levels of proficiency, with exact SCPI command match accuracy ranging from around 56% to 73%, and ranked matching first-position scores achieving around 33% for the best-performing model. We also assess token usage, cost-efficiency, and response times, identifying trade-offs between accuracy and operational efficiency. Additionally, we present a command-line interface (CLI) tool that enables users to generate datasets using the same methodology, allowing for tailored assessments of LLMs. TMIQ and the CLI tool provide a rigorous, reproducible means of evaluating LLMs for production environments, facilitating continuous monitoring and identifying strengths and areas for improvement, and driving innovation in their selections for applications within the Test and Measurement industry.

Single-Photon Counting Receivers for 6G Optical Wireless Communications

Optical wireless communication (OWC) offers several complementary advantages to radio-frequency (RF) wireless networks such as its massive available spectrum; hence, it is widely anticipated that OWC will assume a pivotal role in the forthcoming sixth generation (6G) wireless communication networks. Although significant progress has been achieved in OWC over the past decades, the outage induced by occasionally low received optical power continues to pose a key limiting factor for its deployment. In this work, we discuss the potential role of single-photon counting (SPC) receivers as a promising solution to overcome this limitation. We provide an overview of the state-of-the-art of OWC systems utilizing SPC receivers and identify several critical areas of open problems that warrant further research in the future.

Evaluation of SiPM based Optical Receiver for Low Energy Devices

Silicon Photomultipliers (SiPMs) are photon-counting detectors with great potential to improve the sensitivity of optical receivers. However, its recovery time and dark count rate could limit its dynamic range that effectively detects single photons, hence limiting the sensitivity. Recent studies of SiPMs in communication focus on the speed rather than the power consumption of the receiver. The gain and bandwidth of the designed receiver circuit to read out SiPM output are much higher than required for the target data rate. Additionally, the SiPM experiments for optical communication are performed using an offline method which uses instruments including oscilloscopes and personal computers to process chunks of the transmitted data. In this work, we have developed an embedded realtime system FPGA-based platform to evaluate a commercially available 1 mm-sq SiPM. Moreover, simulations for investigating the relationship between the electrical bandwidth of the receiver circuit and the target data rate to achieve a target Bit Error Rate (BER) are established. The experimental results show that the implemented real-time system achieves a Bit Error Rate (BER) of 1E-3 with less than 5 pW of incident optical power with an average of 11.35 photons per bit at 100 kbps under 620 nm LED, approaching the Poisson limit. It was found that the minimum receiver electrical bandwidth required to maintain the Poisson limit is 50 times higher than the target data rate. The analysis of the minimum receiver bandwidth in photon counting and BER enables the potential future adoption of this receiver technology in high sensitivity applications in underwater and visible light communications, especially for low energy devices.