Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting

Energy arbitrage is one of the most profitable sources of income for battery operators, generating revenues by buying and selling electricity at different prices. Forecasting these revenues is challenging due to the inherent uncertainty of electricity prices. Deep reinforcement learning (DRL) emerged in recent years as a promising tool, able to cope with uncertainty by training on large quantities of historical data. However, without access to future electricity prices, DRL agents can only react to the currently observed price and not learn to plan battery dispatch. Therefore, in this study, we combine DRL with time-series forecasting methods from deep learning to enhance the performance on energy arbitrage. We conduct a case study using price data from Alberta, Canada that is characterized by irregular price spikes and highly non-stationary. This data is challenging to forecast even when state-of-the-art deep learning models consisting of convolutional layers, recurrent layers, and attention modules are deployed. Our results show that energy arbitrage with DRL-enabled battery control still significantly benefits from these imperfect predictions, but only if predictors for several horizons are combined. Grouping multiple predictions for the next 24-hour window, accumulated rewards increased by 60% for deep Q-networks (DQN) compared to the experiments without forecasts. We hypothesize that multiple predictors, despite their imperfections, convey useful information regarding the future development of electricity prices through a "majority vote" principle, enabling the DRL agent to learn more profitable control policies.

The Grasp Reset Mechanism: An Automated Apparatus for Conducting Grasping Trials

Advancing robotic grasping and manipulation requires the ability to test algorithms and/or train learning models on large numbers of grasps. Towards the goal of more advanced grasping, we present the Grasp Reset Mechanism (GRM), a fully automated apparatus for conducting large-scale grasping trials. The GRM automates the process of resetting a grasping environment, repeatably placing an object in a fixed location and controllable 1-D orientation. It also collects data and swaps between multiple objects enabling robust dataset collection with no human intervention. We also present a standardized state machine interface for control, which allows for integration of most manipulators with minimal effort. In addition to the physical design and corresponding software, we include a dataset of 1,020 grasps. The grasps were created with a Kinova Gen3 robot arm and Robotiq 2F-85 Adaptive Gripper to enable training of learning models and to demonstrate the capabilities of the GRM. The dataset includes ranges of grasps conducted across four objects and a variety of orientations. Manipulator states, object pose, video, and grasp success data are provided for every trial.

C-Band Lithium Niobate on Silicon Carbide Surface Acoustic Wave Resonator with Figure-of-Merit of 124 at 6.5 GHz

In this work, we demonstrate a C-band shear-horizontal surface acoustic wave (SH-SAW) resonator with high electromechanical coupling (keff2) of 22% and a quality factor (Q) of 565 based on a thin-film lithium niobate (LN) on silicon carbide (SiC) platform, featuring an excellent figure-of-merit (FoM = keff2*Q ) of 124 at 6.5 GHz, the highest FoM reported in this frequency range. The resonator frequency upscaling is achieved through wavelength ($\lambda$) reduction and the use of thin aluminum (Al) electrodes. The LN/SiC waveguide and synchronous resonator design collectively enable effective acoustic energy confinement for a high FoM, even when the normalized thickness of LN approaches a scale of 0.5$\lambda$ to 1$\lambda$. To perform a comprehensive study, we also designed and fabricated five additional resonators, expending the $\lambda$ studied ranging from 480 to 800 nm, in the same 500 nm-thick transferred Y-cut thin-film LN on SiC. The fabricated SH-SAW resonators, operating from 5 to 8 GHz, experimentally demonstrate a keff2 from 20.3% to 22.9% and a Q from 350 to 575, thereby covering the entire C-band with excellent performance.

Thin-film Lithium Niobate on Insulator Surface Acoustic Wave Devices for 6G Centimeter Bands

In this work, we investigate the frequency scaling of shear-horizontal (S.H.) surface acoustic wave (SAW) resonators based on a lithium niobate on insulator (LNOI) substrate into the centimeter bands for 6G wireless systems. Prototyped resonators with wavelengths ranging between 240 nm and 400 nm were fabricated, and the experimental results exhibit a successful frequency scaling between 9.05 and 13.37 GHz. However, a noticeable performance degradation can be observed as the resonance frequency (fs) scales. Such an effect is expected to be caused by non-ideal helec/{\lambda} for smaller {\lambda} devices. The optimized LNOI SH-SAW with a {\lambda} of 400 nm exhibits a fs of 9.05 GHz, a keff2 of 15%, Qmax of 213 and a FoM of 32, which indicates a successful implementation for device targeting centimeter bands.

23.8 GHz Acoustic Filter in Periodically Poled Piezoelectric Film Lithium Niobate with 1.52 dB IL and 19.4% FBW

This paper reports the first piezoelectric acoustic filter in periodically poled piezoelectric film (P3F) lithium niobate (LiNbO3) at 23.8 GHz with low insertion loss (IL) of 1.52 dB and 3-dB fractional bandwidth (FBW) of 19.4%. The filter features a compact footprint of 0.64 mm2. The third-order ladder filter is implemented with electrically coupled resonators in 150 nm bi-layer P3F 128 rotated Y-cut LiNbO3 thin film, operating in second-order symmetric (S2) Lamb mode. The record-breaking performance is enabled by the P3F LiNbO3 platform, where piezoelectric thin films of alternating orientations are transferred subsequently, facilitating efficient higher-order Lamb mode operation with simultaneously high quality factor (Q) and coupling coefficient (k2) at millimeter-wave (mmWave). Also, the multi-layer P3F stack promises smaller footprints and better nonlinearity than single-layer counterparts, thanks to the higher capacitance density and lower thermal resistance. Upon further development, the reported P3F LiNbO3 platform is promising for compact filters at mmWave.

Millimeter Wave Thin-Film Bulk Acoustic Resonator in Sputtered Scandium Aluminum Nitride Using Platinum Electrodes

This work describes sputtered scandium aluminum nitride (ScAlN) thin-film bulk acoustic resonators (FBAR) at millimeter wave (mmWave) with high quality factor (Q) using platinum (Pt) electrodes. FBARs with combinations of Pt and aluminum (Al) electrodes, i.e., Al top Al bottom, Pt top Al bottom, Al top Pt bottom, and Pt top Pt bottom, are built to study the impact of electrodes on mmWave FBARs. The demonstrated FBAR with Pt top and bottom electrodes achieve electromechanical coupling (k2) of 4.0% and Q of 116 for the first-order symmetric (S1) mode at 13.7 GHz, and k2 of 1.8% and Q of 94 for third-order symmetric (S3) mode at 61.6 GHz. Through these results, we confirmed that even in the frequency band of approximately 60 GHz, ScAlN FBAR can achieve a Q factor approaching 100 with optimized fabrication and acoustic/EM design. Further development calls for stacks with better quality in piezoelectric and metallic layers.

38.7 GHz Thin Film Lithium Niobate Acoustic Filter

In this work, a 38.7 GHz acoustic wave ladder filter exhibiting insertion loss (IL) of 5.63 dB and 3-dB fractional bandwidth (FBW) of 17.6% is demonstrated, pushing the frequency limits of thin-film piezoelectric acoustic filter technology. The filter achieves operating frequency up to 5G millimeter wave (mmWave) frequency range 2 (FR2) bands, by thinning thin-film LiNbO3 resonators to sub-50 nm thickness. The high electromechanical coupling (k2) and quality factor (Q) of first-order antisymmetric (A1) mode resonators in 128 Y-cut lithium niobate (LiNbO3) collectively enable the first acoustic filters at mmWave. The key design consideration of electromagnetic (EM) resonances in interdigitated transducers (IDT) is addressed and mitigated. These results indicate that thin-film piezoelectric resonators could be pushed to 5G FR2 bands. Further performance enhancement and frequency scaling calls for better resonator technologies and EM-acoustic filter co-design.

Measuring a Robot Hand's Graspable Region using Power and Precision Grasps

The variety of robotic hand designs and actuation schemes makes it difficult to measure a hand's graspable volume. For end-users, this lack of standardized measurements makes it challenging to determine a priori if a robot hand is the right size for grasping an object. We propose a practical hand measurement standard, based on precision and power grasps, that is applicable to a wide variety of robot hand designs. The resulting measurements can be used to both determine if an object will fit in the hand and characterize the size of an object with respect to the hand. Our measurement procedure uses a functional approach, based on grasping a hypothetical cylinder, that allows the measurer choose the exact hand orientation and finger configurations that are used for the measurements. This ensures that the measurements are functionally comparable while relying on the human to determine the finger configurations that best match the intended grasp. We demonstrate using our measurement standard with three commercial robot hand designs and objects from the YCB data set.

Grasping Benchmarks: Normalizing for Object Size \& Approximating Hand Workspaces

The varied landscape of robotic hand designs makes it difficult to set a standard for how to measure hand size and to communicate the size of objects it can grasp. Defining consistent workspace measurements would greatly assist scientific communication in robotic grasping research because it would allow researchers to 1) quantitatively communicate an object's relative size to a hand's and 2) approximate a functional subspace of a hand's kinematic workspace in a human-readable way. The goal of this paper is to specify a measurement procedure that quantitatively captures a hand's workspace size for both a precision and power grasp. This measurement procedure uses a {\em functional} approach -- based on a generic grasping scenario of a hypothetical object -- in order to make the procedure as generalizable and repeatable as possible, regardless of the actual hand design. This functional approach lets the measurer choose the exact finger configurations and contact points that satisfy the generic grasping scenario, while ensuring that the measurements are {\em functionally} comparable. We demonstrate these functional measurements on seven hand configurations. Additional hand measurements and instructions are provided in a GitHub Repository.