LLM-Powered Ensemble Learning for Paper Source Tracing: A GPU-Free Approach

We participated in the KDD CUP 2024 paper source tracing competition and achieved the 3rd place. This competition tasked participants with identifying the reference sources (i.e., ref-sources, as referred to by the organizers of the competition) of given academic papers. Unlike most teams that addressed this challenge by fine-tuning pre-trained neural language models such as BERT or ChatGLM, our primary approach utilized closed-source large language models (LLMs). With recent advancements in LLM technology, closed-source LLMs have demonstrated the capability to tackle complex reasoning tasks in zero-shot or few-shot scenarios. Consequently, in the absence of GPUs, we employed closed-source LLMs to directly generate predicted reference sources from the provided papers. We further refined these predictions through ensemble learning. Notably, our method was the only one among the award-winning approaches that did not require the use of GPUs for model training. Code available at https://github.com/Cklwanfifa/KDDCUP2024-PST.

GP-NAS-ensemble: a model for NAS Performance Prediction

It is of great significance to estimate the performance of a given model architecture without training in the application of Neural Architecture Search (NAS) as it may take a lot of time to evaluate the performance of an architecture. In this paper, a novel NAS framework called GP-NAS-ensemble is proposed to predict the performance of a neural network architecture with a small training dataset. We make several improvements on the GP-NAS model to make it share the advantage of ensemble learning methods. Our method ranks second in the CVPR2022 second lightweight NAS challenge performance prediction track.

DQN Control Solution for KDD Cup 2021 City Brain Challenge

We took part in the city brain challenge competition and achieved the 8th place. In this competition, the players are provided with a real-world city-scale road network and its traffic demand derived from real traffic data. The players are asked to coordinate the traffic signals with a self-designed agent to maximize the number of vehicles served while maintaining an acceptable delay. In this abstract paper, we present an overall analysis and our detailed solution to this competition. Our approach is mainly based on the adaptation of the deep Q-network (DQN) for real-time traffic signal control. From our perspective, the major challenge of this competition is how to extend the classical DQN framework to traffic signals control in real-world complex road network and traffic flow situation. After trying and implementing several classical reward functions, we finally chose to apply our newly-designed reward in our agent. By applying our newly-proposed reward function and carefully tuning the control scheme, an agent based on a single DQN model can rank among the top 15 teams. We hope this paper could serve, to some extent, as a baseline solution to traffic signal control of real-world road network and inspire further attempts and researches.

Fault Detection for Covered Conductors With High-Frequency Voltage Signals: From Local Patterns to Global Features

The detection and characterization of partial discharge (PD) are crucial for the insulation diagnosis of overhead lines with covered conductors. With the release of a large dataset containing thousands of naturally obtained high-frequency voltage signals, data-driven analysis of fault-related PD patterns on an unprecedented scale becomes viable. The high diversity of PD patterns and background noise interferences motivates us to design an innovative pulse shape characterization method based on clustering techniques, which can dynamically identify a set of representative PD-related pulses. Capitalizing on those pulses as referential patterns, we construct insightful features and develop a novel machine learning model with a superior detection performance for early-stage covered conductor faults. The presented model outperforms the winning model in a Kaggle competition and provides the state-of-the-art solution to detect real-time disturbances in the field.

Scale- and Context-Aware Convolutional Non-intrusive Load Monitoring

Non-intrusive load monitoring addresses the challenging task of decomposing the aggregate signal of a household's electricity consumption into appliance-level data without installing dedicated meters. By detecting load malfunction and recommending energy reduction programs, cost-effective non-intrusive load monitoring provides intelligent demand-side management for utilities and end users. In this paper, we boost the accuracy of energy disaggregation with a novel neural network structure named scale- and context-aware network, which exploits multi-scale features and contextual information. Specifically, we develop a multi-branch architecture with multiple receptive field sizes and branch-wise gates that connect the branches in the sub-networks. We build a self-attention module to facilitate the integration of global context, and we incorporate an adversarial loss and on-state augmentation to further improve the model's performance. Extensive simulation results tested on open datasets corroborate the merits of the proposed approach, which significantly outperforms state-of-the-art methods.

Fault Location in Power Distribution Systems via Deep Graph Convolutional Networks

This paper develops a novel graph convolutional network (GCN) framework for fault location in power distribution networks. The proposed approach integrates multiple measurements at different buses while takes system topology into account. The effectiveness of the GCN model is corroborated by the IEEE 123-bus benchmark system. Simulation results show that the GCN model significantly outperforms other widely-used machine learning schemes with very high fault location accuracy. In addition, the proposed approach is robust to measurement noise and errors, missing entries, as well as multiple connection possibilities. Finally, data visualization results of two competing neural networks are presented to explore the mechanism of GCN's superior performance.

Convolutional Sequence to Sequence Non-intrusive Load Monitoring

A convolutional sequence to sequence non-intrusive load monitoring model is proposed in this paper. Gated linear unit convolutional layers are used to extract information from the sequences of aggregate electricity consumption. Residual blocks are also introduced to refine the output of the neural network. The partially overlapped output sequences of the network are averaged to produce the final output of the model. We apply the proposed model to the REDD dataset and compare it with the convolutional sequence to point model in the literature. Results show that the proposed model is able to give satisfactory disaggregation performance for appliances with varied characteristics.

Short-term Load Forecasting with Deep Residual Networks

We present in this paper a model for forecasting short-term power loads based on deep residual networks. The proposed model is able to integrate domain knowledge and researchers' understanding of the task by virtue of different neural network building blocks. Specifically, a modified deep residual network is formulated to improve the forecast results. Further, a two-stage ensemble strategy is used to enhance the generalization capability of the proposed model. We also apply the proposed model to probabilistic load forecasting using Monte Carlo dropout. Three public datasets are used to prove the effectiveness of the proposed model. Multiple test cases and comparison with existing models show that the proposed model is able to provide accurate load forecasting results and has high generalization capability.