Evaluation and Optimization of Rendering Techniques for Autonomous Driving Simulation

In order to meet the demand for higher scene rendering quality from some autonomous driving teams (such as those focused on CV), we have decided to use an offline simulation industrial rendering framework instead of real-time rendering in our autonomous driving simulator. Our plan is to generate lower-quality scenes using a game engine, extract them, and then use an IQA algorithm to validate the improvement in scene quality achieved through offline rendering. The improved scenes will then be used for training.

Holmes: An Efficient and Lightweight Semantic Based Anomalous Email Detector

Email threat is a serious issue for enterprise security, which consists of various malicious scenarios, such as phishing, fraud, blackmail and malvertisement. Traditional anti-spam gateway commonly requires to maintain a greylist to filter out unexpected emails based on suspicious vocabularies existed in the mail subject and content. However, the signature-based approach cannot effectively discover novel and unknown suspicious emails that utilize various hot topics at present, such as COVID-19 and US election. To address the problem, in this paper, we present Holmes, an efficient and lightweight semantic based engine for anomalous email detection. Holmes can convert each event log of email to a sentence through word embedding then extract interesting items among them by novelty detection. Based on our observations, we claim that, in an enterprise environment, there is a stable relation between senders and receivers, but suspicious emails are commonly from unusual sources, which can be detected through the rareness selection. We evaluate the performance of Holmes in a real-world enterprise environment, in which it sends and receives around 5,000 emails each day. As a result, Holmes can achieve a high detection rate (output around 200 suspicious emails per day) and maintain a low false alarm rate for anomaly detection.

DualNet: Locate Then Detect Effective Payload with Deep Attention Network

Network intrusion detection (NID) is an essential defense strategy that is used to discover the trace of suspicious user behaviour in large-scale cyberspace, and machine learning (ML), due to its capability of automation and intelligence, has been gradually adopted as a mainstream hunting method in recent years. However, traditional ML based network intrusion detection systems (NIDSs) are not effective to recognize unknown threats and their high detection rate often comes with the cost of high false alarms, which leads to the problem of alarm fatigue. To address the above problems, in this paper, we propose a novel neural network based detection system, DualNet, which is constructed with a general feature extraction stage and a crucial feature learning stage. DualNet can rapidly reuse the spatial-temporal features in accordance with their importance to facilitate the entire learning process and simultaneously mitigate several optimization problems occurred in deep learning (DL). We evaluate the DualNet on two benchmark cyber attack datasets, NSL-KDD and UNSW-NB15. Our experiment shows that DualNet outperforms classical ML based NIDSs and is more effective than existing DL methods for NID in terms of accuracy, detection rate and false alarm rate.

Densely Connected Residual Network for Attack Recognition

High false alarm rate and low detection rate are the major sticking points for unknown threat perception. To address the problems, in the paper, we present a densely connected residual network (Densely-ResNet) for attack recognition. Densely-ResNet is built with several basic residual units, where each of them consists of a series of Conv-GRU subnets by wide connections. Our evaluation shows that Densely-ResNet can accurately discover various unknown threats that appear in edge, fog and cloud layers and simultaneously maintain a much lower false alarm rate than existing algorithms.

Pelican: A Deep Residual Network for Network Intrusion Detection

One challenge for building a secure network communication environment is how to effectively detect and prevent malicious network behaviours. The abnormal network activities threaten users' privacy and potentially damage the function and infrastructure of the whole network. To address this problem, the network intrusion detection system (NIDS) has been used. By continuously monitoring network activities, the system can timely identify attacks and prompt counter-attack actions. NIDS has been evolving over years. The current-generation NIDS incorporates machine learning (ML) as the core technology in order to improve the detection performance on novel attacks. However, the high detection rate achieved by a traditional ML-based detection method is often accompanied by large false-alarms, which greatly affects its overall performance. In this paper, we propose a deep neural network, Pelican, that is built upon specially-designed residual blocks. We evaluated Pelican on two network traffic datasets, NSL-KDD and UNSW-NB15. Our experiments show that Pelican can achieve a high attack detection performance while keeping a much low false alarm rate when compared with a set of up-to-date machine learning based designs.

LuNet: A Deep Neural Network for Network Intrusion Detection

Network attack is a significant security issue for modern society. From small mobile devices to large cloud platforms, almost all computing products, used in our daily life, are networked and potentially under the threat of network intrusion. With the fast-growing network users, network intrusions become more and more frequent, volatile and advanced. Being able to capture intrusions in time for such a large scale network is critical and very challenging. To this end, the machine learning (or AI) based network intrusion detection (NID), due to its intelligent capability, has drawn increasing attention in recent years. Compared to the traditional signature-based approaches, the AI-based solutions are more capable of detecting variants of advanced network attacks. However, the high detection rate achieved by the existing designs is usually accompanied by a high rate of false alarms, which may significantly discount the overall effectiveness of the intrusion detection system. In this paper, we consider the existence of spatial and temporal features in the network traffic data and propose a hierarchical CNN+RNN neural network, LuNet. In LuNet, the convolutional neural network (CNN) and the recurrent neural network (RNN) learn input traffic data in sync with a gradually increasing granularity such that both spatial and temporal features of the data can be effectively extracted. Our experiments on two network traffic datasets show that compared to the state-of-the-art network intrusion detection techniques, LuNet not only offers a high level of detection capability but also has a much low rate of false positive-alarm.

A Transfer Learning Approach for Network Intrusion Detection

Convolution Neural Network (ConvNet) offers a high potential to generalize input data. It has been widely used in many application areas, such as visual imagery, where comprehensive learning datasets are available and a ConvNet model can be well trained and perform the required function effectively. ConvNet can also be applied to network intrusion detection. However, the currently available datasets related to the network intrusion are often inadequate, which makes the ConvNet learning deficient, hence the trained model is not competent in detecting unknown intrusions. In this paper, we propose a ConvNet model using transfer learning for network intrusion detection. The model consists of two concatenated ConvNets and is built on a two-stage learning process: learning a base dataset and transferring the learned knowledge to the learning of the target dataset. Our experiments on the NSL-KDD dataset show that the proposed model can improve the detection accuracy not only on the test dataset containing mostly known attacks (KDDTest+) but also on the test dataset featuring many novel attacks (KDDTest-21) -- about 2.68\% improvement on KDDTest+ and 22.02\% on KDDTest-21 can be achieved, as compared to the traditional ConvNet model.