A Novel Vision Transformer with Residual in Self-attention for Biomedical Image Classification

Biomedical image classification requires capturing of bio-informatics based on specific feature distribution. In most of such applications, there are mainly challenges due to limited availability of samples for diseased cases and imbalanced nature of dataset. This article presents the novel framework of multi-head self-attention for vision transformer (ViT) which makes capable of capturing the specific image features for classification and analysis. The proposed method uses the concept of residual connection for accumulating the best attention output in each block of multi-head attention. The proposed framework has been evaluated on two small datasets: (i) blood cell classification dataset and (ii) brain tumor detection using brain MRI images. The results show the significant improvement over traditional ViT and other convolution based state-of-the-art classification models.

Guided Sampling-based Evolutionary Deep Neural Network for Intelligent Fault Diagnosis

The diagnostic performance of most of the deep learning models is greatly affected by the selection of model architecture and their hyperparameters. The main challenges in model selection methodologies are the design of architecture optimizer and model evaluation strategy. In this paper, we have proposed a novel framework of evolutionary deep neural network which uses policy gradient to guide the evolution of DNN architecture towards maximum diagnostic accuracy. We have formulated a policy gradient-based controller which generates an action to sample the new model architecture at every generation. The best fitness obtained is used as a reward to update the policy parameters. Also, the best model obtained is transferred to the next generation for quick model evaluation in the NSGA-II evolutionary framework. Thus, the algorithm gets the benefits of fast non-dominated sorting as well as quick model evaluation. The effectiveness of the proposed framework has been validated on three datasets: the Air Compressor dataset, Case Western Reserve University dataset, and Paderborn university dataset.

Transfer Learning based Evolutionary Deep Neural Network for Intelligent Fault Diagnosis

The performance of a deep neural network (DNN) for fault diagnosis is very much dependent on the network architecture. Also, the diagnostic performance is reduced if the model trained on a laboratory case machine is used on a test dataset from an industrial machine running under variable operating conditions. Thus there are two challenges for the intelligent fault diagnosis of industrial machines: (i) selection of suitable DNN architecture and (ii) domain adaptation for the change in operating conditions. Therefore, we propose an evolutionary Net2Net transformation (EvoNet2Net) that finds the best suitable DNN architecture for the given dataset. Nondominated sorting genetic algorithm II has been used to optimize the depth and width of the DNN architecture. We have formulated a transfer learning-based fitness evaluation scheme for faster evolution. It uses the concept of domain adaptation for quick learning of the data pattern in the target domain. Also, we have introduced a hybrid crossover technique for optimization of the depth and width of the deep neural network encoded in a chromosome. We have used the Case Western Reserve University dataset and Paderborn university dataset to demonstrate the effectiveness of the proposed framework for the selection of the best suitable architecture capable of excellent diagnostic performance, classification accuracy almost up to 100\%.

Quick Learning Mechanism with Cross-Domain Adaptation for Intelligent Fault Diagnosis

This paper presents a quick learning mechanism for intelligent fault diagnosis of rotating machines operating under changeable working conditions. Since real case machines in industries run under different operating conditions, the deep learning model trained for a laboratory case machine fails to perform well for the fault diagnosis using recorded data from real case machines. It poses the need of training a new diagnostic model for the fault diagnosis of the real case machine under every new working condition. Therefore, there is a need for a mechanism that can quickly transform the existing diagnostic model for machines operating under different conditions. we propose a quick learning method with Net2Net transformation followed by a fine-tuning method to cancel/minimize the maximum mean discrepancy of the new data to the previous one. This transformation enables us to create a new network with any architecture almost ready to be used for the new dataset. The effectiveness of the proposed fault diagnosis method has been demonstrated on the CWRU dataset, IMS bearing dataset, and Paderborn university dataset. We have shown that the diagnostic model trained for CWRU data at zero load can be used to quickly train another diagnostic model for the CWRU data at different loads and also for the IMS dataset. Using the dataset provided by Paderborn university, it has been validated that the diagnostic model trained on artificially damaged fault dataset can be used for quickly training another model for real damage dataset.