Transfer Learning for Neural Networks-based Equalizers in Coherent Optical Systems

In this work, we address the paramount question of generalizability and adaptability of artificial neural networks (NNs) used for impairment mitigation in optical transmission systems. We demonstrate that by using well-developed techniques based on the concept of transfer learning, we can efficaciously retrain NN-based equalizers to adapt to changes in the transmission system using just a fraction of the initial training data and resources. We evaluate the potential of transfer learning to adapt the NN to changes in the launch powers, modulation formats, symbol rates, or even fiber plant (different fiber types and lengths). In our numerical examples, we consider the recently introduced combined NN equalizer consisting of a convolutional layer coupled with bi-directional long-short term memory (biLSTM) recurrent NN elements. We focus our analysis on long-haul coherent optical transmission systems employing two types of transmission fibers: the standard single-mode fiber (SSMF) and the TrueWave Classic (TWC) fiber. We also underline the specific peculiarities that occur when transferring the learning in coherent optical communication systems. Our results demonstrate the effectiveness of transfer learning for the fast adaptation of NN architectures to different transmission regimes and scenarios, paving the way for engineering flexible universal solutions for nonlinearity mitigation.