Abstract:In optical communication systems, fibre nonlinearity is the major obstacle in increasing the transmission capacity. Typically, digital signal processing techniques and hardware are used to deal with optical communication signals, but increasing speed and computational complexity create challenges for such approaches. Highly parallel, ultrafast neural networks using photonic devices have the potential to ease the requirements placed on the digital signal processing circuits by processing the optical signals in the analogue domain. Here we report a silicon photonice-lectronic neural network for solving fibre nonlinearity compensation of submarine optical fibre transmission systems. Our approach uses a photonic neural network based on wavelength-division multiplexing built on a CMOS-compatible silicon photonic platform. We show that the platform can be used to compensate optical fibre nonlinearities and improve the signal quality (Q)-factor in a 10,080 km submarine fibre communication system. The Q-factor improvement is comparable to that of a software-based neural network implemented on a 32-bit graphic processing unit-assisted workstation. Our reconfigurable photonic-electronic integrated neural network promises to address pressing challenges in high-speed intelligent signal processing.