Abstract:Receiver sensitivity is a particularly important metric in optical communication links operating at low signal-to-noise ratios (SNRs), for example in deep-space communication, since it directly limits the maximum achievable reach and data rate. Pulse position modulation (PPM) with direct detection photon-counting detectors are the most power-efficient solution known, however, the sensitivity gain comes at the expense of reduced spectral efficiency. We show that quadrature phase-shift keying (QPSK) modulation with a phase-sensitive ultralow noise pre-amplified coherent receiver outperforms other well-known power-efficient multi-dimensional coherent modulation formats, while simultaneously having higher spectral efficiency. It also results in better sensitivity than PPM for orders up to 64 with ideal direct detection using photon-counting receivers. This is because of the bit error rate characteristics favoring the QPSK format when forward error correction with a large overhead is considered.