End-to-end Learning of Probabilistic and Geometric Constellation Shaping with Iterative Receivers

An end-to-end learning method for constellation shaping with a shaping-encoder assisted transceiver architecture is presented. The shaping encoder, which produces shaping bits with a higher probability of zeros, is used to produce an efficient symbol probability distribution. Both the probability distribution and the constellation geometry are jointly optimized, using end-to-end learning. Optimized constellations are evaluated using two iterative receiver architectures. Bit error rate (BER) performance gain is quantified against standard amplitude phase-shift keying (APSK) and quadrature amplitude modulation (QAM) constellations. A maximum BER gain of 0.3 dB and 0.15 dB are observed under two receivers for the learned constellations compared to standard APSK or QAM. The basic approach is extended to incorporate the full iterative detection and decoding loop, using the deep unfolding technique. A bit error rate gain of 0.1 dB is observed for the iterative scheme with learned constellations under block fading channel conditions, when compared to standard APSK.