Massive MIMO precoding and spectral shaping with low resolution DACs and active constellation extension

Nonlinear precoding and pulse shaping are jointly considered in multi-user massive multiple-input multiple-output (MIMO) systems with low-resolution D/A-converters (DACs) in terms of algorithmic approach as well as large system performance. Two design criteria are investigated: the mean square error (MSE) with active constellation extension (ACE) and the symbol error rate (SER). Both formulations are solved based on a modified version of the generalized approximate message passing (GAMP) algorithm. Furthermore, theoretical performance results are derived based on the state evolution analysis of the GAMP algorithm. The MSE based technique is extended to jointly perform over-the-air (OTA) spectral shaping and precoding for frequency-selective channels, in which the spectral performance is characterized at the transmitter and at the receiver. Simulation and analytical results demonstrate that the MSE based approach yields the same performance as the SER based formulation in terms of uncoded SER. The analytical results provide good performance predictions up to medium SNR. Substantial improvements in detection, as well as spectral performance, are obtained from the proposed combined pulse shaping and precoding approach compared to standard linear methods.