Residual CNN-Based Transceiver With Attention-Aided GAN for Unknown Channels

Intelligent transceivers and wireless channels form an autoencoder (AE) structure, demonstrating a significant improvement in communication performance through end-to-end (E2E) learning. Recently, when the wireless channel is unknown, a residual generative adversarial network (GAN) has been utilized to simulate the real channels, thus facilitating the transmitter training. The quality of these simulated channels directly affects the adaptability of the intelligent transceiver to real-world situations. However, the training instability and the use of fully connected layers limit the residual GAN’s ability to capture effective features of real channels. To address these limitations, we propose an attention-aided residual GAN (AAR-GAN) model. This approach utilizes a convolutional neural network (CNN) to construct the GAN model and applies a squeeze-and-excitation channel attention block to CNN to automatically determine the significance of the feature channel. Furthermore, we employ a residual CNN (RCNN) to construct the transceiver, enabling smoother and more consistent learning, thus improving the communication performance. Simulation results demonstrate that our RCNN-based intelligent transceiver with the AAR-GAN model as an unknown channel significantly improves the bit error rate and block error rate for various bit lengths in the AWGN, Rayleigh fading and real DeepMIMO channels.