Learning Beam Codebooks in mmWave/THz MIMO System
Learning beam patterns for a fully analog system via interaction with the surrounding environment and without explicit channel knowledge
Key ideas
- Formulating beam pattern learning problem as a reinforcement learning problem
- Respecting the non-convex hardware constraint by specifying the phase configurations of the network of fully-analog discrete phase shifters as the state
- Using received measurements to generate the reward used in the reinforcement learning algorithm (or 1-bit feedback in the downlink beamforming case)
- Designing a proper feature vector that does not include explicit channel information to efficiently represent a user’s channel and to perform clustering
Applications
- Learning small-sized site-specific beam codebook, e.g., SSB codebook, to reduce the initial access latency
- Learning codebooks that adapt to the surrounding radio propagation environments
- Learning optimized beam patterns for arrays with unknown, non-uniform geometries
More information about this research direction
Paper: Yu Zhang, Muhammad Alrabeiah, and Ahmed Alkhateeb, “Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO Systems,” in IEEE Transactions on Communications, 2021.
Abstract: Millimeter wave (mmWave) and terahertz MIMO systems rely on pre-defined beamforming codebooks for both initial access and data transmission. Being pre-defined, however, these codebooks are commonly not optimized for specific environments, user distributions, and/or possible hardware impairments. This leads to large codebook sizes with high beam training overhead which increases the initial access/tracking latency and makes it hard for these systems to support highly mobile applications. To overcome these limitations, this paper develops a deep reinforcement learning framework that learns how to iteratively optimize the codebook beam patterns (shapes) relying only on the receive power measurements and without requiring any explicit channel knowledge. The developed model learns how to autonomously adapt the beam patterns to best match the surrounding environment, user distribution, hardware impairments, and array geometry. Further, this approach does not require any knowledge about the channel, array geometry, RF hardware, or user positions. To reduce the learning time, the proposed model designs a novel Wolpertinger-variant architecture that is capable of efficiently searching for an optimal policy in a large discrete action space, which is important for large antenna arrays with quantized phase shifters. This complex-valued neural network architecture design respects the practical RF hardware constraints such as the constant-modulus and quantized phase shifter constraints. Simulation results based on the publicly available DeepMIMO dataset confirm the ability of the developed framework to learn near-optimal beam patterns for both line-of-sight (LOS) and non-LOS scenarios and for arrays with hardware impairments without requiring any channel knowledge.
@ARTICLE{Zhang2021,
author={Zhang, Yu and Alrabeiah, Muhammad and Alkhateeb, Ahmed},
journal={IEEE Transactions on Communications},
title={Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO Systems},
year={2021},
volume={},
number={},
pages={1-1},
doi={10.1109/TCOMM.2021.3126856}}
To reproduce the results in this paper:
Example: Steps to generate the results in this figure
- Download all the files of the GitHub repository given above.
- Run main.py.
- When main.py finishes, run read_beams.py.
- Load beam_codebook.mat in Matlab.
- Run plot_pattern(beams.') in Matlab Command Window, which will give Fig. 7(c) shown right as result.
