Deep Reinforcement Learning for Intelligent Reflecting Surfaces

LIS_DRL

The intelligent reflecting surface (IRS) acts as a deep reinforcement learning (DRL) agent by acquiring a state and a reward from the environment and exerting an action back on the environment

Key ideas

  • Enabling IRSs for mmWave and massive MIMO systems
  • Leveraging DRL to predict interaction beams from a few channel estimates
  • Eliminating the need for collecting large training dataset 
  • Approaching optimal rates with a few active sensors and almost no training overhead

Applications

  • Standalone IRS operation without any control from a base station
  • Almost no beam training overhead in IRS aided massive MIMO systems while still enabling energy-efficient LIS interaction design

More information about this research direction

Paper: A. Taha, Y. Zhang, F. B. Mismar and A. Alkhateeb, “Deep Reinforcement Learning for Intelligent Reflecting Surfaces: Towards Standalone Operation,” 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2020, pp. 1-5.

Abstract: The promising coverage and spectral efficiency gains of intelligent reflecting surfaces (IRSs) are attracting increasing interest. To adopt these surfaces in practice, however, several challenges need to be addressed. One of these main challenges is how to configure the reflecting coefficients on these passive surfaces without requiring massive channel estimation or beam training overhead. Earlier work suggested leveraging supervised learning tools to predict the IRS reflection matrices. While this approach has the potential of reducing the beam training overhead, it requires collecting large datasets for training the neural network models. In this paper, we propose a novel deep reinforcement learning framework for predicting the IRS reflection matrices with minimal beam training overhead. Simulation results show that the proposed online learning framework can converge to the optimal rate that assumes perfect channel knowledge. This represents an important step towards realizing a standalone IRS operation, where the surface configures itself without any control from the infrastructure.

@INPROCEEDINGS{Taha2020,
author={Taha, Abdelrahman and Zhang, Yu and Mismar, Faris B. and Alkhateeb, Ahmed},
booktitle={2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)},
title={Deep Reinforcement Learning for Intelligent Reflecting Surfaces: Towards Standalone Operation},
year={2020},
volume={},
number={},
pages={1-5},
doi={10.1109/SPAWC48557.2020.9154301}}

To reproduce the results in this paper:

Simulation codes (based on DeepMIMO v1):
These simulations use the DeepMIMO scenarios:
Example: Steps to generate the results in this figure
  1. Coming soon.

Having questions or feedback?

Send an email to 

 

Or post your question in the DeepMIMO forum