Differentiated QoS Provisioning in Wireless Networks Based on Deep Reinforcement Learning

Ming-Chu Chou; Guang-Jhe Lin; Chih-Heng Ke; Yeong-Sheng Chen

doi:10.46604/aiti.2024.13655

Authors

Ming-Chu Chou Department of Computer Science and Information Engineering, National Quemoy University, Kinmen, Taiwan, ROC
Guang-Jhe Lin Department of Computer Science and Information Engineering, National Quemoy University, Kinmen, Taiwan, ROC
Chih-Heng Ke Department of Computer Science and Information Engineering, National Quemoy University, Kinmen, Taiwan, ROC
Yeong-Sheng Chen Department of Computer Science, National Taipei University of Education, Taipei, Taiwan, ROC

DOI:

https://doi.org/10.46604/aiti.2024.13655

Keywords:

802.11 wireless networks, Quality of Service (QoS), contention window, deep reinforcement learning

Abstract

Wireless networks manage performance by adjusting the contention window, as they cannot directly detect collisions. Traditional contention window adjustment algorithms, such as the Binary Exponential Backoff (BEB) algorithm, may lead to lower throughput when multiple services with varying bandwidth demands coexist. To address this issue, this study aims to enhance network throughput by enabling differentiated bandwidth allocation for various services. Using deep reinforcement learning, the state space, action space, and reward functions are defined to optimize this differentiation. These definitions are integrated into the Deep Deterministic Policy Gradient (DDPG) technique, implemented in the Access Point (AP) to intelligently adjust the contention window. Leveraging DDPG’s capability for continuous actions, the proposed method provides Quality of Service (QoS) differentiation, ensuring that each service at its respective priority level meets its transmission requirements. Compared to the BEB algorithm, the proposed approach offers improved traffic allocation and higher network bandwidth utilization.

References

T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, et al., “Continuous Control With Deep Reinforcement Learning,” https://doi.org/10.48550/arXiv.1509.02971, September 09, 2015.

L. Li, M. Rong, and G. Zhang, “An Internet of Things QoS Estimate Approach Based on Multi-Dimension QoS,” 9th International Conference on Computer Science & Education, pp. 998-1002, August 2014.

G. Kang, J. Liu, B. Cao, and Y. Xiao, “Diversified QoS-Centric Service Recommendation for Uncertain QoS Preferences,” IEEE International Conference on Services Computing, pp. 288-295, November 2020.

S. Wang, Y. Zhao, L. Huang, J. Xu, and C. H. Hsu, “QoS Prediction for Service Recommendations in Mobile Edge Computing,” Journal of Parallel and Distributed Computing, vol. 127, pp. 134-144, May 2019.

C. Yan, Y. Zhang, W. Zhong, C. Zhang, and B. Xin, “A Truncated SVD-Based ARIMA Model for Multiple QoS Prediction in Mobile Edge Computing,” Tsinghua Science and Technology, vol. 27, no. 2, pp. 315-324, April 2022.

X. Luo, J. Xie, L. Xiong, Z. Wang, and C. Tian, “3-D Deployment of Multiple UAV-Mounted Mobile Base Stations for Full Coverage of IoT Ground Users With Different QoS Requirements,” IEEE Communications Letters, vol. 26, no. 12, pp. 3009-3013, December 2022.

T. Zhang, S. Wang, M. Tang, and J. Nie, “A Novel QoS Interval Prediction Method for Web Services Based on Random Walk,” 4th International Conference on Computers and Artificial Intelligence Technology, pp. 316-321, December 2023.

S. Lu, S. Hu, Y. Duan, J. Gao, and F. Liu, “Contention Window Adaptive Adjustment Strategy for Fairness in Multi-rate IEEE 802.11 Network,” 6th International Conference on Communication and Information Systems, pp. 82-86, October 2022.

H. Yang, F. Jin, Z. Li, and P. Wen, “Clustering Routing Optimization for Ant Colony Mobile Sensor Networks Based on Contention Window of MAC Layer,” 8th International Conference on Computer and Communication Systems, pp. 304-309, April 2023.

J. Li and F. Jian, “Learning Contention Window Selection in Age of Information-Oriented IEEE 802.11 Networks,” IEEE 11th International Conference on Information, Communication and Networks, pp. 903-907, August 2023

S. Jiang and J. Zheng, “A DRQN-Based Initial Contention Window Optimization Algorithm for NR-U and WiFi Coexistence Networks,” GLOBECOM 2022 - 2022 IEEE Global Communications Conference, pp. 6019-6024, December 2022.

B. Suvarna, B. LaluNaick, O. Gandhi, and G. VijayKumar, “A Divide and Conquer Technique for the Contention Window to Improve the QoS of MAC,” International Conference on Green Computing Communication and Electrical Engineering, pp. 1-6, March 2014.

H. Toral-Cruz, J. Ramirez-Pacheco, P. Velarde-Alvarado, and A. K. Pathan, “VoIP in Next-Generation Converged Networks.” Building Next-Generation Converged Networks: Theory and Practice, Boca Raton, FL: CRC Press, pp. 357-380, 2013.

O. O. Aldawibi, S. F. Alahmar, and A. S. Khrwat, “Performance Assessment on Backoff Contention Window for MANETs in Congested Area and Random Movement,” IEEE 1st International Maghreb Meeting of the Conference on Sciences and Techniques of Automatic Control and Computer Engineering MI-STA, pp. 762-766, May 2021.

Rasna, Indrabayu, Dewiani, and A. Achmad, “Evaluation Performance of Contention Window on the Impact Hidden Node Vehicle to Vehicle,” International Seminar on Intelligent Technology and Its Applications, pp. 505-510, July 2023.

J. H. Kwon, D. Kim, and E. J. Kim, “Reinforcement Learning-Based Contention Window Adjustment for Wireless Body Area Networks,” 4th International Conference on Big Data Analytics and Practices, pp. 1-4, August 2023.

S. Ergun, I. Sammour, and G. Chalhoub, “A Survey on How Network Simulators Serve Reinforcement Learning in Wireless Networks,” Computer Networks, vol. 234, article no. 109934, October 2023.

W. Wydmański and S. Szott, “Contention Window Optimization in IEEE 802.11ax Networks With Deep Reinforcement Learning,” IEEE Wireless Communications and Networking Conference, pp. 1-6, March-April 2021.

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, et al., “Playing Atari With Deep Reinforcement Learning,” https://doi.org/10.48550/arXiv.1312.5602, December 19, 2013.

C. H. Ke and L. Astuti, “Applying Multi-Agent Deep Reinforcement Learning for Contention Window Optimization to Enhance Wireless Network Performance,” ICT Express, vol. 9, no. 5, pp. 776-782, October 2023.