การประยุกต์ใช้งาน Q-Learning เพื่อการจัดสรรกำลังที่เหมาะสมในระบบโนมาที่มีผู้ใช้ 2 ราย: DOI: 10.14416/j.ind.tech.2023.04.004

Phetnakorn Aermsa-Ard; Chonticha Wangsamad; Kritsada Mamat

Authors

Phetnakorn Aermsa-Ard Department of Electronic Engineering Technology, College of Industrial Technology, King Mongkut’s University of Technology North Bangkok
Chonticha Wangsamad Department of Electronic Engineering Technology, College of Industrial Technology, King Mongkut’s University of Technology North Bangkok
Kritsada Mamat Department of Electronic Engineering Technology, College of Industrial Technology, King Mongkut’s University of Technology North Bangkok

Keywords:

NOMA, power allocation, Q-Learning

Abstract

This article considers a power domain non-orthogonal multiple access (NOMA) system which is a multiple access technique considered to be used in the 5G technology and beyond. Successive interference cancellation (SIC) is applied to decode user’s signals and power allocation significantly affects the system performance. In this article, we propose to apply Q-learning which is one of the machine learning methods to solve a transmit power allocation problem in a 2-users NOMA system where the objective function is to maximize the minimum transmission rate. We show how to transform NOMA system into O-Learning components namely agent, action, stage, reward, and environment which are very important for the learning process. Numerical results show that the Q-learning offers higher reward in each step. For the system performance, the bit rates of two users in the system are very close to each other when the Q-learning is applied. Furthermore, the Q-learning offers a higher minimum rate than that performed by dynamic power allocation methods in the literature and optimizers in Python’s library.

References

L. Dai, B. Wang, Z. Ding, Z. Wang, S. Chen, and L. Hanzo, A survey of non-orthogonal multiple access for 5G, IEEE Communication Surveys Tutorials, 2018, 20(3), 2294–2323.

M. Zeng, A. Yadav, O.A. Dobre, G.I. Tsiropoulos, and H.V. Poor, On the sum rate of MIMO-NOMA and MIMO-OMA systems, IEEE Wireless Communication Letter, 2017, 6(4), 534–537.

B. Makki, K. Chitti, A. Behravan and M.-S. Alouini, A survey of NOMA: Current status and open research challenges, IEEE Open Journal of the Communications Society, 2020 1, 179-189.

M.M. El-Sayed, A.S. Ibrahim and M.M. Khairy, Power allocation strategies for non-orthogonal multiple access, International Conference on Selected Topics in Mobile & Wireless Networking (MoWNeT-Egytp 2016), Proceeding, 2016, 1-6.

M.A.M. Kaaffah and I. Iskandar, Power allocation effect on capacity of single carrier power domain non-orthogonal multiple access (NOMA), 7th International Conference on Wireless and Telematics (ICWT-Indonesia 2021), Proceeding, 2021, 1-5.

R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, 2nd Ed., MIT Press, Cambridge, Massachusetts, London, 2017.

M. Chen, W. Saad and C. Yin, Optimized uplink-downlink decoupling in LTE-U networks: An echo state approach, IEEE International Conference on Communications (ICC-Malaysia 2016), Proceeding, 2016, 1-6.

H. Sun, X. Chen, Q. Shi, M. Hong, X. Fu and N. D. Sidiropoulos, Learning to optimize: training deep Neural networks for interference management, IEEE Transactions on Signal Processing, 2018, 6(20), 5438-5453.

E. Mete and T. Girici, Q-Learning based scheduling with successive interference cancellation, IEEE Access, 2020, 8, 172034-172042.

https://www.sciencedirect.com/topics/engineering/non-orthogonal-multiple-access. (Accessed on 18 July 2022)

https://medium.com/@nutorbitx/reinforcement-learning. (Accessed on 18 July 2022)