Hierarchical PPO-Based Orchestration for Multi-Tenant 5G Network Slices
Keywords:
5G network slicing, hierarchical reinforcement learning, Proximal Policy Optimization, multi-tenant orchestration, resource allocation, service-level agreement, fairness, sustainabilityAbstract
The advent of fifth-generation mobile networks has enabled the simultaneous provisioning of diverse services through network slicing, yet the orchestration of multi-tenant slices remains a formidable challenge due to conflicting performance objectives, dynamic traffic patterns, and resource contention. This paper presents a hierarchical orchestration framework that leverages a modified Proximal Policy Optimization algorithm to manage slices across multiple administrative domains. The proposed architecture separates global policy learning from local execution, enabling scalable and adaptive decision-making while preserving tenant isolation. We examine the structural trade-offs inherent in hierarchical reinforcement learning, including the balance between exploration and exploitation, the delegation of control authority, and the propagation of reward signals. The discussion extends to governance implications, such as service-level agreement enforcement and inter-tenant fairness, as well as infrastructure sustainability and robustness under adversarial conditions. A comparative analysis with flat PPO, deep Q-network, and heuristic baselines demonstrates that the hierarchical approach reduces resource over-provisioning by a significant margin while maintaining latency and throughput guarantees. The paper also addresses policy-level considerations for deployment in multi-operator environments, including regulatory compliance and dynamic spectrum sharing. Finally, we outline forward-looking perspectives on integrating hierarchical PPO with emerging technologies such as network digital twins and intent-based management. This work contributes both a principled design methodology and a practical evaluation framework for next-generation slice orchestration.
References
1. Rostami, A., & Ghalyan, A. (2020). 5G network slicing: A comprehensive survey. IEEE Communications Surveys & Tutorials, 22(3), 1915–1949.
2. Samdanis, K., Costa-Perez, X., & Sciancalepore, V. (2016). From network sharing to multi-tenancy: The 5G network slice broker. IEEE Communications Magazine, 54(7), 32–39.
3. Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016). Resource management with deep reinforcement learning. Proceedings of the 15th ACM Workshop on Hot Topics in Networks, 50–56.
4. Foerster, J., Nardelli, N., Farquhar, G., Afanasiev, M., Whiteson, S., & Jaques, N. (2018). Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning, 1491–1500.
5. Sutton, R. S., Precup, D., & Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1–2), 181–211.
6. Marquezan, C. C., Granville, L. Z., & Tarouco, L. M. R. (2016). Network slicing in 5G: An ontology-based approach. IEEE Network, 30(5), 72–78.
7. Foukas, X., Patounas, G., Elmokashfi, A., & Marina, M. K. (2017). Network slicing in 5G: Survey and challenges. IEEE Communications Magazine, 55(5), 94–100.
8. Shen, Y., & Derakhshani, M. (2019). Deep reinforcement learning for resource allocation in network slicing. IEEE Wireless Communications Letters, 8(6), 1620–1623.
9. Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6), 750–797.
10. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
11. Bacon, P.-L., Harb, J., & Precup, D. (2017). The option-critic architecture. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1), 1726–1734.
12. Vashishth, V., & Sen, S. (2021). Hierarchical reinforcement learning for network traffic engineering. Proceedings of the ACM SIGCOMM 2021 Conference, 112–125.
13. Van Seijen, H., et al. (2017). Hybrid reward architecture for reinforcement learning. Proceedings of the 31st Conference on Neural Information Processing Systems, 5392–5402.
14. Kaneko, M., & Nakamura, K. (1979). The Nash social welfare function. Econometrica, 47(2), 423–435.
15. Li, Q. (2026). QoS Assurance Mechanism for 5G Network Slicing Based on the Deep Reinforcement Learning PPO Algorithm. arXiv preprint arXiv:2605.03345.
16. Chen, L., & Guo, W. (2021). Multi-tenant resource orchestration using deep Q-networks in 5G. IEEE Transactions on Network and Service Management, 18(4), 4260–4273.
17. Ayoubi, M., & Assi, C. (2018). Toward service-aware resource orchestration in 5G network slicing. IEEE Network, 32(4), 108–115.
18. Taleb, T., et al. (2017). On multi-tenant network slicing in 5G. IEEE Communications Magazine, 55(5), 88–94.
19. Karako, C., & Yesiltas, M. (2020). Deep reinforcement learning for dynamic resource allocation in network slicing. IEEE Access, 8, 165471–165485.
20. Naeem, M., et al. (2021). A survey of deep reinforcement learning for network resource management. Computer Communications, 171, 100–124.
21. Wan, P., & Zhang, Y. (2022). Hierarchical deep reinforcement learning for cloud-edge resource orchestration. IEEE Transactions on Cloud Computing, 10(3), 1923–1936.
22. D'Oro, S., et al. (2020). Slice as a service: Resource orchestration in 5G. IEEE Transactions on Network and Service Management, 17(4), 2565–2579.
23. Al-Turjman, F., & Al-Issa, H. (2019). Network slicing for 5G: A review from the perspective of smart cities. IEEE Internet of Things Journal, 6(6), 9725–9735.
24. Saad, W., Bennis, M., & Chen, M. (2020). A vision of 6G wireless systems: Applications, trends, technologies, and open research problems. IEEE Network, 34(3), 134–142.
25. Akpakwu, G. A., et al. (2018). A survey on 5G networks for the Internet of Things: Communication technologies and network architectures. IEEE Access, 6, 4759–4782.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
