Resource-Aware Serverless Computing for Real-Time Machine Learning Inference in Internet of Things Systems
Keywords:
serverless computing, real-time machine learning, Internet of Things, edge computing, resource management, governance, sustainabilityAbstract
Serverless computing has emerged as a promising execution model for Internet of Things systems that require real-time machine learning inference. By decoupling deployment from fixed infrastructure, serverless platforms can absorb bursty sensor workloads, scale inference functions elastically, and align operational cost with actual usage. However, real-time inference imposes stringent latency requirements that intersect with cold starts, model loading overhead, heterogeneous edge hardware, limited memory, variable network conditions, and multi-tenant contention. This paper presents a system-level examination of resource-aware serverless computing for real-time machine learning inference in Internet of Things ecosystems. It analyzes architectural trade-offs across device, edge, and cloud tiers, and discusses scheduling and placement strategies that account for compute, memory, accelerator, network, and energy resources. The discussion addresses structural tensions between elasticity and locality, statelessness and model reuse, centralized governance and distributed autonomy. It further examines fairness, observability, security, sustainability, and policy implications in multi-tenant inference platforms. Rather than proposing a single optimization algorithm, the paper develops an interdisciplinary perspective on how serverless control planes can evolve to meet real-time inference demands while preserving robustness, fairness, and environmental responsibility in large-scale socio-technical infrastructures.
References
1. Jonas, E., Schleier-Smith, J., Sreekanti, V., Tsai, C. C., Khandelwal, A., Pu, Q., Shankar, K., Carreira, J., Krauth, K., Yadwadkar, N., Gonzalez, J. E., Popa, R. A., Stoica, I., & Patterson, D. A. (2019). Cloud programming simplified: A Berkeley view on serverless computing. arXiv preprint arXiv:1902.03383.
2. Hellerstein, J. M., Faleiro, J., Gonzalez, J. E., Schleier-Smith, J., Sreekanti, V., Tumanov, A., & Wu, C. (2018). Serverless computing: One step forward, two steps back. arXiv preprint arXiv:1812.03651.
3. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30-39.
4. Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637-646.
5. Bonomi, F., Milito, R., Zhu, J., & Addepalli, S. (2012). Fog computing and its role in the internet of things. In Proceedings of the first edition of the MCC workshop on Mobile cloud computing (pp. 13-16). ACM.
6. Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., & Stoica, I. (2017). Clipper: A low-latency online prediction serving system. In 14th USENIX Symposium on Networked Systems Design and Implementation (pp. 613-627). USENIX Association.
7. Olston, C., Fiedel, N., Gorovoy, K., Harmsen, J., Lao, L., Li, F., Rajashekhar, V., Ramesh, S., Soyke, J., Yu, Y., Zhai, A., & Zhang, X. (2017). TensorFlow-Serving: Flexible, high-performance ML serving. arXiv preprint arXiv:1712.06139.
8. Chen, X. (2025, July). Research on Cloud Computing Applications for Optimizing DML Model Training in Serverless Architecture. In International Conference on Frontier Computing (pp. 553-562). Singapore: Springer Nature Singapore.
9. Fingler, H., Akshintala, A., & Rossbach, C. J. (2019). USETL: Unikernels for serverless extract transform and load. In Proceedings of the 10th ACM SIGOPS Asia-Pacific Workshop on Systems (pp. 10-17). ACM.
10. Miao, H., Li, A., Davis, L. S., & Deshpande, A. (2017). Towards unified data and lifecycle management for deep learning. In 2017 IEEE 33rd International Conference on Data Engineering (pp. 571-582). IEEE.
11. Reddi, V. J., Cheng, C., Kanter, D., Mattson, P., Schmuelling, G., Wu, C. J., Anderson, B., Breughe, M., Charlebois, M., Chou, W., Chukka, R., Coleman, C., Davis, S., Deng, P., Diamos, G., Duke, J., Fick, D., Gardner, J. S., Hubara, I., Idgunji, S., Jablin, T. B., Jiao, J., John, T. S., Kanwar, P., Lee, D., Liao, J., Lokhmotov, A., Massa, F., Meng, P., Micikevicius, P., Narayanaswami, R., Osborne, E., Pekhimenko, G., Rajan, A. T. R., Sequeira, D., Sirasao, A., Sun, F., Tang, H., Thomson, M., Wei, G., Wu, C., Xu, L., Yamada, H., Yu, B., Yuan, G., Zhong, A., Zhang, P., & Zhou, Y. (2020). MLPerf inference benchmark. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (pp. 446-459). IEEE.
12. Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020). Convergence of edge computing and deep learning: A comprehensive survey. IEEE Communications Surveys & Tutorials, 22(2), 869-904.
13. Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017). A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys & Tutorials, 19(4), 2322-2358.
14. Baldini, I., Castro, P., Chang, K., Cheng, P., Fink, S., Ishakian, V., Mitchell, N., Muthusamy, V., Rabbah, R., Slominski, A., & Suter, P. (2017). Serverless computing: Current trends and open problems. In Research Advances in Cloud Computing (pp. 1-20). Springer.
15. Barroso, L. A., & Hölzle, U. (2009). The datacenter as a computer: An introduction to the design of warehouse-scale machines. Morgan & Claypool.
16. Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015). Large-scale cluster management at Google with Borg. In Proceedings of the Tenth European Conference on Computer Systems (pp. 1-17). ACM.
17. Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and machine learning. fairmlbook.org.
18. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (pp. 214-226). ACM.
19. Wang, L., Li, M., Zhang, Y., Ristenpart, T., & Swift, M. (2018). Peeking behind the curtains of serverless platforms. In 2018 USENIX Annual Technical Conference (pp. 133-146). USENIX Association.
20. Klimovic, A., Wang, Y., Stuedi, P., Trivedi, A., Pfefferle, J., & Kozyrakis, C. (2018). Pocket: Elastic ephemeral storage for serverless analytics. In 13th USENIX Symposium on Operating Systems Design and Implementation (pp. 427-444). USENIX Association.
21. Jia, Z., & Witchel, E. (2021). Boki: Stateful serverless computing with shared logs. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (pp. 691-707). ACM.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
