Downloads

https://doi.org/10.62836/jcmea.v6i1.0003
Chen, Y. (2026). Distributionally Robust Multi-Timescale Elastic Compute Scheduling for Tail-Latency Controlled Microservices. Journal of Computational Methods in Engineering Applications, 6(1), 0003. https://doi.org/10.62836/jcmea.v6i1.0003
Volume 6, Issue 1, 2026

Distributionally Robust Multi-Timescale Elastic Compute Scheduling for Tail-Latency Controlled Microservices

Elastic compute platforms must provision enough replicas to absorb bursty arrivals while avoiding persistent over-reservation. This paper develops DR-MPC-Elastic, a distributionally robust multi-timescale controller for microservice autoscaling. The method replaces a fixed safety margin with a data-dependent ambiguity radius estimated from the tail of recent forecast residuals. It combines an arrival-service workload model, a Wasserstein distributional uncertainty set, a CVaR tail-risk surrogate, and a mirror-descent projection that produces integer replica decisions with bounded actuation. The empirical case study uses a public Alibaba microservice trace containing 44,903 cleaned records, 390 thirty-second control windows, 381 service identifiers, and 13,060 container identifiers. Compared with a reactive HPA-like policy, DR-MPC-Elastic materially reduces SLA-risk slots while preserving a transparent cost-risk trade-off. The contribution is theoretical, auditable, and implementation-oriented: autoscaling is treated as risk-calibrated online optimization rather than as an opaque reinforcement-learning policy.

elastic compute microservices distributionally robust optimization CVaR model predictive control autoscaling

References

  1. Barroso LA, Clidaras J, Holzle U. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines; Morgan & Claypool: San Rafael, CA, USA, 2013.
  2. Dean J, Barroso LA. The Tail at Scale. Communications of the ACM 2013; 56(2):74–80.
  3. Verma A, Pedrosa L, Korupolu M, et al. Large-Scale Cluster Management at Google with Borg. In Proceedings of the Tenth European Conference on Computer Systems, Bordeaux, France, 22–24 April 2015; pp. 1–17.
  4. Burns B, Grant B, Oppenheimer D, et al. Borg, Omega, and Kubernetes. Communications of the ACM 2016; 59(5): 50–57.
  5. Schwarzkopf M, Konwinski A, Abd-El-Malek M, et al. Omega: Flexible, Scalable Schedulers for Large Compute Clusters. In Proceedings of the 8th ACM European Conference on Computer Systems, Prague, Czech Republic, 14–17 April 2013.
  6. Kubernetes. Horizontal Pod Autoscaling. Kubernetes Documentation. 2025. Available online: https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/ (accessed on 29 June 2026).
  7. CNCF. Kubernetes Event-driven Autoscaling (KEDA) Documentation. 2025. Available online: https://keda.sh/docs/ (accessed on 29 June 2026).
  8. Mao H, Alizadeh M, Menache I, et al. Resource Management with Deep Reinforcement Learning. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks, Atlanta, Georgia, USA, 9–10 November 2016; pp. 50–56.
  9. Mao H, Schwarzkopf M, Venkatakrishnan SB, et al. Learning Scheduling Algorithms for Data Processing Clusters. In Proceedings of the ACM Special Interest Group on Data Communication, Beijing, China, 19 August 2019; pp. 270–288.
  10. Neely MJ. Stochastic Network Optimization with Application to Communication and Queueing Systems; Morgan & Claypool: San Rafael, CA, USA, 2010.
  11. Rockafellar RT, Uryasev S. Optimization of Conditional Value-at-Risk. Journal of Risk 2000; 2: 21–41.
  12. Ben-Tal A, El Ghaoui L, Nemirovski A. Robust Optimization; Princeton University Press: Princeton, NJ, USA, 2009.
  13. Esfahani PM, Kuhn D. Data-Driven Distributionally Robust Optimization Using the Wasserstein Metric. Mathematical Programming 2018; 171: 115–166.
  14. Hazan E. Introduction to Online Convex Optimization. Foundations and Trends in Optimization 2016; 2(3–4): 157–325.
  15. Rawlings JB, Mayne DQ, Diehl M. Model Predictive Control: Theory, Computation, and Design; Nob Hill: Madison, WI, USA, 2017.
  16. Alibaba Cluster Trace Program. Cluster-Trace-Microservices-v2021. GitHub Repository. 2021. Available online: https://github.com/alibaba/clusterdata/tree/master/cluster-trace-microservices-v2021 (accessed on 29 June 2026).
  17. Delimitrou C, Kozyrakis C. Paragon: QoS-Aware Scheduling for Heterogeneous Datacenters. ACM Sigplan Notices 2013; 48(4): 77–88.
  18. Delimitrou C, Kozyrakis C. Quasar: Resource-Efficient and QoS-Aware Cluster Management. ACM Sigplan Notices 2014; 49(4): 127–144.
  19. Shahrad M, Fonseca R, Goiri I, et al. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC 20), Boston, Massachusetts, USA, 15–17 July 2020; pp. 205–218.
  20. Zhang Y, Goiri I, Chaudhry GI, et al. Faster and Cheaper Serverless Computing on Harvested Resources. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, Virtual, 26–29 October 2021; pp. 724–739.
  21. Zhang Y. Azure Functions Invocation Trace 2021. Microsoft Azure Public Dataset. 2021. Available online: https://github.com/Azure/AzurePublicDataset/blob/master/AzureFunctionsInvocationTrace2021.md (accessed on 29 June 2026).
  22. Wang L, Li M, Zhang Y, et al. Peeking Behind the Curtains of Serverless Platforms. In Proceedings of the 2018 USENIX Annual Technical Conference (USENIX ATC 18), Boston, MA, USA, 11–13 July 2018; pp. 133–146.
  23. Kleinrock L. Queueing Systems, Volume 1: Theory; Wiley: Hoboken, NJ, USA, 1975.
  24. Harchol-Balter M. Performance Modeling and Design of Computer Systems; Cambridge University Press: Cambridge, UK, 2013.
  25. Yan H. Real-Time 3D Model Reconstruction through Energy-Efficient Edge Computing. Optimizations in Applied Machine Learning 2022; 2(1). https://doi.org/10.71070/oaml.v2i1.48.
  26. Yan H, Shao D. Enhancing Transformer Training Efficiency with Dynamic Dropout. arXiv 2024. arXiv:2411.03236.
  27. Luo Z, Yan H, Pan X. Optimizing Transformer Models for Resource-Constrained Environments: A Study on Model Compression Techniques. Journal of Computational Methods in Engineering Applications 2023; 3(1): 1–12. https://doi.org/10.62836/jcmea.v3i1.030107.
  28. Yan H, Shao D. Multimodal Medical Image Analysis: Integrating LLM and RAG Deep Learning Strategies. Journal of Advances in Information Technology 2025; 16(4): 568–581. https://doi.org/10.12720/jait.16.4.568-581.
  29. Lu Y, Shao D, Ni X, et al. Emotion-Style Dual Prediction: A Multi-Task Deep Learning Approach for Artistic Images. Cluster Computing 2026; 29(1): 31.
  30. Dai Y. Medical Biopharmaceutical Image Anomaly Detection under Retinex State Space Duality and Frequency Consensus-Driven Transformer. Journal of Computational Methods in Engineering Applications 2026; 6(1): 0001.
  31. Dai Y. Deep Learning-Based Medical Image Segmentation for Early Cancer Detection. Optimizations in Applied Machine Learning 2025; 5(1). https://doi.org/10.71070/oaml.v5i1.144.
  32. Dai Y. MobileMamba-HC: Medical Image Disease Detection in Healthcare Integrating Frequency Adaptive Dilated Convolution and Spatial-Channel Synergistic Attention. Innovations in Applied Engineering and Technology 2026; 5(1): 0002.
  33. Dai Y, Wei L, Yu C. Graph Neural Network-Based Drug-Target Interaction Prediction for Precision Medicine. Optimizations in Applied Machine Learning 2025; 5(1). https://doi.org/10.71070/oaml.v5i1.145.
  34. Li J, Culver TB, Burgis CR, et al. Validating Nitrogen Removal Models with Field Bioretention Data. Journal of Environmental Engineering 2024; 150(8): 04024037.
  35. Li J, Culver TB, Persaud PP, et al. Developing Nitrogen Removal Models for Stormwater Bioretention Systems. Water Research 2023; 243: 120381.
  36. Li J. Nitrogen Removal Models for Stormwater Bioretention Systems. Ph.D. Thesis, University of Virginia, Charlottesville, VA, USA, 2023.
  37. Li J, Culver TB. Review of Process-Based Nitrogen Model for Agricultural Fields with Implications for Nitrogen Simulations in Stormwater BMPs. Environmental Modelling & Software 2022; 151: 105363.
  38. Deng X, Oda S, Kawano Y. Graphene-based Midinfrared Photodetector with Bull’s Eye Plasmonic Antenna. Optical Engineering 2023; 62(9): 097102.
  39. Deng X, Li L, Enomoto M, et al. Continuously Frequency-Tuneable Plasmonic Structures for Terahertz Bio-Sensing and Spectroscopy. Scientific Reports 2019; 9(1): 3498.
  40. Zhang Y, Needleman A. On the Identification of Power-Law Creep Parameters from Conical Indentation. Proceedings. Mathematical, Physical, and Engineering Sciences 2021; 477(2252): 20210233.
  41. Zhang Y, Needleman A. Characterization of Plastically Compressible Solids via Spherical Indentation. Journal of the Mechanics and Physics of Solids 2021; 148: 104283.
  42. Gandhi A, Harchol-Balter M, Das R, et al. Optimal Power Allocation in Server Farms. ACM Sigmetrics Performance Evaluation Review 2009; 37(1): 157–168.
  43. Hellerstein JM, Faleiro J, Gonzalez JE, et al. Serverless Computing: One Step Forward, Two Steps Back. arXiv 2018. arXiv:1812.03651.
  44. Kaffes K, Chong T, Humphries JT, et al. Centralized Core-Granular Scheduling for Serverless Functions. In Proceedings of the ACM Symposium on Cloud Computing, Santa Cruz, CA, USA, 21–23 October 2019; pp. 158–164.
  45. Yu T, Noghabi SA, Raindel S, et al. Characterizing Serverless Platforms with Serverless Bench. In Proceedings of the 11th ACM Symposium on Cloud Computing, Virtual, 19–21 October 2020; pp. 30–44.
  46. Gan Y, Delimitrou C. Seer: Leveraging Big Data to Navigate Performance Debugging in Cloud Microservices. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, Providence, RI, USA, 13–17 April 2019; pp. 19–33.
  47. Qiu H, Banerjee S, Jha S, et al. FIRM: Fine-grained Resource Management for SLO-Oriented Microservices. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), Virtual, 4–6 November 2020; pp. 805–825.
  48. Rausch T, Rashed A, Dustdar S. Optimized Container Scheduling for Data-Intensive Serverless Edge Computing. Future Generation Computer Systems 2021; 114: 259–271.
  49. Tirmazi M, Barker A, Deng N, et al. Borg: The Next Generation. In Proceedings of the Fifteenth European Conference on Computer Systems, Virtual, 27 April 2020; pp. 1–14.
  50. Grandl R, Kandula S, Rao S, et al. Graphene: Packing and Dependency-Aware Scheduling for Data-Parallel Clusters. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA, 2–4 November 2016; pp. 81–97.
  51. Chen Q, Liu C, Xiao Z. Improving MapReduce Performance Using Smart Speculative Execution Strategy. IEEE Transactions on Computers 2013; 63(4): 954–967.
  52. Casanova H, Legrand A, Quinson M. SimGrid: A Generic Framework for Large-Scale Distributed Experiments. In Proceedings of the Tenth International Conference on Computer Modeling and Simulation (uksim 2008), Cambridge, UK, 1–3 April 2008; pp. 126–131.
  53. Moritz P, Nishihara R, Wang S, et al. Ray: A Distributed Framework for Emerging AI Applications. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), Carlsbad, CA, USA, 8–10 October 2018; pp. 561–577.

Supporting Agencies

  1. Funding: This research received no external funding.