Downloads
Download

This work is licensed under a Creative Commons Attribution 4.0 International License.
Dual-Price Hot-Tier Placement and Reliability-Aware Coding for Elastic Cloud Storage Services
Elastic storage services must allocate hot-tier capacity, move objects across tiers, and maintain durability under changing access patterns. This paper develops DTHC-RS, a dual-price hot-tier controller with reliability-aware erasure coding. The algorithm assigns each object a value density based on predicted access probability, object size, bandwidth benefit, and tail-latency benefit. Two online prices regulate the decision: a capacity price for hot-tier pressure and a reliability price for durability pressure. The theoretical derivation connects the admission rule to a Lagrangian relaxation and connects redundancy choice to the data-loss probability of erasure coding. The case study processes 8,737,496 requests from a public Wikimedia 2019 CDN text trace, observes 2,269,608 anonymous objects, and simulates the first 220,000 requests for hot-tier placement. At 5% hot-tier capacity, DTHC-RS obtains a byte-hit ratio of 20.84% and a P95 TTFB proxy of 0.3959 s. LFU achieves a higher byte-hit ratio in this slice, but DTHC-RS gives a lower tail-latency proxy and a clearer reliability-control interface.
References
- Ghemawat S, Gobioff H, Leung ST. The Google File System. In Proceedings of the 2003 ACM Symposium on Operating Systems Principles, Bolton Landing, New York, USA, 19–22 October 2003.
- Chang F, Dean J, Ghemawat S, et al. Bigtable: A Distributed Storage System for Structured Data. ACM Transactions on Computer Systems (TOCS) 2008; 26(2): 4.
- Weil SA, Brandt SA, Miller EL, et al. Ceph: A Scalable, High-Performance Distributed File System. In Proceedings of the 7th Conference on Operating Systems Design and Implementation (OSDI’06), Seattle, WA, USA, 6–8 November 2006.
- DeCandia G, Hastorun D, Jampani M, et al. Dynamo: Amazon’s Highly Available Key-Value Store. ACM SIGOPS Operating Systems Review 2007; 41(6): 205–220.
- Calder B, Wang J, Ogus A, et al. Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, Cascais, Portugal, 23–26 October 2011.
- Huang C, Simitci H, Xu Y, et al. Erasure Coding in Windows Azure Storage. In Proceedings of the 2012 USENIX Annual Technical Conference, Boston, MA, USA, 13–15 June 2012.
- Muralidhar S, Lloyd W, Roy S, et al. F4: Facebook’s Warm BLOB Storage System. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), Broomfield, CO, USA, 6–8 October 2014.
- O’Neil EJ, O’Neil PE, Weikum G. The LRU-K Page Replacement Algorithm for Database Disk Buffering. Acm Sigmod Record 1993; 22(2): 297–306.
- Megiddo N, Modha DS. ARC: A Self-Tuning, Low Overhead Replacement Cache. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST 03), San Francisco, CA, USA, 31 March–2 April 2003.
- Einziger G, Friedman R, Manes B. TinyLFU: A Highly Efficient Cache Admission Policy. ACM Transactions on Storage 2017; 13(4): 35.
- Waldspurger CA, Park N, Garthwaite A, et al. Efficient MRC Construction with SHARDS. In Proceedings of the 3th USENIX Conference on File and Storage Technologies (FAST 15), Santa Clara, CA, USA, 16–19 February 2015.
- Sathiamoorthy M, Asteris M, Papailiopoulos D, et al. XORing Elephants: Novel Erasure Codes for Big Data. Proceedings of the VLDB Endowment 2013; 6(11).
- Dimakis AG, Godfrey PB, Wu Y, et al. Network Coding for Distributed Storage Systems. IEEE Transactions on Information Theory 2010; 56(12): 6362–6376.
- Rashmi KV, Shah NB, Kumar PV. Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points. IEEE Transactions on Information Theory 2011; 57(8): 5227–5239.
- Rashmi KV, Shah NB, Ramchandran K. A Piggybacking Design Framework for Read-and Download-Efficient Distributed Storage Codes. IEEE Transactions on Information Theory 2017; 63(1): 308–330.
- Kadekodi S, Shinde S, et al. Giza: Erasure Coding Objects across Global Data Centers. In Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC 17), Santa Clara, CA, USA, 12–14 July 2023.
- Zhang Y, Huang P, Zhou K, et al. OSCA: An Online-Model Based Cache Allocation Scheme in Cloud Block Storage Systems. In Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC 20), Boston, MA, USA ,15–17 July 2020.
- Li J, Wang Q, Lee PP C, et al. An In-Depth Analysis of Cloud Block Storage Workloads in Large-Scale Production. In Proceedings of the 2020 IEEE International Symposium on Workload Characterization, Virtual, 27–29 October 2020.
- Li J, Wang Q, Lee PP C, et al. An In-Depth Comparative Analysis of Cloud Block Storage Workloads. ACM Transactions on Storage 2023; 19(2): 16.
- Yang J, Cai Y, Rashmi KV. A Large Scale Analysis of Hundreds of In-Memory Cache Clusters at Twitter. ACM Transactions on Storage 2021; 17(3): 17.
- Berger DS, Beckmann N, Chen M, et al. Practical Bounds on Optimal Caching with Variable Object Sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems 2018; 2(2): 32.
- Anwar A, Cheng Y, Gupta A, et al. Improving Docker Registry Design Based on Production Workload Analysis. In Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST 18), Oakland, CA, USA, 12–15 February 2018.
- Narayanan D, Donnelly A, Rowstron A. Write Off-Loading: Practical Power Management for Enterprise Storage. ACM Transactions on Storage (TOS) 2008; 4(3): 10.
- Phothilimthana PM, Kadekodi S, Ghodrati S, et al. Thesios: Synthesizing Accurate Counterfactual I/O Traces from I/O Samples. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, La Jolla, CA, USA, 27 April–1 May 2024.
- Wikimedia Foundation. Data Platform/Data Lake/Traffic/Caching. 2019. Available online: https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Traffic/Caching (accessed on 25 June 2026).
- Berg B, Berger DS, McAllister S, et al. The CacheLib Caching Engine: Design and Experiences at Scale. In Proceedings of the14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), Online, 4–6 November 2020.
- Liu Z, Lee PP C, Lui JCS. Heterogeneity-Aware Erasure-Coded Storage Placement. IEEE Transactions on Parallel and Distributed Systems 2020; 31(11): 2645–2658.
- Ma Y, et al. Kangaroo: Caching Billions of Tiny Objects on Flash. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, Virtual, 26–29 October 2021.
- A Comprehensive Open-Source Cache Trace Dataset. Available online: https://github.com/cacheMon/cache_dataset (accessed on 25 June 2026).
- Saxena M, Swift MM, Zhang Y. FlashTier: A Lightweight, Consistent and Durable Storage Cache. In Proceedings of the 7th ACM european conference on Computer Systems, Bern, Switzerland, 10–13 April 2012.
- Harchol-Balter M. Performance Modeling and Design of Computer Systems: Queueing Theory in Action; Cambridge University Press: Cambridge, UK, 2013.
- Yan H. Real-Time 3D Model Reconstruction through Energy-Efficient Edge Computing. Optimizations in Applied Machine Learning 2022; 2(1).
- Yan H, Shao D. Enhancing Transformer Training Efficiency with Dynamic Dropout. arXiv 2024. https://doi.org/10.48550/arXiv.2411.03236
- Luo Z, Yan H, Pan X. Optimizing Transformer Models for Resource-Constrained Environments: A Study on Model Compression Techniques. Journal of Computational Methods in Engineering Applications 2023: 1–12. https://doi.org/10.62836/jcmea.v3i1.030107
- Yan H, Shao D. Multimodal Medical Image Analysis: Integrating LLM and RAG Deep Learning Strategies. Journal of Advances in Information Technology 2025; 16(4): 568–581. https://doi.org/10.12720/jait.16.4.568-581
- Lu Y, Shao D, Ni X, et al. Emotion-Style Dual Prediction: A Multi-Task Deep Learning Approach for Artistic Images. Cluster Computing 2026; 29(1): 31.
- Dai Y. Medical Biopharmaceutical Image Anomaly Detection Under Retinex State Space Duality and Frequency Consensus-Driven Transformer. Journal of Computational Methods in Engineering Applications 2026; 6(1): 0001.
- Dai Y. Deep Learning-Based Medical Image Segmentation for Early Cancer Detection. Optimizations in Applied Machine Learning 2025; 5(1).
- Dai Y. MobileMamba-HC: Medical Image Disease Detection in Healthcare Integrating Frequency Adaptive Dilated Convolution and Spatial-Channel Synergistic Attention. Innovations in Applied Engineering and Technology 2026; 5(1): 0002.
- Dai Y, Wei L, Yu C. Graph Neural Network-Based Drug-Target Interaction Prediction for Precision Medicine. Optimizations in Applied Machine Learning 2025; 5(1).
- Li J, Culver TB, Burgis CR, et al. Validating Nitrogen Removal Models with Field Bioretention Data. Journal of Environmental Engineering 2024; 150(8): 04024037.
- Li J, Culver TB, Persaud PP, et al. Developing Nitrogen Removal Models for Stormwater Bioretention Systems. Water Research 2023; 243: 120381.
- Li J. Nitrogen Removal Models for Stormwater Bioretention Systems. Ph.D. Thesis, University of Virginia, Charlottesville, VA, USA, 2023.
- Li J, Culver TB. Review of Process-Based Nitrogen Model for Agricultural Fields with Implications for Nitrogen Simulations in Stormwater BMPs. Environmental Modelling & Software 2022; 151: 105363.
- Beckmann N, Sanchez D. Talus: A Simple Way to Remove Cliffs in Cache Performance. In Proceedings of the 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), Burlingame, CA, USA, 7–11 February 2015.
- Deng X, Oda S, Kawano Y. Graphene-Based Midinfrared Photodetector with Bull’s Eye Plasmonic Antenna. Optical Engineering 2023; 62(9): 097102.
- Deng X, Li L, Enomoto M, et al. Continuously Frequency-Tuneable Plasmonic Structures for Terahertz Bio-Sensing and Spectroscopy. Scientific Reports 2019; 9(1): 3498.
- Zhang Y, Needleman A. On the Identification of Power-Law Creep Parameters from Conical Indentation. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 2021; 477(2252): 20210233.
- Zhang Y, Needleman A. Characterization of Plastically Compressible Solids Via Spherical Indentation. Journal of the Mechanics and Physics of Solids 2021; 148: 104283.
- Backblaze Drive Stats: Hard Drive Reliability Test Data. Available online: https://www.backblaze.com/cloud-storage/resources/hard-drive-test-data (accessed on 25 June 2026).
- Ford D, Labelle F, Popovici FI, et al. Availability in Globally Distributed Storage Systems. In Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10), Vancouver, BC, Canada, 4–6 October 2010.
- Cidon A, Stutsman R, Rumble SM, et al. Copysets: Reducing the Frequency of Data Loss in Cloud Storage. In Proceedings of the 2013 USENIX Annual Technical Conference (USENIX ATC 13), San Jose, CA, USA, 26–28 June 2013.
- Terry DB, Theimer MM, Petersen K, et al. Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System. ACM SIGOPS Operating Systems Review 1995; 29(5): 172–182.
- Kleinrock L. Queueing Systems, Volume 1: Theory; Wiley: Hoboken, NJ, USA, 1975.
Supporting Agencies
- Funding: This research received no external funding.