Downloads

Zhou, L., Luo, Z. ., & Pan, X. (2023). Machine Learning-Based System Reliability Analysis with Gaussian Process Regression. Journal of Computational Methods in Engineering Applications, 3(1), 1–23. https://ojs.sgsci.org/journals/jcmea/article/view/151
Volume 3, Issue 1, 2023

Machine Learning-Based System Reliability Analysis with Gaussian Process Regression

Machine learning-based reliability analysis methods have shown great advancements for their computational efficiency and accuracy. Recently, many efficient learning strategies have been proposed to enhance the computational performance. However, few of them explores the theoretical optimal learning strategy. In this article, we propose several theorems that facilitates such exploration. Specifically, cases that considering and neglecting the correlations among the candidate design samples are well elaborated. Moreover, we prove that the well-known U learning function can be reformulated to the optimal learning function for the case neglecting the Kriging correlation. In addition, the theoretical optimal learning strategy for sequential multiple training samples enrichment is also mathematically explored through the Bayesian estimate with the corresponding lost functions. Simulation results show that the optimal learning strategy considering the Kriging correlation works better than that neglecting the Kriging correlation and other state-of-the art learning functions from the literatures in terms of the reduction of number of evaluations of performance function. However, the implementation needs to investigate very large computational resource.

reliability analysis; risk analysis; surrogate models; kriging; gaussian process regression; active learning

References

  1. Rubinstein RY, Kroese DP. Simulation and the Monte Carlo Method; John Wiley & Sons: Hoboken, NJ, USA, 2017.
  2. Fishman G. Monte Carlo: Concepts, Algorithms, and Applications; Springer Science & Business Media: Berlin, Germany, 2017.
  3. Echard B, Gayton N, Lemaire M, Relun N. A Combined Importance Sampling and Kriging Reliability Method for Small Failure Probabilities with Time-Demanding Numerical Models. Reliab Eng Syst Saf 2013; 111: 232–240.
  4. Huang X, Chen J, Zhu H. Assessing Small Failure Probabilities By AK–SS: An Active Learning Method Combining Kriging and Subset Simulation. Struct Saf 2016; 59: 86–95.
  5. Ditlevsen O, Madsen HO. Structural Reliability Methods; John Wiley & Sons: Hoboken, NJ, USA, 1996.
  6. Lemaire M. Structural Reliability; John Wiley & Sons: Hoboken, NJ, USA, 2013.
  7. Faravelli L. Response‐Surface Approach for Reliability Analysis. J Eng Mech 1989; 115(12): 2763–2781.
  8. Roussouly N, Petitjean F, Salaun M. A New Adaptive Response Surface Method for Reliability Analysis. Probabilistic Eng Mech 2013; 32: 103–115.
  9. Sadovskỳ Z, Soares CG. Artificial neural network model of the strength of thin rectangular plates with weld induced initial imperfections. Reliab Eng Syst Saf 2011; 96(6): 713–717.
  10. Schueremans L, Van Gemert D. Benefit of Splines and Neural Networks in Simulation Based Structural Reliability Analysis. Struct Saf 2005; 27(3): 246–261.
  11. Pedroni N, Zio E, Apostolakis GE. Comparison of Bootstrapped Artificial Neural Networks and Quadratic Response Surfaces for the Estimation of the Functional Failure Probability of a Thermal–Hydraulic Passive System. Reliab Eng Syst Saf 2010; 5(4): 386–395.
  12. Bourinet J-M. Rare-Event Probability Estimation With Adaptive Support Vector Regression Surrogates. Reliab Eng Syst Saf 2016; 150: 210–221.
  13. Song H, Choi KK, Lee I, Zhao L, Lamb D. Adaptive Virtual Support Vector Machine for Reliability Analysis of High-Dimensional Problems. Struct Multidiscip Optim 2013; 47(4): 479–491.
  14. Blatman G, Sudret B. An Adaptive Algorithm to Build up Sparse Polynomial Chaos Expansions for Stochastic Finite Element Analysis. Probabilistic Eng Mech 2010; 25(2): 183–197.
  15. Dubreuil S, Berveiller M, Petitjean F, Salaün M. Construction of Bootstrap Confidence Intervals on Sensitivity Indices Computed by Polynomial Chaos Expansion. Reliab Eng Syst Saf 2014; 121: 263–275.
  16. Echard B, Gayton N, Lemaire M. AK-MCS: An Active Learning Reliability Method Combining Kriging and Monte Carlo Simulation. Struct Saf 2011; 33(2): 145–154.
  17. Bichon BJ, Eldred MS, Swiler LP, Mahadevan S, McFarland JM. Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions. AIAA J 2008; 46(10): 2459–2468.
  18. Wang Z, Shafieezadeh A. REAK: Reliability Analysis Through Error Rate-Based Adaptive Kriging. Reliab Eng Syst Saf 2019; 182: 33–45.
  19. Song C, Wang Z, Shafieezadeh A, Xiao R. BUAK-AIS: Efficient Bayesian Updating with Active Learning Kriging-Based Adaptive Importance Sampling. Comput Methods Appl Mech Eng 2022; 391: 114578.
  20. Wang Z, Shafieezadeh A. Metamodel-Based Subset Simulation Adaptable to Target Computational Capacities: The Case for High-Dimensional and Rare Event Reliability Analysis. Struct Multidiscip Optim 2021; 64: 649–675.
  21. Zhang C, Wang Z, Shafieezadeh A. Error Quantification and Control for Adaptive Kriging-Based Reliability Updating with Equality Information. Reliab Eng Syst Saf 2021; 207: 107323.
  22. Wang J, Sun Z, Yang Q, Li R. Two Accuracy Measures of the Kriging Model for Structural Reliability Analysis. Reliab Eng Syst Saf 2017; 167: 494–505.
  23. Lv Z, Lu Z, Wang P. A New Learning Function for Kriging and Its Applications to Solve Reliability Problems in Engineering. Comput Math Appl 2015; 70(5): 1182–1197.
  24. Sun Z, Wang J, Li R, Tong C. LIF: A New Kriging Based Learning Function and Its Application to Structural Reliability Analysis. Reliab Eng Syst Saf 2017; 157: 152–165.
  25. Xiao N-C, Zuo MJ, Zhou C. A New Adaptive Sequential Sampling Method to Construct Surrogate Models for Efficient Reliability Analysis. Reliab Eng Syst Saf 2018; 169: 330–338.
  26. Zhang X, Wang L, Sørensen JD. REIF: A Novel Active-Learning Function Toward Adaptive Kriging Surrogate Models for Structural Reliability Analysis. Reliab Eng Syst Saf 2019; 185: 440–454.
  27. Shi Y, Lu Z, He R, Zhou Y, Chen S. A Novel Learning Function Based on Kriging for Reliability Analysis. Reliab Eng Syst Saf 2020; 198: 106857.
  28. Lelièvre N, Beaurepaire P, Mattrand C, Gayton N. AK-MCSi: A Kriging-Based Method to Deal With Small Failure Probabilities and Time-Consuming Models. Struct Saf 2018; 73: 1–11.
  29. Wen Z, Pei H, Liu H, Yue Z. A Sequential Kriging Reliability Analysis Method With Characteristics of Adaptive Sampling Regions and Parallelizability. Reliab Eng Syst Saf 2016; 153: 170–179.
  30. UQLab Sensitivity Analysis User Manual. Available online: http://www.uqlab.com/userguide-reliability (accessed on13 May 2017).
  31. UQLab Kriging (Gaussian Process Modelling) Manual. Available online: https://www.uqlab.com/kriging-user-manual (accessed on 13 May 2017).
  32. I. Kaymaz. Application of Kriging Method to Structural Reliability Problems. Struct Saf 2005; 27(2): 133–151.
  33. Lophaven SN, Nielsen HB, Søndergaard J. DACE-A Matlab Kriging Toolbox, Version 2.0. Available online: https://orbit.dtu.dk/en/publications/dace-a-matlab-kriging-toolbox-version-20 (accessed on 13 May 2017).
  34. Lophaven SN, Nielsen HB, Søndergaard J. Aspects of the Matlab Toolbox DACE. Informatics and Mathematical Modelling; Technical University of Denmark: Kongens Lyngby, Denmark, 2002.
  35. Zhang C, Wang Z, Shafieezadeh A. Value of Information Analysis via Active Learning and Knowledge Sharing in Error-Controlled Adaptive Kriging. IEEE Access 2020; 8: 51021–51034.
  36. Mohammadi Darestani Y, Wang Z, Shafieezadeh A. Wind Reliability of Transmission Line Models Using Kriging-Based Methods. In Proceedings of the 3th International Conference on Applications of Statistics and Probability in Civil Engineering, Seoul, South Korea, 26–30 May 2019.
  37. Wang Z, Shafieezadeh A. Confidence Intervals for Failure Probability Estimates in Adaptive Kriging-based Reliability Analysis. Reliab Eng Syst Saf 2019; 196(10): 106758.
  38. Song C, Xiao R, Sun B, Zhang C, Wang Z. An Efficient Structural Reliability Analysis Method With Active Learning Kriging-Assisted Robust Adaptive Importance Sampling. Structures 2023; 52: 711–722.
  39. Bichon B, Eldred M, Swiler L, Mahadevan S, McFarland J. Multimodal Reliability Assessment for Complex Engineering Applications Using Efficient Global Optimization. In Proceedings of the 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Honolulu, HI, USA, 23–26 April 2007.
  40. Dubourg V, Sudret B, Deheeger F. Metamodel-Based Importance Sampling for Structural Reliability Analysis. Probabilistic Eng Mech 2013; 33: 47–57.
  41. Johnson OT. Information Theory and the Central Limit Theorem; World Scientific: Singapore, 2004.
  42. Hoeffding W, Robbins H. The Central Limit Theorem for Dependent Random Variables. Duke Math J 1948; 15(3): 773–780.
  43. Poor HV. Elements of Parameter Estimation. In Springer Texts in Electrical Engineering; Springer: New York, NY, USA, 1994.
  44. Chen F, Luo Z, Xu Y, Ke D. Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis. Available online: https://ar5iv.labs.arxiv.org/html/1904.08138 (accessed on 12 December 2023).
  45. Luo Z, Xu H, Chen F. Audio Sentiment Analysis by Heterogeneous Signal Features Learned from Utterance-Based Parallel Neural Network. In Proceedings of the AAAI-19 Workshop on Affective Content Analysis, AffCon2019: Modeling Affect-in-Action, Honolulu, Hawaii, USA, 27 January2019.
  46. Luo Z, Zeng X, Bao Z, Xu M. Deep Learning-Based Strategy for Macromolecules Classification With Imbalanced Data From Cellular Electron Cryotomography. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019.
  47. Luo Z. Knowledge-Guided Aspect-Based Summarization. In Proceedings of the 2023 International Conference on Communications, Computing and Artificial Intelligence (CCCAI), Shanghai, China, 23–25 June 2023.
  48. Luo Z, Xu H, Chen F. Utterance-Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM. ArXiv Prepr. ArXiv181108065.
  49. Chen F, Luo Z. Learning Robust Heterogeneous Signal Features from Parallel Neural Network for Audio Sentiment Analysis. arXiv:1811.08065.
  50. Chen F, Luo Z. Sentiment Analysis Using Deep Robust Complementary Fusion of Multi-Features and Multi-Modalities. In Proceedings of the New In ML 2019, Vancouver, Canada, 9 December 2019.
  51. Zhao Y, Dai W, Z. Wang, Ragab AE. Application of Computer Simulation to Model Transient Vibration Responses of GPLs Reinforced Doubly Curved Concrete Panel Under Instantaneous Heating. Mater Today Commun 2024; 38: 107949.
  52. Dai W, Fatahizadeh M, Touchaei HG, Moayedi H, Foong LK. Application of Six Neural Network-Based Solutions on Bearing Capacity of Shallow Footing on Double-Layer Soils. Steel Compos Struct 2023; 49(2): 231–244.
  53. Dai W. Safety Evaluation of Traffic System with Historical Data Based on Markov Process and Deep-Reinforcement Learning. J Comput Methods Eng Appl 2021; 1(1): 1–14.
  54. Dai W. Design of Traffic Improvement Plan for Line 1 Baijiahu Station of Nanjing Metro. Innov Appl Eng Technol 2023; 2(1). DOI:10.58195/iaet.v2i1.133.
  55. Dai W. Evaluation and Improvement of Carrying Capacity of a Traffic System. Innov Appl Eng Technol 2022; 1–9. DOI:10.58195/iaet.v1i1.001.
  56. Kong C, Li H, Zhang L, Zhu H, Liu T. Link Prediction on Dynamic Heterogeneous Information Networks. In Computational Data and Social Networks; Springer International Publishing: New York, NY, USA.
  57. Zhu H, Wang B. Negative Siamese Network for Classifying Semantically Similar Sentences. In Proceedings of the 2021 International Conference on Asian Language Processing (IALP), Singapore, 11–13 December 2021.
  58. Kong C, Liu J, Li H, Liu Y, Zhu H, Liu T. Drug Abuse Detection via Broad Learning. In Proceedings of the Web Information Systems and Applications: 16th International Conference, WISA 2019, Qingdao, China, 20–22 September 2019.
  59. Kong C, Li H, Zhu H, Xiu Y, Liu J, Liu T. Anonymized User Linkage Under Differential Privacy. In Communications in Computer and Information Science; Springer: Singapore.
  60. Kong C, Zhu H, Li H, Liu J, Wang Z, Qian Y. Multi-agent Negotiation in Real-time Bidding. In 2019 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Taiwan, 20–22 May 2019.
  61. Li Z, Zhu H, Liu H, Song J, Cheng Q. Comprehensive Evaluation of Mal-API-2019 Dataset By Machine Learning in Malware Detection. Int J Comput Sci Inf Technol 2024; 2(1): 1–9.
  62. Tao G, Wang H, Shen Y, Zhai L, Liu B, Wang B, Chen W, Xing S, Chen Y, Gu H-M, Qin S, Zhang D-W. Surf4 (Surfeit Locus Protein 4) Deficiency Reduces Intestinal Lipid Absorption and Secretion and Decreases Metabolism in Mice. Arterioscler Thromb Vasc Biol 2023; 43(4): 562–580.
  63. Shen Y, Gu H-M, Qin S, Zhang D-W. Surf4, Cargo Trafficking, Lipid Metabolism, and Therapeutic Implications. J Mol Cell Biol 2022; 14(9): Mjac063.
  64. Wang M, Alabi A, Gu H-M, Gill G, Zhang Z, Jarad S, Xia X-D, Shen Y, Wang G-Q, Zhang D-W. Identification of Amino Acid Residues in the MT-Loop of MT1-MMP Critical for Its Ability to Cleave Low-Density Lipoprotein Receptor. Front Cardiovasc Med 2022; 9: 917238.
  65. Shen Y, Gu H, Zhai L, Wang B, Qin S, Zhang D. The role of hepatic Surf4 in lipoprotein metabolism and the development of atherosclerosis in apoE-/- mice. Biochim Biophys Acta BBA-Mol Cell Biol Lipids 2022;1867(10): 159196.
  66. Wang B, Shen Y, Zhai L, Xia X, Gu H-M, Wang M, Zhao Y, Chang X, Alabi A, Xing S, Deng S, Liu B, Wang G, Qin S, Zhang D-W. Atherosclerosis-Associated Hepatic Secretion of VLDL but Not PCSK9 Is Dependent on Cargo Receptor Protein Surf4. J. Lipid Res 2021; 62: 100091.
  67. Deng S, Shen Y, Gu H-M, Guo S, Wu S-R, Zhang D. The Role of the C-Terminal Domain of PCSK9 and SEC24 Isoforms in PCSK9 Secretion. Biochim Biophys Acta BBA-Mol Cell Biol Lipids 2020; 1865(6): 158660.
  68. Shen Y, Wang B, Deng Shijun, Zhai L, Gu H-M, Alabi A, Xia X, Zhao Y, Chang X, Qin S, Zhang D-W. Surf4 Regulates Expression of Proprotein Convertase Subtilisin/Kexin Type 9 (PCSK9) but Is Not Required for PCSK9 Secretion in Cultured Human Hepatocytes. Biochim Biophys Acta BBA-Mol Cell Biol Lipids 2020; 1865(2): 158555.
  69. Drezner Z, Wesolowsky GO. On the Computation of the Bivariate Normal Integral. J Stat Comput Simul 1990; 35: 101–107.