Downloads

https://doi.org/10.62836/iaet.v1i1.1006
Pan, X., Luo, Z. ., & Zhou, L. (2022). Comprehensive Survey of State-of-the-Art Convolutional Neural Network Architectures and Their Applications in Image Classification. Innovations in Applied Engineering and Technology, 1(1), 1–16. https://doi.org/10.62836/iaet.v1i1.1006
Volume 1, Issue 1

Comprehensive Survey of State-of-the-Art Convolutional Neural Network Architectures and Their Applications in Image Classification

Image classification is a vital research direction in computer vision all over the world. Before the advent of deep learning, image classification relied on manual feature extraction and conventional machine learning algorithms. However, Convolutional Neural Networks (CNNs) revolutionized this field by automatically learning features from data. The article discusses the fundamental principles of convolutional neural networks and compares various CNN architectures. Key layers such as convolutional, pooling, activation, fully connected, and dropout layers are explained in detail, along with techniques like backpropagation and optimization algorithms. Additionally, common CNN models like LeNet, AlexNet, VGGNet, GoogLeNet, ResNet, SENet, and EfficientNet are introduced, highlighting their characteristics and applications.

image classification; convolutional neural networks; computer vision; machine learning; deep learning

References

  1. LeCun Y, Boser B, Denker J, Henderson D, Howard R, Hubbard W, Jackel L. Handwritten Digit Recognition with a Back–Propagation Network. Advances in Neural Information Processing Systems 1989; 2: 396–404.
  2. Liu Y, Liu L, Yang L, Hao L, Bao Y. Measuring Distance Using Ultra–Wideband Radio Technology Enhanced by Extreme Gradient Boosting Decision Tree (XGBoost). Automation in Construction 2021; 126: 103678.
  3. Liu Y, Bao Y. Review of Electromagnetic Waves–Based Distance Measurement Technologies for Remote Monitoring of Civil Engineering Structures. Measurement 2021; 176: 109193.
  4. Shen Y, Gu H–M, Qin S, Zhang D–W. Surf4, Cargo Trafficking, Lipid Metabolism, and Therapeutic Implications. Journal of Molecular Cell Biology 2022; 14(9): mjac063.
  5. Wang M, Alabi A, Gu H–M, Gill G, Zhang Z, Jarad S, Xia X–D, Shen Y, Wang G–Q, Zhang D–W. Identification of Amino Acid Residues in the MT–Loop of MT1–MMP Critical for Its Ability to Cleave Low–Density Lipoprotein Receptor. Frontiers in Cardiovascular Medicine 2022; 9: 917238.
  6. Shen Y, Gu H–m, Zhai L, Wang B, Qin S, Zhang D–w. The Role of Hepatic Surf4 in Lipoprotein Metabolism and the Development of Atherosclerosis in ApoE-/-Mice. Biochimica et Biophysica Acta (BBA)–Molecular and Cell Biology of Lipids 2022; 1867(10): 159196.
  7. Qiu Y. Estimation of Tail Risk Measures in Finance: Approaches to Extreme Value Mixture Modeling. Master’s Thesis, Johns Hopkins University: Baltimore, MD, USA, May 2019.
  8. Milord J, Yu F, Orton S, Flores L, Marra R. Impact of COVID Transition to Remote Learning on Engineering Self–Efficacy and Outcome Expectations. In Proceedings of the 2021 ASEE Virtual Annual Conference, Virtual Conference, 26–29 July 2021.
  9. Liu Y, Hajj M, Bao Y. Review of Robot–Based Damage Assessment for Offshore Wind Turbines. Renewable and Sustainable Energy Reviews 2022; 158: 112187.
  10. Liu Y, Bao Y. Review on Automated Condition Assessment of Pipelines with Machine Learning. Advanced Engineering Informatics 2022; 53: 101687.
  11. Deng X, Dong Z, Ma X, Wu H, Wang B, Du X. Exploration on Mechanics Design for Scanning Tunneling Microscope. In Proceedings of the 2009 Symposium on Photonics and Optoelectronics, Wuhan, China, 14–16 August 2009.
  12. Deng X, Hu Z, Xiu G, Li D, Yue Y, Song Z, Weng ZK, Xu J, Wang Z. Five–Beam Interference Pattern Model for Laser Interference Lithography. In Proceedings of the The 2010 IEEE International Conference on Information and Automation, Harbin, China, 20–23 June 2010.
  13. Wang B, Shen Y, Zhai L, Xia X, Gu H-M, Wang M, Zhao Y, Chang X, Alabi A, Xing S, Deng S, Liu B, Wang G, Qin S, Zhang D-W. Atherosclerosis–Associated Hepatic Secretion of VLDL but Not PCSK9 Is Dependent on Cargo Receptor Protein Surf4. Journal of Lipid Research 2021; 62: 100091.
  14. Deng S–j, Shen Y, Gu H–M, Guo S, Wu S–R, Zhang D–w. The Role of the C–Terminal Domain of PCSK9 and SEC24 Isoforms in PCSK9 Secretion. Biochimica et Biophysica Acta (BBA)–Molecular and Cell Biology of Lipids 2020; 1865(6): 158660.
  15. Shen Y, Wang B, Deng S, Zhai L, Gu H-M, Alabi A, Xia X, Zhao Y, Chang X, Qin S, Zhang D-W. Surf4 Regulates Expression of Proprotein Convertase Subtilisin/Kexin Type 9 (PCSK9) but Is Not Required for PCSK9 Secretion in Cultured Human Hepatocytes. Biochimica et Biophysica Acta (BBA)–Molecular and Cell Biology of Lipids 2020; 1865(2): 158555.
  16. Mock MB, Zhang S, Pniak B, Belt N, Witherspoon M, Summers RM. Substrate Promiscuity of the NdmCDE N7–Demethylase Enzyme Complex. Biotechnology Notes 2021; 2: 18–25.
  17. Yu F, Milord J, Orton SL, Flores L, Marra R. The Concerns and Perceived Challenges Students Faced When Traditional in–Person Engineering Courses Suddenly Transitioned to Remote Learning. In Proceedings of the 2022 ASEE Annual Conference, Minneapolis, MN, USA, 26–29 June 2022.
  18. Deng X, Li L, Enomoto M, Kawano Y. Continuously Frequency–Tuneable Plasmonic Structures for Terahertz Bio–Sensing and Spectroscopy. Scientific Reports 2019; 9(1): 3498.
  19. Deng X, Simanullang M, Kawano Y. Ge–Core/a–Si–Shell Nanowire–Based Field–Effect Transistor for Sensitive Terahertz Detection. Photonics 2018; 5(2): 13.
  20. Deng X, Kawano Y. Surface Plasmon Polariton Graphene Midinfrared Photodetector with Multifrequency Resonance. Journal of Nanophotonics 2018; 12(2): 026017–026017.
  21. Xia D, Alexander AK, Isbell A, Zhang S, Ou J, Liu XM. Establishing a Co–Culture System for Clostridium Cellulovorans and Clostridium Aceticum for High Efficiency Biomass Transformation. J Sci Heal Univ Ala 2017; 14: 8–13.
  22. Kundu S, Fu Y, Ye B, Beerel PA, Pedram M. Toward Adversary–Aware Non–Iterative Model Pruning Through D Ynamic N Etwork R Ewiring of DNNs. ACM Transactions on Embedded Computing Systems 2022; 21(5): 1–24.
  23. Sun G, Zhan T, Owusu BG, Daniel A–M, Liu G, Jiang W. Revised Reinforcement Learning Based on Anchor Graph Hashing for Autonomous Cell Activation in Cloud–RANs. Future Generation Computer Systems 2020; 104: 60–73.
  24. Li S, Singh K, Riedel N, Yu F, Jahnke I. Digital Learning Experience Design and Research of a Self–Paced Online Course for Risk–Based Inspection of Food Imports. Food Control 2022; 135: 108698.
  25. Horne J, Beddingfield E, Knapp M, Mitchell S, Crawford L, Mills SB, Wrist A, Zhang S, Summers RM. Caffeine and Theophylline Inhibit β–Galactosidase Activity and Reduce Expression in Escherichia coli. ACS Omega 2020; 5(50): 32250–32255.
  26. Chen L, Li S, Bai Q, Yang J, Jiang S, Miao Y. Review of Image Classification Algorithms Based on Convolutional Neural Networks. Remote Sensing 2021; 13(22): 4712.
  27. Wang W, Yang Y, Wang X, Wang W, Li J. Development of Convolutional Neural Network and Its Application in Image Classification: a Survey. Optical Engineering 2019; 58(4): 040901–040901.
  28. Dumoulin V, Visin F. A guide to convolution arithmetic for deep learning. 2016; arXiv:1603.07285.
  29. LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient–Based Learning Applied to Document Recognition. Proceedings of the IEEE 1998; 86(11): 2278–2324.
  30. Krizhevsky A, Sutskever I, Hinton GE. Imagenet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 2017; 60(6): 84–90.
  31. Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural, NIPS’12. In Proceedings of the Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012.
  32. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large–Scale Image Recognition. 2014. arXiv:1409.1556.
  33. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going Deeper with Convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015.
  34. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016.
  35. He K, Zhang X, Ren S, Sun J. Identity Mappings in Deep Residual Networks. In Proceedings of the Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016.
  36. Hu J, Shen L, Sun G. Squeeze–and–Excitation Networks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018.
  37. Tan M, Le Q. Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the ICML 2019: 36th International Conference on Machine Learning, Long Beach, CA, USA, 10–15 June 2019.
  38. Tan M, Le Q. Efficientnetv2: Smaller Models and Faster Training. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Virtual Event, 18–24 July 2021.
  39. Luo Z, Xu H, Chen F. Utterance–Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM. 2018. arXiv:1811.08065.
  40. Chen F, Luo Z. Learning Robust Heterogeneous Signal Features from Parallel Neural Network for Audio Sentiment Analysis. 2018. arXiv:1811.08065.
  41. Chen F, Luo Z. Sentiment Analysis Using Deep Robust Complementary Fusion of Multi–Features and Multi–Modalities. 2019. CoRR abs/1904.08138.
  42. Luo Z, Xu H, Chen F. Audio Sentiment Analysis by Heterogeneous Signal Features Learned from Utterance–Based Parallel Neural Network. In Proceedings of the AffCon@ AAAI 2019, Honolulu, HI, USA, 27 January 2019.
  43. Luo Z, Zeng X, Bao Z, Xu M. Deep Learning–Based Strategy for Macromolecules Classification with Imbalanced Data from Cellular Electron Cryotomography. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019.
  44. Chen F, Luo Z, Xu Y, Ke D. Complementary Fusion of Multi–Features and Multi–Modalities in Sentiment Analysis. 2019. arXiv:1904.08138.

Supporting Agencies

  1. Funding: Not applicable.