Journal of Shanghai Jiaotong University >
Orthogonal Features Extraction Method and Its Application in Convolution Neural Network
Received date: 2020-09-08
Online published: 2021-11-01
In view of feature redundancy in the convolutional neural network, the concept of orthogonal vectors is introduced into features. Then, a method for orthogonal features extraction of convolutional neural network is proposed from the perspective of enhancing the differences between features. By building the structure of parallel branches and designing the orthogonal loss function, the convolution kernels can extract the orthogonal features, enrich the feature diversity, eliminate the feature redundancy, and improve the results of classification. The experiment results on one-dimensional sample dataset show that compared with the traditional convolution neural network, the proposed method can supervise the convolution kernels with different sizes to mine more comprehensive information of orthogonal features, which improves the efficiency of convolutional neural network and lays the foundation for subsequent researches on pattern recognition and compact neural network.
LI Chen, LI Jianxun . Orthogonal Features Extraction Method and Its Application in Convolution Neural Network[J]. Journal of Shanghai Jiaotong University, 2021 , 55(10) : 1320 -1329 . DOI: 10.16183/j.cnki.jsjtu.2020.276
| [1] | KANG X D, XIANG X L, LI S T, et al. PCA-based edge-preserving features for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(12):7140-7151. |
| [2] | WU X H, ZUO W M, LIN L, et al. F-SVM: Combination of feature transformation and SVM learning via convex relaxation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(11):5185-5199. |
| [3] | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2014-09-04) [2020-05-18]. https://arxiv.org/abs/1409.1556 . |
| [4] | 刘娟宏, 胡彧, 黄鹤宇. 端到端的深度卷积神经网络语音识别[J]. 计算机应用与软件, 2020, 37(4):192-196. |
| [4] | LIU Juanhong, HU Yu, HUANG Heyu. End-to-end speech recognition based on deep convolution neural network[J]. Computer Applications and Software, 2020, 37(4):192-196. |
| [5] | LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015: 3431-3440. |
| [6] | ADLER P, FALK C, FRIEDLER S A, et al. Auditing black-box models for indirect influence[J]. Knowledge and Information Systems, 2018, 54(1):95-122. |
| [7] | ZHANG Q L, JIANG Z Q, LU Q S, et al. Split to be slim: An overlooked redundancy in vanilla convolution[C]// Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. California, USA: International Joint Conferences on Artificial Intelligence Organization, 2020: 3195-3201. |
| [8] | HE Y H, ZHANG X Y, SUN J. Channel pruning for accelerating very deep neural networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017: 17453051. |
| [9] | CHEN C F, LEE G G, SRITAPAN V, et al. Deep convolutional neural network on iOS mobile devices[C]// 2016 IEEE International Workshop on Signal Processing Systems (SiPS). Dallas, TX, USA: IEEE, 2016: 130-135. |
| [10] | QIU J X, CHEN C, LIU S C,, et al. SlimConv: Reducing channel redundancy in convolutional neural networks by weights flipping [EB/OL].(2020-03-08) [2020-05-18]. https://www.researchgate.net/publication/339997878_SlimConv_Reducing_Channel_Redundancy_in_Convolutional_Neural_Networks_by_Weights_Flipping . |
| [11] | HAN K, WANG Y H, TIAN Q, et al. GhostNet: More features from cheap operations[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2020: 19874590. |
| [12] | 卢泓宇, 张敏, 刘奕群, 等. 卷积神经网络特征重要性分析及增强特征选择模型[J]. 软件学报, 2017, 28(11):2879-2890. |
| [12] | LU Hongyu, ZHANG Min, LIU Yiqun, et al. Convolution neural network feature importance analysis and feature selection enhanced model[J]. Journal of Software, 2017, 28(11):2879-2890. |
| [13] | BRO R, SMILDE A K. Principal component analysis[J]. Analytical Methods, 2014, 6(9):2812-2831. |
| [14] | CHIANG H L, KADUR T, FETTWEIS G. Analyses of orthogonal and non-orthogonal steering vectors at millimeter wave systems[C]// 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM). Coimbra, Portugal: IEEE, 2016: 16192551. |
| [15] | 孙文赟, 宋昱, 陈昌盛. 基于卷积-反卷积网络的正交人脸特征学习算法[J]. 深圳大学学报(理工版), 2020, 37(5):474-481. |
| [15] | SUN Wenyun, SONG Yu, CHEN Changsheng. An orthogonal facial feature learning method based on convolutional-deconvolutional network[J]. Journal of Shenzhen University (Science and Engineering), 2020, 37(5):474-481. |
| [16] | GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77:354-377. |
| [17] | SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, Massachusetts, USA: IEEE, 2015: 1-9. |
| [18] | LUO W J, LI Y J, URTASUN R, et al. Understanding the effective receptive field in deep convolutional neural networks [EB/OL].(2017-01-15) [2020-05-18]. https://www.researchgate.net/publication/312461699_Understanding_the_Effective_Receptive_Field_in_Deep_Convolutional_Neural_Networks . |
| [19] | MURUGAN P. Feed forward and backward run in deep convolution neural network[EB/OL]. (2017-11-20) [2020-05-18]. https://www.researchgate.net/publication/320975739_Feed_Forward_and_Backward_Run_in_Deep_Convolution_Neural_Network . |
/
| 〈 |
|
〉 |
