Journal of Shanghai Jiao Tong University (Science) ›› 2018, Vol. 23 ›› Issue (2): 297-307.doi: 10.1007/s12204-018-1938-5

Previous Articles     Next Articles

Feature Selection, Deep Neural Network and Trend Prediction

Feature Selection, Deep Neural Network and Trend Prediction

FANG Yan (方艳)   

  1. (School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai 200433, China; School of Finance, Shanghai University of International Business and Economics, Shanghai 201620, China)
  2. (School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai 200433, China; School of Finance, Shanghai University of International Business and Economics, Shanghai 201620, China)
  • Online:2018-04-01 Published:2018-06-19
  • Contact: FANG Yan (方艳) E-mail:yiffanyfang@163.com

Abstract: The literature generally agrees that longer-horizon (over a month) predictions make more sense than short-horizon ones. However, it’s an especially challenging task due to the lack of data (in unit of long horizon) and economic data have a low S/N ratio. We hypothesize that the stock trend is largely dictated by driving factors which are filtered by psychological factors and work on behavioral factors: representative indicators from these three aspects would be adequate in trend prediction. We then extend the Stepwise Regression Analysis (SRA) algorithm to constrained SRA (cSRA) to carry out a further feature selection and lag optimization. During modeling stage, we introduce the Deep Neural Network (DNN) model in stock prediction under the suspicion that economic interactions are too complex for shallow networks to capture. Our experiments indeed show that deep structures generally perform better than shallow ones. Instead of comparing to a kitchen sink model, where over-fitting can easily happen with a shortage of data, we turn around and use a model ensemble approach which indirectly demonstrates our proposed method is adequate.

Key words: feature selection| trend prediction| constrained Stepwise Regression Analysis (cSRA)| Deep Neural Network (DNN)

摘要: The literature generally agrees that longer-horizon (over a month) predictions make more sense than short-horizon ones. However, it’s an especially challenging task due to the lack of data (in unit of long horizon) and economic data have a low S/N ratio. We hypothesize that the stock trend is largely dictated by driving factors which are filtered by psychological factors and work on behavioral factors: representative indicators from these three aspects would be adequate in trend prediction. We then extend the Stepwise Regression Analysis (SRA) algorithm to constrained SRA (cSRA) to carry out a further feature selection and lag optimization. During modeling stage, we introduce the Deep Neural Network (DNN) model in stock prediction under the suspicion that economic interactions are too complex for shallow networks to capture. Our experiments indeed show that deep structures generally perform better than shallow ones. Instead of comparing to a kitchen sink model, where over-fitting can easily happen with a shortage of data, we turn around and use a model ensemble approach which indirectly demonstrates our proposed method is adequate.

关键词: feature selection| trend prediction| constrained Stepwise Regression Analysis (cSRA)| Deep Neural Network (DNN)

CLC Number: