Big Data Framework for Quantitative Trading System

doi:10.1007/s12204-017-1821-9

Journal of shanghai Jiaotong University (Science) ›› 2017, Vol. 22 ›› Issue (2): 193-197.doi: 10.1007/s12204-017-1821-9

Previous Articles Next Articles

Big Data Framework for Quantitative Trading System

DAI Shuji1 (戴书吉), WU Xing1,2* (武星), PEI Mengqi1 (裴孟齐), DU Zhikang1 (杜智康)

(1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; 2. Shanghai Key Laboratory of Financial Information Technology, Shanghai University of Finance and Economics, Shanghai 200433, China)
(1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; 2. Shanghai Key Laboratory of Financial Information Technology, Shanghai University of Finance and Economics, Shanghai 200433, China)

Online:2017-03-31 Published:2017-04-04
Contact: WU Xing(武星) E-mail:xingwu@shu.edu.cn

Abstract

Abstract: Massive trading data are produced in securities market every day. Besides, the amount of relevant social media data is also growing fast. It is a vital problem of making use of these data. Facing on the growing amount of data, using big data framework is a necessary and reasonable method. Then, a big data framework for quantitative trading system is proposed in this paper. In the framework, Apache Spark is chosen as the distributed computing framework to process trading data, and Apache HBase as the distributed database is used to store data. After introducing the whole framework, we discussed data sources and the structure of quantitative trading layer in detail.

Key words: big data| framework| quantitative trading| Apache Spark

摘要： Massive trading data are produced in securities market every day. Besides, the amount of relevant social media data is also growing fast. It is a vital problem of making use of these data. Facing on the growing amount of data, using big data framework is a necessary and reasonable method. Then, a big data framework for quantitative trading system is proposed in this paper. In the framework, Apache Spark is chosen as the distributed computing framework to process trading data, and Apache HBase as the distributed database is used to store data. After introducing the whole framework, we discussed data sources and the structure of quantitative trading layer in detail.

关键词: big data| framework| quantitative trading| Apache Spark

CLC Number:

F 830.91

DAI Shuji1 (戴书吉), WU Xing1,2* (武星), PEI Mengqi1 (裴孟齐), DU Zhikang1 (杜智康). Big Data Framework for Quantitative Trading System[J]. Journal of shanghai Jiaotong University (Science), 2017, 22(2): 193-197.

References 10

[1]	KAUFFMAN R J, HU Y, MA D. Will high-frequencytrading practices transform the financial markets in theAsia Pacific Region? [J]. Financial Innovation, 2015,1(1): 1-27.
[2]	KEARNS M, ORTIZ L. The Penn-Lehman automatedtrading project [J]. IEEE Intelligent Systems, 2003,18(6): 22-31.
[3]	TRELEAVEN P, GALAS M, LALCHAND V. Algorithmictrading review [J]. Communications of theACM, 2013, 56(11): 76-85.
[4]	ZAHARIA M, CHOWDHURY M, FRANKLIN MJ, et al. Spark: Cluster computing with workingsets [C]//Proceedings of the 2nd USENIX Conferenceon Hot Topics in Cloud Computing. Boston, MA:USENIX Association, 2010: 1765-1773.
[5]	SPARK A. Spark programming guide [EB/OL].(2015-10-4). [2016-11-14]. http://spark.apache.org/docs/latest/programming-guide.html.
[6]	SPARK A. Apache spark–lightning-fast cluster computing[EB/OL]. (2014-4-21). [2016-11-14]. http://spark.apache.org/.
[7]	VORA M N. Hadoop-HBase for large-scale data[C]//Computer Science and Network Technology(ICCSNT), 2011 International Conference. Harbin:IEEE, 2011: 601-605.
[8]	NARANG R K. Inside the black box: the simple truthabout quantitative trading [M]. Canada: John Wiley& Sons, 2009.
[9]	GRUNDY B D, KIM Y. Stock market volatility ina heterogeneous information economy [J]. Journal ofFinancial and Quantitative Analysis, 2002, 37(1): 1-27.
[10]	KWON K Y, KISH R J. A comparative study of technicaltrading strategies and return predictability: Anextension of Brock, Lakonishok, and LeBaron (1992)using NYSE and NASDAQ indices [J]. The QuarterlyReview of Economics and Finance, 2002, 42(3): 611-631.

[1]	WEI Lishen, FENG Yuang, FANG Jiakun, AI Xiaomeng, WEN Jinyu. Impact of Renewable Energy Integration on Market-Clearing Results in Spot Market Environment [J]. Journal of Shanghai Jiao Tong University, 2021, 55(12): 1631-1639.
[2]	ZHU Hongwei, LU Zhiqiang. Modeling and Optimization of Resource Constrained Project Scheduling Problem Considering Employee-Timetabling [J]. Journal of Shanghai Jiaotong University, 2020, 54(6): 624-635.
[3]	CAI Qinyi (蔡沁怡), MI Yiqun (宓轶群), CHU Zhaowu (储昭武), ZHENG Yuanyi (郑元义), CHEN Fang (陈方), LIU Yicheng (刘义成). Demand Analysis and Management Suggestion: Sharing Epidemiological Data Among Medical Institutions in Megacities for Epidemic Prevention and Control [J]. J Shanghai Jiaotong Univ Sci, 2020, 25(2): 137-139.
[4]	XIE Youbai. Design Science [J]. Journal of Shanghai Jiaotong University, 2019, 53(7): 873-880.
[5]	GUO Lu, MAO Yong, YAO Huiju. Research on Big Data System Oriented to Integrated Support Design of Equipment [J]. Air & Space Defense, 2019, 2(1): 70-75.
[6]	SONG Renjie1,YU Tong1,CHEN Yuhong2,CHEN Yuyang2,XIA Bin2. A Similar Duplicate Record Detection Algorithm for Big Data Based on MapReduce [J]. Journal of Shanghai Jiaotong University, 2018, 52(2): 214-221.
[7]	WANG Lin,ZOU Yuanyuan,LI Shaoyuan. Two-Layered Model Predictive Control with Performance-Based Trigger [J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1324-1332.
[8]	QI Zongfeng1 (戚宗峰), HAN Shan2 (韩山), LI Jianxun3* (李建勋). Applications of Generalized Rough Set Theory in Evaluation Index System of Radar Anti-Jamming Performance [J]. Journal of shanghai Jiaotong University (Science), 2016, 21(2): 151-158.
[9]	CHEN Si’en1 (陈思恩), YANG Lvqing2* (杨律青), XU Shouhui3 (徐守辉). Analytics: The Real-World Use of Big Data in Financial Services Studying with Judge System Events [J]. Journal of shanghai Jiaotong University (Science), 2016, 21(2): 210-214.
[10]	LIANG Ye (梁野). Multilingual Financial News Retrieval and Smart Recommendation Based on Big Data [J]. Journal of shanghai Jiaotong University (Science), 2016, 21(1): 18-24.
[11]	SUN Lin1,2* (孙琳), WANG Yan-zhang1 (王延章). Identifying the Core Competitive Intelligence Based on Enterprise Strategic Factors [J]. Journal of shanghai Jiaotong University (Science), 2015, 20(1): 118-123.
[12]	WANG Yan-Feng, HU Yu-Li. Influence by Heat Conductivity of Battery Framework to Thermal Process of Underwater Vehicle’s Battery Compartment [J]. Journal of Shanghai Jiaotong University, 2012, 46(05): 683-687.
[13]	DONG Ming. An Analytic Integrated Framework for Distributed Energy Supply Chain Networks [J]. Journal of Shanghai Jiaotong University, 2008, 42(11): 1772-1775.

Big Data Framework for Quantitative Trading System

Big Data Framework for Quantitative Trading System

Knowledge

Abstract

Cite this article

share this article

References 10

Related Articles 13

Recommended Articles

Metrics

Comments