Parallel Resampling Method of Remote Sensing Data Based on Pre-Partitioning for Cloud Computing

Abstract

Abstract:

Abstract： In order to solve the problem of parallel resampling of remote sensing image data in cloud computing, which is the basis for rapid publication of massive remote sensing image date, a parallel resampling method of remote sensing data based on prepartitioning was proposed in combination with the features of MapReduce parallel computing and the characteristics of remote sensing image data processing. Through the prepartitioning mechanism, the image data splitting and parallel resampling tasks can be effectively controlled, and the problem of MapReduce framework application in the unstructured remote sensing data with spatial location features processing was solved, thereby, the efficient parallel resampling of remote sensing image data in cloud computing environment is implemented. In the experiment, a parallel resampling flow on the opensource Hadoop platform was designed according to the parallel resampling method of remote sensing data based on prepartitioning. The experiment and analysis show that the parallel resampling method has a good resampling performance and is capable of achieving the efficient resampling of high resolution remote sensing image data in cloud computing environment.

Key words: cloud computing, pre-partitioning, parallel computing, parallel resampling, remote sensing image

CLC Number:

P 208

CHI Ziwena,b,ZHANG Fenga,b,DU Zhenhonga,b,LIU Renyia,b. Parallel Resampling Method of Remote Sensing Data Based on Pre-Partitioning for Cloud Computing[J]. Journal of Shanghai Jiaotong University, 2014, 48(11): 1627-1632.

References

［1］娄书荣, 孟令奎, 方军, 等. 基于对等网络的多分辨率影像的网络传输模型［J］. 测绘学报, 2011, 40(5): 628634.

LOU Shurong, MENG Linkui, FANG Jun, et al. Network transmission model for multiresolution image based on P2P ［J］. Acta Geodaetica et Cartographica Sinica, 2011, 40(5): 628634.

［2］钟志勇, 陈鹰. 基于遥感影像的空间信息平台的组织与实现［J］. 同济大学学报：自然科学版, 2003, 31(1): 8993.

ZHONG Zhiyong, CHEN Ying. Organization and realization of spatial information platform based on remote sensing image ［J］. Journal of Tongji University: Natural Science, 2003, 31(1): 8993.

［3］程承旗, 吕雪锋, 关丽. 空间数据剖分集群存储系统架构初探［J］. 北京大学学报：自然科学版, 2011, 47(1): 103108.

CHENG Chengqi, LV Xuefen, GUAN Li. Study on system architecture of subdivision storage cluster for global spatial data ［J］. Acta Scientiarum Naturalium Universitatis Pekinensis, 2011, 47(1): 103108.

［4］Yang C, Goodchild M, Huang Q, et al. Spatial cloud computing: How can the geospatial sciences use and help to shape cloud computing ［J］. International Journal of Digital Earth, 2011, 4(4): 305329.

［5］任伏虎, 王晋年. 遥感云服务平台技术研究与实验［J］. 遥感学报, 2012, 16(6): 13311338.

REN Fuhu, WANG Jinnian. Turning remote sensing to cloud services: Technical research and experiment ［J］. Journal of Remote Sensing, 2012, 16(6): 13311338.

［6］Golpayegani N, Halem M. Cloud computing for satellite data processing on high end compute clusters ［C］∥Proceedings of the 2009 IEEE International Conference on Cloud Computing. Piscataway, NJ: IEEE Computer Society, 2009: 8892.

［7］Chen J, Wang H J, Lu H Y. Research for GIS raster data storage and display under the framework of private cloud ［C］∥Proceedings of 2012 International Conference on Computer Science and Electronics Engineering. Piscataway, NJ: IEEE Computer Society, 2012: 449452.

［8］Ghemawat S, Gobioff H, Leung S T. The Google file system ［C］∥Proceedings of the 19th ACM Symposium on Operating Systems Principles. Bolton Landing, NY: ACM, 2003: 2943.

［9］Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters ［J］. Communications of the ACM, 2008, 51(1): 107113.

［10］李建江, 崔健, 王聃, 等. MapReduce并行编程模型研究综述［J］. 电子学报, 2011, 39(11): 26352642.

LI Jianjian, CUI Jian, WANG Dan, et al. Survey of MapReduce parallel programming model ［J］. Acta Electronica Sinica, 2011, 39(11): 26352642.

［11］楼琇林，黄韦艮，周长宝，等. 遥感图像数据重采样的一种快速算法［J］. 遥感学报, 2002, 6(2): 96101.

LOU Xioulin, HUANG Weigen, ZHOU Changbao, et al. A method for fast resampling of remote sensing imagery ［J］. Journal of Remote Sensing, 2002, 6(2): 96101.

［12］TOM W. Hadoop: The definitive guide ［M］. 2nd Edition. Sebastopol: O’Reilly Media Inc, 2010.

［13］刘鹏. 实战Hadoop ［M］. 北京: 电子工业出版社, 2011.

[1]	WANG Xiaoqinga,b，JIN Xianlonga,b，CAO Yuana. ThreeDimensional Simulation of Water Hammer of LargeScale Water Conveyance Tunnel [J]. Journal of Shanghai Jiaotong University, 2016, 50(01): 98-102.
[2]	CHEN Xiao-qinga (陈晓庆), WANG Yu-jieb (王宇杰), SUN Jian-qia*(孙建奇). Graphic Processing Unit Based Phase Retrieval and CT Reconstruction for Differential X-Ray Phase Contrast Imaging [J]. Journal of shanghai Jiaotong University (Science), 2014, 19(5): 550-554.
[3]	DI Dong-chao (狄东超), YE Guan-lin (叶冠林), XIA Xiao-he (夏小和), WANG Jian-hua* (王建华). Application of PETSc in Soil-Water Coupled Geotechnical Problems [J]. Journal of shanghai Jiaotong University (Science), 2013, 18(4): 401-408.
[4]	YIN Gui-Sheng-1, WANG Hai-Ling-1, 2 , ZHANG Jing-1, NI Jun-2, WANG Jian-3. Fast Efficient Collision Detection [J]. Journal of Shanghai Jiaotong University, 2012, 46(06): 962-966.
[5]	WANG Jian-Wei, JIN Xian-Long, CAO Lu-Fen, ZHANG Wei-Wei. Parallel Computation for the Dynamic Response of Connectional Passage in Twin-Tunnel due to Train Load [J]. Journal of Shanghai Jiaotong University, 2012, 46(04): 591-595.
[6]	XU Hua-Nan, LIU Zhe, LIU Can. Remote Sensing Image Denoising by àtrous Wavelet and Nonsubsampled Directional Filter Bank [J]. Journal of Shanghai Jiaotong University, 2012, 46(02): 233-238.
[7]	WU Yi-quan (吴一全), WU Chao (吴超), WU Shi-hua (吴诗婳) . Fusion of Remote Sensing Images Based on Nonsubsampled Contourlet Transform and Region Segmentation [J]. Journal of shanghai Jiaotong University (Science), 2011, 16(6): 722-727.
[8]	WANG Jian-Wei, JIN Xian-Long, WANG Xin, ZHANG Wei-Wei. Seismic Response Analysis of Connectional Passage in Twin-Tunnel Based on Parallel Computing [J]. Journal of Shanghai Jiaotong University, 2011, 45(10): 1557-1561.
[9]	CAO Lufen1,2,JIN Xianlong1,2,WU Huiming3,DU Xinguang1,2 . Parallel Algorithms for Transport Vehicles and Construction Tunnel Dynamic Coupling System [J]. Journal of Shanghai Jiaotong University, 2010, 44(11): 1534-1538.
[10]	ZHEN Xijin,WU Dianliang,ZHU Hongmin,FAN Xiumin. Dynamic Computation of Gap among Objects for Interactive Virtual Assembly [J]. Journal of Shanghai Jiaotong University, 2010, 44(02): 259-0263.