云环境下基于预分片的遥感数据并行重采样方法

摘要/Abstract

摘要：

摘要：面向海量高分辨率遥感影像数据快速发布需求，针对当前云环境下遥感影像数据并行重采样存在的难题，结合云平台MapReduce并行计算框架特性和遥感影像数据处理特点，提出了一种基于预分片的遥感影像数据并行重采样方法，通过预分片机制有效实现了该框架中对影像数据分片和并行重采样任务的控制，解决了MapReduce难以用于并行处理非结构化、具有空间位置特征的遥感影像数据的问题，从而实现了云环境下遥感影像数据的高效并行重采样.通过在开源云平台Hadoop上的实验和分析表明，该方法具有良好的重采样性能，能够实现高分辨率遥感影像数据的高效重采样.

关键词: , 云计算, 预分片, 并行计算, 并行重采样, 遥感影像

Abstract:

Abstract： In order to solve the problem of parallel resampling of remote sensing image data in cloud computing, which is the basis for rapid publication of massive remote sensing image date, a parallel resampling method of remote sensing data based on prepartitioning was proposed in combination with the features of MapReduce parallel computing and the characteristics of remote sensing image data processing. Through the prepartitioning mechanism, the image data splitting and parallel resampling tasks can be effectively controlled, and the problem of MapReduce framework application in the unstructured remote sensing data with spatial location features processing was solved, thereby, the efficient parallel resampling of remote sensing image data in cloud computing environment is implemented. In the experiment, a parallel resampling flow on the opensource Hadoop platform was designed according to the parallel resampling method of remote sensing data based on prepartitioning. The experiment and analysis show that the parallel resampling method has a good resampling performance and is capable of achieving the efficient resampling of high resolution remote sensing image data in cloud computing environment.

Key words: cloud computing, pre-partitioning, parallel computing, parallel resampling, remote sensing image

中图分类号:

P 208

池子文a,b，张丰a,b，杜震洪a,b，刘仁义a,b. 云环境下基于预分片的遥感数据并行重采样方法[J]. 上海交通大学学报（自然版）, 2014, 48(11): 1627-1632.

CHI Ziwena,b,ZHANG Fenga,b,DU Zhenhonga,b,LIU Renyia,b. Parallel Resampling Method of Remote Sensing Data Based on Pre-Partitioning for Cloud Computing[J]. Journal of Shanghai Jiaotong University, 2014, 48(11): 1627-1632.

参考文献

［1］娄书荣, 孟令奎, 方军, 等. 基于对等网络的多分辨率影像的网络传输模型［J］. 测绘学报, 2011, 40(5): 628634.

LOU Shurong, MENG Linkui, FANG Jun, et al. Network transmission model for multiresolution image based on P2P ［J］. Acta Geodaetica et Cartographica Sinica, 2011, 40(5): 628634.

［2］钟志勇, 陈鹰. 基于遥感影像的空间信息平台的组织与实现［J］. 同济大学学报：自然科学版, 2003, 31(1): 8993.

ZHONG Zhiyong, CHEN Ying. Organization and realization of spatial information platform based on remote sensing image ［J］. Journal of Tongji University: Natural Science, 2003, 31(1): 8993.

［3］程承旗, 吕雪锋, 关丽. 空间数据剖分集群存储系统架构初探［J］. 北京大学学报：自然科学版, 2011, 47(1): 103108.

CHENG Chengqi, LV Xuefen, GUAN Li. Study on system architecture of subdivision storage cluster for global spatial data ［J］. Acta Scientiarum Naturalium Universitatis Pekinensis, 2011, 47(1): 103108.

［4］Yang C, Goodchild M, Huang Q, et al. Spatial cloud computing: How can the geospatial sciences use and help to shape cloud computing ［J］. International Journal of Digital Earth, 2011, 4(4): 305329.

［5］任伏虎, 王晋年. 遥感云服务平台技术研究与实验［J］. 遥感学报, 2012, 16(6): 13311338.

REN Fuhu, WANG Jinnian. Turning remote sensing to cloud services: Technical research and experiment ［J］. Journal of Remote Sensing, 2012, 16(6): 13311338.

［6］Golpayegani N, Halem M. Cloud computing for satellite data processing on high end compute clusters ［C］∥Proceedings of the 2009 IEEE International Conference on Cloud Computing. Piscataway, NJ: IEEE Computer Society, 2009: 8892.

［7］Chen J, Wang H J, Lu H Y. Research for GIS raster data storage and display under the framework of private cloud ［C］∥Proceedings of 2012 International Conference on Computer Science and Electronics Engineering. Piscataway, NJ: IEEE Computer Society, 2012: 449452.

［8］Ghemawat S, Gobioff H, Leung S T. The Google file system ［C］∥Proceedings of the 19th ACM Symposium on Operating Systems Principles. Bolton Landing, NY: ACM, 2003: 2943.

［9］Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters ［J］. Communications of the ACM, 2008, 51(1): 107113.

［10］李建江, 崔健, 王聃, 等. MapReduce并行编程模型研究综述［J］. 电子学报, 2011, 39(11): 26352642.

LI Jianjian, CUI Jian, WANG Dan, et al. Survey of MapReduce parallel programming model ［J］. Acta Electronica Sinica, 2011, 39(11): 26352642.

［11］楼琇林，黄韦艮，周长宝，等. 遥感图像数据重采样的一种快速算法［J］. 遥感学报, 2002, 6(2): 96101.

LOU Xioulin, HUANG Weigen, ZHOU Changbao, et al. A method for fast resampling of remote sensing imagery ［J］. Journal of Remote Sensing, 2002, 6(2): 96101.

［12］TOM W. Hadoop: The definitive guide ［M］. 2nd Edition. Sebastopol: O’Reilly Media Inc, 2010.

［13］刘鹏. 实战Hadoop ［M］. 北京: 电子工业出版社, 2011.

[1]	王聚团, 戚晓宁, 黄志明. 水下生产管汇测试技术及其改进研究[J]. 海洋工程装备与技术, 2022, 9(2): 43-49.
[2]	袁振钦, 邹科, 孙亚峰, 刘刚, 屈衍, 李居跃. 基于时域分析法的动态电缆疲劳分析[J]. 海洋工程装备与技术, 2022, 9(2): 50-55.
[3]	王娟, 杨明旺, 郑茂尧, 刘凌云, 赵立君. 高强钢在大型半潜式平台组块建造中的应用[J]. 海洋工程装备与技术, 2022, 9(1): 27-31.
[4]	陈欣, 赵晓磊, 王立坤, 肖德明, 张腾月. 深水大型吸力锚建造技术研究[J]. 海洋工程装备与技术, 2022, 9(1): 32-36.
[5]	尹彦坤, 易涤非. 半潜式生产平台船体结构关键节点工程临界评估[J]. 海洋工程装备与技术, 2022, 9(1): 52-57.
[6]	ZHANG Shengfa (张胜发), TANG Na (唐纳), SHEN Guofeng (沈国峰), WANG Han (王悍), QIAO Shan (乔杉). Universal Software Architecture of Magnetic Resonance-Guided Focused Ultrasound Surgery System and Experimental Study[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 471-481.
[7]	MA Qunsheng (马群圣), CEN Xingxing (岑星星), YUAN Junyi (袁骏毅), HOU Xumin (侯旭敏). Word Embedding Bootstrapped Deep Active Learning Method to Information Extraction on Chinese Electronic Medical Record[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 494-502.
[8]	安庆升, 孙立东, 武秋生. 碳纤维增强复合材料发射筒设计研究[J]. 空天防御, 2021, 4(2): 13-.
[9]	KONG Xiangqiang (孔祥强), MENG Xiangxi (孟祥熙), LI Jianbo (李见波), SHANG Yanping (尚燕平), CUI Fulin (崔福林) . Comparative Study on Two-Stage Absorption Refrigeration Systems with Different Working Pairs[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(2): 155-162.
[10]	ZHUANG Weimin (庄蔚敏), WANG Pengyue (王鹏跃), AO Wenhong (熬文宏), CHEN Gang (陈刚) . Experiment and Simulation of Impact Response of Woven CFRP Laminates with Different Stacking Angles[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(2): 218-230.
[11]	ZHOU Xuhui (周旭辉), ZHANG Wenguang (张文光), XIE Jie (谢颉). Effects of Micro-Milling and Laser Engraving on Processing Quality and Implantation Mechanics of PEG-Dexamethasone Coated Neural Probe[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(1): 1-9.
[12]	HUANG Ningning (黄宁宁), MA Yixin (马艺馨), ZHANG Mingzhu (张明珠), GE Hao (葛浩), WU Huawei (吴华伟). Finite Element Modeling of Human Thorax Based on MRI Images for EIT Image Reconstruction[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(1): 33-39.
[13]	WANG Xianjin, GAO Xu, YU Kuigang . Fixture Locating Modelling and Optimization Research of Aluminum Alloy Sidewall in a High-Speed Train Body[J]. J Shanghai Jiaotong Univ Sci, 2020, 25(6): 706-713.
[14]	QIAO Xing, MA Dan, YAO Xuliang, FENG Baolin. Stability and Numerical Analysis of a Standby System[J]. J Shanghai Jiaotong Univ Sci, 2020, 25(6): 769-778.
[15]	WU Jin, MIN Yu, YANG Xiaodie, MA Simin . Micro-Expression Recognition Algorithm Based on Information Entropy Feature[J]. Journal of Shanghai Jiao Tong University(Science), 2020, 25(5): 589-599.