云环境下基于预分片的遥感数据并行重采样方法

摘要/Abstract

摘要：

摘要：面向海量高分辨率遥感影像数据快速发布需求，针对当前云环境下遥感影像数据并行重采样存在的难题，结合云平台MapReduce并行计算框架特性和遥感影像数据处理特点，提出了一种基于预分片的遥感影像数据并行重采样方法，通过预分片机制有效实现了该框架中对影像数据分片和并行重采样任务的控制，解决了MapReduce难以用于并行处理非结构化、具有空间位置特征的遥感影像数据的问题，从而实现了云环境下遥感影像数据的高效并行重采样.通过在开源云平台Hadoop上的实验和分析表明，该方法具有良好的重采样性能，能够实现高分辨率遥感影像数据的高效重采样.

关键词: 云计算, 预分片, 并行计算, 并行重采样, 遥感影像

Abstract:

Abstract： In order to solve the problem of parallel resampling of remote sensing image data in cloud computing, which is the basis for rapid publication of massive remote sensing image date, a parallel resampling method of remote sensing data based on prepartitioning was proposed in combination with the features of MapReduce parallel computing and the characteristics of remote sensing image data processing. Through the prepartitioning mechanism, the image data splitting and parallel resampling tasks can be effectively controlled, and the problem of MapReduce framework application in the unstructured remote sensing data with spatial location features processing was solved, thereby, the efficient parallel resampling of remote sensing image data in cloud computing environment is implemented. In the experiment, a parallel resampling flow on the opensource Hadoop platform was designed according to the parallel resampling method of remote sensing data based on prepartitioning. The experiment and analysis show that the parallel resampling method has a good resampling performance and is capable of achieving the efficient resampling of high resolution remote sensing image data in cloud computing environment.

Key words: cloud computing, pre-partitioning, parallel computing, parallel resampling, remote sensing image

中图分类号:

P 208

池子文a,b，张丰a,b，杜震洪a,b，刘仁义a,b. 云环境下基于预分片的遥感数据并行重采样方法[J]. 上海交通大学学报（自然版）, 2014, 48(11): 1627-1632.

CHI Ziwena,b,ZHANG Fenga,b,DU Zhenhonga,b,LIU Renyia,b. Parallel Resampling Method of Remote Sensing Data Based on Pre-Partitioning for Cloud Computing[J]. Journal of Shanghai Jiaotong University, 2014, 48(11): 1627-1632.

参考文献

［1］娄书荣, 孟令奎, 方军, 等. 基于对等网络的多分辨率影像的网络传输模型［J］. 测绘学报, 2011, 40(5): 628634.

LOU Shurong, MENG Linkui, FANG Jun, et al. Network transmission model for multiresolution image based on P2P ［J］. Acta Geodaetica et Cartographica Sinica, 2011, 40(5): 628634.

［2］钟志勇, 陈鹰. 基于遥感影像的空间信息平台的组织与实现［J］. 同济大学学报：自然科学版, 2003, 31(1): 8993.

ZHONG Zhiyong, CHEN Ying. Organization and realization of spatial information platform based on remote sensing image ［J］. Journal of Tongji University: Natural Science, 2003, 31(1): 8993.

［3］程承旗, 吕雪锋, 关丽. 空间数据剖分集群存储系统架构初探［J］. 北京大学学报：自然科学版, 2011, 47(1): 103108.

CHENG Chengqi, LV Xuefen, GUAN Li. Study on system architecture of subdivision storage cluster for global spatial data ［J］. Acta Scientiarum Naturalium Universitatis Pekinensis, 2011, 47(1): 103108.

［4］Yang C, Goodchild M, Huang Q, et al. Spatial cloud computing: How can the geospatial sciences use and help to shape cloud computing ［J］. International Journal of Digital Earth, 2011, 4(4): 305329.

［5］任伏虎, 王晋年. 遥感云服务平台技术研究与实验［J］. 遥感学报, 2012, 16(6): 13311338.

REN Fuhu, WANG Jinnian. Turning remote sensing to cloud services: Technical research and experiment ［J］. Journal of Remote Sensing, 2012, 16(6): 13311338.

［6］Golpayegani N, Halem M. Cloud computing for satellite data processing on high end compute clusters ［C］∥Proceedings of the 2009 IEEE International Conference on Cloud Computing. Piscataway, NJ: IEEE Computer Society, 2009: 8892.

［7］Chen J, Wang H J, Lu H Y. Research for GIS raster data storage and display under the framework of private cloud ［C］∥Proceedings of 2012 International Conference on Computer Science and Electronics Engineering. Piscataway, NJ: IEEE Computer Society, 2012: 449452.

［8］Ghemawat S, Gobioff H, Leung S T. The Google file system ［C］∥Proceedings of the 19th ACM Symposium on Operating Systems Principles. Bolton Landing, NY: ACM, 2003: 2943.

［9］Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters ［J］. Communications of the ACM, 2008, 51(1): 107113.

［10］李建江, 崔健, 王聃, 等. MapReduce并行编程模型研究综述［J］. 电子学报, 2011, 39(11): 26352642.

LI Jianjian, CUI Jian, WANG Dan, et al. Survey of MapReduce parallel programming model ［J］. Acta Electronica Sinica, 2011, 39(11): 26352642.

［11］楼琇林，黄韦艮，周长宝，等. 遥感图像数据重采样的一种快速算法［J］. 遥感学报, 2002, 6(2): 96101.

LOU Xioulin, HUANG Weigen, ZHOU Changbao, et al. A method for fast resampling of remote sensing imagery ［J］. Journal of Remote Sensing, 2002, 6(2): 96101.

［12］TOM W. Hadoop: The definitive guide ［M］. 2nd Edition. Sebastopol: O’Reilly Media Inc, 2010.

［13］刘鹏. 实战Hadoop ［M］. 北京: 电子工业出版社, 2011.

[1]	王风, 陈铁林, 樊容, 李跃鹏. 土体中水力劈裂大规模数值模拟及应用[J]. 上海交通大学学报, 2026, 60(3): 463-472.
[2]	刘克, 郝亮, 耿珺, 陈洵, 陆维. 质子交换膜燃料电池仿真模型并行计算与优化[J]. 上海交通大学学报, 2025, 59(11): 1754-1762.
[3]	管延敏, 杨彩虹, 康庄, 周利. 一种改进GPU加速策略在光滑粒子流体动力学方法中的应用[J]. 上海交通大学学报, 2023, 57(8): 981-987.
[4]	张泽龙, 张颖超, 伍波, 董威, 樊友本. 高灰度级高分辨率激光散斑血流实时成像研究[J]. 上海交通大学学报, 2023, 57(5): 552-559.
[5]	张晓慧，柏君励，顾解忡，马宁. 一种不可压缩二维流动的显式逐次超松弛并行算法[J]. 上海交通大学学报, 2019, 53(6): 681-687.
[6]	江列霖，杨培中，史超. 基于负载均衡分区法的建筑火灾并行数值模拟及应用[J]. 上海交通大学学报, 2018, 52(11): 1524-1531.
[7]	李建华. 新型电子政务及其安全保障技术[J]. 上海交通大学学报, 2018, 52(10): 1370-1381.
[8]	吴旭, 陈仁安, 魏德志. 基于云计算的语音实验室的设计及应用[J]. 实验室研究与探索, 2017, 36(5): 239-242.
[9]	史建焘, 李秀坤, 张兆心. 基于CloudStack的网络攻防虚拟实验云平台[J]. 实验室研究与探索, 2017, 36(5): 75-78.
[10]	张素霞, 李元祥, 周则明, 骆建华, 郁文贤. 光学遥感影像舰船型号识别[J]. 上海交通大学学报, 2016, 50(9): 1415-1421.
[11]	刘肃肃, 胡祎乐, 余音. 基于GPU的近场动力学模拟的并行化方法[J]. 上海交通大学学报, 2016, 50(9): 1362-1367.
[12]	郑来文, 赵子沂, 霍宏, 方涛. 基于多核学习的典型地物目标特征描述[J]. 上海交通大学学报, 2016, 50(9): 1339-1347.
[13]	王小庆a,b，金先龙a,b，曹源a. 大规模输水隧道水锤效应三维数值模拟[J]. 上海交通大学学报（自然版）, 2016, 50(01): 98-102.
[14]	印桂生1, 王海玲1, 2, 张菁1, 倪军2, 王建3. 快速高效的碰撞检测算法 [J]. 上海交通大学学报（自然版）, 2012, 46(06): 962-966.
[15]	王建炜, 金先龙, 曹露芬, 张伟伟. 列车载荷下隧道联络通道动态响应的并行计算[J]. 上海交通大学学报（自然版）, 2012, 46(04): 591-595.