Abstract: In order to track the brain tissue shift due to tumor resection during image guided surgery, this paper presented a framework for robust deformable registration based on the theory of optical flow, within which outlier detection is used to set up locallyrefined control points in multilevel free form deformation (FFD) model. Robust Mestimator was also introduced to weaken the effect of outliers in global optimization for getting improved registration result. This proposed method has successfully been applied to a number of registration tasks to demonstrate its applicability and robustness to outliers.