Traditional similarity-based methods generally ignore the diversity of equipment fault modes, the difference in degradation rates, and the inconsistency among monitoring data lengths. Thus, a similarity-based multi-scale ensemble method in multiple fault modes (MFM-MSEN) is proposed to improve remaining useful life (RUL) prediction accuracy and characterize prediction uncertainty. By training the fault mode classification model, designing the time-series weighted prediction strategy, and recognizing the fault mode of equipment, the test equipment is matched with the training equipment with the same fault mode to reduce matching complexity, based on which, a multi-scale ensemble strategy is proposed to overcome the data utilization limitation caused by single-scale matching methods and enhance the generalization ability of the proposed MFM-MSEN method. This strategy matches the similarities between test equipment and training equipment at multiple time scales, and then multiscale prediction results are integrated to fit accurate RUL probability distribution by employing kernel density estimation. Experimental results demonstrate the superiority of the proposed MFM-MSEN method in dealing with the differences in equipment degradation.