Under the constraint requirements of strength and stability, the ANSYS parametric model and genetic algorithm were utilized to optimize the ring-stiffened cylindrical shell. With the constant length and radius of the shell, the minimum weight was considered as the optimal objective, while the optimal variables are shell thickness, frame dimensions, and frame spacing. During optimization, the difficulty in identifying the stability constraints may lead to the fact that the obtained optimal results may not be the global optimal solution. In order to solve this problem, a method for judging instability mode of ring-stiffened cylindrical shell based on the ANSYS path mapping technology combined with incremental search method was proposed and validated by using the optimal results of ringstiffened cylindrical shell. The results indicate that the proposed method to judge instability mode is effective and the optimization algorithm can converge to the optimal results in a limited time.