The fracture toughness of in-service equipment and irradiation materials can be obtained from the ring-notched small punch specimen. The cohesive model was used to describe the ductile fracture behavior and crack propagation process of T91 steel, and the two parameters of material required by this model were calibrated by using the inverse finite element method. The successful implementation of this method requires that the load-displacement curve in the fracture damage stage is sensitive to the two model parameters, which can be optimized by the geometric design of the specimen and notch. The influence of the ratio of diameter to thickness, the depth of notch, and the presence or absence of prefabricated cracks on parameter sensitivity was studied, and the optimized design of the ring-notched specimen was obtained, based on which, a set of parameters was selected for finite element simulation to obtain the load-displacement curve. This curve was chosen as the target, and the genetic algorithm and the random walk algorithm were used for iterative fitting by using the inverse finite element method to extract the parameters of the cohesive model. The calculated results show that the error between the obtained parameters and the pre-selected parameters is less than 1%, which verifies the sensitivity of the specimen design and the accuracy of the inverse finite element method.