To gain the maneuvering model of a typical unconventional ship, a numerical calculation based maneuvering model framework was established based on 3-dimensional mathematical modeling group mode model (MMG). To obtain the hydrodynamic derivatives in the maneuvering model, static oblique towing test (OTT) and dynamic circular motion test (CMT) were conducted in a numerical environment to obtain the forces on ship body. To improve the accuracy of nonlinear hydrodynamic derivatives, a hybrid method of cubic spline interpolation and least squares fitting was put forward. The zigzag and rotation tests of the scaled ship model verified the accuracy of the maneuvering model. The results show that the maneuvering motion model established by numerical calculation and hybrid methods has good accuracy and can be used for modeling unconventional ship maneuvering motion.