Abstract: Oral endoscope image stitching algorithm is studied to obtain wide-field oral images through registration and stitching, which is of great significance for auxiliary diagnosis. Compared with natural images, oral images have lower textures and fewer features. However, traditional feature-based image stitching methods rely heavily on feature extraction quality, often showing an unsatisfactory performance when stitching images with few features. Moreover, due to the hand-held shooting, there are large depth and perspective disparities between the captured images, which also pose a challenge to image stitching. To overcome the above problems, we propose an unsupervised oral endoscope image stitching algorithm based on the extraction of overlapping regions and the loss of deep features. In the registration stage, we extract the overlapping region of the input images by sketching polygon intersection for feature points screening and estimate homography from coarse to fine on a three-layer feature pyramid structure. Moreover, we calculate loss using deep features instead of pixel values to emphasize the importance of depth disparities in homography estimation. Finally, we reconstruct the stitched images from feature to pixel, which can eliminate artifacts caused by large parallax. Our method is compared with both feature-based and previous deep-based methods on the UDIS-D dataset and our oral endoscopy image dataset. The experimental results show that our algorithm can achieve higher homography estimation accuracy, and better visual quality, and can be effectively applied to oral endoscope image stitching.

Key words: oral endoscope image, overlapping region, homography estimation, image stitching

摘要： 口腔内窥镜图像拼接算法通过配准、拼接等处理获取宽视野口腔图像以满足辅助诊断的需求。与自然图像相比，口腔内窥镜图像的纹理特征少。然而，传统的基于特征的图像拼接方法严重依赖于特征提取的质量，在拼接特征较少的图像时，往往无法令人满意。此外，由于手持拍摄，拍摄的图像之间存在较大的深度和视角差异，这也给图像拼接带来了挑战。为了克服上述问题，提出了一种基于重叠区域提取和深度特征丢失的无监督口腔内窥镜图像拼接算法。在配准阶段，通过绘制多边形交点来提取输入图像的重叠区域进行特征点筛选，并在三层特征金字塔结构上由粗到精进行单应性估计。此外，使用深度特征而不是像素值来计算损失，以强调深度差异在单应性估计中的重要性。最后，对拼接后的图像进行从特征到像素的重构，消除了视差过大带来的伪影。我们的方法在UDIS-D数据集和我们的口腔内窥镜图像数据集上与基于特征和先前基于深度的方法进行了比较。实验结果表明，该算法具有较高的单应性估计精度和较好的视觉质量，可有效应用于口腔内窥镜图像拼接。

关键词: 口腔内窥镜图像，重叠区域，单应性估计，图像拼接