To address the issue that most current automatic fabric defect detection methods still require manual selection of training sets and cannot achieve unsupervised learning, an automatic unsupervised defect detection method using the median robust extended local binary pattern (MRELBP) feature for flawless image screening and an under-complete dictionary reconstruction method for defect point detection are proposed. An adaptive dictionary size search algorithm is also proposed to automatically select a suitable dictionary size. First, the algorithm selects the flawless images from fabric images. Then, K-SVD is applied to obtain an under-complete dictionary from the normal image blocks as the training set. Finally, the reconstruction-base scheme is used to identify defects with the structural similarity index measure (SSIM) threshold. Experimental results of 334 plain fabrics with warp, weft, and block defects show that compared to the K-SVD method that uses residual segmentation for defect detection, the proposed method increases the correct detection rate up to 21.81%, reduces the false detection rate up to 0.72%, and a 50% increase in detection speed per image on average. The proposed algorithm achieved an average correct detection rate of 83.29% on the AITEX dataset, demonstrating its effectiveness.