Aiming at the problem that the fasteners in the ballast-less track of high-speed railway become loose, resulting in the deflection or loss of high-speed railway fasteners, this paper proposes a high-speed railway fastener detection algorithm based on deformable convolution improved Faster R-CNN. Deformable Convolution is introduced in the feature extraction network to build Deformable Residual Convolution Block (DRCB), which makes the feature extraction process more focused on the fastener region to achieve accurate extraction of fastener state; and Alpha-IoU is used as the target regression loss function to improve the regression accuracy of high-speed railway fasteners. The experimental results show that in the detection of the fastener state of high-speed railway, the improved Faster R-CNN algorithm has 99.34% detection accuracy for the offset state of high-speed railway fasteners and 76.80% detection accuracy for the lost state, with an average accuracy mean value of 88.07%, compared with other algorithms, the improved Faster R-CNN algorithm has the highest detection accuracy.