The original backbone of the SSD network was replaced with a lightweight MobileNetV2 network. At the same time, the channel attention mechanism SENet structure was introduced to apply different weights to different channels of the feature map, making it more conducive to detection. Using the crack data in three different environments to fuse, a new data set was built so that the network can better learn crack objects in different environments, further improving the network′s recognition ability and speeding up the network′s reasoning speed. The anchor in the original SSD network were set according to the general data set, and the data type concerned in the detection of structural surface cracks was relatively single. The K-means algorithm was used to cluster the anchor suitable for the crack data set, which speeded up the convergence of the model and was more efficient than the original one. The model had higher accuracy. Finally, the accuracy of the improved algorithm is 3.6% higher than that of the previous one, and the detection speed is as much as 2 times that of the previous one, which is enough to meet real -time detection, the algorithm can provide a faster and more accurate detection method for the detection of structural apparent cracks in practical engineering.