Manual surface detection of cable-stayed bridge cables is low accuracy and high labor-intensive.The speed of conventional image processing and convolutional neural networks is too low to meet the re-quirements for timely detection.Therefore,a novel encoder-decoder network is constructed to detect ca-ble surface defects.The optimized MobileNetV2 is used as the encoder to reduce the model parameters and increase the training speed.The UNet idea and pyramid pooling(PSP)module are used in the de-coder to enhance the feature extraction.Moreover,skip connections connect the encoder and decoder to fuse the deep and shallow feature information effectively.The PASCAL VOC dataset is used to pre-train the network to obtain the weight values of the network,which are then loaded into the network to ob-tain the final parameters through the training of defect datasets such as holes,gaps and damages.The ex-periments demonstrate that the novel encoder-decoder network is robust.The mean pixel accuracy,mean intersection over union and the processing time of single image are 89.88%,79.25%and 41.34 ms re-spectively,which are better than the methods,such as PSPNet,UNet and DFANet.In summary,the no-vel network meets the requirements of accuracy and speed for surface defect detection of cable-stayed bridge.