Abstract: To address the issues of low accuracy, false positives, and false negatives in the detection of distant, small, and occluded objects, we proposes an improved algorithm named YOLOv8-DSM (deep small-object model), based on YOLOv8, as a replacement for the original framework. The improvements include: replacing the original framework with DWConv (depth-wise separable convolution) and GELU (Gaussian error linear unit) activation functions to reduce computational complexity and create a lightweight network; designing an SPPFCG (SPPF-Conv-AvgPool) module to replace the SPPF (spatial pyramid pooling - fast) module, minimizing local feature loss while mitigating overfitting and enhancing feature diversity; introducing the DPC2f (dual-pooling C2f) module as a substitute for C2f module to strengthen multiscale feature extraction and improve recognition of varying-sized targets; incorporating a small-target detection head and an SA (shuffle attention) mechanism to boost sensitivity to small-target detection features; proposing the MPDIoU_P (maximum precision distance IoU_ precise) loss function with enhanced penalty terms for width and height errors to refine small-target localization. The optimized network demonstrates superior accuracy in detecting small and obscured objects, accelerated convergence, and improved average precision. Evaluations on the KITTI dataset reveal that YOLOv8-DSM surpass YOLOv8n by4.6% higher mAP (mean average precision), 7.1% higher recall rate, and 5.9×10⁶ fewer parameters. Module comparison and ablation experiments validate the algorithm’s practicality and advantages for small-target and occluded-object detection tasks. Generalization experiments conducted on the VisDrone dataset further validated the effectiveness and generalization ability of our improved model for object detection tasks in traffic scenarios.