Abstract: Public safety issues in autonomous driving receive widespread social attention. The technology of 3D single-object tracking based on point cloud data gains significant attention due to its crucial role in robotics and autonomous driving. Currently, most mainstream trackers leverage the ability of transformers to capture long-range dependencies for similarity matching or enhancing target features. However, this application overlooks the flexibility of transformers in adapting to various pre-trained models and increases the computational burden. To address this limitations, we introduce a pre-trained tracker based on a single-branch framework (SPTracker). The essence of this work lies in the efficient fine-tuning of pre-trained parameters applied to the domain of 3D single-object tracking. However, unique challenges and potential domain gaps make this application less straightforward than intuitively expected. First, when transferring a pre-trained backbone to downstream tasks, structural compatibility between upstream and downstream tasks is often required. Second, compared to the complex objects that point cloud analysis tasks must handle, dynamic tracked objects typically have fewer categories and greater geometric similarity. To overcome these limitations, based on the single-branch framework, we introduce a patch embedding network to unify the position embedding and input of the transformer, thereby enhancing the model’s adaptability between upstream and downstream tasks. Additionally, through efficient fine-tuning of the pre-trained network parameters, we balance the learning capacity of the entire model. Finally, we design an efficient object localization network in a dense bird’s-eye view feature space to achieve better performance with reduced computational overhead. The proposed SPTracker operates at a speed of 52.6 frames per second, reducing the model training time from approximately 48 h to around 16 h, while achieving advanced performance on the KITTI and nuScenes datasets.