To enhance the tracking stability of DeepOCSORT, this paper proposes a novel multi-sensor data fusion-based multi-object tracking (MOT) method. Specifically, we build upon the DeepOCSORT foundation and additionally integrate target velocity information directly measured by light detection and ranging (LiDAR). The introduction of this velocity information is conducted from three perspectives. Firstly, during data association, a penalty term is constructed based on the differences in target velocities to constrain generating matches with consistent velocities. Secondly, use LiDAR velocity for initialization and online updating of the velocity state within the tracker, making tracking predictions more stable. Thirdly, control the degree of dependence on velocity information by adjusting the process noise covariance matrix. Evaluation results on the KITTI dataset demonstrate that compared to the original DeepOCSORT, the proposed improved multi-source heterogeneous information fusion method significantly enhances tracking performance, with maximum improvements of 3.35, 3.26, and 3.71 on the higher order tracking accuracy (HOTA), multi-object tracking accuracy (MOTA), and interaction detection F1 score (IDF1) metrics, respectively. This study provides an effective approach to building a more stable and accurate MOT system.