To address the issues of information fragmentation and insufficient robustness caused by the modular design of traditional perception systems for unmanned vehicles, this paper proposes a Transformer-based multi-modal BEV fusion algorithm. Firstly, the vision branch, built upon BEVFormer, constructs an image spatio-temporal collaborative encoding mechanism. This is combined with a semantic-guided BEV projection to enhance the representation of key regions, mitigating feature distortion caused by lighting variations and motion blur. Secondly, the radar branch introduces a sparse matrix-state space hybrid model （Sparse-SSM）, which tackles the sparsity and discontinuity of point clouds through sparse voxelization and state transition modeling. Finally, a Transformer cross-attention mechanism is employed to achieve precise alignment and fusion of cross-modal features. The model is trained and experimentally validated on the nuScenes dataset. Results demonstrate that compared to the image-only and point cloud-only branches, the proposed method improves the target detection score by 15% and 23.3%, respectively. Compared to the advanced baseline BEVFusion method, our approach achieves a 2.7% increase in average precision and a 2.3% increase in the target detection score, while reducing floating point operations by 20% through its sparse design and achieving faster inference speed. This provides unmanned vehicles with high-precision, highly robust environmental perception capabilities.