Semantic segmentation is a key technology in autonomous driving. Outdoor scene image semantic segmentation faces challenges like environmental complexity and sample imbalance, leading to suboptimal performance. To address these issues, this paper proposes a semantic segmentation network for outdoor scenes based on feature branch enhancement, FBE-Net. FBE-Net adopts an encoder-decoder architecture and designs a feature enhancement branch. It utilizes multi-scale dilated attention to capture key features and enhance overall accuracy, and employs a memory module to address sample imbalance. Simultaneously considering lightweight design. We collected campus scene data using an HD camera, annotated it with semantic labels, and created a campus scene semantic segmentation dataset. Experiments were conducted on the Cityscapes dataset and the self-built dataset. The experimental results showed that FBE-Net achieved a mIoU of 79.64% on the Cityscapes dataset and 78.01% on the self-made dataset, outperforming mainstream semantic segmentation methods.