In order to solve the challenge of breast cancer region segmentation, we improved the U-Net. The convolutional block attention module with prioritized attention (CBAM-PA) and dilated transformer (Dformer) modules were designed to replace the convolutional layers at the encoding side in the base U-Net, the input logic of the U-Net was improved by dynamically adjusting the input size of each layer, and the short connections in the U-Net were replaced with cross-layer connections to enhance the image restoration capability at the decoding side. On the breast ultrasound images (BUSI) dataset, we obtain a Dice coefficient of 0.803 1 and an intersection-over-union (IoU) value of 0.736 2. The experimental results show that the proposed enhancement method effectively improves the accuracy and quality of breast cancer lesion region segmentation.