DeformNeXt-Swin: A Hybrid CNN-Transformer Framework for Breast Lesion Classification in Ultrasound and Mammography

Abstract

Accurate breast lesion classification in ultrasound and mammography requires models that can capture fine local morphology together with broader tissue-level context. This remains challenging because ultrasound images often contain speckle noise, low-contrast lesion margins, and operator-dependent appearance variation, whereas mammograms require recognition of subtle structural patterns such as spiculation, architectural distortion, and tissue asymmetry. We propose DeformNeXt-Swin, a staged hybrid CNN-Transformer architecture that combines a deformable convolution-based front-end with a Swin Transformer-based back-end. The early DeformNeXtSwiGLU stages model irregular lesion boundaries through adaptive sampling and gated response refinement, while the later Swin-SwiGLU stages capture longer-range contextual relationships after spatial features become more compact and semantically enriched. The model was evaluated independently on the BUSI ultrasound dataset and a DDSM-derived mammography dataset under a unified experimental protocol. It was compared with 24 representative CNN, Transformer, and hybrid baselines, including ConvNeXt-Base, EfficientNetV2-S, DeiTBase, Swin-Base, GCViT-Base, PiT-B Distilled, TinyViT, and ViT-Base. Within this controlled benchmark, DeformNeXt-Swin achieved the best performance among the evaluated models, with an accuracy of 0.9750 and an F1-score of 0.9694 on ultrasound, and an accuracy of 0.9529 and an F1-score of 0.9474 on mammography. Expanded ablation experiments associated the performance gains with deformable local modeling, SwiGLU-GRN refinement, and the Stage 2 transition from convolutional to Transformer-based processing. DeformNeXt-Swin provides an effective image-level classification backbone for breast lesion analysis across two distinct imaging modalities.