Leveraging Vision Transformers for High-Precision Classification of Cancer Cell Cultures: A Comparative Study on MDA-MB-231 and PC3 Datasets.

Goal: Accurate classification of cancer cell cultures is critical for understanding tumor behavior, drug responses, and disease progression. Traditional manual evaluation methods are often subjective and prone to errors, necessitating automated approaches based on deep learning. Methods: We design a pipeline using Otsu thresholding, morphological filtering, and watershed segmentation, coupled with class-balanced augmentation. A baseline CNN, two attention-augmented variants: CNN-SE (squeeze-and-excitation) and CNN-CBAM (channel-spatial attention) and a ViT are benchmarked. All models are tuned through a 5-fold cross-validation grid search and evaluated on MDA-MB-231 (triple negative breast cancer) and PC3 (prostate cancer) cell images. The study employed both quantitative and qualitative approaches to comprehensively assess the effectiveness and reliability of the proposed model. Results: Attention modules substantially strengthen CNN performance but ViT achieves the best overall accuracy and generalization, particularly reflecting an advantage in modeling long-range dependencies. Conclusions: Channel and spatial attention narrows the gap between CNNs and transformers, while transformers provide the highest end-to-end performance. These results support attention-augmented CNNs and ViTs as robust complementary tools for automated analysis of cancer cell cultures.