Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI. Phase 1 consists of generating synthetic images to balance all the classes in the dataset. Phase 2 consists of applying different data augmentation techniques and morphological operations to increase the size of the data. Phases 3 & 4 involve developing a ViT model for edge computing systems that can identify patterns and categorize skin lesions from the user's skin visible in the image. In phase 3, after classifying the lesions into the desired class with ViT, we will use explainable AI (XAI) that leads to more explainable results (using activation maps, etc.) while ensuring high predictive accuracy. Real-time images of skin diseases can capture by a doctor or a patient using the camera of a mobile application to perform an early examination and determine the cause of the skin lesion. The whole framework is compared with the existing frameworks for skin lesion detection.
翻译:皮肤癌是人类癌症最流行的形式之一,主要以视觉为主,从临床筛查开始,继续进行脱温检查、组织学评估和样本收集。深演神经网络(CNNs)对分类细微粒对象执行高度隔离和潜在的普遍性任务。该研究提出了一个新的多级预测框架,根据ViT和ViTGAN对皮肤损伤进行分类。以视觉变压器为基础的GANs(Geneative Aversarial Networks)用来解决阶级失衡问题。框架由四个主要阶段组成:ViTGANs、图像处理和可解释的AI。第一阶段是生成合成图像,以平衡数据集中的所有类别。第二阶段是应用不同的数据增强技术和形态操作,以扩大数据规模。第3和第4阶段是开发一个边缘计算系统的ViT模型,可以识别图中可见的用户皮肤损伤模式,并将现有皮肤损伤分类为可观察的。第3阶段,在用ViT类、图像处理和可解释的皮肤检测结果后,我们将用真实的图像来解释,然后用直测测测,然后用可测测测。