Skip to content

A comprehensive guide on how to implement and train a Compact Convolutional Transformer model that is specifically designed for classifying skin cancer.

License

Notifications You must be signed in to change notification settings

MatthewJansen/Application-of-CCT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 

Repository files navigation

Application of Compact Convolutional Transformers (CCTs)

Skin Cancer

Image Source: PapersWithCode | Compact Convolutional Transformers

📄 Overview

In recent years, Transformers have become the go-to for Natural Language Processing (NLP) and have been used in research for Computer Vision and Time Series Forecasting to make groundbreaking discoveries.

There is a clear association between the increasing number of trainable parameters in models and the progress made in these domains, and many researchers may believe this is due to the inability of transformers to adapt to small datasets.

The Compact Convolutional Transformer approach introduced in the paper "Escaping the Big Data Paradigm with Compact Transformers" addresses the issues with Transformer models on small datasets.

The CCT architecture aims to increase performance and provide flexibility for input image sizes whilst demonstrating the independence of Positional Embedding in contrast to other transformer-based architectures. The utilization of the CCT architecture empowers researchers to achieve noteworthy results despite having restricted trainable parameters, small datasets, and a single processing unit, be it a GPU or a CPU.

CCT (Diagram)

Image Source: PapersWithCode | Compact Convolutional Transformers

This notebook aims to provide a comprehensive guide on how to implement and train a CCT model that is specifically designed for classifying skin cancer. The model will accurately identify potential skin cancer cases, which will promote early intervention and improve patient outcomes.

For more information on the Compact Convolutional Transformer architectures and Transformers in general, please refer to the following links:


📁 Dataset

The dataset used in this notebook is known as the International Skin Imaging Collaboration (ISIC) Skin Cancer Detection Dataset. The ISIC is an academia and industry partnership designed to facilitate the application of digital skin imaging to help reduce melanoma mortality. This notebook uses the Kaggle version of this dataset.

For more information check the following:


Requirements

  • Recommended python version: Python +3.8.10 64-bit

[Please note that this section will be updated in due course.]

Project Structure

.
├── LICENSE
├── README.md
└── application-of-compact-convolutional-transformers.ipynb
  • LICENSE | project license (MIT)
  • README.md | project readme file
  • application-of-compact-convolutional-transformers.ipynb | project notebook

Usage

See application-of-compact-convolutional-transformers.ipynb

License

This project is licensed under the terms and conditions of the MIT license.

Releases

No releases published

Packages

No packages published