02_Deepcluster.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to your second assignment!\n",
    "\n",
    "Please submit your solution of this notebook in the Whiteboard at the corresponding Assignment entry. We need you to upload the .ipynb-file and the exported .pdf of this notebook.\n",
    "\n",
    "If you have any questions, ask them in either in the tutorials or in the \"Mattermost\" channel: https://mattermost.imp.fu-berlin.de/biorobotics/channels/ssl_ws_2324\n",
    "\n",
    "\n",
    "In this assignment, we want you to get more comfortable with unsupervised learning algorithms and building models using the machine learning library [PyTorch](https://pytorch.org/). This also includes deadling with latent representations of our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Slide Review\n",
    "\n",
    "[Google Form](https://forms.gle/7EjyGKvmbBUT2N7y7) for the slide review. Please take a minute to scroll over the slides again and improve your lecture.\n",
    "\n",
    "Please make sure to only choose your top 5 slides per lecture!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PapagAI\n",
    "\n",
    "From this week onwards we start the reflective study we told you about on the first lecture.\n",
    "Register on the [PapagAI website](https://www.papag.ai) and write your first reflection about your impressions and challenges in the context of the lectures and tutorials you had this and previous week. The size of reflection can be anywhere bigger than 100 words. You can check out this [YouTube video](https://www.youtube.com/watch?v=QdmZHocZQBk&ab_channel=FernandoRamosL%C3%B3pez) with instructions on how to register, create a reflection and get an ai feedback.\n",
    "\n",
    "Please note, that this task is an obligatory one for this course and make sure each of you does the reflection, not only one person per group."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Please state both names of your group members here:\n",
    "Authors: Jane and John Doe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 2: DeepCluster\n",
    "\n",
    "Inspired by [DeepCluster](https://arxiv.org/pdf/1807.05520.pdf), we want you perform representation learning on the MNIST dataset using kMeans.\n",
    "\n",
    "\n",
    "## Ex. 2.1 Building a PyTorch model\n",
    "Build a model with the follwoing architecture:\n",
    "\n",
    "### *Input layer -> 4x Conv layer with [8,16,32,16] channels (kernel size of your choice) -> 1 FC layer -> output layer* <br>\n",
    "\n",
    "Further information on the training process:\n",
    "- Use ReLU functions as activations for all layers except the output layer,\n",
    "- Use the softmax function on your output,\n",
    "- Your training algorithm should use mini-batches with the size of your choice,\n",
    "- Pick the optimizer you like the most\n",
    "\n",
    "Train your classifier model on the [MNIST](https://pytorch.org/vision/main/generated/torchvision.datasets.MNIST.html) handwritten digits dataset. You should be able to achieve an accuracy of at least 95% on the test set. Plot the loss per training epoch/iteration.  **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 2.2 kMeans\n",
    "\n",
    "Implement the k-Means clustering algorithm. **(RESULT)**\n",
    "\n",
    "Plot the within cluster variance for different k's on the MNIST test set. **(RESULT)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def your_kMeans():\n",
    "    pass\n",
    "    # Code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ex. 2.3 DeepCluster\n",
    "\n",
    "[DeepCluster](https://arxiv.org/pdf/1807.05520.pdf) trains representations using k-Means labels. We want you to do the same with your model and kMeans algorithm above.\n",
    "Train your model from scratch again on a classifier (one fully connected layer) using the newly assigned kMeans labels as targets. **(RESULT)**<br>\n",
    "\n",
    "Evaluate the model performance training only a new classifier (again, one fc layer), and do NOT retrain the whole model. State the accuracy of your final classifier on the MNIST test set. **(RESULT)**\n",
    "\n",
    "Info: If you think your k-Means has flaws or is running too slow for this task, feel free to use a library version, i.e. sklearn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}