2.이진 분류(로지스틱_회귀).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "4.로지스틱 회귀",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yLWxB9XMk28U",
        "colab_type": "text"
      },
      "source": [
        "# Logistic Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dq2UAIVClurv",
        "colab_type": "text"
      },
      "source": [
        "## Hypothesis\n",
        "$$ H(X) = \\frac{1}{1+e^{-WX}} $$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ZbwZhcZl8Rc",
        "colab_type": "text"
      },
      "source": [
        "## Cost\n",
        "$$ cost(W) = -\\frac{1}{m}\\Sigma ylog(H(x)) + (1 - y)(log(1 - H(x)) $$\n",
        "- $Log$ loss for binary classification\n",
        "  - If $ y \\simeq H(x)$, cost converges to 0.\n",
        "  - If $ y \\neq H(x)$, cost converges to $\\infty$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ikgnr9jwnR7X",
        "colab_type": "text"
      },
      "source": [
        "## Weight Update via Gradient Descent\n",
        "$$ W := W - lr\\cdot \\frac{\\partial}{\\partial W}cost(W)$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2lG1GqGEn05x",
        "colab_type": "text"
      },
      "source": [
        "# Prepare Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZFcK8GO2lv0q",
        "colab_type": "code",
        "outputId": "1787ceac-ca9b-4c09-f58c-1727cf654696",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 158
        }
      },
      "source": [
        "import torch\n",
        "import torch.optim as optim\n",
        "\n",
        "torch.manual_seed(1)\n",
        "\n",
        "x_data = [[1, 2], [2, 3], [3, 1], [4, 3], [5, 3], [6, 2]]\n",
        "y_data = [[0], [0], [0], [1], [1], [1]]\n",
        "\n",
        "x_train = torch.FloatTensor(x_data)\n",
        "y_train = torch.FloatTensor(y_data)\n",
        "\n",
        "print(x_train)\n",
        "print(x_train.shape)\n",
        "print(y_train.shape)"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[1., 2.],\n",
            "        [2., 3.],\n",
            "        [3., 1.],\n",
            "        [4., 3.],\n",
            "        [5., 3.],\n",
            "        [6., 2.]])\n",
            "torch.Size([6, 2])\n",
            "torch.Size([6, 1])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "36fb8seLoNer",
        "colab_type": "text"
      },
      "source": [
        "# Train Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lk6PmgvYoXPP",
        "colab_type": "code",
        "outputId": "23d0dbee-3fca-433c-c36c-1f7915243b29",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 210
        }
      },
      "source": [
        "W = torch.ones((2, 1), requires_grad=True)\n",
        "b = torch.zeros(1, requires_grad=True)\n",
        "\n",
        "optimizer = optim.SGD([W, b], lr=1)\n",
        "\n",
        "epochs = 1000\n",
        "for epoch in range(epochs + 1):\n",
        "    # Calculate Cost\n",
        "    hypothesis = torch.sigmoid(x_train.matmul(W) + b)\n",
        "    cost = -(y_train * torch.log(hypothesis) +\n",
        "             (1 - y_train) * torch.log(1 - hypothesis)).mean()\n",
        "    \n",
        "    # Train\n",
        "    optimizer.zero_grad()\n",
        "    cost.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    if epoch % 100 == 0:\n",
        "        print('Epoch {:4d}/{} Cost: {:.6f}'.format(\n",
        "            epoch, epochs, cost.item()\n",
        "        ))"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch    0/1000 Cost: 2.012506\n",
            "Epoch  100/1000 Cost: 0.131138\n",
            "Epoch  200/1000 Cost: 0.079377\n",
            "Epoch  300/1000 Cost: 0.057256\n",
            "Epoch  400/1000 Cost: 0.044908\n",
            "Epoch  500/1000 Cost: 0.036998\n",
            "Epoch  600/1000 Cost: 0.031483\n",
            "Epoch  700/1000 Cost: 0.027413\n",
            "Epoch  800/1000 Cost: 0.024282\n",
            "Epoch  900/1000 Cost: 0.021798\n",
            "Epoch 1000/1000 Cost: 0.019778\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PCDz7KX1prB1",
        "colab_type": "text"
      },
      "source": [
        "# Evaluate Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jQ560dl9pad4",
        "colab_type": "code",
        "outputId": "bb556b89-31b3-4858-dc7e-af39aee3d187",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 228
        }
      },
      "source": [
        "hypothesis = torch.sigmoid(x_train.mm(W) + b)\n",
        "\n",
        "prediction = hypothesis >= torch.FloatTensor([0.5]) # T가 0.5보다 크거나 같으면 True, 아니면 False\n",
        "print(prediction)\n",
        "\n",
        "correct_prediction = prediction.float() == y_train  # T를 1.0, 0.0 으로 바꾼 다음 Y와 비교.\n",
        "print(correct_prediction)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[False],\n",
            "        [False],\n",
            "        [False],\n",
            "        [ True],\n",
            "        [ True],\n",
            "        [ True]])\n",
            "tensor([[True],\n",
            "        [True],\n",
            "        [True],\n",
            "        [True],\n",
            "        [True],\n",
            "        [True]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iBnVXUDiqDAl",
        "colab_type": "code",
        "outputId": "2809ba7c-c586-4dc9-9de0-f0cd41621046",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "accuracy = correct_prediction.sum().item() / len(correct_prediction)\n",
        "print('The model has an accuracy of {:2.2f}% for the training set.'.format(accuracy * 100))"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The model has an accuracy of 100.00% for the training set.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v70dHimtqMx1",
        "colab_type": "code",
        "outputId": "62f79c79-fb7a-4aec-e423-bbba0113bbda",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "XX = [[100, 5]]\n",
        "xx = torch.FloatTensor(XX);\n",
        "hypothesis = torch.sigmoid(xx.matmul(W) + b)\n",
        "prediction = hypothesis >= torch.FloatTensor([0.5])\n",
        "print(prediction)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[True]])\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}