VideoColorizerColab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jantic/DeOldify/blob/master/VideoColorizerColab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **<font color='blue'> Video Colorizer </font>**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "663IVxfrpIAb"
   },
   "source": [
    "#◢ DeOldify - Colorize your own videos!\n",
    "\n",
    "\n",
    "_FYI: This notebook is intended as a tool to colorize gifs and short videos, if you are trying to convert longer video you may hit the limit on processing space. Running the Jupyter notebook on your own machine is recommended (and faster) for larger video sizes._\n",
    "\n",
    "####**Credits:**\n",
    "\n",
    "Big special thanks to:\n",
    "\n",
    "Robert Bell for all his work on the video Colab notebook, and paving the way to video in DeOldify!\n",
    "\n",
    "Dana Kelley for doing things, breaking stuff & having an opinion on everything."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZjPqTBNoohK9"
   },
   "source": [
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "#◢ Verify Correct Runtime Settings\n",
    "\n",
    "**<font color='#FF000'> IMPORTANT </font>**\n",
    "\n",
    "In the \"Runtime\" menu for the notebook window, select \"Change runtime type.\" Ensure that the following are selected:\n",
    "* Runtime Type = Python 3\n",
    "* Hardware Accelerator = GPU \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gaEJBGDlptEo"
   },
   "source": [
    "#◢ Git clone and install DeOldify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-T-svuHytJ-8"
   },
   "outputs": [],
   "source": [
    "!git clone https://github.com/jantic/DeOldify.git DeOldify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd DeOldify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BDFjbNxaadNJ"
   },
   "source": [
    "#◢ Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "00_GcC_trpdE"
   },
   "outputs": [],
   "source": [
    "#NOTE:  This must be the first call in order to work properly!\n",
    "from deoldify import device\n",
    "from deoldify.device_id import DeviceId\n",
    "#choices:  CPU, GPU0...GPU7\n",
    "device.set(device=DeviceId.GPU0)\n",
    "\n",
    "import torch\n",
    "\n",
    "if not torch.cuda.is_available():\n",
    "    print('GPU not available.')\n",
    "\n",
    "from os import path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Lsx7xCXNSVt6"
   },
   "outputs": [],
   "source": [
    "!pip install -r colab_requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MsJa69CMwj3l"
   },
   "outputs": [],
   "source": [
    "import fastai\n",
    "from deoldify.visualize import *\n",
    "from pathlib import Path\n",
    "torch.backends.cudnn.benchmark=True\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", category=UserWarning, message=\".*?Your .*? set is empty.*?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir 'models'\n",
    "!wget https://data.deepai.org/deoldify/ColorizeVideo_gen.pth -O ./models/ColorizeVideo_gen.pth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://media.githubusercontent.com/media/jantic/DeOldify/master/resource_images/watermark.png -O ./resource_images/watermark.png"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tzHVnegp21hC"
   },
   "outputs": [],
   "source": [
    "colorizer = get_video_colorizer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#◢ Instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### source_url\n",
    "Type in a url hosting a video from YouTube, Imgur, Twitter, Reddit, Vimeo, etc.  Many sources work!  GIFs also work.  Full list here: https://ytdl-org.github.io/youtube-dl/supportedsites.html NOTE: If you want to use your own video, upload it first to a site like YouTube. \n",
    "\n",
    "### render_factor\n",
    "The default value of 21 has been carefully chosen and should work -ok- for most scenarios (but probably won't be the -best-). This determines resolution at which the color portion of the video is rendered. Lower resolution will render faster, and colors also tend to look more vibrant. Older and lower quality film in particular will generally benefit by lowering the render factor. Higher render factors are often better for higher quality videos and inconsistencies (flashy render) will generally be reduced, but the colors may get slightly washed out.\n",
    "\n",
    "### watermarked\n",
    "Selected by default, this places a watermark icon of a palette at the bottom left corner of the image.  This is intended to be a standard way to convey to others viewing the image that it is colorized by AI. We want to help promote this as a standard, especially as the technology continues to improve and the distinction between real and fake becomes harder to discern. This palette watermark practice was initiated and lead by the company MyHeritage in the MyHeritage In Color feature (which uses a newer version of DeOldify than what you're using here).\n",
    "\n",
    "### How to Download a Copy\n",
    "Simply right click on the displayed video and click \"Save video as...\"!\n",
    "\n",
    "## Pro Tips\n",
    "1. If a video takes a long time to render and you're wondering how well the frames will actually be colorized, you can preview how well the frames will be rendered at each render_factor by using the code at the bottom. Just stop the video rendering by hitting the stop button on the cell, then run that bottom cell under \"See how well render_factor values perform on a frame here\". It's not perfect and you may still need to experiment a bit especially when it comes to figuring out how to reduce frame inconsistency.  But it'll go a long way in narrowing down what actually works.\n",
    "2. If videos are taking way too much time for your liking, running the Jupyter notebook VideoColorizer.ipynb on your own machine (with DeOldify installed) will generally be much faster (as long as you have the hardware for it).   \n",
    "3. Longer videos (running multiple minutes) are going to have a rough time on Colabs. You'll be much better off using a local install of DeOldify instead in this case.\n",
    "\n",
    "## Troubleshooting\n",
    "The video player may wind up not showing up, in which case- make sure to wait for the Jupyter cell to complete processing first (the play button will stop spinning).  Then follow these alternative download instructions\n",
    "\n",
    "1. In the menu to the left, click Files\n",
    "2. If you don't see the 'DeOldify' folder, click \"Refresh\"\n",
    "3. By default, rendered video will be in /DeOldify/video/result/\n",
    "\n",
    "If a video you downloaded doesn't play, it's probably because the cell didn't complete processing and the video is in a half-finished state."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sUQrbSYipiJn"
   },
   "source": [
    "#◢ Colorize!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "source_url = '' #@param {type:\"string\"}\n",
    "render_factor = 21  #@param {type: \"slider\", min: 5, max: 40}\n",
    "watermarked = True #@param {type:\"boolean\"}\n",
    "\n",
    "if source_url is not None and source_url !='':\n",
    "    video_path = colorizer.colorize_from_url(source_url, 'video.mp4', render_factor, watermarked=watermarked)\n",
    "    show_video_in_notebook(video_path)\n",
    "else:\n",
    "    print('Provide a video url and try again.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## See how well render_factor values perform on a frame here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(10,40,2):\n",
    "    colorizer.vis.plot_transformed_image('video/bwframes/video/00001.jpg', render_factor=i, display_render_factor=True, figsize=(8,8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "X7Ycv_Y9xAHp"
   },
   "source": [
    "---\n",
    "#⚙ Recommended video and gif sources \n",
    "* [/r/Nickelodeons/](https://www.reddit.com/r/Nickelodeons/)\n",
    "* [r/silentmoviegifs](https://www.reddit.com/r/silentmoviegifs/)\n",
    "* https://twitter.com/silentmoviegifs "
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "VideoColorizerColab.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}