movie-nlp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Processing of Movie Reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import the pandas package, then use the \"read_csv\" function to read\n",
    "# the labeled training data\n",
    "import pandas as pd       \n",
    "train = pd.read_csv(\"movie-data/labeledTrainData.tsv\", header=0, \\\n",
    "                    delimiter=\"\\t\", quoting=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>sentiment</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>\"5814_8\"</td>\n",
       "      <td>1</td>\n",
       "      <td>\"With all this stuff going down at the moment ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>\"2381_9\"</td>\n",
       "      <td>1</td>\n",
       "      <td>\"\\\"The Classic War of the Worlds\\\" by Timothy ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>\"7759_3\"</td>\n",
       "      <td>0</td>\n",
       "      <td>\"The film starts with a manager (Nicholas Bell...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>\"3630_4\"</td>\n",
       "      <td>0</td>\n",
       "      <td>\"It must be assumed that those who praised thi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>\"9495_8\"</td>\n",
       "      <td>1</td>\n",
       "      <td>\"Superbly trashy and wondrously unpretentious ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id  sentiment                                             review\n",
       "0  \"5814_8\"          1  \"With all this stuff going down at the moment ...\n",
       "1  \"2381_9\"          1  \"\\\"The Classic War of the Worlds\\\" by Timothy ...\n",
       "2  \"7759_3\"          0  \"The film starts with a manager (Nicholas Bell...\n",
       "3  \"3630_4\"          0  \"It must be assumed that those who praised thi...\n",
       "4  \"9495_8\"          1  \"Superbly trashy and wondrously unpretentious ..."
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(25000, 3)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['id', 'sentiment', 'review'], dtype=object)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.columns.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"With all this stuff going down at the moment with MJ i've started listening to his music, watching the odd documentary here and there, watched The Wiz and watched Moonwalker again. Maybe i just want to get a certain insight into this guy who i thought was really cool in the eighties just to maybe make up my mind whether he is guilty or innocent. Moonwalker is part biography, part feature film which i remember going to see at the cinema when it was originally released. Some of it has subtle messages about MJ's feeling towards the press and also the obvious message of drugs are bad m'kay.<br /><br />Visually impressive but of course this is all about Michael Jackson so unless you remotely like MJ in anyway then you are going to hate this and find it boring. Some may call MJ an egotist for consenting to the making of this movie BUT MJ and most of his fans would say that he made it for the fans which if true is really nice of him.<br /><br />The actual feature film bit when it finally starts is only on for 20 minutes or so excluding the Smooth Criminal sequence and Joe Pesci is convincing as a psychopathic all powerful drug lord. Why he wants MJ dead so bad is beyond me. Because MJ overheard his plans? Nah, Joe Pesci's character ranted that he wanted people to know it is he who is supplying drugs etc so i dunno, maybe he just hates MJ's music.<br /><br />Lots of cool things in this like MJ turning into a car and a robot and the whole Speed Demon sequence. Also, the director must have had the patience of a saint when it came to filming the kiddy Bad sequence as usually directors hate working with one kid let alone a whole bunch of them performing a complex dance scene.<br /><br />Bottom line, this movie is for people who like MJ on one level or another (which i think is most people). If not, then stay away. It does try and give off a wholesome message and ironically MJ's bestest buddy in this movie is a girl! Michael Jackson is truly one of the most talented people ever to grace this planet but is he guilty? Well, with all the attention i've gave this subject....hmmm well i don't know because people can be different behind closed doors, i know this for a fact. He is either an extremely nice but stupid guy or one of the most sickest liars. I hope he is not the latter.\"\n"
     ]
    }
   ],
   "source": [
    "print(train[\"review\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import BeautifulSoup into your workspace\n",
    "from bs4 import BeautifulSoup             \n",
    "\n",
    "# Initialize the BeautifulSoup object on a single movie review     \n",
    "example1 = BeautifulSoup(train[\"review\"][0], 'html.parser')  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"With all this stuff going down at the moment with MJ i've started listening to his music, watching the odd documentary here and there, watched The Wiz and watched Moonwalker again. Maybe i just want to get a certain insight into this guy who i thought was really cool in the eighties just to maybe make up my mind whether he is guilty or innocent. Moonwalker is part biography, part feature film which i remember going to see at the cinema when it was originally released. Some of it has subtle messages about MJ's feeling towards the press and also the obvious message of drugs are bad m'kay.<br /><br />Visually impressive but of course this is all about Michael Jackson so unless you remotely like MJ in anyway then you are going to hate this and find it boring. Some may call MJ an egotist for consenting to the making of this movie BUT MJ and most of his fans would say that he made it for the fans which if true is really nice of him.<br /><br />The actual feature film bit when it finally starts is only on for 20 minutes or so excluding the Smooth Criminal sequence and Joe Pesci is convincing as a psychopathic all powerful drug lord. Why he wants MJ dead so bad is beyond me. Because MJ overheard his plans? Nah, Joe Pesci's character ranted that he wanted people to know it is he who is supplying drugs etc so i dunno, maybe he just hates MJ's music.<br /><br />Lots of cool things in this like MJ turning into a car and a robot and the whole Speed Demon sequence. Also, the director must have had the patience of a saint when it came to filming the kiddy Bad sequence as usually directors hate working with one kid let alone a whole bunch of them performing a complex dance scene.<br /><br />Bottom line, this movie is for people who like MJ on one level or another (which i think is most people). If not, then stay away. It does try and give off a wholesome message and ironically MJ's bestest buddy in this movie is a girl! Michael Jackson is truly one of the most talented people ever to grace this planet but is he guilty? Well, with all the attention i've gave this subject....hmmm well i don't know because people can be different behind closed doors, i know this for a fact. He is either an extremely nice but stupid guy or one of the most sickest liars. I hope he is not the latter.\"\n",
      "\n",
      "\"With all this stuff going down at the moment with MJ i've started listening to his music, watching the odd documentary here and there, watched The Wiz and watched Moonwalker again. Maybe i just want to get a certain insight into this guy who i thought was really cool in the eighties just to maybe make up my mind whether he is guilty or innocent. Moonwalker is part biography, part feature film which i remember going to see at the cinema when it was originally released. Some of it has subtle messages about MJ's feeling towards the press and also the obvious message of drugs are bad m'kay.Visually impressive but of course this is all about Michael Jackson so unless you remotely like MJ in anyway then you are going to hate this and find it boring. Some may call MJ an egotist for consenting to the making of this movie BUT MJ and most of his fans would say that he made it for the fans which if true is really nice of him.The actual feature film bit when it finally starts is only on for 20 minutes or so excluding the Smooth Criminal sequence and Joe Pesci is convincing as a psychopathic all powerful drug lord. Why he wants MJ dead so bad is beyond me. Because MJ overheard his plans? Nah, Joe Pesci's character ranted that he wanted people to know it is he who is supplying drugs etc so i dunno, maybe he just hates MJ's music.Lots of cool things in this like MJ turning into a car and a robot and the whole Speed Demon sequence. Also, the director must have had the patience of a saint when it came to filming the kiddy Bad sequence as usually directors hate working with one kid let alone a whole bunch of them performing a complex dance scene.Bottom line, this movie is for people who like MJ on one level or another (which i think is most people). If not, then stay away. It does try and give off a wholesome message and ironically MJ's bestest buddy in this movie is a girl! Michael Jackson is truly one of the most talented people ever to grace this planet but is he guilty? Well, with all the attention i've gave this subject....hmmm well i don't know because people can be different behind closed doors, i know this for a fact. He is either an extremely nice but stupid guy or one of the most sickest liars. I hope he is not the latter.\"\n"
     ]
    }
   ],
   "source": [
    "print(train[\"review\"][0])\n",
    "print()\n",
    "print(example1.get_text())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " With all this stuff going down at the moment with MJ i ve started listening to his music  watching the odd documentary here and there  watched The Wiz and watched Moonwalker again  Maybe i just want to get a certain insight into this guy who i thought was really cool in the eighties just to maybe make up my mind whether he is guilty or innocent  Moonwalker is part biography  part feature film which i remember going to see at the cinema when it was originally released  Some of it has subtle messages about MJ s feeling towards the press and also the obvious message of drugs are bad m kay Visually impressive but of course this is all about Michael Jackson so unless you remotely like MJ in anyway then you are going to hate this and find it boring  Some may call MJ an egotist for consenting to the making of this movie BUT MJ and most of his fans would say that he made it for the fans which if true is really nice of him The actual feature film bit when it finally starts is only on for    minutes or so excluding the Smooth Criminal sequence and Joe Pesci is convincing as a psychopathic all powerful drug lord  Why he wants MJ dead so bad is beyond me  Because MJ overheard his plans  Nah  Joe Pesci s character ranted that he wanted people to know it is he who is supplying drugs etc so i dunno  maybe he just hates MJ s music Lots of cool things in this like MJ turning into a car and a robot and the whole Speed Demon sequence  Also  the director must have had the patience of a saint when it came to filming the kiddy Bad sequence as usually directors hate working with one kid let alone a whole bunch of them performing a complex dance scene Bottom line  this movie is for people who like MJ on one level or another  which i think is most people   If not  then stay away  It does try and give off a wholesome message and ironically MJ s bestest buddy in this movie is a girl  Michael Jackson is truly one of the most talented people ever to grace this planet but is he guilty  Well  with all the attention i ve gave this subject    hmmm well i don t know because people can be different behind closed doors  i know this for a fact  He is either an extremely nice but stupid guy or one of the most sickest liars  I hope he is not the latter  \n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "# Use regular expressions to do a find-and-replace\n",
    "letters_only = re.sub(\"[^a-zA-Z]\",           # The pattern to search for\n",
    "                      \" \",                   # The pattern to replace it with\n",
    "                      example1.get_text() )  # The text to search\n",
    "print(letters_only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lower_case = letters_only.lower()        # Convert to lower case\n",
    "words = lower_case.split()               # Split into words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml\n",
      "showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download()  # Download text data sets, including stop words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', \"you're\", \"you've\", \"you'll\", \"you'd\", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', \"she's\", 'her', 'hers', 'herself', 'it', \"it's\", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', \"that'll\", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', \"don't\", 'should', \"should've\", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', \"aren't\", 'couldn', \"couldn't\", 'didn', \"didn't\", 'doesn', \"doesn't\", 'hadn', \"hadn't\", 'hasn', \"hasn't\", 'haven', \"haven't\", 'isn', \"isn't\", 'ma', 'mightn', \"mightn't\", 'mustn', \"mustn't\", 'needn', \"needn't\", 'shan', \"shan't\", 'shouldn', \"shouldn't\", 'wasn', \"wasn't\", 'weren', \"weren't\", 'won', \"won't\", 'wouldn', \"wouldn't\"]\n"
     ]
    }
   ],
   "source": [
    "from nltk.corpus import stopwords # Import the stop word list\n",
    "print(stopwords.words(\"english\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['stuff', 'going', 'moment', 'mj', 'started', 'listening', 'music', 'watching', 'odd', 'documentary', 'watched', 'wiz', 'watched', 'moonwalker', 'maybe', 'want', 'get', 'certain', 'insight', 'guy', 'thought', 'really', 'cool', 'eighties', 'maybe', 'make', 'mind', 'whether', 'guilty', 'innocent', 'moonwalker', 'part', 'biography', 'part', 'feature', 'film', 'remember', 'going', 'see', 'cinema', 'originally', 'released', 'subtle', 'messages', 'mj', 'feeling', 'towards', 'press', 'also', 'obvious', 'message', 'drugs', 'bad', 'kay', 'visually', 'impressive', 'course', 'michael', 'jackson', 'unless', 'remotely', 'like', 'mj', 'anyway', 'going', 'hate', 'find', 'boring', 'may', 'call', 'mj', 'egotist', 'consenting', 'making', 'movie', 'mj', 'fans', 'would', 'say', 'made', 'fans', 'true', 'really', 'nice', 'actual', 'feature', 'film', 'bit', 'finally', 'starts', 'minutes', 'excluding', 'smooth', 'criminal', 'sequence', 'joe', 'pesci', 'convincing', 'psychopathic', 'powerful', 'drug', 'lord', 'wants', 'mj', 'dead', 'bad', 'beyond', 'mj', 'overheard', 'plans', 'nah', 'joe', 'pesci', 'character', 'ranted', 'wanted', 'people', 'know', 'supplying', 'drugs', 'etc', 'dunno', 'maybe', 'hates', 'mj', 'music', 'lots', 'cool', 'things', 'like', 'mj', 'turning', 'car', 'robot', 'whole', 'speed', 'demon', 'sequence', 'also', 'director', 'must', 'patience', 'saint', 'came', 'filming', 'kiddy', 'bad', 'sequence', 'usually', 'directors', 'hate', 'working', 'one', 'kid', 'let', 'alone', 'whole', 'bunch', 'performing', 'complex', 'dance', 'scene', 'bottom', 'line', 'movie', 'people', 'like', 'mj', 'one', 'level', 'another', 'think', 'people', 'stay', 'away', 'try', 'give', 'wholesome', 'message', 'ironically', 'mj', 'bestest', 'buddy', 'movie', 'girl', 'michael', 'jackson', 'truly', 'one', 'talented', 'people', 'ever', 'grace', 'planet', 'guilty', 'well', 'attention', 'gave', 'subject', 'hmmm', 'well', 'know', 'people', 'different', 'behind', 'closed', 'doors', 'know', 'fact', 'either', 'extremely', 'nice', 'stupid', 'guy', 'one', 'sickest', 'liars', 'hope', 'latter']\n"
     ]
    }
   ],
   "source": [
    "# Remove stop words from \"words\"\n",
    "words = [w for w in words if not w in stopwords.words(\"english\")]\n",
    "print(words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def review_to_words( raw_review ):\n",
    "    # Function to convert a raw review to a string of words\n",
    "    # The input is a single string (a raw movie review), and \n",
    "    # the output is a single string (a preprocessed movie review)\n",
    "    #\n",
    "    # 1. Remove HTML\n",
    "    review_text = BeautifulSoup(raw_review, 'html.parser').get_text() \n",
    "    #\n",
    "    # 2. Remove non-letters        \n",
    "    letters_only = re.sub(\"[^a-zA-Z]\", \" \", review_text) \n",
    "    #\n",
    "    # 3. Convert to lower case, split into individual words\n",
    "    words = letters_only.lower().split()                             \n",
    "    #\n",
    "    # 4. In Python, searching a set is much faster than searching\n",
    "    #   a list, so convert the stop words to a set\n",
    "    stops = set(stopwords.words(\"english\"))                  \n",
    "    # \n",
    "    # 5. Remove stop words\n",
    "    meaningful_words = [w for w in words if not w in stops]   \n",
    "    #\n",
    "    # 6. Join the words back into one string separated by space, \n",
    "    # and return the result.\n",
    "    return( \" \".join( meaningful_words ))   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "stuff going moment mj started listening music watching odd documentary watched wiz watched moonwalker maybe want get certain insight guy thought really cool eighties maybe make mind whether guilty innocent moonwalker part biography part feature film remember going see cinema originally released subtle messages mj feeling towards press also obvious message drugs bad kay visually impressive course michael jackson unless remotely like mj anyway going hate find boring may call mj egotist consenting making movie mj fans would say made fans true really nice actual feature film bit finally starts minutes excluding smooth criminal sequence joe pesci convincing psychopathic powerful drug lord wants mj dead bad beyond mj overheard plans nah joe pesci character ranted wanted people know supplying drugs etc dunno maybe hates mj music lots cool things like mj turning car robot whole speed demon sequence also director must patience saint came filming kiddy bad sequence usually directors hate working one kid let alone whole bunch performing complex dance scene bottom line movie people like mj one level another think people stay away try give wholesome message ironically mj bestest buddy movie girl michael jackson truly one talented people ever grace planet guilty well attention gave subject hmmm well know people different behind closed doors know fact either extremely nice stupid guy one sickest liars hope latter\n"
     ]
    }
   ],
   "source": [
    "clean_review = review_to_words( train[\"review\"][0] )\n",
    "print(clean_review)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cleaning and parsing the training set movie reviews...\n",
      "\n",
      "Review 1000 of 25000\n",
      "\n",
      "Review 2000 of 25000\n",
      "\n",
      "Review 3000 of 25000\n",
      "\n",
      "Review 4000 of 25000\n",
      "\n",
      "Review 5000 of 25000\n",
      "\n",
      "Review 6000 of 25000\n",
      "\n",
      "Review 7000 of 25000\n",
      "\n",
      "Review 8000 of 25000\n",
      "\n",
      "Review 9000 of 25000\n",
      "\n",
      "Review 10000 of 25000\n",
      "\n",
      "Review 11000 of 25000\n",
      "\n",
      "Review 12000 of 25000\n",
      "\n",
      "Review 13000 of 25000\n",
      "\n",
      "Review 14000 of 25000\n",
      "\n",
      "Review 15000 of 25000\n",
      "\n",
      "Review 16000 of 25000\n",
      "\n",
      "Review 17000 of 25000\n",
      "\n",
      "Review 18000 of 25000\n",
      "\n",
      "Review 19000 of 25000\n",
      "\n",
      "Review 20000 of 25000\n",
      "\n",
      "Review 21000 of 25000\n",
      "\n",
      "Review 22000 of 25000\n",
      "\n",
      "Review 23000 of 25000\n",
      "\n",
      "Review 24000 of 25000\n",
      "\n",
      "Review 25000 of 25000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Get the number of reviews based on the dataframe column size\n",
    "num_reviews = train[\"review\"].size\n",
    "\n",
    "print(\"Cleaning and parsing the training set movie reviews...\\n\")\n",
    "clean_train_reviews = []\n",
    "for i in range( 0, num_reviews ):\n",
    "    # If the index is evenly divisible by 1000, print a message\n",
    "    if( (i+1)%1000 == 0 ):\n",
    "        print(\"Review %d of %d\\n\" % ( i+1, num_reviews ))                                                               \n",
    "    clean_train_reviews.append( review_to_words( train[\"review\"][i] ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating the bag of words...\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Creating the bag of words...\\n\")\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "# Initialize the \"CountVectorizer\" object, which is scikit-learn's\n",
    "# bag of words tool.  \n",
    "vectorizer = CountVectorizer(analyzer = \"word\",   \\\n",
    "                             tokenizer = None,    \\\n",
    "                             preprocessor = None, \\\n",
    "                             stop_words = None,   \\\n",
    "                             max_features = 5000) \n",
    "\n",
    "# fit_transform() does two functions: First, it fits the model\n",
    "# and learns the vocabulary; second, it transforms our training data\n",
    "# into feature vectors. The input to fit_transform should be a list of \n",
    "# strings.\n",
    "train_data_features = vectorizer.fit_transform(clean_train_reviews)\n",
    "\n",
    "# Numpy arrays are easy to work with, so convert the result to an \n",
    "# array\n",
    "train_data_features = train_data_features.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(25000, 5000)\n"
     ]
    }
   ],
   "source": [
    "print(train_data_features.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['abandoned', 'abc', 'abilities', 'ability', 'able', 'abraham', 'absence', 'absent', 'absolute', 'absolutely', 'absurd', 'abuse', 'abusive', 'abysmal', 'academy', 'accent', 'accents', 'accept', 'acceptable', 'accepted', 'access', 'accident', 'accidentally', 'accompanied', 'accomplished', 'according', 'account', 'accuracy', 'accurate', 'accused', 'achieve', 'achieved', 'achievement', 'acid', 'across', 'act', 'acted', 'acting', 'action', 'actions', 'activities', 'actor', 'actors', 'actress', 'actresses', 'acts', 'actual', 'actually', 'ad', 'adam', 'adams', 'adaptation', 'adaptations', 'adapted', 'add', 'added', 'adding', 'addition', 'adds', 'adequate', 'admire', 'admit', 'admittedly', 'adorable', 'adult', 'adults', 'advance', 'advanced', 'advantage', 'adventure', 'adventures', 'advertising', 'advice', 'advise', 'affair', 'affect', 'affected', 'afford', 'aforementioned', 'afraid', 'africa', 'african', 'afternoon', 'afterwards', 'age', 'aged', 'agent', 'agents', 'ages', 'aging', 'ago', 'agree', 'agreed', 'agrees', 'ah', 'ahead', 'aid', 'aids', 'aim', 'aimed', 'air', 'aired', 'airplane', 'airport', 'aka', 'akshay', 'al', 'alan', 'alas', 'albeit', 'albert', 'album', 'alcohol', 'alcoholic', 'alec', 'alert', 'alex', 'alexander', 'alfred', 'alice', 'alicia', 'alien', 'aliens', 'alike', 'alison', 'alive', 'allen', 'allow', 'allowed', 'allowing', 'allows', 'almost', 'alone', 'along', 'alongside', 'already', 'alright', 'also', 'alternate', 'alternative', 'although', 'altman', 'altogether', 'always', 'amanda', 'amateur', 'amateurish', 'amazed', 'amazing', 'amazingly', 'ambiguous', 'ambitious', 'america', 'american', 'americans', 'amitabh', 'among', 'amongst', 'amount', 'amounts', 'amused', 'amusing', 'amy', 'analysis', 'ancient', 'anderson', 'andre', 'andrew', 'andrews', 'andy', 'angel', 'angela', 'angeles', 'angels', 'anger', 'angle', 'angles', 'angry', 'animal', 'animals', 'animated', 'animation', 'anime', 'ann', 'anna', 'anne', 'annie', 'annoyed', 'annoying', 'another', 'answer', 'answers', 'anthony', 'anti', 'antics', 'antonioni', 'antwone', 'anybody', 'anymore', 'anyone', 'anything', 'anyway', 'anyways', 'anywhere', 'apart', 'apartment', 'ape', 'apes', 'appalling', 'apparent', 'apparently', 'appeal', 'appealing', 'appear', 'appearance', 'appearances', 'appeared', 'appearing', 'appears', 'appreciate', 'appreciated', 'appreciation', 'approach', 'appropriate', 'april', 'area', 'areas', 'arguably', 'argue', 'argument', 'arm', 'armed', 'arms', 'army', 'arnold', 'around', 'arrested', 'arrival', 'arrive', 'arrived', 'arrives', 'arrogant', 'art', 'arthur', 'artificial', 'artist', 'artistic', 'artists', 'arts', 'ashamed', 'ashley', 'asian', 'aside', 'ask', 'asked', 'asking', 'asks', 'asleep', 'aspect', 'aspects', 'ass', 'assassin', 'assault', 'assigned', 'assistant', 'associated', 'assume', 'assumed', 'astaire', 'astonishing', 'atlantis', 'atmosphere', 'atmospheric', 'atrocious', 'attached', 'attack', 'attacked', 'attacks', 'attempt', 'attempted', 'attempting', 'attempts', 'attend', 'attention', 'attitude', 'attitudes', 'attorney', 'attracted', 'attraction', 'attractive', 'audience', 'audiences', 'audio', 'aunt', 'austen', 'austin', 'australia', 'australian', 'authentic', 'author', 'authority', 'available', 'average', 'avoid', 'avoided', 'awake', 'award', 'awards', 'aware', 'away', 'awe', 'awesome', 'awful', 'awfully', 'awhile', 'awkward', 'babe', 'baby', 'bacall', 'back', 'backdrop', 'background', 'backgrounds', 'bad', 'badly', 'bag', 'baker', 'bakshi', 'balance', 'baldwin', 'ball', 'ballet', 'balls', 'band', 'bands', 'bang', 'bank', 'banned', 'bar', 'barbara', 'bare', 'barely', 'bargain', 'barry', 'barrymore', 'base', 'baseball', 'based', 'basement', 'basic', 'basically', 'basis', 'basketball', 'bat', 'bath', 'bathroom', 'batman', 'battle', 'battles', 'bay', 'bbc', 'beach', 'bear', 'bears', 'beast', 'beat', 'beaten', 'beating', 'beats', 'beatty', 'beautiful', 'beautifully', 'beauty', 'became', 'become', 'becomes', 'becoming', 'bed', 'bedroom', 'beer', 'began', 'begin', 'beginning', 'begins', 'behave', 'behavior', 'behind', 'beings', 'bela', 'belief', 'beliefs', 'believable', 'believe', 'believed', 'believes', 'believing', 'bell', 'belong', 'belongs', 'beloved', 'belushi', 'ben', 'beneath', 'benefit', 'bergman', 'berlin', 'besides', 'best', 'bet', 'bette', 'better', 'bettie', 'betty', 'beyond', 'bible', 'big', 'bigger', 'biggest', 'biko', 'bill', 'billed', 'billy', 'bin', 'biography', 'bird', 'birds', 'birth', 'birthday', 'bit', 'bite', 'bits', 'bitter', 'bizarre', 'black', 'blade', 'blah', 'blair', 'blake', 'blame', 'bland', 'blank', 'blast', 'blatant', 'bleak', 'blend', 'blew', 'blind', 'blob', 'block', 'blockbuster', 'blond', 'blonde', 'blood', 'bloody', 'blow', 'blowing', 'blown', 'blows', 'blue', 'blues', 'blunt', 'bo', 'board', 'boat', 'bob', 'bobby', 'bodies', 'body', 'bold', 'boll', 'bollywood', 'bomb', 'bond', 'bone', 'bonus', 'book', 'books', 'boom', 'boot', 'border', 'bore', 'bored', 'boredom', 'boring', 'born', 'borrowed', 'boss', 'bother', 'bothered', 'bottle', 'bottom', 'bought', 'bound', 'bourne', 'box', 'boxing', 'boy', 'boyfriend', 'boyle', 'boys', 'brad', 'brady', 'brain', 'brains', 'branagh', 'brand', 'brando', 'brave', 'brazil', 'break', 'breaking', 'breaks', 'breasts', 'breath', 'breathtaking', 'brenda', 'brian', 'bride', 'bridge', 'brief', 'briefly', 'bright', 'brilliance', 'brilliant', 'brilliantly', 'bring', 'bringing', 'brings', 'britain', 'british', 'broad', 'broadcast', 'broadway', 'broke', 'broken', 'brooklyn', 'brooks', 'brosnan', 'brother', 'brothers', 'brought', 'brown', 'bruce', 'brutal', 'brutality', 'brutally', 'buck', 'bucks', 'bud', 'buddies', 'buddy', 'budget', 'buff', 'buffalo', 'buffs', 'bug', 'bugs', 'build', 'building', 'buildings', 'builds', 'built', 'bull', 'bullet', 'bullets', 'bumbling', 'bunch', 'bunny', 'buried', 'burn', 'burned', 'burning', 'burns', 'burt', 'burton', 'bus', 'bush', 'business', 'businessman', 'buster', 'busy', 'butler', 'butt', 'button', 'buy', 'buying', 'cabin', 'cable', 'cage', 'cagney', 'caine', 'cake', 'caliber', 'california', 'call', 'called', 'calling', 'calls', 'calm', 'came', 'cameo', 'cameos', 'camera', 'cameras', 'cameron', 'camp', 'campbell', 'campy', 'canada', 'canadian', 'candy', 'cannibal', 'cannot', 'cant', 'capable', 'capital', 'captain', 'captivating', 'capture', 'captured', 'captures', 'capturing', 'car', 'card', 'cardboard', 'cards', 'care', 'cared', 'career', 'careers', 'careful', 'carefully', 'carell', 'cares', 'caring', 'carl', 'carla', 'carol', 'carpenter', 'carradine', 'carrey', 'carrie', 'carried', 'carries', 'carry', 'carrying', 'cars', 'carter', 'cartoon', 'cartoons', 'cary', 'case', 'cases', 'cash', 'cassidy', 'cast', 'casting', 'castle', 'cat', 'catch', 'catches', 'catching', 'catchy', 'category', 'catherine', 'catholic', 'cats', 'caught', 'cause', 'caused', 'causes', 'causing', 'cave', 'cd', 'celebrity', 'cell', 'celluloid', 'center', 'centered', 'centers', 'central', 'century', 'certain', 'certainly', 'cg', 'cgi', 'chain', 'chair', 'challenge', 'challenging', 'championship', 'chan', 'chance', 'chances', 'change', 'changed', 'changes', 'changing', 'channel', 'channels', 'chaos', 'chaplin', 'chapter', 'character', 'characterization', 'characters', 'charge', 'charisma', 'charismatic', 'charles', 'charlie', 'charlotte', 'charm', 'charming', 'chase', 'chased', 'chases', 'chasing', 'che', 'cheap', 'cheated', 'cheating', 'check', 'checked', 'checking', 'cheek', 'cheese', 'cheesy', 'chemistry', 'chess', 'chest', 'chicago', 'chick', 'chicken', 'chicks', 'chief', 'child', 'childhood', 'childish', 'children', 'chilling', 'china', 'chinese', 'choice', 'choices', 'choose', 'chooses', 'choreographed', 'choreography', 'chorus', 'chose', 'chosen', 'chris', 'christ', 'christian', 'christianity', 'christians', 'christmas', 'christopher', 'christy', 'chuck', 'church', 'cia', 'cinderella', 'cinema', 'cinematic', 'cinematographer', 'cinematography', 'circle', 'circumstances', 'cities', 'citizen', 'city', 'civil', 'civilization', 'claim', 'claimed', 'claims', 'claire', 'clark', 'class', 'classes', 'classic', 'classical', 'classics', 'claus', 'clean', 'clear', 'clearly', 'clever', 'cleverly', 'clich', 'cliche', 'cliff', 'climactic', 'climax', 'clint', 'clip', 'clips', 'clock', 'close', 'closed', 'closely', 'closer', 'closest', 'closet', 'closing', 'clothes', 'clothing', 'clown', 'club', 'clue', 'clues', 'clumsy', 'co', 'coach', 'coast', 'code', 'coffee', 'coherent', 'cold', 'cole', 'collection', 'college', 'colonel', 'color', 'colorful', 'colors', 'colour', 'columbo', 'com', 'combat', 'combination', 'combine', 'combined', 'come', 'comedian', 'comedic', 'comedies', 'comedy', 'comes', 'comfort', 'comfortable', 'comic', 'comical', 'comics', 'coming', 'command', 'comment', 'commentary', 'commented', 'comments', 'commercial', 'commercials', 'commit', 'committed', 'common', 'communist', 'community', 'companies', 'companion', 'company', 'compare', 'compared', 'comparing', 'comparison', 'compassion', 'compelled', 'compelling', 'competent', 'competition', 'complain', 'complaint', 'complete', 'completely', 'complex', 'complexity', 'complicated', 'composed', 'composer', 'computer', 'con', 'conceived', 'concept', 'concern', 'concerned', 'concerning', 'concerns', 'concert', 'conclusion', 'condition', 'conditions', 'confess', 'confidence', 'conflict', 'conflicts', 'confrontation', 'confused', 'confusing', 'confusion', 'connect', 'connected', 'connection', 'connery', 'conscious', 'consequences', 'conservative', 'consider', 'considerable', 'considered', 'considering', 'consistent', 'consistently', 'consists', 'conspiracy', 'constant', 'constantly', 'constructed', 'construction', 'contact', 'contain', 'contained', 'contains', 'contemporary', 'content', 'contest', 'context', 'continue', 'continued', 'continues', 'continuity', 'contract', 'contrary', 'contrast', 'contrived', 'control', 'controversial', 'conventional', 'conversation', 'conversations', 'convey', 'convince', 'convinced', 'convincing', 'convincingly', 'convoluted', 'cook', 'cool', 'cooper', 'cop', 'copies', 'cops', 'copy', 'core', 'corner', 'corny', 'corporate', 'corpse', 'correct', 'correctly', 'corrupt', 'corruption', 'cost', 'costs', 'costume', 'costumes', 'could', 'count', 'counter', 'countless', 'countries', 'country', 'countryside', 'couple', 'couples', 'courage', 'course', 'court', 'cousin', 'cover', 'covered', 'covers', 'cowboy', 'cox', 'crack', 'cracking', 'craft', 'crafted', 'craig', 'crap', 'crappy', 'crash', 'craven', 'crawford', 'crazed', 'crazy', 'cream', 'create', 'created', 'creates', 'creating', 'creation', 'creative', 'creativity', 'creator', 'creators', 'creature', 'creatures', 'credibility', 'credible', 'credit', 'credits', 'creep', 'creepy', 'crew', 'cried', 'crime', 'crimes', 'criminal', 'criminals', 'cringe', 'crisis', 'critic', 'critical', 'criticism', 'critics', 'crocodile', 'cross', 'crowd', 'crucial', 'crude', 'cruel', 'cruise', 'crush', 'cry', 'crying', 'crystal', 'cuba', 'cube', 'cult', 'cultural', 'culture', 'cup', 'cure', 'curiosity', 'curious', 'current', 'currently', 'curse', 'curtis', 'cusack', 'cut', 'cute', 'cuts', 'cutting', 'cynical', 'da', 'dad', 'daddy', 'daily', 'dalton', 'damage', 'damme', 'damn', 'damon', 'dan', 'dana', 'dance', 'dancer', 'dancers', 'dances', 'dancing', 'danes', 'danger', 'dangerous', 'daniel', 'danny', 'dare', 'daring', 'dark', 'darker', 'darkness', 'darren', 'date', 'dated', 'dating', 'daughter', 'daughters', 'dave', 'david', 'davies', 'davis', 'dawn', 'dawson', 'day', 'days', 'de', 'dead', 'deadly', 'deaf', 'deal', 'dealing', 'deals', 'dealt', 'dean', 'dear', 'death', 'deaths', 'debut', 'decade', 'decades', 'deceased', 'decent', 'decide', 'decided', 'decides', 'decision', 'decisions', 'dedicated', 'dee', 'deep', 'deeper', 'deeply', 'defeat', 'defend', 'defense', 'defined', 'definite', 'definitely', 'definition', 'degree', 'del', 'deliberately', 'delight', 'delightful', 'deliver', 'delivered', 'delivering', 'delivers', 'delivery', 'demand', 'demands', 'demented', 'demise', 'demon', 'demons', 'deniro', 'dennis', 'dentist', 'denzel', 'department', 'depicted', 'depicting', 'depiction', 'depicts', 'depressed', 'depressing', 'depression', 'depth', 'der', 'derek', 'descent', 'describe', 'described', 'describes', 'description', 'desert', 'deserve', 'deserved', 'deserves', 'design', 'designed', 'designs', 'desire', 'desired', 'despair', 'desperate', 'desperately', 'desperation', 'despite', 'destiny', 'destroy', 'destroyed', 'destroying', 'destruction', 'detail', 'detailed', 'details', 'detective', 'determined', 'develop', 'developed', 'developing', 'development', 'develops', 'device', 'devil', 'devoid', 'devoted', 'dialog', 'dialogs', 'dialogue', 'dialogues', 'diamond', 'diana', 'diane', 'dick', 'dickens', 'die', 'died', 'dies', 'difference', 'differences', 'different', 'difficult', 'dig', 'digital', 'dignity', 'dimension', 'dimensional', 'din', 'dinner', 'dinosaur', 'dinosaurs', 'dire', 'direct', 'directed', 'directing', 'direction', 'directions', 'directly', 'director', 'directorial', 'directors', 'directs', 'dirty', 'disagree', 'disappear', 'disappeared', 'disappoint', 'disappointed', 'disappointing', 'disappointment', 'disaster', 'disbelief', 'disc', 'discover', 'discovered', 'discovers', 'discovery', 'discuss', 'discussion', 'disease', 'disgusting', 'disjointed', 'dislike', 'disliked', 'disney', 'display', 'displayed', 'displays', 'distance', 'distant', 'distinct', 'distracting', 'distribution', 'disturbed', 'disturbing', 'divorce', 'dixon', 'doc', 'doctor', 'documentaries', 'documentary', 'dog', 'dogs', 'doll', 'dollar', 'dollars', 'dolls', 'dolph', 'domestic', 'domino', 'donald', 'done', 'donna', 'doo', 'doom', 'doomed', 'door', 'doors', 'dorothy', 'double', 'doubt', 'doubts', 'douglas', 'downey', 'downhill', 'downright', 'dozen', 'dozens', 'dr', 'dracula', 'drag', 'dragged', 'dragon', 'drags', 'drake', 'drama', 'dramas', 'dramatic', 'draw', 'drawing', 'drawn', 'draws', 'dreadful', 'dream', 'dreams', 'dreary', 'dreck', 'dress', 'dressed', 'dressing', 'drew', 'drink', 'drinking', 'drive', 'drivel', 'driven', 'driver', 'drives', 'driving', 'drop', 'dropped', 'dropping', 'drops', 'drug', 'drugs', 'drunk', 'drunken', 'dry', 'dub', 'dubbed', 'dubbing', 'dud', 'dude', 'due', 'duke', 'dull', 'dumb', 'duo', 'dust', 'dutch', 'duty', 'dvd', 'dying', 'dynamic', 'eager', 'ear', 'earl', 'earlier', 'early', 'earned', 'ears', 'earth', 'ease', 'easier', 'easily', 'east', 'eastern', 'eastwood', 'easy', 'eat', 'eaten', 'eating', 'eccentric', 'ed', 'eddie', 'edgar', 'edge', 'edgy', 'edie', 'edited', 'editing', 'edition', 'editor', 'education', 'educational', 'edward', 'eerie', 'effect', 'effective', 'effectively', 'effects', 'effort', 'efforts', 'ego', 'eight', 'eighties', 'either', 'elaborate', 'elderly', 'elegant', 'element', 'elements', 'elephant', 'elizabeth', 'ellen', 'elm', 'else', 'elsewhere', 'elvira', 'elvis', 'em', 'embarrassed', 'embarrassing', 'embarrassment', 'emily', 'emma', 'emotion', 'emotional', 'emotionally', 'emotions', 'empathy', 'emperor', 'emphasis', 'empire', 'empty', 'en', 'encounter', 'encounters', 'end', 'endearing', 'ended', 'ending', 'endings', 'endless', 'ends', 'endure', 'enemies', 'enemy', 'energy', 'engage', 'engaged', 'engaging', 'england', 'english', 'enjoy', 'enjoyable', 'enjoyed', 'enjoying', 'enjoyment', 'enjoys', 'enormous', 'enough', 'ensemble', 'ensues', 'enter', 'enterprise', 'enters', 'entertain', 'entertained', 'entertaining', 'entertainment', 'enthusiasm', 'entire', 'entirely', 'entry', 'environment', 'epic', 'episode', 'episodes', 'equal', 'equally', 'equipment', 'equivalent', 'er', 'era', 'eric', 'erotic', 'errors', 'escape', 'escaped', 'escapes', 'especially', 'essence', 'essential', 'essentially', 'established', 'estate', 'et', 'etc', 'ethan', 'eugene', 'europe', 'european', 'eva', 'eve', 'even', 'evening', 'event', 'events', 'eventually', 'ever', 'every', 'everybody', 'everyday', 'everyone', 'everything', 'everywhere', 'evidence', 'evident', 'evil', 'ex', 'exact', 'exactly', 'exaggerated', 'examination', 'example', 'examples', 'excellent', 'except', 'exception', 'exceptional', 'exceptionally', 'excessive', 'exchange', 'excited', 'excitement', 'exciting', 'excuse', 'executed', 'execution', 'executive', 'exercise', 'exist', 'existed', 'existence', 'existent', 'exists', 'exotic', 'expect', 'expectations', 'expected', 'expecting', 'expedition', 'expensive', 'experience', 'experienced', 'experiences', 'experiment', 'experimental', 'experiments', 'expert', 'explain', 'explained', 'explaining', 'explains', 'explanation', 'explicit', 'exploitation', 'exploration', 'explore', 'explored', 'explosion', 'explosions', 'exposed', 'exposure', 'express', 'expressed', 'expression', 'expressions', 'extended', 'extent', 'extra', 'extraordinary', 'extras', 'extreme', 'extremely', 'eye', 'eyed', 'eyes', 'eyre', 'fabulous', 'face', 'faced', 'faces', 'facial', 'facing', 'fact', 'factor', 'factory', 'facts', 'fail', 'failed', 'failing', 'fails', 'failure', 'fair', 'fairly', 'fairy', 'faith', 'faithful', 'fake', 'falk', 'fall', 'fallen', 'falling', 'falls', 'false', 'fame', 'familiar', 'families', 'family', 'famous', 'fan', 'fancy', 'fans', 'fantastic', 'fantasy', 'far', 'farce', 'fare', 'farm', 'farrell', 'fascinated', 'fascinating', 'fashion', 'fashioned', 'fast', 'faster', 'fat', 'fatal', 'fate', 'father', 'fault', 'faults', 'favor', 'favorite', 'favorites', 'favourite', 'fay', 'fbi', 'fear', 'fears', 'feature', 'featured', 'features', 'featuring', 'fed', 'feed', 'feel', 'feeling', 'feelings', 'feels', 'feet', 'felix', 'fell', 'fellow', 'felt', 'female', 'feminist', 'femme', 'fest', 'festival', 'fetched', 'fever', 'fi', 'fianc', 'fiction', 'fictional', 'fido', 'field', 'fields', 'fifteen', 'fight', 'fighter', 'fighting', 'fights', 'figure', 'figured', 'figures', 'files', 'fill', 'filled', 'film', 'filmed', 'filming', 'filmmaker', 'filmmakers', 'films', 'final', 'finale', 'finally', 'financial', 'find', 'finding', 'finds', 'fine', 'finest', 'finger', 'finish', 'finished', 'fire', 'fired', 'firm', 'first', 'firstly', 'fish', 'fisher', 'fishing', 'fit', 'fits', 'fitting', 'five', 'fix', 'flair', 'flash', 'flashback', 'flashbacks', 'flat', 'flaw', 'flawed', 'flawless', 'flaws', 'flesh', 'flick', 'flicks', 'flies', 'flight', 'floating', 'floor', 'flop', 'florida', 'flow', 'fly', 'flying', 'flynn', 'focus', 'focused', 'focuses', 'focusing', 'folk', 'folks', 'follow', 'followed', 'following', 'follows', 'fond', 'fonda', 'food', 'fool', 'fooled', 'foot', 'footage', 'football', 'forbidden', 'force', 'forced', 'forces', 'ford', 'foreign', 'forest', 'forever', 'forget', 'forgettable', 'forgive', 'forgot', 'forgotten', 'form', 'format', 'former', 'forms', 'formula', 'formulaic', 'forth', 'fortunately', 'fortune', 'forty', 'forward', 'foster', 'fought', 'foul', 'found', 'four', 'fourth', 'fox', 'frame', 'france', 'franchise', 'francis', 'francisco', 'franco', 'frank', 'frankenstein', 'frankie', 'frankly', 'freak', 'fred', 'freddy', 'free', 'freedom', 'freeman', 'french', 'frequent', 'frequently', 'fresh', 'friday', 'friend', 'friendly', 'friends', 'friendship', 'frightening', 'front', 'frustrated', 'frustrating', 'frustration', 'fu', 'fulci', 'full', 'fuller', 'fully', 'fun', 'funeral', 'funnier', 'funniest', 'funny', 'furious', 'furthermore', 'fury', 'future', 'futuristic', 'fx', 'gabriel', 'gadget', 'gag', 'gags', 'gain', 'game', 'games', 'gandhi', 'gang', 'gangster', 'gangsters', 'garbage', 'garbo', 'garden', 'gary', 'gas', 'gave', 'gay', 'gem', 'gender', 'gene', 'general', 'generally', 'generated', 'generation', 'generations', 'generic', 'generous', 'genius', 'genre', 'genres', 'gentle', 'gentleman', 'genuine', 'genuinely', 'george', 'gerard', 'german', 'germans', 'germany', 'get', 'gets', 'getting', 'ghost', 'ghosts', 'giallo', 'giant', 'gift', 'gifted', 'ginger', 'girl', 'girlfriend', 'girls', 'give', 'given', 'gives', 'giving', 'glad', 'glass', 'glasses', 'glenn', 'glimpse', 'global', 'glorious', 'glory', 'glover', 'go', 'goal', 'god', 'godfather', 'godzilla', 'goes', 'going', 'gold', 'goldberg', 'golden', 'gone', 'gonna', 'good', 'goodness', 'goofy', 'gordon', 'gore', 'gorgeous', 'gory', 'got', 'gothic', 'gotta', 'gotten', 'government', 'grab', 'grabs', 'grace', 'grade', 'gradually', 'graham', 'grand', 'grandfather', 'grandmother', 'grant', 'granted', 'graphic', 'graphics', 'grasp', 'gratuitous', 'grave', 'gray', 'grayson', 'great', 'greater', 'greatest', 'greatly', 'greatness', 'greed', 'greedy', 'greek', 'green', 'greg', 'gregory', 'grew', 'grey', 'grief', 'griffith', 'grim', 'grinch', 'gripping', 'gritty', 'gross', 'ground', 'group', 'groups', 'grow', 'growing', 'grown', 'grows', 'gruesome', 'guarantee', 'guard', 'guess', 'guessed', 'guessing', 'guest', 'guide', 'guilt', 'guilty', 'gun', 'gundam', 'guns', 'guts', 'guy', 'guys', 'ha', 'hair', 'hal', 'half', 'halfway', 'hall', 'halloween', 'ham', 'hamilton', 'hamlet', 'hammer', 'hand', 'handed', 'handful', 'handle', 'handled', 'hands', 'handsome', 'hang', 'hanging', 'hank', 'hanks', 'happen', 'happened', 'happening', 'happens', 'happily', 'happiness', 'happy', 'hard', 'hardcore', 'harder', 'hardly', 'hardy', 'harm', 'harris', 'harry', 'harsh', 'hart', 'hartley', 'harvey', 'hat', 'hate', 'hated', 'hates', 'hatred', 'haunted', 'haunting', 'hawke', 'hbo', 'head', 'headed', 'heads', 'health', 'hear', 'heard', 'hearing', 'heart', 'hearted', 'hearts', 'heat', 'heaven', 'heavily', 'heavy', 'heck', 'heights', 'held', 'helen', 'helicopter', 'hell', 'hello', 'help', 'helped', 'helping', 'helps', 'hence', 'henry', 'hero', 'heroes', 'heroic', 'heroine', 'heston', 'hey', 'hidden', 'hide', 'hideous', 'hiding', 'high', 'higher', 'highest', 'highlight', 'highlights', 'highly', 'hilarious', 'hilariously', 'hill', 'hills', 'hint', 'hints', 'hip', 'hippie', 'hire', 'hired', 'historical', 'historically', 'history', 'hit', 'hitchcock', 'hitler', 'hits', 'hitting', 'ho', 'hoffman', 'hold', 'holding', 'holds', 'hole', 'holes', 'holiday', 'hollow', 'holly', 'hollywood', 'holmes', 'holy', 'homage', 'home', 'homeless', 'homer', 'homosexual', 'honest', 'honestly', 'honesty', 'hong', 'honor', 'hood', 'hook', 'hooked', 'hop', 'hope', 'hoped', 'hopefully', 'hopeless', 'hopes', 'hoping', 'hopper', 'horrendous', 'horrible', 'horribly', 'horrid', 'horrific', 'horrifying', 'horror', 'horrors', 'horse', 'horses', 'hospital', 'host', 'hot', 'hotel', 'hour', 'hours', 'house', 'household', 'houses', 'howard', 'however', 'hudson', 'huge', 'hugh', 'huh', 'human', 'humanity', 'humans', 'humble', 'humor', 'humorous', 'humour', 'hundred', 'hundreds', 'hung', 'hunt', 'hunter', 'hunters', 'hunting', 'hurt', 'hurts', 'husband', 'husbands', 'hyde', 'hype', 'hysterical', 'ian', 'ice', 'icon', 'idea', 'ideal', 'ideas', 'identify', 'identity', 'idiot', 'idiotic', 'idiots', 'ignorant', 'ignore', 'ignored', 'ii', 'iii', 'ill', 'illegal', 'illness', 'illogical', 'im', 'image', 'imagery', 'images', 'imagination', 'imaginative', 'imagine', 'imagined', 'imdb', 'imitation', 'immediate', 'immediately', 'immensely', 'impact', 'implausible', 'importance', 'important', 'importantly', 'impossible', 'impress', 'impressed', 'impression', 'impressive', 'improve', 'improved', 'improvement', 'inability', 'inane', 'inappropriate', 'incident', 'include', 'included', 'includes', 'including', 'incoherent', 'incompetent', 'incomprehensible', 'increasingly', 'incredible', 'incredibly', 'indeed', 'independent', 'india', 'indian', 'indians', 'indie', 'individual', 'individuals', 'inducing', 'indulgent', 'industry', 'inept', 'inevitable', 'inevitably', 'infamous', 'inferior', 'influence', 'influenced', 'information', 'ingredients', 'initial', 'initially', 'inner', 'innocence', 'innocent', 'innovative', 'insane', 'inside', 'insight', 'inspector', 'inspiration', 'inspired', 'inspiring', 'installment', 'instance', 'instant', 'instantly', 'instead', 'instinct', 'insult', 'insulting', 'integrity', 'intellectual', 'intelligence', 'intelligent', 'intended', 'intense', 'intensity', 'intent', 'intention', 'intentionally', 'intentions', 'interaction', 'interest', 'interested', 'interesting', 'interests', 'international', 'internet', 'interpretation', 'interview', 'interviews', 'intimate', 'intrigue', 'intrigued', 'intriguing', 'introduce', 'introduced', 'introduces', 'introduction', 'invasion', 'invented', 'inventive', 'investigate', 'investigation', 'invisible', 'involve', 'involved', 'involvement', 'involves', 'involving', 'iran', 'iraq', 'ireland', 'irish', 'iron', 'ironic', 'ironically', 'irony', 'irrelevant', 'irritating', 'island', 'isolated', 'israel', 'issue', 'issues', 'italian', 'italy', 'jack', 'jackie', 'jackson', 'jail', 'jake', 'james', 'jamie', 'jane', 'japan', 'japanese', 'jason', 'jaw', 'jaws', 'jay', 'jazz', 'jealous', 'jean', 'jeff', 'jeffrey', 'jennifer', 'jenny', 'jeremy', 'jerk', 'jerry', 'jesse', 'jessica', 'jesus', 'jet', 'jewish', 'jim', 'jimmy', 'joan', 'job', 'jobs', 'joe', 'joel', 'joey', 'john', 'johnny', 'johnson', 'join', 'joined', 'joke', 'jokes', 'jon', 'jonathan', 'jones', 'joseph', 'josh', 'journalist', 'journey', 'joy', 'jr', 'judge', 'judging', 'judy', 'julia', 'julian', 'julie', 'jump', 'jumped', 'jumping', 'jumps', 'june', 'jungle', 'junior', 'junk', 'justice', 'justify', 'justin', 'juvenile', 'kane', 'kansas', 'kapoor', 'karen', 'karloff', 'kate', 'kay', 'keaton', 'keep', 'keeping', 'keeps', 'keith', 'kelly', 'ken', 'kennedy', 'kenneth', 'kept', 'kevin', 'key', 'khan', 'kick', 'kicked', 'kicking', 'kicks', 'kid', 'kidding', 'kidnapped', 'kids', 'kill', 'killed', 'killer', 'killers', 'killing', 'killings', 'kills', 'kim', 'kind', 'kinda', 'kinds', 'king', 'kingdom', 'kirk', 'kiss', 'kissing', 'kitchen', 'knew', 'knife', 'knock', 'know', 'knowing', 'knowledge', 'known', 'knows', 'kolchak', 'kong', 'korean', 'kubrick', 'kudos', 'kumar', 'kung', 'kurosawa', 'kurt', 'kyle', 'la', 'lab', 'lack', 'lacked', 'lacking', 'lackluster', 'lacks', 'ladies', 'lady', 'laid', 'lake', 'lame', 'land', 'landing', 'landscape', 'landscapes', 'lane', 'language', 'large', 'largely', 'larger', 'larry', 'last', 'lasted', 'late', 'lately', 'later', 'latest', 'latin', 'latter', 'laugh', 'laughable', 'laughably', 'laughed', 'laughing', 'laughs', 'laughter', 'laura', 'laurel', 'law', 'lawrence', 'laws', 'lawyer', 'lay', 'lazy', 'le', 'lead', 'leader', 'leading', 'leads', 'league', 'learn', 'learned', 'learning', 'learns', 'least', 'leave', 'leaves', 'leaving', 'led', 'lee', 'left', 'leg', 'legacy', 'legal', 'legend', 'legendary', 'legs', 'lemmon', 'lena', 'length', 'lengthy', 'leo', 'leon', 'leonard', 'les', 'lesbian', 'leslie', 'less', 'lesser', 'lesson', 'lessons', 'let', 'lets', 'letter', 'letters', 'letting', 'level', 'levels', 'lewis', 'li', 'liberal', 'library', 'lie', 'lies', 'life', 'lifestyle', 'lifetime', 'light', 'lighting', 'lights', 'likable', 'like', 'liked', 'likely', 'likes', 'likewise', 'liking', 'lily', 'limited', 'limits', 'lincoln', 'linda', 'line', 'liners', 'lines', 'link', 'lion', 'lips', 'lisa', 'list', 'listed', 'listen', 'listening', 'lit', 'literally', 'literature', 'little', 'live', 'lived', 'lively', 'lives', 'living', 'lloyd', 'load', 'loaded', 'loads', 'local', 'location', 'locations', 'locked', 'logan', 'logic', 'logical', 'lol', 'london', 'lone', 'lonely', 'long', 'longer', 'look', 'looked', 'looking', 'looks', 'loose', 'loosely', 'lord', 'los', 'lose', 'loser', 'losers', 'loses', 'losing', 'loss', 'lost', 'lot', 'lots', 'lou', 'loud', 'louis', 'louise', 'lousy', 'lovable', 'love', 'loved', 'lovely', 'lover', 'lovers', 'loves', 'loving', 'low', 'lower', 'lowest', 'loyal', 'loyalty', 'lucas', 'luck', 'luckily', 'lucky', 'lucy', 'ludicrous', 'lugosi', 'luke', 'lumet', 'lundgren', 'lust', 'lying', 'lynch', 'lyrics', 'macarthur', 'machine', 'machines', 'macy', 'mad', 'made', 'madness', 'madonna', 'mafia', 'magazine', 'maggie', 'magic', 'magical', 'magnificent', 'maid', 'mail', 'main', 'mainly', 'mainstream', 'maintain', 'major', 'majority', 'make', 'maker', 'makers', 'makes', 'makeup', 'making', 'male', 'mall', 'malone', 'man', 'manage', 'managed', 'manager', 'manages', 'manhattan', 'maniac', 'manipulative', 'mankind', 'mann', 'manner', 'mansion', 'many', 'map', 'marc', 'march', 'margaret', 'maria', 'marie', 'mario', 'marion', 'mark', 'market', 'marketing', 'marks', 'marriage', 'married', 'marry', 'mars', 'marshall', 'martial', 'martin', 'marty', 'marvelous', 'mary', 'mask', 'masks', 'mass', 'massacre', 'masses', 'massive', 'master', 'masterful', 'masterpiece', 'masterpieces', 'masters', 'match', 'matched', 'matches', 'mate', 'material', 'matrix', 'matt', 'matter', 'matters', 'matthau', 'matthew', 'mature', 'max', 'may', 'maybe', 'mayor', 'mclaglen', 'mean', 'meaning', 'meaningful', 'meaningless', 'means', 'meant', 'meanwhile', 'measure', 'meat', 'mechanical', 'media', 'medical', 'mediocre', 'medium', 'meet', 'meeting', 'meets', 'mel', 'melodrama', 'melodramatic', 'melting', 'member', 'members', 'memorable', 'memories', 'memory', 'men', 'menace', 'menacing', 'mental', 'mentally', 'mention', 'mentioned', 'mentioning', 'mentions', 'mere', 'merely', 'merit', 'merits', 'meryl', 'mess', 'message', 'messages', 'messed', 'met', 'metal', 'metaphor', 'method', 'methods', 'mexican', 'mexico', 'mgm', 'michael', 'michelle', 'mickey', 'mid', 'middle', 'midnight', 'might', 'mighty', 'miike', 'mike', 'mild', 'mildly', 'mildred', 'mile', 'miles', 'military', 'milk', 'mill', 'miller', 'million', 'millionaire', 'millions', 'min', 'mind', 'minded', 'mindless', 'minds', 'mine', 'mini', 'minimal', 'minimum', 'minor', 'minute', 'minutes', 'miracle', 'mirror', 'miscast', 'miserable', 'miserably', 'misery', 'miss', 'missed', 'misses', 'missing', 'mission', 'mistake', 'mistaken', 'mistakes', 'mistress', 'mitchell', 'mix', 'mixed', 'mixture', 'miyazaki', 'mm', 'mob', 'model', 'models', 'modern', 'modesty', 'molly', 'mom', 'moment', 'moments', 'mon', 'money', 'monk', 'monkey', 'monkeys', 'monster', 'monsters', 'montage', 'montana', 'month', 'months', 'mood', 'moody', 'moon', 'moore', 'moral', 'morality', 'morgan', 'morning', 'moronic', 'morris', 'mostly', 'mother', 'motion', 'motivation', 'motivations', 'motives', 'mountain', 'mountains', 'mouse', 'mouth', 'move', 'moved', 'movement', 'movements', 'moves', 'movie', 'movies', 'moving', 'mr', 'mrs', 'ms', 'mst', 'mtv', 'much', 'multi', 'multiple', 'mummy', 'mundane', 'murder', 'murdered', 'murderer', 'murderous', 'murders', 'murphy', 'murray', 'museum', 'music', 'musical', 'musicals', 'muslim', 'must', 'myers', 'mysteries', 'mysterious', 'mystery', 'nail', 'naive', 'naked', 'name', 'named', 'namely', 'names', 'nancy', 'narration', 'narrative', 'narrator', 'nasty', 'nathan', 'nation', 'national', 'native', 'natural', 'naturally', 'nature', 'navy', 'nazi', 'nazis', 'nd', 'near', 'nearby', 'nearly', 'neat', 'necessarily', 'necessary', 'neck', 'ned', 'need', 'needed', 'needless', 'needs', 'negative', 'neighbor', 'neighborhood', 'neighbors', 'neil', 'neither', 'nelson', 'neo', 'nephew', 'nerd', 'nervous', 'network', 'never', 'nevertheless', 'new', 'newly', 'newman', 'news', 'newspaper', 'next', 'nice', 'nicely', 'nicholas', 'nicholson', 'nick', 'nicole', 'night', 'nightmare', 'nightmares', 'nights', 'nine', 'ninja', 'niro', 'noble', 'nobody', 'noir', 'noise', 'nominated', 'nomination', 'non', 'none', 'nonetheless', 'nonsense', 'nonsensical', 'normal', 'normally', 'norman', 'north', 'nose', 'nostalgia', 'nostalgic', 'notable', 'notably', 'notch', 'note', 'noted', 'notes', 'nothing', 'notice', 'noticed', 'notion', 'notorious', 'novak', 'novel', 'novels', 'nowadays', 'nowhere', 'nuclear', 'nude', 'nudity', 'number', 'numbers', 'numerous', 'nurse', 'nuts', 'nyc', 'object', 'objective', 'obnoxious', 'obscure', 'obsessed', 'obsession', 'obvious', 'obviously', 'occasion', 'occasional', 'occasionally', 'occur', 'occurred', 'occurs', 'ocean', 'odd', 'oddly', 'odds', 'offended', 'offensive', 'offer', 'offered', 'offering', 'offers', 'office', 'officer', 'officers', 'official', 'often', 'oh', 'oil', 'ok', 'okay', 'old', 'older', 'oliver', 'olivier', 'ollie', 'omen', 'one', 'ones', 'online', 'onto', 'open', 'opened', 'opening', 'opens', 'opera', 'operation', 'opinion', 'opinions', 'opportunities', 'opportunity', 'opposed', 'opposite', 'orange', 'order', 'ordered', 'orders', 'ordinary', 'original', 'originality', 'originally', 'orleans', 'orson', 'oscar', 'oscars', 'othello', 'others', 'otherwise', 'ought', 'outcome', 'outer', 'outfit', 'outrageous', 'outside', 'outstanding', 'overacting', 'overall', 'overcome', 'overdone', 'overlook', 'overlooked', 'overly', 'overrated', 'overwhelming', 'owen', 'owner', 'oz', 'pace', 'paced', 'pacing', 'pacino', 'pack', 'package', 'packed', 'page', 'paid', 'pain', 'painful', 'painfully', 'paint', 'painted', 'painting', 'pair', 'pal', 'palace', 'palance', 'palma', 'paltrow', 'pamela', 'pan', 'panic', 'pants', 'paper', 'par', 'parallel', 'paranoia', 'parent', 'parents', 'paris', 'park', 'parker', 'parody', 'part', 'particular', 'particularly', 'parties', 'partly', 'partner', 'parts', 'party', 'pass', 'passable', 'passed', 'passes', 'passing', 'passion', 'passionate', 'past', 'pat', 'path', 'pathetic', 'patience', 'patient', 'patients', 'patrick', 'paul', 'paulie', 'pay', 'paying', 'pays', 'peace', 'peak', 'pearl', 'people', 'peoples', 'per', 'perfect', 'perfection', 'perfectly', 'perform', 'performance', 'performances', 'performed', 'performer', 'performers', 'performing', 'performs', 'perhaps', 'period', 'perry', 'person', 'persona', 'personal', 'personalities', 'personality', 'personally', 'persons', 'perspective', 'pet', 'pete', 'peter', 'peters', 'petty', 'pg', 'phantom', 'phil', 'philip', 'philosophical', 'philosophy', 'phone', 'phony', 'photo', 'photographed', 'photographer', 'photography', 'photos', 'physical', 'physically', 'piano', 'pick', 'picked', 'picking', 'picks', 'picture', 'pictures', 'pie', 'piece', 'pieces', 'pierce', 'pig', 'pile', 'pilot', 'pin', 'pink', 'pit', 'pitch', 'pitt', 'pity', 'place', 'placed', 'places', 'plague', 'plain', 'plan', 'plane', 'planet', 'planned', 'planning', 'plans', 'plant', 'plastic', 'plausible', 'play', 'played', 'player', 'players', 'playing', 'plays', 'pleasant', 'pleasantly', 'please', 'pleased', 'pleasure', 'plenty', 'plight', 'plot', 'plots', 'plus', 'poem', 'poetic', 'poetry', 'poignant', 'point', 'pointed', 'pointless', 'points', 'pokemon', 'polanski', 'police', 'polished', 'political', 'politically', 'politics', 'pool', 'poor', 'poorly', 'pop', 'popcorn', 'pops', 'popular', 'popularity', 'population', 'porn', 'porno', 'portion', 'portrait', 'portray', 'portrayal', 'portrayed', 'portraying', 'portrays', 'position', 'positive', 'possessed', 'possibilities', 'possibility', 'possible', 'possibly', 'post', 'poster', 'pot', 'potential', 'potentially', 'poverty', 'powell', 'power', 'powerful', 'powers', 'practically', 'practice', 'praise', 'pre', 'precious', 'predictable', 'prefer', 'pregnant', 'premiere', 'premise', 'prepared', 'prequel', 'presence', 'present', 'presentation', 'presented', 'presents', 'president', 'press', 'pressure', 'preston', 'presumably', 'pretend', 'pretending', 'pretentious', 'pretty', 'prevent', 'preview', 'previous', 'previously', 'prey', 'price', 'priceless', 'pride', 'priest', 'primarily', 'primary', 'prime', 'prince', 'princess', 'principal', 'print', 'prior', 'prison', 'prisoner', 'prisoners', 'private', 'prize', 'pro', 'probably', 'problem', 'problems', 'proceedings', 'proceeds', 'process', 'produce', 'produced', 'producer', 'producers', 'producing', 'product', 'production', 'productions', 'professional', 'professor', 'profound', 'program', 'progress', 'progresses', 'project', 'projects', 'prom', 'promise', 'promised', 'promises', 'promising', 'proof', 'propaganda', 'proper', 'properly', 'property', 'props', 'prostitute', 'protagonist', 'protagonists', 'protect', 'proud', 'prove', 'proved', 'proves', 'provide', 'provided', 'provides', 'providing', 'provoking', 'pseudo', 'psychiatrist', 'psychic', 'psycho', 'psychological', 'psychotic', 'public', 'pull', 'pulled', 'pulling', 'pulls', 'pulp', 'punch', 'punishment', 'punk', 'puppet', 'purchase', 'purchased', 'pure', 'purely', 'purple', 'purpose', 'purposes', 'pursuit', 'push', 'pushed', 'pushing', 'put', 'puts', 'putting', 'qualities', 'quality', 'queen', 'quest', 'question', 'questionable', 'questions', 'quick', 'quickly', 'quiet', 'quinn', 'quirky', 'quit', 'quite', 'quote', 'quotes', 'rabbit', 'race', 'rachel', 'racial', 'racism', 'racist', 'radio', 'rage', 'rain', 'raines', 'raise', 'raised', 'raising', 'ralph', 'rambo', 'ramones', 'ran', 'random', 'randomly', 'randy', 'range', 'rangers', 'rank', 'ranks', 'rap', 'rape', 'raped', 'rare', 'rarely', 'rat', 'rate', 'rated', 'rather', 'rating', 'ratings', 'rats', 'rave', 'raw', 'ray', 'raymond', 'rd', 'reach', 'reached', 'reaches', 'reaching', 'react', 'reaction', 'reactions', 'read', 'reader', 'reading', 'reads', 'ready', 'real', 'realise', 'realism', 'realistic', 'reality', 'realize', 'realized', 'realizes', 'realizing', 'really', 'reason', 'reasonable', 'reasonably', 'reasons', 'rebel', 'recall', 'receive', 'received', 'receives', 'recent', 'recently', 'recognition', 'recognize', 'recognized', 'recommend', 'recommended', 'record', 'recorded', 'recording', 'red', 'redeeming', 'redemption', 'reduced', 'reed', 'reel', 'reference', 'references', 'reflect', 'reflection', 'refreshing', 'refused', 'refuses', 'regard', 'regarding', 'regardless', 'regret', 'regular', 'reid', 'relate', 'related', 'relation', 'relations', 'relationship', 'relationships', 'relative', 'relatively', 'relatives', 'release', 'released', 'relevant', 'relief', 'relies', 'religion', 'religious', 'remain', 'remained', 'remaining', 'remains', 'remake', 'remarkable', 'remarkably', 'remarks', 'remember', 'remembered', 'remind', 'reminded', 'reminds', 'reminiscent', 'remote', 'remotely', 'removed', 'rendition', 'rent', 'rental', 'rented', 'renting', 'repeat', 'repeated', 'repeatedly', 'repetitive', 'replaced', 'report', 'reporter', 'represent', 'represented', 'represents', 'reputation', 'required', 'requires', 'rescue', 'research', 'resemblance', 'resemble', 'resembles', 'resident', 'resist', 'resolution', 'resort', 'resources', 'respect', 'respected', 'respectively', 'response', 'responsibility', 'responsible', 'rest', 'restaurant', 'restored', 'result', 'resulting', 'results', 'retarded', 'retired', 'return', 'returned', 'returning', 'returns', 'reunion', 'reveal', 'revealed', 'revealing', 'reveals', 'revelation', 'revenge', 'review', 'reviewer', 'reviewers', 'reviews', 'revolution', 'revolutionary', 'revolves', 'rex', 'reynolds', 'rich', 'richard', 'richards', 'richardson', 'rick', 'rid', 'ridden', 'ride', 'ridiculous', 'ridiculously', 'riding', 'right', 'rights', 'ring', 'rings', 'rip', 'ripped', 'rise', 'rises', 'rising', 'risk', 'rita', 'ritter', 'rival', 'river', 'riveting', 'road', 'rob', 'robbery', 'robbins', 'robert', 'roberts', 'robin', 'robinson', 'robot', 'robots', 'rochester', 'rock', 'rocket', 'rocks', 'rocky', 'roger', 'rogers', 'role', 'roles', 'roll', 'rolled', 'rolling', 'roman', 'romance', 'romantic', 'romero', 'romp', 'ron', 'room', 'rooms', 'rooney', 'root', 'roots', 'rose', 'ross', 'roth', 'rotten', 'rough', 'round', 'routine', 'row', 'roy', 'royal', 'rubber', 'rubbish', 'ruby', 'ruin', 'ruined', 'ruins', 'rukh', 'rule', 'rules', 'run', 'running', 'runs', 'rural', 'rush', 'rushed', 'russell', 'russia', 'russian', 'ruth', 'ruthless', 'ryan', 'sabrina', 'sacrifice', 'sad', 'sadistic', 'sadly', 'sadness', 'safe', 'safety', 'saga', 'said', 'sake', 'sally', 'sam', 'samurai', 'san', 'sandler', 'sandra', 'santa', 'sappy', 'sarah', 'sat', 'satan', 'satire', 'satisfied', 'satisfy', 'satisfying', 'saturday', 'savage', 'save', 'saved', 'saves', 'saving', 'saw', 'say', 'saying', 'says', 'scale', 'scare', 'scarecrow', 'scared', 'scares', 'scary', 'scenario', 'scene', 'scenery', 'scenes', 'scheme', 'school', 'sci', 'science', 'scientific', 'scientist', 'scientists', 'scooby', 'scope', 'score', 'scores', 'scotland', 'scott', 'scottish', 'scream', 'screaming', 'screams', 'screen', 'screening', 'screenplay', 'screenwriter', 'script', 'scripted', 'scripts', 'scrooge', 'sea', 'seagal', 'sean', 'search', 'searching', 'season', 'seasons', 'seat', 'second', 'secondly', 'seconds', 'secret', 'secretary', 'secretly', 'secrets', 'section', 'security', 'see', 'seed', 'seeing', 'seek', 'seeking', 'seeks', 'seem', 'seemed', 'seemingly', 'seems', 'seen', 'sees', 'segment', 'segments', 'seldom', 'self', 'selfish', 'sell', 'sellers', 'selling', 'semi', 'send', 'sends', 'sense', 'senseless', 'senses', 'sensitive', 'sent', 'sentence', 'sentimental', 'separate', 'september', 'sequel', 'sequels', 'sequence', 'sequences', 'serial', 'series', 'serious', 'seriously', 'serve', 'served', 'serves', 'service', 'serving', 'set', 'sets', 'setting', 'settings', 'settle', 'seven', 'seventies', 'several', 'severe', 'sex', 'sexual', 'sexuality', 'sexually', 'sexy', 'sh', 'shadow', 'shadows', 'shake', 'shakespeare', 'shaky', 'shall', 'shallow', 'shame', 'shanghai', 'shape', 'share', 'shark', 'sharp', 'shaw', 'shed', 'sheer', 'shelf', 'shelley', 'sheriff', 'shine', 'shines', 'shining', 'ship', 'ships', 'shirley', 'shirt', 'shock', 'shocked', 'shocking', 'shoes', 'shoot', 'shooting', 'shoots', 'shop', 'short', 'shortly', 'shorts', 'shot', 'shots', 'show', 'showcase', 'showdown', 'showed', 'shower', 'showing', 'shown', 'shows', 'shut', 'shy', 'sick', 'sid', 'side', 'sidekick', 'sides', 'sidney', 'sight', 'sign', 'signed', 'significance', 'significant', 'signs', 'silence', 'silent', 'silly', 'silver', 'similar', 'similarities', 'similarly', 'simmons', 'simon', 'simple', 'simplicity', 'simplistic', 'simply', 'simpson', 'sin', 'sinatra', 'since', 'sincere', 'sing', 'singer', 'singers', 'singing', 'single', 'sings', 'sinister', 'sink', 'sir', 'sirk', 'sissy', 'sister', 'sisters', 'sit', 'sitcom', 'site', 'sits', 'sitting', 'situation', 'situations', 'six', 'sixties', 'size', 'skill', 'skills', 'skin', 'skip', 'skull', 'sky', 'slap', 'slapstick', 'slasher', 'slaughter', 'slave', 'sleazy', 'sleep', 'sleeping', 'slice', 'slick', 'slight', 'slightest', 'slightly', 'sloppy', 'slow', 'slowly', 'small', 'smaller', 'smart', 'smile', 'smiling', 'smith', 'smoke', 'smoking', 'smooth', 'snake', 'sneak', 'snl', 'snow', 'soap', 'soccer', 'social', 'society', 'soderbergh', 'soft', 'sold', 'soldier', 'soldiers', 'sole', 'solely', 'solid', 'solo', 'solution', 'solve', 'somebody', 'somehow', 'someone', 'something', 'sometimes', 'somewhat', 'somewhere', 'son', 'song', 'songs', 'sons', 'soon', 'sophisticated', 'sopranos', 'sorry', 'sort', 'sorts', 'soul', 'souls', 'sound', 'sounded', 'sounding', 'sounds', 'soundtrack', 'source', 'south', 'southern', 'soviet', 'space', 'spacey', 'spain', 'spanish', 'spare', 'spark', 'speak', 'speaking', 'speaks', 'special', 'specially', 'species', 'specific', 'specifically', 'spectacular', 'speech', 'speed', 'spell', 'spend', 'spending', 'spends', 'spent', 'spider', 'spielberg', 'spike', 'spin', 'spirit', 'spirited', 'spirits', 'spiritual', 'spite', 'splatter', 'splendid', 'split', 'spock', 'spoil', 'spoiled', 'spoiler', 'spoilers', 'spoke', 'spoken', 'spoof', 'spooky', 'sport', 'sports', 'spot', 'spots', 'spread', 'spring', 'spy', 'square', 'st', 'stack', 'staff', 'stage', 'staged', 'stale', 'stallone', 'stan', 'stand', 'standard', 'standards', 'standing', 'stands', 'stanley', 'stanwyck', 'star', 'stargate', 'staring', 'starred', 'starring', 'stars', 'start', 'started', 'starting', 'starts', 'state', 'stated', 'statement', 'states', 'station', 'status', 'stay', 'stayed', 'staying', 'stays', 'steal', 'stealing', 'steals', 'steel', 'stellar', 'step', 'stephen', 'steps', 'stereotype', 'stereotypes', 'stereotypical', 'steve', 'steven', 'stevens', 'stewart', 'stick', 'sticks', 'stiff', 'still', 'stiller', 'stilted', 'stinker', 'stinks', 'stock', 'stole', 'stolen', 'stomach', 'stone', 'stood', 'stooges', 'stop', 'stopped', 'stops', 'store', 'stories', 'storm', 'story', 'storyline', 'storytelling', 'straight', 'strange', 'strangely', 'stranger', 'strangers', 'streep', 'street', 'streets', 'streisand', 'strength', 'stress', 'stretch', 'stretched', 'strictly', 'strike', 'strikes', 'striking', 'string', 'strip', 'strong', 'stronger', 'strongly', 'struck', 'structure', 'struggle', 'struggles', 'struggling', 'stuart', 'stuck', 'student', 'students', 'studio', 'studios', 'study', 'stuff', 'stumbled', 'stunned', 'stunning', 'stunt', 'stunts', 'stupid', 'stupidity', 'style', 'styles', 'stylish', 'sub', 'subject', 'subjected', 'subjects', 'subplot', 'subplots', 'subsequent', 'substance', 'subtitles', 'subtle', 'subtlety', 'succeed', 'succeeded', 'succeeds', 'success', 'successful', 'successfully', 'suck', 'sucked', 'sucks', 'sudden', 'suddenly', 'sue', 'suffer', 'suffered', 'suffering', 'suffers', 'suffice', 'suggest', 'suggested', 'suggests', 'suicide', 'suit', 'suitable', 'suited', 'suits', 'sullivan', 'sum', 'summary', 'summer', 'sun', 'sunday', 'sunshine', 'super', 'superb', 'superbly', 'superficial', 'superhero', 'superior', 'superman', 'supernatural', 'support', 'supported', 'supporting', 'suppose', 'supposed', 'supposedly', 'sure', 'surely', 'surface', 'surfing', 'surprise', 'surprised', 'surprises', 'surprising', 'surprisingly', 'surreal', 'surrounded', 'surrounding', 'survival', 'survive', 'survived', 'surviving', 'survivor', 'survivors', 'susan', 'suspect', 'suspects', 'suspend', 'suspense', 'suspenseful', 'suspicious', 'sutherland', 'swear', 'swedish', 'sweet', 'swim', 'swimming', 'switch', 'sword', 'symbolism', 'sympathetic', 'sympathy', 'synopsis', 'system', 'table', 'tacky', 'tad', 'tag', 'take', 'taken', 'takes', 'taking', 'tale', 'talent', 'talented', 'talents', 'tales', 'talk', 'talked', 'talking', 'talks', 'tall', 'tame', 'tank', 'tap', 'tape', 'tarantino', 'target', 'tarzan', 'task', 'taste', 'taught', 'taxi', 'taylor', 'tea', 'teach', 'teacher', 'teaching', 'team', 'tear', 'tears', 'tech', 'technical', 'technically', 'technicolor', 'technique', 'techniques', 'technology', 'ted', 'tedious', 'teen', 'teenage', 'teenager', 'teenagers', 'teens', 'teeth', 'television', 'tell', 'telling', 'tells', 'temple', 'ten', 'tend', 'tender', 'tends', 'tense', 'tension', 'term', 'terms', 'terrible', 'terribly', 'terrific', 'terrifying', 'territory', 'terror', 'terrorist', 'terrorists', 'terry', 'test', 'testament', 'texas', 'text', 'th', 'thank', 'thankfully', 'thanks', 'thats', 'theater', 'theaters', 'theatre', 'theatrical', 'theme', 'themes', 'theory', 'therefore', 'thick', 'thief', 'thin', 'thing', 'things', 'think', 'thinking', 'thinks', 'third', 'thirty', 'thomas', 'thompson', 'thoroughly', 'though', 'thought', 'thoughtful', 'thoughts', 'thousand', 'thousands', 'threat', 'threatening', 'three', 'threw', 'thrill', 'thriller', 'thrillers', 'thrilling', 'thrills', 'throat', 'throughout', 'throw', 'throwing', 'thrown', 'throws', 'thru', 'thugs', 'thumbs', 'thus', 'ticket', 'tie', 'tied', 'ties', 'tiger', 'tight', 'till', 'tim', 'time', 'timeless', 'times', 'timing', 'timon', 'timothy', 'tiny', 'tired', 'tiresome', 'titanic', 'title', 'titled', 'titles', 'today', 'todd', 'together', 'toilet', 'told', 'tom', 'tomatoes', 'tommy', 'tone', 'tongue', 'tonight', 'tons', 'tony', 'took', 'tooth', 'top', 'topic', 'topless', 'torn', 'torture', 'tortured', 'total', 'totally', 'touch', 'touched', 'touches', 'touching', 'tough', 'tour', 'toward', 'towards', 'town', 'toy', 'toys', 'track', 'tracks', 'tracy', 'trade', 'trademark', 'tradition', 'traditional', 'tragedy', 'tragic', 'trail', 'trailer', 'trailers', 'train', 'trained', 'training', 'transfer', 'transformation', 'transition', 'translation', 'trap', 'trapped', 'trash', 'trashy', 'travel', 'traveling', 'travels', 'travesty', 'treasure', 'treat', 'treated', 'treatment', 'treats', 'tree', 'trees', 'trek', 'tremendous', 'trial', 'tribe', 'tribute', 'trick', 'tricks', 'tried', 'tries', 'trilogy', 'trio', 'trip', 'trite', 'triumph', 'troops', 'trouble', 'troubled', 'troubles', 'truck', 'true', 'truly', 'trust', 'truth', 'try', 'trying', 'tube', 'tune', 'tunes', 'turkey', 'turn', 'turned', 'turner', 'turning', 'turns', 'tv', 'twelve', 'twenty', 'twice', 'twilight', 'twin', 'twins', 'twist', 'twisted', 'twists', 'two', 'type', 'types', 'typical', 'typically', 'ugly', 'uk', 'ultimate', 'ultimately', 'ultra', 'un', 'unable', 'unaware', 'unbearable', 'unbelievable', 'unbelievably', 'uncle', 'uncomfortable', 'unconvincing', 'underground', 'underlying', 'underrated', 'understand', 'understandable', 'understanding', 'understated', 'understood', 'undoubtedly', 'uneven', 'unexpected', 'unexpectedly', 'unfair', 'unfolds', 'unforgettable', 'unfortunate', 'unfortunately', 'unfunny', 'unhappy', 'uninspired', 'unintentional', 'unintentionally', 'uninteresting', 'union', 'unique', 'unit', 'united', 'universal', 'universe', 'university', 'unknown', 'unless', 'unlike', 'unlikely', 'unnecessary', 'unoriginal', 'unpleasant', 'unpredictable', 'unreal', 'unrealistic', 'unseen', 'unsettling', 'unusual', 'unwatchable', 'uplifting', 'upon', 'upper', 'ups', 'upset', 'urban', 'urge', 'us', 'usa', 'use', 'used', 'useful', 'useless', 'user', 'uses', 'using', 'ustinov', 'usual', 'usually', 'utter', 'utterly', 'uwe', 'vacation', 'vague', 'vaguely', 'valentine', 'valley', 'valuable', 'value', 'values', 'vampire', 'vampires', 'van', 'vance', 'variety', 'various', 'vast', 'vegas', 'vehicle', 'vengeance', 'verhoeven', 'version', 'versions', 'versus', 'veteran', 'vhs', 'via', 'vice', 'vicious', 'victim', 'victims', 'victor', 'victoria', 'victory', 'video', 'videos', 'vietnam', 'view', 'viewed', 'viewer', 'viewers', 'viewing', 'viewings', 'views', 'village', 'villain', 'villains', 'vincent', 'violence', 'violent', 'virgin', 'virginia', 'virtually', 'virus', 'visible', 'vision', 'visit', 'visits', 'visual', 'visually', 'visuals', 'vivid', 'voice', 'voiced', 'voices', 'voight', 'von', 'vote', 'vs', 'vulnerable', 'wacky', 'wait', 'waited', 'waiting', 'waitress', 'wake', 'walk', 'walked', 'walken', 'walker', 'walking', 'walks', 'wall', 'wallace', 'walls', 'walsh', 'walter', 'wandering', 'wang', 'wanna', 'wannabe', 'want', 'wanted', 'wanting', 'wants', 'war', 'ward', 'warm', 'warming', 'warmth', 'warn', 'warned', 'warner', 'warning', 'warren', 'warrior', 'warriors', 'wars', 'washington', 'waste', 'wasted', 'wasting', 'watch', 'watchable', 'watched', 'watches', 'watching', 'water', 'waters', 'watson', 'wave', 'waves', 'way', 'wayne', 'ways', 'weak', 'weakest', 'wealth', 'wealthy', 'weapon', 'weapons', 'wear', 'wearing', 'wears', 'web', 'website', 'wedding', 'week', 'weekend', 'weeks', 'weight', 'weird', 'welcome', 'well', 'welles', 'wells', 'wendy', 'went', 'werewolf', 'wes', 'west', 'western', 'westerns', 'wet', 'whale', 'whatever', 'whats', 'whatsoever', 'whenever', 'whereas', 'whether', 'whilst', 'white', 'whoever', 'whole', 'wholly', 'whoopi', 'whose', 'wicked', 'wide', 'widely', 'widmark', 'widow', 'wife', 'wild', 'wilderness', 'william', 'williams', 'willie', 'willing', 'willis', 'wilson', 'win', 'wind', 'window', 'winds', 'wing', 'winner', 'winning', 'wins', 'winter', 'winters', 'wisdom', 'wise', 'wish', 'wished', 'wishes', 'wishing', 'wit', 'witch', 'witches', 'within', 'without', 'witness', 'witnessed', 'witnesses', 'witty', 'wives', 'wizard', 'wolf', 'woman', 'women', 'wonder', 'wondered', 'wonderful', 'wonderfully', 'wondering', 'wonders', 'wont', 'woo', 'wood', 'wooden', 'woods', 'woody', 'word', 'words', 'wore', 'work', 'worked', 'worker', 'workers', 'working', 'works', 'world', 'worlds', 'worn', 'worried', 'worry', 'worse', 'worst', 'worth', 'worthless', 'worthwhile', 'worthy', 'would', 'wound', 'wounded', 'wow', 'wrap', 'wrapped', 'wreck', 'wrestling', 'wretched', 'write', 'writer', 'writers', 'writes', 'writing', 'written', 'wrong', 'wrote', 'ww', 'wwii', 'ya', 'yard', 'yeah', 'year', 'years', 'yelling', 'yellow', 'yes', 'yesterday', 'yet', 'york', 'young', 'younger', 'youth', 'zero', 'zizek', 'zombie', 'zombies', 'zone']\n"
     ]
    }
   ],
   "source": [
    "# Take a look at the words in the vocabulary\n",
    "vocab = vectorizer.get_feature_names()\n",
    "print(vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "187 abandoned\n",
      "125 abc\n",
      "108 abilities\n",
      "454 ability\n",
      "1259 able\n",
      "85 abraham\n",
      "116 absence\n",
      "83 absent\n",
      "352 absolute\n",
      "1485 absolutely\n",
      "306 absurd\n",
      "192 abuse\n",
      "91 abusive\n",
      "98 abysmal\n",
      "297 academy\n",
      "485 accent\n",
      "203 accents\n",
      "300 accept\n",
      "130 acceptable\n",
      "144 accepted\n",
      "92 access\n",
      "318 accident\n",
      "200 accidentally\n",
      "88 accompanied\n",
      "124 accomplished\n",
      "296 according\n",
      "186 account\n",
      "81 accuracy\n",
      "284 accurate\n",
      "123 accused\n",
      "179 achieve\n",
      "139 achieved\n",
      "124 achievement\n",
      "90 acid\n",
      "971 across\n",
      "1251 act\n",
      "658 acted\n",
      "6490 acting\n",
      "3354 action\n",
      "311 actions\n",
      "83 activities\n",
      "2389 actor\n",
      "4486 actors\n",
      "1219 actress\n",
      "369 actresses\n",
      "394 acts\n",
      "793 actual\n",
      "4237 actually\n",
      "148 ad\n",
      "302 adam\n",
      "98 adams\n",
      "453 adaptation\n",
      "80 adaptations\n",
      "154 adapted\n",
      "810 add\n",
      "439 added\n",
      "166 adding\n",
      "347 addition\n",
      "337 adds\n",
      "113 adequate\n",
      "124 admire\n",
      "621 admit\n",
      "134 admittedly\n",
      "101 adorable\n",
      "510 adult\n",
      "376 adults\n",
      "100 advance\n",
      "90 advanced\n",
      "153 advantage\n",
      "510 adventure\n",
      "204 adventures\n",
      "91 advertising\n",
      "259 advice\n",
      "90 advise\n",
      "346 affair\n",
      "93 affect\n",
      "113 affected\n",
      "104 afford\n",
      "126 aforementioned\n",
      "343 afraid\n",
      "212 africa\n",
      "255 african\n",
      "187 afternoon\n",
      "128 afterwards\n",
      "1121 age\n",
      "233 aged\n",
      "361 agent\n",
      "94 agents\n",
      "249 ages\n",
      "111 aging\n",
      "1033 ago\n",
      "572 agree\n",
      "88 agreed\n",
      "96 agrees\n",
      "119 ah\n",
      "396 ahead\n",
      "106 aid\n",
      "96 aids\n",
      "81 aim\n",
      "120 aimed\n",
      "639 air\n",
      "146 aired\n",
      "92 airplane\n",
      "93 airport\n",
      "194 aka\n",
      "100 akshay\n",
      "376 al\n",
      "351 alan\n",
      "163 alas\n",
      "157 albeit\n",
      "265 albert\n",
      "84 album\n",
      "84 alcohol\n",
      "93 alcoholic\n",
      "81 alec\n",
      "94 alert\n",
      "231 alex\n",
      "121 alexander\n",
      "85 alfred\n",
      "199 alice\n",
      "79 alicia\n",
      "373 alien\n",
      "199 aliens\n",
      "152 alike\n",
      "86 alison\n",
      "463 alive\n",
      "407 allen\n",
      "308 allow\n",
      "325 allowed\n",
      "128 allowing\n",
      "252 allows\n",
      "3139 almost\n",
      "1061 alone\n",
      "1776 along\n",
      "90 alongside\n",
      "1381 already\n",
      "185 alright\n",
      "9156 also\n",
      "88 alternate\n",
      "78 alternative\n",
      "2537 although\n",
      "114 altman\n",
      "112 altogether\n",
      "3239 always\n",
      "101 amanda\n",
      "215 amateur\n",
      "216 amateurish\n",
      "183 amazed\n",
      "1320 amazing\n",
      "174 amazingly\n",
      "80 ambiguous\n",
      "126 ambitious\n",
      "728 america\n",
      "2228 american\n",
      "365 americans\n",
      "92 amitabh\n",
      "783 among\n",
      "160 amongst\n",
      "495 amount\n",
      "90 amounts\n",
      "78 amused\n",
      "509 amusing\n",
      "104 amy\n",
      "88 analysis\n",
      "233 ancient\n",
      "223 anderson\n",
      "79 andre\n",
      "147 andrew\n",
      "151 andrews\n",
      "318 andy\n",
      "230 angel\n",
      "85 angela\n",
      "101 angeles\n",
      "161 angels\n",
      "191 anger\n",
      "185 angle\n",
      "206 angles\n",
      "336 angry\n",
      "342 animal\n",
      "410 animals\n",
      "516 animated\n",
      "826 animation\n",
      "240 anime\n",
      "288 ann\n",
      "251 anna\n",
      "254 anne\n",
      "117 annie\n",
      "143 annoyed\n",
      "998 annoying\n",
      "4325 another\n",
      "361 answer\n",
      "176 answers\n",
      "263 anthony\n",
      "480 anti\n",
      "113 antics\n",
      "82 antonioni\n",
      "88 antwone\n",
      "310 anybody\n",
      "333 anymore\n",
      "2630 anyone\n",
      "2949 anything\n",
      "1117 anyway\n",
      "113 anyways\n",
      "304 anywhere\n",
      "623 apart\n",
      "339 apartment\n",
      "105 ape\n",
      "113 apes\n",
      "125 appalling\n",
      "309 apparent\n",
      "917 apparently\n",
      "448 appeal\n",
      "225 appealing\n",
      "619 appear\n",
      "451 appearance\n",
      "139 appearances\n",
      "371 appeared\n",
      "141 appearing\n",
      "841 appears\n",
      "507 appreciate\n",
      "196 appreciated\n",
      "88 appreciation\n",
      "372 approach\n",
      "221 appropriate\n",
      "101 april\n",
      "337 area\n",
      "116 areas\n",
      "90 arguably\n",
      "111 argue\n",
      "118 argument\n",
      "152 arm\n",
      "95 armed\n",
      "175 arms\n",
      "454 army\n",
      "148 arnold\n",
      "3616 around\n",
      "126 arrested\n",
      "87 arrival\n",
      "114 arrive\n",
      "93 arrived\n",
      "162 arrives\n",
      "99 arrogant\n",
      "1293 art\n",
      "373 arthur\n",
      "94 artificial\n",
      "331 artist\n",
      "339 artistic\n",
      "182 artists\n",
      "310 arts\n",
      "158 ashamed\n",
      "89 ashley\n",
      "237 asian\n",
      "473 aside\n",
      "648 ask\n",
      "295 asked\n",
      "228 asking\n",
      "329 asks\n",
      "213 asleep\n",
      "454 aspect\n",
      "398 aspects\n",
      "267 ass\n",
      "85 assassin\n",
      "88 assault\n",
      "83 assigned\n",
      "148 assistant\n",
      "131 associated\n",
      "229 assume\n",
      "82 assumed\n",
      "132 astaire\n",
      "79 astonishing\n",
      "111 atlantis\n",
      "735 atmosphere\n",
      "148 atmospheric\n",
      "197 atrocious\n",
      "132 attached\n",
      "449 attack\n",
      "158 attacked\n",
      "158 attacks\n",
      "1050 attempt\n",
      "136 attempted\n",
      "163 attempting\n",
      "583 attempts\n",
      "83 attend\n",
      "906 attention\n",
      "237 attitude\n",
      "90 attitudes\n",
      "89 attorney\n",
      "123 attracted\n",
      "143 attraction\n",
      "352 attractive\n",
      "2199 audience\n",
      "476 audiences\n",
      "113 audio\n",
      "173 aunt\n",
      "91 austen\n",
      "82 austin\n",
      "126 australia\n",
      "206 australian\n",
      "166 authentic\n",
      "275 author\n",
      "101 authority\n",
      "388 available\n",
      "720 average\n",
      "775 avoid\n",
      "102 avoided\n",
      "106 awake\n",
      "419 award\n",
      "245 awards\n",
      "277 aware\n",
      "2775 away\n",
      "101 awe\n",
      "485 awesome\n",
      "1724 awful\n",
      "84 awfully\n",
      "78 awhile\n",
      "248 awkward\n",
      "100 babe\n",
      "693 baby\n",
      "90 bacall\n",
      "4972 back\n",
      "105 backdrop\n",
      "619 background\n",
      "101 backgrounds\n",
      "9301 bad\n",
      "662 badly\n",
      "150 bag\n",
      "155 baker\n",
      "130 bakshi\n",
      "161 balance\n",
      "80 baldwin\n",
      "290 ball\n",
      "94 ballet\n",
      "93 balls\n",
      "529 band\n",
      "93 bands\n",
      "108 bang\n",
      "265 bank\n",
      "108 banned\n",
      "387 bar\n",
      "243 barbara\n",
      "114 bare\n",
      "483 barely\n",
      "86 bargain\n",
      "134 barry\n",
      "88 barrymore\n",
      "173 base\n",
      "215 baseball\n",
      "1430 based\n",
      "143 basement\n",
      "519 basic\n",
      "906 basically\n",
      "169 basis\n",
      "88 basketball\n",
      "168 bat\n",
      "93 bath\n",
      "114 bathroom\n",
      "432 batman\n",
      "614 battle\n",
      "136 battles\n",
      "89 bay\n",
      "189 bbc\n",
      "191 beach\n",
      "251 bear\n",
      "140 bears\n",
      "187 beast\n",
      "354 beat\n",
      "124 beaten\n",
      "138 beating\n",
      "111 beats\n",
      "113 beatty\n",
      "2177 beautiful\n",
      "436 beautifully\n",
      "655 beauty\n",
      "697 became\n",
      "1544 become\n",
      "1380 becomes\n",
      "348 becoming\n",
      "384 bed\n",
      "107 bedroom\n",
      "123 beer\n",
      "318 began\n",
      "678 begin\n",
      "1401 beginning\n",
      "795 begins\n",
      "94 behave\n",
      "261 behavior\n",
      "1280 behind\n",
      "128 beings\n",
      "94 bela\n",
      "188 belief\n",
      "92 beliefs\n",
      "711 believable\n",
      "2505 believe\n",
      "208 believed\n",
      "228 believes\n",
      "137 believing\n",
      "112 bell\n",
      "85 belong\n",
      "142 belongs\n",
      "178 beloved\n",
      "87 belushi\n",
      "616 ben\n",
      "104 beneath\n",
      "102 benefit\n",
      "96 bergman\n",
      "99 berlin\n",
      "410 besides\n",
      "6416 best\n",
      "244 bet\n",
      "155 bette\n",
      "5737 better\n",
      "163 bettie\n",
      "89 betty\n",
      "866 beyond\n",
      "134 bible\n",
      "3477 big\n",
      "268 bigger\n",
      "515 biggest\n",
      "89 biko\n",
      "633 bill\n",
      "79 billed\n",
      "375 billy\n",
      "110 bin\n",
      "82 biography\n",
      "113 bird\n",
      "116 birds\n",
      "196 birth\n",
      "137 birthday\n",
      "3054 bit\n",
      "114 bite\n",
      "295 bits\n",
      "164 bitter\n",
      "499 bizarre\n",
      "2033 black\n",
      "95 blade\n",
      "195 blah\n",
      "167 blair\n",
      "110 blake\n",
      "290 blame\n",
      "273 bland\n",
      "122 blank\n",
      "79 blast\n",
      "104 blatant\n",
      "119 bleak\n",
      "113 blend\n",
      "101 blew\n",
      "262 blind\n",
      "122 blob\n",
      "113 block\n",
      "187 blockbuster\n",
      "97 blond\n",
      "270 blonde\n",
      "1185 blood\n",
      "302 bloody\n",
      "202 blow\n",
      "108 blowing\n",
      "184 blown\n",
      "122 blows\n",
      "431 blue\n",
      "100 blues\n",
      "94 blunt\n",
      "156 bo\n",
      "250 board\n",
      "254 boat\n",
      "266 bob\n",
      "138 bobby\n",
      "217 bodies\n",
      "972 body\n",
      "102 bold\n",
      "143 boll\n",
      "169 bollywood\n",
      "235 bomb\n",
      "340 bond\n",
      "114 bone\n",
      "97 bonus\n",
      "2421 book\n",
      "505 books\n",
      "93 boom\n",
      "95 boot\n",
      "124 border\n",
      "181 bore\n",
      "530 bored\n",
      "141 boredom\n",
      "1809 boring\n",
      "385 born\n",
      "84 borrowed\n",
      "415 boss\n",
      "395 bother\n",
      "187 bothered\n",
      "91 bottle\n",
      "421 bottom\n",
      "458 bought\n",
      "179 bound\n",
      "190 bourne\n",
      "637 box\n",
      "126 boxing\n",
      "1560 boy\n",
      "399 boyfriend\n",
      "79 boyle\n",
      "618 boys\n",
      "175 brad\n",
      "109 brady\n",
      "474 brain\n",
      "109 brains\n",
      "186 branagh\n",
      "135 brand\n",
      "158 brando\n",
      "202 brave\n",
      "116 brazil\n",
      "611 break\n",
      "231 breaking\n",
      "259 breaks\n",
      "128 breasts\n",
      "177 breath\n",
      "168 breathtaking\n",
      "84 brenda\n",
      "352 brian\n",
      "136 bride\n",
      "157 bridge\n",
      "395 brief\n",
      "135 briefly\n",
      "273 bright\n",
      "129 brilliance\n",
      "1195 brilliant\n",
      "248 brilliantly\n",
      "869 bring\n",
      "216 bringing\n",
      "630 brings\n",
      "149 britain\n",
      "898 british\n",
      "115 broad\n",
      "114 broadcast\n",
      "242 broadway\n",
      "151 broke\n",
      "278 broken\n",
      "105 brooklyn\n",
      "169 brooks\n",
      "128 brosnan\n",
      "1107 brother\n",
      "557 brothers\n",
      "737 brought\n",
      "271 brown\n",
      "393 bruce\n",
      "303 brutal\n",
      "81 brutality\n",
      "90 brutally\n",
      "175 buck\n",
      "124 bucks\n",
      "83 bud\n",
      "109 buddies\n",
      "258 buddy\n",
      "1836 budget\n",
      "102 buff\n",
      "82 buffalo\n",
      "98 buffs\n",
      "119 bug\n",
      "152 bugs\n",
      "318 build\n",
      "395 building\n",
      "94 buildings\n",
      "114 builds\n",
      "236 built\n",
      "102 bull\n",
      "109 bullet\n",
      "124 bullets\n",
      "85 bumbling\n",
      "813 bunch\n",
      "78 bunny\n",
      "126 buried\n",
      "127 burn\n",
      "114 burned\n",
      "144 burning\n",
      "183 burns\n",
      "167 burt\n",
      "152 burton\n",
      "186 bus\n",
      "136 bush\n",
      "624 business\n",
      "83 businessman\n",
      "81 buster\n",
      "162 busy\n",
      "86 butler\n",
      "123 butt\n",
      "96 button\n",
      "763 buy\n",
      "184 buying\n",
      "175 cabin\n",
      "279 cable\n",
      "296 cage\n",
      "124 cagney\n",
      "204 caine\n",
      "98 cake\n",
      "84 caliber\n",
      "189 california\n",
      "923 call\n",
      "1433 called\n",
      "176 calling\n",
      "260 calls\n",
      "86 calm\n",
      "1673 came\n",
      "259 cameo\n",
      "146 cameos\n",
      "1778 camera\n",
      "110 cameras\n",
      "123 cameron\n",
      "463 camp\n",
      "97 campbell\n",
      "179 campy\n",
      "139 canada\n",
      "226 canadian\n",
      "289 candy\n",
      "84 cannibal\n",
      "1096 cannot\n",
      "202 cant\n",
      "228 capable\n",
      "89 capital\n",
      "326 captain\n",
      "119 captivating\n",
      "287 capture\n",
      "261 captured\n",
      "216 captures\n",
      "88 capturing\n",
      "1225 car\n",
      "150 card\n",
      "131 cardboard\n",
      "99 cards\n",
      "1385 care\n",
      "121 cared\n",
      "1007 career\n",
      "107 careers\n",
      "90 careful\n",
      "134 carefully\n",
      "88 carell\n",
      "228 cares\n",
      "166 caring\n",
      "109 carl\n",
      "85 carla\n",
      "138 carol\n",
      "161 carpenter\n",
      "94 carradine\n",
      "131 carrey\n",
      "117 carrie\n",
      "163 carried\n",
      "163 carries\n",
      "326 carry\n",
      "165 carrying\n",
      "279 cars\n",
      "151 carter\n",
      "545 cartoon\n",
      "205 cartoons\n",
      "108 cary\n",
      "1533 case\n",
      "163 cases\n",
      "236 cash\n",
      "85 cassidy\n",
      "3829 cast\n",
      "622 casting\n",
      "339 castle\n",
      "547 cat\n",
      "447 catch\n",
      "106 catches\n",
      "92 catching\n",
      "91 catchy\n",
      "211 category\n",
      "142 catherine\n",
      "152 catholic\n",
      "110 cats\n",
      "555 caught\n",
      "534 cause\n",
      "237 caused\n",
      "166 causes\n",
      "107 causing\n",
      "98 cave\n",
      "97 cd\n",
      "102 celebrity\n",
      "176 cell\n",
      "107 celluloid\n",
      "232 center\n",
      "109 centered\n",
      "91 centers\n",
      "411 central\n",
      "528 century\n",
      "764 certain\n",
      "1462 certainly\n",
      "96 cg\n",
      "325 cgi\n",
      "122 chain\n",
      "157 chair\n",
      "165 challenge\n",
      "88 challenging\n",
      "81 championship\n",
      "207 chan\n",
      "1067 chance\n",
      "133 chances\n",
      "959 change\n",
      "484 changed\n",
      "386 changes\n",
      "194 changing\n",
      "442 channel\n",
      "86 channels\n",
      "105 chaos\n",
      "150 chaplin\n",
      "88 chapter\n",
      "7023 character\n",
      "123 characterization\n",
      "7154 characters\n",
      "168 charge\n",
      "138 charisma\n",
      "135 charismatic\n",
      "408 charles\n",
      "439 charlie\n",
      "98 charlotte\n",
      "407 charm\n",
      "471 charming\n",
      "438 chase\n",
      "98 chased\n",
      "143 chases\n",
      "145 chasing\n",
      "217 che\n",
      "892 cheap\n",
      "92 cheated\n",
      "103 cheating\n",
      "762 check\n",
      "80 checked\n",
      "139 checking\n",
      "114 cheek\n",
      "158 cheese\n",
      "634 cheesy\n",
      "490 chemistry\n",
      "96 chess\n",
      "93 chest\n",
      "92 chicago\n",
      "233 chick\n",
      "80 chicken\n",
      "89 chicks\n",
      "229 chief\n",
      "1320 child\n",
      "356 childhood\n",
      "117 childish\n",
      "1510 children\n",
      "169 chilling\n",
      "188 china\n",
      "337 chinese\n",
      "528 choice\n",
      "171 choices\n",
      "227 choose\n",
      "85 chooses\n",
      "99 choreographed\n",
      "115 choreography\n",
      "99 chorus\n",
      "202 chose\n",
      "232 chosen\n",
      "421 chris\n",
      "183 christ\n",
      "373 christian\n",
      "79 christianity\n",
      "92 christians\n",
      "623 christmas\n",
      "415 christopher\n",
      "97 christy\n",
      "142 chuck\n",
      "406 church\n",
      "123 cia\n",
      "233 cinderella\n",
      "1491 cinema\n",
      "412 cinematic\n",
      "101 cinematographer\n",
      "983 cinematography\n",
      "103 circle\n",
      "218 circumstances\n",
      "88 cities\n",
      "125 citizen\n",
      "1195 city\n",
      "140 civil\n",
      "93 civilization\n",
      "222 claim\n",
      "100 claimed\n",
      "205 claims\n",
      "173 claire\n",
      "204 clark\n",
      "893 class\n",
      "106 classes\n",
      "1828 classic\n",
      "88 classical\n",
      "233 classics\n",
      "80 claus\n",
      "243 clean\n",
      "786 clear\n",
      "899 clearly\n",
      "533 clever\n",
      "94 cleverly\n",
      "840 clich\n",
      "96 cliche\n",
      "112 cliff\n",
      "93 climactic\n",
      "422 climax\n",
      "110 clint\n",
      "83 clip\n",
      "162 clips\n",
      "96 clock\n",
      "1296 close\n",
      "92 closed\n",
      "140 closely\n",
      "206 closer\n",
      "94 closest\n",
      "103 closet\n",
      "177 closing\n",
      "328 clothes\n",
      "110 clothing\n",
      "107 clown\n",
      "434 club\n",
      "224 clue\n",
      "123 clues\n",
      "105 clumsy\n",
      "604 co\n",
      "102 coach\n",
      "80 coast\n",
      "241 code\n",
      "106 coffee\n",
      "109 coherent\n",
      "571 cold\n",
      "132 cole\n",
      "343 collection\n",
      "496 college\n",
      "106 colonel\n",
      "402 color\n",
      "144 colorful\n",
      "199 colors\n",
      "138 colour\n",
      "206 columbo\n",
      "218 com\n",
      "109 combat\n",
      "231 combination\n",
      "94 combine\n",
      "198 combined\n",
      "3189 come\n",
      "159 comedian\n",
      "315 comedic\n",
      "439 comedies\n",
      "3244 comedy\n",
      "2484 comes\n",
      "80 comfort\n",
      "110 comfortable\n",
      "901 comic\n",
      "171 comical\n",
      "133 comics\n",
      "1062 coming\n",
      "99 command\n",
      "653 comment\n",
      "318 commentary\n",
      "100 commented\n",
      "779 comments\n",
      "239 commercial\n",
      "116 commercials\n",
      "127 commit\n",
      "195 committed\n",
      "509 common\n",
      "103 communist\n",
      "290 community\n",
      "80 companies\n",
      "104 companion\n",
      "515 company\n",
      "325 compare\n",
      "538 compared\n",
      "96 comparing\n",
      "251 comparison\n",
      "81 compassion\n",
      "87 compelled\n",
      "385 compelling\n",
      "140 competent\n",
      "127 competition\n",
      "115 complain\n",
      "139 complaint\n",
      "1035 complete\n",
      "1889 completely\n",
      "427 complex\n",
      "90 complexity\n",
      "178 complicated\n",
      "111 composed\n",
      "82 composer\n",
      "468 computer\n",
      "180 con\n",
      "120 conceived\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "520 concept\n",
      "97 concern\n",
      "269 concerned\n",
      "115 concerning\n",
      "140 concerns\n",
      "143 concert\n",
      "490 conclusion\n",
      "143 condition\n",
      "80 conditions\n",
      "81 confess\n",
      "95 confidence\n",
      "273 conflict\n",
      "93 conflicts\n",
      "79 confrontation\n",
      "377 confused\n",
      "368 confusing\n",
      "163 confusion\n",
      "117 connect\n",
      "147 connected\n",
      "258 connection\n",
      "94 connery\n",
      "79 conscious\n",
      "123 consequences\n",
      "95 conservative\n",
      "513 consider\n",
      "97 considerable\n",
      "483 considered\n",
      "545 considering\n",
      "96 consistent\n",
      "104 consistently\n",
      "144 consists\n",
      "126 conspiracy\n",
      "291 constant\n",
      "416 constantly\n",
      "95 constructed\n",
      "91 construction\n",
      "151 contact\n",
      "156 contain\n",
      "113 contained\n",
      "411 contains\n",
      "211 contemporary\n",
      "366 content\n",
      "83 contest\n",
      "260 context\n",
      "307 continue\n",
      "128 continued\n",
      "261 continues\n",
      "212 continuity\n",
      "130 contract\n",
      "116 contrary\n",
      "224 contrast\n",
      "227 contrived\n",
      "510 control\n",
      "152 controversial\n",
      "109 conventional\n",
      "188 conversation\n",
      "110 conversations\n",
      "171 convey\n",
      "197 convince\n",
      "216 convinced\n",
      "538 convincing\n",
      "93 convincingly\n",
      "121 convoluted\n",
      "159 cook\n",
      "971 cool\n",
      "166 cooper\n",
      "620 cop\n",
      "90 copies\n",
      "301 cops\n",
      "575 copy\n",
      "259 core\n",
      "153 corner\n",
      "258 corny\n",
      "96 corporate\n",
      "127 corpse\n",
      "223 correct\n",
      "82 correctly\n",
      "133 corrupt\n",
      "100 corruption\n",
      "220 cost\n",
      "233 costs\n",
      "223 costume\n",
      "418 costumes\n",
      "7922 could\n",
      "352 count\n",
      "87 counter\n",
      "136 countless\n",
      "151 countries\n",
      "935 country\n",
      "103 countryside\n",
      "1719 couple\n",
      "93 couples\n",
      "148 courage\n",
      "2506 course\n",
      "193 court\n",
      "147 cousin\n",
      "526 cover\n",
      "212 covered\n",
      "135 covers\n",
      "212 cowboy\n",
      "132 cox\n",
      "106 crack\n",
      "79 cracking\n",
      "113 craft\n",
      "167 crafted\n",
      "122 craig\n",
      "1039 crap\n",
      "242 crappy\n",
      "201 crash\n",
      "118 craven\n",
      "82 crawford\n",
      "79 crazed\n",
      "657 crazy\n",
      "78 cream\n",
      "612 create\n",
      "542 created\n",
      "245 creates\n",
      "284 creating\n",
      "129 creation\n",
      "362 creative\n",
      "85 creativity\n",
      "92 creator\n",
      "127 creators\n",
      "340 creature\n",
      "214 creatures\n",
      "153 credibility\n",
      "153 credible\n",
      "530 credit\n",
      "674 credits\n",
      "91 creep\n",
      "638 creepy\n",
      "570 crew\n",
      "117 cried\n",
      "757 crime\n",
      "139 crimes\n",
      "325 criminal\n",
      "174 criminals\n",
      "107 cringe\n",
      "152 crisis\n",
      "124 critic\n",
      "175 critical\n",
      "173 criticism\n",
      "398 critics\n",
      "94 crocodile\n",
      "325 cross\n",
      "226 crowd\n",
      "86 crucial\n",
      "187 crude\n",
      "181 cruel\n",
      "128 cruise\n",
      "102 crush\n",
      "399 cry\n",
      "191 crying\n",
      "83 crystal\n",
      "109 cuba\n",
      "147 cube\n",
      "483 cult\n",
      "184 cultural\n",
      "495 culture\n",
      "124 cup\n",
      "101 cure\n",
      "124 curiosity\n",
      "262 curious\n",
      "257 current\n",
      "113 currently\n",
      "147 curse\n",
      "142 curtis\n",
      "137 cusack\n",
      "1035 cut\n",
      "581 cute\n",
      "274 cuts\n",
      "211 cutting\n",
      "154 cynical\n",
      "80 da\n",
      "491 dad\n",
      "107 daddy\n",
      "163 daily\n",
      "97 dalton\n",
      "108 damage\n",
      "84 damme\n",
      "356 damn\n",
      "89 damon\n",
      "264 dan\n",
      "83 dana\n",
      "744 dance\n",
      "149 dancer\n",
      "116 dancers\n",
      "103 dances\n",
      "529 dancing\n",
      "116 danes\n",
      "215 danger\n",
      "291 dangerous\n",
      "237 daniel\n",
      "330 danny\n",
      "138 dare\n",
      "114 daring\n",
      "1380 dark\n",
      "117 darker\n",
      "193 darkness\n",
      "83 darren\n",
      "436 date\n",
      "267 dated\n",
      "107 dating\n",
      "1138 daughter\n",
      "174 daughters\n",
      "117 dave\n",
      "1024 david\n",
      "129 davies\n",
      "354 davis\n",
      "136 dawn\n",
      "130 dawson\n",
      "2746 day\n",
      "1268 days\n",
      "730 de\n",
      "1881 dead\n",
      "201 deadly\n",
      "82 deaf\n",
      "717 deal\n",
      "267 dealing\n",
      "258 deals\n",
      "137 dealt\n",
      "188 dean\n",
      "146 dear\n",
      "1908 death\n",
      "205 deaths\n",
      "260 debut\n",
      "255 decade\n",
      "177 decades\n",
      "82 deceased\n",
      "1157 decent\n",
      "482 decide\n",
      "705 decided\n",
      "546 decides\n",
      "240 decision\n",
      "105 decisions\n",
      "99 dedicated\n",
      "82 dee\n",
      "653 deep\n",
      "177 deeper\n",
      "320 deeply\n",
      "107 defeat\n",
      "95 defend\n",
      "86 defense\n",
      "87 defined\n",
      "117 definite\n",
      "1580 definitely\n",
      "83 definition\n",
      "204 degree\n",
      "97 del\n",
      "108 deliberately\n",
      "157 delight\n",
      "274 delightful\n",
      "328 deliver\n",
      "243 delivered\n",
      "94 delivering\n",
      "356 delivers\n",
      "184 delivery\n",
      "90 demand\n",
      "113 demands\n",
      "89 demented\n",
      "79 demise\n",
      "183 demon\n",
      "192 demons\n",
      "82 deniro\n",
      "176 dennis\n",
      "152 dentist\n",
      "141 denzel\n",
      "194 department\n",
      "210 depicted\n",
      "82 depicting\n",
      "172 depiction\n",
      "90 depicts\n",
      "99 depressed\n",
      "226 depressing\n",
      "129 depression\n",
      "511 depth\n",
      "86 der\n",
      "151 derek\n",
      "89 descent\n",
      "330 describe\n",
      "233 described\n",
      "102 describes\n",
      "175 description\n",
      "250 desert\n",
      "287 deserve\n",
      "291 deserved\n",
      "591 deserves\n",
      "342 design\n",
      "206 designed\n",
      "91 designs\n",
      "304 desire\n",
      "91 desired\n",
      "86 despair\n",
      "323 desperate\n",
      "179 desperately\n",
      "104 desperation\n",
      "1364 despite\n",
      "104 destiny\n",
      "219 destroy\n",
      "178 destroyed\n",
      "94 destroying\n",
      "142 destruction\n",
      "345 detail\n",
      "108 detailed\n",
      "410 details\n",
      "460 detective\n",
      "164 determined\n",
      "252 develop\n",
      "404 developed\n",
      "102 developing\n",
      "641 development\n",
      "144 develops\n",
      "181 device\n",
      "314 devil\n",
      "104 devoid\n",
      "109 devoted\n",
      "769 dialog\n",
      "142 dialogs\n",
      "1542 dialogue\n",
      "110 dialogues\n",
      "125 diamond\n",
      "101 diana\n",
      "103 diane\n",
      "295 dick\n",
      "94 dickens\n",
      "794 die\n",
      "512 died\n",
      "386 dies\n",
      "376 difference\n",
      "115 differences\n",
      "2385 different\n",
      "695 difficult\n",
      "142 dig\n",
      "126 digital\n",
      "118 dignity\n",
      "83 dimension\n",
      "255 dimensional\n",
      "82 din\n",
      "154 dinner\n",
      "99 dinosaur\n",
      "111 dinosaurs\n",
      "122 dire\n",
      "366 direct\n",
      "1204 directed\n",
      "644 directing\n",
      "1373 direction\n",
      "80 directions\n",
      "193 directly\n",
      "4444 director\n",
      "121 directorial\n",
      "675 directors\n",
      "103 directs\n",
      "334 dirty\n",
      "131 disagree\n",
      "101 disappear\n",
      "99 disappeared\n",
      "98 disappoint\n",
      "917 disappointed\n",
      "420 disappointing\n",
      "404 disappointment\n",
      "320 disaster\n",
      "175 disbelief\n",
      "119 disc\n",
      "266 discover\n",
      "267 discovered\n",
      "240 discovers\n",
      "116 discovery\n",
      "106 discuss\n",
      "118 discussion\n",
      "128 disease\n",
      "221 disgusting\n",
      "102 disjointed\n",
      "150 dislike\n",
      "79 disliked\n",
      "754 disney\n",
      "205 display\n",
      "99 displayed\n",
      "114 displays\n",
      "118 distance\n",
      "124 distant\n",
      "84 distinct\n",
      "107 distracting\n",
      "82 distribution\n",
      "107 disturbed\n",
      "484 disturbing\n",
      "102 divorce\n",
      "94 dixon\n",
      "137 doc\n",
      "625 doctor\n",
      "121 documentaries\n",
      "953 documentary\n",
      "688 dog\n",
      "199 dogs\n",
      "104 doll\n",
      "169 dollar\n",
      "200 dollars\n",
      "96 dolls\n",
      "85 dolph\n",
      "86 domestic\n",
      "98 domino\n",
      "191 donald\n",
      "3096 done\n",
      "115 donna\n",
      "123 doo\n",
      "90 doom\n",
      "104 doomed\n",
      "432 door\n",
      "125 doors\n",
      "142 dorothy\n",
      "399 double\n",
      "757 doubt\n",
      "82 doubts\n",
      "315 douglas\n",
      "79 downey\n",
      "96 downhill\n",
      "188 downright\n",
      "196 dozen\n",
      "103 dozens\n",
      "702 dr\n",
      "121 dracula\n",
      "211 drag\n",
      "138 dragged\n",
      "186 dragon\n",
      "130 drags\n",
      "89 drake\n",
      "1411 drama\n",
      "141 dramas\n",
      "667 dramatic\n",
      "198 draw\n",
      "114 drawing\n",
      "428 drawn\n",
      "118 draws\n",
      "240 dreadful\n",
      "663 dream\n",
      "436 dreams\n",
      "87 dreary\n",
      "81 dreck\n",
      "181 dress\n",
      "293 dressed\n",
      "86 dressing\n",
      "245 drew\n",
      "148 drink\n",
      "175 drinking\n",
      "448 drive\n",
      "125 drivel\n",
      "239 driven\n",
      "195 driver\n",
      "155 drives\n",
      "269 driving\n",
      "205 drop\n",
      "131 dropped\n",
      "85 dropping\n",
      "102 drops\n",
      "403 drug\n",
      "325 drugs\n",
      "292 drunk\n",
      "119 drunken\n",
      "230 dry\n",
      "96 dub\n",
      "224 dubbed\n",
      "153 dubbing\n",
      "83 dud\n",
      "183 dude\n",
      "909 due\n",
      "132 duke\n",
      "816 dull\n",
      "609 dumb\n",
      "121 duo\n",
      "91 dust\n",
      "104 dutch\n",
      "106 duty\n",
      "2345 dvd\n",
      "314 dying\n",
      "112 dynamic\n",
      "94 eager\n",
      "86 ear\n",
      "87 earl\n",
      "662 earlier\n",
      "1605 early\n",
      "100 earned\n",
      "99 ears\n",
      "928 earth\n",
      "110 ease\n",
      "132 easier\n",
      "892 easily\n",
      "170 east\n",
      "83 eastern\n",
      "138 eastwood\n",
      "802 easy\n",
      "275 eat\n",
      "90 eaten\n",
      "278 eating\n",
      "109 eccentric\n",
      "341 ed\n",
      "310 eddie\n",
      "95 edgar\n",
      "441 edge\n",
      "82 edgy\n",
      "107 edie\n",
      "262 edited\n",
      "774 editing\n",
      "89 edition\n",
      "119 editor\n",
      "97 education\n",
      "83 educational\n",
      "204 edward\n",
      "141 eerie\n",
      "633 effect\n",
      "512 effective\n",
      "187 effectively\n",
      "2204 effects\n",
      "792 effort\n",
      "254 efforts\n",
      "128 ego\n",
      "221 eight\n",
      "101 eighties\n",
      "1866 either\n",
      "118 elaborate\n",
      "119 elderly\n",
      "93 elegant\n",
      "392 element\n",
      "783 elements\n",
      "99 elephant\n",
      "175 elizabeth\n",
      "122 ellen\n",
      "84 elm\n",
      "1998 else\n",
      "139 elsewhere\n",
      "153 elvira\n",
      "154 elvis\n",
      "158 em\n",
      "163 embarrassed\n",
      "226 embarrassing\n",
      "96 embarrassment\n",
      "122 emily\n",
      "202 emma\n",
      "396 emotion\n",
      "657 emotional\n",
      "241 emotionally\n",
      "389 emotions\n",
      "84 empathy\n",
      "97 emperor\n",
      "101 emphasis\n",
      "124 empire\n",
      "274 empty\n",
      "90 en\n",
      "175 encounter\n",
      "140 encounters\n",
      "5648 end\n",
      "139 endearing\n",
      "556 ended\n",
      "2358 ending\n",
      "104 endings\n",
      "235 endless\n",
      "984 ends\n",
      "99 endure\n",
      "104 enemies\n",
      "203 enemy\n",
      "317 energy\n",
      "93 engage\n",
      "110 engaged\n",
      "312 engaging\n",
      "305 england\n",
      "986 english\n",
      "1812 enjoy\n",
      "842 enjoyable\n",
      "1245 enjoyed\n",
      "162 enjoying\n",
      "150 enjoyment\n",
      "112 enjoys\n",
      "103 enormous\n",
      "3452 enough\n",
      "149 ensemble\n",
      "88 ensues\n",
      "196 enter\n",
      "104 enterprise\n",
      "131 enters\n",
      "172 entertain\n",
      "237 entertained\n",
      "1442 entertaining\n",
      "878 entertainment\n",
      "86 enthusiasm\n",
      "1460 entire\n",
      "532 entirely\n",
      "145 entry\n",
      "175 environment\n",
      "318 epic\n",
      "1659 episode\n",
      "938 episodes\n",
      "143 equal\n",
      "432 equally\n",
      "84 equipment\n",
      "87 equivalent\n",
      "79 er\n",
      "614 era\n",
      "265 eric\n",
      "201 erotic\n",
      "108 errors\n",
      "538 escape\n",
      "110 escaped\n",
      "163 escapes\n",
      "2535 especially\n",
      "138 essence\n",
      "162 essential\n",
      "257 essentially\n",
      "164 established\n",
      "128 estate\n",
      "103 et\n",
      "1212 etc\n",
      "92 ethan\n",
      "83 eugene\n",
      "216 europe\n",
      "284 european\n",
      "140 eva\n",
      "117 eve\n",
      "12646 even\n",
      "246 evening\n",
      "373 event\n",
      "910 events\n",
      "720 eventually\n",
      "5995 ever\n",
      "3978 every\n",
      "411 everybody\n",
      "167 everyday\n",
      "2222 everyone\n",
      "2321 everything\n",
      "190 everywhere\n",
      "223 evidence\n",
      "127 evident\n",
      "1448 evil\n",
      "468 ex\n",
      "189 exact\n",
      "995 exactly\n",
      "120 exaggerated\n",
      "79 examination\n",
      "1373 example\n",
      "181 examples\n",
      "2070 excellent\n",
      "1129 except\n",
      "401 exception\n",
      "148 exceptional\n",
      "86 exceptionally\n",
      "87 excessive\n",
      "80 exchange\n",
      "230 excited\n",
      "221 excitement\n",
      "515 exciting\n",
      "419 excuse\n",
      "241 executed\n",
      "189 execution\n",
      "132 executive\n",
      "130 exercise\n",
      "300 exist\n",
      "114 existed\n",
      "259 existence\n",
      "161 existent\n",
      "161 exists\n",
      "104 exotic\n",
      "1176 expect\n",
      "404 expectations\n",
      "704 expected\n",
      "588 expecting\n",
      "81 expedition\n",
      "140 expensive\n",
      "1059 experience\n",
      "192 experienced\n",
      "200 experiences\n",
      "170 experiment\n",
      "79 experimental\n",
      "90 experiments\n",
      "176 expert\n",
      "451 explain\n",
      "285 explained\n",
      "107 explaining\n",
      "193 explains\n",
      "290 explanation\n",
      "119 explicit\n",
      "233 exploitation\n",
      "92 exploration\n",
      "118 explore\n",
      "106 explored\n",
      "112 explosion\n",
      "109 explosions\n",
      "117 exposed\n",
      "82 exposure\n",
      "189 express\n",
      "83 expressed\n",
      "169 expression\n",
      "155 expressions\n",
      "115 extended\n",
      "171 extent\n",
      "315 extra\n",
      "173 extraordinary\n",
      "228 extras\n",
      "350 extreme\n",
      "1069 extremely\n",
      "849 eye\n",
      "133 eyed\n",
      "1216 eyes\n",
      "115 eyre\n",
      "178 fabulous\n",
      "1645 face\n",
      "204 faced\n",
      "345 faces\n",
      "177 facial\n",
      "96 facing\n",
      "3523 fact\n",
      "218 factor\n",
      "142 factory\n",
      "224 facts\n",
      "285 fail\n",
      "483 failed\n",
      "119 failing\n",
      "606 fails\n",
      "247 failure\n",
      "455 fair\n",
      "587 fairly\n",
      "214 fairy\n",
      "297 faith\n",
      "178 faithful\n",
      "473 fake\n",
      "122 falk\n",
      "770 fall\n",
      "165 fallen\n",
      "383 falling\n",
      "851 falls\n",
      "193 false\n",
      "230 fame\n",
      "538 familiar\n",
      "239 families\n",
      "3200 family\n",
      "771 famous\n",
      "1911 fan\n",
      "119 fancy\n",
      "1421 fans\n",
      "798 fantastic\n",
      "649 fantasy\n",
      "2978 far\n",
      "122 farce\n",
      "210 fare\n",
      "115 farm\n",
      "89 farrell\n",
      "88 fascinated\n",
      "391 fascinating\n",
      "341 fashion\n",
      "138 fashioned\n",
      "897 fast\n",
      "100 faster\n",
      "275 fat\n",
      "124 fatal\n",
      "271 fate\n",
      "2123 father\n",
      "240 fault\n",
      "97 faults\n",
      "250 favor\n",
      "1232 favorite\n",
      "187 favorites\n",
      "329 favourite\n",
      "121 fay\n",
      "153 fbi\n",
      "538 fear\n",
      "126 fears\n",
      "791 feature\n",
      "192 featured\n",
      "643 features\n",
      "277 featuring\n",
      "82 fed\n",
      "87 feed\n",
      "2950 feel\n",
      "1145 feeling\n",
      "395 feelings\n",
      "810 feels\n",
      "236 feet\n",
      "121 felix\n",
      "346 fell\n",
      "372 fellow\n",
      "1528 felt\n",
      "944 female\n",
      "84 feminist\n",
      "80 femme\n",
      "137 fest\n",
      "399 festival\n",
      "103 fetched\n",
      "116 fever\n",
      "661 fi\n",
      "133 fianc\n",
      "476 fiction\n",
      "188 fictional\n",
      "84 fido\n",
      "290 field\n",
      "79 fields\n",
      "127 fifteen\n",
      "1148 fight\n",
      "111 fighter\n",
      "607 fighting\n",
      "285 fights\n",
      "758 figure\n",
      "187 figured\n",
      "191 figures\n",
      "80 files\n",
      "231 fill\n",
      "551 filled\n",
      "40147 film\n",
      "763 filmed\n",
      "393 filming\n",
      "334 filmmaker\n",
      "566 filmmakers\n",
      "6887 films\n",
      "1329 final\n",
      "267 finale\n",
      "1536 finally\n",
      "105 financial\n",
      "4131 find\n",
      "358 finding\n",
      "948 finds\n",
      "1324 fine\n",
      "278 finest\n",
      "97 finger\n",
      "410 finish\n",
      "302 finished\n",
      "632 fire\n",
      "133 fired\n",
      "81 firm\n",
      "9061 first\n",
      "90 firstly\n",
      "157 fish\n",
      "129 fisher\n",
      "78 fishing\n",
      "489 fit\n",
      "215 fits\n",
      "137 fitting\n",
      "933 five\n",
      "100 fix\n",
      "78 flair\n",
      "149 flash\n",
      "176 flashback\n",
      "240 flashbacks\n",
      "577 flat\n",
      "140 flaw\n",
      "156 flawed\n",
      "125 flawless\n",
      "362 flaws\n",
      "247 flesh\n",
      "1258 flick\n",
      "357 flicks\n",
      "105 flies\n",
      "177 flight\n",
      "89 floating\n",
      "281 floor\n",
      "108 flop\n",
      "107 florida\n",
      "161 flow\n",
      "232 fly\n",
      "357 flying\n",
      "147 flynn\n",
      "509 focus\n",
      "203 focused\n",
      "182 focuses\n",
      "106 focusing\n",
      "116 folk\n",
      "346 folks\n",
      "785 follow\n",
      "373 followed\n",
      "564 following\n",
      "499 follows\n",
      "102 fond\n",
      "111 fonda\n",
      "333 food\n",
      "219 fool\n",
      "95 fooled\n",
      "259 foot\n",
      "651 footage\n",
      "221 football\n",
      "120 forbidden\n",
      "514 force\n",
      "660 forced\n",
      "272 forces\n",
      "304 ford\n",
      "237 foreign\n",
      "191 forest\n",
      "392 forever\n",
      "716 forget\n",
      "205 forgettable\n",
      "131 forgive\n",
      "178 forgot\n",
      "353 forgotten\n",
      "765 form\n",
      "179 format\n",
      "509 former\n",
      "100 forms\n",
      "252 formula\n",
      "98 formulaic\n",
      "190 forth\n",
      "159 fortunately\n",
      "144 fortune\n",
      "103 forty\n",
      "651 forward\n",
      "185 foster\n",
      "81 fought\n",
      "111 foul\n",
      "2572 found\n",
      "912 four\n",
      "176 fourth\n",
      "336 fox\n",
      "246 frame\n",
      "234 france\n",
      "154 franchise\n",
      "91 francis\n",
      "123 francisco\n",
      "98 franco\n",
      "457 frank\n",
      "92 frankenstein\n",
      "90 frankie\n",
      "256 frankly\n",
      "117 freak\n",
      "303 fred\n",
      "287 freddy\n",
      "697 free\n",
      "242 freedom\n",
      "188 freeman\n",
      "789 french\n",
      "87 frequent\n",
      "157 frequently\n",
      "376 fresh\n",
      "198 friday\n",
      "1442 friend\n",
      "193 friendly\n",
      "1788 friends\n",
      "283 friendship\n",
      "195 frightening\n",
      "595 front\n",
      "125 frustrated\n",
      "79 frustrating\n",
      "103 frustration\n",
      "284 fu\n",
      "114 fulci\n",
      "1779 full\n",
      "83 fuller\n",
      "426 fully\n",
      "2694 fun\n",
      "106 funeral\n",
      "173 funnier\n",
      "358 funniest\n",
      "4288 funny\n",
      "83 furious\n",
      "106 furthermore\n",
      "83 fury\n",
      "900 future\n",
      "120 futuristic\n",
      "121 fx\n",
      "90 gabriel\n",
      "110 gadget\n",
      "140 gag\n",
      "262 gags\n",
      "143 gain\n",
      "1281 game\n",
      "340 games\n",
      "121 gandhi\n",
      "421 gang\n",
      "259 gangster\n",
      "94 gangsters\n",
      "459 garbage\n",
      "103 garbo\n",
      "109 garden\n",
      "272 gary\n",
      "195 gas\n",
      "1216 gave\n",
      "609 gay\n",
      "361 gem\n",
      "90 gender\n",
      "277 gene\n",
      "765 general\n",
      "468 generally\n",
      "85 generated\n",
      "235 generation\n",
      "80 generations\n",
      "105 generic\n",
      "124 generous\n",
      "452 genius\n",
      "1254 genre\n",
      "113 genres\n",
      "116 gentle\n",
      "84 gentleman\n",
      "256 genuine\n",
      "251 genuinely\n",
      "862 george\n",
      "91 gerard\n",
      "516 german\n",
      "91 germans\n",
      "227 germany\n",
      "9310 get\n",
      "3204 gets\n",
      "1627 getting\n",
      "484 ghost\n",
      "181 ghosts\n",
      "118 giallo\n",
      "379 giant\n",
      "131 gift\n",
      "84 gifted\n",
      "107 ginger\n",
      "2853 girl\n",
      "636 girlfriend\n",
      "1211 girls\n",
      "3376 give\n",
      "1846 given\n",
      "1576 gives\n",
      "839 giving\n",
      "450 glad\n",
      "157 glass\n",
      "80 glasses\n",
      "118 glenn\n",
      "149 glimpse\n",
      "94 global\n",
      "104 glorious\n",
      "147 glory\n",
      "142 glover\n",
      "5157 go\n",
      "137 goal\n",
      "1207 god\n",
      "134 godfather\n",
      "105 godzilla\n",
      "2441 goes\n",
      "4101 going\n",
      "295 gold\n",
      "123 goldberg\n",
      "259 golden\n",
      "754 gone\n",
      "241 gonna\n",
      "15140 good\n",
      "106 goodness\n",
      "161 goofy\n",
      "245 gordon\n",
      "1038 gore\n",
      "368 gorgeous\n",
      "234 gory\n",
      "3583 got\n",
      "136 gothic\n",
      "129 gotta\n",
      "286 gotten\n",
      "424 government\n",
      "109 grab\n",
      "80 grabs\n",
      "328 grace\n",
      "462 grade\n",
      "118 gradually\n",
      "86 graham\n",
      "307 grand\n",
      "98 grandfather\n",
      "127 grandmother\n",
      "262 grant\n",
      "201 granted\n",
      "239 graphic\n",
      "169 graphics\n",
      "97 grasp\n",
      "237 gratuitous\n",
      "189 grave\n",
      "121 gray\n",
      "89 grayson\n",
      "9058 great\n",
      "174 greater\n",
      "745 greatest\n",
      "154 greatly\n",
      "82 greatness\n",
      "86 greed\n",
      "91 greedy\n",
      "113 greek\n",
      "404 green\n",
      "83 greg\n",
      "79 gregory\n",
      "250 grew\n",
      "158 grey\n",
      "79 grief\n",
      "102 griffith\n",
      "188 grim\n",
      "103 grinch\n",
      "150 gripping\n",
      "196 gritty\n",
      "180 gross\n",
      "353 ground\n",
      "1034 group\n",
      "108 groups\n",
      "224 grow\n",
      "297 growing\n",
      "252 grown\n",
      "134 grows\n",
      "163 gruesome\n",
      "91 guarantee\n",
      "165 guard\n",
      "1310 guess\n",
      "99 guessed\n",
      "151 guessing\n",
      "133 guest\n",
      "125 guide\n",
      "142 guilt\n",
      "198 guilty\n",
      "563 gun\n",
      "94 gundam\n",
      "283 guns\n",
      "143 guts\n",
      "3035 guy\n",
      "1304 guys\n",
      "184 ha\n",
      "505 hair\n",
      "105 hal\n",
      "2094 half\n",
      "173 halfway\n",
      "221 hall\n",
      "244 halloween\n",
      "84 ham\n",
      "113 hamilton\n",
      "145 hamlet\n",
      "111 hammer\n",
      "1257 hand\n",
      "181 handed\n",
      "127 handful\n",
      "167 handle\n",
      "209 handled\n",
      "632 hands\n",
      "228 handsome\n",
      "143 hang\n",
      "215 hanging\n",
      "105 hank\n",
      "140 hanks\n",
      "1043 happen\n",
      "1076 happened\n",
      "383 happening\n",
      "1080 happens\n",
      "152 happily\n",
      "186 happiness\n",
      "965 happy\n",
      "2668 hard\n",
      "102 hardcore\n",
      "112 harder\n",
      "613 hardly\n",
      "177 hardy\n",
      "81 harm\n",
      "256 harris\n",
      "461 harry\n",
      "200 harsh\n",
      "106 hart\n",
      "101 hartley\n",
      "107 harvey\n",
      "212 hat\n",
      "789 hate\n",
      "296 hated\n",
      "104 hates\n",
      "121 hatred\n",
      "217 haunted\n",
      "229 haunting\n",
      "79 hawke\n",
      "108 hbo\n",
      "1541 head\n",
      "169 headed\n",
      "291 heads\n",
      "137 health\n",
      "733 hear\n",
      "1111 heard\n",
      "231 hearing\n",
      "1328 heart\n",
      "225 hearted\n",
      "135 hearts\n",
      "128 heat\n",
      "320 heaven\n",
      "180 heavily\n",
      "492 heavy\n",
      "222 heck\n",
      "85 heights\n",
      "391 held\n",
      "152 helen\n",
      "97 helicopter\n",
      "1025 hell\n",
      "90 hello\n",
      "1895 help\n",
      "324 helped\n",
      "176 helping\n",
      "360 helps\n",
      "155 hence\n",
      "407 henry\n",
      "1056 hero\n",
      "318 heroes\n",
      "115 heroic\n",
      "291 heroine\n",
      "136 heston\n",
      "409 hey\n",
      "342 hidden\n",
      "210 hide\n",
      "103 hideous\n",
      "144 hiding\n",
      "2161 high\n",
      "289 higher\n",
      "106 highest\n",
      "202 highlight\n",
      "125 highlights\n",
      "1147 highly\n",
      "973 hilarious\n",
      "86 hilariously\n",
      "243 hill\n",
      "152 hills\n",
      "147 hint\n",
      "103 hints\n",
      "181 hip\n",
      "84 hippie\n",
      "130 hire\n",
      "188 hired\n",
      "407 historical\n",
      "86 historically\n",
      "1332 history\n",
      "1088 hit\n",
      "209 hitchcock\n",
      "305 hitler\n",
      "272 hits\n",
      "137 hitting\n",
      "126 ho\n",
      "188 hoffman\n",
      "545 hold\n",
      "209 holding\n",
      "300 holds\n",
      "167 hole\n",
      "367 holes\n",
      "148 holiday\n",
      "113 hollow\n",
      "107 holly\n",
      "1907 hollywood\n",
      "163 holmes\n",
      "113 holy\n",
      "133 homage\n",
      "1877 home\n",
      "140 homeless\n",
      "103 homer\n",
      "90 homosexual\n",
      "481 honest\n",
      "453 honestly\n",
      "99 honesty\n",
      "191 hong\n",
      "173 honor\n",
      "162 hood\n",
      "99 hook\n",
      "139 hooked\n",
      "95 hop\n",
      "1447 hope\n",
      "144 hoped\n",
      "214 hopefully\n",
      "82 hopeless\n",
      "273 hopes\n",
      "407 hoping\n",
      "98 hopper\n",
      "134 horrendous\n",
      "1201 horrible\n",
      "214 horribly\n",
      "116 horrid\n",
      "159 horrific\n",
      "95 horrifying\n",
      "3592 horror\n",
      "124 horrors\n",
      "298 horse\n",
      "145 horses\n",
      "356 hospital\n",
      "146 host\n",
      "688 hot\n",
      "397 hotel\n",
      "1188 hour\n",
      "983 hours\n",
      "2184 house\n",
      "84 household\n",
      "105 houses\n",
      "260 howard\n",
      "3537 however\n",
      "141 hudson\n",
      "944 huge\n",
      "111 hugh\n",
      "138 huh\n",
      "1596 human\n",
      "282 humanity\n",
      "319 humans\n",
      "94 humble\n",
      "1311 humor\n",
      "263 humorous\n",
      "441 humour\n",
      "149 hundred\n",
      "151 hundreds\n",
      "98 hung\n",
      "222 hunt\n",
      "239 hunter\n",
      "90 hunters\n",
      "143 hunting\n",
      "382 hurt\n",
      "99 hurts\n",
      "1026 husband\n",
      "79 husbands\n",
      "96 hyde\n",
      "134 hype\n",
      "117 hysterical\n",
      "132 ian\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "282 ice\n",
      "85 icon\n",
      "2043 idea\n",
      "103 ideal\n",
      "595 ideas\n",
      "130 identify\n",
      "256 identity\n",
      "185 idiot\n",
      "145 idiotic\n",
      "118 idiots\n",
      "96 ignorant\n",
      "177 ignore\n",
      "120 ignored\n",
      "366 ii\n",
      "135 iii\n",
      "290 ill\n",
      "90 illegal\n",
      "89 illness\n",
      "98 illogical\n",
      "92 im\n",
      "379 image\n",
      "186 imagery\n",
      "472 images\n",
      "356 imagination\n",
      "140 imaginative\n",
      "737 imagine\n",
      "116 imagined\n",
      "702 imdb\n",
      "85 imitation\n",
      "80 immediate\n",
      "462 immediately\n",
      "102 immensely\n",
      "374 impact\n",
      "108 implausible\n",
      "134 importance\n",
      "931 important\n",
      "127 importantly\n",
      "496 impossible\n",
      "104 impress\n",
      "352 impressed\n",
      "405 impression\n",
      "500 impressive\n",
      "97 improve\n",
      "115 improved\n",
      "85 improvement\n",
      "79 inability\n",
      "95 inane\n",
      "90 inappropriate\n",
      "115 incident\n",
      "375 include\n",
      "272 included\n",
      "321 includes\n",
      "1052 including\n",
      "138 incoherent\n",
      "87 incompetent\n",
      "89 incomprehensible\n",
      "131 increasingly\n",
      "563 incredible\n",
      "626 incredibly\n",
      "722 indeed\n",
      "314 independent\n",
      "179 india\n",
      "405 indian\n",
      "122 indians\n",
      "181 indie\n",
      "237 individual\n",
      "126 individuals\n",
      "83 inducing\n",
      "80 indulgent\n",
      "345 industry\n",
      "175 inept\n",
      "131 inevitable\n",
      "82 inevitably\n",
      "148 infamous\n",
      "94 inferior\n",
      "211 influence\n",
      "107 influenced\n",
      "335 information\n",
      "107 ingredients\n",
      "209 initial\n",
      "178 initially\n",
      "210 inner\n",
      "154 innocence\n",
      "416 innocent\n",
      "110 innovative\n",
      "241 insane\n",
      "600 inside\n",
      "187 insight\n",
      "162 inspector\n",
      "169 inspiration\n",
      "348 inspired\n",
      "135 inspiring\n",
      "121 installment\n",
      "289 instance\n",
      "104 instant\n",
      "128 instantly\n",
      "2190 instead\n",
      "86 instinct\n",
      "213 insult\n",
      "128 insulting\n",
      "82 integrity\n",
      "176 intellectual\n",
      "327 intelligence\n",
      "534 intelligent\n",
      "388 intended\n",
      "344 intense\n",
      "158 intensity\n",
      "117 intent\n",
      "132 intention\n",
      "90 intentionally\n",
      "161 intentions\n",
      "99 interaction\n",
      "1033 interest\n",
      "650 interested\n",
      "3128 interesting\n",
      "82 interests\n",
      "266 international\n",
      "155 internet\n",
      "162 interpretation\n",
      "180 interview\n",
      "158 interviews\n",
      "88 intimate\n",
      "109 intrigue\n",
      "120 intrigued\n",
      "301 intriguing\n",
      "84 introduce\n",
      "313 introduced\n",
      "99 introduces\n",
      "168 introduction\n",
      "91 invasion\n",
      "80 invented\n",
      "97 inventive\n",
      "112 investigate\n",
      "128 investigation\n",
      "200 invisible\n",
      "99 involve\n",
      "1076 involved\n",
      "114 involvement\n",
      "224 involves\n",
      "465 involving\n",
      "93 iran\n",
      "85 iraq\n",
      "115 ireland\n",
      "195 irish\n",
      "94 iron\n",
      "162 ironic\n",
      "123 ironically\n",
      "148 irony\n",
      "85 irrelevant\n",
      "231 irritating\n",
      "535 island\n",
      "98 isolated\n",
      "85 israel\n",
      "287 issue\n",
      "418 issues\n",
      "527 italian\n",
      "153 italy\n",
      "925 jack\n",
      "232 jackie\n",
      "340 jackson\n",
      "172 jail\n",
      "160 jake\n",
      "1068 james\n",
      "128 jamie\n",
      "657 jane\n",
      "292 japan\n",
      "714 japanese\n",
      "330 jason\n",
      "90 jaw\n",
      "87 jaws\n",
      "157 jay\n",
      "109 jazz\n",
      "121 jealous\n",
      "353 jean\n",
      "300 jeff\n",
      "107 jeffrey\n",
      "259 jennifer\n",
      "94 jenny\n",
      "138 jeremy\n",
      "134 jerk\n",
      "380 jerry\n",
      "130 jesse\n",
      "171 jessica\n",
      "289 jesus\n",
      "128 jet\n",
      "164 jewish\n",
      "489 jim\n",
      "274 jimmy\n",
      "302 joan\n",
      "2274 job\n",
      "185 jobs\n",
      "690 joe\n",
      "81 joel\n",
      "128 joey\n",
      "2208 john\n",
      "312 johnny\n",
      "191 johnson\n",
      "167 join\n",
      "90 joined\n",
      "623 joke\n",
      "977 jokes\n",
      "187 jon\n",
      "99 jonathan\n",
      "406 jones\n",
      "232 joseph\n",
      "83 josh\n",
      "112 journalist\n",
      "439 journey\n",
      "293 joy\n",
      "302 jr\n",
      "278 judge\n",
      "102 judging\n",
      "100 judy\n",
      "188 julia\n",
      "86 julian\n",
      "174 julie\n",
      "300 jump\n",
      "92 jumped\n",
      "125 jumping\n",
      "160 jumps\n",
      "90 june\n",
      "188 jungle\n",
      "91 junior\n",
      "190 junk\n",
      "418 justice\n",
      "98 justify\n",
      "89 justin\n",
      "110 juvenile\n",
      "135 kane\n",
      "79 kansas\n",
      "114 kapoor\n",
      "115 karen\n",
      "148 karloff\n",
      "300 kate\n",
      "91 kay\n",
      "308 keaton\n",
      "1601 keep\n",
      "276 keeping\n",
      "642 keeps\n",
      "88 keith\n",
      "429 kelly\n",
      "123 ken\n",
      "108 kennedy\n",
      "116 kenneth\n",
      "750 kept\n",
      "289 kevin\n",
      "425 key\n",
      "99 khan\n",
      "264 kick\n",
      "98 kicked\n",
      "92 kicking\n",
      "133 kicks\n",
      "1199 kid\n",
      "109 kidding\n",
      "128 kidnapped\n",
      "1844 kids\n",
      "1234 kill\n",
      "1111 killed\n",
      "1455 killer\n",
      "245 killers\n",
      "694 killing\n",
      "132 killings\n",
      "530 kills\n",
      "209 kim\n",
      "2783 kind\n",
      "275 kinda\n",
      "191 kinds\n",
      "999 king\n",
      "95 kingdom\n",
      "114 kirk\n",
      "171 kiss\n",
      "83 kissing\n",
      "118 kitchen\n",
      "897 knew\n",
      "147 knife\n",
      "139 knock\n",
      "6166 know\n",
      "447 knowing\n",
      "283 knowledge\n",
      "1080 known\n",
      "901 knows\n",
      "78 kolchak\n",
      "270 kong\n",
      "131 korean\n",
      "130 kubrick\n",
      "128 kudos\n",
      "80 kumar\n",
      "243 kung\n",
      "83 kurosawa\n",
      "149 kurt\n",
      "96 kyle\n",
      "552 la\n",
      "131 lab\n",
      "1058 lack\n",
      "121 lacked\n",
      "277 lacking\n",
      "79 lackluster\n",
      "365 lacks\n",
      "295 ladies\n",
      "848 lady\n",
      "160 laid\n",
      "250 lake\n",
      "742 lame\n",
      "362 land\n",
      "81 landing\n",
      "115 landscape\n",
      "86 landscapes\n",
      "170 lane\n",
      "529 language\n",
      "556 large\n",
      "227 largely\n",
      "143 larger\n",
      "183 larry\n",
      "2933 last\n",
      "99 lasted\n",
      "1211 late\n",
      "96 lately\n",
      "2200 later\n",
      "201 latest\n",
      "94 latin\n",
      "362 latter\n",
      "1374 laugh\n",
      "422 laughable\n",
      "82 laughably\n",
      "366 laughed\n",
      "528 laughing\n",
      "658 laughs\n",
      "244 laughter\n",
      "166 laura\n",
      "121 laurel\n",
      "510 law\n",
      "111 lawrence\n",
      "96 laws\n",
      "210 lawyer\n",
      "93 lay\n",
      "168 lazy\n",
      "143 le\n",
      "1310 lead\n",
      "248 leader\n",
      "622 leading\n",
      "750 leads\n",
      "176 league\n",
      "720 learn\n",
      "254 learned\n",
      "175 learning\n",
      "226 learns\n",
      "3112 least\n",
      "1107 leave\n",
      "683 leaves\n",
      "482 leaving\n",
      "330 led\n",
      "814 lee\n",
      "2125 left\n",
      "112 leg\n",
      "80 legacy\n",
      "87 legal\n",
      "301 legend\n",
      "192 legendary\n",
      "160 legs\n",
      "109 lemmon\n",
      "99 lena\n",
      "336 length\n",
      "89 lengthy\n",
      "96 leo\n",
      "81 leon\n",
      "111 leonard\n",
      "84 les\n",
      "205 lesbian\n",
      "173 leslie\n",
      "2002 less\n",
      "165 lesser\n",
      "256 lesson\n",
      "151 lessons\n",
      "2341 let\n",
      "334 lets\n",
      "139 letter\n",
      "95 letters\n",
      "146 letting\n",
      "963 level\n",
      "235 levels\n",
      "276 lewis\n",
      "114 li\n",
      "121 liberal\n",
      "140 library\n",
      "171 lie\n",
      "290 lies\n",
      "6628 life\n",
      "104 lifestyle\n",
      "187 lifetime\n",
      "976 light\n",
      "361 lighting\n",
      "181 lights\n",
      "382 likable\n",
      "20274 like\n",
      "1516 liked\n",
      "422 likely\n",
      "466 likes\n",
      "89 likewise\n",
      "108 liking\n",
      "132 lily\n",
      "303 limited\n",
      "99 limits\n",
      "176 lincoln\n",
      "105 linda\n",
      "1870 line\n",
      "206 liners\n",
      "1552 lines\n",
      "100 link\n",
      "166 lion\n",
      "109 lips\n",
      "186 lisa\n",
      "585 list\n",
      "125 listed\n",
      "331 listen\n",
      "187 listening\n",
      "109 lit\n",
      "468 literally\n",
      "91 literature\n",
      "6435 little\n",
      "1552 live\n",
      "382 lived\n",
      "88 lively\n",
      "1392 lives\n",
      "1063 living\n",
      "139 lloyd\n",
      "116 load\n",
      "85 loaded\n",
      "100 loads\n",
      "877 local\n",
      "337 location\n",
      "263 locations\n",
      "167 locked\n",
      "79 logan\n",
      "250 logic\n",
      "122 logical\n",
      "106 lol\n",
      "460 london\n",
      "87 lone\n",
      "188 lonely\n",
      "3449 long\n",
      "477 longer\n",
      "4145 look\n",
      "1010 looked\n",
      "2483 looking\n",
      "2413 looks\n",
      "278 loose\n",
      "118 loosely\n",
      "348 lord\n",
      "123 los\n",
      "345 lose\n",
      "140 loser\n",
      "88 losers\n",
      "263 loses\n",
      "220 losing\n",
      "271 loss\n",
      "1554 lost\n",
      "3979 lot\n",
      "799 lots\n",
      "124 lou\n",
      "436 loud\n",
      "225 louis\n",
      "90 louise\n",
      "220 lousy\n",
      "144 lovable\n",
      "6454 love\n",
      "1428 loved\n",
      "424 lovely\n",
      "397 lover\n",
      "291 lovers\n",
      "404 loves\n",
      "316 loving\n",
      "1799 low\n",
      "214 lower\n",
      "92 lowest\n",
      "100 loyal\n",
      "80 loyalty\n",
      "143 lucas\n",
      "262 luck\n",
      "130 luckily\n",
      "258 lucky\n",
      "160 lucy\n",
      "176 ludicrous\n",
      "195 lugosi\n",
      "258 luke\n",
      "100 lumet\n",
      "92 lundgren\n",
      "108 lust\n",
      "146 lying\n",
      "278 lynch\n",
      "104 lyrics\n",
      "98 macarthur\n",
      "324 machine\n",
      "113 machines\n",
      "102 macy\n",
      "499 mad\n",
      "8362 made\n",
      "162 madness\n",
      "111 madonna\n",
      "131 mafia\n",
      "136 magazine\n",
      "105 maggie\n",
      "468 magic\n",
      "194 magical\n",
      "261 magnificent\n",
      "84 maid\n",
      "90 mail\n",
      "2264 main\n",
      "393 mainly\n",
      "202 mainstream\n",
      "93 maintain\n",
      "927 major\n",
      "238 majority\n",
      "8023 make\n",
      "167 maker\n",
      "490 makers\n",
      "4202 makes\n",
      "206 makeup\n",
      "2961 making\n",
      "666 male\n",
      "93 mall\n",
      "80 malone\n",
      "5982 man\n",
      "272 manage\n",
      "425 managed\n",
      "159 manager\n",
      "583 manages\n",
      "121 manhattan\n",
      "83 maniac\n",
      "88 manipulative\n",
      "89 mankind\n",
      "124 mann\n",
      "410 manner\n",
      "161 mansion\n",
      "6675 many\n",
      "84 map\n",
      "79 marc\n",
      "103 march\n",
      "114 margaret\n",
      "175 maria\n",
      "253 marie\n",
      "99 mario\n",
      "86 marion\n",
      "645 mark\n",
      "218 market\n",
      "84 marketing\n",
      "120 marks\n",
      "417 marriage\n",
      "590 married\n",
      "226 marry\n",
      "93 mars\n",
      "80 marshall\n",
      "328 martial\n",
      "367 martin\n",
      "90 marty\n",
      "160 marvelous\n",
      "567 mary\n",
      "221 mask\n",
      "79 masks\n",
      "161 mass\n",
      "145 massacre\n",
      "83 masses\n",
      "192 massive\n",
      "453 master\n",
      "87 masterful\n",
      "612 masterpiece\n",
      "85 masterpieces\n",
      "107 masters\n",
      "588 match\n",
      "89 matched\n",
      "101 matches\n",
      "122 mate\n",
      "760 material\n",
      "197 matrix\n",
      "226 matt\n",
      "1127 matter\n",
      "224 matters\n",
      "156 matthau\n",
      "115 matthew\n",
      "182 mature\n",
      "215 max\n",
      "3386 may\n",
      "2340 maybe\n",
      "96 mayor\n",
      "83 mclaglen\n",
      "1683 mean\n",
      "473 meaning\n",
      "144 meaningful\n",
      "108 meaningless\n",
      "761 means\n",
      "614 meant\n",
      "249 meanwhile\n",
      "104 measure\n",
      "127 meat\n",
      "85 mechanical\n",
      "305 media\n",
      "152 medical\n",
      "365 mediocre\n",
      "132 medium\n",
      "668 meet\n",
      "237 meeting\n",
      "677 meets\n",
      "119 mel\n",
      "188 melodrama\n",
      "122 melodramatic\n",
      "101 melting\n",
      "327 member\n",
      "552 members\n",
      "666 memorable\n",
      "279 memories\n",
      "306 memory\n",
      "1909 men\n",
      "100 menace\n",
      "126 menacing\n",
      "309 mental\n",
      "160 mentally\n",
      "811 mention\n",
      "564 mentioned\n",
      "85 mentioning\n",
      "85 mentions\n",
      "181 mere\n",
      "359 merely\n",
      "112 merit\n",
      "81 merits\n",
      "83 meryl\n",
      "641 mess\n",
      "829 message\n",
      "131 messages\n",
      "91 messed\n",
      "286 met\n",
      "189 metal\n",
      "81 metaphor\n",
      "103 method\n",
      "85 methods\n",
      "184 mexican\n",
      "185 mexico\n",
      "199 mgm\n",
      "1333 michael\n",
      "172 michelle\n",
      "108 mickey\n",
      "319 mid\n",
      "956 middle\n",
      "178 midnight\n",
      "2919 might\n",
      "87 mighty\n",
      "132 miike\n",
      "281 mike\n",
      "133 mild\n",
      "174 mildly\n",
      "115 mildred\n",
      "123 mile\n",
      "260 miles\n",
      "462 military\n",
      "79 milk\n",
      "104 mill\n",
      "169 miller\n",
      "397 million\n",
      "79 millionaire\n",
      "149 millions\n",
      "91 min\n",
      "1995 mind\n",
      "162 minded\n",
      "154 mindless\n",
      "185 minds\n",
      "275 mine\n",
      "213 mini\n",
      "119 minimal\n",
      "84 minimum\n",
      "400 minor\n",
      "789 minute\n",
      "2952 minutes\n",
      "94 miracle\n",
      "168 mirror\n",
      "143 miscast\n",
      "100 miserable\n",
      "124 miserably\n",
      "92 misery\n",
      "883 miss\n",
      "565 missed\n",
      "118 misses\n",
      "594 missing\n",
      "265 mission\n",
      "426 mistake\n",
      "108 mistaken\n",
      "200 mistakes\n",
      "88 mistress\n",
      "125 mitchell\n",
      "367 mix\n",
      "287 mixed\n",
      "103 mixture\n",
      "83 miyazaki\n",
      "83 mm\n",
      "157 mob\n",
      "240 model\n",
      "105 models\n",
      "929 modern\n",
      "122 modesty\n",
      "89 molly\n",
      "367 mom\n",
      "1112 moment\n",
      "1663 moments\n",
      "84 mon\n",
      "2362 money\n",
      "85 monk\n",
      "132 monkey\n",
      "98 monkeys\n",
      "655 monster\n",
      "277 monsters\n",
      "103 montage\n",
      "95 montana\n",
      "148 month\n",
      "272 months\n",
      "432 mood\n",
      "98 moody\n",
      "296 moon\n",
      "227 moore\n",
      "366 moral\n",
      "122 morality\n",
      "275 morgan\n",
      "266 morning\n",
      "83 moronic\n",
      "98 morris\n",
      "941 mostly\n",
      "1524 mother\n",
      "449 motion\n",
      "122 motivation\n",
      "95 motivations\n",
      "103 motives\n",
      "196 mountain\n",
      "110 mountains\n",
      "168 mouse\n",
      "332 mouth\n",
      "727 move\n",
      "322 moved\n",
      "206 movement\n",
      "115 movements\n",
      "530 moves\n",
      "44031 movie\n",
      "7663 movies\n",
      "854 moving\n",
      "1448 mr\n",
      "260 mrs\n",
      "350 ms\n",
      "180 mst\n",
      "86 mtv\n",
      "9765 much\n",
      "140 multi\n",
      "190 multiple\n",
      "108 mummy\n",
      "86 mundane\n",
      "1063 murder\n",
      "260 murdered\n",
      "179 murderer\n",
      "109 murderous\n",
      "367 murders\n",
      "213 murphy\n",
      "81 murray\n",
      "96 museum\n",
      "3056 music\n",
      "992 musical\n",
      "175 musicals\n",
      "85 muslim\n",
      "3249 must\n",
      "97 myers\n",
      "105 mysteries\n",
      "404 mysterious\n",
      "850 mystery\n",
      "98 nail\n",
      "218 naive\n",
      "437 naked\n",
      "1604 name\n",
      "799 named\n",
      "96 namely\n",
      "391 names\n",
      "227 nancy\n",
      "192 narration\n",
      "424 narrative\n",
      "125 narrator\n",
      "342 nasty\n",
      "82 nathan\n",
      "186 nation\n",
      "255 national\n",
      "238 native\n",
      "463 natural\n",
      "267 naturally\n",
      "711 nature\n",
      "162 navy\n",
      "190 nazi\n",
      "114 nazis\n",
      "145 nd\n",
      "824 near\n",
      "109 nearby\n",
      "815 nearly\n",
      "149 neat\n",
      "180 necessarily\n",
      "324 necessary\n",
      "141 neck\n",
      "182 ned\n",
      "1807 need\n",
      "683 needed\n",
      "162 needless\n",
      "841 needs\n",
      "352 negative\n",
      "139 neighbor\n",
      "143 neighborhood\n",
      "89 neighbors\n",
      "134 neil\n",
      "537 neither\n",
      "152 nelson\n",
      "112 neo\n",
      "84 nephew\n",
      "83 nerd\n",
      "97 nervous\n",
      "190 network\n",
      "6484 never\n",
      "236 nevertheless\n",
      "4310 new\n",
      "89 newly\n",
      "79 newman\n",
      "331 news\n",
      "113 newspaper\n",
      "1716 next\n",
      "2012 nice\n",
      "299 nicely\n",
      "92 nicholas\n",
      "113 nicholson\n",
      "293 nick\n",
      "82 nicole\n",
      "2163 night\n",
      "318 nightmare\n",
      "108 nightmares\n",
      "130 nights\n",
      "161 nine\n",
      "86 ninja\n",
      "100 niro\n",
      "152 noble\n",
      "452 nobody\n",
      "419 noir\n",
      "137 noise\n",
      "222 nominated\n",
      "98 nomination\n",
      "898 non\n",
      "1032 none\n",
      "163 nonetheless\n",
      "286 nonsense\n",
      "82 nonsensical\n",
      "457 normal\n",
      "304 normally\n",
      "127 norman\n",
      "213 north\n",
      "146 nose\n",
      "91 nostalgia\n",
      "97 nostalgic\n",
      "168 notable\n",
      "119 notably\n",
      "199 notch\n",
      "718 note\n",
      "144 noted\n",
      "123 notes\n",
      "4290 nothing\n",
      "366 notice\n",
      "272 noticed\n",
      "104 notion\n",
      "172 notorious\n",
      "86 novak\n",
      "958 novel\n",
      "168 novels\n",
      "167 nowadays\n",
      "443 nowhere\n",
      "129 nuclear\n",
      "198 nude\n",
      "596 nudity\n",
      "1006 number\n",
      "403 numbers\n",
      "270 numerous\n",
      "125 nurse\n",
      "88 nuts\n",
      "86 nyc\n",
      "142 object\n",
      "79 objective\n",
      "164 obnoxious\n",
      "119 obscure\n",
      "235 obsessed\n",
      "162 obsession\n",
      "1066 obvious\n",
      "1163 obviously\n",
      "106 occasion\n",
      "193 occasional\n",
      "256 occasionally\n",
      "112 occur\n",
      "114 occurred\n",
      "131 occurs\n",
      "103 ocean\n",
      "582 odd\n",
      "160 oddly\n",
      "108 odds\n",
      "104 offended\n",
      "211 offensive\n",
      "378 offer\n",
      "192 offered\n",
      "109 offering\n",
      "346 offers\n",
      "567 office\n",
      "280 officer\n",
      "107 officers\n",
      "106 official\n",
      "1601 often\n",
      "1449 oh\n",
      "142 oil\n",
      "1016 ok\n",
      "706 okay\n",
      "4525 old\n",
      "656 older\n",
      "220 oliver\n",
      "107 olivier\n",
      "95 ollie\n",
      "85 omen\n",
      "26788 one\n",
      "953 ones\n",
      "89 online\n",
      "328 onto\n",
      "664 open\n",
      "155 opened\n",
      "979 opening\n",
      "259 opens\n",
      "396 opera\n",
      "90 operation\n",
      "959 opinion\n",
      "89 opinions\n",
      "85 opportunities\n",
      "393 opportunity\n",
      "122 opposed\n",
      "266 opposite\n",
      "85 orange\n",
      "951 order\n",
      "79 ordered\n",
      "109 orders\n",
      "267 ordinary\n",
      "3376 original\n",
      "171 originality\n",
      "290 originally\n",
      "85 orleans\n",
      "96 orson\n",
      "861 oscar\n",
      "146 oscars\n",
      "87 othello\n",
      "1595 others\n",
      "670 otherwise\n",
      "114 ought\n",
      "118 outcome\n",
      "104 outer\n",
      "101 outfit\n",
      "124 outrageous\n",
      "596 outside\n",
      "417 outstanding\n",
      "87 overacting\n",
      "1434 overall\n",
      "154 overcome\n",
      "120 overdone\n",
      "86 overlook\n",
      "128 overlooked\n",
      "249 overly\n",
      "114 overrated\n",
      "109 overwhelming\n",
      "87 owen\n",
      "263 owner\n",
      "102 oz\n",
      "550 pace\n",
      "300 paced\n",
      "295 pacing\n",
      "202 pacino\n",
      "152 pack\n",
      "82 package\n",
      "157 packed\n",
      "390 page\n",
      "358 paid\n",
      "379 pain\n",
      "417 painful\n",
      "240 painfully\n",
      "192 paint\n",
      "102 painted\n",
      "136 painting\n",
      "241 pair\n",
      "91 pal\n",
      "84 palace\n",
      "79 palance\n",
      "108 palma\n",
      "94 paltrow\n",
      "85 pamela\n",
      "103 pan\n",
      "116 panic\n",
      "94 pants\n",
      "226 paper\n",
      "229 par\n",
      "89 parallel\n",
      "93 paranoia\n",
      "119 parent\n",
      "762 parents\n",
      "405 paris\n",
      "371 park\n",
      "243 parker\n",
      "246 parody\n",
      "4042 part\n",
      "730 particular\n",
      "1079 particularly\n",
      "87 parties\n",
      "127 partly\n",
      "277 partner\n",
      "1191 parts\n",
      "550 party\n",
      "420 pass\n",
      "80 passable\n",
      "246 passed\n",
      "106 passes\n",
      "191 passing\n",
      "296 passion\n",
      "99 passionate\n",
      "1263 past\n",
      "142 pat\n",
      "197 path\n",
      "468 pathetic\n",
      "81 patience\n",
      "158 patient\n",
      "89 patients\n",
      "223 patrick\n",
      "897 paul\n",
      "119 paulie\n",
      "610 pay\n",
      "184 paying\n",
      "106 pays\n",
      "205 peace\n",
      "81 peak\n",
      "94 pearl\n",
      "9285 people\n",
      "100 peoples\n",
      "161 per\n",
      "1598 perfect\n",
      "144 perfection\n",
      "637 perfectly\n",
      "150 perform\n",
      "2896 performance\n",
      "1821 performances\n",
      "192 performed\n",
      "103 performer\n",
      "150 performers\n",
      "131 performing\n",
      "84 performs\n",
      "1681 perhaps\n",
      "766 period\n",
      "123 perry\n",
      "1596 person\n",
      "127 persona\n",
      "629 personal\n",
      "156 personalities\n",
      "338 personality\n",
      "446 personally\n",
      "90 persons\n",
      "266 perspective\n",
      "166 pet\n",
      "80 pete\n",
      "770 peter\n",
      "83 peters\n",
      "83 petty\n",
      "145 pg\n",
      "89 phantom\n",
      "80 phil\n",
      "162 philip\n",
      "99 philosophical\n",
      "109 philosophy\n",
      "321 phone\n",
      "84 phony\n",
      "86 photo\n",
      "126 photographed\n",
      "118 photographer\n",
      "406 photography\n",
      "95 photos\n",
      "307 physical\n",
      "151 physically\n",
      "124 piano\n",
      "452 pick\n",
      "330 picked\n",
      "122 picking\n",
      "169 picks\n",
      "1484 picture\n",
      "453 pictures\n",
      "142 pie\n",
      "1536 piece\n",
      "424 pieces\n",
      "82 pierce\n",
      "102 pig\n",
      "207 pile\n",
      "300 pilot\n",
      "82 pin\n",
      "85 pink\n",
      "88 pit\n",
      "155 pitch\n",
      "194 pitt\n",
      "229 pity\n",
      "2411 place\n",
      "188 placed\n",
      "411 places\n",
      "146 plague\n",
      "568 plain\n",
      "421 plan\n",
      "322 plane\n",
      "502 planet\n",
      "101 planned\n",
      "124 planning\n",
      "204 plans\n",
      "95 plant\n",
      "148 plastic\n",
      "87 plausible\n",
      "2237 play\n",
      "2588 played\n",
      "300 player\n",
      "286 players\n",
      "1633 playing\n",
      "2214 plays\n",
      "234 pleasant\n",
      "124 pleasantly\n",
      "1045 please\n",
      "127 pleased\n",
      "309 pleasure\n",
      "632 plenty\n",
      "91 plight\n",
      "6585 plot\n",
      "285 plots\n",
      "645 plus\n",
      "85 poem\n",
      "96 poetic\n",
      "92 poetry\n",
      "156 poignant\n",
      "3225 point\n",
      "135 pointed\n",
      "504 pointless\n",
      "814 points\n",
      "93 pokemon\n",
      "106 polanski\n",
      "1097 police\n",
      "85 polished\n",
      "608 political\n",
      "106 politically\n",
      "208 politics\n",
      "153 pool\n",
      "1897 poor\n",
      "713 poorly\n",
      "316 pop\n",
      "112 popcorn\n",
      "89 pops\n",
      "550 popular\n",
      "83 popularity\n",
      "113 population\n",
      "366 porn\n",
      "98 porno\n",
      "104 portion\n",
      "145 portrait\n",
      "264 portray\n",
      "508 portrayal\n",
      "601 portrayed\n",
      "227 portraying\n",
      "229 portrays\n",
      "179 position\n",
      "518 positive\n",
      "108 possessed\n",
      "98 possibilities\n",
      "107 possibility\n",
      "999 possible\n",
      "709 possibly\n",
      "483 post\n",
      "120 poster\n",
      "100 pot\n",
      "612 potential\n",
      "91 potentially\n",
      "133 poverty\n",
      "213 powell\n",
      "948 power\n",
      "620 powerful\n",
      "317 powers\n",
      "234 practically\n",
      "100 practice\n",
      "171 praise\n",
      "309 pre\n",
      "121 precious\n",
      "854 predictable\n",
      "175 prefer\n",
      "178 pregnant\n",
      "78 premiere\n",
      "712 premise\n",
      "171 prepared\n",
      "85 prequel\n",
      "410 presence\n",
      "616 present\n",
      "160 presentation\n",
      "415 presented\n",
      "207 presents\n",
      "251 president\n",
      "127 press\n",
      "80 pressure\n",
      "80 preston\n",
      "127 presumably\n",
      "112 pretend\n",
      "95 pretending\n",
      "269 pretentious\n",
      "3664 pretty\n",
      "123 prevent\n",
      "97 preview\n",
      "630 previous\n",
      "204 previously\n",
      "79 prey\n",
      "296 price\n",
      "86 priceless\n",
      "151 pride\n",
      "226 priest\n",
      "104 primarily\n",
      "106 primary\n",
      "202 prime\n",
      "300 prince\n",
      "197 princess\n",
      "104 principal\n",
      "200 print\n",
      "192 prior\n",
      "493 prison\n",
      "89 prisoner\n",
      "89 prisoners\n",
      "270 private\n",
      "95 prize\n",
      "142 pro\n",
      "2841 probably\n",
      "1451 problem\n",
      "886 problems\n",
      "112 proceedings\n",
      "88 proceeds\n",
      "301 process\n",
      "229 produce\n",
      "555 produced\n",
      "436 producer\n",
      "491 producers\n",
      "108 producing\n",
      "232 product\n",
      "1790 production\n",
      "183 productions\n",
      "335 professional\n",
      "208 professor\n",
      "139 profound\n",
      "253 program\n",
      "120 progress\n",
      "91 progresses\n",
      "492 project\n",
      "131 projects\n",
      "115 prom\n",
      "219 promise\n",
      "97 promised\n",
      "84 promises\n",
      "207 promising\n",
      "148 proof\n",
      "203 propaganda\n",
      "229 proper\n",
      "167 properly\n",
      "87 property\n",
      "106 props\n",
      "109 prostitute\n",
      "238 protagonist\n",
      "146 protagonists\n",
      "165 protect\n",
      "188 proud\n",
      "265 prove\n",
      "249 proved\n",
      "358 proves\n",
      "305 provide\n",
      "207 provided\n",
      "350 provides\n",
      "118 providing\n",
      "166 provoking\n",
      "112 pseudo\n",
      "122 psychiatrist\n",
      "85 psychic\n",
      "247 psycho\n",
      "265 psychological\n",
      "120 psychotic\n",
      "564 public\n",
      "342 pull\n",
      "273 pulled\n",
      "121 pulling\n",
      "184 pulls\n",
      "111 pulp\n",
      "173 punch\n",
      "84 punishment\n",
      "106 punk\n",
      "81 puppet\n",
      "96 purchase\n",
      "90 purchased\n",
      "562 pure\n",
      "169 purely\n",
      "116 purple\n",
      "439 purpose\n",
      "84 purposes\n",
      "85 pursuit\n",
      "124 push\n",
      "127 pushed\n",
      "118 pushing\n",
      "2380 put\n",
      "380 puts\n",
      "369 putting\n",
      "207 qualities\n",
      "1301 quality\n",
      "347 queen\n",
      "183 quest\n",
      "685 question\n",
      "91 questionable\n",
      "479 questions\n",
      "338 quick\n",
      "639 quickly\n",
      "284 quiet\n",
      "83 quinn\n",
      "174 quirky\n",
      "83 quit\n",
      "3738 quite\n",
      "145 quote\n",
      "98 quotes\n",
      "88 rabbit\n",
      "365 race\n",
      "189 rachel\n",
      "99 racial\n",
      "153 racism\n",
      "177 racist\n",
      "284 radio\n",
      "111 rage\n",
      "201 rain\n",
      "79 raines\n",
      "164 raise\n",
      "170 raised\n",
      "94 raising\n",
      "147 ralph\n",
      "114 rambo\n",
      "83 ramones\n",
      "239 ran\n",
      "361 random\n",
      "85 randomly\n",
      "87 randy\n",
      "234 range\n",
      "86 rangers\n",
      "102 rank\n",
      "113 ranks\n",
      "124 rap\n",
      "365 rape\n",
      "126 raped\n",
      "442 rare\n",
      "315 rarely\n",
      "108 rat\n",
      "626 rate\n",
      "507 rated\n",
      "2733 rather\n",
      "928 rating\n",
      "166 ratings\n",
      "87 rats\n",
      "82 rave\n",
      "176 raw\n",
      "378 ray\n",
      "92 raymond\n",
      "123 rd\n",
      "246 reach\n",
      "115 reached\n",
      "102 reaches\n",
      "95 reaching\n",
      "108 react\n",
      "249 reaction\n",
      "136 reactions\n",
      "1964 read\n",
      "80 reader\n",
      "685 reading\n",
      "93 reads\n",
      "336 ready\n",
      "4737 real\n",
      "126 realise\n",
      "279 realism\n",
      "757 realistic\n",
      "987 reality\n",
      "654 realize\n",
      "317 realized\n",
      "199 realizes\n",
      "98 realizing\n",
      "11736 really\n",
      "2323 reason\n",
      "117 reasonable\n",
      "119 reasonably\n",
      "596 reasons\n",
      "103 rebel\n",
      "226 recall\n",
      "112 receive\n",
      "262 received\n",
      "83 receives\n",
      "510 recent\n",
      "579 recently\n",
      "90 recognition\n",
      "195 recognize\n",
      "113 recognized\n",
      "1667 recommend\n",
      "489 recommended\n",
      "283 record\n",
      "104 recorded\n",
      "85 recording\n",
      "819 red\n",
      "326 redeeming\n",
      "143 redemption\n",
      "118 reduced\n",
      "163 reed\n",
      "88 reel\n",
      "166 reference\n",
      "249 references\n",
      "92 reflect\n",
      "81 reflection\n",
      "206 refreshing\n",
      "78 refused\n",
      "154 refuses\n",
      "166 regard\n",
      "174 regarding\n",
      "125 regardless\n",
      "189 regret\n",
      "266 regular\n",
      "86 reid\n",
      "235 relate\n",
      "202 related\n",
      "94 relation\n",
      "102 relations\n",
      "966 relationship\n",
      "361 relationships\n",
      "126 relative\n",
      "213 relatively\n",
      "89 relatives\n",
      "807 release\n",
      "986 released\n",
      "132 relevant\n",
      "242 relief\n",
      "106 relies\n",
      "238 religion\n",
      "310 religious\n",
      "209 remain\n",
      "78 remained\n",
      "120 remaining\n",
      "439 remains\n",
      "583 remake\n",
      "309 remarkable\n",
      "105 remarkably\n",
      "91 remarks\n",
      "1702 remember\n",
      "258 remembered\n",
      "157 remind\n",
      "347 reminded\n",
      "297 reminds\n",
      "175 reminiscent\n",
      "163 remote\n",
      "189 remotely\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "108 removed\n",
      "90 rendition\n",
      "719 rent\n",
      "214 rental\n",
      "337 rented\n",
      "177 renting\n",
      "143 repeat\n",
      "204 repeated\n",
      "119 repeatedly\n",
      "123 repetitive\n",
      "162 replaced\n",
      "96 report\n",
      "213 reporter\n",
      "103 represent\n",
      "99 represented\n",
      "132 represents\n",
      "191 reputation\n",
      "190 required\n",
      "132 requires\n",
      "231 rescue\n",
      "222 research\n",
      "107 resemblance\n",
      "80 resemble\n",
      "113 resembles\n",
      "85 resident\n",
      "84 resist\n",
      "130 resolution\n",
      "89 resort\n",
      "89 resources\n",
      "499 respect\n",
      "83 respected\n",
      "79 respectively\n",
      "122 response\n",
      "88 responsibility\n",
      "275 responsible\n",
      "1803 rest\n",
      "119 restaurant\n",
      "90 restored\n",
      "632 result\n",
      "82 resulting\n",
      "274 results\n",
      "161 retarded\n",
      "82 retired\n",
      "624 return\n",
      "117 returned\n",
      "126 returning\n",
      "307 returns\n",
      "151 reunion\n",
      "189 reveal\n",
      "256 revealed\n",
      "122 revealing\n",
      "183 reveals\n",
      "133 revelation\n",
      "555 revenge\n",
      "849 review\n",
      "240 reviewer\n",
      "267 reviewers\n",
      "717 reviews\n",
      "188 revolution\n",
      "103 revolutionary\n",
      "154 revolves\n",
      "128 rex\n",
      "129 reynolds\n",
      "587 rich\n",
      "847 richard\n",
      "90 richards\n",
      "92 richardson\n",
      "105 rick\n",
      "118 rid\n",
      "88 ridden\n",
      "413 ride\n",
      "964 ridiculous\n",
      "112 ridiculously\n",
      "155 riding\n",
      "3313 right\n",
      "184 rights\n",
      "312 ring\n",
      "185 rings\n",
      "323 rip\n",
      "138 ripped\n",
      "235 rise\n",
      "79 rises\n",
      "89 rising\n",
      "162 risk\n",
      "80 rita\n",
      "135 ritter\n",
      "161 rival\n",
      "289 river\n",
      "99 riveting\n",
      "435 road\n",
      "252 rob\n",
      "92 robbery\n",
      "80 robbins\n",
      "951 robert\n",
      "142 roberts\n",
      "248 robin\n",
      "93 robinson\n",
      "217 robot\n",
      "112 robots\n",
      "155 rochester\n",
      "876 rock\n",
      "102 rocket\n",
      "138 rocks\n",
      "86 rocky\n",
      "203 roger\n",
      "170 rogers\n",
      "3188 role\n",
      "1112 roles\n",
      "338 roll\n",
      "83 rolled\n",
      "183 rolling\n",
      "107 roman\n",
      "694 romance\n",
      "854 romantic\n",
      "117 romero\n",
      "81 romp\n",
      "183 ron\n",
      "945 room\n",
      "99 rooms\n",
      "87 rooney\n",
      "122 root\n",
      "82 roots\n",
      "244 rose\n",
      "82 ross\n",
      "96 roth\n",
      "96 rotten\n",
      "186 rough\n",
      "244 round\n",
      "201 routine\n",
      "132 row\n",
      "228 roy\n",
      "87 royal\n",
      "90 rubber\n",
      "275 rubbish\n",
      "94 ruby\n",
      "203 ruin\n",
      "227 ruined\n",
      "84 ruins\n",
      "86 rukh\n",
      "188 rule\n",
      "231 rules\n",
      "1218 run\n",
      "992 running\n",
      "513 runs\n",
      "110 rural\n",
      "139 rush\n",
      "138 rushed\n",
      "208 russell\n",
      "79 russia\n",
      "302 russian\n",
      "121 ruth\n",
      "101 ruthless\n",
      "224 ryan\n",
      "84 sabrina\n",
      "127 sacrifice\n",
      "996 sad\n",
      "112 sadistic\n",
      "575 sadly\n",
      "112 sadness\n",
      "227 safe\n",
      "98 safety\n",
      "101 saga\n",
      "2196 said\n",
      "246 sake\n",
      "134 sally\n",
      "456 sam\n",
      "125 samurai\n",
      "186 san\n",
      "174 sandler\n",
      "98 sandra\n",
      "276 santa\n",
      "94 sappy\n",
      "192 sarah\n",
      "293 sat\n",
      "128 satan\n",
      "261 satire\n",
      "106 satisfied\n",
      "87 satisfy\n",
      "216 satisfying\n",
      "220 saturday\n",
      "128 savage\n",
      "1023 save\n",
      "276 saved\n",
      "143 saves\n",
      "276 saving\n",
      "3167 saw\n",
      "5395 say\n",
      "946 saying\n",
      "1110 says\n",
      "209 scale\n",
      "219 scare\n",
      "98 scarecrow\n",
      "304 scared\n",
      "189 scares\n",
      "988 scary\n",
      "183 scenario\n",
      "5378 scene\n",
      "407 scenery\n",
      "5207 scenes\n",
      "106 scheme\n",
      "1659 school\n",
      "658 sci\n",
      "549 science\n",
      "119 scientific\n",
      "337 scientist\n",
      "140 scientists\n",
      "136 scooby\n",
      "97 scope\n",
      "1030 score\n",
      "91 scores\n",
      "84 scotland\n",
      "584 scott\n",
      "93 scottish\n",
      "268 scream\n",
      "271 screaming\n",
      "117 screams\n",
      "2493 screen\n",
      "173 screening\n",
      "695 screenplay\n",
      "161 screenwriter\n",
      "3026 script\n",
      "118 scripted\n",
      "147 scripts\n",
      "93 scrooge\n",
      "264 sea\n",
      "156 seagal\n",
      "263 sean\n",
      "298 search\n",
      "147 searching\n",
      "773 season\n",
      "237 seasons\n",
      "232 seat\n",
      "1962 second\n",
      "104 secondly\n",
      "348 seconds\n",
      "611 secret\n",
      "138 secretary\n",
      "93 secretly\n",
      "113 secrets\n",
      "206 section\n",
      "194 security\n",
      "11475 see\n",
      "101 seed\n",
      "2099 seeing\n",
      "179 seek\n",
      "161 seeking\n",
      "86 seeks\n",
      "2175 seem\n",
      "1363 seemed\n",
      "347 seemingly\n",
      "3618 seems\n",
      "6679 seen\n",
      "537 sees\n",
      "255 segment\n",
      "139 segments\n",
      "80 seldom\n",
      "1185 self\n",
      "112 selfish\n",
      "232 sell\n",
      "99 sellers\n",
      "129 selling\n",
      "210 semi\n",
      "231 send\n",
      "139 sends\n",
      "2325 sense\n",
      "101 senseless\n",
      "79 senses\n",
      "181 sensitive\n",
      "395 sent\n",
      "105 sentence\n",
      "146 sentimental\n",
      "149 separate\n",
      "93 september\n",
      "818 sequel\n",
      "224 sequels\n",
      "875 sequence\n",
      "728 sequences\n",
      "394 serial\n",
      "3416 series\n",
      "989 serious\n",
      "1002 seriously\n",
      "167 serve\n",
      "158 served\n",
      "203 serves\n",
      "209 service\n",
      "86 serving\n",
      "2455 set\n",
      "852 sets\n",
      "634 setting\n",
      "175 settings\n",
      "104 settle\n",
      "359 seven\n",
      "112 seventies\n",
      "1420 several\n",
      "98 severe\n",
      "1683 sex\n",
      "710 sexual\n",
      "150 sexuality\n",
      "134 sexually\n",
      "445 sexy\n",
      "103 sh\n",
      "181 shadow\n",
      "117 shadows\n",
      "95 shake\n",
      "296 shakespeare\n",
      "80 shaky\n",
      "133 shall\n",
      "259 shallow\n",
      "671 shame\n",
      "81 shanghai\n",
      "159 shape\n",
      "367 share\n",
      "141 shark\n",
      "206 sharp\n",
      "100 shaw\n",
      "82 shed\n",
      "247 sheer\n",
      "103 shelf\n",
      "95 shelley\n",
      "244 sheriff\n",
      "109 shine\n",
      "149 shines\n",
      "129 shining\n",
      "347 ship\n",
      "79 ships\n",
      "103 shirley\n",
      "118 shirt\n",
      "385 shock\n",
      "209 shocked\n",
      "339 shocking\n",
      "121 shoes\n",
      "460 shoot\n",
      "479 shooting\n",
      "142 shoots\n",
      "263 shop\n",
      "1865 short\n",
      "122 shortly\n",
      "148 shorts\n",
      "2051 shot\n",
      "947 shots\n",
      "6295 show\n",
      "90 showcase\n",
      "84 showdown\n",
      "489 showed\n",
      "159 shower\n",
      "775 showing\n",
      "994 shown\n",
      "2308 shows\n",
      "167 shut\n",
      "125 shy\n",
      "487 sick\n",
      "82 sid\n",
      "1277 side\n",
      "105 sidekick\n",
      "157 sides\n",
      "180 sidney\n",
      "323 sight\n",
      "274 sign\n",
      "89 signed\n",
      "80 significance\n",
      "182 significant\n",
      "106 signs\n",
      "130 silence\n",
      "439 silent\n",
      "888 silly\n",
      "139 silver\n",
      "852 similar\n",
      "98 similarities\n",
      "98 similarly\n",
      "84 simmons\n",
      "272 simon\n",
      "1023 simple\n",
      "91 simplicity\n",
      "101 simplistic\n",
      "1965 simply\n",
      "95 simpson\n",
      "157 sin\n",
      "245 sinatra\n",
      "2906 since\n",
      "86 sincere\n",
      "270 sing\n",
      "274 singer\n",
      "79 singers\n",
      "520 singing\n",
      "918 single\n",
      "148 sings\n",
      "163 sinister\n",
      "87 sink\n",
      "185 sir\n",
      "93 sirk\n",
      "82 sissy\n",
      "818 sister\n",
      "207 sisters\n",
      "709 sit\n",
      "166 sitcom\n",
      "244 site\n",
      "95 sits\n",
      "451 sitting\n",
      "669 situation\n",
      "486 situations\n",
      "386 six\n",
      "83 sixties\n",
      "125 size\n",
      "180 skill\n",
      "267 skills\n",
      "212 skin\n",
      "305 skip\n",
      "78 skull\n",
      "321 sky\n",
      "121 slap\n",
      "176 slapstick\n",
      "491 slasher\n",
      "89 slaughter\n",
      "117 slave\n",
      "164 sleazy\n",
      "325 sleep\n",
      "179 sleeping\n",
      "80 slice\n",
      "93 slick\n",
      "137 slight\n",
      "122 slightest\n",
      "541 slightly\n",
      "117 sloppy\n",
      "1131 slow\n",
      "412 slowly\n",
      "1646 small\n",
      "105 smaller\n",
      "406 smart\n",
      "289 smile\n",
      "87 smiling\n",
      "493 smith\n",
      "118 smoke\n",
      "134 smoking\n",
      "128 smooth\n",
      "110 snake\n",
      "84 sneak\n",
      "97 snl\n",
      "152 snow\n",
      "284 soap\n",
      "101 soccer\n",
      "582 social\n",
      "674 society\n",
      "83 soderbergh\n",
      "297 soft\n",
      "162 sold\n",
      "348 soldier\n",
      "422 soldiers\n",
      "120 sole\n",
      "97 solely\n",
      "503 solid\n",
      "100 solo\n",
      "88 solution\n",
      "139 solve\n",
      "299 somebody\n",
      "757 somehow\n",
      "2339 someone\n",
      "5077 something\n",
      "1218 sometimes\n",
      "965 somewhat\n",
      "483 somewhere\n",
      "1357 son\n",
      "1008 song\n",
      "908 songs\n",
      "149 sons\n",
      "1222 soon\n",
      "129 sophisticated\n",
      "78 sopranos\n",
      "771 sorry\n",
      "1472 sort\n",
      "190 sorts\n",
      "420 soul\n",
      "122 souls\n",
      "1320 sound\n",
      "175 sounded\n",
      "97 sounding\n",
      "645 sounds\n",
      "764 soundtrack\n",
      "206 source\n",
      "472 south\n",
      "164 southern\n",
      "118 soviet\n",
      "755 space\n",
      "83 spacey\n",
      "85 spain\n",
      "281 spanish\n",
      "107 spare\n",
      "79 spark\n",
      "522 speak\n",
      "405 speaking\n",
      "202 speaks\n",
      "2113 special\n",
      "85 specially\n",
      "96 species\n",
      "135 specific\n",
      "101 specifically\n",
      "248 spectacular\n",
      "200 speech\n",
      "249 speed\n",
      "137 spell\n",
      "508 spend\n",
      "133 spending\n",
      "187 spends\n",
      "536 spent\n",
      "87 spider\n",
      "154 spielberg\n",
      "133 spike\n",
      "152 spin\n",
      "544 spirit\n",
      "127 spirited\n",
      "106 spirits\n",
      "125 spiritual\n",
      "179 spite\n",
      "100 splatter\n",
      "123 splendid\n",
      "144 split\n",
      "92 spock\n",
      "214 spoil\n",
      "123 spoiled\n",
      "406 spoiler\n",
      "570 spoilers\n",
      "91 spoke\n",
      "169 spoken\n",
      "171 spoof\n",
      "124 spooky\n",
      "108 sport\n",
      "228 sports\n",
      "379 spot\n",
      "143 spots\n",
      "91 spread\n",
      "124 spring\n",
      "205 spy\n",
      "93 square\n",
      "360 st\n",
      "79 stack\n",
      "109 staff\n",
      "714 stage\n",
      "111 staged\n",
      "87 stale\n",
      "91 stallone\n",
      "144 stan\n",
      "815 stand\n",
      "448 standard\n",
      "355 standards\n",
      "249 standing\n",
      "398 stands\n",
      "168 stanley\n",
      "161 stanwyck\n",
      "2087 star\n",
      "83 stargate\n",
      "93 staring\n",
      "183 starred\n",
      "484 starring\n",
      "1695 stars\n",
      "1700 start\n",
      "963 started\n",
      "283 starting\n",
      "1220 starts\n",
      "533 state\n",
      "132 stated\n",
      "190 statement\n",
      "335 states\n",
      "321 station\n",
      "182 status\n",
      "787 stay\n",
      "182 stayed\n",
      "109 staying\n",
      "182 stays\n",
      "247 steal\n",
      "144 stealing\n",
      "212 steals\n",
      "95 steel\n",
      "102 stellar\n",
      "373 step\n",
      "327 stephen\n",
      "146 steps\n",
      "98 stereotype\n",
      "245 stereotypes\n",
      "177 stereotypical\n",
      "483 steve\n",
      "241 steven\n",
      "94 stevens\n",
      "468 stewart\n",
      "469 stick\n",
      "117 sticks\n",
      "133 stiff\n",
      "5623 still\n",
      "117 stiller\n",
      "93 stilted\n",
      "102 stinker\n",
      "97 stinks\n",
      "255 stock\n",
      "115 stole\n",
      "190 stolen\n",
      "167 stomach\n",
      "356 stone\n",
      "133 stood\n",
      "95 stooges\n",
      "1084 stop\n",
      "230 stopped\n",
      "158 stops\n",
      "519 store\n",
      "1179 stories\n",
      "152 storm\n",
      "11983 story\n",
      "804 storyline\n",
      "173 storytelling\n",
      "864 straight\n",
      "926 strange\n",
      "166 strangely\n",
      "155 stranger\n",
      "81 strangers\n",
      "103 streep\n",
      "699 street\n",
      "269 streets\n",
      "152 streisand\n",
      "245 strength\n",
      "98 stress\n",
      "144 stretch\n",
      "83 stretched\n",
      "129 strictly\n",
      "131 strike\n",
      "135 strikes\n",
      "137 striking\n",
      "124 string\n",
      "143 strip\n",
      "1096 strong\n",
      "133 stronger\n",
      "222 strongly\n",
      "128 struck\n",
      "206 structure\n",
      "327 struggle\n",
      "155 struggles\n",
      "198 struggling\n",
      "80 stuart\n",
      "350 stuck\n",
      "391 student\n",
      "361 students\n",
      "517 studio\n",
      "181 studios\n",
      "250 study\n",
      "1174 stuff\n",
      "78 stumbled\n",
      "82 stunned\n",
      "405 stunning\n",
      "136 stunt\n",
      "139 stunts\n",
      "1701 stupid\n",
      "158 stupidity\n",
      "1601 style\n",
      "114 styles\n",
      "158 stylish\n",
      "371 sub\n",
      "709 subject\n",
      "79 subjected\n",
      "107 subjects\n",
      "121 subplot\n",
      "87 subplots\n",
      "121 subsequent\n",
      "219 substance\n",
      "190 subtitles\n",
      "434 subtle\n",
      "102 subtlety\n",
      "150 succeed\n",
      "106 succeeded\n",
      "167 succeeds\n",
      "585 success\n",
      "524 successful\n",
      "161 successfully\n",
      "177 suck\n",
      "251 sucked\n",
      "280 sucks\n",
      "248 sudden\n",
      "538 suddenly\n",
      "91 sue\n",
      "181 suffer\n",
      "150 suffered\n",
      "250 suffering\n",
      "201 suffers\n",
      "84 suffice\n",
      "377 suggest\n",
      "81 suggested\n",
      "145 suggests\n",
      "316 suicide\n",
      "311 suit\n",
      "94 suitable\n",
      "111 suited\n",
      "121 suits\n",
      "192 sullivan\n",
      "159 sum\n",
      "178 summary\n",
      "374 summer\n",
      "193 sun\n",
      "190 sunday\n",
      "117 sunshine\n",
      "500 super\n",
      "671 superb\n",
      "126 superbly\n",
      "112 superficial\n",
      "119 superhero\n",
      "311 superior\n",
      "308 superman\n",
      "201 supernatural\n",
      "393 support\n",
      "81 supported\n",
      "899 supporting\n",
      "397 suppose\n",
      "1516 supposed\n",
      "365 supposedly\n",
      "2683 sure\n",
      "417 surely\n",
      "192 surface\n",
      "102 surfing\n",
      "715 surprise\n",
      "801 surprised\n",
      "203 surprises\n",
      "302 surprising\n",
      "466 surprisingly\n",
      "208 surreal\n",
      "134 surrounded\n",
      "134 surrounding\n",
      "106 survival\n",
      "260 survive\n",
      "91 survived\n",
      "104 surviving\n",
      "85 survivor\n",
      "101 survivors\n",
      "186 susan\n",
      "301 suspect\n",
      "156 suspects\n",
      "84 suspend\n",
      "739 suspense\n",
      "192 suspenseful\n",
      "88 suspicious\n",
      "160 sutherland\n",
      "113 swear\n",
      "112 swedish\n",
      "572 sweet\n",
      "83 swim\n",
      "97 swimming\n",
      "97 switch\n",
      "195 sword\n",
      "114 symbolism\n",
      "229 sympathetic\n",
      "199 sympathy\n",
      "111 synopsis\n",
      "370 system\n",
      "181 table\n",
      "79 tacky\n",
      "97 tad\n",
      "99 tag\n",
      "3507 take\n",
      "986 taken\n",
      "2192 takes\n",
      "955 taking\n",
      "790 tale\n",
      "933 talent\n",
      "586 talented\n",
      "268 talents\n",
      "166 tales\n",
      "842 talk\n",
      "126 talked\n",
      "954 talking\n",
      "220 talks\n",
      "121 tall\n",
      "111 tame\n",
      "80 tank\n",
      "115 tap\n",
      "234 tape\n",
      "82 tarantino\n",
      "211 target\n",
      "291 tarzan\n",
      "174 task\n",
      "437 taste\n",
      "95 taught\n",
      "91 taxi\n",
      "315 taylor\n",
      "136 tea\n",
      "137 teach\n",
      "314 teacher\n",
      "82 teaching\n",
      "823 team\n",
      "138 tear\n",
      "324 tears\n",
      "82 tech\n",
      "305 technical\n",
      "195 technically\n",
      "80 technicolor\n",
      "150 technique\n",
      "136 techniques\n",
      "245 technology\n",
      "198 ted\n",
      "218 tedious\n",
      "339 teen\n",
      "325 teenage\n",
      "221 teenager\n",
      "217 teenagers\n",
      "198 teens\n",
      "181 teeth\n",
      "904 television\n",
      "1718 tell\n",
      "613 telling\n",
      "880 tells\n",
      "122 temple\n",
      "829 ten\n",
      "215 tend\n",
      "101 tender\n",
      "86 tends\n",
      "152 tense\n",
      "541 tension\n",
      "167 term\n",
      "433 terms\n",
      "1638 terrible\n",
      "274 terribly\n",
      "433 terrific\n",
      "142 terrifying\n",
      "123 territory\n",
      "197 terror\n",
      "103 terrorist\n",
      "89 terrorists\n",
      "112 terry\n",
      "236 test\n",
      "83 testament\n",
      "210 texas\n",
      "158 text\n",
      "745 th\n",
      "439 thank\n",
      "182 thankfully\n",
      "472 thanks\n",
      "345 thats\n",
      "828 theater\n",
      "229 theaters\n",
      "319 theatre\n",
      "228 theatrical\n",
      "816 theme\n",
      "423 themes\n",
      "192 theory\n",
      "335 therefore\n",
      "116 thick\n",
      "158 thief\n",
      "360 thin\n",
      "4526 thing\n",
      "3688 things\n",
      "7296 think\n",
      "1179 thinking\n",
      "437 thinks\n",
      "739 third\n",
      "141 thirty\n",
      "254 thomas\n",
      "85 thompson\n",
      "350 thoroughly\n",
      "4566 though\n",
      "3437 thought\n",
      "98 thoughtful\n",
      "219 thoughts\n",
      "145 thousand\n",
      "153 thousands\n",
      "116 threat\n",
      "126 threatening\n",
      "2295 three\n",
      "112 threw\n",
      "111 thrill\n",
      "895 thriller\n",
      "151 thrillers\n",
      "157 thrilling\n",
      "129 thrills\n",
      "122 throat\n",
      "1360 throughout\n",
      "402 throw\n",
      "173 throwing\n",
      "409 thrown\n",
      "167 throws\n",
      "97 thru\n",
      "111 thugs\n",
      "136 thumbs\n",
      "418 thus\n",
      "124 ticket\n",
      "102 tie\n",
      "149 tied\n",
      "104 ties\n",
      "87 tiger\n",
      "182 tight\n",
      "213 till\n",
      "309 tim\n",
      "12724 time\n",
      "121 timeless\n",
      "3235 times\n",
      "169 timing\n",
      "94 timon\n",
      "118 timothy\n",
      "214 tiny\n",
      "380 tired\n",
      "92 tiresome\n",
      "190 titanic\n",
      "1497 title\n",
      "121 titled\n",
      "172 titles\n",
      "1243 today\n",
      "112 todd\n",
      "2243 together\n",
      "130 toilet\n",
      "1063 told\n",
      "785 tom\n",
      "88 tomatoes\n",
      "113 tommy\n",
      "500 tone\n",
      "156 tongue\n",
      "96 tonight\n",
      "134 tons\n",
      "522 tony\n",
      "1100 took\n",
      "86 tooth\n",
      "1847 top\n",
      "160 topic\n",
      "101 topless\n",
      "154 torn\n",
      "300 torture\n",
      "116 tortured\n",
      "631 total\n",
      "1307 totally\n",
      "472 touch\n",
      "170 touched\n",
      "206 touches\n",
      "435 touching\n",
      "479 tough\n",
      "154 tour\n",
      "285 toward\n",
      "637 towards\n",
      "1279 town\n",
      "169 toy\n",
      "107 toys\n",
      "399 track\n",
      "105 tracks\n",
      "165 tracy\n",
      "140 trade\n",
      "81 trademark\n",
      "162 tradition\n",
      "257 traditional\n",
      "362 tragedy\n",
      "348 tragic\n",
      "83 trail\n",
      "377 trailer\n",
      "102 trailers\n",
      "411 train\n",
      "97 trained\n",
      "218 training\n",
      "95 transfer\n",
      "100 transformation\n",
      "91 transition\n",
      "117 translation\n",
      "124 trap\n",
      "188 trapped\n",
      "504 trash\n",
      "99 trashy\n",
      "246 travel\n",
      "101 traveling\n",
      "105 travels\n",
      "84 travesty\n",
      "202 treasure\n",
      "320 treat\n",
      "275 treated\n",
      "235 treatment\n",
      "97 treats\n",
      "175 tree\n",
      "104 trees\n",
      "259 trek\n",
      "127 tremendous\n",
      "152 trial\n",
      "84 tribe\n",
      "138 tribute\n",
      "171 trick\n",
      "137 tricks\n",
      "773 tried\n",
      "1274 tries\n",
      "216 trilogy\n",
      "112 trio\n",
      "492 trip\n",
      "152 trite\n",
      "115 triumph\n",
      "89 troops\n",
      "522 trouble\n",
      "145 troubled\n",
      "84 troubles\n",
      "174 truck\n",
      "2333 true\n",
      "1743 truly\n",
      "321 trust\n",
      "700 truth\n",
      "1830 try\n",
      "2473 trying\n",
      "79 tube\n",
      "145 tune\n",
      "119 tunes\n",
      "173 turkey\n",
      "1359 turn\n",
      "925 turned\n",
      "122 turner\n",
      "344 turning\n",
      "1251 turns\n",
      "2782 tv\n",
      "115 twelve\n",
      "299 twenty\n",
      "387 twice\n",
      "89 twilight\n",
      "126 twin\n",
      "81 twins\n",
      "606 twist\n",
      "199 twisted\n",
      "437 twists\n",
      "6906 two\n",
      "1124 type\n",
      "247 types\n",
      "778 typical\n",
      "130 typically\n",
      "354 ugly\n",
      "229 uk\n",
      "248 ultimate\n",
      "521 ultimately\n",
      "140 ultra\n",
      "190 un\n",
      "247 unable\n",
      "83 unaware\n",
      "116 unbearable\n",
      "434 unbelievable\n",
      "116 unbelievably\n",
      "335 uncle\n",
      "149 uncomfortable\n",
      "186 unconvincing\n",
      "174 underground\n",
      "82 underlying\n",
      "235 underrated\n",
      "1643 understand\n",
      "97 understandable\n",
      "275 understanding\n",
      "90 understated\n",
      "179 understood\n",
      "107 undoubtedly\n",
      "107 uneven\n",
      "250 unexpected\n",
      "79 unexpectedly\n",
      "80 unfair\n",
      "104 unfolds\n",
      "143 unforgettable\n",
      "208 unfortunate\n",
      "1352 unfortunately\n",
      "267 unfunny\n",
      "96 unhappy\n",
      "123 uninspired\n",
      "108 unintentional\n",
      "135 unintentionally\n",
      "198 uninteresting\n",
      "128 union\n",
      "634 unique\n",
      "89 unit\n",
      "216 united\n",
      "217 universal\n",
      "197 universe\n",
      "131 university\n",
      "286 unknown\n",
      "675 unless\n",
      "585 unlike\n",
      "211 unlikely\n",
      "307 unnecessary\n",
      "83 unoriginal\n",
      "110 unpleasant\n",
      "82 unpredictable\n",
      "84 unreal\n",
      "226 unrealistic\n",
      "83 unseen\n",
      "99 unsettling\n",
      "310 unusual\n",
      "106 unwatchable\n",
      "81 uplifting\n",
      "859 upon\n",
      "158 upper\n",
      "266 ups\n",
      "154 upset\n",
      "189 urban\n",
      "102 urge\n",
      "3794 us\n",
      "164 usa\n",
      "1803 use\n",
      "1879 used\n",
      "94 useful\n",
      "128 useless\n",
      "102 user\n",
      "540 uses\n",
      "801 using\n",
      "83 ustinov\n",
      "965 usual\n",
      "981 usually\n",
      "240 utter\n",
      "454 utterly\n",
      "102 uwe\n",
      "159 vacation\n",
      "129 vague\n",
      "90 vaguely\n",
      "80 valentine\n",
      "82 valley\n",
      "93 valuable\n",
      "524 value\n",
      "467 values\n",
      "419 vampire\n",
      "249 vampires\n",
      "495 van\n",
      "79 vance\n",
      "192 variety\n",
      "603 various\n",
      "100 vast\n",
      "95 vegas\n",
      "236 vehicle\n",
      "94 vengeance\n",
      "118 verhoeven\n",
      "2157 version\n",
      "253 versions\n",
      "109 versus\n",
      "201 veteran\n",
      "282 vhs\n",
      "169 via\n",
      "92 vice\n",
      "116 vicious\n",
      "416 victim\n",
      "377 victims\n",
      "236 victor\n",
      "233 victoria\n",
      "80 victory\n",
      "1730 video\n",
      "136 videos\n",
      "192 vietnam\n",
      "963 view\n",
      "210 viewed\n",
      "1262 viewer\n",
      "785 viewers\n",
      "750 viewing\n",
      "88 viewings\n",
      "182 views\n",
      "255 village\n",
      "600 villain\n",
      "278 villains\n",
      "145 vincent\n",
      "1091 violence\n",
      "523 violent\n",
      "149 virgin\n",
      "94 virginia\n",
      "214 virtually\n",
      "139 virus\n",
      "89 visible\n",
      "304 vision\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "262 visit\n",
      "89 visits\n",
      "523 visual\n",
      "259 visually\n",
      "253 visuals\n",
      "109 vivid\n",
      "1156 voice\n",
      "98 voiced\n",
      "218 voices\n",
      "111 voight\n",
      "184 von\n",
      "222 vote\n",
      "273 vs\n",
      "91 vulnerable\n",
      "84 wacky\n",
      "718 wait\n",
      "95 waited\n",
      "549 waiting\n",
      "85 waitress\n",
      "140 wake\n",
      "511 walk\n",
      "195 walked\n",
      "142 walken\n",
      "117 walker\n",
      "439 walking\n",
      "213 walks\n",
      "367 wall\n",
      "116 wallace\n",
      "133 walls\n",
      "86 walsh\n",
      "221 walter\n",
      "90 wandering\n",
      "81 wang\n",
      "158 wanna\n",
      "117 wannabe\n",
      "3703 want\n",
      "1352 wanted\n",
      "298 wanting\n",
      "1287 wants\n",
      "2051 war\n",
      "126 ward\n",
      "227 warm\n",
      "109 warming\n",
      "86 warmth\n",
      "153 warn\n",
      "174 warned\n",
      "177 warner\n",
      "314 warning\n",
      "117 warren\n",
      "120 warrior\n",
      "82 warriors\n",
      "332 wars\n",
      "272 washington\n",
      "1457 waste\n",
      "560 wasted\n",
      "150 wasting\n",
      "6973 watch\n",
      "307 watchable\n",
      "2236 watched\n",
      "122 watches\n",
      "4603 watching\n",
      "547 water\n",
      "108 waters\n",
      "85 watson\n",
      "173 wave\n",
      "93 waves\n",
      "8026 way\n",
      "236 wayne\n",
      "804 ways\n",
      "761 weak\n",
      "99 weakest\n",
      "109 wealth\n",
      "153 wealthy\n",
      "148 weapon\n",
      "168 weapons\n",
      "186 wear\n",
      "326 wearing\n",
      "166 wears\n",
      "114 web\n",
      "112 website\n",
      "304 wedding\n",
      "460 week\n",
      "207 weekend\n",
      "202 weeks\n",
      "139 weight\n",
      "663 weird\n",
      "214 welcome\n",
      "10662 well\n",
      "251 welles\n",
      "121 wells\n",
      "98 wendy\n",
      "1463 went\n",
      "272 werewolf\n",
      "121 wes\n",
      "476 west\n",
      "600 western\n",
      "164 westerns\n",
      "96 wet\n",
      "116 whale\n",
      "732 whatever\n",
      "85 whats\n",
      "319 whatsoever\n",
      "270 whenever\n",
      "149 whereas\n",
      "856 whether\n",
      "280 whilst\n",
      "1492 white\n",
      "202 whoever\n",
      "3078 whole\n",
      "82 wholly\n",
      "84 whoopi\n",
      "986 whose\n",
      "118 wicked\n",
      "281 wide\n",
      "87 widely\n",
      "155 widmark\n",
      "105 widow\n",
      "2140 wife\n",
      "432 wild\n",
      "80 wilderness\n",
      "596 william\n",
      "344 williams\n",
      "85 willie\n",
      "320 willing\n",
      "92 willis\n",
      "200 wilson\n",
      "491 win\n",
      "274 wind\n",
      "257 window\n",
      "102 winds\n",
      "177 wing\n",
      "232 winner\n",
      "348 winning\n",
      "164 wins\n",
      "137 winter\n",
      "105 winters\n",
      "89 wisdom\n",
      "359 wise\n",
      "957 wish\n",
      "95 wished\n",
      "154 wishes\n",
      "85 wishing\n",
      "234 wit\n",
      "322 witch\n",
      "92 witches\n",
      "832 within\n",
      "3267 without\n",
      "210 witness\n",
      "88 witnessed\n",
      "89 witnesses\n",
      "273 witty\n",
      "105 wives\n",
      "95 wizard\n",
      "120 wolf\n",
      "2795 woman\n",
      "1819 women\n",
      "1038 wonder\n",
      "126 wondered\n",
      "1656 wonderful\n",
      "324 wonderfully\n",
      "358 wondering\n",
      "125 wonders\n",
      "101 wont\n",
      "80 woo\n",
      "271 wood\n",
      "330 wooden\n",
      "400 woods\n",
      "240 woody\n",
      "926 word\n",
      "885 words\n",
      "98 wore\n",
      "4372 work\n",
      "635 worked\n",
      "129 worker\n",
      "165 workers\n",
      "794 working\n",
      "1278 works\n",
      "3835 world\n",
      "143 worlds\n",
      "88 worn\n",
      "117 worried\n",
      "142 worry\n",
      "1468 worse\n",
      "2731 worst\n",
      "2277 worth\n",
      "126 worthless\n",
      "187 worthwhile\n",
      "362 worthy\n",
      "12436 would\n",
      "94 wound\n",
      "84 wounded\n",
      "431 wow\n",
      "87 wrap\n",
      "92 wrapped\n",
      "127 wreck\n",
      "103 wrestling\n",
      "79 wretched\n",
      "670 write\n",
      "1153 writer\n",
      "661 writers\n",
      "94 writes\n",
      "1304 writing\n",
      "1616 written\n",
      "1821 wrong\n",
      "573 wrote\n",
      "85 ww\n",
      "158 wwii\n",
      "103 ya\n",
      "82 yard\n",
      "462 yeah\n",
      "2362 year\n",
      "4516 years\n",
      "92 yelling\n",
      "106 yellow\n",
      "1535 yes\n",
      "116 yesterday\n",
      "2753 yet\n",
      "809 york\n",
      "3660 young\n",
      "503 younger\n",
      "275 youth\n",
      "384 zero\n",
      "85 zizek\n",
      "740 zombie\n",
      "518 zombies\n",
      "147 zone\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Sum up the counts of each vocabulary word\n",
    "dist = np.sum(train_data_features, axis=0)\n",
    "\n",
    "# For each, print the vocabulary word and the number of times it \n",
    "# appears in the training set\n",
    "for tag, count in zip(vocab, dist):\n",
    "    print(count, tag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Distributed Word Vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Read 25000 labeled train reviews, 25000 labeled test reviews, and 50000 unlabeled reviews\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# Read data from files \n",
    "train = pd.read_csv( \"movie-data/labeledTrainData.tsv\", header=0, \n",
    " delimiter=\"\\t\", quoting=3 )\n",
    "test = pd.read_csv( \"movie-data/testData.tsv\", header=0, delimiter=\"\\t\", quoting=3 )\n",
    "unlabeled_train = pd.read_csv( \"movie-data/unlabeledTrainData.tsv\", header=0, \n",
    " delimiter=\"\\t\", quoting=3 )\n",
    "\n",
    "# Verify the number of reviews that were read (100,000 in total)\n",
    "print(\"Read %d labeled train reviews, %d labeled test reviews, \" \\\n",
    " \"and %d unlabeled reviews\\n\" % (train[\"review\"].size,  \n",
    " test[\"review\"].size, unlabeled_train[\"review\"].size ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import various modules for string cleaning\n",
    "from bs4 import BeautifulSoup\n",
    "import re\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "def review_to_wordlist( review, remove_stopwords=False ):\n",
    "    # Function to convert a document to a sequence of words,\n",
    "    # optionally removing stop words.  Returns a list of words.\n",
    "    #\n",
    "    # 1. Remove HTML\n",
    "    review_text = BeautifulSoup(review, 'html.parser').get_text()\n",
    "    #  \n",
    "    # 2. Remove non-letters\n",
    "    review_text = re.sub(\"[^a-zA-Z]\",\" \", review_text)\n",
    "    #\n",
    "    # 3. Convert words to lower case and split them\n",
    "    words = review_text.lower().split()\n",
    "    #\n",
    "    # 4. Optionally remove stop words (false by default)\n",
    "    if remove_stopwords:\n",
    "        stops = set(stopwords.words(\"english\"))\n",
    "        words = [w for w in words if not w in stops]\n",
    "    #\n",
    "    # 5. Return a list of words\n",
    "    return(words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml\n"
     ]
    }
   ],
   "source": [
    "# Download the punkt tokenizer for sentence splitting\n",
    "import nltk.data\n",
    "nltk.download()   \n",
    "\n",
    "# Load the punkt tokenizer\n",
    "tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')\n",
    "\n",
    "# Define a function to split a review into parsed sentences\n",
    "def review_to_sentences( review, tokenizer, remove_stopwords=False ):\n",
    "    # Function to split a review into parsed sentences. Returns a \n",
    "    # list of sentences, where each sentence is a list of words\n",
    "    #\n",
    "    # 1. Use the NLTK tokenizer to split the paragraph into sentences\n",
    "    raw_sentences = tokenizer.tokenize(review.strip())\n",
    "    #\n",
    "    # 2. Loop over each sentence\n",
    "    sentences = []\n",
    "    for raw_sentence in raw_sentences:\n",
    "        # If a sentence is empty, skip it\n",
    "        if len(raw_sentence) > 0:\n",
    "            # Otherwise, call review_to_wordlist to get a list of words\n",
    "            sentences.append( review_to_wordlist( raw_sentence, \\\n",
    "              remove_stopwords ))\n",
    "    #\n",
    "    # Return the list of sentences (each sentence is a list of words,\n",
    "    # so this returns a list of lists\n",
    "    return sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parsing sentences from training set\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:219: UserWarning: \"b'.'\" looks like a filename, not markup. You should probably open this file and pass the filehandle into Beautiful Soup.\n",
      "  ' Beautiful Soup.' % markup)\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://www.happierabroad.com\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parsing sentences from unlabeled set\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://www.archive.org/details/LovefromaStranger\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://www.loosechangeguide.com/LooseChangeGuide.html\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://www.msnbc.msn.com/id/4972055/site/newsweek/\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:219: UserWarning: \"b'..'\" looks like a filename, not markup. You should probably open this file and pass the filehandle into Beautiful Soup.\n",
      "  ' Beautiful Soup.' % markup)\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://www.youtube.com/watch?v=a0KSqelmgN8\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n",
      "/home/luis/opt/anaconda3/lib/python3.6/site-packages/bs4/__init__.py:282: UserWarning: \"http://jake-weird.blogspot.com/2007/08/beneath.html\"\" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.\n",
      "  ' that document to Beautiful Soup.' % decoded_markup\n"
     ]
    }
   ],
   "source": [
    "sentences = []  # Initialize an empty list of sentences\n",
    "\n",
    "print(\"Parsing sentences from training set\")\n",
    "for review in train[\"review\"]:\n",
    "    sentences += review_to_sentences(review, tokenizer)\n",
    "\n",
    "print(\"Parsing sentences from unlabeled set\")\n",
    "for review in unlabeled_train[\"review\"]:\n",
    "    sentences += review_to_sentences(review, tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "795538\n"
     ]
    }
   ],
   "source": [
    "print(len(sentences))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['with', 'all', 'this', 'stuff', 'going', 'down', 'at', 'the', 'moment', 'with', 'mj', 'i', 've', 'started', 'listening', 'to', 'his', 'music', 'watching', 'the', 'odd', 'documentary', 'here', 'and', 'there', 'watched', 'the', 'wiz', 'and', 'watched', 'moonwalker', 'again']\n"
     ]
    }
   ],
   "source": [
    "print(sentences[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-01-28 17:38:52,653 : INFO : 'pattern' package not found; tag filters are not available for English\n",
      "2018-01-28 17:38:52,659 : INFO : collecting all words and their counts\n",
      "2018-01-28 17:38:52,661 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types\n",
      "2018-01-28 17:38:52,753 : INFO : PROGRESS: at sentence #10000, processed 225803 words, keeping 17776 word types\n",
      "2018-01-28 17:38:52,829 : INFO : PROGRESS: at sentence #20000, processed 451892 words, keeping 24948 word types\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training model...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-01-28 17:38:52,918 : INFO : PROGRESS: at sentence #30000, processed 671314 words, keeping 30034 word types\n",
      "2018-01-28 17:38:53,002 : INFO : PROGRESS: at sentence #40000, processed 897814 words, keeping 34348 word types\n",
      "2018-01-28 17:38:53,090 : INFO : PROGRESS: at sentence #50000, processed 1116962 words, keeping 37761 word types\n",
      "2018-01-28 17:38:53,170 : INFO : PROGRESS: at sentence #60000, processed 1338403 words, keeping 40723 word types\n",
      "2018-01-28 17:38:53,247 : INFO : PROGRESS: at sentence #70000, processed 1561579 words, keeping 43333 word types\n",
      "2018-01-28 17:38:53,336 : INFO : PROGRESS: at sentence #80000, processed 1780886 words, keeping 45714 word types\n",
      "2018-01-28 17:38:53,423 : INFO : PROGRESS: at sentence #90000, processed 2004995 words, keeping 48135 word types\n",
      "2018-01-28 17:38:53,525 : INFO : PROGRESS: at sentence #100000, processed 2226966 words, keeping 50207 word types\n",
      "2018-01-28 17:38:53,621 : INFO : PROGRESS: at sentence #110000, processed 2446580 words, keeping 52081 word types\n",
      "2018-01-28 17:38:53,716 : INFO : PROGRESS: at sentence #120000, processed 2668775 words, keeping 54119 word types\n",
      "2018-01-28 17:38:53,813 : INFO : PROGRESS: at sentence #130000, processed 2894303 words, keeping 55847 word types\n",
      "2018-01-28 17:38:53,905 : INFO : PROGRESS: at sentence #140000, processed 3107005 words, keeping 57346 word types\n",
      "2018-01-28 17:38:54,019 : INFO : PROGRESS: at sentence #150000, processed 3332627 words, keeping 59055 word types\n",
      "2018-01-28 17:38:54,132 : INFO : PROGRESS: at sentence #160000, processed 3555315 words, keeping 60617 word types\n",
      "2018-01-28 17:38:54,239 : INFO : PROGRESS: at sentence #170000, processed 3778655 words, keeping 62077 word types\n",
      "2018-01-28 17:38:54,337 : INFO : PROGRESS: at sentence #180000, processed 3999236 words, keeping 63496 word types\n",
      "2018-01-28 17:38:54,423 : INFO : PROGRESS: at sentence #190000, processed 4224449 words, keeping 64794 word types\n",
      "2018-01-28 17:38:54,515 : INFO : PROGRESS: at sentence #200000, processed 4448603 words, keeping 66087 word types\n",
      "2018-01-28 17:38:54,614 : INFO : PROGRESS: at sentence #210000, processed 4669967 words, keeping 67390 word types\n",
      "2018-01-28 17:38:54,713 : INFO : PROGRESS: at sentence #220000, processed 4894968 words, keeping 68697 word types\n",
      "2018-01-28 17:38:54,803 : INFO : PROGRESS: at sentence #230000, processed 5117545 words, keeping 69958 word types\n",
      "2018-01-28 17:38:54,886 : INFO : PROGRESS: at sentence #240000, processed 5345050 words, keeping 71167 word types\n",
      "2018-01-28 17:38:54,967 : INFO : PROGRESS: at sentence #250000, processed 5559165 words, keeping 72351 word types\n",
      "2018-01-28 17:38:55,052 : INFO : PROGRESS: at sentence #260000, processed 5779146 words, keeping 73478 word types\n",
      "2018-01-28 17:38:55,138 : INFO : PROGRESS: at sentence #270000, processed 6000435 words, keeping 74767 word types\n",
      "2018-01-28 17:38:55,220 : INFO : PROGRESS: at sentence #280000, processed 6226314 words, keeping 76369 word types\n",
      "2018-01-28 17:38:55,306 : INFO : PROGRESS: at sentence #290000, processed 6449474 words, keeping 77839 word types\n",
      "2018-01-28 17:38:55,395 : INFO : PROGRESS: at sentence #300000, processed 6674077 words, keeping 79171 word types\n",
      "2018-01-28 17:38:55,476 : INFO : PROGRESS: at sentence #310000, processed 6899391 words, keeping 80480 word types\n",
      "2018-01-28 17:38:55,560 : INFO : PROGRESS: at sentence #320000, processed 7124278 words, keeping 81808 word types\n",
      "2018-01-28 17:38:55,643 : INFO : PROGRESS: at sentence #330000, processed 7346021 words, keeping 83030 word types\n",
      "2018-01-28 17:38:55,727 : INFO : PROGRESS: at sentence #340000, processed 7575533 words, keeping 84280 word types\n",
      "2018-01-28 17:38:55,816 : INFO : PROGRESS: at sentence #350000, processed 7798803 words, keeping 85425 word types\n",
      "2018-01-28 17:38:55,920 : INFO : PROGRESS: at sentence #360000, processed 8019466 words, keeping 86596 word types\n",
      "2018-01-28 17:38:56,040 : INFO : PROGRESS: at sentence #370000, processed 8246654 words, keeping 87708 word types\n",
      "2018-01-28 17:38:56,149 : INFO : PROGRESS: at sentence #380000, processed 8471801 words, keeping 88878 word types\n",
      "2018-01-28 17:38:56,246 : INFO : PROGRESS: at sentence #390000, processed 8701551 words, keeping 89907 word types\n",
      "2018-01-28 17:38:56,325 : INFO : PROGRESS: at sentence #400000, processed 8924500 words, keeping 90916 word types\n",
      "2018-01-28 17:38:56,405 : INFO : PROGRESS: at sentence #410000, processed 9145850 words, keeping 91880 word types\n",
      "2018-01-28 17:38:56,483 : INFO : PROGRESS: at sentence #420000, processed 9366930 words, keeping 92912 word types\n",
      "2018-01-28 17:38:56,568 : INFO : PROGRESS: at sentence #430000, processed 9594467 words, keeping 93932 word types\n",
      "2018-01-28 17:38:56,651 : INFO : PROGRESS: at sentence #440000, processed 9821218 words, keeping 94906 word types\n",
      "2018-01-28 17:38:56,735 : INFO : PROGRESS: at sentence #450000, processed 10044980 words, keeping 96036 word types\n",
      "2018-01-28 17:38:56,836 : INFO : PROGRESS: at sentence #460000, processed 10277740 words, keeping 97088 word types\n",
      "2018-01-28 17:38:56,941 : INFO : PROGRESS: at sentence #470000, processed 10505665 words, keeping 97933 word types\n",
      "2018-01-28 17:38:57,039 : INFO : PROGRESS: at sentence #480000, processed 10726049 words, keeping 98862 word types\n",
      "2018-01-28 17:38:57,134 : INFO : PROGRESS: at sentence #490000, processed 10952793 words, keeping 99871 word types\n",
      "2018-01-28 17:38:57,218 : INFO : PROGRESS: at sentence #500000, processed 11174449 words, keeping 100765 word types\n",
      "2018-01-28 17:38:57,310 : INFO : PROGRESS: at sentence #510000, processed 11399724 words, keeping 101699 word types\n",
      "2018-01-28 17:38:57,409 : INFO : PROGRESS: at sentence #520000, processed 11623075 words, keeping 102598 word types\n",
      "2018-01-28 17:38:57,498 : INFO : PROGRESS: at sentence #530000, processed 11847473 words, keeping 103400 word types\n",
      "2018-01-28 17:38:57,609 : INFO : PROGRESS: at sentence #540000, processed 12072088 words, keeping 104265 word types\n",
      "2018-01-28 17:38:57,712 : INFO : PROGRESS: at sentence #550000, processed 12297639 words, keeping 105133 word types\n",
      "2018-01-28 17:38:57,822 : INFO : PROGRESS: at sentence #560000, processed 12518929 words, keeping 105997 word types\n",
      "2018-01-28 17:38:57,940 : INFO : PROGRESS: at sentence #570000, processed 12748076 words, keeping 106787 word types\n",
      "2018-01-28 17:38:58,042 : INFO : PROGRESS: at sentence #580000, processed 12969572 words, keeping 107665 word types\n",
      "2018-01-28 17:38:58,149 : INFO : PROGRESS: at sentence #590000, processed 13195097 words, keeping 108501 word types\n",
      "2018-01-28 17:38:58,246 : INFO : PROGRESS: at sentence #600000, processed 13417295 words, keeping 109218 word types\n",
      "2018-01-28 17:38:58,333 : INFO : PROGRESS: at sentence #610000, processed 13638318 words, keeping 110092 word types\n",
      "2018-01-28 17:38:58,422 : INFO : PROGRESS: at sentence #620000, processed 13864643 words, keeping 110837 word types\n",
      "2018-01-28 17:38:58,514 : INFO : PROGRESS: at sentence #630000, processed 14088929 words, keeping 111610 word types\n",
      "2018-01-28 17:38:58,597 : INFO : PROGRESS: at sentence #640000, processed 14309712 words, keeping 112416 word types\n",
      "2018-01-28 17:38:58,688 : INFO : PROGRESS: at sentence #650000, processed 14535468 words, keeping 113196 word types\n",
      "2018-01-28 17:38:58,808 : INFO : PROGRESS: at sentence #660000, processed 14758258 words, keeping 113945 word types\n",
      "2018-01-28 17:38:58,920 : INFO : PROGRESS: at sentence #670000, processed 14981651 words, keeping 114643 word types\n",
      "2018-01-28 17:38:59,019 : INFO : PROGRESS: at sentence #680000, processed 15206483 words, keeping 115354 word types\n",
      "2018-01-28 17:38:59,116 : INFO : PROGRESS: at sentence #690000, processed 15428676 words, keeping 116131 word types\n",
      "2018-01-28 17:38:59,227 : INFO : PROGRESS: at sentence #700000, processed 15657382 words, keeping 116943 word types\n",
      "2018-01-28 17:38:59,343 : INFO : PROGRESS: at sentence #710000, processed 15880371 words, keeping 117596 word types\n",
      "2018-01-28 17:38:59,453 : INFO : PROGRESS: at sentence #720000, processed 16105658 words, keeping 118221 word types\n",
      "2018-01-28 17:38:59,543 : INFO : PROGRESS: at sentence #730000, processed 16332039 words, keeping 118954 word types\n",
      "2018-01-28 17:38:59,639 : INFO : PROGRESS: at sentence #740000, processed 16553072 words, keeping 119668 word types\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-01-28 17:38:59,728 : INFO : PROGRESS: at sentence #750000, processed 16771399 words, keeping 120295 word types\n",
      "2018-01-28 17:38:59,836 : INFO : PROGRESS: at sentence #760000, processed 16990803 words, keeping 120930 word types\n",
      "2018-01-28 17:38:59,941 : INFO : PROGRESS: at sentence #770000, processed 17217940 words, keeping 121703 word types\n",
      "2018-01-28 17:39:00,053 : INFO : PROGRESS: at sentence #780000, processed 17448086 words, keeping 122402 word types\n",
      "2018-01-28 17:39:00,141 : INFO : PROGRESS: at sentence #790000, processed 17675162 words, keeping 123066 word types\n",
      "2018-01-28 17:39:00,193 : INFO : collected 123504 word types from a corpus of 17798263 raw words and 795538 sentences\n",
      "2018-01-28 17:39:00,195 : INFO : Loading a fresh vocabulary\n",
      "2018-01-28 17:39:00,307 : INFO : min_count=40 retains 16490 unique words (13% of original 123504, drops 107014)\n",
      "2018-01-28 17:39:00,308 : INFO : min_count=40 leaves 17239118 word corpus (96% of original 17798263, drops 559145)\n",
      "2018-01-28 17:39:00,427 : INFO : deleting the raw counts dictionary of 123504 items\n",
      "2018-01-28 17:39:00,434 : INFO : sample=0.001 downsamples 48 most-common words\n",
      "2018-01-28 17:39:00,436 : INFO : downsampling leaves estimated 12749794 word corpus (74.0% of prior 17239118)\n",
      "2018-01-28 17:39:00,437 : INFO : estimated required memory for 16490 words and 300 dimensions: 47821000 bytes\n",
      "2018-01-28 17:39:00,521 : INFO : resetting layer weights\n",
      "2018-01-28 17:39:00,895 : INFO : training model with 6 workers on 16490 vocabulary and 300 features, using sg=0 hs=0 sample=0.001 negative=5 window=10\n",
      "2018-01-28 17:39:01,910 : INFO : PROGRESS: at 0.63% examples, 399029 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:02,912 : INFO : PROGRESS: at 1.38% examples, 438273 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:03,927 : INFO : PROGRESS: at 2.12% examples, 445038 words/s, in_qsize 12, out_qsize 1\n",
      "2018-01-28 17:39:04,928 : INFO : PROGRESS: at 2.87% examples, 451700 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:05,935 : INFO : PROGRESS: at 3.46% examples, 435222 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:39:06,987 : INFO : PROGRESS: at 4.26% examples, 443386 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:07,990 : INFO : PROGRESS: at 5.04% examples, 451306 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:08,995 : INFO : PROGRESS: at 5.81% examples, 455267 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:10,005 : INFO : PROGRESS: at 6.50% examples, 452565 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:11,042 : INFO : PROGRESS: at 7.16% examples, 447145 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:12,046 : INFO : PROGRESS: at 7.86% examples, 447314 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:13,072 : INFO : PROGRESS: at 8.52% examples, 444253 words/s, in_qsize 12, out_qsize 1\n",
      "2018-01-28 17:39:14,088 : INFO : PROGRESS: at 9.18% examples, 441967 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:15,095 : INFO : PROGRESS: at 9.84% examples, 440875 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:16,095 : INFO : PROGRESS: at 10.33% examples, 432080 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:17,131 : INFO : PROGRESS: at 10.99% examples, 430469 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:18,141 : INFO : PROGRESS: at 11.65% examples, 430126 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:19,145 : INFO : PROGRESS: at 12.27% examples, 428422 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:20,149 : INFO : PROGRESS: at 12.98% examples, 429469 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:21,170 : INFO : PROGRESS: at 13.69% examples, 430106 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:22,195 : INFO : PROGRESS: at 14.37% examples, 430219 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:23,213 : INFO : PROGRESS: at 15.18% examples, 433364 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:39:24,214 : INFO : PROGRESS: at 15.93% examples, 435298 words/s, in_qsize 12, out_qsize 1\n",
      "2018-01-28 17:39:25,229 : INFO : PROGRESS: at 16.72% examples, 438014 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:26,239 : INFO : PROGRESS: at 17.48% examples, 439754 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:39:27,253 : INFO : PROGRESS: at 18.26% examples, 441827 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:28,260 : INFO : PROGRESS: at 19.00% examples, 442550 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:29,278 : INFO : PROGRESS: at 19.79% examples, 444594 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:30,280 : INFO : PROGRESS: at 20.56% examples, 446219 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:31,292 : INFO : PROGRESS: at 21.34% examples, 447590 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:32,321 : INFO : PROGRESS: at 22.12% examples, 448623 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:33,338 : INFO : PROGRESS: at 22.92% examples, 450239 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:34,342 : INFO : PROGRESS: at 23.72% examples, 451671 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:35,363 : INFO : PROGRESS: at 24.55% examples, 453666 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:36,375 : INFO : PROGRESS: at 25.33% examples, 454621 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:37,389 : INFO : PROGRESS: at 26.15% examples, 456285 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:38,390 : INFO : PROGRESS: at 26.94% examples, 457446 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:39,394 : INFO : PROGRESS: at 27.72% examples, 458526 words/s, in_qsize 12, out_qsize 2\n",
      "2018-01-28 17:39:40,418 : INFO : PROGRESS: at 28.54% examples, 459856 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:41,449 : INFO : PROGRESS: at 29.29% examples, 459972 words/s, in_qsize 12, out_qsize 1\n",
      "2018-01-28 17:39:42,458 : INFO : PROGRESS: at 30.09% examples, 461207 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:43,459 : INFO : PROGRESS: at 30.89% examples, 462285 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:44,490 : INFO : PROGRESS: at 31.68% examples, 463151 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:45,491 : INFO : PROGRESS: at 32.44% examples, 463664 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:46,509 : INFO : PROGRESS: at 33.25% examples, 464600 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:47,529 : INFO : PROGRESS: at 34.04% examples, 465326 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:48,540 : INFO : PROGRESS: at 34.84% examples, 466106 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:49,554 : INFO : PROGRESS: at 35.66% examples, 467279 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:50,562 : INFO : PROGRESS: at 36.45% examples, 467867 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:51,585 : INFO : PROGRESS: at 37.26% examples, 468582 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:52,599 : INFO : PROGRESS: at 38.06% examples, 469346 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:53,608 : INFO : PROGRESS: at 38.88% examples, 470127 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:39:54,642 : INFO : PROGRESS: at 39.70% examples, 470941 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:55,655 : INFO : PROGRESS: at 40.49% examples, 471504 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:56,665 : INFO : PROGRESS: at 41.30% examples, 472191 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:57,673 : INFO : PROGRESS: at 42.06% examples, 472251 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:58,673 : INFO : PROGRESS: at 42.88% examples, 472999 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:39:59,679 : INFO : PROGRESS: at 43.66% examples, 473298 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:00,698 : INFO : PROGRESS: at 44.48% examples, 473847 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:01,709 : INFO : PROGRESS: at 45.26% examples, 474191 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:02,734 : INFO : PROGRESS: at 46.08% examples, 474763 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:03,736 : INFO : PROGRESS: at 46.89% examples, 475264 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:04,754 : INFO : PROGRESS: at 47.70% examples, 475864 words/s, in_qsize 12, out_qsize 0\n",
      "2018-01-28 17:40:05,764 : INFO : PROGRESS: at 48.47% examples, 476057 words/s, in_qsize 11, out_qsize 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-01-28 17:40:06,771 : INFO : PROGRESS: at 49.30% examples, 476803 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:07,777 : INFO : PROGRESS: at 50.08% examples, 477122 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:08,780 : INFO : PROGRESS: at 50.89% examples, 477654 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:09,784 : INFO : PROGRESS: at 51.70% examples, 478367 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:10,798 : INFO : PROGRESS: at 52.50% examples, 478697 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:11,815 : INFO : PROGRESS: at 53.28% examples, 478893 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:12,828 : INFO : PROGRESS: at 54.08% examples, 479210 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:13,835 : INFO : PROGRESS: at 54.89% examples, 479749 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:14,836 : INFO : PROGRESS: at 55.69% examples, 480126 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:15,839 : INFO : PROGRESS: at 56.50% examples, 480563 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:16,854 : INFO : PROGRESS: at 57.28% examples, 480732 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:40:17,859 : INFO : PROGRESS: at 58.10% examples, 481236 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:18,874 : INFO : PROGRESS: at 58.91% examples, 481574 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:19,878 : INFO : PROGRESS: at 59.70% examples, 481889 words/s, in_qsize 12, out_qsize 0\n",
      "2018-01-28 17:40:20,900 : INFO : PROGRESS: at 60.50% examples, 482167 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:21,906 : INFO : PROGRESS: at 61.30% examples, 482447 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:22,915 : INFO : PROGRESS: at 62.07% examples, 482440 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:23,940 : INFO : PROGRESS: at 62.88% examples, 482609 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:24,942 : INFO : PROGRESS: at 63.69% examples, 482974 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:25,951 : INFO : PROGRESS: at 64.50% examples, 483213 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:26,953 : INFO : PROGRESS: at 65.28% examples, 483404 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:27,965 : INFO : PROGRESS: at 66.00% examples, 483042 words/s, in_qsize 9, out_qsize 3\n",
      "2018-01-28 17:40:28,988 : INFO : PROGRESS: at 66.73% examples, 482622 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:30,001 : INFO : PROGRESS: at 67.43% examples, 482190 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:31,004 : INFO : PROGRESS: at 68.08% examples, 481430 words/s, in_qsize 11, out_qsize 1\n",
      "2018-01-28 17:40:32,006 : INFO : PROGRESS: at 68.73% examples, 480687 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:33,010 : INFO : PROGRESS: at 69.53% examples, 481034 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:34,013 : INFO : PROGRESS: at 70.31% examples, 481159 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:35,025 : INFO : PROGRESS: at 71.11% examples, 481451 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:36,032 : INFO : PROGRESS: at 71.77% examples, 480863 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:37,046 : INFO : PROGRESS: at 72.47% examples, 480476 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:38,070 : INFO : PROGRESS: at 73.19% examples, 480124 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:39,100 : INFO : PROGRESS: at 73.92% examples, 479819 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:40:40,105 : INFO : PROGRESS: at 74.63% examples, 479575 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:41,120 : INFO : PROGRESS: at 75.27% examples, 478711 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:42,146 : INFO : PROGRESS: at 75.97% examples, 478311 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:43,163 : INFO : PROGRESS: at 76.68% examples, 477959 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:44,193 : INFO : PROGRESS: at 77.38% examples, 477554 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:45,198 : INFO : PROGRESS: at 78.13% examples, 477546 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:46,204 : INFO : PROGRESS: at 78.90% examples, 477601 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:47,207 : INFO : PROGRESS: at 79.63% examples, 477534 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:48,222 : INFO : PROGRESS: at 80.40% examples, 477616 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:49,237 : INFO : PROGRESS: at 81.19% examples, 477763 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:50,244 : INFO : PROGRESS: at 82.03% examples, 478197 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:51,246 : INFO : PROGRESS: at 82.80% examples, 478268 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:52,264 : INFO : PROGRESS: at 83.64% examples, 478648 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:53,270 : INFO : PROGRESS: at 84.44% examples, 478883 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:54,287 : INFO : PROGRESS: at 85.23% examples, 478997 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:55,297 : INFO : PROGRESS: at 86.02% examples, 479206 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:56,299 : INFO : PROGRESS: at 86.86% examples, 479563 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:40:57,301 : INFO : PROGRESS: at 87.63% examples, 479677 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:58,311 : INFO : PROGRESS: at 88.44% examples, 479994 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:40:59,313 : INFO : PROGRESS: at 89.21% examples, 480099 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:00,317 : INFO : PROGRESS: at 90.02% examples, 480440 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:41:01,344 : INFO : PROGRESS: at 90.83% examples, 480616 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:02,348 : INFO : PROGRESS: at 91.60% examples, 480761 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:03,349 : INFO : PROGRESS: at 92.41% examples, 481043 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:04,349 : INFO : PROGRESS: at 93.18% examples, 481141 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:05,350 : INFO : PROGRESS: at 93.99% examples, 481410 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:06,352 : INFO : PROGRESS: at 94.79% examples, 481671 words/s, in_qsize 10, out_qsize 1\n",
      "2018-01-28 17:41:07,382 : INFO : PROGRESS: at 95.59% examples, 481767 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:08,406 : INFO : PROGRESS: at 96.40% examples, 481934 words/s, in_qsize 9, out_qsize 2\n",
      "2018-01-28 17:41:09,439 : INFO : PROGRESS: at 97.23% examples, 482182 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:10,470 : INFO : PROGRESS: at 98.04% examples, 482372 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:11,487 : INFO : PROGRESS: at 98.87% examples, 482613 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:12,517 : INFO : PROGRESS: at 99.66% examples, 482691 words/s, in_qsize 11, out_qsize 0\n",
      "2018-01-28 17:41:12,866 : INFO : worker thread finished; awaiting finish of 5 more threads\n",
      "2018-01-28 17:41:12,882 : INFO : worker thread finished; awaiting finish of 4 more threads\n",
      "2018-01-28 17:41:12,893 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2018-01-28 17:41:12,908 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2018-01-28 17:41:12,915 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2018-01-28 17:41:12,918 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2018-01-28 17:41:12,920 : INFO : training on 88991315 raw words (63748884 effective words) took 132.0s, 482893 effective words/s\n",
      "2018-01-28 17:41:12,922 : INFO : precomputing L2-norms of word weight vectors\n",
      "2018-01-28 17:41:13,179 : INFO : saving Word2Vec object under 300features_40minwords_10context, separately None\n",
      "2018-01-28 17:41:13,180 : INFO : not storing attribute syn0norm\n",
      "2018-01-28 17:41:13,182 : INFO : not storing attribute cum_table\n",
      "2018-01-28 17:41:13,572 : INFO : saved 300features_40minwords_10context\n"
     ]
    }
   ],
   "source": [
    "# Import the built-in logging module and configure it so that Word2Vec \n",
    "# creates nice output messages\n",
    "import logging\n",
    "logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',\\\n",
    "    level=logging.INFO)\n",
    "\n",
    "# Set values for various parameters\n",
    "num_features = 300    # Word vector dimensionality                      \n",
    "min_word_count = 40   # Minimum word count                        \n",
    "num_workers = 6       # Number of threads to run in parallel\n",
    "context = 10          # Context window size                                                                                    \n",
    "downsampling = 1e-3   # Downsample setting for frequent words\n",
    "\n",
    "# Initialize and train the model (this will take some time)\n",
    "from gensim.models import word2vec\n",
    "print(\"Training model...\")\n",
    "model = word2vec.Word2Vec(sentences, workers=num_workers, \\\n",
    "            size=num_features, min_count = min_word_count, \\\n",
    "            window = context, sample = downsampling)\n",
    "\n",
    "# If you don't plan to train the model any further, calling \n",
    "# init_sims will make the model much more memory-efficient.\n",
    "model.init_sims(replace=True)\n",
    "\n",
    "# It can be helpful to create a meaningful model name and \n",
    "# save the model for later use. You can load it later using Word2Vec.load()\n",
    "model_name = \"300features_40minwords_10context\"\n",
    "model.save(model_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'kitchen'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.doesnt_match(\"man woman child kitchen\".split())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'berlin'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.doesnt_match(\"france england germany berlin\".split())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('terrible', 0.7741101980209351),\n",
       " ('atrocious', 0.741277813911438),\n",
       " ('horrible', 0.7327954769134521),\n",
       " ('abysmal', 0.7309484481811523),\n",
       " ('dreadful', 0.7246018648147583),\n",
       " ('appalling', 0.680578887462616),\n",
       " ('horrendous', 0.6762707829475403),\n",
       " ('horrid', 0.6622831225395203),\n",
       " ('lousy', 0.6418094038963318),\n",
       " ('amateurish', 0.61811363697052)]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.most_similar(\"awful\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'paris'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.doesnt_match(\"paris berlin london austria\".split())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-01-28 17:47:49,379 : INFO : loading Word2Vec object from 300features_40minwords_10context\n",
      "2018-01-28 17:47:49,652 : INFO : loading wv recursively from 300features_40minwords_10context.wv.* with mmap=None\n",
      "2018-01-28 17:47:49,654 : INFO : setting ignored attribute syn0norm to None\n",
      "2018-01-28 17:47:49,655 : INFO : setting ignored attribute cum_table to None\n",
      "2018-01-28 17:47:49,656 : INFO : loaded 300features_40minwords_10context\n"
     ]
    }
   ],
   "source": [
    "# Load the model that we created in Part 2\n",
    "from gensim.models import Word2Vec\n",
    "model = Word2Vec.load(\"300features_40minwords_10context\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  9.69553515e-02,   4.92838845e-02,  -6.73670024e-02,\n",
       "        -1.53345615e-01,   1.95551366e-02,  -8.07441920e-02,\n",
       "        -6.14578612e-02,  -1.36809628e-02,   4.44527045e-02,\n",
       "        -5.24167195e-02,   6.32067323e-02,  -3.78362089e-02,\n",
       "        -4.67547365e-02,  -1.41874952e-02,   1.14649348e-02,\n",
       "        -2.57511213e-02,  -1.98574346e-02,  -6.51483908e-02,\n",
       "        -1.14198297e-01,   5.99479750e-02,  -1.14032105e-01,\n",
       "        -4.74299751e-02,  -4.89998646e-02,   2.47632270e-03,\n",
       "         3.12357806e-02,  -2.77043935e-02,   8.84354673e-03,\n",
       "         5.01944609e-02,   5.70105873e-02,  -1.24108442e-03,\n",
       "        -4.24729139e-02,  -6.90879896e-02,   3.93572859e-02,\n",
       "        -1.20190345e-02,  -1.05416283e-01,  -3.79511304e-02,\n",
       "        -5.15912361e-02,  -6.77533215e-03,  -1.66365393e-02,\n",
       "         1.11106902e-01,   6.68367594e-02,   6.46341071e-02,\n",
       "        -4.68867384e-02,  -3.31970155e-02,   1.03970291e-02,\n",
       "         1.51010463e-02,  -2.17614081e-02,  -3.27847116e-02,\n",
       "         5.18615078e-03,  -3.08635756e-02,  -3.70131396e-02,\n",
       "        -4.96021099e-02,   4.59346361e-03,  -1.34515157e-02,\n",
       "         1.69179756e-02,   1.66675702e-01,  -7.03025982e-02,\n",
       "        -6.54496104e-02,  -2.31736973e-02,   1.84433088e-02,\n",
       "        -5.34142833e-04,  -7.50509128e-02,  -6.74867555e-02,\n",
       "        -6.87828520e-03,  -5.82785495e-02,   8.15067589e-02,\n",
       "         1.96095519e-02,   7.78857097e-02,   1.40340887e-02,\n",
       "        -4.81825482e-05,  -7.37227648e-02,  -4.36792187e-02,\n",
       "        -7.23650753e-02,  -1.70922820e-02,   1.01114780e-01,\n",
       "        -9.65598524e-02,   1.87324695e-02,   8.98239389e-03,\n",
       "        -1.06081441e-02,   4.53123488e-02,   2.67432891e-02,\n",
       "         1.21558085e-02,  -2.74295658e-02,  -1.40912503e-01,\n",
       "        -6.30872399e-02,   8.49828050e-02,   4.00019586e-02,\n",
       "         2.80457418e-02,   5.24044707e-02,  -4.25699167e-02,\n",
       "         7.75380107e-03,  -8.43040943e-02,  -2.17896141e-02,\n",
       "         1.87671762e-02,   3.32920589e-02,  -6.91094324e-02,\n",
       "         1.85137987e-02,  -1.08663570e-02,   2.16859393e-02,\n",
       "         1.26684587e-02,   1.20974980e-01,  -9.64547768e-02,\n",
       "         9.36978962e-03,   2.66931020e-03,   1.90574434e-02,\n",
       "         3.63732432e-03,   2.77100727e-02,   8.12758803e-02,\n",
       "         2.62813247e-03,   1.84525317e-03,   3.50642614e-02,\n",
       "         1.15501892e-03,   6.31750003e-02,   5.57333268e-02,\n",
       "         1.00093566e-01,   2.57272739e-02,   1.59198593e-03,\n",
       "        -2.44334657e-02,   1.75533891e-02,   6.84264153e-02,\n",
       "         8.21762625e-03,   6.19971529e-02,  -1.21543415e-01,\n",
       "         2.01093182e-02,  -3.41904424e-02,   1.00837667e-02,\n",
       "         2.91616358e-02,   1.11745447e-02,   1.56950522e-02,\n",
       "        -2.06220727e-02,  -1.18599802e-01,  -4.48952951e-02,\n",
       "        -5.30369654e-02,   1.59285948e-01,  -5.81206493e-02,\n",
       "         3.04355677e-02,   1.41447093e-02,  -3.15094218e-02,\n",
       "         1.66936368e-01,   1.60851218e-02,  -1.09731471e-02,\n",
       "         7.36989528e-02,  -9.50221345e-03,   7.67644569e-02,\n",
       "         1.26751922e-02,  -7.22781792e-02,   7.27755278e-02,\n",
       "        -3.72999795e-02,  -8.51948932e-02,   4.83974218e-02,\n",
       "        -6.05681464e-02,  -5.83723607e-03,   4.54516113e-02,\n",
       "         4.30254340e-02,  -3.28833833e-02,  -1.36285638e-02,\n",
       "        -3.31400381e-03,   3.78298052e-02,  -2.78346110e-02,\n",
       "         9.85038579e-02,  -4.48835194e-02,   3.97099881e-03,\n",
       "        -3.29626836e-02,  -6.24940246e-02,  -4.44760434e-02,\n",
       "         4.35462520e-02,   7.19824107e-03,   1.64811850e-01,\n",
       "        -1.66277468e-01,  -2.98136547e-02,  -6.46835268e-02,\n",
       "         8.03391449e-03,   1.25552863e-01,  -6.47489801e-02,\n",
       "        -1.73357520e-02,  -8.32063854e-02,  -6.05713911e-02,\n",
       "        -9.29322187e-03,   2.60421354e-02,  -6.19801059e-02,\n",
       "        -3.96040119e-02,   1.49373516e-01,   7.77761340e-02,\n",
       "        -7.54793454e-03,   4.14819680e-02,  -3.16713080e-02,\n",
       "        -8.07341002e-03,  -9.13824663e-02,   7.93391764e-02,\n",
       "        -8.60175118e-03,   8.16265419e-02,   8.10887888e-02,\n",
       "         7.89608806e-02,  -8.40203743e-03,  -4.31708619e-02,\n",
       "         6.02057502e-02,  -4.62326780e-02,   7.88846165e-02,\n",
       "         6.65386319e-02,   4.83069830e-02,  -4.77133021e-02,\n",
       "        -8.15227777e-02,  -2.63391458e-03,  -5.81673943e-02,\n",
       "        -5.48278391e-02,  -2.40905471e-02,  -2.12516189e-02,\n",
       "        -3.25199366e-02,   6.85278177e-02,  -8.34854245e-02,\n",
       "        -3.90139110e-02,  -1.34667248e-01,   2.66760290e-02,\n",
       "        -4.65327017e-02,   1.30426222e-02,   6.13243654e-02,\n",
       "         4.50791307e-02,  -1.21483959e-01,   3.05953342e-02,\n",
       "         8.03667009e-02,   2.10659411e-02,  -8.06460604e-02,\n",
       "         7.82781281e-03,   7.59778246e-02,  -3.29880342e-02,\n",
       "         1.38975745e-02,  -7.01197311e-02,   7.47979358e-02,\n",
       "        -2.98366696e-02,   6.20604306e-03,   4.32075150e-02,\n",
       "        -9.08533186e-02,   4.58785705e-02,  -3.08170747e-02,\n",
       "        -2.04317849e-02,  -1.55823054e-02,   8.05313885e-02,\n",
       "        -1.63606629e-02,   2.76848767e-02,  -4.71248440e-02,\n",
       "        -2.87401266e-02,   9.11899954e-02,   9.06135961e-02,\n",
       "         5.69309555e-02,   2.20458233e-03,  -1.01121798e-01,\n",
       "        -3.44884326e-03,  -9.86941382e-02,  -7.39022791e-02,\n",
       "        -5.78689165e-02,   7.05976598e-03,  -8.12644735e-02,\n",
       "        -3.51299345e-02,  -1.11121990e-01,   3.81803513e-02,\n",
       "        -1.08967666e-02,   1.45097068e-02,  -1.04735885e-02,\n",
       "        -4.91847135e-02,   2.85750218e-02,   9.25643928e-03,\n",
       "        -8.13276228e-03,   3.98202352e-02,  -8.64937436e-03,\n",
       "         3.01343482e-03,  -5.25922689e-04,   5.29863760e-02,\n",
       "         3.32569443e-02,   1.49755867e-03,  -6.77014440e-02,\n",
       "        -8.00659508e-02,   6.14638366e-02,  -5.89858703e-02,\n",
       "        -2.48622447e-02,  -1.38854543e-02,  -4.39253673e-02,\n",
       "         5.02748489e-02,  -4.49225195e-02,  -5.30078113e-02,\n",
       "        -1.49963563e-02,   2.42127441e-02,   5.43537550e-02,\n",
       "        -3.87147442e-02,   9.45682824e-02,   6.53410852e-02,\n",
       "        -8.65837559e-02,  -2.81715728e-02,   7.78791821e-03,\n",
       "         1.19835325e-03,   7.39150587e-03,  -4.92669046e-02,\n",
       "         2.39007324e-02,  -3.67729254e-02,  -8.46352894e-03,\n",
       "        -3.88797261e-02,   5.04800177e-04,   3.79522424e-03,\n",
       "        -2.30229739e-02,  -4.87304963e-02,  -4.48953286e-02], dtype=float32)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model[\"flower\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np  # Make sure that numpy is imported\n",
    "\n",
    "def makeFeatureVec(words, model, num_features):\n",
    "    # Function to average all of the word vectors in a given\n",
    "    # paragraph\n",
    "    #\n",
    "    # Pre-initialize an empty numpy array (for speed)\n",
    "    featureVec = np.zeros((num_features,),dtype=\"float32\")\n",
    "    #\n",
    "    nwords = 0\n",
    "    # \n",
    "    # Index2word is a list that contains the names of the words in \n",
    "    # the model's vocabulary. Convert it to a set, for speed \n",
    "    index2word_set = set(model.wv.index2word)\n",
    "    #\n",
    "    # Loop over each word in the review and, if it is in the model's\n",
    "    # vocaublary, add its feature vector to the total\n",
    "    for word in words:\n",
    "        if word in index2word_set: \n",
    "            nwords = nwords + 1\n",
    "            featureVec = np.add(featureVec,model[word])\n",
    "    # \n",
    "    # Divide the result by the number of words to get the average\n",
    "    featureVec = np.divide(featureVec,nwords)\n",
    "    return featureVec\n",
    "\n",
    "\n",
    "def getAvgFeatureVecs(reviews, model, num_features):\n",
    "    # Given a set of reviews (each one a list of words), calculate \n",
    "    # the average feature vector for each one and return a 2D numpy array \n",
    "    # \n",
    "    # Initialize a counter\n",
    "    counter = 0\n",
    "    # \n",
    "    # Preallocate a 2D numpy array, for speed\n",
    "    reviewFeatureVecs = np.zeros((len(reviews),num_features),dtype=\"float32\")\n",
    "    # \n",
    "    # Loop through the reviews\n",
    "    for review in reviews:\n",
    "       #\n",
    "       # Print a status message every 1000th review\n",
    "       if counter%1000 == 0:\n",
    "           print(\"Review %d of %d\" % (counter, len(reviews)))\n",
    "       # \n",
    "       # Call the function (defined above) that makes average feature vectors\n",
    "       reviewFeatureVecs[counter] = makeFeatureVec(review, model, \\\n",
    "           num_features)\n",
    "       #\n",
    "       # Increment the counter\n",
    "       counter = counter + 1\n",
    "    return reviewFeatureVecs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Review 0 of 25000\n",
      "Review 1000 of 25000\n",
      "Review 2000 of 25000\n",
      "Review 3000 of 25000\n",
      "Review 4000 of 25000\n",
      "Review 5000 of 25000\n",
      "Review 6000 of 25000\n",
      "Review 7000 of 25000\n",
      "Review 8000 of 25000\n",
      "Review 9000 of 25000\n",
      "Review 10000 of 25000\n",
      "Review 11000 of 25000\n",
      "Review 12000 of 25000\n",
      "Review 13000 of 25000\n",
      "Review 14000 of 25000\n",
      "Review 15000 of 25000\n",
      "Review 16000 of 25000\n",
      "Review 17000 of 25000\n",
      "Review 18000 of 25000\n",
      "Review 19000 of 25000\n",
      "Review 20000 of 25000\n",
      "Review 21000 of 25000\n",
      "Review 22000 of 25000\n",
      "Review 23000 of 25000\n",
      "Review 24000 of 25000\n",
      "Creating average feature vecs for test reviews\n",
      "Review 0 of 25000\n",
      "Review 1000 of 25000\n",
      "Review 2000 of 25000\n",
      "Review 3000 of 25000\n",
      "Review 4000 of 25000\n",
      "Review 5000 of 25000\n",
      "Review 6000 of 25000\n",
      "Review 7000 of 25000\n",
      "Review 8000 of 25000\n",
      "Review 9000 of 25000\n",
      "Review 10000 of 25000\n",
      "Review 11000 of 25000\n",
      "Review 12000 of 25000\n",
      "Review 13000 of 25000\n",
      "Review 14000 of 25000\n",
      "Review 15000 of 25000\n",
      "Review 16000 of 25000\n",
      "Review 17000 of 25000\n",
      "Review 18000 of 25000\n",
      "Review 19000 of 25000\n",
      "Review 20000 of 25000\n",
      "Review 21000 of 25000\n",
      "Review 22000 of 25000\n",
      "Review 23000 of 25000\n",
      "Review 24000 of 25000\n"
     ]
    }
   ],
   "source": [
    "# ****************************************************************\n",
    "# Calculate average feature vectors for training and testing sets,\n",
    "# using the functions we defined above. Notice that we now use stop word\n",
    "# removal.\n",
    "\n",
    "clean_train_reviews = []\n",
    "for review in train[\"review\"]:\n",
    "    clean_train_reviews.append( review_to_wordlist( review, \\\n",
    "        remove_stopwords=True ))\n",
    "\n",
    "trainDataVecs = getAvgFeatureVecs( clean_train_reviews, model, num_features )\n",
    "\n",
    "print(\"Creating average feature vecs for test reviews\")\n",
    "clean_test_reviews = []\n",
    "for review in test[\"review\"]:\n",
    "    clean_test_reviews.append( review_to_wordlist( review, \\\n",
    "        remove_stopwords=True ))\n",
    "\n",
    "testDataVecs = getAvgFeatureVecs( clean_test_reviews, model, num_features )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}