Topic models (Python).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Topic models\n",
    "\n",
    "## Dataset for the exercise\n",
    "\n",
    "* [New York Times Comments](https://www.kaggle.com/aashita/nyt-comments/data), set of reders' comments for articles published in the New York Times\n",
    "\n",
    "## Overarching research question\n",
    "\n",
    "The comments allow a perspective to study what kind of concerns people raise when commenting to online articles.\n",
    "Examine if meaninful themes emerge from the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## data collection from files.\n",
    "## to keep the dataset fairly small, we conduct random data selection here.\n",
    "## this is *ONLY* to ensure that the model is suitable for teaching purposes\n",
    "\n",
    "import os\n",
    "import csv\n",
    "import random\n",
    "\n",
    "random.seed(1)\n",
    "\n",
    "path = 'data/nyt-comments/'\n",
    "files = os.listdir( path ) ## see all files in directory\n",
    "files = filter( lambda file_name: file_name.startswith(\"Comments\"), files ) ## choose only data files for April\n",
    "files = map( lambda file_name: path + file_name, files ) ## add path to file names\n",
    "\n",
    "documents = []\n",
    "\n",
    "for file in files:\n",
    "    for entry in csv.DictReader( open( file ) ):\n",
    "        \n",
    "        if random.random() > .99: ## choose content randomly\n",
    "            comment = entry['commentBody']\n",
    "\n",
    "            documents.append( comment )\n",
    "            \n",
    "            \n",
    "print(\"Data sample size\", len(documents) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## From text data to document-term matrix\n",
    "\n",
    "To analyse textual data we transform it to a document term matrix, where in rows we have documents (different comments) and columns represent each word in the dataset.\n",
    "\n",
    "Note how we **preprocess** the text during this quantification. We remove stopwords (through a set of common English stopwords; we could also create our own lists), stem the content of comments to ensure language is treated well and lower case everything in the content. Thus, the `document_terms` is a huge sparse matrix in the end. Preprocessing is its own kind of art, as it can [influence results](https://www.cambridge.org/core/product/identifier/S1047198717000444/type/journal_article)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "import nltk\n",
    "from nltk.stem.snowball import EnglishStemmer\n",
    "\n",
    "stemmer = EnglishStemmer()\n",
    "\n",
    "def stem( text ):\n",
    "    words = nltk.word_tokenize(text)\n",
    "    return [ stemmer.stem(w) for w in words ]\n",
    "    \n",
    "\n",
    "tf_vectorizer = CountVectorizer(max_df=0.90, min_df=10, stop_words='english', analyzer = stem, lowercase = True)\n",
    "\n",
    "document_terms = tf_vectorizer.fit_transform(documents)\n",
    "document_terms_names = tf_vectorizer.get_feature_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## From document-term matrix to analysis\n",
    "\n",
    "Finally we run the Latent Dirichlet Allocation process to our matrix to create topics.\n",
    "Similar to k-means, we choose the number of topics; there are also other parameters which could be used to _fine tune_ topic models, see [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.LatentDirichletAllocation.html) for details.\n",
    "When I use these we with their default parameters as none of them solves the challenge that [topic models work on a different abstration level than humans](http://doi.wiley.com/10.1002/asi.23786)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import LatentDirichletAllocation\n",
    "\n",
    "lda = LatentDirichletAllocation( n_components = 5 )\n",
    "model = lda.fit( document_terms )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for topic_number, words in enumerate( model.components_ ):\n",
    "        print( \"Topic\", topic_number+1 )\n",
    "        for word in words.argsort()[:-6:-1]:\n",
    "            print( \"\\t\", document_terms_names[word] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## see the distribution of a document to different topics\n",
    "model.transform( document_terms[0] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tasks\n",
    "\n",
    "* Compute the distribution of all documents to each topic. Where could you use this?\n",
    "* Modify the code and examine a few potential topic numbers. What differences can you detect?\n",
    "* Modify the preprocessing and remove all words which are shorter than four characters. What do you learn now?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model evaluation\n",
    "\n",
    "There are many different approaches to evaluate topic models (see, [1](http://doi.acm.org/10.1145/1553374.1553515), [2](https://journal.fi/politiikka/article/view/79629) for examples).\n",
    "We can evaluate the suitability of topic models using statistical measurements like loglikelihood, but [some say](http://www.umiacs.umd.edu/~jbg/docs/nips2009-rtl.pdf) that this might be a bad practice - and [others](https://journal.fi/politiikka/article/view/79629) recommend it.\n",
    "Here we show how to do it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.score( document_terms )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tasks\n",
    "\n",
    "* Evaluate a set of different topics based on this score. Which one would you choose?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}