Mobile_App_Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Profitable App Profiles for the App Store and Google Play Markets\n",
    "\n",
    "## Analyzing Mobile App Data\n",
    "\n",
    "This project is aimed towards analyzing mobile apps' data in order to understand what types of apps are more likely to attract more users. In this project, as hypothetical data analysts for an app development company, we are studying which apps are most likely to engage the most users, and therefore generate most revenue.\n",
    "\n",
    "We work towards providing free mobile apps on both the Apple App Store and Google Play, and our main source of revenue is advertisements within those applications. By finding out which mobile apps engage the most users, we can provide our software development team with sufficient information to maximize revenue."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Opening and Exploring the Data\n",
    "\n",
    "As of September 2018, there approximately 2 million iOS apps available on the Apple App Store and 2.1 million Android apps on Google play. The statistic is shown in the following graph:\n",
    "![Number of Mobile Apps in Various Stores](https://s3.amazonaws.com/dq-content/350/py1m8_statista.png) <p style=\"text-align:center\"> Source: <a href=\"https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/\" target=\"_blank\"> Statista </a></p>\n",
    "\n",
    "Given that collecting data from over 4 million applications requires a significant amount of time and money, we will try analyzing a smaller sample of the data instead. For that, we can use the following sources:\n",
    "- [This dataset](https://www.kaggle.com/lava18/google-play-store-apps) contains data (collected in August 2018) about approximately 10,000 Android apps available on Google Play. The dataset can be downloaded directly from [this link](https://dq-content.s3.amazonaws.com/350/googleplaystore.csv).\n",
    "- [This dataset](https://www.kaggle.com/ramamet4/app-store-apple-data-set-10k-apps) contains data (collected in July 2017) about approximately 7,000 iOS apps available on the App Store. The dataset can be downloaded directly from [this link](https://dq-content.s3.amazonaws.com/350/AppleStore.csv).\n",
    "\n",
    "Let's start by opening and exploring these two data sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from csv import reader\n",
    "\n",
    "#Starting with the Google Play data set\n",
    "opened_file = open(\"googleplaystore.csv\")\n",
    "read_file = reader(opened_file)\n",
    "android = list(read_file)\n",
    "android_heading = android[0]\n",
    "android = android[1:]\n",
    "\n",
    "#Now the App Store data set\n",
    "opened_file = open(\"AppleStore.csv\")\n",
    "read_file = reader(opened_file)\n",
    "ios = list(read_file)\n",
    "ios_heading = ios[0]\n",
    "ios = ios[1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To explore the data sets more efficiently, we create a function called `explore_data()` that we can use to repeatedly print rows in a readable way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def explore_data(dataset, start, end, rows_and_cols=False):\n",
    "    a_slice = dataset[start:end]\n",
    "    for row in a_slice:\n",
    "        print(row)\n",
    "        print('\\n') #To seperate each row\n",
    "    \n",
    "    if rows_and_cols:\n",
    "        print(\"Number of rows: \", len(dataset))\n",
    "        print(\"Number of columns: \", len(dataset[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's print the headings row and the first 3 rows of the andriod apps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['App', 'Category', 'Rating', 'Reviews', 'Size', 'Installs', 'Type', 'Price', 'Content Rating', 'Genres', 'Last Updated', 'Current Ver', 'Android Ver']\n",
      "\n",
      "\n",
      "['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['Coloring book moana', 'ART_AND_DESIGN', '3.9', '967', '14M', '500,000+', 'Free', '0', 'Everyone', 'Art & Design;Pretend Play', 'January 15, 2018', '2.0.0', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up']\n",
      "\n",
      "\n",
      "Number of rows:  10841\n",
      "Number of columns:  13\n"
     ]
    }
   ],
   "source": [
    "print(android_heading)\n",
    "print('\\n')\n",
    "explore_data(android, 0, 3, rows_and_cols=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that this dataset has 10,841 andriod apps, and their are 13 categories for each:\n",
    "\n",
    "| Category | Description |\n",
    "| :--- | :---: |\n",
    "| \"App\" | App Name|\n",
    "| \"Category\" | App Category |\n",
    "| \"Rating\" | Average App Rating |\n",
    "| \"Reviews\" | Total Number of Reviews |\n",
    "| \"Size\" | App Size (in Megabytes) |\n",
    "| \"Installs\" | Number of Installations |\n",
    "| \"Type\" | Free or Paid |\n",
    "| \"Price\" | Price of App in USD |\n",
    "| \"Content Rating\" | Minimum Age Requirement |\n",
    "| \"Genres\" | All Generes of App (including Category) |\n",
    "| \"Last Updated\" | Date of Last App Update |\n",
    "| \"Current ver\" | Current Version of App |\n",
    "| \"Andriod ver\" | Minimum Required Andriod Version |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Next, let's explore the iOS apps data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['id', 'track_name', 'size_bytes', 'currency', 'price', 'rating_count_tot', 'rating_count_ver', 'user_rating', 'user_rating_ver', 'ver', 'cont_rating', 'prime_genre', 'sup_devices.num', 'ipadSc_urls.num', 'lang.num', 'vpp_lic']\n",
      "\n",
      "\n",
      "['284882215', 'Facebook', '389879808', 'USD', '0.0', '2974676', '212', '3.5', '3.5', '95.0', '4+', 'Social Networking', '37', '1', '29', '1']\n",
      "\n",
      "\n",
      "['389801252', 'Instagram', '113954816', 'USD', '0.0', '2161558', '1289', '4.5', '4.0', '10.23', '12+', 'Photo & Video', '37', '0', '29', '1']\n",
      "\n",
      "\n",
      "['529479190', 'Clash of Clans', '116476928', 'USD', '0.0', '2130805', '579', '4.5', '4.5', '9.24.12', '9+', 'Games', '38', '5', '18', '1']\n",
      "\n",
      "\n",
      "Number of rows:  7197\n",
      "Number of columns:  16\n"
     ]
    }
   ],
   "source": [
    "print(ios_heading)\n",
    "print(\"\\n\")\n",
    "explore_data(ios, 0, 3, rows_and_cols=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that his data set has 7,197 iOS apps, and there are 16 categories for each:\n",
    "\n",
    "| Column name | Description |\n",
    "| :--- | :---:|\n",
    "| \"id\" | App ID |\n",
    "| \"track_name\" | App Name |\n",
    "| \"size_bytes\" | App Size (in Bytes) |\n",
    "| \"currency\" | Currency For Price Column (USD) |\n",
    "| \"price\" | Price of App |\n",
    "| \"rating_count_tot\" | Number of User Ratings (All Versions) |\n",
    "| \"rating_count_ver\" | Number of User Ratings (Current Version) |\n",
    "| \"user_rating\" | Average User Rating (All Versions) |\n",
    "| \"user_rating_ver\" | Average User Rating (Current Version) |\n",
    "| \"ver\" | Current App Version |\n",
    "| \"cont_rating\" | Minimum Age Requirement |\n",
    "| \"prime_genre\" | Primary Genre of App |\n",
    "| \"sup_devices.num\" | Number of Devices Supported |\n",
    "| \"ipadSc_urls.num\" | Number of Screenshots Shown for Display |\n",
    "| \"lang.num\" | Number of Supported Languages |\n",
    "| \"vpp_lic\" | Vpp Device Based Licensing Enabled |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deleting Wrong Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Google Play dataset has a dedicated [discussion section](), and an issue with the data has been raised in [this discussion]().\n",
    "According to the discussion, entry 10472 has an incorrect rating value of 19 (maximum rating is 5). We can print the row to see that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Life Made WI-Fi Touchscreen Photo Frame', '1.9', '19', '3.0M', '1,000+', 'Free', '0', 'Everyone', '', 'February 11, 2018', '1.0.19', '4.0 and up']\n",
      "\n",
      "\n",
      "['App', 'Category', 'Rating', 'Reviews', 'Size', 'Installs', 'Type', 'Price', 'Content Rating', 'Genres', 'Last Updated', 'Current Ver', 'Android Ver']\n"
     ]
    }
   ],
   "source": [
    "print(android[10472])\n",
    "print('\\n')\n",
    "print(android_heading)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the error happens because the entry is missing a value in the `Category`. To fix this issue, we will delete entry nummber 10472."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10841\n",
      "10840\n"
     ]
    }
   ],
   "source": [
    "print(len(android)) #should print 10841\n",
    "del(android[10472]) #don't run more than once\n",
    "print(len(android)) #should print 10840"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upon inspecting the discussion section in the App Store dataset, there are no input errors. On the other hand, there are duplicates in the Google Play dataset. We can remove those for more accurate results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Removing Duplicate Entries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part One"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check for duplicate entries in the Google Play dataset, we can do the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of duplicate apps in the Google Play dataset:  1181\n",
      "\n",
      "\n",
      "Some duplicate apps:  ['Quick PDF Scanner + OCR FREE', 'Box', 'Google My Business', 'ZOOM Cloud Meetings', 'join.me - Simple Meetings', 'Box', 'Zenefits', 'Google Ads', 'Google My Business', 'Slack']\n"
     ]
    }
   ],
   "source": [
    "duplicate_apps = []\n",
    "unique_apps = []\n",
    "\n",
    "for app in android:\n",
    "    name = app[0]\n",
    "    if name in unique_apps:\n",
    "        duplicate_apps.append(name)\n",
    "    else:\n",
    "        unique_apps.append(name)\n",
    "        \n",
    "print(\"The number of duplicate apps in the Google Play dataset: \", len(duplicate_apps))\n",
    "print('\\n')\n",
    "print(\"Some duplicate apps: \", duplicate_apps[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's print some of the duplicates to see if they are exactly the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80805', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n",
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80805', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n",
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80804', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n"
     ]
    }
   ],
   "source": [
    "for app in android:\n",
    "    name = app[0]\n",
    "    if name == 'Quick PDF Scanner + OCR FREE':\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Google Ads', 'BUSINESS', '4.3', '29313', '20M', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 30, 2018', '1.12.0', '4.0.3 and up']\n",
      "['Google Ads', 'BUSINESS', '4.3', '29313', '20M', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 30, 2018', '1.12.0', '4.0.3 and up']\n",
      "['Google Ads', 'BUSINESS', '4.3', '29331', '20M', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 30, 2018', '1.12.0', '4.0.3 and up']\n"
     ]
    }
   ],
   "source": [
    "for app in android:\n",
    "    name = app[0]\n",
    "    if name == 'Google Ads':\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Slack', 'BUSINESS', '4.4', '51507', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'August 2, 2018', 'Varies with device', 'Varies with device']\n",
      "['Slack', 'BUSINESS', '4.4', '51507', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'August 2, 2018', 'Varies with device', 'Varies with device']\n",
      "['Slack', 'BUSINESS', '4.4', '51510', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'August 2, 2018', 'Varies with device', 'Varies with device']\n"
     ]
    }
   ],
   "source": [
    "for app in android:\n",
    "    name = app[0]\n",
    "    if name == 'Slack':\n",
    "        print(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above data, we can see that duplicate entries have the exact same data except for the fourth position of the row (index 3), the `Reviews` column. We can infer that the data was collected at different points in time, and therefore, it is safe to assume that the entries with the highest number of reviews are the most recent ones."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, we can keep the rows with the highest number of reviews and remove the other duplicate entries. To do that we will:\n",
    "- Create an empty dictionary where the key is the app name and the value is the maximum number of reviews\n",
    "- Use the dictionary to update the `andriod` data set to contain only the entries with the maximum number of reviews"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part Two"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_reviews = {}\n",
    "\n",
    "for app in android:\n",
    "    name = app[0]\n",
    "    reviews = float(app[3]) #cast to float since the reviews are stored as strings\n",
    "    if name in max_reviews and max_reviews[name] < reviews:\n",
    "            max_reviews[name] = reviews\n",
    "    else:\n",
    "        max_reviews[name] = reviews"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We saw previously that the duplicates in the Google Play dataset are 1181 apps. Therefore, the length of `max_reviews` should be the length of the `andriod` dataset minus 1181."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of andriod dataset without duplicates: 9659\n",
      "Length of max_reviews dictionary:  9659\n"
     ]
    }
   ],
   "source": [
    "print(\"Length of andriod dataset without duplicates:\", len(android) - 1181)\n",
    "print(\"Length of max_reviews dictionary: \", len(max_reviews))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will use the `max_reviews` dictionary to remove the duplicates from the dataset itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of cleaned dataset:  9659\n"
     ]
    }
   ],
   "source": [
    "android_clean = []\n",
    "already_added = []\n",
    "\n",
    "for app in android:\n",
    "    name = app[0]\n",
    "    n_reviews = float(app[3])\n",
    "    if max_reviews[name] == n_reviews and name not in already_added:\n",
    "        android_clean.append(app)\n",
    "        already_added.append(name)\n",
    "        \n",
    "print(\"Length of cleaned dataset: \", len(android_clean)) #expected: 9659"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the cleaned dataset has 9,659 apps/entries as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['Sketch - Draw & Paint', 'ART_AND_DESIGN', '4.5', '215644', '25M', '50,000,000+', 'Free', '0', 'Teen', 'Art & Design', 'June 8, 2018', 'Varies with device', '4.2 and up']\n",
      "\n",
      "\n",
      "Number of rows:  9659\n",
      "Number of columns:  13\n"
     ]
    }
   ],
   "source": [
    "explore_data(android_clean, 0, 3, True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['ZOOM Cloud Meetings', 'BUSINESS', '4.4', '31614', '37M', '10,000,000+', 'Free', '0', 'Everyone', 'Business', 'July 20, 2018', '4.1.28165.0716', '4.0 and up']\n",
      "\n",
      "\n",
      "['Quick PDF Scanner + OCR FREE', 'BUSINESS', '4.2', '80804', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'February 26, 2018', 'Varies with device', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['Slack', 'BUSINESS', '4.4', '51510', 'Varies with device', '5,000,000+', 'Free', '0', 'Everyone', 'Business', 'August 2, 2018', 'Varies with device', 'Varies with device']\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for app in android_clean:\n",
    "    name = app[0]\n",
    "    if name == 'Slack' or name == 'ZOOM Cloud Meetings' or name == 'Quick PDF Scanner + OCR FREE':\n",
    "        print(app)\n",
    "        print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that all duplicates have been removed successfully, and the remaining entries are the ones with the highest reviews."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check the App Store dataset to ensure that there are no duplicates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of apps in iOS dataset:  7197\n",
      "Number of unique apps in iOS dataset:  7197\n",
      "Number of duplicates in iOS dataset:  0\n"
     ]
    }
   ],
   "source": [
    "duplicate_ios = []\n",
    "unique_ios = []\n",
    "\n",
    "for app in ios:\n",
    "    app_id = app[0]\n",
    "    if app_id in unique_ios:\n",
    "        duplicate_ios.append(app_id)\n",
    "    else:\n",
    "        unique_ios.append(app_id)\n",
    "        \n",
    "print(\"Number of apps in iOS dataset: \", len(ios))\n",
    "print(\"Number of unique apps in iOS dataset: \", len(unique_ios))\n",
    "print(\"Number of duplicates in iOS dataset: \", len(duplicate_ios))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that there are no duplicates in the iOS dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Removing Non-English Apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part One"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After further exploring the datasets, we can see that some of the apps in both datasets have non-English names and are directed at a non-English speaking audience. However, we develop apps for an English-speaking audience at our company, so we will remove said apps from the datasets to obtain more relevant and accurate results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "English text usually includes letters from the English alphabet, numbers composed of digits from 0 to 9, punctuation marks (., !, ?, ;), and other symbols (+, \\*, /). It is safe to assume that any app name that contains a character other than those is not an app with an English name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The characters described above use standard ASCII encoding, which makes each character have a corresponding number between 0 and 127. The python `ord()` built-in function can check for that, and we can use it to filter out any apps with non-English characters and remove it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_english(string):\n",
    "    for char in string:\n",
    "        if ord(char) > 127:\n",
    "            return False\n",
    "        \n",
    "    return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can find certain apps in the dataset without english names and test the function with those apps. This can be done by manually looking through the dataset until a few apps are found."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "爱奇艺PPS -《欢乐颂2》电视剧热播\n",
      "【脱出ゲーム】絶対に最後までプレイしないで 〜謎解き＆ブロックパズル〜\n",
      "\n",
      "\n",
      "中国語 AQリスニング\n",
      "Instachat 😜\n",
      "Docs To Go™ Free Office Suite\n"
     ]
    }
   ],
   "source": [
    "print(ios[813][1])\n",
    "print(ios[6731][1])\n",
    "print('\\n')\n",
    "print(android_clean[4360][0])\n",
    "print(android_clean[3091][0])\n",
    "print(android_clean[180][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "False\n",
      "False\n",
      "False\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "print(is_english(ios[813][1]))\n",
    "print(is_english(ios[6731][1]))\n",
    "print(is_english(android_clean[4360][0]))\n",
    "print(is_english(android_clean[3091][0]))\n",
    "print(is_english(android_clean[180][0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that some apps that have English names have 'non-English' characters, such as `Instachat 😜` and `Docs To Go™ Free Office Suite`. Losing these apps could lead to the loss of useful data in the analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8482\n",
      "128540\n"
     ]
    }
   ],
   "source": [
    "print(ord('™'))\n",
    "print(ord('😜'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both characters return a value greater than 127."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part Two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To minimize data loss, we'll remove apps that have more than 3 ASCII characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "False\n",
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "def is_english(string):\n",
    "    non_ascii = 0\n",
    "    \n",
    "    for char in string:\n",
    "        if ord(char) > 127:\n",
    "            non_ascii += 1\n",
    "            \n",
    "    if non_ascii > 3:\n",
    "        return False\n",
    "    else:\n",
    "        return True\n",
    "    \n",
    "# name: 爱奇艺PPS -《欢乐颂2》电视剧热播, expected: False\n",
    "print(is_english(ios[813][1]))\n",
    "\n",
    "# name: 中国語 AQリスニング, expected: False\n",
    "print(is_english(android_clean[4360][0]))\n",
    "\n",
    "# name: Instachat 😜, expected: True\n",
    "print(is_english(android_clean[3091][0]))\n",
    "\n",
    "# name: Docs To Go™ Free Office Suite, expected: True\n",
    "print(is_english(android_clean[180][0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can use the updated `is_english()` function to filter out the apps with non-English names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['U Launcher Lite – FREE Live Cool Themes, Hide Apps', 'ART_AND_DESIGN', '4.7', '87510', '8.7M', '5,000,000+', 'Free', '0', 'Everyone', 'Art & Design', 'August 1, 2018', '1.2.4', '4.0.3 and up']\n",
      "\n",
      "\n",
      "['Sketch - Draw & Paint', 'ART_AND_DESIGN', '4.5', '215644', '25M', '50,000,000+', 'Free', '0', 'Teen', 'Art & Design', 'June 8, 2018', 'Varies with device', '4.2 and up']\n",
      "\n",
      "\n",
      "Number of rows:  9614\n",
      "Number of columns:  13\n",
      "\n",
      "\n",
      "['284882215', 'Facebook', '389879808', 'USD', '0.0', '2974676', '212', '3.5', '3.5', '95.0', '4+', 'Social Networking', '37', '1', '29', '1']\n",
      "\n",
      "\n",
      "['389801252', 'Instagram', '113954816', 'USD', '0.0', '2161558', '1289', '4.5', '4.0', '10.23', '12+', 'Photo & Video', '37', '0', '29', '1']\n",
      "\n",
      "\n",
      "['529479190', 'Clash of Clans', '116476928', 'USD', '0.0', '2130805', '579', '4.5', '4.5', '9.24.12', '9+', 'Games', '38', '5', '18', '1']\n",
      "\n",
      "\n",
      "Number of rows:  6183\n",
      "Number of columns:  16\n"
     ]
    }
   ],
   "source": [
    "android_eng = []\n",
    "ios_eng = []\n",
    "\n",
    "for app in android_clean:\n",
    "    name = app[0]\n",
    "    if is_english(name):\n",
    "        android_eng.append(app)\n",
    "        \n",
    "for app in ios:\n",
    "    name = app[1]\n",
    "    if is_english(name):\n",
    "        ios_eng.append(app)\n",
    "        \n",
    "explore_data(android_eng, 0, 3, True)\n",
    "print('\\n')\n",
    "explore_data(ios_eng, 0, 3, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After filtering out 'non-English' apps, we are left with 9614 android apps and 6183 iOS apps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Isolating the Free Apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As previously mentioned, we only build apps that are free to download and install, and our main source of revenue consists of in-app ads. Therefore, we should isolate the free apps from the data sets in order for our analysis to be more tailored to our requirements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final android dataset size:  8864\n",
      "Final iOS dataset size:  3222\n"
     ]
    }
   ],
   "source": [
    "android_free = []\n",
    "ios_free = []\n",
    "\n",
    "for app in android_eng:\n",
    "    price = app[7]\n",
    "    if price == \"0\":\n",
    "        android_free.append(app)\n",
    "        \n",
    "for app in ios_eng:\n",
    "    price = app[4]\n",
    "    if price == \"0.0\":\n",
    "        ios_free.append(app)\n",
    "        \n",
    "print(\"Final android dataset size: \", len(android_free))\n",
    "print(\"Final iOS dataset size: \", len(ios_free))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After cleaning both datasets, we have 8864 android apps remaining and 3222 iOS apps remaining."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Most Common Apps by Genre"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part One - Summary and Next Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we have cleaned the data by:\n",
    "- Removing inaccurate data\n",
    "- Removing duplicate app entries\n",
    "- Removing non-English spps\n",
    "- Isolating the free apps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our goal is to use the cleaned datasets in order to determine which apps are more likely to attract the most users, therefore generating the greatest revenue."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To minimize risks and overhead, our validation strategy has three steps:\n",
    "1. Build a minimal Android version of the app, and add it to Google Play\n",
    "2. If the app has good response from the users, we develop it further\n",
    "3. If the app is profitable after six months, we build an iOS version of the app and add it to the App Store."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the end goal is to add the app on both Google Play and the App Store, we need to look for app profiles that are successful in both markets. Let's begin the analysis by determining the most common genres for each market. To do this, we will build frequency tables for a few columns in our datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part Two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll build two functions we can use to analyze the frequency tables:\n",
    "- One function to generate frequency tables that show percentages\n",
    "- Another function we can use to display the percentages in descending order"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The columns that we will use those functions on are the `prime_genre` column of the App Store dataset, and the `Genres` and `Category` columns of the Google Play dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def freq_table(dataset, index):\n",
    "    n_table = {}\n",
    "    total = 0\n",
    "    \n",
    "    for row in dataset:\n",
    "        total += 1\n",
    "        value = row[index]\n",
    "        if value in n_table:\n",
    "            n_table[value] += 1\n",
    "        else:\n",
    "            n_table[value] = 1\n",
    "    \n",
    "    p_table={} # We convert the frequencies to percentages\n",
    "    for key in n_table:\n",
    "        percentage = (n_table[key] / total) * 100\n",
    "        p_table[key] = percentage\n",
    "        \n",
    "    return p_table\n",
    "\n",
    "def display_table(dataset, index):\n",
    "    table = freq_table(dataset, index)\n",
    "    table_display = []\n",
    "    for key in table:\n",
    "        key_to_tuple = (table[key], key)\n",
    "        table_display.append(key_to_tuple)\n",
    "    \n",
    "    table_sorted = sorted(table_display, reverse=True)\n",
    "    for entry in table_sorted:\n",
    "        print(entry[1], \":\", entry[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part Three"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the functions ready, we can start by analyzing the `prime_genre` column of the App Store dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Games : 58.16263190564867\n",
      "Entertainment : 7.883302296710118\n",
      "Photo & Video : 4.9658597144630665\n",
      "Education : 3.662321539416512\n",
      "Social Networking : 3.2898820608317814\n",
      "Shopping : 2.60707635009311\n",
      "Utilities : 2.5139664804469275\n",
      "Sports : 2.1415270018621975\n",
      "Music : 2.0484171322160147\n",
      "Health & Fitness : 2.0173805090006205\n",
      "Productivity : 1.7380509000620732\n",
      "Lifestyle : 1.5828677839851024\n",
      "News : 1.3345747982619491\n",
      "Travel : 1.2414649286157666\n",
      "Finance : 1.1173184357541899\n",
      "Weather : 0.8690254500310366\n",
      "Food & Drink : 0.8069522036002483\n",
      "Reference : 0.5586592178770949\n",
      "Business : 0.5276225946617008\n",
      "Book : 0.4345127250155183\n",
      "Navigation : 0.186219739292365\n",
      "Medical : 0.186219739292365\n",
      "Catalogs : 0.12414649286157665\n"
     ]
    }
   ],
   "source": [
    "display_table(ios_free, 11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see from the table above that of the free English apps in the App Store, over 58% of them are games. Entertainment followed with \\~7.88%, followed by photo & video (\\~4.97%), education (\\~3.66%) and social networking (\\~3.29%)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From this, we can develop a general impression that the App Store (at least the part with free and English apps) is dominated by apps that are designed for fun (games, entertainment, social media, photo & video, music, sports, etc.). However, just because the apps are more common on the App store doesn't necessarily imply that these apps have the greatest number of users. Sometimes, the supply and demand of apps don't match. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can look at the frequency table of the `Genres` and `Category` columns of the Google Play dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FAMILY : 19.223826714801444\n",
      "GAME : 9.510379061371841\n",
      "TOOLS : 8.461191335740072\n",
      "BUSINESS : 4.580324909747293\n",
      "LIFESTYLE : 3.9034296028880866\n",
      "PRODUCTIVITY : 3.892148014440433\n",
      "FINANCE : 3.7003610108303246\n",
      "MEDICAL : 3.5424187725631766\n",
      "SPORTS : 3.4183212996389893\n",
      "PERSONALIZATION : 3.3167870036101084\n",
      "COMMUNICATION : 3.2490974729241873\n",
      "HEALTH_AND_FITNESS : 3.068592057761733\n",
      "PHOTOGRAPHY : 2.944494584837545\n",
      "NEWS_AND_MAGAZINES : 2.7978339350180503\n",
      "SOCIAL : 2.6624548736462095\n",
      "TRAVEL_AND_LOCAL : 2.33528880866426\n",
      "SHOPPING : 2.2450361010830324\n",
      "BOOKS_AND_REFERENCE : 2.1435018050541514\n",
      "DATING : 1.861462093862816\n",
      "VIDEO_PLAYERS : 1.782490974729242\n",
      "MAPS_AND_NAVIGATION : 1.3989169675090252\n",
      "FOOD_AND_DRINK : 1.2409747292418771\n",
      "EDUCATION : 1.128158844765343\n",
      "LIBRARIES_AND_DEMO : 0.9363718411552346\n",
      "AUTO_AND_VEHICLES : 0.9250902527075812\n",
      "ENTERTAINMENT : 0.8799638989169676\n",
      "HOUSE_AND_HOME : 0.8235559566787004\n",
      "WEATHER : 0.8009927797833934\n",
      "EVENTS : 0.7107400722021661\n",
      "PARENTING : 0.6543321299638989\n",
      "ART_AND_DESIGN : 0.6430505415162455\n",
      "COMICS : 0.6204873646209386\n",
      "BEAUTY : 0.5979241877256317\n"
     ]
    }
   ],
   "source": [
    "display_table(android_free, 1) #Category column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Google Play generes are quire different than the App Store ones. Less apps on Google Play are designed for fun and more are aimed towards practical purposes (family, tools, business, lifestyle, productivity, etc.) relative to the App Store. However, after further investigation we found that the `Family` category (which accounts for the most apps on Google Play: more than 19%) represents games that are aimed towards younger children. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With that being said, while games are still the most common apps, it can still be seen that the categories are more versatile (greater representation) on Google Play. Next, we can look at the Genres column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tools : 8.449909747292418\n",
      "Entertainment : 6.069494584837545\n",
      "Education : 5.347472924187725\n",
      "Business : 4.580324909747293\n",
      "Productivity : 3.892148014440433\n",
      "Lifestyle : 3.892148014440433\n",
      "Finance : 3.7003610108303246\n",
      "Medical : 3.5424187725631766\n",
      "Sports : 3.463447653429603\n",
      "Personalization : 3.3167870036101084\n",
      "Communication : 3.2490974729241873\n",
      "Action : 3.1024368231046933\n",
      "Health & Fitness : 3.068592057761733\n",
      "Photography : 2.944494584837545\n",
      "News & Magazines : 2.7978339350180503\n",
      "Social : 2.6624548736462095\n",
      "Travel & Local : 2.3240072202166067\n",
      "Shopping : 2.2450361010830324\n",
      "Books & Reference : 2.1435018050541514\n",
      "Simulation : 2.0419675090252705\n",
      "Dating : 1.861462093862816\n",
      "Arcade : 1.861462093862816\n",
      "Video Players & Editors : 1.782490974729242\n",
      "Casual : 1.7486462093862816\n",
      "Maps & Navigation : 1.3989169675090252\n",
      "Food & Drink : 1.2409747292418771\n",
      "Puzzle : 1.128158844765343\n",
      "Racing : 0.9927797833935018\n",
      "Role Playing : 0.9363718411552346\n",
      "Libraries & Demo : 0.9363718411552346\n",
      "Auto & Vehicles : 0.9250902527075812\n",
      "Strategy : 0.9138086642599278\n",
      "House & Home : 0.8235559566787004\n",
      "Weather : 0.8009927797833934\n",
      "Events : 0.7107400722021661\n",
      "Adventure : 0.6768953068592057\n",
      "Comics : 0.6092057761732852\n",
      "Beauty : 0.5979241877256317\n",
      "Art & Design : 0.5979241877256317\n",
      "Parenting : 0.4963898916967509\n",
      "Card : 0.4399819494584838\n",
      "Casino : 0.42870036101083037\n",
      "Trivia : 0.41741877256317694\n",
      "Educational;Education : 0.39485559566787\n",
      "Board : 0.3835740072202166\n",
      "Educational : 0.3722924187725632\n",
      "Education;Education : 0.33844765342960287\n",
      "Word : 0.2594765342960289\n",
      "Casual;Pretend Play : 0.236913357400722\n",
      "Music : 0.2030685920577617\n",
      "Racing;Action & Adventure : 0.16922382671480143\n",
      "Puzzle;Brain Games : 0.16922382671480143\n",
      "Entertainment;Music & Video : 0.16922382671480143\n",
      "Casual;Brain Games : 0.13537906137184114\n",
      "Casual;Action & Adventure : 0.13537906137184114\n",
      "Arcade;Action & Adventure : 0.12409747292418773\n",
      "Action;Action & Adventure : 0.10153429602888085\n",
      "Educational;Pretend Play : 0.09025270758122744\n",
      "Simulation;Action & Adventure : 0.078971119133574\n",
      "Parenting;Education : 0.078971119133574\n",
      "Entertainment;Brain Games : 0.078971119133574\n",
      "Board;Brain Games : 0.078971119133574\n",
      "Parenting;Music & Video : 0.06768953068592057\n",
      "Educational;Brain Games : 0.06768953068592057\n",
      "Casual;Creativity : 0.06768953068592057\n",
      "Art & Design;Creativity : 0.06768953068592057\n",
      "Education;Pretend Play : 0.056407942238267145\n",
      "Role Playing;Pretend Play : 0.04512635379061372\n",
      "Education;Creativity : 0.04512635379061372\n",
      "Role Playing;Action & Adventure : 0.033844765342960284\n",
      "Puzzle;Action & Adventure : 0.033844765342960284\n",
      "Entertainment;Creativity : 0.033844765342960284\n",
      "Entertainment;Action & Adventure : 0.033844765342960284\n",
      "Educational;Creativity : 0.033844765342960284\n",
      "Educational;Action & Adventure : 0.033844765342960284\n",
      "Education;Music & Video : 0.033844765342960284\n",
      "Education;Brain Games : 0.033844765342960284\n",
      "Education;Action & Adventure : 0.033844765342960284\n",
      "Adventure;Action & Adventure : 0.033844765342960284\n",
      "Video Players & Editors;Music & Video : 0.02256317689530686\n",
      "Sports;Action & Adventure : 0.02256317689530686\n",
      "Simulation;Pretend Play : 0.02256317689530686\n",
      "Puzzle;Creativity : 0.02256317689530686\n",
      "Music;Music & Video : 0.02256317689530686\n",
      "Entertainment;Pretend Play : 0.02256317689530686\n",
      "Casual;Education : 0.02256317689530686\n",
      "Board;Action & Adventure : 0.02256317689530686\n",
      "Trivia;Education : 0.01128158844765343\n",
      "Travel & Local;Action & Adventure : 0.01128158844765343\n",
      "Tools;Education : 0.01128158844765343\n",
      "Strategy;Education : 0.01128158844765343\n",
      "Strategy;Creativity : 0.01128158844765343\n",
      "Strategy;Action & Adventure : 0.01128158844765343\n",
      "Simulation;Education : 0.01128158844765343\n",
      "Role Playing;Brain Games : 0.01128158844765343\n",
      "Racing;Pretend Play : 0.01128158844765343\n",
      "Puzzle;Education : 0.01128158844765343\n",
      "Parenting;Brain Games : 0.01128158844765343\n",
      "Music & Audio;Music & Video : 0.01128158844765343\n",
      "Lifestyle;Pretend Play : 0.01128158844765343\n",
      "Lifestyle;Education : 0.01128158844765343\n",
      "Health & Fitness;Education : 0.01128158844765343\n",
      "Health & Fitness;Action & Adventure : 0.01128158844765343\n",
      "Entertainment;Education : 0.01128158844765343\n",
      "Communication;Creativity : 0.01128158844765343\n",
      "Comics;Creativity : 0.01128158844765343\n",
      "Casual;Music & Video : 0.01128158844765343\n",
      "Card;Brain Games : 0.01128158844765343\n",
      "Card;Action & Adventure : 0.01128158844765343\n",
      "Books & Reference;Education : 0.01128158844765343\n",
      "Art & Design;Pretend Play : 0.01128158844765343\n",
      "Art & Design;Action & Adventure : 0.01128158844765343\n",
      "Arcade;Pretend Play : 0.01128158844765343\n",
      "Adventure;Education : 0.01128158844765343\n"
     ]
    }
   ],
   "source": [
    "display_table(android_free, 9) #Genres column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference between the `Genres` and the `Category` columns is not crystal clear, but one thing we can notice is that the `Genres` column is much more granular (it has more categories). We're only looking for the bigger picture at the moment, so we'll only work with the `Category` column moving forward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Up to this point, we found that the App Store is dominated by apps designed for fun, while Google Play shows a more balanced landscape of both practical and for-fun apps. Now we'd like to get an idea about the kind of apps that have most users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Most Popular Apps by Genre on the App Store"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to find out what genres are the most popular is to calculate the average number of installs for each app genre. For the Google Play dataset, we can find this information in the `Installs` column. However, we do not have an equivalent column in the App Store dataset. As a workaround, we can use the `rating_count_tot` column to get the total number of user ratings as a proxy for the number of users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we calculate the average number of user ratings per app genre on the App Store:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Social Networking : 71548.34905660378\n",
      "Photo & Video : 28441.54375\n",
      "Games : 22788.6696905016\n",
      "Music : 57326.530303030304\n",
      "Reference : 74942.11111111111\n",
      "Health & Fitness : 23298.015384615384\n",
      "Weather : 52279.892857142855\n",
      "Utilities : 18684.456790123455\n",
      "Travel : 28243.8\n",
      "Shopping : 26919.690476190477\n",
      "News : 21248.023255813954\n",
      "Navigation : 86090.33333333333\n",
      "Lifestyle : 16485.764705882353\n",
      "Entertainment : 14029.830708661417\n",
      "Food & Drink : 33333.92307692308\n",
      "Sports : 23008.898550724636\n",
      "Book : 39758.5\n",
      "Finance : 31467.944444444445\n",
      "Education : 7003.983050847458\n",
      "Productivity : 21028.410714285714\n",
      "Business : 7491.117647058823\n",
      "Catalogs : 4004.0\n",
      "Medical : 612.0\n"
     ]
    }
   ],
   "source": [
    "genre_dict = freq_table(ios_free, 11)\n",
    "users_dict = {}\n",
    "\n",
    "for genre in genre_dict:\n",
    "    total = 0\n",
    "    len_genre = 0\n",
    "    for app in ios_free:\n",
    "        genre_app = app[11]\n",
    "        if genre == genre_app:\n",
    "            app_rating = float(app[5])\n",
    "            total += app_rating\n",
    "            len_genre += 1\n",
    "    avg_rating_num = total / len_genre\n",
    "    users_dict[genre] = avg_rating_num\n",
    "    print(genre, \":\", avg_rating_num)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Genre with the maximum number of users:\n",
      "Navigation : 86090.33333333333\n"
     ]
    }
   ],
   "source": [
    "print(\"Genre with the maximum number of users:\")\n",
    "print(max(users_dict, key=users_dict.get), \":\", max(users_dict.values()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On average, we can see that the `Navigation` genre has the most user reviews on average (\\~86,000) followed by `Reference` (\\~74,000). The number of reviews of the navigation apps is heavily influenced by Google Maps and Waze, which have over half a million reviews on the app store combined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waze - GPS Navigation, Maps & Real-time Traffic : 345046\n",
      "Google Maps - Navigation & Transit : 154911\n",
      "Geocaching® : 12811\n",
      "CoPilot GPS – Car Navigation & Offline Maps : 3582\n",
      "ImmobilienScout24: Real Estate Search in Germany : 187\n",
      "Railway Route Search : 5\n"
     ]
    }
   ],
   "source": [
    "for app in ios_free:\n",
    "    if app[11] == 'Navigation':\n",
    "        print(app[1], ':', app[5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This pattern is also seen in the `Social Networking` and `Music` genres, as a few giants in the industry have a lot of reviews, skewing the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our aim is to find the most popular genres, but genres like `Social Networking`, `Music`, and `Navigation` might seem more popular than they are due to the presence of a few giants in those genres. Most apps struggle to get past the 10,000 user reviews threshold, but those giants lead to the average ratings being skewed in the favour of their genres, which could be misleading."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to solve this problem is to remove the giants from the analysis, and then compute the averages again. However, we will leave that level of detailed analysis for later and stick to a big picture view for now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the `Reference` genre (as it has the second highest average) and analyze what we find."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bible : 985920\n",
      "Dictionary.com Dictionary & Thesaurus : 200047\n",
      "Dictionary.com Dictionary & Thesaurus for iPad : 54175\n",
      "Google Translate : 26786\n",
      "Muslim Pro: Ramadan 2017 Prayer Times, Azan, Quran : 18418\n",
      "New Furniture Mods - Pocket Wiki & Game Tools for Minecraft PC Edition : 17588\n",
      "Merriam-Webster Dictionary : 16849\n",
      "Night Sky : 12122\n",
      "City Maps for Minecraft PE - The Best Maps for Minecraft Pocket Edition (MCPE) : 8535\n",
      "LUCKY BLOCK MOD ™ for Minecraft PC Edition - The Best Pocket Wiki & Mods Installer Tools : 4693\n",
      "GUNS MODS for Minecraft PC Edition - Mods Tools : 1497\n",
      "Guides for Pokémon GO - Pokemon GO News and Cheats : 826\n",
      "WWDC : 762\n",
      "Horror Maps for Minecraft PE - Download The Scariest Maps for Minecraft Pocket Edition (MCPE) Free : 718\n",
      "VPN Express : 14\n",
      "Real Bike Traffic Rider Virtual Reality Glasses : 8\n",
      "教えて!goo : 0\n",
      "Jishokun-Japanese English Dictionary & Translator : 0\n"
     ]
    }
   ],
   "source": [
    "for app in ios_free:\n",
    "    if app[11] == 'Reference':\n",
    "        print(app[1], ':', app[5]) # print name and number of ratings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the average number is likely skewed by apps like 'Bible' and 'Dictionary.com'. However, while these apps have a lot of reviews, the genre still has several other apps with a high review count (greater than the 10,000 review threshold); 6 to be exact. This is different compared to the navigation or social media apps, as those only have a lot of reviews for the widely-known apps only."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given all of that, it seems that an app that fits this reference genre might be a good idea to go with. With that in mind, it is important to note how apps that are aimed towards entertainment also have a lot of users. Therefore, an idea that would encompass both of these genres would be a book app that has features to make it stand out and be more engaging that another app."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data shows that even though games and for-fun apps are the most downloaded, they don't have the most users, which represents saturation in the market when it comes to for-fun apps. Therefore, the book app described above overcomes this problem by being different that any other entertainment app."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other genres that would also fit our criteria include book, weather, food and drink, and finance apps. These apps aren't very prevalent, but still have a good average number of reviews without any giants skewing the results. The book app described above would overlap with the book genre, which seems to be the most beneficial given the other factors described previously."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This book app should include features that make it stand out and more engaging for users. These features could include daily quotes from the book, quizzes and trivia about the book, and a built-in dictionary so users don't have to exit the app to check the meaning of a word."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's analyze the Google Play market to see if this idea fits well there as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Most Populatr Apps by Genre on Google Play"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the Google Play market, we actually have data about the number of installs, so we should be able to get a clearer picture about genre popularity. However, the install numbers don't seem precise enough — we can see that most values are open-ended (100+, 1,000+, 5,000+, etc.):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1,000,000+ : 15.749097472924186\n",
      "100,000+ : 11.563628158844766\n",
      "10,000,000+ : 10.503158844765343\n",
      "10,000+ : 10.209837545126353\n",
      "1,000+ : 8.393501805054152\n",
      "100+ : 6.915613718411552\n",
      "5,000,000+ : 6.825361010830325\n",
      "500,000+ : 5.561823104693141\n",
      "50,000+ : 4.7721119133574\n",
      "5,000+ : 4.512635379061372\n",
      "10+ : 3.5424187725631766\n",
      "500+ : 3.2490974729241873\n",
      "50,000,000+ : 2.3014440433213\n",
      "100,000,000+ : 2.1322202166064983\n",
      "50+ : 1.917870036101083\n",
      "5+ : 0.78971119133574\n",
      "1+ : 0.5076714801444043\n",
      "500,000,000+ : 0.2707581227436823\n",
      "1,000,000,000+ : 0.22563176895306858\n",
      "0+ : 0.04512635379061372\n",
      "0 : 0.01128158844765343\n"
     ]
    }
   ],
   "source": [
    "display_table(android_free, 5) # the Installs columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One problem with this data is that is not precise. For instance, we don't know whether an app with 100,000+ installs has 100,000 installs, 200,000, or 350,000. However, we don't need very precise data for our purposes — we only want to get an idea which app genres attract the most users, and we don't need perfect precision with respect to the number of users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, we're going to leave the numbers as they are, which means that we'll consider that an app with 100,000+ installs has 100,000 installs, and an app with 1,000,000+ installs has 1,000,000 installs, and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first step is to convert the number of installs to floats so that we can use the numbers to compute an average."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ART_AND_DESIGN : 1986335.0877192982\n",
      "AUTO_AND_VEHICLES : 647317.8170731707\n",
      "BEAUTY : 513151.88679245283\n",
      "BOOKS_AND_REFERENCE : 8767811.894736841\n",
      "BUSINESS : 1704192.3399014778\n",
      "COMICS : 817657.2727272727\n",
      "COMMUNICATION : 38326063.197916664\n",
      "DATING : 854028.8303030303\n",
      "EDUCATION : 1768500.0\n",
      "ENTERTAINMENT : 9146923.076923076\n",
      "EVENTS : 253542.22222222222\n",
      "FINANCE : 1387692.475609756\n",
      "FOOD_AND_DRINK : 1924897.7363636363\n",
      "HEALTH_AND_FITNESS : 4167457.3602941176\n",
      "HOUSE_AND_HOME : 1331540.5616438356\n",
      "LIBRARIES_AND_DEMO : 638503.734939759\n",
      "LIFESTYLE : 1437816.2687861272\n",
      "GAME : 12914435.883748516\n",
      "FAMILY : 5180161.789906103\n",
      "MEDICAL : 123064.7898089172\n",
      "SOCIAL : 23253652.127118643\n",
      "SHOPPING : 7036877.311557789\n",
      "PHOTOGRAPHY : 17840110.40229885\n",
      "SPORTS : 4274688.722772277\n",
      "TRAVEL_AND_LOCAL : 13984077.710144928\n",
      "TOOLS : 10801391.298666667\n",
      "PERSONALIZATION : 5201482.6122448975\n",
      "PRODUCTIVITY : 16772838.591304347\n",
      "PARENTING : 542603.6206896552\n",
      "WEATHER : 5074486.197183099\n",
      "VIDEO_PLAYERS : 24790074.17721519\n",
      "NEWS_AND_MAGAZINES : 9549178.467741935\n",
      "MAPS_AND_NAVIGATION : 4056941.7741935486\n"
     ]
    }
   ],
   "source": [
    "cat_table = freq_table(android_free, 1)\n",
    "installs_dict = {}\n",
    "\n",
    "for category in cat_table:\n",
    "    total = 0\n",
    "    len_category = 0\n",
    "    for app in android_free:\n",
    "        category_app = app[1]\n",
    "        if category == category_app:\n",
    "            installs_app = app[5]\n",
    "            installs_app = installs_app.replace('+', '')\n",
    "            installs_app = installs_app.replace(',', '')\n",
    "            installs_app = float(installs_app)\n",
    "            total += installs_app\n",
    "            len_category += 1\n",
    "    average_install_num = total / len_category\n",
    "    installs_dict[category] = average_install_num\n",
    "    print(category, \":\", average_install_num)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Genre with the maximum number of users:\n",
      "COMMUNICATION : 38326063.197916664\n"
     ]
    }
   ],
   "source": [
    "print(\"Genre with the maximum number of users:\")\n",
    "print(max(installs_dict, key=installs_dict.get), \":\", max(installs_dict.values()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On average, communication apps have the most installs: more than 38 million installs. Let's look at this category in further detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Messenger – Text and Video Chat for Free : 1,000,000,000+\n",
      "Gmail : 1,000,000,000+\n",
      "imo beta free calls and text : 100,000,000+\n",
      "imo free video calls and chat : 500,000,000+\n",
      "Android Messages : 100,000,000+\n",
      "Google Duo - High Quality Video Calls : 500,000,000+\n",
      "UC Browser - Fast Download Private & Secure : 500,000,000+\n",
      "Skype - free IM & video calls : 1,000,000,000+\n",
      "Who : 100,000,000+\n",
      "GO SMS Pro - Messenger, Free Themes, Emoji : 100,000,000+\n",
      "WhatsApp Messenger : 1,000,000,000+\n",
      "Google Chrome: Fast & Secure : 1,000,000,000+\n",
      "Firefox Browser fast & private : 100,000,000+\n",
      "Messenger Lite: Free Calls & Messages : 100,000,000+\n",
      "LINE: Free Calls & Messages : 500,000,000+\n",
      "Hangouts : 1,000,000,000+\n",
      "Kik : 100,000,000+\n",
      "KakaoTalk: Free Calls & Text : 100,000,000+\n",
      "Opera Mini - fast web browser : 100,000,000+\n",
      "Opera Browser: Fast and Secure : 100,000,000+\n",
      "Telegram : 100,000,000+\n",
      "Truecaller: Caller ID, SMS spam blocking & Dialer : 100,000,000+\n",
      "UC Browser Mini -Tiny Fast Private & Secure : 100,000,000+\n",
      "Viber Messenger : 500,000,000+\n",
      "WeChat : 100,000,000+\n",
      "Yahoo Mail – Stay Organized : 100,000,000+\n",
      "BBM - Free Calls & Messages : 100,000,000+\n"
     ]
    }
   ],
   "source": [
    "num_big = 0\n",
    "num = 0\n",
    "\n",
    "for app in android_free:\n",
    "    if app[1] == 'COMMUNICATION' and (app[5] == '1,000,000,000+'\n",
    "                                      or app[5] == '500,000,000+'\n",
    "                                      or app[5] == '100,000,000+'):\n",
    "        num_big += 1\n",
    "        print(app[0], ':', app[5]) # Name then no. of installs\n",
    "        \n",
    "for app in android_free:\n",
    "    if app[1] == 'COMMUNICATION':\n",
    "        num += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of communication apps with 100,000,000+ installs: 27\n",
      "Number of communication apps in total: 288\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of communication apps with 100,000,000+ installs:\", num_big)\n",
    "print(\"Number of communication apps in total:\", num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the data is skewed by a few apps (27 of the total 288) that have a lot of downloads, especiall apps like WhatsApp and Google Chrome, with over a billion downloads each."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we were to remove the apps that have over 100,000,000 installs, the average installs would be reduced by approximately 10 times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3593510.3486590036"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "under_100_m = []\n",
    "\n",
    "for app in android_free:\n",
    "    n_installs = app[5]\n",
    "    n_installs = n_installs.replace(',', '')\n",
    "    n_installs = n_installs.replace('+', '')\n",
    "    if (app[1] == 'COMMUNICATION') and (float(n_installs) < 100000000):\n",
    "        under_100_m.append(float(n_installs))\n",
    "        \n",
    "sum(under_100_m) / len(under_100_m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the average number of installs went from approximately 38 million to 3.5 million after the big apps were removed. The runner-up category, video players, has the same issue with giants skewing the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Motorola Gallery : 100,000,000+\n",
      "Google Play Movies & TV : 1,000,000,000+\n",
      "YouTube : 1,000,000,000+\n",
      "Dubsmash : 100,000,000+\n",
      "MX Player : 500,000,000+\n",
      "VLC for Android : 100,000,000+\n",
      "VivaVideo - Video Editor & Photo Movie : 100,000,000+\n",
      "VideoShow-Video Editor, Video Maker, Beauty Camera : 100,000,000+\n",
      "Motorola FM Radio : 100,000,000+\n"
     ]
    }
   ],
   "source": [
    "num_big = 0\n",
    "num = 0\n",
    "\n",
    "for app in android_free:\n",
    "    if app[1] == 'VIDEO_PLAYERS' and (app[5] == '1,000,000,000+'\n",
    "                                      or app[5] == '500,000,000+'\n",
    "                                      or app[5] == '100,000,000+'):\n",
    "        num_big += 1\n",
    "        print(app[0], ':', app[5]) # Name then no. of installs\n",
    "        \n",
    "for app in android_free:\n",
    "    if app[1] == 'VIDEO_PLAYERS':\n",
    "        num += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of video player apps with 100,000,000+ installs: 9\n",
      "Number of video player apps in total: 158\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of video player apps with 100,000,000+ installs:\", num_big)\n",
    "print(\"Number of video player apps in total:\", num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main concern here is that these apps seem more popular than they are; the giants make the categories seem better for our goal when they are not. Penetrating a market with apps that already dominate it would be greatly difficult, and it is highly unlikely that a minimalistic app (at least at first) would stand out and attract an audience."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The game genre seems quite popular, and with it we avoid any major skewing issues. However, as previously mentioned, since the game genre has the most apps in the market and does not have the most installs/users, we can infer that the market is saturated, and it would be difficult to stand out. Therefore, we would want to avoid that market, if possible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the reference genre seemed to provide an app idea with the App Store analysis, we can look at it again in the Google Play market."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "E-Book Read - Read Book for free : 50,000+\n",
      "Download free book with green book : 100,000+\n",
      "Wikipedia : 10,000,000+\n",
      "Cool Reader : 10,000,000+\n",
      "Free Panda Radio Music : 100,000+\n",
      "Book store : 1,000,000+\n",
      "FBReader: Favorite Book Reader : 10,000,000+\n",
      "English Grammar Complete Handbook : 500,000+\n",
      "Free Books - Spirit Fanfiction and Stories : 1,000,000+\n",
      "Google Play Books : 1,000,000,000+\n",
      "AlReader -any text book reader : 5,000,000+\n",
      "Offline English Dictionary : 100,000+\n",
      "Offline: English to Tagalog Dictionary : 500,000+\n",
      "FamilySearch Tree : 1,000,000+\n",
      "Cloud of Books : 1,000,000+\n",
      "Recipes of Prophetic Medicine for free : 500,000+\n",
      "ReadEra – free ebook reader : 1,000,000+\n",
      "Anonymous caller detection : 10,000+\n",
      "Ebook Reader : 5,000,000+\n",
      "Litnet - E-books : 100,000+\n",
      "Read books online : 5,000,000+\n",
      "English to Urdu Dictionary : 500,000+\n",
      "eBoox: book reader fb2 epub zip : 1,000,000+\n",
      "English Persian Dictionary : 500,000+\n",
      "Flybook : 500,000+\n",
      "All Maths Formulas : 1,000,000+\n",
      "Ancestry : 5,000,000+\n",
      "HTC Help : 10,000,000+\n",
      "English translation from Bengali : 100,000+\n",
      "Pdf Book Download - Read Pdf Book : 100,000+\n",
      "Free Book Reader : 100,000+\n",
      "eBoox new: Reader for fb2 epub zip books : 50,000+\n",
      "Only 30 days in English, the guideline is guaranteed : 500,000+\n",
      "Moon+ Reader : 10,000,000+\n",
      "SH-02J Owner's Manual (Android 8.0) : 50,000+\n",
      "English-Myanmar Dictionary : 1,000,000+\n",
      "Golden Dictionary (EN-AR) : 1,000,000+\n",
      "All Language Translator Free : 1,000,000+\n",
      "Azpen eReader : 500,000+\n",
      "URBANO V 02 instruction manual : 100,000+\n",
      "Bible : 100,000,000+\n",
      "C Programs and Reference : 50,000+\n",
      "C Offline Tutorial : 1,000+\n",
      "C Programs Handbook : 50,000+\n",
      "Amazon Kindle : 100,000,000+\n",
      "Aab e Hayat Full Novel : 100,000+\n",
      "Aldiko Book Reader : 10,000,000+\n",
      "Google I/O 2018 : 500,000+\n",
      "R Language Reference Guide : 10,000+\n",
      "Learn R Programming Full : 5,000+\n",
      "R Programing Offline Tutorial : 1,000+\n",
      "Guide for R Programming : 5+\n",
      "Learn R Programming : 10+\n",
      "R Quick Reference Big Data : 1,000+\n",
      "V Made : 100,000+\n",
      "Wattpad 📖 Free Books : 100,000,000+\n",
      "Dictionary - WordWeb : 5,000,000+\n",
      "Guide (for X-MEN) : 100,000+\n",
      "AC Air condition Troubleshoot,Repair,Maintenance : 5,000+\n",
      "AE Bulletins : 1,000+\n",
      "Ae Allah na Dai (Rasa) : 10,000+\n",
      "50000 Free eBooks & Free AudioBooks : 5,000,000+\n",
      "Ag PhD Field Guide : 10,000+\n",
      "Ag PhD Deficiencies : 10,000+\n",
      "Ag PhD Planting Population Calculator : 1,000+\n",
      "Ag PhD Soybean Diseases : 1,000+\n",
      "Fertilizer Removal By Crop : 50,000+\n",
      "A-J Media Vault : 50+\n",
      "Al-Quran (Free) : 10,000,000+\n",
      "Al Quran (Tafsir & by Word) : 500,000+\n",
      "Al Quran Indonesia : 10,000,000+\n",
      "Al'Quran Bahasa Indonesia : 10,000,000+\n",
      "Al Quran Al karim : 1,000,000+\n",
      "Al-Muhaffiz : 50,000+\n",
      "Al Quran : EAlim - Translations & MP3 Offline : 5,000,000+\n",
      "Al-Quran 30 Juz free copies : 500,000+\n",
      "Koran Read &MP3 30 Juz Offline : 1,000,000+\n",
      "Hafizi Quran 15 lines per page : 1,000,000+\n",
      "Quran for Android : 10,000,000+\n",
      "Surah Al-Waqiah : 100,000+\n",
      "Hisnul Al Muslim - Hisn Invocations & Adhkaar : 100,000+\n",
      "Satellite AR : 1,000,000+\n",
      "Audiobooks from Audible : 100,000,000+\n",
      "Kinot & Eichah for Tisha B'Av : 10,000+\n",
      "AW Tozer Devotionals - Daily : 5,000+\n",
      "Tozer Devotional -Series 1 : 1,000+\n",
      "The Pursuit of God : 1,000+\n",
      "AY Sing : 5,000+\n",
      "Ay Hasnain k Nana Milad Naat : 10,000+\n",
      "Ay Mohabbat Teri Khatir Novel : 10,000+\n",
      "Arizona Statutes, ARS (AZ Law) : 1,000+\n",
      "Oxford A-Z of English Usage : 1,000,000+\n",
      "BD Fishpedia : 1,000+\n",
      "BD All Sim Offer : 10,000+\n",
      "Youboox - Livres, BD et magazines : 500,000+\n",
      "B&H Kids AR : 10,000+\n",
      "B y H Niños ES : 5,000+\n",
      "Dictionary.com: Find Definitions for English Words : 10,000,000+\n",
      "English Dictionary - Offline : 10,000,000+\n",
      "Bible KJV : 5,000,000+\n",
      "Borneo Bible, BM Bible : 10,000+\n",
      "MOD Black for BM : 100+\n",
      "BM Box : 1,000+\n",
      "Anime Mod for BM : 100+\n",
      "NOOK: Read eBooks & Magazines : 10,000,000+\n",
      "NOOK Audiobooks : 500,000+\n",
      "NOOK App for NOOK Devices : 500,000+\n",
      "Browsery by Barnes & Noble : 5,000+\n",
      "bp e-store : 1,000+\n",
      "Brilliant Quotes: Life, Love, Family & Motivation : 1,000,000+\n",
      "BR Ambedkar Biography & Quotes : 10,000+\n",
      "BU Alsace : 100+\n",
      "Catholic La Bu Zo Kam : 500+\n",
      "Khrifa Hla Bu (Solfa) : 10+\n",
      "Kristian Hla Bu : 10,000+\n",
      "SA HLA BU : 1,000+\n",
      "Learn SAP BW : 500+\n",
      "Learn SAP BW on HANA : 500+\n",
      "CA Laws 2018 (California Laws and Codes) : 5,000+\n",
      "Bootable Methods(USB-CD-DVD) : 10,000+\n",
      "cloudLibrary : 100,000+\n",
      "SDA Collegiate Quarterly : 500+\n",
      "Sabbath School : 100,000+\n",
      "Cypress College Library : 100+\n",
      "Stats Royale for Clash Royale : 1,000,000+\n",
      "GATE 21 years CS Papers(2011-2018 Solved) : 50+\n",
      "Learn CT Scan Of Head : 5,000+\n",
      "Easy Cv maker 2018 : 10,000+\n",
      "How to Write CV : 100,000+\n",
      "CW Nuclear : 1,000+\n",
      "CY Spray nozzle : 10+\n",
      "BibleRead En Cy Zh Yue : 5+\n",
      "CZ-Help : 5+\n",
      "Modlitební knížka CZ : 500+\n",
      "Guide for DB Xenoverse : 10,000+\n",
      "Guide for DB Xenoverse 2 : 10,000+\n",
      "Guide for IMS DB : 10+\n",
      "DC HSEMA : 5,000+\n",
      "DC Public Library : 1,000+\n",
      "Painting Lulu DC Super Friends : 1,000+\n",
      "Dictionary : 10,000,000+\n",
      "Fix Error Google Playstore : 1,000+\n",
      "D. H. Lawrence Poems FREE : 1,000+\n",
      "Bilingual Dictionary Audio App : 5,000+\n",
      "DM Screen : 10,000+\n",
      "wikiHow: how to do anything : 1,000,000+\n",
      "Dr. Doug's Tips : 1,000+\n",
      "Bible du Semeur-BDS (French) : 50,000+\n",
      "La citadelle du musulman : 50,000+\n",
      "DV 2019 Entry Guide : 10,000+\n",
      "DV 2019 - EDV Photo & Form : 50,000+\n",
      "DV 2018 Winners Guide : 1,000+\n",
      "EB Annual Meetings : 1,000+\n",
      "EC - AP & Telangana : 5,000+\n",
      "TN Patta Citta & EC : 10,000+\n",
      "AP Stamps and Registration : 10,000+\n",
      "CompactiMa EC pH Calibration : 100+\n",
      "EGW Writings 2 : 100,000+\n",
      "EGW Writings : 1,000,000+\n",
      "Bible with EGW Comments : 100,000+\n",
      "My Little Pony AR Guide : 1,000,000+\n",
      "SDA Sabbath School Quarterly : 500,000+\n",
      "Duaa Ek Ibaadat : 5,000+\n",
      "Spanish English Translator : 10,000,000+\n",
      "Dictionary - Merriam-Webster : 10,000,000+\n",
      "JW Library : 10,000,000+\n",
      "Oxford Dictionary of English : Free : 10,000,000+\n",
      "English Hindi Dictionary : 10,000,000+\n",
      "English to Hindi Dictionary : 5,000,000+\n",
      "EP Research Service : 1,000+\n",
      "Hymnes et Louanges : 100,000+\n",
      "EU Charter : 1,000+\n",
      "EU Data Protection : 1,000+\n",
      "EU IP Codes : 100+\n",
      "EW PDF : 5+\n",
      "BakaReader EX : 100,000+\n",
      "EZ Quran : 50,000+\n",
      "FA Part 1 & 2 Past Papers Solved Free – Offline : 5,000+\n",
      "La Fe de Jesus : 1,000+\n",
      "La Fe de Jesús : 500+\n",
      "Le Fe de Jesus : 500+\n",
      "Florida - Pocket Brainbook : 1,000+\n",
      "Florida Statutes (FL Code) : 1,000+\n",
      "English To Shona Dictionary : 10,000+\n",
      "Greek Bible FP (Audio) : 1,000+\n",
      "Golden Dictionary (FR-AR) : 500,000+\n",
      "Fanfic-FR : 5,000+\n",
      "Bulgarian French Dictionary Fr : 10,000+\n",
      "Chemin (fr) : 1,000+\n",
      "The SCP Foundation DB fr nn5n : 1,000+\n"
     ]
    }
   ],
   "source": [
    "for app in android_free:\n",
    "    if app[1] == 'BOOKS_AND_REFERENCE':\n",
    "        print(app[0], ':', app[5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The book and reference genre includes a variety of apps: software for processing and reading ebooks, various collections of libraries, dictionaries, tutorials on programming or languages, etc. It seems there's still a small number of extremely popular apps that skew the average:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Google Play Books : 1,000,000,000+\n",
      "Bible : 100,000,000+\n",
      "Amazon Kindle : 100,000,000+\n",
      "Wattpad 📖 Free Books : 100,000,000+\n",
      "Audiobooks from Audible : 100,000,000+\n"
     ]
    }
   ],
   "source": [
    "for app in android_free:\n",
    "    if app[1] == 'BOOKS_AND_REFERENCE' and (app[5] == '1,000,000,000+'\n",
    "                                            or app[5] == '500,000,000+'\n",
    "                                            or app[5] == '100,000,000+'):\n",
    "        print(app[0], ':', app[5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the only app that greatly skews the average is Google Play Books, with the rest having 100,000,000+. Since there are only a few really popular apps with not as many downloads as the other categories (showing that it is not saturated), this market shows potential. Let's see how many apps are in the middle in terms of popularity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wikipedia : 10,000,000+\n",
      "Cool Reader : 10,000,000+\n",
      "Book store : 1,000,000+\n",
      "FBReader: Favorite Book Reader : 10,000,000+\n",
      "Free Books - Spirit Fanfiction and Stories : 1,000,000+\n",
      "AlReader -any text book reader : 5,000,000+\n",
      "FamilySearch Tree : 1,000,000+\n",
      "Cloud of Books : 1,000,000+\n",
      "ReadEra – free ebook reader : 1,000,000+\n",
      "Ebook Reader : 5,000,000+\n",
      "Read books online : 5,000,000+\n",
      "eBoox: book reader fb2 epub zip : 1,000,000+\n",
      "All Maths Formulas : 1,000,000+\n",
      "Ancestry : 5,000,000+\n",
      "HTC Help : 10,000,000+\n",
      "Moon+ Reader : 10,000,000+\n",
      "English-Myanmar Dictionary : 1,000,000+\n",
      "Golden Dictionary (EN-AR) : 1,000,000+\n",
      "All Language Translator Free : 1,000,000+\n",
      "Aldiko Book Reader : 10,000,000+\n",
      "Dictionary - WordWeb : 5,000,000+\n",
      "50000 Free eBooks & Free AudioBooks : 5,000,000+\n",
      "Al-Quran (Free) : 10,000,000+\n",
      "Al Quran Indonesia : 10,000,000+\n",
      "Al'Quran Bahasa Indonesia : 10,000,000+\n",
      "Al Quran Al karim : 1,000,000+\n",
      "Al Quran : EAlim - Translations & MP3 Offline : 5,000,000+\n",
      "Koran Read &MP3 30 Juz Offline : 1,000,000+\n",
      "Hafizi Quran 15 lines per page : 1,000,000+\n",
      "Quran for Android : 10,000,000+\n",
      "Satellite AR : 1,000,000+\n",
      "Oxford A-Z of English Usage : 1,000,000+\n",
      "Dictionary.com: Find Definitions for English Words : 10,000,000+\n",
      "English Dictionary - Offline : 10,000,000+\n",
      "Bible KJV : 5,000,000+\n",
      "NOOK: Read eBooks & Magazines : 10,000,000+\n",
      "Brilliant Quotes: Life, Love, Family & Motivation : 1,000,000+\n",
      "Stats Royale for Clash Royale : 1,000,000+\n",
      "Dictionary : 10,000,000+\n",
      "wikiHow: how to do anything : 1,000,000+\n",
      "EGW Writings : 1,000,000+\n",
      "My Little Pony AR Guide : 1,000,000+\n",
      "Spanish English Translator : 10,000,000+\n",
      "Dictionary - Merriam-Webster : 10,000,000+\n",
      "JW Library : 10,000,000+\n",
      "Oxford Dictionary of English : Free : 10,000,000+\n",
      "English Hindi Dictionary : 10,000,000+\n",
      "English to Hindi Dictionary : 5,000,000+\n"
     ]
    }
   ],
   "source": [
    "for app in android_free:\n",
    "    if app[1] == 'BOOKS_AND_REFERENCE' and (app[5] == '1,000,000+'\n",
    "                                            or app[5] == '5,000,000+'\n",
    "                                            or app[5] == '10,000,000+'\n",
    "                                            or app[5] == '50,000,000+'):\n",
    "        print(app[0], ':', app[5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This niche seems to be dominated by software for processing and reading ebooks, as well as various collections of libraries and dictionaries, so it's probably not a good idea to build similar apps since there'll be some significant competition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also notice there are quite a few apps built around the book Quran, which suggests that building an app around a popular book can be profitable. It seems that taking a popular book (perhaps a more recent book) and turning it into an app could be profitable for both the Google Play and the App Store markets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, the market seems to be have a lot of libraries and book processing softwares, so our app needs to provide extra features (as mentioned in the App Store analysis section) so it can stand out."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, we analyzed data about the App Store and Google Play mobile apps with the goal of recommending an app profile that can be profitable for both markets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We concluded that taking a popular book (perhaps a more recent book) and turning it into an app could be profitable for both the Google Play and the App Store markets. The markets are already full of libraries, so we need to add some special features besides the raw version of the book. This might include daily quotes from the book, an audio version of the book, quizzes on the book, a forum where people can discuss the book, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the app proves to be successful, we can then expand it to include extra books that are popular and provide all the features provided for the original book for the new books."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}