readme.txt

       (                     (   (       )                                
   (   )\ ) (       )   )    )\ ))\ ) ( /(             (    *   )      )  
   )\ (()/( )\ ) ( /(( /(   (()/(()/( )\())   (  (     )\ ` )  /(   ( /(  
 (((_) /(_)|()/( )\())\())__ /(_))(_)|(_)\    )\ )\  (((_) ( )(_))  )(_)) 
 )\___(_))  /(_)|(_)((_)\___(_))(_))   ((_)  ((_|(_) )\___(_(_())  ((_)   
((/ __/ __|(_) // _(_) (_)  | _ \ _ \ / _ \ _ | | __((/ __|_   _|  |_  )  
 | (__\__ \ / _ \_, /| |    |  _/   /| (_) | || | _| | (__  | |     / /   
  \___|___/ \___//_/ |_|    |_| |_|_\ \___/ \__/|___| \___| |_|    /___|  
                                                                          
CS691 PROJECT 2
==========================================================
Set up:
I followed this to set up Jupyter, which also set up PySpark (included).
I think Jupyter is an excellent tool for working out problems once you
get used to it. I highly recommend it.

https://blog.sicara.com/get-started-pyspark-jupyter-guide-tutorial-ae2fe84f594f

I did not have to do anything else to run the scripts through the Python
interpreter. 
==========================================================
How to run: 
All .py files can be run from the python interpreter as follows:

$python q1.py /[PATHTOFILE]/training_set_tweets.txt /[PATHTOFILE]/training_set_users.txt
==========================================================
Versions Used to Build:
#python
$python --version
Python 3.6.5 :: Anaconda, Inc.

#spark
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.4.1
      /_/
         
Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_201)
Type in expressions to have them evaluated.
Type :help for more information.


#java (does it matter? probably)
$ java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)