ME 297 - Introduction to Data Science for Mechanical Engineers

This is the handout and homework repository for the course "ME 297 Introduction to Data Science for Mechanical Engineers" which is currently taught (Fall 2020) by Prof. Ilias Bilionis at Purdue University. The course is fully online with the videos being accessible through Brightspace. This is an experimental course that will likely evolve into a requirement for all Purdue Mechanical Engineering students.

The material is published under the GNU General Public License. You can reuse it in your own courses as soon as you also include the same license and cite this repository. Please send me an email if you do as I would love to know!

Lecture Notebooks

Below, I provide links that open up directly on Google Colab. If you want to view the Jupyter notebooks locally, please see the section named Running the notebooks on your personal computer.

• Lecture 1, Part A - Introduction to Predictive Modeling

  • Hands-on Activity 1.1 (The uncertainty propagation probability problem)
  • Hands-on Activity 1.2 (The model calibration problem)

• Lecture 1, Part B - Python Basics

  • Hands-on Activity 1.3 (Python as a calculator)
  • Hands-on Activity 1.4 (Python variables and types)
  • Hands-on Activity 1.5 (Basic tuples and lists)
  • Hands-on Activity 1.6 (Basics of strings and printing)
  • Hands-on Activity 1.7 (Conditionals)
  • Hands-on Activity 1.8 (Loops)

• Lecture 2 - Advanced Python Concepts

  • Hands-on Activity 2.1 (Functions)
  • Python Basics Activity 8 (Classes)
  • Python Basics Activity 9 (Numpy arrays and linear algebra)
  • Python Basics Activity 10 (The Python data analysis library)
  • Python Basics Activity 11 (Basic plotting)

• Lecture 3 - Basics of Probability Theory

  • Reading Activity 3
  • Hands-on Activity 3.1 (Experiment with the "randomness" of a coin toss)

• Lecture 4 - Discrete Random Variables

  • Reading Activity 4
  • Hands-on Activity 4.1 (Discrete random variables)

• Lecture 5 - Continuous Random variables

  • Reading Activity 5
  • Hands-on Activity 5.1 (The uniform distribution)
  • Hands-on Activity 5.2 (The Gaussian distribution)

• Lecture 6 - Collections of Random variables

  • Reading Activity 6.1
  • Hands-on Activity 6.1 (Collections of random variables)
  • Reading Activity 6.2
  • Hands-on Activity 6.2 (The multivariate normal - diagonal covariance case)
  • Hands-on Activity 6.3 (The multivariate normal - full covariance case)

• Lecture 7 - The Monte Carlo Method for Estimating Expectations

  • Hands-on Activity 7.1 (The law of large numbers)
  • Hands-on Activity 7.2 (Estimating the variance)
  • Hands-on Activity 7.3 (Estimating the cumulative distribution function)
  • Hands-on Activity 7.4 (Estimating the probability density function via histograms)
  • Hands-on Activity 7.5 (Estimating predictive quantiles)
  • Hands-on Activity 7.6 (Application – Uncertainty propagation through an initial value problem)

• Lecture 8 - Analytical Examples of Bayesian Inference

  • Reading Activity 8
  • Hands-on Activity 8.1 (Bayesian parameter estimation)
  • Hands-on Activity 8.2 (Credible intervals)
  • Hands-on Activity 8.3 (Decision-making)
  • Hands-on Activity 8.4 (Predictive checking)

• Lecture 9 - Linear Regression via Least Squares

  • Reading Activity 9
  • Hands-on Activity 9.1 (Linear regression with a single variable)
  • Hands-on Activity 9.2 (Polynomial regression)
  • Hands-on Activity 9.3 (The generalized linear model)
  • Hands-on Activity 9.4 (Measures of predictive accuracy)

• Lecture 10 - Bayesian Linear regression

  • Reading Activity 10
  • Hands-on Activity 10.1 (Probabilistic interpretation of least squares – Estimating the measurement noise)
  • Hands-on Activity 10.2 (Maximum a posteriori estimate – Avoiding overfitting)
  • Hands-on Activity 10.3 (Bayesian linear regression)
  • Hands-on Activity 10.4 (The point-predictive distribution – Separating epistemic and aleatory uncertainty)

• Lecture 11 - Classification

  • Reading Activity 11
  • Hands-on Activity 11.1 (Logistic regression with a single variable)
  • Hands-on Activity 11.2 (Logistic regression with many features)
  • Hands-on Activity 11.3 (Making decisions)
  • Hands-on Activity 11.4 (Diagnostics for classification)
  • Hands-on Activity 11.5 (Multi-class logistic regression)

• Lecture 12 - Clustering and Density Estimation

  • Reading Activity 12
  • Hands-on Activity 12.1 (Clustering using k-means)
  • Hands-on Activity 12.2 (Density estimation using Gaussian mixture)

• Lecture 13 - Dimensionality Reduction

  • Reading Activity 13
  • Hands-on Activity 13.1 (Principal component analysis)
  • Hands-on Activity 13.2 (Clustering high-dimensional data)
  • Hands-on Activity 13.3 (Density estimation with high-dimensional data)

• Lecture 14 - Deep Neural Networks

  • Reading Activity 14
  • Hands-on Activity 14.1 (The stochastic gradient descent algorithm and variants)

• Lecture 15 - Deep Neural Networks Continued

  • Reading Activity 15
  • Hands-on Activity 15.1 (Image classification)

Homework Notebooks

• Homework 1
• Homework 2
• Homework 3
• Homework 4
• Homework 5
• Homework 6
• Homework 7
• Homework 8

Running the notebooks on Google Colab

Make sure you have a Google account before you start. Then, you just click on the links above.

Converting your homework notebooks to PDF on Google Colab

One solution is to "print" your notebook to a PDF. However, we have observed that sometimes the figures get a bit messed up. One solution is to run the notebooks on your own laptop, and the do "File-> Download as-> PDF via Latex (.pdf)." See below if you want to take that route. Now, it is possible to do the same thing on Google Colab. Follow the instructions in this notebook.

Running the notebooks on your personal computer

Find and download the right version of Anaconda for Python 3.7 from Continuum Analytics. This package contains most of the software we are going to need. Note: You do need Python 3 and note Python 2. The notebooks will not work with Python 2.

OS Specific Instructions

Microsoft Windows

• We need C, C++, Fortran compilers, as well as the Python sources. Start the command line by opening "Anaconda Prompt" from the start menu. In the command line type:

conda config --append channels https://repo.continuum.io/pkgs/free
conda install mingw libpython

• Finally, you need git. As you install it, make sure to indicate that you want to use "Git from the command line and also from 3rd party software".

Apple OS X

• Download and install the latest version of Xcode.

Linux

If you are using Linux, I am sure that you can figure it out on your own.

Installation of Required Python Packages

Independently of the operating system, use the command line to install the following Python packages:

• Seaborn, for beautiful graphics:

conda install seaborn

• PyMC3 for MCMC sampling:

conda install pymc3

• GPy for Gaussian process regression:

pip install GPy

• pydoe for generating experimental designs:

pip install pydoe

• fipy for solving partial differential equations using the finite volume method:

pip install fipy

*** Windows Users ***

You may receive the error

ModuleNotFoundError: No module named 'future'

If so, please install future and then install fipy:

pip install future

• scikit-learn for some standard machine learning algorithms implemented in Python:

conda install scikit-learn

• graphviz for visualizing probabilistic graphical models:

pip install graphviz

Running the notebooks

• Open the command line.
• cd to your favorite folder.
• Then, type:

git clone https://github.com/PurdueMechanicalEngineering/me-297-intro-to-data-science.git

• This will download the contents of this repository in a folder called me-297-intro-to-data-science.
• Enter the me-297-intro-to-data-science folder:

cd me-297-intro-to-data-science

• Start the jupyter notebook by typing the command:

jupyter notebook

• Use the browser to navigate the course, experiment with code etc.
• If the course content has been updated, type the following command (while being inside me-297-intro-to-data-science) to get the latest version:

git pull origin master

Keep in mind, that if you have made local changes to the repository, you may have to commit them before moving on.