Session 7 Extension - Eigenvalues and Eigenvectors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Eigenvalues and eigenvectors**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Viewing matrices as linear transformations**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far, we haven't mentioned one of the most useful interpretations of matrices, and that is as *linear transformations* of vectors of appropriate size. So, an (n x n) dimensional square matrix would define a linear transformation of an n-dimensional *'vector space'*.\n",
    "<br><br>\n",
    "What is a 'vector space'? This is just a fancy way of saying the set of all possible vectors of the appropriate dimension, together with the following operations: (i) adding those vectors, (ii) multiplying those vectors by a scalar. So, a 2D vector space would consist of all vectors of the form (a_x, a_y) (where a_x and a_y are real numbers), together with the operations of adding those vectors and multiplying them by a scalar (i.e. just another real number). \n",
    "<br><br>\n",
    "Anyway, maths waffle. A *linear transformation* in that space just turns one vector in the space into another vector in the same space. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sympy import Matrix, init_printing\n",
    "init_printing()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 2\\\\3 & 4\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  2⎤\n",
       "⎢    ⎥\n",
       "⎣3  4⎦"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2D example. Here's our 'transformation' matrix A\n",
    "A = Matrix( [ [1,2], [3,4] ] )\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1\\\\1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎣1⎦"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here's the vector we're going to make A act upon\n",
    "u = Matrix([[1], [1]])\n",
    "u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}3\\\\7\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡3⎤\n",
       "⎢ ⎥\n",
       "⎣7⎦"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A acts upon u by left multiplying the vector, generating a new vector v\n",
    "v = A*u\n",
    "v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Short diversion - reflections and rotations** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reflections**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}-1 & 0\\\\0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡-1  0⎤\n",
       "⎢     ⎥\n",
       "⎣0   1⎦"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reflect in the y-axis, i.e. x goes to -x, y goes to y\n",
    "# Matrix performing this operation (we often just call this matrix the 'operator'):\n",
    "R_in_y = Matrix( [ [-1,0], [0,1] ] )\n",
    "R_in_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}-1\\\\1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡-1⎤\n",
       "⎢  ⎥\n",
       "⎣1 ⎦"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now make the reflection matrix operate on u\n",
    "R_in_y * u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1 & 0\\\\0 & -1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1  0 ⎤\n",
       "⎢     ⎥\n",
       "⎣0  -1⎦"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reflect in the x-axis, i.e. x goes to x, y goes to -y\n",
    "R_in_x = Matrix( [ [1,0], [0,-1] ] )\n",
    "R_in_x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1\\\\-1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1 ⎤\n",
       "⎢  ⎥\n",
       "⎣-1⎦"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Again, make the reflection matrix operate on u\n",
    "R_in_x * u"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Rotations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAA4AAAATCAYAAACgADyUAAAABHNCSVQICAgIfAhkiAAAAP1JREFUOI3N0qtKRFEUxvEfooODBoNBOcF5ArNoFQwmg8UiZvEFBMUkmLRqMAo+gD6E2LxgdBiZYtBgGI94CWcNHA5nj8Xggs0H6+O/12Vv/jia2MUDeuhgHyODoGncIsc5DnCHbxynoAau8YaFUn4cbXxiqg7cjps3a7yj8FarRhMv6CZm2QlwC4ZKxgomcIaPGnA0NIfhkrEcmmGvBlwM7VSNdrTy22mVobFI3tRUothqXq7WnzEL7SbAJcXCLqpgI/Q9AW6EnlaNyWj1qgaawxcuE5e6V/yM2VJuBo+K922lwLWo+oxDnOA1oPkU1I91xYfu4SngbCDx/+MHAjk837FNgbsAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\theta$$"
      ],
      "text/plain": [
       "θ"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sympy import symbols, sin, cos, pi\n",
    "theta = symbols('theta')\n",
    "theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}\\cos{\\left (\\theta \\right )} & - \\sin{\\left (\\theta \\right )}\\\\\\sin{\\left (\\theta \\right )} & \\cos{\\left (\\theta \\right )}\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡cos(θ)  -sin(θ)⎤\n",
       "⎢               ⎥\n",
       "⎣sin(θ)  cos(θ) ⎦"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rot_anticlock_by_theta = Matrix( [ [cos(theta), -sin(theta)], [sin(theta), cos(theta)] ] )\n",
    "rot_anticlock_by_theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0 & -1\\\\1 & 0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0  -1⎤\n",
       "⎢     ⎥\n",
       "⎣1  0 ⎦"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rot_anticlock_by_90 = rot_anticlock_by_theta.subs(theta, pi/2)\n",
    "rot_anticlock_by_90"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1\\\\0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎣0⎦"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now make this operate on a vector at the '3 o'clock' position (i.e. purely along +ve x)\n",
    "u = Matrix([[1], [0]])\n",
    "u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0\\\\1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0⎤\n",
       "⎢ ⎥\n",
       "⎣1⎦"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# vector goes to '12 o'clock' position (i.e. purely along +ve y)\n",
    "v = rot_anticlock_by_90 * u\n",
    "v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0 & 1\\\\-1 & 0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0   1⎤\n",
       "⎢     ⎥\n",
       "⎣-1  0⎦"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now let's rotate the vector back to where it started\n",
    "rot_clock_by_90 = rot_anticlock_by_theta.subs(theta, -pi/2)\n",
    "rot_clock_by_90"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0 & 1\\\\-1 & 0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0   1⎤\n",
       "⎢     ⎥\n",
       "⎣-1  0⎦"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We could also have obtained this by taking the inverse of the anticlock_by_90 matrix\n",
    "rot_clock_by_90 = rot_anticlock_by_90.inv()\n",
    "rot_clock_by_90"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1\\\\0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎣0⎦"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Apply this inverse operation to the '12 o'clock' vector\n",
    "rot_clock_by_90 * v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Eigenvectors and eigenvalues**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's just dive straight in, the *eigenvectors* of a matrix are the vectors that remain *'unchanged in direction'* by the operation represented by the matrix. The corresponding *eigenvalues* are the *'scaling factors'* of the length of the matrix. \n",
    "<br><br>\n",
    "Let's consider an example. Consider the 'reflection in y' operator:    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}-1 & 0\\\\0 & 1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡-1  0⎤\n",
       "⎢     ⎥\n",
       "⎣0   1⎦"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R_in_y = Matrix( [ [-1,0], [0,1] ] )\n",
    "R_in_y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What sort of vectors will not have their directions altered by this operator? Thinking about it, we realise that vectors perpendicular, parallel and anti-parallel to the y-axis will not change 'direction' under this operation: while they may flip sign and/or be rescaled in length, they will *remain on the line established by the original vector*. Every other vector will change direction, i.e. move off their original line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0\\\\1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0⎤\n",
       "⎢ ⎥\n",
       "⎣1⎦"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Parallel to y\n",
    "u = Matrix([[0], [1]])\n",
    "u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0\\\\1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0⎤\n",
       "⎢ ⎥\n",
       "⎣1⎦"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# unchanged in direction or length\n",
    "R_in_y*u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0\\\\-1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0 ⎤\n",
       "⎢  ⎥\n",
       "⎣-1⎦"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Anti-parallel to y\n",
    "u = Matrix([[0], [-1]])\n",
    "u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}0\\\\-1\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡0 ⎤\n",
       "⎢  ⎥\n",
       "⎣-1⎦"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# unchanged in direction or length\n",
    "R_in_y*u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}1\\\\0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎣0⎦"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Perpendicular to y\n",
    "u = Matrix([[1], [0]])\n",
    "u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left[\\begin{matrix}-1\\\\0\\end{matrix}\\right]$$"
      ],
      "text/plain": [
       "⎡-1⎤\n",
       "⎢  ⎥\n",
       "⎣0 ⎦"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# unchanged in direction, but flipped in sign (i.e. length rescaled by -1)\n",
    "R_in_y*u"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SymPy can find the eigenvectors and eigenvalues for us via the .eigenvects() method of a Matrix object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$$\\left [ \\left ( -1, \\quad 1, \\quad \\left [ \\left[\\begin{matrix}1\\\\0\\end{matrix}\\right]\\right ]\\right ), \\quad \\left ( 1, \\quad 1, \\quad \\left [ \\left[\\begin{matrix}0\\\\1\\end{matrix}\\right]\\right ]\\right )\\right ]$$"
      ],
      "text/plain": [
       "⎡⎛       ⎡⎡1⎤⎤⎞  ⎛      ⎡⎡0⎤⎤⎞⎤\n",
       "⎢⎜-1, 1, ⎢⎢ ⎥⎥⎟, ⎜1, 1, ⎢⎢ ⎥⎥⎟⎥\n",
       "⎣⎝       ⎣⎣0⎦⎦⎠  ⎝      ⎣⎣1⎦⎦⎠⎦"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R_in_y.eigenvects()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see we get a list of tuples. The first element of each tuple gives the eigenvalue, the second element the 'multiplicity' (i.e. how many linearly independent eigenvectors this one eigenvector represents), and the third element the eigenvector itself. So, each eigenvector is associated with a corresponding eigenvalue. \n",
    "<br><br>\n",
    "The result of the .eigenvects() method confirms what we saw above. Vectors 'on' the [0,1] line are unchanged, while vectors on the [1,0] line are inverted through the origin but remain on the line. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJgAAAAWCAYAAAA8eFWJAAAABHNCSVQICAgIfAhkiAAAAqpJREFUaIHt2k+IjHEYwPHPWpLSWntBEidZLpKUg5qctiSEiwuOUqQcOJmSSOTg5qApTkpsOSgHiiIlCq2LWuXfgcPuHlatf4ffO7t2rHfnnZl35p3xfmv6TfN7n9/zzDO/95nned4fOTktZB+eYAQ/sL615uRkiCIm8Bm3sTrpAmvxE2O4itNY2jj7JtmDy3iIUfzC9RT0NFtXvWTdLwWcw93o+qdJlR6OBI8kFUzIi0jPGIak68hm6qqXdvLLMyEYLaycmBMj1BeNQwkU1cIxIbz24FAH6aqXdvLLG3RhceXE3Bih7micqEFhEu6nvH6rdNVLO/mlvEe6KyfiIlg7URLC+oHWmpEpSjLgk7gN1huN480wJKet+RaNiyon/rXBurBFuAPepWRUIzmJftxqtSEZopk+GY7GQuVE5QYbwEWh5NyAK0KfI27hXwleaVVBn4REcySl9duRZvrkmhCILuAGzmAVfyf5AzgavX8l9EbieGsqPFbDxwTX5rQPH3AWl7A3+uyeqcg2jR7sEHKv92aoDFKkoHm9qWbqqpeCbPtlWyTzCOswrzwxUw42ikHcxHLhfzwnJ46t0VjEa3+0tuKqyHJy3xdzTVZYhjVmqGIaQEkGyv0aSNMnlZQbrMOVE3GN1vIuTLtXtjN6MfWsc7Pww8IXHJ9ljbPYj4N/yDVKV/n7f5/FhkZTr1+q9UkjdNXko1PCnVtIIlQDRfGV53AVa5RUF2Vq0fVcSBv+egySMkX1+aWk+sjbKF0rq9A1yYlIaFcSoQ6jVzimdL7VhmScQWGvLEkitDsSuoMVOuexUhK2C22YNI4pdQLzsUnotY1IuEcW4KXpoTI/cJhTpmj63jg100VxSf44NgolaL9w1ieuq5/zf/FASOq/4rFwpiwnJyeng/gNwWXbj8AHdKMAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$$\\left \\{ -1 : 1, \\quad 1 : 1\\right \\}$$"
      ],
      "text/plain": [
       "{-1: 1, 1: 1}"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# If we just want the eigenvalues, can use the .eigenvals() method, but this is less useful, generally,\n",
    "# than getting the eigenvectors and eigenvalues together with .eigenvects()\n",
    "R_in_y.eigenvals()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}