lisa.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LISA Demo"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[LISA (Linux Integrated System Analysis)](https://lisa-linux-integrated-system-analysis.readthedocs.io/en/master/index.html) is a framework to record ftrace traces and analyze them as Panda dataframes.\n",
    "\n",
    "This notebook shows a typical LISA-use case:\n",
    "\n",
    "* Connects to the VM\n",
    "* Collects a trace while executing a workload\n",
    "* Displays the trace\n",
    "* Analyses the trace\n",
    "\n",
    "It can serve as a template for your own experiments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Target configuration"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Connect to the VM started by VSCode using SSH at `localhost:5555` with a [Target](https://lisa-linux-integrated-system-analysis.readthedocs.io/en/master/target.html#lisa.target.Target) object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lisa.target import Target\n",
    "import os\n",
    "\n",
    "target = Target(\n",
    "    kind='linux',\n",
    "    name='vscode-vm',\n",
    "    host='localhost',\n",
    "    port='5555',\n",
    "    username='root',\n",
    "    keyfile=os.path.expanduser('~/.ssh/vm-cli_rsa'),\n",
    "    strict_host_check=False,\n",
    "    kernel_src=\"../\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up a workload"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a [RT App](https://lisa-linux-integrated-system-analysis.readthedocs.io/en/master/workloads.html#lisa.wlgen.rta.RTA), convenient for scheduler experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lisa.wlgen.rta import RTA, RTAPhase, PeriodicWload\n",
    "\n",
    "rtapp_profile = {\n",
    "    f'tsk{cpu}_{i}': RTAPhase(\n",
    "        prop_wload=PeriodicWload(\n",
    "            duty_cycle_pct=20,\n",
    "            period=16e-3,\n",
    "            duration=1,\n",
    "        )\n",
    "    )\n",
    "    for cpu in range(target.number_of_cpus)\n",
    "    for i in range(3)\n",
    "}\n",
    "\n",
    "workload = RTA.from_profile(target, rtapp_profile, name='lisa_workload')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tracing the workload"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to specify the ftrace events we want to record. You can find a list of ftrace events in `/sys/kernel/debug/tracing/available_events`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "events = [\n",
    "    'sched_switch',\n",
    "    'sched_wakeup',\n",
    "    'sched_wakeup_new',\n",
    "    'task_rename',\n",
    "]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's record a trace while running the workload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lisa.trace import FtraceCollector\n",
    "\n",
    "trace_path = os.path.join(workload.res_dir, 'trace.dat')\n",
    "ftrace_collector = FtraceCollector(target, events=events, buffer_size=10240, output_path=trace_path)\n",
    "\n",
    "with workload, ftrace_collector:\n",
    "    workload.run()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the trace"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Trace](https://lisa-linux-integrated-system-analysis.readthedocs.io/en/master/trace_analysis.html#id1) class wraps a `trace.dat`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lisa.trace import Trace\n",
    "\n",
    "trace = Trace(trace_path, target.plat_info, events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking at the trace"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have `kernelshark` installed, it can be opened with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# trace.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysing the trace"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With data frames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = trace.df_event('sched_switch')\n",
    "df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [standard DataFrame operations](https://pandas.pydata.org/docs/user_guide/10min.html) are available, so you can filter/slice it however you wish:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[df.next_comm == 'systemd'].iloc[0:3]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### With trace analysis functions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LISA ships [a number of trace analysis](https://lisa-linux-integrated-system-analysis.readthedocs.io/en/master/trace_analysis.html) methods under `trace.ana`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "trace.ana.tasks.df_tasks_runtime()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from lisa.trace import TaskID\n",
    "\n",
    "df = trace.ana.tasks.df_task_states(TaskID(pid=0, comm='systemd'), stringify=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lisa.analysis.tasks import TaskState\n",
    "\n",
    "df[df.curr_state == TaskState.TASK_ACTIVE]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### With plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trace.ana.tasks.plot_task_total_residency(TaskID(pid=0, comm=\"systemd\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trace.ana.tasks.plot_tasks_wakeups_heatmap(bins=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trace.ana.tasks.plot_tasks_forks_heatmap(bins=200)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}