notebooks/samples/CyberML - Anomalous Access Detection.ipynb

{"cells":[{"cell_type":"markdown","source":["# Access Anomalies Example\n\nHere we demonstrate a novel CyberML model which can learn user access patterns and then automatically detect anomalous user access based on learned behavior.\nThe model internally uses Collaborative Filtering for Implicit Feedback as published here: http://yifanhu.net/PUB/cf.pdf\nand is based on Apache Spark's implementation of this: https://spark.apache.org/docs/2.2.0/ml-collaborative-filtering.html.\n\nThis notebook demonstrates a usage example of Anomalous Resource Access model.\nAll the model requires is a dataset in which there are 'users' which access 'resources'.\nThe model is based on Collaborative Filtering and it uses Machine Learning to learn access patterns of users and resources.\nWhen a user accesses a resource which is outside of the user's learned profile then this access recieves a high anomaly score.\n\nIn this notebook we provide a usage example and a synthetic dataset in which there are 3 departments:\n(1) Finance, (2) HR and (3) Engineering.\nIn the training data users access only a subset of resources from thier own departments.\nTo evaluate the model we use two datasets.\nThe first contains access patterns unseen during training in which users access resources within their departments (again, resources they didn't access during training but within their department).\nThe latter contains users accessing resources from outside their department.\nWe then use the model to assign anomaly scores expecting that the first get low anomaly scores and the latter recieve high anomaly scores.\nThis is what this example demonstrates.\n\nNote: the data does NOT contain information about departments, this information is implictly learned by the model by analyzing the access patterns."],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"d19c5fe5-6268-4b04-a79b-6c44d619e722"}}},{"cell_type":"markdown","source":["# Create an Azure Databricks cluster and install the following libs\n\n1. In Cluster Libraries install from library source Maven:\nCoordinates: com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc3\nRepository: https://mmlspark.azureedge.net/maven\n\n2. In Cluster Libraries install from PyPI the library called plotly"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"7c9b804f-9057-4a35-b886-30d580c2de99"}}},{"cell_type":"markdown","source":["# Setup & Initialization"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"ea7264f0-4f2d-424e-90b1-333b5f43ea7d"}}},{"cell_type":"code","source":["# this is used to produce the synthetic dataset for this test\nfrom mmlspark.cyber.dataset import DataFactory\n\n# the access anomalies model generator\nfrom mmlspark.cyber.anomaly.collaborative_filtering import AccessAnomaly\n\nfrom pyspark.sql import functions as f, types as t"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"f5e0b2a9-42a3-41c3-be6c-505e234c2f38"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["spark.sparkContext.setCheckpointDir('dbfs:/checkpoint_path/')"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"0ddf4890-452f-4ea0-ab6a-fb4ef985960d"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"markdown","source":["# Loadup datasets"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"a3a2b930-2e90-4a70-8a8b-78005676b5dc"}}},{"cell_type":"code","source":["factory = DataFactory(\n  num_hr_users = 25,\n  num_hr_resources = 50,\n  num_fin_users = 35,\n  num_fin_resources = 75,\n  num_eng_users = 15,\n  num_eng_resources = 25,\n  single_component = True\n)\n\ntraining_pdf = factory.create_clustered_training_data(ratio=0.4)\n\n# a tenant id is used when independant datasets originate from different tenants, in this example we set all tenants-ids to the same value\ntraining_df = spark.createDataFrame(training_pdf).withColumn('tenant_id', f.lit(0))\ningroup_df = spark.createDataFrame(factory.create_clustered_intra_test_data(training_pdf)).withColumn('tenant_id', f.lit(0))\noutgroup_df = spark.createDataFrame(factory.create_clustered_inter_test_data()).withColumn('tenant_id', f.lit(0))"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"3bd49a68-930f-42ab-a3a8-a45fecf769b0"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["training_df.show()"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"eee03097-fc4f-469c-ae91-7ff786bd7074"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["print(training_df.count())\nprint(ingroup_df.count())\nprint(outgroup_df.count())"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"462abebd-045f-4d36-b39d-99231626c921"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"markdown","source":["# Model setup & training"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"62411fbc-be0a-43ee-aeec-f06ada15e320"}}},{"cell_type":"code","source":["access_anomaly = AccessAnomaly(\n  tenantCol='tenant_id',\n  userCol='user',\n  resCol='res',\n  likelihoodCol='likelihood',\n  maxIter=1000\n)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"bd7a7b5c-e349-4493-b057-7486628a009b"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["model = access_anomaly.fit(training_df)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"04602cd1-952f-40b6-a574-be98119295af"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"markdown","source":["# Apply model & show result stats"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"71888616-235e-468a-bc1c-11458de860bb"}}},{"cell_type":"code","source":["ingroup_scored_df = model.transform(ingroup_df)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"5649ed1a-d958-4132-8b85-d15bd004c892"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["ingroup_scored_df.agg(\n  f.min('anomaly_score').alias('min_anomaly_score'),\n  f.max('anomaly_score').alias('max_anomaly_score'),\n  f.mean('anomaly_score').alias('mean_anomaly_score'),\n  f.stddev('anomaly_score').alias('stddev_anomaly_score'),\n).show()"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"8d83324a-90f7-46c7-b4d0-b53e599a6312"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["outgroup_scored_df = model.transform(outgroup_df)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"11c968a2-e0fc-4d7b-bb10-fada6e9f0964"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["outgroup_scored_df.agg(\n  f.min('anomaly_score').alias('min_anomaly_score'),\n  f.max('anomaly_score').alias('max_anomaly_score'),\n  f.mean('anomaly_score').alias('mean_anomaly_score'),\n  f.stddev('anomaly_score').alias('stddev_anomaly_score'),\n).show()"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"c52e8f6c-9b8d-4762-b1ce-01af9802b58a"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"markdown","source":["# Examine results"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"96fc88bf-597f-4454-b06d-2817ae2bd929"}}},{"cell_type":"code","source":["#\n# Select a subset of results to send to Log Analytics\n#\n\nfull_res_df = outgroup_scored_df.orderBy(f.desc('anomaly_score')).cache()\n\nfrom pyspark.sql.window import Window\n\nw = Window.partitionBy(\n                  'tenant_id',\n                  'user',\n                  'res'  \n                ).orderBy(\n                  f.desc('anomaly_score')\n                )\n\n# select values above threshold\nresults_above_threshold = full_res_df.filter(full_res_df.anomaly_score > 1.0)\n\n# get distinct resource/user and corresponding timestamp and highest score\nresults_to_la = results_above_threshold.withColumn(\n                  'index', f.row_number().over(w)\n                  ).orderBy(\n                    f.desc('anomaly_score')\n                  ).select(\n                    'tenant_id',\n                    f.col('user'),\n                    f.col('res'),\n                    'anomaly_score'\n                  ).where(\n                    'index == 1'\n                  ).limit(100).cache()\n\n# add a fake timestamp to the results\nresults_to_la = results_to_la.withColumn('timestamp', f.current_timestamp())\n  \ndisplay(results_to_la)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"f92d6660-c19d-41dd-87f7-4683dba19527"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"markdown","source":["# Display all resource accesses by users with highest anomalous score"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"3d1f7d63-97f7-405f-b8ac-a1a6ce67f0bc"}}},{"cell_type":"code","source":["from plotly import __version__\nfrom plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot, offline\n\nimport numpy as np\nimport pandas as pd\n\nprint (__version__) # requires version >= 1.9.0\n\n# run plotly in offline mode\noffline.init_notebook_mode()"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"21be9733-adb3-45e8-bf11-d450a551ff6b"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["#Find all server accesses of users with high predicted scores\n# For display, limit to top 25 results\nresults_to_display = results_to_la.orderBy(\n                  f.desc('anomaly_score')\n                ).limit(25).cache()\ninteresting_records = full_res_df.join(results_to_display, ['user'], 'left_semi')\nnon_anomalous_records = interesting_records.join(results_to_display, ['user', 'res'], 'left_anti')\n\ntop_non_anomalous_records = non_anomalous_records.groupBy(\n                          'tenant_id',\n                          'user', \n                          'res'\n                        ).agg(\n                          f.count('*').alias('count'),\n                        ).select(\n                          f.col('tenant_id'),\n                          f.col('user'),\n                          f.col('res'),\n                          'count'\n                        )\n\n#pick only a subset of non-anomalous record for UI\nw = Window.partitionBy(\n                  'tenant_id',\n                  'user',\n                ).orderBy(\n                  f.desc('count')\n                )\n\n# pick top non-anomalous set\ntop_non_anomalous_accesses = top_non_anomalous_records.withColumn(\n                  'index', f.row_number().over(w)\n                  ).orderBy(\n                    f.desc('count')\n                  ).select(\n                    'tenant_id',\n                    f.col('user'),\n                    f.col('res'),\n                    f.col('count')\n                  ).where(\n                    'index in (1,2,3,4,5)'\n                  ).limit(25)\n\n# add back anomalous record\nfileShare_accesses = (top_non_anomalous_accesses\n                          .select('user', 'res', 'count')\n                          .union(results_to_display.select('user', 'res', f.lit(1).alias('count'))).cache())"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"20e040fa-7c89-4eab-9e1d-41e0fdfdd94f"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["# get unique users and file shares\nhigh_scores_df = fileShare_accesses.toPandas()\nunique_arr = np.append(high_scores_df.user.unique(), high_scores_df.res.unique())\n\nunique_df = pd.DataFrame(data = unique_arr, columns = ['name'])\nunique_df['index'] = range(0, len(unique_df.index))\n\n# create index for source & target and color for the normal accesses\nnormal_line_color = 'rgba(211, 211, 211, 0.8)'\nanomolous_color = 'red'\nx = pd.merge(high_scores_df, unique_df, how='left', left_on='user', right_on='name').drop(['name'], axis=1).rename(columns={'index' : 'userIndex'})\nall_access_index_df = pd.merge(x, unique_df, how='left', left_on='res', right_on='name').drop(['name'], axis=1).rename(columns={'index' : 'resIndex'})\nall_access_index_df['color'] = normal_line_color\n\n# results_to_display index, color and \ny = results_to_display.toPandas().drop(['tenant_id', 'timestamp', 'anomaly_score'], axis=1)\ny = pd.merge(y, unique_df, how='left', left_on='user', right_on='name').drop(['name'], axis=1).rename(columns={'index' : 'userIndex'})\nhigh_scores_index_df = pd.merge(y, unique_df, how='left', left_on='res', right_on='name').drop(['name'], axis=1).rename(columns={'index' : 'resIndex'})\nhigh_scores_index_df['count'] = 1\nhigh_scores_index_df['color'] = anomolous_color\n\n# substract 1 for the red entries in all_access df\nhsi_df = high_scores_index_df[['user','res', 'count']].rename(columns={'count' : 'hsiCount'})\nall_access_updated_count_df = pd.merge(all_access_index_df, hsi_df, how='left', left_on=['user', 'res'], right_on=['user', 'res'])\nall_access_updated_count_df['count'] = np.where(all_access_updated_count_df['hsiCount']==1, all_access_updated_count_df['count'] - 1, all_access_updated_count_df['count'])\nall_access_updated_count_df = all_access_updated_count_df.loc[all_access_updated_count_df['count'] > 0]\nall_access_updated_count_df = all_access_updated_count_df[['user','res', 'count', 'userIndex', 'resIndex', 'color']]\n\n# combine the two tables\nframes = [all_access_updated_count_df, high_scores_index_df]\ndisplay_df = pd.concat(frames, sort=True)\n# display_df.head()"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"bd6e5803-2496-45f9-8a75-e7957848fad2"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":["data_trace = dict(\n    type='sankey',\n    domain = dict(\n      x =  [0,1],\n      y =  [0,1]\n    ),\n    orientation = \"h\",\n    valueformat = \".0f\",\n    node = dict(\n      pad = 10,\n      thickness = 30,\n      line = dict(\n        color = \"black\",\n        width = 0\n      ),\n      label = unique_df['name'].dropna(axis=0, how='any')\n    ),\n    link = dict(\n      source = display_df['userIndex'].dropna(axis=0, how='any'),\n      target = display_df['resIndex'].dropna(axis=0, how='any'),\n      value = display_df['count'].dropna(axis=0, how='any'),\n      color = display_df['color'].dropna(axis=0, how='any'),\n  )\n)\n\nlayout =  dict(\n    title = \"All resources accessed by users with highest anomalous scores\",\n    height = 772,\n    font = dict(\n      size = 10\n    ),    \n)\n\nfig = dict(data=[data_trace], layout=layout)\n\np = plot(fig, output_type='div')\n\ndisplayHTML(p)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"c72004cb-4368-4386-8192-4f8730503dcc"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0},{"cell_type":"code","source":[""],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"8facc7d8-1241-48c5-9e42-a1a3740784a0"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"type":"ipynbError","data":"","errorSummary":"","arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>"]}}],"execution_count":0}],"metadata":{"application/vnd.databricks.v1+notebook":{"notebookName":"byoml_demo","dashboards":[],"language":"python","widgets":{},"notebookOrigID":659272127833844}},"nbformat":4,"nbformat_minor":0}