Skip to content

Instrumentation Prototyping Environment

Jump to bottom
Liam Mc Carthy edited this page Jun 22, 2022 · 12 revisions

Docker Instructions

Docker Desktop Download additional Instructions for Win 10/11 Home editions

Download the correct installer and run as normal.

Apache Superset Instructions

  • Download zip from: Superset -- This bypasses a common error (CRLF) during git clone you might otherwise run into when using docker on Windows devices.
  • Pull Docker Images:
docker-compose -f docker-compose-arlis.yml pull
  • Build Docker Container(s)
docker-compose -f docker-compose-arlis.yml up
  • Navigate to localhost:8088
  • For prototype env user/pass = admin

These instructions are adapted from the Superset Startup Instructions

Instrumenting Superset

adding UserALE.us to Superset

By default, Superset does not include UserALE on composition, but the ARLIS build comes with a packaged implementation. We utilized a simple, 'script-tag' method to inject UserALE.js in all Superset windows:

  • create a top-level directory @ master with the following structure In step three of the startup instructions, instead of entering:
Superset
-*custom
--*js
---*lib

Superset
-custom
--js
---lib
----*userale-2.3.0.min.js
  • then add a new .html file to your custom folder--this will allow for injecting the UserALE.js script into Superset windows:
Superset
-custom
-*tail_js_custom_extra.html
--js
---lib
----userale-2.3.0.min.js
  • add UserALE.js as a script tag to the .html file, along with custom settings:
  <script
    src="{{ url_for('static', filename='js/lib/userale-2.3.0.min.js') }}" #works out of the box if downloaded via source or npm (flagon-userale)
    data-url="http://localhost:8000/" #custom settings for userale.js
    data-user="superset-user"
    data-version="2.3.0"
    data-tool="Superset"
  ></script>
  • you will also need to modify certain docker scripts so that the .html page is built along with the superset front-end resources. This includes mods to the docker templates and to the docker-compose file, see links for examples. Note that this is a one-time operations. Additional scripts, configurations can now be added directly through the custom .html page.

If your custom script doesn't appear to be working or is using an outdated version of itself, try cleaning out your browser's file cache. The issue may be due to your browser referencing an outdated version of your custom script stored in memory. Alternatively, if your browser offers an incognito mode or similar function which does not store caches, try using that to run Superset.

Using the UserALE.js API and adding additional Custom Scripts

  • to add a new custom script, add it to your js/lib folder:
Superset
-custom
-tail_js_custom_extra.html
--js
---lib
----userale-2.3.0.min.js
----*custom.js
  • then point to it in your .html page"
  <script
    src="{{ url_for('static', filename='js/lib/userale-2.3.0.min.js') }}" #works out of the box if downloaded via source or npm (flagon-userale)
    data-url="http://localhost:8000/" #custom settings for userale.js
    data-user="superset-user"
    data-version="2.3.0"
    data-tool="Superset"
  ></script>
  <script
    src="{{ url_for('static', filename='js/lib/custom.js') }}"
  ></script>
  • through your new custom.js script you can add additional UserALE functionality including filters and custom logging using the UserALE.js API: 
#example filter
window.userale.filter(function (log) { #note: use `window` to call userale.* functions if UserALE.js is added as a script (vs. module)
    var type_array = ['mouseup', 'mouseover', 'mousedown', 'keydown', 'dblclick', 'blur', 'focus', 'input', 'wheel'];
    var logType_array = ['interval'];
    return !type_array.includes(log.type) && !logType_array.includes(log.logType);
});

Elastic Stack Instructions

Elastic Stack

git clone https://github.com/UMD-ARLIS/docker-elk #clone local version
cd docker-elk
git fetch origin #pull all remote heads (makes all branches available)
git checkout flagon-configs (pull our branch with added instrumentation)

Elastic Stack Startup instructions

Startup docker-elk

cd docker-elk
docker-compose up

ref to remove images, containers

Kickstart Elastic Index

see this guide: Register an index in Kibana to see the logs

Modifications on Elastic Stack Instructions

not actually an issue with docker-elk As with Superset, use code>download ZIP via GitHub to download the Elastic Stack, which includes Elasticsearch, Logstash, and Kibana.

Clone this wiki locally