Introducing a caching layer

.landscape.yml

@@ -9,6 +9,7 @@ pylint:
   disable:
     - cyclic-import
     - invalid-name
+    - logging-format-interpolation
   options:
     docstring-min-length: 10
 pep8:
@@ -20,6 +21,3 @@ ignore-paths:
 ignore-patterns:
     - ^example/doc_.*\.py$
     - (^|/)docs(/|$)
-python-targets:
-    - 2
-    - 3

README.md

@@ -44,32 +44,24 @@ Dashed provides:
     slicing and dicing large, realtime datasets
 
 
-Buzz Phrases
-------------
-
-* Analytics at the speed of thought!
-* Instantaneous learning curve
-* Realtime analytics when querying [Druid.io](http://druid.io)
-* Extentsible to infinity
-
 Database Support
 ----------------
 
 Dashed was originally designed on to of Druid.io, but quickly broadened
 its scope to support other databases through the use of SqlAlchemy, a Python
 ORM that is compatible with
-[most common databases](http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html). 
+[most common databases](http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html).
 
 
 What is Druid?
 -------------
 From their website at http://druid.io
 
-*Druid is an open-source analytics data store designed for 
-business intelligence (OLAP) queries on event data. Druid provides low 
-latency (real-time) data ingestion, flexible data exploration, 
-and fast data aggregation. Existing Druid deployments have scaled to 
-trillions of events and petabytes of data. Druid is best used to 
+*Druid is an open-source analytics data store designed for
+business intelligence (OLAP) queries on event data. Druid provides low
+latency (real-time) data ingestion, flexible data exploration,
+and fast data aggregation. Existing Druid deployments have scaled to
+trillions of events and petabytes of data. Druid is best used to
 power analytic dashboards and applications.*
 
 
@@ -109,50 +101,28 @@ your datasources for Dashed to be aware of, and they should show up in
 `Menu -> Datasources`, from where you can start playing with your data!
 
 Configuration
--------------
+=======
+[most common databases](http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html).
 
-To configure your application, you need to create a file (module) 
-`dashed_config.py` and make sure it is in your PYTHONPATH. Here are some
-of the parameters you can copy / paste in that configuration module:
 
-```
-#---------------------------------------------------------
-# Dashed specifix config
-#---------------------------------------------------------
-ROW_LIMIT = 5000
-WEBSERVER_THREADS = 8
-
-DASHED_WEBSERVER_PORT = 8088
-#---------------------------------------------------------
+Installation & Configuration
+----------------------------
 
-#---------------------------------------------------------
-# Flask App Builder configuration
-#---------------------------------------------------------
-# Your App secret key
-SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h'
+(See in the documentation)
+[http://mistercrunch.github.io/panoramix-docs/installation.html]
 
-# The SQLAlchemy connection string.
-SQLALCHEMY_DATABASE_URI = 'sqlite:////tmp/dashed.db'
 
-# Flask-WTF flag for CSRF
-CSRF_ENABLED = True
-
-# Whether to run the web server in debug mode or not
-DEBUG = True
-```
-
-This file also allows you to define configuration parameters used by
-Flask App Builder, the web framework used by Dashed. Please consult
-the [Flask App Builder Documentation](http://flask-appbuilder.readthedocs.org/en/latest/config.html) for more information on how to configure Dashed.
-
-
-* From the UI, enter the information about your clusters in the 
-``Admin->Clusters`` menu by hitting the + sign. 
+What is Druid?
+-------------
+From their website at http://druid.io
 
-* Once the Druid cluster connection information is entered, hit the 
-``Admin->Refresh Metadata`` menu item to populate
+*Druid is an open-source analytics data store designed for
+business intelligence (OLAP) queries on event data. Druid provides low
+latency (real-time) data ingestion, flexible data exploration,
+and fast data aggregation. Existing Druid deployments have scaled to
+trillions of events and petabytes of data. Druid is best used to
+power analytic dashboards and applications.*
 
-* Navigate to your datasources
 
 More screenshots
 ----------------

TODO.md

@@ -2,7 +2,6 @@
 List of TODO items for Dashed
 
 ## Important
-* **Caching:** integrate with flask-cache
 * **Getting proper JS testing:** unit tests on the Python side are pretty
     solid, but now we need a test suite for the JS part of the site,
     testing all the ajax-type calls
@@ -14,7 +13,6 @@ List of TODO items for Dashed
 * **Stars:** set dashboards, slices and datasets as favorites
 * **Homepage:** a page that has links to your Slices and Dashes, favorited
     content, feed of recent actions (people viewing your objects)
-* **Comments:** allow for people to comment on slices and dashes
 * **Dashboard URL filters:** `{dash_url}#fltin__fieldname__value1,value2`
 * **Default slice:** choose a default slice for the dataset instead of
     default endpoint
@@ -34,9 +32,11 @@ List of TODO items for Dashed
 * **Slack integration** - TBD
 * **Sexy Viz Selector:** the visualization selector should be a nice large
     modal with nice thumbnails for each one of the viz
+* **Comments:** allow for people to comment on slices and dashes
 
 
 ## Easy-ish fix
+* Kill switch for Druid in docs
 * CREATE VIEW button from SQL editor
 * Test button for when editing SQL expression
 * Slider form element

dashed/__init__.py

@@ -6,6 +6,7 @@
 from flask.ext.appbuilder import SQLA, AppBuilder, IndexView
 from flask.ext.appbuilder.baseviews import expose
 from flask.ext.migrate import Migrate
+from flask.ext.cache import Cache
 
 
 APP_DIR = os.path.dirname(__file__)
@@ -18,6 +19,9 @@
 app = Flask(__name__)
 app.config.from_object(CONFIG_MODULE)
 db = SQLA(app)
+
+cache = Cache(app, config=app.config.get('CACHE_CONFIG'))
+
 migrate = Migrate(app, db, directory=APP_DIR + "/migrations")
 
 

dashed/assets/javascripts/dashboard.js

@@ -22,7 +22,7 @@ var Dashboard = function (dashboardData) {
       dashboard.slices.forEach(function (data) {
         var slice = px.Slice(data, dash);
         $("#slice_" + data.slice_id).find('a.refresh').click(function () {
-          slice.render();
+          slice.render(true);
         });
         sliceObjects.push(slice);
         slice.render();
@@ -90,7 +90,7 @@ var Dashboard = function (dashboardData) {
       var gridster = $(".gridster ul").gridster({
         autogrow_cols: true,
         widget_margins: [10, 10],
-        widget_base_dimensions: [100, 100],
+        widget_base_dimensions: [95, 95],
         draggable: {
           handle: '.drag'
         },
@@ -113,6 +113,16 @@ var Dashboard = function (dashboardData) {
           };
         }
       }).data('gridster');
+
+      // Displaying widget controls on hover
+      $('.chart-header').hover(
+        function () {
+          $(this).find('.chart-controls').fadeIn(300);
+        },
+        function () {
+          $(this).find('.chart-controls').fadeOut(300);
+        }
+      );
       $("div.gridster").css('visibility', 'visible');
       $("#savedash").click(function () {
         var expanded_slices = {};
@@ -168,6 +178,11 @@ var Dashboard = function (dashboardData) {
       $('#filters').click(function () {
         alert(dashboard.readFilters());
       });
+      $('#refresh_dash').click(function () {
+        dashboard.slices.forEach(function (slice) {
+          slice.render(true);
+        });
+      });
       $("a.remove-chart").click(function () {
         var li = $(this).parents("li");
         gridster.remove_widget(li);
@@ -226,4 +241,5 @@ var Dashboard = function (dashboardData) {
 
 $(document).ready(function () {
   Dashboard($('.dashboard').data('dashboard'));
+  $('[data-toggle="tooltip"]').tooltip({ container: 'body' });
 });

dashed/assets/javascripts/explore.js

@@ -53,19 +53,18 @@ function prepForm() {
   });
 }
 
-function renderSlice() {
+function druidify() {
+  $('.query-and-save button').attr('disabled', 'disabled');
+  $('.btn-group.results span,a').attr('disabled', 'disabled');
+  $('div.alert').remove();
+  $('#is_cached').hide();
+  history.pushState({}, document.title, slice.querystring());
   prepForm();
   slice.render();
 }
 
 function initExploreView() {
 
-  function druidify() {
-    $('div.alert').remove();
-    history.pushState({}, document.title, slice.querystring());
-    renderSlice();
-  }
-
   function get_collapsed_fieldsets() {
     var collapsed_fieldsets = $("#collapsed_fieldsets").val();
 
@@ -199,9 +198,7 @@ function initExploreView() {
     bindOrder: 'sortableStop'
   });
   $("form").show();
-  $('[data-toggle="tooltip"]').tooltip({
-    container: 'body'
-  });
+  $('[data-toggle="tooltip"]').tooltip({ container: 'body' });
   $(".ui-helper-hidden-accessible").remove(); // jQuery-ui 1.11+ creates a div for every tooltip
 
   function set_filters() {
@@ -319,7 +316,7 @@ $(document).ready(function () {
   $('.slice').data('slice', slice);
 
   // call vis render method, which issues ajax
-  renderSlice();
+  druidify();
 
   // make checkbox inputs display as toggles
   $(':checkbox')

dashed/assets/javascripts/modules/dashed.js

@@ -169,16 +169,54 @@ var px = (function () {
         }
         return qrystr;
       },
+      getWidgetHeader: function () {
+        return this.container.parents("li.widget").find(".chart-header");
+      },
       jsonEndpoint: function () {
         var parser = document.createElement('a');
         parser.href = data.json_endpoint;
-        var endpoint = parser.pathname + this.querystring() + "&json=true";
+        var endpoint = parser.pathname + this.querystring();
+        endpoint += "&json=true";
+        endpoint += "&force=" + this.force;
         return endpoint;
       },
       done: function (data) {
         clearInterval(timer);
         token.find("img.loading").hide();
         container.show();
+
+        var cachedSelector = null;
+        if (dashboard === undefined) {
+          cachedSelector = $('#is_cached');
+          if (data !== undefined && data.is_cached) {
+            cachedSelector
+              .click(function () {
+                slice.render(true);
+              })
+              .attr('title', 'Served from data cached at ' + data.cached_dttm + '. Click to force-refresh')
+              .show()
+              .tooltip('fixTitle');
+          } else {
+            cachedSelector.hide();
+          }
+        } else {
+          var refresh = this.getWidgetHeader().find('.refresh');
+          if (data !== undefined && data.is_cached) {
+            refresh
+            .addClass('danger')
+            .attr(
+              'title',
+              'Served from data cached at ' + data.cached_dttm + '. Click to force-refresh')
+            .tooltip('fixTitle');
+          } else {
+            refresh
+            .removeClass('danger')
+            .attr(
+              'title',
+              'Click to force-refresh')
+            .tooltip('fixTitle');
+          }
+        }
         if (data !== undefined) {
           $("#query_container").html(data.query);
         }
@@ -194,7 +232,8 @@ var px = (function () {
         $('#csv').click(function () {
           window.location = data.csv_endpoint;
         });
-        $('.btn-group.results span').removeAttr('disabled');
+        $('.btn-group.results span,a').removeAttr('disabled');
+        $('.query-and-save button').removeAttr('disabled');
         always(data);
       },
       error: function (msg) {
@@ -204,6 +243,8 @@ var px = (function () {
         container.show();
         $('span.query').removeClass('disabled');
         $('#timer').addClass('btn-danger');
+        $('.btn-group.results span,a').removeAttr('disabled');
+        $('.query-and-save button').removeAttr('disabled');
         always(data);
       },
       width: function () {
@@ -228,8 +269,11 @@ var px = (function () {
           }, 500);
         });
       },
-      render: function () {
-        $('.btn-group.results span').attr('disabled', 'disabled');
+      render: function (force) {
+        if (force === undefined) {
+          force = false;
+        }
+        this.force = force;
         token.find("img.loading").show();
         container.hide();
         container.html('');