Merge remote-tracking branch 'origin/master' into michal/6429

* origin/master: (29 commits) fix typo add new contributing doc, cleanup readme #6483 (#6484) wheel is not usd to skip this release step #6470 Removed line that breaks use of virtualenvs (#6471) #6459 remove internal classes from javadoc (#6463) #6474 removed python 2 and added python 3.6 to setup.py (#6476) #6412 fix LoD menu breaks in lab (#6467) #6431: fix for unexpected return value (#6473) remove version lock, jupyterlab-manager is fixed #6439 Optimalisation of scala test execution. (#6454) #6157 version has 'tags/' (#6468) fix KernelExecutionTest (#6469) #6443: fix displaying widgets by fixing parent messages (#6464) jarek/6452: remove jvm-repr workaround (#6465) disable broken test disable broken test #6358 Fix docker language warnings. (#6448) update documentation to workaround jupyterlab/jupyterlab#3215 #5813 move js dir to top, second PR (#6446) #5643 make javadoc accessible (#6450) ...
twosigma · Dec 11, 2017 · ab49b0c · ab49b0c
2 parents 28c9e7f + ba92ddc
commit ab49b0c
Show file tree

Hide file tree

Showing 266 changed files with 1,227 additions and 5,248 deletions.
diff --git a/.gitignore b/.gitignore
@@ -22,6 +22,7 @@ beakerx/static/
 beakerx.egg-info/
 beakerx/beakerx/static/
 beakerx/beakerx/kernel/
+beakerx/beakerx/javadoc/
 beakerx/dist
 
 kernel/build/

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,42 @@
+<!--
+    Copyright 2017 TWO SIGMA OPEN SOURCE, LLC
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
+
+           http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+
+# Contributing
+
+We welcome developers to extend and improve BeakerX in ways that can
+benefit everyone. In order for us to accept your code or pull request,
+we need for you to fill out and email back to us a scan of a signed copy of the
+[Contributor License Agreement](http://beakernotebook.com/cla.zip).
+
+BeakerX uses [Google Java
+style](https://google.github.io/styleguide/javaguide.html), and all
+Java code needs unit tests.  For JavaScript we use [Google JS
+style](https://google.github.io/styleguide/jsguide.html) with
+[require](http://requirejs.org/) instead of goog.  All files should
+end with newline and have a copyright and license banner.
+
+# Core Team Process
+
+For those with write access to the github repository, please use the
+following system for naming your branches and submitting your work.
+
+* Make one PR for each issue.  Each PR should be a branch from master
+  that can be merged independently of any other issue.
+* Name your branch "YourName/IssueNumber", eg "spot/6483".
+* Each commit should have the issue number in its message like "#6483"
+  so that github links it.
+* An admin will review, test, write feedback, and eventually merge the PR and close the issue.
+* Do not rebase your PRs.
diff --git a/README.md b/README.md
@@ -70,7 +70,7 @@ conda install -y -c conda-forge jupyterlab
 (cd beakerx; pip install -e . --verbose)
 beakerx-install
 jupyter labextension install @jupyter-widgets/jupyterlab-manager
-(cd beakerx/jslab; jupyter labextension install .)
+(cd js/lab; jupyter labextension install .)
 ```
 
 ### Update after Java change
@@ -88,9 +88,13 @@ The notebook extensions are installed to run out of the repo, so just
 a local build should suffice:
 
 ```
-(cd beakerx/js; yarn install)
+(cd js/notebook; yarn install)
 ```
 
+### Run Tests
+
+The Java unit tests are run with every build. See [test/README.md] for how to run the e2e tests.
+
 ## Groovy with Interactive Plotting and Tables:
 <img width="900" alt="screen shot" src="https://user-images.githubusercontent.com/963093/28300136-585f9f7c-6b4b-11e7-8827-b5807d3fc9a8.png">
 
@@ -106,18 +110,11 @@ docker run -p 8888:8888 beakerx/beakerx
 
 ## Contributing
 
-We welcome developers to extend and improve BeakerX in ways that can
-benefit everyone. In order for us to accept your code or pull request,
-we need for you to fill out and email back to us a scan of a signed copy of the
-[Contributor License Agreement](http://beakernotebook.com/cla.zip).
+See [CONTRIBUTING.md](CONTRIBUTING.md).
 
-BeakerX uses [Google Java
-style](https://google.github.io/styleguide/javaguide.html), and all
-Java code needs unit tests.  For JavaScript we use [Google JS
-style](https://google.github.io/styleguide/jsguide.html) with
-[require](http://requirejs.org/) instead of goog.  All files should
-end with newline and have a copyright and license banner.
+## Releasing
 
+See [RELEASE.md](RELEASE.md).
 
 ## Attribution
 

diff --git a/RELEASE.md b/RELEASE.md
@@ -1,3 +1,19 @@
+<!--
+    Copyright 2017 TWO SIGMA OPEN SOURCE, LLC
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
+
+           http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+
 Making a BeakerX release
 ===========================
 
@@ -13,14 +29,13 @@ Create the release
 ------------------
 
 Update the version in `beakerx/beakerx/_version.py` and
-`beakerx/js/package.json` and `beakerx/jslab/package.json`.  Commit
+`beakerx/js/package.json` and `js/lab/package.json`.  Commit
 the change and push the git tag.
 
 ```bash
 git clean -xfd
 cd beakerx
 python setup.py sdist
-python setup.py bdist_wheel --universal
 ```
 
 Make a test installation with pip into a fresh environment, if it works then continue with:
@@ -65,8 +80,8 @@ Publish on npmjs
 To update the embedded version of our widget library:
 
 - Do a full build.
-- Run `(cd beakerx/js; npm publish)`
-- Run `(cd beakerx/jslab; npm publish)`
+- Run `(cd js/notebook; npm publish)`
+- Run `(cd js/lab; npm publish)`
 
 Release to Docker Hub
 ---------------------

diff --git a/StartHere.ipynb b/StartHere.ipynb
@@ -13,39 +13,39 @@
     "[The Jupyter documentation](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Running%20Code.html) covers interacting with code cells, markdown, and notebooks. The tutorials below show the features of the BeakerX extension.\n",
     "\n",
     "### Languages\n",
-    "[Groovy](doc/groovy/Groovy.ipynb), [Java](doc/java/Java.ipynb), [Scala](doc/scala/Scala.ipynb), [Clojure](doc/clojure/Clojure.ipynb), [SQL](doc/sql/SQL.ipynb), [Kotlin](doc/kotlin/Kotlin.ipynb).\n",
+    "[Groovy](doc/groovy/Main.ipynb), [Java](doc/java/Main.ipynb), [Scala](doc/scala/Main.ipynb), [Clojure](doc/clojure/Main.ipynb), [SQL](doc/sql/Main.ipynb), [Kotlin](doc/kotlin/Main.ipynb).\n",
     "\n",
     "### Magics\n",
-    "[Timing](doc/groovy/TimeExecutionMeasurementWithMagicCommands.ipynb), [Classpath and Imports](doc/groovy/AddJarToClasspathMagicCommands.ipynb), [Properties, Heap Size and other JVM parameters](doc/groovy/java_args.ipynb), [Defining New Magics](doc/groovy/LoadMagicCommand.ipynb).\n",
+    "[Timing](doc/groovy/TimeExecutionMeasurementWithMagicCommands.ipynb), [Classpath and Imports](doc/groovy/AddJarToClasspathMagicCommands.ipynb), [Properties, Heap Size and other JVM parameters](doc/groovy/JavaArgs.ipynb), [Defining New Magics](doc/groovy/LoadMagicCommand.ipynb).\n",
     "\n",
     "### Autotranslation\n",
-    "[Python](doc/python/autotranslation_python.ipynb), [Groovy](doc/groovy/autotranslation_groovy_demo.ipynb)\n",
+    "[Python](doc/python/AutoTranslation.ipynb), [Groovy](doc/groovy/AutoTranslation.ipynb)\n",
     "\n",
     "### Groovy Plotting and Charting\n",
-    "[Example and Interaction](doc/groovy/chartingTutorial.ipynb), [Time Series and General APIs and Features](doc/groovy/PlotFeatures.ipynb), [Category Plots and Bar Charts](doc/groovy/categoryPlot.ipynb), [Levels of Detail](doc/groovy/levelsOfDetails.ipynb), [Histograms](doc/groovy/histogram.ipynb), [Heatmaps](doc/groovy/heatmap.ipynb), [Treemaps](doc/groovy/treemap.ipynb), [Plot Actions](doc/groovy/plotActions.ipynb), [Plot Seamless Updates](doc/groovy/Plots_python_style.ipynb), [Second Y Axis](doc/groovy/2ndYaxis.ipynb).\n",
+    "[Example and Interaction](doc/groovy/Charting.ipynb), [Time Series and General APIs and Features](doc/groovy/PlotFeatures.ipynb), [Category Plots and Bar Charts](doc/groovy/CategoryPlot.ipynb), [Levels of Detail](doc/groovy/LevelsOfDetails.ipynb), [Histograms](doc/groovy/Histogram.ipynb), [Heatmaps](doc/groovy/Heatmap.ipynb), [Treemaps](doc/groovy/Treemap.ipynb), [Plot Actions](doc/groovy/PlotActions.ipynb), [Plot Seamless Updates](doc/groovy/PlotsPythonStyle.ipynb), [Second Y Axis](doc/groovy/2ndYaxis.ipynb).\n",
     "\n",
     "### Table Display\n",
-    "[Groovy API including Actions](doc/groovy/tableApi.ipynb), [64-Bit Integers and BigNums](doc/groovy/bigInts.ipynb), [Automatic Display of Simple Data Structures](doc/groovy/MapLikeTable.ipynb), [Handling of Large Tables](doc/groovy/big_table_test.ipynb).\n",
+    "[Groovy API including Actions](doc/groovy/TableAPI.ipynb), [64-Bit Integers and BigNums](doc/groovy/BigInts.ipynb), [Automatic Display of Simple Data Structures](doc/groovy/MapLikeTable.ipynb), [Handling of Large Tables](doc/groovy/BigTable.ipynb).\n",
     "\n",
     "### BeakerX Plotting in Other Languages\n",
-    "[JavaScript](doc/python/plot-js-api.ipynb), [Python](doc/python/pythonChartingAPI.ipynb), [Scala](doc/scala/Plot.ipynb).\n",
+    "[JavaScript](doc/python/PlotJSAPI.ipynb), [Python](doc/python/ChartingAPI.ipynb), [Scala](doc/scala/Plot.ipynb).\n",
     "\n",
     "### Python\n",
     "\n",
-    "[Tables including pandas integration](doc/python/tableApi_python.ipynb), [Time Series](doc/python/pythonChartingAPI.ipynb), [Heatmaps](doc/python/heatmap_python.ipynb),  [Category Plots](doc/python/categoryPlot_python.ipynb), [Treemaps](doc/python/treemap_python.ipynb), [Histograms](doc/python/histogram_python.ipynb), [EasyForm](doc/python/EasyFormPythonDemos.ipynb), [Output Containers](doc/python/OutputContainersAndLayoutManagers_python.ipynb).\n",
+    "[Tables including pandas integration](doc/python/TableAPI.ipynb), [Time Series](doc/python/ChartingAPI.ipynb), [Heatmaps](doc/python/Heatmap.ipynb),  [Category Plots](doc/python/CategoryPlot.ipynb), [Treemaps](doc/python/Treemap.ipynb), [Histograms](doc/python/Histogram.ipynb), [EasyForm](doc/python/EasyForm.ipynb), [Output Containers](doc/python/OutputContainersAndLayoutManagers.ipynb).\n",
     "\n",
     "### Scala\n",
-    "[Tables](doc/scala/TableApi.ipynb), [Plotting](doc/scala/Plot.ipynb), [EasyForm](doc/scala/ScalaDemos.ipynb).\n",
+    "[Tables](doc/scala/TableAPI.ipynb), [Plotting](doc/scala/Plot.ipynb), [EasyForm](doc/scala/EasyForm.ipynb).\n",
     "\n",
     "### Media Outputs and Displayer Customization\n",
     "\n",
     "[Media and MIME Outputs](doc/groovy/Mime.ipynb), [Display of Null](doc/groovy/showNullExecutionResult.ipynb), [Custom Displayers and `jvm-repr`](doc/groovy/JvmRepr.ipynb).\n",
     "\n",
     "### Forms, Widgets, and Interaction\n",
-    "[EasyForm](doc/groovy/EasyFormDemos.ipynb), [Output Containers and Layout Managers](doc/groovy/OutputContainersAndLayoutManagers.ipynb), [Groovy interface to Jupyter JS Widgets](doc/groovy/JavaWidgets.ipynb), [Interactive recomputation](doc/groovy/interactive.ipynb).\n",
+    "[EasyForm](doc/groovy/EasyForm.ipynb), [Output Containers and Layout Managers](doc/groovy/OutputContainersAndLayoutManagers.ipynb), [Groovy interface to Jupyter JS Widgets](doc/groovy/JavaWidgets.ipynb), [Interactive recomputation](doc/groovy/Interactive.ipynb).\n",
     "\n",
     "### Automation\n",
-    "[Progress Reporting API](doc/groovy/ProgressUpdateReporting.ipynb), [Initialization Cells](doc/python/init-cells.ipynb), [Get Code](doc/groovy/CodeCell.ipynb).\n",
+    "[Progress Reporting API](doc/groovy/ProgressUpdateReporting.ipynb), [Initialization Cells](doc/python/InitCells.ipynb), [Get Code](doc/groovy/CodeCell.ipynb).\n",
     "\n",
     "### Tablesaw\n",
     "[Tablesaw](doc/groovy/Tablesaw.ipynb) Pandas for the JVM.\n",

diff --git a/beakerx/MANIFEST.in b/beakerx/MANIFEST.in
@@ -1,5 +1,6 @@
 recursive-include beakerx/static *
 recursive-include beakerx/kernel *
+recursive-include beakerx/javadoc *
 
 include LICENSE
 include NOTICE

diff --git a/beakerx/beakerx/handlers.py b/beakerx/beakerx/handlers.py
@@ -13,13 +13,14 @@
 # limitations under the License.
 
 from notebook.utils import url_path_join
-from notebook.base.handlers import APIHandler
+from notebook.base.handlers import APIHandler, IPythonHandler
 
 import json
 from tornado import web
 from .environment import *
 import beakerx
 import tornado
+import os
 
 
 class SettingsHandler(APIHandler):
@@ -53,11 +54,25 @@ def get(self):
         self.finish(json.dumps(data))
 
 
+class JavaDoc(web.StaticFileHandler, IPythonHandler):
+    def initialize(self):
+        beakerx_path = os.path.dirname(beakerx.__file__)
+        web.StaticFileHandler.initialize(self, path=os.path.join(beakerx_path, 'javadoc'))
+
+    @web.authenticated
+    def get(self, path):
+        self.set_header('Content-Type', 'text/html')
+        return web.StaticFileHandler.get(self, path)
+
+
 def load_jupyter_server_extension(nbapp):
     web_app = nbapp.web_app
     host_pattern = '.*$'
     settings_route_pattern = url_path_join(web_app.settings['base_url'], '/beakerx', '/settings')
     version_route_pattern = url_path_join(web_app.settings['base_url'], '/beakerx', '/version')
+    javadoc_route_pattern = url_path_join(web_app.settings['base_url'], '/beakerx', '/javadoc/(.*)')
     web_app.add_handlers(host_pattern, [(settings_route_pattern, SettingsHandler)])
     web_app.add_handlers(host_pattern, [(version_route_pattern, VersionHandler)])
+    web_app.add_handlers(host_pattern, [(javadoc_route_pattern, JavaDoc)])
+
     nbapp.log.info("[beakerx] enabled")
diff --git a/beakerx/beakerx/static/html/settings_tab.html b/beakerx/beakerx/static/html/settings_tab.html
@@ -15,15 +15,25 @@
 -->
 <div id="beakerx" class="tab-pane">
     <style>
-        .bko-spacing .form-control { margin-right: 10px; }
-        #beakerx_info .beakerx_site_link { display: inline-block; margin: 5px 0; }
-        #beakerx_info .beakerx_site_link img { height: 35px; }
+        .bko-spacing .form-control {
+            margin-right: 10px;
+        }
+
+        #beakerx_info .beakerx_site_link {
+            display: inline-block;
+            margin: 5px 0;
+        }
+
+        #beakerx_info .beakerx_site_link img {
+            height: 35px;
+        }
     </style>
     <div id="beakerx_env_toolbar" class="list_toolbar row">
         <div class="col-xs-12 no-padding">
-           <span id="beakerx_info" class="toolbar_info"></span>
+            <span id="beakerx_info" class="toolbar_info"></span>
         </div>
-    </div><br/>
+    </div>
+    <br/>
 
     <div class="beakerx_container">
         <form id="beakerx_jvm_settings_form" class="form-horizontal">

diff --git a/beakerx/beakerx/tabledisplay/tabledisplay.py b/beakerx/beakerx/tabledisplay/tabledisplay.py
@@ -155,7 +155,7 @@ def convert_type(object_type, value):
                 return "integer"
         if object_type == "uint64":
             return "int64"
-        if is_date(value):
+        if object_type.startswith("datetime64"):
             return "time"
         if isinstance(value, str):
             return "string"

diff --git a/beakerx/beakerx/tabledisplay/tableitems.py b/beakerx/beakerx/tabledisplay/tableitems.py
@@ -43,15 +43,15 @@ class ColumnType(Enum):
 
 class DateType:
     type = "Date"
-    
+
     def __init__(self, value):
-        self.timestamp = parse(value).timestamp() * 1000
+        self.timestamp = parse(str(value)).timestamp() * 1000
 
 
 class DataBarsRenderer:
     type = "DataBars"
     includeText = True
-    
+
     def __init__(self, x):
         self.includeText = x
 
@@ -60,7 +60,7 @@ class DecimalStringFormat:
     type = "decimal"
     minDecimals = 4
     maxDecimals = 4
-    
+
     def __init__(self, min=4, max=4):
         self.minDecimals = min
         self.maxDecimals = max
@@ -96,8 +96,8 @@ class TableDisplayCellHighlighter:
     FULL_ROW = HighlightStyle.FULL_ROW
     SINGLE_COLUMN = HighlightStyle.SINGLE_COLUMN
     defaultStyle = HighlightStyle.FULL_ROW
-    
+
     @staticmethod
     def getHeatmapHighlighter(colName, style=defaultStyle, minVal=None, maxVal=None, minColor=None, maxColor=None):
         return HeatmapHighlighter (colName, style, minVal, maxVal, minColor, maxColor)
-        
+
diff --git a/beakerx/setup.py b/beakerx/setup.py
@@ -30,17 +30,19 @@
 cmdclass = create_cmdclass(develop_wrappers=[
     'js',
     'java',
+    'javadoc',
 ], distribute_wrappers=[
     'js',
-    'java'
+    'java',
+    'javadoc',
 ])
 cmdclass['js'] = install_node_modules(
-    path='js',
-    build_dir=os.path.join(here, 'js', 'dist'),
-    source_dir=os.path.join(here, 'js', 'src')
+    path='../js/notebook',
+    build_dir=os.path.join(here, '../js/notebook', 'dist'),
+    source_dir=os.path.join(here, '../js/notebook', 'src')
 )
 cmdclass['java'] = run_gradle(cmd='build')
-
+cmdclass['javadoc'] = run_gradle(cmd='base:javadoc')
 setup_args = dict(
     name                = 'beakerx',
     description         = 'BeakerX: Beaker Extensions for Jupyter Notebook',
@@ -66,12 +68,11 @@
         'Intended Audience :: Developers',
         'Intended Audience :: Science/Research',
         'Topic :: Multimedia :: Graphics',
-        'Programming Language :: Python :: 2',
-        'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3',
         'Programming Language :: Python :: 3.3',
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
+        'Programming Language :: Python :: 3.6',
     ],
     entry_points={
         'console_scripts': [

diff --git a/beakerx/setupbase.py b/beakerx/setupbase.py
@@ -53,7 +53,6 @@ def list2cmdline(cmd_list):
 here = os.path.abspath(os.path.dirname(sys.argv[0]))
 root = os.path.abspath(os.path.join(here, os.pardir))
 kernel_path = os.path.join(root, 'kernel')
-site_packages = site.getsitepackages()[0]
 is_repo = os.path.exists(os.path.join(root, '.git'))
 node_modules = os.path.join(here, 'js', 'node_modules')
 node_modules_path = ':'.join([