NUP-2394: network API code example #3520

marionleborgne · 2017-04-05T01:04:56Z

The code example for the network API is done but the PR is blocked on NUP-2396. Currently the example has to call customCompute() on the classifierRegion since classification of continuous values is not supported by the network API. I'm moving on to the blocking issue (NUP-2396) to fix this.

Todos, for when I get back to this issue:

remove call to customCompute() and make use of new region links created in NUP-2396.
take care of TODOs in nupic/docs/examples/params/model.yaml by adding new params documentation based on feedback collected in google doc.

cc @rhyolight

rhyolight · 2017-04-05T03:56:45Z

Only input so far, is if we're going to go all in on YAML model params for this example, we should also change the algorithm quick start to use the same YAML file to load params.

marionleborgne · 2017-04-05T17:13:57Z

OK, I'll update the algorithm example.

…94-network-api # Conflicts: # docs/examples/opf/model_params.py

…xamples (OPF, network API and algo)

* Use new network links created in NUP-2396 * Remove customCompute() usage

marionleborgne · 2017-04-08T02:14:27Z

@rhyolight This PR is blocked by #3526.

Changelog:

Use new links between SensorRegion and SDRClassifierRegion that allow to pass bucketIdx and actValue so that the classifier can predict continuous values - like in the 2 other quick-start examples.
Update YAML with more comments based on @scottpurdy 's feedback.

Once #3526 is merged, this is ready to be reviewed.

rhyolight · 2017-04-10T15:25:17Z

@marionleborgne I also noticed you are writing results to an output file instead of to the console, so the RST will need to be updated to reflect these changes in the descriptive text.

marionleborgne · 2017-04-10T17:32:10Z

@rhyolight Actually, I am still writing to the console. I just added writing to an output file, so that I can compare the results of the 3 examples. I'll update the descriptive text.

rhyolight · 2017-04-10T17:34:00Z

Sounds good. I think the examples should be as simple as possible. Writing to the console is fine.

…

--------- Matt Taylor OS Community Manager Numenta

On Mon, Apr 10, 2017 at 10:32 AM, Marion Le Borgne ***@***.*** > wrote: @rhyolight <https://github.com/rhyolight> Actually, I am still writing to the console. I just added writing to an output file, so that I can compare the results of the 3 examples. I'll update the descriptive text. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3520 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA8zhIuzbihVJoZM7mHiXacYNGHcaGnks5rumebgaJpZM4Mznwm> .

marionleborgne · 2017-04-10T17:48:33Z

After discussion with @rhyolight, the plan is to take out writing to file once we fix the discrepancy between prediction results. Out of the scope of this PR. Task tracked here: https://jira.numenta.com/browse/NUP-2400

scottpurdy · 2017-04-10T22:20:46Z

docs/examples/params/model.yaml

@@ -0,0 +1,317 @@
+# Type of model that the rest of these parameters apply to.


I'm not sure that we want to have the full documentation for all parameters in every model configuration file. Perhaps we could centralize this information somewhere and link to that reference page at the top of this file?

Do you mean other places in the codebase where we have model params with detailed comments? If yes, I agree.

I think this file is a great place for it, because it will show up here (currently python, but soon yaml):

http://docs.numenta.org/quick-start/example-model-params.html

Perhaps we should remove the comments from other places and refer them to the API docs?

I see, yeah might be fine here. I think the real source of truth should be the algorithm doc strings but those are spread out throughout the code so this can be the place where we include them all. We may want to have references back to the individual algorithms in case they get out of sync.

…94-network-api

…94-network-api # Conflicts: # docs/examples/opf/model_params.py

marionleborgne · 2017-04-17T16:40:56Z

@rhyolight @scottpurdy: can someone review this PR? nothing is blocking it since #3526 was merged last week.

scottpurdy

One thing that was holding up my feedback is a research discussion on the network api. If we are going to keep saying that that is the way to use things then I don't think we should include a conflicting example (full example without network api) in the docs. But let's hold off on that discussion until after I have that discussion with Subutai, Marcus, others that are interested. And it doesn't have to hold up this PR.

scottpurdy · 2017-04-14T00:27:25Z

docs/examples/algo/complete-example.py

@@ -1,144 +1,171 @@
 import csv
 import datetime
-
+import yaml


goes after numpy

scottpurdy · 2017-04-14T00:28:00Z

docs/examples/algo/complete-example.py

 from nupic.research.spatial_pooler import SpatialPooler
 from nupic.research.temporal_memory import TemporalMemory
 from nupic.algorithms.sdr_classifier_factory import SDRClassifierFactory

-_INPUT_FILE_PATH = "../data/gymdata.csv"
+_NUM_RECORDS = 3000
+_INPUT_FILE_PATH = '../data/gymdata.csv'


use double quotes here and elsewhere

scottpurdy · 2017-04-14T00:40:15Z

docs/examples/algo/complete-example.py


  sp = SpatialPooler(
    # How large the input encoding will be.
    inputDimensions=(encodingWidth),
    # How many mini-columns will be in the Spatial Pooler.
-    columnDimensions=(2048),
+    columnDimensions=(spParams['columnCount']),


We probably don't want to pull out all the params one at a time. Can you extend spParams with any changes needed and then construct the SP with SpatialPooler(**spParams)?

This is @rhyolight 's example. I think he pulled the params one by one so that he can describe them in the the quick-start guide: http://nupic.docs.numenta.org/quick-start/algorithms.html

scottpurdy · 2017-04-14T00:40:52Z

docs/examples/algo/complete-example.py

@@ -1,144 +1,171 @@
 import csv


Are these examples embedded in a web page? Is that why there is no copyright header?

scottpurdy · 2017-04-14T00:44:06Z

docs/examples/util/check_predictions.py

+if netResults.error.mean() != algResults.error.mean():
+  warnings.warn('Algorithms and Network API predictions are different.')
+
+plt.show()


missing newline at eof

scottpurdy · 2017-04-14T00:45:32Z

docs/examples/util/run-all-complete-examples.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash


What's this for? It would be nice to have this be a test that just validates there are no exceptions in the examples.

Agreed.

I'll add a unit test to make sure the 3 examples can run without throwing an exception.

I'll add the other test you are talking about as part of: https://jira.numenta.com/browse/NUP-2401 (Fix results discrepancy between OPF, network API and algorithm API)

I'll take out the scripts under docs/util.

scottpurdy · 2017-04-14T00:45:43Z

docs/examples/util/check_predictions.py

@@ -0,0 +1,84 @@
+import os


What's this for?

A script to inspect the prediction results of the 3 examples:

docs/algo/complete-example.py

docs/opf/complete-example.py

docs/networkapi/complete-example.py

Adding this info at the top of the file.

* Add unit test to make sure all 3 examples don't thow any exception * Change single quotes to double quotes everywhere * Remove utlity script to plot saved prediction results from all 3 examples * Remove part where examples save predictions to file * Rename networkapi to network for better readability

marionleborgne · 2017-04-17T20:22:13Z

@scottpurdy thanks for the review, I addressed the feedback. Some things to note:

I added a unit test (as you suggested) to make sure the 3 examples can run and don't throw any exception. This does not check if the prediction results are correct.
I will add a unit test for the prediction results consistency between network API, OPF and algo API as part of https://jira.numenta.com/browse/NUP-2401.

…e sniptets

* Make the example code easier to follow in the quick-start section.

oxtopus · 2017-04-19T20:32:53Z

docs/examples/algo/complete-example.py

 import numpy
+import os
+import yaml


why yaml? i.e. what benefit does it have over a native python module?

In all 3 examples we are using a single YAML file with all the params. This YAML is heavily commented to explain each param. Using YAML vs native python module makes it a bit more obvious in the docs that it's a configuration file. Another (small) benefit is that it displays in a more compact manner in the docs and it's easier to read than the native python module.

Python model params adds the extra requirement of a local import, an extra directory, and an empty __init__.py file.

rhyolight · 2017-04-24T16:52:37Z

@marionleborgne Will review today

rhyolight · 2017-04-24T17:48:56Z

docs/examples/algo/complete-example.py

    # Random number generator seed.
-    seed=1956,
+    seed=spParams["seed"],
+    # TODO: is this useful?


Probably not. Feel free to remove.

@marionleborgne If you choose not to make the change above, let me know so I can merge as-is.

I went ahead and merged. Will revisit.

Marion Le Borgne added 3 commits April 4, 2017 16:58

NUP-2394: use YAML file for model params

80bb517

NUP-2394: network API quick-start example (WIP)

54a9d10

NUP-2394: run inference outside network

69841ff

Marion Le Borgne added 5 commits April 5, 2017 12:02

NUP-2394: save and check all complete-examples.py predictions

6956e0c

Merge branch 'master' of https://github.com/numenta/nupic into NUP-23…

956f569

…94-network-api # Conflicts: # docs/examples/opf/model_params.py

NUP-2394: use YAML params in "algo" code example

fb95987

NUP-2394: update comments of YAML params based on feedback

dc3cdbf

NUP-2394: scripts to compare predictions results between the 3 code e…

a771b84

…xamples (OPF, network API and algo)

marionleborgne force-pushed the NUP-2394-network-api branch from 6a93c41 to a771b84 Compare April 6, 2017 19:01

marionleborgne mentioned this pull request Apr 8, 2017

NUP-2396: Allow SensorRegion to pass actValue and bucketIdx to SDRClassifierRegion #3526

Merged

Marion Le Borgne added 2 commits April 7, 2017 19:08

NUP-2394: Run classification inside network. Details:

93160cf

* Use new network links created in NUP-2396 * Remove customCompute() usage

NUP-2394: Show RMSE in plot titles

4ce9edc

scottpurdy reviewed Apr 10, 2017

View reviewed changes

Marion Le Borgne added 2 commits April 13, 2017 17:28

Merge branch 'master' of https://github.com/numenta/nupic into NUP-23…

9c77a66

…94-network-api

Merge branch 'master' of https://github.com/numenta/nupic into NUP-23…

21d82b1

…94-network-api # Conflicts: # docs/examples/opf/model_params.py

scottpurdy reviewed Apr 17, 2017

View reviewed changes

Marion Le Borgne added 2 commits April 17, 2017 13:18

NUP-2394: Fix YAML with new CLA model name (HTMPrediction)

dc72590

Marion Le Borgne added 3 commits April 18, 2017 13:16

NUP-2394: make model_params camel case for consistency and update cod…

fac2380

…e sniptets

NUP-2394: re-order network creation logic:

3c205af

* Make the example code easier to follow in the quick-start section.

NUP-2394: fix indentation

d7e8593

oxtopus reviewed Apr 19, 2017

View reviewed changes

marionleborgne mentioned this pull request Apr 20, 2017

NUP-2405: quick-start guide for the Network API #3557

Merged

rhyolight approved these changes Apr 24, 2017

View reviewed changes

rhyolight merged commit 92fb883 into numenta:master Apr 24, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NUP-2394: network API code example #3520

NUP-2394: network API code example #3520

marionleborgne commented Apr 5, 2017 •

edited

Loading

rhyolight commented Apr 5, 2017

marionleborgne commented Apr 5, 2017

marionleborgne commented Apr 8, 2017

rhyolight commented Apr 10, 2017

marionleborgne commented Apr 10, 2017

rhyolight commented Apr 10, 2017 via email

marionleborgne commented Apr 10, 2017

scottpurdy Apr 10, 2017

rhyolight Apr 10, 2017 •

edited

Loading

scottpurdy Apr 10, 2017

marionleborgne commented Apr 17, 2017

scottpurdy left a comment

scottpurdy Apr 14, 2017

scottpurdy Apr 14, 2017

scottpurdy Apr 14, 2017

marionleborgne Apr 17, 2017

scottpurdy Apr 14, 2017

marionleborgne Apr 17, 2017

scottpurdy Apr 14, 2017

scottpurdy Apr 14, 2017

marionleborgne Apr 17, 2017 •

edited

Loading

scottpurdy Apr 14, 2017

marionleborgne Apr 17, 2017

marionleborgne commented Apr 17, 2017

oxtopus Apr 19, 2017

marionleborgne Apr 19, 2017

rhyolight Apr 24, 2017

rhyolight commented Apr 24, 2017

rhyolight Apr 24, 2017

rhyolight Apr 24, 2017

rhyolight Apr 24, 2017

		@@ -0,0 +1,317 @@
		# Type of model that the rest of these parameters apply to.

NUP-2394: network API code example #3520

NUP-2394: network API code example #3520

Conversation

marionleborgne commented Apr 5, 2017 • edited Loading

rhyolight commented Apr 5, 2017

marionleborgne commented Apr 5, 2017

marionleborgne commented Apr 8, 2017

rhyolight commented Apr 10, 2017

marionleborgne commented Apr 10, 2017

rhyolight commented Apr 10, 2017 via email

marionleborgne commented Apr 10, 2017

Choose a reason for hiding this comment

rhyolight Apr 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marionleborgne commented Apr 17, 2017

scottpurdy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marionleborgne Apr 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marionleborgne commented Apr 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhyolight commented Apr 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marionleborgne commented Apr 5, 2017 •

edited

Loading

rhyolight Apr 10, 2017 •

edited

Loading

marionleborgne Apr 17, 2017 •

edited

Loading