Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dependency from $NUPIC environment variable. Fixes numenta/nupic#1815, numenta/nupic#1947 #1970

Closed
wants to merge 2 commits into from
Closed

Remove dependency from $NUPIC environment variable. Fixes numenta/nupic#1815, numenta/nupic#1947 #1970

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
636c63a
Remove dependency from $NUPIC environment variable
pehlert Mar 30, 2015
a206231
Fixed improper reference to NUPIC_ROOT in configuration unit tests
pehlert Mar 30, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion examples/opf/experiments/multistep/base/permutations_simple_3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
'~/nupic/eng/lib/python2.6/site-packages/nupic/frameworks/opf/expGenerator/ExpGenerator.pyc'
"""

from nupic import NUPIC_ROOT
from nupic.swarming.permutationhelpers import *

# The name of the field being predicted. Any allowed permutation MUST contain
Expand Down Expand Up @@ -58,7 +59,7 @@
'predictedField': 'field2',
'predictionSteps': [1,3],

'dataSource': 'file://%s' % (os.path.join(os.environ['NUPIC'], 'examples', 'opf',
'dataSource': 'file://%s' % (os.path.join(NUPIC_ROOT, 'examples', 'opf',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

examples is in the root of the repo which is a level above NUPIC_ROOT. And pkg_resources would probably be better here but up to you

'experiments', 'multistep', 'datasets', 'simple_3.csv')),

'__field2_encoder' : PermuteEncoder(fieldName='field2',
Expand Down
4 changes: 4 additions & 0 deletions nupic/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,7 @@
# http://numenta.org/licenses/
# ----------------------------------------------------------------------
__version__ = "0.3.0.dev0"

import os
NUPIC_ROOT = os.environ.get('NUPIC', os.path.dirname(os.path.realpath(__file__)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried get the absolute path of __file__ and so discard $NUPIC completely?

Copy link
Member

@breznak breznak Mar 30, 2015 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi David, my change will consider $NUPIC if it is set or get the path from __file__ as you suggested. I didn't remove $NUPIC altogether because I wanted to avoid side-effects. If you guys are sure nobody relies on it, I'm more than happy to remove it for good. 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait what Matthew thinks ;) keeping it might be better for sidemodules or community projects that may rely on it. There are a lot and Matthew probably has the best overview. How about just a try-except implementation? 

-------- Original message --------
From: Pascal Ehlert notifications@github.com
Date:03/30/2015 15:52 (GMT+01:00)
To: numenta/nupic nupic@noreply.github.com
Cc: Pascal Weinberger passiweinberger@gmail.com
Subject: Re: [nupic] Remove dependency from $NUPIC environment variable. Fixes #1815, #1947 (#1970)
In nupic/**init**.py:

@@ -19,3 +19,7 @@

http://numenta.org/licenses/

----------------------------------------------------------------------

version = "0.3.0.dev0"
+
+import os
+NUPIC_ROOT = os.environ.get('NUPIC', os.path.dirname(os.path.realpath(file)))
Hi David, my change will consider $NUPIC if it is set or consider the path from file as you suggested. I didn't remove $NUPIC altogether because I wanted to avoid side-effects. If you guys are sure nobody relies on it, I'm more than happy to remove it for good.

—
Reply to this email directly or view it on GitHub.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You guys are right. We could keep $NUPIC as extra (but not required) variable. So this script should get the value of $NUPIC and in case of it is not found, get the absolute path of this __file__..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this is clear, but that's what it does right now. os.environ.get('NUPIC', 'foo') will return either the value of $NUPIC (if it's set) or 'foo'. See https://docs.python.org/2/library/stdtypes.html#dict.get for reference.

So $NUPIC ist completely optional.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I'll stay with that solution?


3 changes: 2 additions & 1 deletion nupic/support/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
from pkg_resources import resource_string, resource_filename

from configuration import Configuration
from nupic import NUPIC_ROOT
from nupic.support.fshelpers import makeDirectoryFromAbsolutePath


Expand Down Expand Up @@ -409,7 +410,7 @@ def initLogging(verbose=False, console='stdout', consoleLevel='DEBUG'):
# If NTA_LOG_DIR is not defined, set it now. This is used by the logging
# config file to set the path for the log files
if 'NTA_LOG_DIR' not in os.environ:
os.environ['NTA_LOG_DIR'] = os.path.join(os.environ['NUPIC'], 'logs')
os.environ['NTA_LOG_DIR'] = os.path.join(NUPIC_ROOT, '../logs')
if not os.path.exists(os.environ['NTA_LOG_DIR']):
makeDirectoryFromAbsolutePath(os.path.abspath(os.environ['NTA_LOG_DIR']))

Expand Down
7 changes: 4 additions & 3 deletions tests/integration/nupic/opf/expgenerator_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
from pkg_resources import resource_filename
import unittest2 as unittest

from nupic import NUPIC_ROOT
from nupic.data import dictutils
from nupic.database.ClientJobsDAO import ClientJobsDAO
from nupic.support import aggregationDivide
Expand Down Expand Up @@ -118,7 +119,7 @@ def setUp(self):
global g_myEnv
if not g_myEnv:
# Setup environment
params = type('obj', (object,), {'installDir' : os.environ['NUPIC']})
params = type('obj', (object,), {'installDir' : NUPIC_ROOT})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.environ['NUPIC'] was a directory above NUPIC_ROOT

g_myEnv = MyTestEnvironment(params)


Expand Down Expand Up @@ -1947,7 +1948,7 @@ def _getTestList():
# Our custom options (that don't get passed to unittest):
customOptions = ['--installDir', '--verbosity', '--logLevel']
parser.add_option(
"--installDir", dest="installDir", default=os.environ['NUPIC'],
"--installDir", dest="installDir", default=NUPIC_ROOT,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, change in the location

help="Path to the NTA install directory [default: %default].")

parser.add_option("--verbosity", default=0, type="int",
Expand All @@ -1961,7 +1962,7 @@ def _getTestList():

# The following are put here to document what is accepted by the unittest
# module - we don't actually use them in this code bas.
parser.add_option("--verbose", dest="verbose", default=os.environ['NUPIC'],
parser.add_option("--verbose", dest="verbose", default=NUPIC_ROOT,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, change in the location

help="Verbose output")
parser.add_option("--quiet", dest="quiet", default=None,
help="Minimal output")
Expand Down
7 changes: 3 additions & 4 deletions tests/integration/nupic/opf/opf_description_template_test/opf_description_template_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import sys
import unittest2 as unittest

from nupic import NUPIC_ROOT
from nupic.frameworks.opf.opfhelpers import (
loadExperimentDescriptionScriptFromDir,
getExperimentDescriptionInterfaceFromModule
Expand All @@ -47,9 +48,7 @@ class MyTestEnvironment(object):

def __init__(self):

nupicDir = os.environ['NUPIC']

examplesDir = os.path.join(nupicDir, "examples")
examplesDir = os.path.join(NUPIC_ROOT, "examples")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, change in the location


_debugOut("examplesDir=<%s>" % (examplesDir,))

Expand All @@ -58,7 +57,7 @@ def __init__(self):

# This is where we find OPF binaries (e.g., run_opf_experiment.py, etc.)
# In the autobuild, it is a read-only directory
self.__opfBinDir = os.path.join(nupicDir, "scripts")
self.__opfBinDir = os.path.join(NUPIC_ROOT, "scripts")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, change in the location

assert os.path.exists(self.__opfBinDir), \
"%s is not present in filesystem" % self.__opfBinDir
_debugOut("self.__opfBinDir=<%s>" % self.__opfBinDir)
Expand Down
7 changes: 3 additions & 4 deletions tests/integration/nupic/opf/opf_experiment_results_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import time
import unittest2 as unittest

from nupic import NUPIC_ROOT
from nupic.data.file_record_stream import FileRecordStream


Expand All @@ -55,9 +56,7 @@ def testExperimentResults(self):
examples/prediction directory in the install tree (same as predictionDir)

"""
nupicDir = os.environ['NUPIC']

opfDir = os.path.join(nupicDir, "examples", "opf")
opfDir = os.path.join(NUPIC_ROOT, "examples", "opf")

testDir = opfDir

Expand Down Expand Up @@ -95,7 +94,7 @@ def testExperimentResults(self):
# Run from the test directory so that we can find our experiments
os.chdir(testDir)

runExperiment = os.path.join(nupicDir, 'scripts', 'run_opf_experiment.py')
runExperiment = os.path.join(NUPIC_ROOT, 'scripts', 'run_opf_experiment.py')

# A list of experiments to run. Valid attributes:
# experimentDir - Required, path to the experiment directory containing
Expand Down
3 changes: 2 additions & 1 deletion tests/integration/nupic/opf/opf_experiments_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,15 @@
import traceback
import unittest2 as unittest

from nupic import NUPIC_ROOT
from nupic.frameworks.opf.experiment_runner import (
runExperiment, initExperimentPrng)



# Globals
EXCLUDED_EXPERIMENTS = [] # none for now
PREDICTION_DIR = os.path.join(os.environ['NUPIC'], 'examples', 'opf')
PREDICTION_DIR = os.path.join(NUPIC_ROOT, 'examples', 'opf')
RUN_ALL_ITERATIONS = False


Expand Down
3 changes: 2 additions & 1 deletion tests/swarming/nupic/swarming/swarming_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@

from optparse import OptionParser

from nupic import NUPIC_ROOT
from nupic.database.ClientJobsDAO import ClientJobsDAO
from nupic.support import configuration, initLogging
from nupic.support.unittesthelpers.testcasebase import (unittest,
Expand Down Expand Up @@ -255,7 +256,7 @@ def _generateHSJobParams(self,

# Form the stream definition
if dataPath is None:
dataPath = os.path.join(os.environ['NUPIC'], 'examples', 'prediction',
dataPath = os.path.join(NUPIC_ROOT, 'examples', 'prediction',
'data', 'extra', 'qa', "hotgym", "qa_hotgym.csv")
streamDef = dict(
version = 1,
Expand Down
7 changes: 3 additions & 4 deletions tests/unit/nupic/support/configuration_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,10 @@
from xml.etree.ElementTree import ParseError
except ImportError:
from xml.parsers.expat import ExpatError as ParseError
import nupic

import nupic.support.configuration_base as configuration

from nupic import NUPIC_ROOT

import nupic.support.configuration_base as configuration

class ConfigurationTest(unittest.TestCase):

Expand Down Expand Up @@ -519,7 +518,7 @@ def testGetConfigPathsForNone(
configuration.Configuration._configPaths = None # pylint: disable=W0212
result = configuration.Configuration.getConfigPaths()
self.assertTrue(isinstance(result, list))
self.assertEqual(result, [os.path.join(os.environ['NUPIC'],
self.assertEqual(result, [os.path.join(NUPIC_ROOT,
'config', 'default')])

@patch.object(configuration.Configuration, '_configPaths',
Expand Down
4 changes: 2 additions & 2 deletions tests/unit/nupic/support/custom_configuration_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
except ImportError:
from xml.parsers.expat import ExpatError as ParseError

import nupic
from nupic import NUPIC_ROOT

import nupic.support.configuration_custom as configuration

Expand Down Expand Up @@ -803,7 +803,7 @@ def testGetConfigPathsForNone(
configuration.Configuration._configPaths = None # pylint: disable=W0212
result = configuration.Configuration.getConfigPaths()
self.assertTrue(isinstance(result, list))
self.assertEqual(result, [os.path.join(os.environ['NUPIC'],
self.assertEqual(result, [os.path.join(NUPIC_ROOT,
'config', 'default')])

@patch.object(configuration.Configuration, '_configPaths',
Expand Down