Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python -m spacy.en.download no longer valid? #45

Open
scevola44 opened this issue Nov 14, 2017 · 28 comments
Open

python -m spacy.en.download no longer valid? #45

scevola44 opened this issue Nov 14, 2017 · 28 comments

Comments

@scevola44
Copy link

Running the line of code no longer download the spaCy's models, because there are now three different ones: en_core_web_sm, en_core_web_md, en_core_web_lg, en_vectors_web_lg.
Which one (if any) is the correct one to download during setup, and should the setup file be changed at line 125 in order to follow the change?

@kengz
Copy link
Owner

kengz commented Nov 15, 2017

Hi, yep spaCy moved to v2.0.
Since wordvec is needed, download the with this new command python -m spacy download en_core_web_md.

If it works, could you please submit a pull request with the changes in the setup file?

@scevola44
Copy link
Author

scevola44 commented Nov 15, 2017

I see. I'll try to get it working, although in the file ./aiva/node_modules/spacy-nlp/src/py/nlp.py in line 2 where it says from spacy.en import English I'm unsure what changes I should make (I'm not an expert of python)

Does this mean though that if a language doesn't have the "_md" module it won't be possible to use the bot with that specific language?

@scevola44
Copy link
Author

Also, even changing all files where the outdated command was used (6, IIRC), I'm still getting an error when launching npm start --debug :

aiva@4.2.0 start /home/scevola/aiva
bin/start

[ ------- Starting on local machine, not Docker ------- ]

aiva@4.2.0 development /home/scevola/aiva
mkdir -p ./logs; NODE_ENV=development ./node_modules/forever/bin/forever --minUptime=1000 --spinSleepTime=1000 --killSignal=SIGTERM --uid aiva-development -m 10 -a index.js | tee ./logs/aiva-development.log

[Wed Nov 15 2017 09:14:09 GMT+0000 (utc)] ERROR
[Wed Nov 15 2017 09:14:09 GMT+0000 (utc)] ERROR
error: Forever detected script exited with code: 0
error: Script restart attempt #1

And then the script restarts 10 times until it fails. In the log file (production.log, since development.log is empty) I have this:

[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Authenticated database successfully
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Starting aiva process
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Starting poly-socketio server on port: 6466, expecting 4 IO clients
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for js at 6466
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for python at 6466
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Deploying bot aivadev with adapter SLACK
[Tue Nov 14 2017 16:42:06 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for python at 6466
[Tue Nov 14 2017 16:42:07 GMT+0000 (utc)] �[32mINFO�[39m js 1WGMS9oIUx-nmeGsAAAA joined, 3 remains
[Tue Nov 14 2017 16:42:07 GMT+0000 (utc)] ERROR Invalid service token provided, please follow the upgrade instructions

@kengz
Copy link
Owner

kengz commented Nov 15, 2017

oh that's no longer a spacy issue, it's now a token issue with the config file. What services are you using? (slack bot, ngrok, fb messenger)?

@scevola44
Copy link
Author

I'm using slack, following your blog's advice. You think the issue might be with the bot's token?

@kengz
Copy link
Owner

kengz commented Nov 15, 2017

ahh, unlikely it's Slack. On the config file default.json under ADAPTERS, could you delete the keys for services not used (so leave just Slack).

@scevola44
Copy link
Author

I get the same output

@kengz
Copy link
Owner

kengz commented Nov 15, 2017

did some digging, seems like an error towards your slack token. See the source of error method
It throws when ur token does not begin with the 'xoxb-', 'xoxp-' strings. Check if your slack token is correct and the latest?

@kengz
Copy link
Owner

kengz commented Nov 15, 2017

gotta run will look again evening if it's still erroring, thanks for ur patience

@scevola44
Copy link
Author

scevola44 commented Nov 15, 2017

My token starts with 'xoxb-272[...]', so it would seem that it shouldn't raise that error.
I even tried "regenerating" the token on Slack and updating it in default.json, but the result is the same

@scevola44
Copy link
Author

Okay, I managed to "move on", and get new error messages. I did a clean reset (recloned the aiva folder and -hopefully- fixed most of the python code related to the outdated linguistic models), and now the stdout is:

npm start --debug

> aiva@4.2.0 start /home/scevola/aiva
> bin/start

[ ------- Starting on local machine, not Docker ------- ]

> aiva@4.2.0 development /home/scevola/aiva
> mkdir -p ./logs; NODE_ENV=development ./node_modules/forever/bin/forever --minUptime=1000 --spinSleepTime=1000 --killSignal=SIGTERM --uid aiva-development -m 10 -a index.js | tee ./logs/aiva-development.log

[Thu Nov 16 2017 08:05:12 GMT+0000 (utc)] INFO Authenticated database successfully
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG 
Sequelize [Node: 8.9.1, CLI: 2.8.0, ORM: 3.30.4]

Loaded configuration file "config/db.json".
Using environment "development".
No migrations were executed, database schema was already up to date.

[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Starting aiva process
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG globalConfig {
  "BOTNAME": "labibot",
  "PORTS": {
    "NEO4J": 7476,
    "SOCKETIO": 6466,
    "SLACK": 8345,
    "TELEGRAM": 8443,
    "FB": 8545
  },
  "NGROK_AUTH": null,
  "ADMINS": [
    "alberto.bandini.93@gmail.com"
  ],
  "TEST": {
    "HUBOT_SHELL_USER_ID": "ID0000001",
    "HUBOT_SHELL_USER_NAME": "alice"
  }
}
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Starting poly-socketio server on port: 6466, expecting 4 IO clients
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG adapterConfig {
  "ACTIVATE": true,
  "HUBOT_SLACK_TOKEN": "xoxb-272773609590-91TJwfHhbMnDHAcUvTmirULH"
}
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG import js lib from client.js
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Starting socketIO client for js at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Starting socketIO client for python at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG Set SLACK PORT: 8345
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] DEBUG No WEBHOOK set for adapter SLACK
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Deploying bot labibot with adapter SLACK
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO Starting socketIO client for python at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] INFO js 0zyAW_VV2mx93iq4AAAA joined, 3 remains

[Thu Nov 16 2017 08:05:14 GMT+0000 (utc)] ERROR Traceback (most recent call last):
File "/home/scevola/aiva/node_modules/spacy-nlp/src/client.py", line 14, in

[Thu Nov 16 2017 08:05:14 GMT+0000 (utc)] ERROR from py import *
  File "/home/scevola/aiva/node_modules/spacy-nlp/src/py/nlp.py", line 2, in <module>
    from spacy import en_core_web_md  # NLP with spaCy https://spacy.io
ImportError: cannot import name en_core_web_md

[Thu Nov 16 2017 08:05:14 GMT+0000 (utc)] ERROR Traceback (most recent call last):
  File "/home/scevola/aiva/lib/client.py", line 14, in <module>
    from py import *

File "/home/scevola/aiva/lib/py/convo_classifier.py", line 5, in
from autocorrect import spell
ImportError: No module named autocorrect

[Thu Nov 16 2017 08:05:15 GMT+0000 (utc)] INFO Logged in as labibot of LabiTech
[Thu Nov 16 2017 08:05:18 GMT+0000 (utc)] INFO Slack client now connected
[Thu Nov 16 2017 08:05:18 GMT+0000 (utc)] DEBUG Started global js socketIO client for SLACK at 6466
[Thu Nov 16 2017 08:05:18 GMT+0000 (utc)] INFO global-client-js OncOSpTy8AyIcbU1AAAB joined, 2 remains
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] ERROR {}
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] ERROR Expected number of IO clients failed to join in time
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] INFO Shutting down
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] INFO Exit: killed ioClient.js children
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] INFO Exit: killed ioClient.js children
error: Forever detected script exited with code: 1
error: Script restart attempt #1

I highlighted the ones I think are the 2 main errors (hope it doesn't look too ugly). The first still has to do with spaCy's module. I didn't know how to fix line 2 of client.py and improvised something, but it doesn't seem to work.
As for the second error, I do have autocorrect installed, and the spell module too. It's all under the python3 folder, so I'm uncertain as to why it doesn't find it.

This is the content of aiva-development.log(only log file):

[Thu Nov 16 2017 08:05:12 GMT+0000 (utc)] �[32mINFO�[39m Authenticated database successfully
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Starting aiva process
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Starting poly-socketio server on port: 6466, expecting 4 IO clients
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for js at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for python at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Deploying bot labibot with adapter SLACK
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m Starting socketIO client for python at 6466
[Thu Nov 16 2017 08:05:13 GMT+0000 (utc)] �[32mINFO�[39m js 0zyAW_VV2mx93iq4AAAA joined, 3 remains
[Thu Nov 16 2017 08:05:15 GMT+0000 (utc)] INFO Logged in as labibot of LabiTech
[Thu Nov 16 2017 08:05:18 GMT+0000 (utc)] INFO Slack client now connected
[Thu Nov 16 2017 08:05:18 GMT+0000 (utc)] �[32mINFO�[39m global-client-js OncOSpTy8AyIcbU1AAAB joined, 2 remains
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] �[32mINFO�[39m Shutting down
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] �[32mINFO�[39m Exit: killed ioClient.js children
[Thu Nov 16 2017 08:06:53 GMT+0000 (utc)] �[32mINFO�[39m Exit: killed ioClient.js children

@scevola44
Copy link
Author

scevola44 commented Nov 16, 2017

So, I "solved" the first problem. in /home/scevola/aiva/node_modules/spacy-nlp/src/py/nlp.py line 2 and 3 should be

import spacy
nlp = spacy.load('en_core_web_md')

However, in this way it looks for the spacy package in python 2.7.x (default package for Ubuntu 16.04, at least), and not in 3.x. Not knowing much of Python coding, I simply downloaded spacy's language models into Python 2.7.x too, and now that ERROR is gone.

EDIT:
I tried the same procedure for the ImportError: No module named autocorrectand it works!
The bot's brain launched, but now I have a different issue, that I believe might be cause by the "mixed" imports from 2.7 and 3.x.
image

I tried the same commands you gave in your test example, but I don't get a "saved to brain" message, and in the terminal I get

[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG IO on pass, msg: {
  "hash": "global-client-js_8ccb1e14f9eb28e852fc9d929d8bd8c0",
  "from": "global-client-js",
  "input": "Bob brought the pizza to Alice.",
  "to": "convo_classifier.py",
  "intent": "classify"
} fn: undefined
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG IO on pass, msg: {
  "output": null,
  "from": "cgkb-py",
  "hash": "global-client-js_7def1ed0c267abf4bed2b368aa8bd522",
  "to": "global-client-js"
} fn: undefined
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG IO on pass, msg: {
  "output": null,
  "from": "py",
  "hash": "global-client-js_8ccb1e14f9eb28e852fc9d929d8bd8c0",
  "to": "global-client-js"
} fn: undefined
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hasher.handle invoking cb for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hasher.handle invoking cb for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] ERROR 
[Thu Nov 16 2017 09:56:33 GMT+0000 (utc)] INFO Knowledge saved to brain
[Thu Nov 16 2017 09:56:35 GMT+0000 (utc)] DEBUG [In log]: labibot Bob brought the pizza to Alice.

but no ERROR is present in development.log

This apparently happens also when giving it the sample command for booking a flight. Instead of the json output shown in the blog, I get this as an output

[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG Started global js socketIO client for SLACK at 6466
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG Added a callback to hasher for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] INFO global-client-js CRLj1zBnJoZWRRgpAAAB joined, -8 remains
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] INFO All 4 IO clients have joined
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG [Out log]: Your wish is my command
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG IO on pass, msg: {
"hash": "global-client-js_26e4da2159e48091e432980ad167244a",
"from": "global-client-js",
"input": "Book a flight from New York to London for Sunday",
"to": "nlp.cgkb-py",
"intent": "parse"
} fn: undefined
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG IO on pass, msg: {
"output": null,
"from": "cgkb-py",
"hash": "global-client-js_26e4da2159e48091e432980ad167244a",
"to": "global-client-js"
} fn: undefined
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG Added a callback to hasher for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG IO on pass, msg: {
"hash": "global-client-js_fce40c4ddcee0dad5825e419d4d71f8d",
"from": "global-client-js",
"input": "Book a flight from New York to London for Sunday",
"to": "convo_classifier.py",
"intent": "classify"
} fn: undefined
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hasher.handle invoking cb for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG IO on pass, msg: {
"output": null,
"from": "py",
"hash": "global-client-js_fce40c4ddcee0dad5825e419d4d71f8d",
"to": "global-client-js"
} fn: undefined
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hasher.handle invoking cb for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] DEBUG hash string exists for global-client
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] ERROR
[Thu Nov 16 2017 10:08:30 GMT+0000 (utc)] INFO Knowledge saved to brain

Also, the neo4j interface doesn't show me the graph
image

@scevola44
Copy link
Author

PS: I also got a funny response after restarting the bot
image
And in the terminal it labeled my request as topic: weather

@kengz
Copy link
Owner

kengz commented Nov 16, 2017

great to see u made progress :)
The error would be from the socket failing to post to python, the process needs start in python3 (cuz of some encoding error in the socket library in python2).
So the problem is 2 fold:

  1. the socket connection needs to happen from Slack -> Nodejs -> sent over SocketIO to python3 -> processed in spacy -> sent back the parsed results to Nodejs -> Node records on Neo4j and replies to user. The last 2 steps is where it fails and hence an empty neo4j db. This is indicated by the output: null error message.
    Recommended solution: start and embed ur runtime in python3 with the installs, use something like Conda:
if which conda >/dev/null; then
  echo "Conda is already installed"
else
  curl -O https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
  bash Miniconda3-latest-Linux-x86_64.sh -b -u
fi
conda create -n aiva python=3.6 -c conda-forge -y
# every time when u use it, active the python3 environment in your terminal session:
source activate aiva
# now u can run the installations and AIVA commands from.
pip install -r requirements.txt
# ... install spacy and things u did earlier
# to deactivate, just do this from within the environment
source deactivate

Shd fix problem 1.

  1. Problem 2 is with the weird response. So, Your wish is my command is the immediate reply triggered in the script when the input to spacy is received. Once done processing from the above chain (thru spacy and neo4j), it passes a output: {some response} instead of output: null, which is picked up and sent to Slack. Since in your terminal it was output: null, nothing is sent after.
    The No idea, I live in the cloud and it's always sunny ;) is from some other script that got triggered, likely default Hubot script.

Let me know how this goes!

@scevola44
Copy link
Author

I'm unsure at what point I should run the code you sent. I tried running it in the terminal, but line 7 gives an error
conda: command not found
but it seemed to have installed conda when I gave the previous commands

@kengz
Copy link
Owner

kengz commented Nov 16, 2017

oh just start a new terminal session to install Conda, it's from this. You're on linux right?
Try to install conda first with:

curl -O https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Then open a new bash session and check conda --version. Should return a number

@scevola44
Copy link
Author

scevola44 commented Nov 16, 2017

Yes, I'm on Linux, Ubuntu 16.04 :)
Should I also re-run the setup file for aiva? Cause I repeated the install process for the dependencies then did npm start --debug, but the result seems to be the same

@kengz
Copy link
Owner

kengz commented Nov 17, 2017

rerunning the setup file is fine, it only updates on existing ones.

The bottom of this issue had similar encounter with socketIO, might be worthwhile to look.

Meanwhile let's debug it in a hacky way.

  1. start a shell session, do source activate aiva, and run npm run server. This is to start a server independently with the python services.
  2. start another shell session, do npm start --debug, then do the slack chat steps

show the logs from both windows pls?

@scevola44
Copy link
Author

I am now getting an error at step 1. In terminal it shows (sending only the last lines)

[Fri Nov 17 2017 08:11:33 GMT+0000 (utc)] INFO py gG5ltQwSSk8i5NNKAAAC joined, 1 remains
[Fri Nov 17 2017 08:12:52 GMT+0000 (utc)] ERROR {}
[Fri Nov 17 2017 08:12:52 GMT+0000 (utc)] ERROR Expected number of IO clients failed to join in time
[Fri Nov 17 2017 08:12:52 GMT+0000 (utc)] INFO Exit: killed ioClient.js children
[Fri Nov 17 2017 08:12:52 GMT+0000 (utc)] INFO Exit: killed ioClient.js children
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! aiva@4.2.0 server: node src/start-io.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the aiva@4.2.0 server script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/scevola/.npm/_logs/2017-11-17T08_12_52_059Z-debug.log
(aiva) scevola@UbuScevola:~/aiva$ npm start --debug

And in the npm log file

0 info it worked if it ends with ok
1 verbose cli [ '/home/scevola/.nvm/versions/node/v8.9.1/bin/node',
1 verbose cli '/home/scevola/.nvm/versions/node/v8.9.1/bin/npm',
1 verbose cli 'run',
1 verbose cli 'server' ]
2 info using npm@5.5.1
3 info using node@v8.9.1
4 verbose run-script [ 'preserver', 'server', 'postserver' ]
5 info lifecycle aiva@4.2.0preserver: aiva@4.2.0
6 info lifecycle aiva@4.2.0
server: aiva@4.2.0
7 verbose lifecycle aiva@4.2.0server: unsafe-perm in lifecycle true
8 verbose lifecycle aiva@4.2.0
server: PATH: /home/scevola/.nvm/versions/node/v8.9.1/lib/node_modules/npm/bin/node-gyp-bin:/home/scevola/aiva/node_modules/.bin:/home/scevola/miniconda3/envs/aiva/bin:/home/scevola/miniconda3/bin:/home/scevola/.nvm/versions/node/v8.9.1/bin:/home/scevola/bin:/home/scevola/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
9 verbose lifecycle aiva@4.2.0server: CWD: /home/scevola/aiva
10 silly lifecycle aiva@4.2.0
server: Args: [ '-c', 'node src/start-io.js' ]
11 silly lifecycle aiva@4.2.0server: Returned: code: 1 signal: null
12 info lifecycle aiva@4.2.0
server: Failed to exec server script
13 verbose stack Error: aiva@4.2.0 server: node src/start-io.js
13 verbose stack Exit status 1
13 verbose stack at EventEmitter. (/home/scevola/.nvm/versions/node/v8.9.1/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:280:16)
13 verbose stack at emitTwo (events.js:126:13)
13 verbose stack at EventEmitter.emit (events.js:214:7)
13 verbose stack at ChildProcess. (/home/scevola/.nvm/versions/node/v8.9.1/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at emitTwo (events.js:126:13)
13 verbose stack at ChildProcess.emit (events.js:214:7)
13 verbose stack at maybeClose (internal/child_process.js:925:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
14 verbose pkgid aiva@4.2.0
15 verbose cwd /home/scevola/aiva
16 verbose Linux 4.4.0-98-generic
17 verbose argv "/home/scevola/.nvm/versions/node/v8.9.1/bin/node" "/home/scevola/.nvm/versions/node/v8.9.1/bin/npm" "run" "server"
18 verbose node v8.9.1
19 verbose npm v5.5.1
20 error code ELIFECYCLE
21 error errno 1
22 error aiva@4.2.0 server: node src/start-io.js
22 error Exit status 1
23 error Failed at the aiva@4.2.0 server script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

@scevola44
Copy link
Author

scevola44 commented Nov 17, 2017

Also, I checked out issue #22 and (to the best of my skills, not being familiar with the python environment), I excluded python from default.json, ran npm start --debug, then opened a new command window, ran client.py, and tested out aiva.
In the first terminal I get the usual errors upon sending a message to the bot

[Fri Nov 17 2017 08:34:17 GMT+0000 (utc)] INFO Knowledge saved to brain
global-client-js promise exception with input: undefined, to: {"input":"ping","to":"convo_classifier.py","intent":"classify"}
[Fri Nov 17 2017 08:34:25 GMT+0000 (utc)] ERROR
global-client-js promise exception with input: undefined, to: {"input":"Book a flight from New York to London for Sunday","to":"convo_classifier.py","intent":"classify"}
[Fri Nov 17 2017 08:34:37 GMT+0000 (utc)] ERROR

In the second one I get this

py handle fails. <class 'TypeError'>

Every time I send a message to the bot

@scevola44
Copy link
Author

Any new ideas/suggestions?

@kengz
Copy link
Owner

kengz commented Jan 15, 2018

could u update spacy-nlp to v1.0.8? There's a recent patch in the logger to print thee error log from python better. Would help debugging this

@ryanblakeley
Copy link

ryanblakeley commented Mar 27, 2018

I think I'm bumping into a related bug. I cloned the repo, followed the setup without deviating. npm start fails with

[Tue Mar 27 2018 19:56:48 GMT+0000 (UTC)] INFO Authenticated database successfully
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO Starting aiva process
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO Starting poly-socketio server on port: 6466, expecting 4 IO clients
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO Starting socketIO client for js at 6466
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO Starting socketIO client for python at 6466
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO Starting socketIO client for python at 6466
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] INFO js 6IgMkV_zfVKoYnlqAAAA joined, 3 remains
[Tue Mar 27 2018 19:56:50 GMT+0000 (UTC)] ERROR Traceback (most recent call last):
  File "/Users/ryanblakeley/Code/aiva/node_modules/spacy-nlp/src/client.py", line 12, in <module>
    import websocket
ImportError: No module named websocket

@ryanblakeley
Copy link

Is it supposed to be websockets with an s at the end?

@ryanblakeley
Copy link

ryanblakeley commented Mar 27, 2018

python3 -m pip show websocket

looks like websocket is installed

If I open a python shell with python3 and then input import websocket that also seems to work.
Can't figure out why in client.py this same command is throwing the error. But maybe this is just by naivety with python and not a problem with aiva

@kengz
Copy link
Owner

kengz commented Mar 28, 2018

yeah it's websocket. Have u tried installing it in a virtualenv and running it in it too?

@ryanblakeley
Copy link

I have not tried that. I'm pretty inexperienced with python, so this is probably just an issue with my system, rather than a bug with the project. I haven't done anything with virtualenv. But no worries about sorting this one out. I'm working with Rasa atm.

@theTechie
Copy link

@kengz - looks like i have a similar problem. I did follow the instructions mentioned above to update scripts as per spaCy 2.0. and use custom conda env But still getting this error

TypeError: Cannot read property 'output' of undefined

at this place

(node:15046) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'output' of undefined
(node:15046) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

global-client-js promise exception with input: undefined*, to: {"input":"Hi","to":"convo_classifier.py","intent":"classify"}
[Tue Apr 03 2018 17:33:24 GMT+0000 (UTC)] ERROR
[Tue Apr 03 2018 17:33:31 GMT+0000 (UTC)] ERROR operation timed out```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants