Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to run in windows (wip) #9

Merged
merged 1 commit into from
Mar 6, 2019
Merged

Conversation

davalapar
Copy link
Contributor

@davalapar davalapar commented Mar 4, 2019

Fix for #1, still a few problems though:

  • Figure out causes
    • Warnings C4244, C4309 etc
    • /benchmark/benchmark.js just exits / calling simdjson.parse(something) just exits node process, no warnings / error
C:\simdjson_nodejs>yarn
yarn install v1.13.0-20181107.1254
warning package-lock.json found. Your project contains lock files generated by t
ools other than Yarn. It is advised not to mix package managers in order to avoi
d resolution inconsistencies caused by unsynchronized lock files. To clear this
warning, remove package-lock.json.
[1/4] Resolving packages...
success Already up-to-date.
$ node-gyp rebuild
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@10.10.0 | win32 | x64
gyp info spawn C:\Python27\python.exe
gyp info spawn args [ 'C:\\simdjson_nodejs\\node_modules\\node-gyp\\gyp\\gyp_mai
n.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-G',
gyp info spawn args   'msvs_version=2015',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\simdjson_nodejs\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\simdjson_nodejs\\node_modules\\node-gyp\\addon.gypi',

gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\1234\\.node-gyp\\10.10.0\\include\\node\\commo
n.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\1234\\.node-gyp\\10.10.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\simdjson_nodejs\\node_modules\\node-gy
p',
gyp info spawn args   '-Dnode_lib_file=C:\\Users\\1234\\.node-gyp\\10.10.0\\<(ta
rget_arch)\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=C:\\simdjson_nodejs',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'C:\\simdjson_nodejs\\build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MS
Build\15.0\Bin\MSBuild.exe
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args   '/clp:Verbosity=minimal',
gyp info spawn args   '/nologo',
gyp info spawn args   '/p:Configuration=Release;Platform=x64' ]
Building the projects in this solution one at a time. To enable parallel build,
please add the "/m" switch.
  main.cpp
  nonavx2.cpp
  win_delay_load_hook.cc
     Creating library C:\simdjson_nodejs\build\Release\simdjson.lib and object
  C:\simdjson_nodejs\build\Release\simdjson.exp
  Generating code
  All 79 functions were compiled because no usable IPDB/IOBJ from previous comp
  ilation was found.
  Finished generating code
  simdjson.vcxproj -> C:\simdjson_nodejs\build\Release\\simdjson.node
  main.cpp
  simdjson.cpp
  bindings.cpp
  win_delay_load_hook.cc
c:\simdjson_nodejs\simdjson\src\simdjson.h(42): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxp
roj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(46): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxp
roj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(42): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(46): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(42): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(46): warning C4244: 'return': conver
sion from 'unsigned __int64' to 'int', possible loss of data (compiling source
file ..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35592): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35644): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35646): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35649): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35651): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35677): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35691): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\src\simdjson.cpp
) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36473): warning C4244: 'initializing
': conversion from 'int64_t' to 'int', possible loss of data (compiling source
file ..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxp
roj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36557): warning C4146: unary minus o
perator applied to unsigned type, result still unsigned (compiling source file
..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36652): warning C4244: 'initializing
': conversion from '__int64' to 'int', possible loss of data (compiling source
file ..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxp
roj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36721): warning C4244: 'initializing
': conversion from 'int64_t' to 'double', possible loss of data (compiling sour
ce file ..\simdjson\src\simdjson.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.v
cxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(163): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(246): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(258): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35592): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35644): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35646): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35649): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35651): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35677): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35691): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\main.cpp) [C:\si
mdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35592): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35644): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35646): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35649): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35651): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35677): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(35691): warning C4309: 'argument': t
runcation of constant value (compiling source file ..\simdjson\bindings.cpp) [C
:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36473): warning C4244: 'initializing
': conversion from 'int64_t' to 'int', possible loss of data (compiling source
file ..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36557): warning C4146: unary minus o
perator applied to unsigned type, result still unsigned (compiling source file
..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36652): warning C4244: 'initializing
': conversion from '__int64' to 'int', possible loss of data (compiling source
file ..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36721): warning C4244: 'initializing
': conversion from 'int64_t' to 'double', possible loss of data (compiling sour
ce file ..\simdjson\main.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36473): warning C4244: 'initializing
': conversion from 'int64_t' to 'int', possible loss of data (compiling source
file ..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36557): warning C4146: unary minus o
perator applied to unsigned type, result still unsigned (compiling source file
..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36652): warning C4244: 'initializing
': conversion from '__int64' to 'int', possible loss of data (compiling source
file ..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.h(36721): warning C4244: 'initializing
': conversion from 'int64_t' to 'double', possible loss of data (compiling sour
ce file ..\simdjson\bindings.cpp) [C:\simdjson_nodejs\build\simdjson-avx2.vcxpr
oj]
c:\simdjson_nodejs\simdjson\bindings.cpp(72): warning C4244: 'argument': conver
sion from 'int64_t' to 'double', possible loss of data [C:\simdjson_nodejs\buil
d\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(439): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(494): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(617): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(672): warning C4309: 'argument': t
runcation of constant value [C:\simdjson_nodejs\build\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(850): warning C4244: '=': conversi
on from 'uint64_t' to 'uint32_t', possible loss of data [C:\simdjson_nodejs\bui
ld\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(862): warning C4244: '=': conversi
on from 'uint64_t' to 'uint32_t', possible loss of data [C:\simdjson_nodejs\bui
ld\simdjson-avx2.vcxproj]
c:\simdjson_nodejs\simdjson\src\simdjson.cpp(874): warning C4244: '=': conversi
on from 'uint64_t' to 'uint32_t', possible loss of data [C:\simdjson_nodejs\bui
ld\simdjson-avx2.vcxproj]
     Creating library C:\simdjson_nodejs\build\Release\simdjson-avx2.lib and ob
  ject C:\simdjson_nodejs\build\Release\simdjson-avx2.exp
  Generating code
  All 539 functions were compiled because no usable IPDB/IOBJ from previous com
  pilation was found.
  Finished generating code
  simdjson-avx2.vcxproj -> C:\simdjson_nodejs\build\Release\\simdjson-avx2.node
gyp info ok
Done in 8.35s.

@davalapar
Copy link
Contributor Author

davalapar commented Mar 5, 2019

Hmmm so far the benchmarks won't run and testing it and running parse function just abruptly exits it without any error / warning.

Do you got ideas on what could be causing this? I've checked a couple issues in node-gyp repo and it seems that on warnings like these (shown in pr) they are telling people that it's not a problem in node-gyp ¯\_(ツ)_/¯

C:\simdjson_nodejs>node
> x = require("./index")
{ hasAVX2: [Function], isValid: [Function], parse: [Function] }
> x
{ hasAVX2: [Function], isValid: [Function], parse: [Function] }
> x.parse('{ "a": 1}');

C:\simdjson_nodejs>

@luizperes
Copy link
Owner

Wow, that is a very weird one. What does console.log show? Also, could you do a try-catch just in case?
I've never seen it before.

@davalapar
Copy link
Contributor Author

davalapar commented Mar 6, 2019

So yeah, it turns out that the main issue here (despite my recently updated Visual Studio Build Tools 2017 for latest node-gyp) is that my workstation processor (AMD A8 6XXX) does not support AVX2 instructions.

We've discovered this during a teamviewer session where we tried installing other npm modules that also uses AVX2 and noticed that all their build phases during installation are failing.

Long story short, the following must be in check:

Thanks @luizperes for your time and support!

@luizperes
Copy link
Owner

Hi @davalapar, as per #1, I will merge this PR! Thanks!

@luizperes luizperes merged commit d58f5b2 into luizperes:master Mar 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants