Hints to build with Win10 / VS2019 / Python 3.7 #28

incensesmoke · 2019-11-27T23:09:31Z

For a Win10/VS2019 build with a Python3.7 wrapper and SWIG4.0.1 I had to make the following changes in the current master BTK-core:

Support building for Python 3 #26 Support building for Python 3
Visual Studio 2017 warning about std::tr1 #23 Avoid tr1 with VS2017: 2b7a9a8
Moved the *.tpp includes from the .h to the .cpp. #16 Moved the *.tpp includes from the .h to the .cpp.
Fix the problem with using BTK in other libraries (like OpenSim) on Windows with MSVC. #12 Fix the problem with using BTK in other libraries (like OpenSim) on Windows with MSVC.
Adapt ...\BTKCore-master\Utilities\eigen3\Eigen\src\Core\VectorwiseOp.h on line 585 to include the following:
#ifdef EIGEN_VECTOROP_PLUGIN
#include EIGEN_VECTOROP_PLUGIN
#endif

Perhaps this can save somebody else some trouble. As a n00b this took me a long time to figure out ;) Now it passes all tests and I'm happy!

Edit: spelling

The text was updated successfully, but these errors were encountered:

jimmyDunne · 2019-11-27T23:17:55Z

May want to try https://github.com/pyomeca/ezc3d

incensesmoke · 2019-11-27T23:30:54Z

Thanks for the link! I'll look into it

melund · 2019-12-02T12:10:42Z

@incensesmoke Could you push the changes you did to your fork of BTK?

I would really love to get conda-forge to build conda packages of BTK for Python 3.

Enable Python 3 build Moved the *.tpp includes from the .h to the .cpp. Fix the problem with using BTK in other libraries (like OpenSim) on Windows. Problem -- Including the BTK headers causes an error on Windows with VS2015. " .... cstdint: 'uint8_t' is not a member of global namespace .... ". The error appears to be in the standard library but is actually caused by the fact that Utilities/stdint.h is included for MSVC builds and the directory Utilities is in the include path for the compiler. When trying to search for a header named stdint.h, the compiler keeps running into Utilities/stdint.h even when the original intention was to find the header in the system standard path. Solution -- Rename the file Utilities/stdint.h to something else that does not conflict with standard library header of the same name therefore avoiding compiler choosing this header when standard library header is meant to be found. Fix RPATH for OSX. Fix from Biomechanical-ToolKit#28 Remove xrange

Biomechanical-ToolKit#28

Enable Python 3 build Moved the *.tpp includes from the .h to the .cpp. Fix the problem with using BTK in other libraries (like OpenSim) on Windows. Problem -- Including the BTK headers causes an error on Windows with VS2015. " .... cstdint: 'uint8_t' is not a member of global namespace .... ". The error appears to be in the standard library but is actually caused by the fact that Utilities/stdint.h is included for MSVC builds and the directory Utilities is in the include path for the compiler. When trying to search for a header named stdint.h, the compiler keeps running into Utilities/stdint.h even when the original intention was to find the header in the system standard path. Solution -- Rename the file Utilities/stdint.h to something else that does not conflict with standard library header of the same name therefore avoiding compiler choosing this header when standard library header is meant to be found. Fix RPATH for OSX. Fix from Biomechanical-ToolKit#28 Remove xrange Avoid tr1 with VS2017 Biomechanical-ToolKit#28

melund · 2020-01-09T13:53:41Z

I have created a conda package for BTK which supports Python 3. It can be installed with:

conda install -c conda-forge btk

It currently has support for Windows. If anyone would help getting Linux/mac working that would great. I don't think it will be hard, I just haven't had the opportunity to try other compilers.

The build recipe is here:
https://github.com/conda-forge/btk-feedstock

I hope this will serve for anyone who needs BTK for older projects but can't use Python 2.7. I will not recommend it for new projects since BTK development has stopped.

Thanks a lot to:
@allardhoeve, @klshrinidhi, @chrisdembia, @aroy-hc
for all your fixes and commits which help get BTK Python 3 support.

also thanks to:
@incensesmoke, @CameronJGrant, @aaa34169, @jjnurminen, @demotu
Your posts and comments different places on Github helped me a lot to get the build working.

aaa34169 · 2020-01-09T16:10:45Z

Hi Morten Sound good. Just one question : have you run all native implemented unit tests ? regards Fabien Le jeu. 9 janv. 2020 à 14:53, Morten Enemark Lund <notifications@github.com> a écrit :

…

I have create a conda package for BTK which supports Python 3. It can be installed with: conda install -c conda-forge btk It currently has support for Windows. If anyone would help getting Linux/mac working that would great. I don't think it will be hard, I just haven't had the opportunity to try other compilers. The build recipe is here: https://github.com/conda-forge/btk-feedstock I hope this will serve for anyone who needs BTK for older projects but can't use Python 2.7. I will not recommend it for new projects since BTK development has stopped. Thanks a lot to: @allardhoeve <https://github.com/allardhoeve>, @klshrinidhi <https://github.com/klshrinidhi>, @chrisdembia <https://github.com/chrisdembia>, @aroy-hc <https://github.com/aroy-hc> for all your fixes and commits which help get BTK Python 3 support. also thanks to: @incensesmoke <https://github.com/incensesmoke>, @CameronJGrant <https://github.com/CameronJGrant>, @aaa34169 <https://github.com/aaa34169>, @jjnurminen <https://github.com/jjnurminen>, @demotu <https://github.com/demotu> Your posts and comments different places on Github helped me a lot to get the build working. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#28?email_source=notifications&email_token=AAXUMBLTMSIQWAZ5LCJQJQTQ44T6PA5CNFSM4JSNGRI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIQL3NY#issuecomment-572571063>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXUMBLITOWVTZRSYLAJQGTQ44T6PANCNFSM4JSNGRIQ> .

-- Fabien

---------------------------------------------------------------------------------------------- Fabien Leboeuf Ingénieur "analyste du mouvement" du Pole 10, CHU Nantes, France Chercheur associé de l'Université de Salford, Manchester, Royaume uni Laboraratoire d'analyse du mouvement 85 rue saint Jacques 44093 Nantes, FRANCE ------------------------------------------------------------------------------------------------

melund · 2020-01-09T18:36:28Z

Good question and the answer is yes. The conda build recipe is setup to do that as part of the build. The build can only succeed if all the tests pass.

I had to adapt the test runner a little bit to handle the setup on the CI server used by conda forge. You can see it here:
https://github.com/conda-forge/btk-feedstock/blob/master/recipe/run_test.py

aaa34169 · 2020-01-09T19:20:03Z

well done. i have not seen you used circle-ci. Envoyé de mon iPhone

…

Le 9 janv. 2020 à 19:36, Morten Enemark Lund ***@***.***> a écrit : Good question and the answer is yes. The conda build recipe is setup to do that as part of the build. The build can only succeed if all the tests pass. I had to adapt the test runner a little bit to handle the setup on the CI server used by conda forge. You can see it here: https://github.com/conda-forge/btk-feedstock/blob/master/recipe/run_test.py — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

incensesmoke · 2020-01-09T19:25:23Z

Thanks @melund for doing the work I gave up on! Is it a lot of effort to also make a PyPi package? Perhaps to merge/augment @aaa34169's contribution there?

melund · 2020-01-09T21:21:07Z

Is it a lot of effort to also make a PyPi package?

I actually have no idea. I guess it is possible these days using wheels. But I think packages like this is exactly what conda is for. Distributing binaries with the python and real tracking of dependencies.

jjnurminen · 2020-01-10T10:28:38Z

@melund Thanks, this is great! Installs and works without a hitch on Win10 64-bit/Python 3.7.

jjnurminen · 2020-01-13T14:57:39Z

@melund Now that you established the build procedure, would it be possible to also build a 2.7 version of btk into conda-forge with small effort? I know this may sound silly, but I still need to maintain 2.7/3.7 versions of my package for some time. It would be great to be able to get the same btk conda-forge dependency regardless of whether Python 2.7 or 3.7 is used.

melund · 2020-01-13T15:26:49Z

I don't know if conda forge still supports building 2.7 packages.

You can try to make a PR to the recipe
https://github.com/conda-forge/btk-feedstock/blob/master/recipe/meta.yaml

where you change the 3.6 restriction. The CI system should then build a 2.7 version to check the PR. If 2.7 is supported that is :)

jjnurminen · 2020-01-13T15:38:44Z

@melund I tried, let's see what happens. Was not 100% sure about the syntax of the yaml file...

melund · 2020-01-13T20:06:54Z

It doesn't seem to do anything. I don't think conda forge builds for 2.7 any more.

jjnurminen · 2020-01-14T09:14:40Z

It doesn't seem to do anything. I don't think conda forge builds for 2.7 any more.

Ok, thanks. On quick googling, I couldn't find any mention of conda-forge dropping 2.7 though. Maybe I'll try again later.

melund · 2020-01-14T11:45:51Z

@jjnurminen Wait. I think the automatic CI generation removes the 2.7 build scripts when the recipe excludes python 2.7. I tried to re-render the recipe and now 2.7 is available:
conda-forge/btk-feedstock#3

We will see if also works. Maybe the Python 3 patch needs to be removed for 2.7

melund · 2020-01-14T11:47:01Z

It is running here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=110936&view=logs&j=171a126d-c574-5c8c-1269-ff3b989e923d

jjnurminen · 2020-01-14T12:43:45Z

Looks like the build failed, cannot find numpy?

Enable Python 3 build Moved the *.tpp includes from the .h to the .cpp. Fix the problem with using BTK in other libraries (like OpenSim) on Windows. Problem -- Including the BTK headers causes an error on Windows with VS2015. " .... cstdint: 'uint8_t' is not a member of global namespace .... ". The error appears to be in the standard library but is actually caused by the fact that Utilities/stdint.h is included for MSVC builds and the directory Utilities is in the include path for the compiler. When trying to search for a header named stdint.h, the compiler keeps running into Utilities/stdint.h even when the original intention was to find the header in the system standard path. Solution -- Rename the file Utilities/stdint.h to something else that does not conflict with standard library header of the same name therefore avoiding compiler choosing this header when standard library header is meant to be found. Fix RPATH for OSX. Fix from Biomechanical-ToolKit#28 Remove xrange Avoid tr1 with VS2017 Biomechanical-ToolKit#28 Fix problem with test Fixes a problem with unittest.assertAlmostEqual not working on numpy ndarrays. Reporting that ndarray doesn't have a __round__ methods. Fix test error

jjnurminen · 2020-02-14T14:43:32Z

Hi @melund and others

I ran into an issue.

With Python 2.7, I used to convert strings to latin1, which btk ate happily:

c3dfile = 'some_file.c3d'
reader = btk.btkAcquisitionFileReader()
c3dfile = c3dfile.encode('latin-1')
reader.SetFilename(c3dfile)

With Python 3, the SWIG interface apparently eats Python 3 string type directly. So the above would simply read:

c3dfile = 'some_file.c3d'
reader = btk.btkAcquisitionFileReader()
reader.SetFilename(c3dfile)

Still good. However, using non-ASCII chars (which are rather common in Finnish):

c3dfile = 'söme_file.c3d'
reader = btk.btkAcquisitionFileReader()
reader.SetFilename(c3dfile)

 RuntimeError: File doesn't exist

even if the file actually exists. I tried encoding to UTF-8, but that results in

TypeError: in method 'btkAcquisitionFileReader_SetFilename', argument 2 of type 'std::string const &'

Any idea how we could get correct support for non-ASCII filenames? Maybe some compile-time flag for the SWIG interface?

melund · 2020-02-17T09:46:34Z

I don't think you need to do any encoding since Python 3 uses utf-8. This works out of the box for me:

jjnurminen · 2020-02-17T13:27:56Z

That's weird, I verified that I get a RuntimeError for any filename with non-ASCII characters. I suppose you're also on a Windows 10 machine?

melund · 2020-02-17T17:17:14Z

Does it help if you put # -*- coding: utf-8 -*-, as the first line of your Python files?

jjnurminen · 2020-02-19T09:49:08Z

@melund no, it doesn't seem to help. But strangely, the above code works just fine on another Windows 10 machine. So I suspect this particular Windows box has some special locale settings or whatever that screws things up. Anyway, thank you.

aaa34169 · 2020-08-07T13:26:13Z

@melund

Hello, morten, i try to recompile your btk code ( branch tagged python3 ) . Unfortunaly i ran into an issue when the compiler try to wrap the code in python.

my configuration is :

windows 10
cmake 3.18.1
swig 4.0.2
here is how i configure cmake :

at [98% ] Linking CXX shared module to ....\bin_btk.pyd
i have this errror message :
Error LNK2001 symbole externe non résolu "public: virtual short __cdecl btk::ByteOrderBinaryFileStream<class btk::VAXLittleEndianFormat>::ReadI16(void)" (?ReadI16@?$ByteOrderBinaryFileStream@VVAXLittleEndianFormat@btk@@@btk@@UEAAFXZ) _btk C:\Users\fleboeuf\Documents\Programmation\BTK\BUILD btkCore melund\build-python3_0.4.dev0-4-sln\Wrapping\Python\BTKIO.lib(btkC3DFileIO.obj)

repeated for other method of btkC3DFileIO.obj

I am not an expert in c++ compilation and appreciate any help. maybe @chrisdembia, share with us his expertise

regards

Fabien

melund · 2020-08-07T21:20:58Z

@aaa34169 Did you see how I build it here. Note, the patch I apply. But I guess you get that implicitly by building of the python 3 branch. Though I can't recall it
.

You can try to clone the repository from conda-forge
https://github.com/conda-forge/btk-feedstock
and then run

conda build recipe

It requires that you have a mini-conda installation, the conda-build package installed, a working c compiler (visual studio). But then the script should install the necessary dependencies (cmake,swig, jom) and run the build. Maybe that will help you figure out what is wrong with your build setup.

Why do you want to build it your self?

aaa34169 · 2020-08-08T08:09:49Z

thanks morten I want to figure out what is wrong with c3d file containing a non-ascii character ( for example : "staticé.c3d") I installed your package and if you test by your own you will see that btkFileReader doesn t find the file ! It s weird. In python 3, management of unicode should be moire simple. There is something wrong with windows machine only . (that work fine on linux). I suspect that come from swig and mapping of : const std::string& @jussi Nurminen <jjnurminen@gmail.com> : from your previous email, you apparently managed to work with finnish accent, didn t you ! any secret? Le ven. 7 août 2020 à 23:21, Morten Enemark Lund <notifications@github.com> a écrit :

…

@aaa34169 <https://github.com/aaa34169> Did you see how I build it here. Note, the patch I apply. But I guess you get that implicitly by building of the python 3 branch. Though I can't recall it . You can try to clone the repository from conda-forge https://github.com/conda-forge/btk-feedstock and then run conda build recipe It requires that you have a mini-conda installation, the conda-build package installed, a working c compiler (visual studio). But then the script should install the necessary dependencies (cmake,swig, jom) and run the build. Maybe that will help you figure out what is wrong with your build setup. Why do you want to build it your self? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXUMBPMAFX6Q4IPUTDVMZDR7RV4TANCNFSM4JSNGRIQ> .

-- Fabien

---------------------------------------------------------------------------------------------- Fabien Leboeuf Ingénieur "analyste du mouvement" du Pole 10, CHU Nantes, France Chercheur associé de l'Université de Salford, Manchester, Royaume uni Laboraratoire d'analyse du mouvement 85 rue saint Jacques 44093 Nantes, FRANCE ------------------------------------------------------------------------------------------------

jjnurminen · 2020-08-17T17:08:57Z

@aaa34169 it's probably related to Windows locale or language settings. I'll try to figure it out.

melund added a commit to melund/BTKCore that referenced this issue Dec 3, 2019

Fix from Biomechanical-ToolKit#28

7b993f6

melund added a commit to melund/BTKCore that referenced this issue Dec 11, 2019

Avoid tr1 with VS2017

531de6e

Biomechanical-ToolKit#28

CameronJGrant mentioned this issue Jan 21, 2020

Python3 support Biomechanical-ToolKit/BTKPython#2

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hints to build with Win10 / VS2019 / Python 3.7 #28

Hints to build with Win10 / VS2019 / Python 3.7 #28

incensesmoke commented Nov 27, 2019 •

edited

Loading

jimmyDunne commented Nov 27, 2019

incensesmoke commented Nov 27, 2019

melund commented Dec 2, 2019

melund commented Jan 9, 2020 •

edited

Loading

aaa34169 commented Jan 9, 2020 via email

melund commented Jan 9, 2020

aaa34169 commented Jan 9, 2020 via email

incensesmoke commented Jan 9, 2020

melund commented Jan 9, 2020

jjnurminen commented Jan 10, 2020

jjnurminen commented Jan 13, 2020 •

edited

Loading

melund commented Jan 13, 2020

jjnurminen commented Jan 13, 2020

melund commented Jan 13, 2020

jjnurminen commented Jan 14, 2020

melund commented Jan 14, 2020

melund commented Jan 14, 2020

jjnurminen commented Jan 14, 2020

jjnurminen commented Feb 14, 2020

melund commented Feb 17, 2020

jjnurminen commented Feb 17, 2020

melund commented Feb 17, 2020

jjnurminen commented Feb 19, 2020

aaa34169 commented Aug 7, 2020

melund commented Aug 7, 2020

aaa34169 commented Aug 8, 2020 via email

jjnurminen commented Aug 17, 2020

Hints to build with Win10 / VS2019 / Python 3.7 #28

Hints to build with Win10 / VS2019 / Python 3.7 #28

Comments

incensesmoke commented Nov 27, 2019 • edited Loading

jimmyDunne commented Nov 27, 2019

incensesmoke commented Nov 27, 2019

melund commented Dec 2, 2019

melund commented Jan 9, 2020 • edited Loading

aaa34169 commented Jan 9, 2020 via email

melund commented Jan 9, 2020

aaa34169 commented Jan 9, 2020 via email

incensesmoke commented Jan 9, 2020

melund commented Jan 9, 2020

jjnurminen commented Jan 10, 2020

jjnurminen commented Jan 13, 2020 • edited Loading

melund commented Jan 13, 2020

jjnurminen commented Jan 13, 2020

melund commented Jan 13, 2020

jjnurminen commented Jan 14, 2020

melund commented Jan 14, 2020

melund commented Jan 14, 2020

jjnurminen commented Jan 14, 2020

jjnurminen commented Feb 14, 2020

melund commented Feb 17, 2020

jjnurminen commented Feb 17, 2020

melund commented Feb 17, 2020

jjnurminen commented Feb 19, 2020

aaa34169 commented Aug 7, 2020

melund commented Aug 7, 2020

aaa34169 commented Aug 8, 2020 via email

jjnurminen commented Aug 17, 2020

incensesmoke commented Nov 27, 2019 •

edited

Loading

melund commented Jan 9, 2020 •

edited

Loading

jjnurminen commented Jan 13, 2020 •

edited

Loading