Unable to install from source on Windows

[flothesof]

Hi,
I'm trying to install from source with latest devel.
However, it doesn't work.

(tofu_dev) C:\Users\FL232714\Documents\GitHub\tofu>pip install -e ".[dev]"
Obtaining file:///C:/Users/FL232714/Documents/GitHub/tofu
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Anaconda3\envs\tofu_dev\python.exe' 'C:\Anaconda3\envs\tofu_dev\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\FL232714\AppData\Local\Temp\tmprlzaqcsh'
       cwd: C:\Users\FL232714\Documents\GitHub\tofu
  Complete output (16 lines):
  Traceback (most recent call last):
    File "C:\Anaconda3\envs\tofu_dev\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 207, in <module>
      main()
    File "C:\Anaconda3\envs\tofu_dev\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Anaconda3\envs\tofu_dev\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 54, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\FL232714\AppData\Local\Temp\pip-build-env-dzkqyuzc\overlay\Lib\site-packages\setuptools\build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\FL232714\AppData\Local\Temp\pip-build-env-dzkqyuzc\overlay\Lib\site-packages\setuptools\build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "C:\Users\FL232714\AppData\Local\Temp\pip-build-env-dzkqyuzc\overlay\Lib\site-packages\setuptools\build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 21, in <module>
      import _updateversion as up
  ModuleNotFoundError: No module named '_updateversion'
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\Anaconda3\envs\tofu_dev\python.exe' 'C:\Anaconda3\envs\tofu_dev\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\FL232714\AppData\Local\Temp\tmprlzaqcsh' Check the logs fo
r full command output.

I'm a bit confused, this used to work.

Since the _updateversion.py is just a single function, I'm going to put it inside the setup.py and see if it fixes this.

[lasofivec]

"from source" means you pulled the git repo right ?
what version of tofu is this ?
what version of pip?

and silly question: if you do a "ls" do you see the file ?

[flothesof]

from source" means you pulled the git repo right ? yes, I started with SHA 6988b9f
what version of tofu is this ? '1.4.2a3-2-gfaeb7af'
what version of pip? pip 19.2.2
and silly question: if you do a "ls" do you see the file ? yes, it is there

[Didou09]

I just tried on linux this morning installing the new alpha4 from source with pip, and I got the same error as @flothesof

(empty) [ DV226270 spica ~ ] pip install tofu==1.4.2a4
Collecting tofu==1.4.2a4
  Downloading https://files.pythonhosted.org/packages/cd/36/9f98c7ebf4d03f120aa113574f6f45342aef6d3245fcb6be3e64d2cdac2e/tofu-1.4.2a4.tar.gz (35.6MB)
     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 35.6MB 12.7MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Home/DV226270/.conda/envs/empty/bin/python /Home/DV226270/.conda/envs/empty/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpiza0pryo
       cwd: /tmp/pip-install-ihp04ea_/tofu
  Complete output (16 lines):
  Traceback (most recent call last):
    File "/Home/DV226270/.conda/envs/empty/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
      main()
    File "/Home/DV226270/.conda/envs/empty/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Home/DV226270/.conda/envs/empty/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 91, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-mwzc1vpm/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-mwzc1vpm/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-mwzc1vpm/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 21, in <module>
      import _updateversion as up
  ModuleNotFoundError: No module named '_updateversion'
  ----------------------------------------
ERROR: Command errored out with exit status 1: /Home/DV226270/.conda/envs/empty/bin/python /Home/DV226270/.conda/envs/empty/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpiza0pryo Check the logs for full command output.

Very strange, it looks like the _updateversion.py module is not included in the pip sdist, so it is not found.

Solutions:

• Understand why it is not included anymore (it was before)
• and / or do as suggested by @flothesof and include the function updateversion() directly into the setup.py (but we wont be able to run it as standalone anymore).

@lasofivec, is there something changed to the sdist in your opinion ?
Did you try installing with pip recently ?

[Didou09]

I think it is because we forgot to specify that _updateversion.py should be included in the sdist:
https://docs.python.org/3.7/distutils/sourcedist.html

From what I understand, we can include it via the setup options and / or via a MANIFEST.in

[Didou09]

But, I just created a sdist locally:

python setup.py sdist --dist-dir .

and unzipped the tar.gz to see its content:
tar -xvzf tofu-1.4.2a4.tar.gz

The _updateversion.py is already in there...

And it is also in the tar.gz that was uploaded to pipy...

I don't understand why it is not found and pip install time. Any idea ?

[Didou09]

From the error message I seem to understand that _updateversion.py shall be included in the build-dependencies, because it fails at

Getting requirements to build wheel ... error

This is logical, because we import _updateversion.py at the top of setup.py.

Suggested hotfix given the previous discussions, time emergency:
=> copy updateversion() into setup.py as suggested by @flothesof
=> also keep _updateversion.py as standalone for use when necessary

We'll need to discuss later how to design a more elegant solution, but it should do for the time being
Agreed ?

[Didou09]

Tested, solves the problem, see PR #299

[lasofivec]

the thing is I am not able to recreate the error...

I am able to recreate the error

[Didou09]

Are you sure you tried with the alpha4 ?
And from a folder from which _updateversion was absent ?

[Didou09]

Fixed in devel by PR #299

[flothesof]

Just tested this with devel from commit 2d9b183, it now works for me.

Closing.

[Didou09]

See 'Notes for the future' in Issue #295

[lasofivec]

I don't think the solution we took here was the right one. I understood a little better why _updateversion.py is no longer packaged. There a couple of issues explaining this better, eg:

pypa/pip#6375

And

pypa/setuptools#1698 (comment)

"How the semantics of a setuptools.build_meta differ from directly invoking setup.py; at the moment, the only deliberate difference is that with setuptools.build_meta, the source root is not on sys.path when you invoke it, meaning that you should either stop importing things from the local path, or add sys.path.insert(0, os.dirname(file)) to the top of your setup.py file."

So there are two solutions, for both it would mean to take out the version of the function in the setup.py:

• adding sys.path.insert(0, os.dirname(__file__)) to the top of your setup.py file
• taking out build-backend = "setuptools.build_meta" in pyproject.toml

Since the documentation on this is still very light, I would vote for the latter.