[package] cpython/all: Can't be built using Visual Studio Build Tools

[bog-dan-ro]

Package and Environment Details

• Package Name/Version: cpython/*
• Operating System+version: Windows 11 in a docker container
• Compiler+version: msvc 2022(17)

Conan profile

Configuration for profile default:

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=Visual Studio
compiler.version=17
build_type=Release
[options]
[conf]
[build_requires]
[env]

Steps to reproduce

devenv is not available on Visual Studio Build Tools and the build fails.
Is _upgrade_single_project_file step really necessary?

Logs

No response

[AtnNn]

I have the same issue. Setting CONAN_SKIP_VS_PROJECTS_UPGRADE=True does not help. The conanfile uses upgrade_project=False and runs devenv /upgrade directly, regardless of the visual studio version. There is a comment which says:

devenv /upgrade <project.vcxproj> will upgrade ALL projects referenced by the project.
By temporarily moving the solution project, only one project is upgraded
This is needed for static cpython or for disabled optional dependencies (e.g. tkinter=False)
Restore it afterwards because it is needed to build some targets.

[uilianries]

Cpython recipe does not enforce project upgrade:

conan-center-index/recipes/cpython/all/conanfile.py

Line 406 in b60e3d7

msbuild.build(project_file, upgrade_project=False, build_type="Debug" if self.settings.build_type == "Debug" else "Release",

Should it be something configured in the project itself?

Usually we recommend people installing Visual Studio Community edition.

The environment variable CONAN_SKIP_VS_PROJECTS_UPGRADE should work.

[bog-dan-ro]

Cpython recipe does not enforce project upgrade:

conan-center-index/recipes/cpython/all/conanfile.py

Line 406 in b60e3d7

msbuild.build(project_file, upgrade_project=False, build_type="Debug" if self.settings.build_type == "Debug" else "Release",

Should it be something configured in the project itself?

Usually we recommend people installing Visual Studio Community edition.

The environment variable CONAN_SKIP_VS_PROJECTS_UPGRADE should work.

But here
https://github.com/conan-io/conan-center-index/blob/master/recipes/cpython/all/conanfile.py#L313
it calls devenv unconditionally.

[bog-dan-ro]

strange, after I set CONAN_SKIP_VS_PROJECTS_UPGRADE I have another error:

PS C:\> conan install --build=missing -o shared=True -o with_tkinter=False -pr:b=default  cpython/3.9.7@
...
openssl/1.1.1l: Already installed!
sqlite3/3.36.0: Already installed!
sqlite3/3.36.0: Appending PATH env var with : C:\.conan\65d112\1\bin
xz_utils/5.2.5: Already installed!
zlib/1.2.11: Already installed!
cpython/3.9.7: WARN: Build folder is dirty, removing it: C:\.conan\d3e983\1
cpython/3.9.7: Copying sources to build folder
cpython/3.9.7: Building your package in C:\.conan\d3e983\1
cpython/3.9.7: Generator txt created conanbuildinfo.txt
cpython/3.9.7: Generator 'MSBuildDeps' calling 'generate()'
cpython/3.9.7: ERROR: Traceback (most recent call last):
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\site-packages\conans\client\generators\__init__.py", line 181, in write_generators
    generator.generate()
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\site-packages\conan\tools\microsoft\msbuilddeps.py", line 110, in generate
    generator_files = self._content()
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\site-packages\conan\tools\microsoft\msbuilddeps.py", line 333, in _content
    result.update(self._package_props_files(dep, build=False))
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\site-packages\conan\tools\microsoft\msbuilddeps.py", line 303, in _package_props_files
    result[comp_filename] = self._dep_props_file(full_comp_name, comp_filename,
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\site-packages\conan\tools\microsoft\msbuilddeps.py", line 229, in _dep_props_file
    dom = minidom.parseString(content_multi)
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\xml\dom\minidom.py", line 1998, in parseString
    return expatbuilder.parseString(string)
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\xml\dom\expatbuilder.py", line 925, in parseString
    return builder.parseString(string)
  File "C:\Users\ContainerAdministrator\scoop\apps\python\current\lib\xml\dom\expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 6, column 33

ERROR: Error in generator 'MSBuildDeps': not well-formed (invalid token): line 6, column 33

PS C:\>

[bog-dan-ro]

I found why, the previous error was because I used shared=True...

Building it statically, it fails when running devenv unconditionally.

PS C:\>  conan install --build=missing -o shared=False -o with_tkinter=False -pr:b=default  cpython/3.9.7@
...
Installing (downloading, building) binaries...
zlib/1.2.11: Already installed!
cpython/3.9.7: Copying sources to build folder
cpython/3.9.7: Building your package in C:\.conan\a45956\1
cpython/3.9.7: Generator txt created conanbuildinfo.txt
cpython/3.9.7: Generator 'MSBuildDeps' calling 'generate()'
cpython/3.9.7: Aggregating env generators
cpython/3.9.7: Calling build()
cpython/3.9.7: Patching runtime
cpython/3.9.7: Building 3 Visual Studio projects: ('pythoncore', 'python', 'pythonw')
cpython/3.9.7: [1/3] Building project 'pythoncore'...
Conan:vcvars already set
'devenv' is not recognized as an internal or external command,
operable program or batch file.
cpython/3.9.7:
cpython/3.9.7: ERROR: Package 'cdb6fd543ab5677fd09e2c988f468d700c5445bc' build failed
cpython/3.9.7: WARN: Build folder C:\.conan\a45956\1
ERROR: cpython/3.9.7: Error in build() method, line 425
        self._msvc_build()
while calling '_msvc_build', line 405
        self._upgrade_single_project_file(project_file)
while calling '_upgrade_single_project_file', line 325
        self.run("devenv \"{}\" /upgrade".format(project_file), run_environment=True)
        ConanException: Error 1 while executing devenv "source_subfolder\PCbuild\pythoncore.vcxproj" /upgrade
PS C:\>

[uilianries]

@bog-dan-ro My recommendation is using Visual Studio Community Edition. In the past we tried to use Build Tools for CI and was a nightmare trying to skip devenv or even installing as a separated tool. Some projects need devenv because are based only .sln files and they are not compatible if you don't update.

[bog-dan-ro]

@uilianries I created a dummy devenv.bat file (it contains only an @echo off) and I could build cpython as a static library (I didn't try to use it yet). So it seems that in this particular case, devenv is not really needed.

Shall I open a new bug report for #12373 (comment) (conan install --build=missing -o shared=True -o with_tkinter=False -pr:b=default cpython/3.9.7@)?

It's a bit more complicated to use Visual Studio Community Edition. If you have a payed license you can freely use Build Tools on your CI, but you can not use Visual Studio Community Edition... The only option is to buy a new license and install the pro version.
That's why it will be great if we can somehow skip devenv ...

I also reported this problem to microsoft here https://developercommunity.visualstudio.com/t/devenv-is-missing-from-build-tools-2022/10138030 ...

[uilianries]

@bog-dan-ro The bug report is this PR. If you can build without project upgrade, the recipe should be updated.

[Ahajha]

In case this is still relevant, I believe this is fixed in the CPython Conan 2 PR: #21387

No project upgrade occurs.