Experimental 'genvslite' WIP. (#11049)

* Capture all compile args from the first round of ninja backend generation for all languages used in building the targets so that these args, defines, and include paths can be applied to the .vcxproj's intellisense fields for all buildtypes/configurations.

Solution generation is now set up for mutiple build configurations (buildtypes) when using '--genvslite'.

All generated vcxprojs invoke the same high-level meson compile to build all targets;  there's no selective target building (could add this later).  Related to this, we skip pointlessly generating vcxprojs for targets that aren't buildable (BuildTarget-derived), which aren't of interest to the user anyway.

When using --genvslite, no longer inject '<ProjectReference ...>' dependencies on which a generated .vcxproj depends because that imposes a forced visual studio build dependency, which we don't want, since we're essentially bypassing VS's build in favour of running 'meson compile ...'.

When populating the vcxproj's shared intellisense defines, include paths, and compiler options fields, we choose the most frequent src file language, since this means more project src files can simply reference the project shared fields and fewer files of non-primary language types need to populate their full set of intellisense fields.  This makes for smaller .vcxproj files.

Paths for generated source/header/etc files, left alone, would be added to solution projects relative to the '..._vs' build directory, where they're never generated;  they're generated under the respective '..._[debug/opt/release]' ninja build directories that correspond to the solution build configuration. Although VS doesn't allow conditional src/header listings in vcxprojs (at least not in a simple way that I'm aware of), we can ensure these generated sources get adjusted to at least reference locations under one of the concrete build directories (I've chosen '..._debug') under which they will be generated.

Testing with --genvslite has revealed that, in some cases, the presence of 'c:\windows\system32;c:\windows' on the 'Path' environment variable (via the make-style project's ExecutablePath element) is critical to getting the 'meson compile ...' build to succeed.  Not sure whether this is some 'find and guess' implicit defaults behaviour within meson or within the MSVC compiler that some projects may rely on. Feels weird but not sure of a better solution than forcibly adding these to the Path environment variable (the Executable Path property of the project).

Added a new windows-only test to windowstests.py ('test_genvslite') to exercise the --genvslite option along with checking that the 'msbuild' command invokes the 'meson compile ...' of the build-type-appropriate-suffixed temporary build dir and checks expected program output.

Check and report error if user specifies a non-ninja backend with a 'genvslite' setup, since that conflicts with the stated behaviour of genvslite.  Also added this test case to 'WindowsTests.test_genvslite'

I had problems tracking down some problematic environment variable behaviour, which appears to need a work-around. See further notes on VSINSTALLDIR, in windowstests.py, test_genvslite.
'meson setup --help' clearly states that positional arguments are ... [builddir] [sourcedir].  However, BasePlatformTests.init(...) was passing these in the order [sourcedir] [builddir].  This was producing failures, saying, "ERROR: Neither directory contains a build file meson.build." but when using the correct ordering, setup now succeeds.

Changed regen, run_tests, and run_install utility projects to be simpler makefile projects instead, with commands to invoke the appropriate '...meson.py --internal regencheck ...' (or install/test) on the '[builddir]_[buildtype]' as appropriate for the curent VS configuration.  Also, since the 'regen.vcxproj' utility didn't work correctly with '--genvslite' setup build dirs, and getting it to fully work would require more non-trivial intrusion into new parts of meson (i.e. '--internal regencheck', '--internal regenerate', and perhaps also 'setup --reconfigure'), for now, the REGEN project is replaced with a simpler, lighter-weight RECONFIGURE utility proj, which is unlinked from any solution build dependencies and which simply runs 'meson setup --reconfigure [builddir]_[buildtype] [srcdir]' on each of the ninja-backend build dirs for each buildtype.
Yes, although this will enable the building/compiling to be correctly configured, it can leave the solution/vcxprojs stale and out-of-date, it's simple for the user to 'meson setup --genvslite ...' to fully regenerate an updated, correct solution again. However, I've noted this down as a 'fixme' to consider implementing the full regen behaviour for the genvslite case.

* Review feedback changes -
- Avoid use of 'captured_compile_args_per_buildtype_and_target' as an 'out' param.
- Factored a little msetup.py, 'run(...)' macro/looping setup steps, for genvslite, out into a 'run_genvslite_setup' func.

* Review feedback:  Fixed missing spaces between multi-line strings.

* 'backend_name' assignment gets immediately overwritten in 'genvslite' case so moved it into else/non-genvslite block.

* Had to bump up 'test cases/unit/113 genvslites/...' up to 114; it collided with a newly added test dir again.

* Changed validation of 'capture' and 'captured_compile_args_...' to use MesonBugException instead of MesonException.

* Changed some function param and closing brace indentation.

docs/markdown/Builtin-options.md

@@ -76,6 +76,7 @@ machine](#specifying-options-per-machine) section for details.
 | -------------------------------------- | ------------- | -----------                                                    | -------------- | ----------------- |
 | auto_features {enabled, disabled, auto} | auto         | Override value of all 'auto' features                          | no             | no                |
 | backend {ninja, vs,<br>vs2010, vs2012, vs2013, vs2015, vs2017, vs2019, vs2022, xcode, none} | ninja | Backend to use    | no             | no                |
+| genvslite {vs2022}                     | vs2022        | Setup multi-builtype ninja build directories and Visual Studio solution | no | no |
 | buildtype {plain, debug,<br>debugoptimized, release, minsize, custom} | debug | Build type to use                       | no             | no                |
 | debug                                  | true          | Enable debug symbols and other information                     | no             | no                |
 | default_library {shared, static, both} | shared        | Default library type                                           | no             | yes               |
@@ -106,6 +107,26 @@ configure with no backend at all, which is an error if you have targets to
 build, but for projects that need configuration + testing + installation allows
 for a lighter automated build pipeline.
 
+#### Details for `genvslite`
+
+Setup multiple buildtype-suffixed, ninja-backend build directories (e.g.
+[builddir]_[debug/release/etc.]) and generate [builddir]_vs containing a Visual
+Studio solution with multiple configurations that invoke a meson compile of the
+setup build directories, as appropriate for the current configuration (builtype).
+
+This has the effect of a simple setup macro of multiple 'meson setup ...'
+invocations with a set of different buildtype values.  E.g.
+`meson setup ... --genvslite vs2022 somebuilddir` does the following -
+```
+meson setup ... --backend ninja --buildtype debug somebuilddir_debug
+meson setup ... --backend ninja --buildtype debugoptimized somebuilddir_debugoptimized
+meson setup ... --backend ninja --buildtype release somebuilddir_release
+```
+and additionally creates another 'somebuilddir_vs' directory that contains
+a generated multi-configuration visual studio solution and project(s) that are
+set to build/compile with the somebuilddir_[...] that's appropriate for the
+solution's selected buildtype configuration.
+
 #### Details for `buildtype`
 
 <a name="build-type-options"></a> For setting optimization levels and

docs/markdown/snippets/gen_vslite.md

@@ -0,0 +1,11 @@
+## Added a new '--genvslite' option for use with 'meson setup ...'
+
+To facilitate a more usual visual studio work-flow of supporting and switching between
+multiple build configurations (buildtypes) within the same solution, among other
+[reasons](https://github.com/mesonbuild/meson/pull/11049), use of this new option
+has the effect of setting up multiple ninja back-end-configured build directories,
+named with their respective buildtype suffix.  E.g. 'somebuilddir_debug',
+'somebuilddir_release', etc. as well as a '_vs'-suffixed directory that contains the
+generated multi-buildtype solution.  Building/cleaning/rebuilding in the solution
+now launches the meson build (compile) of the corresponding buildtype-suffixed build
+directory, instead of using Visual Studio's native engine.

mesonbuild/backend/backends.py

@@ -256,6 +256,13 @@ def get_backend_from_name(backend: str, build: T.Optional[build.Build] = None, i
         return nonebackend.NoneBackend(build, interpreter)
     return None
 
+
+def get_genvslite_backend(genvsname: str, build: T.Optional[build.Build] = None, interpreter: T.Optional['Interpreter'] = None) -> T.Optional['Backend']:
+    if genvsname == 'vs2022':
+        from . import vs2022backend
+        return vs2022backend.Vs2022Backend(build, interpreter, gen_lite = True)
+    return None
+
 # This class contains the basic functionality that is needed by all backends.
 # Feel free to move stuff in and out of it as you see fit.
 class Backend:
@@ -280,7 +287,17 @@ def __init__(self, build: T.Optional[build.Build], interpreter: T.Optional['Inte
         self.src_to_build = mesonlib.relpath(self.environment.get_build_dir(),
                                              self.environment.get_source_dir())
 
-    def generate(self) -> None:
+    # If requested via 'capture = True', returns captured compile args per
+    # target (e.g. captured_args[target]) that can be used later, for example,
+    # to populate things like intellisense fields in generated visual studio
+    # projects (as is the case when using '--genvslite').
+    #
+    # 'captured_compile_args_per_buildtype_and_target' is only provided when
+    # we expect this backend setup/generation to make use of previously captured
+    # compile args (as is the case when using '--genvslite').
+    def generate(self,
+                 capture: bool = False,
+                 captured_compile_args_per_buildtype_and_target: dict = None) -> T.Optional[dict]:
         raise RuntimeError(f'generate is not implemented in {type(self).__name__}')
 
     def get_target_filename(self, t: T.Union[build.Target, build.CustomTargetIndex], *, warn_multi_output: bool = True) -> str:

mesonbuild/backend/ninjabackend.py

@@ -38,7 +38,7 @@
 from ..compilers import Compiler
 from ..linkers import ArLikeLinker, RSPFileSyntax
 from ..mesonlib import (
-    File, LibType, MachineChoice, MesonException, OrderedSet, PerMachine,
+    File, LibType, MachineChoice, MesonBugException, MesonException, OrderedSet, PerMachine,
     ProgressBar, quote_arg
 )
 from ..mesonlib import get_compiler_for_source, has_path_sep, OptionKey
@@ -575,7 +575,13 @@ def detect_prefix(out):
 
         raise MesonException(f'Could not determine vs dep dependency prefix string. output: {stderr} {stdout}')
 
-    def generate(self):
+    def generate(self,
+                 capture: bool = False,
+                 captured_compile_args_per_buildtype_and_target: dict = None) -> T.Optional[dict]:
+        if captured_compile_args_per_buildtype_and_target:
+            # We don't yet have a use case where we'd expect to make use of this,
+            # so no harm in catching and reporting something unexpected.
+            raise MesonBugException('We do not expect the ninja backend to be given a valid \'captured_compile_args_per_buildtype_and_target\'')
         ninja = environment.detect_ninja_command_and_version(log=True)
         if self.environment.coredata.get_option(OptionKey('vsenv')):
             builddir = Path(self.environment.get_build_dir())
@@ -614,6 +620,14 @@ def generate(self):
             self.build_elements = []
             self.generate_phony()
             self.add_build_comment(NinjaComment('Build rules for targets'))
+
+            # Optionally capture compile args per target, for later use (i.e. VisStudio project's NMake intellisense include dirs, defines, and compile options).
+            if capture:
+                captured_compile_args_per_target = {}
+                for target in self.build.get_targets().values():
+                    if isinstance(target, build.BuildTarget):
+                        captured_compile_args_per_target[target.get_id()] = self.generate_common_compile_args_per_src_type(target)
+
             for t in ProgressBar(self.build.get_targets().values(), desc='Generating targets'):
                 self.generate_target(t)
             self.add_build_comment(NinjaComment('Test rules'))
@@ -652,6 +666,9 @@ def generate(self):
         self.generate_compdb()
         self.generate_rust_project_json()
 
+        if capture:
+            return captured_compile_args_per_target
+
     def generate_rust_project_json(self) -> None:
         """Generate a rust-analyzer compatible rust-project.json file."""
         if not self.rust_crates:
@@ -2922,6 +2939,34 @@ def _generate_single_compile_target_args(self, target: build.BuildTarget, compil
         commands += compiler.get_include_args(self.get_target_private_dir(target), False)
         return commands
 
+    # Returns a dictionary, mapping from each compiler src type (e.g. 'c', 'cpp', etc.) to a list of compiler arg strings
+    # used for that respective src type.
+    # Currently used for the purpose of populating VisualStudio intellisense fields but possibly useful in other scenarios.
+    def generate_common_compile_args_per_src_type(self, target: build.BuildTarget) -> dict[str, list[str]]:
+        src_type_to_args = {}
+
+        use_pch = self.environment.coredata.options.get(OptionKey('b_pch'))
+
+        for src_type_str in target.compilers.keys():
+            compiler = target.compilers[src_type_str]
+            commands = self._generate_single_compile_base_args(target, compiler)
+
+            # Include PCH header as first thing as it must be the first one or it will be
+            # ignored by gcc https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100462
+            if use_pch and 'mw' not in compiler.id:
+                commands += self.get_pch_include_args(compiler, target)
+
+            commands += self._generate_single_compile_target_args(target, compiler, is_generated=False)
+
+            # Metrowerks compilers require PCH include args to come after intraprocedural analysis args
+            if use_pch and 'mw' in compiler.id:
+                commands += self.get_pch_include_args(compiler, target)
+
+            commands = commands.compiler.compiler_args(commands)
+
+            src_type_to_args[src_type_str] = commands.to_native()
+        return src_type_to_args
+
     def generate_single_compile(self, target: build.BuildTarget, src,
                                 is_generated=False, header_deps=None,
                                 order_deps: T.Optional[T.List['mesonlib.FileOrString']] = None,

mesonbuild/backend/nonebackend.py

@@ -14,6 +14,8 @@
 
 from __future__ import annotations
 
+import typing as T
+
 from .backends import Backend
 from .. import mlog
 from ..mesonlib import MesonBugException
@@ -23,7 +25,15 @@ class NoneBackend(Backend):
 
     name = 'none'
 
-    def generate(self):
+    def generate(self,
+                 capture: bool = False,
+                 captured_compile_args_per_buildtype_and_target: dict = None) -> T.Optional[dict]:
+        # Check for (currently) unexpected capture arg use cases -
+        if capture:
+            raise MesonBugException('We do not expect the none backend to generate with \'capture = True\'')
+        if captured_compile_args_per_buildtype_and_target:
+            raise MesonBugException('We do not expect the none backend to be given a valid \'captured_compile_args_per_buildtype_and_target\'')
+
         if self.build.get_targets():
             raise MesonBugException('None backend cannot generate target rules, but should have failed earlier.')
         mlog.log('Generating simple install-only backend')