Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[boost] stop renaming libs and add has_synchro for boost-atomic #27694

Closed
wants to merge 76 commits into from
Closed

[boost] stop renaming libs and add has_synchro for boost-atomic #27694

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
92f23c1
add compiler version to vcpkg-cmake-get-vars
Neumann-A Nov 7, 2022
30cba5f
boost library naming stuff
Neumann-A Nov 7, 2022
a82885e
fix missing ,
Neumann-A Nov 7, 2022
3d72073
has_synchronization_lib check for boost-atomic.
Neumann-A Nov 7, 2022
5d6c981
use correct ver dep
Neumann-A Nov 7, 2022
24ac419
v db
Neumann-A Nov 7, 2022
3395c01
update helper script
Neumann-A Nov 7, 2022
f74ac8b
Merge remote-tracking branch 'origin/master' into HEAD
BillyONeal Nov 8, 2022
c32d32e
install CMake 3.24.3 FindBoost.cmake
Neumann-A Nov 10, 2022
60c2435
remove more renaming.
Neumann-A Nov 10, 2022
fb66ad5
v db
Neumann-A Nov 10, 2022
cd3c953
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Nov 11, 2022
87f2900
more suffixes restored.
Neumann-A Nov 11, 2022
806c1cd
v db
Neumann-A Nov 12, 2022
e7f64cd
guard policy 102
Neumann-A Nov 12, 2022
d3bc6b9
test in CI
Neumann-A Nov 12, 2022
8ae0528
more policy adjustments
Neumann-A Nov 12, 2022
f31254a
add pass entry to ci baseline
Neumann-A Nov 13, 2022
30aa236
fix vcpkg-ci-boost on arm
Neumann-A Nov 13, 2022
ebcd03a
more fixes
Neumann-A Nov 13, 2022
3f1edd0
more dep adjustment
Neumann-A Nov 13, 2022
f8a9515
more platform deps
Neumann-A Nov 13, 2022
d44217d
more dep and supports fixes
Neumann-A Nov 13, 2022
4b32e9c
remove uwp from testing
Neumann-A Nov 13, 2022
040d793
v db and script
Neumann-A Nov 13, 2022
28d00f9
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Nov 14, 2022
481e069
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Dec 2, 2022
ea70fa2
v db
Neumann-A Dec 2, 2022
f8402bf
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Dec 3, 2022
a84154c
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Dec 5, 2022
ee6a9a3
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Dec 21, 2022
4877304
details
Neumann-A Dec 21, 2022
9fd2c02
v db
Neumann-A Dec 21, 2022
d7f25d4
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Jan 16, 2023
9e3cff0
v db
Neumann-A Jan 16, 2023
e5f4c7c
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Jan 17, 2023
504a863
revert v db fix 2023
Neumann-A Jan 17, 2023
5e49004
v db
Neumann-A Jan 17, 2023
71c1126
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Jan 19, 2023
088058b
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Jan 20, 2023
66c202f
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Jan 27, 2023
6d77309
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Feb 8, 2023
62ac681
port version bump
Neumann-A Feb 8, 2023
8c0c931
v db
Neumann-A Feb 8, 2023
0608d38
restore removal of renaming
Neumann-A Feb 9, 2023
2fbcda1
v db
Neumann-A Feb 9, 2023
994a6b9
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Feb 10, 2023
dff3332
v db
Neumann-A Feb 10, 2023
4ba314e
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Feb 27, 2023
ee70360
v db
Neumann-A Feb 27, 2023
e1289cc
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Feb 27, 2023
016860e
revert vdb and boost-random
Neumann-A Feb 27, 2023
691d499
generate update
Neumann-A Feb 27, 2023
7b73f99
v db
Neumann-A Feb 27, 2023
016589e
Merge branch 'microsoft:master' into fix_boost_clang_cl
Neumann-A Mar 8, 2023
20b6ea4
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Mar 18, 2023
13442c7
v db
Neumann-A Mar 18, 2023
ee0067c
remove ws
Neumann-A Mar 18, 2023
aec8ad6
v db
Neumann-A Mar 18, 2023
05de907
Merge remote-tracking branch 'upstream/master' into fix_boost_clang_cl
Neumann-A Apr 19, 2023
0827185
v db
Neumann-A Apr 19, 2023
f8515fd
v changes
Neumann-A Apr 19, 2023
ea84a1f
v db
Neumann-A Apr 19, 2023
fd7fe2b
Merge branch 'master' of https://github.com/microsoft/vcpkg into fix_…
JonLiu1993 May 9, 2023
619f3a9
fix android triplets
Neumann-A May 9, 2023
84c8720
v db
Neumann-A May 9, 2023
4e51b1d
merge
JavierMatosD Jun 16, 2023
052810f
Merge remote-tracking branch 'origin/master' into HEAD
JavierMatosD Jun 16, 2023
07782c6
remove boost-atomic stuff since it's already merged
JavierMatosD Jun 16, 2023
de9e3c0
newline at end of file
JavierMatosD Jun 16, 2023
7eaefb8
Merge remote-tracking branch 'origin/master' into HEAD
BillyONeal Jun 20, 2023
8e3df40
Revert do-nothing changes.
BillyONeal Jun 20, 2023
9a7f23f
run generate-ports
JavierMatosD Jun 21, 2023
7cc2ec2
revert mystery change from running generate-ports
JavierMatosD Jun 21, 2023
a94f089
version db
JavierMatosD Jun 21, 2023
1861d39
fix boost-asio
JavierMatosD Jun 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ports/boost-modular-build-helper/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ if(VCPKG_DETECTED_MSVC)

if(VCPKG_DETECTED_CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
set(USER_CONFIG_TOOLSET clang-win)
string(REGEX MATCH "[^\\\.]+" USER_CONFIG_TOOLSET_VERSION "${VCPKG_DETECTED_CMAKE_CXX_COMPILER_VERSION}")
else()
set(USER_CONFIG_TOOLSET msvc)
if(VCPKG_DETECTED_MSVC_VERSION LESS "1900")
Expand Down
56 changes: 34 additions & 22 deletions ports/boost-modular-build-helper/boost-modular-build.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ get_filename_component(BOOST_BUILD_INSTALLED_DIR "${CMAKE_CURRENT_LIST_DIR}" DIR
get_filename_component(BOOST_BUILD_INSTALLED_DIR "${BOOST_BUILD_INSTALLED_DIR}" DIRECTORY)

set(BOOST_VERSION "${VERSION}")

string(REGEX MATCH "^([0-9]+)\\.([0-9]+)\\.([0-9]+)" BOOST_VERSION_MATCH "${BOOST_VERSION}")
if("${CMAKE_MATCH_3}" GREATER 0)
set(BOOST_VERSION_ABI_TAG "${CMAKE_MATCH_1}_${CMAKE_MATCH_2}_${CMAKE_MATCH_3}")
Expand All @@ -16,6 +17,9 @@ endif()
function(boost_modular_build)
cmake_parse_arguments(_bm "" "SOURCE_PATH;BOOST_CMAKE_FRAGMENT" "" ${ARGN})

vcpkg_cmake_get_vars(cmake_vars_file)
include("${cmake_vars_file}")

if(NOT DEFINED _bm_SOURCE_PATH)
message(FATAL_ERROR "SOURCE_PATH is a required argument to boost_modular_build.")
endif()
Expand All @@ -38,18 +42,38 @@ function(boost_modular_build)
else()
message(FATAL_ERROR "Could not find b2 in ${BOOST_BUILD_PATH}")
endif()
if(NOT VCPKG_CMAKE_SYSTEM_NAME OR VCPKG_CMAKE_SYSTEM_NAME STREQUAL "WindowsStore")
set(BOOST_LIB_PREFIX)
if(VCPKG_PLATFORM_TOOLSET MATCHES "v14.")
set(BOOST_LIB_RELEASE_SUFFIX -vc140-mt.lib)
set(BOOST_LIB_DEBUG_SUFFIX -vc140-mt-gd.lib)
if(VCPKG_DETECTED_CMAKE_SYSTEM_PROCESSOR STREQUAL "IA64")
string(APPEND BOOST_ARCHITECTURE_TAG "i")
elseif(VCPKG_DETECTED_CMAKE_SYSTEM_PROCESSOR MATCHES "^[xX]86"
OR VCPKG_DETECTED_CMAKE_SYSTEM_PROCESSOR MATCHES "^(([xX]86_)?[xX]64|AMD64)")
string(APPEND BOOST_ARCHITECTURE_TAG "x")
elseif(VCPKG_DETECTED_CMAKE_SYSTEM_PROCESSOR MATCHES "^ARM")
string(APPEND BOOST_ARCHITECTURE_TAG "a")
elseif(VCPKG_DETECTED_CMAKE_SYSTEM_PROCESSOR STREQUAL "MIPS")
string(APPEND BOOST_ARCHITECTURE_TAG "m")
endif()
if(VCPKG_DETECTED_CMAKE_SIZEOF_VOID_P EQUAL "8")
string(APPEND BOOST_ARCHITECTURE_TAG "64")
else()
string(APPEND BOOST_ARCHITECTURE_TAG "32")
endif()

set(BOOST_LIB_PREFIX "")
if(VCPKG_DETECTED_CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
if(VCPKG_PLATFORM_TOOLSET MATCHES "v(14.)")
set(BOOST_LIB_RELEASE_SUFFIX -vc${CMAKE_MATCH_1}-mt-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
set(BOOST_LIB_DEBUG_SUFFIX -vc${CMAKE_MATCH_1}-mt-gd-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
elseif(VCPKG_PLATFORM_TOOLSET MATCHES "v120")
set(BOOST_LIB_RELEASE_SUFFIX -vc120-mt.lib)
set(BOOST_LIB_DEBUG_SUFFIX -vc120-mt-gd.lib)
set(BOOST_LIB_RELEASE_SUFFIX -vc120-mt-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
set(BOOST_LIB_DEBUG_SUFFIX -vc120-mt-gd-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
else()
set(BOOST_LIB_RELEASE_SUFFIX .lib)
set(BOOST_LIB_DEBUG_SUFFIX d.lib)
set(BOOST_LIB_RELEASE_SUFFIX -${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
set(BOOST_LIB_DEBUG_SUFFIX -${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib) # Note: FindBoost.cmake will not look for d suffixed libraries
endif()
elseif(VCPKG_DETECTED_CMAKE_CXX_COMPILER_ID STREQUAL "Clang" AND VCPKG_DETECTED_CMAKE_CXX_COMPILER_FRONTEND_VARIANT STREQUAL "MSVC")
string(REGEX MATCH "[^\\\.]+" clang_major_ver "${VCPKG_DETECTED_CMAKE_CXX_COMPILER_VERSION}")
set(BOOST_LIB_RELEASE_SUFFIX -clangw${clang_major_ver}-mt-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
set(BOOST_LIB_DEBUG_SUFFIX -clangw${clang_major_ver}-mt-gd-${BOOST_ARCHITECTURE_TAG}-${BOOST_VERSION_ABI_TAG}.lib)
else()
set(BOOST_LIB_PREFIX lib)
if(VCPKG_TARGET_ARCHITECTURE STREQUAL "wasm32")
Expand Down Expand Up @@ -108,8 +132,6 @@ function(boost_modular_build)
list(APPEND configure_options "-DBOOST_CMAKE_FRAGMENT=${_bm_BOOST_CMAKE_FRAGMENT}")
endif()

vcpkg_cmake_get_vars(cmake_vars_file)

vcpkg_check_features(
OUT_FEATURE_OPTIONS feature_options
FEATURES
Expand Down Expand Up @@ -152,17 +174,7 @@ function(boost_modular_build)
get_filename_component(DIRECTORY_OF_LIB_FILE ${LIB} DIRECTORY)
string(REPLACE "libboost_" "boost_" NEW_FILENAME ${OLD_FILENAME})
string(REPLACE "-s-" "-" NEW_FILENAME ${NEW_FILENAME}) # For Release libs
string(REPLACE "-vc141-" "-vc140-" NEW_FILENAME ${NEW_FILENAME}) # To merge VS2017 and VS2015 binaries
string(REPLACE "-vc142-" "-vc140-" NEW_FILENAME ${NEW_FILENAME}) # To merge VS2019 and VS2015 binaries
string(REPLACE "-vc143-" "-vc140-" NEW_FILENAME ${NEW_FILENAME}) # To merge VS2022 and VS2015 binaries
string(REPLACE "-sgd-" "-gd-" NEW_FILENAME ${NEW_FILENAME}) # For Debug libs
string(REPLACE "-sgyd-" "-gyd-" NEW_FILENAME ${NEW_FILENAME}) # For Debug libs
string(REPLACE "-gyd-" "-gd-" NEW_FILENAME ${NEW_FILENAME}) # For Debug libs with python debugging
string(REPLACE "-x32-" "-" NEW_FILENAME ${NEW_FILENAME}) # To enable CMake 3.10 and earlier to locate the binaries
string(REPLACE "-x64-" "-" NEW_FILENAME ${NEW_FILENAME}) # To enable CMake 3.10 and earlier to locate the binaries
string(REPLACE "-a32-" "-" NEW_FILENAME ${NEW_FILENAME}) # To enable CMake 3.10 and earlier to locate the binaries
string(REPLACE "-a64-" "-" NEW_FILENAME ${NEW_FILENAME}) # To enable CMake 3.10 and earlier to locate the binaries
string(REPLACE "-${BOOST_VERSION_ABI_TAG}" "" NEW_FILENAME ${NEW_FILENAME}) # To enable CMake > 3.10 to locate the binaries
string(REGEX REPLACE "-s?gy?d-" "-gd-" NEW_FILENAME ${NEW_FILENAME}) # For Debug libs
if("${DIRECTORY_OF_LIB_FILE}/${NEW_FILENAME}" STREQUAL "${DIRECTORY_OF_LIB_FILE}/${OLD_FILENAME}")
# nothing to do
elseif(EXISTS "${DIRECTORY_OF_LIB_FILE}/${NEW_FILENAME}")
Expand Down
2 changes: 1 addition & 1 deletion ports/boost-modular-build-helper/vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$comment": "Automatically generated by scripts/boost/generate-ports.ps1",
"name": "boost-modular-build-helper",
"version": "1.82.0",
"port-version": 3,
"port-version": 4,
"description": "Internal vcpkg port used to build Boost libraries",
"license": "MIT",
"dependencies": [
Expand Down
17 changes: 16 additions & 1 deletion ports/boost-uninstall/portfile.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,19 @@ set(VCPKG_POLICY_EMPTY_PACKAGE enabled)
message(STATUS "\nPlease use the following command when you need to remove all boost ports/components:\n\
\"./vcpkg remove boost-uninstall:${TARGET_TRIPLET} --recurse\"\n")

configure_file("${CMAKE_CURRENT_LIST_DIR}/vcpkg-cmake-wrapper.cmake" "${CURRENT_PACKAGES_DIR}/share/boost/vcpkg-cmake-wrapper.cmake" @ONLY)
vcpkg_download_distfile(
FILE_PATH
URLS https://gitlab.kitware.com/cmake/cmake/-/raw/v3.24.3/Modules/FindBoost.cmake
FILENAME FindBoost-CMake-3.24.3.cmake
SHA512 db16fa222ec28135d6dd13d62661d01850f93d68b3cbc5d035da92e6c4337285676885b9d27bc7f0dda7dd620de6536c804ef760f78b3959cd5d1d50cc27a7df
)

file(INSTALL "${FILE_PATH}" DESTINATION "${CURRENT_PACKAGES_DIR}/share/boost" RENAME "FindBoost.cmake")
vcpkg_replace_string("${CURRENT_PACKAGES_DIR}/share/boost/FindBoost.cmake" [[include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake)]] [[include(${CMAKE_ROOT}/Modules/FindPackageHandleStandardArgs.cmake)]])
vcpkg_replace_string("${CURRENT_PACKAGES_DIR}/share/boost/FindBoost.cmake" [[cmake_policy(GET CMP0093 _Boost_CMP0093
PARENT_SCOPE # undocumented, do not use outside of CMake
)]] "if(POLICY CMP0093)\ncmake_policy(GET CMP0093 _Boost_CMP0093\n PARENT_SCOPE # undocumented, do not use outside of CMake\n )\nelse()\nset(_Boost_CMP0093 OLD)\nendif()\n")
vcpkg_replace_string("${CURRENT_PACKAGES_DIR}/share/boost/FindBoost.cmake" [[cmake_policy(GET CMP0074 _Boost_CMP0074)]] "if(POLICY CMP0074)\ncmake_policy(GET CMP0074 _Boost_CMP0074)\nelse()\nset(_Boost_CMP0074 OLD)\nendif()\n")
vcpkg_replace_string("${CURRENT_PACKAGES_DIR}/share/boost/FindBoost.cmake" [[cmake_policy(SET CMP0102 NEW)]] "if(POLICY CMP0102)\ncmake_policy(SET CMP0102 NEW)\nendif()\n")

configure_file("${CMAKE_CURRENT_LIST_DIR}/vcpkg-cmake-wrapper.cmake" "${CURRENT_PACKAGES_DIR}/share/boost/vcpkg-cmake-wrapper.cmake" @ONLY)
Comment on lines +13 to +21
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been messing around with this to see if this would work with older versions of CMake. This doesn't seem to work with CMake 3.12 or below.
To repro you can do the following:

vcpkg install boost
vcpkg install boost-regex

I have a list file that looks like this:

cmake_minimum_required(VERSION 3.0)

project(HelloWorld)

# Find Boost
find_package(Boost REQUIRED COMPONENTS regex)

# Add executable
add_executable(HelloWorld main.cpp)

# Link Boost libraries
target_link_libraries(HelloWorld Boost::regex)

//main.cpp

#include <iostream>
#include <boost/version.hpp>
#include <boost/regex.hpp>

int main()
{
    std::cout << "Hello, world!\n";
    std::cout << "Using Boost version " << BOOST_VERSION << "\n";

    // Test the Boost regex library
    std::string input = "Hello, world!";
    boost::regex pattern("world");
    bool result = boost::regex_search(input, pattern);
    std::cout << "Regex search result: " << result << "\n";

    return 0;
}

cmake-3.12.0.exe -G "Ninja" -DCMAKE_TOOLCHAIN_FILE=C:\dev\vcpkg\scripts\buildsystems\vcpkg.cmake ..
You get the following error:

CMake Warning at C:/dev/vcpkg/installed/x64-windows/share/boost/FindBoost.cmake:1387 (message):
  New Boost version may have incorrect or missing dependencies and imported
  targets
Call Stack (most recent call first):
  C:/dev/vcpkg/installed/x64-windows/share/boost/FindBoost.cmake:1510 (_Boost_COMPONENT_DEPENDENCIES)
  C:/dev/vcpkg/installed/x64-windows/share/boost/FindBoost.cmake:2131 (_Boost_MISSING_DEPENDENCIES)
  C:/dev/vcpkg/installed/x64-windows/share/boost/vcpkg-cmake-wrapper.cmake:13 (_find_package)
  C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:800 (include)
  CMakeLists.txt:6 (find_package)


CMake Error at C:/Program Files/cmake-3.12.0-win64-x64/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find Boost (missing: regex) (found version "1.81.0")
Call Stack (most recent call first):
  C:/Program Files/cmake-3.12.0-win64-x64/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  C:/dev/vcpkg/installed/x64-windows/share/boost/FindBoost.cmake:2389 (find_package_handle_standard_args)
  C:/dev/vcpkg/installed/x64-windows/share/boost/vcpkg-cmake-wrapper.cmake:13 (_find_package)
  C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:800 (include)
  CMakeLists.txt:6 (find_package)

The project compiles and links successfully when I use cmake 3.24.3 as expected.

Copy link
Contributor Author

@Neumann-A Neumann-A Mar 1, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did your CMake 3.12 test use VS2017 or below? If not the test is invalid, if yes please provide a --trace-expand log.
(Also mixing a boost build with VS2022 and consuming it in VS2017 intentionally won't work with this PR; So you need to build everything with VS2017)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did your CMake 3.12 test use VS2017 or below?

I tested with VS2022, but I was actually able to build successfully by setting Boost_COMPILER to "-vc143" in the vcpkg-cmake-wrapper.cmake.

(Also mixing a boost build with VS2022 and consuming it in VS2017 intentionally won't work with this PR; So you need to build everything with VS2017)

I believe this is a scenario that we care about and was possible before this PR. I.e., VS2022 with CMake 3.12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is a scenario that we care about

No the scenario you care about is the other way around. Building boost with VS2017 and consuming with VS2022. The other way around is unsupported by VS itself but was possible before.

I tested with VS2022, but I was actually able to build successfully by setting Boost_COMPILER to "-vc143"

I am not fixing virtual problems. VS2022 + CMake 3.12 isn't supported by CMake itself. As such it is probably unable to deduce MSVC_TOOLSET_VERSION correctly which is required for FindBoost.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current behavior does: a) leads to irritation about the lib name b) doesn't work with clang-cl c) Is wrong in that regard that VS2015 could use boost build with VS2022 which certainly will end in linker errors.....

Putting aside my backward compatibility concerns for the moment, could you please expand on how this PR addresses your three points?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why this is true

Boost_COMPILER set _boost_COMPILER in FindBoost here:
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L1913

Lookup for library names is done e.g. around here :
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L2230

So by setting Boost_COMPILER you are telling FindBoost to look for boost libs with a fixed infix (v140) however using clang the infix is (clangw<major>). Renaming the infix should be avoided since lld-link and link linked libraries are not necessary link compatible if lld-link left IR metadata in the library (due to -flto or something similar)

The whole name guesing FindBoost normally does is below
https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L861

Copy link
Contributor

@ras0219-msft ras0219-msft Jun 27, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is super helpful, thank you.

I have a few observations on this though:

  1. This is a unique-to-boost behavior. Every user of every[1] other library would find the libraries in mismatched compiler scenarios.
  2. This at best causes a slight shift-left of the failure -- it moves the failure from link time (when it will certainly fail) to configure time. However, it can also cause massive shift-right if it causes CMake to find an unrelated, unmanaged copy of boost from some other folder on the machine. Then, you're tempting ODR fate and who knows what kind of problems.
  3. This doesn't even correctly detect LTO incompatibility in the case of MSVC -- with MSVC, /LTCG requires using exactly the same compiler (down to the same build). With the boost infix as-is, v143 from 17.6.x will happily find the LTCG library from 17.6.0 and fail to link.
  4. This breaks cases where everything would work totally fine. For example, if you don't pass -flto and the libraries are link compatible. It seems like extremely bad behavior to break things that work just to slightly shift around where the failure occurs in a narrow set of circumstances (passing -flto AND mixing compilers).

Given this, I think pinning Boost_COMPILER sounds like the best path forward given all the concerns. It lets us remove our boost library name customizations (Yay!) while still giving customers the same unsurprising experience they get with every other library. It also, importantly, doesn't break any cases that would work[2].

Another PS from above: it makes total sense that we don't care about supporting new Visual Studio Generators with old CMake. However, using a simple developer command prompt with Ninja is broadly compatible between new and old CMake/MSVC and we'd like to avoid breaking that without a highly compelling reason.

[1] I think wxwidgets might also do something similar
[2] Where "work" excludes suspicious ODR-violating things like having multiple copies of Boost available to find and relying on CMake to always pick the "right" one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, it can also cause massive shift-right if it causes CMake to find an unrelated, unmanaged copy of boost from some other folder on the machine.

This should probably forbidden as soon as vcpkg installs the wrapper. However, the current state also makes exactly this possible with clang-cl (and even msvc if the dep is forgotten). The libs get installed with the lib infix -clangw<ver>- vcpkg hardcodes the compiler to be v140. Somewhere on a cmake search path I have either the libs with the infix or i have them without them since FindBoost also looks for libs names without it: https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L2234
So the shift-right is currently also possible independent of this change. (and also for MSVC if you for example forget to install a boost dependency but still look it up and have it available somewhere).

The above would also happen with per port customization switching compilers between clang-cl and cl in the boost ports but that is probably niche usage ;)

I also wouldn't consider this massiv shift-right as too important since to get to that point there has to be a massiv incompetence from the user side. E.g. currently you could simply set Boost_DIR|ROOT to somewhere external or use CONFIG to get a different boost version then provided by vcpkg.

This doesn't even correctly detect LTO incompatibility

LTO was just one example there are more cases where clang-cl/lld-link will generate a binary which cannot be read/linked with link.exe. Ask BillyONeal which metadata he skipped when he implemented the dumpbin check into vcpkg. But that is really not the point here.

Given this, I think pinning Boost_COMPILER sounds like the best path forward given all the concerns.

but to which value? Remember that the wrapper is installed by boost-uninstall and technically there could be a switch in compiler between different boost ports. From my perspective vcpkg either uses the correct infix without setting the variable or removes the infix itself completely (which would make external lookup due to the lib name impossible unless other cmake variables are used to control the lookup behavior).

It lets us remove our boost library name customizations

For me the optimal way forward would be if boost would generate the cmake configs the monolithic build installs instead of assuming https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L1013 and below will be correct. However getting b2 to do that is not quite as simply since it requires the whole information about the dependents and is probably not a quick solution. I tried once a few years ago to get it to work however failed to find enough information about how b2 works to get it to work. Basically you copy the rule form the monolithic build and adjust it to whatever vcpkg does. however fixing the missing parts wasn't as easy/straight forward.

However, using a simple developer command prompt with Ninja is broadly compatible between new and old CMake/MSVC and we'd like to avoid breaking that without a highly compelling reason.

If If FindBoost does not support the boost version you are installing you are out of luck any way (https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindBoost.cmake#L1657)
That being said newer boost + old cmake + newer VS + ninja seems like a very niche combination for me especially since newer VS + old cmake might fail proper compiler detection logic. (It is not all about the VS Generator; Ninja also needs to pass compiler/feature detection which might be ok in general I just wouldn't bet on it.)

So TL;DR:

  • The main point of this PR is to make clang-cl usage work (which it currently does not)
  • A slight shift-left is a fact,
  • The shift-right is also a current state and not introduced by this PR.
  • Should the compiler infix be removed completely? (However this means that MSVC builds will find clang-cl builds which might fail at link time. It also means that MSVC v140 will find libs build with v143 which will also fail at link time if an old linker is used [but that is also current state] ) / or should it be replaced by a fixed value? (which value?)
  • Optimal solution would be cmake configs.cmake but unlikely to get implemented.

Copy link
Contributor

@ras0219-msft ras0219-msft Jun 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If If FindBoost does not support the boost version you are installing you are out of luck any way

In my experience, the vast majority of projects that build successfully with boost version N will also succeed on boost version N+1 with everything else held constant (though the CMake warning is very annoying). The main problem I've seen with old CMake + new Boost is that new libraries aren't available for find_package(Boost COMPONENTS ...). It's theoretically possible that new dependency edges could break the static libraries, but I haven't seen that to happen with any frequency in practice.

Remember that the wrapper is installed by boost-uninstall and technically there could be a switch in compiler between different boost ports.

This is a very good point. I think this indicates that we must pin the infix to a consistent value (possibly empty) for any sort of sane result. (edit: Possible alternative at the end)

Some observations:

  • My recollection of the past is that we couldn't use the "system" naming scheme (the closest approximation to an "empty" infix) because that broke multi-config generators in older CMake versions -- the "system" naming scheme did not include a debug annotation.

  • As long as we set(Boost_COMPILER ...) in the wrapper to some value (possibly determined at port build time) and homogenize the library names, that should fix clang-cl (and all other compilers).

  • I'm still open to making the library names track the "default" names as closely as possible, which sounds like boost-uninstall or boost-modular-build-helper should run a compiler query pass to pick an infix, which will then be consumed and adhered to by every other boost port (regardless of port customizations). This does mean that if you choose to, for example, build boost-filesystem with MSVC v142 instead of v143 the resulting binaries may be mislabeled -- all boost binaries would be marked as "v143" -- but it seems to be as close as we can possibly get given my understanding of the circumstances.

The one other moonshot that might be interesting to explore is whether it's possible to pass the library names into FindBoost directly, skipping all detection. If that's the case, we could just have each boost-xyz port install its own pseudo-targets file and glob them all into the boost-uninstall wrapper:

# share/boost/vcpkg-cmake-wrapper.cmake
file(GLOB _boost_target_files "${CMAKE_CURRENT_LIST_DIR}/boost-*-wrapper.cmake")
foreach(_boost_target_file IN LISTS _boost_target_files)
  include("${_boost_target_file}")
endforeach()
# share/boost/boost-regex-wrapper.cmake
# generated and installed by boost-regex
set(Boost_regex_RELEASE_LIBRARY "${CMAKE_CURRENT_LIST_DIR}/../../lib/boost_regex_clangw11_x64.lib")
set(Boost_regex_DEBUG_LIBRARY "${CMAKE_CURRENT_LIST_DIR}/../../lib/boost_regex_clangw11_x64_gd.lib")

If this approach works, we wouldn't need to add any compiler detection pass and we could remove ALL of our lib name regularization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did not include a debug annotation.

correct that is why system naming is out of the question

pass the library names into FindBoost directly, skipping all detection.

That is always possible since FindBoost just uses find_library and thus CACHE variables. However, I am neither commiting time for either setting the infix to empty or prefilling the CACHE. I am currently happy with my overlay which is this PR and I would rather spend exploring a solution for #32274

11 changes: 9 additions & 2 deletions ports/boost-uninstall/vcpkg-cmake-wrapper.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,14 @@ set(Boost_NO_BOOST_CMAKE ON)
unset(Boost_USE_STATIC_RUNTIME CACHE)
if("${CMAKE_VS_PLATFORM_TOOLSET}" STREQUAL "v120")
set(Boost_COMPILER "-vc120")
endif()
if(DEFINED CMAKE_MODULE_PATH)
set(Z_VCPKG_BACKUP_BOOST_CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH}")
endif()
list(INSERT CMAKE_MODULE_PATH 0 "${CMAKE_CURRENT_LIST_DIR}")
_find_package(${ARGS})
if(DEFINED Z_VCPKG_BACKUP_BOOST_CMAKE_MODULE_PATH)
set(CMAKE_MODULE_PATH "${Z_VCPKG_BACKUP_BOOST_CMAKE_MODULE_PATH}")
else()
set(Boost_COMPILER "-vc140")
unset(CMAKE_MODULE_PATH)
endif()
_find_package(${ARGS})
2 changes: 1 addition & 1 deletion ports/boost-uninstall/vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$comment": "Automatically generated by scripts/boost/generate-ports.ps1",
"name": "boost-uninstall",
"version": "1.82.0",
"port-version": 2,
"port-version": 3,
"description": "Internal vcpkg port used to uninstall Boost",
"license": "MIT"
}
1 change: 1 addition & 0 deletions ports/vcpkg-cmake-get-vars/cmake_get_vars/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ set(COMPILERS ${VCPKG_LANGUAGES} RC)
foreach(prog IN LISTS COMPILERS)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${prog}_COMPILER)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${prog}_COMPILER_ID)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${prog}_COMPILER_VERSION)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${prog}_COMPILER_FRONTEND_VARIANT)
endforeach()
# Variables to check
Expand Down
2 changes: 1 addition & 1 deletion ports/vcpkg-cmake-get-vars/vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "vcpkg-cmake-get-vars",
"version-date": "2023-03-02",
"version-date": "2023-03-18",
"documentation": "https://vcpkg.io/en/docs/README.html",
"license": "MIT",
"dependencies": [
Expand Down
3 changes: 2 additions & 1 deletion scripts/boost/generate-ports.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ else {
$defaultPortVersion = 2
$portVersions = @{
"boost-atomic" = 3;
"boost-modular-build-helper" = 3;
"boost-modular-build-helper" = 4;
"boost-uninstall" = 3;
}

function Get-PortVersion {
Expand Down
1 change: 1 addition & 0 deletions scripts/ci.baseline.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1745,6 +1745,7 @@ vcpkg-ci-arrow:x64-windows-static=pass
vcpkg-ci-arrow:x64-windows-static-md=pass
vcpkg-ci-arrow:x64-osx=pass
vcpkg-ci-arrow:x64-linux=pass
vcpkg-ci-boost:arm64-windows=pass
vcpkg-ci-boost:x64-linux=pass
vcpkg-ci-boost:arm64-windows=pass
vcpkg-ci-boost:x64-osx=pass
Expand Down
2 changes: 1 addition & 1 deletion scripts/test_ports/cmake-user/vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
"dependencies": [
{
"$package": "Boost",
"name":"boost",
"name": "boost",
"platform": "!uwp"
Neumann-A marked this conversation as resolved.
Show resolved Hide resolved
},
{
Expand Down
24 changes: 12 additions & 12 deletions scripts/test_ports/vcpkg-ci-boost/vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"features": [
"mpi"
],
"platform":"!(arm & windows)"
"platform": "!(arm & windows)"
},
{
"name": "boost-asio",
Expand All @@ -24,15 +24,15 @@
"features": [
"numa"
],
"platform":"!uwp & !arm & !emscripten"
"platform": "!uwp & !arm & !emscripten"
},
{
"name": "boost-locale",
"default-features": false,
"features": [
"icu"
],
"platform":"!uwp"
"platform": "!uwp"
},
{
"name": "boost-mpi",
Expand All @@ -43,8 +43,8 @@
"platform": "!static & !(arm & windows)"
},
{
"name":"boost-odeint",
"default-features":false
"name": "boost-odeint",
"default-features": false
},
{
"name": "boost-odeint",
Expand All @@ -56,31 +56,31 @@
},
{
"name": "boost-python",
"default-features":false,
"features":[
"default-features": false,
"features": [
"python2"
],
"platform":"!(arm & windows)"
"platform": "!(arm & windows)"
},
{
"name": "boost-python",
"default-features": false,
"features": [
"python3"
],
"platform":"!uwp"
"platform": "!uwp"
},
{
"name":"boost-regex",
"default-features":false
"name": "boost-regex",
"default-features": false
},
{
"name": "boost-regex",
"default-features": false,
"features": [
"icu"
],
"platform":"!uwp"
"platform": "!uwp"
}
]
}
5 changes: 5 additions & 0 deletions versions/b-/boost-modular-build-helper.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"versions": [
{
"git-tree": "2144707e458c92cee8b812024688c1640bf1a04d",
"version": "1.82.0",
"port-version": 4
},
{
"git-tree": "98f75b07bdab1080c9470eb115836035d7142397",
"version": "1.82.0",
Expand Down
5 changes: 5 additions & 0 deletions versions/b-/boost-uninstall.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"versions": [
{
"git-tree": "ebf33093b547d7a0bbf0ab6960f0f1f5623c0e71",
"version": "1.82.0",
"port-version": 3
},
{
"git-tree": "dcca7487581acb01e55bddd3ec306ff543c24ce6",
"version": "1.82.0",
Expand Down
8 changes: 4 additions & 4 deletions versions/baseline.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -654,7 +654,7 @@
},
"boost-asio": {
"baseline": "1.82.0",
"port-version": 3
"port-version": 2
},
"boost-assert": {
"baseline": "1.82.0",
Expand Down Expand Up @@ -934,7 +934,7 @@
},
"boost-modular-build-helper": {
"baseline": "1.82.0",
"port-version": 3
"port-version": 4
},
"boost-move": {
"baseline": "1.82.0",
Expand Down Expand Up @@ -1186,7 +1186,7 @@
},
"boost-uninstall": {
"baseline": "1.82.0",
"port-version": 2
"port-version": 3
},
"boost-units": {
"baseline": "1.82.0",
Expand Down Expand Up @@ -8397,7 +8397,7 @@
"port-version": 1
},
"vcpkg-cmake-get-vars": {
"baseline": "2023-03-02",
"baseline": "2023-03-18",
"port-version": 0
},
"vcpkg-get-python-packages": {
Expand Down
5 changes: 5 additions & 0 deletions versions/v-/vcpkg-cmake-get-vars.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"versions": [
{
"git-tree": "72a79de1a7eb46fb166643beb75f3fc9882b6b87",
"version-date": "2023-03-18",
"port-version": 0
},
{
"git-tree": "c6eb09f11e34173a4bfc31252d02d6aea6c25d8f",
"version-date": "2023-04-13",
Expand Down