Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Click house/v22.5 #4

Closed
wants to merge 41 commits into from
Closed

Click house/v22.5 #4

wants to merge 41 commits into from

Conversation

rschu1ze
Copy link
Member

No description provided.

mkruskal-google and others added 30 commits April 24, 2023 11:22
…ig-22

Fix declared dependencies for pkg-config
Without any weak ordering, btree_map fails to compile on windows 2022.

Closes protocolbuffers#12289

PiperOrigin-RevId: 527019995
…2584)

* Mark update script executable

* Squashed 'third_party/utf8_range/' changes from 72c943dea..cd1507d14

cd1507d14 Fix typo in pkg-config file to properly link the utf8_range libraries
3f179d134 Add pkg-config file declaring Abseil dependency of utf8_range

git-subtree-dir: third_party/utf8_range
git-subtree-split: cd1507d1479815fbcd8ff24dc05a978a62098bae
Without this, certain failures will be silently ignored.

PiperOrigin-RevId: 527718574
fix(libprotoc): export useful symbols from .so
We found last week that Clang's support for `musttail` on 32-bit Linux does not
seem to work reliably, so let's disable it for now.

PiperOrigin-RevId: 528817403
…sttail-22

Turn off `clang::musttail` on i386
…e version specified in protobuf_deps.

PiperOrigin-RevId: 528838071
* Bump Abseil submodule to 20230125.3

* Fix gtest installation setup with submodules
PiperOrigin-RevId: 529429609
This partially addresses protocolbuffers#12201

Closes protocolbuffers#12660

COPYBARA_INTEGRATE_REVIEW=protocolbuffers#12660 from mkruskal-google:absl-update ce07c45
PiperOrigin-RevId: 529456996
These more closely follow the standard practices of our users, where dependencies are pre-installed instead of using our provided sub-modules.  This will prevent issues such as protocolbuffers#12201 from reoccuring.

Additionally, this cl bumps our Abseil dependency to the latest release, and fixes a GTest issue that went previously unnoticed.

PiperOrigin-RevId: 529490402
This should fix protocolbuffers#12374, protocolbuffers#12375, and protocolbuffers#12450. The `protobuf_PROTOC_EXEC` variable
is not defined, and I think `protobuf::protoc` is what we should be using
instead.

PiperOrigin-RevId: 525591320
This also downgrades our tests to use cmake 3.10, compatible with our support window.

Closes protocolbuffers#12672

PiperOrigin-RevId: 529840763
Note: gcc only supports docker images down to 9.5, and the 7.3 image is very old and problematic.  A follow-up change might enable testing for GCC 7.3, which is our minimal supported version
PiperOrigin-RevId: 529885733
mkruskal-google and others added 11 commits May 5, 2023 23:03
…es-22

Backport increased C++ test coverage + fixes
When the protobuf libraries have been compiled as shared libraries the users of the library need to add `-DPROTOBUF_USE_DLLS` to their build line. Otherwise some symbols are missing.

Fixes protocolbuffers#12699

FWIW, I am not sure this is an ideal fix.  It may be better to fix the headers such that no macros change the ABI.

Closes protocolbuffers#12700

COPYBARA_INTEGRATE_REVIEW=protocolbuffers#12700 from coryan:fix-define-protobuf-use-dlls-in-pkg-config-file 13c792e
PiperOrigin-RevId: 530116678
My sketch to fix protocolbuffers#12672 was wrong.  This works for realsies.

Closes protocolbuffers#12698

COPYBARA_INTEGRATE_REVIEW=protocolbuffers#12698 from coryan:fix-cmake-typo-in-string-join-workaround 182d2e2
PiperOrigin-RevId: 530116824
On Wndows, `size_t` is 64-bits, and `int` is 32-bits. That makes conversions from `size_t` to `int` potentially lossy, and they generate warnings.  In this case an `int` variable was assigned to `size_t` and then passed to functions consuming `int`. Seems simpler to use `auto` and avoid these problems altogether.

Closes protocolbuffers#12701

COPYBARA_INTEGRATE_REVIEW=protocolbuffers#12701 from coryan:fix-warnings-repeated-field-warnings-in-msvc b1ec34d
PiperOrigin-RevId: 530134611
Warnings in header files can be a problem for consumers that enable `/WX` (or `-Werror`).  In this case, using `... & -align` produces a warning (C4146) with MSVC. The fix is to use equivalent expression `...  & ~(align - 1)`, which was already used in the same file.

Fixes protocolbuffers#12675

Closes protocolbuffers#12697

COPYBARA_INTEGRATE_REVIEW=protocolbuffers#12697 from coryan:fix-msvc-warnings-in-arena-align 835f3b4
PiperOrigin-RevId: 530137165
Cherry pick various portability fixes from mainline
This crash was due to the fact that we were passing `nullptr` as a `const
char*` parameter and relying on that implicitly converting into an empty
`absl::string_view`. `absl::string_view` supports that functionality, but
starting with C++17 its behavior changes since it's just a type alias for
`std::string_view`. `std::string_view` does not have any special conversion for
nullptr and so we were just getting crashes.

PiperOrigin-RevId: 530431663
@rschu1ze rschu1ze closed this Nov 12, 2023
@CLAassistant
Copy link

CLAassistant commented Nov 12, 2023

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 9 committers have signed the CLA.

✅ rschu1ze
❌ mkruskal-google
❌ deannagarcia
❌ acozzette
❌ coryan
❌ protobuf-github-bot
❌ protobuf-team-bot
❌ zhangskz
❌ fowles
You have signed the CLA already but the status is still pending? Let us recheck it.

rschu1ze pushed a commit that referenced this pull request Nov 12, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit that referenced this pull request Nov 15, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit that referenced this pull request Nov 16, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit that referenced this pull request Nov 17, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants