Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attempt to fix Stress test (MSan) #17344

Merged
merged 1 commit into from
Nov 24, 2020
Merged

Attempt to fix Stress test (MSan) #17344

merged 1 commit into from
Nov 24, 2020

Conversation

nikitamikhaylov
Copy link
Member

@nikitamikhaylov nikitamikhaylov commented Nov 23, 2020 •
edited
Loading

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
PODArray does not initialize "new" elements when resizing, unlike std::vector. This probably fixes this failure from #17309.

@robot-clickhouse robot-clickhouse added the pr-build Pull request with build/testing/packaging improvement label Nov 23, 2020
@nikitamikhaylov
Copy link
Member Author 

==255==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x10dcb857 in DB::parseIPv6(char const*, unsigned char*) (/usr/bin/clickhouse+0x10dcb857)
    #1 0x10dc9aa3 in DB::FunctionIPv6StringToNum::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x10dc9aa3)
    #2 0x106cca8c in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x106cca8c)
    #3 0x106cbe7c in DB::DefaultExecutable::executeDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x106cbe7c)
    #4 0x1079ee55 in DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079ee55)
    #5 0x1079d25d in DB::ExecutableFunctionAdaptor::defaultImplementationForConstantArguments(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079d25d)
    #6 0x1079ebd3 in DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079ebd3)
    #7 0x107a1133 in DB::ExecutableFunctionAdaptor::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x107a1133)
    #8 0x28f36964 in DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node*, std::__1::allocator<DB::ActionsDAG::Node*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsDAG.cpp:211:35
    #9 0x28f3395c in DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::Context const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsDAG.cpp:165:12
    #10 0x2a791cc0 in DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:501:51
    #11 0x2a7a1c24 in DB::ActionsMatcher::Data::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.h:155:27
    #12 0x2a7a1c24 in DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:938:14
    #13 0x2a793fe2 in DB::ActionsMatcher::visit(std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:546:9
    #14 0x2a7ace8f in DB::ActionsMatcher::visit(DB::ASTExpressionList&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:654:17
    #15 0x2a79436c in DB::ActionsMatcher::visit(std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:550:9
    #16 0x2a74d28f in DB::InDepthNodeVisitor<DB::ActionsMatcher, true, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InDepthNodeVisitor.h:34:13
    #17 0x2a70d528 in DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ActionsDAG>&, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.cpp:400:48
    #18 0x2a72b1ab in DB::SelectQueryExpressionAnalyzer::appendSelect(DB::ExpressionActionsChain&, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.cpp:853:5
    #19 0x2a7409c2 in DB::ExpressionAnalysisResult::ExpressionAnalysisResult(DB::SelectQueryExpressionAnalyzer&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool, bool, bool, std::__1::shared_ptr<DB::FilterInfo> const&, DB::Block const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.cpp:1225:24
    #20 0x29f201e5 in DB::InterpreterSelectQuery::getSampleBlockImpl() /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:517:23
    #21 0x29f0d9a0 in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&)::$_2::operator()(bool) const /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:405:25
    #22 0x29efaf68 in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:408:5
    #23 0x29ef58f9 in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:147:7
    #24 0x2a80c095 in std::__1::__unique_if<DB::InterpreterSelectQuery>::__unique_single std::__1::make_unique<DB::InterpreterSelectQuery, std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&>(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3028:32
    #25 0x2a80c095 in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:84:42
    #26 0x29dc4762 in std::__1::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::__1::make_unique<DB::InterpreterSelectWithUnionQuery, std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions>(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions&&) /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3028:32
    #27 0x29dc4762 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::QueryProcessingStage::Enum) /build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:107:16
    #28 0x2ae5cba3 in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:462:28
    #29 0x2ae58094 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:813:30
    #30 0x2cb88d02 in DB::TCPHandler::runImpl() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:254:24
    #31 0x2cbb4bd5 in DB::TCPHandler::run() /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1334:9
    #32 0x37a13aea in Poco::Net::TCPServerConnection::start() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
    #33 0x37a15109 in Poco::Net::TCPServerDispatcher::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20
    #34 0x37ea34b2 in Poco::PooledThread::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
    #35 0x37e9e898 in Poco::(anonymous namespace)::RunnableHolder::run() /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:55:11
    #36 0x37e9ad2b in Poco::ThreadImpl::runnableEntry(void*) /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
    #37 0x7f248a565668 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9668)
    #38 0x7f248a47c2b2 in clone (/lib/x86_64-linux-gnu/libc.so.6+0x1222b2)

  Uninitialized value was created by a heap allocation
    #0 0x860c46d in malloc (/usr/bin/clickhouse+0x860c46d)
    #1 0x87d2727 in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/Allocator.h:218:27
    #2 0x883c3d5 in Allocator<false, false>::alloc(unsigned long, unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
    #3 0x883c3d5 in void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::alloc<>(unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/PODArray.h:120:64
    #4 0x883c3d5 in void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::realloc<>(unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/PODArray.h:142:13
    #5 0x883c3d5 in void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::reserve<>(unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/PODArray.h:223:13
    #6 0x883c3d5 in void DB::PODArrayBase<1ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::resize<>(unsigned long) /build/obj-x86_64-linux-gnu/../src/Common/PODArray.h:236:9
    #7 0x1f194e7d in DB::FunctionBase64Conversion<DB::TryBase64Decode>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x1f194e7d)
    #8 0x106cca8c in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x106cca8c)
    #9 0x106cbe7c in DB::DefaultExecutable::executeDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x106cbe7c)
    #10 0x1079ee55 in DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079ee55)
    #11 0x1079d25d in DB::ExecutableFunctionAdaptor::defaultImplementationForConstantArguments(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079d25d)
    #12 0x1079ebd3 in DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x1079ebd3)
    #13 0x107a1133 in DB::ExecutableFunctionAdaptor::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const (/usr/bin/clickhouse+0x107a1133)
    #14 0x28f36964 in DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node*, std::__1::allocator<DB::ActionsDAG::Node*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsDAG.cpp:211:35
    #15 0x28f3395c in DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, DB::Context const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsDAG.cpp:165:12
    #16 0x2a791cc0 in DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:501:51
    #17 0x2a7a1c24 in DB::ActionsMatcher::Data::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.h:155:27
    #18 0x2a7a1c24 in DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:938:14
    #19 0x2a793fe2 in DB::ActionsMatcher::visit(std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:546:9
    #20 0x2a79a2be in DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:850:13
    #21 0x2a793fe2 in DB::ActionsMatcher::visit(std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:546:9
    #22 0x2a7ace8f in DB::ActionsMatcher::visit(DB::ASTExpressionList&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:654:17
    #23 0x2a79436c in DB::ActionsMatcher::visit(std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) /build/obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:550:9
    #24 0x2a74d28f in DB::InDepthNodeVisitor<DB::ActionsMatcher, true, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) /build/obj-x86_64-linux-gnu/../src/Interpreters/InDepthNodeVisitor.h:34:13

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/usr/bin/clickhouse+0x10dcb857) in DB::parseIPv6(char const*, unsigned char*)

@nikitamikhaylov nikitamikhaylov merged commit 44e4a4f into ClickHouse:master Nov 24, 2020
@nikitamikhaylov
Copy link
Member Author

Noticed, that there is performance degradation.

@nikitamikhaylov nikitamikhaylov mentioned this pull request Nov 24, 2020
Merged
@alexey-milovidov
Copy link
Member

If the error is bogus, you can leave the code if it's compiled with MSan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
pr-build Pull request with build/testing/packaging improvement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nikitamikhaylov @alexey-milovidov @robot-clickhouse