-
Notifications
You must be signed in to change notification settings - Fork 982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Readme updates #1
Conversation
Hi, Thanks for the interest, but I think I'm going to be nixing both of these changes. We haven't decided how to technically do external contributions (we need a way to pull them back into our main repository and then mirror it out on Github), and for small changes like this, it's infinitely more pain right now than it's worth. |
@sesse Ok, I see what you mean. Until you've figured out how to take external contributors, maybe it'd be better to disable pull requests all together on this repo? |
Or maybe you can't disable pull requests... Anyways, It'd be cool if you guys figure out how to take on external contributors! |
The commit 8bfb028 (Improve zippy decompression speed) introduce a crash when snappy is compiled with _FORTIFY_SOURCE with musl libc. Backtrace reveals that it it comes from using memcpy with overlap, which is undefined behavior. We avoid the undefined behavior due to memcpy with overlap by using a temporary uint64_t. Bactrace from core dump created with `make check`: ```` (gdb) bt #0 memcpy (__n=8, __os=0xb38c8367eea, __od=0xb38c8367eeb) at /usr/include/fortify/string.h:48 google#1 snappy::(anonymous namespace)::UnalignedCopy64 (src=0xb38c8367eea, dst=0xb38c8367eeb) at snappy.cc:92 google#2 0x00006fb4c7c31717 in snappy::(anonymous namespace)::IncrementalCopy ( buf_limit=0xb38c8380ee0 "", op_limit=<optimized out>, op=<optimized out>, src=0xb38c8367eea " .\001") at snappy.cc:178 google#3 snappy::SnappyArrayWriter::AppendFromSelf (len=<optimized out>, offset=<optimized out>, this=<synthetic pointer>) at snappy.cc:1131 google#4 snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter> ( writer=<synthetic pointer>, this=0x7f1d26737050) at snappy.cc:715 google#5 snappy::InternalUncompressAllTags<snappy::SnappyArrayWriter> ( uncompressed_len=<optimized out>, writer=<synthetic pointer>, decompressor=0x7f1d26737050) at snappy.cc:799 google#6 snappy::InternalUncompress<snappy::SnappyArrayWriter> ( writer=<synthetic pointer>, r=0x7f1d26737000) at snappy.cc:789 google#7 snappy::RawUncompress (compressed=compressed@entry=0x7f1d267370c0, uncompressed=0xb38c8367ee0 " content: .\001") at snappy.cc:1149 google#8 0x00006fb4c7c3194d in snappy::RawUncompress (compressed=<optimized out>, n=<optimized out>, uncompressed=<optimized out>) at snappy.cc:1144 google#9 0x00000b38c5c6261a in snappy::BM_UFlat (iters=99, arg=<optimized out>) at snappy_unittest.cc:1371 google#10 0x00000b38c5c69846 in snappy::Benchmark::Run (this=0xb38c8323c40) at snappy-test.cc:192 google#11 0x00000b38c5c5f3fd in RunSpecifiedBenchmarks () at snappy-test.h:485 google#12 main (argc=1, argv=0x7f1d267374b8) at snappy_unittest.cc:1515 ```` Signed-off-by: Natanael Copa <natanael.copa@docker.com>
* remove -<snappy> from version having -<snappy> as part of the version makes it near impossibl to use this as a hex dependency as matchers won't match it. * change version to 1.2.0 * change makefile version * Remove versioning section from README
* Support building on windows * Delete erlang.mk * Fix travis-ci
As `i + offset` is promoted to a "negative" size_t, UBSan would complain when adding the resulting offset to `dst`: ``` /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/snappy.cc:343:43: runtime error: addition of unsigned offset to 0x6120003c5ec1 overflowed to 0x6120003c5ec0 #0 0x7f9ebd21769c in snappy::(anonymous namespace)::Copy64BytesWithPatternExtension(char*, unsigned long) /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/snappy.cc:343:43 google#1 0x7f9ebd21769c in std::__1::pair<unsigned char const*, long> snappy::DecompressBranchless<char*>(unsigned char const*, unsigned char const*, long, char*, long) /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/snappy.cc:1160:15 ```
googletest 1.13.0 requires C++14 at least, otherwise you get build errors like these. please note, that this is a source directory with some patches applied from Arch Linux to use the system-installed googletest library. cd /tmp/snappy-1.1.10 cp -v ../snappy.pc.in . '../snappy.pc.in' -> './snappy.pc.in' patch -p1 < ../cmake_add_pkgconfig.patch patching file CMakeLists.txt Hunk google#1 succeeded at 209 (offset 22 lines). Hunk google#2 succeeded at 423 (offset 22 lines). patch -p1 < ../system_gtest.patch patching file CMakeLists.txt Hunk google#2 succeeded at 320 (offset 28 lines). Hunk google#3 succeeded at 369 (offset 28 lines). patch -p1 < ../reenable_rtti.patch patching file CMakeLists.txt mkdir build-demo cd build-demo CXXFLAGS+=" -DNDEBUG" \ cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=/usr/lib -DBUILD_SHARED_LIBS=no -DSNAPPY_USE_BUNDLED_GTEST=OFF -DSNAPPY_USE_BUNDLED_BENCHMARK_LIB=OFF -- The C compiler identification is GNU 13.1.0 -- The CXX compiler identification is GNU 13.1.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for sys/mman.h -- Looking for sys/mman.h - found -- Looking for sys/resource.h -- Looking for sys/resource.h - found -- Looking for sys/time.h -- Looking for sys/time.h - found -- Looking for sys/uio.h -- Looking for sys/uio.h - found -- Looking for unistd.h -- Looking for unistd.h - found -- Looking for windows.h -- Looking for windows.h - not found -- Looking for zlibVersion in z -- Looking for zlibVersion in z - found -- Looking for lzo1x_1_15_compress in lzo2 -- Looking for lzo1x_1_15_compress in lzo2 - found -- Looking for LZ4_compress_default in lz4 -- Looking for LZ4_compress_default in lz4 - found -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX - Failed -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX2 -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX2 - Failed -- Performing Test HAVE_CLANG_MAVX -- Performing Test HAVE_CLANG_MAVX - Success -- Performing Test HAVE_CLANG_MBMI2 -- Performing Test HAVE_CLANG_MBMI2 - Success -- Performing Test SNAPPY_HAVE_NO_MISSING_FIELD_INITIALIZERS -- Performing Test SNAPPY_HAVE_NO_MISSING_FIELD_INITIALIZERS - Success -- Performing Test HAVE_BUILTIN_EXPECT -- Performing Test HAVE_BUILTIN_EXPECT - Success -- Performing Test HAVE_BUILTIN_CTZ -- Performing Test HAVE_BUILTIN_CTZ - Success -- Performing Test HAVE_ATTRIBUTE_ALWAYS_INLINE -- Performing Test HAVE_ATTRIBUTE_ALWAYS_INLINE - Failed -- Performing Test SNAPPY_HAVE_SSSE3 -- Performing Test SNAPPY_HAVE_SSSE3 - Failed -- Performing Test SNAPPY_HAVE_X86_CRC32 -- Performing Test SNAPPY_HAVE_X86_CRC32 - Failed -- Performing Test SNAPPY_HAVE_NEON_CRC32 -- Performing Test SNAPPY_HAVE_NEON_CRC32 - Failed -- Performing Test SNAPPY_HAVE_BMI2 -- Performing Test SNAPPY_HAVE_BMI2 - Failed -- Performing Test SNAPPY_HAVE_NEON -- Performing Test SNAPPY_HAVE_NEON - Failed -- Looking for mmap -- Looking for mmap - found -- Looking for sysconf -- Looking for sysconf - found -- Found PkgConfig: /usr/bin/pkg-config (found version "1.9.5") -- Checking for one of the modules 'gtest_main' -- Checking for one of the modules 'benchmark' -- Configuring done (5.7s) -- Generating done (0.0s) -- Build files have been written to: /tmp/snappy-1.1.10/build-demo make -w -j 1 VERBOSE=1 make: Entering directory '/tmp/snappy-1.1.10/build-demo' /usr/bin/cmake -S/tmp/snappy-1.1.10 -B/tmp/snappy-1.1.10/build-demo --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /tmp/snappy-1.1.10/build-demo/CMakeFiles /tmp/snappy-1.1.10/build-demo//CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Entering directory '/tmp/snappy-1.1.10/build-demo' make -f CMakeFiles/snappy.dir/build.make CMakeFiles/snappy.dir/depend make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' cd /tmp/snappy-1.1.10/build-demo && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/snappy-1.1.10 /tmp/snappy-1.1.10 /tmp/snappy-1.1.10/build-demo /tmp/snappy-1.1.10/build-demo /tmp/snappy-1.1.10/build-demo/CMakeFiles/snappy.dir/DependInfo.cmake --color= make[2]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make -f CMakeFiles/snappy.dir/build.make CMakeFiles/snappy.dir/build make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' [ 7%] Building CXX object CMakeFiles/snappy.dir/snappy-c.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10/build-demo -I/tmp/snappy-1.1.10 -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -MD -MT CMakeFiles/snappy.dir/snappy-c.cc.o -MF CMakeFiles/snappy.dir/snappy-c.cc.o.d -o CMakeFiles/snappy.dir/snappy-c.cc.o -c /tmp/snappy-1.1.10/snappy-c.cc [ 14%] Building CXX object CMakeFiles/snappy.dir/snappy-sinksource.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10/build-demo -I/tmp/snappy-1.1.10 -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -MD -MT CMakeFiles/snappy.dir/snappy-sinksource.cc.o -MF CMakeFiles/snappy.dir/snappy-sinksource.cc.o.d -o CMakeFiles/snappy.dir/snappy-sinksource.cc.o -c /tmp/snappy-1.1.10/snappy-sinksource.cc ... make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' [ 64%] Building CXX object CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10 -I/tmp/snappy-1.1.10/build-demo -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -DGTEST_HAS_PTHREAD=1 -MD -MT CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o -MF CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o.d -o CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o -c /tmp/snappy-1.1.10/snappy_unittest.cc In file included from /usr/include/gtest/gtest-message.h:57, from /usr/include/gtest/gtest-assertion-result.h:46, from /usr/include/gtest/gtest.h:64, from /tmp/snappy-1.1.10/snappy_unittest.cc:39: /usr/include/gtest/internal/gtest-port.h:270:2: error: #error C++ versions less than C++14 are not supported. 270 | #error C++ versions less than C++14 are not supported. | ^~~~~ In file included from /usr/include/gtest/gtest-printers.h:115, from /usr/include/gtest/gtest-matchers.h:48, from /usr/include/gtest/internal/gtest-death-test-internal.h:47, from /usr/include/gtest/gtest-death-test.h:43, from /usr/include/gtest/gtest.h:65: /usr/include/gtest/internal/gtest-internal.h:636:58: error: wrong number of template arguments (0, should be 1) 636 | typedef ::std::map<std::string, CodeLocation, std::less<>> RegisteredTestsMap; | ^ In file included from /usr/include/c++/13.1.0/string:49, from /usr/include/c++/13.1.0/random:42, from /tmp/snappy-1.1.10/snappy_unittest.cc:32: /usr/include/c++/13.1.0/bits/stl_function.h:403:12: note: provided for 'template<class _Tp> struct std::less' 403 | struct less : public binary_function<_Tp, _Tp, bool> | ^~~~ /usr/include/gtest/internal/gtest-internal.h:636:59: error: template argument 3 is invalid 636 | typedef ::std::map<std::string, CodeLocation, std::less<>> RegisteredTestsMap; | ^~ /usr/include/gtest/internal/gtest-internal.h: In member function 'bool testing::internal::TypedTestSuitePState::AddTestName(const char*, int, const char*, const char*)': /usr/include/gtest/internal/gtest-internal.h:613:23: error: request for member 'insert' in '((testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'int'} 613 | registered_tests_.insert( | ^~~~~~ /usr/include/gtest/internal/gtest-internal.h: In member function 'bool testing::internal::TypedTestSuitePState::TestExists(const std::string&) const': /usr/include/gtest/internal/gtest-internal.h:619:30: error: request for member 'count' in '((const testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'const testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'const int'} 619 | return registered_tests_.count(test_name) > 0; | ^~~~~ /usr/include/gtest/internal/gtest-internal.h: In member function 'const testing::internal::CodeLocation& testing::internal::TypedTestSuitePState::GetCodeLocation(const std::string&) const': /usr/include/gtest/internal/gtest-internal.h:623:40: error: qualified-id in declaration before 'it' 623 | RegisteredTestsMap::const_iterator it = registered_tests_.find(test_name); | ^~ /usr/include/gtest/internal/gtest-internal.h:624:5: error: 'it' was not declared in this scope; did you mean 'int'? 624 | GTEST_CHECK_(it != registered_tests_.end()); | ^~~~~~~~~~~~ /usr/include/gtest/internal/gtest-internal.h:624:5: error: request for member 'end' in '((const testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'const testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'const int'} 624 | GTEST_CHECK_(it != registered_tests_.end()); | ^~~~~~~~~~~~ /usr/include/gtest/internal/gtest-internal.h:625:12: error: 'it' was not declared in this scope; did you mean 'int'? 625 | return it->second; | ^~ | int make[2]: *** [CMakeFiles/snappy_unittest.dir/build.make:76: CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o] Error 1 make[2]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make[1]: *** [CMakeFiles/Makefile2:144: CMakeFiles/snappy_unittest.dir/all] Error 2 make[1]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make: *** [Makefile:146: all] Error 2 make: Leaving directory '/tmp/snappy-1.1.10/build-demo'
Nice to see that snappy is on github! I've done a few small changes to the readme