[FLASH-406] CI run gtest cases #163

JaySon-Huang · 2019-08-07T04:07:51Z

Changes:

Comment in PR with /run-integration-tests will also run gtest. code
- introduce unittests binary:
  - unit_tests_dbms recursively glob all tests defined in gtest_*.cpp under dbms/ defined at dbms/CMakeLists.txt . Binary size is about 240 MB.
  - unit_tests_libcommon. Binary size is about 4 MB defined at libs/libcommon/src/tests/CMakeLists.txt
- Build unittests written with gtest and copy to docker image (only in Debug mode). docker/builder/build.sh
- */CMakeLists.txt mark all tests binary as EXCLUDE_FROM_ALL, so that we can ignore those binaries when running make all

TODO:
Tests define with independent main entrance can change to use gtest framework and be integrated to CI flow

JaySon-Huang · 2019-08-07T05:33:02Z

/run-integration-tests

zanmato1984 · 2019-08-07T05:56:52Z

Seems cmake has EXCLUDE_FROM_ALL option for add_subdirectory command (https://cmake.org/cmake/help/v3.0/command/add_subdirectory.html), so we'll be able to specify EXCLUDE_FROM_ALL when adding tests subdirectories and no need to specify for each individual test?

2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh

JaySon-Huang · 2019-08-07T06:23:08Z

Seems cmake has EXCLUDE_FROM_ALL option for add_subdirectory command (https://cmake.org/cmake/help/v3.0/command/add_subdirectory.html), so we'll be able to specify EXCLUDE_FROM_ALL when adding tests subdirectories and no need to specify for each individual test?

Good idea, I will change it

zanmato1984 · 2019-08-07T06:33:15Z

tests/docker/run.sh

@@ -24,7 +24,7 @@ docker-compose down

 docker-compose up -d --scale tiflash0=0 --scale tikv-learner0=0 --scale tikv0=0 --scale tidb0=0 --scale pd0=0

-docker-compose exec -T tics0 bash -c 'cd /tests ; ./run-test.sh mutable-test'
+docker-compose exec -T tics0 bash -c 'cd /tests ; ./run-test.sh mutable-test && ./run-gtest.sh '


Do our gtests rely on a running server? If not, I'd suggest shutting down all docker services and start a new container of tics with hijacked entry command (the one that runs our gtests).

dbms/src/Storages/Transaction/SchemaBuilder.cpp

JaySon-Huang · 2019-08-07T08:21:34Z

/run-integration-tests

zanmato1984 · 2019-08-07T09:21:29Z

dbms/CMakeLists.txt

@@ -266,7 +266,8 @@ if (ENABLE_TESTS)

    # attach all dbms gtest sources
    grep_gtest_sources(${ClickHouse_SOURCE_DIR}/dbms dbms_gtest_sources)
-    add_executable(unit_tests_dbms ${dbms_gtest_sources})
-    target_link_libraries(unit_tests_dbms gtest_main dbms)
+    add_executable(unit_tests_dbms EXCLUDE_FROM_ALL ${dbms_gtest_sources})


I think we should name it gtests_dbms? The same as other unit_tests_*.

These name is defined in Clickhouse, and I just use them...
Agree that gtest_* is more easy to understand, changes done.

zanmato1984 · 2019-08-07T09:24:08Z

tests/docker/docker-compose.yaml

+  # tics-gtest container is for gtests cases
+  tics-gtest:
+    image: hub.pingcap.net/tiflash/tics:${TAG:-master}
+    ports:


We don't actually need these ports and volumes for gtests, do we?

Oh maybe we need log and config, I'm not sure.

Ports and some volumes are removed. Keep config files.

tests/docker/docker-compose.yaml

JaySon-Huang · 2019-08-07T10:48:45Z

/run-integration-tests

JaySon-Huang · 2019-08-08T03:09:29Z

@zanmato1984 PTAL again

JaySon-Huang · 2019-08-08T03:55:44Z

/run-integration-tests

zanmato1984

LGTM

* [flash-365] support db filters * format * address comments. * pass tests * change config file * change config again * address comment

2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh

…g/tics into ci-run-gtest-cases-to-master

JaySon-Huang · 2019-08-08T06:29:45Z

/run-integration-tests

* ci run unittests defined using gtest * fix bug in CMakeLists * 1. disable zookeeper test cases 2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh * use EXCLUDE_FROM_ALL in add_subdirectory instead of add_executable * new container of tics just for gtest * disable tics-gtest in unused command * address comment * address comment * add continue_on_error option in run-gtest.sh * ci run unittests defined using gtest * fix bug in CMakeLists * 1. disable zookeeper test cases 2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh * use EXCLUDE_FROM_ALL in add_subdirectory instead of add_executable * new container of tics just for gtest * disable tics-gtest in unused command * address comment * address comment * add continue_on_error option in run-gtest.sh

JaySon-Huang added 2 commits August 7, 2019 11:34

ci run unittests defined using gtest

db76348

fix bug in CMakeLists

56fde86

1. disable zookeeper test cases

e645009

2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh

zanmato1984 reviewed Aug 7, 2019

View reviewed changes

dbms/src/Storages/Transaction/SchemaBuilder.cpp Show resolved Hide resolved

JaySon-Huang added 2 commits August 7, 2019 15:47

use EXCLUDE_FROM_ALL in add_subdirectory instead of add_executable

5b168fe

new container of tics just for gtest

5bbb9c1

disable tics-gtest in unused command

b378f5a

zanmato1984 reviewed Aug 7, 2019

View reviewed changes

address comment

e798b4c

zanmato1984 reviewed Aug 7, 2019

View reviewed changes

tests/docker/docker-compose.yaml Show resolved Hide resolved

JaySon-Huang added 2 commits August 8, 2019 10:43

address comment

478e76a

add continue_on_error option in run-gtest.sh

cb9cb89

zanmato1984 approved these changes Aug 8, 2019

View reviewed changes

hanfei1991 and others added 8 commits August 8, 2019 13:46

[flash-365] support db filters (pingcap#164)

8997bd3

* [flash-365] support db filters * format * address comments. * pass tests * change config file * change config again * address comment

ci run unittests defined using gtest

4534d24

fix bug in CMakeLists

4cba67d

1. disable zookeeper test cases

42b8cf2

2. use template struct SchemaBuilder instead of template class 3. more portable run-gtest.sh

use EXCLUDE_FROM_ALL in add_subdirectory instead of add_executable

dd34b4f

new container of tics just for gtest

65dbd64

disable tics-gtest in unused command

6ceff3a

address comment

2b9eaa4

JaySon-Huang added 3 commits August 8, 2019 13:52

address comment

e369ca5

add continue_on_error option in run-gtest.sh

6c9b8fe

Merge branch 'ci-run-gtest-cases-to-master' of github.com:JaySon-Huan…

5adbc6a

…g/tics into ci-run-gtest-cases-to-master

JaySon-Huang changed the title ~~CI run gtest cases~~ [FLASH-406] CI run gtest cases Aug 8, 2019

JaySon-Huang merged commit c1c75da into pingcap:master Aug 8, 2019

zanmato1984 deleted the ci-run-gtest-cases-to-master branch August 8, 2019 07:28

CalvinNeo mentioned this pull request Nov 7, 2024

update proxy of master to raftstore-proxy #9590

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLASH-406] CI run gtest cases #163

[FLASH-406] CI run gtest cases #163

JaySon-Huang commented Aug 7, 2019 •

edited

Loading

JaySon-Huang commented Aug 7, 2019

zanmato1984 commented Aug 7, 2019

JaySon-Huang commented Aug 7, 2019

zanmato1984 Aug 7, 2019

JaySon-Huang Aug 7, 2019

JaySon-Huang commented Aug 7, 2019

zanmato1984 Aug 7, 2019

JaySon-Huang Aug 7, 2019

zanmato1984 Aug 7, 2019

zanmato1984 Aug 7, 2019 •

edited

Loading

JaySon-Huang Aug 7, 2019

JaySon-Huang commented Aug 7, 2019

JaySon-Huang commented Aug 8, 2019

JaySon-Huang commented Aug 8, 2019

zanmato1984 left a comment

JaySon-Huang commented Aug 8, 2019

[FLASH-406] CI run gtest cases #163

[FLASH-406] CI run gtest cases #163

Conversation

JaySon-Huang commented Aug 7, 2019 • edited Loading

JaySon-Huang commented Aug 7, 2019

zanmato1984 commented Aug 7, 2019

JaySon-Huang commented Aug 7, 2019

zanmato1984 Aug 7, 2019

Choose a reason for hiding this comment

JaySon-Huang Aug 7, 2019

Choose a reason for hiding this comment

JaySon-Huang commented Aug 7, 2019

zanmato1984 Aug 7, 2019

Choose a reason for hiding this comment

JaySon-Huang Aug 7, 2019

Choose a reason for hiding this comment

zanmato1984 Aug 7, 2019

Choose a reason for hiding this comment

zanmato1984 Aug 7, 2019 • edited Loading

Choose a reason for hiding this comment

JaySon-Huang Aug 7, 2019

Choose a reason for hiding this comment

JaySon-Huang commented Aug 7, 2019

JaySon-Huang commented Aug 8, 2019

JaySon-Huang commented Aug 8, 2019

zanmato1984 left a comment

Choose a reason for hiding this comment

JaySon-Huang commented Aug 8, 2019

JaySon-Huang commented Aug 7, 2019 •

edited

Loading

zanmato1984 Aug 7, 2019 •

edited

Loading