test_conv_embed_db (ctest -> gtest) #2168

alexandraBara · 2023-05-25T17:30:10Z

Moving part of the conv_2d test to gtest.
Test is automatically picked up by gtest.

test/gtest/conv_2d_implicit_gemm.cpp

test/gtest/conv_2d.hpp

test/gtest/conv_2d_implicit_gemm.cpp

test/CMakeLists.txt

atamazov

@alexandraBara Sorry if I am too soon but I hopt you'll find this useful.

test/gtest/conv_embed_db.cpp

…essed other comments

atamazov · 2023-06-07T20:24:10Z

@JehandadKhan Hmm... Do we really want to move test_conv_embed_db (which should be run only when we embed databases) under the gtest's jurisdiction?

atamazov · 2023-06-07T20:26:06Z

@alexandraBara Please rename this PR. It moves test_conv_embed_db.

atamazov · 2023-06-22T15:12:57Z

@xinlipn

CI is still running "Performance Tests - gfx90a", which should be a residual issue and has supposedly been disabled.

Maybe. We need a kind of approval from @junliume that this PR has passed CI tests.

atamazov · 2023-06-22T15:14:41Z

@xinlipn BTW testing is not covered in the PR description. I means testing of the test itself, does it perform as expected.

Have you run this test with MIOPEN_LOG_LEVEL=5 and checked if the log matches your expectations?

atamazov

Some final review comments.

test/gtest/conv_embed_db.cpp

atamazov · 2023-06-22T22:51:01Z

test/gtest/conv_embed_db.cpp

+
+std::string GetFloatArg()
+{
+    static const auto tmp = miopen::GetEnv("MIOPEN_TEST_FLOAT_ARG");


Please reuse env.hpp from the library.

test/gtest/conv_embed_db.cpp

atamazov · 2023-06-22T23:07:40Z

test/gtest/conv_embed_db.cpp

+#if MIOPEN_EMBED_DB
+
+    const auto& handle = get_handle();
+    if((handle.GetDeviceName() == "gfx900" || handle.GetDeviceName() == "gfx906") &&


Factor out device checks to a function.

Suggested change

if((handle.GetDeviceName() == "gfx900" || handle.GetDeviceName() == "gfx906") &&

if(IsTestSupportedForDevice(handle) &&

atamazov · 2023-06-23T18:46:06Z

test/gtest/conv_embed_db.cpp

+    case miopenInt8x4:
+    case miopenInt32:
+    case miopenDouble:
+        MIOPEN_THROW(miopenStatusBadParm,


Please do not use MIOPEN_THROW in tests. This one is for library.

just fail the test

test/gtest/conv_embed_db.cpp

atamazov · 2023-06-23T23:26:34Z

test/gtest/conv_embed_db.cpp

+#if MIOPEN_EMBED_DB
+
+    const auto& handle = get_handle();
+    const auto& envVar = miopen::GetEnv("MIOPEN_TEST_FLOAT_ARG");


Please use miopen::GetStringEnv() as it implements caching. Example:

MIOPEN_DECLARE_ENV_VAR(MIOPEN_TEST_FLOAT_ARG) ... static std::string GetFloatArg() { const char* p = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{}); if(p == nullptr) { // Fail the test (preferred) or return "--float" by default. } return {p}; }

This matches the proven pattern that we use in MIOpen when we need to read a string.

atamazov · 2023-06-23T23:37:57Z

@xinlipn
Please note the following comments:

These were hidden as resolved which is not actually so. Due to that I had do duplicate some review comments. Let's hide review comments after they explicitly marked [Resolved], if possible.

Please also provide an answer to #2168 (comment). Thank you!

xinlipn · 2023-06-24T04:01:19Z

@xinlipn BTW testing is not covered in the PR description. I means testing of the test itself, does it perform as expected.

Have you run this test with MIOPEN_LOG_LEVEL=5 and checked if the log matches your expectations?

Will upload

@xinlipn BTW testing is not covered in the PR description. I means testing of the test itself, does it perform as expected.

Have you run this test with MIOPEN_LOG_LEVEL=5 and checked if the log matches your expectations?

@atamazov , CTest and gTest cover the same test cases, after having it reviewed by JD, there are a few issues, e.g. warning message that test_drive was called more than once. Have been working on them...

atamazov

Some refactoring is highly recommended.

atamazov · 2023-06-27T23:41:34Z

test/gtest/conv_embed_db.cpp

+    if(IsTestSupportedForDevice(handle) &&
+       (p_envVar != nullptr && std::strcmp(p_envVar, "--float") == 0))


[R] This can be factored out to a function, like this:

Suggested change

if(IsTestSupportedForDevice(handle) &&

(p_envVar != nullptr && std::strcmp(p_envVar, "--float") == 0))

if(IsTestSupportedForDevice(handle) && IsTestRunWith("--float"))

And of course you do not need the following definition above

const char* const p_envVar = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{});

Proposed function:

static bool IsTestRunWith(const char* float_arg) { assert(float_arg != nullptr); const char* const p = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{}) return p != nullptr && std::strcmp(p, float_arg) == 0; }

Done, on a side note, google test Macro ASSERT_NE(float_arg, nullptr) can only be used in a void function, otherwise there will be a compile error

"cannot initialize return object of type 'bool' with an rvalue of type 'void'"

atamazov · 2023-06-27T23:41:55Z

test/gtest/conv_embed_db.cpp

+    const char* const p_envVar = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{});
+
+    if(IsTestSupportedForDevice(handle) &&
+       (p_envVar != nullptr && std::strcmp(p_envVar, "--half") == 0))


atamazov · 2023-06-27T23:42:01Z

test/gtest/conv_embed_db.cpp

+    const char* const p_envVar = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{});
+
+    if(IsTestSupportedForDevice(handle) &&
+       (p_envVar != nullptr && std::strcmp(p_envVar, "--int8") == 0))


atamazov · 2023-06-27T23:42:07Z

test/gtest/conv_embed_db.cpp

+    const char* const p_envVar = miopen::GetStringEnv(MIOPEN_TEST_FLOAT_ARG{});
+
+    if(IsTestSupportedForDevice(handle) &&
+       (p_envVar != nullptr && std::strcmp(p_envVar, "--bfloat16") == 0))


atamazov · 2023-06-28T22:14:15Z

Almost done. Two things left:

test_conv_embed_db (ctest -> gtest) #2168 (comment)
resolve merge conflicts

xinlipn · 2023-07-05T17:42:40Z

@xinlipn BTW testing is not covered in the PR description. I means testing of the test itself, does it perform as expected.

Have you run this test with MIOPEN_LOG_LEVEL=5 and checked if the log matches your expectations?

@atamazov , there was some discrepancies in terms of initiated solvers for test coverage between develop branch and this branch, e.g. Forward convolution: ConvAsmImplicitGemmV4R1DynamicFwd_1x1 in Develop vs ConvHipImplicitGemmV4R1Fwd in current branch for
Input tensor: 128, 512, 7, 7
Weights tensor: 2048, 512, 1, 1
Output tensor: 128, 2048, 7, 7

Took some time to remove the discrepancies by updated cmake flags on Develop branch.

Log from Develop branch:

devleop_make_test_conv_embed_db_stdout.log

Log from current branch

branch_ctest.log

Please note MIOPEN_LOG_LEVEL=5 has been enabled, but it has affected the gtest serial spew. I have been working on this, which may need a separate ticket.

Request re-review

alexandraBara and others added 12 commits May 13, 2023 00:12

conv_2d_wrapper gtest

89c8fe2

pulling in env vars

dc3b8b9

sending tuple values

6f1d5b0

updates to env vars

ba9cd2a

stdout test capture to fail based in stderr stream

9f3fe10

clang tidy

251bb89

MIOPEN_FLOAT_TEST_ARG assigments to all gtests

61e49ec

added all precision based tests

17aedb8

some rework

1b5e8c1

renamed file

f5d3cf1

code cleanup

c5cbb1a

Merge branch 'develop' into alex_gtest

8a8176d

alexandraBara added the testing label May 25, 2023

alexandraBara requested review from iq136boy, JehandadKhan and shurale-nkn May 25, 2023 17:30

alexandraBara marked this pull request as draft May 25, 2023 17:30

shurale-nkn previously requested changes May 25, 2023

View reviewed changes

test/gtest/conv_2d_implicit_gemm.cpp Outdated Show resolved Hide resolved

JehandadKhan previously requested changes May 25, 2023

View reviewed changes

test/gtest/conv_2d.hpp Outdated Show resolved Hide resolved

test/gtest/conv_2d_implicit_gemm.cpp Outdated Show resolved Hide resolved

test/gtest/conv_2d_implicit_gemm.cpp Outdated Show resolved Hide resolved

addressed reviews

ae29c5d

atamazov reviewed May 31, 2023

View reviewed changes

test/CMakeLists.txt Outdated Show resolved Hide resolved

atamazov suggested changes May 31, 2023

View reviewed changes

This comment was marked as outdated.

Sign in to view

JehandadKhan marked this pull request as ready for review June 1, 2023 17:49

xinlipn added 4 commits June 2, 2023 00:41

Cleaned up test case creation, skip tests for gfx908 and gfx90a, addr…

beb7fcb

…essed other comments

Bring back GetEnvVars(), removed duplicated wrw test

d86eb8d

Replace trivial for loop with std::transform to fix hip tidy warning

4760a5a

Cleaned up code

29afc6d

atamazov suggested changes Jun 22, 2023

View reviewed changes

Refactored code eg reusing existing API

0979c73

atamazov reviewed Jun 23, 2023

View reviewed changes

atamazov suggested changes Jun 23, 2023

View reviewed changes

Replace GetEnv() witn GetStringEnv(), MIOPEN_THROW() with FAIL()

c987d64

atamazov suggested changes Jun 27, 2023

View reviewed changes

Refactor code by creating IsTestRunWith()

20532ed

xinlipn added 3 commits July 5, 2023 11:31

Moved and renamed conv_2d.hpp in gtest folder one level up

45e89ca

Relocate conv2d.hpp

a6e6f1a

Fix static check error

444ee2b

xinlipn force-pushed the alex_gtest branch from 23c7d02 to 444ee2b Compare July 7, 2023 07:12

Fix conflicts and merge branch 'develop' into alex_gtest

ee3adb8

xinlipn added the TESTING_CI_PASSED label Jul 8, 2023

Fix conflicts and merge branch 'develop' into alex_gtest

878aa00

xinlipn added TESTING_CI_PASSED and removed TESTING_CI_PASSED labels Jul 11, 2023

xinlipn added 2 commits July 13, 2023 09:50

Fix Read the Docs build failed error in CI

35f72a8

Fix conflic and merge branch 'develop' into alex_gtest

e4b5296

junliume approved these changes Jul 16, 2023

View reviewed changes

junliume merged commit 976f8f9 into develop Jul 16, 2023

junliume deleted the alex_gtest branch August 28, 2023 23:26

atamazov mentioned this pull request Dec 24, 2023

#2599 silently removes some test executables and, consequently, the following tests from "Full test" stages: #2636

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_conv_embed_db (ctest -> gtest) #2168

test_conv_embed_db (ctest -> gtest) #2168

alexandraBara commented May 25, 2023

atamazov left a comment

This comment was marked as outdated.

atamazov commented Jun 7, 2023 •

edited

Loading

atamazov commented Jun 7, 2023 •

edited

Loading

atamazov commented Jun 22, 2023

atamazov commented Jun 22, 2023

atamazov left a comment

atamazov Jun 22, 2023

xinlipn Jun 23, 2023

atamazov Jun 23, 2023

atamazov Jun 22, 2023

xinlipn Jun 23, 2023

atamazov Jun 23, 2023

atamazov Jun 23, 2023

atamazov Jun 23, 2023

atamazov Jun 27, 2023

atamazov Jun 23, 2023

atamazov Jun 23, 2023

atamazov Jun 27, 2023

atamazov commented Jun 23, 2023

xinlipn commented Jun 24, 2023

atamazov left a comment

atamazov Jun 27, 2023 •

edited

Loading

xinlipn Jun 28, 2023

atamazov Jun 28, 2023

atamazov Jun 27, 2023

atamazov Jun 28, 2023

atamazov Jun 27, 2023

atamazov Jun 28, 2023

atamazov Jun 27, 2023

atamazov Jun 28, 2023

atamazov commented Jun 28, 2023

xinlipn commented Jul 5, 2023 •

edited

Loading

	if((handle.GetDeviceName() == "gfx900" \|\| handle.GetDeviceName() == "gfx906") &&
	if(IsTestSupportedForDevice(handle) &&

		if(IsTestSupportedForDevice(handle) &&
		(p_envVar != nullptr && std::strcmp(p_envVar, "--float") == 0))

	if(IsTestSupportedForDevice(handle) &&
	(p_envVar != nullptr && std::strcmp(p_envVar, "--float") == 0))
	if(IsTestSupportedForDevice(handle) && IsTestRunWith("--float"))

test_conv_embed_db (ctest -> gtest) #2168

test_conv_embed_db (ctest -> gtest) #2168

Conversation

alexandraBara commented May 25, 2023

atamazov left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

atamazov commented Jun 7, 2023 • edited Loading

atamazov commented Jun 7, 2023 • edited Loading

atamazov commented Jun 22, 2023

atamazov commented Jun 22, 2023

atamazov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atamazov commented Jun 23, 2023

xinlipn commented Jun 24, 2023

atamazov left a comment

Choose a reason for hiding this comment

atamazov Jun 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atamazov commented Jun 28, 2023

xinlipn commented Jul 5, 2023 • edited Loading

atamazov commented Jun 7, 2023 •

edited

Loading

atamazov commented Jun 7, 2023 •

edited

Loading

atamazov Jun 27, 2023 •

edited

Loading

xinlipn commented Jul 5, 2023 •

edited

Loading