Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Topi][Testing] Float16 unittests for dense, conv2d, depthwise conv2d #8529

Merged
merged 6 commits into from
Aug 7, 2021

Conversation

Lunderberg
Copy link
Contributor

The goal of this PR is to add float16 test cases to topi schedules used for ResNet50. This PR is split up into independent commits, as described below, for ease of review.

[Topi][Testing] Minor cleanup for python reference implementations

  • Use input dtype for dilate/conv2d accumulate in python
    impl. Previously, the python implementations of dilation and conv2d
    would use numpy default dtype in some cases, rather than the input
    data's dtype.

  • Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).

  • Refactored to avoid duplication, use common get_pad_tuple functionality.

[Topi][UnitTests] Added float16 tests to test_topi_dense.py

[Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py

[Topi][Float16] Added float16 tests for depthwise conv2d.

@Lunderberg Lunderberg changed the title Topi float16 [Topi][Testing] Float16 unittests for dense, conv2d, depthwise conv2d Jul 22, 2021
@Lunderberg
Copy link
Contributor Author

Potential reviewers: @masahi (implemented primarily for vulkan float16 support), @tmoreau89 (float16 progress)

Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I'm afraid of flaky tests due to the rounding errors. Can we at least fix the random seed in the test to reduce the possibility?

@Lunderberg
Copy link
Contributor Author

In general I'm afraid of flaky tests due to the rounding errors. Can we at least fix the random seed in the test to reduce the possibility?

Good call, especially with the much larger relative error of float16. Change has been made.

Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Lunderberg Lunderberg force-pushed the topi_float16 branch 2 times, most recently from e249fd5 to 1b5363b Compare July 28, 2021 13:06
@Lunderberg Lunderberg force-pushed the topi_float16 branch 2 times, most recently from df89fd9 to 97a86a7 Compare August 3, 2021 14:54
@Lunderberg
Copy link
Contributor Author

Bumping to restart CI after #8656 .

- Use input dtype for dilate/conv2d accumulate in python
  impl. Previously, the python implementations of dilation and conv2d
  would use numpy default dtype in some cases, rather than the input
  data's dtype.

- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).

- Refactored to avoid duplication, use common get_pad_tuple functionality.
Intended to avoid flaky test failures later due to rounding errors.
- ref_data must be a test fixture, not acquired through
  request.getfixturevalue, in order to have the random_seed be known.

- dilate_python's return value didn't follow `out_dtype`.

- The test_topi_conv3d tests had the reference results computed in
  float64, due to dilate_python() not respecting the input data type.
  With the correct dtype, the tolerances needed to be slightly widened.
@Lunderberg
Copy link
Contributor Author

Rebased to main to restart CI. Looks like it was a timeout in the macOS compilation.

@masahi masahi merged commit bf3669d into apache:main Aug 7, 2021
@Lunderberg Lunderberg deleted the topi_float16 branch August 7, 2021 03:08
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Aug 11, 2021
…apache#8529)

* [Topi][Testing] Minor cleanup for python reference implementations

- Use input dtype for dilate/conv2d accumulate in python
  impl. Previously, the python implementations of dilation and conv2d
  would use numpy default dtype in some cases, rather than the input
  data's dtype.

- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).

- Refactored to avoid duplication, use common get_pad_tuple functionality.

* [Topi][UnitTests] Added float16 tests to test_topi_dense.py

* [Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py

* [Topi][Float16] Added float16 tests for depthwise conv2d.

* [UnitTests] Explicitly set seed for float16 tests

Intended to avoid flaky test failures later due to rounding errors.

* [UnitTests] Fixed a few failing unit tests.

- ref_data must be a test fixture, not acquired through
  request.getfixturevalue, in order to have the random_seed be known.

- dilate_python's return value didn't follow `out_dtype`.

- The test_topi_conv3d tests had the reference results computed in
  float64, due to dilate_python() not respecting the input data type.
  With the correct dtype, the tolerances needed to be slightly widened.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
ylc pushed a commit to ylc/tvm that referenced this pull request Sep 29, 2021
…apache#8529)

* [Topi][Testing] Minor cleanup for python reference implementations

- Use input dtype for dilate/conv2d accumulate in python
  impl. Previously, the python implementations of dilation and conv2d
  would use numpy default dtype in some cases, rather than the input
  data's dtype.

- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).

- Refactored to avoid duplication, use common get_pad_tuple functionality.

* [Topi][UnitTests] Added float16 tests to test_topi_dense.py

* [Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py

* [Topi][Float16] Added float16 tests for depthwise conv2d.

* [UnitTests] Explicitly set seed for float16 tests

Intended to avoid flaky test failures later due to rounding errors.

* [UnitTests] Fixed a few failing unit tests.

- ref_data must be a test fixture, not acquired through
  request.getfixturevalue, in order to have the random_seed be known.

- dilate_python's return value didn't follow `out_dtype`.

- The test_topi_conv3d tests had the reference results computed in
  float64, due to dilate_python() not respecting the input data type.
  With the correct dtype, the tolerances needed to be slightly widened.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
…apache#8529)

* [Topi][Testing] Minor cleanup for python reference implementations

- Use input dtype for dilate/conv2d accumulate in python
  impl. Previously, the python implementations of dilation and conv2d
  would use numpy default dtype in some cases, rather than the input
  data's dtype.

- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).

- Refactored to avoid duplication, use common get_pad_tuple functionality.

* [Topi][UnitTests] Added float16 tests to test_topi_dense.py

* [Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py

* [Topi][Float16] Added float16 tests for depthwise conv2d.

* [UnitTests] Explicitly set seed for float16 tests

Intended to avoid flaky test failures later due to rounding errors.

* [UnitTests] Fixed a few failing unit tests.

- ref_data must be a test fixture, not acquired through
  request.getfixturevalue, in order to have the random_seed be known.

- dilate_python's return value didn't follow `out_dtype`.

- The test_topi_conv3d tests had the reference results computed in
  float64, due to dilate_python() not respecting the input data type.
  With the correct dtype, the tolerances needed to be slightly widened.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
Anndrey24 added a commit to Anndrey24/tvm that referenced this pull request May 17, 2024
This commit adds fp16 test cases to the conv2d NHWC TOPI schedules for `arm_cpu`.
Following the example of apache#8529, the numpy reference conv2d output is computed in fp32 instead of fp16, while the absolute tolerance varies for each test case according to the size of the summed axis and the output's largest element.
ekalda pushed a commit that referenced this pull request May 22, 2024
…17007)

This commit adds fp16 test cases to the conv2d NHWC TOPI schedules for `arm_cpu`.
Following the example of #8529, the numpy reference conv2d output is computed in fp32 instead of fp16, while the absolute tolerance varies for each test case according to the size of the summed axis and the output's largest element.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants