Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Opset13][FP8] FakeConvert evaluate and reference impl #21034

Merged
merged 62 commits into from
Nov 24, 2023
Merged

[Opset13][FP8] FakeConvert evaluate and reference impl #21034

merged 62 commits into from
Nov 24, 2023

Conversation

mitruska
Copy link
Contributor

@mitruska mitruska commented Nov 13, 2023 •
edited
Loading

Details:

Based on: POC by @andreyanufr

  • Enabled types: fp32, fp16, bf16
  • Created reference function and moved the code from the op class and evaluate
  • Updated Evaluate approach
  • destination_type as string is a temp solution, can be updated to ov::element_type when possible
  • Added tests

The following mapping for the POC names can be applied:

  • f8e4m3 -> "HF8" -> convertfp16_hf8_bias7
  • f8e5m2 -> "BF8" -> convertfp16_bf8
  • Usage of simple loop for single element scale and per channel scale (aligned with POC),
    added usage of autobroadcast_select to enable other numpy broadcast cases.

Follow ups TO DO:

  • More tests for data/scale/shift different shapes
  • Tests refactor (Move the tests from eval to template tests)

Spec PR:

Tickets:

  • 119976

@github-actions github-actions bot added category: Core OpenVINO Core (aka ngraph) category: IE Tests OpenVINO Test: plugins and common category: CPP API OpenVINO CPP API bindings labels Nov 13, 2023
@mitruska mitruska added the WIP work in progress label Nov 13, 2023
mitruska
mitruska commented Nov 15, 2023
View reviewed changes
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
src/core/src/op/fake_convert.cpp Outdated Show resolved Hide resolved
@mitruska
Add var to keep scale_shift lambda
692a1de
@mitruska
Use lamda only for autobroadcast
dcdc814
@mitruska
More checks for per channel scale
0f6ff15
@mitruska
Remove union half_t
26a34ff
@mitruska
Change template to fp16 type for emulate
378e827
@mitruska
Use f8e4m3_min_val as constexpr
3a34460
@mitruska
Update unsigned short to uint16_t
a23ae00
@mitruska
Minor style refactor
1761035
@mitruska
Minor comments update
01c8037
@mitruska
Update fake_convert_details namespace to func
c78802b
@mitruska
Update apply_conversion return type
d88e87c
@mitruska
Add doxygen text
90a2011
@mitruska
Update supported type assert
c996766
@mitruska
Remove duplicated tests
16c710e
@mitruska
Update namespace move to cpp
08a0049
@mitruska
Fill init with zeroes instead of reserve
bff2a8f
@mitruska
Use add div mul sub reference for applying scale
ad5bfa3
@mitruska
Use autobroadcast_select for applying scale
3292b62
@mitruska
Bring back opt broacast cases
b825749
@mitruska
Merge remote-tracking branch 'upstream/master' into mitruska/fake_con…
a5fae1c 
…vert_eval
@mitruska
Reuse scale_shift as func
690880a
@mitruska
Adjust s and o var names
3bb5473
@mitruska
Update multiplication in loop tu accumulate
e7f5c96
praasz
praasz approved these changes Nov 23, 2023
View reviewed changes
src/core/src/op/fake_convert.cpp Show resolved Hide resolved
src/core/reference/src/op/fake_convert.cpp Show resolved Hide resolved
@nikita-malininn nikita-malininn mentioned this pull request Nov 23, 2023
Merged
@mitruska mitruska merged commit 63e7e2d into openvinotoolkit:master Nov 24, 2023
nikita-malininn added a commit to openvinotoolkit/nncf that referenced this pull request Dec 15, 2023
@nikita-malininn
[PTQ][OV] FP8 implementation (#2283)
5f2c20e 
### Changes

- Added FP8 implementation
- Added `Mode` parameter

### Reason for changes

- New FP8 implementation

### Related tickets

- 119805

### Tests

- `tests/openvino/native/quantization/test_graphs.py`
- `tests/openvino/native/test_model_transformer.py`

On top of openvinotoolkit/openvino#21034 -
**Merged**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@praasz praasz praasz approved these changes

@andreyanufr andreyanufr andreyanufr approved these changes

@AlexKoff88 AlexKoff88 Awaiting requested review from AlexKoff88

@mmikolajcz mmikolajcz Awaiting requested review from mmikolajcz

Assignees

@mitruska mitruska

Labels
category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings
Projects
None yet
Milestone
2023.3
Development

Successfully merging this pull request may close these issues.
4 participants
@mitruska @mmikolajcz @praasz @andreyanufr