Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] Convert operator supports u2, u3, u6 types #23490

Merged
merged 24 commits into from
Mar 31, 2024
Merged

[core] Convert operator supports u2, u3, u6 types #23490

merged 24 commits into from
Mar 31, 2024

Conversation

praasz
Copy link
Contributor

@praasz praasz commented Mar 16, 2024 •
edited
Loading

Details:

  • Add new types u2, u3, u6 to Convert operator.
  • Expand element::Iterator to support NF4 and BitProxy to support NF4 conversion.
  • Update tensor to calculate correctly byte size for u3, u6 types.
  • Fix NF4 <-> conversion to always use byte pack/unpack and quantization when convert to/from floating point. In future the conversion for NF4 will be limited to f32 -> NF4.

Tickets:

@github-actions github-actions bot added category: inference OpenVINO Runtime library - Inference category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra category: TEMPLATE OpenVINO Template plugin category: CPP API OpenVINO CPP API bindings labels Mar 16, 2024
@praasz
Copy link
Contributor Author

praasz commented Mar 16, 2024 •
edited
Loading

Blocked by PR #23279 .

@praasz praasz added this to the 2024.1 milestone Mar 16, 2024
@praasz praasz force-pushed the feature/add-u2-u3-u6-to-convert branch from 0afc7c0 to 23a0e32 Compare March 18, 2024 21:26
@github-actions github-actions bot removed category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra category: CPP API OpenVINO CPP API bindings labels Mar 18, 2024
@praasz praasz marked this pull request as ready for review March 18, 2024 21:40
@praasz praasz requested review from a team as code owners March 18, 2024 21:40
@mlukasze
Copy link
Contributor

Blocked by PR #23279 .

blocker has been merged

mitruska
mitruska approved these changes Mar 25, 2024
View reviewed changes
src/core/src/op/convert.cpp Show resolved Hide resolved
@praasz praasz added this pull request to the merge queue Mar 26, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 26, 2024
@praasz praasz added this pull request to the merge queue Mar 26, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 26, 2024
@praasz praasz added this pull request to the merge queue Mar 26, 2024
@akladiev akladiev removed this pull request from the merge queue due to the queue being cleared Mar 26, 2024
@praasz praasz enabled auto-merge March 26, 2024 15:21
@praasz praasz force-pushed the feature/add-u2-u3-u6-to-convert branch from 05691fd to b601124 Compare March 29, 2024 12:54
@praasz praasz added this pull request to the merge queue Mar 31, 2024
Merged via the queue into openvinotoolkit:master with commit bb20710 Mar 31, 2024
108 checks passed
@praasz praasz deleted the feature/add-u2-u3-u6-to-convert branch March 31, 2024 08:45
bbielawx pushed a commit to bbielawx/openvino that referenced this pull request Apr 12, 2024
@praasz @mlukasze @bbielawx
[core] Convert operator supports u2, u3, u6 types (openvinotoolkit#…
4c396bd 
…23490)

### Details:
 - Add new types `u2, u3, u6` to Convert operator.
- Expand `element::Iterator` to support NF4 and `BitProxy` to support
NF4 conversion.
 - Update tensor to calculate correctly byte size for `u3, u6` types.
- Fix NF4 <-> conversion to always use byte pack/unpack and quantization
when convert to/from floating point. In future the conversion for NF4
will be limited to f32 -> NF4.

### Tickets:
 - [CVS-127000](https://jira.devtools.intel.com/browse/CVS-127000)
 - [CVS-128024](https://jira.devtools.intel.com/browse/CVS-128024)

---------

Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
alvoron pushed a commit to alvoron/openvino that referenced this pull request Apr 29, 2024
@praasz @mlukasze @alvoron
[core] Convert operator supports u2, u3, u6 types (openvinotoolkit#…
3344e63 
…23490)

### Details:
 - Add new types `u2, u3, u6` to Convert operator.
- Expand `element::Iterator` to support NF4 and `BitProxy` to support
NF4 conversion.
 - Update tensor to calculate correctly byte size for `u3, u6` types.
- Fix NF4 <-> conversion to always use byte pack/unpack and quantization
when convert to/from floating point. In future the conversion for NF4
will be limited to f32 -> NF4.

### Tickets:
 - [CVS-127000](https://jira.devtools.intel.com/browse/CVS-127000)
 - [CVS-128024](https://jira.devtools.intel.com/browse/CVS-128024)

---------

Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@t-jankowski t-jankowski t-jankowski approved these changes

@mitruska mitruska mitruska approved these changes

@AlexKoff88 AlexKoff88 Awaiting requested review from AlexKoff88

@vurusovs vurusovs Awaiting requested review from vurusovs

Assignees
No one assigned
Labels
category: Core OpenVINO Core (aka ngraph) category: inference OpenVINO Runtime library - Inference category: TEMPLATE OpenVINO Template plugin Code Freeze
Projects
None yet
Milestone
2024.1
Development

Successfully merging this pull request may close these issues.
4 participants
@praasz @mlukasze @mitruska @t-jankowski