Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ort openvino npu 1.17 master #19966

Merged
merged 59 commits into from
Mar 22, 2024
Merged

Ort openvino npu 1.17 master #19966

merged 59 commits into from
Mar 22, 2024

Conversation

sfatimar
Copy link
Contributor

Description

Add NPU to list of device supported.
Added changes for Support to OV 2024.0
Nuget packages removes packaging of OpenVINO DLL
Bug Fixes with Python API
Reverted Dockerfiles not being maintained.

Motivation and Context

NPU Device has been introduced by Intel in latest client systems
OpenVINO 2024.0 release is out.

sspintel and others added 30 commits March 12, 2024 23:14
@sspintel
Add OpenVINO 2023.2 support
9e9c1fb
@sspintel
num_of_threads mapped to inference_num_threads property of CPU
f71ca3a
@sspintel
Use ONNX FrontEnd convert_model to facilitate external weights path
b656d06
@sspintel
Fix Lint issues
cce97c0
@sspintel
inference_num_threadas is applicable only for the CPU device
babc244
@sspintel
Fix Lint issues
b6e4af0
@preetha-intel @sspintel
Add an user option to disable default dynamic model execution for imp…
6b8c867 
…roved performance
@sspintel
Fix for runtime errors with dynamic shapes flag
4f7fe28
@sspintel
Fix conflict in gitignore
6704415
@sspintel
Revert removal of USE_OPENVINO macro in provider registration; Add ch…
359b71b 
…ecks for proxy plugins for NUM_STREAMS
@preetha-intel @sspintel
Enable OV CPU fallback for NPU compilation failures
fb6d981
@preetha-intel @sspintel
Add NPU device in supported list of openvino devices
d2879b9
@preetha-intel @sspintel
Handle dynamic shapes fallback for NPU to OV CPU
b7db23f
@preetha-intel @sspintel
Remove NPU operator from static mapping
08f5b8c
@sspintel
Add NPU device; Revert num_of_threads to 1 to be default
d9fe4c1
@sspintel
Add support for LayerNormalization Op; NPU to go through ReadModel ->…
ff7070b 
… CompileModel stages
@sspintel
Fix an issue with provider options getting overwritten
b4b2c59 
Since there's a common GlobalContext, when we create two Sessions in the same user code,
the provider options of the first model gets overwritten with the second model's.

To overcome this, this PR creates an instance of GlobalContext for each instance of
OpenVINOExecutionProvider and uses it throughout wherever access to the GlobalContext's parameters are required.
@sspintel
Add device_precision access for UnsupportedOpModes
4121684
@sspintel
Fix an issue that shared global_context across subgraphs
e8fd60a
@sspintel
Remove static mapping of LayerNorm op for the NPU; Remove unused MLAS…
439afd3 
… fallback code
@sspintel
Add support for UINT16 DTYPE in initializers, NPU, and CPU devices
593d870
@sspintel
Temporarily disable model domain check as it is yet to be supported b…
de8194c 
…y the onnx frontend
@sspintel
Allow overriding NPU compiler type through an environmental variable
8fe5760
@sspintel
Remove deprecated model domain check
281ddf6
@sspintel
Remove unused parameter op_map
49d7f4a
@preetha-intel @sspintel
Enable dynamic backend execution for NPU
0466f29
@preetha-intel @sspintel
OV NPU fallback for OV CPU
e4cc1c9
@hmamidix @sspintel
The default should be false
da26118
@hmamidix @sspintel
Resetting num of threads to 0
6adbc90
@vthaniel @sspintel
Nuget does not need openvino windows dll
16c9a3e
@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@sfatimar
Copy link
Contributor Author

Seems like one of unit test is giving accuracy mismatch. It worked fine in our systems. Checking on this

@sfatimar
Copy link
Contributor Author

@jywu-msft Disabled the unit test , Can you please re run

@sfatimar sfatimar mentioned this pull request Mar 21, 2024
Closed
@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@sfatimar
Copy link
Contributor Author

The pipeline failed again :( . This time for Gelu Python Tests. One of the attributes approximate in Gelu is "none" in ONNX however we had defined it as "" string . Since it was part of latest ONNX Test suite and our onnx was not updated, this occurred. Hopefully it will pass this time.

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, onnxruntime-python-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed

@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Big Models, Linux QNN CI Pipeline, Windows ARM64 QNN CI Pipeline, Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@jywu-msft jywu-msft merged commit eab35c2 into microsoft:main Mar 22, 2024
69 checks passed
This was referenced Mar 24, 2024
Closed
Closed
Open
mszhanyi pushed a commit that referenced this pull request Mar 26, 2024
Revert "Ort openvino npu 1.17 master (#19966)"
daa53a2 
This reverts commit eab35c2.
ilya-lavrenov pushed a commit to ilya-lavrenov/onnxruntime that referenced this pull request Mar 28, 2024
@sfatimar @sspintel
@preetha-intel @hmamidix @vthaniel @saurabhkale17 @ilya-lavrenov
Ort openvino npu 1.17 master (microsoft#19966)
bbf7646 
Add NPU to list of device supported.
Added changes for Support to OV 2024.0
Nuget packages removes packaging of OpenVINO DLL
Bug Fixes with Python API
Reverted Dockerfiles not being maintained.

NPU Device has been introduced by Intel in latest client systems
OpenVINO 2024.0 release is out.

---------

Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com>
Co-authored-by: Preetha Veeramalai <preetha.veeramalai@intel.com>
Co-authored-by: Ubuntu <ubuntu@ubuntu-118727.iind.intel.com>
Co-authored-by: hmamidix <hemax.sowjanya.mamidi@intel.com>
Co-authored-by: vthaniel <vishnudas.thaniel.s@intel.com>
Co-authored-by: saurabhkale17 <saurabh1.kale@intel.com>
luncliff added a commit to luncliff/vcpkg that referenced this pull request Apr 20, 2024
@luncliff
[onnxruntime] bump to 1.17.3 for OpenVINO 2024.0+
ef70bf6 
* microsoft/onnxruntime#19966
* rewrite patches with the new version
This was referenced Apr 20, 2024
Draft
Merged
TedThemistokleous pushed a commit to TedThemistokleous/onnxruntime that referenced this pull request May 7, 2024
@sfatimar @sspintel
@preetha-intel @hmamidix @vthaniel @saurabhkale17
Ort openvino npu 1.17 master (microsoft#19966)
166204d 
### Description
Add NPU to list of device supported. 
Added changes for Support to OV 2024.0
Nuget packages removes packaging of OpenVINO DLL 
Bug Fixes with Python API 
Reverted Dockerfiles not being maintained. 



### Motivation and Context
NPU Device has been introduced by Intel in latest client systems
OpenVINO 2024.0 release is out.

---------

Co-authored-by: Suryaprakash Shanmugam <suryaprakash.shanmugam@intel.com>
Co-authored-by: Preetha Veeramalai <preetha.veeramalai@intel.com>
Co-authored-by: Ubuntu <ubuntu@ubuntu-118727.iind.intel.com>
Co-authored-by: hmamidix <hemax.sowjanya.mamidi@intel.com>
Co-authored-by: vthaniel <vishnudas.thaniel.s@intel.com>
Co-authored-by: saurabhkale17 <saurabh1.kale@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jywu-msft jywu-msft jywu-msft approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@sfatimar @jywu-msft @sspintel @preetha-intel @hmamidix @vthaniel @saurabhkale17