Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failed to build TensorRT Engine with Custom Plugins on Windows #70

Merged
merged 3 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
cmake_minimum_required(VERSION 3.15.0) # 设置CMake的最低版本要求
cmake_policy(SET CMP0091 NEW) # 允许在CMake 3.10+中自动设置项目名作为二进制目录名
cmake_policy(SET CMP0146 OLD) # 忽略对find_package的过时警告
project(TensorRT-YOLO VERSION 5.0.0 LANGUAGES CXX CUDA) # 定义项目名称、版本和使用的编程语言(C++和CUDA)
project(TensorRT-YOLO VERSION 5.0.2 LANGUAGES CXX CUDA) # 定义项目名称、版本和使用的编程语言(C++和CUDA)

# 设置 C++ 标准
set(CMAKE_CXX_STANDARD 17) # 设置C++标准为17
Expand Down
7 changes: 2 additions & 5 deletions docs/cn/model_export.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,8 @@ trtexec --onnx=model.onnx --saveEngine=model.engine --minShapes=images:1x3x640x6
trtexec --onnx=yolov8n-obb.onnx --saveEngine=yolov8n-obb.engine --fp16 --staticPlugins=./lib/plugin/libcustom_plugins.so --setPluginsToSerialize=./lib/plugin/libcustom_plugins.so

# YOLO11-OBB 动态 batch
trtexec --onnx=yolov11n-obb.onnx --saveEngine=yolov11n-obb.engine --fp16 --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 --dynamicPlugins=./lib/plugin/custom_plugins.dll --setPluginsToSerialize=./lib/plugin/custom_plugins.dll
trtexec --onnx=yolov11n-obb.onnx --saveEngine=yolov11n-obb.engine --fp16 --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 --staticPlugins=./lib/plugin/custom_plugins.dll --setPluginsToSerialize=./lib/plugin/custom_plugins.dll
```

> [!NOTE]
> 在使用 `--dynamicPlugins` 和 `--setPluginsToSerialize` 参数构建包含自定义插件的动态模型时,如果遇到错误 `[E] Error[4]: IRuntime::deserializeCudaEngine: Error Code 4: API Usage Error (Cannot register the library as plugin creator of EfficientRotatedNMS_TRT exists already.)`,这通常意味着引擎构建已成功,但加载并反序列化引擎时检测到插件重复注册。这种情况下,可以忽略该错误。

> [!IMPORTANT]
> 强烈推荐您使用Linux操作系统。经过测试,在Windows和Linux系统中使用NVIDIA 20、30、40系列显卡时发现,对于经过显存修改的2080Ti 22G显卡,在Windows环境下构建包含自定义插件的动态模型时,可能会遇到“`IPluginRegistry::loadLibrary: Error Code 3: API Usage Error (SymbolAddress for getCreators could not be loaded, check function name against library symbol)`”的错误。然而,在Linux系统中,该问题并未出现。目前尚不清楚这一问题是由于Windows系统本身的缺陷,还是由于显卡显存的修改所导致。因此,为了避免潜在的兼容性问题,强烈建议您在Linux系统下进行相关操作。
> 在使用 `--staticPlugins` 和 `--setPluginsToSerialize` 参数构建包含自定义插件的动态模型时,如果遇到错误 `[E] Error[4]: IRuntime::deserializeCudaEngine: Error Code 4: API Usage Error (Cannot register the library as plugin creator of EfficientRotatedNMS_TRT exists already.)`,这通常意味着引擎构建已成功,但加载并反序列化引擎时检测到插件重复注册。这种情况下,可以忽略该错误。
7 changes: 2 additions & 5 deletions docs/en/model_export.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,8 @@ trtexec --onnx=model.onnx --saveEngine=model.engine --minShapes=images:1x3x640x6
trtexec --onnx=yolov8n-obb.onnx --saveEngine=yolov8n-obb.engine --fp16 --staticPlugins=./lib/plugin/libcustom_plugins.so --setPluginsToSerialize=./lib/plugin/libcustom_plugins.so

# YOLO11-OBB dynamic batch
trtexec --onnx=yolov11n-obb.onnx --saveEngine=yolov11n-obb.engine --fp16 --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 --dynamicPlugins=./lib/plugin/custom_plugins.dll --setPluginsToSerialize=./lib/plugin/custom_plugins.dll
trtexec --onnx=yolov11n-obb.onnx --saveEngine=yolov11n-obb.engine --fp16 --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 --staticPlugins=./lib/plugin/custom_plugins.dll --setPluginsToSerialize=./lib/plugin/custom_plugins.dll
```

> [!NOTE]
> When using the `--dynamicPlugins` and `--setPluginsToSerialize` parameters to build a dynamic model with custom plugins, encountering the error `[E] Error[4]: IRuntime::deserializeCudaEngine: Error Code 4: API Usage Error (Cannot register the library as plugin creator of EfficientRotatedNMS_TRT exists already.)` typically indicates that the engine has been successfully built, but an attempt to load and deserialize the engine detected a duplicate plugin registration. This error can be safely ignored.

> [!IMPORTANT]
> We strongly recommend using the Linux operating system. After testing, it has been found that when using NVIDIA 20, 30, and 40 series graphics cards in both Windows and Linux systems, a modified 2080Ti 22G graphics card may encounter the error `IPluginRegistry::loadLibrary: Error Code 3: API Usage Error (SymbolAddress for getCreators could not be loaded, check function name against library symbol)` when building dynamic models with custom plugins in the Windows environment. However, this issue does not occur in the Linux system. It is currently unclear whether this problem is due to defects in the Windows system itself or due to the modification of the graphics card's memory. Therefore, to avoid potential compatibility issues, it is strongly advised that you perform related operations in the Linux system.
> When using the `--staticPlugins` and `--setPluginsToSerialize` parameters to build a dynamic model with custom plugins, encountering the error `[E] Error[4]: IRuntime::deserializeCudaEngine: Error Code 4: API Usage Error (Cannot register the library as plugin creator of EfficientRotatedNMS_TRT exists already.)` typically indicates that the engine has been successfully built, but an attempt to load and deserialize the engine detected a duplicate plugin registration. This error can be safely ignored.
6 changes: 3 additions & 3 deletions plugin/common/vfcCommon.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -73,13 +73,13 @@ ILogger* getPluginLogger()
} // namespace plugin
} // namespace

extern "C" TENSORRTAPI void setLoggerFinder(nvinfer1::ILoggerFinder* finder)
extern "C" DEPLOYAPI void setLoggerFinder(nvinfer1::ILoggerFinder* finder)
{
nvinfer1::plugin::gLoggerFinder.setLoggerFinder(finder);
}

#if (TENSORRT_VERSION >= 100000)
extern "C" TENSORRTAPI IPluginCreatorInterface* const* getCreators(int32_t& nbCreators)
extern "C" DEPLOYAPI IPluginCreatorInterface* const* getCreators(int32_t& nbCreators)
{
nbCreators = 2;
static EfficientRotatedNMSPluginCreator efficientRotatedNMSPluginCreator;
Expand All @@ -91,7 +91,7 @@ extern "C" TENSORRTAPI IPluginCreatorInterface* const* getCreators(int32_t& nbCr
#endif

#if (TENSORRT_VERSION < 100100)
extern "C" TENSORRTAPI IPluginCreator* const* getPluginCreators(int32_t& nbCreators)
extern "C" DEPLOYAPI IPluginCreator* const* getPluginCreators(int32_t& nbCreators)
{
nbCreators = 2;
static EfficientRotatedNMSPluginCreator efficientRotatedNMSPluginCreator;
Expand Down
12 changes: 9 additions & 3 deletions plugin/common/vfcCommon.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@
#include "NvInferRuntimeCommon.h"
#include "NvInferVersion.h"

#ifdef _MSC_VER
#define DEPLOYAPI __declspec(dllexport)
#else
#define DEPLOYAPI __attribute__((visibility("default")))
#endif

#define TENSORRT_VERSION (NV_TENSORRT_MAJOR * 10000L + NV_TENSORRT_MINOR * 100L + NV_TENSORRT_PATCH * 1L)

using namespace nvinfer1;
Expand All @@ -36,14 +42,14 @@ ILogger* getPluginLogger();
} // namespace plugin
} // namespace nvinfer1

extern "C" TENSORRTAPI void setLoggerFinder(nvinfer1::ILoggerFinder* finder);
extern "C" DEPLOYAPI void setLoggerFinder(nvinfer1::ILoggerFinder* finder);

#if (TENSORRT_VERSION >= 100000)
extern "C" TENSORRTAPI IPluginCreatorInterface* const* getCreators(int32_t& nbCreators);
extern "C" DEPLOYAPI IPluginCreatorInterface* const* getCreators(int32_t& nbCreators);
#endif

#if (TENSORRT_VERSION < 100100)
extern "C" TENSORRTAPI IPluginCreator* const* getPluginCreators(int32_t& nbCreators);
extern "C" DEPLOYAPI IPluginCreator* const* getPluginCreators(int32_t& nbCreators);
#endif

#endif // TRT_PLUGIN_VFC_COMMON_H
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

# Project settings -----------------------------------------------------------------------------------------------------
[project]
version = "5.0.0"
version = "5.0.2"
readme = "README.md"
name = "tensorrt_yolo"
requires-python = ">=3.8"
Expand Down
2 changes: 1 addition & 1 deletion xmake.lua
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-- 设置项目信息
set_project("TensorRT-YOLO")
set_version("5.0.0")
set_version("5.0.2")
set_languages("cxx17")
set_allowedplats("windows", "linux")

Expand Down