Fix cpp ci #2958
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,10 +18,6 @@ if(CLANG_FORMAT_EXE) | |
| ${PROJECT_SOURCE_DIR}/test/*.hh | ||
| ) | ||
|
|
||
| endif() | ||
|
|
||
|
|
||
|
|
@@ -31,6 +27,21 @@ find_package(fmt REQUIRED) | |
| find_package(gflags REQUIRED) | ||
| find_package(Torch REQUIRED) | ||
|
|
||
| include(FetchContent) | ||
|
|
||
| FetchContent_Declare( | ||
| yaml-cpp | ||
| GIT_REPOSITORY https://github.com/jbeder/yaml-cpp.git | ||
| GIT_TAG 0.8.0 # Can be a tag (yaml-cpp-x.x.x), a commit hash, or a branch name (master) | ||
| ) | ||
| FetchContent_GetProperties(yaml-cpp) | ||
|
|
||
| if(NOT yaml-cpp_POPULATED) | ||
| message(STATUS "Fetching yaml-cpp...") | ||
| FetchContent_Populate(yaml-cpp) | ||
| add_subdirectory(${yaml-cpp_SOURCE_DIR} ${yaml-cpp_BINARY_DIR}) | ||
| endif() | ||
|
|
||
|
Comment on lines
+30
to
+44
There was a problem hiding this comment. we need build yaml-cpp with -DCMAKE_CXX_FLAGS="-fPIC" (see https://github.com/pytorch/serve/blob/master/cpp/build.sh#L133C84-L133C109) |
||
| set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}") | ||
|
|
||
| include_directories(${TORCH_INCLUDE_DIRS}) | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,36 +5,34 @@ | |
| * cmake version: 3.18+ | ||
| ## Installation and Running TorchServe CPP | ||
|
|
||
| This installation instruction assumes that TorchServe is already installed through pip/conda/source. If this is not the case install it after the `Install dependencies` step through your preferred method. | ||
|
|
||
| ### Install dependencies | ||
| ``` | ||
| cd serve | ||
| python ts_scripts/install_dependencies.py --cpp --environment dev [--cuda=cu121|cu118] | ||
| ``` | ||
| ### Building the backend | ||
| Don't forget to install or update TorchServe at this point if it wasn't previously installed. | ||
| ``` | ||
| ## Dev Build | ||
| cd cpp | ||
| ./build.sh [-g cu121|cu118] | ||
|
|
||
| ``` | ||
|
|
||
| ### Run TorchServe | ||
| ``` | ||
| mkdir model_store | ||
| torchserve --ncs --start --model-store model_store | ||
| ``` | ||
|
|
||
| ### Clean the build directory | ||
| To clean the build directory in order to rebuild from scratch simply delete the cpp/_build directory with | ||
| ``` | ||
| rm -rf cpp/_build | ||
| ``` | ||
|
|
||
|
Comment on lines
+29
to
+35
There was a problem hiding this comment. can we keep this so that cx can clean up env in case of debugging? There was a problem hiding this comment. @lxning Not sure if I understand. This is meant to be executed if you want to rebuild from scratch. Like a There was a problem hiding this comment. We haven't set clean rule at this moment. We need let cx know how to clean. |
||
| ## Backend | ||
| TorchServe cpp backend can run as a process, which is similar to [TorchServe Python backend](https://github.com/pytorch/serve/tree/master/ts). By default, TorchServe supports torch scripted model in cpp backend. Other platforms such as MxNet, ONNX can be supported through custom handlers following the TorchScript example [src/backends/handler/torch_scripted_handler.hh](https://github.com/pytorch/serve/blob/master/cpp/src/backends/handler/torch_scripted_handler.hh). | ||
| ### Custom Handler | ||
|
|
@@ -89,11 +87,11 @@ python -c "import ts; from pathlib import Path; print((Path(ts.__file__).parent | |
| 3. Make sure you have the right conda/venv environment activated during building that you're also using to run TorchServe. | ||
|
|
||
| Q: Build on Mac fails with `Library not loaded: @rpath/libomp.dylib` | ||
| A: Install libomp with brew and link in /usr/local/lib | ||
| ```bash | ||
| brew install libomp | ||
| sudo ln -s /opt/homebrew/opt/libomp/lib/libomp.dylib /usr/local/lib/libomp.dylib | ||
| ``` | ||
|
|
||
| Q: When loading a handler which uses a model exported with torch._export.aot_compile the handler dies with "error: Error in dlopen: MODEL.SO : undefined symbol: SOME_SYMBOL". | ||
| A: Make sure that you are using matching libtorch and Pytorch versions for inference and export, respectively. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we check cmake version before we proceed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lxning not sure what you mean? Inside the workflow file? What would be the consequences of the check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
install_dependency with option "cpp" will install cmake (eg. mac, linux). Currently some cmake version can cause cpp compile error. Better to check cmake version before cijob proceed to save debug time.