Add CI for cpp_backend branch #1916
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maaquib
approved these changes
Oct 20, 2022
lxning
approved these changes
Oct 31, 2022
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 18, 2024
* add workaround solution from nvidia * add comments * expand runtimeType * add runtimeType in model config * add unit test * revert test/buildspec_gpu.yml * update testng.xml * update json files * fmt * fmt * init cpp dir * init code structure * Init socket code and otf protocol * add log api * decouple backend and model worker; impl torchscript load model; build scripts [ci skip] * delete src/CMakeLists.txt * init model archive manifest loader * add manifest and unit test * integrate manifest into backend; add unit test * update otf message internal structure; add inference request message * update otfmessage function return [skip ci] * add torch base handler * support dynamic load shared lib * disable install libtorch * update utils/CMakeLists.txt * add dynamic lib loader unit test * [skip CI] update src/utils/CMakeLists.txt * install kineto in build.sh * [skip ci] add handler factory * [skip ci] update inference request message * vision handler inference impl. * [skip ci] update torchscript backend api * change model_path to model_dir [skip ci] * [skip ci] torchscripted handler load model pass postive test * [skip ci] fix dll test teardown * [skip ci] add mnist inference positive test * update torchscripted base handler postprocess * [skip ci] add model instance status in backend * [skip ci]add prediction test for base and dll handler * [skip ci] clean up * add batch inference test * [skip ci] add dll close * [skip ci] file_system clean up * [skip ci] add mnist scripted model pt file for testing * [skip ci] torch_scripted/torch_scripted_backend_test refactory * [skip ci] torch_scripted_backend_test refactory * [skip ci] extend LoadPredict api * [skip ci] add negative test in torch_scripted_backend_test * explicit set ABI=1 * support different build step for linux and mac * [skip ci] update folly installation * add sudo for folly dep installation * [skip ci] update libtorch cuda installation * [skip ci] update sudo mac * [skip ci] update cuda option flag * [skip ci] set cuda compiler * [skip ci] skip install kineto on linux * [skip ci] fix cude compile path * add documnetation * update gcc version description * add cpp log config file option * add parameters * update setup.py for package cpp * set cpp libpath env * add unit test * [skip ci] install lib * [skip ci] add cpp log config path for cpp backend start * CPP OTFProtocol implementation for inference request and response (#1817) * Add folly logging * Adding model response serializer * Slight refactor * Adding test for otf protocol * Address review comments * Adding some logging tests * Refactoring socket methods into its own class to enable mocking for testing * otf protocol implementation for inference request and response * rebase from #1814 * rebase from #1814 * refactor after PR#1814 rebase * add unit tests for inference request and response otf protocol * add cpp backend test for otf_protocol and handler * Update logging flag to read log config file * Address test review comments * Remove response end from LoadModelResponse data struct * Removing googletest since folly already has it * Adding errno to socket send and receive failures * address review comments * refactor LoadRequest and Response OTF protocol to remove use of new and minor improvements * address review comments Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * update model archiver for cpp * Bug fix in cpp integration (#1887) * bug fixes in java - cpp integration * revert arg changes * add clang-tidy in build (#1896) * replace strcpy with strncpy (#1898) * add clang-tidy in build * replace strcpu with strncpy * [WIP] cpp backend with Java integ (#1890) * Fix build script * Fix socket issues * Fix socket name truncation for uds * Fix expected log lines format from frontend * Removing some debug logs * Address review comments * Remove incorrect log line * Fix inference issue * Update filesystem import * Fix path of default logging file * Make build.sh executable * add clang-tidy and clang-format for cpp backend lint (#1910) * add clang-tidy in build * replace strcpu with strncpy * fix warnings for torchscripte backends * add .clang-tidy and update CMakeLists * add clang-format * remove unused parameter in cpp basehandler (#1917) * add clang-tidy in build * replace strcpu with strncpy * fix warnings for torchscripte backends * add .clang-tidy and update CMakeLists * add clang-format * remove unused parameters in basehandler and update mnist handler * remove libmnist_handler.dylib * remove not necessary func softmax in mnist example handler * fix clang-tidy warnings (#1915) * CPP mnist_base postman test (#1907) * add mnist base cpp postman integration test * refactor based on #1917 * add response body validation * disable grpc inference api test for cpp backend model * fix typo * install clang-format on linux (#1926) * Add CI for cpp_backend branch (#1916) * Create ci-cpu-cpp.yml * Update ci-cpu-cpp.yml * Update ci-cpu-cpp.yml * Update ci-cpu-cpp.yml * Metrics helper classes implementation for C++ backend (#1874) * Metrics helper classes implementation Dimension, Units and Metric * Refactor metrics helper classes 1) Move metrics helper classes from src/backends to src/utils 2) Update Metric class to store a vector of values instead of a single value * Fix metrics headers include guard to follow naming convention * Refactor metrics implementation to follow the API described in the metrics refactor RFC: #1492 * Revert changes to the following CMakeLists files since no change is required as part of the metrics implementation: cpp/src/backends/CMakeLists.txt cpp/test/CMakeLists.txt * Fix compiler warnings related to std::experimental::filesystem * Refactor metrics helper classes to simplify backend metrics implementation by emitting logs when the metrics API is called instead of storing them until the completion of an inference request to flush the metrics * Infer dimension names order from config file and use the same order for dimension values argument in the metrics API. Fix clang-tidy warnings. * Refactor backend metrics unit tests to use same regex as frontend to parse metric logs * install cpp via install_from_src (#1883) * add clang-tidy in build * replace strcpu with strncpy * fix warnings for torchscripte backends * add .clang-tidy and update CMakeLists * add clang-format * remove unused parameters in basehandler and update mnist handler * remove libmnist_handler.dylib * remove not necessary func softmax in mnist example handler * feature install cpp from install_from_src * add --install-dependencies in setup.py * fix typo * update MANIFEST.in and readme * update readme * code cleanup * update readme * update logging path * fix backend worker started checking * update readme * Update README.md * YAML metrics configuration handling for C++ backend (#1941) * fix yaml_cpp installation in build script (#1996) * fix yaml_cpp installation * build request id strings for one batch * Metrics cache implementation and integration with C++ backend (#1975) * Metrics cache implementation for C++ backend * Metrics cache integration with C++ backend Co-authored-by: Naman Nandan <namannan@amazon.com> * Revert "Metrics cache implementation and integration with C++ backend (#1975)" (#2011) This reverts commit 3451bb7. * Metrics cache implementation and integration with C++ backend (#2012) * Metrics cache implementation for C++ backend * Metrics cache integration with C++ backend Co-authored-by: Naman Nandan <namannan@amazon.com> * Fix lint error * Fix lint error * Fix model-archiver after cpp merge * Adjust signature of workerLifeCycleMnist.startWorker in test * Fix unit tests after merging master into cpp_backend * Fix linting error * Install dependencies for cpp backend * Fix unit tests after cpp merge * Fix formatting * Move installation of cpp deps to ts_scripts/install_dependencies.py * Build cpp backend for regression and sanity tests * Fix formatting * Fix typo * Temp fix hanging after starting cpp worker * Add pytest for cpp backend * Roll back building of cpp abckend in ci regression and sanity tests; install deps in cpp ci * Fix formatting * Remove mnist_custom_cpp.mar file from postman test as we do not build cpp backend for general regression test * Remove cpp model archive in additional place * Remove cpp build from setup.py * Remove additional ref to build_cpp in setup.py * fix code link * Update README.md * Update libtorch versions + move installation of cpp backend to build.sh * Prepare cpp build workflow for merge into master * Update cuda version in cpp/build.sh * Remove reference to LDP * Fix WorkerLifeCycleTest * rm src/test/resources/config_test_cpp.properties * Remove debug prints * Skip cpp backend test if cpp backend is not available --------- Co-authored-by: lxning <lninga@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: lxning <23464292+lxning@users.noreply.github.com> Co-authored-by: rohithkrn <rohith.nallamaddi@gmail.com> Co-authored-by: Naman Nandan <namankt55@gmail.com> Co-authored-by: Naman Nandan <namannan@amazon.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a bare minimum ci for cpp_backend branch. Runs
./build.shscript.