Fixed response status message for inferencing on model with zero workers #305
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
harshbafna
suggested changes
May 6, 2020
frontend/server/src/main/java/org/pytorch/serve/http/ManagementRequestHandler.java
Show resolved
Hide resolved
frontend/server/src/main/java/org/pytorch/serve/http/InferenceRequestHandler.java
Outdated
Show resolved
Hide resolved
frontend/server/src/main/java/org/pytorch/serve/http/ManagementRequestHandler.java
Outdated
Show resolved
Hide resolved
frontend/server/src/main/java/org/pytorch/serve/http/ManagementRequestHandler.java
Outdated
Show resolved
Hide resolved
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
dhaniram-kshirsagar
changed the title
mjpsl
changed the title
harshbafna
reviewed
May 7, 2020
frontend/server/src/main/java/org/pytorch/serve/http/ManagementRequestHandler.java
Outdated
Show resolved
Hide resolved
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
harshbafna
previously requested changes
May 7, 2020
frontend/server/src/main/java/org/pytorch/serve/http/InferenceRequestHandler.java
Show resolved
Hide resolved
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
vdantu
reviewed
May 7, 2020
| @@ -329,8 +337,19 @@ private void updateModelWorkers( | |||
| modelManager.scaleRequestStatus(modelName, modelVersion); | |||
| if (HttpResponseStatus.OK.equals(v)) { | |||
| if (status) { | |||
| NettyUtils.sendJsonResponse( | |||
| ctx, new StatusResponse("Workers scaled"), v); | |||
| String msg = | |||
There was a problem hiding this comment.
I am curious to understand if this message needs modification. Can't this be a generic message of how many workers are we scaling this to? If we elaborate on this message, then customers who maintain logs will know that POST /models, which is register model, was updated with n number of workers. Adding a flag to determine the code path, ie., if this is coming from register-model or from scale model, seems a little hacky.
There was a problem hiding this comment.
The idea is to keep the response message consistent for registration request with/without initial workers. I agree its a bit hacky.
There was a problem hiding this comment.
Keeping these changes as message give clear information.
There was a problem hiding this comment.
@vdantu Let me know if this LGTY
There was a problem hiding this comment.
I am still not convinced that passing a boolean flag to determine the error message is the the only option. But, this is is not a blocker for the PR. We can look into how to refactor this code in the future (may be return the future to the method above and populate the error message there or some such).
mjpsl
changed the title
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Validated all the scale worker and registration API and working as expected. Logs : |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
maaquib
approved these changes
May 27, 2020
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
mycpuorg
added a commit
that referenced
this pull request
Jun 9, 2020
* fixed resnet152 custom handler to work with GPU * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler * issue templates (#273) * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler Co-authored-by: mycpuorg <mail@mycpu.org> * Fixes #205 - Docker install not finding GPUs (#262) Invokes docker with --runtime=nvidia for GPU Added the option to specify the GPU / specific GPU ids in start.sh Fixed the documentation for start.sh script Fixed the JDK version in Dockerfile.gpu Co-authored-by: Aaqib <maaquib@gmail.com> * Fix pylint issue (#163) * pylint fixes and code cleanup * UT fixes, pylint updates per latest version and updated sanity script to run pylint * fixed pylint and wrapt installation issue * freeze pylint version to 2.4.4 due to bug in version 2.5 * removed duplicate statement to install pylint Co-authored-by: Aaqib <maaquib@gmail.com> * Updated exception handling for worker threads shutdown. (#172) * updated code to now log errors only in case when exception is not genreated by unregister api call * java formatting * fixed illigalstateexception * removed the unrequired handling for illegal monitor state exception Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> * changes for Homebrew issue 98 (#126) * changes for Homebrew issue 98 * adding changes for path in benchmark.py * CI Build corrected * benchmark changes corrected path * changed PATH for homebrew * Update Readme Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh * Updated Readme.md removed unnecessory lines * corrected indentation corrected indentation for new lines added * Update Readme.md fil added separate commands for CPU and GPU based Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * added changes to clean model loaded through URL from model store on unregisted * added UTs * Moving the plugins SDK to org.pytorch. Added plugins example. (#261) * Moved plugins SDK to org.pytorch * Updating the repo for publishing plugins SDK * Enabled bintray maven repo temporarily * Reset tests * Added GPG plugin to pom file to sign the plugins SDK * Update the SDK version * Removed bintray repo * Modify the buildspec to not deploy the SDK Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Added model zoo (#299) Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Brad Heintz <bradheintz@fb.com> * Removed snapshot dependency on model (#152) * Snapshot is now independent of model * fixed checkstyle issue * fixed formatting * doc review: configuration and custom_service * addressed review comments * fixed incorrect merge * fixed review comments and corrected the error messages * adding remaining fixes for string constants Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 (#285) * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 * Script to clone Pytorch Repo & Run Post Man Scripts * Basic Tests for Inference & Managment APIs * AWS Code Build buildspec.yml * Refactor postman tests, add empty postman collection for todos * Add a readme for adding tests * Add more management tests * Fix typo * Snapshot Regression tests - #57 * Fix typo * Comment out snapshot test for now * Weird typos are showing up, try saving everything * Need to manually save every request in postman... * When reregister don't use s3 * More workarounds for rereg * Fix test messages * Remove test for empty reqs * Change get to post * Fix pytest for subprocess - #57 * Enable pytest - #57 * Inference API Testcases / Rename Management Testcases #57 * Wipe Model MARs in Modelstore between tests #54 * Fix Inference Endpoint Execution #54 * Fix file paths - #57 * Fix Initial Workers #57 * Fix Inference Assertions #57 * Fix DenseNet Inference #57 * Fix Densenet Register #54 * Redirect Logs to File #54 * Fix Test Execution Log #54 * README, Build Badge #57 * Format ReadMe #57 * Update README - Add Instruction to add new tests / Run tests locally * Update README * Update branch for build Codebuild Badge Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> * Issue 312 install from src (#322) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update instructions for installation from source This commit aims to address Issue #312 Tested on a fresh EC2 P3.2 instance: * Followed instructions under https://github.com/pytorch/serve#install-torchserve-for-development * Started the server manually from command line * Ran a ping query Received a response from server with "Healthy" status Signed-off-by: Manoj Rao <mail@mycpu.org> * add separate files for installation one each for linux and macos * update README and remove relative path from script * Install script - Dont uninstall if not previously installed - #132 * explicitly build the source code with gradle prior to install * remove upgrade instructions in favor of clean reinstall from source * update instructions to contain the correct script name Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Manoj Rao <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> * fix for macos script (#338) * Move Issue Template to .github dir * Updated sanity script to use install_from_src_ubuntu script (#350) * updated sanity script to use install_from_src_ubuntu script * added java version check * fixed issue with if statement * fixed variable name * fixed repeated cd command * updated artifacts * removed artifacts from buildspec * Corrected HTTP response code for Model version does not exists exception (#282) * issue 196 patch * Added openapi missing responses. - incorporated review comments * updated method signature after resolving merge conflicts Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Corrected paths in command and added notes (#354) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Refactored UT for better reporting (#159) * refactored UTs * upgraded testng version and updated testcase names * added testng.xml to handle sequential execution of test classes * added waveglow example * Updated readme for waveglow example * fixed failing UTs after merging master * removed unintentionally pushed waveglow example * fixed failing test case * moved set-default to utils and added couple of more add cases for default version * java formatting * corrected set default negative test case * fixed java formatting Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Huggingface Transformers generalization (#302) * Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model * modified to include mapping of predicitons and decoding received text * move back the original text * moved the index_to_name to bert folder * Transformers/ Bert serving with Torchserve readme * deleting an unused dictionary * just cleaning some of the commented out parts * cleaning up * Cleaning up * uncommenting the logging and cleaning the comments * Download and save the checkpoints, config file and vocab of the BERT from Transformers for demonstration purposes * mapping of predicitons to labels * a custom handler for transformers. * README. * sample of input text. * modifying the descriptions * modifying the descriptions * adding comments * adding commnets * Fromatting modifications * modified comments * added the citations * adding hugginface as the type of transformer model we are referring to * renaming the folder to huggingface_transformers * Removing the bert folder * simple modification * modifed the script to use a setup_config.json for loading and saving models * added the section for transfomer_handler_generalized * the generalized handler * set_up config intialization * edited for transfomer generalization * modified to use just input_ids as inputs for models * modified for question answering * modified to include the number of labels * corrected typos * modifed variable names * edited for clarity * modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case * modifed values for QA * modifed the variable names * added comments * modified to add clarity * add comment in preprocessing of QA * added warning for the case mode is not selected properly and combined constructor and initlaize fucntion * clean up * clean up * combined constructor and initliaze func * combined constructor and initliaze func * Separated the constructor and initliaze func * Modified to remove the classification part and update with generalized version * resolved the coflict and separated the costructor and initliaze func * removed the handler for classification case as the generalized version handles the case Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: mycpuorg <mail@mycpu.org> * added waveglow example (#258) * added waveglow example * Updated readme for waveglow example * enhanced example to remove runtime checkpoint download * updated handler to return the audio file content instead of dumping it to tmp dir * minor change to handler * added pipefail * added python snippet to run inference * updated readme with know issue with the example Co-authored-by: Aaqib <maaquib@gmail.com> * made build suite compatible with latest gradle (#348) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * moved env section after context * Deleting redundant ISSUE_TEMPLATE at root dir * Fix typo in scripts/install_from_src_macos * Corrected OpenApi description (#328) * Correct query parameter name (Registration Api)in OpenApi description * removed non existing path parameter Co-authored-by: Aaqib <maaquib@gmail.com> * Regression suite fixes and added test cases (#344) * added test model zoo test cases updated regression script * Updated Readme * update -y flag for install * Updated docker command as per new imagename and tag generated in stagging branch * Fix sudo usage in regression test script * - addressed review comments - reverted changes related to sudo in regression script. * added https test collection * - updated failed test cases on staging branch - added environment and updated newman cmd with --insecure for https * updated script * updated pytest * update curl check for serve up * Revert "update curl check for serve up" This reverts commit 0047eae. * Revert "update curl check for serve up" This reverts commit 0047eae. - update model zoo - added resnet test case * added workers to resnet model * Update readme to run test suite on desired branch * enhanced inference test cases to driven by datafile * Updated readme Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Fix typo in mnist readme (#330) Co-authored-by: Aaqib <maaquib@gmail.com> * Fixed typo in ab installation. (#327) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Fixed typo in ab installation. Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Setup Nightly Build for TorchServe (#371) * Add Branch Flag for Nightly CodeBuild #57 * Update ENV Variable for Nightly Build Name * Nigtly Build - Fix FileName for Build Artifacts * Update README for accessing test logs * Whitspace changes * Build new images for build - README Instruction * Fix for intermittent gradle build failure (#352) * Fix for intermittent null pointer exception * Fixed comments Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * documentation: Remove redundant curl options in example (#336) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * fix-doc * fix remaining "-X POST"s Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Corrected port number validation range (#304) * Corrected port number validation range * added UT for issue 291 * added ut dependency file formatting * correct param in config_test_env.properties * removed incorrect flags from test case config files * added negative test cases * removed properties file and changes in test case Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Fixes version number reporting for TorchServe (#360) * Correct version and added vesion.txt for getting version info * fixed description test cases and build issue * Added snapshot test fixes and resolved relative path issue for version.txt * python formatting * added version.txt in MANIFEST.in * addressed review comments * added version.txt for model archiver as well - addressed review comments * -fixed python UT -added ts python UT run in sanity script * removed blank lines from snapshot files Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * fixes installation script exits without error if no java present on machine (#376) * fixes installation script exits without error if no java present on machine * added changes for macos script Co-authored-by: Aaqib <maaquib@gmail.com> * Dockerfile enhancements (#308) * dockerfile now points to latest torchserve on pypi and added separate files for dev * Minor changes * Made build and run commands consistent * Fixing formatting * Consistency issue * Enhanced Dockerfile for binary per #178 and other misc. * Fixed gpu related issues in start script * Corrected the heading font * Added mar creation steps from container * added buildkit support for dev docker files * Fix for buildkit * Restoring psutil and future as absence leads intermittent hangged build * Fixed review comments for doc * Fixed typo Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Aaqib <maaquib@gmail.com> * Updated configuration.md for missing config properties (#337) * Updated configuration.md for missing parameters * changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT - removes ambiguity - follows naming convention * addressed review comments. * removed debug property * updated configuration.md Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Adding missing document links to README (#379) * Maven tests should output only errors (#326) Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> * fixed code for build failure in python virtual environment (#341) * fixed code for build failure in python virtual environment * added exception missed on resolving conflicts Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> * updated object detector handler to use only default cuda device for torchvision version less then 0.6 (#265) * updated object detector handler to use default cuda device for torchvision version 0.5.0 * updated version check logic * Updated documentation for torchvision version dependency * enhanced sanity script to validate all default handlers * fixed sanity script * added torchtext install step in sanity script * fixed issue related to inference after restart with snapshot. * moved torchtext to codebuild docker files * added more log statements * enhanced run inference function * fixed register model function * added unregister model function * incorporated code review comments * added back inference call for snapshot test * fixed if block Co-authored-by: Aaqib <maaquib@gmail.com> * Adding a barebones ec2 template to run torchserve on ubuntu (#301) * Adding a barebones ec2 template to run torchserve on ubuntu * Adding VPC with Private subnet * Address review comments #1 * Address review comments #2 * Rebase for #365 (#389) * Updated README.md for #364 See #364 for a deep description of the problem * updated README.md * updated README.md Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> * Document the docker settings for running in production mode (#388) * Docker Production Options #234 * Docker Production Options - Typo Fixes / Fit & Finish #234 * Typos - Docker Docs #234 * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Huggingface transformers integration test (#382) * Add Huggingface transformers example integration tests * Add transformers intallation * Fix a missing bracket * move transformers installation to chain installations Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Updated docker README.md (#401) * Update README.md * Fixed cd command * Made source cloning steps explicit!! * Fixed online editing mistake Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Fixed response status message for inferencing on model with zero workers (#305) * Changes for issue_29 * Changed error Messages * changed message for scalling request * changed defaultVersion string * Corrected error message * changed the code for best practice * resolved review comment for version * updated management api doc as per new responses in worker scale api Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * made model server port tests independent (#396) * Added backend profiling and fixed benchmark's install dependency scripts (#402) * added backend profiling * updated install dependency scripts to make them jmeter version independent * updated readme * Fix benchmarks/README typo and JMeter versioning issue * updated documentation Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * Add sphinx directory and conf.py (#274) * Add sphinx folder that contains files for docs build * Delete .DS_Store * Delete .DS_Store * Add conf.py in docs/ * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update to the latest version of sphinx * Add recommonmark to extensions Add recommonmark to extensions so sphinx project has markdown support. Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html * Update source suffix to use latest recommonmark syntax Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html Co-authored-by: Aaron Markham <markhama@amazon.com> * Remove old recommonmark parser syntax * Pin sphinx version to work with latest version of recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Pin version latest version recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Delete theme file generated during build * de-duplicate install package of psutil (#329) The Dockerfile of GPU install package psutil duplicated. * Remove pytorch-sphinx-theme files * Update to match latest staging_0_1_1 * Update docker directory to match staging_0_1_1 branch * Delete .DS_Store * Add sphinxcontrib.katex and torch Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Image classifier example readme fixes (#405) * Changed the model name in torch.jit.save to be consistent with docs * Remove extra handler flag and fix the torchservr start * fix the doc * fix the doc Co-authored-by: mycpuorg <mail@mycpu.org> * Porting memory fix changes to TorchServe (#422) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Benchmark documentation fix (#425) * Fix Benchmark Documentation * Documentaion fixes * .circleci: Add initial circleci configuration (#431) Signed-off-by: Eli Uriegas <eliuriegas@fb.com> * Torchscript support huggingface transfrmers (#421) * Add support for torchscript * Add support for torchscript * Add key values for torchscript support * Add Torchscript suport * Add Torchscript support * Add Torchscript support in loading the models * keep the example to QA * modify to reflect Torchscript example? * Add torchscript support guides * Add fintued BERT for QA * Add QA artifacts * Add seqenece classification artifacts * Token classification artifacts * Correct typos * Add max length to tokenizer * clean up * enhance the doc * change setup to seq classification * Add detail to torchscript support * set the model for QA * Add torchscript bert seq classification example * Add bert seq classification example * Modify the packaging model command * Modify torchscript packaging command * clean up * clean up * correct spacing between list items Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Integration test huggingface transfromers torchscript support (#433) * made model server port tests independent * Add integration tests for Huggingface transfromers with new handler that support torchscript Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Apachebench benchmark - cleanup script / documentation * Revert "Apachebench benchmark - cleanup script / documentation" This reverts commit c8f8682. * Documentation updates for clarity on Eager mode / Torch scripted models (#439) * Model Zoo updates for Eager Mode / Torchscripted models * Update examples/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Apachebench benchmarking - Script / Documentation updates (#438) * Apachebench benchmarking script - Script / Documentation updates * Stop TS after benchmark completion. Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com> Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Brad Heintz <bradheintz@fb.com> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal> Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com> Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> Co-authored-by: alvations <alvations@gmail.com> Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Jessica Lin <jplin@fb.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
maaquib
added a commit
that referenced
this pull request
Jun 9, 2020
* fixed resnet152 custom handler to work with GPU * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler * issue templates (#273) * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler Co-authored-by: mycpuorg <mail@mycpu.org> * Fixes #205 - Docker install not finding GPUs (#262) Invokes docker with --runtime=nvidia for GPU Added the option to specify the GPU / specific GPU ids in start.sh Fixed the documentation for start.sh script Fixed the JDK version in Dockerfile.gpu Co-authored-by: Aaqib <maaquib@gmail.com> * Fix pylint issue (#163) * pylint fixes and code cleanup * UT fixes, pylint updates per latest version and updated sanity script to run pylint * fixed pylint and wrapt installation issue * freeze pylint version to 2.4.4 due to bug in version 2.5 * removed duplicate statement to install pylint Co-authored-by: Aaqib <maaquib@gmail.com> * Updated exception handling for worker threads shutdown. (#172) * updated code to now log errors only in case when exception is not genreated by unregister api call * java formatting * fixed illigalstateexception * removed the unrequired handling for illegal monitor state exception Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> * changes for Homebrew issue 98 (#126) * changes for Homebrew issue 98 * adding changes for path in benchmark.py * CI Build corrected * benchmark changes corrected path * changed PATH for homebrew * Update Readme Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh * Updated Readme.md removed unnecessory lines * corrected indentation corrected indentation for new lines added * Update Readme.md fil added separate commands for CPU and GPU based Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * added changes to clean model loaded through URL from model store on unregisted * added UTs * Moving the plugins SDK to org.pytorch. Added plugins example. (#261) * Moved plugins SDK to org.pytorch * Updating the repo for publishing plugins SDK * Enabled bintray maven repo temporarily * Reset tests * Added GPG plugin to pom file to sign the plugins SDK * Update the SDK version * Removed bintray repo * Modify the buildspec to not deploy the SDK Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Added model zoo (#299) Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Brad Heintz <bradheintz@fb.com> * Removed snapshot dependency on model (#152) * Snapshot is now independent of model * fixed checkstyle issue * fixed formatting * doc review: configuration and custom_service * addressed review comments * fixed incorrect merge * fixed review comments and corrected the error messages * adding remaining fixes for string constants Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 (#285) * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 * Script to clone Pytorch Repo & Run Post Man Scripts * Basic Tests for Inference & Managment APIs * AWS Code Build buildspec.yml * Refactor postman tests, add empty postman collection for todos * Add a readme for adding tests * Add more management tests * Fix typo * Snapshot Regression tests - #57 * Fix typo * Comment out snapshot test for now * Weird typos are showing up, try saving everything * Need to manually save every request in postman... * When reregister don't use s3 * More workarounds for rereg * Fix test messages * Remove test for empty reqs * Change get to post * Fix pytest for subprocess - #57 * Enable pytest - #57 * Inference API Testcases / Rename Management Testcases #57 * Wipe Model MARs in Modelstore between tests #54 * Fix Inference Endpoint Execution #54 * Fix file paths - #57 * Fix Initial Workers #57 * Fix Inference Assertions #57 * Fix DenseNet Inference #57 * Fix Densenet Register #54 * Redirect Logs to File #54 * Fix Test Execution Log #54 * README, Build Badge #57 * Format ReadMe #57 * Update README - Add Instruction to add new tests / Run tests locally * Update README * Update branch for build Codebuild Badge Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> * Issue 312 install from src (#322) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update instructions for installation from source This commit aims to address Issue #312 Tested on a fresh EC2 P3.2 instance: * Followed instructions under https://github.com/pytorch/serve#install-torchserve-for-development * Started the server manually from command line * Ran a ping query Received a response from server with "Healthy" status Signed-off-by: Manoj Rao <mail@mycpu.org> * add separate files for installation one each for linux and macos * update README and remove relative path from script * Install script - Dont uninstall if not previously installed - #132 * explicitly build the source code with gradle prior to install * remove upgrade instructions in favor of clean reinstall from source * update instructions to contain the correct script name Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Manoj Rao <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> * fix for macos script (#338) * Move Issue Template to .github dir * Updated sanity script to use install_from_src_ubuntu script (#350) * updated sanity script to use install_from_src_ubuntu script * added java version check * fixed issue with if statement * fixed variable name * fixed repeated cd command * updated artifacts * removed artifacts from buildspec * Corrected HTTP response code for Model version does not exists exception (#282) * issue 196 patch * Added openapi missing responses. - incorporated review comments * updated method signature after resolving merge conflicts Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Corrected paths in command and added notes (#354) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Refactored UT for better reporting (#159) * refactored UTs * upgraded testng version and updated testcase names * added testng.xml to handle sequential execution of test classes * added waveglow example * Updated readme for waveglow example * fixed failing UTs after merging master * removed unintentionally pushed waveglow example * fixed failing test case * moved set-default to utils and added couple of more add cases for default version * java formatting * corrected set default negative test case * fixed java formatting Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Huggingface Transformers generalization (#302) * Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model * modified to include mapping of predicitons and decoding received text * move back the original text * moved the index_to_name to bert folder * Transformers/ Bert serving with Torchserve readme * deleting an unused dictionary * just cleaning some of the commented out parts * cleaning up * Cleaning up * uncommenting the logging and cleaning the comments * Download and save the checkpoints, config file and vocab of the BERT from Transformers for demonstration purposes * mapping of predicitons to labels * a custom handler for transformers. * README. * sample of input text. * modifying the descriptions * modifying the descriptions * adding comments * adding commnets * Fromatting modifications * modified comments * added the citations * adding hugginface as the type of transformer model we are referring to * renaming the folder to huggingface_transformers * Removing the bert folder * simple modification * modifed the script to use a setup_config.json for loading and saving models * added the section for transfomer_handler_generalized * the generalized handler * set_up config intialization * edited for transfomer generalization * modified to use just input_ids as inputs for models * modified for question answering * modified to include the number of labels * corrected typos * modifed variable names * edited for clarity * modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case * modifed values for QA * modifed the variable names * added comments * modified to add clarity * add comment in preprocessing of QA * added warning for the case mode is not selected properly and combined constructor and initlaize fucntion * clean up * clean up * combined constructor and initliaze func * combined constructor and initliaze func * Separated the constructor and initliaze func * Modified to remove the classification part and update with generalized version * resolved the coflict and separated the costructor and initliaze func * removed the handler for classification case as the generalized version handles the case Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: mycpuorg <mail@mycpu.org> * added waveglow example (#258) * added waveglow example * Updated readme for waveglow example * enhanced example to remove runtime checkpoint download * updated handler to return the audio file content instead of dumping it to tmp dir * minor change to handler * added pipefail * added python snippet to run inference * updated readme with know issue with the example Co-authored-by: Aaqib <maaquib@gmail.com> * made build suite compatible with latest gradle (#348) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * moved env section after context * Deleting redundant ISSUE_TEMPLATE at root dir * Fix typo in scripts/install_from_src_macos * Corrected OpenApi description (#328) * Correct query parameter name (Registration Api)in OpenApi description * removed non existing path parameter Co-authored-by: Aaqib <maaquib@gmail.com> * Regression suite fixes and added test cases (#344) * added test model zoo test cases updated regression script * Updated Readme * update -y flag for install * Updated docker command as per new imagename and tag generated in stagging branch * Fix sudo usage in regression test script * - addressed review comments - reverted changes related to sudo in regression script. * added https test collection * - updated failed test cases on staging branch - added environment and updated newman cmd with --insecure for https * updated script * updated pytest * update curl check for serve up * Revert "update curl check for serve up" This reverts commit 0047eae. * Revert "update curl check for serve up" This reverts commit 0047eae. - update model zoo - added resnet test case * added workers to resnet model * Update readme to run test suite on desired branch * enhanced inference test cases to driven by datafile * Updated readme Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Fix typo in mnist readme (#330) Co-authored-by: Aaqib <maaquib@gmail.com> * Fixed typo in ab installation. (#327) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Fixed typo in ab installation. Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Setup Nightly Build for TorchServe (#371) * Add Branch Flag for Nightly CodeBuild #57 * Update ENV Variable for Nightly Build Name * Nigtly Build - Fix FileName for Build Artifacts * Update README for accessing test logs * Whitspace changes * Build new images for build - README Instruction * Fix for intermittent gradle build failure (#352) * Fix for intermittent null pointer exception * Fixed comments Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * documentation: Remove redundant curl options in example (#336) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * fix-doc * fix remaining "-X POST"s Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Corrected port number validation range (#304) * Corrected port number validation range * added UT for issue 291 * added ut dependency file formatting * correct param in config_test_env.properties * removed incorrect flags from test case config files * added negative test cases * removed properties file and changes in test case Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Fixes version number reporting for TorchServe (#360) * Correct version and added vesion.txt for getting version info * fixed description test cases and build issue * Added snapshot test fixes and resolved relative path issue for version.txt * python formatting * added version.txt in MANIFEST.in * addressed review comments * added version.txt for model archiver as well - addressed review comments * -fixed python UT -added ts python UT run in sanity script * removed blank lines from snapshot files Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * fixes installation script exits without error if no java present on machine (#376) * fixes installation script exits without error if no java present on machine * added changes for macos script Co-authored-by: Aaqib <maaquib@gmail.com> * Dockerfile enhancements (#308) * dockerfile now points to latest torchserve on pypi and added separate files for dev * Minor changes * Made build and run commands consistent * Fixing formatting * Consistency issue * Enhanced Dockerfile for binary per #178 and other misc. * Fixed gpu related issues in start script * Corrected the heading font * Added mar creation steps from container * added buildkit support for dev docker files * Fix for buildkit * Restoring psutil and future as absence leads intermittent hangged build * Fixed review comments for doc * Fixed typo Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Aaqib <maaquib@gmail.com> * Updated configuration.md for missing config properties (#337) * Updated configuration.md for missing parameters * changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT - removes ambiguity - follows naming convention * addressed review comments. * removed debug property * updated configuration.md Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Adding missing document links to README (#379) * Maven tests should output only errors (#326) Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> * fixed code for build failure in python virtual environment (#341) * fixed code for build failure in python virtual environment * added exception missed on resolving conflicts Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> * updated object detector handler to use only default cuda device for torchvision version less then 0.6 (#265) * updated object detector handler to use default cuda device for torchvision version 0.5.0 * updated version check logic * Updated documentation for torchvision version dependency * enhanced sanity script to validate all default handlers * fixed sanity script * added torchtext install step in sanity script * fixed issue related to inference after restart with snapshot. * moved torchtext to codebuild docker files * added more log statements * enhanced run inference function * fixed register model function * added unregister model function * incorporated code review comments * added back inference call for snapshot test * fixed if block Co-authored-by: Aaqib <maaquib@gmail.com> * Adding a barebones ec2 template to run torchserve on ubuntu (#301) * Adding a barebones ec2 template to run torchserve on ubuntu * Adding VPC with Private subnet * Address review comments #1 * Address review comments #2 * Rebase for #365 (#389) * Updated README.md for #364 See #364 for a deep description of the problem * updated README.md * updated README.md Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> * Document the docker settings for running in production mode (#388) * Docker Production Options #234 * Docker Production Options - Typo Fixes / Fit & Finish #234 * Typos - Docker Docs #234 * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Huggingface transformers integration test (#382) * Add Huggingface transformers example integration tests * Add transformers intallation * Fix a missing bracket * move transformers installation to chain installations Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Updated docker README.md (#401) * Update README.md * Fixed cd command * Made source cloning steps explicit!! * Fixed online editing mistake Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Fixed response status message for inferencing on model with zero workers (#305) * Changes for issue_29 * Changed error Messages * changed message for scalling request * changed defaultVersion string * Corrected error message * changed the code for best practice * resolved review comment for version * updated management api doc as per new responses in worker scale api Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * made model server port tests independent (#396) * Added backend profiling and fixed benchmark's install dependency scripts (#402) * added backend profiling * updated install dependency scripts to make them jmeter version independent * updated readme * Fix benchmarks/README typo and JMeter versioning issue * updated documentation Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * Add sphinx directory and conf.py (#274) * Add sphinx folder that contains files for docs build * Delete .DS_Store * Delete .DS_Store * Add conf.py in docs/ * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update to the latest version of sphinx * Add recommonmark to extensions Add recommonmark to extensions so sphinx project has markdown support. Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html * Update source suffix to use latest recommonmark syntax Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html Co-authored-by: Aaron Markham <markhama@amazon.com> * Remove old recommonmark parser syntax * Pin sphinx version to work with latest version of recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Pin version latest version recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Delete theme file generated during build * de-duplicate install package of psutil (#329) The Dockerfile of GPU install package psutil duplicated. * Remove pytorch-sphinx-theme files * Update to match latest staging_0_1_1 * Update docker directory to match staging_0_1_1 branch * Delete .DS_Store * Add sphinxcontrib.katex and torch Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Image classifier example readme fixes (#405) * Changed the model name in torch.jit.save to be consistent with docs * Remove extra handler flag and fix the torchservr start * fix the doc * fix the doc Co-authored-by: mycpuorg <mail@mycpu.org> * Porting memory fix changes to TorchServe (#422) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Benchmark documentation fix (#425) * Fix Benchmark Documentation * Documentaion fixes * .circleci: Add initial circleci configuration (#431) Signed-off-by: Eli Uriegas <eliuriegas@fb.com> * Torchscript support huggingface transfrmers (#421) * Add support for torchscript * Add support for torchscript * Add key values for torchscript support * Add Torchscript suport * Add Torchscript support * Add Torchscript support in loading the models * keep the example to QA * modify to reflect Torchscript example? * Add torchscript support guides * Add fintued BERT for QA * Add QA artifacts * Add seqenece classification artifacts * Token classification artifacts * Correct typos * Add max length to tokenizer * clean up * enhance the doc * change setup to seq classification * Add detail to torchscript support * set the model for QA * Add torchscript bert seq classification example * Add bert seq classification example * Modify the packaging model command * Modify torchscript packaging command * clean up * clean up * correct spacing between list items Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Integration test huggingface transfromers torchscript support (#433) * made model server port tests independent * Add integration tests for Huggingface transfromers with new handler that support torchscript Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Apachebench benchmark - cleanup script / documentation * Revert "Apachebench benchmark - cleanup script / documentation" This reverts commit c8f8682. * Documentation updates for clarity on Eager mode / Torch scripted models (#439) * Model Zoo updates for Eager Mode / Torchscripted models * Update examples/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Apachebench benchmarking - Script / Documentation updates (#438) * Apachebench benchmarking script - Script / Documentation updates * Stop TS after benchmark completion. Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com> Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Brad Heintz <bradheintz@fb.com> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal> Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com> Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> Co-authored-by: alvations <alvations@gmail.com> Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Jessica Lin <jplin@fb.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
maaquib
added a commit
that referenced
this pull request
Jun 9, 2020
* fixed resnet152 custom handler to work with GPU * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler * issue templates (#273) * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler Co-authored-by: mycpuorg <mail@mycpu.org> * Fixes #205 - Docker install not finding GPUs (#262) Invokes docker with --runtime=nvidia for GPU Added the option to specify the GPU / specific GPU ids in start.sh Fixed the documentation for start.sh script Fixed the JDK version in Dockerfile.gpu Co-authored-by: Aaqib <maaquib@gmail.com> * Fix pylint issue (#163) * pylint fixes and code cleanup * UT fixes, pylint updates per latest version and updated sanity script to run pylint * fixed pylint and wrapt installation issue * freeze pylint version to 2.4.4 due to bug in version 2.5 * removed duplicate statement to install pylint Co-authored-by: Aaqib <maaquib@gmail.com> * Updated exception handling for worker threads shutdown. (#172) * updated code to now log errors only in case when exception is not genreated by unregister api call * java formatting * fixed illigalstateexception * removed the unrequired handling for illegal monitor state exception Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> * changes for Homebrew issue 98 (#126) * changes for Homebrew issue 98 * adding changes for path in benchmark.py * CI Build corrected * benchmark changes corrected path * changed PATH for homebrew * Update Readme Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh * Updated Readme.md removed unnecessory lines * corrected indentation corrected indentation for new lines added * Update Readme.md fil added separate commands for CPU and GPU based Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * added changes to clean model loaded through URL from model store on unregisted * added UTs * Moving the plugins SDK to org.pytorch. Added plugins example. (#261) * Moved plugins SDK to org.pytorch * Updating the repo for publishing plugins SDK * Enabled bintray maven repo temporarily * Reset tests * Added GPG plugin to pom file to sign the plugins SDK * Update the SDK version * Removed bintray repo * Modify the buildspec to not deploy the SDK Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Added model zoo (#299) Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Brad Heintz <bradheintz@fb.com> * Removed snapshot dependency on model (#152) * Snapshot is now independent of model * fixed checkstyle issue * fixed formatting * doc review: configuration and custom_service * addressed review comments * fixed incorrect merge * fixed review comments and corrected the error messages * adding remaining fixes for string constants Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 (#285) * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 * Script to clone Pytorch Repo & Run Post Man Scripts * Basic Tests for Inference & Managment APIs * AWS Code Build buildspec.yml * Refactor postman tests, add empty postman collection for todos * Add a readme for adding tests * Add more management tests * Fix typo * Snapshot Regression tests - #57 * Fix typo * Comment out snapshot test for now * Weird typos are showing up, try saving everything * Need to manually save every request in postman... * When reregister don't use s3 * More workarounds for rereg * Fix test messages * Remove test for empty reqs * Change get to post * Fix pytest for subprocess - #57 * Enable pytest - #57 * Inference API Testcases / Rename Management Testcases #57 * Wipe Model MARs in Modelstore between tests #54 * Fix Inference Endpoint Execution #54 * Fix file paths - #57 * Fix Initial Workers #57 * Fix Inference Assertions #57 * Fix DenseNet Inference #57 * Fix Densenet Register #54 * Redirect Logs to File #54 * Fix Test Execution Log #54 * README, Build Badge #57 * Format ReadMe #57 * Update README - Add Instruction to add new tests / Run tests locally * Update README * Update branch for build Codebuild Badge Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> * Issue 312 install from src (#322) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update instructions for installation from source This commit aims to address Issue #312 Tested on a fresh EC2 P3.2 instance: * Followed instructions under https://github.com/pytorch/serve#install-torchserve-for-development * Started the server manually from command line * Ran a ping query Received a response from server with "Healthy" status Signed-off-by: Manoj Rao <mail@mycpu.org> * add separate files for installation one each for linux and macos * update README and remove relative path from script * Install script - Dont uninstall if not previously installed - #132 * explicitly build the source code with gradle prior to install * remove upgrade instructions in favor of clean reinstall from source * update instructions to contain the correct script name Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Manoj Rao <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> * fix for macos script (#338) * Move Issue Template to .github dir * Updated sanity script to use install_from_src_ubuntu script (#350) * updated sanity script to use install_from_src_ubuntu script * added java version check * fixed issue with if statement * fixed variable name * fixed repeated cd command * updated artifacts * removed artifacts from buildspec * Corrected HTTP response code for Model version does not exists exception (#282) * issue 196 patch * Added openapi missing responses. - incorporated review comments * updated method signature after resolving merge conflicts Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Corrected paths in command and added notes (#354) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Refactored UT for better reporting (#159) * refactored UTs * upgraded testng version and updated testcase names * added testng.xml to handle sequential execution of test classes * added waveglow example * Updated readme for waveglow example * fixed failing UTs after merging master * removed unintentionally pushed waveglow example * fixed failing test case * moved set-default to utils and added couple of more add cases for default version * java formatting * corrected set default negative test case * fixed java formatting Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Huggingface Transformers generalization (#302) * Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model * modified to include mapping of predicitons and decoding received text * move back the original text * moved the index_to_name to bert folder * Transformers/ Bert serving with Torchserve readme * deleting an unused dictionary * just cleaning some of the commented out parts * cleaning up * Cleaning up * uncommenting the logging and cleaning the comments * Download and save the checkpoints, config file and vocab of the BERT from Transformers for demonstration purposes * mapping of predicitons to labels * a custom handler for transformers. * README. * sample of input text. * modifying the descriptions * modifying the descriptions * adding comments * adding commnets * Fromatting modifications * modified comments * added the citations * adding hugginface as the type of transformer model we are referring to * renaming the folder to huggingface_transformers * Removing the bert folder * simple modification * modifed the script to use a setup_config.json for loading and saving models * added the section for transfomer_handler_generalized * the generalized handler * set_up config intialization * edited for transfomer generalization * modified to use just input_ids as inputs for models * modified for question answering * modified to include the number of labels * corrected typos * modifed variable names * edited for clarity * modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case * modifed values for QA * modifed the variable names * added comments * modified to add clarity * add comment in preprocessing of QA * added warning for the case mode is not selected properly and combined constructor and initlaize fucntion * clean up * clean up * combined constructor and initliaze func * combined constructor and initliaze func * Separated the constructor and initliaze func * Modified to remove the classification part and update with generalized version * resolved the coflict and separated the costructor and initliaze func * removed the handler for classification case as the generalized version handles the case Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: mycpuorg <mail@mycpu.org> * added waveglow example (#258) * added waveglow example * Updated readme for waveglow example * enhanced example to remove runtime checkpoint download * updated handler to return the audio file content instead of dumping it to tmp dir * minor change to handler * added pipefail * added python snippet to run inference * updated readme with know issue with the example Co-authored-by: Aaqib <maaquib@gmail.com> * made build suite compatible with latest gradle (#348) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * moved env section after context * Deleting redundant ISSUE_TEMPLATE at root dir * Fix typo in scripts/install_from_src_macos * Corrected OpenApi description (#328) * Correct query parameter name (Registration Api)in OpenApi description * removed non existing path parameter Co-authored-by: Aaqib <maaquib@gmail.com> * Regression suite fixes and added test cases (#344) * added test model zoo test cases updated regression script * Updated Readme * update -y flag for install * Updated docker command as per new imagename and tag generated in stagging branch * Fix sudo usage in regression test script * - addressed review comments - reverted changes related to sudo in regression script. * added https test collection * - updated failed test cases on staging branch - added environment and updated newman cmd with --insecure for https * updated script * updated pytest * update curl check for serve up * Revert "update curl check for serve up" This reverts commit 0047eae. * Revert "update curl check for serve up" This reverts commit 0047eae. - update model zoo - added resnet test case * added workers to resnet model * Update readme to run test suite on desired branch * enhanced inference test cases to driven by datafile * Updated readme Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Fix typo in mnist readme (#330) Co-authored-by: Aaqib <maaquib@gmail.com> * Fixed typo in ab installation. (#327) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Fixed typo in ab installation. Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Setup Nightly Build for TorchServe (#371) * Add Branch Flag for Nightly CodeBuild #57 * Update ENV Variable for Nightly Build Name * Nigtly Build - Fix FileName for Build Artifacts * Update README for accessing test logs * Whitspace changes * Build new images for build - README Instruction * Fix for intermittent gradle build failure (#352) * Fix for intermittent null pointer exception * Fixed comments Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * documentation: Remove redundant curl options in example (#336) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * fix-doc * fix remaining "-X POST"s Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Corrected port number validation range (#304) * Corrected port number validation range * added UT for issue 291 * added ut dependency file formatting * correct param in config_test_env.properties * removed incorrect flags from test case config files * added negative test cases * removed properties file and changes in test case Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Fixes version number reporting for TorchServe (#360) * Correct version and added vesion.txt for getting version info * fixed description test cases and build issue * Added snapshot test fixes and resolved relative path issue for version.txt * python formatting * added version.txt in MANIFEST.in * addressed review comments * added version.txt for model archiver as well - addressed review comments * -fixed python UT -added ts python UT run in sanity script * removed blank lines from snapshot files Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * fixes installation script exits without error if no java present on machine (#376) * fixes installation script exits without error if no java present on machine * added changes for macos script Co-authored-by: Aaqib <maaquib@gmail.com> * Dockerfile enhancements (#308) * dockerfile now points to latest torchserve on pypi and added separate files for dev * Minor changes * Made build and run commands consistent * Fixing formatting * Consistency issue * Enhanced Dockerfile for binary per #178 and other misc. * Fixed gpu related issues in start script * Corrected the heading font * Added mar creation steps from container * added buildkit support for dev docker files * Fix for buildkit * Restoring psutil and future as absence leads intermittent hangged build * Fixed review comments for doc * Fixed typo Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Aaqib <maaquib@gmail.com> * Updated configuration.md for missing config properties (#337) * Updated configuration.md for missing parameters * changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT - removes ambiguity - follows naming convention * addressed review comments. * removed debug property * updated configuration.md Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Adding missing document links to README (#379) * Maven tests should output only errors (#326) Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> * fixed code for build failure in python virtual environment (#341) * fixed code for build failure in python virtual environment * added exception missed on resolving conflicts Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> * updated object detector handler to use only default cuda device for torchvision version less then 0.6 (#265) * updated object detector handler to use default cuda device for torchvision version 0.5.0 * updated version check logic * Updated documentation for torchvision version dependency * enhanced sanity script to validate all default handlers * fixed sanity script * added torchtext install step in sanity script * fixed issue related to inference after restart with snapshot. * moved torchtext to codebuild docker files * added more log statements * enhanced run inference function * fixed register model function * added unregister model function * incorporated code review comments * added back inference call for snapshot test * fixed if block Co-authored-by: Aaqib <maaquib@gmail.com> * Adding a barebones ec2 template to run torchserve on ubuntu (#301) * Adding a barebones ec2 template to run torchserve on ubuntu * Adding VPC with Private subnet * Address review comments #1 * Address review comments #2 * Rebase for #365 (#389) * Updated README.md for #364 See #364 for a deep description of the problem * updated README.md * updated README.md Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> * Document the docker settings for running in production mode (#388) * Docker Production Options #234 * Docker Production Options - Typo Fixes / Fit & Finish #234 * Typos - Docker Docs #234 * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Huggingface transformers integration test (#382) * Add Huggingface transformers example integration tests * Add transformers intallation * Fix a missing bracket * move transformers installation to chain installations Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Updated docker README.md (#401) * Update README.md * Fixed cd command * Made source cloning steps explicit!! * Fixed online editing mistake Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Fixed response status message for inferencing on model with zero workers (#305) * Changes for issue_29 * Changed error Messages * changed message for scalling request * changed defaultVersion string * Corrected error message * changed the code for best practice * resolved review comment for version * updated management api doc as per new responses in worker scale api Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * made model server port tests independent (#396) * Added backend profiling and fixed benchmark's install dependency scripts (#402) * added backend profiling * updated install dependency scripts to make them jmeter version independent * updated readme * Fix benchmarks/README typo and JMeter versioning issue * updated documentation Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * Add sphinx directory and conf.py (#274) * Add sphinx folder that contains files for docs build * Delete .DS_Store * Delete .DS_Store * Add conf.py in docs/ * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update to the latest version of sphinx * Add recommonmark to extensions Add recommonmark to extensions so sphinx project has markdown support. Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html * Update source suffix to use latest recommonmark syntax Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html Co-authored-by: Aaron Markham <markhama@amazon.com> * Remove old recommonmark parser syntax * Pin sphinx version to work with latest version of recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Pin version latest version recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Delete theme file generated during build * de-duplicate install package of psutil (#329) The Dockerfile of GPU install package psutil duplicated. * Remove pytorch-sphinx-theme files * Update to match latest staging_0_1_1 * Update docker directory to match staging_0_1_1 branch * Delete .DS_Store * Add sphinxcontrib.katex and torch Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Image classifier example readme fixes (#405) * Changed the model name in torch.jit.save to be consistent with docs * Remove extra handler flag and fix the torchservr start * fix the doc * fix the doc Co-authored-by: mycpuorg <mail@mycpu.org> * Porting memory fix changes to TorchServe (#422) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Benchmark documentation fix (#425) * Fix Benchmark Documentation * Documentaion fixes * .circleci: Add initial circleci configuration (#431) Signed-off-by: Eli Uriegas <eliuriegas@fb.com> * Torchscript support huggingface transfrmers (#421) * Add support for torchscript * Add support for torchscript * Add key values for torchscript support * Add Torchscript suport * Add Torchscript support * Add Torchscript support in loading the models * keep the example to QA * modify to reflect Torchscript example? * Add torchscript support guides * Add fintued BERT for QA * Add QA artifacts * Add seqenece classification artifacts * Token classification artifacts * Correct typos * Add max length to tokenizer * clean up * enhance the doc * change setup to seq classification * Add detail to torchscript support * set the model for QA * Add torchscript bert seq classification example * Add bert seq classification example * Modify the packaging model command * Modify torchscript packaging command * clean up * clean up * correct spacing between list items Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Integration test huggingface transfromers torchscript support (#433) * made model server port tests independent * Add integration tests for Huggingface transfromers with new handler that support torchscript Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Apachebench benchmark - cleanup script / documentation * Revert "Apachebench benchmark - cleanup script / documentation" This reverts commit c8f8682. * Documentation updates for clarity on Eager mode / Torch scripted models (#439) * Model Zoo updates for Eager Mode / Torchscripted models * Update examples/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Apachebench benchmarking - Script / Documentation updates (#438) * Apachebench benchmarking script - Script / Documentation updates * Stop TS after benchmark completion. Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com> Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Brad Heintz <bradheintz@fb.com> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal> Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com> Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> Co-authored-by: alvations <alvations@gmail.com> Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Jessica Lin <jplin@fb.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
dhanainme
added a commit
that referenced
this pull request
Jun 25, 2020
* fixed resnet152 custom handler to work with GPU * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler * issue templates (#273) * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler Co-authored-by: mycpuorg <mail@mycpu.org> * Fixes #205 - Docker install not finding GPUs (#262) Invokes docker with --runtime=nvidia for GPU Added the option to specify the GPU / specific GPU ids in start.sh Fixed the documentation for start.sh script Fixed the JDK version in Dockerfile.gpu Co-authored-by: Aaqib <maaquib@gmail.com> * Fix pylint issue (#163) * pylint fixes and code cleanup * UT fixes, pylint updates per latest version and updated sanity script to run pylint * fixed pylint and wrapt installation issue * freeze pylint version to 2.4.4 due to bug in version 2.5 * removed duplicate statement to install pylint Co-authored-by: Aaqib <maaquib@gmail.com> * Updated exception handling for worker threads shutdown. (#172) * updated code to now log errors only in case when exception is not genreated by unregister api call * java formatting * fixed illigalstateexception * removed the unrequired handling for illegal monitor state exception Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> * changes for Homebrew issue 98 (#126) * changes for Homebrew issue 98 * adding changes for path in benchmark.py * CI Build corrected * benchmark changes corrected path * changed PATH for homebrew * Update Readme Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh * Updated Readme.md removed unnecessory lines * corrected indentation corrected indentation for new lines added * Update Readme.md fil added separate commands for CPU and GPU based Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * added changes to clean model loaded through URL from model store on unregisted * added UTs * Moving the plugins SDK to org.pytorch. Added plugins example. (#261) * Moved plugins SDK to org.pytorch * Updating the repo for publishing plugins SDK * Enabled bintray maven repo temporarily * Reset tests * Added GPG plugin to pom file to sign the plugins SDK * Update the SDK version * Removed bintray repo * Modify the buildspec to not deploy the SDK Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Added model zoo (#299) Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Brad Heintz <bradheintz@fb.com> * Removed snapshot dependency on model (#152) * Snapshot is now independent of model * fixed checkstyle issue * fixed formatting * doc review: configuration and custom_service * addressed review comments * fixed incorrect merge * fixed review comments and corrected the error messages * adding remaining fixes for string constants Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 (#285) * Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 * Script to clone Pytorch Repo & Run Post Man Scripts * Basic Tests for Inference & Managment APIs * AWS Code Build buildspec.yml * Refactor postman tests, add empty postman collection for todos * Add a readme for adding tests * Add more management tests * Fix typo * Snapshot Regression tests - #57 * Fix typo * Comment out snapshot test for now * Weird typos are showing up, try saving everything * Need to manually save every request in postman... * When reregister don't use s3 * More workarounds for rereg * Fix test messages * Remove test for empty reqs * Change get to post * Fix pytest for subprocess - #57 * Enable pytest - #57 * Inference API Testcases / Rename Management Testcases #57 * Wipe Model MARs in Modelstore between tests #54 * Fix Inference Endpoint Execution #54 * Fix file paths - #57 * Fix Initial Workers #57 * Fix Inference Assertions #57 * Fix DenseNet Inference #57 * Fix Densenet Register #54 * Redirect Logs to File #54 * Fix Test Execution Log #54 * README, Build Badge #57 * Format ReadMe #57 * Update README - Add Instruction to add new tests / Run tests locally * Update README * Update branch for build Codebuild Badge Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> * Issue 312 install from src (#322) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update instructions for installation from source This commit aims to address Issue #312 Tested on a fresh EC2 P3.2 instance: * Followed instructions under https://github.com/pytorch/serve#install-torchserve-for-development * Started the server manually from command line * Ran a ping query Received a response from server with "Healthy" status Signed-off-by: Manoj Rao <mail@mycpu.org> * add separate files for installation one each for linux and macos * update README and remove relative path from script * Install script - Dont uninstall if not previously installed - #132 * explicitly build the source code with gradle prior to install * remove upgrade instructions in favor of clean reinstall from source * update instructions to contain the correct script name Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Manoj Rao <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> * fix for macos script (#338) * Move Issue Template to .github dir * Updated sanity script to use install_from_src_ubuntu script (#350) * updated sanity script to use install_from_src_ubuntu script * added java version check * fixed issue with if statement * fixed variable name * fixed repeated cd command * updated artifacts * removed artifacts from buildspec * Corrected HTTP response code for Model version does not exists exception (#282) * issue 196 patch * Added openapi missing responses. - incorporated review comments * updated method signature after resolving merge conflicts Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Corrected paths in command and added notes (#354) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Refactored UT for better reporting (#159) * refactored UTs * upgraded testng version and updated testcase names * added testng.xml to handle sequential execution of test classes * added waveglow example * Updated readme for waveglow example * fixed failing UTs after merging master * removed unintentionally pushed waveglow example * fixed failing test case * moved set-default to utils and added couple of more add cases for default version * java formatting * corrected set default negative test case * fixed java formatting Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Huggingface Transformers generalization (#302) * Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model * modified to include mapping of predicitons and decoding received text * move back the original text * moved the index_to_name to bert folder * Transformers/ Bert serving with Torchserve readme * deleting an unused dictionary * just cleaning some of the commented out parts * cleaning up * Cleaning up * uncommenting the logging and cleaning the comments * Download and save the checkpoints, config file and vocab of the BERT from Transformers for demonstration purposes * mapping of predicitons to labels * a custom handler for transformers. * README. * sample of input text. * modifying the descriptions * modifying the descriptions * adding comments * adding commnets * Fromatting modifications * modified comments * added the citations * adding hugginface as the type of transformer model we are referring to * renaming the folder to huggingface_transformers * Removing the bert folder * simple modification * modifed the script to use a setup_config.json for loading and saving models * added the section for transfomer_handler_generalized * the generalized handler * set_up config intialization * edited for transfomer generalization * modified to use just input_ids as inputs for models * modified for question answering * modified to include the number of labels * corrected typos * modifed variable names * edited for clarity * modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case * modifed values for QA * modifed the variable names * added comments * modified to add clarity * add comment in preprocessing of QA * added warning for the case mode is not selected properly and combined constructor and initlaize fucntion * clean up * clean up * combined constructor and initliaze func * combined constructor and initliaze func * Separated the constructor and initliaze func * Modified to remove the classification part and update with generalized version * resolved the coflict and separated the costructor and initliaze func * removed the handler for classification case as the generalized version handles the case Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: mycpuorg <mail@mycpu.org> * added waveglow example (#258) * added waveglow example * Updated readme for waveglow example * enhanced example to remove runtime checkpoint download * updated handler to return the audio file content instead of dumping it to tmp dir * minor change to handler * added pipefail * added python snippet to run inference * updated readme with know issue with the example Co-authored-by: Aaqib <maaquib@gmail.com> * made build suite compatible with latest gradle (#348) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * moved env section after context * Deleting redundant ISSUE_TEMPLATE at root dir * Fix typo in scripts/install_from_src_macos * Corrected OpenApi description (#328) * Correct query parameter name (Registration Api)in OpenApi description * removed non existing path parameter Co-authored-by: Aaqib <maaquib@gmail.com> * Regression suite fixes and added test cases (#344) * added test model zoo test cases updated regression script * Updated Readme * update -y flag for install * Updated docker command as per new imagename and tag generated in stagging branch * Fix sudo usage in regression test script * - addressed review comments - reverted changes related to sudo in regression script. * added https test collection * - updated failed test cases on staging branch - added environment and updated newman cmd with --insecure for https * updated script * updated pytest * update curl check for serve up * Revert "update curl check for serve up" This reverts commit 0047eae. * Revert "update curl check for serve up" This reverts commit 0047eae. - update model zoo - added resnet test case * added workers to resnet model * Update readme to run test suite on desired branch * enhanced inference test cases to driven by datafile * Updated readme Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Fix typo in mnist readme (#330) Co-authored-by: Aaqib <maaquib@gmail.com> * Fixed typo in ab installation. (#327) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Fixed typo in ab installation. Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Setup Nightly Build for TorchServe (#371) * Add Branch Flag for Nightly CodeBuild #57 * Update ENV Variable for Nightly Build Name * Nigtly Build - Fix FileName for Build Artifacts * Update README for accessing test logs * Whitspace changes * Build new images for build - README Instruction * Fix for intermittent gradle build failure (#352) * Fix for intermittent null pointer exception * Fixed comments Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * documentation: Remove redundant curl options in example (#336) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * fix-doc * fix remaining "-X POST"s Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Corrected port number validation range (#304) * Corrected port number validation range * added UT for issue 291 * added ut dependency file formatting * correct param in config_test_env.properties * removed incorrect flags from test case config files * added negative test cases * removed properties file and changes in test case Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Fixes version number reporting for TorchServe (#360) * Correct version and added vesion.txt for getting version info * fixed description test cases and build issue * Added snapshot test fixes and resolved relative path issue for version.txt * python formatting * added version.txt in MANIFEST.in * addressed review comments * added version.txt for model archiver as well - addressed review comments * -fixed python UT -added ts python UT run in sanity script * removed blank lines from snapshot files Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * fixes installation script exits without error if no java present on machine (#376) * fixes installation script exits without error if no java present on machine * added changes for macos script Co-authored-by: Aaqib <maaquib@gmail.com> * Dockerfile enhancements (#308) * dockerfile now points to latest torchserve on pypi and added separate files for dev * Minor changes * Made build and run commands consistent * Fixing formatting * Consistency issue * Enhanced Dockerfile for binary per #178 and other misc. * Fixed gpu related issues in start script * Corrected the heading font * Added mar creation steps from container * added buildkit support for dev docker files * Fix for buildkit * Restoring psutil and future as absence leads intermittent hangged build * Fixed review comments for doc * Fixed typo Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Aaqib <maaquib@gmail.com> * Updated configuration.md for missing config properties (#337) * Updated configuration.md for missing parameters * changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT - removes ambiguity - follows naming convention * addressed review comments. * removed debug property * updated configuration.md Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Adding missing document links to README (#379) * Maven tests should output only errors (#326) Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> * fixed code for build failure in python virtual environment (#341) * fixed code for build failure in python virtual environment * added exception missed on resolving conflicts Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> * updated object detector handler to use only default cuda device for torchvision version less then 0.6 (#265) * updated object detector handler to use default cuda device for torchvision version 0.5.0 * updated version check logic * Updated documentation for torchvision version dependency * enhanced sanity script to validate all default handlers * fixed sanity script * added torchtext install step in sanity script * fixed issue related to inference after restart with snapshot. * moved torchtext to codebuild docker files * added more log statements * enhanced run inference function * fixed register model function * added unregister model function * incorporated code review comments * added back inference call for snapshot test * fixed if block Co-authored-by: Aaqib <maaquib@gmail.com> * Adding a barebones ec2 template to run torchserve on ubuntu (#301) * Adding a barebones ec2 template to run torchserve on ubuntu * Adding VPC with Private subnet * Address review comments #1 * Address review comments #2 * Rebase for #365 (#389) * Updated README.md for #364 See #364 for a deep description of the problem * updated README.md * updated README.md Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> * Document the docker settings for running in production mode (#388) * Docker Production Options #234 * Docker Production Options - Typo Fixes / Fit & Finish #234 * Typos - Docker Docs #234 * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Huggingface transformers integration test (#382) * Add Huggingface transformers example integration tests * Add transformers intallation * Fix a missing bracket * move transformers installation to chain installations Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Updated docker README.md (#401) * Update README.md * Fixed cd command * Made source cloning steps explicit!! * Fixed online editing mistake Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Fixed response status message for inferencing on model with zero workers (#305) * Changes for issue_29 * Changed error Messages * changed message for scalling request * changed defaultVersion string * Corrected error message * changed the code for best practice * resolved review comment for version * updated management api doc as per new responses in worker scale api Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * made model server port tests independent (#396) * Added backend profiling and fixed benchmark's install dependency scripts (#402) * added backend profiling * updated install dependency scripts to make them jmeter version independent * updated readme * Fix benchmarks/README typo and JMeter versioning issue * updated documentation Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * Add sphinx directory and conf.py (#274) * Add sphinx folder that contains files for docs build * Delete .DS_Store * Delete .DS_Store * Add conf.py in docs/ * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes #297 * remove minus y (#311) update pip install instructions * Update to the latest version of sphinx * Add recommonmark to extensions Add recommonmark to extensions so sphinx project has markdown support. Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html * Update source suffix to use latest recommonmark syntax Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html Co-authored-by: Aaron Markham <markhama@amazon.com> * Remove old recommonmark parser syntax * Pin sphinx version to work with latest version of recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Pin version latest version recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Delete theme file generated during build * de-duplicate install package of psutil (#329) The Dockerfile of GPU install package psutil duplicated. * Remove pytorch-sphinx-theme files * Update to match latest staging_0_1_1 * Update docker directory to match staging_0_1_1 branch * Delete .DS_Store * Add sphinxcontrib.katex and torch Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * fix for response time out issue during build * Image classifier example readme fixes (#405) * Changed the model name in torch.jit.save to be consistent with docs * Remove extra handler flag and fix the torchservr start * fix the doc * fix the doc Co-authored-by: mycpuorg <mail@mycpu.org> * Porting memory fix changes to TorchServe (#422) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Benchmark documentation fix (#425) * Fix Benchmark Documentation * Documentaion fixes * .circleci: Add initial circleci configuration (#431) Signed-off-by: Eli Uriegas <eliuriegas@fb.com> * Torchscript support huggingface transfrmers (#421) * Add support for torchscript * Add support for torchscript * Add key values for torchscript support * Add Torchscript suport * Add Torchscript support * Add Torchscript support in loading the models * keep the example to QA * modify to reflect Torchscript example? * Add torchscript support guides * Add fintued BERT for QA * Add QA artifacts * Add seqenece classification artifacts * Token classification artifacts * Correct typos * Add max length to tokenizer * clean up * enhance the doc * change setup to seq classification * Add detail to torchscript support * set the model for QA * Add torchscript bert seq classification example * Add bert seq classification example * Modify the packaging model command * Modify torchscript packaging command * clean up * clean up * correct spacing between list items Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Integration test huggingface transfromers torchscript support (#433) * made model server port tests independent * Add integration tests for Huggingface transfromers with new handler that support torchscript Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Apachebench benchmark - cleanup script / documentation * Revert "Apachebench benchmark - cleanup script / documentation" This reverts commit c8f8682. * Documentation updates for clarity on Eager mode / Torch scripted models (#439) * Model Zoo updates for Eager Mode / Torchscripted models * Update examples/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Apachebench benchmarking - Script / Documentation updates (#438) * Apachebench benchmarking script - Script / Documentation updates * Stop TS after benchmark completion. * Adding system diagram and brief explanation of workings (#413) * Adding system diagram and brief explanation of workings * Adding number of worker recommendations * Adding description of netty configuration params * Update header for worker threads readme section Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Added TODO statement * rebased to master * rebased to master Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com> Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Brad Heintz <bradheintz@fb.com> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal> Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com> Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> Co-authored-by: alvations <alvations@gmail.com> Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Jessica Lin <jplin@fb.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
htappen
pushed a commit
to htappen/serve
that referenced
this pull request
Jun 30, 2020
* fixed resnet152 custom handler to work with GPU * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler * issue templates (pytorch#273) * issue template * Moving issue templates to root folder per pull request template * adding feature request template * Minor changes related to sections and names. * Update feature_template.md * Update bug_template.md * doc template * minor changes * Minor changes to upload custom model or handler Co-authored-by: mycpuorg <mail@mycpu.org> * Fixes pytorch#205 - Docker install not finding GPUs (pytorch#262) Invokes docker with --runtime=nvidia for GPU Added the option to specify the GPU / specific GPU ids in start.sh Fixed the documentation for start.sh script Fixed the JDK version in Dockerfile.gpu Co-authored-by: Aaqib <maaquib@gmail.com> * Fix pylint issue (pytorch#163) * pylint fixes and code cleanup * UT fixes, pylint updates per latest version and updated sanity script to run pylint * fixed pylint and wrapt installation issue * freeze pylint version to 2.4.4 due to bug in version 2.5 * removed duplicate statement to install pylint Co-authored-by: Aaqib <maaquib@gmail.com> * Updated exception handling for worker threads shutdown. (pytorch#172) * updated code to now log errors only in case when exception is not genreated by unregister api call * java formatting * fixed illigalstateexception * removed the unrequired handling for illegal monitor state exception Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> * changes for Homebrew issue 98 (pytorch#126) * changes for Homebrew issue 98 * adding changes for path in benchmark.py * CI Build corrected * benchmark changes corrected path * changed PATH for homebrew * Update Readme Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh * Updated Readme.md removed unnecessory lines * corrected indentation corrected indentation for new lines added * Update Readme.md fil added separate commands for CPU and GPU based Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * added changes to clean model loaded through URL from model store on unregisted * added UTs * Moving the plugins SDK to org.pytorch. Added plugins example. (pytorch#261) * Moved plugins SDK to org.pytorch * Updating the repo for publishing plugins SDK * Enabled bintray maven repo temporarily * Reset tests * Added GPG plugin to pom file to sign the plugins SDK * Update the SDK version * Removed bintray repo * Modify the buildspec to not deploy the SDK Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Added model zoo (pytorch#299) Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Brad Heintz <bradheintz@fb.com> * Removed snapshot dependency on model (pytorch#152) * Snapshot is now independent of model * fixed checkstyle issue * fixed formatting * doc review: configuration and custom_service * addressed review comments * fixed incorrect merge * fixed review comments and corrected the error messages * adding remaining fixes for string constants Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Regression Tests Suite - AWS Code Build Infra - Initial Commit for pytorch#57 (pytorch#285) * Regression Tests Suite - AWS Code Build Infra - Initial Commit for pytorch#57 * Script to clone Pytorch Repo & Run Post Man Scripts * Basic Tests for Inference & Managment APIs * AWS Code Build buildspec.yml * Refactor postman tests, add empty postman collection for todos * Add a readme for adding tests * Add more management tests * Fix typo * Snapshot Regression tests - pytorch#57 * Fix typo * Comment out snapshot test for now * Weird typos are showing up, try saving everything * Need to manually save every request in postman... * When reregister don't use s3 * More workarounds for rereg * Fix test messages * Remove test for empty reqs * Change get to post * Fix pytest for subprocess - pytorch#57 * Enable pytest - pytorch#57 * Inference API Testcases / Rename Management Testcases pytorch#57 * Wipe Model MARs in Modelstore between tests pytorch#54 * Fix Inference Endpoint Execution pytorch#54 * Fix file paths - pytorch#57 * Fix Initial Workers pytorch#57 * Fix Inference Assertions pytorch#57 * Fix DenseNet Inference pytorch#57 * Fix Densenet Register pytorch#54 * Redirect Logs to File pytorch#54 * Fix Test Execution Log pytorch#54 * README, Build Badge pytorch#57 * Format ReadMe pytorch#57 * Update README - Add Instruction to add new tests / Run tests locally * Update README * Update branch for build Codebuild Badge Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> * Issue 312 install from src (pytorch#322) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes pytorch#297 * remove minus y (pytorch#311) update pip install instructions * Update instructions for installation from source This commit aims to address Issue pytorch#312 Tested on a fresh EC2 P3.2 instance: * Followed instructions under https://github.com/pytorch/serve#install-torchserve-for-development * Started the server manually from command line * Ran a ping query Received a response from server with "Healthy" status Signed-off-by: Manoj Rao <mail@mycpu.org> * add separate files for installation one each for linux and macos * update README and remove relative path from script * Install script - Dont uninstall if not previously installed - pytorch#132 * explicitly build the source code with gradle prior to install * remove upgrade instructions in favor of clean reinstall from source * update instructions to contain the correct script name Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Manoj Rao <mail@mycpu.org> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> * fix for macos script (pytorch#338) * Move Issue Template to .github dir * Updated sanity script to use install_from_src_ubuntu script (pytorch#350) * updated sanity script to use install_from_src_ubuntu script * added java version check * fixed issue with if statement * fixed variable name * fixed repeated cd command * updated artifacts * removed artifacts from buildspec * Corrected HTTP response code for Model version does not exists exception (pytorch#282) * issue 196 patch * Added openapi missing responses. - incorporated review comments * updated method signature after resolving merge conflicts Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Corrected paths in command and added notes (pytorch#354) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Refactored UT for better reporting (pytorch#159) * refactored UTs * upgraded testng version and updated testcase names * added testng.xml to handle sequential execution of test classes * added waveglow example * Updated readme for waveglow example * fixed failing UTs after merging master * removed unintentionally pushed waveglow example * fixed failing test case * moved set-default to utils and added couple of more add cases for default version * java formatting * corrected set default negative test case * fixed java formatting Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Huggingface Transformers generalization (pytorch#302) * Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model * modified to include mapping of predicitons and decoding received text * move back the original text * moved the index_to_name to bert folder * Transformers/ Bert serving with Torchserve readme * deleting an unused dictionary * just cleaning some of the commented out parts * cleaning up * Cleaning up * uncommenting the logging and cleaning the comments * Download and save the checkpoints, config file and vocab of the BERT from Transformers for demonstration purposes * mapping of predicitons to labels * a custom handler for transformers. * README. * sample of input text. * modifying the descriptions * modifying the descriptions * adding comments * adding commnets * Fromatting modifications * modified comments * added the citations * adding hugginface as the type of transformer model we are referring to * renaming the folder to huggingface_transformers * Removing the bert folder * simple modification * modifed the script to use a setup_config.json for loading and saving models * added the section for transfomer_handler_generalized * the generalized handler * set_up config intialization * edited for transfomer generalization * modified to use just input_ids as inputs for models * modified for question answering * modified to include the number of labels * corrected typos * modifed variable names * edited for clarity * modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case * modifed values for QA * modifed the variable names * added comments * modified to add clarity * add comment in preprocessing of QA * added warning for the case mode is not selected properly and combined constructor and initlaize fucntion * clean up * clean up * combined constructor and initliaze func * combined constructor and initliaze func * Separated the constructor and initliaze func * Modified to remove the classification part and update with generalized version * resolved the coflict and separated the costructor and initliaze func * removed the handler for classification case as the generalized version handles the case Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: mycpuorg <mail@mycpu.org> * added waveglow example (pytorch#258) * added waveglow example * Updated readme for waveglow example * enhanced example to remove runtime checkpoint download * updated handler to return the audio file content instead of dumping it to tmp dir * minor change to handler * added pipefail * added python snippet to run inference * updated readme with know issue with the example Co-authored-by: Aaqib <maaquib@gmail.com> * made build suite compatible with latest gradle (pytorch#348) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * moved env section after context * Deleting redundant ISSUE_TEMPLATE at root dir * Fix typo in scripts/install_from_src_macos * Corrected OpenApi description (pytorch#328) * Correct query parameter name (Registration Api)in OpenApi description * removed non existing path parameter Co-authored-by: Aaqib <maaquib@gmail.com> * Regression suite fixes and added test cases (pytorch#344) * added test model zoo test cases updated regression script * Updated Readme * update -y flag for install * Updated docker command as per new imagename and tag generated in stagging branch * Fix sudo usage in regression test script * - addressed review comments - reverted changes related to sudo in regression script. * added https test collection * - updated failed test cases on staging branch - added environment and updated newman cmd with --insecure for https * updated script * updated pytest * update curl check for serve up * Revert "update curl check for serve up" This reverts commit 0047eae. * Revert "update curl check for serve up" This reverts commit 0047eae. - update model zoo - added resnet test case * added workers to resnet model * Update readme to run test suite on desired branch * enhanced inference test cases to driven by datafile * Updated readme Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Fix typo in mnist readme (pytorch#330) Co-authored-by: Aaqib <maaquib@gmail.com> * Fixed typo in ab installation. (pytorch#327) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes pytorch#297 * remove minus y (pytorch#311) update pip install instructions * Fixed typo in ab installation. Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Setup Nightly Build for TorchServe (pytorch#371) * Add Branch Flag for Nightly CodeBuild pytorch#57 * Update ENV Variable for Nightly Build Name * Nigtly Build - Fix FileName for Build Artifacts * Update README for accessing test logs * Whitspace changes * Build new images for build - README Instruction * Fix for intermittent gradle build failure (pytorch#352) * Fix for intermittent null pointer exception * Fixed comments Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Aaqib <maaquib@gmail.com> * documentation: Remove redundant curl options in example (pytorch#336) * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes pytorch#297 * remove minus y (pytorch#311) update pip install instructions * fix-doc * fix remaining "-X POST"s Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> * Corrected port number validation range (pytorch#304) * Corrected port number validation range * added UT for issue 291 * added ut dependency file formatting * correct param in config_test_env.properties * removed incorrect flags from test case config files * added negative test cases * removed properties file and changes in test case Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * Fixes version number reporting for TorchServe (pytorch#360) * Correct version and added vesion.txt for getting version info * fixed description test cases and build issue * Added snapshot test fixes and resolved relative path issue for version.txt * python formatting * added version.txt in MANIFEST.in * addressed review comments * added version.txt for model archiver as well - addressed review comments * -fixed python UT -added ts python UT run in sanity script * removed blank lines from snapshot files Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> * fixes installation script exits without error if no java present on machine (pytorch#376) * fixes installation script exits without error if no java present on machine * added changes for macos script Co-authored-by: Aaqib <maaquib@gmail.com> * Dockerfile enhancements (pytorch#308) * dockerfile now points to latest torchserve on pypi and added separate files for dev * Minor changes * Made build and run commands consistent * Fixing formatting * Consistency issue * Enhanced Dockerfile for binary per pytorch#178 and other misc. * Fixed gpu related issues in start script * Corrected the heading font * Added mar creation steps from container * added buildkit support for dev docker files * Fix for buildkit * Restoring psutil and future as absence leads intermittent hangged build * Fixed review comments for doc * Fixed typo Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Aaqib <maaquib@gmail.com> * Updated configuration.md for missing config properties (pytorch#337) * Updated configuration.md for missing parameters * changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT - removes ambiguity - follows naming convention * addressed review comments. * removed debug property * updated configuration.md Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Adding missing document links to README (pytorch#379) * Maven tests should output only errors (pytorch#326) Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Aaqib <maaquib@gmail.com> * fixed code for build failure in python virtual environment (pytorch#341) * fixed code for build failure in python virtual environment * added exception missed on resolving conflicts Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> * updated object detector handler to use only default cuda device for torchvision version less then 0.6 (pytorch#265) * updated object detector handler to use default cuda device for torchvision version 0.5.0 * updated version check logic * Updated documentation for torchvision version dependency * enhanced sanity script to validate all default handlers * fixed sanity script * added torchtext install step in sanity script * fixed issue related to inference after restart with snapshot. * moved torchtext to codebuild docker files * added more log statements * enhanced run inference function * fixed register model function * added unregister model function * incorporated code review comments * added back inference call for snapshot test * fixed if block Co-authored-by: Aaqib <maaquib@gmail.com> * Adding a barebones ec2 template to run torchserve on ubuntu (pytorch#301) * Adding a barebones ec2 template to run torchserve on ubuntu * Adding VPC with Private subnet * Address review comments #1 * Address review comments pytorch#2 * Rebase for pytorch#365 (pytorch#389) * Updated README.md for pytorch#364 See pytorch#364 for a deep description of the problem * updated README.md * updated README.md Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> * Document the docker settings for running in production mode (pytorch#388) * Docker Production Options pytorch#234 * Docker Production Options - Typo Fixes / Fit & Finish pytorch#234 * Typos - Docker Docs pytorch#234 * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> * grammar check - docker/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Huggingface transformers integration test (pytorch#382) * Add Huggingface transformers example integration tests * Add transformers intallation * Fix a missing bracket * move transformers installation to chain installations Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Updated docker README.md (pytorch#401) * Update README.md * Fixed cd command * Made source cloning steps explicit!! * Fixed online editing mistake Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> * Fixed response status message for inferencing on model with zero workers (pytorch#305) * Changes for issue_29 * Changed error Messages * changed message for scalling request * changed defaultVersion string * Corrected error message * changed the code for best practice * resolved review comment for version * updated management api doc as per new responses in worker scale api Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * made model server port tests independent (pytorch#396) * Added backend profiling and fixed benchmark's install dependency scripts (pytorch#402) * added backend profiling * updated install dependency scripts to make them jmeter version independent * updated readme * Fix benchmarks/README typo and JMeter versioning issue * updated documentation Co-authored-by: Aaqib Ansari <maaquib@gmail.com> * Add sphinx directory and conf.py (pytorch#274) * Add sphinx folder that contains files for docs build * Delete .DS_Store * Delete .DS_Store * Add conf.py in docs/ * documentation: fixed issues in installation and quick start in README.md * documentation: Reverting conda related doc changes from PR#286 Fixes pytorch#297 * remove minus y (pytorch#311) update pip install instructions * Update to the latest version of sphinx * Add recommonmark to extensions Add recommonmark to extensions so sphinx project has markdown support. Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html * Update source suffix to use latest recommonmark syntax Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html Co-authored-by: Aaron Markham <markhama@amazon.com> * Remove old recommonmark parser syntax * Pin sphinx version to work with latest version of recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Pin version latest version recommonmark Co-authored-by: Aaron Markham <markhama@amazon.com> * Delete theme file generated during build * de-duplicate install package of psutil (pytorch#329) The Dockerfile of GPU install package psutil duplicated. * Remove pytorch-sphinx-theme files * Update to match latest staging_0_1_1 * Update docker directory to match staging_0_1_1 branch * Delete .DS_Store * Add sphinxcontrib.katex and torch Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaqib Ansari <maaquib@gmail.com> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> * Image classifier example readme fixes (pytorch#405) * Changed the model name in torch.jit.save to be consistent with docs * Remove extra handler flag and fix the torchservr start * fix the doc * fix the doc Co-authored-by: mycpuorg <mail@mycpu.org> * Porting memory fix changes to TorchServe (pytorch#422) Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Benchmark documentation fix (pytorch#425) * Fix Benchmark Documentation * Documentaion fixes * .circleci: Add initial circleci configuration (pytorch#431) Signed-off-by: Eli Uriegas <eliuriegas@fb.com> * Torchscript support huggingface transfrmers (pytorch#421) * Add support for torchscript * Add support for torchscript * Add key values for torchscript support * Add Torchscript suport * Add Torchscript support * Add Torchscript support in loading the models * keep the example to QA * modify to reflect Torchscript example? * Add torchscript support guides * Add fintued BERT for QA * Add QA artifacts * Add seqenece classification artifacts * Token classification artifacts * Correct typos * Add max length to tokenizer * clean up * enhance the doc * change setup to seq classification * Add detail to torchscript support * set the model for QA * Add torchscript bert seq classification example * Add bert seq classification example * Modify the packaging model command * Modify torchscript packaging command * clean up * clean up * correct spacing between list items Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Integration test huggingface transfromers torchscript support (pytorch#433) * made model server port tests independent * Add integration tests for Huggingface transfromers with new handler that support torchscript Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com> * Apachebench benchmark - cleanup script / documentation * Revert "Apachebench benchmark - cleanup script / documentation" This reverts commit c8f8682. * Documentation updates for clarity on Eager mode / Torch scripted models (pytorch#439) * Model Zoo updates for Eager Mode / Torchscripted models * Update examples/README.md Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: Aaqib <maaquib@gmail.com> * Apachebench benchmarking - Script / Documentation updates (pytorch#438) * Apachebench benchmarking script - Script / Documentation updates * Stop TS after benchmark completion. Co-authored-by: harshbafna <harsh_bafna@persistent.co.in> Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com> Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com> Co-authored-by: Aaqib <maaquib@gmail.com> Co-authored-by: mycpuorg <mail@mycpu.org> Co-authored-by: Harsh Bafna <harshbafna619@gmail.com> Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal> Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com> Co-authored-by: Vamshidhar Dantu <dantu@amazon.com> Co-authored-by: Brad Heintz <bradheintz@fb.com> Co-authored-by: eslesar-aws <eslesar@amazon.com> Co-authored-by: Aaron Markham <markhama@amazon.com> Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal> Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal> Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com> Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com> Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal> Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com> Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com> Co-authored-by: alvations <alvations@gmail.com> Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal> Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch> Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal> Co-authored-by: Jessica Lin <jplin@fb.com> Co-authored-by: Mario Cho <hephaex@gmail.com> Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Update error message when attempting inference on model with 0 workers
Fixes #29
Type of change
Feature/Issue validation/testing
-[x] Test 1: Register model with 0 initial_workers Asynchronously
curl -X POST "http://localhost:8081/models?url=https://torchserve.s3.amazonaws.com/mar_files/resnet-18.mar&synchronous=false"
{
"status": "Model "resnet-18" Version: 1.0 registered with 0 initial workers. Use scale API to add workers for the model."
}
-[x] Test 2: Register model with 0 initial_workers Synchronously
curl -X POST "http://localhost:8081/models?url=https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar&synchronous=true"
{
"status": "Model "densenet161" Version: 1.0 registered with 0 initial workers. Use scale API to add workers for the model."
}
-[x] Test 3: Register model with 3 initial_workers Synchronously
curl -X POST "http://localhost:8081/models?url=https://torchserve.s3.amazonaws.com/mar_files/squeezenet1_1.mar&initial_workers=3&synchronous=true"
{
"status": "Model "squeezenet1_1" Version: 1.0 registered with 3 initial workers"
}
-[x] Test 4: Register model with 3 initial_workers Asynchronously
curl -X POST "http://localhost:8081/models?url=https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar&initial_workers=3&synchronous=false"
{
"status": "Processing worker updates..."
}
-[x] Test 5: Inference on model which has 0 workers
curl -X POST http://127.0.0.1:8080/predictions/resnet-18 -T kitten.jpg
{
"code": 503,
"type": "ServiceUnavailableException",
"message": "Model "resnet-18" has no worker to serve inference request. Please use scale workers API to add workers."
}
-[x] Test 6: Inference on model which has workers
curl -X POST http://127.0.0.1:8080/predictions/densenet161 -T kitten.jpg
torchserve_sanity.log
-[x] Test 7: Scale Mode with non zero workers
curl -X PUT "http://localhost:8081/models/resnet-18?min_worker=5&max_worker=10&synchronous=true"
{
"status": "5 Workers scaled for model resnet-18"
}
Checklist: