Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed Text2Speech synthesis as NVIDIA models' URL required an update #762

Merged
merged 8 commits into from
Nov 25, 2020
Merged

Fixed Text2Speech synthesis as NVIDIA models' URL required an update #762

merged 8 commits into from
Nov 25, 2020

Conversation

alvarobartt
Copy link
Contributor

Description

As the NVIDIA Waveglow and Tacotron2 PyTorch pre-trained models have been updated, their URLs too and the previous versions of those models seem to be unavailable, so that the URLs required an update.

This fix should probably be applied to all NVIDIA model URLs in case they do not work. As the previous URLs are currently unavailable, the error code while trying to retrieve those either using wget or curl is HTTP 404.

More information regarding all the NVIDIA Catalog Models can be found at: https://ngc.nvidia.com/catalog/models?orderBy=modifiedDESC&pageNumber=1&query=&quickFilter=models&filters=

Fixes #761

References shared by @harshbafna that should probably be also fixed:

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 3627eb3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 3627eb3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

alvarobartt
alvarobartt commented Oct 29, 2020
View reviewed changes
Copy link
Contributor Author

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVIDIA PyTorch model download tested and MAR file generation using torch-model-archiver too. It works! 🤗

harshbafna
harshbafna suggested changes Oct 29, 2020
View reviewed changes
Copy link
Contributor

@harshbafna harshbafna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alvarobartt: The tacotron2 and waveglow model weight file names are being changed here which are used in waveglow handler, due to which the model registration will fail.

Please update the handler as well.

@alvarobartt
Copy link
Contributor Author

You're right @harshbafna I forgot to commit those changes... Now everything should be right! 👍🏻

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: df9bdf3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: df9bdf3
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

alvarobartt
alvarobartt commented Oct 29, 2020
View reviewed changes
Copy link
Contributor Author

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated handler refs to the updated version of Waveglow and Tacotron2 NVIDIA models as requested by @harshbafna

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 99b8cb2
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 99b8cb2
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@chauhang
Copy link
Contributor

@alvarobartt Can you please check if picking the waveglow/tacotron2 models from PyTorch Hub might be a better option, so that we don't have to deal with breaking path changes in future?

https://pytorch.org/hub/nvidia_deeplearningexamples_waveglow/
https://pytorch.org/hub/nvidia_deeplearningexamples_tacotron2/

@alvarobartt
Copy link
Contributor Author

Hi @chauhang! Yes, indeed that's a better option/way to proceed, the thing is that the NVIDIA DeepLearning examples repository on the branch torchhub is also facing the same issue, which means that the torch.hub.load() function of those models won't work... Anyway, I submitted a PR to NVIDIA/DeepLearningExamples that you can find at NVIDIA/DeepLearningExamples#727 👍🏻

@alvarobartt
Copy link
Contributor Author

alvarobartt commented Oct 30, 2020 •
edited
Loading

Quick update! It seems that people from NVIDIA already solved this using some temp fallback to backup model checkpoints for all the models mentioned above (Tacotron2, Waveglow) and also for SSD. As you can see at NVIDIA/DeepLearningExamples@3e961c3 So on, this issue regarding torch.hub.load seems to be temporarily fixed (I guess they discarded my PR as they included those new temporary URLs) but not quite sure about that.

Anyway, tomorrow I will likely update the URLs for the Tacotron2 and Waveglow model checkpoints downloaded in the create_mar.sh file as well as the names of those models in the waveglow_handler.py. So maybe currently this should be merged using those URLs, but maybe an issue should be kept open so as to get back to this once NVIDIA properly updates its model URLs so that the torch.hub.load() function works as expected, even though it currently does with this temporary fix they made.

P.S.: Indeed they ended up updating the URLs just like I did on the PR I submitted (NVIDIA/DeepLearningExamples#727), but instead of merging it they updated it theirselved (don't ask me why) in this commit NVIDIA/DeepLearningExamples@b759609, so the URLs I included in the create_mar.sh script are valid and updated! 👍🏻

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 0bbe4bf
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 0bbe4bf
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

alvarobartt
alvarobartt commented Nov 3, 2020
View reviewed changes
Copy link
Contributor Author

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created requirements.txt for Waveglow as requested by @chauhang and @harshbafna in #761, so as to avoid its manual installation on the server as TorchServe already handles this via that file on the MAR.

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: eb85b62
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: faf9dd5
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: eb85b62
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: faf9dd5
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@alvarobartt alvarobartt mentioned this pull request Nov 4, 2020
Closed
@harshbafna harshbafna mentioned this pull request Nov 23, 2020
Closed
@harshbafna harshbafna linked an issue Nov 23, 2020 that may be closed by this pull request
Text to speeh synthesizer create_mar.sh running failed #794
Closed
@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 668ec43
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 668ec43
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 4144e54
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 4144e54
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@chauhang chauhang added this to the v0.3.0 milestone Nov 25, 2020
@chauhang chauhang merged commit 38e8e02 into pytorch:master Nov 25, 2020
@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-cpu
  • Commit ID: 4144e54
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-neo-ci-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: torch-serve-build-gpu
  • Commit ID: 4144e54
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@kbumsik kbumsik mentioned this pull request Oct 13, 2022
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@harshbafna harshbafna harshbafna approved these changes

@chauhang chauhang chauhang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v0.3.0
Development

Successfully merging this pull request may close these issues.

Text to speeh synthesizer create_mar.sh running failed Cannot create model archive in waveglow example
4 participants
@alvarobartt @sagemaker-neo-ci-bot @chauhang @harshbafna