[ViT] Support fine-tuning with different image resolution #5025
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💊 CI failures summary and remediationsAs of commit 73093f0 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
datumbox
reviewed
Dec 8, 2021
datumbox
approved these changes
Dec 9, 2021
There was a problem hiding this comment.
@sallysyw I pushed to your branch a change to fix the typing issues. The problem here is that the OrderedDict is not subscriptable. Using quotes will do the trick.
I'm approving to unblock your work, but it's important to follow up with another PR that adds some tests to cover the method.
|
Hey @sallysyw! You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py |
facebook-github-bot
pushed a commit
that referenced
this pull request
Dec 21, 2021
…5025) Summary: * add from_checkpoint method for vit * remove useless change * Making interpolate_embeddings a utility function * remove logging * fix type hint * fix return type check * ad retuurns in docsting & unify type hint * remove useless import * fix issue: 'type' object is not subscriptable * Fixing typing issues * Making interpolation mode configurable * formatting Reviewed By: prabhat00155 Differential Revision: D33253466 fbshipit-source-id: 79bf6855f2dcee3c2fef6c05c243a0dc8dfee25e Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As discussed in #4594, we should be able to interpolate embeddings from one resolution to a different one when training ViT models.
This PR adds the support for it.
References: ClassyVision Implementation
Experiments:
cc @datumbox