V0.5.0 forward merge #339
Closed
V0.5.0 forward merge #339
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
…NVIDIA#301) This PR ensures that users can run the PEFT SDG tutorial using arbitrary API endpoints by exposing the URL that is used for synthetic data generation. Signed-off-by: Mehran Maghoumi <Maghoumi@users.noreply.github.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
* add column param Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * read_pickle and black Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * optional param Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * add parquet comment Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
* keep_filename_column param Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * update pytests Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> * run black Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> --------- Signed-off-by: Sarah Yurick <sarahyurick@gmail.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
* Add partial image implementation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor requirements Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix bugs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change from_map to map_partitions Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add super constructor Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add kwargs for load_object_on_worker Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Get proper epoch size Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Complete embedding creation loop Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change devices Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add device Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor embedding creation and add classifier Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix bugs in classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor model names Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add model name Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix classifier bugs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Allow postprocessing for classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix name and add print Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix variable name Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add NSFW Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update init for import Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix embedding size Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add fused classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix missing index Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update metdata for fused classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add export to webdataset Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix missing id col Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Sort embeddings by id Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add timm Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update init file Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add autocast to timm Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update requirements and transform Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add additional interpolation support Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix transform normalization Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove open_clip Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add index path support to wds Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Address Vibhu's feedback Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add import guard for image dataset Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change default device Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove commented code Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove device id Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix index issue Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add docstrings and standardize variable names Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add image curation tutorial Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add initial image docs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove tutorial Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add dataset docs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add embedder documentation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Revert embedding column name change Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update user guide for images Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update README with RAPIDS nightly instructions Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix formatting issues in image documentation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove extra newline in README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Address most of Sarah's feedback Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add section summary Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix errors and REWORD GPU bullets in README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix how table of contents displays with new sections Signed-off-by: Ryan Wolf <rywolf@nvidia.com> --------- Signed-off-by: Ryan Wolf <rywolf@nvidia.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: Vibhu Jawa <vibhujawa@gmail.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: Ryan Wolf <rywolf@nvidia.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
* Speedup fuzzy dedup by avoiding merge Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Remove unused function Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Clean up PR based on Praateeks reviews Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * style fixes Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * style fixes Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Remove dangling print Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Add handling for multiple columns Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Nuking convert to strings Signed-off-by: Vibhu Jawa <vibhujawa@gmail.com> * Nuking convert to strings Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Verify it works on exp-01 Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> * Add dask profile options and add overwrite Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> --------- Signed-off-by: Vibhu Jawa <vjawa@nvidia.com> Signed-off-by: Vibhu Jawa <vibhujawa@gmail.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: Ryan Wolf <rywolf@nvidia.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: Ryan Wolf <rywolf@nvidia.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
* Change download for NSFW model Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix model init Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix embedding size Signed-off-by: Ryan Wolf <rywolf@nvidia.com> --------- Signed-off-by: Ryan Wolf <rywolf@nvidia.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
ayushdg
force-pushed
the
v0.5.0_forward_merge
branch
from
c219a8b
to
1ad8b01
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Merge in changes from the 0.5.0 release into main.
Specifically excludes the changes around our requirements.txt being pinned to specific versions in the previous release.
Also bumps our current version to
0.6.0.dev0for folks installing from source.Usage
N/A
Checklist