feat: depth estimation dataset guide. #5379
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The documentation is not available anymore as the PR was closed or merged. |
stevhliu
approved these changes
Dec 20, 2022
docs/source/depth_estimation.mdx
Outdated
| >>> train_dataset.set_transform(transforms) | ||
| ``` | ||
|
|
||
| You can verify the transformation worked by indexing into the `pixel_values` and `labels` of an example: |
There was a problem hiding this comment.
I would combine the two code snippets below.
Suggested change
| You can verify the transformation worked by indexing into the `pixel_values` and `labels` of an example: | |
| You can verify the transformation worked by indexing into the `pixel_values` and `labels` of an example image: |
There was a problem hiding this comment.
The succession made it visually simpler to read through. Hence decided to keep it that way.
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
|
Thanks for the changes, looks good to me! |
|
@stevhliu I have pushed some quality improvements both in terms of code and content. Would you be able to re-review? |
sayakpaul
commented
Dec 23, 2022
Comment on lines
+127
to
+130
| * Random cropping | ||
| * Random brightness and contrast | ||
| * Random gamma correction | ||
| * Random hue saturation |
There was a problem hiding this comment.
I had to drop horizontal flipping. Here's why.
The behavior seems to be unstable with datasets when applied via set_transform(). Consider the following as an example.
You'll notice that subplots tagged as "wrong" have flipped images but their corresponding depth maps are not flipped. This is not a consistent case, though. Subplots tagged as "right", for example. You'll notice that the images and their depth maps have both been flipped there (compare them to the originals and you should be able to spot the difference).
I have verified if there's a problem with albumentations but that doesn't seem to be the case. Refer to this notebook: https://gist.github.com/sayakpaul/63ae5f7971f634894ab37e76e6b3879c. You'll notice that I took individual samples from the dataset and passed them through the aug chain and the results were fine.
Have you experienced this before? Is anything obviously wrong you can spot in the code?
Let me know if anything's unclear.
There was a problem hiding this comment.
I'm unable to reproduce this issue 🙁 . Take a look at this simplified notebook.
There was a problem hiding this comment.
Yes @nateraw. But it doesn't seem to be consistent, as I stated here:
You'll notice that subplots tagged as "wrong" have flipped images but their corresponding depth maps are not flipped. This is not a consistent case, though. Subplots tagged as "right", for example.
Sometimes, the augmented depth maps are correct, i.e., the flip transformation has been correctly applied, and sometimes, that doesn't seem to be the case. You'll notice in the collage here.
There was a problem hiding this comment.
Yea I noticed that, but I ran it over and over and never got that behavior. Do you have a minimal reproduction of the behavior?
There was a problem hiding this comment.
I don't. Strangely, it only comes up when I try to put it inside a collection of images.
I will try to reproduce the behavior minimally and let you know here.
There was a problem hiding this comment.
You have this behavior because you call train_dataset[idx]["pixel_values"] and train_dataset[idx]["labels"] separately when plotting. For each call, it randomly gets flipped or not. To fix this, just do
>>> for i, idx in enumerate(random_indices):
... ax = plt.subplot(3, 3, i + 1)
... example = train_dataset[idx]
... input_image = example["pixel_values"]
... depth_target = example["labels"]
... image_viz = merge_into_row(input_image, depth_target)
... plt.imshow(image_viz.astype("uint8"))
... plt.axis("off")There was a problem hiding this comment.
My god! Thanks for saving me!
stevhliu
approved these changes
Jan 3, 2023
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Show benchmarksPyArrow==6.0.0 Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
|
Show benchmarksPyArrow==6.0.0 Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
|
Show benchmarksPyArrow==6.0.0 Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
Show updated benchmarks!Benchmark: benchmark_array_xd.json
Benchmark: benchmark_getitem_100B.json
Benchmark: benchmark_indices_mapping.json
Benchmark: benchmark_iterating.json
Benchmark: benchmark_map_filter.json
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a guide for prepping datasets for depth estimation.
PR to add documentation images is up here: https://huggingface.co/datasets/huggingface/documentation-images/discussions/22