Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix size_t issues across JPEG versions and platforms #4439

Merged
merged 4 commits into from
Sep 17, 2021

Conversation

datumbox
Copy link
Contributor

@datumbox datumbox commented Sep 17, 2021

This PR fixes (again) the size_t issues we've been facing due to JPEG. It seems that overnight, the version on windows changed from 9b-hb83a4c4_2 to 9d-h2bbff1b_0. It seems that the Windows version has fixed the issue from 9d>= and aligned itself with the ones of the rest of the platforms.

To ensure that our patch works with all versions, I pinned and tested jpeg on the following versions:

  • jpeg <=9b
  • jpeg >=9d (same as unpinned)

We skip tests for jpeg ==9c because we get Unsatisfiable dependencies errors in conda.

Copy link
Member

@fmassa fmassa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving conditioned on green CI.

I have a question about removing Windows that might be good to double-check

// For windows backward compatibility, we define JpegSizeType as different types
// according to the libjpeg version used, in order to prevent compilation
// errors.
#if defined(_WIN32) || !defined(JPEG_LIB_VERSION_MAJOR) || \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No more Windows checks? Ok with me, but IIUC there used to be discrepancies between Windows / Linux libjpeg that we might need to take into account?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The situation seems to have been resolved in later versions of Windows (9d>=) hence blunt windows filter needs to go. I'll pin different versions of JPEG in a bit to see what exactly is the right condition we should use here.

@datumbox
Copy link
Contributor Author

datumbox commented Sep 17, 2021

All CI tests green when unpinned, except of cmake_windows_gpu which is unrelated, see #4440.

@datumbox datumbox changed the title Use size_t for JPEG on Windows Fix size_t issues across JPEG versions and platforms Sep 17, 2021
@datumbox datumbox merged commit 1140ecf into pytorch:main Sep 17, 2021
@datumbox datumbox deleted the jpeg_type_windows branch September 17, 2021 14:28
datumbox added a commit to datumbox/vision that referenced this pull request Sep 17, 2021
facebook-github-bot pushed a commit that referenced this pull request Sep 30, 2021
Summary:
* Windows use size_t

* Temporarily pin jpeg <=9b

* Temporarily pin jpeg ==9c

* Remove pinning.

Reviewed By: datumbox

Differential Revision: D31268029

fbshipit-source-id: 6d6514abe195e71f05996fee7e4e1be7fad4a137
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants