Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Download WordPress bleeding edge from GitHub #18773

Merged
merged 3 commits into from
Nov 27, 2019

Conversation

aduth
Copy link
Member

@aduth aduth commented Nov 26, 2019

Previously: #17004
Relevant Slack discussion: https://wordpress.slack.com/archives/C02QB2JS7/p1574809705101600 (link requires registration)

This pull request seeks to update the Travis build preparation to install WordPress latest rather than the latest nightly build. The intention here is to resolve intermittent build failures caused by a corrupted ZIP file:

    # Download and unpack WordPress.
    curl -sL https://wordpress.org/nightly-builds/wordpress-latest.zip -o /tmp/wordpress-latest.zip
...
curl: (56) SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
[/tmp/wordpress-latest.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

Upon further investigation, I was led to the conclusion that the downloads become interrupted when a new nightly version is uploaded, evidenced by the recentness of the last-modified header at the time of debugging:

⇒ curl -I https://wordpress.org/nightly-builds/wordpress-latest.zip
...
date: Tue, 26 Nov 2019 23:11:06 GMT
last-modified: Tue, 26 Nov 2019 23:00:06 GMT

Since nightly builds are updated with regular frequency, this isn't too surprising. But the build tooling is not tolerant of these disrupted downloads, and will subsequently error.

Alternatives considered include:

  • Because each nightly has a tagged identifier (e.g. 5.4-alpha-46786), if we could determine the tagged version at the time of build start and download it directly (rather than latest.zip), we could avoid a download interruption.
    • Conclusion: From what I can tell, there is no archives of alpha ZIPs; only latest.zip is available for download.
  • We could test the integrity of the zip file after the cURL command exits, to check whether it is complete and, if not complete, assume that a new version was uploaded and try again.
    • Conclusion: It's unclear to me whether we can immediately start to download the newest latest.zip after a failed request, or if there is a delay in the availability of the new version matching the time it takes for that new version to be uploaded.

Ultimately, while there might be some value in running the nightly version in the Travis build, it does not seem to be of critical importance; or at least, it is more important that the builds be stable. Furthermore, prior to #17004 we were running "latest" (not nightly), so this matches a known previous behavior.

For these reasons, the changes propose simply to replace the nightly download with a download of the latest stable version. I acknowledge that with these changes, we should consider to reopen the issue at #12839.

Testing Instructions:

Travis build should pass.

@aduth aduth added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Nov 26, 2019
Copy link
Member

@ntwb ntwb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM 👍🏼

@aduth
Copy link
Member Author

aduth commented Nov 27, 2019

There was some further discussion in Slack after submitting the pull request.

@dd32 points out that the nightly download shouldn't be disrupted as I suspected it was, even if a new version is uploaded while a download is in-progress.

@peterwilsoncc offered the suggestion to download the ZIP via GitHub (https://github.com/WordPress/WordPress/archive/master.zip). As best I can tell, this should serve just as well as the nightly (perhaps even more up-to-date than the nightly?). At the very least, it could be worth testing to see if it has any improvement.

@aduth
Copy link
Member Author

aduth commented Nov 27, 2019

Updated to download the latest master version from GitHub. Seems like this will be the least disruptive option, even an improvement so far as running a more recent version than that offered by the nightlies.

@aduth aduth merged commit d384e16 into master Nov 27, 2019
@aduth aduth deleted the fix/build-failure-nightly-download branch November 27, 2019 05:56
@aduth aduth changed the title Testing: Use WordPress latest for Travis build (rather than nightly) Testing: Download WordPress bleeding edge from GitHub Dec 6, 2019
@aduth
Copy link
Member Author

aduth commented Dec 6, 2019

Sadly, the errors still seem to occur, even downloading from GitHub:

$ if [[ "$INSTALL_WORDPRESS" = "true" ]]; then
    # Download and unpack WordPress.
    curl -sL https://github.com/WordPress/WordPress/archive/master.zip -o /tmp/wordpress-latest.zip
    unzip -q /tmp/wordpress-latest.zip -d /tmp
...
curl: (56) SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
[/tmp/wordpress-latest.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /tmp/wordpress-latest.zip or
        /tmp/wordpress-latest.zip.zip, and cannot find /tmp/wordpress-latest.zip.ZIP, period.
mv: cannot stat ‘/tmp/WordPress-master/*’: No such file or directory

Anecdotally, it doesn't seem to occur for me quite as frequently as before, but it remains a problem nonetheless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants