Add build workflow #30644
Add build workflow #30644
Conversation
This comment was marked as off-topic.
This comment was marked as off-topic.
Lesmiscore
force-pushed
the
build-yml
branch
from
0b4d932
to
9d679fa
Compare
|
Has this PR been completely forgotten about? |
|
maybe ping @dirkf |
|
No. Great work by everyone, and thanks, and apologies for taking so long to follow this up. To do:
|
You'll need to use pyinstaller for this and not py2exe |
|
This is kinda PoC anyways... |
IMO that's unnecessary, because afaik all those devices will be compatible with yt-dlp (at least the macbooks, I'm not sure about the other unixes).
and speaking of the signing key, I wonder if @dirkf would be willing to share the signing key with @pukkandan, so that yt-dlp can also be signed (and hence won't be flagged by any antiviruses, although that's not a major issue nowadays) |
The signing part implies .sig files like there For a signing keys for EXE files (which you meant) must NEVER be done as you're asking to share private keys (possibly) containing dirkf's personal info! edit: youtube-dl.exe from version 2021.12.17 isn't signed, for a PE file |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
| - name: Get Changelog | ||
| id: get_changelog | ||
| run: | | ||
| changelog=$(cat ChangeLog | grep -oPz '(?s)(?<=version ${{ steps.bump_version.outputs.ytdl_version }}\n{2}).+?(?=\n{2,3}version)') || true |
There was a problem hiding this comment.
you should change this
| changelog=$(cat ChangeLog | grep -oPz '(?s)(?<=version ${{ steps.bump_version.outputs.ytdl_version }}\n{2}).+?(?=\n{2,3}version)') || true | |
| changelog=$(grep -oPz '(?s)(?<=version ${{ steps.bump_version.outputs.ytdl_version }}\n{2}).+?(?=\n{2,3}version)' ChangeLog) || true |
There was a problem hiding this comment.
The advantage of cat file | whatever is that it puts the file at the start of the command, so decoupling it from the pipe being applied to it. < file whatever should have the same benefit without any feline intervention, but isn't widely used.
|
... Sorry for barging in 😉, I not being a python coder 😊, but,
so my hope/wish/suggestion 😉 is any future binary release (based on this PR) by the new management also contain "native" PS: The abandoned Kind regards 😃 |
As far as I remember, py2exe doesn't work well with it. Hence why |
|
While ffmpeg isn't a hard requirement, so much yt-dl[p] functionality depends on it that I can't imagine not having it, except in some niche application of yt-dl. Also, it solves the problem for the old Pythons that yt-dl is trying to support but which are not supported by |
|
FWIW, there have been cases in the past where I wanted to fetch geo-fenced media content served over HLSe (AES-128 encrypted), and the only geo-location circumvention means available to me was a SOCKS5 proxy (or a SSH tunnel).
--proxy "socks5://proxyhost:proxyport"so it passes this down to [hlsnative] which, when PS: It wasn't obvious to me back then, but, in the absence of
Nope... The standalone Windows binaries built either by i.e. no type of "installation" is required at all... |
Do you think that's wise? https://nvd.nist.gov/vuln/detail/CVE-2013-7459
|
|
The official response is to "Switch to pycryptodome", as a quick fix.
But it seems like a good quick fix to release something. A better fix would be to move to the I would go with the better fix. But that's just me. |
|
yt-dl can't use cryptography because of missing legacy platform support. It can use the common API of Pycrypto and Pycryptodome, relying on the user having installed the latter if suitable, or the former, or bundling the former otherwise. Though I'm not sure that Pycrypto will help ARM64 since it expects Py<=3.3. If there is a known security issue in Pycrypto it should be possible to fix it, though someone (maybe a Bellingcat-type open source investigator) who is really concerned about such issues should be running yt-dl in a sandbox. |
Debian still maintains pycrypto and have fixed the CVE-2013-7459 vulnerability (#849495).
Someone should be able to build from source, I guess? |
|
Imho, this is being unnecessarily delayed and side tracked. Getting any release out, even one without deps or even unsigned will cover most people's needs and should be the priority imo. Automation, signing, pycryptodome, standalone unix binaries are all useful to have, but is not essential to a release and can be added later incrementally. |
Well, indeed, both Be that as it may, in the linked article it's stated that CVE has been patched in the pycrypto Additionally, the pycrypto GH repo has seen in 2022 some minor activity, after a long period of inertia, but has now been made a
The "official" Win-binaries produced by the former team of devs seem to contain only a subset of the full which is understandable, because "we" only need
For those interested and willing to trust third-party compiled wheels, GitHub holds: py3.5 wheels of v2.6.1: py3.6 wheels of v2.6.1/v2.7a1: py3.7 wheels of v2.6.1: py3.8 wheels of v2.6.1: ... and py3.4 wheels for v2.6.1 were available on For the reasons I described in my previous post, I'd prefer @nicolaasjan: You routinely compile your own, py3.4-based, Windows builds, with |
This comment was marked as off-topic.
This comment was marked as off-topic.
Why not? It's certainly there, in wheel format for py3.4 x86: especially since I specifically requested its creation and the module's author indulged 👍 ... Well, this comment 😉 from there describes an identical situation... BTW, and I think it might be an interesting read for the current maintainer, there's a related discussion between the two main former devs (dstftw and remitamine) on the topic of |
This comment was marked as off-topic.
This comment was marked as off-topic.
| env: | ||
| PUSH_VERSION_COMMIT: ${{ secrets.PUSH_VERSION_COMMIT }} | ||
| if: "env.PUSH_VERSION_COMMIT == ''" | ||
| run: echo ::set-output name=version_suffix::$(date -u +"%H%M%S") |
There was a problem hiding this comment.
| run: echo ::set-output name=version_suffix::$(date -u +"%H%M%S") | |
| run: echo "version_suffix=$(date -u +"%H%M%S")" >> $GITHUB_OUTPUT |
set-output is depreciated
| make issuetemplates | ||
| - name: Push to release | ||
| id: push_release | ||
| run: echo ::set-output name=head_sha::$(git rev-parse HEAD) |
There was a problem hiding this comment.
| run: echo ::set-output name=head_sha::$(git rev-parse HEAD) | |
| run: echo "head_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT |
set-output is depreciated
| - name: Get SHA2-256SUMS for youtube-dl | ||
| id: sha256_bin | ||
| run: echo "::set-output name=sha256_bin::$(sha256sum youtube-dl | awk '{print $1}')" | ||
| - name: Get SHA2-256SUMS for youtube-dl.tar.gz | ||
| id: sha256_tar | ||
| run: echo "::set-output name=sha256_tar::$(sha256sum youtube-dl.tar.gz | awk '{print $1}')" | ||
| - name: Get SHA2-512SUMS for youtube-dl | ||
| id: sha512_bin | ||
| run: echo "::set-output name=sha512_bin::$(sha512sum youtube-dl | awk '{print $1}')" | ||
| - name: Get SHA2-512SUMS for youtube-dl.tar.gz | ||
| id: sha512_tar | ||
| run: echo "::set-output name=sha512_tar::$(sha512sum youtube-dl.tar.gz | awk '{print $1}')" |
There was a problem hiding this comment.
| - name: Get SHA2-256SUMS for youtube-dl | |
| id: sha256_bin | |
| run: echo "::set-output name=sha256_bin::$(sha256sum youtube-dl | awk '{print $1}')" | |
| - name: Get SHA2-256SUMS for youtube-dl.tar.gz | |
| id: sha256_tar | |
| run: echo "::set-output name=sha256_tar::$(sha256sum youtube-dl.tar.gz | awk '{print $1}')" | |
| - name: Get SHA2-512SUMS for youtube-dl | |
| id: sha512_bin | |
| run: echo "::set-output name=sha512_bin::$(sha512sum youtube-dl | awk '{print $1}')" | |
| - name: Get SHA2-512SUMS for youtube-dl.tar.gz | |
| id: sha512_tar | |
| run: echo "::set-output name=sha512_tar::$(sha512sum youtube-dl.tar.gz | awk '{print $1}')" | |
| - name: Get SHA2-256SUMS for youtube-dl | |
| id: sha256_bin | |
| run: echo "sha256_bin=$(sha256sum youtube-dl | awk '{print $1}')" >> $GITHUB_OUTPUT | |
| - name: Get SHA2-256SUMS for youtube-dl.tar.gz | |
| id: sha256_tar | |
| run: echo "sha256_tar=$(sha256sum youtube-dl.tar.gz | awk '{print $1}')" >> $GITHUB_OUTPUT | |
| - name: Get SHA2-512SUMS for youtube-dl | |
| id: sha512_bin | |
| run: echo "sha512_bin=$(sha512sum youtube-dl | awk '{print $1}')" >> $GITHUB_OUTPUT | |
| - name: Get SHA2-512SUMS for youtube-dl.tar.gz | |
| id: sha512_tar | |
| run: echo "sha512_tar=$(sha512sum youtube-dl.tar.gz | awk '{print $1}')" >> $GITHUB_OUTPUT |
set-output is depreciated
| - name: Create Release | ||
| id: create_release | ||
| uses: actions/create-release@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| tag_name: ${{ steps.bump_version.outputs.ytdl_version }} | ||
| release_name: youtube-dl ${{ steps.bump_version.outputs.ytdl_version }} | ||
| commitish: ${{ steps.push_release.outputs.head_sha }} | ||
| body: ${{ env.changelog }} | ||
| draft: false | ||
| prerelease: false | ||
| - name: Upload youtube-dl Unix binary | ||
| id: upload-release-asset | ||
| uses: actions/upload-release-asset@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| upload_url: ${{ steps.create_release.outputs.upload_url }} | ||
| asset_path: ./youtube-dl | ||
| asset_name: youtube-dl | ||
| asset_content_type: application/octet-stream | ||
| - name: Upload Source tar | ||
| uses: actions/upload-release-asset@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| upload_url: ${{ steps.create_release.outputs.upload_url }} | ||
| asset_path: ./youtube-dl.tar.gz | ||
| asset_name: youtube-dl-${{ steps.bump_version.outputs.ytdl_version }}.tar.gz | ||
| asset_content_type: application/gzip |
There was a problem hiding this comment.
This action is archived.
| - name: Create Release | |
| id: create_release | |
| uses: actions/create-release@v1 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| with: | |
| tag_name: ${{ steps.bump_version.outputs.ytdl_version }} | |
| release_name: youtube-dl ${{ steps.bump_version.outputs.ytdl_version }} | |
| commitish: ${{ steps.push_release.outputs.head_sha }} | |
| body: ${{ env.changelog }} | |
| draft: false | |
| prerelease: false | |
| - name: Upload youtube-dl Unix binary | |
| id: upload-release-asset | |
| uses: actions/upload-release-asset@v1 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| with: | |
| upload_url: ${{ steps.create_release.outputs.upload_url }} | |
| asset_path: ./youtube-dl | |
| asset_name: youtube-dl | |
| asset_content_type: application/octet-stream | |
| - name: Upload Source tar | |
| uses: actions/upload-release-asset@v1 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| with: | |
| upload_url: ${{ steps.create_release.outputs.upload_url }} | |
| asset_path: ./youtube-dl.tar.gz | |
| asset_name: youtube-dl-${{ steps.bump_version.outputs.ytdl_version }}.tar.gz | |
| asset_content_type: application/gzip | |
| - name: Create Release | |
| id: create_release | |
| uses: ncipollo/release-action@v1 | |
| with: | |
| tag: ${{ steps.bump_version.outputs.ytdl_version }} | |
| name: youtube-dl ${{ steps.bump_version.outputs.ytdl_version }} | |
| commit: ${{ steps.push_release.outputs.head_sha }} | |
| body: ${{ env.changelog }} | |
| draft: false | |
| prerelease: false | |
| artifacts: | | |
| ./youtube-dl,./youtube-dl.tar.gz | |
For other inputs see the readme: https://github.com/ncipollo/release-action
| - name: Upload youtube-dl.exe Windows binary | ||
| id: upload-release-windows | ||
| uses: actions/upload-release-asset@v1 | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| with: | ||
| upload_url: ${{ needs.build_unix.outputs.upload_url }} | ||
| asset_path: ./youtube-dl.exe | ||
| asset_name: youtube-dl.exe | ||
| asset_content_type: application/vnd.microsoft.portable-executable |
There was a problem hiding this comment.
| - name: Upload youtube-dl.exe Windows binary | |
| id: upload-release-windows | |
| uses: actions/upload-release-asset@v1 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| with: | |
| upload_url: ${{ needs.build_unix.outputs.upload_url }} | |
| asset_path: ./youtube-dl.exe | |
| asset_name: youtube-dl.exe | |
| asset_content_type: application/vnd.microsoft.portable-executable | |
| - name: Update Release | |
| id: update-release-windows | |
| uses: ncipollo/release-action@v1 | |
| with: | |
| allowUpdates: true | |
| tag: ${{ needs.build_unix.outputs.ytdl_version }} | |
| name: youtube-dl ${{ needs.build_unix.outputs.ytdl_version }} | |
| commit: ${{ needs.build_unix.outputs.head_sha }} | |
| body: ${{ env.changelog }} | |
| draft: false | |
| prerelease: false | |
| artifacts: | | |
| ./youtube-dl.exe |
| - name: Get SHA2-256SUMS for youtube-dl.exe | ||
| id: sha256_win | ||
| run: echo "::set-output name=sha256_win::$((Get-FileHash youtube-dl.exe -Algorithm SHA256).Hash.ToLower())" | ||
| - name: Get SHA2-512SUMS for youtube-dl.exe | ||
| id: sha512_win | ||
| run: echo "::set-output name=sha512_win::$((Get-FileHash youtube-dl.exe -Algorithm SHA512).Hash.ToLower())" |
There was a problem hiding this comment.
| - name: Get SHA2-256SUMS for youtube-dl.exe | |
| id: sha256_win | |
| run: echo "::set-output name=sha256_win::$((Get-FileHash youtube-dl.exe -Algorithm SHA256).Hash.ToLower())" | |
| - name: Get SHA2-512SUMS for youtube-dl.exe | |
| id: sha512_win | |
| run: echo "::set-output name=sha512_win::$((Get-FileHash youtube-dl.exe -Algorithm SHA512).Hash.ToLower())" | |
| - name: Get SHA2-256SUMS for youtube-dl.exe | |
| id: sha256_win | |
| run: echo "sha256_win=$((Get-FileHash youtube-dl.exe -Algorithm SHA256).Hash.ToLower())" >> $GITHUB_OUTPUT | |
| - name: Get SHA2-512SUMS for youtube-dl.exe | |
| id: sha512_win | |
| run: echo "sha512_win=$((Get-FileHash youtube-dl.exe -Algorithm SHA512).Hash.ToLower())" >> $GITHUB_OUTPUT |
set-output is depreciated
| outputs: | ||
| version_suffix: ${{ steps.version_suffix.outputs.version_suffix }} | ||
| ytdl_version: ${{ steps.bump_version.outputs.ytdl_version }} | ||
| upload_url: ${{ steps.create_release.outputs.upload_url }} | ||
| sha256_bin: ${{ steps.sha256_bin.outputs.sha256_bin }} | ||
| sha512_bin: ${{ steps.sha512_bin.outputs.sha512_bin }} | ||
| sha256_tar: ${{ steps.sha256_tar.outputs.sha256_tar }} | ||
| sha512_tar: ${{ steps.sha512_tar.outputs.sha512_tar }} |
There was a problem hiding this comment.
| outputs: | |
| version_suffix: ${{ steps.version_suffix.outputs.version_suffix }} | |
| ytdl_version: ${{ steps.bump_version.outputs.ytdl_version }} | |
| upload_url: ${{ steps.create_release.outputs.upload_url }} | |
| sha256_bin: ${{ steps.sha256_bin.outputs.sha256_bin }} | |
| sha512_bin: ${{ steps.sha512_bin.outputs.sha512_bin }} | |
| sha256_tar: ${{ steps.sha256_tar.outputs.sha256_tar }} | |
| sha512_tar: ${{ steps.sha512_tar.outputs.sha512_tar }} | |
| outputs: | |
| version_suffix: ${{ steps.version_suffix.outputs.version_suffix }} | |
| ytdl_version: ${{ steps.bump_version.outputs.ytdl_version }} | |
| upload_url: ${{ steps.create_release.outputs.upload_url }} | |
| sha256_bin: ${{ steps.sha256_bin.outputs.sha256_bin }} | |
| sha512_bin: ${{ steps.sha512_bin.outputs.sha512_bin }} | |
| sha256_tar: ${{ steps.sha256_tar.outputs.sha256_tar }} | |
| sha512_tar: ${{ steps.sha512_tar.outputs.sha512_tar }} | |
| head_sha: ${{ steps.push_release.outputs.head_sha }} |
|
🤷 seems like dirkf is working privately so |
|
Apologies for contacting you here/this way, but 😉 ... https://github.com/ytdl-patched/youtube-dl/tags and produced via GHA; most sadly, the https://github.com/ytdl-patched/youtube-dl/actions/workflows/rebase-on-upstream.yml thus even the latest tag offered (2023.04.28.334) is built on stale source code from Mar 14th 2023 😭 ... As I'm sure you're aware by now, your kind service is currently the source for updated among which the latest "fix" for Google/YT's throttling issue (which, thankfully, doesn't appear to still be an "issue" with latest yt-player Huge thanks ❤️ in advance for an eventual resolution/fix... Best regards 😄 |
|
I suppose this arises from the following chain of unpleasantness:
But now seems to be running, many thanks. |
|
Thanks Lesmiscore for your immediate action on the reported issue 🥇 ... Latest "nightly" tag as of this writing, 2023.04.29.1919, is indeed built on 211cbfd 👍 ! However, to futureproof the service, it's MHO the relevant As I'm totally clueless myself about GHA, this is purely academic on my part, but I'm confident savvy people monitoring this PR would come up with applicable solutions (hopefully 😜 ) ... dirkf: The https://github.com/ytdl-patched/youtube-dl repo doesn't (by design) have its issue tracker enabled; as the administrator of this repo, what would be the recommended place/way one should report a future "breakage" of the unofficial "daily" |
|
For now, post in the yt-dl tracker. |
Today's "daily" youtube-dl build, v2023.05.26.810, is still built on d1c6c5c from May 11th, whereas Regards. |
|
Looks like the builds succeed again after manually triggered. |
This comment was marked as resolved.
This comment was marked as resolved.
Please follow the guide below
xinto all the boxes [ ] relevant to your pull request (like that [x])Before submitting a pull request make sure you have:
(Forgive me for not checking some of them)
In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:
yt-dlp build workflow and devscripts/update-version.py for latter.
What is the purpose of your pull request?
Description of your pull request and other information
The goal for this PR is to give this repository a build workflow, enough to create a release except signatures (aka .sig).
It also reproduces how the
youtube-dl.exewas built.The files are reduced version to create a daily releases, so I need some fixes.
Big thanks for yt-dlp build workflow which this PR is based on.
(forgive me for being lazy while ESA is in session)ESA ended.TODO
.114514)Known Notes:
PUSH_VERSION_COMMIT. Value can be "anything".Closes #31067.