Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken URLs to lists.linuxfoundation.org #2018

Merged
merged 4 commits into from
Nov 11, 2024

Conversation

kanzure
Copy link
Contributor

@kanzure kanzure commented Nov 7, 2024

This pull request updates lists.linuxfoundation.org urls to gnusha.org/url.

Background

Unfortunately, lists.linuxfoundation.org is no longer hosting the bitcoin-dev and lightning-dev static email archives, nor any others for that matter. There are a lot of broken links around the web now, including more than 2,500 in Bitcoin Optech.

Redirect web service

gnusha.org/url is a web service that redirects old style archive urls to a new location where the content can be viewed.

For example, a bitcoin-dev mailing list archive url like:

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html

using https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html

is redirected to:

https://gnusha.org/pi/bitcoindev/ogoa6o$o4s$1@blaine.gmane.org/

while a lightning-dev url like:

https://lists.linuxfoundation.org/pipermail/lightning-dev/2015-November/000313.html

using https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/lightning-dev/2015-November/000313.html

is redirected to:

https://diyhpl.us/~bryan/irc/bitcoin/bitcoin-dev/linuxfoundation-pipermail/lightning-dev/2015-November/000313.txt

The redirect mappings were generated from my archives and you can review the scripts I made to do that here:
https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b

Alternatives

Instead of using gnusha.org/url, another option would be to resolve all of the archive urls to their email Message-ID redirects for the bitcoin-dev mailing list and use those urls instead. In fact, either gnusha.org/pi/bitcoindev or https://mailing-list.bitcoindevs.xyz/bitcoindev/ could be used for this purpose.

Those links would look like:

https://gnusha.org/pi/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_ZUiBg1HGFyA@mail.gmail.com/
https://mailing-list.bitcoindevs.xyz/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_ZUiBg1HGFyA@mail.gmail.com/

If this is the preferred solution for Bitcoin Optech, then let me know how I can help. Take a look here.

lightning-dev status and issues

I do not have a complete lightning-dev.mbox archive at this time. The gnusha.org/url redirect service uses Linux Foundation pipermail .txt.gz files that I archived back in February 2024. These .txt.gz files do not contain email attachments. Someone would have to get a lightning-dev.mbox archive, convert it into public-inbox, and host a public-inbox for lightning-dev somewhere online, and then attachments could be accessed that way. If someone were to do that, I would be happy to update the gnusha.org/url redirect service to point to public-inbox emails by email Message-ID instead of the .txt files being used now.

FYI

Additional notes for this pull request:

  • img/posts/taproot-workshop/taproot-workshop.pdf is not updated in this pull request.
  • a few links to attachments were modified since gnusha.org/url does not support attachment links at the moment.

More background

For more information see:

Unfortunately, lists.linuxfoundation.org is no longer hosting the
bitcoin-dev and lightning-dev static email archives, nor any others for
that matter.

gnusha.org/url is a web service that redirects old style archive urls to
a new location where the content can be viewed.

"img/posts/taproot-workshop/taproot-workshop.pdf" is not updated.

For more information see:
bitcoin/bitcoin#29782 (comment)
bitcoin/bitcoin#31240
bitcointranscripts/bitcointranscripts#566
https://x.com/kanzure/status/1853779672514826334
https://gnusha.org/url/
A few lists.linuxfoundation.org urls were linking over to an attachment
file, which is not supported by the gnusha.org/url service at this time.

Instead, these links have been replaced by a link to the attachment in
the public-inbox archive on gnusha.org/pi/bitcoindev.

See 8bedb57 for more information.
Copy link
Contributor

@kouloumos kouloumos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@bitschmidty bitschmidty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • manually spot checked a few replacements
  • grep'd for missed replacements
  • ran a link checker before and after this PR to ensure nothing missed (noted 4 that were caught/errored)

I think it ok to leave the noted PDF as is for now

If LF comes back online, we can consider reverting this if there is a reason to do so.

[zmn splicing cut-through]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-April/001153.html
[pickhardt bolt splicing]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-June/001322.html
[russell splicing]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-October/001434.html
[ln threads]: https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-November/thread.html
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

broken link

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for lightning-dev this is not fixable at this time. Someone will have to contribute a mbox file and make a public-inbox archive etc... Or if you are really insistent, then you can simply turn it into a wayback machine link.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(post-merge comment) This one can be turned into a Wayback Machine link for completeness.

_posts/cs/newsletters/2024-02-14-newsletter.md Outdated Show resolved Hide resolved
kanzure added a commit to kanzure/bips that referenced this pull request Nov 9, 2024
Unfortunately, lists.linuxfoundation.org is no longer hosting the
bitcoin-dev mailing list static archives nor any other mailing list
static archives.

gnusha.org/url is a web service that redirects old style archive urls to
a new location where the content can be viewed.

For more information see:
https://gnusha.org/url/
https://x.com/kanzure/status/1853779672514826334
bitcoin/bitcoin#29782 (comment)
bitcoin/bitcoin#31240
bitcoinops/bitcoinops.github.io#2018
@bitschmidty bitschmidty merged commit 7d0572a into bitcoinops:master Nov 11, 2024
2 checks passed
@bitschmidty
Copy link
Contributor

Thanks for this PR and for hosting the service, @kanzure !

@kanzure kanzure deleted the fix-linuxfoundation-urls branch November 11, 2024 12:25
@kanzure kanzure restored the fix-linuxfoundation-urls branch November 11, 2024 12:47
@kanzure kanzure deleted the fix-linuxfoundation-urls branch November 11, 2024 12:47
@kanzure
Copy link
Contributor Author

kanzure commented Nov 11, 2024

another option would be to resolve all of the archive urls to their email Message-ID redirects for the bitcoin-dev mailing list and use those urls instead

this is implemented here:
https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b#file-resolver-py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants