Github userscript request

[CollinChaffin]

Hey Mottie!

Since you have by far created the most comprehensive (and most functional) GM/TM userscripts for Github, I have one I've been trying to figure out myself that is a hole that nobody has yet filled. Perhaps this simply cannot be accomplished via userscript but I'll post here to get your thoughts.

Site/Match: gist.github.com
Functionality: Gists - the one "button" nobody has managed to add that is sorely needed is, upon executing a GIST search, a new "RAW" button for each gist returned to allow a direct pull of the raw source. Currently, you must first visit each Gist, then either click the "Raw", or an added "Rawgit" button, etc. It is a bit absurd that each file must be individually opened first before then having to click RAW to save it.
Optional: An optional add either with this or probably should be a whole new request (but I'm lazy tonight) is to add the (still - unbelievably) totally void/missing DATE SORT of Gists returned from search. They (can) actually be sorted but there is no graphical method today on the page and I and others have literally been posting and complaining to Github for many years begging for this and other basic search features to be added. At least this one can be added by the community ourselves and I'm surprised nobody has added this one too.

Again your scripts are becoming an everyday requirement for Github users, and frankly Github should be paying you because they also really just make Github more fun to use!

[Mottie]

Hi @CollinChaffin!

Thanks! I'm glad you're finding the scripts I've created useful 😸. I probably should apply to work at GitHub since I am looking for a job right now 😉

To clarify, you'd like to add a "raw" and "rawgit" button to each gist listing? Would you like it in a similar location to the gist to dabblet links?

As for the sorting of gists, I have created a search autocomplete userscript that you can use on https://gist.github.com/search, but apparently I'll need to update it to work on the result page (e.g. https://gist.github.com/search?q=css).

[Mottie]

Preliminary investigation has revealed that adding a "raw" and "rawgit" button would require accessing GitHub's API to get the raw url for individual files. For example, my repo shown in the above example has this link in "my gists" list

https://gist.github.com/Mottie/3290897 (overall gist link)

Inside, each revision has a unique sha plus the file name:

https://gist.github.com/Mottie/3290897/raw/675f659f2e6e3f078c983c3589eaf06817dee56b/dabblet.css
https://gist.github.com/Mottie/3290897/raw/675f659f2e6e3f078c983c3589eaf06817dee56b/dabblet.html
https://gist.github.com/Mottie/3290897/raw/675f659f2e6e3f078c983c3589eaf06817dee56b/settings.json

If this was implemented, a dropdown would need to be added to allow choosing a specific file. I don't think adding a separate link for each file would work, especially if there were more than 3 files.

To avoid needing to include a personal access token, I could add a basic button that doesn't access the API until you click on it, but honestly, I'm not sure how much better that would be than to actually click on the gist and open it in a new tab.

[CollinChaffin]

Hey @Mottie! Sorry I don't think I explained it very well. The Gist search results have no way currently at all to download via any RAW link - you must first visit the individual Gist page, at which time via your other scripts I already do have a RAW, a RAWGit, even a different script HEAD button, etc. All that works fine once the individual Gist page is loaded. Again, what's missing is any link or button to the actual RAW/HEAD Gists to download them straight from the search result which is this request.

The second part I am actually going to work on too I have several scripts I have yet to release myself so if you don't have the time to look at adding the much needed SORT option of the Gist results - I can work on that.....it's really the above since you are the "button" master and already added the RAWGit button to the individual gist, and as you pointed out I also run (along with about two dozen others of yours) the search autocomplete. :)

I do see what you mean, in the search results for Gists I believe you are correct that the Gists listed when visited can actually contain multiple files - so adding a button to the single Gists listed in the search results may not be perfect or even all-inclusive, but even if it literally only provided a RAW download link for simply the very first file in each Gist entry on the search results, it's still one more than today and I would still use it and guess others would too.

Thanks again and if it winds up looking like too much work don't worry about it I honestly thought adding a download button in the search results would be easier than it probably is.....big surprise! :)

[CollinChaffin]

Just an idea after looking at everything again......not sure I understand the need for a persona token etc or even worrying about throttling etc. since I can use a download mgr and quickly download a few gist files back to back without needing a token since it's simply executing a handful of normal http get requests in the context of the user. Even if I highlight an entire Gist search result page with IDM or another download manager and tell it to download and save all those HTML pages, I have not seen any issue with throttling or hitting any limits etc., or am I wrong about this?

Either way, something you said made me think. Consider this -- we may be over-thinking it - what about just a "Download All" button? A single button next to each gist in the search results and probably best just to the left of the "2 Files" as shown here:

So, here is the high-level in my head around this and you tell me what you think:

TWO Functions:

1. httprequest to initiate active download of a gist in browser
2. httprequest to scrape/parse the individual Gist page for RAW links (*COMPLETE VIA YOUR RAWGit BUTTON FUNCTION)

Nested foreach loops on Gist search result page load:

1. Executes on gist search URL: Begin outer loop each Gist search result of the httprequest of each individual Gist page (CALL FUNCTION 1 ABOVE FOR EACH GIST RESULT)
2. Inner loop - store the RAW URL for every file in the Gist in array found in the Gist's detail page downloaded in Add this cool feature to github.io #1
3. End of outer loop - initiate download of all file stored in array (CALL FUNCTION 2 ABOVE FOR EACH FILE FOUND IN GIST DETAIL)

So, for the user, upon click the browser will ether pop open the saveas normal dialog to select save location for a single file, or for each of multiple downloads just as it does on MS and other sites where a multi-file download is initiated. Even if every GIst on a search result had 10 files, for my browser to automatically request every (10 at a time) Gist detail pages, and then auto-request 5 files in each Gist - if I do that manually today without a GM script back to back quickly, it does not appear to be hitting any limits. If I keep going (which I just did for almost 30 min and about 50 gists), I still never hit any issue so I do not know what kind of rate limiting is in place for straight gist downloading, but unless my testing is off I don't think this would cause issues but if I've missed something that you think throws a wrench into the works in attempting this just let me know!

I'm anxious to hear your thoughts on this!

[Mottie]

A personal token isn't necessary unless you plan on accessing the API a lot! And it is better to use the API than to scrape the HTML pages for data since GitHub is constantly changing their layout class names.

I put together a script that will add a dropdown that contains links to the raw code for all the latest files in the gist. Here's a screenshot.

I'll start working on the wiki page now... it should be available soon 😸

Oh, and let me know if the 🍣 (sushi) emoji next to the dropdown is annoying.. I can leave it off 😉.

[Mottie]

Done... link to wiki page: https://github.com/Mottie/GitHub-userscripts/wiki/Gist-raw-links

[CollinChaffin]

WOW! This is awesome! I can confirm it's tested working great here! The only thing I noticed - more of just an observation since I use a download manager anyway, is in the dropdown list of files, if I mouseover the statusbar does in fact show the correct RAW URLs, and if I then either use my alt-click DM or right-click and if I select to save link as....then it DOES save each RAW file perfectly........but the weird observation I haven't come across this before so maybe it's the way that dropdown element is created but single-clicking on each RAW file does nothing, no matter which Gist or type of file it's weird but again every file will save as just fine using the right click target.

Amazing work - Github really should hire you!

CHECK THIS ONE OUT FOLKS!!

[CollinChaffin]

Hey @Mottie one thing I just came across that I'm still trying to debug......for some reason if I disable (through trial and error) the "Wide Github" script, this one will draw the initial dropdown, but clicking it will not actually populate and open to show any files. Not sure why or if this makes sense to you but were you developing it with the wide github script enabled so there is a dependency?

[Mottie]

Hmm, yeah, I guess I never clicked on those links LOL... I'll look into the problem.

I don't really use GitHub wide, so I'm not sure why there would be a problem. I'll check that out too.

[CollinChaffin]

Hey here are 2 screenshots I just took to help.....this one is the elements with the "wide github" script enabled and everything working:

and after the only thing I did was disable the other "Wide Github" script doing a page refresh and inspecting the same dropdown elements:

As you can see, there is a difference but I don't understand WHY LOL......but you're so quick on this one if you look at this it will probably make sense.....and again I thought you wrote the Wide Github because this new one actually only works with that other one ENABLED, so it does't break this one it fixes it (which is why I figured it was one of yours.)

[Mottie]

LOL, give me a minute to investigate... I actually didn't write GitHub Wide style or the GitHub raw button script.

[Mottie]

hmm, are you talking about this GitHub wide userstyle? Or are you using a userscript to do that?

[CollinChaffin]

Sorry no it's here:

https://github.com/xthexder/wide-github

As a userscript. And again, your new script here only works when I actually ENABLE the wide userscript (which is why my initial tests worked). If I disable all other github related scripts and only enable this new one, the dropdown won't function. If I then enable as the only other github related userscript in TM the above wide one, then yours works. That's why I assumed without looking that was one of yours that maybe you had enabled during DEV of this one.......I can't figure out why it's happening but you can see in the elements I screenshotted it almost is hanging up on something. I'll try doing some more debug in dev tools too.

[Mottie]

Very odd... which browser are you testing this in? The screenshots look like Chrome... I tested the userscript successfully in Chrome and Firefox.

I'll have an update available in a second... it'll fix the left click, but I'm still not sure why it's hanging up on you.

[CollinChaffin]

Chrome. I'm doing more debugging but here is the search link I have open to test. The dropdowns get painted but looking at the JS Log, when I enable the wide script, I see your script logging as it parses the links. Now, with the only github related script (and the only single script that TM shows active on this page), it does NOT log in the console that it is even loading the Gists to populate the lists. No idea why, though.

EDIT: sorry here is link:

https://gist.github.com/search?l=PowerShell&p=3&q=function&ref=searchresults&utf8=%E2%9C%93

[Mottie]

Oh... well, the userscript uses some of GitHub's built-in functionality to open the dropdown. I see there is a script error on that page in ticky-browser.js (some GitHub script) which is preventing any further javascript from running.

I guess I could bypass GitHub's code to open the dropdown... that should fix the problem.

[CollinChaffin]

Hmm okay makes sense I just threw some quick poor mans console.log calls into your functions when I click the dropdown sure enough I get my console log hit that addBindings is then being called, but then it never then gets past the:

if (target.classList.contains("ghrl-get-list"))

So that seems to back up what you just said about the dropdown call......?

[Mottie]

Oh duh... I just need to add a z-index to the link... the file name content is overlapping it.

[CollinChaffin]

Weird I just re-tested with WIDE on again and sure enough my 2nd breakpoint got hit properly. All that wide script does for Gists is:

// Gists
".gist-content-wrapper .container {" +
"width: auto !important;" +
"margin-left: 20px !important;" +
"margin-right: 20px !important;" +
"min-width: 980px;" +
"}" +

But I'm guessing it's because it's physically forcing that addition?

[CollinChaffin]

AHH LOL! Nice I missed that too!

[CollinChaffin]

Hence why the damn wide appeared to fix it, LOL!

[CollinChaffin]

LOL, and that also explained why my single left click didn't work! 2 birds with one stone - 100% working great now!

[Mottie]

Excellent! I've just updated it again to use the newest version raw url - see #18.