Managing the KerbalStuff shutdown?

[bgse]

Might be just my impression, but Kerbalstuff seems to be increasingly unreliable as of late.

Since a vast amount of packages have at least some dependencies to other packages hosted on KS, this is sort of an 'all eggs in one basket' situation.

What could the first steps towards a contingency plan look like for CKAN?

As a band-aid solution, I could imagine some sort of staging process when applying changes, e.g. start downloading packages and defer installation until all selected packages have been successfully downloaded.

Related: https://github.com/KSP-CKAN/CKAN-meta/issues/946

[StarkRG]

I'd suggest having backup download locations, if the primary download fails try the second.

Additionally you could provide a method of allowing the user to supply CKAN with a zip in the (hopefully) rare case that both the primary and secondary download sites fail (or the, again hopefully rare, cases where the dev hasn't provided a secondary download location).

[politas]

"Mirror FOSS mods to S3, allow clients to fallback to mirror, #935" was raised some time ago, and is still being worked on (if slowly)

[LeeroyDing]

Kerbal Stuff is shutting down -- SirCmpwn

[godarklight]

Seeing as KS is open source, I hope the database can be made available when a new hoster can be found?

[TeddyDD]

This is quiet bad news...

[ghost]

I've opened an issue over on KS's repository regarding a possible relaunch.

[ILM126]

I just found out minutes ago that KS is shutting down :
This was all sudden, didn't even know about this :|
Would love to help out in what ever way I can ^_^

[godarklight]

Anyone willing to help jump on IRC. irc.esper.net channels #ckan and #kspmodders. Atm we've got someone setting up KS on their server :)

[ILM126]

Weird, the usual IRC sites I normally go on isn't working...
Even the irc.esper.net one, is there another location where I can connect from?

[ghost]

Install Hexchat, it should have esper.net set up already.

[godarklight]

https://kiwiirc.com/client/irc.esper.net perhaps?

[ILM126]

Thanks :D

[TeddyDD]

I think we need new issue (or rename this) to keep track on KS shutdown. This will be hell lot of work and have long term consequences.

[pjf]

@TeddyDD : Done!

Immediate pre-caffeinated thoughts on the CKAN side of things:

• There's a torrent of all the KS mods which seems to be getting lots of downloads. I'd recommend grabbing a copy of that if you're at all unsure. (I think it's about 66GB in total.)
• We have hosting credits with AWS, so as @politas mentioned some version of Mirror FOSS mods to S3, allow clients to fallback to mirror #935 would work as a short-term solution. This wouldn't require client-side changes; we can update the metadata to point to new buckets that host the mods. However a client-side change is still likely to be the better way to go as it means we can implement fallbacks properly.
• Various CKAN machines including our testing and index servers are likely to have copies of most or all of the mods from KS, although I may have to check with @techman83 to know for sure. These should already have the hash codes we use for caching attached to them, so if we're doing a Mirror FOSS mods to S3, allow clients to fallback to mirror #935-esque solution that could potentially speed things.
• Long term the discussions are happening in IRC as mentioned earlier in this ticket. I'm not yet up-to-date with those conversations.

[ghost]

And we got a GH org going: KSP-SpaceDock/SpaceDock-Project#1

[TeddyDD]

I think #487 is related, and should be reconsidered.
Edit: this too #62

[techman83]

@pjf The Indexer has everything back to November.

[pjf]

@techman83 : Brilliant! Thank you!

[pjf]

Going with the shortest path to getting mods onto users, Jattenalle has set up cdn.kspstuff.com. As we speak, I'm in the process of:

• Using the filelist I've been given to create a hash indexed on mods + version.
• Walking through the CKAN-meta directory looking for KS indexed mods that have a kspstuff file that matches it.
• Rewriting the metadata where found to point to the new location.

I am literally wearing my Superlman t-shirt right now.

[techman83]

That's awesome!

[ILM126]

@pjf You're here to save the mods of KSP! Great 👍

[netkan-bot]

Hey there! I'm a fun-loving automated bot who's responsible for making sure old support tickets get closed out. As we haven't seen any activity on this ticket for a while, we're hoping the problem has been resolved and I'm closing out the ticket automaically. If I'm doing this in error, please add a comment to this ticket to let us know, and we'll re-open it!