Adding a mod to the CKAN
Before anything else, thank you for wanting to contribute to CKAN.
This page is a wiki; please feel free to contribute, and be bold with your edits!
Adding a mod to the CKAN is simply a matter of writing a small JSON file to describe the mod, its relationship with other mods, and the installation process. If the mod is released on SpaceDock, or if the mod is packaged in the usual fashion, then the process is even easier!
If you need a hand to write or submit your file, feel free to hop on IRC (#ckan on irc.esper.net) or open a new issue on NetKAN and we'll be happy to help you!
Ensure the mod you want to add hasn't already been added into CKAN. Under some circumstances the CKAN client "hides" mods so can be unreliable. Performing a couple searches (search bar at the top of the page) of the CKAN metadata repository and the NetKAN repository for the mod's name, spacedock page, github page and/or author name is a good way to check. (You should also try looking in the CKAN app and searching while your filter is set to "all", and/or search this page.)
(Hopefully) temporary important information: Make sure you check the opt-out list before proceeding.
Also be aware that some mod authors (e.g. RoverDude) manage their own metadata files. In this case it may be more appropriate to make a pull request on the mod author's repository to add the new .netkan and then subsequently create a pull request (or let the author make a pull request) in the metadata repository using the $kref string followed by a link (see the CKAN specification for more details) to the actual .netkan in the author's repo. (See the submitting section for more information.)
It's recommended that you go to the CKAN release page and download the latest ckan.exe. You'll also need the latest netkan.exe, which is periodically updated by our build system. These utilities require a Mono/.NET 4.5 environment (and libcurl on linux/mac) to run, and hence are compatible with Windows, Linux, and Mac!
Quick Downloads
Using a capable text editor can help you create valid JSON (the data format used for .ckan and .netkan files).
GitHub's own Atom editor has two useful plugins, atom-linter and linter-jsonlint, which—when both are installed—will warn you about any errors. Select JSON as the grammar to use when editing .ckan and .netkan files; you configure this to happen automatically by installing the file-types add-on and adding this to your config.cson:
"file-types":
ckan: "source.json"
netkan: "source.json"Some useful snippets to add to Atom for netkan files can be found in this gist.
Here's a straightforward guide to get you on the way:
Step 1. Create the metadata file. Start by finding the guide that matches the hosting solution the mod uses.
- Mods on SpaceDock can start here
- Mods on GitHub can start here (Mods hosted on both SpaceDock and GitHub should follow the SpaceDock guide)
- Mods on neither SpaceDock nor GitHub can start here
Step 2. Verify the metadata file. See Verifying Metadata Files section
Step 3. Submit the mod to the CKAN repository. See Submitting section
While we love submissions of new .netkan files, it's also completely fine to request a mod be added. If you're the author of said mod, please make sure you mention that. If you know the author is on GitHub, it's polite to tag them in the request when you open it.
When uploading your mod, look in the bottom right corner for a checkbox labeled "Add mod to CKAN":
If you check this checkbox, your mod will be marked with a CKAN tag, which allows the CKAN team to find it for indexing.
Because the mod is hosted on SpaceDock, once you set up the .netkan file the first time you shouldn't need to update it since CKAN can get required information from the SpaceDock API.
Copy paste the following into a new file titled <identifier>.netkan (see later for how to choose an identifier):
{
"spec_version" : 1,
"identifier" : "<identifier>",
"$kref" : "#/ckan/spacedock/<sdid>",
"license" : "<license string>"
}The spec_version is the version of CKAN required to read the metadata, you should not change this.
You need to change <identifier> to a short, unique, name for the mod that will identify the mod inside the system (end users will not see this name). It may only consist of ASCII-letters, ASCII-digits and - (dash) and it should match the name of the directory that you wish to copy to GameData. Ex: A mod titled "My first ever mod!! I'm so great!$##" that installs to "BestModEver" should have the identifier "BestModEver" because that's where the mod installs to.
If the identifier is not identical to the folder that should be installed you will need to add more information into the .netkan file in order to tell CKAN where to install files to. See the "Special Installations" section.
The <sdid> should be replaced with the SpaceDock ID of the mod. This is an integer number and you can find it in the URL of the mod's page:
spacedock.info/mod/<ksid>/YourModName
The license will usually need to be coded in ex. "license": "CC-BY-NC-SA-4.0". See here for an all-inclusive list of allowable licenses.
If the license on the SpaceDock page exactly matches a license specified in the schema, you can replace the "license" line with "x_netkan_license_ok" : true. This tells the CKAN bot that SpaceDocks's API for licensing information will provide an allowable license string.
If the mod contains a valid .version file for KSP-AVC see this section
If the mod has "Dependencies or Recommended Companion Mods", see the section with that title, if not you can move on to the Verifying Metadata Files section.
Copy paste the following into a new file titled <identifier>.netkan (see later for how to choose an identifier):
{
"spec_version" : 1,
"identifier" : "<identifier>",
"$kref" : "#/ckan/github/<user>/<repo>",
"ksp_version" : "1.0.2",
"license" : "<license string>",
"resources": {
"homepage": "<url>"
}
}The spec_version is the version of CKAN required to read the metadata, if possible, you should not change this.
You need to change <identifier> to a short, unique, name for the mod that will identify the mod inside the system (end users will not see this name). It may only consist of ASCII-letters, ASCII-digits and - (dash) and it should match the name of the directory that you wish to copy to GameData. Ex: A mod titled "My first ever mod!! I'm so great!$##" that installs to "BestModEver" should have the identifier "BestModEver" because that's where the mod installs to.
If the identifier is not identical to the folder that should be installed you will need to add more information into the .netkan file in order to tell CKAN where to install files to. See the "Special Installations" section.
You will need to change <user> to the GitHub username, and <repo> to the repository name (Eg: for a mod hosted at https://github.com/pjf/DogeCoinFlag the <user> is pjf and the <repo> is DogeCoinFlag meaning $kref should be #/ckan/github/pjf/DogeCoinFlag).
The CKAN bot will attempt to turn the Github Repository name into a nice full name, inserting spaces before uppercase letters or instead of dashes. If this is not successful, you may wish to add a name field containing the full name you want users to see for the mod under name
The CKAN bot will similarly use the Repository owner to populate the author's name, which you can replace with one or more authors in an author tag, and if there is a readme.md in the repository, that will be used to create the mod description that users will see, which you can replace with an abstract tag.
If the license of the GitHub repo exactly matches a license specified in the schema, you can replace the license line with "x_netkan_license_ok" : true. This tells the CKAN bot that GitHub's API for licensing information will provide an allowable license string.
Remember, you will have to update the .netkan with the newest ksp_version manually when KSP updates, since Github can't provide us with this information.
Besides when KSP updates though, the mod should remain up to date when new versions of the mod are released.
resources conveys extra information to the CKAN clients that users can see. The most useful of which is homepage which should be changed to the best source of information regarding the mod (usually the mod's KSP thread)
If the mod contains a valid .version file for KSP-AVC see this section
If the mod has "Dependencies or Recommended Companion Mods", see the section with that title, if not you can move on to the Verifying Metadata Files section.
NOTE: The Curse API used by CKAN is disabled as of March 17, 2018. This prevents CKAN from automatically indexing mods hosted on Curse.
Expand this section to see the old instructions, but note that they will not work unless the API is re-enabled!
Because the mod is hosted on Curse, once you set up the .netkan file the first time you shouldn't need to update it since CKAN can get required information from the Curse website.
Copy paste the following into a new file titled <identifier>.netkan (see later for how to choose an identifier):
{
"spec_version" : 1,
"identifier" : "<identifier>",
"abstract" : "A short description of what the mod does",
"$kref" : "#/ckan/curse/<curseid>",
"license" : "<license string>",
"resources": {
"homepage": "<url>"
}
}The spec_version is the version of CKAN required to read the metadata, you should not change this unless adding other elements to the netkan such as install directives.
You need to change <identifier> to a short, unique, name for the mod that will identify the mod inside the system (end users will not see this name). It may only consist of ASCII-letters, ASCII-digits and - (dash) and it should match the name of the directory that you wish to copy to GameData. Ex: A mod titled "My first ever mod!! I'm so great!$##" that installs to "BestModEver" should have the identifier "BestModEver" because that's where the mod installs to.
If the identifier is not identical to the folder that should be installed you will need to add more information into the .netkan file in order to tell CKAN where to install files to. See the "Special Installations" section.
The <curseid> should be replaced with the Curse ID of the mod. This is an integer number and you can find it in the URL of the mod's page:
mods.curse.com/ksp-mods/kerbal/<curseid>-mod-name
The license will usually need to be coded in ex. "license": "CC-BY-NC-SA-4.0". See here for an all-inclusive list of allowable licenses.
If the license on the Curse page exactly matches a license specified in the schema, you can replace the "license" line with "x_netkan_license_ok" : true. This tells the CKAN bot that Curse's API for licensing information will provide an allowable license string.
resources conveys extra information to the CKAN clients that users can see. The most useful of which is homepage which should be changed to the best source of information regarding the mod (usually the mod's KSP thread)
If the mod contains a valid .version file for KSP-AVC see this section
If the mod has "Dependencies or Recommended Companion Mods", see the section with that title, if not you can move on to the Verifying Metadata Files section.
If the mod uses a hosting solution other than SpaceDock, Github or Curse, it might still be possible to include it in the CKAN as long as the hosting allows automated downloads and you are willing to fill in a little bit of information.
Copy paste the following into a new file titled <identifier>.netkan (see later for how to choose an identifier):
{
"spec_version": 1,
"identifier" : "<identifier>",
"name" : "User-friendly name for the mod",
"author" : [ "author1", "author2", "author3" ],
"abstract" : "A short description of what the mod does",
"version" : "1.2.3",
"ksp_version" : "1.0.2",
"license" : "GPL-3.0",
"resources": {
"homepage": "<url>"
},
"download": "<download_url>"
}Please make sure that automated downloads are possible and allowed by the hosting solution.
The spec_version is the version of CKAN required to read the metadata, you should not change this.
You need to change <identifier> to a short, unique, name for the mod that will identify the mod inside the system (end users will not see this name). It may only consist of ASCII-letters, ASCII-digits and - (dash) and it should match the name of the directory that you wish to copy to GameData. Ex: A mod titled "My first ever mod!! I'm so great!$##" that installs to "BestModEver" should have the identifier "BestModEver" because that's where the mod installs to.
If the identifier is not identical to the folder that should be installed you will need to add more information into the .netkan file in order to tell CKAN where to install files to. See the "Special Installations" section.
You will need to add the full name you want users to see for the mod under name, the author's name under author and a description that users will see under abstract.
Each new release of the mod will require manually editing the .netkan with the new version and each update to KSP will require the ksp_version to be updated.
You will also have to choose the appropriate license that the mod is under for license. See here for an all-inclusive list of allowable licenses.
resources conveys extra information to the CKAN clients that users can see. The most useful of which is homepage which should be changed to the best source of information regarding the mod (usually the mod's KSP thread)
Each version of the mod will require manually inputting the download location. Simply replace <download_url> with the direct link to the mod's .zip
Currently our bots will not be able to pick up new releases automatically for hosts other than SpaceDock or GitHub. (As of the time of writing, Curse integration will be added in the near-future)
If the mod contains a valid .version file for KSP-AVC see this section
If the mod has "Dependencies or Recommended Companion Mods", see the section with that title, if not you can move on to the Verifying Metadata Files section.
If the distribution contains a valid .version file for KSP-AVC, we can search it for information about which versions of KSP the mod works with. This is done with just one special line:
"$vref" : "#/ckan/ksp-avc"You should leave out the ksp_version field if you use $vref.
Sometimes your mod's metadata might need to change due to underlying changes in a new release. For example, a new version of your mod may depend on a particular version of another mod, or it may introduce a conflict with some mod that doesn't apply to older versions. Or you may wish to change your install paths. It can be hard to figure out the timing of such changes so that the old version is not accidentally changed and the new version is not released under the old metadata.
In these cases you can put the version-specific metadata in your downloaded ZIP file, in a file with the .ckan extension. The indexing bot will read this file and add any properties it contains to that version's metadata. For example, if a new version requires Ringworld 0.1.0 and conflicts with TransferWindowPlanner, putting this file in the ZIP would take care of those requirements without affecting other versions:
{
"spec_version": "v1.16",
"depends": [
{ "name": "Ringworld", "version": "0.1.0" }
],
"conflicts": [
{ "name": "TransferWindowPlanner" }
]
}Note that an internal .ckan file can only add properties; it cannot override or remove properties that are set in your main netkan entry!
If the mod has relationships to other mods, you can easily tell CKAN by adding just a few lines:
{
"spec_version" : 1,
"identifier" : "<identifier>",
"$kref" : "#/ckan/spacedock/<ksid>",
"depends": [
{ "name": "ModuleManager" }
],
"recommends": [
{ "name": "RasterPropMonitor" }
],
"suggests": [
{ "name": "Firespitter" },
{ "name": "KerbalEngineerRedux" }
]
}Dependencies
With a depends relationship when a user tries to install the mod using CKAN, its dependencies will be also be automatically installed. CKAN won't install a mod unless it can resolve all of its dependencies. When a mod is uninstalled all mods that have a depends relationship to that mod will also be uninstalled.
Sometimes dependencies are bundled with a mod for regular users. In these cases you should use an explicit depends relationship rather than getting the CKAN client to install bundled files. Doing so will not only prevent users from upgrading dependencies, but will also prevent other mods from using them.
Remember to make sure to specify the mod's identifier in the "name" field, or CKAN won't be able to resolve the dependency. You can look at the CKAN-meta repository to find out what mods are supported and find their identifier.
Don't use depends unless you really have to: use them only when they are absolutely necessary for the mod to function properly. Unnecessary depends relatioships can leave a mod un-installable if the dependency cannot be met (especially common when a new version of KSP is released).
Recommended and Suggested Companion Mods
recommends and suggests are entered into the metadata in the same way as depends but they define a different type of relationship:
-
recommendssays that modules should be installed with the mod, but they're not absolutely required. These are installed by default unless the user deselects them. See here for a screenshot of howrecommendsare presented to users. -
suggestsindicates these modules are great things to install as well, but won't be installed by default. This should be the preferred method to suggest another mod to the user. See here for a screenshot of howsuggestsare presented to users.
If the mod has some more elaborate installation instructions, you can use an "install stanza" to make sure everything ends up in the right place. The simplest case is when the mod name is different from the directory you need to install; for example, if an author has multiple mods installed to the same directory you could use:
{
"spec_version" : 1,
"identifier" : "<identifier>",
"$kref" : "#/ckan/spacedock/<ksid>",
"install": [
{
"file" : "GameData/MyAuthorName",
"install_to" : "GameData"
}
]
}In this case if a .zip file has a path to the mod of GameData/MyAuthorName/MyMod the MyMod folder will be installed to the user as GameData/MyAuthorName/MyMod. In some cases directory structures aren't important. For example if the .zip file has a path to the mod of hello/world/MyMod then you would want "file" : "hello/world/MyMod" so that the "MyMod" folder from the zip is installed directly to the "GameData" folder for the user.
You can have multiple install sections if you need to install multiple parts and/or have ship files to copy:
"install": [
{
"file" : "GameData/MyAuthorName/MyMod",
"install_to" : "GameData",
"comment" : "This is the main install for MyMod"
},
{
"file" : "GameData/TextureReplacer",
"install_to" : "GameData",
"comment" : "This would be TextureReplacer configs for MyMod"
},
{
"file" : "Extras/Ships/VAB",
"install_to" : "Ships",
"comment" : "This would correctly install ships from the mod"
}
]Please don't use this to install any mods bundled in the archive (like ModuleManager). This will cause conflicts. Please use an explicit depends instead (see above section).
The paths are resolved as follows:
- The input path is resolved relative to the packaged zip file: please note that "file" can (and usually does) point to folders.
- The "install_to" path is one of these four special folders:
GameDataShipsTutorialsGameRoot
You've created the .netkan file, and now you need to test to see if it works. You'll be using the netkan.exe utility which will verify the file and generate the metadata for the mod. This is a command-line only program, and you can run it simply by providing the file's name:
netkan.exe --verbose <identifier>.netkan
This will produce a .ckan file (the mod's full/inflated metadata) in the same directory. You can leave the --verbose switch out, but then the tool will be silent unless something goes wrong.
Having inflated the metadata using the netkan tool, you should now try installing the mod:
You can use the ckan GUI to test installing the mod: File -> Install from .ckan
Or the command line:
ckan.exe install -c <identifier>-<version>.ckan
And you can then view the files installed:
ckan.exe show <identifier>
If that worked, great! The mod is now officially compatible with the CKAN: the next step is submitting
You can now open an issue on the NetKAN and include a copy of the .netkan or if you're familiar with git and GitHub, then we love pull-requests (to NetKAN, not CKAN-meta) even more! If you send a pull-request and the tests come back negative that's normal. netkan.exe and ckan.exe weren't built as or meant to be a full test suite. Common errors that will get past user tools but not Jenkins include invalid JSON structure (an extra or missing comma) and the name of a .netkan not being identical to the identifier. If you're able to read the console output from the bot: excellent. If not don't fret a member of the team will inform you of changes that are required.
Not sure what pull-requests are but want to learn? See the Pull Requests section
If you are still confused, you might be able to solve your problems by taking a look at the metadata files of other mods. The NetKAN has many examples which our indexer uses. If you feel comfortable using git, you can also send us pull requests to this location!
If you need it, the complete CKAN metadata spec is available, in both human readable and machine readable format. The CKAN provides both a lot of power and flexibility, so check out the spec for more things you can do and the ways to use them.
The indexing bot's most recently checked mods and error messages are listed on this page:
This is mostly useful for mods hosted on GitHub, Curse, and SpaceDock, as mods hosted elsewhere cannot be checked for new releases automatically.
Note that the status page reflects only the bot's processing, and does not change when a mod's metadata is modified manually via pull requests on GitHub. However, such updates will be shown after the bot's next pass is completed.
The ckan bots need to run some checks on the file before they can accept it as valid. Unfortunately, this comes with an unpleasant restriction: the file will only be accepted if the license matches one defined in the CKAN schema.
Please choose a license from that list to ensure that the bots can recognize it.
By design, the CKAN does not allow the overwriting of files, nor does it allow installation order to be specified. This not only gives us a number of strong mathematical assurances (we can analyze dependencies as a set, rather than a graph), but it also means we can immediately detect conflicts, and we don't have to worry about mods installed "earlier" in the install overwriting those from "later" in the install when they're upgraded.
The ideal solution (when you can't use ModuleManager) to this is to split configuration from content. TACLS is a great example here; TACLS give you the core, but depends upon the virtual "TACLS-Config" package. That can be provided by TACLS-Config-Stock, or TACLS-Config-RealismOverhaul, which both conflict and provide TACLS-Config, which means that you can only have one installed at a time. If you're running RealismOverhaul, then it depends upon TACLS-Config-RealismOverhaul, so you get the right one. And if you're not running RO, then you'll get the stock config, because the RO config depends upon RO.
Note that this makes it absolutely clear which package owns which files. If you were to upgrade TACLS, you'd still keep the RealismOverhaul configs for it if they were installed.
Of course, the module authors needn't split their configuration from their content at all; instead our metadata documents specify the TACLS config file that comes with TACLS for "Stock", and the one that comes with RealismOverhaul for RO. No extra files are downloaded, but the end result is we still have strong assurances as to file ownership and consistency.
If you need a hand to write or submit the file, feel free to hop on IRC (#ckan on irc.esper.net) or open a new issue in the NetKAN repository and we'll be happy to help you!
You can also check out the forum thread that contains more information and discussion.
Submitting files via pull requests makes things a little easier on our end. Here's the easiest way to submit one simply using a web browser.
Step 1: Sign up for a GitHub account and then be logged in.
Step 2: Navigate over to KSP-CKAN's NETKAN repository (or just click).
Step 3: Click on the "NetKAN" folder and you'll see the .netkan files for a large number of mods(or just click).
Step 4: Click the "New File" button.
Step 5: At the top insert the file name (example.netkan) then copy + paste the contents of the .netkan file you created earlier into the text box. Click on "Propose new file" then add any relevant details in the comment area and click "Create pull request".
Step 6: Smile. Someone will review the file and either request changes or integrate your contribution into database. The mod will be listed in CKAN shortly thereafter.
What's happening on the backend/more technical stuff? When you clicked on the "New File" link, GitHub creates a copy of the entire NetKAN repository under your name (called a fork). The file you created (the .netkan file) exists in your fork of the NetKAN repository. The "pull request" is you sending the CKAN team a request for them to take the .netkan file you created and incorporate it into the official repository. Why so complicated? For science!