Editor-agnostic built-in package check

[ComradeVanti]

I just thought of an idea for checking whether a package is built-in without having to have any editor installed. I would like a sanity check on this to see if I am mistaken or missing anything.

Lets take three example packages:

1. com.unity.xr.interaction.toolkit: Unity package, that is not built-in
2. com.unity.modules.audio: Unity package, that is built-in
3. com.unity.very.cool: Looks like a Unity package, but does not exist

So first built-in packages are not found on the unity package registry. So if we make requests to the unity registry with all three packages we get: 1 exists, 2 and 3 don't exist. So based on that 1 is not a built-in package, but 2 and 3 are.

But this obviously misses packages which don't exist on the unity registry, not because they are built-in, but because they don't exist at all. So we also need a way to determine if a package is an official unity package. For this I propose checking if there is a manual page for it, so for example https://docs.unity3d.com/Manual/com.unity.xr.interaction.toolkit.html and so on. For the example packages here, a web request to the corresponding manual page will return 200 for 1 and 2 and 404 for 3.

So a package is built-in if A. a manual page for it exists and B. it does not exist on the unity registry.

[ComradeVanti]

Obviously this only gives us a yes/no of whether the package is built-in but does not get us closer to getting the packages package.json without installing the editor. But it's a start.

[ComradeVanti]

@favoyang thoughts on this?

[ComradeVanti]

One special case that can cause problems here are packages that are only built-in for some editors/package versions.

For example com.unity.ugui seems to be built-in for versions < 3.0.0 but will be on the registry >= 3.0.0.

Then there are also packages like com.unity.render-pipelines.universal which were on the registry and then became built-in from a certain version on.

So the built-in check can not be done for a whole package, but only specific versions of it.

[ComradeVanti]

I am also current working on overhauling the output of the deps command (I will open a separate discussion of this soon)

Here are some examples using this new approach.

openupm deps com.unity.xr.interaction.toolkit --deep

notice com.unity.xr.interaction.toolkit@3.0.3 [Unity]
notice |_ com.unity.inputsystem@1.8.1
notice |_ com.unity.mathematics@1.2.6
notice |_ com.unity.ugui@1.0.0
notice |_ com.unity.xr.core-utils@2.2.3
notice |_ com.unity.modules.audio@1.0.0
notice |_ com.unity.modules.imgui@1.0.0
notice |_ com.unity.modules.physics@1.0.0
notice |_ com.unity.modules.xr@1.0.0

notice com.unity.inputsystem@1.8.1 [Unity]
notice |_ com.unity.modules.uielements@1.0.0

notice com.unity.ugui@1.0.0 [Unity] [Built-in]

notice com.unity.xr.core-utils@2.2.3 [Unity]
notice |_ com.unity.modules.xr@1.0.0

notice com.unity.modules.audio@1.0.0 [Unity] [Built-in]

notice com.unity.modules.imgui@1.0.0 [Unity] [Built-in]

notice com.unity.modules.physics@1.0.0 [Unity] [Built-in]

notice com.unity.modules.xr@1.0.0 [Unity] [Built-in]

notice com.unity.modules.uielements@1.0.0 [Unity] [Built-in]

notice com.unity.modules.xr@1.0.0 [Unity] [Built-in]

openupm deps com.unity.postprocessing@14.0.9 --deep

See #133

notice com.unity.postprocessing@14.0.9 [Unity] [Built-in]

openupm deps com.unity.polyspatial --deep

See #133

notice com.unity.polyspatial@2.0.0-pre.3 [Unity]
notice |_ com.unity.nuget.newtonsoft-json@3.0.2
notice |_ com.unity.render-pipelines.universal@14.0.1
notice |_ com.unity.collections@2.4.0
notice |_ com.unity.textmeshpro@3.0.7
notice |_ com.unity.xr.core-utils@2.3.0
notice |_ com.unity.ext.flatsharp@1.1.1
notice |_ com.unity.modules.particlesystem@1.0.0
notice |_ com.unity.inputsystem@1.4.4
notice |_ com.unity.modules.video@1.0.0
notice |_ com.unity.ugui@1.0.0

notice com.unity.render-pipelines.universal@14.0.1 [Unity] [Built-in]

notice com.unity.collections@2.4.0 [Unity]
notice |_ com.unity.burst@1.8.12
notice |_ com.unity.nuget.mono-cecil@1.11.4
notice |_ com.unity.test-framework@1.4.3
notice |_ com.unity.test-framework.performance@3.0.3

notice com.unity.textmeshpro@3.0.7 [Unity]
notice |_ com.unity.ugui@1.0.0

notice com.unity.xr.core-utils@2.3.0 [Unity]
notice |_ com.unity.modules.xr@1.0.0

notice com.unity.modules.particlesystem@1.0.0 [Unity] [Built-in]

notice com.unity.inputsystem@1.4.4 [Unity]
notice |_ com.unity.modules.uielements@1.0.0

notice com.unity.modules.video@1.0.0 [Unity] [Built-in]

notice com.unity.ugui@1.0.0 [Unity] [Built-in]

notice com.unity.burst@1.8.12 [Unity]
notice |_ com.unity.mathematics@1.2.1
notice |_ com.unity.modules.jsonserialize@1.0.0

notice com.unity.test-framework@1.4.3 [Unity]
notice |_ com.unity.ext.nunit@2.0.3
notice |_ com.unity.modules.imgui@1.0.0
notice |_ com.unity.modules.jsonserialize@1.0.0

notice com.unity.test-framework.performance@3.0.3 [Unity]
notice |_ com.unity.test-framework@1.1.31
notice |_ com.unity.modules.jsonserialize@1.0.0

notice com.unity.ugui@1.0.0 [Unity] [Built-in]

notice com.unity.modules.xr@1.0.0 [Unity] [Built-in]

notice com.unity.modules.uielements@1.0.0 [Unity] [Built-in]

notice com.unity.modules.jsonserialize@1.0.0 [Unity] [Built-in]

notice com.unity.modules.imgui@1.0.0 [Unity] [Built-in]

notice com.unity.modules.jsonserialize@1.0.0 [Unity] [Built-in]

notice com.unity.test-framework@1.1.31 [Unity]
notice |_ com.unity.ext.nunit@1.0.6
notice |_ com.unity.modules.imgui@1.0.0
notice |_ com.unity.modules.jsonserialize@1.0.0

notice com.unity.modules.jsonserialize@1.0.0 [Unity] [Built-in]

Seems about right, no?

[ComradeVanti]

Made a draft PR so you can look at the code: #364. It still uses the current dependency output formatting

[favoyang]

You have listed all the possibilities for com.unity scope.

1 Ship-with-editor package (Core + Buit-in) and Unity registry package

• a core Unity package com.unity.2d.sprite
• a built-in package com.unity.modules.audio
• a Unity registry package com.unity.xr.interaction.toolkit
• was a Unity registry package and then became a ship-with-editor package com.unity.render-pipelines.universal
• was a ship-with-editor package and then became a registry package com.unity.ugui

2 Other Unity package (use com.unity scope, made by Unity, hosted somewhere else)

• an openupm hosted Unity scope package com.unity.goql, and more.

3 Non-Unity package

• a Unity scope package that exists on the current project (Packages/com.unity.edge-case, a very edge case that I don't care about)
• a Unity scope package that doesn't exist com.unity.typo

A reliable way to check if a package is a Unity package would be

• Check ship-with-editor packages from the installed Editor
• Query the Unity registry
• If both fail then treat it as a normal package.

The proposal to use the Manual URL to verify the Unity package seems a tricky way. Though it may not that reliable if Unity changes the website restructure.

[ComradeVanti]

I will look into how long the unity manual page structure has been the same way. If it is very stable we could think about actually using it.

[ComradeVanti]

Seems like the documentation web-page had this shape since at least 2020, though not all built-in packages seem to have been listed back then. I'll look around for any other possible ways we can check the built-in status without having the editor installed.

[ComradeVanti]

Another note. It initially seemed like the above method looks up whether the packaege is a unity package, regardless of editor. This is not correct. When using the basic manual url it will look up the latest lts editor version, as can be seen here.

When using a different version, for example here 2020.2 the url changes to include the version.

[ComradeVanti]

This means that, theoretically, very old Unity packages could eventually stop being listed in these docs for the new Unity editor versions. Just something to keep in mind.

[ComradeVanti]

Personally I feel like checking whether a doc page exists to determine if a package is an official Unity package, seems reliable enough. If you agree @favoyang I would finish implementing this. If not, then we can look for some other way.

[favoyang]

Personally I feel like checking whether a doc page exists to determine if a package is an official Unity package, seems reliable enough. If you agree @favoyang I would finish implementing this. If not, then we can look for some other way.

I agree it is pragmatic enough.

[ComradeVanti]

The related PR #364 is now closed an merged. Closing the discussion