[Spec] Block Versioning

[hexabits]

Various features being worked on necessitate that an entire block scope be limited to a range or set of versions to minimize how many distinct NIF versions "see" that block. Range or set are the two important ways that block versions need to be referenced.

Note: The syntax here will be relying on Issues #69 and #70 , so it is important to be familiarized with those tickets. Most importantly, a "version" has become more than simply the 4-component Version number, and these proposed attributes will not be using expressions. Therefore they will be using the new ID attribute to reference these distinct versions.

Range

Some blocks can simply be defined as implemented in a specific version or removed in a specific version, or both. They are available to all versions within that range.

Example

<niobject name="NiPhysXDynamicSrc" inherit="NiPhysXRigidBodySrc" since="V20_2_0_8">
<niobject name="NiMesh" inherit="NiRenderObject" since="V20_5_0_0">
<niobject name="NiPSysModifier" abstract="1" inherit="NiObject" since="V10_1_0_0" until="V20_5_0_0">

Note: The until for NiPSysModifier is actually the version it was deprecated. It's possible that some game still uses NiPSys* instead of the replacement system. This is addressed below in For Discussion.

Set

Some blocks need to have each version they belong to listed in a non-continuous fashion. Bethesda's custom blocks are a perfect example, as are nearly all custom blocks.

When a block is for only one single version, using since/until is a bit clunky and verbose, so sets of versions can also have a size of 1.

Note: Space-separated is the correct way to write lists in XML.

Example

<niobject name="Ni3dsAlphaAnimator" inherit="NiObject" versions="V2_3">
<niobject name="AvoidNode" inherit="NiNode" versions="V4_0_0_2">
<niobject name="FxWidget" inherit="NiNode" versions="V10_0_1_0">
<niobject name="BSTriShape" inherit="NiAVObject" versions="V20_2_0_7_SSE V20_2_0_7_FO4">
<niobject name="BSDynamicTriShape" inherit="BSTriShape" versions="V20_2_0_7_SSE">
<!-- Reference Issue #70 for these. In short they are tokens representing a list of many versions. -->
<niobject name="bhkRefObject" versions="#BETHESDA#">
<niobject name="bhkPoseArray" versions="#FO3#" />

For Discussion

Deprecated vs Removed

The attribute until implies the finality of "removed", yet most info available to us only denotes the version in which the block was deprecated. So alternatively,

since, until => introduced, deprecated, removed

...Instead? The word "introduced" was chosen because "implemented" clashes a bit too much with the proposed block contracts specification syntax (#55). In fact since could still be kept, but split until into deprecated/removed, whichever is applicable and available.

Deprecation as part of the block attributes would have a number of uses, namely warning that a NIF being loaded is using deprecated systems, or if creating a NIF yourself, that you might want to use the newer alternatives.

One important difference between until and removed is that until is inclusive and removed is not. This should be taken into consideration when deciding which set of attributes makes the most sense.

Opinion: I believe that since/until is most natural, and deprecated can be a third optional attribute associated with these attributes. And that way you are also making the deprecation knowledge part of the spec.

Inheritance

Technically, the versions a block belongs to must confine any children of that block to only those versions. The children can override the set of versions, but only from the versions on the parent. For example, a child block cannot apply to 30.0.0.2 if the parent was removed in 20.5.0.0. Verifying this need not fall on parsers or interpreters however. It would be part of the XML's eventual test suite.

The question is, do we repeat the version info on every block even if it's inherited? This is repetitious, but also explicit. It also simplifies parsing as you do not need to follow each parent up the hierarchy to get a set of versions for a child block.

Opinion: Marching through ancestors to get the set of applicable versions is unwieldy for parsers. Putting the versions on each block is verbose but also explicit, and easy to validate there are no violations (children listing versions outside their parents' sets).

Mutual exclusivity by version

Can a block have more than one definition if they are mutually exclusive due to versioning?

Some blocks like NiGeometry/NiGeometryData and NiParticlesData are extremely complex due to changes by Bethesda to the actual inheritance of the types. For some NIF versions, NiGeometry was removed and replaced with BSGeometry. The current nif.xml goes through a lot of effort to fake this inheritance change.

However, there is likely a much simpler way and that's with aliasing, e.g. If Version == V20_2_0_7_SSE, NiGeometryData -> BSGeometryData, NiGeometry -> BSGeometry, and so on. This essentially reroutes the RTTI type to a new type for specific versions. Although this would likely harm interoperability of classes and damage the API for stuff like niflib and prohibit code reuse. So discussion of this feature would need its own ticket.

Opinion: Type redirection to simplify some of the blocks messed up by custom versions sounds very appealing, but needs more thought, and its own ticket. Using the block versions to have exclusive types seems suboptimal compared to conditional type aliasing.

Scope

Should this apply to only niobject and compound or do enum and bitflag benefit as well? The reasoning for applying it to those types as well is mostly for cleanliness when it comes to code generation which targets only certain versions or ignores unsupported versions. If you do not generate the code that uses the enums, you do not need the enums either.

Opinion: Should be applicable for all the datatypes, except maybe <basic>.

Referencing types which are unavailable or "hidden"

Any parser/intepreter/generator will need to be able to deal with the notion of unavailable types for a version. For example:

<niobject name="NiMesh" inherit="NiRenderObject" since="V20_5_0_0">
    <!-- ... -->
    <!-- Epic Mickey -->
    <add name="Has Extra EM Data" type="bool" ver1="20.6.5.0" ver2="20.6.5.0" vercond="(User Version &gt; 9)" />
    <add name="Extra EM Data" type="ExtraMeshDataEpicMickey" cond="Has Extra EM Data" ver1="20.6.5.0" ver2="20.6.5.0" vercond="(User Version &gt; 9)" />
</niobject>

Note that since the "Has" row is explicitly versioned, everything but the cond in the next row is actually unnecessary. The "Has" can never be true if it doesn't exist in the current version. So this is also currently valid in the XML:

<add name="Has Extra EM Data" type="bool" ver1="20.6.5.0" ver2="20.6.5.0" vercond="(User Version &gt; 9)" />
<add name="Extra EM Data" type="ExtraMeshDataEpicMickey" cond="Has Extra EM Data" />

However, the ExtraMeshDataEpicMickey type--which only applies to 20.6.5.0--may be entirely ignored, or in the case of generation completely absent if the code was generated without support for that custom version.

So, any parser will need to be able to resolve the type="<T>" and if <T> is not found due to versioning, actually completely remove the <add> from its generation, AST, or whatever data structures being used during parsing.

[hexabits]

I have added two sections to the bottom of the issue, "Scope" and "Referencing types...".

As of right now, I am adding since/until or versions to enums, bitflags, compounds, and niobjects. Having the version info available for enumerations is useful enough to justify adding the attribute to it.

[Candoran2]

When using since and until, should the version order be taken from the order in which the versions appear in the xml, or from comparing version, then user version, and then bs version? Best I can tell that's functionally equivalent at the moment due to the way the versions are ordered in the xml, but it's still useful to specify.

[neomonkeus]

I would think the latter is more robust.

[Candoran2]

Alright, in that case it ties in neatly with the proposed verattr tag, since you can use that to specify the order the attributes should be evaluated in:

<verattr name="num" access="#VER#" index="0" />
<verattr name="user" access="#USER#" index="1" />
<verattr name="bsver" access="#BSVER#" index="2" />

[neomonkeus]

Yep, I would still keep things ordered for now for the cases that don't support the verattr which realistically would only be NifiSkope....hows your C++ 😜

[Candoran2]

Yep, I would still keep things ordered for now for the cases that don't support the verattr which realistically would only be NifiSkope....hows your C++ 😜

I have no experience with C++, so I can't make any promises as to actual functionality, but I've at least gotten NifSkope to accept the verattr token in the xml and open files without issue just now. Building it was the real headache.