Replace the Env Meta Interface Version with two new meta fields for the Protocol Version and the Pre-Release Version

[leighmcculloch]

I'd like to propose that we deprecate and stop using the Env Meta Interface Version and replace it with two new Env Meta fields:

• Protocol Version
• Pre-Release Version

The interface version is currently a 64 bit integer containing the protocol version in the high 32 bits and the pre-release version in the low 32 bits.

The protocol version is the Stellar protocol version, such as 20 currently and 21 upcoming.

The pre-release version is currently zero (0) for any formal release that is destined for testnet/pubnet, and a non-zero incrementing value for any release to futurenet or any release that isn't destined for pubnet. It is typically a value like 1, 2, 3, etc, and then resets to 0 when the protocol release is made formally.

The interface version as such takes two numbers that are easy to understand like 20 and 0, or 20 and 1, and turns it into 85899345920 or 85899345921 respectively. This combined number is simple and easy to unwind, but also confusing and unintuitive because to the human eye it is opaque.

This has been confusing for developers who have dug into what's stored in the files and those building developer tooling. Of course with a little explanation and documentation we are bringing people up to speed who need to know or who are curious, but given that the XDR format, and structures we are using would support this being self documented I think we can fix the way we're storing these values for future releases.

The Env Meta XDR that is embedded in contracts is defined to be human understandable and we're able to use the existing discriminant to add new fields with new data. Splitting up the existing interface version into the two separate fields achieves just that.

To illustrate what the change would involve, I've opened the following pull request demonstrating the change in XDR. The diff is also included below.

diff --git a/Stellar-contract-env-meta.x b/Stellar-contract-env-meta.x
index 330726d..5bacb49 100644
--- a/Stellar-contract-env-meta.x
+++ b/Stellar-contract-env-meta.x
@@ -11,13 +11,24 @@ namespace stellar
 
 enum SCEnvMetaKind
 {
-    SC_ENV_META_KIND_INTERFACE_VERSION = 0
+    // Env interface version (protocol version << 32 | prerelease version).
+    // Deprecated.
+    SC_ENV_META_KIND_INTERFACE_VERSION = 0,
+    // Env minimum protocol version that the contract will run on.
+    SC_ENV_META_KIND_MINIMUM_PROTOCOL_VERSION = 1,
+    // Env pre-release version that the contract is built against. Present for
+    // contracts built against pre-release builds of the env only.
+    SC_ENV_META_KIND_PRE_RELEASE_VERSION = 2
 };
 
 union SCEnvMetaEntry switch (SCEnvMetaKind kind)
 {
 case SC_ENV_META_KIND_INTERFACE_VERSION:
     uint64 interfaceVersion;
+case SC_ENV_META_KIND_MINIMUM_PROTOCOL_VERSION:
+    uint32 minProtocolVersion;
+case SC_ENV_META_KIND_PRE_RELEASE_VERSION:
+    uint32 preReleaseVersion;
 };
 
 }

Ref: stellar/stellar-xdr#179

A similar proposal by @sisuresh was made here to remove the pre-release version: stellar/rs-soroban-env#1378

[wildework]

I fully support the break up of that Hyper into multiple values. One thing that still brothers me a little bit is the naming scheme, specifically the PRE_RELEASE part, because at the moment they're not obviously related to one another. Perhaps SC_ENV_META_KIND_PRE_PROTOCOL_VERSION would be better?

Alongside this update, I would be super happy to see the RPC method getNetwork display the range of supported protocol and pre-protocol versions! Right now I don't know of an easy way to check what each network supports. Sorry for the off-topic!

[leighmcculloch]

The pre-release value is actually the version of the env.json interface before it's been released in a formal release. Even though the env.json interface is part of the protocol, it's not the protocol in general, it's just a part of it, so I don't think calling it the pre-protocol version is appropriate.

Maybe we should go back to the version being the 'env host fn interface version' and stop resetting it to zero? (cc @graydon)

[leighmcculloch]

Or at least rename it to SC_ENV_META_KIND_HOST_FN_INTERFACE_PRE_RELEASE_VERSION (and keep resetting it to zero)?

[wildework]

How about SC_ENV_META_KIND_HOST_INTERFACE_VERSION_BUMP, makes it sound more like what it is - an incrementing value from zero.

[leighmcculloch]

Bump is a verb in this sense, and this should probably be a noun so that it describes the thing, not the action associated with it. How about: SC_ENV_META_KIND_HOST_FN_INTERFACE_DEV_VERSION?

[wildework]

SC_ENV_META_KIND_HOST_DEVELOPMENT_ITERATION?

[sisuresh]

@dmkozh mentioned today that some downstream consumers misunderstood what "protocol version" means in the interface version, so if we replace the fields with new ones, we should also rename protocol version to max supported protocol version.

[wildework]

I'd love to move forward with splitting the version into two parts, so at this point I'm happy with any name. I feel like naming discussions often can take up a lot of cycle without yielding much benefit.

Let's just do it and we can explain the names later if people have issues.

[leighmcculloch]

@dmkozh Would the soroban version major be it's own value incremented on its own schedule, or would it still be the protocol version?

[dmkozh]

There can't be a separate schedule for the interface as it's a protocol change. My idea is to distinguish naming between what the contract dev sets and the actual protocol running. So while the 'soroban version' is based on the protocol version, it doesn't mean that one can't create v20 contract while the network runs protocol 21, and this contract will work just fine. Again, I'm fine with any term that doesn't overload the term 'protocol version'.

[leighmcculloch]

stellar/soroban-rpc#10 has some discussion regarding this.

I think the issues discussed in stellar/soroban-rpc#10 are pretty different to the issue being discussed here. The issue there is how to communicate about and manage a multi-env-version server product.

The issue here is simpler, it's how a contract signals the minimum version it requires.

Today in SDK v20 and Protocol 20 if your contract is built against soroban-env-* v20.y.z, then your minimum version is 20. (Except if you build against an unreleased or prereleased soroban-env-* then we say that your min and max version is (20, the pre-release number).)

This proposal isn't changing the simplicity of ☝🏻. The goal is still for a contract to signal what its minimum version is based off the soroban-env-* it is compiled against.

This proposal is asking for the components of the version to be named and transparent rather than opaque.

[leighmcculloch]

So while the 'soroban version' is based on the protocol version, it doesn't mean that one can't create v20 contract while the network runs protocol 21, and this contract will work just fine.

I don't think we have this issue. Contracts built against env v20 are already signalling their min version, not an exact version. And this proposal doesn't change that:

A contract built against soroban-env-* v20.y.z will contain SC_ENV_META_KIND_MINIMUM_PROTOCOL_VERSION that will be compared <= with the current protocol version.

A contract built against soroban-env-* v20.0.0-preview.1 will also contain SC_ENV_META_KIND_PRE_RELEASE_VERSION that when present will require an exact match to execute.

[leighmcculloch]

I'd like to make an alternative proposal that I believe does not need a protocol change and will get 80% of the benefit, and is backwards compatible.

We replace the uint64 interface version with a struct containing two uint32 fields. There will be no binary change, but applications that use the XDR structure as is (such as JSON-XDR format in developer tooling) will see the sub-fields isolated from one another rather than a large confusing number.

 case SC_ENV_META_KIND_INTERFACE_VERSION:
-    uint64 interfaceVersion;
+    struct {
+        uint32 protocolVersion;
+        uint32 preReleaseVersion;
+    } interfaceVersion;

This alternative suggestion doesn't have the advantage of saving 4 bytes in contracts, but we can still continue to debate that change separately. That seems like a small benefit when we can solve the primary problem with a backwards compatible change.

[leighmcculloch]

The change is proposed here:

• Split the interface version into protocol and pre-release-version (curr) stellar-xdr#212

I think we can bikeshed forever about the naming of these fields. I propose we keep the names simple as defined in the PR because the benefit is 80% in splitting the fields, and not in reinventing the concepts they represent.

[leighmcculloch]

This change, that isn't a protocol change, and just a restructure of the XDR, has been made to the XDR and Soroban Env in:

• Split the interface version into protocol and pre-release-version (curr) stellar-xdr#212
• Split the interface version into protocol and pre-release-version (next) stellar-xdr#213
• Update with stellar-xdr containing split interface rs-stellar-xdr#389
• Update with stellar-xdr with split interface version rs-soroban-env#1453