CIP-33: Reference scripts #161
Conversation
| , ? ref_script : plutus_script | ||
| ] | ||
| ``` | ||
| TODO: can we use a more generic type that allows _any_ script in a forwards-compatible way? |
There was a problem hiding this comment.
You could do something like this:
any_script = [ 1, timelock_scipt ] / [ 2, plutus_script ] ; ... / [ n, other_script ]
Our future selves would also be grateful if the tags matched the ones used for the script buckets in either the auxiliary data or the witness sets (which don't match each other 😓)
There was a problem hiding this comment.
Why not just allow attaching arbitrary data to the UTXO? This would allow it to be used for other purposes too, besides scripts. It does overlap a bit with having inline datums, but I think it's important to consider that the datum might also contain information necessary only for consuming it, and not relevant for the referencer. By having this extended to arbitrary data, we could choose to ignore the datum (which would contain irrelevant information) and only use the information we are interested in.
There was a problem hiding this comment.
@redxaxder I'd be very happy to just use a production for a discriminated union of scripts if we wanted to promote one (of the two that we apparently have...). Let me know which one!
@L-as The "data" here is used for a specific purpose in the ledger. Just having the field contain arbitrary data has the same issue as putting it in the datum: we then need rules for when it "counts" as a reference script for the purposes of the ledger, and when it's "just" uninterpreted data. Having a field specifically for this purpose avoids any confusion.
On a practical note, the issue of functional confusion between referencing and spending is much less problematic for datums. You can just put a pair in the datum and only look at the first component when spending and the second when referencing. A trick like this isn't viable for reference scripts themselves, where we need the serialized data to exactly match the expected hash.
There was a problem hiding this comment.
I suggest using the same indices as the auxiliary data, since they're not interleaved with other stuff
There was a problem hiding this comment.
@michaelpj I don't see why you'd need to know when to "count" it as a reference script or uninterpreted data. Were it arbitrary data, to get the data for a hash specified in a transaction, you would in addition to the attached scripts, txInfoData, etc., also check the "reference data" in each input and output. Perhaps there's an issue with this i'm missing?
There was a problem hiding this comment.
What's the type of the extra_txout_data field?
Is it Data? Well, scripts are an incompatible format with Data, so we'd have to e.g. parse the Data and look for a nested bytestring or something.
Is it just a raw ByteString? Now we have an inelegant situation where we just try to match it up with various kinds of thing, and use that to decide how to deserialise it. Which is quite tricky and also not how we treat any other part of the transaction.
I think you're suggesting the latter?
There was a problem hiding this comment.
A raw bytestring would also be much less useful if the output contains data that isn't supposed to be a hash pre-image for something in the transaction, and just contains extra data you might want to look at. Scripts would then have no easy way of interpreting it (unless we provided parseData :: ByteString -> Data, which I'd rather not).
There was a problem hiding this comment.
Using elements on a list of flexible size has always created problems down the road for me. You always need to check length size and access context by the index, even if you use a data structure to parse.
I would already consider for the third entry to be either the datumhash or a map where I can name the content, either datumhash, the inline datum, a timelockscript, a plutus script, and whatever else might come in the future. This lets you put any data, and if you need have a clear way to implement a data parser as the key on the map would specify the parser.
| , ? ref_script : plutus_script | ||
| ] | ||
| ``` | ||
| TODO: can we use a more generic type that allows _any_ script in a forwards-compatible way? |
There was a problem hiding this comment.
Why not just allow attaching arbitrary data to the UTXO? This would allow it to be used for other purposes too, besides scripts. It does overlap a bit with having inline datums, but I think it's important to consider that the datum might also contain information necessary only for consuming it, and not relevant for the referencer. By having this extended to arbitrary data, we could choose to ignore the datum (which would contain irrelevant information) and only use the information we are interested in.
|
I like the idea. I'm not sold on read-only inputs however. I propose an alternative here which I believe would have similar power but allow us to keep the affine can only be read once nature of UTXOs #159 (comment). |
|
As a dApp builder, I'm super excited about these changes. I've also spoken to a lot of other dApp developers who are equally eager. There's one really useful property that this (and all other proposals) could have, and I'm not equipped to evaluate if this is true of the current proposals or not, so I'll just lay it at your feet: the ability to construct transactions with existing scripts that take advantage of these features. Lets take script compression for example: if the script compression is applied after the script hash is taken (and conversely, the script hash is checked after decompressing), then existing scripts on chain can automatically get huge benefit from script compression. But if the script hash is of the compressed script, then it requires you to deploy different scripts on chain to take advantage. Similarly with this proposal, if it's possible to take a UTXO created today locked by a script, and then once this is released, publish the script as an inline datum, and then spend the previously created UTXO, this is super valuable. It allows existing protocols to seamlessly upgrade to take advantage of these features without deploying new scripts. Deploying new scripts is often extremely painful if not impossible for existing dApps. It sounds like CIP-31,32,33 have this property, but calling it out as an explicit goal would be valuable, I believe. |
It won't be an inline datum per se, but yes, this should just be a drop-in thing that you can use with existing scripts and script outputs. |
A UTXO created today stores the hash of the script that locks it. If you want to spend it today, you need to include the script source in the transaction witnesses. As I understand it, if you want to spend the UTXO after CIP 33 is implemented, you'll be able to either provide the script source in the witnesses (as before) or reference-input a UTXO that contains the script source in its "reference script" field. In other words, you'll only need to update your off-chain dApp code to take advantage of CIP 33. Per CIP 33:
|
|
@GeorgeFlerovsky that's what I thought but I didn't want to assume. 😅 |
|
I think this CIP is ready to be merged as Draft. I will revisit it and update to Active once the implementation has progressed and the CDDL is nailed down. |
| ## Abstract | ||
|
|
||
| We propose to allow scripts ("reference scripts") to be attached to outputs, and to allow reference scripts to be used to satisfy script requirements during validation, rather than requiring the spending transaction to do so. | ||
| This will allow transactions using common scripts to be much smaller. |
There was a problem hiding this comment.
Sorry for probably stupid question or if I missed this point in the discussion. Do you think that this idea could be extended to something like content addressable imports in plutus core - something similar to dhall imports or unison imports or nix artifact build and caching strategy?
This way we could possibly build an ecosystem of libraries on the chain which would be "cheap" to use.
There was a problem hiding this comment.
No, that's drastically different and more difficult. This proposal is entirely external to the language: it just treats scripts as black boxes and lets you get those boxes from other places. Something that actually intruded into the language itself would be massively more complicated.
|
After discussing with the ledger team, there's a strong desire to be as conservative a possible about converting the script context for old language versions, to avoid getting us into a situation where scripts make incorrect assumptions because we omit information rather than failing. It's therefore likely that we're going to change this CIP and CIP-31 to instead not allow V1 scripts to be spent in transactions with new features (as is already in CIP-32), so we never have to provide partial information to scripts. That does mean that this feature won't be usable with old scripts. I think given that we're hoping that V2 will be quite attractive this won't be too much of a problem. |
|
@michaelpj can we schedule some time to discuss this? |
|
This is something that could IMHO be discussed with the CDA, but intuitively, I honestly don't disagree with the decision. There are many unknowns, and just making old scripts noninteroperable with new features is also a solution. After thinking about it a bit, i think that for the majority of ledger changes that would necessitate changes to how scripts see the context, there is very rarely a good solution for exposing that to older scripts. |
@michaelpj I never did get time to chat with you about this, so let me elaborate on why this is concerning to us (and sorry for it taking a while to provide this context, this fell off my radar while I was waiting for a response). There are transaction flows that we currently cannot construct because we anticipated being able to rely on script references to reduce transaction size (based on our conversations in this thread). It's not because of a single script size, but because of the combination of scripts involved. I don't mind if the semantics of invoking the script remain the same. For example, if old scripts are unable to use normal transaction inputs with inline datums, and are unable to see reference inputs, that doesn't concern me. But, it's fairly critical to us that we be able to satisfy "Here is the script bytecode that hashes to this script address" witness criteria via a script reference. This criteria is checked by the node, not by the script, so to my mind this should be possible without interfering with the script context and putting existing scripts at risk. This would allow us to reduce the size of the transactions involving already-deployed scripts. It's not clear from your description if this will be the case. Without this property, we will be unable to upgrade our protocol until transaction sizes reach nearly 18kb (which seems far off). This would be extremely painful for us. Can you help clarify if the scenario I described above will be supported? Happy to hop on a call if you want more context. |
That's the alternative behaviour: just omit the information. Our argument is that silently omitting information is always risky, because it presents an inaccurate picture of the transaction to the script, and this can lead to subtle bugs. So we should adopt a principle of never silently omitting information: better to do the thing that is definitely correct. But if we want this to be a principle, we should apply it consistently.
The crux of the matter is not that it would change the behaviour of existing scripts, but that the script would be shown the wrong thing.
If what you say is true, then this proposal will indeed not help you out of that particular hole. |
This CIP proposes adding "reference scripts" which allow satisfying script witnessing requirements using scripts attached to outputs rather than scripts provided by the spending transaction.