Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redefine opcode prices and make them adjustable #1875

Closed
roman-khimov opened this issue Aug 25, 2020 · 26 comments · Fixed by #2045
Closed

Redefine opcode prices and make them adjustable #1875

roman-khimov opened this issue Aug 25, 2020 · 26 comments · Fixed by #2045
Labels
Discussion Initial issue state - proposed but not yet accepted
Milestone

Comments

@roman-khimov
Copy link
Contributor

Summary or problem description
We have a number of issues recently around our opcode pricing (#1862, #1863, and not so recently in #1433), so there seems to be a problem of defining some fair reasonable price for various opcodes.

Do you have any solution you want to propose?
It comes in two parts, one for fairness and another one for reasonableness.

The fairness part is that instead of defining opcode prices in GAS directly let's first define execution time relations between different opcodes. Let's call it a coefficient. We can do that using the simplest instruction possible as a reference --- NOP. So executing a NOP takes some time and one NOP is our basic execution unit which has a coefficient of one.

How to get a coefficient for PUSH1, for example? Execute it and compare its execution time with NOP execution time. Maybe it's about the same. Maybe it's 2, 3, 5 times more. But you can get a coefficient for it, say 2. Then do the same for PUSHINT8, then for INC, then ADD, etc. Obviously it wouldn't be easy as to ADD things we need to PUSH them first and we need to test EQUAL for various data lengths and some control flow instructions are even more involved, but it can be done, it just takes some time.

To really nail this fairness part we should use two full independent Neo 3 VM implementations, one neo-vm itself and another neo-go. They have a lot of similarities, but they also have a number of differences, so if we're to define a set of tests in VM code (like neo-vm unit tests) we can run them in both VMs and find some average values.

Of course these coefficients may change in the future as VM implementations evolve. But in practice I think they will be stable enough for a year of two. These coefficients can thus be hardcoded in a (versioned) VM. Now to the reasonableness.

The reasonableness part is actually almost the same as was proposed in #1433 (comment) (that even had an implementation in #1453), but with a little twist. As we have now opcode coefficients defined we only need to set the base NOP price in GAS and everything else would be calculated automatically.

That value IMO should be a part of Policy contract, because in practice everyone would be recalculating this price to some USD and comparing with other options. And GAS/USD ratio can jump in hardly predictable ways (that's why I think "1 GAS per second" policy is not enough, there are a lot of prices for 1 second of execution and they're way lower than dollar value of one GAS), so to be reasonable and to stay reasonable this base price needs to be adjustable.

Neo Version

  • Neo 3

Where in the software does this update applies to?

  • Other: Economics of execution
@roman-khimov roman-khimov added the Discussion Initial issue state - proposed but not yet accepted label Aug 25, 2020
@EdgeDLT
Copy link

EdgeDLT commented Aug 25, 2020

Agree completely! Same should be true for syscalls, oracle calls... Maybe in a decade it will have reached a price equilibrium, but for now at least GAS cannot be treated like it's a stablecoin.

It is volatile, and that volatility needs to be addressed. Coefficients and a modifiable base price seems like a good way to do it, definitely an improvement over a simple multiplier.

@Qiao-Jin
Copy link
Contributor

Qiao-Jin commented Aug 26, 2020

I think there could be another advantage of this feature. If most smart contract users think the opcode price too high to continue with Neo smart contract, we may lower the fee to attract more people. If there are too many smart contracts, do the opposite.

However there could be some problems: users might worry about the dynamic price as they are not certain about how much budget they will need even in terms of GAS, which is very inconvenient for stable users. Moreover, modifying opcode price by committee members with no concrete rule sounds like delegate cartels.

@roman-khimov
Copy link
Contributor Author

modifying opcode price by committee members with no concrete rule

This is a valid concern, but to me that just reminds that we need more clear governance rules in general, what questions are solved by the committee, what's the procedure, what are the limits, how conflicts are resolved, things like that.

Like this particular question, it can be solved with a simple policy, for example "committee can change base opcode price no more often than once a year or if GAS price changes five-fold since the previous change relative to XDR". Maybe with some limit to the change step. And something like this IMO will give enough predictability for end users.

@erikzhang
Copy link
Member

The fairness part is that instead of defining opcode prices in GAS directly let's first define execution time relations between different opcodes.

The current model is like this.

@roman-khimov
Copy link
Contributor Author

The current model is like this.

Was it really measured? Take INC, ADD, MUL and SHL as an example. Is there a three-fold difference between SHL and INC or 1.5 diff between ADD and MUL? Then compare it to my favorite pet peeve which is ECDsaVerifyPrice, it's some 3333 times bigger than MUL. Is this ratio correct? Maybe. But there is no data behind it. I think it'd be nice to have something more solid backing these relations.

@erikzhang
Copy link
Member

I used to make a tool to test the CPU time consumed by each instruction. But it is based on the early NEO3 code. I will rewrite it based on the latest NEO3 code in a while.

@Qiao-Jin
Copy link
Contributor

Qiao-Jin commented Sep 2, 2020

modifying opcode price by committee members with no concrete rule

This is a valid concern, but to me that just reminds that we need more clear governance rules in general, what questions are solved by the committee, what's the procedure, what are the limits, how conflicts are resolved, things like that.

Like this particular question, it can be solved with a simple policy, for example "committee can change base opcode price no more often than once a year or if GAS price changes five-fold since the previous change relative to XDR". Maybe with some limit to the change step. And something like this IMO will give enough predictability for end users.

Cannot agree. Limiting times that committee members can change opcodes price still doesn't solve the problem. There is still no rule how to change the price. Suppose you are a common user, how can u trust the network if the price can be changed by will at any second?

Furthermore, there are many leaks in such rules. For example, you suppose committee members can change price at most some times a year. You know what? If committee members change the price to 0, or 100000000 per opcode and use up all possible times to change price, does it mean overall network must suffer such price for one year, even if committee members are shuffled? Yes, this is not likely to happen exactly because such price is very unreasonable, but what if committee members change the price to a seemingly reasonable standard but eventually ruin the network? What will you do at that time, hard folk?

When you put forward such issues you should tink about the practicability of it.

Opcode as well as some interops price should be redefined. However, changing opcodes price by committee is too radical.

@roman-khimov
Copy link
Contributor Author

However, changing opcodes price by committee is too radical.

If we don't trust the committee then what's the point of its existence? And how to solve these tricky questions like opcode prices? The committee can ruin the network already in many ways, that's the power it has. However if we don't give this particular aspect of the power to the committee then what's the protocol for opcode price change? At the moment that would be "someone creates a patch and a bunch of core developers review and merge it", is it any better than the committee? I don't think so.

I think if we're to have the committee it should run the network, it should have all the powers needed and at the same time it should have explicit transparent rules for every question this committee is in charge of (like #1867, for example). And committee decisions should be expressed as signed transactions on the network, again bringing more transparency to the process of any kind of policy change (changes in the node's code are exactly the opposite of transparent).

This particular question, think of it as of central bank decision on base interest rate. There is some kind of committee there too. And it can ruin country's economy with its decision. Yet it somehow makes a decision and usually it works.

@Qiao-Jin
Copy link
Contributor

Qiao-Jin commented Sep 2, 2020

If we don't trust the committee then what's the point of its existence

Trust relays in how much rights granted. Too much rights weaken trust.

And how to solve these tricky questions like opcode prices

So you think dynamic price by committee is the only way? Why don't set up a reasonable price list at the start after careful calculation?

The committee can ruin the network already in many ways

But previous members should not impact future network env all right? Plz note that this is pointing out why "committee can change base opcode price no more often than once a year or if GAS price changes five-fold since the previous change relative to XDR" is unreasonable. Before puting forward a funtional change you should think carefully about what will be the impact rather than at will.

I think if we're to have the committee it should run the network

When did I deny the existence of committee? What I doubt is HOW MUCH POWER SHOULD BE GRANTED EXACTLY TO THEM, to prevent abusing and destroying environment.

This particular question, think of it as of central bank decision on base interest rate. There is some kind of committee there too. And it can ruin country's economy with its decision. Yet it somehow makes a decision and usually it works.

Then why do we work on NEO if a centralized authority works with no problem??

@roman-khimov
Copy link
Contributor Author

Why don't set up a reasonable price list at the start after careful calculation?

This particular question has been discussed already, that price is being expressed in GAS and I still can't buy Coca-Cola for GAS at the nearest store. The price of execution should be economically viable for dApps to use the blockchain. That price would be checked against not only blockchain of the day, but also things like AWS Lambda which is expressed in USD. Ask yourself --- would Neo 2 survive without gas_free when GAS/USD ratio jumped ten-fold? It was fun for coin speculations for sure, but it's not for a network that tries to provide some real service at reasonable price.

@Qiao-Jin
Copy link
Contributor

Qiao-Jin commented Sep 2, 2020

The price of execution should be economically viable for dApps to use the blockchain

From your reason, then this price should be bound to exchange price rather than set by committee. Ask yourself, would you trust a committee or direct bounding to exchange price for stabilization?

would Neo 2 survive without gas_free when GAS/USD ratio jumped ten-fold

So what? It only shows that pricing itself is unreasonable. It doesn't show that we need a dynamic opcode price, or dynamic pricing by committee.

@EdgeDLT
Copy link

EdgeDLT commented Sep 2, 2020

So what? It only shows that pricing itself is unreasonable. It doesn't show that we need a dynamic opcode price, or dynamic pricing by committee.

Can't agree with this. It's not acceptable that one day an action costs $1, and the next week the same thing might cost $50 because GAS has gone up 50x. At some points it was possible to deploy a contract with storage on Neo2 for $500, at other times the price was $30,000.

Ethereum too is currently killing itself over ludicrous fees. It is true that this is mostly a consequence of congestion rather than price volatility, but the result is the same, an unusable network.

So you think dynamic price by committee is the only way? Why don't set up a reasonable price list at the start after careful calculation?

It is fine to start with a reasonable price list, but that price list doesn't stay reasonable for long because it is denominated in GAS, which will change in price. We need a way to scale opcode fees in accordance with GAS price so that no matter what the market price of GAS is, the cost to use the network is the same reasonable sum as before.

@diskooooo
Copy link
Contributor

So what? It only shows that pricing itself is unreasonable. It doesn't show that we need a dynamic opcode price, or dynamic pricing by committee.

Can't agree with this. It's not acceptable that one day an action costs $1, and the next week the same thing might cost $50 because GAS has gone up 50x. At some points it was possible to deploy a contract with storage on Neo2 for $500, at other times the price was $30,000.

Ethereum too is currently killing itself over ludicrous fees. It is true that this is mostly a consequence of congestion rather than price volatility, but the result is the same, an unusable network.

So you think dynamic price by committee is the only way? Why don't set up a reasonable price list at the start after careful calculation?

It is fine to start with a reasonable price list, but that price list doesn't stay reasonable for long because it is denominated in GAS, which will change in price. We need a way to scale opcode fees in accordance with GAS price so that no matter what the market price of GAS is, the cost to use the network is the same reasonable sum as before.

100% agreed. Again: GAS is volatile and traded freely in the markets (thus prone to speculation); it cannot under any circumstances be expected that GAS has a stable value.

@vncoelho
Copy link
Member

vncoelho commented Sep 2, 2020

Hi @EdgeDLT and @diskooooo,
Glad to hear from you.

From my perspective, we need to be careful even when speculating about GAS price.
Different than other blockchain projects, NEO has GAS generated in a way to provide Computational Capabilities for NEO holders.
In the same way it will now reward the Committee (or it can also be seen as providing them computational resources). But this rewards are "kind of minimal" because it is being projected to be 5% divided by 21 members.

There are projects that provide a parcel of the block for holders of the token (thus, in every block, holders have slots for transactions).

I believe that GAS can be, yes, more stable, it all depends on the blockchain use and a balance between demand and offer of those interested in using it.

1 GAS for 1 second does not looked bad to me when I first read @erikzhang talking about this. I believe that we need to attach it to some metric. Furthermore, it should be adjusted during years long execution.
In the same way that @roman-khimov is emphasizing here:

The fairness part is that instead of defining opcode prices in GAS directly let's first define execution time relations between different opcodes.

@EdgeDLT
Copy link

EdgeDLT commented Sep 2, 2020

Hey @vncoelho. I don't think it's really a case of speculating about GAS price. You can expect that the economic design will eventually lead GAS to stability, but it is not stable today, or tomorrow, and it won't be stable next year either.

1 GAS for 1 second sounds great in principle. Today that means $1.90 for one second, not a great deal even now. What about if 1 GAS is $50, then I should pay $50 for 1 second of compute? It does not make sense to me; we must adjust for volatility.

How about something similar to Justin Drake's proposal on Ethereum for an enshrined price feed? Use the oracle service to get the current GAS value every x blocks and use it to adjust opcode pricing to the market rate for that epoch.

@vncoelho
Copy link
Member

vncoelho commented Sep 2, 2020

I am not sure, @EdgeDLT.
I prefer to make the calculus based on 1 GAS for 1 second and watch the flow. Validators on NEO are reduced, differently than what is being proposed by Ethereum (I am not saying that I am more in favor of one approach than another. What we propose for dBFT 3.0 with double speakers is a great path for scalability of consensus). In this sense, those validators can afford most of the computational costs due to the reputation they win and also incentives.
In addition to what we are discussing, perhaps loosing the minimal free gas is also a big loss to the ecosystem, independently of the price we set for opcodes in GAS.

@roman-khimov
Copy link
Contributor Author

We've created a benchmark for this in neo-go, check out nspcc-dev/neo-go#1493, you can run it locally to check some of these results and there are results from my machine posted there too (100000 iterations for each test, that's the basis for the table below). The benchmark creates a VM instance to run just one opcode that we're targeting preparing any environment for it when needed. An instruction is run via Step() method as if it's a regular script processing (including instruction fetching).

Here is the table based on neo-go testing results and considering numbers outlined in #2004 for neo-vm. All of these are calculated against the golden standard of NOP for every case, absolute numbers mean nothing here, we're concentrating on relative performance for now. "Current" is current relation derived from opcode prices and "Optimal" is just a bit shorter word for "Recommended", that's what I'd propose to use based on data we have.

For some opcodes we have very stable results for both implementations, for some it's not that easy, but we can discuss each opcode and tests for it additionaly if needed. What's more interesting is that NOP-based coefficients seem to work fine in that they provide enough granularity for this and allow to create some meaningful table.

I should notice though that this subject is a bit more complex than these numbers because of implicit type conversions, reference counting and huge difference between regular expected usages and edge cases. All of these distort the picture considerably, but we have to make some choice anyway, so I've added some per-instruction comments.

We also have patterns based on our instruction set that need to be accounted for. Lots of instructions have short and long (_L) versions, they differ in size, but their execution time tends to be about the same, so we don't want to differentiate their coefficients (contract size cost should motivate people here). Instructions operating on two arguments cost a bit more than operating on one argument. Using parameters can also cost a bit more.

Instruction Current neo-go neo-vm Optimal Comment
PUSHINT8 1 3.8-4.0 5.0 4 Shorter integers are a bit cheaper.
PUSHINT16 1 4.0-4.1 N/A 4
PUSHINT32 1 4.0-4.3 N/A 4
PUSHINT64 1 4.1-4.4 N/A 4
PUSHINT128 4 4.5-4.7 N/A 5 And longer ones can pay a bit more.
PUSHINT256 4 4.6-5.2 5.0 5
PUSHA 4 11-948 5.6 20 Pointers are not that easy, though in practice neo-go shouldn't hit this edge case.
PUSHNULL 1 2.4 2.2 2
PUSHDATA1 6 3.7-3.9 4.6 6
PUSHDATA2 433 3.9-42 4.6 40
PUSHDATA4 3666 72-1014 4.6 1000 Pushing a lot of data should be demotivated a bit. And copying this data costs something.
PUSHM1..16 1 3.5-3.9 5.0 4
NOP 1 1 1 1 The standard.
JMP 2.3 1.9 1.5 2
JMPL 2.3 1.9 1.3 2
JMPIF 2.3 2.0 2.1 2
JMPIFL 2.3 2.0 2.1 2
JMPIFNOT 2.3 2.0-2.2 2.2 2
JMPIFNOTL 2.3 2.0-2.3 2.3 2
JMPEQ 2.3 2.2-5.8 3.2 4 Still cheaper than NUMEQUAL+JMP
JMPEQL 2.3 2.2-5.6 3.2 4
JMPNE 2.3 2.2-6.1 3.3 4
JMPNEL 2.3 2.3-5.6 3.3 4
JMPGT 2.3 2.2-5.5 N/A 4
JMPGTL 2.3 2.2-5.6 N/A 4
JMPGE 2.3 2.2-5.7 N/A 4
JMPGEL 2.3 2.2-5.6 N/A 4
JMPLT 2.3 2.2-5.6 N/A 4
JMPLTL 2.3 2.2-5.7 N/A 4
JMPLE 2.3 2.2-6.3 N/A 4
JMPLEL 2.3 2.2-5.6 N/A 4
CALL 733 5.5 1.8 10 It creates new execution context, so making it equal to JMP is not fair.
CALLL 733 5.5 1.7 10
CALLA 733 13-952 8.4 20 Going through pointers can be more involved, again shouldn't happen in practice for neo-go.
ABORT 1 6.2 N/A 0 It's an über-RET.
ASSERT 1 1.4-6.0 1.9 2
THROW 733 2.8-241 14 100 There are edge cases, exceptions are not easy.
TRY 3.3 4.4 5.6 5
TRYL 3.3 4.5 N/A 5
ENDTRY 3.3 2.3-2.7 1.6 3
ENDTRYL 3.3 2.2-2.3 N/A 3
ENDFINALLY 3.3 3.2-241 14 100 It can contain unhandled exception and incur the same overhead as THROW.
RET 0 1.9 0.8 0
SYSCALL 0 2.2 N/A 2 Not strictly necessary, syscalls are supposed to add some GAS themselves, but we can account for some generic overhead still.
DEPTH 2 4.6-6.3 4.7 5
DROP 2 1.4-5.4 1.9 2
NIP 2 1.4-1.8 2.2 2
XDROP 13 1.2-51 1019 100
CLEAR 13 1.6-2.6 552 50
DUP 2 3.0-77 2.1 3 Creating element should be a bit more costly than shuffling elements around (like SWAP/ROT/etc)
OVER 2 3.4-77 2.3 3
PICK 2 2.9-69 2.5 10 Ability to go at any depth should be paid for.
TUCK 2 2.1-38 2.6 4
SWAP 2 1.2-1.3 2.7 2
ROT 2 1.1-1.2 2.7 2
ROLL 13 1.3-17 3.7 15 Rolling all the stack should be paid for.
REVERSE3 2 1.2 1.4 2
REVERSE4 2 1.2 1.5 2
REVERSEN 13 1.3-26 60 40
INITSSLOT 13 2.6-7.3 117 50
INITSLOT 27 3.8-41 444 100 Twice INITSSLOT.
LDSFLD0..6 2 2.2 N/A 2
LDSFLD 2 2.8 3.3 3 Maybe not worth differentiating from LDSFLD0, but it does cost a bit more.
STSFLD0..6 2 1.2 N/A 2
STSFLD 2 1.9 3.6 3
LDLOC0..6 2 2.2 N/A 2
LDLOC 2 2.9 3.2 3
STLOC0..6 2 1.2 N/A 2
STLOC 2 2.0 3.5 3
LDARG0..6 2 2.2 N/A 2
LDARG 2 3.0 3.3 3
STARG0..6 2 1.2 N/A 2
STARG 2 2.0 3.5 3
NEWBUFFER 2667 3.8-472 276 300
MEMCPY 2667 4.5-462 4156 2000
CAT 2667 4.7-713 21764 4000 Twice MEMCPY.
SUBSTR 2667 8.9-1008 21365 4000
LEFT 2667 7.8-1022 21166 4000
RIGHT 2667 9.4-1008 21887 4000
INVERT 3.3 2.8-5.3 8.2 6 One argument.
AND 6.7 4.1-9.0 8.1 9 Two arguments.
OR 6.7 4.0-8.0 8.2 9
XOR 6.7 4.1-9.2 8.2 9
EQUAL 6.7 3.1-16 3.4-245 50
NOTEQUAL 6.7 2.7-15 N/A 50
SIGN 3.3 3.4-6.1 7.1 6
ABS 3.3 2.8-5.3 7.2 6
NEGATE 3.3 2.8-7.2 6.2 6
INC 3.3 4.4-6.8 7.6 7
DEC 3.3 4.5-6.9 7.6 7
ADD 6.7 4.0-8.3 9.2 9
SUB 6.7 4.0-8.2 9.1 9
MUL 10 3.8-6.7 9.8 9
DIV 10 3.9-8.3 9.7 9
MOD 10 4.7-8.6 8.6 9
SHL 10 4.6-6.3 31 9
SHR 10 3.9-6.6 48 9
NOT 3.3 2.8-2.9 2.5 3
BOOLAND 6.7 3.0-3.1 3.3 4 Operating on two arguments, a bit harder than NOT.
BOOLOR 6.7 2.9-3.3 3.2 4
NZ 3.3 2.8-4.9 2.6 3
NUMEQUAL 6.7 3.1-6.3 3.5 4
NUMNOTEQUAL 6.7 2.9-6.4 3.5 4
LT 6.7 3.4-6.6 3.5 4
LTE 6.7 3.1-6.4 3.5 4
GT 6.7 2.9-6.2 3.6 4
GTE 6.7 3.0-6.3 3.5 4
MIN 6.7 3.2-6.8 10 9
MAX 6.7 3.2-7.3 8.7 9
WITHIN 6.7 2.9-7.4 4.4 9
PACK 233 6.3-110 3874 500 Like NEWARRAY.
UNPACK 233 6.7-434 1656 500
NEWARRAY0 13 5.1 17 10
NEWARRAY 500 6.0-64 1896 500
NEWARRAYT 500 6.0-472 1926 800
NEWSTRUCT0 13 4.9 18 10
NEWSTRUCT 500 5.6-66 1963 500
NEWMAP 6.7 4.6 24 20
SIZE 5 4.9-34 20 20
HASKEY 9000 6.0-99 40 50 HASKEY/PICKITEM/REMOVE are quite similar in their real cost.
KEYS 17 583 32 200 Actually expected to be similar to VALUES.
VALUES 233 5.2-108 42945 400
PICKITEM 9000 3.6-103 25 50
APPEND 500 3.8-154 34785 500
SETITEM 9000 4.4-166 31810 500
REVERSEITEMS 17 36-4766 15337 5000
REMOVE 17 28-100 25 50
CLEARITEMS 13 47-54 9202 500
ISNULL 2 2.8 4.6 3
ISTYPE 2 3.1 8.3 6
CONVERT 2667 3.3-85 3.8 50

@Qiao-Jin
Copy link
Contributor

In some cases you forgot considering Struct.Clone().

@ixje
Copy link
Contributor

ixje commented Oct 16, 2020

Do you have, or can you create, a cost difference for typical smart contract use-cases? From eyeballing the table it feels like with the recommended prices execution will be more expensive.

@roman-khimov
Copy link
Contributor Author

I think we can replay preview3 testnet transactions to see how proposed coefficients affect GAS cost of real executions (probably next week). In general, yes, this makes some basic opcodes cost more, but at the same time making things like PICKITEM way cheaper, so the end result depends on usage patterns. But then we're talking fair coefficients here at the moment, while the base NOP price can be as low as 1 Satoshi.

@ixje
Copy link
Contributor

ixje commented Oct 16, 2020

I'd be interested to see the results of that exercise.

@roman-khimov
Copy link
Contributor Author

If we're to take preview3 testnet at the height of 331056 with just 186 transactions inside, we can get the following picture.

Сalculating overall GAS consumption for all executions gives us a cost of 241.7818188 GAS, with proposed changes that would be 241.1211635 or 99.73%, pretty close, but still a little less. Overall there are 9893 instructions executed with the following distribution:

   4051 PUSHDATA1
    519 SYSCALL
    479 NOP
    297 LDARG0
    263 DUP
    247 SWAP
    206 PACK
    196 RET
    185 PUSH0
    177 INITSLOT
    154 LDLOC0
    147 CONVERT
    144 CALL_L
    143 PUSH1
    142 STLOC0
    136 LDLOC1
    135 LDARG1
    130 ISNULL
    120 PICKITEM
    118 PUSH3
     96 ASSERT
     94 JMPIFNOT_L
     88 SETITEM
     80 STLOC1
     80 JMPIFNOT
     76 JMP_L
     71 REVERSE3
     68 JMP
     65 PUSH2
     64 SIZE
     60 PUSHINT64
     60 LDLOC2
     56 ROT
     56 JMPIF
     56 APPEND
     52 LDARG2
     50 NUMNOTEQUAL
     43 LT
     42 STLOC2
     41 PUSHINT32
     40 JMPIF_L
     39 CAT
     36 NEWSTRUCT
     35 PUSHDATA2
     31 NEWARRAY0
     30 NEWARRAY
     30 ABS
     29 STSFLD0
     29 INITSSLOT
     28 INC
     27 STSFLD
     26 STSFLD1
     23 PUSHINT8
     19 LDSFLD1
     12 DROP
     11 NUMEQUAL
     11 ADD
     10 STLOC3
     10 NOT
     10 LDLOC3
      9 STSFLD6
      9 STSFLD5
      9 STSFLD4
      9 STSFLD3
      9 STSFLD2
      7 PUSHINT16
      7 LTE
      6 STLOC4
      6 LDSFLD0
      6 LDLOC4
      5 SUB
      5 STLOC5
      5 LDLOC5
      4 PUSHNULL
      4 EQUAL
      4 CALL
      3 MOD
      2 STARG1
      2 PUSH9
      2 PUSH7
      2 MUL
      1 PUSH5
      1 PUSH4
      1 PUSH14
      1 GT
      1 DIV

141 of 186 transactions did really burn a little more of GAS for their execution with the biggest relative increase for 173dcbc4a88995a0cf7bdd006923d148f787f76ca75621dc4c440ca6d9afbc73 that burned 0 GAS and now suddenly required 60×10⁻⁸. It's a very special transaction that does exactly this:

0        SYSCALL    Neo.Native.Deploy (123e7fe8)    <<

So I guess it's not very representative. As for other 140 transactions, the biggest relative increase happened for 1dd1dced6a2dace4346bfa0a8aeef380e7f3153a523d868253dad1a24d17f36d which paid 1008450×10⁻⁸ and after the proposed changes would have to pay 1017700×10⁻⁸ which is a 0.917% increase, it's a bit more useful with the following code inside:

0        PUSH0                                                    <<
1        PACK                                                     
2        PUSHDATA1    73796d626f6c ("symbol")                     
10       PUSHDATA1    75a50886ca922a9d4d0f1bfe85e30c44927a5979    
32       SYSCALL      System.Contract.Call (627d5b52)             

So from 140 transactions we have 14 that increased their cost for more than 0.1% and others are even less than that. And all of these 140 are heavily dominated by some syscall in their cost, so I don't think they're really interesting, they just don't execute enough of VM code. What's interesting is that we have 45 transactions that actually cost less using the alternative pricing.

The winners here are 2b5592173c587673e076337f6bc500f17362ee3f39a2d4a6b7c69f57162490d9, 89fa34dd37ebe06bb2c3cc19d014c3bc37e545e07bca1d5d8d8fe86ae61886c8, 46ccaabc95617de3adb2747419a45187cdef5342f30fdbdb1985a4e0d4884c35 and 5538fc911c3f4323bf55eeb47b44ca7320bdd6a0c14f6a6f01930b8468878c6f. All of these transactions use 13799010×10⁻⁸ GAS with current pricing and 7221070×10⁻⁸ with proposed scheme which means that they're actually 47.7% cheaper now.

They're quite similar, doing things like this:

0        PUSHINT64    20000000000 (00c817a804000000)              <<
9        PUSHDATA1    bb8e1fb6aebb8928a41c08fa4f6145a929601c23    
31       PUSHDATA1    a497e2ce7a554207fc1798d39b9bb3baba2cb0dc    
53       PUSH3                                                    
54       PACK                                                     
55       PUSHDATA1    7472616e73666572 ("transfer")               
65       PUSHDATA1    8b87611a7e673b1c13ca90f39aa0cc36e330c255    
87       SYSCALL      System.Contract.Call (627d5b52)             

So there is some non-native NEP-5 contract behind this that runs some real opcodes in VM.

Other than that, from 45 transactions 19 decreased their cost by less than 0.2%, from quick glance those are mostly contract-deploying transactions that benefited a little from cheaper PUSHDATA2. Then there are 11 transactions with 2-5% gain, 5 that decreased by 5-10%, 3 in 14-19% range, 1 with almost 25%, 2 with 36-39% and 4 champions mentioned above. What's interesting is that I see more of real code executed there, for real contracts these changes actually tend to decrease GAS cost even with the current NOP price of 30×10⁻⁸.

BTW, speaking of real contracts with real opcodes, we have NeoFS contract there too (written in Go and compiled with neo-go, of course) and its invocations in 33ce82de7487063ca8cf278949268f5e50314e2d375ef8b3f479d5dd1e782041 and b0f53ef871b426a26ea4a80f9a47cc78bec64d7125ec899ef4de82701a37bba6 decreased in cost by 14.2% and 2.6% respectively.

So, I think that:

  • NOP-relative scheme is absolutely viable
  • coefficients outlined above tend to decrease actual GAS cost of execution for real contracts

We can use this table as the basis for the new pricing scheme, maybe adjusting some particular opcodes based on additional discussions (I can do some recalculations relatively easy now).

@Qiao-Jin
Copy link
Contributor

How did u count time consuming in ur test? Did u count overall vm running time or time used by specified Opcode?

@Qiao-Jin
Copy link
Contributor

Qiao-Jin commented Oct 26, 2020

absolute numbers mean nothing here, we're concentrating on relative performance for now

You fellows may be concentrating on relative performance, for now, but it's unwise to neglect real time consuming, as it's important to know how much time each Opcode can cost at most. Even though time cost varies across different machines, a rough value is needed to see whether some opcodes offend the "at most 1 second per gas consumed" rule. And yes, if you are interested in relative performance, feel free to calculate by division by yourself:)

@diskooooo
Copy link
Contributor

I'm seeing a Fee Ratio change submitted in #2032.

Surely a step in the right direction, but why not do this truly dynamically by using an oracle request to get the real time $ cost of GAS? It's the only way to ensure $ equivalents for OPcode fees are somewhat static in a highly volatile market, isn't it?

When doing storage or computing on AWS, you also expect time spent or bytes used to have the same $ cost always...

@Tommo-L
Copy link
Contributor

Tommo-L commented Oct 29, 2020

Surely a step in the right direction, but why not do this truly dynamically by using an oracle request to get the real time $ cost of GAS? It's the only way to ensure $ equivalents for OPcode fees are somewhat static in a highly volatile market, isn't it?

I think it's not a good idea that our protocol need the outside GAS/NEO price. If we want use the price, we'd better do it in contract level, not neo-core.

roman-khimov added a commit to roman-khimov/neo that referenced this issue Nov 3, 2020
This defines a new Policy contract setting, BaseExecFee which contains a price
of basic VM execution unit --- NOP instruction. It then redefines all opcode
prices as coefficients applied to that base fee.

The following invariants hold for this:
 * transaction fees are expressed in GAS
 * BaseExecFee is expressed in GAS
 * opcode price is a coefficient now
 * syscall price is expressed in GAS
 * ApplicationEngine.AddGas() also accepts GAS values, so it's a caller's
   responsibility to calculate that in whatever fashion he likes.

Caveats:
 * while coefficients are based on table from neo-project#1875 (which is built on
   instruction execution time relations for two complete VM implementations),
   some modifications were applied:
   - it's impossible for SYSCALL to have non-0 cost now (tests just hang)
   - all slot operations have the same price
 * block and consensus payloads are adjusted to use BaseExecFee, but probably
   ECDsaVerifyPrice is still a constant
 * it's not really tested other than unit tests

In general, this allows to define any execution cost in NOPs and then use
BaseExecFee to get the price. The same principle could be applied to storage
pricing based on FeePerByte (StoragePrice could just be `100 * FeePerByte`),
but that's a bit different topic.

Closes neo-project#1875.
roman-khimov added a commit to roman-khimov/neo that referenced this issue Nov 3, 2020
This defines a new Policy contract setting, BaseExecFee which contains a price
of basic VM execution unit --- NOP instruction. It then redefines all opcode
prices as coefficients applied to that base fee.

The following invariants hold for this:
 * transaction fees are expressed in GAS
 * BaseExecFee is expressed in GAS
 * opcode price is a coefficient now
 * syscall price is expressed in GAS
 * ApplicationEngine.AddGas() also accepts GAS values, so it's a caller's
   responsibility to calculate that in whatever fashion he likes.

Caveats:
 * while coefficients are based on table from neo-project#1875 (which is built on
   instruction execution time relations for two complete VM implementations),
   some modifications were applied:
   - it's impossible for SYSCALL to have non-0 cost now (tests just hang)
   - all slot operations have the same price
 * block and consensus payloads are adjusted to use BaseExecFee, but probably
   ECDsaVerifyPrice is still a constant
 * it's not really tested other than unit tests

In general, this allows to define any execution cost in NOPs and then use
BaseExecFee to get the price. The same principle could be applied to storage
pricing based on FeePerByte (StoragePrice could just be `100 * FeePerByte`),
but that's a bit different topic.

Closes neo-project#1875.
@shargon shargon mentioned this issue Nov 11, 2020
@erikzhang erikzhang added this to the NEO 3.0 milestone Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion Initial issue state - proposed but not yet accepted
Projects
None yet
8 participants