Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only enable JS runtime limits during execution #5730

Merged

Conversation

achamayou
Copy link
Member

@achamayou achamayou commented Oct 13, 2023

Alternative to #5729 in addressing #5589. Also fixes #5731.

This change removes the interpreter heap, stack and runtime caps outside of JS code execution, which avoids having to handle those situations in the preparation of the runtime and of the response. In those contexts, the limits set by the environment (OS and/or SGX) apply instead, as they do in the rest of the code.

Note that this of course does not solve the issue of leaks in case a C++ exception is thrown in a wrapped function called from JS, this will need separate changes.

Also changed runtime options setting so that governance always has at least the default values available, to avoid getting to an unrecoverable situation where the values are so low that no further governance is possible.

@ghost
Copy link

ghost commented Oct 13, 2023

only_enable_runtime_limits_during_js_exec@77292 aka 20231016.22 vs main ewma over 20 builds from 76888 to 77284

Click to see table

main

build_id build_number pi_basic_mt_virtual_cft^ pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem Commit latency factor tpcc_virtual_cft^ ls_virtual_cft^ tpcc_sgx_cft^ tpcc_sgx_cft_mem pi_ls_virtual_cft^ pi_basic_virtual_cft^ pi_basic_js_virtual_cft^ ls_jwt_virtual_cft^ pi_ls_jwt_virtual_cft^ ls_js_virtual_cft^ ls_sgx_cft^ ls_sgx_cft_mem pi_ls_sgx_cft^ pi_ls_sgx_cft_mem ls_full_js_virtual_cft^ pi_basic_sgx_cft^ pi_basic_sgx_cft_mem ls_js_jwt_virtual_cft^ pi_basic_js_sgx_cft^ pi_basic_js_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem hist_sgx_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
76888 20231004.30 67387.1 27780.7 2.30851e+07 0.810034 17390.9 45776.9 5604.3 8.59996e+07 48301 51883.1 4390.1 17189.2 20048.9 15378.1 13985 1.88908e+07 14115.9 1.05021e+07 14886.4 15556.5 1.46964e+07 10356.8 1434.9 1.25993e+07 6863.79 1.67936e+07 42002.2 7032.1 6.30784e+06 5815.76 1.67936e+07 5751.88 1.67936e+07 3969.63 1.67936e+07 823286 1.18056e+06 8.1555e+06 3.03318e+07
76915 20231004.42 80712.1 27485.2 2.51822e+07 0.774478 17274.8 46033.9 5624.97 8.59996e+07 48315.9 53167.5 4402.4 17273.2 19310.2 17384.9 14042.4 1.88908e+07 14174.9 1.05021e+07 14995.5 15627.4 1.25993e+07 10159.5 1433.2 1.25993e+07 6910.89 1.88908e+07 40358.8 6998.2 6.30784e+06 5784.96 1.67936e+07 5748.49 1.67936e+07 4003.02 1.67936e+07 835408 1.17924e+06 8.12898e+06 3.07817e+07
76922 20231005.1 65412.6 28029.7 2.51822e+07 0.810974 17459.2 45898.2 5626.69 8.59996e+07 48742 54118.6 4456.3 17231.8 20034.9 17323.3 14039 1.88908e+07 14080.5 1.05021e+07 14832.9 15577.1 1.46964e+07 10275.8 1432.3 1.25993e+07 6863.93 1.67936e+07 42574.2 7085.2 6.30784e+06 5810.75 1.67936e+07 5719.35 1.67936e+07 3992.52 1.67936e+07 829315 1.17285e+06 8.14651e+06 3.07836e+07
76933 20231005.5 81465 27882.8 2.30851e+07 0.805698 17274.6 45946.3 5569.11 8.59996e+07 48589.4 54166.2 4463.6 17329.5 19980 17094.5 14005.2 1.88908e+07 14146.2 1.05021e+07 16949.3 15512.2 1.46964e+07 9763.9 1432.2 1.25993e+07 6865.95 1.88908e+07 39270.6 6977.1 6.30784e+06 5765.37 1.67936e+07 5766.41 1.67936e+07 3994.82 1.67936e+07 837034 1.18428e+06 8.10377e+06 3.25923e+07
76941 20231006.1 65485.1 28097.1 2.30851e+07 0.799935 17296.4 45618.5 5581.89 8.59996e+07 48175.1 53457.3 4479 17334.7 19718.8 17075.6 14055.4 1.88908e+07 14129.3 1.05021e+07 14603.4 15654.4 1.25993e+07 9842.12 1434.8 1.25993e+07 6890.62 1.67936e+07 39294.9 6976.6 6.30784e+06 5781.49 1.67936e+07 5744.04 1.67936e+07 4009.45 1.67936e+07 839192 1.18477e+06 8.15264e+06 3.23033e+07
76953 20231006.6 78324.4 27710.2 2.51822e+07 0.797479 16945.5 43917.9 5610.73 8.59996e+07 48958.9 53263.2 4423.9 17245.1 18721.7 17292 14024 1.88908e+07 14122.5 1.05021e+07 14773.2 15585.3 1.25993e+07 9748.83 1444.6 1.25993e+07 7252.76 1.67936e+07 45657 7058.2 6.30784e+06 5820.86 1.67936e+07 5773.73 1.67936e+07 4008.93 1.67936e+07 835912 1.17953e+06 8.07625e+06 2.6895e+07
76967 20231006.11 64719 28032.2 2.30851e+07 0.799426 17341.4 45691.3 5542.39 8.59996e+07 48875.1 54004.5 4469.9 17300.7 19158.2 16934.3 13951.2 1.88908e+07 14077.9 1.05021e+07 14583.1 15519.1 1.25993e+07 9795.4 1421.7 1.25993e+07 6820.04 1.88908e+07 47130.7 6884.1 6.30784e+06 5770.08 1.67936e+07 5760.09 1.88908e+07 4004.62 1.67936e+07 835081 1.17659e+06 8.17304e+06 3.09852e+07
76976 20231007.2 69696.2 27833 2.51822e+07 0.797827 17276.4 45702.6 5640.74 8.59996e+07 48619 53436.8 4409.7 17184.3 19733.2 17169.5 14032.5 1.88908e+07 14190 1.05021e+07 14871.6 15631.1 1.46964e+07 9779.02 1433.9 1.25993e+07 7287.64 1.67936e+07 45049.2 7044.3 6.30784e+06 5786.37 1.67936e+07 5747.56 1.67936e+07 4006.09 1.67936e+07 830418 1.18267e+06 8.17291e+06 3.07891e+07
76982 20231009.2 81026 27935.7 2.51822e+07 0.792855 17231.2 45523.7 5580.01 8.59996e+07 48581 54131 4253.1 17397.5 19156 17135.7 13963.1 1.88908e+07 14141.8 1.05021e+07 16869.3 15380.7 1.25993e+07 9793.25 1439.9 1.25993e+07 7196.39 1.67936e+07 46259.3 7023.9 6.30784e+06 5773.77 1.67936e+07 5474.35 1.67936e+07 3981.48 1.67936e+07 832879 1.18218e+06 8.14943e+06 3.15184e+07
76994 20231009.5 88389.5 27792.2 2.30851e+07 0.799524 17106.4 45848 5591.29 8.59996e+07 48779.1 53666.3 4397.6 17242 19596.7 16914.5 13986.8 1.88908e+07 14144 1.05021e+07 16715.5 15585.3 1.25993e+07 9855.69 1431.3 1.25993e+07 6852.61 1.67936e+07 44311.9 6940.5 6.30784e+06 5808.48 1.67936e+07 5783.34 1.67936e+07 3987.05 1.67936e+07 831717 1.18619e+06 8.15293e+06 3.07093e+07
77015 20231010.4 62049.7 27777.8 2.30851e+07 0.83228 17372.9 45785.1 5592.78 8.59996e+07 48171.9 53651.8 4441.2 17407.2 19739.4 17105.5 14023.3 1.88908e+07 14107 1.05021e+07 16857.9 15597.5 1.25993e+07 9954.93 1436 1.25993e+07 6898.77 1.88908e+07 41221.6 7117.8 6.30784e+06 5804.14 1.67936e+07 5786.43 1.67936e+07 4016.78 1.67936e+07 825345 1.18742e+06 8.12844e+06 3.06711e+07
77052 20231010.17 87071.8 27851.7 2.51822e+07 0.805827 17123.9 45707.2 5614.03 8.59996e+07 48521.1 53548.8 4435.4 17238.9 19725.4 17064.9 14004.1 1.88908e+07 14072.2 1.05021e+07 16932.1 15532.7 1.25993e+07 9900.42 1434.8 1.25993e+07 6879.16 1.88908e+07 39661.9 6931.4 6.30784e+06 5811.67 1.67936e+07 5725.25 1.88908e+07 3978.87 1.67936e+07 835356 1.17508e+06 8.12795e+06 3.02453e+07
77065 20231010.21 76791.9 27866.2 2.51822e+07 0.805254 17453.5 45981.3 5593.79 8.59996e+07 47639.7 53782.5 4471.3 17159.7 19727 17221.9 13984.2 1.88908e+07 14076.6 1.05021e+07 16835.2 15457.8 1.46964e+07 9888.21 1431.7 1.25993e+07 6839.77 1.67936e+07 47354.4 6882.8 6.30784e+06 5802.59 1.67936e+07 5713.9 1.67936e+07 3982.91 1.67936e+07 831464 1.17023e+06 8.14609e+06 3.06028e+07
77080 20231011.6 77100.4 28231.3 2.30851e+07 0.805739 17356.8 45541.3 5614.98 8.59996e+07 47443.3 53459.3 4480.6 17192.9 19626.7 16932.3 14016.4 1.67936e+07 14058.7 1.05021e+07 16704.5 15607.1 1.25993e+07 9841.59 1435.6 1.25993e+07 6865.39 1.88908e+07 44338.6 6917 6.30784e+06 5768.86 1.67936e+07 5749.11 1.67936e+07 3974.5 1.67936e+07 839572 1.1863e+06 8.14557e+06 3.07489e+07
77087 20231011.8 83135 27891.2 2.51822e+07 0.821071 17648.9 45716.6 5620.28 8.59996e+07 48214.1 53475.5 4431.6 17316.9 19004.2 17758.2 14043.4 1.88908e+07 14129.4 1.05021e+07 16880.7 15619 1.46964e+07 9849.81 1427.7 1.25993e+07 6872.29 1.67936e+07 40652.5 7079.7 6.30784e+06 5780.82 1.67936e+07 5765.08 1.67936e+07 3974.44 1.67936e+07 827510 1.16961e+06 8.14036e+06 3.30899e+07
77124 20231011.20 81677.3 27859.9 2.51822e+07 0.810686 17272.2 45763.1 5576.3 8.59996e+07 48479 54961.6 4431 17497.2 19917.1 17216.9 13934.9 1.88908e+07 14029.2 1.05021e+07 16806.9 15421.8 1.25993e+07 9875.23 1419.9 1.25993e+07 6782.48 1.67936e+07 42920 7028.9 6.30784e+06 5759.88 1.67936e+07 5495.58 1.88908e+07 3980.45 1.67936e+07 829281 1.17704e+06 8.02253e+06 3.09324e+07
77138 20231011.24 79701.7 27788.7 2.51822e+07 0.817166 17450.2 43714.5 5505.15 8.59996e+07 47439 54645.3 4386.8 16999.7 19561.8 17388.6 13944.3 1.88908e+07 14007.9 1.05021e+07 14746.3 15414.3 1.46964e+07 9915 1420.9 1.25993e+07 6826.31 1.88908e+07 39122 6933.7 6.30784e+06 5805.03 1.67936e+07 5491.25 1.67936e+07 3992.65 1.67936e+07 838714 1.18152e+06 8.09934e+06 3.09754e+07
77148 20231012.2 88890.3 28066.3 2.51822e+07 0.806977 17218.1 46048.4 5622.22 8.59996e+07 48655.6 55217 4449.2 17077.4 19720.8 17064.5 14012.7 1.88908e+07 14136.8 1.05021e+07 15135.1 15607.9 1.25993e+07 9799.03 1437.4 1.25993e+07 6885.22 1.88908e+07 45326.3 6981.9 6.30784e+06 5813.57 1.67936e+07 5723.57 1.88908e+07 4000.45 1.67936e+07 828634 1.17897e+06 8.14758e+06 3.07383e+07
77236 20231016.5 59377 27858.7 2.51822e+07 0.79596 17148.6 45783.9 5620.83 8.59996e+07 48719.4 54902.8 4441 17077.1 18804.1 17198.3 13995.4 1.88908e+07 14108.1 1.05021e+07 15291.6 15609.7 1.46964e+07 9993.77 1442 1.25993e+07 6827.74 1.67936e+07 44154.4 7065.1 6.30784e+06 5817.83 1.67936e+07 5743.02 1.67936e+07 4006.32 1.67936e+07 838342 1.17813e+06 8.15154e+06 3.18804e+07
77284 20231016.20 76928.2 27801.1 2.30851e+07 0.800376 17222.7 45566.9 5523.42 8.59996e+07 48808.1 54701.8 4442.5 17018.1 19157.5 17650.8 13858.5 1.88908e+07 14093.4 1.05021e+07 14639.8 15419 1.46964e+07 9852.83 1423.4 1.05021e+07 6815.4 1.88908e+07 43502.1 6872.8 6.30784e+06 5802.5 1.67936e+07 5472.72 1.67936e+07 3973.37 1.67936e+07 833609 1.17767e+06 8.15537e+06 3.06513e+07

only_enable_runtime_limits_during_js_exec

build_id build_number pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem Commit latency factor tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_basic_sgx_cft^ pi_basic_sgx_cft_mem pi_basic_js_sgx_cft^ pi_basic_js_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem pi_basic_mt_virtual_cft^ tpcc_virtual_cft^ ls_virtual_cft^ hist_sgx_cft^ pi_ls_virtual_cft^ pi_basic_virtual_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^ pi_basic_js_virtual_cft^ ls_jwt_virtual_cft^ pi_ls_jwt_virtual_cft^ ls_js_virtual_cft^ ls_full_js_virtual_cft^ ls_js_jwt_virtual_cft^
77266 20231016.14 28000.2 2.51822e+07 0.799662 5605.34 8.59996e+07 14020.8 1.88908e+07 14154.4 1.05021e+07 15613.6 1.25993e+07 1440.9 1.25993e+07 6876.57 1.67936e+07 6924.9 6.30784e+06 5780.87 1.67936e+07 5774.3 1.67936e+07 3983.7 1.67936e+07 80941.7 17214.1 45889.7 45162 48432 54261.2 835570 1.18519e+06 8.1336e+06 3.07729e+07 4430.3 17363.2 19962.5 17289.2 14918.4 9772.41
77270 20231016.15 27821.4 2.30851e+07 0.795861 5545.26 8.59996e+07 13940.5 1.88908e+07 14001.6 1.05021e+07 15443.3 1.46964e+07 1432.3 1.25993e+07 6871.5 1.88908e+07 6973.3 6.30784e+06 5803.73 1.67936e+07 5497.87 1.67936e+07 3958.77 1.67936e+07 65520.3 17275.1 45769.9 41754.5 48113 54475.8 833800 1.17825e+06 8.13703e+06 3.0862e+07 4406.9 17253.9 19366.7 17298.2 14908.3 9917.5
77276 20231016.17 27673.5 2.30851e+07 0.801142 5594.22 8.59996e+07 14021 1.88908e+07 14117.9 1.05021e+07 15663.4 1.25993e+07 1434.4 1.25993e+07 7267.51 1.67936e+07 7028.4 6.30784e+06 5809.11 1.67936e+07 5722.77 1.67936e+07 3999.77 1.67936e+07 79547.2 17112.2 45641.9 42391.8 47877.1 53751.2 835868 1.17467e+06 8.15468e+06 3.06165e+07 4428.9 17022.5 19723.5 17355.3 14667.6 9875.93
77282 20231016.19 28061 2.30851e+07 0.803299 5580.29 8.59996e+07 13983.3 1.88908e+07 14074.4 1.05021e+07 15321.8 1.46964e+07 1428 1.25993e+07 6819.45 1.88908e+07 6971.1 6.30784e+06 5772.37 1.67936e+07 5723.5 1.67936e+07 3993.4 1.67936e+07 75371.4 17289.6 43997.1 42634.9 47966.9 54207.4 828886 1.1843e+06 8.13402e+06 3.06243e+07 4445.3 17126.4 19940.2 17531.7 14699.4 9793.22
77292 20231016.22 27707.4 2.51822e+07 0.805827 5512.33 8.59996e+07 14048.9 1.88908e+07 14101 1.05021e+07 15553.7 1.46964e+07 1427.6 1.25993e+07 6866.59 1.88908e+07 6878.1 6.30784e+06 5798.14 1.67936e+07 5489.19 1.88908e+07 3983.24 1.67936e+07 85024.6 17296.6 46054.7 39188.3 47983.7 53759 835176 1.17901e+06 8.1368e+06 3.14289e+07 4396.9 17194.4 19633.6 17014.5 14696.2 10417.5

images

@achamayou achamayou marked this pull request as ready for review October 13, 2023 17:02
@achamayou achamayou requested a review from a team October 13, 2023 17:02
src/js/wrap.cpp Outdated Show resolved Hide resolved
@achamayou achamayou added auto-backport Automatically backport this PR to LTS branch 4.x-todo PRs which should be backported to 4.x labels Oct 16, 2023
@achamayou achamayou enabled auto-merge (squash) October 16, 2023 15:55
@achamayou achamayou merged commit c2f42d4 into microsoft:main Oct 16, 2023
24 of 29 checks passed
@ghost
Copy link

ghost commented Oct 16, 2023

💔 All backports failed

Status Branch Result
release/4.x Backport failed because of merge conflicts

You might need to backport the following PRs to release/4.x:
- Use new governance API throughout our infra (#5698)
- Add basic backpressure mechanism if primary has too many in-flight transactions (#5692)
- Implement Azure-compliant governance interface (#5660)
- Add CLI argument for SNP context directory (#5686)
- Pass enclave path as CLI argument rather than in configuration (#5665)

Manual backport

To create the backport manually run:

backport --pr 5730

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

@achamayou achamayou added this to the 4.x milestone Oct 17, 2023
@ghost ghost mentioned this pull request Oct 23, 2023
achamayou added a commit to achamayou/CCF that referenced this pull request Oct 24, 2023
(cherry picked from commit c2f42d4)

# Conflicts:
#	.daily_canary
#	CHANGELOG.md
#	src/js/wrap.cpp
#	tests/js-custom-authorization/custom_authorization.py
@achamayou achamayou added the backported This PR was successfully backported to LTS branch label Oct 24, 2023
achamayou added a commit to achamayou/CCF that referenced this pull request Oct 25, 2023
(cherry picked from commit c2f42d4)

# Conflicts:
#	CHANGELOG.md
#	src/js/wrap.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4.x-todo PRs which should be backported to 4.x auto-backport Automatically backport this PR to LTS branch backported This PR was successfully backported to LTS branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handler invocations from js_kv_map_foreach get a full max_exec_time each
2 participants