feat: add optimization mode to vyper compiler (vyperlang#3493)

this commit adds the `--optimize` flag to the vyper cli, and as an
option in vyper json. it is to be used separately from the
`--no-optimize` flag. this commit does not actually change codegen,
just adds the flag and threads it through the codebase so it is
available once we want to start differentiating between the two modes,
and sets up the test harness to test both modes.

it also makes the `optimize` and `evm-version` available as source code
pragmas, and adds an additional syntax for specifying the compiler
version (`#pragma version X.Y.Z`). if the CLI / JSON options conflict
with the source code pragmas, an exception is raised.

this commit also:
* bumps mypy - it was needed to bump to 0.940 to handle match/case, and
  discovered we could bump all the way to 0.98* without breaking
  anything
* removes evm_version from bitwise op tests - it was probably important
  when we supported pre-constantinople targets, which we don't anymore

.github/workflows/test.yml

@@ -79,8 +79,8 @@ jobs:
     strategy:
       matrix:
         python-version: [["3.10", "310"], ["3.11", "311"]]
-        # run in default (optimized) and --no-optimize mode
-        flag: ["core", "no-opt"]
+        # run in modes: --optimize [gas, none, codesize]
+        flag: ["core", "no-opt", "codesize"]
 
     name: py${{ matrix.python-version[1] }}-${{ matrix.flag }}
 

docs/compiling-a-contract.rst

@@ -99,6 +99,11 @@ See :ref:`searching_for_imports` for more information on Vyper's import system.
 Online Compilers
 ================
 
+Try VyperLang!
+-----------------
+
+`Try VyperLang! <https://try.vyperlang.org>`_ is a JupterHub instance hosted by the Vyper team as a sandbox for developing and testing contracts in Vyper. It requires github for login, and supports deployment via the browser.
+
 Remix IDE
 ---------
 
@@ -109,22 +114,33 @@ Remix IDE
    While the Vyper version of the Remix IDE compiler is updated on a regular basis, it might be a bit behind the latest version found in the master branch of the repository. Make sure the byte code matches the output from your local compiler.
 
 
+.. _evm-version:
+
 Setting the Target EVM Version
 ==============================
 
-When you compile your contract code, you can specify the Ethereum Virtual Machine version to compile for, to avoid particular features or behaviours.
+When you compile your contract code, you can specify the target Ethereum Virtual Machine version to compile for, to access or avoid particular features. You can specify the version either with a source code pragma or as a compiler option. It is recommended to use the compiler option when you want flexibility (for instance, ease of deploying across different chains), and the source code pragma when you want bytecode reproducibility (for instance, when verifying code on a block explorer).
+
+.. note::
+   If the evm version specified by the compiler options conflicts with the source code pragma, an exception will be raised and compilation will not continue.
+
+For instance, the adding the following pragma to a contract indicates that it should be compiled for the "shanghai" fork of the EVM.
+
+.. code-block:: python
+
+   #pragma evm-version shanghai
 
 .. warning::
 
-    Compiling for the wrong EVM version can result in wrong, strange and failing behaviour. Please ensure, especially if running a private chain, that you use matching EVM versions.
+    Compiling for the wrong EVM version can result in wrong, strange, or failing behavior. Please ensure, especially if running a private chain, that you use matching EVM versions.
 
-When compiling via ``vyper``, include the ``--evm-version`` flag:
+When compiling via the ``vyper`` CLI, you can specify the EVM version option using the ``--evm-version`` flag:
 
 ::
 
     $ vyper --evm-version [VERSION]
 
-When using the JSON interface, include the ``"evmVersion"`` key within the ``"settings"`` field:
+When using the JSON interface, you can include the ``"evmVersion"`` key within the ``"settings"`` field:
 
 .. code-block:: javascript
 
@@ -213,9 +229,10 @@ The following example describes the expected input format of ``vyper-json``. Com
         // Optional
         "settings": {
             "evmVersion": "shanghai",  // EVM version to compile for. Can be istanbul, berlin, paris, shanghai (default) or cancun (experimental!).
-            // optional, whether or not optimizations are turned on
-            // defaults to true
-            "optimize": true,
+            // optional, optimization mode
+            // defaults to "gas". can be one of "gas", "codesize", "none",
+            // false  and true (the last two are for backwards compatibility).
+            "optimize": "gas",
             // optional, whether or not the bytecode should include Vyper's signature
             // defaults to true
             "bytecodeMetadata": true,

docs/structure-of-a-contract.rst

@@ -9,16 +9,47 @@ This section provides a quick overview of the types of data present within a con
 
 .. _structure-versions:
 
-Version Pragma
+Pragmas
 ==============
 
-Vyper supports a version pragma to ensure that a contract is only compiled by the intended compiler version, or range of versions. Version strings use `NPM <https://docs.npmjs.com/about-semantic-versioning>`_ style syntax.
+Vyper supports several source code directives to control compiler modes and help with build reproducibility.
+
+Version Pragma
+--------------
+
+The version pragma ensures that a contract is only compiled by the intended compiler version, or range of versions. Version strings use `NPM <https://docs.npmjs.com/about-semantic-versioning>`_ style syntax.
+
+As of 0.3.10, the recommended way to specify the version pragma is as follows:
 
 .. code-block:: python
 
-    # @version ^0.2.0
+    #pragma version ^0.3.0
+
+The following declaration is equivalent, and, prior to 0.3.10, was the only supported method to specify the compiler version:
+
+.. code-block:: python
+
+    # @version ^0.3.0
+
+
+In the above examples, the contract will only compile with Vyper versions ``0.3.x``.
+
+Optimization Mode
+-----------------
+
+The optimization mode can be one of ``"none"``, ``"codesize"``, or ``"gas"`` (default). For instance, the following contract will be compiled in a way which tries to minimize codesize:
+
+.. code-block:: python
+
+   #pragma optimize codesize
+
+The optimization mode can also be set as a compiler option. If the compiler option conflicts with the source code pragma, an exception will be raised and compilation will not continue.
+
+EVM Version
+-----------------
+
+The EVM version can be set with the ``evm-version`` pragma, which is documented in :ref:`evm-version`.
 
-In the above example, the contract only compiles with Vyper versions ``0.2.x``.
 
 .. _structure-state-variables:
 

setup.py

@@ -28,7 +28,7 @@
         "flake8-bugbear==20.1.4",
         "flake8-use-fstring==1.1",
         "isort==5.9.3",
-        "mypy==0.910",
+        "mypy==0.982",
     ],
     "docs": ["recommonmark", "sphinx>=6.0,<7.0", "sphinx_rtd_theme>=1.2,<1.3"],
     "dev": ["ipython", "pre-commit", "pyinstaller", "twine"],

tests/ast/test_pre_parser.py

@@ -1,6 +1,7 @@
 import pytest
 
-from vyper.ast.pre_parser import validate_version_pragma
+from vyper.ast.pre_parser import pre_parse, validate_version_pragma
+from vyper.compiler.settings import OptimizationLevel, Settings
 from vyper.exceptions import VersionException
 
 SRC_LINE = (1, 0)  # Dummy source line
@@ -51,14 +52,14 @@ def set_version(version):
 @pytest.mark.parametrize("file_version", valid_versions)
 def test_valid_version_pragma(file_version, mock_version):
     mock_version(COMPILER_VERSION)
-    validate_version_pragma(f" @version {file_version}", (SRC_LINE))
+    validate_version_pragma(f"{file_version}", (SRC_LINE))
 
 
 @pytest.mark.parametrize("file_version", invalid_versions)
 def test_invalid_version_pragma(file_version, mock_version):
     mock_version(COMPILER_VERSION)
     with pytest.raises(VersionException):
-        validate_version_pragma(f" @version {file_version}", (SRC_LINE))
+        validate_version_pragma(f"{file_version}", (SRC_LINE))
 
 
 prerelease_valid_versions = [
@@ -98,11 +99,85 @@ def test_invalid_version_pragma(file_version, mock_version):
 @pytest.mark.parametrize("file_version", prerelease_valid_versions)
 def test_prerelease_valid_version_pragma(file_version, mock_version):
     mock_version(PRERELEASE_COMPILER_VERSION)
-    validate_version_pragma(f" @version {file_version}", (SRC_LINE))
+    validate_version_pragma(file_version, (SRC_LINE))
 
 
 @pytest.mark.parametrize("file_version", prerelease_invalid_versions)
 def test_prerelease_invalid_version_pragma(file_version, mock_version):
     mock_version(PRERELEASE_COMPILER_VERSION)
     with pytest.raises(VersionException):
-        validate_version_pragma(f" @version {file_version}", (SRC_LINE))
+        validate_version_pragma(file_version, (SRC_LINE))
+
+
+pragma_examples = [
+    (
+        """
+    """,
+        Settings(),
+    ),
+    (
+        """
+    #pragma optimize codesize
+    """,
+        Settings(optimize=OptimizationLevel.CODESIZE),
+    ),
+    (
+        """
+    #pragma optimize none
+    """,
+        Settings(optimize=OptimizationLevel.NONE),
+    ),
+    (
+        """
+    #pragma optimize gas
+    """,
+        Settings(optimize=OptimizationLevel.GAS),
+    ),
+    (
+        """
+    #pragma version 0.3.10
+    """,
+        Settings(compiler_version="0.3.10"),
+    ),
+    (
+        """
+    #pragma evm-version shanghai
+    """,
+        Settings(evm_version="shanghai"),
+    ),
+    (
+        """
+    #pragma optimize codesize
+    #pragma evm-version shanghai
+    """,
+        Settings(evm_version="shanghai", optimize=OptimizationLevel.GAS),
+    ),
+    (
+        """
+    #pragma version 0.3.10
+    #pragma evm-version shanghai
+    """,
+        Settings(evm_version="shanghai", compiler_version="0.3.10"),
+    ),
+    (
+        """
+    #pragma version 0.3.10
+    #pragma optimize gas
+    """,
+        Settings(compiler_version="0.3.10", optimize=OptimizationLevel.GAS),
+    ),
+    (
+        """
+    #pragma version 0.3.10
+    #pragma evm-version shanghai
+    #pragma optimize gas
+    """,
+        Settings(compiler_version="0.3.10", optimize=OptimizationLevel.GAS, evm_version="shanghai"),
+    ),
+]
+
+
+@pytest.mark.parametrize("code, expected_pragmas", pragma_examples)
+def parse_pragmas(code, expected_pragmas):
+    pragmas, _, _ = pre_parse(code)
+    assert pragmas == expected_pragmas

tests/base_conftest.py

@@ -12,6 +12,7 @@
 
 from vyper import compiler
 from vyper.ast.grammar import parse_vyper_source
+from vyper.compiler.settings import Settings
 
 
 class VyperMethod:
@@ -111,14 +112,16 @@ def w3(tester):
     return w3
 
 
-def _get_contract(w3, source_code, no_optimize, *args, **kwargs):
+def _get_contract(w3, source_code, optimize, *args, **kwargs):
+    settings = Settings()
+    settings.evm_version = kwargs.pop("evm_version", None)
+    settings.optimize = optimize
     out = compiler.compile_code(
         source_code,
         # test that metadata gets generated
         ["abi", "bytecode", "metadata"],
+        settings=settings,
         interface_codes=kwargs.pop("interface_codes", None),
-        no_optimize=no_optimize,
-        evm_version=kwargs.pop("evm_version", None),
         show_gas_estimates=True,  # Enable gas estimates for testing
     )
     parse_vyper_source(source_code)  # Test grammar.
@@ -135,13 +138,15 @@ def _get_contract(w3, source_code, no_optimize, *args, **kwargs):
     return w3.eth.contract(address, abi=abi, bytecode=bytecode, ContractFactoryClass=VyperContract)
 
 
-def _deploy_blueprint_for(w3, source_code, no_optimize, initcode_prefix=b"", **kwargs):
+def _deploy_blueprint_for(w3, source_code, optimize, initcode_prefix=b"", **kwargs):
+    settings = Settings()
+    settings.evm_version = kwargs.pop("evm_version", None)
+    settings.optimize = optimize
     out = compiler.compile_code(
         source_code,
         ["abi", "bytecode"],
         interface_codes=kwargs.pop("interface_codes", None),
-        no_optimize=no_optimize,
-        evm_version=kwargs.pop("evm_version", None),
+        settings=settings,
         show_gas_estimates=True,  # Enable gas estimates for testing
     )
     parse_vyper_source(source_code)  # Test grammar.
@@ -173,17 +178,17 @@ def factory(address):
 
 
 @pytest.fixture(scope="module")
-def deploy_blueprint_for(w3, no_optimize):
+def deploy_blueprint_for(w3, optimize):
     def deploy_blueprint_for(source_code, *args, **kwargs):
-        return _deploy_blueprint_for(w3, source_code, no_optimize, *args, **kwargs)
+        return _deploy_blueprint_for(w3, source_code, optimize, *args, **kwargs)
 
     return deploy_blueprint_for
 
 
 @pytest.fixture(scope="module")
-def get_contract(w3, no_optimize):
+def get_contract(w3, optimize):
     def get_contract(source_code, *args, **kwargs):
-        return _get_contract(w3, source_code, no_optimize, *args, **kwargs)
+        return _get_contract(w3, source_code, optimize, *args, **kwargs)
 
     return get_contract
 

tests/cli/vyper_json/test_get_settings.py

@@ -3,7 +3,6 @@
 import pytest
 
 from vyper.cli.vyper_json import get_evm_version
-from vyper.evm.opcodes import DEFAULT_EVM_VERSION
 from vyper.exceptions import JSONError
 
 
@@ -31,7 +30,3 @@ def test_early_evm(evm_version):
 @pytest.mark.parametrize("evm_version", ["istanbul", "berlin", "paris", "shanghai", "cancun"])
 def test_valid_evm(evm_version):
     assert evm_version == get_evm_version({"settings": {"evmVersion": evm_version}})
-
-
-def test_default_evm():
-    assert get_evm_version({}) == DEFAULT_EVM_VERSION

tests/compiler/asm/test_asm_optimizer.py

@@ -1,6 +1,7 @@
 import pytest
 
 from vyper.compiler.phases import CompilerData
+from vyper.compiler.settings import OptimizationLevel, Settings
 
 codes = [
     """
@@ -72,7 +73,7 @@ def __init__():
 
 @pytest.mark.parametrize("code", codes)
 def test_dead_code_eliminator(code):
-    c = CompilerData(code, no_optimize=True)
+    c = CompilerData(code, settings=Settings(optimize=OptimizationLevel.NONE))
     initcode_asm = [i for i in c.assembly if not isinstance(i, list)]
     runtime_asm = c.assembly_runtime
 
@@ -87,7 +88,7 @@ def test_dead_code_eliminator(code):
     for s in (ctor_only_label, runtime_only_label):
         assert s + "_runtime" in runtime_asm
 
-    c = CompilerData(code, no_optimize=False)
+    c = CompilerData(code, settings=Settings(optimize=OptimizationLevel.GAS))
     initcode_asm = [i for i in c.assembly if not isinstance(i, list)]
     runtime_asm = c.assembly_runtime