incompatible_static_name_resolution: Implement Name resolution proposal

[laurentlb]

https://github.com/bazelbuild/proposals/blob/master/designs/2018-06-18-name-resolution.md

[laurentlb]

Allow shadowing of builtins in bzl files
395fbbd

[dslomov]

What is missing for migration: migration docs, length of migration window. After these are done, please add "migration-ready" label.

[dslomov]

There are still no migration docs in this issue. Please add some.

[alandonovan]

I'm sorry for pointing this out after you've done most of the work (but at least it hasn't been released), but I'm no longer convinced this change is an improvement. Though it makes the handling of globals and locals consistent, disallowing the first print(len) in this example,

print(len); len=1; print(len)

it creates a problem no smaller than the one it solves, yet gratuitously different from Python: the restriction was motivated by the idea that a reader shouldn't have to remember where they are in the file to know which binding of len they are referring to---in particular, they don't have to look "up". But now they have to look "down" to see whether a global declaration follows. Even though it's statically provable that the first reference to len will fail dynamically, there is no static error.

Having to look down is more surprising than having to look up; the need to look down for local variables in Python works because the function bodies fit on a page, but global variables can be thousands of lines away.

I think we should revert the spec to follow Python. (Or at the very least make the forward reference a static error too.)

[laurentlb]

Having to look down is more surprising than having to look up

This can easily be reported by a linter. IDEs should be able to jump to definition too.

[alandonovan]

IDEs should be able to jump to definition too.

That's not relevant. The choice here is between two alternative static name resolution algorithms.

From related discussion at google/starlark-go#14 (comment):

No-one disputes the value of static name resolution; that's not the issue here.

The issue is that we can statically resolve the first len in print(len); len=1; print(len) in two different ways: the Python way, which resolves to the definition "above", that is, the predeclared one, or the Skylark-spec way, which resolves the definition "below".

The Python way is clearly useful. For example, you might want to define a local variant of a built-in (print, say), using the same name:

builtin_print = print
def print(x): ....builtin_print(x)...
...
print(...) # calls the user-defined one
There are plenty of other examples in BUILD files.

But the Skylark-spec way is literally never useful. It's a guaranteed dynamic error.

The Python way is static, inconsistent but familiar, and useful. The Skylark-spec way is static, consistent but surprising, and not useful. How is that better?

[brandjon]

I don't think it's that important that it be a static error as opposed to a linter error. Plenty of things can be determined to fail statically, such as division by literal zero.

The Python way is clearly useful. For example, you might want to define a local variant of a built-in (print, say), using the same name

It can also be mis-used as follows:

print(...)    # 1
def f(x):
    print(x)  # 2

def print(x): ...

print(...)    # 3
def g(x):
    print(x)  # 4

In Python, only (1) uses the builtin print. The use case you give could be supported by adding a builtins module to reference the original print, if we feel it's important.

[alandonovan]

Correction: I was under the misapprehension that Python's name resolution statically resolved the distinction between predeclared names and global names, but this is not the case. It treats all global names the same, and the LOAD_GLOBAL op does a dynamic lookup first in globals, then in builtins.

So, rephrasing: perhaps we should specify that name resolution statically resolves all global names to either a predeclared identifier or a global, based on the lexical environment. This is more static (and more efficient) than the Python approach, yet still allows 'print(len); len=1; print(len)' to work in the obvious way. Jon's program from the previous note would resolve uses 1 and 2 to the predeclared print, and 3 and 4 to the user-defined print. Users only need to "look up".

[EricCousineau-TRI]

FTR, new link for doc is at:
https://github.com/bazelbuild/proposals/blob/master/designs/2018-06-18-name-resolution.md#lexical-binding-of-variables (permalink via sha)