Toplevel definitions in forgotten class files break compilation

[odersky]

Toplevel definitions have the downside that they can accumulate on the output path and influence subsequent compilations in unexpected ways. Here's an example:

First, compile file A.scala

@main def Test() = println("test")

Next, compile file B.scala

object Test

If the output directory is on the class path, this gives an error indicating that Test needs to be applied to (). What happens is that the compiler will load and merge the denotations of Test in both A.scala and B.scala and then will get confused since it thinks Test is a method.

The problem is more general: Whenever there are several files with toplevel definitions, these definitions go into separate wrapper objects that are named after their file names. This means that compiling one file with toplevel definitions does not invalidate the definitions in another file. This is normally intended, but it does mean that "junk" in long forgotten class files can come back in unexpected ways. For instance, if one has first toplevel definitions in a file A.scala, and then renames the file to B.scala, possibly with with some changes, the definitions of A and B will compete as overloaded variants with each other, as long as A$package.class is on the classpath.

The normal way to mitigate the problem is to clean compile whenever a file containing toplevel definitions is renamed. But there's one situation where this is impractical: Toplevel definitions in the empty package. These are created by most simple tests, all the time. It is impractical to require a clean compile before compiling an additional test.

I believe it's better to hide toplevel definitions in the empty package from being visible in other compilation units, unless both units are compiled together. Otherwise we are simply too vulnerable of getting junk in the system that can cause mysterious compilation errors.

Another, more complicated fix would be to drop shadowed toplevel definitions looking at file creation dates. That is, if we have two toplevel definitions of method foo with matching signatures in two different wrappers, go back to the wrapper's associated files and get the file creation date of each. Only keep the method in the newer file. This protects against conflicting versions of foo, but not against overloading. We'd still have the problem if we have toplevel definitions like this:

def foo(x: Long): Unit = ...
val bar = foo(1)

and then compile them with a junk class file that contains a definition of foo over Int. One could make the filtering even more aggressive by dropping all same-named methods independent of their signatures, but that would mean that we cannot define overloaded variants of existing methods
in new toplevel definitions anymore. So, no easy solution in sight...

[odersky]

I settled on the solution to invalidate toplevel definitions with the same name as a later one, which does outlaw overloaded toplevel definitions in several files. I think that one has the least downside of all considered alternatives.