Recompile problems after deps update

[lukaszsamson]

Environment

• Elixir & Erlang versions (elixir --version): elixir 1.6.1, erlang 20.2.3
• Operating system: OSX 10.13

Current behavior

Umbrella app with two apps: app and app_web

mix test

passes, no problems

mix deps.update --all
...
* Updating plug (Hex package)

mix test
==> plug
Compiling 44 files (.ex)
warning: Enum.partition/2 is deprecated, use Enum.split_with/2
  lib/plug/adapters/cowboy.ex:46
Generated plug app
...

all plug tests fail with

** (FunctionClauseError) no function clause matching in Plug.Parsers.call/2
...
stacktrace:
       (plug) lib/plug/parsers.ex:218: Plug.Parsers.call/2
       (app_web) lib/app_web/endpoint.ex:1: App.Web.Endpoint.plug_builder_call/2
       (app_web) lib/app_web/endpoint.ex:54: App.Web.Endpoint."call (overridable 2)"/2
       (app_web) lib/app_web/endpoint.ex:1: App.Web.Endpoint.call/2
       (phoenix) lib/phoenix/test/conn_test.ex:224: Phoenix.ConnTest.dispatch/5

should not happen, no breaking changes in plug between 1.4.3 and 1.4.5 (elixir-plug/plug@fae09fc affects internal api)

mix clean
mix test
==> app
Compiling 128 files (.ex)

== Compilation error in file test/support/factory.ex ==
** (RuntimeError) You tried to use ExMachina.Ecto, but the Ecto module is not loaded. Please add ecto to your dependencies.
    lib/ex_machina/ecto.ex:54: ExMachina.Ecto.verify_ecto_dep/0
    expanding macro: ExMachina.Ecto.__using__/1
    test/support/factory.ex:2: App.Factory (module)
    (elixir) expanding macro: Kernel.use/2
    test/support/factory.ex:2: App.Factory (module)
==> app_web
Compiling 143 files (.ex)
** (ArgumentError) argument error
    (stdlib) :ets.lookup_element(:elixir_modules, App.MyModule, 2)
    (elixir) lib/kernel/lexical_tracker.ex:37: Kernel.LexicalTracker.to_pid/1
    (elixir) lib/kernel/lexical_tracker.ex:16: Kernel.LexicalTracker.remote_references/1
    (mix) lib/mix/compilers/elixir.ex:230: Mix.Compilers.Elixir.each_module/4
    (elixir) lib/kernel/parallel_compiler.ex:293: Kernel.ParallelCompiler.wait_for_messages/6
    (elixir) lib/kernel/parallel_compiler.ex:129: Kernel.ParallelCompiler.spawn_workers/3
    (mix) lib/mix/compilers/elixir.ex:180: Mix.Compilers.Elixir.compile_manifest/9

ecto is in the deps, and the compiler should not crash.

mix test

this time it worked

Expected behavior

proper recompilation of changed deps and affected aplications

[josevalim]

When you update Plug, Mix should recompile any application that depends on Plug. But that doesn't seem to be happening. Do you have a sample application that reproduces the issue or a minimal case? Otherwise it will be very hard to track this.

[josevalim]

Ping.

[lukaszsamson]

I don't have a working repro. I'll try to isolate this and reproduce.

[lukaszsamson]

I was able to reproduce the plug issue the following steps:

1. mix phx.new plug_error_repro --umbrella
2. override plug dep to 1.4.3
3. mix deps.get && mix test
4. remove override
5. mix deps.update --all && mix test

However it only happened when I had this extension running. It seems ElixirLS is doing its compilation in the background and it messes the compilation requested by mix test. If that's the case then maybe some locking is necessary to prevent concurrent compilation...

[josevalim]

Thanks a lot!

Let's ping @JakeBecker but generally speaking Elixir will indeed fail with multiple projects commands running on the same build. It is best if editors run their commands on a separate _build directory. It that is not easily doable today, let's find a way to make it so.

[JakeBecker]

ElixirLS could override the build directory after loading the project's Mix file, but this wouldn't work for umbrella apps because each sub-app specifies the build directory and there's no simple way to hook in and change it when running mix compile from the umbrella app. If there were an easy way to override it, I'd be happy to make that change in ElixirLS.

[josevalim]

@JakeBecker only the configuration of the parent project matters so the build_path inside the umbrella child is ignored when they are used as a dependency (which is the case in any umbrella project). This applies to all dependencies too. So I am thinking that if you change the build_path for the parent, you should be covered. So you should be good to go?

[JakeBecker]

Ah, it does seem to work even for umbrellas if I do this before loading the mixfile: Mix.ProjectStack.post_config(build_path: ".elixir_ls/build")

Assuming this API isn't in immediate danger of going away, I'll push an update to ElixirLS with this shortly.

[josevalim]

That API is not going to change but it is private. If you are loading the project with Mix.Project.in_project, you should be able to pass this option directly. :)