[shadow-cljs] Invalid :refer, var taoensso.encore/have does not exist

[BorisKourt]

[:app] Compiling ...
------ ERROR -------------------------------------------------------------------
 File: jar:file:/Users/<~>/.m2/repository/com/taoensso/timbre/6.0.2/timbre-6.0.2.jar!/taoensso/timbre/appenders/core.cljc:1:1
--------------------------------------------------------------------------------

   1 | (ns taoensso.timbre.appenders.core
-------^------------------------------------------------------------------------
Invalid :refer, var taoensso.encore/have does not exist
--------------------------------------------------------------------------------
   2 |   "Core Timbre appenders without any special dependency requirements.
   3 |   These can be aliased into the main Timbre ns for convenience."
   4 |   {:author "Peter Taoussanis (@ptaoussanis)"}
   5 |   (:require
--------------------------------------------------------------------------------

When compiling for production with shadow-cljs I get this error from the last few versions of Timbre.

Version 5.2.1 is working fine.

I have not been able to pinpoint an exact place in the changelog that starts to have this issue. But I guess it is related to the shuffling of community appenders?

Thanks for any suggestions!

[ptaoussanis]

@BorisKourt Hi Boris, apologies for the trouble!

It looks like you need to update your version of Encore.
I've just pushed [com.taoensso/timbre "6.0.3"] to Clojars, which includes a newer bundled version of Encore.

That should solve your problem, just make sure to run lein clean (or equivalent) to clear out any old build artifacts.

[BorisKourt]

Thank you for such a fast response, unfortunately (with a clean run as well) I am still getting this issue on 6.0.3:

[:app] Compiling ...
------ ERROR -------------------------------------------------------------------
 File: jar:file:/Users/<~>/.m2/repository/com/taoensso/timbre/6.0.3/timbre-6.0.3.jar!/taoensso/timbre/appenders/core.cljc:1:1
--------------------------------------------------------------------------------

   1 | (ns taoensso.timbre.appenders.core
-------^------------------------------------------------------------------------
Invalid :refer, var taoensso.encore/have does not exist
--------------------------------------------------------------------------------
   2 |   "Core Timbre appenders without any special dependency requirements.
   3 |   These can be aliased into the main Timbre ns for convenience."
   4 |   {:author "Peter Taoussanis (@ptaoussanis)"}
   5 |   (:require
--------------------------------------------------------------------------------

I also already had encore at latest on Clojars:

[com.taoensso/timbre "6.0.3" :exclusions [com.taoensso/encore]]
[com.taoensso/encore "3.42.0"]

(test with and without exclusions earlier)

[ptaoussanis]

@BorisKourt Is there a chance something else is bringing in an older version of Encore?
Please try check for encore in the output of lein deps :tree or equivalent.

I'm not too familiar with shadow-cljs, but this appears to be an identical issue to taoensso/encore#64, which should have hopefully been resolved with Encore [com.taoensso/encore "3.37.1"] and later.

If you can confirm that you really have the latest Encore and are seeing the issue, could you please try with [com.taoensso/encore "3.37.1"] precisely? If that's working, but later versions are not - it would seem to indicate that I inadvertently broke shadow-cljs support again.

In that case, I'll try get it running locally so that I can debug properly.

Thanks, and again sorry for the trouble!

[BorisKourt]

Hey I checked using deps :tree, there were no references to older encore that I could search for. I double checked my excludes and made sure that my manual encore include should be the main one referenced.

I then downgraded to 3.37.1 but am unable to test if that solved this issue due to:

Syntax error macroexpanding at (timbre.cljc:15:1).
Execution error (ExceptionInfo) at taoensso.encore/assert-min-encore-version (encore.cljc:1499).
Insufficient `com.taoensso/encore` version, you may have a dependency conflict: see http://goo.gl/qBbLvC for solutions.

I am not sure which version of timbre to downgrade to to fix this. Since if I go far enough back the original issue can go away.

Thanks for taking the time!

[ptaoussanis]

Ah, sorry about that. I've just pushed [com.taoensso/encore "3.37.1-SNAPSHOT"], which is identical to to 3.37.1 but removes the version check.

Could you please try with that?

If it's still no good, I'll try get setup with shadow-cljs later today or tomorrow so that I can debug properly myself.

[BorisKourt]

Hey, this is working with this snapshot.

[ptaoussanis]

Okay, great- thanks for the confirmation 👍
That means that I must have somehow broken shadow support again after 3.37.1.

Will try get setup with shadow so I can debug properly myself, after which I'll push new versions of Encore and Timbre.
Might be a couple days.

In the meantime, I believe that the following combination should work:

[com.taoensso/encore "3.37.1"]
[com.taoensso/timbre "6.0.2"]

Again, sorry about the nuisance.
Thanks a lot for reporting this, and for all the assistance debugging.

[BorisKourt]

Thank you very much! I will keep an eye out for new versions.

[ptaoussanis]

@BorisKourt I'm having some trouble reproducing the problem.
I've not used shadow-cljs before, so it's very possible I'm making a mistake somewhere.

Here's the minimal project I'm using: https://github.com/ptaoussanis/temp-timbre-364

shadow-cljs.edn contains:

 :dependencies
 [[com.taoensso/encore "3.42.0"]
  [com.taoensso/timbre  "6.0.3"]]

And using shadow-cljs - server version: 2.20.13

The init ns contents:

(ns foo
  (:require
   [taoensso.encore :as enc]
   [taoensso.timbre :as timbre]
   [taoensso.timbre.appenders.core]
   ))

(defn init []
  (js/alert    (enc/have string? "hello 1"))
  (timbre/info (enc/have string? "hello 2"))
  )

I'm then running:

npx shadow-cljs browser-repl
npx shadow-cljs watch frontend

and opening http://localhost:8080 in a browser.
So far, I'm not seeing any obvious compilation issues - and Timbre does seem to be operating normally.

Is there something in particular I need to do to trigger the error?
Do you see errors with my example project?

[ptaoussanis]

Update: can confirm that my example project does confirm the problem with [com.taoensso/encore "3.36.0"], but [com.taoensso/encore "3.37.1"] or newer appears fine from what I can tell.

Is it possible that your shadow-cljs has outdated build artifacts? Might it be worth trying to clear your entire js output dir if you haven't already.

[BorisKourt]

I will try to create my own repro example. I am using shadow via lein, directly from the luminus +shadow template. Though it has now been many months in development, so a lot going on in here.

Regarding the second comment:

As far as I know I have cleared all cached output. It seems like its all in target and that gets completely wiped on lein clean I am not sure if shadow does any additional cache work. If it was the persistent wouldn't it still break with [com.taoensso/encore "3.37.1-SNAPSHOT"]?

[ptaoussanis]

Not sure about luminus + shadow template, not familiar with either of those. But at least with the vanilla shadow-cljs, it seems to generate js files in the public dir by default.

If it was the persistent wouldn't it still break with [com.taoensso/encore "3.37.1-SNAPSHOT"]?

I would definitely think so, yes. But not sure why I can't reproduce on my side. Weird things can definitely occur if stale build artifacts are somehow lingering. If it's non-trivial for you to try create your own repro (which I can totally understand), I'd maybe recommend starting with my example project - just to rule out the possibility that you're seeing different results even with that.

What version of Cljs, and what compiler options are you using for your own project that's exhibiting the problem? Is this an :advanced compile?

[BorisKourt]

Quick answer first before I dig in further, this is an :advanced production compile. The original issue only happens in the production build, I mention that but should have been more clear about the CLJS settings.

[ptaoussanis]

Thanks for clarifying 👍

Just tried npx shadow-cljs release on the example project, and that seems to build okay using :advanced mode from what I can tell.

Will continue investigating so long.

[BorisKourt]

Just an update. I tried creating a new Luminus app with +shadow-cljs and adding timbre and encore. After doing a bunch of permutations of versions and settings I couldn't reproduce the issue.

Sadly going back to the original project and trying with anything but the snapshot you provided I am still getting the error. I am trying to figure out what the main differences are between this existing project and a fresh Luminus setup. But I am assuming that its something on my end and not yours.

[ptaoussanis]

Thanks for keeping me updated 👍

My best guess would still be that there's stale build artifacts squirrelled away somewhere. I'd try search in your base project dir for any .js files - might turn up an unexpected cache dir?

[BorisKourt]

I have just tested removing: .shadow-cljs, node_modules and target directories. resources/public/js only contains manually added libraries in this project. Searching for JS files in the project directory only yielded those.

With these deps I still get the same error:

   [com.taoensso/timbre "6.0.3" :exclusions [com.taoensso/encore]]
   [com.taoensso/encore "3.42.0"]]

With these I do not:

   [com.taoensso/timbre "6.0.2" :exclusions [com.taoensso/encore]]
   [com.taoensso/encore "3.37.1"]]

Its really weird that it persists if I change these back and forth.

[ptaoussanis]

Its really weird that it persists if I change these back and forth.

This is definitely unexpected.

If we've ruled out dependency conflict and stale build artifacts, then the only other explanation I can think of right now is that there's indeed a problem in Encore or Timbre, but it requires some specific context (e.g. compiler options and/or versions of other software) to show up.

I'm not so familiar with shadow-cljs though, and completely unfamiliar with luminus - so could be I'm missing something else.

Is it possible that your old project (the one exhibiting) problems is using an older version of shadow-cljs, or ClojureScript? Would it be possible to share the config related to your Cljs build?

I've just pushed [com.taoensso/encore "3.42.0-timbre364"] to Clojars. Could you let me know if that reliably works for you? (Should work with any Timbre version).

[BorisKourt]

Hey, I ran with:

   [com.taoensso/timbre "6.0.3" :exclusions [com.taoensso/encore]]
   [com.taoensso/encore "3.42.0-timbre364"]]

And it still gives me the error.

---

I am using the latest version of most libraries (regularly check via antq), unless there are breaking changes. So I am on latest shadow-cljs (which provides the CLJS compiler.)

I will try to see if I can duplicate this project and just strip away everything else. See if its something with the way its configured.

[ptaoussanis]

Are you by any chance using clj-kondo in your project that's experiencing the trouble?

[BorisKourt]

Hey, I have isolated it to just shadow, timbre and encore: https://github.com/BorisKourt/timbre-repro

Hopefully it will be possible to reproduce the error on your end. Steps:

• npm i
• lein uberjar

I am not sure if you need shadow-cljs installed globally or not.

On my machine that triggers the error every time.

---

I have removed all dev profiles and dependencies. So only lein uberjar will work. But that seems to be where the error happens consistently.

[ptaoussanis]

Can confirm that I'm seeing the same error with your repo and steps above 👍
Thanks for the repro! Will investigate and report back.

[ptaoussanis]

So that was quite a trip down the rabbit hole ^^

Would you please try one more time using [com.taoensso/encore "3.42.0-timbre364b"]?
That should hopefully finally fix the issue. This is working on my side.

Thanks again for the repro, that was a huge help - and I understand it must have taken a lot of effort to find.

Quick summary

It looks like there might be some sort of issue with shadow-cljs, or something else in your reproducible example (some config, interaction, etc.).

I've found what I hope is a reliable fix/workaround that can be made in Encore, though I don't understand exactly what the underlying issue is or why the workaround seems to help.

Digging into this further could be a lot of effort for not much benefit, but I'll leave some crumbs below for posterity or if someone else feels curious enough to dig.

Usual disclaimer that I may be missing something stupid/obvious!

Details

Basically the problem appears to be this:

If we have a form like (ns my-ns (:require [taoensso.encore :as enc :refer [have])), the ClojureScript compiler is generally able to infer that have is a macro.

However, somehow, the Clojure (not ClojureScript) defalias implementation in Encore seems to be tripping up this inference while compiling ClojureScript code.

The Invalid :refer, var taoensso.encore/have does not exist error appears to be caused by the compiler having the incorrect expectation that an enc/have var should be present in ClojureScript. But in fact, have is a macro - so only available in Clojure.

This confusion appears to happen only when using shadow-cljs, and even then only in specific circumstances (e.g. your specific repro), though I'm not certain what the key factors are.

BTW your repro example can actually be simplified and Timbre can be removed.
I've updated my example project to demonstrate both the simplified repro, and to confirm the fix below. (See the commit history for details).

The fix/workaround

It looks like adding an alter-meta! call to Encore's Clojure defalias implementation resolves the error.

I might be missing an obvious reason why the extra alter-meta! is necessary, but in any case it at least seems plausible that the extra call is necessary to reliably tag the var as a macro.

I'm not familiar with any of the relevant compiler internals, so can't speculate as to what might be going on. Encore's defalias is certainly doing some unusual/hacky things, so it wouldn't be surprising if it was tickling some sort of undefined/fragile behaviour.

[BorisKourt]

Thanks for taking the time on this, really appreciate it! That is quite an unexpected issue.

Everything appears to be working with [com.taoensso/encore "3.42.0-timbre364b"]!

I wonder who this should be passed on to? I can't quite tell if its shadow exposing a clojure bug or just a shadow issue.

[ptaoussanis]

You're very welcome. Likewise thanks for all the help, and sorry for all your time sunk on this.

[com.taoensso/encore "3.43.0"] and [com.taoensso/timbre "6.0.4"] are now on Clojars.

I'm inclined not to bother anyone else about this unless the problem recurs.
The issue was clearly either being caused by enc/defalias itself, or defalias was triggering some underlying cause elsewhere.

If there is an underlying cause, it seems likely that it's triggered only in obscure circumstances. defalias is definitely abusing the tooling, so it wouldn't surprise me if it touched an edge-case somewhere.

The current fix/workaround appears to be reliable, so I'm happy to leave it as is for now.

If the problem does recur, I'll dig further into what might actually be going on underneath - either to change the approach, or to ping the relevant upstream folks with clearer info.

Does that sound reasonable to you?

[BorisKourt]

That sounds totally reasonable. Lets close this one :)