Skip to content

circleci/lein-deps-plus

Repository files navigation

deps-plus plugin

deps-plus is a Leiningen plugin designed to help with analyzing project dependencies and reviewing dependency changes.

Installation

Add [com.circleci/deps-plus "0.1.0-SNAPSHOT"] to your :user profile plugins in ~/.lein/profiles.clj. For example,

{:user {:plugins [[com.circleci/deps-plus "0.1.0-SNAPSHOT"]]}}

Usage

To run a deps-plus task simply run lein deps-plus <task name> from a Leiningen project.

Tasks

  • list - Lists all dependencies of the current project.

  • list-save - Save the project dependency list to "deps.list".

  • list-diff - Compare project dependencies to those previously saved with list-save.

  • why - Shows all paths to given dependency as a tree. This is similar to lein deps :why, but instead of only showing the single resolved path to dependency chosen by Maven, all copies and paths to the dependency (including those excluded by conflict resolution) are shown.

Checks

check-classpath-conflicts

Checks for classpath conflicts. Classpath conflicts occur when resources which should be unique (e.g. Java class files, Clojure namespaces, etc) are found in multiple dependencies. Classpath conflicts can lead to unpredictable behavior at runtime as the version of the resource which is loaded can depend on subtle factors like the ordering of JARs on the classpath, or the order in which JARs are merged into an uberjar. Classpath conflicts are often caused by dependencies being duplicated (e.g. two versions of a dependency with different Maven coordinates) or dependencies which bundle other dependencies (e.g. shaded JARs). In these cases the conflict can be resolved by replacing the offending dependency with a more appropriate one (e.g. a non-shaded version).

Example: Apache logging classes

Found 6 classpath conflicts between commons-logging:commons-logging:jar:1.2:compile and org.slf4j:jcl-over-slf4j:jar:1.7.30:compile
  org/apache/commons/logging/Log.class
  org/apache/commons/logging/LogConfigurationException.class
  org/apache/commons/logging/LogFactory.class
  org/apache/commons/logging/impl/NoOpLog.class
  org/apache/commons/logging/impl/SimpleLog$1.class
  org/apache/commons/logging/impl/SimpleLog.class

These packages both implement the same classes, but the latter implements the former with SLF4J.

Solution: Pick which package should be chosen (there’s no practical way to tell which version of the class has been chosen in the past). In this case, we’ll pick org.slf4j/jcl-over-slf4j. So we add a top level exclusion for commons-logging/commons-logging, preventing any dependency from pulling it in:

  :exclusions [... commons-logging/commons-logging ...]

Example: com.google.javascript:closure-compiler

The jar for the Closure Compiler includes a shaded version of Gson without renaming the classes:

$ jar tf ~/.m2/repository/com/google/javascript/closure-compiler/v20160315/closure-compiler-v20160315.jar | grep 'com/google/gson'
com/google/gson/
com/google/gson/Gson$5.class
com/google/gson/JsonNull.class
...

This always causes classpath conflicts when we also declare a direct dependency on Gson, even if the version of Gson is the same in both places. You’ll have to deal with this type of conflict on a case by case basis. Here, there is an unshaded version of the Closure Compiler available that does not include the Gson classes.

check-downgraded

A dependency is considered "downgraded" if the resolved version is older than the version specified by some other included dependency. This can occur if :managed-dependencies is used to pin a dependency version or if Maven conflict resolution happens to not select the latest version.

check-families

Checks for inconsistent dependency versions within families. Dependencies families are sets of dependencies which share a common group ID and version. Inconsistent family versions can occur if the :managed-dependencies for a family are incomplete or inconsistent. This situation can also occur when different family members are brought in through different transitive dependency paths. To fix this issue consider adding a complete set of :managed-dependencies entries for the family.

Example: io.netty family

Suspicious combination, io.netty:netty-codec-socks:jar:4.1.34.Final:compile and io.netty:netty-buffer:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-codec-socks:jar:4.1.34.Final:compile and io.netty:netty-codec:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-codec-socks:jar:4.1.34.Final:compile and io.netty:netty-common:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-codec-socks:jar:4.1.34.Final:compile and io.netty:netty-transport:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-handler-proxy:jar:4.1.34.Final:compile and io.netty:netty-buffer:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-handler-proxy:jar:4.1.34.Final:compile and io.netty:netty-codec-http:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-handler-proxy:jar:4.1.34.Final:compile and io.netty:netty-codec:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-handler-proxy:jar:4.1.34.Final:compile and io.netty:netty-common:jar:4.1.50.Final:compile
Suspicious combination, io.netty:netty-handler-proxy:jar:4.1.34.Final:compile and io.netty:netty-transport:jar:4.1.50.Final:compile

These netty dependencies are all in the same family, but they have different versions (4.1.34.Final vs. 4.1.50.Final). In this case, the project.clj declares a managed dependency on io.netty/netty-codec-http2 but not the other members of the family:

  :managed-dependencies [... [io.netty/netty-codec-http2 "4.1.50.Final"] ...]

The related projects are coming in through transitive dependencies, and have different versions:

$ lein deps-plus why io.netty:netty-codec-socks:jar:4.1.34.Final
...
   ;; +- [circleci/my-project "0.1.0"]
...
   ;; |  \- [io.grpc/grpc-netty "1.20.0"]
   ;; |     \- [io.netty/netty-handler-proxy "4.1.34.Final"]
   ;; |        \- [io.netty/netty-codec-socks "4.1.34.Final"] (chosen)
...

Solution: add a managed dependency for the family. Pick the desired version for the family (in this case, we’ll pick 4.1.50.Final). At a minimum, for every package mentioned in the list of suspicious combinations that has the undesired version, add an entry in the :managed-dependencies:

  :managed-dependencies [[io.netty/netty-codec-socks "4.1.50.Final"]
                         [io.netty/netty-handler-proxy "4.1.50.Final"]
                         ...]

For completeness, you can add every member of the family the project uses to managed dependencies.

check-management-conflicts

Checks for dependency management conflicts. A dependency management conflict occurs when a dependency has versions specified in both :dependencies and :managed-dependencies. To resolve this issue you can remove the version number from :dependencies. If you wish to override a managed dependency version inherited from a parent project you should do so in your own :managed-dependencies section.

Example: org.clojure/core.async

org.clojure:core.async:jar:1.2.603 conflicts with managed dependency org.clojure:core.async:jar:1.3.610

Solution 1: if the exact version of core.async does not matter, remove the version number from the org.clojure/core.async version in your dependencies to automatically get the version provided by clj-parent:

  :dependencies [... [org.clojure/core.async] ...]

This solution also applies when the versions are identical:

org.clojure:core.async:jar:1.3.610 conflicts with managed dependency org.clojure:core.async:jar:1.3.610

Solution 2: if it is necessary to pin the version 1.2.603, move the dependency to the managed dependencies:

  :managed-dependencies [... [org.clojure/core.async "1.2.603"] ...]

check-pedantic

Checks for version conflicts and version ranges. This check is similar to Leiningen’s :pedantic? :abort mode, but suggests :managed-dependencies instead of :exclusions. In general, expect to see warnings when:

  1. A top-level dependency is overridden by another version

  2. A transitive dependency is overridden by an older version

Unlike Leiningen, this task ignores plugin dependencies since these are unaffected by managed dependencies. By default, each suggested managed dependency is shown alongside a dependency tree for the conflict. Pass the :quiet flag to suppress the output of these trees.

Example: cheshire

Found 7 dependency conflicts.
Considering adding the following :managed-dependencies,

...
   ;; +- [cheshire/cheshire "5.9.0"] (chosen)
...
   ;; \- [circleci/my-project "0.1.0"]
   ;;    +- [circleci/the-other-project "0.1.0"]
   ;;    |  +- [circleci/rollcage "1.0.203"]
   ;;    |  |  \- [cheshire/cheshire "5.8.1"] (omitted)
   ;;    |  +- [cheshire/cheshire "5.10.0"] (omitted)
   ;;    |  \- [amperity/vault-clj "0.7.0"]
   ;;    |     \- [cheshire/cheshire "5.8.1"] (omitted)
   ;;    \- [cheshire/cheshire "5.10.0"] (omitted)
   [cheshire/cheshire "5.9.0"]
...

This shows all of the different versions of cheshire/cheshire, including which versions were chosen (would actually be used when the program runs) vs. which were excluded. check-pedantic complains because multiple dependencies ask for different versions of cheshire/cheshire, and the newest version (transitively "5.10.0"), is omitted.

Solution 1: if the version of cheshire/cheshire from the :dependencies is not required for correctness, remove it as an explicit dependency and retry. If the warning disappears, you can see that the newest version wins with why:

$ lein deps-plus why cheshire
   ;; +- [circleci/the-other-project "0.1.0"]
   ;; |  +- [circleci/rollcage "1.0.203"]
   ;; |  |  \- [cheshire/cheshire "5.8.1"] (omitted)
   ;; |  +- [cheshire/cheshire "5.10.0"] (chosen)
   ;; |  \- [amperity/vault-clj "0.7.0"]
   ;; |     \- [cheshire/cheshire "5.8.1"] (omitted)
...

Solution 2: add a managed dependency for the preferred version. Pick the version that should be included (in this case, we’ll pick "5.9.0"). This is the version check-pedantic suggests at the bottom of the dependency knot. It’s also the version that the project explicitly requires as a :dependency. Move it to a managed dependency:

  :managed-dependencies [... [cheshire/cheshire "5.9.0"] ...]

Releasing

deps-plus is pushed to clojars.org as a SNAPSHOT release.

Bump the version only when backwards-incompatible changes are made.

The following should be updated on the main branch if there are new releases:

  • project.clj - version

  • README.adoc - dependency coordinates

  • CHANGELOG.adoc - summary of changes

License

Distributed under the Eclipse Public License.