Allow specifying package versions

[mbauman]

It'd be really cool to allow the following

@require DataFrames > v"0.6.1" begin
    # blah blah blah
end

I think this should be possible, but it'll take some work to augment the data structures and method signatures to allow the optional version number. The macro parsing could also get complicated quickly, since if you give a mouse a cookie, the mouse will soon want v"0.5.1" < DataFrames <= v"0.6.1".

[MikeInnes]

What kind of use case are you thinking of?

You should be able to already do something like:

@require DataFrames if v"0.5.1" < Pkg.installed("DataFrames") <= v"0.6.1"
  # blah blah blah
end

which might be enough.

[mbauman]

Yes, that would work just as well with a lot less work. Since we can't specify versions in REQUIRE for lazy (and non-required) modules loaded by @require, I just wanted to fail gracefully when I start depending upon new module features.

[MikeInnes]

Ok, makes sense – I'll add that snippet to the readme when I get a chance.

[oxinabox]

I think adding that as sugar to the macro might be nice.

It also might be nice to have with it a flag to give a warning

by having the result be:

@require DataFrames if v"0.5.1" < Pkg.installed("DataFrames") <= v"0.6.1"
  # blah blah blah
else
    warn("DataFrames version: " * Pkg.installed("DataFrames")) * "was loaded. * current_module() * has additional functionality for DataFrames; but only suports v"0.5.1" < version <= v"0.6.1")
end

But with that said It is no pressing feature, I don't have any usecase blocked by it personally

[oxinabox]

Latest hack for this:
invenia/NamedDims.jl#54

@require Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" let
    version = Pkg.installed()["Tracker"]
    allowed_versions = Pkg.Types.semver_spec("2.2")
    if tracker_version ∈ allowed_versions
        include("tracker_compat.jl")
    else
        @warn "Tracker version not compatible with NamedDims. Tracker Compatability functionality has been disabled." version allowed_versions
    end
end

[oxinabox]

@KristofferC pointed out that the above doesn't work,
for your dependencies dependencies.

Because Pkg.installed() only reads Project.toml

Here is my latest alternativive to Pkg.installed() for finding verisions.

using Pkg
function pkg_version(_module::Module)
    pkg_id = Base.PkgId(_module)
    env = Pkg.Types.Context().env
    pkg_info = Pkg.Types.manifest_info(env, pkg_id.uuid)
    return VersionNumber(pkg_info["version"])
end

Note it working on CategoricalArrays which I only have via DataFrames.jl

julia> Pkg.installed()["CategoricalArrays"]
ERROR: KeyError: key "CategoricalArrays" not found
Stacktrace:
 [1] getindex(::Dict{String,Union{Nothing, VersionNumber}}, ::String) at ./dict.jl:478
 [2] top-level scope at none:0

julia> pkg_version(DataFrames.CategoricalArrays)
v"0.5.2"

[oxinabox]

How do we feel about the API:

@require Tracker="9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c" compat="~1.2, 2" warn=false begin
      include("trackercompath.jl")
end

Components:

• Pkgname & UUID
• Optional: compat bounds, which use the same rules as the compat in Project.toml.
• Optional: warn=false, to disable a warning being printed. Where the warning would include:
  • Required package name
  • Required package actual version
  • Requiring package name (from Base.moduleroot(__module__))
  • Allowed versions
• Body to run if accepted.

[KristofferC]

If people start using this it will get extremely annoying. No longer will the resolver give you a set of versions that are specified to work with each other, you will at runtime get an error message that a version that you use is not supported. So you pin that or something and then another package starts complaining. There is no longer a central place where package compatibility is recorded. Dependency hell v2, here we come.

Shudder.

[oxinabox]

I agree, but...
Almost all uses of Requires.jl right now already live in that world though.
Only without a feature to check what the version is,
you don't get told that it isn't compatible,
it just fails in unexpected ways.

Some times it should be possibly to gracefully fail the version check,
and then can do warn=false. and it would disable features.
We probably do need the ability to do multiple different compat specifiers.
So that one can do one thing for one version and different for another.

But I agree, determining compability at run time, rather than install time, is not the long term solution.
We do need to solve the conditional/optional dependency problem in Pkg.

[MikeInnes]

Yeah, I agree 100% with Kristoffer's analysis, but at the same time you have to compare to the right counterfactual -- the situation we have right now is not actually better, just more implicit, and packages that need this are going to find some way to do it whether it's in Requires or not. And we'll be in that situation as long as the response from Pkg/Base folks is just "you shouldn't do this" without a better forward path.

That being said, there's potentially a big difference between making something possible and making it easy. Maybe this should just remain something that's unsupported, undocumented and spoken of in hushed tones. That avoids people thinking it's a good idea because there's a convenient syntax for it.

[alyst]

Does Project.toml allow specifying compat for packages that are not in deps? That could be an alternative (and no version checks in Requires.jl). Or an explicit optcompat section.

[oxinabox]

Does Project.toml allow specifying compat for packages that are not in deps? That could be an alternative (and no version checks in Requires.jl). Or an explicit optcompat section.

No, it does not.
It would be an interesting option.

[oxinabox]

This is still hurting me,
if I made a PR to add pkg_version(Module) to Requires.jl,
and a example to the docs with a warning saying
"this API is exerimental and still being investigated, please comment in this issue."