Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parameter identifiability example from docs fails #2881

Closed
moble opened this issue Jul 20, 2024 · 5 comments
Closed

Parameter identifiability example from docs fails #2881

moble opened this issue Jul 20, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@moble
Copy link

moble commented Jul 20, 2024

Describe the bug 🐞

Following the example on this page results in

ERROR: Measured quantities (output functions) were not provided and no outputs were found.

I'm not sure that this package is really to blame. Maybe @pogudingleb could provide some insight?

Expected behavior

I would have thought that marking y1(t) and y2(t) with [output = true] would be enough to communicate that these are outputs.

Minimal Reproducible Example 👇

I've just copied the example code, but run it in a temp environment:

using Pkg
Pkg.activate(; temp=true)
Pkg.add(["StructuralIdentifiability", "ModelingToolkit"])


using StructuralIdentifiability, ModelingToolkit
using ModelingToolkit: t_nounits as t, D_nounits as D

@mtkmodel Biohydrogenation begin
    @variables begin
        x4(t)
        x5(t)
        x6(t)
        x7(t)
        y1(t), [output = true]
        y2(t), [output = true]
    end
    @parameters begin
        k5
        k6
        k7
        k8
        k9
        k10
    end
    # define equations
    @equations begin
        D(x4) ~ -k5 * x4 / (k6 + x4)
        D(x5) ~ k5 * x4 / (k6 + x4) - k7 * x5 / (k8 + x5 + x6)
        D(x6) ~ k7 * x5 / (k8 + x5 + x6) - k9 * x6 * (k10 - x6) / k10
        D(x7) ~ k9 * x6 * (k10 - x6) / k10
        y1 ~ x4
        y2 ~ x5
    end
end

# define the system
@mtkbuild de = Biohydrogenation()

# query local identifiability
# we pass the ode-system
local_id_all = assess_local_identifiability(de, prob_threshold = 0.99)

Error & Stacktrace ⚠️

ERROR: Measured quantities (output functions) were not provided and no outputs were found.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] macro expansion
   @ ~/.julia/packages/StructuralIdentifiability/GXfvN/ext/ModelingToolkitSIExt.jl:299 [inlined]
 [3] _assess_local_identifiability(ode::ODESystem; measured_quantities::Vector{…}, funcs_to_check::Vector{…}, prob_threshold::Float64, type::Symbol)
   @ ModelingToolkitSIExt ~/.julia/packages/TimerOutputs/Lw5SP/src/TimerOutput.jl:237
 [4] _assess_local_identifiability
   @ ~/.julia/packages/TimerOutputs/Lw5SP/src/TimerOutput.jl:230 [inlined]
 [5] (::ModelingToolkitSIExt.var"#42#43"{Vector{Array{Equation}}, Vector{Array}, Float64, Symbol, ODESystem})()
   @ ModelingToolkitSIExt ~/.julia/packages/StructuralIdentifiability/GXfvN/ext/ModelingToolkitSIExt.jl:274
 [6] with_logstate(f::Function, logstate::Any)
   @ Base.CoreLogging ./logging.jl:515
 [7] with_logger
   @ ./logging.jl:627 [inlined]
 [8] assess_local_identifiability(ode::ODESystem; measured_quantities::Vector{…}, funcs_to_check::Vector{…}, prob_threshold::Float64, type::Symbol, loglevel::Base.CoreLogging.LogLevel)
   @ ModelingToolkitSIExt ~/.julia/packages/StructuralIdentifiability/GXfvN/ext/ModelingToolkitSIExt.jl:273
 [9] top-level scope
   @ ~/.julia/dev/PostNewtonian/notes/modeling_toolkit/structural_identifiability_bug.jl:42
Some type information was truncated. Use `show(err)` to see complete types.

Environment (please complete the following information):

  • Output of using Pkg; Pkg.status()
Status `/private/var/folders/bh/rg4bdsl965l0572z3vtmwkr40000gn/T/jl_B1llWc/Project.toml`
  [961ee093] ModelingToolkit v9.25.0
  [220ca800] StructuralIdentifiability v0.5.8
  • Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
Status `/private/var/folders/bh/rg4bdsl965l0572z3vtmwkr40000gn/T/jl_B1llWc/Manifest.toml`
  [47edcb42] ADTypes v1.6.1
⌅ [c3fe647b] AbstractAlgebra v0.41.10
  [1520ce14] AbstractTrees v0.4.5
  [7d9f7c33] Accessors v0.1.36
  [79e6a3ab] Adapt v4.0.4
  [66dad0bd] AliasTables v1.1.3
  [ec485272] ArnoldiMethod v0.4.0
  [4fba245c] ArrayInterface v7.12.0
  [4c555306] ArrayLayouts v1.10.2
  [a9b6321e] Atomix v0.1.0
  [e2ed5e7c] Bijections v0.1.7
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [2a0fbf3d] CPUSummary v0.2.6
  [00ebfdb7] CSTParser v3.4.3
  [49dc2e85] Calculus v0.5.1
  [d360d2e6] ChainRulesCore v1.24.0
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [861a8166] Combinatorics v1.0.2
  [a80b9123] CommonMark v0.8.12
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.0
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.15.0
  [b152e2b5] CompositeTypes v0.1.4
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.5
  [adafc99b] CpuId v0.3.1
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [2b5f629d] DiffEqBase v6.151.5
  [459566f4] DiffEqCallbacks v3.6.2
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.5.9
  [31c24e10] Distributions v0.25.109
  [ffbed154] DocStringExtensions v0.9.3
  [5b8099bc] DomainSets v0.7.14
  [fa6b7ba4] DualNumbers v0.6.8
⌅ [7c1d4256] DynamicPolynomials v0.5.7
⌅ [06fc5a27] DynamicQuantities v0.13.2
  [4e289a0a] EnumX v1.0.4
  [f151be2c] EnzymeCore v0.7.7
  [e2ba6199] ExprTools v0.1.10
  [7034ab61] FastBroadcast v0.3.4
  [9aa1b823] FastClosures v0.3.2
  [29a986be] FastLapackInterface v2.0.4
  [1a297f60] FillArrays v1.11.0
  [64ca27bc] FindFirstFunctions v1.2.0
  [6a86dc24] FiniteDiff v2.23.1
  [1fa38f19] Format v1.3.7
  [f6369f11] ForwardDiff v0.10.36
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [d9f16b24] Functors v0.4.11
  [46192b85] GPUArraysCore v0.1.6
  [c27321d9] Glob v1.3.1
  [86223c79] Graphs v1.11.2
  [0b43b601] Groebner v0.7.5
  [3e5b6fbb] HostCPUFeatures v0.1.17
  [34004b35] HypergeometricFunctions v0.3.23
  [615f187c] IfElse v0.1.1
  [d25df0c9] Inflate v0.1.5
  [18e54dd8] IntegerMathUtils v0.1.2
  [8197267c] IntervalSets v0.7.10
  [3587e190] InverseFunctions v0.1.15
  [92d709cd] IrrationalConstants v0.2.2
  [c8e1da08] IterTools v1.10.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.5.0
  [682c06a0] JSON v0.21.4
  [98e50ef6] JuliaFormatter v1.0.58
  [ccbc3e58] JumpProcesses v9.11.1
  [ef3ab10e] KLU v0.6.0
  [ba0b0d4f] Krylov v0.9.6
  [b964fa9f] LaTeXStrings v1.3.1
  [2ee39098] LabelledArrays v1.16.0
  [984bce1d] LambertW v0.4.6
  [23fbe1c1] Latexify v0.16.4
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.1.9
  [d3d80556] LineSearches v7.2.0
  [7ed4a6bd] LinearSolve v2.30.2
  [2ab3a3ac] LogExpFunctions v0.3.28
  [bdcacae8] LoopVectorization v0.12.171
  [d8e11817] MLStyle v0.4.17
  [1914dd2f] MacroTools v0.5.13
  [d125e4d3] ManualMemory v0.1.8
  [bb5d69b7] MaybeInplace v0.1.3
  [e1d29d7a] Missings v1.2.0
  [961ee093] ModelingToolkit v9.25.0
  [46d2c3a1] MuladdMacro v0.2.4
  [102ac46a] MultivariatePolynomials v0.5.6
  [d8a4904e] MutableArithmetics v1.4.5
  [d41bc354] NLSolversBase v7.8.3
  [77ba4419] NaNMath v1.0.2
⌅ [2edaba10] Nemo v0.45.7
  [8913a72c] NonlinearSolve v3.13.1
  [6fe1bfb0] OffsetArrays v1.14.1
  [bac558e1] OrderedCollections v1.6.3
  [90014a1f] PDMats v0.11.31
  [65ce6f38] PackageExtensionCompat v1.0.2
  [3e851597] ParamPunPam v0.4.1
  [d96e819e] Parameters v0.12.3
  [69de0a69] Parsers v2.8.1
  [e409e4f3] PoissonRandom v0.4.4
  [f517fe37] Polyester v0.7.15
  [1d0040c9] PolyesterWeave v0.2.2
  [d236fae5] PreallocationTools v0.4.22
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [08abe8d2] PrettyTables v2.3.2
  [27ebfcd6] Primes v0.5.6
  [92933f4c] ProgressMeter v1.10.2
  [43287f4e] PtrArrays v1.2.0
  [1fd47b50] QuadGK v2.9.4
  [fb686558] RandomExtensions v0.4.4
  [e6cf234a] RandomNumbers v1.5.3
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.26.0
  [f2c3362d] RecursiveFactorization v0.2.23
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [79098fc4] Rmath v0.7.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.43
  [0bca4576] SciMLBase v2.43.0
  [c0aeaf25] SciMLOperators v0.3.8
  [53ae85a6] SciMLStructures v1.4.1
  [efcf1570] Setfield v1.1.1
  [727e6d20] SimpleNonlinearSolve v1.10.1
  [699a6c99] SimpleTraits v0.9.4
  [a2af1166] SortingAlgorithms v1.2.1
  [47a9eef4] SparseDiffTools v2.19.0
  [0a514795] SparseMatrixColorings v0.3.5
  [e56a9233] Sparspak v0.3.9
  [276daf66] SpecialFunctions v2.4.0
  [aedffcd0] Static v1.1.1
  [0d7ed370] StaticArrayInterface v1.5.1
  [90137ffa] StaticArrays v1.9.7
  [1e83bf80] StaticArraysCore v1.4.3
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.3
  [4c63d2b9] StatsFuns v1.3.1
  [7792a7ef] StrideArraysCore v0.5.7
  [892a3eda] StringManipulation v0.3.4
  [220ca800] StructuralIdentifiability v0.5.8
  [2efcf032] SymbolicIndexingInterface v0.3.26
  [19f23fe9] SymbolicLimits v0.2.1
  [d1185830] SymbolicUtils v2.1.2
  [0c5d862f] Symbolics v5.34.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
⌅ [8ea1fca8] TermInterface v0.4.1
  [8290d209] ThreadingUtilities v0.5.2
  [a759f4b9] TimerOutputs v0.5.24
  [0796e94c] Tokenize v0.5.29
  [d5829a12] TriangularSolve v0.2.1
  [410a4b4d] Tricks v0.1.8
  [781d530d] TruncatedStacktraces v1.4.0
  [5c2747f8] URIs v1.5.1
  [3a884ed6] UnPack v1.0.2
  [1986cc42] Unitful v1.21.0
  [a7c27f48] Unityper v0.1.6
  [013be700] UnsafeAtomics v0.2.1
  [3d5dd08c] VectorizationBase v0.21.70
  [19fa3120] VertexSafeGraphs v0.2.0
  [e134572f] FLINT_jll v300.100.300+0
  [1d5cc7b8] IntelOpenMP_jll v2024.2.0+0
  [856f044c] MKL_jll v2024.2.0+0
⌅ [656ef2d0] OpenBLAS32_jll v0.3.24+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.4.2+0
  [1317d2d5] oneTBB_jll v2021.12.0+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [781609d7] GMP_jll v6.2.1+6
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [3a97d323] MPFR_jll v4.2.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.8.0+1
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
  • Output of versioninfo()
Julia Version 1.10.4
Commit 48d4fd48430 (2024-06-04 10:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 12 × Apple M2 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 8 default, 0 interactive, 4 GC (on 8 virtual cores)
Environment:
  JULIA_EDITOR = code
@moble moble added the bug Something isn't working label Jul 20, 2024
@pogudingleb
Copy link

@moble
Thanks for reporting! This is indeed a big in the interface, this particular way of defining models does not quite work in the current versions. I am refactoring right now ModelingToolkit interface, I will fix this as well.

For the time being, you can just pass the observed quantities to the function directly like this

using StructuralIdentifiability, ModelingToolkit
using ModelingToolkit: t_nounits as t, D_nounits as D

@mtkmodel Biohydrogenation begin
    @variables begin
        x4(t)
        x5(t)
        x6(t)
        x7(t)
        y1(t), [output = true]
        y2(t), [output = true]
    end
    @parameters begin
        k5
        k6
        k7
        k8
        k9
        k10
    end
    # define equations
    @equations begin
        D(x4) ~ -k5 * x4 / (k6 + x4)
        D(x5) ~ k5 * x4 / (k6 + x4) - k7 * x5 / (k8 + x5 + x6)
        D(x6) ~ k7 * x5 / (k8 + x5 + x6) - k9 * x6 * (k10 - x6) / k10
        D(x7) ~ k9 * x6 * (k10 - x6) / k10
    end
end

# define the system
@mtkbuild de = Biohydrogenation()

x4, x5, x6, x7 = unknowns(de)

# query local identifiability
# we pass the ode-system
local_id_all = assess_local_identifiability(de, measured_quantities = [x4, x5], prob_threshold = 0.99)

@moble
Copy link
Author

moble commented Jul 20, 2024

Thanks for looking into this, Gleb! Since posting this, I took a look too and found ModelingToolkit.observed. Is this what you need? In particular, could you change the signatures in your extension to use that as a default value, like this:

function StructuralIdentifiability.find_identifiable_functions(
    ode::ModelingToolkit.ODESystem;
    measured_quantities = ModelingToolkit.observed(ode),

Thanks for your help!

@pogudingleb
Copy link

Thanks for looking into this, Gleb! Since posting this, I took a look too and found ModelingToolkit.observed. Is this what you need? In particular, could you change the signatures in your extension to use that as a default value, like this:

function StructuralIdentifiability.find_identifiable_functions(
    ode::ModelingToolkit.ODESystem;
    measured_quantities = ModelingToolkit.observed(ode),

Thanks for your help!

Thanks! We indeed used observed before (but in a less nice way as you proposed) but this code was lost in some transformations and, apparently, was not covered in the tests. I will fix this for the next release.

@pogudingleb pogudingleb mentioned this issue Aug 17, 2024
Merged
3 tasks
@pogudingleb
Copy link

@moble Thanks again for reporting this!
The example should work with the new release of StructuralIdentifiability (0.5.9) as you wrote it originally.

@moble
Copy link
Author

moble commented Sep 10, 2024

Works like a charm. Thanks!

@moble moble closed this as completed Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@moble @pogudingleb