From 4ee5db5ee514bf750d8c576f59d2f948a397230c Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Sun, 10 May 2020 15:46:38 -0500 Subject: [PATCH 01/21] WIP: blog post on invalidations --- blog/2020/05/invalidations.md | 518 ++++++++++++++++++++++++++++++++++ 1 file changed, 518 insertions(+) create mode 100644 blog/2020/05/invalidations.md diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md new file mode 100644 index 0000000000..df82993dc7 --- /dev/null +++ b/blog/2020/05/invalidations.md @@ -0,0 +1,518 @@ +@def authors = "Tim Holy" +@def published = "10 May 2020" +@def title = "Analyzing sources of compiler latency in Julia: method invalidations" +@def rss_pubdate = Date(2020, 5, 10) +@def rss = """Julia is fast, but compiling Julia code takes time. This post analyzes why it's sometimes necessary to repeat that work, and what might be done to fix it.""" + + +[The Julia programming language][Julia] has wonderful flexibility with types, and this allows you to combine packages in unanticipated ways to solve new kinds of problems. +Crucially, it achieves this flexibility without sacrificing performance. +It does this by running versions of code or algorithms that have been "specialized" for the specific types you are using. +Creating these specializations is compilation, and when done on the fly (as is common in Julia) this is called "just in time" (JIT) compilation. +Unfortunately, JIT-compilation takes time, and this contributes to *latency* +when you first run Julia code. +This problem is often summarized as "time-to-first-plot," though there is nothing specific about plotting other than the fact that plotting libraries tend to involve large code bases, and these must be JIT-compiled. + +Many people have spent a lot of time analyzing and reducing Julia's latency. +These efforts have met with considerable success: the upcoming Julia 1.5 feels "snappier" than any recent version I've used. +But the job of reducing latency is not over yet. +Recently I got interested in a specific source of this latency, and this blog post is a summary of some of what I've learned about the scope of this problem and opportunities for further improvement. + +# Method invalidation: what is it and when does it happen? + +## An example: compilation in the presence of Union-splitting + +When Julia compiles a method for specific types, it saves the resulting code +so that it can be used by any caller. +This is crucial to performance, because it means generally compilation has to be done only once for a specific type. +To keep things really simple, I'm going to use a very artificial example. + +```julia +f(x::Int) = 1 +f(x::Bool) = 2 + +function applyf(container) + x1 = f(container[1]) + x2 = f(container[2]) + return x1 + x2 +end +``` + +If you call `applyf([100, 200])`, it will compile a version of `applyf` +knowing that only `f(::Int)` will be used: the call will be "hard-wired" into the code that Julia produces. +You can see this using `@code_typed`: + +``` +julia> @code_typed applyf([100,200]) +CodeInfo( +1 ─ Base.arrayref(true, container, 1)::Int64 +│ Base.arrayref(true, container, 2)::Int64 +└── return 2 +) => Int64 +``` + +The compiler itself knows that the answer will be 2, as long as the input array +has elements indexable by 1 and 2. (Those `arrayref` statements enforce +bounds-checking, and ensure that Julia will throw an appropriate error if you +call `applyf([100])`.) + +If you pass a `Vector{Bool}`, it will compile `applyf` again, this time specializing it for `x::Bool`: + +``` +julia> @code_typed applyf([true,false]) +CodeInfo( +1 ─ Base.arrayref(true, container, 1)::Bool +│ Base.arrayref(true, container, 2)::Bool +└── return 4 +) => Int64 +``` + +In this case, you can see that Julia knew those two `arrayref` statements would +return a `Bool`, and since it knows the value of `f(::Bool)` it just went +ahead and computed the result for you. + +At the end of these experiments, hidden away in Julia's "method cache" there will +be two `MethodInstance`s of `applyf`, one specialized for `Vector{Int}` and the other specialized for `Vector{Bool}`. +You don't normally see these, but Julia manages them for you; anytime you write +code that calls `applyf`, it checks to see if this previous compilation work can be reused. + +For the purpose of this blog post, things start to get especially interesting if we try the following: +``` +julia> c = Any[1, false]; + +julia> applyf(c) +3 + +julia> @code_typed applyf(c) +CodeInfo( +1 ── %1 = Base.arrayref(true, container, 1)::Any +│ %2 = (isa)(%1, Bool)::Bool +└─── goto #3 if not %2 +2 ── goto #6 +3 ── %5 = (isa)(%1, Int64)::Bool +└─── goto #5 if not %5 +4 ── goto #6 +5 ── %8 = Main.f(%1)::Int64 +└─── goto #6 +6 ┄─ %10 = φ (#2 => 2, #4 => 1, #5 => %8)::Int64 +│ %11 = Base.arrayref(true, container, 2)::Any +│ %12 = (isa)(%11, Bool)::Bool +└─── goto #8 if not %12 +7 ── goto #11 +8 ── %15 = (isa)(%11, Int64)::Bool +└─── goto #10 if not %15 +9 ── goto #11 +10 ─ %18 = Main.f(%11)::Int64 +└─── goto #11 +11 ┄ %20 = φ (#7 => 2, #9 => 1, #10 => %18)::Int64 +│ %21 = Base.add_int(%10, %20)::Int64 +└─── return %21 +) => Int64 +``` + +This may seem a lot more complicated, but the take-home message is actually +quite simple. First, look at those `arrayref` statements: they are annotated +`Any`, meaning that Julia can't predict in advance what they will return. +Immediately after each reference, notice that there are two `isa` statements +followed by a `ϕ`; all of this is essentially equivalent to + +```julia +if isa(x, Bool) + value = 2 +elseif isa(x, Int) + value = 1 +else + value = f(x)::Int +end +``` + +This is [union-splitting], a key optimization for performance in the face of uncertain types. + +## Triggering method invalidation + +One point in particular is worth noting: what's up with that final `f(x)::Int` call? +Didn't it already handle all the possibilities? +Well, all the possibilities *so far*. +But we might next define some new method of `f` for a different type: + +``` +f(::String) = 3 +``` + +and Julia has prepared the way to make sure it will still give the right answer even if we add new methods to `f`. + +However, if you try `@code_typed applyf(c)` again, you'll notice something curious: +Julia has gone to the trouble to create a new-and-improved implementation of `applyf`, +one which also union-splits for `String`. +This brings us to the topic of this blog post: the old compiled method has been *invalidated*. +Given new information---which here comes from defining or loading new methods--- +Julia changes its mind about how things should be implemented, +and this forces Julia to recompile `applyf`. + +If you add fourth and fifth methods, + +``` +f(::AbstractArray) = 4 +f(::Missing) = 5 +``` + +then Julia produces + +``` +julia> @code_typed applyf(c) +CodeInfo( +1 ─ %1 = Base.arrayref(true, container, 1)::Any +│ %2 = Main.f(%1)::Any +│ %3 = Base.arrayref(true, container, 2)::Any +│ %4 = Main.f(%3)::Any +│ %5 = (%2 + %4)::Any +└── return %5 +) => Any +``` + +There are now so many possibilities that Julia just gives up and +uses "runtime dispatch" to decide what method of `f` to call. +It doesn't even try to enforce the fact that `f` returns and `Int`, +in part because determining such facts takes time (adding to compiler latency) +and because functions with many methods typically tend to return multiple types +anyway. + +Compiling each of these new implementations takes JIT-time. +If Julia knew in advance that you'd arrive at this place, it would never have bothered to produce that first, heavily-optimized version of `applyf`. +But the performance benefits of such optimizations are so large that, when applicable, they are well worth it. +For example, if you start a fresh Julia session and just define the `f(::Int)` +and `f(::Bool)` methods, then + +``` +julia> using BenchmarkTools + +julia> @btime applyf($c) + 4.659 ns (0 allocations: 0 bytes) +3 + +julia> f(::String) = 3 +f (generic function with 3 methods) + +julia> f(::AbstractArray) = 4 +f (generic function with 4 methods) + +julia> f(::Missing) = 5 +f (generic function with 5 methods) + +julia> @btime applyf($c) + 33.537 ns (0 allocations: 0 bytes) +3 +``` +It's almost a tenfold difference. +If `applyf` is performance-critical, you'll be very happy that Julia tries to give you the best version it can, given the available information. +But this leaves the door open to invalidation, which means recompilation the next time you use `applyf`. +If method invalidation happens often, this might contribute to making Julia "feel" sluggish. + +# How common is method invalidation? + +Unfortunately, method invalidation is pretty common. +First, let's get some baseline statistics. +Using the [MethodAnalysis] package (which is at a very early stage of development +at the time of this writing), you can find out that a fresh Julia session +(albeit one that has loaded the MethodAnalysis package and used it to perform some analysis) has almost 50,000 `MethodInstance`s tucked away in its cache. +These are mostly for `Base` and the standard libraries. + +Using some not-yet merged work in both Julia itself and [SnoopCompile], we can count the number of invalidations when we load various packages into a fresh Julia session: + +| Package | # of unique invalidations | +|:------- | ------------------:| +| Example | 0 | +| Revise | 27 | +| FixedPointNumbers | 429 | +| SIMD | 2799 | +| StaticArrays | 2852 | +| Optim | 3171 | +| Images | 3638 | +| Flux | 3697 | +| Plots | 4002 | +| DataFrames | 4048 | +| JuMP | 4666 | +| Makie | 6118 | +| DifferentialEquations | 6777 | + +You can see that key packages used by large portions of the Julia ecosystem invalidate +hundreds or thousands of MethodInstances, sometimes more than 10% of the total +number of MethodInstances present before loading the package. + +# How serious is method invalidation? + +The next time you want to call functionality that gets invalidated, +you have to wait for recompilation. +We can illustrate this using everyone's favorite example, plotting: + +```julia +julia> using Plots + +julia> @time display(plot(rand(5))) + 7.717729 seconds (15.27 M allocations: 797.207 MiB, 3.59% gc time) +``` + +As is well known, it's much faster the second time, because it's already compiled: + +```julia +julia> @time display(plot(rand(5))) + 0.311226 seconds (19.93 k allocations: 775.055 KiB) +``` + +Now load a package that does a lot of invalidation, and try again: + +```julia +julia> using SIMD + +julia> @time display(plot(rand(5))) + 7.238336 seconds (26.50 M allocations: 1.338 GiB, 7.88% gc time) +``` + +Because so much got invalidated by loading SIMD, Julia had to recompile many methods before it could once again produce a plot, so that in terms of time it was almost as expensive as the first usage. +The size of the effect varies substantially depending on what task you are trying to achieve and what packages you load to do the invalidation. +(If you do this with StaticArrays rather than SIMD there's no increase in latency, +but that's because StaticArrays was already loaded as an internal dependency of Plots.) + +It's worth noting that you can escape much of this cost by loading all +packages at the outset. +Loading time is somewhat increased by invalidation, but the cost of +first-time usage is typically larger. When possible, it's best to get +the invalidation out of the way before starting your work. + +# Can and should this be fixed? + +Invalidations affect latency on their own, but they also impact other potential strategies for reducing latency. +For example, julia creates "precompile" (`*.ji`) files to speed up package usage. +Currently, it saves type-inferred but not "native" code to its precompile files. +In principle, saving native code would eliminate latency (for the method and type combinations that have been precompiled), but this will be ineffective if most of this code ends up getting invalidated. +Indeed, it could make it even worse, because you're doing work (loading more stuff from disk) without much reward. +Consequently, reducing invalidation seems likely to be useful on its own, and a necessary prerequisite to other potential strategies. + +As to "can it be fixed?", that depends on the goal. +We will never get rid of invalidation altogether; as the `applyf` example above shows, +invalidation is sometimes necessary if you want both good runtime performance and interactive usage, and this combination is one of the best things about Julia. +The real question is whether there are *unnecessary* invalidations, +or a strategy to limit their impact. +Determining the answer to that question requires that we develop an understanding the common reasons for the large number of invalidations listed in the table above. + +# An analysis of the causes of invalidation + +This section relies on a recent [pull request to Julia][PRJulia] and +the [invalidations branch][PRSC] of [SnoopCompile]. +If you try to replicate these, remember that invalidations occur only +for methods that have been compiled, which generally means you have to +execute them. + +As we analyze causes of invalidation, you'll note that in some cases we can begin to think about how they might be fixed. +However, we'll save more detailed recommendations for the final section. + +## New methods with greater specificity + +It will be simplest to start with a case we already understand, the `applyf` example above. In a fresh Julia session, + +``` +f(x::Int) = 1 +f(x::Bool) = 2 +function applyf(container) + x1 = f(container[1]) + x2 = f(container[2]) + return x1 + x2 +end +c = Any[1, false]; +applyf(c) + +using SnoopCompile +``` + +Then, +``` +julia> invalidation_trees(@snoopr f(x::String) = 3) +1-element Array{SnoopCompile.MethodInvalidations,1}: + insert f(x::String) in Main at REPL[7]:1 invalidated: + mt_backedges: signature Tuple{typeof(f),Any} triggered MethodInstance for applyf(::Array{Any,1}) (0 children) more specific +``` + +Let's walk through this output a bit. +`@snoopr` turns on some debugging code inside Julia, and then executes the supplied statment; +it returns a fairly opaque list that can be parsed by `invalidation_trees`. +Entries in the returned array correspond to method additions (or deletions, if relevant) that triggers one or more invalidations. +In this case, the output means that the new `f(x::String)` method triggered an invalidation of `applyf(::Array{Any,1})`, +due to intersection with the signature `f(::Any)`. +`(0 children)` means that `applyf(::Vector{Any})` does not yet have any methods that called it and which in turn need to be invalidated. +Finally, `more specific` (which is printed in cyan) indicate that the new method was strictly more specific than the one that got invalidated. + +As we mentioned above, there are good reasons to think this invalidation is "necessary," meaning that it is an unavoidable consequence of the choices made to optimize runtime performance while also allowing one to dynamically extend functions. +However, that doesn't mean there is nothing that you, as a developer, could do to eliminate this invalidation. +Perhaps there is no real need to ever call `applyf` with a `Vector{Any}`; +perhaps you can fix one of its upstream callers to supply a concretely-type vector. +Or perhaps it may be well-typed, but inference fails to realize that; +you could annotate the result of some call with `Vector{String}`, for instance, +if you were certain of the result type. +Of course, in some cases you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. + +## New methods with ambiguous specificity + +Now let's try a real-world case, where the outcomes are more complex. + +``` +julia> invalidation_trees(@snoopr using FixedPointNumbers) +6-element Array{SnoopCompile.MethodInvalidations,1}: + insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: + backedges: MethodInstance for promote_rule(::Type{Union{}}, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) less specific + MethodInstance for promote_rule(::Type{S} where S<:Integer, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) ambiguous + + insert oneunit(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:93 invalidated: + backedges: MethodInstance for oneunit(::Type{T} where T<:AbstractChar) triggered MethodInstance for first(::Base.OneTo{T}) where T<:AbstractChar (0 children) ambiguous + 7 mt_cache + + insert one(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:94 invalidated: + mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (0 children) ambiguous + 5 mt_cache + + insert sizeof(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:100 invalidated: + backedges: MethodInstance for sizeof(::DataType) triggered MethodInstance for Base.CyclePadding(::DataType) (25 children) ambiguous + MethodInstance for sizeof(::Type) triggered MethodInstance for padding(::DataType) (3 children) more specific + MethodInstance for sizeof(::Type{T} where T) triggered MethodInstance for array_subpadding(::Type{T} where T, ::Type{T} where T) (0 children) more specific + 1 mt_cache + + insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: + backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific + 3 mt_cache + + insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: + mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous + backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered MethodInstance for +(::AbstractChar, ::UInt8) (157 children) ambiguous + MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) triggered MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (197 children) ambiguous + 1 mt_cache +``` + +This list is ordered from least-consequential to most consequential in terms of total number of invalidations. +The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 MethodInstances. +(It is not certain that these methods are all disjoint from one another.) +In contrast, the first entry is responsible for just two invalidations. + +One does not have to look at this list for very long to see that the majority of the invalidated methods are due to [method ambiguity]. +From + +```julia +julia> which(Char, (Int32,)) +(::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 +``` + +You may find it surprising that this method signature is ambiguous with `(::Type{X})(x::Real) where X<:FixedPoint`: after all, an `AbstractChar` is quite different from a `FixedPoint` number. +We can discover why with + +``` +julia> typeintersect(FixedPoint, AbstractChar) +Union{} +``` + +which shows that there is one type, the "empty type", that lies in their intersection. + +There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation. +If such a change gets made to Julia, then all the ones marked "ambiguous" should magically disappear. +Consequently, we can turn our attention to other cases. +`reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint` is strictly more specific than `reduce_empty(::Function, ::Type{T} where T)`. +It's curious that this is listed as a backedge of `reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber})`, since the latter is a concrete signature on its own. +This case too seems like something that should be fixed in Julia's invalidation logic. + +Moving backward, we get to `sizeof(::Type{X}) where X<:FixedPoint`. +Simply put, this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. +So we've discovered the first place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. + +You'll also notice one example where the new method is *less specific*. +It is not clear why such methods should be invalidating, and this may be a Julia bug. + +## Partial specialization + +If you try + +```julia +julia> trees = invalidation_trees(@snoopr using StaticArrays) +``` + +you'll see a much longer output. +A large number of invalidations derive from the fact that StaticArrays defines a method for `!=`, which invalidates the fallback definition + +```julia +!=(x, y) = !(x == y) +``` + +Since such definitions account for hundreds of nominal invalidations, it would be well worth considering whether it is possible to delete the custom `!=` methods. +For example, if they are purely for internal use you could modify each caller to +use the default method. + +The vast majority of the rest appear to derive from ambiguities. +However, one more interesting case we've not seen before is + +```julia +julia> tree = trees[end-7] +insert unsafe_convert(::Type{Ptr{T}}, m::Base.RefValue{FA}) where {N, T, D, FA<:FieldArray{N,T,D}} in StaticArrays at /home/tim/.julia/packages/StaticArrays/mlIi1/src/FieldArray.jl:124 invalidated: + mt_backedges: signature Tuple{typeof(Base.unsafe_convert),Type{Ptr{_A}} where _A,Base.RefValue{_A} where _A} triggered MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) (159 children) more specific +``` + +In this case, the signature that triggered the invalidation, `Base.unsafe_convert(::Type{Ptr{_A}} where _A, ::Base.RefValue{_A} where _A)`, +has been only partially specified: it depends on a type parameter `_A`. +Where does such a signature come from? +You can extract this line with + +``` +julia> trigger = tree[:mt_backedges, 1] +MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) at depth 0 with 159 children + +julia> trigger.children +5-element Array{SnoopCompile.InstanceTree,1}: + MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) at depth 1 with 0 children + MethodInstance for unsafe_convert(::Type{Ptr{T}}, ::Base.RefValue{Tuple{Vararg{T,N}}}) where {N, T} at depth 1 with 2 children + MethodInstance for _show_default(::Base.GenericIOBuffer{Array{UInt8,1}}, ::Any) at depth 1 with 113 children + MethodInstance for _show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at depth 1 with 37 children + MethodInstance for _show_default(::IOContext{REPL.Terminals.TTYTerminal}, ::Any) at depth 1 with 2 children +``` + +and see all the `MethodInstance`s that called this one. +You'll notice three `_show_default` `MethodInstance`s here; +a little digging reveals that this is defined as + +```julia +function _show_default(io::IO, @nospecialize(x)) + t = typeof(x) + ... +end +``` + +So the `@nospecialize` annotation, designed to reduce the number of cases when `_show_default` needs to be recompiled, causes the methods *it* uses to become triggers for invalidation. +So here we see that a technique that very successfully reduces latencies also has a side effect of increasing the number of invalidations. +Fortunately, these cases of partial specialization also seem to count as ambiguities, and so if ambiguous matches are eliminated it should also solve partial specialization. + +## Some summary statistics + +Let's go back to our table above, and augment it with "sources" of invalidation: + +| Package | greater specificity | lesser specificity | ambiguity | +|:------- | ------------------:| --------:| -----:| +| Example | 0 | 0 | 0 | 0 | +| Revise | 6 | 0 | 0 | +| FixedPointNumbers | 139 | 0 | 381 | +| SIMD | 3040 | 0 | 1017 | +| StaticArrays | 1382 | 13 | 2540 | +| Optim | 1385 | 13 | 2941 | +| Images | 1513 | 113 | 3102 | +| Flux | 1177 | 49 | 4107 | +| Plots | 1104 | 48 | 4604 | +| DataFrames | 2725 | 0 | 2680 | +| JuMP | 1549 | 14 | 5164 | +| Makie | 5147 | 92 | 4145 | +| DifferentialEquations | 3776 | 53 | 7419 | + +The numbers in this table don't add up to those in the first, for a variety of reasons (here there is no attempt to remove duplicates, here we don't count "mt_cache" invalidations which were included in the first table, etc.). +In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column indicates invalidations that should either be fixed in packages, Julia's own code, or will need to remain unfixed. +The good news is that much will likely be fixed by "automated" means, +but it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. + +[Julia]: https://julialang.org/ +[union-splitting]: https://julialang.org/blog/2018/08/union-splitting/ +[MethodAnalysis]: https://github.com/timholy/MethodAnalysis.jl +[SnoopCompile]: https://github.com/timholy/SnoopCompile.jl +[PRJulia]: https://github.com/JuliaLang/julia/pull/35768 +[PRSC]: https://github.com/timholy/SnoopCompile.jl/pull/79 +[method ambiguity]: https://docs.julialang.org/en/latest/manual/methods/#man-ambiguities-1 From 104bc71d36ffe3d8decdf275fa488e578d3296bc Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Mon, 11 May 2020 07:42:33 -0500 Subject: [PATCH 02/21] Expand the section on fixing invalidations and add a summary --- blog/2020/05/invalidations.md | 374 ++++++++++++++++++++++++++++------ 1 file changed, 309 insertions(+), 65 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index df82993dc7..1a7dca2a0a 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -1,7 +1,7 @@ @def authors = "Tim Holy" -@def published = "10 May 2020" +@def published = "11 May 2020" @def title = "Analyzing sources of compiler latency in Julia: method invalidations" -@def rss_pubdate = Date(2020, 5, 10) +@def rss_pubdate = Date(2020, 5, 11) @def rss = """Julia is fast, but compiling Julia code takes time. This post analyzes why it's sometimes necessary to repeat that work, and what might be done to fix it.""" @@ -27,7 +27,7 @@ so that it can be used by any caller. This is crucial to performance, because it means generally compilation has to be done only once for a specific type. To keep things really simple, I'm going to use a very artificial example. -```julia +``` f(x::Int) = 1 f(x::Bool) = 2 @@ -116,7 +116,7 @@ quite simple. First, look at those `arrayref` statements: they are annotated Immediately after each reference, notice that there are two `isa` statements followed by a `ϕ`; all of this is essentially equivalent to -```julia +``` if isa(x, Bool) value = 2 elseif isa(x, Int) @@ -126,7 +126,7 @@ else end ``` -This is [union-splitting], a key optimization for performance in the face of uncertain types. +This is [union-splitting], a performance optimization for cases where an object might be of one of several types. ## Triggering method invalidation @@ -219,21 +219,21 @@ These are mostly for `Base` and the standard libraries. Using some not-yet merged work in both Julia itself and [SnoopCompile], we can count the number of invalidations when we load various packages into a fresh Julia session: -| Package | # of unique invalidations | -|:------- | ------------------:| -| Example | 0 | -| Revise | 27 | -| FixedPointNumbers | 429 | -| SIMD | 2799 | -| StaticArrays | 2852 | -| Optim | 3171 | -| Images | 3638 | -| Flux | 3697 | -| Plots | 4002 | -| DataFrames | 4048 | -| JuMP | 4666 | -| Makie | 6118 | -| DifferentialEquations | 6777 | +| Package | Version | # of unique invalidations | +|:------- | -------:| ------------------:| +| Example | 0.5.3 | 0 | +| Revise | 2.6.6 | 27 | +| FixedPointNumbers | 0.8.0 | 429 | +| SIMD | 2.8.0 | 2799 | +| StaticArrays | 0.12.3 | 2852 | +| Optim | 0.21.0 | 3171 | +| Images | 0.22.2 | 3638 | +| Flux | 0.10.4 | 3697 | +| Plots | 1.2.3 | 4002 | +| DataFrames | 0.21.0 | 4048 | +| JuMP | 0.21.2 | 4666 | +| Makie | 0.10.0 | 6118 | +| DifferentialEquations | 6.13.0 | 6777 | You can see that key packages used by large portions of the Julia ecosystem invalidate hundreds or thousands of MethodInstances, sometimes more than 10% of the total @@ -245,7 +245,7 @@ The next time you want to call functionality that gets invalidated, you have to wait for recompilation. We can illustrate this using everyone's favorite example, plotting: -```julia +``` julia> using Plots julia> @time display(plot(rand(5))) @@ -254,14 +254,23 @@ julia> @time display(plot(rand(5))) As is well known, it's much faster the second time, because it's already compiled: -```julia +``` julia> @time display(plot(rand(5))) 0.311226 seconds (19.93 k allocations: 775.055 KiB) ``` -Now load a package that does a lot of invalidation, and try again: +Moreover, if you decide you want some additional functionality and decide to load a new package, sometimes it's essentially as fast again: -```julia +``` +julia> using StaticArrays + +julia> @time display(plot(rand(5))) + 0.305394 seconds (19.96 k allocations: 781.836 KiB) +``` + +But if you load a package that does a lot of invalidation: + +``` julia> using SIMD julia> @time display(plot(rand(5))) @@ -270,8 +279,6 @@ julia> @time display(plot(rand(5))) Because so much got invalidated by loading SIMD, Julia had to recompile many methods before it could once again produce a plot, so that in terms of time it was almost as expensive as the first usage. The size of the effect varies substantially depending on what task you are trying to achieve and what packages you load to do the invalidation. -(If you do this with StaticArrays rather than SIMD there's no increase in latency, -but that's because StaticArrays was already loaded as an internal dependency of Plots.) It's worth noting that you can escape much of this cost by loading all packages at the outset. @@ -284,24 +291,24 @@ the invalidation out of the way before starting your work. Invalidations affect latency on their own, but they also impact other potential strategies for reducing latency. For example, julia creates "precompile" (`*.ji`) files to speed up package usage. Currently, it saves type-inferred but not "native" code to its precompile files. -In principle, saving native code would eliminate latency (for the method and type combinations that have been precompiled), but this will be ineffective if most of this code ends up getting invalidated. +In principle, saving native code would eliminate latency for the method and type combinations that have been precompiled, but this will be ineffective if most of this code ends up getting invalidated. Indeed, it could make it even worse, because you're doing work (loading more stuff from disk) without much reward. -Consequently, reducing invalidation seems likely to be useful on its own, and a necessary prerequisite to other potential strategies. +Consequently, reducing invalidation seems likely to be useful on its own, and a necessary prerequisite to other potential strategies for reducing latency. As to "can it be fixed?", that depends on the goal. We will never get rid of invalidation altogether; as the `applyf` example above shows, invalidation is sometimes necessary if you want both good runtime performance and interactive usage, and this combination is one of the best things about Julia. The real question is whether there are *unnecessary* invalidations, -or a strategy to limit their impact. +or an unexploited strategy to limit their impact. Determining the answer to that question requires that we develop an understanding the common reasons for the large number of invalidations listed in the table above. # An analysis of the causes of invalidation This section relies on a recent [pull request to Julia][PRJulia] and the [invalidations branch][PRSC] of [SnoopCompile]. -If you try to replicate these, remember that invalidations occur only +If you try to replicate these results, remember that invalidations occur only for methods that have been compiled, which generally means you have to -execute them. +execute them first. As we analyze causes of invalidation, you'll note that in some cases we can begin to think about how they might be fixed. However, we'll save more detailed recommendations for the final section. @@ -335,7 +342,7 @@ julia> invalidation_trees(@snoopr f(x::String) = 3) Let's walk through this output a bit. `@snoopr` turns on some debugging code inside Julia, and then executes the supplied statment; it returns a fairly opaque list that can be parsed by `invalidation_trees`. -Entries in the returned array correspond to method additions (or deletions, if relevant) that triggers one or more invalidations. +Entries in the returned array correspond to method additions (or deletions, if relevant) that trigger one or more invalidations. In this case, the output means that the new `f(x::String)` method triggered an invalidation of `applyf(::Array{Any,1})`, due to intersection with the signature `f(::Any)`. `(0 children)` means that `applyf(::Vector{Any})` does not yet have any methods that called it and which in turn need to be invalidated. @@ -345,80 +352,121 @@ As we mentioned above, there are good reasons to think this invalidation is "nec However, that doesn't mean there is nothing that you, as a developer, could do to eliminate this invalidation. Perhaps there is no real need to ever call `applyf` with a `Vector{Any}`; perhaps you can fix one of its upstream callers to supply a concretely-type vector. -Or perhaps it may be well-typed, but inference fails to realize that; -you could annotate the result of some call with `Vector{String}`, for instance, -if you were certain of the result type. -Of course, in some cases you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. +In some cases, though, you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. ## New methods with ambiguous specificity Now let's try a real-world case, where the outcomes are more complex. ``` -julia> invalidation_trees(@snoopr using FixedPointNumbers) -6-element Array{SnoopCompile.MethodInvalidations,1}: +julia> trees = invalidation_trees(@snoopr using FixedPointNumbers) +5-element Array{SnoopCompile.MethodInvalidations,1}: insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: backedges: MethodInstance for promote_rule(::Type{Union{}}, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) less specific MethodInstance for promote_rule(::Type{S} where S<:Integer, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) ambiguous - - insert oneunit(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:93 invalidated: - backedges: MethodInstance for oneunit(::Type{T} where T<:AbstractChar) triggered MethodInstance for first(::Base.OneTo{T}) where T<:AbstractChar (0 children) ambiguous - 7 mt_cache + 3 mt_cache insert one(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:94 invalidated: - mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (0 children) ambiguous - 5 mt_cache + mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (1 children) ambiguous + 1 mt_cache insert sizeof(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:100 invalidated: backedges: MethodInstance for sizeof(::DataType) triggered MethodInstance for Base.CyclePadding(::DataType) (25 children) ambiguous MethodInstance for sizeof(::Type) triggered MethodInstance for padding(::DataType) (3 children) more specific MethodInstance for sizeof(::Type{T} where T) triggered MethodInstance for array_subpadding(::Type{T} where T, ::Type{T} where T) (0 children) more specific - 1 mt_cache + 7 mt_cache insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific - 3 mt_cache insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered MethodInstance for +(::AbstractChar, ::UInt8) (157 children) ambiguous MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) triggered MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (197 children) ambiguous - 1 mt_cache + 6 mt_cache ``` -This list is ordered from least-consequential to most consequential in terms of total number of invalidations. +This list is ordered from least- to most-consequential in terms of total number of invalidations. The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 MethodInstances. -(It is not certain that these methods are all disjoint from one another.) +(There is no guarantee that these methods are all disjoint from one another; +the results are represented as a tree, where each node links to its callers.) In contrast, the first entry is responsible for just two invalidations. One does not have to look at this list for very long to see that the majority of the invalidated methods are due to [method ambiguity]. -From +Consider the line `backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered...`. +We can see which method this is by the following: -```julia +``` julia> which(Char, (Int32,)) (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 ``` +or directly as + +``` +julia> tree = trees[end] +insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: + mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous + backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered MethodInstance for +(::AbstractChar, ::UInt8) (157 children) ambiguous + MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) triggered MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (197 children) ambiguous + 6 mt_cache + + +julia> mi, invtree = tree[:backedges,1] +MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) => MethodInstance for +(::AbstractChar, ::UInt8) at depth 0 with 157 children + +julia> mi.def +(::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 +``` + +`trees[end]` selected the last (most consequential) method and the invalidations it triggered; indexing this with `:backedges` selected the category (`:mt_backedges`, `:backedges`, or `:mt_cache`), and the integer index selected the particular entry from that category. +This returns a pair `MethodInstance => InstanceTree`, where the latter is a type encoding the tree. +We'll see how to work with `InstanceTree`s in a moment, for now we want to focus on the `mi` portion of that pair. + You may find it surprising that this method signature is ambiguous with `(::Type{X})(x::Real) where X<:FixedPoint`: after all, an `AbstractChar` is quite different from a `FixedPoint` number. We can discover why with ``` -julia> typeintersect(FixedPoint, AbstractChar) -Union{} +julia> tree.method.sig +Tuple{Type{X},Real} where X<:FixedPoint + +julia> mi.specTypes +Tuple{Type{T} where T<:AbstractChar,Int32} + +julia> typeintersect(tree.method.sig, mi.specTypes) +Tuple{Type{Union{}},Int32} +``` + +These two signatures have non-empty intersection. +The second parameter, `Int32`, makes sense as the intersection of `Int32` and `Real`. +The first arises from + +``` +julia> typeintersect(Type{<:FixedPoint}, Type{<:AbstractChar}) +Type{Union{}} ``` -which shows that there is one type, the "empty type", that lies in their intersection. +which shows that there is one Type, the "empty Type", that lies in their intersection. -There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation. +There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation---if it were to be called by the compiled code, it would trigger an error anyway. If such a change gets made to Julia, then all the ones marked "ambiguous" should magically disappear. Consequently, we can turn our attention to other cases. + +Let's look at the next item up the list: + +``` +julia> tree = trees[end-1] +insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: + backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific +``` + `reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint` is strictly more specific than `reduce_empty(::Function, ::Type{T} where T)`. -It's curious that this is listed as a backedge of `reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber})`, since the latter is a concrete signature on its own. -This case too seems like something that should be fixed in Julia's invalidation logic. +This might look like one of those "necessary" invalidations. +Below we'll analyze this in far greater detail and discover some possible fixes. -Moving backward, we get to `sizeof(::Type{X}) where X<:FixedPoint`. +Moving backward another step, we get to `sizeof(::Type{X}) where X<:FixedPoint`. Simply put, this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. -So we've discovered the first place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. +So we've discovered an easy place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. You'll also notice one example where the new method is *less specific*. It is not clear why such methods should be invalidating, and this may be a Julia bug. @@ -427,14 +475,14 @@ It is not clear why such methods should be invalidating, and this may be a Julia If you try -```julia +``` julia> trees = invalidation_trees(@snoopr using StaticArrays) ``` you'll see a much longer output. A large number of invalidations derive from the fact that StaticArrays defines a method for `!=`, which invalidates the fallback definition -```julia +``` !=(x, y) = !(x == y) ``` @@ -445,7 +493,7 @@ use the default method. The vast majority of the rest appear to derive from ambiguities. However, one more interesting case we've not seen before is -```julia +``` julia> tree = trees[end-7] insert unsafe_convert(::Type{Ptr{T}}, m::Base.RefValue{FA}) where {N, T, D, FA<:FieldArray{N,T,D}} in StaticArrays at /home/tim/.julia/packages/StaticArrays/mlIi1/src/FieldArray.jl:124 invalidated: mt_backedges: signature Tuple{typeof(Base.unsafe_convert),Type{Ptr{_A}} where _A,Base.RefValue{_A} where _A} triggered MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) (159 children) more specific @@ -473,7 +521,7 @@ and see all the `MethodInstance`s that called this one. You'll notice three `_show_default` `MethodInstance`s here; a little digging reveals that this is defined as -```julia +``` function _show_default(io::IO, @nospecialize(x)) t = typeof(x) ... @@ -483,6 +531,7 @@ end So the `@nospecialize` annotation, designed to reduce the number of cases when `_show_default` needs to be recompiled, causes the methods *it* uses to become triggers for invalidation. So here we see that a technique that very successfully reduces latencies also has a side effect of increasing the number of invalidations. Fortunately, these cases of partial specialization also seem to count as ambiguities, and so if ambiguous matches are eliminated it should also solve partial specialization. +In the statistics below, we'll lump partial specialization in with ambiguity. ## Some summary statistics @@ -506,8 +555,202 @@ Let's go back to our table above, and augment it with "sources" of invalidation: The numbers in this table don't add up to those in the first, for a variety of reasons (here there is no attempt to remove duplicates, here we don't count "mt_cache" invalidations which were included in the first table, etc.). In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column indicates invalidations that should either be fixed in packages, Julia's own code, or will need to remain unfixed. -The good news is that much will likely be fixed by "automated" means, -but it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. +The good news is that these counts reveal that much will likely be fixed by "automated" means. +However, it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. + +# Fixing invalidations + +You may have noticed that two packages, `Example` and `Revise`, trigger far fewer invalidations that the rest of the packages in our analysis. +`Example` is quite trivial, but `Revise` and its dependencies are quite large. +How does it avoid this problem? +First, Revise does not extending very many Base methods; +most of its methods are to functions it "owns," and the same is true for its dependencies. +Second, in the closing days of Julia 1.5's merge window, +Revise (and Julia) underwent a process of tracking down invalidations and eliminating them; +for comparison, on Julia 1.4, Revise triggers more than a 1000 non-unique invalidations. +The success of this effort gives one hope that other packages too may one day have fewer invalidations. + +As stated above, there is reason to hope that most of the invalidations marked as "ambiguous" will be fixed by changes to Julia's compiler. +Here our focus is on those marked "more specific," since those are cases where it is hard to imagine a generic fix. + +## Fixing a case of type-instability + +In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving a type-instability. +For example, one set of invalidations happened because `CodeTracking`, a dependency of Revise's, defines new methods for `Base.PkgId`. +It turns out that this triggered an invalidation of `_tryrequire_from_serialized`, which is used to load packages; +a negative consequence is that Revise introduced a slight latency upon loading the *next* package. +However, it turned out to be an easy fix: one section of `_tryrequire_from_serialized` had a passage + +``` +for M in mod::Vector{Any} + if PkgId(M) == modkey && module_build_id(M) === build_id + return M + end +end +``` + +and since `M` had type `Any`, the compiler couldn't predict which version of `PkgId` would be called. +It sufficed to add + +``` + M = M::Module +``` + +immediately after the `for` statement to fix the problem. +Not only does this fix the invalidation, but it lets the compiler generate better code. + +The other case was similar: a call from `Pkg` of `keys` on an AbstractDict of unknown type +(due to a higher `@nospecialize` call). +Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default consequence of calling `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. + +## Redirecting call chains + +Let's return to our FixedPointNumbers `reduce_empty` example above. +A little prodding as done above reveals that this corresponds to the definition + +``` +julia> mi, invtree = tree[:backedges, 1] +MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 136 children + +julia> mi +MethodInstance for reduce_empty(::Function, ::Type{T} where T) + +julia> mi.def +reduce_empty(op, T) in Base at reduce.jl:309 +``` + +If you look up this definition, you'll see it's + +``` +reduce_empty(op, T) = _empty_reduce_error() +``` + +which indicates that it is the fallback method for reducing over an empty collection, and indeed calling this results in an error: + +``` +julia> op = Base.BottomRF(Base.max) +Base.BottomRF{typeof(max)}(max) + +julia> Base.reduce_empty(op, VERSION) +ERROR: ArgumentError: reducing over an empty collection is not allowed +Stacktrace: + [1] _empty_reduce_error() at ./reduce.jl:299 + [2] reduce_empty(::Function, ::VersionNumber) at ./reduce.jl:309 + [3] reduce_empty(::Base.BottomRF{typeof(max)}, ::VersionNumber) at ./reduce.jl:324 + [4] top-level scope at REPL[36]:1 +``` + +This essentially means that no "identity element" has been defined for this operation and type. + +Can we avoid this fallback? +One approach is to define the method directly: modify Julia to add + +``` +reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = _empty_reduce_error() +``` + +so that we get the same result but don't rely on the fallback. +But perhaps a better approach is to see who's calling it: + +``` +julia> invtree.mi +MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) + +julia> invtree.mi.def +reduce_empty(op::Base.BottomRF, T) in Base at reduce.jl:324 + +julia> invtree.children +5-element Array{SnoopCompile.InstanceTree,1}: + MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children + MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{Int64}) at depth 1 with 39 children + MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Pkg.Resolve.FieldValue}) at depth 1 with 21 children + MethodInstance for mapreduce_empty(::typeof(identity), ::Pkg.Resolve.var"#132#134"{Pkg.Resolve.var"#smx#133"{Pkg.Resolve.Graph,Pkg.Resolve.Messages}}, ::Type{Int64}) at depth 1 with 10 children + MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Int64}) at depth 1 with 23 children +``` + +This illustrates how to work with an `InstanceTree`: you access the MethodInstance through `.mi` and its callers through `.children`. +Let's start with the first one: + +``` +julia> node = invtree.children[1] +MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children +``` + +We can display the whole tree using `show(node)`: + +``` +julia> show(node) + MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) + MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}) + MethodInstance for foldl_impl(::Base.BottomRF{typeof(max)}, ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) + MethodInstance for mapfoldl_impl(::typeof(identity), ::typeof(max), ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) + MethodInstance for #mapfoldl#201(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapfoldl), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for mapfoldl(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for #mapreduce#205(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapreduce), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for mapreduce(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for maximum(::Set{VersionNumber}) + MethodInstance for set_maximum_version_registry!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec) + MethodInstance for collect_project!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::Dict{Base.UUID,Array{Pkg.Types.PackageSpec,1}}) + MethodInstance for collect_fixed!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Base.UUID,String}) + MethodInstance for resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) + ⋮ +``` + +This indicates that this invalidation path resulted from a call to `maximum` from a function in `Pkg`, `set_maximum_version_registry!`. A little digging reveals that it is defined as + +``` +function set_maximum_version_registry!(ctx::Context, pkg::PackageSpec) + pkgversions = Set{VersionNumber}() + for path in registered_paths(ctx, pkg.uuid) + pathvers = keys(load_versions(ctx, path; include_yanked=false)) + union!(pkgversions, pathvers) + end + if length(pkgversions) == 0 + pkg.version = VersionNumber(0) + else + max_version = maximum(pkgversions) + pkg.version = VersionNumber(max_version.major, max_version.minor, max_version.patch, max_version.prerelease, ("",)) + end +end +``` + +From that error above, we know that this invalidation is produced by an error-path triggered by trying to reduce over an empty collection. +Interestingly, we can see that `set_maximum_version_registry!` handles an empty collection by other means: +there is, in fact, no chance that `maximum(pkgversions)` will ever reach the error case. +However, the compiler does not realize that, and therefore generates code that has been prepared to call `reduce_empty`, and that makes it vulnerable to invalidation. + +There are a couple of potential fixes. +From here, we see that a potentially better definition `reduce_empty` for `VersionNumber`s might be + +``` +reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = VersionNumber(0) +``` + +In that case we could delete the `length` check and rely on this to produce the desired outcome. +One should take this approach only if one can be convinced that version 0 can act as a universal "neutral element" for reductions involving `max` over `VersionNumber`s. +Adding this to `Pkg` would be type-piracy, since `max`, `BottomRF`, and `VersionNumber` are all defined in `Base`, so fixing it this way would best be done in Base. + +But there are other possible fixes. +A little higher up the tree we see a call to `mapreduce`, and this presents another opportunity because `mapreduce` allows you to supply an `init` value: + +``` +julia> mapreduce(identity, max, Set(VersionNumber[]); init=VersionNumber(0)) +v"0.0.0" +``` + +Perhaps we could just call this instead of `maximum`. +However, it's a bit uglier than the original; +perhaps a nicer approach would be to allow one to supply `init` as a keyword argument to `maximum` itself. +While this is not supported on Julia versions up through 1.5, it's a feature that seems to make sense, and this analysis suggests that it might also allow developers to make code more robust against certain kinds of invalidation. + +# Summary + +Julia's remarkable flexibility and outstanding code-generation open many new horizons. +These advantages come with a few costs, and here we've explored one of them, method invalidation. +While Julia's core developers have been aware of its cost for a long time, +we're only now starting to get tools to analyze it in a manner suitable for a larger population of users and developers. +Because it's not been easy to measure previously, it would not be surprising if there are numerous opportunities to reduce it, waiting to be discovered. +One might hope that the next period of development might see significant strides in new ways of getting packages to work together without stomping on each other's toes. [Julia]: https://julialang.org/ [union-splitting]: https://julialang.org/blog/2018/08/union-splitting/ @@ -516,3 +759,4 @@ but it appears that there will need to be a second round in which package develo [PRJulia]: https://github.com/JuliaLang/julia/pull/35768 [PRSC]: https://github.com/timholy/SnoopCompile.jl/pull/79 [method ambiguity]: https://docs.julialang.org/en/latest/manual/methods/#man-ambiguities-1 +[sentinel]: https://en.wikipedia.org/wiki/Sentinel_value From 071d1214f05e19a98ba7de4f03b3979b2d6add49 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Mon, 11 May 2020 08:56:47 -0500 Subject: [PATCH 03/21] Demote all headers 1 level --- blog/2020/05/invalidations.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 1a7dca2a0a..f13b193a7a 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -18,9 +18,9 @@ These efforts have met with considerable success: the upcoming Julia 1.5 feels " But the job of reducing latency is not over yet. Recently I got interested in a specific source of this latency, and this blog post is a summary of some of what I've learned about the scope of this problem and opportunities for further improvement. -# Method invalidation: what is it and when does it happen? +## Method invalidation: what is it and when does it happen? -## An example: compilation in the presence of Union-splitting +### An example: compilation in the presence of Union-splitting When Julia compiles a method for specific types, it saves the resulting code so that it can be used by any caller. @@ -128,7 +128,7 @@ end This is [union-splitting], a performance optimization for cases where an object might be of one of several types. -## Triggering method invalidation +### Triggering method invalidation One point in particular is worth noting: what's up with that final `f(x)::Int` call? Didn't it already handle all the possibilities? @@ -208,7 +208,7 @@ If `applyf` is performance-critical, you'll be very happy that Julia tries to gi But this leaves the door open to invalidation, which means recompilation the next time you use `applyf`. If method invalidation happens often, this might contribute to making Julia "feel" sluggish. -# How common is method invalidation? +## How common is method invalidation? Unfortunately, method invalidation is pretty common. First, let's get some baseline statistics. @@ -239,7 +239,7 @@ You can see that key packages used by large portions of the Julia ecosystem inva hundreds or thousands of MethodInstances, sometimes more than 10% of the total number of MethodInstances present before loading the package. -# How serious is method invalidation? +## How serious is method invalidation? The next time you want to call functionality that gets invalidated, you have to wait for recompilation. @@ -286,7 +286,7 @@ Loading time is somewhat increased by invalidation, but the cost of first-time usage is typically larger. When possible, it's best to get the invalidation out of the way before starting your work. -# Can and should this be fixed? +## Can and should this be fixed? Invalidations affect latency on their own, but they also impact other potential strategies for reducing latency. For example, julia creates "precompile" (`*.ji`) files to speed up package usage. @@ -302,7 +302,7 @@ The real question is whether there are *unnecessary* invalidations, or an unexploited strategy to limit their impact. Determining the answer to that question requires that we develop an understanding the common reasons for the large number of invalidations listed in the table above. -# An analysis of the causes of invalidation +## An analysis of the causes of invalidation This section relies on a recent [pull request to Julia][PRJulia] and the [invalidations branch][PRSC] of [SnoopCompile]. @@ -313,7 +313,7 @@ execute them first. As we analyze causes of invalidation, you'll note that in some cases we can begin to think about how they might be fixed. However, we'll save more detailed recommendations for the final section. -## New methods with greater specificity +### New methods with greater specificity It will be simplest to start with a case we already understand, the `applyf` example above. In a fresh Julia session, @@ -354,7 +354,7 @@ Perhaps there is no real need to ever call `applyf` with a `Vector{Any}`; perhaps you can fix one of its upstream callers to supply a concretely-type vector. In some cases, though, you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. -## New methods with ambiguous specificity +### New methods with ambiguous specificity Now let's try a real-world case, where the outcomes are more complex. @@ -471,7 +471,7 @@ So we've discovered an easy place where a developer could do something to produc You'll also notice one example where the new method is *less specific*. It is not clear why such methods should be invalidating, and this may be a Julia bug. -## Partial specialization +### Partial specialization If you try @@ -533,7 +533,7 @@ So here we see that a technique that very successfully reduces latencies also ha Fortunately, these cases of partial specialization also seem to count as ambiguities, and so if ambiguous matches are eliminated it should also solve partial specialization. In the statistics below, we'll lump partial specialization in with ambiguity. -## Some summary statistics +### Some summary statistics Let's go back to our table above, and augment it with "sources" of invalidation: @@ -558,7 +558,7 @@ In general terms, the last two columns should probably be fixed by changes in ho The good news is that these counts reveal that much will likely be fixed by "automated" means. However, it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. -# Fixing invalidations +## Fixing invalidations You may have noticed that two packages, `Example` and `Revise`, trigger far fewer invalidations that the rest of the packages in our analysis. `Example` is quite trivial, but `Revise` and its dependencies are quite large. @@ -573,7 +573,7 @@ The success of this effort gives one hope that other packages too may one day ha As stated above, there is reason to hope that most of the invalidations marked as "ambiguous" will be fixed by changes to Julia's compiler. Here our focus is on those marked "more specific," since those are cases where it is hard to imagine a generic fix. -## Fixing a case of type-instability +### Fixing a case of type-instability In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving a type-instability. For example, one set of invalidations happened because `CodeTracking`, a dependency of Revise's, defines new methods for `Base.PkgId`. @@ -603,7 +603,7 @@ The other case was similar: a call from `Pkg` of `keys` on an AbstractDict of un (due to a higher `@nospecialize` call). Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default consequence of calling `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. -## Redirecting call chains +### Redirecting call chains Let's return to our FixedPointNumbers `reduce_empty` example above. A little prodding as done above reveals that this corresponds to the definition @@ -743,7 +743,7 @@ However, it's a bit uglier than the original; perhaps a nicer approach would be to allow one to supply `init` as a keyword argument to `maximum` itself. While this is not supported on Julia versions up through 1.5, it's a feature that seems to make sense, and this analysis suggests that it might also allow developers to make code more robust against certain kinds of invalidation. -# Summary +## Summary Julia's remarkable flexibility and outstanding code-generation open many new horizons. These advantages come with a few costs, and here we've explored one of them, method invalidation. From b9adb3acbb0e2b1c39faff45be0c9a7ccb04b686 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Mon, 11 May 2020 09:02:20 -0500 Subject: [PATCH 04/21] Use julia-repl for REPL blocks --- blog/2020/05/invalidations.md | 52 ++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 25 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index f13b193a7a..5d0c1f94c0 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -42,7 +42,7 @@ If you call `applyf([100, 200])`, it will compile a version of `applyf` knowing that only `f(::Int)` will be used: the call will be "hard-wired" into the code that Julia produces. You can see this using `@code_typed`: -``` +```julia-repl julia> @code_typed applyf([100,200]) CodeInfo( 1 ─ Base.arrayref(true, container, 1)::Int64 @@ -58,7 +58,7 @@ call `applyf([100])`.) If you pass a `Vector{Bool}`, it will compile `applyf` again, this time specializing it for `x::Bool`: -``` +```julia-repl julia> @code_typed applyf([true,false]) CodeInfo( 1 ─ Base.arrayref(true, container, 1)::Bool @@ -77,7 +77,8 @@ You don't normally see these, but Julia manages them for you; anytime you write code that calls `applyf`, it checks to see if this previous compilation work can be reused. For the purpose of this blog post, things start to get especially interesting if we try the following: -``` + +```julia-repl julia> c = Any[1, false]; julia> applyf(c) @@ -158,7 +159,7 @@ f(::Missing) = 5 then Julia produces -``` +```julia-repl julia> @code_typed applyf(c) CodeInfo( 1 ─ %1 = Base.arrayref(true, container, 1)::Any @@ -183,7 +184,7 @@ But the performance benefits of such optimizations are so large that, when appli For example, if you start a fresh Julia session and just define the `f(::Int)` and `f(::Bool)` methods, then -``` +```julia-repl julia> using BenchmarkTools julia> @btime applyf($c) @@ -245,7 +246,7 @@ The next time you want to call functionality that gets invalidated, you have to wait for recompilation. We can illustrate this using everyone's favorite example, plotting: -``` +```julia-repl julia> using Plots julia> @time display(plot(rand(5))) @@ -254,14 +255,14 @@ julia> @time display(plot(rand(5))) As is well known, it's much faster the second time, because it's already compiled: -``` +```julia-repl julia> @time display(plot(rand(5))) 0.311226 seconds (19.93 k allocations: 775.055 KiB) ``` Moreover, if you decide you want some additional functionality and decide to load a new package, sometimes it's essentially as fast again: -``` +```julia-repl julia> using StaticArrays julia> @time display(plot(rand(5))) @@ -270,7 +271,7 @@ julia> @time display(plot(rand(5))) But if you load a package that does a lot of invalidation: -``` +```julia-repl julia> using SIMD julia> @time display(plot(rand(5))) @@ -332,7 +333,8 @@ using SnoopCompile ``` Then, -``` + +```julia-repl julia> invalidation_trees(@snoopr f(x::String) = 3) 1-element Array{SnoopCompile.MethodInvalidations,1}: insert f(x::String) in Main at REPL[7]:1 invalidated: @@ -358,7 +360,7 @@ In some cases, though, you might really need to call `applyf` with a `Vector{Any Now let's try a real-world case, where the outcomes are more complex. -``` +```julia-repl julia> trees = invalidation_trees(@snoopr using FixedPointNumbers) 5-element Array{SnoopCompile.MethodInvalidations,1}: insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: @@ -396,14 +398,14 @@ One does not have to look at this list for very long to see that the majority of Consider the line `backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered...`. We can see which method this is by the following: -``` +```julia-repl julia> which(Char, (Int32,)) (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 ``` or directly as -``` +```julia-repl julia> tree = trees[end] insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous @@ -426,7 +428,7 @@ We'll see how to work with `InstanceTree`s in a moment, for now we want to focus You may find it surprising that this method signature is ambiguous with `(::Type{X})(x::Real) where X<:FixedPoint`: after all, an `AbstractChar` is quite different from a `FixedPoint` number. We can discover why with -``` +```julia-repl julia> tree.method.sig Tuple{Type{X},Real} where X<:FixedPoint @@ -441,7 +443,7 @@ These two signatures have non-empty intersection. The second parameter, `Int32`, makes sense as the intersection of `Int32` and `Real`. The first arises from -``` +```julia-repl julia> typeintersect(Type{<:FixedPoint}, Type{<:AbstractChar}) Type{Union{}} ``` @@ -454,7 +456,7 @@ Consequently, we can turn our attention to other cases. Let's look at the next item up the list: -``` +```julia-repl julia> tree = trees[end-1] insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific @@ -475,7 +477,7 @@ It is not clear why such methods should be invalidating, and this may be a Julia If you try -``` +```julia-repl julia> trees = invalidation_trees(@snoopr using StaticArrays) ``` @@ -493,7 +495,7 @@ use the default method. The vast majority of the rest appear to derive from ambiguities. However, one more interesting case we've not seen before is -``` +```julia-repl julia> tree = trees[end-7] insert unsafe_convert(::Type{Ptr{T}}, m::Base.RefValue{FA}) where {N, T, D, FA<:FieldArray{N,T,D}} in StaticArrays at /home/tim/.julia/packages/StaticArrays/mlIi1/src/FieldArray.jl:124 invalidated: mt_backedges: signature Tuple{typeof(Base.unsafe_convert),Type{Ptr{_A}} where _A,Base.RefValue{_A} where _A} triggered MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) (159 children) more specific @@ -504,7 +506,7 @@ has been only partially specified: it depends on a type parameter `_A`. Where does such a signature come from? You can extract this line with -``` +```julia-repl julia> trigger = tree[:mt_backedges, 1] MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) at depth 0 with 159 children @@ -608,7 +610,7 @@ Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default consequence Let's return to our FixedPointNumbers `reduce_empty` example above. A little prodding as done above reveals that this corresponds to the definition -``` +```julia-repl julia> mi, invtree = tree[:backedges, 1] MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 136 children @@ -627,7 +629,7 @@ reduce_empty(op, T) = _empty_reduce_error() which indicates that it is the fallback method for reducing over an empty collection, and indeed calling this results in an error: -``` +```julia-repl julia> op = Base.BottomRF(Base.max) Base.BottomRF{typeof(max)}(max) @@ -652,7 +654,7 @@ reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = _empty_reduc so that we get the same result but don't rely on the fallback. But perhaps a better approach is to see who's calling it: -``` +```julia-repl julia> invtree.mi MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) @@ -671,14 +673,14 @@ julia> invtree.children This illustrates how to work with an `InstanceTree`: you access the MethodInstance through `.mi` and its callers through `.children`. Let's start with the first one: -``` +```julia-repl julia> node = invtree.children[1] MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children ``` We can display the whole tree using `show(node)`: -``` +```julia-repl julia> show(node) MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}) @@ -733,7 +735,7 @@ Adding this to `Pkg` would be type-piracy, since `max`, `BottomRF`, and `Version But there are other possible fixes. A little higher up the tree we see a call to `mapreduce`, and this presents another opportunity because `mapreduce` allows you to supply an `init` value: -``` +```julia-repl julia> mapreduce(identity, max, Set(VersionNumber[]); init=VersionNumber(0)) v"0.0.0" ``` From 05fe10b7214086692a970f913796bf95f8af636a Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Mon, 11 May 2020 09:49:22 -0500 Subject: [PATCH 05/21] Add table of contents and polish --- blog/2020/05/invalidations.md | 71 ++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 35 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 5d0c1f94c0..1fa0b7f8d6 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -4,6 +4,7 @@ @def rss_pubdate = Date(2020, 5, 11) @def rss = """Julia is fast, but compiling Julia code takes time. This post analyzes why it's sometimes necessary to repeat that work, and what might be done to fix it.""" +\toc [The Julia programming language][Julia] has wonderful flexibility with types, and this allows you to combine packages in unanticipated ways to solve new kinds of problems. Crucially, it achieves this flexibility without sacrificing performance. @@ -24,7 +25,7 @@ Recently I got interested in a specific source of this latency, and this blog po When Julia compiles a method for specific types, it saves the resulting code so that it can be used by any caller. -This is crucial to performance, because it means generally compilation has to be done only once for a specific type. +This is crucial to performance, because it means that compilation generally has to be done only once for a specific type. To keep things really simple, I'm going to use a very artificial example. ``` @@ -146,8 +147,7 @@ However, if you try `@code_typed applyf(c)` again, you'll notice something curio Julia has gone to the trouble to create a new-and-improved implementation of `applyf`, one which also union-splits for `String`. This brings us to the topic of this blog post: the old compiled method has been *invalidated*. -Given new information---which here comes from defining or loading new methods--- -Julia changes its mind about how things should be implemented, +Given new information--which here comes from defining or loading new methods--Julia changes its mind about how things should be implemented, and this forces Julia to recompile `applyf`. If you add fourth and fifth methods, @@ -173,14 +173,14 @@ CodeInfo( There are now so many possibilities that Julia just gives up and uses "runtime dispatch" to decide what method of `f` to call. -It doesn't even try to enforce the fact that `f` returns and `Int`, +It doesn't even try to enforce the fact that `f` returns an `Int`, in part because determining such facts takes time (adding to compiler latency) and because functions with many methods typically tend to return multiple types anyway. Compiling each of these new implementations takes JIT-time. If Julia knew in advance that you'd arrive at this place, it would never have bothered to produce that first, heavily-optimized version of `applyf`. -But the performance benefits of such optimizations are so large that, when applicable, they are well worth it. +But the performance benefits of such optimizations are so large that, when applicable, they can be well worth it. For example, if you start a fresh Julia session and just define the `f(::Int)` and `f(::Bool)` methods, then @@ -214,9 +214,9 @@ If method invalidation happens often, this might contribute to making Julia "fee Unfortunately, method invalidation is pretty common. First, let's get some baseline statistics. Using the [MethodAnalysis] package (which is at a very early stage of development -at the time of this writing), you can find out that a fresh Julia session -(albeit one that has loaded the MethodAnalysis package and used it to perform some analysis) has almost 50,000 `MethodInstance`s tucked away in its cache. +at the time of this writing), you can find out that a fresh Julia session has almost 50,000 `MethodInstance`s tucked away in its cache. These are mostly for `Base` and the standard libraries. +(There are some additional `MethodInstance`s that get created to load the MethodAnalysis package and do this analysis, but these are surely a very small fraction of the total.) Using some not-yet merged work in both Julia itself and [SnoopCompile], we can count the number of invalidations when we load various packages into a fresh Julia session: @@ -237,8 +237,8 @@ Using some not-yet merged work in both Julia itself and [SnoopCompile], we can c | DifferentialEquations | 6.13.0 | 6777 | You can see that key packages used by large portions of the Julia ecosystem invalidate -hundreds or thousands of MethodInstances, sometimes more than 10% of the total -number of MethodInstances present before loading the package. +hundreds or thousands of `MethodInstance`s, sometimes more than 10% of the total +number of `MethodInstance`s present before loading the package. ## How serious is method invalidation? @@ -328,13 +328,13 @@ function applyf(container) end c = Any[1, false]; applyf(c) - -using SnoopCompile ``` Then, ```julia-repl +julia> using SnoopCompile + julia> invalidation_trees(@snoopr f(x::String) = 3) 1-element Array{SnoopCompile.MethodInvalidations,1}: insert f(x::String) in Main at REPL[7]:1 invalidated: @@ -344,16 +344,17 @@ julia> invalidation_trees(@snoopr f(x::String) = 3) Let's walk through this output a bit. `@snoopr` turns on some debugging code inside Julia, and then executes the supplied statment; it returns a fairly opaque list that can be parsed by `invalidation_trees`. -Entries in the returned array correspond to method additions (or deletions, if relevant) that trigger one or more invalidations. +Entries in the array returned by `invalidation_trees` correspond to method additions (or deletions, if relevant) that trigger one or more invalidations. In this case, the output means that the new `f(x::String)` method triggered an invalidation of `applyf(::Array{Any,1})`, due to intersection with the signature `f(::Any)`. `(0 children)` means that `applyf(::Vector{Any})` does not yet have any methods that called it and which in turn need to be invalidated. -Finally, `more specific` (which is printed in cyan) indicate that the new method was strictly more specific than the one that got invalidated. +Finally, `more specific` (which is printed in cyan) indicate that the new method `f(::String)` was strictly more specific than the signature `f(::Any)` used by the `applyf` `MethodInstance` that got invalidated. As we mentioned above, there are good reasons to think this invalidation is "necessary," meaning that it is an unavoidable consequence of the choices made to optimize runtime performance while also allowing one to dynamically extend functions. However, that doesn't mean there is nothing that you, as a developer, could do to eliminate this invalidation. Perhaps there is no real need to ever call `applyf` with a `Vector{Any}`; perhaps you can fix one of its upstream callers to supply a concretely-type vector. +Or perhaps you could define more `f` methods at the outset, so that Julia has a better understanding of the different types that `applyf` needs to handle. In some cases, though, you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. ### New methods with ambiguous specificity @@ -389,7 +390,7 @@ julia> trees = invalidation_trees(@snoopr using FixedPointNumbers) ``` This list is ordered from least- to most-consequential in terms of total number of invalidations. -The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 MethodInstances. +The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 `MethodInstance`s. (There is no guarantee that these methods are all disjoint from one another; the results are represented as a tree, where each node links to its callers.) In contrast, the first entry is responsible for just two invalidations. @@ -450,8 +451,8 @@ Type{Union{}} which shows that there is one Type, the "empty Type", that lies in their intersection. -There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation---if it were to be called by the compiled code, it would trigger an error anyway. -If such a change gets made to Julia, then all the ones marked "ambiguous" should magically disappear. +There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation--if it were to be called by the compiled code, it would trigger an error anyway. +If this gets changed in Julia, then all the ones marked "ambiguous" should magically disappear. Consequently, we can turn our attention to other cases. Let's look at the next item up the list: @@ -537,9 +538,9 @@ In the statistics below, we'll lump partial specialization in with ambiguity. ### Some summary statistics -Let's go back to our table above, and augment it with "sources" of invalidation: +Let's go back to our table above, and count the number of invalidations in each of these categories: -| Package | greater specificity | lesser specificity | ambiguity | +| Package | more specific | less specific | ambiguous | |:------- | ------------------:| --------:| -----:| | Example | 0 | 0 | 0 | 0 | | Revise | 6 | 0 | 0 | @@ -565,23 +566,22 @@ However, it appears that there will need to be a second round in which package d You may have noticed that two packages, `Example` and `Revise`, trigger far fewer invalidations that the rest of the packages in our analysis. `Example` is quite trivial, but `Revise` and its dependencies are quite large. How does it avoid this problem? -First, Revise does not extending very many Base methods; -most of its methods are to functions it "owns," and the same is true for its dependencies. +First, Revise does not extend very many Base methods; +most of its methods are for functions it "owns," and the same is true for its dependencies. Second, in the closing days of Julia 1.5's merge window, Revise (and Julia) underwent a process of tracking down invalidations and eliminating them; for comparison, on Julia 1.4, Revise triggers more than a 1000 non-unique invalidations. The success of this effort gives one hope that other packages too may one day have fewer invalidations. As stated above, there is reason to hope that most of the invalidations marked as "ambiguous" will be fixed by changes to Julia's compiler. -Here our focus is on those marked "more specific," since those are cases where it is hard to imagine a generic fix. +Here our focus is on those marked "more specific," since those are cases where it is harder to imagine a generic fix. -### Fixing a case of type-instability +### Fixing type instabilities -In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving a type-instability. +In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving type-instabilities. For example, one set of invalidations happened because `CodeTracking`, a dependency of Revise's, defines new methods for `Base.PkgId`. -It turns out that this triggered an invalidation of `_tryrequire_from_serialized`, which is used to load packages; -a negative consequence is that Revise introduced a slight latency upon loading the *next* package. -However, it turned out to be an easy fix: one section of `_tryrequire_from_serialized` had a passage +It turns out that this triggered an invalidation of `_tryrequire_from_serialized`, which is used to load packages. +Fortunately, it turned out to be an easy fix: one section of `_tryrequire_from_serialized` had a passage ``` for M in mod::Vector{Any} @@ -601,9 +601,11 @@ It sufficed to add immediately after the `for` statement to fix the problem. Not only does this fix the invalidation, but it lets the compiler generate better code. -The other case was similar: a call from `Pkg` of `keys` on an AbstractDict of unknown type -(due to a higher `@nospecialize` call). -Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default consequence of calling `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. +The other case was a call from `Pkg` of `keys` on an AbstractDict of unknown type +(due to a caller's `@nospecialize` annotation). +Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default return value of `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. +The benefits of this change in Pkg's code went far beyond helping Revise; any package depending on the OrderedCollections package (which is a dependency of Revise and what actually triggered the invalidation) got the same benefit. +With these and a few other relatively simple changes, loading Revise no longer forces Julia to recompile much of Pkg's code the next time you try to update packages. ### Redirecting call chains @@ -627,7 +629,7 @@ If you look up this definition, you'll see it's reduce_empty(op, T) = _empty_reduce_error() ``` -which indicates that it is the fallback method for reducing over an empty collection, and indeed calling this results in an error: +which indicates that it is the fallback method for reducing over an empty collection, and as you might expect from the name, calling it results in an error: ```julia-repl julia> op = Base.BottomRF(Base.max) @@ -642,7 +644,7 @@ Stacktrace: [4] top-level scope at REPL[36]:1 ``` -This essentially means that no "identity element" has been defined for this operation and type. +This essentially means that no "neutral element" has been defined for this operation and type. Can we avoid this fallback? One approach is to define the method directly: modify Julia to add @@ -678,7 +680,7 @@ julia> node = invtree.children[1] MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children ``` -We can display the whole tree using `show(node)`: +We can display this whole branch of the tree using `show(node)`: ```julia-repl julia> show(node) @@ -751,8 +753,8 @@ Julia's remarkable flexibility and outstanding code-generation open many new hor These advantages come with a few costs, and here we've explored one of them, method invalidation. While Julia's core developers have been aware of its cost for a long time, we're only now starting to get tools to analyze it in a manner suitable for a larger population of users and developers. -Because it's not been easy to measure previously, it would not be surprising if there are numerous opportunities to reduce it, waiting to be discovered. -One might hope that the next period of development might see significant strides in new ways of getting packages to work together without stomping on each other's toes. +Because it's not been easy to measure previously, it would not be surprising if there are numerous opportunities for improvement waiting to be discovered. +One might hope that the next period of development might see significant improvement in getting packages to work together without stomping on each other's toes. [Julia]: https://julialang.org/ [union-splitting]: https://julialang.org/blog/2018/08/union-splitting/ @@ -761,4 +763,3 @@ One might hope that the next period of development might see significant strides [PRJulia]: https://github.com/JuliaLang/julia/pull/35768 [PRSC]: https://github.com/timholy/SnoopCompile.jl/pull/79 [method ambiguity]: https://docs.julialang.org/en/latest/manual/methods/#man-ambiguities-1 -[sentinel]: https://en.wikipedia.org/wiki/Sentinel_value From 6b99fff88a2ce1b6da06dc9c07c8c0f5f918e2f5 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Mon, 11 May 2020 11:38:34 -0500 Subject: [PATCH 06/21] Discuss the effect of lack of specialization in the reduce_empty example --- blog/2020/05/invalidations.md | 51 ++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 1fa0b7f8d6..a1546f3d28 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -415,7 +415,7 @@ insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/ti 6 mt_cache -julia> mi, invtree = tree[:backedges,1] +julia> mi, node = tree[:backedges,1] MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) => MethodInstance for +(::AbstractChar, ::UInt8) at depth 0 with 157 children julia> mi.def @@ -465,7 +465,21 @@ insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in F `reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint` is strictly more specific than `reduce_empty(::Function, ::Type{T} where T)`. This might look like one of those "necessary" invalidations. -Below we'll analyze this in far greater detail and discover some possible fixes. +However, even though it's marked "more specific," the new method `reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint` can't be reached from a call `reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber})`: + +```julia-repl +julia> mi, node = tree[:backedges, 1] +MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 143 children + +julia> node.mi # this is the MethodInstance that called `mi` +MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) + +julia> typeintersect(tree.method.sig, node.mi.specTypes) +Union{} +``` + +This is is a consequence of the fact that Julia's compiler has chosen not to specialize the argument types, and `reduce_empty(::Function, ::Type{T} where T)` is broader than what could be determined from the caller. +Thus, while it looks like a case of greater specificity, in fact this is more analogous to the "partial specialization" described below. Moving backward another step, we get to `sizeof(::Type{X}) where X<:FixedPoint`. Simply put, this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. @@ -613,7 +627,7 @@ Let's return to our FixedPointNumbers `reduce_empty` example above. A little prodding as done above reveals that this corresponds to the definition ```julia-repl -julia> mi, invtree = tree[:backedges, 1] +julia> mi, node = tree[:backedges, 1] MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 136 children julia> mi @@ -635,18 +649,19 @@ which indicates that it is the fallback method for reducing over an empty collec julia> op = Base.BottomRF(Base.max) Base.BottomRF{typeof(max)}(max) -julia> Base.reduce_empty(op, VERSION) +julia> Base.reduce_empty(op, VersionNumber) ERROR: ArgumentError: reducing over an empty collection is not allowed Stacktrace: [1] _empty_reduce_error() at ./reduce.jl:299 - [2] reduce_empty(::Function, ::VersionNumber) at ./reduce.jl:309 - [3] reduce_empty(::Base.BottomRF{typeof(max)}, ::VersionNumber) at ./reduce.jl:324 - [4] top-level scope at REPL[36]:1 + [2] reduce_empty(::Function, ::Type{T} where T) at ./reduce.jl:309 + [3] reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{T} where T) at ./reduce.jl:324 + [4] top-level scope at REPL[2]:1 ``` This essentially means that no "neutral element" has been defined for this operation and type. -Can we avoid this fallback? +For the purposes of illustration, let's ignore the fact that this might be a case where the fix might in principle be made in the compiler. +Using ordinary Julia code, can we avoid this fallback? One approach is to define the method directly: modify Julia to add ``` @@ -654,16 +669,20 @@ reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = _empty_reduc ``` so that we get the same result but don't rely on the fallback. -But perhaps a better approach is to see who's calling it: -```julia-repl -julia> invtree.mi -MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) +Given our observation above that this *apparent* invalidating method is not actually reachable by this call chain, +another approach is to force the compiler to specialize the method by adding type parameters: -julia> invtree.mi.def -reduce_empty(op::Base.BottomRF, T) in Base at reduce.jl:324 +``` +reduce_empty(op::F, ::Type{T}) where {F,T} = _empty_reduce_error() +``` -julia> invtree.children +While there's little actual reason to force specialization on a method that just issues an error, in this case it does have the effect of allowing the compiler to realize that our new method is not reachable from this call path. + +For addressing this purely at the level of Julia code, perhaps the best approach is to see who's calling it: + +```julia-repl +julia> node.children 5-element Array{SnoopCompile.InstanceTree,1}: MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{Int64}) at depth 1 with 39 children @@ -676,7 +695,7 @@ This illustrates how to work with an `InstanceTree`: you access the MethodInstan Let's start with the first one: ```julia-repl -julia> node = invtree.children[1] +julia> node = node.children[1] MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children ``` From cd7378b097abcca06aa0022173389410f4dccb89 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Tue, 12 May 2020 12:04:50 -0500 Subject: [PATCH 07/21] Apply suggestions from code review Co-authored-by: Matt Bauman --- blog/2020/05/invalidations.md | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index a1546f3d28..949aba03ef 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -39,7 +39,15 @@ function applyf(container) end ``` -If you call `applyf([100, 200])`, it will compile a version of `applyf` +Here I've defined two functions; `f` is a function with two very simple methods, +and `arrayf` is a function with just one method that supports `Any` argument at all. +When you call `applyf`, Julia will compile _specialized_ versions on demand for the +particular types of `container` that you're using at that moment (even though I didn't +use a single type in its definition!). + +If you call `applyf([100, 200])`, Julia will compile and use a version of `applyf` specifically +created for `Vector{Int}`. Since the element type (`Int`) is a part of the `container`'s type, it +will compile this specialization knowing that only `f(::Int)` will be used: the call will be "hard-wired" into the code that Julia produces. You can see this using `@code_typed`: @@ -57,7 +65,7 @@ has elements indexable by 1 and 2. (Those `arrayref` statements enforce bounds-checking, and ensure that Julia will throw an appropriate error if you call `applyf([100])`.) -If you pass a `Vector{Bool}`, it will compile `applyf` again, this time specializing it for `x::Bool`: +If you pass a `Vector{Bool}`, it will compile `applyf` again, this time specializing it for `Bool` elements: ```julia-repl julia> @code_typed applyf([true,false]) @@ -70,7 +78,7 @@ CodeInfo( In this case, you can see that Julia knew those two `arrayref` statements would return a `Bool`, and since it knows the value of `f(::Bool)` it just went -ahead and computed the result for you. +ahead and computed the result at compile time for you. At the end of these experiments, hidden away in Julia's "method cache" there will be two `MethodInstance`s of `applyf`, one specialized for `Vector{Int}` and the other specialized for `Vector{Bool}`. @@ -143,7 +151,7 @@ f(::String) = 3 and Julia has prepared the way to make sure it will still give the right answer even if we add new methods to `f`. -However, if you try `@code_typed applyf(c)` again, you'll notice something curious: +However, if you try `@code_typed applyf(Any[1, false])` again, you'll notice something curious: Julia has gone to the trouble to create a new-and-improved implementation of `applyf`, one which also union-splits for `String`. This brings us to the topic of this blog post: the old compiled method has been *invalidated*. @@ -176,7 +184,7 @@ uses "runtime dispatch" to decide what method of `f` to call. It doesn't even try to enforce the fact that `f` returns an `Int`, in part because determining such facts takes time (adding to compiler latency) and because functions with many methods typically tend to return multiple types -anyway. +anyway. Adding further methods of `f` would no longer cause invalidations of this very generic implementation or any of its callers. Compiling each of these new implementations takes JIT-time. If Julia knew in advance that you'd arrive at this place, it would never have bothered to produce that first, heavily-optimized version of `applyf`. @@ -535,7 +543,7 @@ julia> trigger.children ``` and see all the `MethodInstance`s that called this one. -You'll notice three `_show_default` `MethodInstance`s here; +You'll notice three `_show_default` `MethodInstance`s with the bulk of the children here; a little digging reveals that this is defined as ``` @@ -627,6 +635,10 @@ Let's return to our FixedPointNumbers `reduce_empty` example above. A little prodding as done above reveals that this corresponds to the definition ```julia-repl +julia> tree = trees[end-1] + insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: + backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific + julia> mi, node = tree[:backedges, 1] MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 136 children From 69c5eaacdc0a5a462aa77bbe2b37ea2f83395ec9 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Tue, 12 May 2020 16:02:22 -0500 Subject: [PATCH 08/21] fix --- blog/2020/05/invalidations.md | 208 ++++++++++++---------------------- 1 file changed, 70 insertions(+), 138 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 949aba03ef..8208623248 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -404,7 +404,7 @@ the results are represented as a tree, where each node links to its callers.) In contrast, the first entry is responsible for just two invalidations. One does not have to look at this list for very long to see that the majority of the invalidated methods are due to [method ambiguity]. -Consider the line `backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered...`. +Consider the line `...char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32)`. We can see which method this is by the following: ```julia-repl @@ -418,21 +418,23 @@ or directly as julia> tree = trees[end] insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous - backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered MethodInstance for +(::AbstractChar, ::UInt8) (157 children) ambiguous - MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) triggered MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (197 children) ambiguous - 6 mt_cache + backedges: superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) (187 children) ambiguous + superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (198 children) ambiguous + 3 mt_cache +julia> tree.method +(::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 -julia> mi, node = tree[:backedges,1] -MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) => MethodInstance for +(::AbstractChar, ::UInt8) at depth 0 with 157 children +julia> node = tree[:backedges,1] +MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) at depth 0 with 187 children -julia> mi.def +julia> node.mi.def (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 ``` `trees[end]` selected the last (most consequential) method and the invalidations it triggered; indexing this with `:backedges` selected the category (`:mt_backedges`, `:backedges`, or `:mt_cache`), and the integer index selected the particular entry from that category. -This returns a pair `MethodInstance => InstanceTree`, where the latter is a type encoding the tree. -We'll see how to work with `InstanceTree`s in a moment, for now we want to focus on the `mi` portion of that pair. +This returns an `InstanceTree`, where the latter is a type encoding the tree. +(`:mt_backedges` will return a `sig=>node` pair, where `sig` is the invalidated signature.) You may find it surprising that this method signature is ambiguous with `(::Type{X})(x::Real) where X<:FixedPoint`: after all, an `AbstractChar` is quite different from a `FixedPoint` number. We can discover why with @@ -441,10 +443,10 @@ We can discover why with julia> tree.method.sig Tuple{Type{X},Real} where X<:FixedPoint -julia> mi.specTypes +julia> node.mi.specTypes Tuple{Type{T} where T<:AbstractChar,Int32} -julia> typeintersect(tree.method.sig, mi.specTypes) +julia> typeintersect(tree.method.sig, node.mi.specTypes) Tuple{Type{Union{}},Int32} ``` @@ -463,33 +465,8 @@ There are good reasons to believe that the right way to fix such methods is to e If this gets changed in Julia, then all the ones marked "ambiguous" should magically disappear. Consequently, we can turn our attention to other cases. -Let's look at the next item up the list: - -```julia-repl -julia> tree = trees[end-1] -insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: - backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific -``` - -`reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint` is strictly more specific than `reduce_empty(::Function, ::Type{T} where T)`. -This might look like one of those "necessary" invalidations. -However, even though it's marked "more specific," the new method `reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint` can't be reached from a call `reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber})`: - -```julia-repl -julia> mi, node = tree[:backedges, 1] -MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 143 children - -julia> node.mi # this is the MethodInstance that called `mi` -MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) - -julia> typeintersect(tree.method.sig, node.mi.specTypes) -Union{} -``` - -This is is a consequence of the fact that Julia's compiler has chosen not to specialize the argument types, and `reduce_empty(::Function, ::Type{T} where T)` is broader than what could be determined from the caller. -Thus, while it looks like a case of greater specificity, in fact this is more analogous to the "partial specialization" described below. - -Moving backward another step, we get to `sizeof(::Type{X}) where X<:FixedPoint`. +For now we'll skip `trees[end-1]`, and consider `tree[end-2]` which results from defining +`sizeof(::Type{X}) where X<:FixedPoint`. Simply put, this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. So we've discovered an easy place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. @@ -498,65 +475,42 @@ It is not clear why such methods should be invalidating, and this may be a Julia ### Partial specialization -If you try +Let's return now to ```julia-repl -julia> trees = invalidation_trees(@snoopr using StaticArrays) -``` - -you'll see a much longer output. -A large number of invalidations derive from the fact that StaticArrays defines a method for `!=`, which invalidates the fallback definition +julia> tree = trees[end-1] +insert reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 invalidated: + backedges: superseding reduce_empty(op, T) in Base at reduce.jl:309 with MethodInstance for reduce_empty(::Function, ::Type{T} where T) (137 children) more specific +julia> node = tree[:backedges, 1] +MethodInstance for reduce_empty(::Function, ::Type{T} where T) at depth 0 with 137 children ``` -!=(x, y) = !(x == y) -``` - -Since such definitions account for hundreds of nominal invalidations, it would be well worth considering whether it is possible to delete the custom `!=` methods. -For example, if they are purely for internal use you could modify each caller to -use the default method. -The vast majority of the rest appear to derive from ambiguities. -However, one more interesting case we've not seen before is +That certainly looks like a less specific method than the one we defined. +We can look at the callers of this `reduce_empty` method: ```julia-repl -julia> tree = trees[end-7] -insert unsafe_convert(::Type{Ptr{T}}, m::Base.RefValue{FA}) where {N, T, D, FA<:FieldArray{N,T,D}} in StaticArrays at /home/tim/.julia/packages/StaticArrays/mlIi1/src/FieldArray.jl:124 invalidated: - mt_backedges: signature Tuple{typeof(Base.unsafe_convert),Type{Ptr{_A}} where _A,Base.RefValue{_A} where _A} triggered MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) (159 children) more specific +julia> node.children +5-element Array{SnoopCompile.InstanceTree,1}: + MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 1 with 39 children + MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{Int64}) at depth 1 with 39 children + MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Pkg.Resolve.FieldValue}) at depth 1 with 21 children + MethodInstance for mapreduce_empty(::typeof(identity), ::Pkg.Resolve.var"#132#134"{Pkg.Resolve.var"#smx#133"{Pkg.Resolve.Graph,Pkg.Resolve.Messages}}, ::Type{Int64}) at depth 1 with 10 children + MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Int64}) at depth 1 with 23 children ``` -In this case, the signature that triggered the invalidation, `Base.unsafe_convert(::Type{Ptr{_A}} where _A, ::Base.RefValue{_A} where _A)`, -has been only partially specified: it depends on a type parameter `_A`. -Where does such a signature come from? -You can extract this line with +If we look at the source for these definitions, we can figure out that they'd call `reduce_empty` with one of two functions, `max` and `identity`. +Neither of these is consistent with the method for `add_sum` we've defined: ```julia-repl -julia> trigger = tree[:mt_backedges, 1] -MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) at depth 0 with 159 children - -julia> trigger.children -5-element Array{SnoopCompile.InstanceTree,1}: - MethodInstance for unsafe_convert(::Type{Ptr{Nothing}}, ::Base.RefValue{_A} where _A) at depth 1 with 0 children - MethodInstance for unsafe_convert(::Type{Ptr{T}}, ::Base.RefValue{Tuple{Vararg{T,N}}}) where {N, T} at depth 1 with 2 children - MethodInstance for _show_default(::Base.GenericIOBuffer{Array{UInt8,1}}, ::Any) at depth 1 with 113 children - MethodInstance for _show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at depth 1 with 37 children - MethodInstance for _show_default(::IOContext{REPL.Terminals.TTYTerminal}, ::Any) at depth 1 with 2 children +julia> tree.method +reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 ``` -and see all the `MethodInstance`s that called this one. -You'll notice three `_show_default` `MethodInstance`s with the bulk of the children here; -a little digging reveals that this is defined as - -``` -function _show_default(io::IO, @nospecialize(x)) - t = typeof(x) - ... -end -``` +What's happening here is that we're running up against the compiler's heuristics for specialization: +it's not actually possible that any of these callers would end up calling our new method, +but because the compiler decides to create a "generic" version of the method, the signature gets flagged by the invalidation machinery as matching. -So the `@nospecialize` annotation, designed to reduce the number of cases when `_show_default` needs to be recompiled, causes the methods *it* uses to become triggers for invalidation. -So here we see that a technique that very successfully reduces latencies also has a side effect of increasing the number of invalidations. -Fortunately, these cases of partial specialization also seem to count as ambiguities, and so if ambiguous matches are eliminated it should also solve partial specialization. -In the statistics below, we'll lump partial specialization in with ambiguity. ### Some summary statistics @@ -566,17 +520,17 @@ Let's go back to our table above, and count the number of invalidations in each |:------- | ------------------:| --------:| -----:| | Example | 0 | 0 | 0 | 0 | | Revise | 6 | 0 | 0 | -| FixedPointNumbers | 139 | 0 | 381 | -| SIMD | 3040 | 0 | 1017 | -| StaticArrays | 1382 | 13 | 2540 | -| Optim | 1385 | 13 | 2941 | -| Images | 1513 | 113 | 3102 | -| Flux | 1177 | 49 | 4107 | -| Plots | 1104 | 48 | 4604 | -| DataFrames | 2725 | 0 | 2680 | -| JuMP | 1549 | 14 | 5164 | -| Makie | 5147 | 92 | 4145 | -| DifferentialEquations | 3776 | 53 | 7419 | +| FixedPointNumbers | 170 | 0 | 387 | +| SIMD | 3903 | 0 | 187 | +| StaticArrays | 989 | 0 | 3133 | +| Optim | 1643 | 0 | 2921 | +| Images | 1749 | 14 | 3671 | +| Flux | 1991 | 26 | 3460 | +| Plots | 1542 | 11 | 4302 | +| DataFrames | 4919 | 0 | 783 | +| JuMP | 2145 | 0 | 4670 | +| Makie | 6233 | 46 | 5526 | +| DifferentialEquations | 5152 | 18 | 6218 | The numbers in this table don't add up to those in the first, for a variety of reasons (here there is no attempt to remove duplicates, here we don't count "mt_cache" invalidations which were included in the first table, etc.). In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column indicates invalidations that should either be fixed in packages, Julia's own code, or will need to remain unfixed. @@ -635,17 +589,10 @@ Let's return to our FixedPointNumbers `reduce_empty` example above. A little prodding as done above reveals that this corresponds to the definition ```julia-repl -julia> tree = trees[end-1] - insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: - backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific - -julia> mi, node = tree[:backedges, 1] -MethodInstance for reduce_empty(::Function, ::Type{T} where T) => MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 0 with 136 children - -julia> mi -MethodInstance for reduce_empty(::Function, ::Type{T} where T) +julia> node = tree[:backedges, 1] +MethodInstance for reduce_empty(::Function, ::Type{T} where T) at depth 0 with 137 children -julia> mi.def +julia> node.mi.def reduce_empty(op, T) in Base at reduce.jl:309 ``` @@ -691,43 +638,25 @@ reduce_empty(op::F, ::Type{T}) where {F,T} = _empty_reduce_error() While there's little actual reason to force specialization on a method that just issues an error, in this case it does have the effect of allowing the compiler to realize that our new method is not reachable from this call path. -For addressing this purely at the level of Julia code, perhaps the best approach is to see who's calling it: - -```julia-repl -julia> node.children -5-element Array{SnoopCompile.InstanceTree,1}: - MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children - MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{Int64}) at depth 1 with 39 children - MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Pkg.Resolve.FieldValue}) at depth 1 with 21 children - MethodInstance for mapreduce_empty(::typeof(identity), ::Pkg.Resolve.var"#132#134"{Pkg.Resolve.var"#smx#133"{Pkg.Resolve.Graph,Pkg.Resolve.Messages}}, ::Type{Int64}) at depth 1 with 10 children - MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Int64}) at depth 1 with 23 children -``` - -This illustrates how to work with an `InstanceTree`: you access the MethodInstance through `.mi` and its callers through `.children`. -Let's start with the first one: - -```julia-repl -julia> node = node.children[1] -MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) at depth 1 with 38 children -``` - -We can display this whole branch of the tree using `show(node)`: +For addressing this purely at the level of Julia code, perhaps the best approach is to see who's calling it. We looked at `node.children` above, but now let's get a more expansive view: ```julia-repl julia> show(node) - MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) - MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}) - MethodInstance for foldl_impl(::Base.BottomRF{typeof(max)}, ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) - MethodInstance for mapfoldl_impl(::typeof(identity), ::typeof(max), ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) - MethodInstance for #mapfoldl#201(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapfoldl), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for mapfoldl(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for #mapreduce#205(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapreduce), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for mapreduce(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for maximum(::Set{VersionNumber}) - MethodInstance for set_maximum_version_registry!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec) - MethodInstance for collect_project!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::Dict{Base.UUID,Array{Pkg.Types.PackageSpec,1}}) - MethodInstance for collect_fixed!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Base.UUID,String}) - MethodInstance for resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) +julia> show(node) +MethodInstance for reduce_empty(::Function, ::Type{T} where T) + MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) + MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) + MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}) + MethodInstance for foldl_impl(::Base.BottomRF{typeof(max)}, ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) + MethodInstance for mapfoldl_impl(::typeof(identity), ::typeof(max), ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) + MethodInstance for #mapfoldl#201(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapfoldl), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for mapfoldl(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for #mapreduce#205(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapreduce), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for mapreduce(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) + MethodInstance for maximum(::Set{VersionNumber}) + MethodInstance for set_maximum_version_registry!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec) + MethodInstance for collect_project!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::Dict{Base.UUID,Array{Pkg.Types.PackageSpec,1}}) + MethodInstance for collect_fixed!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Base.UUID,String}) ⋮ ``` @@ -778,6 +707,9 @@ However, it's a bit uglier than the original; perhaps a nicer approach would be to allow one to supply `init` as a keyword argument to `maximum` itself. While this is not supported on Julia versions up through 1.5, it's a feature that seems to make sense, and this analysis suggests that it might also allow developers to make code more robust against certain kinds of invalidation. +As this hopefully illustrates, there's often more than one way to "fix" an invalidation. +Finding the best approach may require that we as a community develop experience with this novel consideration. + ## Summary Julia's remarkable flexibility and outstanding code-generation open many new horizons. From a443aad94e19436676fd0594fff4959cc6d64326 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Tue, 12 May 2020 16:09:49 -0500 Subject: [PATCH 09/21] more fix --- blog/2020/05/invalidations.md | 38 +++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 8208623248..bb56386d81 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -15,7 +15,7 @@ when you first run Julia code. This problem is often summarized as "time-to-first-plot," though there is nothing specific about plotting other than the fact that plotting libraries tend to involve large code bases, and these must be JIT-compiled. Many people have spent a lot of time analyzing and reducing Julia's latency. -These efforts have met with considerable success: the upcoming Julia 1.5 feels "snappier" than any recent version I've used. +These efforts have met with considerable success: the upcoming Julia 1.5 feels snappier than any recent version I've used. But the job of reducing latency is not over yet. Recently I got interested in a specific source of this latency, and this blog post is a summary of some of what I've learned about the scope of this problem and opportunities for further improvement. @@ -371,30 +371,34 @@ Now let's try a real-world case, where the outcomes are more complex. ```julia-repl julia> trees = invalidation_trees(@snoopr using FixedPointNumbers) -5-element Array{SnoopCompile.MethodInvalidations,1}: - insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: - backedges: MethodInstance for promote_rule(::Type{Union{}}, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) less specific - MethodInstance for promote_rule(::Type{S} where S<:Integer, ::Type{Float64}) triggered MethodInstance for promote_type(::Type{Float64}, ::Type{S} where S<:Integer) (0 children) ambiguous - 3 mt_cache - +6-element Array{SnoopCompile.MethodInvalidations,1}: insert one(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:94 invalidated: - mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (1 children) ambiguous + mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (0 children) ambiguous 1 mt_cache + insert oneunit(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:93 invalidated: + backedges: superseding oneunit(::Type{T}) where T in Base at number.jl:300 with MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (1 children) more specific + 3 mt_cache + + insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: + backedges: superseding promote_rule(::Type{var"#s822"} where var"#s822"<:AbstractIrrational, ::Type{T}) where T<:Real in Base at irrationals.jl:42 with MethodInstance for promote_rule(::Type{Union{}}, ::Type{Float64}) (1 children) ambiguous + superseding promote_rule(::Type{var"#s92"} where var"#s92", ::Type{var"#s91"} where var"#s91") in Base at promotion.jl:235 with MethodInstance for promote_rule(::Type{S} where S<:Integer, ::Type{Float64}) (1 children) more specific + 6 mt_cache + insert sizeof(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:100 invalidated: - backedges: MethodInstance for sizeof(::DataType) triggered MethodInstance for Base.CyclePadding(::DataType) (25 children) ambiguous - MethodInstance for sizeof(::Type) triggered MethodInstance for padding(::DataType) (3 children) more specific - MethodInstance for sizeof(::Type{T} where T) triggered MethodInstance for array_subpadding(::Type{T} where T, ::Type{T} where T) (0 children) more specific - 7 mt_cache + backedges: superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::DataType) (26 children) more specific + superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::Type) (4 children) more specific + superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::Type{T} where T) (1 children) more specific + 4 mt_cache - insert reduce_empty(::typeof(Base.mul_prod), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:225 invalidated: - backedges: MethodInstance for reduce_empty(::Function, ::Type{T} where T) triggered MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) (136 children) more specific + insert reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 invalidated: + backedges: superseding reduce_empty(op, T) in Base at reduce.jl:309 with MethodInstance for reduce_empty(::Function, ::Type{T} where T) (137 children) more specific insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous - backedges: MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) triggered MethodInstance for +(::AbstractChar, ::UInt8) (157 children) ambiguous - MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) triggered MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (197 children) ambiguous - 6 mt_cache + backedges: superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) (187 children) ambiguous + superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (198 children) ambiguous + 3 mt_cache ``` This list is ordered from least- to most-consequential in terms of total number of invalidations. From 3ee5511f4b431f25bcadf52ac7f7cb360718f2bc Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Tue, 12 May 2020 16:38:40 -0500 Subject: [PATCH 10/21] Cleanups --- blog/2020/05/invalidations.md | 37 ++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index bb56386d81..ada64a0b46 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -42,8 +42,8 @@ end Here I've defined two functions; `f` is a function with two very simple methods, and `arrayf` is a function with just one method that supports `Any` argument at all. When you call `applyf`, Julia will compile _specialized_ versions on demand for the -particular types of `container` that you're using at that moment (even though I didn't -use a single type in its definition!). +particular types of `container` that you're using at that moment, even though I didn't +specify a single type in its definition. If you call `applyf([100, 200])`, Julia will compile and use a version of `applyf` specifically created for `Vector{Int}`. Since the element type (`Int`) is a part of the `container`'s type, it @@ -80,12 +80,12 @@ In this case, you can see that Julia knew those two `arrayref` statements would return a `Bool`, and since it knows the value of `f(::Bool)` it just went ahead and computed the result at compile time for you. -At the end of these experiments, hidden away in Julia's "method cache" there will +After calling `applyf` with both sets of arguments, hidden away in Julia's "method cache" there will be two `MethodInstance`s of `applyf`, one specialized for `Vector{Int}` and the other specialized for `Vector{Bool}`. You don't normally see these, but Julia manages them for you; anytime you write code that calls `applyf`, it checks to see if this previous compilation work can be reused. -For the purpose of this blog post, things start to get especially interesting if we try the following: +For the purpose of this blog post, things start to get especially interesting if use a container that can store elements with different types (here, type `Any`): ```julia-repl julia> c = Any[1, false]; @@ -215,7 +215,7 @@ julia> @btime applyf($c) It's almost a tenfold difference. If `applyf` is performance-critical, you'll be very happy that Julia tries to give you the best version it can, given the available information. But this leaves the door open to invalidation, which means recompilation the next time you use `applyf`. -If method invalidation happens often, this might contribute to making Julia "feel" sluggish. +If method invalidation happens often, this might contribute to making Julia feel sluggish. ## How common is method invalidation? @@ -405,7 +405,7 @@ This list is ordered from least- to most-consequential in terms of total number The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 `MethodInstance`s. (There is no guarantee that these methods are all disjoint from one another; the results are represented as a tree, where each node links to its callers.) -In contrast, the first entry is responsible for just two invalidations. +In contrast, the first three entries are responsible for a tiny handful of invalidations. One does not have to look at this list for very long to see that the majority of the invalidated methods are due to [method ambiguity]. Consider the line `...char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32)`. @@ -471,11 +471,11 @@ Consequently, we can turn our attention to other cases. For now we'll skip `trees[end-1]`, and consider `tree[end-2]` which results from defining `sizeof(::Type{X}) where X<:FixedPoint`. -Simply put, this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. +There's a perfectly good default definition, and this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. So we've discovered an easy place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. -You'll also notice one example where the new method is *less specific*. -It is not clear why such methods should be invalidating, and this may be a Julia bug. +Rarely (in other packages) you'll notice cases where the new method is *less specific*. +It is not clear why such methods should be invalidating, and this may be either a SnoopCompile or Julia bug. ### Partial specialization @@ -490,7 +490,9 @@ julia> node = tree[:backedges, 1] MethodInstance for reduce_empty(::Function, ::Type{T} where T) at depth 0 with 137 children ``` -That certainly looks like a less specific method than the one we defined. +Our new method certainly looks more specific method than the one that triggered the invalidation, +so at face value this looks like one of those "necessary" invalidations we'd be hard-pressed to avoid. +However, appearances can be deceptive. We can look at the callers of this `reduce_empty` method: ```julia-repl @@ -523,7 +525,7 @@ Let's go back to our table above, and count the number of invalidations in each | Package | more specific | less specific | ambiguous | |:------- | ------------------:| --------:| -----:| | Example | 0 | 0 | 0 | 0 | -| Revise | 6 | 0 | 0 | +| Revise | 7 | 0 | 0 | | FixedPointNumbers | 170 | 0 | 387 | | SIMD | 3903 | 0 | 187 | | StaticArrays | 989 | 0 | 3133 | @@ -537,8 +539,8 @@ Let's go back to our table above, and count the number of invalidations in each | DifferentialEquations | 5152 | 18 | 6218 | The numbers in this table don't add up to those in the first, for a variety of reasons (here there is no attempt to remove duplicates, here we don't count "mt_cache" invalidations which were included in the first table, etc.). -In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column indicates invalidations that should either be fixed in packages, Julia's own code, or will need to remain unfixed. -The good news is that these counts reveal that much will likely be fixed by "automated" means. +In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column is a mixture of ones that might be removed by changes in Julia (if they are due to partial specialization) or ones that should either be fixed in packages, Base and the standard libraries, or will need to remain unfixed. +The good news is that these counts reveal that more than half of all invalidations will likely be fixed by "automated" means. However, it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. ## Fixing invalidations @@ -559,7 +561,7 @@ Here our focus is on those marked "more specific," since those are cases where i ### Fixing type instabilities In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving type-instabilities. -For example, one set of invalidations happened because `CodeTracking`, a dependency of Revise's, defines new methods for `Base.PkgId`. +For example, one set of invalidations happened because CodeTracking, a dependency of Revise's, defines new methods for `Base.PkgId`. It turns out that this triggered an invalidation of `_tryrequire_from_serialized`, which is used to load packages. Fortunately, it turned out to be an easy fix: one section of `_tryrequire_from_serialized` had a passage @@ -582,7 +584,7 @@ immediately after the `for` statement to fix the problem. Not only does this fix the invalidation, but it lets the compiler generate better code. The other case was a call from `Pkg` of `keys` on an AbstractDict of unknown type -(due to a caller's `@nospecialize` annotation). +(due to partial specialization). Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default return value of `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. The benefits of this change in Pkg's code went far beyond helping Revise; any package depending on the OrderedCollections package (which is a dependency of Revise and what actually triggered the invalidation) got the same benefit. With these and a few other relatively simple changes, loading Revise no longer forces Julia to recompile much of Pkg's code the next time you try to update packages. @@ -646,7 +648,6 @@ For addressing this purely at the level of Julia code, perhaps the best approach ```julia-repl julia> show(node) -julia> show(node) MethodInstance for reduce_empty(::Function, ::Type{T} where T) MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) @@ -712,7 +713,7 @@ perhaps a nicer approach would be to allow one to supply `init` as a keyword arg While this is not supported on Julia versions up through 1.5, it's a feature that seems to make sense, and this analysis suggests that it might also allow developers to make code more robust against certain kinds of invalidation. As this hopefully illustrates, there's often more than one way to "fix" an invalidation. -Finding the best approach may require that we as a community develop experience with this novel consideration. +Finding the best approach may require some experimentation. ## Summary @@ -721,7 +722,7 @@ These advantages come with a few costs, and here we've explored one of them, met While Julia's core developers have been aware of its cost for a long time, we're only now starting to get tools to analyze it in a manner suitable for a larger population of users and developers. Because it's not been easy to measure previously, it would not be surprising if there are numerous opportunities for improvement waiting to be discovered. -One might hope that the next period of development might see significant improvement in getting packages to work together without stomping on each other's toes. +One might hope that the next period of development might see significant improvement in getting packages to work together gracefully without stomping on each other's toes. [Julia]: https://julialang.org/ [union-splitting]: https://julialang.org/blog/2018/08/union-splitting/ From 2ea09cf45a155c2538ca457e967d81879ceebe55 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 13 May 2020 14:33:18 -0500 Subject: [PATCH 11/21] Apply suggestions from code review Co-authored-by: Ian Fiske <135570+ianfiske@users.noreply.github.com> --- blog/2020/05/invalidations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index ada64a0b46..0cc5cdf477 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -40,7 +40,7 @@ end ``` Here I've defined two functions; `f` is a function with two very simple methods, -and `arrayf` is a function with just one method that supports `Any` argument at all. +and `applyf` is a function with just one method that supports `Any` argument at all. When you call `applyf`, Julia will compile _specialized_ versions on demand for the particular types of `container` that you're using at that moment, even though I didn't specify a single type in its definition. From b3d410aaab982d4060643d001a0c978007f2fb3e Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 13 May 2020 14:36:33 -0500 Subject: [PATCH 12/21] tkf suggestion --- blog/2020/05/invalidations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 0cc5cdf477..10e68749da 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -630,7 +630,7 @@ Using ordinary Julia code, can we avoid this fallback? One approach is to define the method directly: modify Julia to add ``` -reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = _empty_reduce_error() +reduce_empty(::typeof(max), ::Type{VersionNumber}) = _empty_reduce_error() ``` so that we get the same result but don't rely on the fallback. From 700a9f9e977be01112cf50ca0dc3dfbab866436c Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 13 May 2020 14:47:07 -0500 Subject: [PATCH 13/21] Get away from KeySet solution --- blog/2020/05/invalidations.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index 10e68749da..ff043346b0 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -584,8 +584,9 @@ immediately after the `for` statement to fix the problem. Not only does this fix the invalidation, but it lets the compiler generate better code. The other case was a call from `Pkg` of `keys` on an AbstractDict of unknown type -(due to partial specialization). -Replacing `keys(dct)` with `Base.KeySet(dct)` (which is the default return value of `keys`) eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. +(due to inference failure). +Resolving that inference problem eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. + The benefits of this change in Pkg's code went far beyond helping Revise; any package depending on the OrderedCollections package (which is a dependency of Revise and what actually triggered the invalidation) got the same benefit. With these and a few other relatively simple changes, loading Revise no longer forces Julia to recompile much of Pkg's code the next time you try to update packages. From d70c02abf82a7c861dc47c00f4490b59b6823f4b Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Sat, 1 Aug 2020 12:36:54 -0500 Subject: [PATCH 14/21] Better explanation of fixing inference --- blog/2020/05/invalidations.md | 81 +++++++++++++++++++++++++++-------- 1 file changed, 63 insertions(+), 18 deletions(-) diff --git a/blog/2020/05/invalidations.md b/blog/2020/05/invalidations.md index ff043346b0..f3316e76c5 100644 --- a/blog/2020/05/invalidations.md +++ b/blog/2020/05/invalidations.md @@ -560,35 +560,74 @@ Here our focus is on those marked "more specific," since those are cases where i ### Fixing type instabilities -In engineering Julia and Revise to reduce invalidations, at least two cases were fixed by resolving type-instabilities. -For example, one set of invalidations happened because CodeTracking, a dependency of Revise's, defines new methods for `Base.PkgId`. -It turns out that this triggered an invalidation of `_tryrequire_from_serialized`, which is used to load packages. -Fortunately, it turned out to be an easy fix: one section of `_tryrequire_from_serialized` had a passage +Most of the time, Julia is very good at inferring a concrete type for each object, +and when successful this eliminates the risk of invalidations. +However, as illustrated by our `applyf` example, certain input types can make inference impossible. +Fortunately, it's possible to write your code in ways that eliminate or reduce the impact of such invalidations. + +An excellent resource for avoiding inference problems is Julia's [performance tips] page. +That these tips appear on a page about performance, rather than invalidation, is a happy state of affairs: +not only are you making your (or Julia's) code more robust against invalidation, you're almost certainly making it faster. + +Virtually all the type-related tips on that page can be used to reduce invalidations. +Here, we'll present a couple of examples and then focus on some of the more subtle issues. + +#### Add annotations for containers with abstractly-typed elements + +As the performance page indicates, when working with containers, concrete-typing is best: +`Vector{Int}` will typically be faster in usage and more robust to invalidation than `Vector{Any}`. +When possible, using concrete typing is highly recommended. +However, there are cases where you sometimes need elements to have an abstract type. +In such cases, one common fix is to annotate elements at the point of usage. + +For instance, Julia's [IOContext] structure is defined roughly as ``` -for M in mod::Vector{Any} - if PkgId(M) == modkey && module_build_id(M) === build_id - return M - end +struct IOContext{IO_t <: IO} <: AbstractPipe + io::IO_t + dict::ImmutableDict{Symbol, Any} end ``` -and since `M` had type `Any`, the compiler couldn't predict which version of `PkgId` would be called. -It sufficed to add +There are good reasons to use a value-type of `Any`, but that makes it impossible for the compiler to infer the type of any object looked up in an IOContext. +Fortunately, you can help! +For example, the documentation specifies that the `:color` setting should be a `Bool`, and since it appears in documentation it's something we can safely enforce. +Changing ``` - M = M::Module +iscolor = get(io, :color, false) ``` -immediately after the `for` statement to fix the problem. -Not only does this fix the invalidation, but it lets the compiler generate better code. +to either of + +``` +iscolor = get(io, :color, false)::Bool # the rhs is Bool-valued +iscolor::Bool = get(io, :color, false) # `iscolor` must be Bool throughout scope +``` + +makes computations performed with `iscolor` robust against invalidation. +For example, the SIMD package defines a new method for `!`, which is typically union-split on non-inferred arguments. +Julia's `with_output_color` function computes `!iscolor`, +and without the type annotation it becomes vulnerable to invalidation. +Adding the annotation fixed hundreds of invalidations in methods that directly or indirectly call `with_output_color`. +(Such annotations are not necessary for uses like `if iscolor...`, `iscolor ? a : b`, or `iscolor && return nothing` because these are built into the language and do not rely on dispatch.) + +#### Force runtime dispatch + +In some circumstances it may not be possible to add a type annotation: `f(x)::Bool` will throw an error if `f(x)` does not return a `Bool`. +To avoid breaking generic code, sometimes it's necessary to have an alternate strategy. + + +Above we looked at cases where Julia can't specialize the code due to containers with abstract elements. +There are also circumstances where specialization is undesirable because it would force Julia to compile too many variants. +For example, consider Julia's `methods(f::Function)`: +by default, Julia specializes `myfunc(f::Function)` for the particular function `f`, but for something like `methods` which might be called hundreds or thousands of times with different `f`s and which is not performance-critical, that amount of compilation would be counterproductive. +Consequently, Julia allows you to annotate an argument with `@nospecialize`, and so `methods` is defined as `function methods(@nospecialize(f), ...)`. + +`@nospecialize` is a very important and effective tool for reducing compiler latency, but ironically it can also make you more vulnerable to invalidation. +For example, consider the following definition: -The other case was a call from `Pkg` of `keys` on an AbstractDict of unknown type -(due to inference failure). -Resolving that inference problem eliminated a very consequential invalidation, one that triggered seconds-long latencies in the next `Pkg` command after loading Revise. -The benefits of this change in Pkg's code went far beyond helping Revise; any package depending on the OrderedCollections package (which is a dependency of Revise and what actually triggered the invalidation) got the same benefit. -With these and a few other relatively simple changes, loading Revise no longer forces Julia to recompile much of Pkg's code the next time you try to update packages. ### Redirecting call chains @@ -716,6 +755,10 @@ While this is not supported on Julia versions up through 1.5, it's a feature tha As this hopefully illustrates, there's often more than one way to "fix" an invalidation. Finding the best approach may require some experimentation. +## Notes + +MethodInstances with no backedges may be called by runtime dispatch. (Not sure how those get `::Any` type annotations, though.) + ## Summary Julia's remarkable flexibility and outstanding code-generation open many new horizons. @@ -732,3 +775,5 @@ One might hope that the next period of development might see significant improve [PRJulia]: https://github.com/JuliaLang/julia/pull/35768 [PRSC]: https://github.com/timholy/SnoopCompile.jl/pull/79 [method ambiguity]: https://docs.julialang.org/en/latest/manual/methods/#man-ambiguities-1 +[IOContext]: https://docs.julialang.org/en/latest/base/io-network/#Base.IOContext +[performance tips]: https://docs.julialang.org/en/latest/manual/performance-tips/ From 3fd655e675f28e34ee96872150103b7d79ef5ab2 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 12 Aug 2020 17:40:07 -0500 Subject: [PATCH 15/21] Streamline the invalidations blog post --- .../blog/2020-invalidations/SIMD_ascend.png | Bin 0 -> 46781 bytes .../2020-invalidations/SIMD_invalidations.png | Bin 0 -> 159095 bytes blog/2020/05/invalidations.md | 818 +++++------------- 3 files changed, 203 insertions(+), 615 deletions(-) create mode 100644 _assets/blog/2020-invalidations/SIMD_ascend.png create mode 100644 _assets/blog/2020-invalidations/SIMD_invalidations.png diff --git a/_assets/blog/2020-invalidations/SIMD_ascend.png b/_assets/blog/2020-invalidations/SIMD_ascend.png new file mode 100644 index 0000000000000000000000000000000000000000..a45d372b36580e0e496e79925b1194ee80e5e3c1 GIT binary patch literal 46781 zcmc$_1yo#1yDbVKf#AU%5(pOD-GT*ocXxN~;10pvgF6Iw4<6hlSR)OMHqh85d+-0< z|Gjt5Iq%%@#<)G!SY5rUdaYVt*;jMUFG5jX5*3LM2?hoRRqBhFG7Joy!%KSiE&R(F zU6I)P+T|JDP&0s9-9qh~)TuhwJ%YZ|0v8FIm{H{_D$z zFef5Fk+T9E4k=jXGy=P-=x=o|9>^iv2o-RazkKsY0Vv=h2ZdLt9{D5v%3lMDMd zWTt85zsy}7v{jw_g<)l38*};MRhGdXeAs^~?bM3kK!ruAjbG=&w!J!FISZBMAP=gQ zJJek=qm-MV$S+X*k&bAA_ogPrTo-we!=SfLR$SW4Q@PsNX&jxG?P8LR!II*}6wYswM9kZ2e zBU3pRm|)w>El|79MQq$yeZXDVc`!)!xo@31b@{uBZmkN$MI}4W_N-;7@14uu=v0po z_^2#(;U(3h{m?oN_OnOih-pwsbVTBaZ(>P4K+&mx%Ci(>@QSL2HT(INS7bD3GL|Nb zH6gsyHKcS^o<12xk`J@_Vc>fi-WRuufrdhfa|jcXQKLiy&yzMUa)d^~m)Fr7w3=qu zmxYvz5SMPLzbbNCvf3qaLwqV2)qXtR-_Yp-$6mp4Gjd038M<*9tEb2t=R1qgWVeZv!g<3DVhOfK`pML;_aW5*gmWQ@GxP!x(XB(*@00k$R-0K%4ws22VR@-S0pl zgzcauvcuuZd>s3^kdZJ6u>TlSfaDUHOWL(g_kN>=UpoHEp|Q(%+BCkAnA#QYhS|gi>^WdvOOKAKlty{_)`OZ zp8%uGNONCRd(7^@rFl7$-5bX>oSc2$IMzoD%Y3TFwi&zaZZ+Iy-8mA5nGm^Tv*1A~io>8g^=JVi34Ky}~~! zW;EQp+aZn?>?xlQD)Tzl(D$CaxnJ;i1<9P}O%Dsz-EN^ZiND;cQi$kge1F2dYq@)3 zR>i#&n<)d(kV1lSOt9D$zQ#uVGN-J4NDNFk`JCZD)E6SBjxFrOu|3=4sc@<^qN`nI zXT!MDvTWaW!z>wVYy@3WXN==q_ z9C1Wc3YWohWK=4q-PI?82exwo^Vfu>b0)9*QQRR>Kh*q3BLle+$euXGtlQ?~F8}KjcpDi8WxZG9bTfL){%8GhlXF zv9Vow;E_ab*}!pOr3T8HiiC&EFrn^DEMGcBfP02*Sf`K~Ag_;$qfO*Au}32D)O=#i z$D;{8A~c#o^^pzpG=a4AS#_8 zgrvQzB(&&Uf@qwj&J%X`WDPb;SD$^brbLp$lU224*+6h8CJ-k~yy2Y(V=$XY69U!{ z^~Me$6G4SD%;9Xl;qt)1f;G&6NdnDwM1gd8xLrUr^uFF^*Twf-M|Qu}k@>C5BZ;hk zBhE$?u2Y%ilNGrd$;mK${hLUM%mErDEd4YH$DR z+HBJM=&>G*dw*XY(3Vk+%|8?PpfhsLwr~0`Q23 z2cbLW`}b&OaNg{uexpfJzDr%FN}(7hV~tO;z=n(q(g*zUJDkEup<$G5Kis#=SRl7a z>WcOmi>7 z>o{9pTQ59AsBO4H64Dk$HSCSbwNkAi@s;bu4~i&?vK-b*gN%V{t5X{-4lFzL_f27J zKYSQIWu#(MP)sDh@+J?w9`Kc5*H7L_6mPt;oZJKknU|}@9ZlMMPF|Qi$8q`Bq>bZG zCVfwXEpYoDp^9Ams9QMbuzQJ{t)*E^CZlh3Ar8Irhkp?NxsuZ6Y zAN%yMLTrv14U9It)If+M>qtw2GD!H#ug{UUu~HgQ0ZuA4(a}}4TclMeu~N6-5V_G&6A()E>q_14jLMr8fDUyJL2l6F+YtYm~8lXCp`r2Kb1lX z9SoWQ@}15(XJR3v;{1x?eWit1Q|&kT&;B?br!2j~u03(y67wV66LX()e06r7q<}Xq^SL zOo(9BkuFNRxx_9N&+}waq{Hb9VaWA47LJ9>?#o!AE<6Ply{RcbzsD)i9}r=#l40}Q zU6DX3>2xaQ+9dn{P2o7vMllL%ACG$#$r0TRMK?c>0QIXf2&6<2@Xi{F-cO6ly}uBH ziKl+W{o|TnJ`8W#Mc6MTdM&0JOm17GRi}N)o8sY>;hgC~>kEgQohatc)*9G-bxL2yUQ(AUxJ;|daAgtp7(BY?aBor%WlOx zt6Ng>xb#J3T9p|eynCtwCr)@{cp%wX3eNUnq$e7lnYnjc7yu@0F~+&Xj_+A5t>HyI zvjj4HZAEfB;(OO|K)3(cnwmHa$J534HL8s36&?rwS1i1kxx*`B!8>9!4re?zzRtd| zinYVXd-4tcg=6o{!tM>3nOe0XU9#~4pl^5X$#AjXUVC=Om_6OSGpf~W6^mIGl4a1J zHG<+NB)u7h&S+Jw9KycEw-JNqV}Fk#X8hu- zz!nt7?7##Lxw*UfZ0FS85e~YEbd-Xf-udAGs9`V4b&5 z6fr7*Vk{Cj>ae~5YKfRarFydahyHMx=HYfDhK`dNZoKRyzXcJQ-K%LA{0&g=u^&?d zhEj-=(TS|B6oL3_ zj-*Xg?nk2X@nga|104pOCiUe-oUV)Pr?@($(sBYCUjH%8>oLZzjAft8Hoau zbQJD&c=%-1uq2hs&88>~@Ff+9JR4nA_@`|0R)3I&V-4gXF#O%Bp$}l-|YQxPRYo zy*Gd$)gtTq-Iarav`BRuy{wd~MG&b`8}We|aWNt`smf|#z0%n&H5mo#UFzf$#)zc@ zpJIJ^I;)z9-1HZ;ku7V(1r*)D5K)2cB`oVfXRd&_?eOW1Y4s=N$af46!Eir= zSW~EjQZq_U3VqeMSKB{6LgG%welFETXIP1s1iXE9Z^fS)`N#jprTq62bwxw1Z2 zqEVeL_~}qAvmc>$Rn%rR{iD`Yhs@X!EBALF!Ij zjm@)34)eh*F ztyKk)k#-WoGf$|UzvdlzfWQiim5Fie>K}|nY@>dvBVATC{YYX$nGn%JP1Y=W3Dzl- zVL(tUC^#%rV7~pjK>CPw(m=XzODle;KU9(09Sdwf_T&%iGMI8jybx!vpt}bSaN(f} zSS{?|jg~w&#%{DOr9B@QScPIsu`v#=IWg1VNAemcl|>_nJ>Ti)owhwc0D)m@I%5#^#s!+WybnnmtO|k^*gi=b@<#;^Lm*HjFMBNP29ZCy9_Yg9Q0Hha3De}^t3qb-0|9_C zoMh&lS>QK^)h{NTo<(cqM)Tu~Bqh7W!lc|(`~z2c>@=Aretw&u3pZGvRPse$ArDrV z;^cy0QfwoND;oLFn$53G3QcuI{y=J8ri;isl8lgz`sZ@EWKaLuC)Pj7KSOOb*xJX71;43nC_L218(r4onW2p} zDe1izMc2OJ_y)Ck1(2`(wrIM_;m*`Lvxm&KNWH56a$SrZIMYzl;BYv=Mz-`@d+zJ9 zJjLT@cq17V?jP(y?5NCyWmk1@u?RD7pSXf{fAI0WgZYCB`Thz6W-Ju@9vXYLtrouc zu#UX_?{)Wp?8PTCw0nm*JrT0C^#dEz{dzeWhy72EY_D-6l_j>_` z4$!}j&cA`P;&A0l;bRK@+a}vW2TfZHkI^B>nUyqOx2*?C0^)t@GFV)!ib8v*!*5iE zbS;4NQo@naYb?8*+lF~Nbahq#&L@kkhwN%N?$X=lGGv$jPm^D zH0mNxQGLQ0BwYWYRIDk?WWzy_IyNOcJ>7e^{NCLV-&LU?09Ds;v@l$MEDI+}0krL8$@egL661F_Z89wa2IIDtyq<41|HB%iOex&0z$wfP2~##d)RY4=~- zBBm|SKN$Q|p>3|0irP%|D|k44wacfLlma6)wprfPpAXr_lcZ~w5rnueebn)MMB1dr z5GE->Lia?ZJz;WW*(z)GoBR?B$10o>wgDgExJSvD1Xo6MeMxYQ#5h#d{U*w6aEw+A zXA9MoU^osVO+SG;Be~s)EU%1Mf%9_Enf~j$s4n)!qT+;N?w#$YUkYFUWl!c+iw)Mw zU^|VnmD#UgPcD5>kUfqR1cndv69__H9QtUgA5^qp3Rv@Oq4eDZnD(|HVYRWjU)P<{fFB)cKOG2dDl4- zk|nRq>**PnPvmW7|GLqQ$EeJNW_}H^P}wD`@_X#Y~ax zf>8PW(w45Zc*4&cLE2 z^Zxq#NP>TC;s?e#-#WVWuny^V&UvTymbe(5sl=TAjejQ9IL5u@;oKQ^z7RS|CfF_2 z+2A{0c&Bs2&E63CSr9WqNo|t}5(_KI$A2bLbTNooA{CZeahp;@l0=^tjtT*VyYQMh zavf&NKb>#@F7Z){z&vJ&;ZV2rbu3QIQju1~14o3NkomV+^3VS)mG5qb?Z&A7sl{i0 z-(7<1f5wN>+b?HS|2!_FCkS~18_ycz6Ner)*B+EQvDdE|pl&ieQ*_1)) zM!v0nbgPWHh6cf80J|*tMZ!1wQE!G?@9rNky5d5>b1m($)nAX;rygkjMHJzTdpR+! z{D$N2PBLCkbK`jZygQ!zD6{Hf^E01XPGoj|h;&F$gjPDyN+yj1qE()9^Q{QKz>yU< z-*^_SqtP0}L1@Pfd|%YJR3lh=z9itj=;vTW-;U}vd3?Ke-gsezZz8m9->Tkz-gkjT zyuueas>RT!=tk2$m~9IAYRsvT@ISeugAV}#GZd*V%y$wL*9vI8LiRk5pEfdQuYc3} z^eT6(Ts&>Z&&6m@0@uA+T2}9Ug&eR-7}qy}@t;VmJOMxn&R_(`vm+%=68)jM2$C|8 zyA7S1BfUzzGHFgyg)Z@>9;m_a=*QMY;c))~+Q}%V$yS!Z?fn`H#DJ#uzp z1k(MHb}ef?moka!&0+xSBecKnS@xVoOu&1q17d4$=P{A17auwu63xWMpU z-!D3egg<-gK3!|J89leE%N3mSsS>+zCo%{W$1@c8yNImnz0#;Zc+9@DrySPr*3<>V zn=x%vW)FPA{9tsfaoytJrM^~nJ*0}8%Rl~dtdNb{aRI@j_pW9YU6tKY>_|ygq_7^*aYdyOmH36IOsX48HuJD1G=!vg=e%P| zC{|_k6ei>ZN{Bb+$lXR{@|kIT5JLaT5gVT9WJoia{9RWt$$>B?={q)i1L9>B1yTcI zV#;Ud1O_8n*ttWv10PgMURLN1RQI8nPh~-`wWKD9CCN^qgc^<6rJkhb+B5z_X!7GO z#-Vo+=%x&G2)yKyO)65tSkAgO4?=GA8m1=$7O5t9ils+=p1K5+!saSIO0|8Aa$!6g|r zVNNL!=Tm?@`XJTJ3_8e85%QPaLru-W>j4H=PVU6d*(3eOJ5F}*P3?c)i%PNJGIK zvC#?r9!3VdY+@SLSP|VWAQ5D&*rHQ6` zy1^Azf3Uo}*c+#hr}#Q(Oy;_|*4(sBdkEv$4m@Aa0X4rjqo}#DiFzhykJDH>+Mb{@ zJKC-8T_SJpXRelslhmq(apUM+3O^i{G7T1z>XrCTWa(utIriqlor{3tu1pqwA#wV% zRbY}43ipr-aZcupw_Wbj{7VZ!bRsz1-3PNlq)h$Y0NeOm=Z?HX#{)nw&N1G9QT$1;rs70|pqMn$6Heb=DtFn&XlD z?x$(|NQC@-K+yBO{sC$+7$kwUf4N%M32O6>6rD!Ud<3gPIqBSYw6(wO)IFq?&v=QkY2ML- z0d1Jw0F4XL?h#w%jOlg z&47UYW@Ws-yZlV-n4FvLTQ!fe4d}Bus=UQsaNO5F|%jS|H}+Rcb}pRz#dJ+!@}%kh&&4rr|2<SFAE(hI8KAH7g1c*qUX(y@8|{uJ z7Or^xw)lKbApx5S{D_EMde>qBn-F}YTTWl!-Bh8y{&=90fy=%!lKY04lOR^gwOm^) zbjHQ^f=1)0$Y%AIlEGKyjt@+zR&S7ncE%zsBBX1L-RUAR3Eed|pL9gkYkm7yeV>f! zMgat3U&ghEguGy=O)yp_wZ)!^h149~G8|f6_o!`j_D$s{(|-oOLoPBP9Lj6}?qF^P z#4QdSwc{U*HpDh52?JBNm$RCx^xkIh#;t3(Fo|>l#bK4OFr|jz5)n~i%}RA8C_qbZ z*c%8Kej`PccRg5oAE_$n9US?%{ZPv)dmj1RMfAPa>q7Fukr^qn(fGDwgI{YfM2o0l zC9aNLZbhBkXk7yoIaVm1K02K@Z_?r;ykow{Tph9BbO(9a{e8Mzlj$P9wMKAKqT&SS z2tUhnqv=F>PA;i{OHYP{|GR?lZ+jA;6mz#`^1o29#rC-Q0mKM_(=&P zHQ!x{AN8FHq_6pVz`-w?nJCf7%w5aAuM!2@n0&pU4X;MONJs_qpeuy|Css!9YRD&-^{T9o;RQ{A@~ zfzE;l3tM`XTAe}cTf%wNsBjx4UdZVF2KaE4yv&ZmTJj2qJ-6w-ngQ#p@+KNy96=Jz z7EiHeVyWH%IeJlB&raxq>29z%OO0wpPv79NJA>eiI~~&)o=?e9m&mq0KYp!aw>uqS z`k-W)c6J5%npQ=;(w6&`hR!j%aJeb30rU@D`kcdUt`JnsR=QgII!>}il!gNMLScN_muOKT?0mONk6^T4W|+(?`!9$bwx)w=P!E?hfy>W zGQUlNzg@j%%$A1{C7&K?^(%%sX76^^y56NuGy^;M3i-(GNB`V^Kd!cTx&I5|+ig#q zdht|x%71J{l?TG!k?bqy$e&hb8dh-9B^A5uLi}SXqVVX5jEqnDr=9LsR`o9yj zS(jVvx$HxPc3#!vBKr=JlC#+EHRxMGT-qvbGfJ=QdF3uZ$3O7hE7uf;t}Lvyp%#lkdmfc;+=bM`K7n_^C%j8he3Le zj{<0G$NadpfRMK2*roXwOZ9BB16tl$05^DQM92sSp z=SugzJy`MO^HiacLtJbPt}Gnp`Ep4&3oh+H)bM;Faf^2XoW{+$mLy&@rHnF&@f1Og zDVLgm`fG{D{*I*F-`q3YAMSaR;xF#mq*^U1(wtfSFDm!p<|s>~o>E9-5-X>4OMiF8 zxlr|?>o(X;8qb_$bD*OYd^YNW$)quA5kaN)5r5smB)f z6LI;fde(Q^JZ;7TcrLB}Yh%zhI|JUMsLfuSc!GC3S_(k}!|{0;%NZJEzNV;oM*nZK%s^wBp?HT}+1cYdxjm2&Zq3T4t5f0KH;zm9s&s<2HAf|SO;-A>v?oG==is*hWCAmKnrcRF z0+=`0Hu!*Rvhi)K2)(+Qbgcs;Yqp(=)U!m6%3l1fj1lyrXq?_F}^)V6KeeVr1$B z8(Tc!xHm3R?MP?z zdzpq{w~osYxJ2NO!c$*1YgcRyHKNNG|IeicS_j>e*A1J=>Dahqc#RMj|F$5>`!R{`Do?>BZHfu4b!1V+-$p?ym*~L0twZo>m zsX<8Ti43$SZC*3m%#=a1Wo&mjuKodGh;+&!tJ*hvl4nV0p)I_nD1gnKC+La%Lelz+ zt-677iZhPOeoV19x(-|*u(?7Vu7fsz4IBLCM+@?6(oW$H`0~NXeNbS-D+1W3*Tt!M z?sf>|fuFfuW~Y|frk5Rj6<#D0PD6u_ld;}I=O!;Db|cL9?AN|qeHu=j@U0_8em~&M zAVg`kR6ZjB072bS@B)T$F97l{d-B*DWzy@tgHWI&enV;cjgIsytH+B==SN5odage zzWwnW74L-$n~N7A=gr_YAleW)Z|EIK(YA6OvG~Q9G^y9ZET=7BHww1ZT2MaObwSq+ zLw{&R`%>5H=NNxKFY@NktIqnCxb(QFgM&ObVd=Z^5;EZ*7Pbel)^k2MUBk_)q-FK$XoKzuk%aGJNIP|F*E3DEM4dkh~Q*IZD=&&Gpp^tIi#tG;P>+OK2(2c-^4tIV%Qirjz#Z@0kss{{liLK8P9M9hXH3xJZIZoMjuEe?g)yZ!- zhPrZ1rX_pjqH3{0df6y%RQ7XzJe%*U zRw2N&jWuY`|M&OSh2CM|CNoj0nwg|)r-8=rY>@?9dX1QIgT?P~oM*A9FZX(|3Wug? zmgD@Mj0TEUch+lhH{7kVqU1**ClWbkQ*ZZ8zlH$yy|PcAx}CZC34(l3BUe{%CnTV- zxPZ2Zg$62Sk3kc9t+udVbcgtQo#%V_bn0(R8uPnOQ>vk#6yeuC&3%{f!>J*ZJLENg ze?7@)x-_n{lA7}Kun@@Hu^hB`uWv4sL{t<#d}p#pTlq$cKR*Qq?Y_DVlsE z9~%{lUTARL@UY+{w=l7xvD^TKIh+kWn?|UkU-w6xZ$KMFT}z4MzpN~@v*ev(f*E%^ z(Mx1C7ghOifUr^nh&p_3sq=(Ty_%%{tyzqF9YwqscE~4HyVvn+ z8&SXu(d1c!U~Pw$Z-@Vk@+(e7kOOg0$Be1&Z)H^+Z%%ZhNy!hT1hr0TtbQBS1L?G> ztj$I${uec~YscG3)m^DS9#+fU-EqbiP|YtS*=~RFCQevP+;Vz{Xq&gBRT=3T0->`g zWDD`d`k8lJ%@*ZAwUWCR6#VR1y-T??B6y0Pv20aJK3W@D2O$5-4ojDPiVgkjSkb-7r7++5CjQ>HnBC)lc`$?xKvAnF}#>bT6qb6=bV$kG?r@A30|o4}}83C?lEy5CU$ z=h5mRNoU5V{mnEQ!pwja_vEu#JgHyp6aqWEI;_Pzx4T|dKg>|RX>gwIXkICOWAJow zewo9?oQ@-j82Te|EV8n2!WI5Qi2~(40v0Em$ejdRa6ZJ=?@Yj0F zGRDtKUD44bW8oMox7DQ*SLb%)$50+23xU>m3Ea%p=>3dwMwZX$>5yu=p&y=%2qz}% zhc-7Z_%ze^HvZ2Y8`lnK8*4vexfsIIpFFv65BvgS{v%aK^V*kmjpZtfJG-qfYl1`) zFEG@3N*Kw)uBS0Y>~Fk`q}2_VJj^=RkDW|6CO=CjQL?7P$S09#H71>W4^5x6NMy6j zMg7(}1g_PD>8G>!FeV%NfpwnI$xAK)#1A&ae%(;AfMx-KDe0}~^YL2E`cr6qgxAz$&hmGibJ31iRqKB=#_z8mg% z#Roid?Pw|Pn6!PW$T%(5d;Fn*XGj0>cZ>!< zy|o(U?nWx)FDs&Eq0o)jBz9_@11pON+Yfh|P`NY_i~Em#=wTL3g!35;-TBCPwZ{`9 z9q0CN4;!13u>uN$ZW$JYgE661$34@@Ssi=G^}C{BJa)cO#SH6rJBwMI8OX;xPgzZ& zf&kah5I`-%aLolOwqHFhM^vk^{bDTi6oBbYt)i+7`K<2VQ%B`D)HK-g1gD%B`wD<@+i6(J9v_9mKj&N6f#r6o?<_J zz%QaHq5hnsb#EogYiuABrO{dCmmMR~Qmpm}kLLBKnCn)a!n?dt@~(`=G=;gGXdV4C zg%gIikvatBYR)1~HR8Y#QQD?$k`&Sj9#0kxTd66d#{xeAD>dsQ_QTVz0|kd*agfA)Rjr^kVE|C2v(TtZ%zj5Rbz!FyL1?H|sZD57odctU&HBswp1XU3{#@xa(|zIDnHZZ?5}EK&YcY7E%$e5OG3ks9 z$xOA)p4@%2F^mtI0_Hc-g#hWXT=71pO-_T|^c7d1PcUAl}6 zW8ML3aL}`neex?^=H%2-nVr8II*wDN8X%=G{Jd_eOeDOkjI2(?nNi0Ta^fRGdn}yU zD8Jh#tT4;X$AMU)h-F2j5;0jzdEAE^ib+E7=;X)fKbuW^T+RQhN3QwGo9AT%)3jIZ&ay6=bx^yT>U5@Tpu*Yg4ozZ}~v^;%Pi`N;~ zu2P}AU(+FUm8bCyPme0iZR(evj`p%&GO`<#_iT4cK54|rs>n}rTW{RJ;(-UB?B_=KdgTTb zHfG&^BO!g80fG4m%<C=CFo$fA+=`P~44-eN^LGf6 zE<%bte;w@PYWg}Ntdc7@(k$f0_?Tas^KF7KnrBUZWVU@x2FAHCLt!e^(Tn%(KD7QD z0}%ZqYxGnpu3Dwbe6m{3wqKh4&sUN;C}i~S4&ZsZ zh`cV}7Q3U8D~taTjDBysU%oxDBf4R9)@L=qKOa@}L0ysG>h9WC(2pe}8Eeyclz`1l zr(?9I#GW?o-;wCMQkp@cimdE=&Jk;4GIz z|NjGit>LF#T$u}F9%HB~^_L2#6U=FO^@If=ta z{(Obj^J;G8I<~a2x|k7-TB>%JdaS(O)S#|0rb?B!s)u{OOA6UNpyS+U&~(VL_~RhT zyL8du0(~9@&w|f;|YeexL!Y1IrhHMFA--3yT_{B@zLR>Wvxn|=d zDCVAwZL;3Mhg-CLv7o?gGh;j>Hxoc38fpO6U_S?<&3ee-@-dX!+?=Ubz?Og_R&2GI&I2 zxD|uc%KSm`;9L0-R7D#DdmU$=l@w9#_`eAK8{V~(Ua@vlu$;8N^6Z9?@GtD@XXW3r zQBTul!T9|0?|N@g;5*9qADtM)`^(LX-5BHV_y)85zSjKLks@%OgNw27m+Oc1(=k+q zAlly*7!88S>yO4-=gvq<-noWW)t|GWXs?-|kzRn)z1?%+STNd_u{GLz1KdWk)zo9NY zNi5fY!7!Wx*Q$RniqDw;P3aiX&tWsiL+qqVtAfdz)1-D3`D!hzNz->E=+%N~>{{l# z&f%fTjF;Iii?6sDNQtfSKdgN|BeTkavSlBX@^NDl^k$*do~_Z<%&tFcpoLMJIM#@q z+d80mY(*&TSSEpn1)rI>XHL%THIW_K*(b%R%_cto?BI*YI>8L^W*sw+_TRSo<9|o+hfAt&^kzDnq=1H6bSRv? z$?rkgrPC|HI%0Jm(^qj94VJdK2N2w8^)kpUM$y*S=ug8evwy zT-BKkZ`ay8Vx%LWde4~sNRcqT;htxz#%5L?m{AZJ46^m=+Hu?`KKl8iHs^)nlHEn7poHqN|f z@;2oq$z2vD09f$ugaRh&32+oGN*t*NqC^)jx;Kj{`^@M_v9fP{C5YB6yyEr8)uUEY zn|GM0sCiheW{lOB5Gcu6eRPoKM zTN|jc;>b8mY4vVJX$Dzayg=_XoGX8x3rh8zC=;t{MQLSQ4^+FFo*WJU9wOlWyzr~G&N4E?4`gvV1(UEt6z~4>Gq4`gU+<6qQ|O>WYsPrX5$fuVc+K|nn`5pi9}Ljv z_pRC5^X0~L;&A4=^vr=|jNRfny?{&PtjDYKNGFS_^SvvFi*1ROv*B9P{cMYK$sc%( zvl;xk2=A56cc>^=#;umn7Y=QV`%m4 z#1Q2Fyj`$G@h|r*sciY^wKHUZBy`7bF{~fR#X8$XfFoA1+ZvCB-jJt<4s9YBhX?m* zgs3J+kR`~Wo43mEH{R;`av3t4z>ja2Oi;h=)Uy_%Ih#ZALy@5}Qg0XJsH!ZxmsX3; zy?dIQBWTD1gJO7?ZE2z)oBrmxVTxRY(X%xTUN!bw=`mIGxRfa}K2d%Yq4mG2FoT*H z^A$SrZj8eI>M76F)TEphC(wqg7A(7M^xD1S-1F|Lp{B#kA5AQjlJMu6cRbTnSXfCY z<>hN~Ovd@XlvY=H<@cYi$A%qEBiO>2lD4Pb;wJ>e8I723Hv^=ezpm_tzD~G{z9}x> z$SiZw!Dzo=!zF9e@$NppsbMyxcP77EHTo1YX=^qUc*DbS1rxD-kNbDCY4%EvB%rS2 zO%|B!lgR=ckc7b8BOyD(%gq-$`wTZ9_zaYH1JF!sWPo-UggsyQ;7%o%$G8Tt;>ozXw zPZ2eEe6BsRsDAWGPsJ9St_FGK0J(|C`jSln^7`&(kY5r>i|zf^vh&T7%wRAaaSZCN zvN8;?n#A}zcnE#Wyv2{;tCNm-NU-zW8gMCn!MIfC%LaR&p)U2Jzp7Nyz*sruBVLVR z&@f```b9rAgckz?<52W!QBD`cXz*Whv4q62Zw_u zxVvkR;OPls_XRb-FvO|+t;;PbIR^)qDhZ$ z~1xoRlsdUJ`f zt|j}QJuTijZ7H*1&H=?{7ZGay)pS=_EKLfm~6X<;e+{uGC2|!)z%7UyN z>4fJ1{vtv1{%-?yggp}h`*YdW8G0*+Qz1!l)_%E0i81CbBK>-w_Ipciqi=1O{Y^u= zYzXwxF+UsU)ad|sF%&X7`GZP8_5vmPlEq~BJe7QdODR30XPQ!cuz!1H0pbrtNHX)8 zhf`~QG34pozLoa_{Vh0d9E^OjDoL1?k(pf@$>giv#yFi~NAKX5xtcF_E_9;2SmWoq zXz&m+dhc}^7Ox>hGUNHwvc{tD1MH&^s<_CR0ecEQ0++s$*^2a@v1?dv=DgHA_;F*$ zMP6T|1}AGxrS?p0~m&IjmI&>ZRL6#Z8h<*kZJ4L!na47Sg02q#U`~ez96#J zM5(&v5^Ndapeeqc?!Sauq1r$o26O<80uc!|>?Fnl--}wk&wK0IQ$QYC$cT~!FMU*! z;T{77+<(*44J903!O=TdPnZP;&ek7H=DjZt_W4cPTm@drSio+n+IM1Qc;4)tt!3U- zIp}S&CAFNH{BY|8T%DB z0WO%dl6-x?l-ED@TEF-80`G~GOrr%#XSH>r$b7C<+d&=S>#0M?_~AN@4NL>xcH+XFFxerPn$%mh@Fi7LnO>3 zekSQAY&AiUgW(+R1ukSMW21nI%paDgKOXTsj=A?GE;q)EfQ8icO%I)3Ot9<(v z8>2Woj$tD~otJF6h|=8%3)-vxTCAlY9@((-i0qwmW&0KH-Avf^SYV0A>EPr|p#{uH zZLUUwP6ee23@Q%URF0+hEZ;nEM#nI`httG0v8f7DbI=6$=eYlD^kZFe=5ekmJF)bH zWUn<#Ij4O-rkR)gt{88|#unNI8bu`JaB&1|x=`Tln2i+y%IFYHwjeLUNK!BM9UgvT z=>?ZVt*#ooBlU{4No*XAySCfqZn>f1hc2jO>Vb7F;(}~ff1@JK~kD+2%zjQL~Cp5qJ>f_W!DzN z*@jWIQTN`yJE?0CZk}_01O~VaG`f?I#Lt)$);-Hqln&!q@qRCc-lt%(q_zyqnh}@N zo*X2DnJ~`bJFeCfnac*Hf_z1jrKQ7UAHpy{r&5%Wc6q~EW7&bW2`+DQ{)$YGw38i^ zeWIwz)<1N2lCO2-b;3EYRZ1iP4j3;2!DZ@c8IytzHY=&`NjbI(J^U3r35_;#hUyw( z3GGCu#jDNlC9ZvI7kWB<$~W>9&a>fEAKkT8!y($7EPJQ2QKMPC2X-PUES9j4M+PI} zU@$dPIjZK7(?dYdkvNwg} z2wiRzIl}pk!Wxz1PS~a$$!IjRu^a?sBoAT`u{ySio^c8wR715$sno?r4mf|MJO~wkq|0}Jx>CV-X^Y5t^?QQ8nF4nuf?eE4rH`^0i2#IQP=+|TF&HjOD4N!hHxgO@-yN2upduTlG(53T=2=vbx@6 z;bnBV#ku5SS!Q7k%6_X%!v0%(3;5VAOko^cF-)g2bvF7lOdCEU!z9`)1G(Gr+N>%s ziV5brByg8JbVm!nltQ7EQNoReZvx)@|Fe=5Bwzg~Vb@hvtOR2moM2{3q%|Kh{>lzM z{gUcZfIcu<;x(t|NA?}MB28+l5ES}7<~(Y}nWsvwKhGd2QQ{XlT>Z3I2jsFgFEbLl zwL*d^wYXRk73YjJaXW{)+ptxP3v#T37;&7ktX^J)lW-3t(AcIc_K~Fz1~{Ai5w$W9 zxW_&4$B|kXG&@T9q6a$jLf(klNemQj30E6X$A`A=20Vdw_ez3AR)JF@o9JoVa+r=3=tliL3XYdfL=;W<(+_%~X zmF-zHg{5V}T(sbd9@l2WwuU2+(AA@@707Sr4*QGdl`JO0Z#VK2TOg|VV(e*l3>;y1KosunxI1>wezGuOo)SAFVE&$VJAnCXUAZK1@|DqKH3|z>CCRYM0e<0ZQ;2i#Ny=yp?=6?tRB~)-u+q?5 zOAk9TbvG^t6JMMmk5gLg&P4FyvI)bBT4kZW+FPd@kmJE0IK?#rmRU)0DR?0{MurDF zWs$oRrm#4V68iQJ{xc83YZ!BL!MLREH+&iCj0P9T)`yt{GMh=wU+#GmRIZogLt(#v zN-lSob>0T!+S+IDiTzL-k;=$o`7(eZ@x_cGFIkL2Q7?8EGQ$&p)PHoTU9EXR4R+)H z`b*5oWk%=r1yi`h=I3J`)dY_b$Ap1a;)5*VR?35{)6K8LDo?*1x_wdj@|A4+E z+KqzIG?v+F*rOi(>N_psXt}pw;l!6AkS*!H>wBm0w~C9n`1lwwTm#qpwEoZtCi(O$ ztE9_9XH{@t_-?(dSqZCOZg$pTarBn_<^qR5rqVZ>rent7Eg7t)E#fQ0C!IgI0|xub zSq|3zTlpDMk^0xd9Ym?uh7y1#nN)ANTEZ;Xg7n*Ah0Df`(L5NSdC>IU;;-V9uiQi{{ zRHmJeQbAq==bG_5+?=*dJ9+)z1lg)>A_@}8TTHeqvih6vJQ{$jtk=t>e_J?^LoMU~ zcJ%gZ`aMO6{I5Lc&r}B{WIX+^KmB$_826`eTaF@+)IJ+7OJZuCcEUO~`kbq?lXGuJ zI{U=I%4Rn>zouYs*Yet0X_2xWI){$x@1>`-PW+$c9`>}a)$Ujum<|$h+ij{9UwDj? z`u95V;V=!hZ(pKh&{{-%rcy?3R-rx_mX}pF3!cep@JTs%@>kA-6M#yp0m{AuJ4(O* zMev+}AJ{gMLl&7sU1B^zf~6o+A{%VF;^68PtbVNd(#U|&9WJ69`bgk;xz}MUduT<< zWB+y+-SCB@jZNB37px^3>$=KIt5vouosi2Cvdc@5&-k)?^!{NBdry0 z&GlSQ>2D)(hJ_oY5tds51a1(n4rKnE#?p~Wyrc@uJJPeZO7TU%dhVykLagXpE`}1$ zmu}B!BRqX^la&Z%;S)nQAmlGlR`K{~lKY)$htnz9APL)r>|XGfYDUTxVM zqwFd$%ydgcP^n zG0siqlx{>aRF)C5eX_bc5$Dg0TI8Ag=FoT_ZSx9Ma@3y6(}-D3D`igj{2?ww z99EYb^0pnj%%;auL-B}vR>_8H>ML(L5xNTqE@tDYO_YLP2hC1Gk%cuv{WYTkAU#Ya zY0j<(6Zkj7$+?sRJW3=VJ~eHQU*5=jLAu=Tmm&@Jo7BI>0v>f*7-84$s?R~)g%eVk zcr$kr>bE*~J4Vt>S%HaC()4a}gH|-%!A!92OxE3bC^CJ2vD54!@ zG*tDJDDo=h)$QeFbWB!O9jgWu@=LRe{!zpBrHHx_P-jp5q`xc$$AWtX*{?=l9#C z@Si#&Ri8<`1`oVSX=z#H-c!?t$!6r@c@UrA36agp*_sU z6s0VcwannRTpuu_p8iy{>WGKVTDg-yH12j8XBmq=)L$lggpvAMNIa}JKg>|*zWsd4 ztim4*R`h0mqx=ISGg&vnK8*y2(B!H!JX*6(XNs_>|2GB8cs@kOW;`3xLY>)D1zI;psF)zJdVth z)EWaQc}pS|2WteLp|nPIo2`{-w8f{fWV#MJ*-VCPmMb~()IoK^Uq7ka(aEpDYu9_O%oZYU%!Z&mNs@%y4Fq}!LF z2Cu8ZuGKweiDb?A=5getup#WmN2IqG$Wr(H=jyr%?^jMN2door+7VmP?2)keI~nPC z5&ZHXgS@MJ8J1)fIIo{$hS9@pjG3}7x)e8LG8{O`l!)|*b)jAfPQkN|j1ICNG+BDo z5yxCoRk{+I+DWW&)Uh}#`$$|Wq}bU5K>3bOS~Nw|l7m!7oZ=el z)qcIoIy`%Ff3#s;4BS#Hio|zZ}Z8hdTcZ0z!1i7>^)_M-r?V1i!j-_U==E~ zDw?ivoXy_H2))p5ak_nl&jjQ^@yQ=r>`ev@hg~^jm#FDlrRAe;de+|eCs3eO3X&2* z%zt)L+dfwA*Eh^50+TMZ`3I?~^9{qRW0}Xyo2y)OIPiIIjaoxd9jEz~2v3UgpQ*wu zhpNA+Rs_^bi|hx!WIPVXDhyaw@w7$ASh`Bi&ubKKW;N`#(5SwAMi58>v`j@V9Cr zwZGF$v8~PUjn|>G5_d(>*x-9V6&^e*bc>2P$5qM52s}Ym?z%LDv=Oylsh@GR7F03; zC9*&cbt1)E7R}qyuOIx$c-YFL@%{#_^$=d|Z$8sYsYmbdm|}S2JDaXvrL(1Wgk@sy zUj#`Fb@~WgXX+;hKHb4{lil<@9Kmq_()uaSmkkwp=I@#mu>1DFQ$_fk-Q(W@w0P{+ zQpQd)>#nW?PkJZqvYM$@Ohu2m0}LHqi4Nh`&0?~Cpv6UH`~lq(Z0=mM z?Dt2FWqEt&MPML9yjF^X*v*N&+S@>Cxa!hSQ?u;Rs!=`BI{isQgtH0{J`}yVNF_WF z-$FFviR@{@$d^U?gyUa^n~{#QcHa?%jIQW@6>=*rzskI-XulFZYCG@H+(J>^qWn&*}>v2oQL7VG1GZX%D+rQE~B{ zxzkndR?FNJ|G~|tX<|AgiAi6buxV#_s`VFmfaBgAZc!Jn+FwPWKz<1~qy@XM)zl0o zjaC;G`4wtCDG2#ou~tZ|jK2Yk2lRsXl4Qhs!D25hDoUBe6-`*u%)oKP3sNqSFB?EF zDuq9gQHyx*m*q@Baq$|a!nTh~febX7&UoDHv1D>UE#w2~ zWyn;39>;#Z86yA62I;A5_e*iW|5Q`GoB8!Tc-UHG#1DUr^r9#Mf^Rb!d`mlQ1lhaAte@1vyF zy1n6ZT8tm3EqA1KM{E%VlF3OGZ$?C8e!(M+?+K!q47ZjqbGj?;nfrrSY=*IPV#8RG zQJkEKu*Z*BG+PrC!{WpP4i_|RRYEd6)xHWHl+IJQsmj=IHa=Izq{jZ5@O#z#ng`e;MOOv-Wj!Zo~JJ=@u-o}w8`4Jtu|aknO9i)UOPAO z>_ZmNtHVmusgt?u&&%be9Q>s*TP|IElh1`<CspFnmd|yQ5atrS)QE_5>5} zfhlUQ1S_L}bY8gWc}D#;-!u70w8nM|KrYehMJMipvQM1D1YwKHl*;pI^ z1ov;UnX~I#FKoK@q~M(j7g0LpN$k(|V&6LsF6gKp;uIE&aFv?1FUy`3^!k=|3P7oxs z@tT#+#v8A>&Wlqb{tpuawsYCVh+}7Zsn{-^Rn~Mv!Dq1x^iq%|oE|?iXdfQuxkt;x zd;qPmo^7PU)<%C42h@#*{2LP*Vsx!f=jz zqA_p!a-HHvg>obw$k>$~wjwOlA}3+YqSgZ6;N7VC)h9}yG}AB!($rRBy)TF|3#1dc zI$44Nij5tc;HkV6V|`p!U7ALb_QidO%sR~UWh#HdrCR~3-TeYxy5f5m*UxUkZCuW$ z)px)Chh=0mUy2$tK83OFwDz%>Me%!MdH>dcPpvyJ;A@2arst<}is%}Z5wVwhiJ;Y( z%2+NtY4)}wuoLYm`e>{@(g)b%Yv4$g>6|-0OvaSi+;9HEgObFE)t5ijk4BIuo(TNd z*w!af@!}GYK10ytM0;*L=doAVl#+xIZk^Cb6`danx?GBKtiYn*PvA`csw$*Ax3<5z z7@Oh?3VC^+yJnZ8e7t&Le{klp{qZc`WYcGyi=I7U;r~R(NXR(l-T$#v?w}~}U7`bO zT`UNks24ho&H-=S$y(cQ#pv|_7CrS+FL#Joxj%laFL@&4{c43*pNF_0iYO4 zK#hc|Cr-LoVI0#HeZfm=XLvzDRF7E$V{!1n!4Iz&PnFaZvx2zzdm7rvKW6YwsJC@x zhiaI6d>l-bP6ev~H~c=oOswu^-ezLK35_<-OOrp5K$MvjNbwm}5zDIf&&fz-{F@yf zT8N5Y){@VF39(kV8OTeJ=;FN`?e6>hP?DV(%ZGNsU2m}?r?ci zxsjongN5Gfjd7_Z;?xuh721NLkh{1jsGLh25$Jn_(!$2ZIEPIP9TW%8-r>S8 z3J>r3(!oqziV0}GJg{yyc*G#9?~)oCa=gH^1Jj=BJsZgimvdhLTWy$s3hv~T+AE%^ zfEqj2#53Dqw9xft$TFwPAHII#{j_oYtu1eL@!1$~gfVZdqMqwBpy$;Lz*lZrPB?0a z|C*qh3s&2gN|~!rkfXIe>QKVm@vVa9zGQ-5zWE7A@3OfV_bzy%S@}1O1IK+yM4T$8 zStjpz^}#!ab?{#A-iF4N96+zy>lw?UNuvlKvHx&ksZIUPk^y$2(GG`VBFI>&5S^6s zt)`BJxNqEST|7jN)&az!rQUAOK5ONYxm;`kczn;ki?%)_l+n4>AGKjVT|ZyDc<%e6 zrJjXUyFWd!&*rr3vQ5w$CHq9aH5HaN!HyLK<~u4G8YsPx_nozzcm;Wylb`P%cVW9R zylwB>o+`@sLpzf5N6_9?*sprhItc&*Vmhk608agg`*Lx^x>;A|V=bXnPNEJY=b zob640^@vrKEV8QYoXVgE64yO*IsxaqCAva@PH2mq6hY*X{`Vl*iK;Xpco;C~)!#Z&TeLc7O=NYs7YL39=g? za8e(_^}tKl^#M-NzzkU!&17Mn@fT~@gwA5IP2I) zM;5N1b)WA^!CTBzhQHBxE)Ei|Ue&epalr0jD;~|yErWM_=j4sEeg5`8ob8=d1Aht# z7*%^otRr;`+BH=-L$(zK@^@F*=v(29@sI8eWZM-yT#U6oWa-jUc}mT`HSVa7h9!i9 zWVAE|0%_+jo?P@-{e29FJsXgN6EOR_i)a9H`dzobW!&GsZbPusrBBJpRG|&wezH0V zTAE=1p&a{MQ??F5nTmiCVY%_0QkM8>F@4MuQ^X~tMc=*0Ughps51kOaM-psB9sP0_ z=HtLve9~{5iitM7$7)<;tQEawE(|tujhwOO<^N*Kq{9D^$;11jHG{`iqI2R*vTn+Q zCe$!LUoRrACvggT>9Qc+A=7DBO|L;iR3%Wb;4`?aD2keULxyB&(tC@J#|+YSeOGV+ zj*8de^2zX~?KE9?%+5U?+DS+t=;@n9Ss+qrsIxE+_VS|NfH-3i0rVLNmU@zwR+awr}h-8F&$-EMa!9T6dbLPrrh1V zkLrhUq^ADvw18CYA-qFUz$c^PehjZ;%(!!O&x3_LtfsV-sOOuY(uosOGLt0~# zoy2H5b`Rm~qdqAeQ^Lh2*R<(1PqNF9V!%B#+KE#Zsf=0G(a$9DmsaCKpK9aF5Uk9o zC(N2b>7TgVlOhRG*_4QL`aM`_e-uZx!3~wly^?d z`|A%%3Ug|y_}8L;C6+phbQlS`|4~@;CRRZPn-qC``ig6gQn!WQ5M*66XU>Pd0O%t5 zh3$k)^&C@|%ST&e95#LzC-qO-qH|3_Ee;h~MCX?&a#U0=*c%C=X|O7%D>OET0C|Or zg6?fF85x{Oa|~^Mu7<{B|4<-NFvxK?J+%x;ZOUMitpD7xtKjZc!;u29&I#g&3W^~k{I0oSKHKn2YKV`zjOUp*cgw zH3>__N(Jqwx1w=uG+o}z4;5JuqDdUm>7LCa4Ub!8e?!%QIjgo93dCO&L?2zUAB=FM z57*fYrFq~5Pq67AEkPU1p)xK*s(KjO6M*L~>yv!$MzIdC(L?BJn$ye>I2BK{T?v`N zHPr3>rSV zy2H-2g=IB~J)rUuwCYxeBbuE9>H8R)gX`m%CeYPIkMBvv=l343q8)Tlyv}ar!g%#^b^OeJ* zL=pQ#wJJh%C3i-;@U?LPyO% zyk3gh8ZM}@?J);aURBrH6i z-H#iARL+PwJb1Ak2}M@1`mOZUVP5CtN0C!;ZD3Blambn)#hya5loY(HFA2&Y7P;Ff zpN>1l?ueo8ou3Q^I%}d*MHUZV5nP}Pko$?_b!eG)qPCv`@4Cu$@_DqSn<+& ztIkIwB{Dl#@!c~+%~s9d)K1U%aP0bdnuzorGDzs!-47Pe4W#&jY$O)q05h8w#h)UU zO*Mi8$0>xM8mUTC_k0EsrQ#S?z)tH9Z2_0A&%MEUUxb)Iiv=X(a5Q)z*^$c*uT{`H z9T1IuwJG!uEmH51yPv@yM!ToPxCCf%B`M-HALBT}_1C_!YX>24e2a4+Zs z4TX#I?UekF-h)N@65xlvdf8VSOo7mv?bCzUn_K`_iPrT-ODy%xc&*(n*{VpKqUzaT zlYYSwd-qQZec&u2470D41ezMM>ETwd-yx!|>pRVGHSK%3#$Ju0S)3eV&c*;G)56#! zQpLGtG|a<^2RI{cE6ySqMaAB|PTFLYaI}#ufC$W?7gZ5HwA5YU8lMf*;pEZpLCK*f z)V^I!MFM5}4%(mLp(G3oA6okvb9nF`Fm8l3mk`mE{j>dfrn}c3oH}@Y0cSmNGaI!A zX|P1RdRPlQ{t3E0ebCv=2M0_YYdqf-cglOkbYIjFB|uk*e%7aSt^bw>-OewFBn}Y(^|( zsiZb`LlnkfThn12Y=&TLW}`Befo|b9DMks73;IVWG-Q#hU6^~+3^1B)D?62==E0L? z$xIuN+eo;(IqUqq|0cjM9Nhx#PY^l0m3?8@+85CHoG~O~oj2{q8)g!lNKUF+w;PJN z7l}WetIFNo@bY_KkRPrEZ*&g)zSj)XF*J$Yf{i6E5Xj7Uk6p|7ZYLSH&Wwh>dx3f2 zkR-Y@$>6@mf4_+oppHB8BOwB4<^u_^eD^;(C@D>9iV5kXlsNE9Z0-*E-?*pT8~1F<7sht;IaIzaj5w_2f+ezh zN`l4y;G$yt4S&YGrt)>ao`IJ*%sRNCdSfA|;hat7diEw%L+6H)3q-EvR`O&~4WHN2 zHdPFR_xi4z{bxARS0**N@aT=V_Q(_8oE-?u9`4aQ*41N)I|0VSywM$pztoo!b$s8H zr~H+r(K8^Oq?zrIPY636qKX8KetJb3?{L^RBX89LPMr6g|GMvB!DRnra0jW9z+T_J z4Gc>T8aF#hFK&ty6scIFZA17T6_Ik>OJ<3ve``b?(iiFd>}r7^@1zE!x?%-;_|<8Bv->c$i79a+cH5!E*6Y*VzJbNt`-3ofP?-h*gVk zqR`u5vHzmQdR(&)A5+j9vf?P*S-H$lD3waz+NlP3`DeBxqcB3P?sWSQIVN#4flA3- zGnymETcA}{`C_Auz-2%GOwKX;WUxopxvdX@_cBRD)x9n+a3AanjMZ9k%V{2JTpzfV zhe4TZA#KHYd)!4WTFcPSU>+3L->~C!Xsr#I0I?#Ah@;P`%x*6dXQo%e4kL zBxH<8H-NkQNr;9%&N#-z>*`FX<_a(9ITxYy-WpdM?aHAbNII_*N)MRde(T>z9dFvw zueA|}o?W6|g}dCItcN}4HvO+6u{%!E)Etg-$@Exw9F5adA3ci3580a|!hXxF! z;s?6i>ptqw=Ft##mF-Oaf8_J=)?BeUMD#sRO$Tp6_pf#*+wXH3t^PzO#SCbHs{ zW55s(ded>KO*yBK`&6+NvT^*m3YvIFU!5SB^YPyVacIgyZzd)IGVRB6-m%_vW1#se zZi4syA1GPx4F!)lRb8uT9yEWjj>G)DnvftVH`bTW4yF(AH4{y|R^6BP&W9 zsw}<-tw9ym$~don?AjCs$x-P>q%jX!bLQ#MA0gV(b0*#7P3Q>rM0gb26s=uCiQhr| ztkeoD-o8Az24;FQqIHD^bYt;^fL80eBBJp`UM5BVl;~}74|RE7esP^m@5^v9FVc_? z2j#;mvq5j+#?xGGHPns*CFJCd$gq9WdL|0tfgrhL+10rjt~}hd4S5E5Z0m0xbwHqfW``!Tv#rH|JZAhgB@;+<%Doo(qR%_=m!f zz??OQD^V%WE5L_J^lEp3UIzw#Q99|=D>m1Z@t>S!qOE^Wp}&ondk2|}7Qz?#YWl|#4e`?OS6XQ>rl z0iM6K9wcvz_viBMi6rQp$hR(^_AXZXhb8Q+cBJs{yb@*T@4V7FF=h6!cM!4j!YyBL z-OZcJ%ZHTpG+j~X0~u{gIsfob(;pT2!47-tm{Fbp({WIF)ljxKYB)&b3y36?r&3n4 z1ROW#IeK@qFFeieH6wf(BIWA!ruWlypH`I(*uI02= zQsPLV{vZ5Nj4lh^AtRjh=ox-+G0du-7UklU(#w|$7JO7W`4E063n}MG1t9D+;=}OG zKdve`{V!MBFAx^pC+Gyo0GnyVV?Aow?d5OMu54xOMkYWXo=@jlxaTCJEO6p>>b<%q zcv*vT-ZQI@e~ZSv5)w5Npr)~hUztq9R&X9&angJ?(3fDqXFOi}I;&pSILjJoi@Xoa z+OhHaq#pOe4O~iG62Vs#cwK#$SJKAr4ZQ-zxRv?E&n*JhH*@uOlikDqciO>PrT)x< z+NU5vfxBm}t@lYa6LRr0!jMpu01D~$xb0If{*dKDp~(*D=WSzrtJHr zxHO%cNpN*p@AWjJ=+)-k>}-OpZ3fs;O3N}GNGSnC8?0}@a^c|*{>pzhuFKnd1Q!{e zQbDV@RnBN1*&b24bF9HPs3GO>*5>iwq>n82e^OAxXM^L<<1=xC)YSp7foNFnks&G0 zCfxqKT??P!5y<(tqMRQfm!}=}SOl!E29KV~_rL)uhA)ES9f=z(fM$gsi-&P}pDy9a z7D0RxNuvgHLiCZ3ZUgc!Mg!eHMKiHoIuLrhe`Ym|L;WP>r%9+PRlnr#2+l4z-%OJt z#p4`wb4GB8o{7)c{ukQ#&EqXfKq>z}ab5raKj$RWl@i*~;?f35v@f1`>Eh^T zv8;Z3T0{&X-QDcF{W`6K?vg1qJ9#G!Pi8`!R%Sym%DRQc5AY?KNy4rsw{8S)IyNB% z%}Cfn|F}Wf2-;Z(bHoOT%kUN5|J4Nt{a>_!-g$2<!AB5L(=V%;z6g=W{NYEOKUrawJ1*8d$bvWwp>za=U>4rf1y0H!n207m%KRabnhLee&$bCzn_tF=$W!JhkpO+k6Ovtr8 znTjjH5-UoboX)M(Vzwq17mneF(-bw4h@+fv^6@2;9ONRF+=l3=SCpm!&cM2$bBSn&4 zDUzXRXj|md$kg-JSHy^jq+G5XUk6foOnEwU(+Sr}*@VQ}Kbyu#0`jk>xmO%X&Qa68 zUJ7Zlk`d-qY<%}PvPaS#+bG!#Zak(eR=8;O;be9YHHtMV= zmgn`?z#uWg+Ua0^dU^_{6)9(~saS)A>QhXCYW-@B3mIRuPlS&-euGaz@ZGA<=vq>- z>}Qs<;q{phLWLrfBfws20{@OuhbaBXd?jK*;O}z&4HoObmD=|nml0w@q>cifqT|?E z8W?Pq+2EgDrk6}=gbH1QZ7f7yJZaRoPgLAKpWq=r-&=kzH|dz;J1OyJHF2fJ{gUf* z6Y~=e|MP*E&}-RpM25Z5$$(A?h2LGY)~B?E{U}*e#qu}2A#y8Di+2Yil|QnX&!R77 zTRLL7d*-*&F3D}PSW1o%mYWkb>lte;AGDWKvkARu-cU z`glIdsx9GBr}JIfvt~EK#I_OW9}IK>A#CV$c&b7!EL;m3mgD99R^ysz@jIaiz(0f= z2r{>^!@H#}!L@@A-RR-n!Lmk0g75W0zjG4Gb561~4U!yZl?j?CU;t_&N1zg!DJaks zNOt;4(tl2pP++ZpO`G9xu3&mwpEX~7(*|t4lxSzlQc(>%eyk2%JKf2v%tH~gMPKqC z{btOA1#7mde)M4r#m%!aLDaEhdAzY)pALEVl)dP4L9?DE;I4Q3mG1$#SAdAHS)r3? zR!$2_L3x;da8dkOi#F|2A-rODa|N;2p8q*DC`r+;iR`#VzfvJeq~PYIR61E=Tn-{K zJD$sx8z`|@<^MEG6+eWf%C1#c8a6+Y{dN#Pdv+rf&Ae zWqf55C1pGbuCWpic6vhF$iwwS_~6vIPE%Pg40!R+cV)@?P)8atDoutR^^@p|vTFG=|vBiBG%*-19XvYq^MzsO|h9#%{ZH0j9 z3SvRAKmbSv^g;N?xO5C@RF&-t&WRboD6%f=MLs7#g`+SoEV(gw*x@5k@DF|uD)bJo z!CxAzCEHy;NZ(s*6r4(5!wHTyoaQZYxe>@H&pVAuh_xHe;B1hkK)pm2oP*U#;4ywGhxhV?cg++oju}x4u+v4g&iyZ%vh=d<{${SS98F!TkUN(3O(IeBgx6T!% z8Nf1iRCrRr?aOJn+Ei->GSPkzLxu=E$MnD@?R*ARV9!3Q`>IU7Xk)xOlTJ;*t~@<%*`VdJq^34ivzepvK@v{vP0vxIA0*^f8Y`)$*!O zw7^BK&vYM{YZBzDHiQ33jnev;mO~>o0e{mrvzvcCR%dnUUBm_Ah2I6u~?7-oyjZ zFer?jVk?-3@gd=n2EQf=|6m02j2|4VM?v4YA zzGdd%ktx{~w@I`*Ys&E%H6}|t>x_^tl^pEO5SO~^#1>-y`` z+CsMKM~rimrAGSaR|Z^_uUxI>wiyu9$nv&OfDq%GaKE z1*V7o0lUZf-3WiZ<0sWrHr$n=TS6Kv?a9y4JtNK*Xt|*m9bHq!q)XcWF;UqEEA}th4~;|6grq9TeBL?Q24i;O-XO0t5|>ySuwP0fI}A z;K2g~cXxMpZ5)C-G!Wdq;r8A~?s@l~ed^V#di~E_v)8KCRdcSf<{aPn^%zZdcIiWY zBkq17-0sLkDCbm0r5&wS{IqhMGK$aK9j8 zGNosqQ$QZNZ{$%Sk=eerWOt-gxeThIF-2Ole0#{wle8VSAedlirXF33D?6XBzO~Lq zDfZk~cU|lBnb19LB3%lWA&NC7Qe{>z1Z^np(_uK+&SDe$0i~Gw^Cz26 zCJo*0_ntH;wY|B6sk95Qh04eny4|zrsh+Eu^=lRYtKf@nuz_l!A#Zmx9Ka{10(5%l zkF4eK1lFseM}3JUbNqdTA%~D6MA9|%$IWUFe zj-c+n;&TIYnVu0dc~jC|OJ(TP$C~?u4VkRuZ#I0)^#(8h9KTS)NckD+-i#HS-gg$k z+bam|FZ3S$9ZbN2Xn8vV;&(F&pAHOFC;<60CFI*bPI~b7^E>?gDoCRQgC6(^(o`3Z zo=-aV+teOO2BoeYM@h!ByGCe)rZ_8?-vbl-tY z#yMm#vaL)Z3tg65u5t^7S_ZCoMgm1b9{6x=aml4X*sd;DzekF?WT}#W+$M=3d9amx z2rN%&a^Z&V%gpy(s~so*QnWe(b=OVhF$S0^V8(k=%q*LM>ho||&0p7=9PuW7Geaf+ zNdwxyb?cqRGiz36L%0G19^_YY3ipCIpN$G8m4V9Me!D$sfY_O89(`O$>Y@#z=6)~d zhg0p_XFv?|FQg{H4T)8h$ErT+3r|U^tI3Mct|?B{s{U*X$wB*v48MRi$1s-AYWt@s0&n5_XwHfPQrV|0lbOu+MuM&#m* z*^z1Jo-|2fEeiB^h%o9P-!X&5xO2mhnnRCWqE;Cp`M#F4nhXkOli#17U2fj+I3uOH zf25|-(KE;fJiQgw1&34lOK%K(ep2YIT%dS~g4M>86tihfUp7G(%rp=_l9XJ!SaWC= zR8s96@v}0wmnub;@=8`OQz%J0BH+IxRz#LjtpiFGRxL`u-SLmqfh-sO+f3xo=|mL! z5#)~+YNf{ELzg|}e3{{nAhdh4Jz|UFvaxus#N#&IZ=Lgln0q8hrQu>UdydjVHXmQL zCJVQMGvo{*73(BOKsC*pLg%P-&#E8K7$J`m?wL{5lX`j{xETX98XMoJ<@OLF;fl}1;;BDlppou;o8HRSBVv8{4!}>HzRmi{kbz!~?A%OZT{)b5h$5NRv`G>XWF%K? z!#*shE0i%&?!=3zG}zvzG_)UVy{bxkdnL)4Zx#~y*vDG#?m7qa5eLEahzAHipG#Jy`@s;&J;R0l4@gJ+uYy1$YS1!EnY zNPbHa6gCbUcEUvSc}6(Dq@1D+5Cg1r@hAF-hSK`67VUrVUAbb>RLI&pX8~&04|2*L zyVQ{1uiy3#-LTPq-mg7PP7De3GRk3Iuc1@=^K}mmIWYLN|7I!{e_Qff5p9#g>v)y( z?WV)j4~ZK#Psy+>oB#ubp%HS(UZTox`^A%$$$*D zRr`Jp(+N38Fm(u+ze?`hyRHoB5%6Z}lj;ab5Mc>F?!m*l(O)~S(^bpHPRN*)QsKTI z@^JM2=YC48<-{K(#w-VTn;y*&8Q3Y*1FZA4xIPxQy6 z=v8Ro(%BVNS}3;Kgr!8VXVOx+Grq`j6;|>E7zNWTxJC*44%&+a`+dSgkO9Bk zdBfTS=-^q7s3=ng6lFtXkW4po_Yv>YZ%?iW}XJ zUfPColPo4v-Gc9C1;DmN`yjQ0z?->J?#Y9R(T2SZ%B-_FVW`Q%3+Q=*5EB5;u6O{Ekl8o;u11dMRO~O{Fv{D@ROBrcf2Ge_R z=jafcOmGHetC(+&L^tipRGgjT_h@*_lg{%a`V+?Omp;w6j8g5?TCp`+2wwgwsPlF_ zhI(o`0WChDXBs#dC;?tqw?eccwW3ZAaqpLqEnm0{Gu@Vc4v0vvA{}-N6RF?msIg0R z@67WLWS}O*0vg&BjXoQio;vVPW^mAT*3cRKX?53-0|VauH^>LqX+)p=RLM)IrLK1r zfv()<$ejuI56aZAH696eF^cUMmx)#z`KUJioAT$k7%Mapn15o75aBC}IrZiR;VMVf zL!zK9uPx@u(xdt|y*Bi4-`T>sX?XPpG;1AoV8z?$pnavHBTn#liNS({&-+2L^`9!I z*PA5_cb!ne8x9Vag)tP*r?fH12NlbK{(IPTqi2H zNPpm)bCkAw;1NSouRgcqQb>%f6n^6H-lny_L?GRp+DKF!hJ^f8W>Kt26f?|i%;65E zISf#JC<@Tc8EmW|Z*a=yln(z@@-D}6rD};xEik8z1mNv~$TXVo&`w{R?ic8=rH82GtCwH6;lixEz=PVz>)Tr`$OwE_|EC^>@0XC#O!H615puGf}1@EI%L; z#$GK^I(44+=Vp6?)8SE=BjDMqPb`toSTm{9$fgH;$$)VAWIDP1lQL!o)}r@cuVIMWYc=YfY>>AI zW-Us5;cKkFqCS!O9B`hq4}w`Ur58T2SuXPEv3DM;Q{<)V5t>oEfzM|m&+ zxm37+fm~FX+t|z@9Ou@HzLt^NKS(Vs+*~Gy;(}>R>`>~R%l4)T?aPt?)XEj3_G1G( z*lzV3^ymX}y+Jgw)XC`gheo-}^zINO)=6;wd56d2@oJ_<^_lp@L9fVnDfXURWgIoy zKx0;K<4ldcq+5Da-C-rWo~+3UtEBphl%q24{78Z+HOXS^TEE~aH(*3 z+GcCQMtrdleeLmfZ|0zUt+&)z*mr;4J?c=xbQeDpe|f&5 zaH?m=W45*WY%X1<(z2mJ^?$`kH1;g1*(QJdsWuC{s1xmQ>Ms^RAsU78gt-X$l#K|p z1o2-G0gtZ)kn-G{n8^|iZ&a)8ae{ZM^LCrDwqSS9`#!G-q((U})Nq*d4|%K25phLb z$OizX&a57N&(t)D6SHzc*Gw8oih-y6n(xBp#O-waOPifHzd$X2@B6&O5jS4fXk>Xy z{Z%1%T*I%|TJ$#3iql!(Brd(#jW=jfO_ZVDP6jp*J0K5}eXj+wTuKrh z%L{pHw~l8{xF)d2+%T9J?~ZqN`Ajp^lzn?nCu?*1rQH*6MqML?)V(4qde?(R;P!;J zDiW`hU@s@$_iVsaI>mhA!wh(2ca;(3NzMY9Db(#fl9@DkvL@nAE%c{N?(6$K=!QbJ zMK}kIUkF_TS-$i!26$8;aPmYMskL#m@`URC~QHpG+O3OJH8#Qfm(~%_>lQVUu_LzyQvPG;t&MDmP zL#FAR8)4dAR6)Qj+h7cz&~TYtT|_z_!0^EIB3&f*3Q7uJvqV0sRq-Q(Y66X+oSDvo znw~`_t&GM2WEvG?civt4e)GCkqVazCAzNdiHW2t$(%?HRW}*J$%=%o~T2b%7Xd1*s z;2eU90l$JQ;*;3w3o#T=n=fN-KKlX)<70#sD44kt$G+DXKVGYsGty##JTp%MeT|Ol z;uBqhPEvY?Lp8J}p95-+B$em2wx!KJW`5Q9vMmKw15s>8NV1*2$k9ZR4MCSV{hS)O z>OLd9Rp=_hqf8L8(@LE;O=4@8utt_9zs+P>@J;VY?TDFi)u4zc+n%iT9aR`hLN&ke zwapJw;%j2sJ%9BMOTt-HSMr^R(Qi2{-!(=w$>{*D*Ra#wfU!dymM1hV;2=ptz9Tpf z(bJJhw$L}q%1%{ga4YKeMI z-NaaWKTibM8)Ouy`1GI`u7Ms=w2SM087l~x$NxYteu9i0qXyVl)PLGSN?w8Fr_a{X z5)05UyZ_Y@bA;$xwzd=Px+xIO{%Br(oylr;2JA5y)olopypXo8Mt@Bl) zs-M^Ve{;U{mV#vLyS(qqgp|tvJ;T^sd*A7a^$rF$C~=di?!iqLI^&pFi>o@k*a$`( z7tE7UqsJFRUXlB-zhc)Mp-`Tk42d4#sU;lLPLQtT5TZt=^!c)W!D<)nZLl&?`2%6@;_HV_cyw%Sm%?8)A6rL4 z4A{B%60<4}p@pZ#cD1#|HYeogP9qdbG1I zv=o5Q4Lokt(QGPY4IB4Z}#F|L2A*rW$vbyvk7nwLh1MIvuwv^3~PvFN0`@>~wRAHbOi*Ge0*}QZeKd?e+;i-aN9t?rJlOJvcH>r!koV zyTHw|{|!DYyTb0aeYSj}2cABEHQwQm4p#Gh=Qi!Xsvz^~xHKD@R5)DVg#0{|rW@wl z9F%VUwCp=KX8*J7Ut!4pDRA}C^?_beDIXnNcn8PHNn5@J&*&4a6TSUJiH7)0B{Wh_ zw4ui{UZR9nDY>y7xgzj6>3Szng++q^liFr3Csmuk#zzSQyx>vzPn=>vn}DQ1MvWag zA&w3R-o;L-S3{{y7qO|8(+S@zP;JX+18Wm)jrH1xWLFLa0W*&?bniUP&Glj?HO{uD z_|f$8DxHsyZ6q2Q;~UsiDe`u;3GnB^=>_(uzu%VOoy5t^y2LS|uG=^Bb%yPCJF(*y;Fq(1n`RzfV#=CfG#!BC;UY!4#NELeK_se?f2?qMpGS0_vt%{=!y8_%U|S zctQ0l;o->6`Ymnx6X2va9yVUbBvTuF2f#h6JMnyfsYFmkgfvo48{yFMg?cbc7XdQz z0LSxWCXEb6T#tQC-{avKw~qVVn|F=P`JngM16#@@2R75OUnW@fg2wt1ys)Ezw%-88lLELu`DWEFvDEC-=TR~bzl~v3NMCOcknZr? zY#hMflYfE2NomBTTHj8dHAa0+9G{04VzcYVvn9EIny{-JdicV5%qBVE0~{(~gSlS0 zjH)8D$D>i2t>(&Oxi=rvdvx|S9kML5{0m?v-g*6I%9b?*WueNEX)ylEzAgDvIjO{? zo?cd%rZ;!p>CQf*dq{^Ccri3XwX~qi3ei>K-x4zm%cOEyhm{QC*8PziI!#ov6rp%; zg@I&bt)YTu=dlNeJwY|qyR<$%G+o&yWJ6id*P4SvYG#2;kVQG~Yb)TRpPHzQ#SATr zU{M}98&zcWSF~wsGITGN0C=>SO#Y^k8P#TLSSQz4sL|w`t)lNj_Kz$%@*{_lJX^z> z6>~<*vA$s7_QRDo7ioL+!mqkTMV-M5+6sK0Qsu|Sc|LP&0VGb>iX^r?XUtFuzkxIV zC5|XBlzS5|N$&b|heWd8-@D{oIu)CdbFs0BH?ud&6!ArVY~=n_{t9VP0;QK64wnAj zUX(yT{;)D2lYRPq8p+j3OY9Hrv;WN-3-B6YkNz!h{MmPFj4=B;(NDgL0-uJP%r_p> zeyRPtEnPDs8w$XjF61Bx-TsV#C@Yp2?{ToB_$`UdUjTz@^qZMNjQPgY)`;QQZ$QMx z?K3c8wg~VHbi%E099Qu>G!Z@TI0=q9GFMofFkfS3LV>^O7SV4vjint}bc+oQro|oo z_h`<#yx&dyWdu#fg=dEGiulJ)rAgB>cBw$#L;FO8qRL_wu#Q+*SxgbRd*kigFYApm z(u*O#UI16pcSxMswW;+dS6~e9yoe7+cv5RL#2v~(;p=`;t-Ig4vY&gIPU;>S(wZAg zh&!ykuC23lxecFMwRn3ah-Pt=j-dBWPE5a*g43qJiG@{E?60WWg&b{(i7uR$d!YWt z)lOU+U-&|(N7Ni)4_A#~C32G~w zI;1v)5@Mb9Z}Ok8KAu?(lo#k~#hw$)x`pJ#k4`=8lYDsP3m28`H2Fp1W8+3Ry7)?` zybTE3S8!x1p*`{967pH11jjkLG70=vjiI$XX~xZBru}QGr@UtgHFN&xiD?@*v%tmT z%Z`vVTogOAUFU_*-=tQU1r~sN95cH{WH}sP8$ZPLV?PD*!$?@gj;0vpvS7V(-?mI* z@n4+({JoY>I@pZmklqv-XZhy-<>UAD4dAesd`joGIhhPdaJ!F(#~$pRqLSxl`Wk?F ze87)It4WgaI-AZG_emMijC92FfflG5OZ5Zv8P1^um>qo(CY4pDZAZ-M_#gu+fcLik}4s@Z@~^G3qT+MM`tOY`^d1;;S>vp%T}S zt&$f9W#c8rp!)HO(_~8yKgHE7dUg9x=0L_g-+Dnf>L=HtEI3hqny#a~RFQg!7Li z6}CSqw{B6|PX%Q*Nbx26xGr$4NCXW8*<#7$!GP@mPt z@#GjOsaJ_}2Ey>!uc@x~C=;ZpwmrcN-g54}%wMfa2W}G~QDbYah?C-GYS9gRQ|M}O zy!wO_rxPG1_)@PIsyg^hE9@kOf&N_4S5u_^4gqm$rbASB)B#MkWLFEd8TQFw9rGk@ zPFTKBIBCXL-vF}nqB2?8efp=Y%d;H)_n)#Z^R||kk42igVPg@ycaeE!=^D&C%^c{w zlkA^2ZaD+nbd6M!Yt``rRzON~QG_PS-&EiKUEapC0@%VliL(8Jm@eFRW+5ChsMA$J zyMp27R70JgY!;8yyPc5n?p|cQJ2G7y;yUHe1O0u9pQd8{6a>k*ibtC)?nCsSL9Pk1 zTg>UrpZkWWsgPx{pN*$VM{i1Oas9hAKh&rSJyZkU z%S+}qem}SFUleosyL0%xvoo&-p+Wgx;6Oht1+vY#!1Zh5Rb*Z=pGTCs&6jnDFFwo5 zrKRjv`x+L%I>!nhW*sl1#3AVb;{CERs?MV(sjeRMeY#h&eWfZf7$Ec!uz&w#>n zoexywzGm&Ut>xoN^P3j=A~UnXa#8pL7tbyn9TrK{d^hucZ(SNs1*5)Txs~rYpsOoO zJh_Z>um6)1SFmvIYnXEGe+V175DVdZPrgEbzKw?j!WR6ynTpb|&SgPWiFFg96%knx z>Y2s5D$bMD(jvk(GX5OFQ=xz1b>((_>&9l=%e)HhoIIsUU}{2fLB%Fui@Munn56p0 zI)MKTl4KWlbN{WYF30G_$7mO|pWu608WwzEAR?e31T#f$rs45I++7qpI(^GNQNf07 zWQ0xSw@SN9e7EYq739e;@0zJlnaas@*)uRs(Xu}_SU*Xtb2P0XOhL5{iSf@Es}$jy zU#U>zWz6!~Lr`^!#dzeqE0kS@!mKv;Tq#c))!@dpL;CwKyfGfs`o>*8?-N3h3-&j+ zt}8bO4SD%89YTLqDPAQ?z%0J@-v(-4++O!w)xvL$z=9jf6T72SlbiVQqlUN&8;^k1 z+C^c0+x?Rnk8B4!2;3eP3fUZ(6sK61-9e|aL(|6>jxXHj+c$WCk=;9O0HtKBBfFA# z-RVgtD9UU;+KowlwZkdiN;@S|MI~3{T+n_pR(G%dM8|OYSa;dSveM&su~rpah)`+~ zuRw{q)vucQtC*7;o9RMRb&gX~7#ktHLx0WY)ID&UDt*kuJN+PbqDQp7w!Q*_ha!lu zLZC}gN?Jfrbfm33Ut!pi>%71RZG}A!-r`qsjWj${>o-2~?GN7TH%VRAKvfBG`Un~h z`;xwPM|ik#0-&V3JF9u1!#tv>Liel2$5LxUw&_dqR{;ib&IVdPl1tGDj0%g1BK
A0l7W=JRG7dLU(Ik_uLif9q)iuC%Hsa3$7{pH@Ek;iy$UnrUfN z5RmW)QcM&RR!A!Lluqm4-twk8E{btzMmNW`E~mn73|8;=zI8EARaG94_1Z0U?l*4y zOgOTqty|6h6eXy5f5}||13J{8xm_e>UYjMCgw)w+WL+Fz?CXE#dKyX^*=xdV;><&&9no!tp6A=Lk%mGo&c-EYGNfORl{*cno-wAZ@foVVk-)4C z@b;BCJ*&Os5mx3;{C@)Z-LtwZ;Z-M)9jfAo%y?5$==a8Y1zenm0zJiY>Z#tWvDW&k zjwzm*blN#5GG>_+Lm0bYsGEZvx%*4Z9Y5L_`Sx#-)XjHUq5Or>+31Pvl&$fIq!9> z$(VB}!+Z^%aXoY+-m?lrxZQf(xT<;n?RVeqLKugebrVr-+xQbh&@SIlhTDJ_*Xf!} zaGTzLxtgP5h}qmDY6d?}Ql?}Tjf*P-w( zNz?2o+Ur4WF7Fz`&42VXKl(7+)ToY4I;$yUGxw`BXWsz6OKO5KT&v z0R5Fd@%J(2e5it1G_u9@?n}IGa0Q)Nz31{P52IsHb}GcI#;4q{IOi8OUbu4(#gbE8 zejNBpMcQ^M9ln&@vN4kKYa>KU#`+Sd`l>u%5bt2+(iPTzdQlt=&k$OQy8`GIK|d@1{7bL(nasGxMCqO-Qdq~Q*q?h zI;J}E2L0#DAl7MF>b+=Q1z@>$8#!Ixv_k*DU>pG}mt0n7xGTskmv6GpXA14JkDE0cQ-M63O=0KrOYq+T%Er9V35xPW;@`Indkylh# zwM|Rh3Ize|**A)mP&g9pV+YA!<{8P=H(Nl%iIUM-#}iWDL$dG_+MRnwx;E0Z0N;#U zCm3;*i}97Of&U|v@vc)y2|L!b1rIyWu`ScZMk~w-2dG>cWDpGGXo{-J#&!=JP>bXFkub)G0xIbE;J~P2T4lQ+tD zDh|gqU*Ehd43f)^-Y~5>(|%7Em%m+W^X99KUtfSM9-Uo2=Kux z#!&A84UyOQX0k39iIV>&lT!Yr5(Cik7EgOhUH)*#sPeV-+s1_D)p#0n;=9SK#haDL z4riBBF3@}xP!Y1S1ut^Z|6OJYX~dyw79{=(m_$bxPn-vh*7|Q0Ys~f A1poj5 literal 0 HcmV?d00001 diff --git a/_assets/blog/2020-invalidations/SIMD_invalidations.png b/_assets/blog/2020-invalidations/SIMD_invalidations.png new file mode 100644 index 0000000000000000000000000000000000000000..b85efbc649e26e6bc9bf3065cd751c9fe754a9f0 GIT binary patch literal 159095 zcmeFY^T7myHKgS!WJcM0wUcS*1i+}&M@3hr7^xE2lt&A$7b zot*Cb-tIr3$1k;NerCM+};mqyqZB1F6O`J?k?VK&_T}}`> zpIdkfCoT3-)g$Y0#oYzZbZPMHUHd6a;>3s(C6m-BER@q9nv zC2Jt5{z7^qR(PEOCk==4jymc0Ck*O>Zw;*{6yFi0dYBQVef)Bronp;S#&q4&;UOcg zaT5<4p|L80ij@Du66WfTFUm^XQUpoE!vD4*vf{qtzYD^*yk&eM|1Od0`2SEP)pc{q z`1@pEVB2fmZxXi5gbJj;%d@fcDA6Wh=NdnGw^wTH4(58pu9hAwVX;id<5|>kP^&wh zPBJXx*{Y1DMH;$c6=Dp3y~ued(?MTx6dFyYmLs%Y z`A(@+#q|5tu1jpW!nQ?T44yY821XF2yzc2a5B)_NldyX6=6EkUU_`AbI#=Y)uQ!92 zc41)sltD@VUQqlJO6*cEf;&-Nm3i`d$2wi#~gn=6(H*!i6pO8VUZ0 znG)xlso0p)qyH1fmE``lw{UUU#foY9biAT^O#b6S|CqmbF++E-03*mS{2R#TtyKSvZgjQH&1v- zzH2#Ql{)%Gw8nTy2lgae0?K*f_H^-s)$j2qg&;P2&y1!-8hA9V8(Vyx9AdxsJ|{X5o28+< z#0{G`4AYsE5_C(wyG7$b!O4>21oYwJrCb2X+jjQDB7mF&BcWZDwwQ%XMPwKeC(jEVD4(4H-^k zlCf8(?Q`N8I2oIj%7X-5>Fy@&d_QKis{~9YqZhAATjij+O8WE*3r=>9 z(ie@RuS*$Apba+DMEm_jvOPH&OUkP&!!q$uKARYrmeU;{6uLV*M?2vVX`%Hslivy% zNu6@j2b^{@claFW$NCFKYUJ*9mwv)ebn(X}|8;EJ;eVz1gHpl7_7vUL%41mqusTh_ z8^so?_Ri{Dr)}~~&rjfU5Wx=6*aFucYKyMO$m0|Iivx{trb7@xAkM_m#`h&#TEBG+ zfGgvMw@?kup3+>w9C~|*zYYYMvUsvxGE#uU+*Sx%D-dnH@l1N7d~YrEI9lWmC2cjF z&(bnfQ~*DPjPX?@RCj*YDWO|UBu=bl7ah-AMjvR6lpGcOy)w5X=I}0QSQU(6o6%1^ z`yS_qx=h0E>jh?wz4gd6)paa{ad0bz^LB&R0g$ z7`Zwxu))H@5mjDza^{SewQX+V{)P6~|FFI+&`^m+dsv1s-#h;uFzn(G7$jcBG zWZqq5dpvr)PuaHD-?~9TI0;)W5p01Iuk{PsW09d<@1&@j-t9_SDNTQbvFG^L;H);Y z2VQYW^5qzQu5uXQX<)5C29sYWB91KzBC+AOLjcPVIyuQ9S zH{kX91mLz-Sz*ceh7nfLal#FI{gEUVtli3!tOu+gR^AtWF)IO9Z^Ts7g>NNm?fs~5 zYgG&~1@UDV!C@l1F>5j}Ts+-*I7P@Fc`YAokD92wcqQB!_6_R&il3rS(zd-ig;9J0 z;7jwjd))F>D1Le5)&*ycf3nrKDj&r{w%)Y7+X6^Z8q0nOM*BPo9IviS2#<7FpB)Q& zNmj;7I#8H|X1dg((p*kLBV$U+d(@PTH#L*vX5;4I*nBphUU&@4#>SZCXcZ^XcJ&DL z6X_fpUFtCN*kMEAPu=^1iR;@LiqML|oSJBW&%{~~pJVNfu_Vp+y6eQ`#f`O>AZ^(b zn@Jeb`Eoef0=0T&R2WIoPDaCc-S9qF*a~LoSEMRQ3hTP9!yugl9S&!9Z`)v zsxe4>#U<0K#&P{H|m6?&0*TjP$Z1ButcTvOGWBd5e3E< z*{1AGwNca7{!T5F3fS~jaOtR>A{bCk>`C!Tfp&dryDO=Jd!(s^yv_i@Ia~odd*~yf$(i7uJ ze&5$Js_x%cE0Ib(Z~*O73pc8ibZm4*o0I(Y1Tr&D-;Y5^B@V;@U)Fm5G=(vwQwF5~ zN&jl6y8wjzK<<3T)5$nN)(>AZ)cJA-9@muT@4Xp0%!~!yKg6*TLA8GF%())eLpnD9kC= z@}5oQoLzai+F6ww9L<6K7WLCw%yrDCg;pFf&5}A5y7cap9HG)>?WGnH0@p}HlY2?# z$^8;MGI_T%T=OM+U3w%>Hh&EF@+mNH-S4&E-T~Q&GC9*}u&+~jcVIT!cOHdD+fJTa^IGgj1Y;n)*D21PkJ5rJZ zE8NDB7p2_cHw3E%M>YJbL!IT8yJqGV& z?IgaTomJE)>Sa1V$l#4U-R+}D?6cWfbDOKSowxH_I`ShtbPYRcytqlWJ2of1L zQ_^(u>Og3P@ER(mt^S#KTjC}8HlxY_C>M1m66{4l<(Q>2FdtfOVbDo36*%Q=aSIyo zfnFxM51iIKG(*ahPM(M;uANLBLF7JlyLfYJRsbsikOVEI1~Nt-h{*ze681=#TzGg9 zy8H{!kh3y+ezR+T!|x>6?dU!4V}T*^c!wXC+TPZ{lH}`3zLZe$I4Nc^2`>tUOq8B) ziv;+1n(6Z`HFk*B%dXW-6Ampm3CSI);32R-WI(Tq)XD(l0}W6?t#&X0ZQELkRmWT* zkFBKEd@(Xb8H3zhICwww?Q)q4{<)u9W7!8XWP~u+al4xA+V*m-b+_}6k zSoE4Q9a}3{PX756b3WdtrsSzw3SNx>E!B^hu8zMt*=DFr*zJhj;%M>(t< zu}ayPrxWt@jX|T?=K+e;ps4u(D^xmQY^V0g2sAsm^&=Hx3>If?u(clrEqre0{& zo?FnrBcMbPChAK1np|6ci=s<^$lqGsDaQLQcQ?y$7f)Z1(-q#!bX*=W?VVX?lMIBmC2ueGH*PzBw|KbCvU>j7H@Wb2j24*6mC-H`Dk&=B}VmVH4D50``}=-2+GW^R1S` zxxP43!{lV1T`OD7W3f;mWK@o%H!jq~jNN@s{RQa6)LZUVl)7A=dE$?`Wfgf~KTxw2 zI+^lH-G5=%Dx}Dpi8d$75R%8K5$jxgHeBv4I&g!x6Tgvo^UmjaWyT{)BA*W~TB6$= z_qB4ch?gu?;l!#LKsTPs-%@+fEv2#<4h${WC^juJ!6};r-g7X5}yx?dnuf(3Mbkff^WSDKepf;vzzJL7Ovh*ClUqL8sU}B~r`sl%@Ea|8HwgR9CIFC8^ zwZB)UI;?y_J*0@{1;J6@QH{}cmGaLNCN&Ao>YPZEDH&u58s1;74#l78+^upO9Unk= zSFi+8mn4MZNUKRV&hixP}~M_-7iU^!gYqcdQ=?> zmEbt0V2^&RrFA;quzT?6lGtoXMx?1PFd8M+-`T9GdL`JoWnA6o13 zAcapMD4U=ZPOKNk>2v-ifl}veVYW^;dLeg^7P#JPZgK;K%@O(kPCf?;Ok8gq#Jbl0 zYdWf&t;@q4Yj1wI3yiM$M-Bfgw)YJA2sKmsdE}o~^{#uGFEhbluC1OquHl!=!=xfg z^yns|yZ^To+o+nIs9Uk3JLS@+R&=rp!WkKYzA0kSg&zFyhUEv{qJf1zpY-gh(74as zX0-MlH7RHn&j+|QCUkxFs2-5pdTju8qI0~CoiIVOt0Ej5`HdmIL5D?^*)n~{svCYJ zkNN4vF+g=ClcCfYxH5WFEhe5daCgX6S5h77iQTB_ubuZQq@4}t0cs_H>6L|wHfgDC zfziqQb3BLz6>R#!24BQXkUiIvX>A>Bo~d${C{@vpPS(c?ON@yyn5+{=F*^~-8HLjP z$2m}@>LuRD>1Ora^%FhWPM>WpG^_g@cS!^7!mP1mj4gH80qDy@7(U*2zl{Tuabk3< z%H(?e-lk);AF0~xNEG5%Bpq!_FOZW{2N%>l?R=q3U6kU!oCc&jD=GE8iKGI?ODu07 zkVcQOeogA>5Ql13pAR5rDV1R$yiBD%ouRuuS#~DEI&2i*yz7z}aJ5pXUZ~|)+k)6C zEb5)IL6CQ5t9|v@jm&MkSlu%AWN2rR*EZw0VGOzA40*8ME?#qFgP2^NVmg50tNtLG zuW1FD?}b`|>X}|h!w2@ZdA_wljLE!%PJONAGv=*vlE(q(_$Bz=P`zbz5vUOxxb*m`;a*F}_Tq z+=mv3-mlN?0;D(Z0;dl|5SZUsImSI{i8kgI7`7^=J**Mi5T4_J_3k-sm6dc?;KmFL z_+z+eAqO@nP-iQVjwpHSMzbtyJ;XAD4v;%ah$0&IA^0^&v z_-+=Qqto!>;~m`D#JqK`tt-1TWoSJvKG5Fy_8er+{`MR8f%uA~(j3IaANDCRi=%?= zrNNsi;$`0Eh}kcTHX3axmv9-4ZtlV6&p{x{FzJH(i+oEW8O@E*HIb&Kuv4LzakIao z-_ZPZ06oSlb{l*s_G~=e+SoWwqPI5uvIza%KRPu3V&eF%Fn!4HJUr1sZTi`sZ0$;~ z&v)cUN({mqB>>2b0=jP1~JyP0C}^;d}ba3!tKC67%$|!`cQoXS}Khze5>|Q#mYWW?=Lz;s;ZnV}?22WiM%9EYvv)eZ9Tw z!IJp*@77*gUlOa<=d6RuC-`Y8A4@!Rt!9f_bl#)!lD6FGuUvdWwB;M`8m})oyhAgu zNvkCp;Md4+ksf=x$Z)?N7zKXp2?=Jtw`9?Z+|1gzU{EB+^dmiqp*ZbV6b$zV#JWW4 zTe(`#&0T_RxH$}}8~KV0j;Rc7-?O>>`t)XQEqs`9)O&Di97e~)8*Nm6-!urLmYqG^ zakH0%c=i(PqMmy#=Qi}Y-L3E_xHgdv5YWcHrVPt)L?m- zr%Wnfv?LVn@Yrs;w!@@Y1UEZ$e>ML|0bRoR3yV%=DflcN1;DH0^5

Te$^*o$?QI zh)DXEE!jKZXjTqM?-?V0zCyMG0xW)S3;^BrWST!WlnW*=wJ7~i9dMkAU6OVbK-k*C zZ=U1osvY?~H6cJd^`el_yb^e}VAF+zegrkT`L6Jd zn?VlLcczURgN(MEevHKeFz<_&Zm zYh+Fbwi~BEHU!i2r{ndaodh`_(NNY?yYAz?6*z}O=JF*IAp8_L%%f3| zP33e(DtrP1fZmgWzIw4it>T=D=zw?Y4v&D&R5Aj)hsBNqvi)X(rQR=rCM!>2>V^v8SktHW5sYY4jnwRt?D zV?lfhr0evmhil2BCXU~PRc0d3Q>}&!)qInq-p?uA{F;?gS@z5@5mjTY{6P6@SVj`9 ztp*|TvDl@PpAK~Iet`sD?YNV7l%or@FA+lfttlo7dsIGcdwZ z-5cKIrl8tdJ{1Vw6O=ecszxW5X%9Qa>Y*pkiPIhNn zpI&V=q(y|FpvUs`qInu(D8NC`RV)Ti4(lJ;L%@tiRcCgfx!r4mL$cD|@F2eTmo+*$ zDe22<->&X*^T4_XqfPnjGTGE3I*k|dC8Hj0q$0%^k*fi#-T$bCBp&TxG5{`ajZJUO^w+7uQ z542>iJuV;g_4<%_n?HUS`lb~9F#XnZ;(%Eoi6J|?BUi}C5^(z<>w*bOCmT$x!Yh+L zV-|G$`ctU5>grhRiM4nEZ%5LyC#}=sX~x4T@^*`|%7@&oSKX52 zN3S?MKI;$@X!k}Q{Nl0=!*UK(PpWi??`LxB8hSc)nVESo`a<5a+}uHG1{sauclB-J z3Ni>>N7zfUq1RUY=S0lCuF^a41=L(xbJJ;AOt zK~~+3$}CpGhQ*(O11_Yisa|1b9J{4u6(k6`gSil;T%m`m^I*omU3cCgZf-gbl147P zAm6Rli;u@S(>5D{_83)k^UbfVz*}oq))e7OHo={V;n(-%N$rD?21J4c4NKLnaF!ZN zVr`k>py%RB)55&cZ0J{>Nxgt3)j$L3o-+w}nHh&dU5qLNmv1g_v!U3jj3VHv`Bj=& z7A2lZC$Oc{12HXDhV=L9Y(g%3IEfxdH~%Mcw5RYfr`9nNnrag{_d7D*m{fzq!Jqo# zr&WW_R`#!J5&SsU2L1^vXM2R1Oj=?d3mjdre*sDKZB59PaHZJg`pGXP;Rb^|;`;rM zXjUWg=i^|*Wd{0?jhwTw^t#BVF|rymW4kqtGnmC|z0UOU13$a@1lHFi?NI?g`wn80 z6n1y=)3u;N0&+PrVOzMtivDqhQP<`fm88aQ)Hs|q$UzEN=7uNk={H4Kvq@-KvFOHo z9kM6iTxiKv4(N6~{t-$$NN%-q+f!N1%Do7S25<8o;GVj@PB-uNqYk$X3!6M2UhHe* z3O&Q}2WBpbdZM98EED4>XYp4imm z3c=(<-;nf{`VO4=+qar1MuO$Oyk@#Gkg*cr7H;%ZtjaYWmi0Zm^FVALkSl8UrW|x9 z1>FoCi`f=zD6{~hkdb4Htowg>v=-oaI}pHApsz<&{$QmcVM#g^jxZ3Za4$sl7}_sH zmtN{NzwYcBL%;;I3vo=2JHf_VV_isi+Dw9BFc2rjmh)ej<1v|z5 zud!46Z9J!c42G*~rnWze4pdM6$Yz}Ah!g?OfZXf<1jvOd-^2sDWp3f8ZjD66Ul<(r zw04q@s+yV?PLg!8yK4e)NiM;2L2k>is11X+A<$wI>EWnr4q1j4R-ImJ^U!mPx&Apj z*uBV*as}Pm38{?Snk3=X{blFjH;`uJ~L?L{+K&SEic?Z^4%i$6em_+H!HYgVM-_3hOjKYBJVIXGFGdD$sJQ}!Kd(>Gq++^4DK=IanURmIiOc$Emv2c{{D-*Ec_ z6%gR1w7X!_k_kgM*kF7;pI|a=&6E$Cbn)kzn!y#c!g^$V74qA1Xc`YpIvhAQS-Rik ze)H~m(WOiIZ764I1VW}NL%B0w@?gpzerUZG&GulkCg0*4EizgmbSV=_zRqdAB2TW1 zI$3nx>7k?GbU5=`b#)8?^NBRrj0b-(H?U!HPbz%9P233SZ>Uu20g>rESYq1g4b1P* zRtwO&^WMS+7r=bY-a;yi3Z`!Bx4Nq}nt#ArjT2D4kuqWJO`gSNB%T{@&F%8Xk-Z>N zK>9-5WCb|Y_F~6^{_%Picwjb{^bX#sGzGd(>oh#m(&$+R8i0UwcLrk)N z`H1~b`(j%ZhRsXxe|+q}u9jJ!k0{js>}@482P~^(pV_WuXUdZ;gpCo~Gwu^tXuF@Z zpGJKN{&TZ<|K!$$oA;3&T}e>nSMnn_sE^ec$L>_$bT-s71QfZ!tCY9&;-^@HhuZZl zR2pSQpT_jvXpfDw*M7+S+~TY(vWG66y*0xq_56f$YT)fl)G>WDndv4kV+1^cwy5ru zd|$CtUtv%+%Z~6y5pD?GGpH+UTUtSv`tyEH zIz@fOXoWe%J(8muHOF#@OXDwVEw;tH-jBc--wrmLM%&IuUR^$uD?XvrI6rf(lf8ka zT*=9rp7O0|AxmKsK|#^AqAOP-IN6xgTmCD1+B)b+G9OoKds_DPaK3$~WUJ@_(M%PJ zqN>%{^VZ;?`#9gu92fT(noISpTxtvhhRvRWEA7ub7TWZD4%ItQe2Zay24U{lKD5E6 znVsNr{raLM5M4ki zOYE+Yqank1y4WCa>B(C6%l#r*L49kSKq1SafKBIxZ%`nXkd)~XCIpFgW#*NA=EkQ9 z9CF?Hme8)D!~#9&_kR++t>ysXPlT-tV|{9@NH0OGW&^EjH3vz_KkCUL-J-_@tI693 z>xWpCQygWad+{5(ZI598^She=6U#mtmv9K1=jYgQ%`w>f3Ys#xdDIJHD3_?^xuP_W zTMZ#k&zZ_Bi}Bahj~x6cbEu^pP|KtNy}l)&*BC)Ylipfsjr+40_^4FUW3#8wgtzmb z9!D}}GEsEdGta!PSjpkmqku|51k=&`Qwo|tA z9=59@HN;&S@N#GkZ@qOp@^V|Sp3IB4FS6oOJoY~njy@upcN*B}BWzKAs1}ZXwB}$^ z2!1Bh(en2xoSvucL^@wHVipP!G1eXeb15HbZ11?kvwv=A#KnT?I9MFG znc_lmoZcm@C&J4SY3?n;5-`}&tCnh{Xfz|08;xj4OFJOFPdXo^P8Ay147#G@YZr3X z`*hQyZ;NcvMTle5yO;HJP@c~;M;z~%wK=}#$a9vH<-cYT_Q)I{@Hg=o<-NHYthdps zj||E6AKC?pN^y9ys#wp60W=x_^2@^s0U^u}S3C{Xz?g$+?JOMeYn-apR5L}5K4X?o zwX=_i<@2)NYx*l=FM;FReV5ydU-+O{N0eJW^rb;XoHr#2<3Pax^EcXxX~VgaZnZ45 z!68w1DhDl$hZoM0{I5IWe(s8q#!@Rt9-2JmJja3hR))sqUXAw&@&vOS5AkEjjmq0c zAvRHufsl?=OtrfIEwfKUo3ic|TeByQi@4<^i%UP4?i1=hZBD&17&Geb{@kD!ZfJdB zhF|U-CefjlJsrBDFaTs6?PeC60;N`Qco9)8;f*IQQT$;DLoqcAxxXXLn_s#dPrbJ8 z(|&=)r?Rp2@d&#%jq`^RS909hr&haY zJ&nKyp~&iu{tlnI3PL^FtNe!2VUK4)p73H?M#SiClcb2g--ZxGa;^YW7SZP5sWV!a zEoXg+chveM>ax$_cd{DUaH;*fL0`6|lOk{dL;PrPu6o0sV4vInRov(rvx7)XhF6XK zf|tTPF!p=MkE21ZgShYSt8BkOtSi}fo8hAgIl>Bc;!IW9(<2u$@4#kf{Z))&@_tbo z0v)4@$X2JDZCdEIo4z!5`5p+17#6Yar|v$J_HeM={qc2ua9Z>Jyz7ke%TRbsN&E;! zZ)PRz;%Xj`P#$i8-NW*sZd&Am7)ftpBnHKItFadn^(W&2<6uovo!o9Tvor1dXcl&i zr=mKm4f(CHJsaU%y~34eg`rh>M);tk9ZEih)yJiKK`t74mQ0Vap~8c@57?;Q2Gm#LnS{JnUNq@Gyw(*(Ezs3Np5QV?3nq7QL$S>OCzYePkXF;4uH4q5guF8i}_YpVSESpAzWWR ztL)=zc0;~6l*|Q(!LF+fhm8)+27hn+dHJ8d7e0`js_q4e8KBU-of@_yul;{v#Sobe zwh-SaNVT!)6$-EQ@JoPZZ}%C^VD0?o`~SV_a==xVd9b-lY|LCXZFer2VWGdMUWaSE zp7q5nTi#9;&lu`M%zVuodT`2z`EjZRyD&>LbntqSwX6995FAn6?`&lN(8LLEj?s&E zJ`kwKV~FmgiHXv|uRhMkb|Fp+U8{+(QT!RaaT1q{WQ>-{pL%;R+92|+KZ+;eMX=b- z)Z0Vb3?3+WPHRW5JSL43^t;}~NObL>2U#W782QEAf~VmJ-^k@2O^C-@_{z`-w{*1k zo%An1LJ2Ve&0j*|QHy6bX;nFBsg>n*y@)Wsn-Pv)_hyw&@Rwuj&p4yAhjXj-UzdGs zHOH=CMQ3SgY`AA>El+7g&)V((TYkVyP+c-&(H=ZJfI)w^LXp+-$g45=Xf_aV-F(y3 zol+tw_hkHMzY1U0!*6Pc1UeeMq2RSvJb7yCz4*q7uFIFb>bk;N@38UJadg4O7#W_< z?X`msL#65Bz(k>&(DCK@b~OV$V(!|fpkP1_!a|Ax_qMxDB=klzCnA)^r&_Ht8qb^*!6<(W$FjghRc7v-*3lp(@asSVZ6 zkDLn215C(z|EA6FNHF!}Rf{MnEqWbwt>=nr_5!&yi2bVLf!@M}XA=z3Ppk)DV!)z= z&fbmRO@oZaq8KHMDN?E(Hr^gzgZz>zbJ4gs)|@((ud%Cmh#Om6e{Q1AAI+0tI(e1; znA#aJE6<3eOWm4vwR(pA*7H)b=q9(66pvHIuSf`j6OuAQH^5P;@!mTlk+2KwSUSz; z4!8HR)hwOM`*n%@nK(E!*tw=TWHMg9~JwZOXv1tu6Mq?c)W*~jlq;#XwTSN z!vf#ZF>Pe8EY?u$TsIcIhHcR?3pPi@1MQonB2aGPD}`^q+7@5F{QM^XeAo<)R6IP( zLdODs@gQ89BkOh^v}w8W44}|ok(0>IjR~&|`+b7dD2sKa!v7|N0csXtBsdVb3QNcJ zJ3=s#v|^?kOBv_$CJn>LWQjsdP6vOvby)to@-l@)c$6Q-EL6zqO=+(_Sj5(f!_gTv zuAWtXx^gtU1rTj!>^5JG;MSUM;S|e%pOnufByX}bKklM8k%(p9(FKh$XEzfU<*_}Y zCVjPj%>n}scRVCL&A!|Kla$EY&=~%3VvsNWSiaC()=0~d z@BWvf`FQe*HaE5+q<3{zV65<9dh;Nv@mbI5MZA%FnSXHdnk+-YtNA@Tg*|{}JZE&_ zU^J8^+Ztz(Hx~`r4L$lW0ob@bP@hp{CCsE)U$hN%O~cucSsG8~6S%e&Wqpnw7-R?2 z*UlvRfj(U4Z(dM>tMDwxx^hETm65E*{}m);M-KM+tb8e7-FNkZ@fH6c^uF#Z)#wJ~ zM9ayH_vYskPg;k34uVS(e-@pTzN5(|7grcr7#Z{Im_48K>rQ{sLi3k;dizmta*9L77u1lgBx0tX|TxRN_>#I{Il`w-u}kFr%hRH1^og`~5=8 z24WR?#9r!$wX@y9QX(jlJ~(n>v~hx5Nj)Ri?*oWF#Z}MoJC>GtxX>9_%jNIO>iCzm z_GzP=_D$7GQycCGStN2R<%~K#Bfdl|sR;goj>1~4x>_c^oS(6yk+!@ml$mvX z?<=%1-N$ShENRVDU?YrRV}@QBhO7zN+9y>3zH#IlBWGg!>;2)NDUr9kJu$*_n9SwO#VlZ zihvhyn1VsyDFe_~&j2D)7ze#?BP&Woh)qOGzmj(^>fnfB@LM}SIF}kP^!{8RBfXjO zFjPd9PX1Pq^>zIY-t21j?%MYI%*_XSc|s_`(I8~Z(TRy1*Cy<`N~D7w7}n9lBl~to zH&Ugfi`-A^^`2P9z(6$;8VboRWsr#6chw>06pE(1sUl+q45RU|QcDg~^fgdr50;4C zyL~xE^<3j5GXf~VaVL&qEJv>9(Djrz^fJ?clgE@(mXcZZ2oOwQO=jk|)UQQ1?UW>A z#n|N8w3b#`g_nQp{pqKZ2f|XXB91+@NVnGR!Xi!KW>-&lY~Ml4AuVD-E>jl1MR+;A zo9*N2SC^4GlD+_B6U|vq4x;8*+z0-B>qTboY*5NQr}@5BPb~GOQUwnd-sKJ_gbIba7Ja;rsJjnPz#>i$Cer+@x?1tAqm}Ig8jtPMQJ*yVJ0}W z^C(>y*DtA~u_GD}A0$OY*B*$lKCd)0Zoq|sOI<`(qTnm*i%+)5 z@vbqQo?20P@hP}W z;6v*U#tW{_C~O2pd&TIw+5&^dE3!Qv8;^}W(=H^s@#HCOZf-Bgti;d|Il6#Q*4Oqm z)6Gm_O%+xL0nYHkdmPr7+K_}5y0e49%L>(POs)2$cb0-ThmQnlo?XpWs`u3*>6fFU zfMu>6{zP@Xb{=?}u(YLXH6IaTu{rx9XiF|hLjhW!#0llw{X1`a1!u4A0RA!z-2b)d zT=!;yhm)-b{|VfK1Ugw<8m#sIAg^;ya6}k|W7ewYzTM!vCwvJd0ajhmH!8NXzAg-R zKazSxNJ5% zhkIuHS$6#Gi941rR;+(>riuo5%Cv+^hJ`+Nu*dP4tOVDbD4)N3l6D(5`CQBIdS zryrD3Tgf8A!`CHB?sd)`7~%wd$lYb;JR@hXQ~SNbK-84(2^FTpY=Dx*0=o7Lb@8YD z*VxTbW&e#oO_q-|-=6eSJwH>?@?_^hTSf-`O1f(nK;hvNE4Cj)ZK!z9m*hHwcOoLgW4~@SL?8k>%J|XeWINey`K2B zjk~NfNr6Qmh^*n$bJz)fT6haMm^#{aQ*FK|;1oJHA9!kxs$1gwlCwLfm%w{>3`78R z4o|&dE-(Oq&cbJyb)mc*4T2LAVAdu@MV#!DS*aYuyAudMzO9l5zp)Gp_% z*os5^ewkwzeSa%G3uul%t<{1}0sOC+L~8In)7(HSvVDs8P9DF_zMsNCQysC%dj5Wc zn+Rvkj|6h5R@t5h&5KQDXX#X~be#&1AwurkDA^=x6rQ18IJV=+bRFO{ zOQtvF1@0>hC);ePc2^3sWtl@BgR^b(nUx7uk={rzHG&E|ax zVooCh($A$wy_`!uP@PsO|8|p$!5)l{K*O&mEBoy8dd`#iNn3s?W5dfw!JUSS)c{rP zSHY{^1n}^!Zztc}tB4$U&yPYEe18NYJ!i1;0w)U^gNfgM`xbohZPmN;+Q*;zL%`Om z^Dq+5+HHAqeUhBn)oQJ|nWB(fn?L18h1hv3^dthaZ=~MQL6SYgzdAT%ZO%72YMuJZ zo!#3QP3BW4wFiQVOnGc`NjAb#s=gB=6Y!RmBQ2A@Hyj@*KAf*C+<)v;z~!--fTdM? zH5ShafjAcw#qYkdX*yD_jos#<7GqKoaplpGwdq#`5Oc=I zcau3Z;{T+VgDg+j9>xxpE%h~5RJRu1I$K|0sJx$;)AGSYBi_l8Yq@8Apra*+{I?bW zSL1X$r-12RP~|Zwpk9MQ0-Z?iTchCBFXBjNV-&%r-}HrAxS0Ut<>->pn3~pys@0<+kT_RQQcC) zDfi{lZ+GU-3N-cBP8W0*4c&tR~?b%LbTw*hG3*1$ZwZyN^S72 zh4}2mwoOdAsP%#L4m@rX*iWP2O(~z6P3$$=ybCGlXMn|{ltmdq;?EN1!k6^&(VL>C z{6x|Kki%08;cydU=kU+kc)4sIjQomMOO$V6Ox zTfBrOlG1M@Q|w5sW$v@Ku0Q<;hrVHbQ725w->z(0&FhoZOm+!k@%adpG=yS;x+~cX zd~I0<3F4$e2z$KicKC=M=TnPh)Ef1Ht7@lj>3=7l-;hz)r<8I76K-GLd>6OT9B%H! zhaPc!ePHH_Qyz+iY5zKY0>8KF#b1phDsk_w7HnUlu4WVKud)+z#9ESV(9-)3AEYgB zI{EPtp!*?aJSYzyOz2AsPKHRM#2ULau5UKdPSp=rot7)I{_u{EO_qp}xRna;di@azyrM~?{P!HM?BxqJ z9cV00ZKKnRiaZvG$CJ|Yz0tI5Ckf)o)ewKs@k_W`eB@-J|6lf?Xf!)eT(-lENEh06 zg&n7O5NI!`aae9O949dU*orlNmK~icgoML5LE)9&-in^`kQnIPCgu*C3g>B$=`&tC2lu zGolig-QwlBOkW&eSH)ezarG;7458~ocO+7@_aQ>A!vlVpRX+!zSpjr(!=4Hnya7gUCrMi8Dn1`f7;E9 zGwiK?PX(HspY(iqag2%+zT6TGbS#7vmatONHHNpSq^U{0!rL)T)LY0)Df8lhReL)K zH7C&^PwpT6`VK8f^~eKqbQp9bq6cA+NiKuZx*D1U=JQe*>Bqv~F>kTO79=_Bu_lp{ zydYScODK?N6OSDWB%s;>NOxW&`*cuMj2HbfJobjbA71&7L{`0^{xZTGW_CcMl;e~0 z-tqMSWk%Z-M%56lOf^ZJ{^Sy6~QyUKB{oQAoe3!~6B5<$2?Lvpq`33E&O`6pQj zmB4SKE-{Jg!|2p@{F0XFo(Gq4vc;~|f|rXVrBhxcs&C8_;?-UjBqZb)uJlA8BF{T9 zwsCaYF=i?!s4(vGI1$n%xF^`TQ0ex(g%7MoUSUp#+{Ld*!UGR*$4)Czj6xIYtc)q}^futl@yYS%=y{t1cWfIbnq})t7p{DYC_x#)5Zk)>ROV*ZA ziJuju&?5pF^+o21OX&^@SC-`z+5U$7Lxo~_f|#5{j7CvO5lZ!0_8RnwMvKg~CwLz8 z6XX%Vatdy;W{i*VW1cm@WMzu1O%LP__Jv>=zaT3sSh)H;-)gop9tdTQdxj>_!cn%HR3D4$kORZb_I zSa8t}x9SgzUa1LwFp_=g2=iLxZvhf)rPfn>e~&HfwJg>*VEgPn99@F$=k9NMc4XJieCdey$BqzEe`=oN zZU16-dXjW`?t-&^LqZXJ`SR_76)l%;#8_DC>*nevSnv2pwqAV9R}|jwj*A7m2K(Jl zN0nrb(zoV#Yba<+rx0W5_FY--Jh+|$pwzJtN8=vSim0h?XwSVemULyQbUvG!BUrI8 zXC9YqT}We44-UL{eYDV!E43>3gTTK&TYR9L8APgMm^>c01Z}mz(LoH|KfitcRrLAI z@tWQ;qBvZs-|xY?JL8TWc^&*|_k~?mE)$&EIav>Hniqe3i!~pu9cq4n6q9L<1!IhH zDjDQ5b43)U+d7|Z4NNpVONMVr9mZs34ACLlAw4Y852#V&RRk(S6{i+ySj2rcD%t1e3%luC1M)qfnTF43E zZ%|&{BbX#(O9mK2pfY$#w6Sz7Kb`#d3vxweZeTS{S<8$S*2!KWlb8wKp)#V({psKO zII(QVFuljd#(Y|Gf4pxluC|L`CnFTt&%%m@_JjN;Yol7-2%1h)>R1}Jnh+Tlx{0t? z$?Ct<8K(rA+$KmFa^|qbZ-4!2PI^kYVX4pkY&i@UxjnPrK8iqsWVdH6Nj1N;h2f2h zP65j0zU)4$(D!FN4#gZ358WL003DH|VARET0!$3h`_p)!w=c_tC8)bExRWi*N>U&j zs4*bYnEaXJe83EK;sXDcE+{p8+FFha6N1QeR|;BdEz8hcu0vnU6M&jNkpxye@F!ZqnKtq{xiKwmNsWKML`wN zz(GXv+Mpm2B?YTM`AgWSOh*RtF~2SM_@IG7iuz|Ju7-cym50wSn9}ewQN9H`w0^Ym zLNrU$K0B|^h^6Ims2{_L^_iJ#X#Ep>M;ADv8XGQptBR|CfRz+f%)wo6ZCf}6lvC8{ zyG+@S-|3vT$<77HTcHq{eW_ic+~@a0ry)abE6He<5Gv_t&NuL{{9u#I`b z$*-o0$EYlbzkGZY<3@2$t!ZL%xR9E)3nuAPy0#sbb+k@{CSSg4cdw=~RT4g*hqlju>HBFuGh!Zd!rYDM?Y_dYN7* z&b*C!h3@(@v9z9~2R@l|UNfxvFAWz!{=!}0^!*zkPx{HO1k#XaFvd~_7Vz$5F^euJ zXmsfA03-H55BAq^)A%l^(R*=I)+z@W5s*otc?6&#TI1FtFLn;r7D#0U$OvB3< zg|ff9sIIrs8vR%C6BXf;qC(O2q3C;;yL(-`B6}zP^+6nuUOh(4vHbHfQ)YuVIgNLl zH!zXEXCH_zWeG!v%2bW@BGAjj);s+1d2r$XQIE(=9;857FoFl3qCCv=bnl9!o83 z54@tLM1KvF9}znTvwm`fL}dG%cT6-&4scAc*K=ROxzrx%GDRr$abtcY37OCyCRnx9 z8=+)}&I~jDJ;IUUh?O#o?VsZlHCzoueaj5ryr$BgoWGPYGtRfXL349K?A<=w9>Z0h zi^InfGd46DGi6J{I8ged>rcRkAHPQ`^z;2A(1*yiLe zj9p~9q|gBWZ&Ed%v8pz_+=Pj`J`;hqC{;y5MhDvH*|k?3U(%bdgqLS_gZs044Usb1 zgKTh5_qqNS5Apps%azSxSlq@{yTIVG)n~#mrK0caJLrpELni)gAiELPF?!S zjiI3Jg~I9l>;0L`m0WL#Wl|Luk2j%rH!u|4-d1|{xX{}X9>Mtz_TT%hMFcK7NXa5S zFq8j;Wc(a`?=cTA2g~Gk`)(MF*y>_QwYOkl1AW{&HNE}~wYB+(ECN3A-^5oh#a!Ae zdV(B-fvU??RMXR-P8Gr5fe=Lj?h}K93wMgG><#*y@e^I_cmmp{t+gW+-ZHZ!r*mjN zk>nO~-NdR&nfkCo7)2Bd>rsOObtS~t|KwjyTo`uEx&?C{8GGdj53QyAHJh&?t+cJD zl(Sqz(ufm}Hq~Qgdy$mN)=?^~YcS1sT90tR@DcwOe~P0z?m;tAe3lKb)40NHju)1b zXO;vcEsWOQZc>3E=qh059jwK>#jtTN9BckeTcdGpA2%EFRdzOB#3O@ zhHHqjkf~OlqXVfp!L^oa)J9*e6Y##o2WbiQOWt5H#m|3a9??1r0wZ zRv0K}Z;i;aQ{9$I;dW-L#Nc!?C$VN2BY>L>gn9{KqFr7DWb&j%q=2q+JXIz-Ta8rA zBKp+q%ci2tK}uE*!QYo#fte2%*z!O((y%J1+MDyWh0dUFgcpZ1sVJ5ST=&}GBHqcY z`RR|Yfm@s<=xg(c4i~te%#j<)X&yN=W_o^cZYg^Iz64#cOg&7$t5@X#l7?>Eax&Ql z&s>@Jzm{-~gT*)V4Ws_W+sRR7WdD!Xi8_581U_LqDoM~_9G#s63DTn_2fZHRH&yyX z7R48EwR*6uF1`;gtivWPh%202bRJut{=Jz&iD78XOwC;&qG%e>E$kUQtlYNjRO#u2 znK7wzk^jVT2zoX$T<@Xj@nQ-~H8l1QqEpM_Sg1VIZQ*o(8LKS1wv@*x&xH9Dg1zC& zp?E88P^bn3xoYF5Rpd^^;$8@t>;4r;yYY^aE&(*KD$3V<%*$(gO?SpRxNH40^WpNQ zV7Pb%R>PhHJBbS4_9yhu&$a+_nmwXq<>S%X(V|O%lT7 zc4V75(M;J1rn+s?W;xY{?sF48e1?1_SnLR+<&PKWm@!}e24b{<_;%P+ft)|S8&dAy7D zA>(*Tx=e;yXwra(zGbrz8Pu%J?fw^^qg?9B{&xZ{E4Uh@ZCSzkJz3o?AX6%ABi@=B zx~rIOEX+c=YWRy_Njo#Gqy!91fPt$vyZC|WY!C?1YPWdQ37v5Wd3>fb{y!Uc7$^1A z2RKNqwjM}cr>LX@h|88OAU(c`ql zv7h^u7XY67L&UI0+10k4$O83W7JRz>%+62lge6Oc;E6IY_MznS&Vh-zHdMAs!E(Jp z#rt|g%1dz1#xU_1fd1rinFIT*;+xYPM>whb(@|nR*f_0|gb0w*+UgljJvXPIy}CwT z#H~p_GPhn+{IQ|20o|9JGhZgdEvd4M*9PTLFCZ^kZF{jN{w@Y4$Au@tu(!O4#hd7q zqVRz-?U-_ugf74=tIU@|Mp9FQQCWY!Buf%^;I>73fOHwU+f(HI=FOzDBoi<_k}I;# z<16IDH&UCIog}<@?X!b&kvWs&+eQ2uZ_hWI*G36B5P~Rou&GN`k4Fnk&D+A8=Gr}) zt)=6cv4lO_l95w^Y{eR<<(zdJ#T(=qsJGjk?B7=>(r`1p&1%+l>YHg9P8k&s4kPI7 zNhAE+65p_dx<02LTCRBt{9Df&9Uh@;_uX6sI^1fY#Uf(|xrr%PA|xv z1^Q%RanWbD3!znKte^mFFaj&nzn>>V)sfu|b4$jh-dcuPS}k_9*qBEgAhn&9$Dz#b zO6suPJ%7$#`G;T=_wEfV76JOO1>T+5SCz>z97n*V>f!Lg9dSd(U!`Vq8o7Sm{DHV?j z+0_dNIeQQLOnA1NSzGjcKYb7F#Y`@Y`Ud&PRCQ`yXvcE2)dWmMOnagy@Ykk2W%id3 zfjV`75(rfT^HhRxoG0e9BrUo1BvlV?BA+!nBq+qxQsNH ziZ!2$XP2gbS~|y7`pW>eaM*_j1B-?b3zik9Kj$==EssetM}0mtU56hKNcDU3ekf?l zK}YPFlncCe8`7qq$s{8YQJ5G6YyBP*v+J0aQ4(?Ne;p_U2$gSfn{_3ts7b-ZAaSx% z5|oTknvTVK^I!TFBRu_m+=#saS?i1BvhN^F&)XbL_eI>MxufB}l93+({J5*?5~zDH zdBlyVIpi69yzwNJcsy_IoH|3G1gRw{3sm|kYiMF*hw~<7$%QY2a;i`F>qOKcyfdc` z3&;GZp>rwx(aoIf&sK75e}6l+O%z%JG)SZ4``A(gJjFZs=6>w&$E{ugYvw9|_FqaW zi!kxqievn6>PAMtxmB~9N12it9P-6Y_d#!)t*Qinirok)L;0NIr2^zmA0E`BZ{gES zvb@VqwHjvLomq@~rg27bc75p)5Otq72eceMhpaJPNvwd-SUkX zo*_#{!lNziAA2DWq8y5Mm|9D`k5-oB^KVcK^@a$VTA3>Ouig0Hz1hsls>j7$wNpsz zXZ_5)VDNLfGALqyZyj^Z*hQD z*Wvn5+LuQ=!0SbhQ`bZz@w}XPX5ibGJ8|VoUi;>Ga^|2${7w?Wbly}Jm<(vl34N836* z31|U0U3gnzw$}j`FZzPwhlc;7LB%w|Izs&}M}fT8E5f&THB)&c{~gz$A|0|g_@%=U z<;A?8e-mURj&JCE?@*1e*mTxAP(y9U6-Hs!C%5HCeQ}b?*gKGX4L`kGDW6t66B5Os z8!bL3yK?2V`80)lUF4jOr>bSeyc4?euBFxw5sZd`1 zKz_oEwv$kT&lA8ClH|l%gdLcSj0CaBzi1+JAgz*+s2#QB^rf#o8AaVDn&tS;*Icp_ zVQ(mt*yqT8Fsf{TiZp3BWRmIA7Y#E_HJ(a5AL8u_NwNi0A;wU%n0aH4ic@pN`lC~L zn~ZwE@n$DJ#}oXoG#>=cGp6=rwZk2What4gjw%S|mjlbuib*LEnUYz<>se3}SN-Cc~X$CcQXe;qbonvwduIq+eGcZt0VK4S^lCM2I?^6PM@G4VjNXB zmG=B4vz*pjA;zk4wG>}vhqhklp0=A}^o~HPrvM>jfv|UF291~%n@{Di+Jz#$zn@heE}Sx}`0m8h*2UI>-yLT~#g<02}?cXK#`w z262Y*!;4auQjO+-K)`(6XadHkno&i3|3)#JM-kJ120?v`gSX;oR9!~i{W^;Ee`{xh z!_A+QW$JQ!il2ey%bf_qs;4L_Jpqc;LND1 z?s1K58KtW9yuRX@>I23Ra)V2jG(~*Tx>z#Bl_d>x<1_Y)N$~01FVn~W)as?jH%;!0 zibs`Jw|9&qwVVgkpz<48E^IO8)G*_;wW7(!{8Zc@x}%ARBgW0(YF+O-oWIDH(UVlS z%yhIAEn7K!@tmmE7!L>P@6O9ZQga9l9TY+<%?+YrDE2+R5E=~va=4P!Vvd#&CkK%r zBnO(=uG0UP1t1eK$xG+R3Pv_{Dj5!U6}dV+;Pg1P24rY9S@TQh==TD94cUVV6-_qL z8RExqYfXoZoY&!!EzPPiBViHJ zM&>{tR`&k2IKe?f?IWkAbe{6Nu!>O+TrpQ8s-YZLhV^G17 zO2V#KFNC&7j@O6 z4I9;WKLe8y4Y50%iM?!B^CIeq8#K;LPXv(ekE$5c$$JHB@q4oqK2nr2cuAhl@T)Y? za|S=O<3tGB=7*dZBAC+F&TyB*8`0xaEKTvie1c&($hZ52@qtN`rKD!q-fo1PH#~>9 zJh~;#z3rDO(43Yf^fDCQtDdd{x@$z81h{&asB+=GSNfWEJi!ipmxhBVpVeL{a*yy# znvyO@B^#9k>2L|Y_wRWZ%11og#Lx4Xi3DBo^|i|?S{e+coC)>E=xJw^sLgykqo0tK z2i_`}MXg|v@#vl_s#(dXq%=35HM-@qvJFq0BF|*jR4Kya$t)Zy4{UCW=moWrR~F}F zC#VBesa{}Z&j(tzeYk2bBmxm(zP2}InVVS!ml?UeIs<>1gGWrl1BMmyp`T_|tK>K! zos4a9c70U%Fi+5)iZbiF;|$ulBdBfCKN>@l2)( zV_u3$u`YkzS_U{u<{QrfTIqoX$JII+fXOn@LLjO8kxDt+bg04fQ|=fE%(i{bcNYe< z;YO;PB)j16)~SfhP2?=?-eT_-%aJ(EQy9{GsvsKvq}_5Ykz}@aJ|NT7G!9#%$`6wS zXJa~^iDY(^xglQ-)ZDQjJOSU6u_Bu(j` z{0}o5j>aOXWCWxodgm|o<{b}iI5eiuT`qKA5g*h{?Jq(Xmz&7tKF^X>2E*0OZjA3E zd6_Qtyv6k8lm}<}tKYpyuznK-G>&o>HFr48&Z)?u8-NQ#7FyUL>AoVrbxcOKi1i;8U)f4eW3#_OvE8|=#s`I25$MI9_A zmD^Qnin%2_5q*0>D(Y6}+SW$W9U2V!J2;A&B^B(_Zu@vLDoNe)91~bHqak(B%~LtT z@uer8zIlBAqRO+E-By8}54Wgj&O(|-+P6Dz%5&<`8CdT^05oLQ8qfEXPF!XN!*IG1 z2>T6rE8UDvdm2}+m4j~U2$FL`SfAZizY9L+5}8(BSQg-_2;|0|aL z=V8k~ItV03)xE;!4;L@yi%d;K@{wc@7jM3~&&sLgscy8rSd)?pU+;`!BPX?=WN3fN z$o@O(gyOczf=yW()^Idlv5anWpv<}`OeY!2;n;6M;s)|XwQBV5$M*}>X7Xd{gw)pY zZez*6;l!($lTan!D;BTu=%b&|z0uVgJEFJHwO0q$Jft>ej&~5qHxv!crMC3x7dDZ^ zCZXeU#W~&?d8bGjQ2{}|(2AuC2E_Wi*Ft(P08mh@qz2NPm=7_j)a?J z!Vgl_Eb>945}PIn0IzuG*fJmRKQGrort{e#3xkw@*hyYyPszzCbB^+9LUal#*Dk*J z-_7wQKYb(C-9a9U+|Xh@408N_r)PtDY#{p+FU#@p_i8_;O)l_&afO`p0r>H)ybNBF zvklVaj4e&wO=AJL{*}&8Qfj>q0&RFa*Z3oXTk z`5^FUFR@PqZT);P3m|N?J7*hNt;M@~pBs!Syz^EokEg1NTlqiS23(la>tCh@=03cc z+MYa~W5qk4DI2!ir$sMCSl0JPH?_S;VsL`CX3|fDM6j9R$Di5E<oc4cy<-X&lSuNBo1?xq~63 zzeOL~*ITUO2EjMV$r$JaZYCm>wI(MUWvKSJ2R+|2AePa8g0`>dU)wf9y|kYKD$lpr zUmdx__G7g<+U2JpIcv>ziC*#pxd7OpneTQ}(65^dpNx7yIlO+>i_=2bT75}#&uF4#M8Sj=yh&jz_1hE&#N&ykSKDKWcF#eOP-4kk1L4sy zOZhi<+cc|U`L)il_HArk#GF$%JsxXp`IRFY>q~1=_I`hhpsq8is$aO? zXN^5}k1MmsN_37HUy-t3oG?B(28&cLg&QBAs;G~MH(oq=y#@*QbgJIfA`1+No$-Nn z=5JoNn_RI}3|YDNDUu`x=c`}sD9y@vyz+^`n4?PK<=`h+yWE#cbDVRiF*jwFD}01` zOzQHhFO6{1J%Jq!{^r7rqEuY`5+x<5=&XX2Oy7H;<`XbVLd}`H znRC*Uk5i_Lh%r-^nXmthMS5u?QB(L4_Gg^ed0fR<8{QGQ-WuGwB1h=O9)zvN;9hBky7n#n)=gKp9dS3b;>eGi`mc;^Y-}OHxtT1Wo|M7w`1eEN zviH>=DnKJi)w#-^>83G06o%2}k`C&A9flK~X*&?VI(E^g_S_9NBh z=DJ=D!DVshU!o$oJyc>C0d*Gg@>Cm>%$lsg*{tQ}i%1dEY zYwve(@KHbx*f*&6OaWYneVh%AYvOQH!u{rcwK@u43_9Ok7a#6EM1aa2lwUAS-p97z{+xt(XX!vMG|b7w9@(kKhz-%VU>cR zLRfo{de-5NReps8WgZ+ZDqUUtHOLD7bBu!kOxI|;!j63g7!;aJzS`M%juufMzIDGx z*Srg>Ovd(Jtr3tp>0FpL!8~^DW$|4l9WaQn68-zkA!X&W+iiA`q~aZQ zWXFjMe$1K}kkrx)tAexA6wmm`!&oMaRD0Ot*1`7<9WG;sNYdCaam;UQ+ww2!8;R@g z3V6N#>O9K%9s_q+^5w-^Y6C%pkcN9{u4cC!?;Wl^xe49gvV4G;Bm>G0OSx2Wmj{## z{P-!D>%x1JqLRK}?$KS$$R{RkqhoUO4EZ~_Jyha)#87D0s>Se^V)^xXKKXgMh#n`| z*7A7v3&D-)01QobcKpal!od$~ezxx{?)F0L`Cpn4S4CkZJox~0j43Up%%^w8{U^WY zrhsb@PgGqFMvB%;-al~D3qQdwY9c#*rtM(*NB2?`CY@R;a!SO%N5}n6A7;V_&9L6i zq#S%-r&q%*I!86Yt^Z!zvtTp;vffsRN1(mp6Fuee90olibLB7coqPY@fsrlQ^>&6C z7Q7#x`po<}J>S*sJV|Cu7sS2qattPXUMVICg7xG(K{wq#631IZOG%zIO%1Z3%3IMw zaSZ<>NjXiL$oBHYr`_p8SniqtzYZUs7gM{|D(=qtwFKP0GQAs3Z<3^`j`}lr?Frl* zpMkc1d!hV=D?jwL)1VoIz2vl;Ozs*K0o^Y*NSW2$ICx|4)#3&h(uybiH&4`O*h6rn z%YK&R43kGes)3vE7w8_lN5n`6gZ6Dlv|jRW!{zmXHLW{A(a@9AHy*pObZ5hyLnFN~ zNCUP;*UGvXEwv;D0*{)$AkKl<$Y%pLCWdK%UTo(Ohd44?0@g^+uuI2-ihd?eipgxe zBw>|-vJ}}L56VZU8=-*ejbb$~=Es3a4WA3Lp-4l{Y=Y6j&P{NC3$z{M*e~gt%hC0Z zxA;x;Nfb4_@#)(>>x~Eb5WTnOqOIQH$!14JAD(JC*Up&o32JWsXZFb`db8P%Lk)Dh zLS;)=&EmT@=lu@a%t~TE@S&Tm4i|@sSx-1|wck+epYO{#9x;i*W=k%6xY^lr7ZHh2 z6h1OBljRT|XA-RK-PoS3h({D)2VYj2?AxxnTqxki+Cz_-H*Q--7S@D)K4-`-fSA@N z|7~B4VF+1PF+KW6M%a&xAmVboLbx#bvX|yZytpr_UMje5`9AfR``xoKB2K21>C1X| z5IjWu55z0@-32`DxVn!N4dwd3*|xYhpi%Kr2|WqMi}k01KZ0!+Q%epZNp&hy6+Q~= z`jO}P6ff5UZQl?eE0N3Y+D~)#*8=a?-(4*5hCb;E_;aT$+}Av$>$m8mCR#xSKwc^y zK=mRlwll`{64Tj?z2I$MRYiEcI{9f40JG9mvT#wB>Oqw=>W! zFLEu$bro20d2IhAmZ|f>lq@fOhnV<_UB%kH-rD@jH*AiYOx8gZ)Z&d0dqa+pO;@9< zpP^-+tb*ReBl!@2N`mG==CI0{S#68DEe@9xiu6FaW|uPL!@$oy@59LbxzMNT%M+v>;AOz>@HO+?AJnMfe`0?a~buf*NeV@ZS|ts1*Yq} zFTMiH(MHq1t#!`jn#*6Sn{v}1)1=T?5G4})&swB*Ce)Rk9kx{znD#vU)Gp3?v%?Hx zhFo{6D}8yn#z&iq%UW&ho(e91(=IX!J3n!j9Z_Fd z?y>LymNWi)MEUiNgPBW{$;v$r>T+RAo_pCCBI-l}VQ zrI@CZr?Ad*#Y;r~1(|S?Za?%pW}N$^pZn=<=yK+&X@w1=gz6Vtkkk9b?Mu&Kox0>9HVoPz~`tI(}lq7w1-QhnQn427q|E8 z?D)cQ`mqK*rFXRrYids_vLZENN2|DbTk4h@oCrAbLB~#teh*B&0hFbD#1F5v7(26s zc>u-0_nXO*aN0oshkW1nO&F6{g?i%hKY4eN|Dy3UOQXdE%#faKXQ#dAu=*lDLG9k3 zGrglcvxG3SoXk~Xl>zz;`WENjIWVN|3@#By0N~SGII_Y{9;T#jf6f{Fx3UVc8P>pz z6M@vzCdr)UA9R&;9AaSoBV@iP(=~-4JG2vP-5;TUMu6DNH=j3^^jpN{`o73Yq{*L` z;yk?@Q4?~lp=~mv4}?S@+9e^9kK97%efoO^sND3e;M;YFLT|Q^#1BEXJeJJpvX{=x zYv~vtTJn0($L-pG$~>Wg5!Q`>YE3UHpR@hv`0a^$Bo7Csj(gR;|&y3FiBod zU1sY&gSDuGI*D5+rTK1wdaP>cIdO0J*|FwGNmfUXAy~L6@a@sNtzSQ<5Ug4u2yEwb z_tyL}QNR2681@yCtzZLo>wMMnX*UtxJB?(Eqd(Zg2UmFtCLnB6kf$x>@cKlV{0EGs z=OlP1p27Q9TGDVj87#`tbtYjXh00Bzwj`bmJJE{O)m^egD`&&loGx|QRyD)yQo5dW zYi{uvUpDVY7sLK_t+%gI0u^$L{JrUuAN~{}$Y|e6lWZS*s?DNSwzTwt@aNrUloY!y z19j@8j66d?a%f*TG5M+n|nPxEGzr+yJc=Xt*%~0qkk2Bv_`EI z@TmC3DPwpM4*>NQfw_;#N9k+##lv1FEYhQ&u4I5-h@HmhkTDTPvg1}ue@&jNyPwwY zIsyglX<&84>lE{j)mO#->TY~|ct;NtMc4rxL?1Ix@La?f9kZdEC#G+hH^S1y!||F( z2ggN^gX4FI|Fs6v{<8)KfwipEbf4Y%iglEceFI`(1&d=2q0jbwUMERj}YYNiXG~)S-Z_3#z=-4EAjFDk7w< zT;xSJ+km!+tz<#LPE*eaMNU}~yq*F0c)J5HtGia;wndZoD9IG|ltiLcgcTim!9O%M3J z8tQSm=KuE2(Q>(wBiT(RHQX2s%e^9S%S7nA7J3tD3O=bzlU~e%iXGv7R^buK`8nyg zaIi=>8#0i4>=pM=9NT9!PhGbv`{%%?D%0cb_lBnD*Ka6olY*P#FT1Wxgp-mW{Vn+& zpj+Kh{ZU(t^kMJ%vu5h-@GqL#+`;8OSnz{`7KVjXGFy3Bi3XE)P+g#TPg@jh=oN&qX4{R#VKxi-^2z_& zzYDcH7EYNzdA5$mE!(h`eP1E=d1GuWKk+$|V21P#_1yk-JJd`h$!rX*@I>tT<@n)o z(53`d?5-C(i1efA`C0&nb|XYrZkJd%JJN!Mgb9yR^I?;zDU2sa1aB{lzx7K*Yyonm z2#f2|e(>%+siF&&aQChGI)4h6ixfu4`YI!ywe=tUVxOwVM9xCauS`?OHCKa&9+u3&e81-||v<)g_Io%13J9|ynl3ma0c zx~3>WWrnFTZ3*IvAt=`S>e9J$Gueft+(a6AoQ6{0{)_KRHeYeu*&fQ$aA3`OWsl+I z6dea3qq*CetW(Twqa3lUVyd$^cazC$alPEnkD*uQU%s;=;O6ff7HW|}X0EF!Xbg-@ zIg)_Ma|2lN)c1(wA66`5Y+7F(>Pmk3`zm`M&x;4TgTbchil*eAa{tQ$c(68sJeE=0 zr<{ugGB*uueSh}I%K41cMM-u-{=9SXV44tIn}Be=Xa@aeRRmHP6yn|tMHarR2+a)m zf|9cCJnPWzh;W8kJm|+8K-BWu&#p-ltEzN2rMltp;cPI%4esG_xv9kU!9JiC_r{=C zRk3sJdjzWtVK5X*r_Ir+&DLIzBaW}0{AdXgSz*P5x-j)0qfZp;VmNjPe%P%siBJpE zi1~2DUe;+PbqDwJzy5(s&2Akxm%z@dl9)!uWSdKOA%KxEzhqeX=|Azac!XXf#*f%}3k*z}JQEi~3(CirMv*%WR zfDTv2chj@Wqg|opx+jB9pUj`w*KnlbT(!0){0UL5$#^1mP6y26I)~jQsikj`Au1vc z^il>I_TN>0;pQAD`<+Rnl<;5#g#ChBo5H0EQIW%!6J}!xySE%JgZ@8I(qBX_^_=a1 zl&^i~#%W;-tOZ!tZ$H=+8Dw{hOeEDI-Tq%~39pv{T7hXxm~NV3}btk<&ykJ*B8hvA>Xt8tc3 z8+kt_H$24D6>Yvg@~|`)Vxf6ns?vu2BGTAe7WD~OI#N}*%F7RkfWcG|i6?=ZQH-aU ziTrWgi7rt@R+yN*=4E#9CP7482$H3TJ@U4yO*zQss(;{r$y^4e-s?^rsd9Rr>7ZDi zanTpU;XpkS58HA^+|<^I2mA@t*_{Xia zH<=cWgfG7!iTOwgja#1V$=@|&q!@G}g3l@W&4_oGnU%|Jqn~=N_;*Ce)qTfTKx*Dc z3+PaKSXMLjhE4#{Je;@VyW{FJ455EY+}8_;wM^?A-2{2fXE6&#YxAX=FH%;*Y&iS% zP!jN)eU7;Tqf@C4uUXa8Q;Ee2#v+HVVWiRu3})S$tF^c;dq%_2W;>jU!=Vfx4`Z4y zNm2)O!s;KrCO#4gBGaW9XXoe37h_`t(rlGjEkr&K8$q z82S>i>JCn!znXiSWA{?URh$l+%It4n_Jm#zIO;m8yi#Mp5l#-t9Bx%T_i;EdlDQ@; zTy=X8q!jx|A}Re72wruc`rSN3A4gSQrcYnZpJOI#m=f0% zAG9&_NUORN4U7~-1jE-RZ@&Ee84YsY@wk-yAFHtuo4Av6o#e8l=2v0Js>lQNG)ylH zwX2D+1gieu(?y(i+=(M+sv`ZQ%ny|u$4;us$S4CtoA+L?kJT7>>2OL%N(UZ8vJ!XO zz)LcKM|jtjj*~U3Ou*(yB#u)|WGNaV%@e7F{W53aS$>y{S^cO7Em9L&QCyAS@ z?pRlx=DY)csG}pDv^Db0=n*UeKhs~TRr>KgFWCn?%AF+K>$UoG9pPunT_K@+Bpt}^%zy=z||tl&AH7GW^i zmkv!p<>yfo=~`giK5Ar}!TV)I~Ji1TH9O2|SY3{4B`MmrxT%Kt2h9}16Y0_naR9KPhLh#9o@c>952-5d&>E=3n zA*lREYJ9T?P-oC(ZjY;QX3Y?M^!y4X>HdjiJLz+PByl=v2EM7ZQHiZjcr>S z+sVe}lVoE(;pP3+sp{&kuK$3kshR2iPWK1B)J_);508r>PGv{glj!X(^iwDJqnFGQ z86vjt{F{U7ELRfWS}@^AD-mHZfizv0W`7@LoxHL8obb&~ zaR2LZ3R&4wnqFP#1VI`(^*pNcMhwA5`@~45lqOq;O!_mUoX2hQk?VAP5z`RJRX1{1 zY^(_6OdERtnQS@C3hQ}$eJLMyr)QW{@sJwagk_#hE3j==mTHBeetRQIa^JqUmW8{` z=0LCZBuXVMV)!wb5WT`XZ)_|zzsccY?0>3_h6crz^LTfn@ya=g?uV`SQ3fh-2Rvc8 z`{i|TOg0`V*Lm#jGbqP+YqiT64Mk)2)!tbEo|R<^qtN3^Ht)qK-k5o@SZ3mODUpC| z3q>2(*_(ztkkV)Vcd>aL!s?cSTBVzts#KqMc7igCA|D`|78Vnui(a6=F_FqI;CQ!5 z2z+Vp)HJvn8J1hG2EiLkaNc+_&~9`6<*Wsg@d!#nA(tZ$&veZtCM5DpT9R>pW2wbf z58Wz2$RqaLXI1SI2D~;~V!p~t)@C_^DxiO;fnW;2MMa{pg97re>wZuK#m8TBJSY(FRJK4QdSp5z+ZC!x752(%}n=mCP)t(d*AhXavD*OHoA=Se+srIy zY|{gt^D0G>NR1>2g_8f22~@9HRmQ<2E5l#!$XA?=aF@~m2cguouR#BNhG?Bm`qNTz zilZltmV6KMDU-?wOCzzIN7TuUn@d)6gO(c5tq#gBCaxp}r_@r1RPdqj(y<1kZ?q@z z>Pbki0gkOA@v!=_gen%tF7Z5H1e+J*_&rTrR{Y@&&t;4~l5p_RLpQfk!OFBRc7PY4U_aS;%s`&n>rl2%zHALS-gxOW$hnhbkU%ysnqFM}skcrk3$`F6#6^7X0n+ zb)>lm?;u%ytuT%*@3Rs`DDr9-l{_U7= zMGtL~3`FkhKUuFd6u#qQ=f%rVNHg@7*aaz`YEyGie&!v}CTT|Rj5Nc;%Q8>(eCUF= z5BtnEG-p-=f|6{1-c2QvqjJ+h^gjoo(`I`nKP-Q4o*IyQ2MNq=S^T=;U*n(;8z$)w zF~7336XHTu$W(6gk31*h2kAN2gmyG9*Wu5qFS1&@6r4Rs=;aEugfpS8Qy>If%=4>i!)XiJ99_t*6B;$-PBP8fVs?;PB0+++*Q zIDOnz!F)8cU*qi8G0A3Mf!4!5uhQydF5RNwj}eLD6myjRkt^Oa0AfV}Q#QHm!ZT7o z75tmIL$#RfSpZ9-Ne5p0nLxPSeE)sXOzlYzH{MVlzkR85lSrSpD^dh=DakiSWj)&> za=b0y>yZ66A7GN%Spss8c}dRtH~dmB>KxBxVsg*sc!GOugP5RR`a!3Jc zX@S1hbhJeSxF5ihdG-9OzZRBcU|-%aq>8jw|IhBJYKu$&vx_-lSwiNhC{xz1hc4;V zn|qa{E#`%AGdc_AFv*inu(FAY2bbLb=Tc{7XLXAkv&P zX*>JkWU`pZmlhkSfN78UiwLHN8Yl(t+yg)o)bD!4Rd?-N=uQG7w*h2_nI9m{;pHT@wla(NFs*{0~& zA(SiK;KgNax|7^)q0prn!})qG%&1)@uo##pxhTX4K}1VTc_Hkwgr^_m{7xl%2pQCZ z_?8Zu&W~Q2L6Bv_Ui`j+e!rK%xPDR)E*VMB$HW#d3g{l`oXL_VJ^#xn%g0R&{MY5Z zY*42aU*y}43Rp7AEMFDZ@`t9|LC^<*o-MG2Tct?2LUxC+yZyPWFT&(UG|%pr?5DAT z{+n6_|BOmyT)p6*h@?I-cZ8Xqo%Q6x#=O6 zVcetsc$+7)k2e3hhW~NO0wbT~<>^R_X#~*6)zZIGP3kb8A3ky@bSHzUr$!ML zEQPdZl~j*{Z%!)NBqP&?ubA}1uDs;2`n6841fchz)?%F{t5|)gjb7H9IC!v%o7JPf z8>|FJkf@a0u~&6(i)$)py3Rr4R;U%W?HPl~ni?K+Zoro{yj}SD@K!`ET*B<%eAorST&9kWwTSu@W}_NVg92vn(O`dL zW~_R;0so`Do0`c^oCy6{oXOIX1oJn*z!KrfR&wY`Nld)Au)#6|e`GFq5~3wSY~puL zzVMnB&s_}q+4JuZ{0V3GkBvRs4`$zDTHm3DLZhLk24dksYlxiMiWGrp$>O!OMy`}q z@3(~=fh_o_Rf-9WjgUM8^~05on+^-eIe+CpMNuS+;BC)kO48g*?xx6j6ekJW>G2;{ z4p-G^V+X?FUO$Bsatl-KIuLrlzaNHekeo&=tw)lhdAf;ujZMZCDLyh7_nU3 z=L-TjXBH z0Te)`+(TL*3)YC3UluOQEW?fenbd|)zVT-`rODVSJh2%|O2l;8;M!Fz(2t1Zk%YqB ze|+*-sN@VFXbA|t-Viy1{W5!R+R;o!qbJy!PCe~2-g)}{>o(QVAHV~buMlx_*P_(} zun>|`IB`B=au3z#Vpkw@9$?gu1C0FHpIh*7Mg--Bz-~nTt;v|@769q8tr+1PB{u|h zuh+Zl<+4L5-%dEQ8wJ3_xu^b^MYM4I^7oymLGoY=J7e+5>NYW&4Y9f;+zugTa=6#N zeg>Z7I~>)R8V5f$o!HK5ty;5ILp_e%H!D5huwwhKSNBl;QdEFFEo!KuiEAYP; zxJ>FzGVld)(Fjmnniozexcmy?ZC5`njKuD0(c|QQ!C#7`U{VM`UMx}y_<%I?v%J!R zoD@sN4j@+W1NZ;>$yOPtOsUMk$6Q`4?+NR(h$P!9L837~KYKaCZ;?Q&w%~6N zyhZ(AKSws<52`JhM9VI~ zCndCFf!U+t2>@1A7u^iNMJz2#ga4e<_26@4Iuz~G%zL&>p_~RxGTp| zL(N!tF|(Tkg#N^flP-+vdPK#@6oUV^7Bbb!?5!B4Rx6~ktJL={_-4i(*{aETf3{4t zAGhTyFP#Qx&2>A~CC+>IjQR=Hg9TS=S&X8>ZB)aM-#e06pRa1D=T=P@C=;s7Y2bmq z#3FbA-zBQlP6&J@4J2XVw(%DKN*soj#Kxi|SK<^umD$^wJ6|wEF+iQ^?M2S%tyez@ zgP&5wejg~A9x)Yn2tM*Ge7Ptcm&9C$;49)y@tg$+c!!91Ph5&jk!C+}4C%~w0cP`_ ztcG)%oW&`(5N9s7a?{6Ai79Ws{QXLnymG;uYpWCeXMayOF)reD*=QC?vMX^Shz%3K z_m5dfgCBEtTvwdmU98S*DcSQ_BH)EB$u6T_p=jMq`=M$UmJI@Mx(K}g2kYCtFOBay ziwJi8Q05gF+nK`P0VH)U417ixJ1TfkR7fi{r@_8;gTyGx^Bcp;^UO_8!Uq z6kCaHCzsQeeSXOthC2yOsdsT=*I8TlKK{Ic>lX8|Zl8N#xb*T?JN5xnbkfk+{k9HL zfGL>4mj@orERTwZG-JO?>{i%QZC&=jz=9MZQ#UYzR{#i=Hy3+wRD-D1Or41~!}eWq zzx^S1f(eQVMD*VXXl8e!-}r=heXsqrr<2a^(ue*)NVa-2-;YBdyzPwX$T-aBx4?~z=35E=0XsIYKp~j7ql_mkB=@8X3Us<-1cyXRMc>I83nBkq-GAnt2(*I zsB-L_og-^YqLhWxm_ z=g4IF%&BDp&KMx-c~kS3OwZS5n9d-_XObj3;N~A_pZ^I7D)*C#)7gv~N`gm#wnkS@ zz*xBuv8L72APws*?@;F%E9uv33odiwCWFI4ZY`PSf`cM-csQ*kR-98_s<>DpRGyfuiC(>Xw^8vI6sY=R42vW64Az?WAyq-|s|VnUKPk&G0BBBqT%u)$bU`J+FL3 zd|WeUN~NK^10t^G9taxj!71UCSCqlc5R2Vv$-OFAT3#m;6B!qL2mMQY$@WeBns)=* zF^5OZgP-5;AUh!2;k%LKGqV$^u7_cW`O}_;=<&+AMwk>3(b31cZYes=GgtBI2E3c< z@~sKK+s`NC#A$_d9G(5rCQO;;<*Y+E%v;dpegTz{<^{kMATgPMU$f;+tuCG3M!hl@ z2PE`R`5h9*sfI=Ly=5sFG&CQTwgW%Na~@p7bvzE1TyY&P^@RJ$orz)42K&A9EChI8 z=XCLj)C&H7BrNq$Iaa-BDJdI6Z#Jhl&iM|Nz38|$JevTgdVQoGb=RbFURNi20C5t#x#JQzv0#Fu+Bz?!+x>-cEwr; zJ;>j|E9#~3XqaD@i*#MF(|`W2vA65EY)9TGR31s^zy$Yyk4gN>9tg(&M^Ms@4wnZ7 z1r7A0PheIm1RRN{-=!as55nKx{W#+Bozy`B`;L&J#ES*1WlC?iyYhw@hDmhdg(4sH zoqzlLufN<)X%G?XrRUz{ILfcSQvwq5i=GtKNt=p)D@eJ&~Dn4sQ!-ga-Z$2TD#nFR+6>-~a}oFD%4(bL^G zV+8Y=`Ly2)M6V!d`lZ(2Tbo^!5fj zxk9h|*zRRs1bZf--jX&WCf?|a_PQ0OjreP_-HtUis5BmTWRn6DP@;s&)W3tFbZ@{q zp`n{KiJo}n1-(>L&)E7%qp~ph%~$y}FXTOcn;E5B|3nMAk;v-=xd6y%KGXi;En^wf z4QA!QtxI1QWBMb9xw!=`8|URZT+KVb*8C3X%sajHl*l0mYb_r-Z8ryTciC%x1nN(Z z-h`U~hfp9Q0ytUzNEb(1>q9Z9v7A0ZrCW0=fsw&jJp$_ZGVb2@RlR$6PzfNqGq~j# z)?JKM?BX;ZyTT%bw;I06a9DzV5?+|JBajq;s?2VU7T#P2_>{dWZ&b!G-xF%wD%^`0 z9rU|2+x5RGn)u_KO8mPxV8XY$D|L8Q(kp{|E|Zc=VH)?O`IS?#I+|g0e{pB-^DR9< zmVB!}VewlYVy(Q;;O`^$?f59&;oU6MZoJi0x^1p_0bN2B;mpqaaM%r|cxa6{#;>0Q zS?>Rm+R^38_~KLDKe2a!|Nb6zrv=<;hxg1y_gqfz;qV@*5!-7O#AL`9#@d@rr#=05 zP-3*@nLAgj6yOB{Y~!d+PQc@xlYU`#0%&3lma}GZ+g+IDsVuCb_mdJD{NIQm#1#NJ z4edbim6IY5y`AvG%$yF0|G@=VPay*SO32v@782zsA+G=RW;ghGDcoWTrG0*Elfvo? zxpeeY!Eim4*;z9v^%Jzsg%gcL#_3mp)EhXEnCjnj>BSS!zy) znY@!(m1t298z{N_;KO6L<@SyQ*R|<|YWm|KRrHP5t_0K92ZHe%=2}FZhziKGq>2B4 zt{-_DNcFgd?t~ z5s5h9_EG@+?<;I+DllXfr}QP_U$3Jxq z&fjySFHF+MtQ+mB82*Cxg=lY>?R4Gq=1h?h(BSQ#&UyIQ%o|@o=woF{#{ML-Wa)$1 zRqr1z?nRXfXt{h?q^DrIigM*OL;V8wAn?iLn(Tjq1+EMZ4OK0EaWR}j2s(Rm`!RH_#OY?zBVDrQSNqYIv@*|2x@8eUr!`8SU` zoDR7MhnED{4nCfOQ4M~Ve$nc1#Sy20nU!`Mo2=Y09!)&H>~-UW@}UkixsZK)PAn5J zCe3+gX+7Re`F4@-X6D}Q=o zer7~)@fhhcnxLFmbN7v&^iOTapORFT*z`B(`(YJl;qI0c)|R$_`);iYCBl7zIBq}U z^Frb9R%*9V)*6V7X``rxy z9w%G5&d<{TTR#sGoLh(!3S&inFDNo$VcvDjC5n9GXh7JZL#_d2GhvYudJDk8WS8n! z_IAwHH9qaX3*&3xFKcP+fJ1w3fn_2tCLbb`H-zO93Z%5&Kv{|nrM4N(kv67KdI*5ILm|RsGvcgm;3f z>Am=g*bkXbd(Z;eM`g64S!$xJPMU29Ao{U@WDAZ~#9Uq*%UdxcKOE|^PZ#sGuRfH@ z1NNxHD%Xt3i&3K~8P_=IK|86-ku!v~dQy^NeOBsX3Z!7wxyJjzw%VJwVW0k0xwV|P*W3|z5VIxI%%6(ez6Jq^yp0MNUw&wc; zFJFFVm<(Qr(|B{>$(|Slx4ibu(|BvFKS_b;7!&lhIq&m7-t;j1W5GMCC()~eIlAiw z`cu`pY6-Oxe51$sw3_$E#A0gMw9e~)4w(UWL~qH#YnYuW#{n~Yt@2zJN3a4L{x84o z5o|3kN>lK9d{g7P1NI)rstuZ(1XX=w6Q15@o;rvSDDV&KMeeb-i>U}SQd!#fw+mSC zHrN9<{&qd=xRak^#WmaJZV>xG^lxPfbt^Ojy4^czUTYI>)dsE15i6Zqg<<=70Iwq) z(`%hk@TUn)0$h=P8Sac9t=^q!`gf=CLWe-3Id}Kre8ab>T`YkXbpKf_9yZ8-wpbhM zj>~7R^UOq~yl)P~jq5z6N8aO89ER9S)X!Wp64MeKhF3~t5{85&92ACH{csnJa+(P> zf0)1Yj(nsY%My=b9wwouY5YgLP4WN6?V6mggC-d;B;6}e5mm5n#{=qTG%@=l8Y4>&{~mbLV9erk%w~gb0m+Ht2tg1B`)g ztj^uht5ZEkW}&4xcMP~LraOV13avd1BD`OC$$m8P`5`}j z+dpGFEx292lRe{vQ5#BVI%C*I0`Mc`4xH^lEzfqI%@mrTLZk2X%$w zc!GQh)lZn)hXABEjzaqFQ0-<^=Vo4?h{qNs@5@8#tjFOWXkUIFTDHX;aq*giW=v5M zUn|2fR& zvCL@ox3SmrrCy)uq-4E?FPrYfn_t3Mk5-wn0ek}bSg+Ix-Wuqq5ZEoQ0cEuL21zHt z_qim?U!;^N4Kmc(Whkh>rEaisr8FQ{!elp*-7=dCVR91M4R_a1YIO8+h;ye`N2~Mf zd0Fb{WuDk992{ZY39Kdwr>Yj8CONdquSou6Ab#KWR3!!;wEgU;oSRpFwT}H(wF&`} zsy*g7lnZXtiKTm2Z8r{cWkSy9-Qo~>?`MUiUQC!qhVDW&gv+arv!9xT7mNX7-d6$n z(wFVQ?afSTB+jdO}v7P|EBh^iA25iTTk}Er&(E(Y-;S*4Gq%C0pDa{pK|Rd zqj#-u&U=CJ53XT^pcM!RSL$cH%!y?X!zSO1W_Oc6=NPtF*J+ysTE1CG%fVTG&hJ|F z`rh-^K1!b^O^bEKVQ0q+)0%0sSt(XEALl3m`m9cG+Zg2vUySdH`Gi&0w?;DiR^X3a z5RUnLSpWg^L^VywLD2{WoTg(*5pg&AqnBmlCFpa$Ly6vADcUc(D3R88&myrM!{Qf* z?-_Y_wvMU?9a$V9Ir%R)grkWKvZxWW^T$guolTrFKc7qQT}HP3P7dxr(M~-)_7FUO z>kX~zZ*5WVJzFhIC@#9QCJ~?Ziv`ph`OitA7zj_Vn{6_ATh`ftnLIr+$u}mK3cQnL zy#kC63so8ctM~#JLJ~da$!nUYoJZ&_M|`q`?I3`t9bUcN;|xZu@7PMk;^Bv%M4QR; zAnX0ptZl}Ne+r#2-mW{N8u2$5XYT^)ANR75KKk`hk}SDBAzRpjg2c_zCfFKAa`>fA zmbRPiZ<(niG9xy^?PfwmuE4Rfxl)fHpy%Rw3YYJyL4(&jlq??-6gm%+>)-0FrL`CU zE<-U4;(1$t{)yGSSO1tfiWu1y&I<FH_q>VhmI>4wvN`V)W&iGa!nH-4iO25eG*{gZoXKMuc>n`-`~G@RuHkICFVKn8 ziR!0$zM$w?*zYv=t1WhGge9moO{NpDGJN?`;RyYAhfBE8a{iHv=3i4!Ygqg7mTnXz zPB2di?m?Z*MLrC&0{&rVJm6#gy*5kRiTYnC_c89NQoDu%NsxAVW2^@%qS^tM{Sv2s zReU`u|rNV~ZcJ7T1(B3)Ck zcB_S#Ul|W)r4r@}B_4glTXI6c6WC>>AkQ+)GJpZ`YhSpDgZ2XR*vQd2xrT^$0sTgL zR>cGa-92ig#O}bm8`N4OoY^I9%bF~74ZL3du zKW4d9psdk^s0A@z#QmdBTbJLP5>o)2we|%@n9nL&xGv^h61|1+xsWVV5wS1mUGf*F zJN26fpIs`!TZgoHE|Xy6u7}15d)MMw44E!U3UmO$`-#OUAO2i)!tPNFBTFmcnf$I# zZcB4c)DMQZkEvv!10kVx{ok{*{Aq{k?T$AP)I3nz>|wj13&uzEIDx*d@ds3D^egzi znIyLt(tor={;kE_%rCbi&9t3#i4Og@#&bEy(|l>$eUuYVHpc91+-dm~2cL<_h2LuiV_hUFX? zx0xTo!Y4HP$gWJCF3v?G+dQ|+Mhd&;?USVX7k&vWs@$he7IOXy3EvTI3W7EFaZt3v z^4_U~pJLk>%V~(^poto75C-quy34zSgcqPu0omJZ^ti8j?G9A5fMV|9M(B+#y1u^+ zIls#2j-7Z=CVrE2pswrx$uSTX`_|8q-oLF$BQ$-X;!F1y9 zWNg521h)zcjGF69c(t!Al(sVU=95*6!>L$Y6t*a3;6z-v_5|sr?5|-ENt9L{p9LbO zcn^uvL|gWy^W#alT)m7y0%F;X>H}th>GbHt^$chNy_D5%a%R&3_ze`w?}ULmb!Au% zK!ZAEuvE1^6>+I|jV}rAY=fEs!~e}|OtWI|JA?>(DAN(>zfw!|t%oUG69PZL zJpBEAIeDI{c|EWHy~L;t1FHB@(DUDdxda4)ySxaRBLYM;T_Mw@TW{gI9pk30j}XlL zn$lsj{I{6S@F8ci1TXpcfNa7mTIlQ7{a+3*%2Le>#68X7<*Ah|X; zK=;N;Q2t|#GnpeQS@>ZcEm1j&;pmJ&3F!3tP%E-FY=QMS1F+?)LHCsRO8et$m3ANT zV)G&26zDhBYw6~T_U!RsVI-;hq}a_D+$-~|u1vsdTKb|crL$O0t%UDXae|bXHMQiY z)EXO)6f$urL|R!)#>l!rfW=4n10xNLA}+8; z?F7P?pBS5a0(WqigyV85QW3lwi{iP0+wP2S{UN+b>~KO|tC(aY$jQ%zm(#=Vj&EC1 zVcz&jIVFX%yD5AU$Ff<;VO>`6=TE_qk**u3*%HAE!4S%a8-Xzt{8Nhou0UUe$_Pb& z&LA^Kj}n2h4)xRbE!s!D7!hm%7^X+LX&RjZ`%Y-YT`&RvND~`vs1a&>am1$f{$a06Jb5u}P#W|1im!_|0VHMn5cUK$Z>RJ7cCX>cY8CDI5 z&gh18d>G1jF@5z?H9W;gQwP&Az)V07QymEHpC9K;9RABubtmiq-#?SVnQAWfGWjCy zxW1e5=|^{o;}#Ax<#LtBlN{A)X2S3FH$pn*u(PC!RMa%-sN8UsHF?$onLycbs5HE^ z$y|h5>(6I1^RsQWbVM6D#@9*Ur2m8e!eG{f1<+F+<7b192Kk9MJb+^@VciEsvf_M7 zMj?yyRgCiR)Ko$E-v)@s;|=K_nC<#W7p7b~9p@u1x+QBHwT<>PEwn2^kv&4np<`Nv zE7?=eU`SI(*HbpGv!!h#lk2Ljm<#z_l&ud{x~q<AVL#K+Ni_WY@C{ zQBxo_G3aAz2K^LTb;i7sekWKf^#Cr)dcz8v#e|ePsv8gFVtMm$7z9=3qxHRR5NHYc-2ov*gRt{qvnOSzSQg^=x@GB^s61HB_OzI~>okL~?)e-fTuKIWUG-{T=zrxgti$sC4 zz9%CDL2WdUdG^7KwSPSb^$yLUJs5^9VjBIkPp_L`=v8O=wP9EmGl7}iYQd<@jUPoW z!F$}SEpfO?67hoG9n_P*DMc^36PplBc2$PQmHm4k#L?`uMA%GT9{QNQ>(Kr|w9|=l@T4q%Ne9wL`SbD!G8PlgB}H&-_4G?yE2IW`ZDn z&5)F@rR?spoo3^Qso2Op$b#{h0bg-qE*TBAc$h$3%Ngd~*rGn(F3Mj(c6UQCZg=s^ z7hGEyvIEx9`dg@xDO3*thM6)xM7=!h;0%D|3ARJ$hFjYZ%Ed?sgG8tzd7i zx0#Lj?TTN7BKEwfV^PABtO2;kHGa}$F6)I9Z3}6mVvV7Bz6ZD*g@t${(R`&mNy0B# zH^?|ov2cGKQnR)RG2Ew!dh5b((r-}=tm#^biEY#W=cJ@TY}M8x@Z+rjz&&3z@8wB8d3TOTEa%Efba70wg)U%#`gxO}IN zc4Mhxa_i=Z`1|ch>P2GHW|4GqG*$XTp{rjlivHhr#Q`kfmfHRb=~(^cZ_)c*r1jO7 zI(oc+zN{+o%9D8ds@wIVlD-pt-QCG7+Zur~7%@zXERRIGuHkUWme6;n)@cfTg}a&= zwLN1=Q3U8NIO^8WBWtd{i_*y$-cefd>SzuVYqou9zb!IS$BA*rEG*=U$>q6x@EHwx zEfY<42qY9|h%vJ2=YH^`&s(46({{cCpw|!OKduP*eI6=J2(u~h@L=elR7`}-R*4=` z&^TT%!u>UHv>X1e3k4fmPg1lBv~?3zCTg1OjFGpSck7)Mtm-fdEG4D70ho6F2hXD{ zZVpmQ&^WvYA}b=dm(tUpGM7)L!4hhFLF5a+IrYZ}i!b_|$XZ;eI?dcP)RI#MhIE`i zUu>!|^DyF;kCu&nhh`_7^t^1#WZxlp?DXBP*p%aVI%>NHlVEIM<4}JJ$8n8~s78gS zy(5!$*zyXwEA?nEVs#fdjoYA>B>t}1-@mZ?mH-+W-qioed|{B_Wq5#YT0M%M@&qR*f?l7thW+}f2`oiN_N=6&&Xg_XO)&^mq;IHs z{&AAq!*0B2n-S$~m36e`W&TY>q~&N+yJY-bFCrIf?moQEkqk*5tGT-<@I4^6I+YSBNcY**hn1=cRmjZcUpMqJfkV^~(Cf+7cZAm;- zTkt(MuZ1 zH1xNF(-j3KsqtC;6se0^Zp2{4jz2!Tp}`KuZb4w%7xmotK6WVn{U2$=?5_|R?cK#u zcJIXw>h$@vgz#uV$Dt_C9k{OU3#cnV2H;0p$us^cH!Dp1Q!hZyY;2{sOckhe9cW)fzpbiU>-Zziy~}Cp zPpZ(<1CHHDHEOT$jg19Bd@~t@vPjgtej{SS6B`0?;XLmdjA1`Eq`_M*W?>PfE`dhG zP{z_PT5S|aE!G4=)hA4~N`z(a9|%Li zf;t)9+L?^olQLkj0jyv8(tdu!7^@1t=H|&{ytj90L|^_iY(X!a==HB|(1~qno)F!K zFG}*8pE6VF_M-GOdre*%MdU(7;T39N#Q%DqI);i8tBADKf|`j}5wdL%8o&q$*1IF` z#LI-r=Pwh8*jf)XF;zWxqB7>&XvMhJAv*Q+*4%scT(#Jd2=?>@wSHJuR>ZIu%=p6r zcPKj~3jF)J&$x!(y#YzUX}~4xvueZ;m&DX08Ai1f(T z0)Hy~wG&K9^9Az3pHGLiQ6*LZHcGtf5UwJB=1TL|qkkh(kSFxG5|K)%LP%>|jmC^n zNoR4i|9=XZF3kQ9?a2aAsG2|zgx0XWoV8sGt^r}F`eypGLuV~Bx<^RD2xJQqU2KhZzq1qUi$66On_Hgve zd9eksjC7x;lIp6Gdy{^HhT{%pX~iy|KmOT00+L-G8}UOz5mJafK${&^w04$OyPg(3 zC-L@0nr3LqK{GJAE_YkQk1Y87 zw#{fqb+k76{m=g4*;Xt%t}qcw6FBls(*HB|V;-$-u3z>6G$5~YPJGX=!9n+Dl$igw zW_n>G*qJ*JC#GpDO|u?-uGK1xXjBLT~M1Cl9?M?1TdxjpZuJZ4i& zcLl37hDAuBos37#8r3=Lzd#@A`PyE?xh+qGQLYT}Du7%Bq}KGcM|`P9w^{Hg*ahR0 z5JQ#R%#pBDmwq9sJ3U!@&6PSwHf!koqt>iM$^W>g<1M!!h&1HY~Q%t-48={ z|JJ4M#{JSAe*f6Z&*!Yj^D#VbwUe#PN+oVStApyZ6!Djn_o7mLl1p7@dm_8O5PgBzLJL06#S1ir1m3+PzI^*FK3`V5C69*ZGl*nc zb?Vm6IO%j+l7pqX?;VB8Tt1|rfX581WtQ;%1K$ebe8byl7Rr4Zpxexk(E!G*=QJu? z1P+y6I}Ur65S8{?&R{OLE%{UX5YO8)5(GV2i90)avx`TOW$9t6U=`3a)&qD1NggRi zwX(cTDt|qU&?2&?x4DnCKAdr=Yd!OUHcQp7%}QD>4&Ev~?TJ{c`n#bobH?p!34a|- z(cq!nKagrt+P#7ObDfSK8SdC_OH5cFBxS3~N>8J~4okrr7280xG7WI+P_Wx^noMo- zBDjx>4Jjx(bUIFqF21NOG6J>{+ySjrxqYN=Y_lDg%(sB(^;67LyTU!k5biX?d*|ix z9x{iyJjF%~C)%Sh!xq(m5)i(mk`Tk6f||mLf5T47xOza3UsN(j;X$nkpXeZHvuLHi zsqkaA;=op#lL11brWqmlBqTj=V{5#qMUNPl9F{c6m#^PWdB2R5kZxR)pEx#NhzaV48V>wvbhozY>U%FI)x!pF`ZGFTRsA z1j+R0UXSVXFUJ0vDy>KxnypI><-sWej`1fupga*XjL1)IjNOM|jdaqZ#z`GUXJSC*6Rgv>e3?^H$bcpr?83eIjOEJ%O)J32h zCCB1GD~_-{{REA=os|Tv7*0gs-VgF|U9lMmI%cj{VL?kxLM9+>`iF+%_d33kD?X^} zWr+v9aJadv4-S4(N6We-7z68Ejp<1E)8a^N6NnjY z(w=Pq;AB!gJG%)c#y=Y&!O~;_w7aQw69(rqo+q@o`_Dng*kgoRvPN=j;7 zQpv&~RFW;33gH4~olO@eO@)nBNj1PeOB$_pAe-PnscEy{)l%0M=c&GZrdbPn#&{M3%gOeP1uVwhfpRU!k8+-{6oxnSp?z!yR%#$4_ zgH?uN@NR8F$)q%kANZj&W_Wd5EL!gPkSy7*Fg4Fj7ha6K_nge(XAm*9q#QZ>R|5E0 zNDzT@A|fo&LOKxraG(r`uO|9GdPAi92eg&s)?6*1`5v(rEMD zE20-H_xZ*JwCNh}1*Z2Lw5HfKuGbm!%~e(USUq1z`gf2v-ViNgzmUzW51j$ok9tmi z(PA}+d`)AG)TF6tb)5YeP%@2t+TO>j&^$&B(kX9vV;j71EE`NHn zuK)-&)!7VX(mR;+F9TRm`=^gj!!`g%%2AlvOR@gck~{7Bk0>OJ27{yJfgw4=jQ)lG z$@{WH*6pPw|8Lf~bZq3`x%cq6t9qnTofBpLd`>_k@NIFq(ln~Di!qjdCdz1d?FkLd z4Hw3%L&$lnSKI$a-dn{+7BpRgrZzJ(Gc$9W-DYNnHZwCbGq%}oGqc@hW@dJosm!?k z{+*F#e`%zZcBOsTe5i-2tQ%QbdE;h8o)bs#)*yjg@ht-A)H z!2ep`XKM(febUh4|^S9=s6K)9!-fUUkqmI@gjfdN5KSg z%qQidby-3$cDSpBam!;Bb)%n1jUe>zd+g#E>J+o&M0b5q4g|CDGR^)@GFQv+dN~+H zWyN3xVj?Zcw1un2gb_r&ala~q+JDUVdNFLg!sBu&y`{wVa6LNU;yrQvK>Om~_Wz>t z6;YjTMDyH!&+%ZG^;>A7LK>b3;*ZIScY8ovf1Qqo+b|6L{ZpeEI&VuI<~L*fc`Me^ zTUl-`VG{W}b`gkBJXejJ6R_%#!qIDC$6fKb2o(V>tiJ+NFULAw<-2w^fj(H~`DdsM zByNcX-UhalvQT&I)H*Ogv5c^=6|U=%ws1rJFvlmKNE}t@={!$}_ACoy)HlDcrLZ}E z*CU;Y%>SLB|E5N&SAJl(xPQtpg{P?g)AK9LqsFgwut>ina@Cd}@xyfSysun`%IK4A z*G@~72i|klGGs3|aX4mV*C=Gsba51pTBEUhCRw9ueLfD8PzE0}P1lE{jfAt}hehj- zzxu@~8?`dhAmr8PxlLP2+GjnM=P^hNyXxqsgvzH$t*)pn9}PavS}17Zz;pSV#`}Fc zp0WgKA{Ju(i?Sa;a%*T!4Wcpi0>ekGG71C50*hE{Zzq!EzWqwF7s(fUZaW~k5ZRL_ z^d;PQ&npe7jQsX3oy7y;X+K-5Ct?g!o~eC$cq*Y-nk;FOGsNhh_WqD*$5&+fdF- z255K;Q0-x+f4phi$#KIEsLd`h22*rw#4(CwHHQ@OVA=@e> z6Cl%?=*&@53`i0nbJKwR_;g>k1A_G!D@`(DAJmXZ=FMvPLWn==CDDM1^F6UD7Tjnv zFE*Zf>tm{n(iFcJNB>1Fah6vXd@ovacvS-;Wjf;xl=_ZMK)?w(*iDo>?z z_mVUL6JtD6++?kT1bX8yHXu=G=M<8T%KtO1UBUn7d4>NhuRu^^W3Lf#m^exG?XVKb zw0pXOQ%|BRwCPB8S$}@#X>o4OlNDLtzdaDwbN_f?`KKfxQG|XcRf{SNb)gY7zuHUK z;!EIn1V!R8(;zZmTul5IW0p0nzEop6X=wZ)|Gwhh`rlQ;gEP!>qQ@aG%85oFqya`b zDd^ncgBvxTl)d138)`%8;Sr2ExZ90kXy^XYa!KX59HjY3z3{6Hjkm84=k{ z47K-Fwqv+~BEjv;qo;idrD>C4FKoAX@#kjnv=Wvd=`=D1hs#1*bA`?<;<9!{03lvp zUT8>drt-0X^MZE^59PTOeMZtjnV!kD-LXD5kzahdW}e#Y^G3fB!o&=U#w20fXn3gG zc;`GNC?H=(=8D82$_OPGZ$U1`vG|4+>Q4rsgq4O#B)EEb6hXhn0H>|g>0DoTsVL>M zC8WR}min&+Ly4LRGw0fjL~9|7y~yZ#J}^jq9F@%cZFaWXU`6#1OqT)OLeZTy`W(eb z-)_FUrG`35QcglyF@vo4h6w3v0}IgweR`y2fr6qUpe4}l?dc8+3|?F>QBo2x*FUIo zDOfD=DqgewON3r2R08}}naU69yKc!p9S+_hC=M77c9gv%%7`UV#e=HyvD<_iM|Bs5 zV8t<gmeIWYf&t0nHBD%73& z46DV=)IkqrVtX-;kI`@BuBs@Ou8CNx%H$m#LC-X&;c8X;H{s~%M-vU3+JJTcZ5MrI>MRkK_I z|Jh#<-z&HB@_LeZDSf84o^Clom+;GtN+*WkZ}y)&mDQ0Q!>rPKRMr2P%!twL5embQ zMfn9gUhbxuHf;6wtir%DZ5c`|Szz;tFoE=D!)FaNDjF_``9PNy^A1S}TD&4`zdH8V|JvQAl;TM3K`N&eR*(pB zyP|X~Cn0EAuXoHTz6ZzfVG=z(^n;;2Ht*}?1&mzNavC4Wg>sFEg(ZOV0dpTeefjA;DW<+k;W! zaDn0ORs{#nEo{cYBz`u=^w;H#A7&?HW>vX5STCHBVhEC2(2;Rxy_u|#P(4fO=w@5S zB#~Z`rpGEGumg8WDBpK9piWS=9b3o7trb z%9Xq-bO`>pE!Q|=S-Eu|{M!Nj?;9Yn$lmAd`}hBE8vor1UjDaJ;8mb4gffobmRcep zY@r;2mJE^iO~`ZhUyb^&FCK=fW*(DC%vO8CgoIbi%0rVjZ=~->Gfrb)Y_*u~KaiB1 zzD5;ztk--Gz6fYag6q-aPV=du(n}|>9<3&tco3r{iPh4tFwkU(<1%6#6@P{}Da=wD_z9+VAsbkQ89gds z<9cCMSc!}Ju^8xjU>|h1-AYXhc@n1xXJG`Kz9asE!ZN~N0_)QR$=na?c5P8XRJ?G? zh6*fzv`4(0tmM!o+!l>%Oti>qSE4IB6-)3aO+_N=vJo5Kn+X+lI*k3?X0M>cRM;T; z4dH|P#^Ps-;Fec0^@+xz)!v0)O>{ftlMY$(a=xQId3vpS!qcZYmW(oCRd?=V8yC80 zE-?bt1p2i)yjm2zdpUHCmB06#I+-85z)yX#ES@+N-|exvugHAU`;zJFzw3eUX&*y{ zaWyf*K5Ck5e{W0SN3<_7#sc;gHo%~m;@Mz zMCi-qCFlf-B(uI8n|L101ljYM59%G8S{={qiC1mz=b^H!?AD&Dt6tDgTHX8heZjp@yU#~@^Cd$$*B&+>7H%VVxk!B@BVML1+rW<7M>_u8k0 zEK3}lX&5dpCOHTioWSpH_+rRnRfxgC^QoU`b#0Qdj$N`~R8>E;iPd`RgE26(g&a3C z@H)qmko%_zPmE40gghjUjHQKz6p1kC;d(W^)GjeOU*X8%a3-w(FrVeuD_HZT#^_4Q z+tFPi^+>4PAqtSt-uNBh9MXMI55F1p7X%wcl40mX2+lcQuKdgM*GOjpVpsr;l$p~r zEny4(1YU2EWjjk7S3h$Zkui?}1QiYkJvN6S!E)w?QZk%FINim&%J=^iEBRT>5ETrX zcNuZJA|mDeWW1GVmmR|ZtuM6zFt$7iV6A{k^W1x^r=W9NUK8XIzMZg*LPEWMH(-!> zk(|HXl42YqR(~+r!xTd8;hB+)uI4-{Qs%Jl$={pau}$K^dUIf1+eo?83sob_X}7Rl z&l*oT?wjm2_TP$1zI+s;-s`*BdG|QhBtM`ccOIbKzhpuo&>G}~MXmi(3}xDJ{Sy)m zf|+@;%l=5;J$C<6F}R+Q#r>Ln^TcdeWAzU@Gy_cx*HLVtZ6P64Nz{l)S$JA;ptM#6IkG9K5`&ZRoQ>Jc(K0lp?=Z4 zg3hCOeo)l1x<)M0+udA-uEZoDsawW2 zMgE8aoMLR5>Q0Lc{Z@^JhEJvWf&!mDYk3VgFuB=2QM)a&i8h;~@WjnJZOji6kc%r) zUtU)lLb`~?&nka(e@UyJuw@c;;H<4ypdD~ZZdEq!_OX3P+`No1lxryyADe=}g~giP z?mZi~bCwU=8MzD`a7S-<@z~Z791oC+_6)xg>LCLdjyKxTpntEYtC7TyuBXMt#mSS* zeFySU$UiER!N>f+#!g^Xq}*$V&wqy9oEit#S%fvP-5Q+~qjuk*t>-YtKlE$O6NbUY zvj#4(e7OsP^q>D^Muf0V&sRzUp}<)(9$Jp?@LUYcE9#!h!Igus_P+PVOuyQ#2s0t1MHM@d^NzIZO}r& zDA#Im_RcjV6t3w_h3$LU%k!SL%P~qxjHjq8(RKi>QmGmBLhcwg%gf6*sjbBZ$BR19 zT}*~JcrQ>KZS}*1!v9So?p?^JMs$2W+sI)4Fy@i`%dNBe=#8!uFF4tQc0+#<@OCQY z6(>)=W%FmXWYi>dN`&L(3s`+oT(e9KvCE+?nW zlrVo-1QJ&BSFkfBO~jwcZAe5dxvZjNDqtG8OD&TKcmS2)uZ1p4*~&m9xK9x$rzjr1rKDhRd5-Fn>okzuFM_*PM(m z-Suw|FL3g?{BBExDQ&mi&*bG`S0Bt|y6$O`br`WKx{M(=7eCUA2I65JBPF}yPifx~ zU1o15{tsGWH3@CBfPwN3cOk$m`oS1TjGoMv@+!e?G7PtkPVDI-;xDS@{kj{t;&fO= z6{&mScL(a>W`85zdW?m~9cGv6*n};{z2*3vpUSr7!F$)a8ENjq4%FdIJ|oph*?f-J zDoiTFML&j3HXNUezw;$o$o`tazfBY%#vnVJPUv4qAKxIud%28x+d;#7B}h5$x(xuS zxybnA#0*=0vH*}6fpODrY|(S|h_o$DfbM7sS}Ziex?u(fT=>8SbMAJ~?4+041$ysS~D7`^e|PrucZKE*#dkh4CJ>bqiFWf%?$%MYOL;yghp5RVjD z=Vx1>c?jNK+v40gFdi;=Qv7GkKXahNTX)1`^kZG$K{q(NmG;iW7Y`(rFhAl-HrP|{ zY?)nZTr>tdX~LQ(m_IrR7BU&QZN8bp{%Th}nH3!BKVyd*+k`PVl)dgR zV*Ith1}i9;_G2jpvoVgQNhR&q5q)0kG0@SkPrKGliOQHT-F`csNN2U}Jm&OfjTr~_ zW9W0P`+}D0c!wxR8PBpt`+TwFW5kH0M9Nw`?j}0$T2?KY`6$RUmj{d2}4)!kHC^N~tW%~Q0Q=40hST^SuQC<=* zbX7GHR_fe8(dMXjOn>1}5yI6?^@Zi!@@l5`WEC1+RenISb8TnWc5vT)X2?|Wzq5x2 zSv6jZF)UXz$_;EhnZ~xjZ4>%*w^MdbFCAOXF4*iSMb@r!Z%=u1oPmQ;;Jw^Dyj$4_ zX415J`*ZkR?UipCXt&v}@EXwaaL;-k*>+m7)wT|7X)f(D^$lb$8>E{Ihn?B*JhDRt zTRHEl%K2T}HM~r;W?3HZuenU;OPOz6uVi%Sl`MN-sirkO0CSZ~NE90Vd^wio$Jtyn zUh8Ctpy+fXh@(x|S3j%q?^qbzNO8g)!>2UcNGi`M!WND7vV#iaAY*m$z6IOS~n?h-v%<6j8p19=lPQiK#V26K~3 zDZgMi+|YJ<*+Z3PPJm1%K3<0aPnQrX3KG|#zZWj7~_r_x`Z_9(& zQgM6x7ii#K{!Ym6LkQ`gyy5sBe~4X75H*~n*;>78amEvk4cA#J|6sCm!yT`yP?7Gw z!AwTd{^t~wY2Q5-1@OMzsFJ`xSorRL5FLBUEL|Z4n*}c2E!Wi1j3#!lBxCGo*Py9F=4I4DNvL)t~f97K2ad64;q z)x26FZhZE!ZR!_N9@3Hcd}Nlbh>+hv|_iJH1{WdH$?GTd~mT@Q*aF`h)Z;28=KNUbo~zP zQ1J&ML}4+fh>bQ*oL*Onf!3fohS3gc3Ob(884zE&Y`D%CciEtySPQZh71`@&r^2;} z_y->d#e>dZY0oIo{+L(=I*Q%z>XmaP14>vzj~VSHIuTpjcCdkcxQBB`sVF1%>7whI zB&=0DMFDF&s!tTT1LGxMA3K{%^qGu%p%zAjdV#8FGeR7LA6e&dnAr;*;_08)W}(2h zBbS6jqCf;yoAM7+E*IlVe?Xg6TXY7lqN!W({Y#9;$onRKO-OF7B_qJbh`1{kX9Fx1n}4dM5U{GLQ?K zt(ui4Wx7TwsqeEC5Dj9kc9=+jY! zIBwzdY!*caB?M#k**^IqF5k47+(^TfoXN%$cmoga+L2N?v)CZki_+h~43db|TQxXy zsn#M7d+;O+wxsq|SXt@Wp-kN6LWypLiqvTbIzw@$hX1U2NXm=va3r-#SI zsyR0flGy5AJqwk#RMkG=sjpPi6Qm>QZlB2f_zK(y&>i=2Rsfd!VK>X<+bg9;Av(Sf zzC*iJJBASM`}b7d)zJyOUNzr6U=csJ-S}a-FZPmVu#f%v)ezM;72r^EQv$=r`YqeU zHg&i%W90)@v?J9g&3dyuDC06wqNn4638BbtCpxCFui4&7ZnEQ0w%_CuWQI_0RExb3 zkd-i*H$WSU*RJg5gj>LB=x`X?fm;8j+-CDbED`oWw*nVy3X3dBiXFFNqk@xfnI!Y>&*|fFHa*$YMAL?nR+EYn+a0)2L_t&b}nE zKy;4$`Mq!s*qCs}0wrqslA3^&Z@|CX6g)S^XlR6lv%6+c#USS&)_5;t*uU8QHv)?m zlBHrWp5LT!I=W8>o6ssUop7&cvR>D=`TJW&u4AZ}zjLAnY;_KC49?iZsYSOsOH%9n zk-jib@!)+=o_Upp4#wkto1I@{QMI`lz(-`TcO5667M_R-kDP_doU#?*P~iR@q_h0knG2mk7KdT z)jzkzDg1!ikEQQ23tuV5ezOl$WvWeaanoqgJ+~DqhtPRMH^8CWt~1iYVf@|nCkqEw zZ*lI4-pSK9g5OgRi0_VFAb?qH>OBBVeJ6LzIdfT&*)BR&L)57BW>*S?F)K@Z$`y)C zk6e5n22AF=AX#b7%;LbII9)rN;F{@?^~j6&h9GOeK$%{Jw--b$J44_8#9XHAWtt*l zHEcJT|6@v|mu=~t?#FxtRBd5`yp01hj6RH}DjZBfI!wD_#w!Jm?s{F2E$EZK7UuhQ z!L_jeBZ6IDysQb^1D_YD$yRra?GAqL3$Xo`x#3SXmg3{i-Tdc7oegA|U@ZyJR`F#u zUH(c!N1EtM3*7QU>-04rM$AK|8MY3mm}c|q722@h42imulKb{(K>_??<;p)c6jeCg z-YDFhDJ&~~nSV6T9VEu5C8I{PDZZF}LKv<)o|kT(*0XB>j$*gZ@&zxWTz|SS#1PRE zA(nejjF%e;Rkh%05al0(w0~-Pa-4~9t;u|s;rISAWxUl+G}QUu*Tm#Q`r1uM} z(*t*~bdkX!iF+;r@o}Ld!xNuMOnka;x8y>{T%5M<>AC| z-vD{>5CxDh5gfE2ZrjiF9uRG0@@k_gk`wg=wvv~zdzLm=g#dylxsG*(=A5sy01RuUZPd#{t?u&DUs zwuBd27R-rJs8XDg$*L<#^O)VqkhT2JMNdQlkPccX|L0Ve>LG{Z+PqNc@>&``@zeis zqnt;mp%c=iA78R%oZC6?rDQuZZDC*+8zqXZ_T6uH^D=QaE7 zK*$l5U(kf8grtM%H-Fp}pdRDfMngDSr}Ov{<8Ie*eZB(uzU#%e6;Ka`nJ4Vhm?){C z)|fdQIZdObqb5xr2bi)3kfoFBud8HHA|TKkn1?3ixm@r58>&cuwsV7Ibujej9Ufmg zS=vc%hcby|xD*-r!sX@>%%}&lauqy6^~34u-n8^R^1bRRn61?4yym2rcXqPJ;>pmr z8;gto35kce#lhc5A3xQ^kUBW*@z$O-?_8QJh;>J*;d7VY9r{WGv}46Yj9l539#zE8 zh1LuRIBa3bzvg1pL&~a{<7K2 zvJHnYflra+B5$j0M%UG!G$J1h2DA5S3Tac(;!6oa)*|%QvOU_P4;>qBN3DpNsZGu~ z0#vhYZW!d>@E8Ju+z}$*1C_E-CA3!HKxK7?Mh|A}8~kq3L*w2h<0MZa2ZeG3jPBs| z6Mzm^n(Aicdadi8&pgL7)&!5U&g_(y^AehJEv&P%;L&-Jb<1#JnmC)&AhoM~N;yr1NCxoDs~uViv2M&_V zSPI8SoN#*BLY=;#{Ecj{I}18YZhg9_@)kWPnXVscf+cq1&U*~Rh=NU)&PWWu8QDZf zh?@-ys2qz2@l;S~w-d+eL%n}qk_ZqdUi1cKIWVAy0%~fF+ofaXR<4LKOBCXFbdCk$ zhk4ylHdO^0u9kmv2_~~)t4U2$6fu3Ei_-isd%fDkx@v+U^$n>92I`+&8@KeMHu0n0LecESaLNO;&{tqqHn`j{yOsyt{}?aP&G5}d z+9TfwNC$@u_8pqI?*7SZevKWwFtE0hk~CR{I2comSFd|m?SiK4*^?|~gg*RYoSqVt zTw50koo_#xP1>+bs#p0&^yOyYa#T=?*F75U(JDjdL2mxQkk!WBXpx<~M2OxfP1OD# zaIyo&KgEg6vK(@Ls^!CfA@upF5QCM&qDN9PO_}Foic^xJ|Dsg4B5@HhISEG_U<^5& zp93;KiMW-9Kw#7eDrhsc+$V>{CGqYnt}-j)qxyRc$X0n9;TIk|9DQ`abe0F}J?$ZB zIQ4b!-3mS|{F-boq-c<&VwD_Rf}lq>4H(92;uVJ@Eo279c2FZn%~l1a{I}d05I&yL9m$x}7h2)9+X@avdD#t*1Xwtv2TGk1;PA5+!%pThNZuW!`bTKaLZWis_0*FT1@ z#vZ3=nQ)RLqEVES?sjtGc+dx$e*XAA?0*g8D~TVe6Ji8%DS4y8;O-4oa@ac^yD%aV zZn?+P846(~NiMsIldz!VY>9157-Z`@Ix0%b|C*-aq|YH(2x~Q$9t_bc-&t#!h%n1P z!12#SLu)zAqB;yg^-MhL!?%44^5jaLr8?@FZt`K;M-xInr&Dc-#ajO_`B9^p(Q8>UrZjq~Qa|-A>hL9`A&&L0i zzP0W16pdx8A7_5g$S~0Xc0`Lq!XKC&70U}GHl`1^9;9gSlcX7LzW?+7uKAbCQ9=;A zuP?+$u)Lhu!Tol1+tE_qY;00B?Fm@==(r(NLrshWi(J2UIe(w|ZqrKP2VvEGU+kny z+I!J3vXPf*d*+z&6CV~8!OKgDgFI_SFnVu?j^~KebJUcYJiDN{EF|J?cbhO7-=icG z2B2-W>QXS7Yk(JH=&>|-zi`^F`}XOr8~FBObjxguCBCUka%lzXlb!ghLF#0!tgV{#*;4kO7$fjI#sE4fVW|EPqj@83k8m$~Y_g_wx``jckEP$W zT|=D+jfHEga|B(eeA<2T7qoCl=WzMpVV{T<%)NKP=o=t#2Q6$c<-I#1t|`C1wdtOZ zknD%e!^gH(EUO_C5mGRI{41Mz$(=$>s@MG;wF%_H_x;d7LTTu=FBwEdtKQWvA>H23 z0~6}A6Yxy}9)I0lG{%M0PVd!udKZ!l=R_&?&%cPV^|;UJHV6H1 zaGUmtrnWZs+Qa{QgfOU0S*62txb3!yu&7s=Fw-Jtr3&+yGp)rq03XEGcC-rz!hvHZ z4RW^GnxG#SkPeYTgml<#k+H?dz-z#tPXkO6diSnus#3))@RR2D-`j48eIW@i)Q7Bh zQzO3n?A>XLz@;JmC}Zhgm*cUgx0;i*uE~F7y8BxxWxFljVIz7k7~MrLD`fdlp?(Ma zt^O+&Ru1*wXV6QQ{0i)|VG3&BVd8G%5NFQS`|mrs0(HK!MDa=f=bYE1 zrC*9|+~O*z>vYerPJCLjMb3ihPoYBw4OX)l!g3rQ_{=WjCBvV5FW)d454;M}{pc&| zX!WA+WdC~|X67cf4pIHM+TmwWV(k{9C``dhJa+c7$t$Lz6Gx%sThFpfl1v`hW*fT&zh7k4Kuo z2)R?wjno1|Omf{VxYGCsV_d)#g~#4^w0)3SqQb(hcx?tn4=8N6>8@1hrlgD+nZJBZ zFuaP=YR5(%7z)5PSA8@(dsX(?q>Q^_YwRjXO}~Vc%=1+zwmg9JQKy6ZNLX1-X8@H3F z%9Q0ZUUeDHeyrs-UQLx<8>Wvq;(z&5F?#eRdl&`Jz52%)We^+W@8ovq?WV43&=X$9 z?Qf9q5ptAtfbZ<6^8)Jnd|0WPrnR7!s|zv2kB>A^3Jnmu=Qm z4PQB}Ih?~eXkyiE(URNLtqJqoJqJ|MqnN{_{7VL5csp^XM}1ykG|*(#SunlT0hpA? z)`kv&knh?b656$S^pS*RiwB3FLo`rtxnVl!WXE2S)z`1Dq2&s&&J>RA!NB>*qBAQk zK}*Z8Zl#g>KOe=(WE_HJbv=QghCBWDBiTuLo5`RP6ee$nV2z>#2r~hJ%T+6}6 z%*$$uebP>l9hrB|6F!TF_fdQLemV#Bq0kvaT7 zA_Q6P98Sn64bBj-55C*ZE08e3ioE`fvwh?rKiGQ035lXT+w^hs0^WCm995F>7ZI!0 zhA*}<7_Z9E!`Tg`<41YTMcvp@6$8%|?TUk~IYM$4$Nz6IfK=x+p7Mg-&&XaGf5y$P z^H`n}Rr3OV=J5nDAYibzDzT5Q(xuPN$9b86H<~{f4$q$c!Ypkdz2U{NbET#*R`EMI zAtli=qhGTpXoxaKcXR~dV9gT(c(QdHh?llY*86Di)?=dNk4vDoj38$;kH0m$1SOH{ zUtf=~u!L#`2g;PXq}j`?95Owa^E;);DP?@w<}qzZLaPbIm6Gy%9^(p z?VGXO3x*adrB1+_ymux>lj!Ya@e>A$_BpaG4HX6Art(Z`Ni4J+nGGiEr*g4f{ye*s z?D35jTO=k*FILWtvtm1LC+`3Bzz^l~FXbg^HQ%-O)dR|(-UHk1w)yhN@~ z#g36;VEb2CW2)E zD@(6Rwra^<_L2tsgB2}y)7dSMiU1o4z^wQ>!%4(XHWTzlb#=47LU0Jl%*4(28vr`p z3e*~te-Bx*)J=P~)YisAnF;w9xySy@Z*@Cp(Ux!LYPft3v66g}reOYDjr4tA^MeJtnb8SY{(nRHu~AqbTLg8!#ofwN zqI)GJHvmjl*=zP5gGv7teZH7^nY2d__KrDfW%@;30VAfzJ=pRG9dF#J*vUuL!b=ZN z^lQ8Oxg#4%UIvwmiO|?@*Z9)yx2X8Xxg@<{FiR6&)GZlWF;$_yU?ghR<(g15oM<`; z_J>^giad!w3*!0opv|S0lKDrFt}pWZ35yht67>Wn*>x9~74CS7LVbw48uNjB=A z;y;l2!h#KP^krKXB=H0tDIc1>`hi9#o!AyG0V&zk{&FJQm_DtLQ9cBo$2(FB+W{Ct z1>}djq5PNgh0e6&Av1O4@c(9gxs%ePyl#<~KNk{_*0jUaMD*%mOEwhYKk|m&4+ zHI!CjhEW%AD*$*>MLHzhp!Ivy)_yaa4$7AmNXRGwT{URN*`_I4Z{IpnG!Y6w;b;X= z{7Mt-RdhEwvG$~e6)?~Oc>ytdERRbyvDWHQo)JvW3i|a{vV7Ve)Q*~(PPd`EVXMWMIa_{TTJ>c!f20JE@!H608~2p| z1B*D!1Yez3Byq-*dx`IMf?>EUb}%)?f)qnMD7rGZT>#ig;Y5n*a}OTNzT^_QVp(XQ zOsY3%+^AL4s*C051(=5VQ-#d}I*2|5?~TN~!R4bw50B7~iK0T!wFGmL2CofN6AgkTvw7Gk$9QAp3ZArmh>M&wUWD7 z;SED!qJ!^7@D7B1HzX!64{Q-UDF2GTVvR02bcW+Zp9}k3;w~h1NUBeGn&Gw?!G0?h zYVNX_L*<@TupheP}K`$+0P;eJ6QYQZn)oo^6)EdEc(3f6_;? z`;&yNI-2^tf5(n0H=OZ0@DFuZoW z0G=qu>?f!!t6)upe#Fja%xx^5=7wKj&IVd)%~C?CeE)7Q&cw!o60G)qP5J&v9*Y4s z;lr&R(|`CSPK0dB19jf?{j~&j^szK;zm#4{TR+F5Fmt16w4Lw-zUJoUM3`s2MC2fK zk&c80lA}uRolr>B@64tl6%bqCjMq=;Cl(455oN8J$zO5h1ZC=E$3*&;BB$ACSw@B8 z2pLGZADEpYp$s*8R$2U_#EAq!qF51GI&P-|VV@NQ6?(V!Te9?}f-*oNp!8riBWi zeyeJbzRn+}i~sWa;zi4PH1@2!g880z$?VpR-ZE)<{{t0CDurMCOIJUwiVY6WrqBmb z)hyZnFuCr#-@IIbb*UAp_$VeT{zOC3w+qr_fZ&*p%ECuF#YOX8VTDf3>8pmEuR}uN zj34lNRC?hgFJdH&F1aI*IiNIA6zuDe_6LCknZt5igowG;N`U%Ecq_FCO;7A-lJKGg zeF65`>^OL0->R%;_@~sTUxq$-I~GYDD1x{vKtO?a-M|)lla4Q4*gsn(wu>u0i>iEm z2ZHq?|C3zB^JiQap?QD)UvBBr0uN!Cv2;UH#%CP{9d=M9I7el~cWG#!n-V5Lya7$0 z_=aWj49=)&lPmEQ#+3%{Bc%dcK zgBW#aC-m{-ox1nBnU+RQB2Fv(@l}O-u_j4q8mX$zpSDn5uT7@x*b$gQ(_H%&>8#k^KqGwoMO%F*nTYRM2~7 z=Y_rrkh5rVY{g}Pq_QO+;r7<>p z_hHbG{`pex;GANPu74U#Xczb-8edemk93KIF3^PHb%-#3d+;k>8!oVPRXXx8T(Lf%cn#9B& zpi()y<(dMcleeneyF1rR56I<>FCg%A`-VANKsOATOQT5tLZF^?KULOv1D?Xkk^iu2 zAi!+T0G~ZkS=^5WET+5I068@8)Enu0(#!gM^=1kJJN34c<`jUa&=sNyz?O<$!b&q( z+`9|Gj2ZvX4V`#*i|yvJP08V)&q!%%E%32^UvMWa6uF=9?yn9>2(l>j|MDVMTk@pV zYQ;73zr6Q9x@^f)3Wok)dxs$V=Kb=4{qKEW`7@u<|1X#QzQnyJ>n+WHvlem!G~HME zU+w>=!yI62uHnDSLM$iKe<9u(oc}7aTgg6DNW`3z(8LV(CL`A3>y4(NP18#3 z+29f2U**i~LiL24GJ$a0LXS{aYVq~wnu0BMur;#n@Z`pQVm>m;W3@JBi9MQ(Ai6m9 z4rE9QcSWoMVb6tGIs^$P6!6;znk{E2>fZp2(8godwtaShvz5K8CPA48>%-pH$j2v5 zj{Jl1?ZVd}{Du~z=}sZ%jvar^FP~iv{$?9cHmVqlHc5=r3Le$pTPV?Zdq5)7QfJbU z{kvDi&odF8Xb({$u`3SD{amN(%q}Cl7&1%L_Db?7+M}9a~xDA zw@ox`Hfx5YaUz;_qF?e&pLn!beAK*{0qcByEKrFXA7x5I6}g7G@LVH#As4=^ljAr zv(L5%sxFD$_Pi#vn!9=naj?sgR#JIwB6WxP=|mh_BSfA9t@92)A>p-mVF_Z5kXGnY zlOy!6)k3A>o8pWIx9C3x8+0G6p?Qu_x0t1P{dgfp;1cob4GrwxyIq z{k{7F0sI-u4x@&$nj+kmCE%Ohm!RuS?(jO)m08;n9%r|6!aE~_4^nz}c$}Fppmf`< z>#4?PmFv7t`Ux<(ts&G+SNquW`g@+w$-cHK-_1FCx~EKi@I}wQklsJ(XyUqTcgvky zWnLrPwNus}NA`4#h5ch=-kZ@PU$@#(_vZFR_41A2I)@3$8C#Z6Q#7LtiV=~MR<=PpR%mQ* z6nMDoHuLW{;OSJ(dXUQ${^RcfcgC{DaIbcJ`^W2PoZBjftL5O<1{M|74+D3o(Iyjh zohNK5dxp2%npED4CC{q!CvPZ9K@QmM52|Geq(Z?fR5d+R3Hw>DiITRsgR9J78i5D* zp0nNJvNQI8n@HzO0Ty@zsL+N(Mx@3>Pz!ga94ExOBVl}3NQ*8g1(UwbkyAihBo~>p z*CQkBP<8|l9JVpfkh?c4wTcgP=egZ+ExU5di!Y2z)*x^a;!ckzCJOFff!pNJ>SY)VhP73!-0q-U)3so1h0l;M+3`)syOE- z^_pc!$)VQ%pQw4FMo^i5?y8l(X|=EJ+!tIXok)v2iWY_Dnh6i}r#E`twopRSG`6UX zxydwcC*Qv{3L(vpu;&c=7PQpyAI^jN?LZx=^nuJO8Wi<9iR{}`GE7Zj3>`5+*;7}~ z>*7vJOv{-Oz_-H+O)AftBYt=K!||br8=XXqt36}q50_daQ=2j)r5;NIO{$SA37N-x zo*F3JH``*DK9}~mP*UloUA(GOe!GU271o4_SVJ@Zrt%76z<%*_vBXQujQz{^-`wX& zo@|oiX^$UJSrwI+vFjt!vfS483$onzG{LWt5JyxN$RDgnNN{sbW-8|J)?*ik0qEI(4tBTYUB&|*d2t-M>be+Q2U_p3^EYruykG8K+?;yU5bl5qwlq{PgS(G6 zAul^->YG9(c$*pbG@}9nspFk1a}EZ;%LUuJW!+rr@f1$3jE9$158UApJ+mk>9a5^g z!FM@29&u!k3`nV6O5jgV=&L9eNrm9YF&t9FKccXNaQeqi@`IKtEqu7f!%T3eW8^5g zQx=L6b_@aD@Jgc3h~;YBoHr&Hy6zB)q_x+1Lu7ErzJ+9uE%bHmY3{r|Yb@@fzo8Em zRDDpJ{FfUqLG_G^0S(V+2F6X4++r2_3ye;k8JIcQBGB!80_R^(7>+_7%ITn+Bm~VE zK9Z<%G9xH2NJ?uBG)hWOfBW|3??LqXZF+z8W^pbgS(t3&Zu?i-!x~Qeh*b#?=cNVw zXEGM$)@_oLaT?cQBll%9Zg|1%EuO4IJX0q|I90#uwvfmL@~PB32d!ddRPl1x*f+oz z$y6E62vkD850txa#HC$PX&B#!>vk{w{)phwlBHhL=&(cAE|9)(R3Qj=;#bIGI{4ie z@pi*vs;Hd)%kfl@6)4^N*g(mAsd+{s-vU=*vIL*Ql&VniCVKjpro-hl+ZtcWHU>S# z(hyE;r-ZRRPonIJ{%Vg2K*aH5Y=JC*kNTsot7@9GXdMqqu^8tCSV_>Z$to@}UQR(J zLRyiUhOVdhuZwdFoGECu*Bq=0(s}$N`=z3n=W>0wy6r~n+IX?~tkH}vBbAmo6@fY= zz1~W%J4W{`>rhAMZg1*}L?P${>Fs)(lJ~u4`wwd^k1>M`cN;WPlkk$9?<1u5ubn{o z!wUfsoCT)3dlseEvDb#O59S8lA z#gv8Y2x|^=rpRo0#RM@OFxr-Bv^nhSZS_9tnM(Fl2^(qp(B)@KWfpg3WSJ+6F*#~F zC`DznJIP*=ursw}#zMc+O&~7+=J<;d!W%*yJjhv4uf>Y)sJx1?-vVST?Knn~Y6@-W37cSnb}?sTo~(jowbgKMtq=hwX^6-e+@o_{bC zn95b1Cg)FOyK0QRC?R!qIGN79*)b*Y9!)W7=XD5Xkt}4Jp<9RmDA}$)?#jZ?;PL`xd`F&(U+;oY+q{3B1e@>GeKN@szWI?Lk4n2kt%Y0A zxo?>=(Vrbp$j_Tw!Hn|jT=G_)mhTe3+<;XXS$Ijz>r7S4eyG%E&Zghahly2gf8C$K zOA+FG*S!i(JEoq+;9R7e;q%<9S2oY8v+H zZ07C1lRTyL*PUA)i_JouB1(1h^++3kik)Pl7dD)&jI)T%os1Ay3)msw`K{}h?SGIs&kbFu6)t$ugfjmQmgK_Zr-r=+7 zCDfIwO3Q?~+ta0Hlx)|oKBml$sGnuD{qh0tqv=5Ccf~Yos@_WcQ9AMTAh&vA0^gj{ zV8?0N+uewW51JMIX})N!He%uJ1r6wOj(&F7yqPDohG#FMDjE{9m7XG658=#9@aMFwC)CEM~N?{v68k{e~6t(MoV{FTG0 zwd*7guUx1@m2ZRH+4WG(R15q08m}OsXK`MA9FP`YwJy z&I~$e%Z*OYew$3~Ub^0j0&sr%2rEO}RN))${(Ic&%#QM6PUN1osty6W z_YhCMz z4kmXb+g<-aG#&)n`IhbbQUnjxyZ|ziwe_l+bKFCm8LC{))v$07XTkmvWZ|lJl;}oW z<-%fo%?bO)ms8mce!`2-Ka@Juy=>K)#amW))Y{R;s+1EsAQ?rHn9B+yhB;$k@j%lzb-usf3FqOH@oj%D*7hFdG>hNDi^~oC&fL4pDjz^XoWRw*aiAtC zAw`G<2=4}rNEC4=DB8b8B{=X!(s(>dDZfRP9|%<=!3>l&ig;G_Y~V^TxVe+Wi2pe@ zeygIha~PI-SbUr|242;2Cixbn$`Va>OvP)2W=UZIyx&(r7OgPp@m1H7nHO#zUP&91 zfO;H=iSbV1)jsoBsU91<*t!RJBmX4&BuQqP@?>eAS;6wfDSTghQeqxQ%nJb>kad91 z`)YfQUysNfJhbD*UQ1x9yS;#m2GYXd**pCfTMJ=zxQh;;&;P`^m_o(xY(3Py#vOTa z1`HSybA6pm3l9upd08<>g|TGkDdp_*`0Nt|P00tf@kBb;3^(AG7>w@sXvo-jaTnnt zFE+hShyy3s-jL=Z0CA27yZ7*tIr>6xuu#UBH?~iaL=d?h&#=egRo1@jratMwbbBX3 z#QoK~cregn2qC6|5-~a!#>KfX3e4xh+Mxo{OSs{BtasX;KwFTKllN7kO|p3MfLdDm zEB8sDUrFeo$RCA_IiUX*sBEK0)qu#QW*~KXMhs@{Sx=IS@No_9Zk)Ru>R;N}2o!cE zE;gqzQd#&ZHDRkEvr91fI#`ZhSC02P`M#CL5a1jC5W7Mc<1JeHFjP-0WVAP6~rkWNQ_LCynXz|+>!7%lU#RkQuhF5%brzPLb2wFD11&XPpO;PMff;JsFBYEl_ zme@z5F!E|r)VJiQj)R+428Loil)m$^tv{9?5U;f?TH!wPA_~7`I_kRNpWsC*mY|#T zrhR}GXV&g-#%iIktryrS&1R!sk5h}A(b#j9h;P^#5st2Kgk~qYG(eONFK24YqqZ9V zmV$-`UG;v1v$jp9}%tRc!~S6tm!twnMnk5 z&~Y6ZeC^GLFTK*n6MG-|EGK?9-BN0Tf~l4rx{Uovni?21)CujygQwmO+?gPN>f=87 zrM=ptpDNPo@*LD0Y8ox*_ThRvYGe+P9fP~_Xsh3)rG~opo+wPD!`I9fXa=vWO64Ei zq2KmubT2B0u(3scHEYTERA+V)rhUcqibAWxLGm5;+yA|5+H0gyywk(4G7qxkRZ| zl-_85Ia0sm9xJnZHHFEE$i*C9QkoqH_U9`o`_HWhbz{KiU?WFTl|)AHhpUgJfP>Z|A{=M!wa z$&h0YR&jfWP6E36WW3+0?QsKGxTprOe z6sf$oEEjwU>AE(s$rxNL?s75bh{M_DdbG`t2(fwi5;RhCNIIGObgh^nC}h!v^^2yM zyXpi{Dw8vClQ*YoZ$3frrI9BDu040w;l)>Q3*3;LzUxroBUImJ-4tfBqo#W<`xu`N za3stub2jm{CajPKO*lVK@S@M_kX&!@&CGMIhM;?GPJ$ti?5qsPt9$RactA?Z49~dp zSK>vHOK&>08zpLU7~b!8r@&Aec&DTIGs1*0EU(n@CSz2Jne))ha%rW|dMX$^sn%w~0yfV)KUt5%rK;r7bRp`2 zv1~z;M1S~D1n+a+!w0RY*k#Q1#ujuk{eFvXl9u?0(xKsSoET#9jRw{&{fo8X8$$E^ z4-l#Fp|>tv7hc;RVMG2ghnw1MEqNm0kA%i755Xyq@7csiHqA79ETh{!RMLZ(jc2@i z(e{Ia)UzQuxQG;SgyBx>e(i6lE1u4{5@Y1$;1BffAA|%IbZXHhRmt#p%qi2j&qzM4 zY7WU}bOf=zmOm@5dETOd@(SvHOVo0aMwq1e1sTYp`J>uzdzAwiaRRU#90Dh3L#Zp2 zKiJue=D&OGBbl5epPE#;a;DFJsuQ_N9?m5?p~`y1M09-$o*aNzYG_8|=N=T$F#beY zkNj3ffCEiI=t0B5E)=-hZ(G zgV`My#s>W8GNkWOQ#)I?Wj7mlon3n4vk!h3D#$|lUre}LoQ-_uORpUBeRE4OG71zn zM$-YpXkV603Re4rQA5Q61$cb<6V_LoYZ0mHxzzjY#xw$W?csauby6H^6mXz8&E`NY$7H_6XiYwlV#UkN2$$G7{KTT#3 zOO98Tv(#0buw6slv@fD~cw{DmrCr#AW=$L%W;3}@Inv|}so31NWs_1pqas0|A0;Dc zdBCQ%yt%3wW6W4RlTD+5ehk*4tD0W}Va39c?T`vUpF#@$eKfjsE z?!Mqm_ePG8lwgLPozUhtgbOf-3GpLo(sEtM z>9DKShGH!jx5ZppR>|zMg6B4tPMSwl!>LYh%tPYE)+?koU(^A6{R*M`Ie|CC)HA38 z-j6gZO-|mE6Bd&FkQUs!Yr_+(CM7u014146aU{rNuGOLkI-DN96IbN$)Le%*S+U9= zm4Rt4mZ@1eLwMI(f`AlU!%8Neu0q=$d#^cs(MaNaENmdq#*?L~R!*V?|Y<<1824Y}rx zZp|q=8}D2Q)2GyqD5Orfxm0!8Jv*jzCNV`jXD;e297mT~Ug8cIOw|$x%{w{JrOezK z-kkEdfAvo6+)0HY6PhO|cE?v7UQxpliv>`0a~WiRS9{E)+HzIq#$tbGNPjN1wbqI8 zgSte>p0Kl3*;TitD_^n*HR(#2QwrACnf39uZIu}Zemd*#Bc7YR40&He+u*a+A_EU2 z6g86_=Vimtt8o|Kyyu>%y&f1TWR0)$oPCQ9UF%1RA0Ve7h}sTUAQsCGS6M+b>@vxP zeL=pgXW?(Q;GY057}S2X+j(Rzh$I{+gQsd`%=mJ!L0NP3)E;XhEZlTLjG9$KYo8F_SLNN6*p%C0NDm;q!gERM@9Ra9NeeN6H3QUmHZftI2AYp9VxtEI_tomv=E#gP zJC<+&4VrvhwA@-0lYo=9V$npIIPKDcy3er_zETQ`p>|8R?-eN!u$q{vt1o68XXQyI zf7Kk$6%LLe`L{IXtUNIJFE5_4iEw1qM44ug{mt;Pq8xN6b@f6VT#4;-2pW}cRsoxC zb+xMn@h7u+T0UJSv@R?j6gF(Htf+uoc5&0aWqbSwJ{?F{9??!7FM^KetFf2m>sbP4 zjJ-uk7Mly`Mhc~Q1Y{=YiCcWe&RL318#sE9(zxUP+&ONlU+h_^NJ*~QB2GFl1G9iB zEc!hbwK2{HbT7XZsR!x}oGZob**E7bq|n_w-M`(zKkdG6=~Za9$Rxza=Rua0L`|%@ zL|s})*9&nnSA|q;DNjw+e9o*KgUv(z7RO7V_JwHQmO3R0lC#F@dB83kuN89Kk zopn=B*RV3Dl~gUMNlzN-_Iw&{!w*7!FFv8$x2&$$M*(gRqcbla@6-9N`;-oHnLcW^ z1pfZOrk<&D^hgphJv~Lu*yvH`Os2UzXe=%DSwvH?d+WKVkc|h-qGg+1m9@?&mMg}Q zVv?KpIkASFebaiBdM4VcI0ni~ORwt+Fw~ooGdZ;hphR>Y9O$lUn*-Wk>jiqs4HS%D)PhW^ieJiosAP~_VuGk{EV zGDILElwa71Ba@0$PBqF(X{oqi_;?P5tj0e;sN%)eUzx(shk}5hD|sq3maMJ%W<5G0 z%fgIl@cS#a7*_`}u}ue}K});rJe|XhU^*gq`BPZ6Ilm>G;rw-v`5+u#wSIeY7nxUQ z{DMbGis=j^FQbzJ96sir)XDs2_x9nVoLlT5-+3wDhZOctt4C}{+?INO0+FD0X9e_h zHwzEji`k{&S8E<`%t6-0178nP8-R}eLHfA`tnN$5oFk2iOa5%%(|HF1rgkcG&G=vD zXa}p!MsDvLv-8-&Lqf1WqkUm2@}l{H>14{qxfMEawOA&NHZPheto;xfy|d3`Q)RsX zV`Z{H1b2hsQ46>DfD!BY#J&{jqLOzhXsHEEq}nW8Xh8gx%))9c{-rU3W)*=bp(p|-Yi@X-i^$3sRh`0^&; z@z(31x6)?w**r2Le{XsQUjy%oz}dP~94}3_@&F1PP%gibZHcX?kKTG0{*j3+Y%h(O znOk4s73urt%`)GF=EJpMWfnmtRsJY*h2Q6&EqX4iZF7D0o%iTfnk@0Q`0$0V+i!wy z+{j4}eX*jQTTl8qZuOaLZhJqG~h6R*%V8l2`K zW4e)iIU-3tAB?!8go{*dYvoa3Ub^!f-MJdzLR6FIw!1Lk_M*55bWb|Um(bS$fAXHt z;C8Q;jz>3K-(*4-+;cgYo7Ng7XkL!=QfzqUT1H-9m7~AIzl*Zsb}X;+vdcBMb>}MZ z$PnVBLvHU#P3dpp@xti|5R!h-7DH4>@ZBEa*OQ)D zt)|3iM6d7s`)_`&i<81tU3G?*tnhl?Q=M=r!&lf5Ddy-&7lj=vR-de4Z}*pqyo_cS z`6-p>;$|T!35KDdygbcY8vJ?eX(qQnseMjZNjy@uPu1(T>TW zyo@++RYC;lYqG8ioqjE;Wk;m{EtB8=I>oatGK1FCaIwqe7mFvr7?GqUP5rYa67P&8R@J=EAJFODeV;FZ@$dLK(rdLsTQ%gNrLgH660h=mw@d>ba zdkxnK&(Tl${EK2EY&~?b1Z>5f2Fe_qat~EwDfc~%p@k9FhYgUzr#-Xo7eq*A(zI-d zo^^}6sn<{!6^XF8e`2>l$6A&EVl7UQ*KG2XFvV++3Y0VCS5@5+qBW{4jvdfZ)6ll< z<2ZU3{BL=fkL#*rv#nwKzXAr$r%T9x4;XfEy}f1OFQPt(0!^@zwJcWsC#_#cM`RF5 z?(?Lduq|XSJO1M5VUuxCCVHBbDZ9D{E`P}6AnTZLn>d#&o`JeyWDIKm;;cb8)+*nG zIQshfoO2?y_9Gl%aCR9Z%6Z!6Q zg{yLmq*CeTUxLwK2P{;Xh15-;q)DU=mn37PxuvC#LF?m*?Ks#S@HeSCvQ4k0I@X(mhCi1AYTUx*O4gq)A2D z>}0!{rI$};atAtXRM5;=h#ONKPDN7gF|K7)^YCXBALV8TODs3locla zZd(_=aD#!bCKny&JtI4=@-8)65Up#z+E0lu;ApfC_xR}oWY$p=&d1ntzslr(-eFZl zb_Y$q$+`Xxx3KPk(j;E%4@j$2t7k@K6eXR(AVs zN4gGoHfAsVTXr=v2=e8@Ck)SxT`X`{EyKcwianph6hMe*Qg-%{Z38T5YS z3+ybY#dcleCDpj&j?hjSqWCmT403N;rf`!u>9(U8`P&#aFb?ekeHOK+3 zK0n=0yb2h=VIn3JkY?Hv`sA;1b&nxC=!)L^yQMKcjCgdSkB?7PxWol}&6|axk)fpW z(k?Dmi2ceN+gpigCoA5CUnq`ycMFnIafUl|+e&J6_6H7O|y z4|X=pskFu2#m=5py$ZdRc9*gI#4?A_79x#)Lx{xS&>;ca1LC7yL|{E6X2MsA5!{w6 zjF);+Fo_k(&!Ra1yyEVAOkjg$I7^ieUelji>hJZ(pAr`C6` z0KUHL4pirrqoS|8Zoc0)Zfaofo(8J2+&s{cpt?Ib6 z1`tKNMjxb(mC>sSr_X&TiRF;Im&lTOz&z=^^}0lx-&*-~vFL;I9MOfE>Q?v+4N9T( zti$N8(_M%Nob#BWmB7op0HV{4!b~NnKHRh9s$YQJhnT8|Q!kIE#={e{=4oO?=BIXm z+8Sw(<||^j387JwwKtsb0~vz%TOLgQJkk)mLyeVD90}S|wUw`&frB(!t=s9#qjGPL5!3!c$5Rtd1ED+57a5YR>J{VG8;9KJHpb_-?UgYeU zTl84*?FfM(f&~%^A+}&D)eqw-6$6I+;tnQ@CtLT?HInnTg*B@c+XvzbLI2`f|MhxK zp!ehRH%ff??l%Z3L7Kq}O~O?CnqLiRmL+?|ZN$(84L{h^Zdqx$tiDbvz3ExvfKj{o zLrrVMJ`n`O1FusDzjqKS$eLWOh|7Acw(k*&BKXTvWC}9%v2sf=dF%4P$++V*n+H3t$cWqI4ZhU++ji&^7&|U{uJdr36YD)r!1%rF zmiq=>zPOV1>+z)wdj7u8K)}Zw%X2l^n6!BaZxcB0Na1?1_^83M$s=Y;|6hlL>S_|g zTMKr&=C79+d^K}txx&Pwdl3yW%jib*`JMy7f#bgDYznarEvFdCJ8$E`{MVzQ48ymx zzcTD^q>*xP6e?!?OzFU@+k3s^wYqpMf+w$ahECXZ6cx~z8g9n1jcp`SYuU%8RdR4J zpZZ$)SSG7&Jy7}ofOgA$@9FY|Vf%l-K>oeqKT_?53pSxlLdfJE{`&YRx?a&}C~a~! z&h%ps8=w9`yLU47)Zb1+3opXnkGLgdzU*=DU^q2io9+&j zd9HWd4v!K&KTHp@0PEUg7Gs9ebkF2>#39d4*u9$gK^r%pQ$w(tR1+{j`xEUU&j_iZK{RJ-jI zoj&SMct$J`bf?(M+M3B_QbIG(_IF>{6PO`CZZ-6uDO$9j?(ZV`8UlZb^Ul>>&>L%X zH3uy}Ig+}p4$<-E6t$W!(zcTu?x~XWb=GOLT-8+B5sXY zybx>^^c8~e=NEY`mz|Fql32(5-w^}&jw%s)omSH6b1|2Ec4+x^LzQ$=lQxo0f;f{H2vEp+4P8tNk}*m`Ay9ikf|o_duc7`@=ayF$q;WE-Rv7vAv|lno~m~dC!Vo$ z4gdsMjb2|441Mnh1{vJNqu(02Ys)?c)|a0THW{;X>s`n!Y19)X1<^it%Oj!^P=5RiSWC43eT;{On!*|M8@vjWcIkQ_Mab+IlrU^iRhWD~fo-5@N$`>9 zUFE&h$kF0Rd!7S3`~Sh=c>amQWg&dzn~JTGB!g*d5vqJHT(d;*7Z|AeMm7~^W%a68 zPSw$h4kr)nPk9^FUqmGxaHZH~B3BSg?KHc~_B~4_)yB05y@R>18kCoJD>!BFlbw)f zo5ze0cjxU-B4HqR&o1Mb#(pXv5juz-ZY5J5`-p+C^h#Wh#09^{IO~}%lGjxKzEF%Y z4{m1Jx!#ap`ha{XEP3~PxzNyT&I_SfeD-BVLs?2E-dC}MkR5vv`J%tnhBG+7dGjPj zQYZx_yR)5`rVbPHNXkKhQSgMc$cVEyS6(?r+{6SeIT@*&3a{Gc2@_WqIJz3SyM*mn zDz)i$+l2U`ZNY%uf~jNl1w0;I_+I;idR_FxUu=VGD-%`U{P~XJ+eqC3-_E2=kiHr%o_RYkxQbkE{vEfn zD)gM8T>DZjroWvr zxzs%Mi9ZhY@7@)lVs}tJ9#-I~1D7zkf;2*6N2*`qUp6pKwF(q=wICHqZ`1tL?^AVQ zm%_0~PQ_mirFhUq`rgLZnO$@Xw(b+#;rp6pPMC19R!=~Uz z&$u^p?3ch?-0#tIm+#a6C-}*NbKZ#J6gs;#>=^}bD06YrRKppPgZYt%aCLCVD}CH+ zVe;Epn3x)_LVxtrsa<{^!t_=|Lr{EL(>f+M39562ZyG;U zU)mQq(>uuhgZi5|;(+`E3#2=2s)JRI_jToAQN;s{oJWKVeh4j;p;cw#o43^TC6|@lXyltQG(w%F05jR5U^=T+)y-$MLGl4+SMu+N&WE~Jw)Xf` zP;b@xya2$dM}O9uFsfmJB(Kd~&b#i00|qW(J~16u8G>8Ar?ngl$YyGi4up%vScW=F zg9U{acSl?uJN1sZt){kXnYtc+i3~$`;|cUuJezO-0R0D0{9tGP1hqkWbJ}i0I{fti zXd?WbVZYp6_%HJnsT5J51Q(SQHxPrAyXOk@gBGY$4GV!?5MaHi7GKh+`=!VOjZTm* zdKTYKq>>V_Meb&s&U`#5`eux|dUP8RR?3E9oVNNxAbg@Za1TKEl&a?c@cWb9x%V~I z${AE$8Z(Gu((SrJ9ks2lwLYJNiH26>2J$I^rfcr5LJLXYXVZ@g3DXjhuj9dOfgN2#ke)OtE9zV$EX~J8o2a|{|dqNQ5Tc9r$IgcD;(g}mZ>piK}|Pk zyW~cKlb$S6lkey>iZ_KW|AQyBcbI(cPFcMYw#3WjUsJ@8X1&Bx5=vB|6ssd2#NCgA z2SDVLs{g8CA$Q-jp5vR(8h)82!aVw4EWnolb3;>R_B+iO22SE-FPNRuH#VIY?`eqI z&_Fm7;J=~tK|lt<4>`fDwSwQe;Kz zK{R#fvfGquVA-o-7F+};8dtIoRb=L8Q}>KZRiJr@BcF8O;(DNH!*xFgH1*8g^Z zBVXp?BG7;?W4+ylfXzj(PM6K%0&1FIg;RxtIvxo{Eb)jIa3$UIklTF_9QXC(muFMW zP1k_j4Z9?_q(m}}-!F@{v0QSdd~ck0-46GC3N6}&whm2=C;r)P+6ecGwJ5>JJ_U-+ z*khIwnyAs!*l!J|{wP`%V0E-jk{N%4E%!!RSXfXVyFx}|`ag0D<%rnC+TLgE@t8g* zd;}IB*EwJJJjocTN%64cLk>BWaBDM*24r7jEtUv~!bjl_7Jv5ytQM}TK=BA3%!uPJ z_w6`kp0sgwwtIMDuEH^*Rl|+h?Dv7(KgqMJ&c{K`ESLQ1v6O+WSTP0 zyIq#C44`nSP@&(E0AzSG4Wyt3Z^=8}n<}%*%wT>*@*5@!;3=KX-Oj?{PvwpQzQgV2$)eNB5}GS0wpiau7$Y5JWVVxzthxICr(^rxw@H`0?A zeA< zKHPUSq&>)w@x6*>=2tuh28H(`6L9%*M4RJxz2 z0Oy<5-{U!@Ws5FT&+$tG!~d|cChq9Y)lvBjp1#bTbu4;Q78PWhXK~7?QJeTYYL#K* z#cO#wlv4RkwR%uBc^arRf;X`Kp%*V#frUS=lat2j1XJbK&(mtcb0bA>ef{n*=7|tl zrVI6D*>$@x%B~ooE$0ih3YGY71|(&XkAqxM@Wa>*86>|peWU0eLZsjU2uRB$VbiG) z$>~bDY_`^on+Q9y>sxHQmh=IsP&s0J4p`x5;{`i8?2u!FsK2K33t9wtE4rzgtCVq~ z5laJKF~QU0M+o-pIK3_h#w;D#*)^Aa+-hUUpwmPs4iy?vgrbyFR|C&*7r}~d_L&sV zY{&AnM0kD@p&t{L3iL}AZ3>3r&{10V`=Wwo4PlsFpSIx3jbddeZw!eopymJ;)Cib? zD!kOT!c;En3j;}n2zbY({*t{!a)E+vfnegc{)!o!46~_&qS8q@Ep0m$vu&i^8N-Um z_HS0lgawSmzq*2l7)~WUH;O9?MD07Lg#(B*+$SMlzCewpFuH2O~q zdq&-(LJ0@kieI6FYF2oeFo7z-5miR#wh?IHz>)ptMBw>$c1oXyF=wJ=ZOu%mURl4b z`VA*h%E9f+e9P$`bqVLsu8!no=~JZa8y##MnlIW_**V~%&l7o}@FF5TAqIFTw7-4X zqiNed_%Z%mpyXPN2KPs5SqJ6y=*tLuJdj$0yT557Ug_3A7j*8MX zJ}Bo)7^975-7l>akW6DWu$anTP7ei?Oq+YY_|;wcrD}hsjB?#^LI<-@7 zw#uGdMeq8^?W6pR?M`2d_ZFOyH2sZFI;;6Xk(`=DQ*NPyr=25|hpeL-eiubi+y0+J~kFw+obj zFdU|uFMO)v^M8RJ{Spm*L6Z-EDvx**0}mGCLhj0~%K-N0X6yyO+VJnU;GbnKb7_Or z>b@LF@CEb`*ZUSLW}2MuiA%{*)z}Sjv$mxwif<2oFN1^4QAA)S5dWARAFUNzmPl** z;=B~WYHLq@U9>HgJe5eMlk(T7gDm;QliHrs)%QS|%6^*Z(OW~93v-k*K@n&A-_I!R1vI)oHIH#E96J1$&RE>^_VQAZ3l#ehp z)aUy~@|G~f-|k^q>C{A&f8sv1H;=Yb=fP9lWpr|Ld%uF49=1jO*L8IJhD09Tt3wv> ze5MisDHe(46E7OCLRtxa;7;SZO>JMFHhM2?lB<%yE%FX!MKNj9I0I@*N_pf*nX#ohiG@h>;I zUujE|p|LqQ_LuSs8sLiDEM{si6f^lM1_r0b4@E>D{n4qbl}u)_eQ$jaU^^TFRe)RO zNJo|gnbt1Wf80x@^oD{N=O$x*0DYnu0E!n!1HHd0SM)t58}_+jh#n^29gG$h%XA15 z4$*_dDfNNlp^&eqYNZ+wZdq$4LFnX~>) zWKFJs#4G`nf5b02t#ZwEO~R4$nKuvV+rOIxFwl?1!i8g#>-M$Y#umJ+yGi(AN(F&O z$9X)cux5x8#VP-Bzt9JHi<8rP3GfM`OncyA4Eq$Bk$=bn({ zDsZH?U|pe@gMrovv-+Y{%H!V#wfttyPLuU&mTQrx;M;I<_*y_O#gah+{k*-X0RXgrmzU7Bt9cbv(<(KW7A|k^AET}k6gv= zMY0&3Q&~WU1R{yd>?9kQy@#4{%0}?{95d8AWohq+f1`%_DS(=G*h6`v9!xrN8kwxs zWT|9)-@uhPT!*qQRg$b3q_p7^o(JJoHiyW+AjE)jbmls5)6rAJ!OOP`nS{|;SR z9UPSL^N7N7=6|xZvK-A2Del_3b&U8?b?Icw(t-9hkzW6o94*=htjrAMrn?mxb|hL8 zS4r4*Lc8$Zd@&)_u4uw&HDA#@goJDM+8(?+wiM+lc9X9tEW7WDyea+=Pae8-MZ>!&m< z5B)TCT#&3G$m}_^qKZn z<`zySh~r=39}*vr1u}=Xs$Iv6Agkf#huCB(AnqtJ9wnNZ5xkjEC0f4Dty?VOA4AWz zhlwZnDT8g}PdtjBofkoWLWfMx6%yYN_+h@DBsoAn792%C)RPpcRh^$RTXI0%v-M%x?Y((eu&|58~+ zqJBN>OsqO>;MNbaDWb^i+G@aBq-g86xB<>k!c=V)Ui6_@Vg@BD=z|-W9WqHBgg-GG z5Ej{(^hxe$AnuD}OcqU?tl4~bR0)0bbGn4#)=wgRk%pkir|t1M$gSc128YyiXtY=e z#sMOm{d_I%Y!t|zzY#NWJ5&E;5tZ`vBuxbuVX)1DPKwMYd}wC6@wpZ|v4_m?WIpN; z+*b+E^MTpdEL?p#yh^9$&+MA&EWdqFe8()U6L|QGL+xJGUMH=Io7-f!F37ktorzb< z>7|&UMFmk2WU3@vjt5PvGLdpH`nxuZ&t|Go3B_0G(l5A$W}bJeq| z_S&`Usl9(_y)9E>uSqDV=ZM?67)!SCH^avPk_yPsdmgEEf|)`MFmG}2)I*z- zQc~~6iOo?fZCWVjKpGiW9cap2K4E|bwdt@oIXYycgPannCH$565}CI6R~+F_iszo4 za9o~oZbcqs>*|4*CpOIXi;79+8wfACNZ}8RMnf@!8}&lAd}V?5o8$^;@c~C+@|dzn&d=SZ#^03LFoX*<9fXdD3*=btxO0io4s`#3P!h zOu^Dyc@P-N_ZLIKl({!nr=w%8e)vs6c2*a=(}^;4MoA3kdR3@ktVEu5zyk~PjLCBm zr@RKf+dqRf!KdpguPeTIslsj&N1Q8!zRWXWYUIEpCl>bK8R2z0do=<~)q(iGUWJ1d zHY`&F7yKC@WY4h!_tmQJz;M)9Q}R<4(wvE0SX@+CbB4E$qzKIv+GOZ!Qx8@`99Rn1 zuLaJyl$;*L2Z=sSg^E{0joEHx#LY?Vj-&1gkKn9kxI-dk@+^3yN&$SiFzP`NKgF6 z;c{~RcZW((nI$b7b0?M)cU3j87DCs>)5D$?D0lW2&Z!3@QO%dM`LhAO+p{vYI}p&86#1%*Q6~8>wKnSjoNLkI7WgS|hc5d0uxAt$ zA=i;Qvy7=A6sT>OnL7xbyY5RE6D*Ha_RBhFY>vkA6kSR#YHsTNv85Vl`wH;p3Lhv! zswO25z&mWJ(ce8?OL4iAAoh*YsnEZ9VrpOJs{XMHMRJ0OGX$S8tIB>UbdXNAm)PDv zM8U3=l$0U14lJb}Q+qRj7@I^(0EQ7*ZN9~*c`)Co%zeKn!QBF!@m#LGDpk4HW)HI) z=TD|#(|dY+Iqhn>8D-mYC&(HK{PQtO8zZgXaiIiwT@x!bR+_A*K!K#;j8x_@NO%+u zN5yRN@iB<^A)l`lj)wZCb7y&Ff>F1ZeO!K%c;pT%!p_DzHMg(sj9zH6>$6oONpy*` zL4dP6AIWJfFn8T;xhla)JM>^C$%?QVnz)nj=Q&dUqgY@%d!+d3r~%_qkGU(5qS4<`4JFZl?KE zpFC4_-Cw1sx+VsVz!aK*m%pdyFsjc=UC*k432i&U&)2qd5lX7ysG&|nj+IS~YMEDz zRTMF#LfGOHdnPdqwy!k zsjZbT%lNSCsdcip#k|)>{0es!Nhy1+7Bqr6Ozz%Vd+mFi=3wRRT0_nQnil*g7Wu;r z^tv8yUOB6b*%i0E)gLVf>}6ieGcsgh=7sXQpEa6NkEZ5f<_&ZDw2Ymt$BDAVEB6tU z7$(3fMl&bEmT+dHgHldz?`8nXt7I;$x(2MaB<1YFa7w5|=C+r_(I7xa)k$5-s-phe zc_D~(A-=)F{8}=JbBrXFdX27@y{?f_Ps9{enWl0w``)e5&Sk%|$)e8HM!`9V(Rc&| z$2g(zZh57a{$*lIF+Ql$Li5YmPmn4`Xp*r28mg7ZqW)m3uqEqUgw{~nHMlf20%YFu zymEFt*hesDJY87`La-QIc#TwRY^JyBz?Eo2IU3U_!>PG4Ww;Mu+6UYX$_w^nf`EsMW* zM^^-Vc@CHwLvWHi^=hY;1PeYFM@Z+QmcpFu~ z!Ph&i)e7IH#_%=aoXI}#x#g7M*r!lX*|x}ov5Jimk+Hxknw>V8x3?tgTEwQd9Z;!A zam8mNS?iQy=CL%632dKt=;D!+e9T-(e@4H9HG7J{LoIGRbU>sm8M@aoQlheNB4?!- zb6DF`kCVH+)V40&M90U-Tpboxh<+GbW_wr^xlG64Ytc!vwLNwyf_bcIxtk=_sfDptJi=bCM7_gqeLa!bIwT0SrN{H)7ul>xQ;RfwLs(ct(JF1#1H zmgDWysmoh;-Gg$cy%gOFkg=DyYm(ec)ZYLz=}3$t`K5<_?i|XkxU&83AHB_x>)k=m zq7Hjh>k5y=Wv)TgnOy^Jg6Q6Q($97$J_I)#hyH3^+l2QT4-EUA9%`vJGTHdZ&f>(Il`%irR;6QI$)1=wlDPAF-3pAlb7?arN#AYi77?4kb|fRH7_MTk?$%Vtu&g&#WiN zrTs}3nx}(4UW!zn?nyacxPnT$=s+yC1%tY6p>GlS93%@rJE{sh)+2o+0KC6*~K+fdc!<}#t`|Y5;ILO|MRThu5+afCjyqjv1$;g z1J>Qd9gAqn{R)`2{c7KHb6a2}gBYI_^FmIfg@sv#TQ+LoVbNDy2 z&=AN<_$!cpV^LPoyzw?y`$b3_CrvVTfIplY-ObtE3KE>LRhE&O<9Zt)IJh1c);2Qk zl{Ih7Yj+xyW6z?D5p+GY*PG$e64U-G)k-XIFK#4om7{s*6yBxYuBU#|+3LNf*lN7e zv12okD=tEr`=wz9tCvp#dJl-yxpW^!+#7W>)xZSG2d<8P4fa^$Ld za>#1xXeei>0@~DdizlVG_S&l0{!$a>a;B{4o17w!(UvB2va^Asv?Sm+XySzPL6wux zhun}HU5oahpHwrVHiw0Iw}?M|S#nwMFJ*TAV3Ux+?{=>sc8S z@9|z!9Cq}bQ^QLQSeqsVNQP9X49{9EY%U|`BT;Z$n{x%G8~5X98d3lF&>!Sv)Lf_; zS$>_*o=kpL;rA#aXxwKb)PR-IZ!_ag#J7_3*6`ey^4-#qJv}+XxlS?sjrw|vbtwV4 z4tFS>mo=_{;SRad>VK)MF*oW^lQo~SGC`&^JJl5QLq;Y0Z?ZqTfqO2+^MilQ#CVCo zDq;3LAUW8%x6^{S)%vILCk3gw)@)kt5B(LyQ3L^G1zDU0Xz@Q+$0xxvotT9*=SMfr zdy@HBk}A$WO;1(wtY{2n%c||Gw-^!Cf+s*_AkdR|YY`x#{%v z83Gy(=hp}K-&E$#FS6IWtt(3BjR-NMNYl#} z7eg4opl952g}&T*8fE?{-fj97=2cQj)=A>2BzpN^H2f2@F_LSrNW#7|sfh<_Yy#2k z{X2mlW2+@n+TV$g%Md19Y{4lgL0n4UZ zbrbv$3I1V+S<2GivsL=)xf0;kcW=qf?;@H;I+>nR<9}tO>Ai*Te?B!*+iLL0K;nsl z_gyCoYU~W8Ie(F;zbK_RTS>iOTMLNv@C-wLO#!W^4(71pi5_+@l-=u6oi$Q9SGHfn zO-$#(HSwtWYgpH1pLft>$(wRw$UFnC1}sk9+!3S3<}o|?ztYQV2iX@6JCc7MJoL#OZ1|xQ?YtZY8aXm*~8ZA`1?)V}08#JMMjUtr_TYd*BqHD7t z>;VLf4L}WU%)^iO9S(xF)by9l&dB@t>81C zLIz`BWW+Af-Ag4L9P-)UjpQfoQ&yHwTC6W-9xECAp;3 zC&%ev!qZgZ)_ICbMW<<2&-|m%q23j;bY$LsV4W;^&6&`*TkGQ+|E>` z;UW)m7+AM)8B6o6(2QNw86=PyczPj@Z0Q*0xeyn)_z439ab?K7My zE(Fp_GxJdquk!ydpxB81(@DXu(LzjQSG&ubGr(*qAlM(43v`}1(dYlSY`h%z+M|u< z0cTSFeRK`xOO#q`*JdBe`LeBKtQ854ggn*nA z$emu9rP4!HrA|28rtK3l6F*%Q{e8z&kd*9Nhb*_c?`d$$+lFCK9eQ07Z=c`uxg`|< zwr~5`aDq!`>m05q@BIpPC3(8oMQMsyO9ccU^AS)O_P(FWJIJh&BdOpP@oB zQ=KKW`5lo9J2zPc`#rO?3|7@qlzi*_r7)*?{5YY`vILSs^?oz%)y4!vm=Sq={PfC9y!c8T(``E8UukMw8LhVYu#9O=Mf^-5B zqcga@#+>Wv9`wvv(O0)2Y0PU5rxp0_ZXkB-nkSRU4o-)r?&i!EHKDFpOR{tWc1#9E zrfUj|>h6(IXM?-bhi1jfD)L;@F6Fz|mi$T1NnW@u`tM~NUKjGSGaKWN+;g2QEH#%=mI9(A=BZ-jMBMU(v=aVmU_(MPF4`E+-Za915n3XinQt})bbu^ca0hA{;y z=zc@HLSO4MCadPS6Hl{xgP`jvF>z-k3>mu>vVmFtYUK2dZ$m58%Ux{FAWkOsMH5+M zT25~nqKVDm!@=e{@avyO^sC0(1YO41*zP_Jr%`!Jgn6bsW#hG z2Xcx#|5v#X2lWUN(!0~JRm=$+(9Q{BteE)B)#Y6$-R9^&v!DR7Ae;|i^J>j1eA{=Y zUR19|Sy{CG(OExpC;YTGRNDEC!F-l16A9Db=CU4{DGbk}sLriHpG}`Cvf_hZhp{n? z)&~@9yhlU&1fHxFdJ`DQvX_9hu3NIkcrC4qB__n*%%G;GWo761t+SO-jmwuT1N#k* zRHL0hEQ5_3!{cpj4mv2L;f3kc)#0t4}p<`IDk2;%EfuIOL46 z-4p$GPGR;Kp4>bk7DN_}-d)Oinj>av?y|y+Vah{Qy6{{>Paw46o>Tm;Yrd@d5Lvu1 z4@>d^+P^#@NhZ}()61Su%N;Qz$3OQ}A8Wp~jU?e&YO-z5k0y6MesQ?-TH^M=JTruQ z+>CPD+0yr_`tfN1zQG51P9eOyv-1mA@84+;1*f?vrtE1lO{9p{dVy-+jU0AdB-*WD z_N9@`=+6EacAeN{hk>p5GFh=zH&Shf@V#hv+*{1-tGXR8on`M~3sKSwv73^qR8~nR)F~QPdYNQI98P$| zr7SNec@SJ0Ozr&)A_&nNzcjt?*FjZ(Q^a)lRtT44l7HOZowelB6S;cO@6Q((YA;A7 z%}k%q3kQX>%W4xVxhrseHhV5Gar^|kLtZXLwgu~UDqN|Bj*=BO`dfjJ7@p!eMRr>J z!U`u2{;aKeXSx}#jK9j0V9gT3fmDSAZ`5|91~C)U-4kVOQDvtOzjH1bOT-@uf5}C# zmhkEt1C(JK>+J(`y|Dy zBupQa*~bjX3Jui-0WN!^B)t8_p6WdSr)vBD5<&VjQ_%t>Rk1po!u=Khqg)h%HD#rV zO=#)TFGifKaL()5FF?-YqWFmxQB88KN<)PLyyqkGZ3)GuwqF@{EBt1rv#b-wlRM4( zwRDW$rRm6%6Lb&oF1L0kw_lup0i%@AZD>V#v?3ZQ7y{#|0>j?8i}Mq;H4eL%Cg(v* zR(qVqSsp&b9SU~RU6Zl|^qX%AnKj|njw4Ga$(kxyH|E7Rw)js8QqH4}Y6z4irs0$=#*%-P0pxFcg2iKBNV!Ah?(=#AxY($-?oqaKn6O}uf_qSz39d?f-sy(5|qfhStXh|{7z zQ5Iqin$}QdfP!eo;!a3M@wQSE#@~}d{jP>>*>y5<>hC*F@!0dSa~Ud$snlWWUkikY zS-p2oMtIf}x#$kI`6h4RRRZvZnl}7bme5x(a$qhvxZ%8ZZI%GH-Hs+!OO9mkXY_Tv zSD!8*R|vRZ&K8}HW-t2YNKn+tO$lshUkZBKkLgivYNA0_>Pmq=_KB@bN&;qD-}(7( zvLdKLxFBDliziMX+(amRBV2EH+47#|<-03$2zZnpaSAG(WKT|xPNg}%iPDtNeM~*z zHw%kv|Kt|i%dCYrE(Sf-h{zrKgePi_f5?F1#T!>1&3gBCl5m1As5*~Qh?9x_%=34w z!=5D#nC3iD12sd;E96cy2f6ie?-`P$c~M8V05y)5k=FvvS3TGwC!S*Zo&9EBzOfBA z(p?7_y=t$}glIPCG9T|g6co2Ftf$BLhk|PaH_1PL} z5=Wf;lAGI6fIJw$Zt@lqSvZNd<;XM}>^Tuu zHb7C7%5WM5EY(V%08I)@bPS0dn0Twg73WgH?kVH=o~fHOIy30lNM!@fliG?;-2Qcw zCvP}hw2mBUCZyk9!+DT5V}z#gs!#n5cN`Mbj5RTa7Kdt2Y=3sFwCQcQ-d*Ro^TVY% zpGF*`3x?|e>nK~|4QOEK#tc;7B&G0Av$OY_$bv=^Ry9)Dm>CkKenojnC;}$-1k(c# zWWWj*KAZ$VWz+VRtT%ib&80Lk@q1FhNA+dbQ9o0SaXTU+W`oVBthP`vCeiyo)tUhN z7RL}e4^cF{Lj`ay;-$qsI~6zsE*7E(Bb}sa zc#g}zv;7013Xyl|Bjpa)2e68(evG_Cc!2m$ly(PUb9Nk~?E^VTPOWq1EVG?ALa}S5 z)An1UpBcZJSOXH4H}Pu&?QWwJtO3YH4}W!*U1$*Jn?8C+4(r=SXT}S#r!5}WZIW7w zfaNaBLqkank_+8Z`)HxDzRD=l8%7J^`=Qc_on5ufxX13dCGsBPG%+XxkA5=3SwBZi4?jxP?TtPhA#FdG1%P6 z4t6ocuz)seey#D(Hv$~uU}(Hr`iE<}BUeQ_#8&I6L7cXU{ZuM`8k>UE5+vViN>q@8 zG_3WykuPStkT|!T;)4f4b}wl})_>=U@j}8|K0{Q(9r&KiS;UG8zp(xOJzI%e)j#wR zq5li`a|v;;?w9b1Y83xzJ3jCpy=~;#^M_4GW;bnG?(|EVf}E43c(>}hEeaQe)hnNC z9m|19Z1qJ1#OB}=kCpqWe{z=m)Nf0&2hPB!16L66&F#*xB-0Nc0}8<7TOvZGUdsYE zR8xf5wbPsGlRb&Nw?mR?hu) zXzh~oo(~64BV*nh|Ex+O8SEr4?3`zUf%{o&gWzhBY=XKR&M8=h zVfy-Fgv(NZe5_IhI;Jg5EgTly2ul`vd}D>2LX_@0MpWjhzu5CZ-c$#ZdEnEM0dB~E zXa)8O^H`O=k$i=zVjdGwcKJz&lmumXtFnk@wXc+4yAj2oM`8CUBn1?61NNOt}V(i#!%^N=R}4sX2z`Dijr3-|Ex8T zh)A$V0|3Wq#~9`Fn@Q1zg@S#BL{%Z>&WL`UyP_AMu>NQy2Un!;B&ls&8aruY!BE5T zv1KwUZciUy`|TU^s;AN{JrDN7T3H*sAVg`-X8QrC{O(MI>LmNAFSELaPPrhJ+&|DF zVt!r%j@rfK!%Q-q8i(H1IGfW`PoM~u#;)q8|aAd#*=jci%}~j%VC2} zR77VcL9*RaiAk3>Q_a0Mf!BknCOdwpS}7HGb@SW81{WR)`p;wBD496iF3Vn1e35h_ zy5{tWxCuN8%bXi4BtzVR7KS(pV{PTXnb+EV9xC_$PVCwEije5FP z^%B%YWT8fB(pC<6SIPn`m{_qOS-q)2m)tV9P{R46pWCNj{B9H;7Q) zxJF81bf&&i8_Aao=I?Ff@yq5$Wf~o>kyDO5DI#4sd)MEHA0|Z(sK#!12X!st9xlgl znG#h0ZFs8SHge?5`4bg&hE`DReXws8%j(96J=?8~;B^h@L;5V2AV8)5J!oC*wRNUt zNagbU`qNoEfZjtZzx`hZeA-nBp9ZEhu$iQM$OAaQcK8B=36B}fuxX!LbY&!dZ@=(0 zrnlFlxwvhK-be`xU~T2zZQS@R51BPJBckJ z+Q#z4!5!&!ac^{)A{X(4KDp^L^mZjNs+T1xL(}-klO9FeCTM6FzUp$ko8|FE$wo3o zO!Le7$jL6-X4_P*8k6U@OoB+G?lx4nIi>^$aP0*o)0zCkN_P#Ki)3YIJuYJuHTu0L`FM5@XguT4e0m?qePn5l~-DJ&$JehrZ z;O^QV%{CGP?2EQmqtajDk!*5C6?Hyl;-|#mK0R3}({Kpa=6Eu9%i@j6nPx1q@|-DY zBY7ubX5s^zs#@5^6}Zjqqr{&%K2z@z4aLrO*DTIetqqSs1M-; zUa51*JxBJ+&!fMN^MR_oLaYSjGhdgd3cVvcdK{7my?n_TK3ly(gK&EJWJJK%#N~&u z`X{f@i9;`Z5<)yJ1mJTcTkbA;%DwORb5=_m>LG5oP^u&kL35_{ZaVrp9@nGp!uJkK zMK7VpU=Q8xL(|Zwc>bJ^V(WmtQbcPYg&XR&^+OVZ{kbM%Nm#PCb9Q>)i0L-4D+lpM zq}$_!p;vp?mXv_99$1#vvqh`vb8^<%7aX1sXvu}LDxcW~3eyxSy9@u_8=i-^n<5&a z(RMWC=<6kJxfhMDcPD9~3P+CX$u}nytE-xHn|^Fvp77TMJ2Hc3{pT&z@lFg;3BImx zVa~0-Hf4#b5FZoUJZDn`k+ zijy@Z>pg4XWRw}U!=!hX5uCkTQfZU+Qe4TY&h8}-I$S?zu~1y@kv9C;FVmH&i~Er= znfPTW=o=wsM&bLw&me@m-D3dcSI6MnC8{W@u#Z89B#IO4Xjy+me3HYlGU~)<#r@C| zIJ2Tz)vHmpPHmTxHM~#}Qo|FsLdMF`xQm6Y97#%%!%UO;3#`wl=25+)>m^1x4))$u ztDlvO9z|suy`#lDReU@!+E|Qcc7H8pf;EbS8|5VHK}Qh5s9#1LIz)M*2kjJu&f90l zdxfOk6Bc7`dNW6M#6mgm@_5F>P2#3)5D#MA$bw7=>ge|he&V}zu=jB~$>-89wfwqVq3)VT%rI@TLf7Uf~Pw z@|N1;$7DgmvnSG;EWIgMj6zQgj*1uhD{hZ;U^{wu8q2>*mNCm9%`ns!FzMPUG+rdCInC9ja9rASdM@HIXzqb=bwaGM8vFB#Hw3&6-LB1Ui*&D8JQtP0PFkEMQZIy2saq5WL@W>U71s3pDgqR&zj@uE$ zKD}fu9wv)_Dl~JT%kYUSui=nrtWxi&raBLLoBLxK?11=XBg7F)pa1ygD~*k!2+j@U z0{VcbmjEKK*C8d0W1j8L@HC!BSa&#%Fe@rv2V@y5=QVGYh zX{$Xeh-+^t_Z3U|W;x@I`^CxstE1O^^>5D_@_lMjRt}}JUIFhgK`mN;O!DTnrpnTH!_O&WX7|$13&2d`|55+8^`dPP4-$^^J z>t)#tm!AjpeWDz+i;y2oi$YYoQv`$4g@Zbv`Aw#CY?BI_0r`l%;FW!joEG}Ps=tVK z74Ot&-8h?O_nmr^1AA}KAs)A*Wpq!cE1pGdWPZdcP8}gtL;Eq;pOB%nRn<3ngy1l{ zGv_;|-TmCH)!twYFH2BV^;o_M9cv!d+pHpd22b$-Eb{3K^A1&b7}xPvT~hr~Q*9g8 zI9d|mj#mY;4GrgemG$W;Q3Hxr=Ti!DR(0*sh>9v%+`kjqWb~rH=mS8r*%HzdDG6h# zOZsw?$$5SM*XIz`&-@e9$|f)xKd*T2FpTR7vdFnyZ>>+KmRs4{B}4cl@Z(|3Zjkrb zgnww$!*@z^I>68iEp~(68~Lz_BY@l`gDGX7@VmT*y?xC>yRFgCI(5jxavnv?*t!@Jtbc-^7XRSW1&`GkZs{5 zp2h%$Puf578vWmS{c99*eK7Gs`byLAo0H7S`yDF8LE&FdR_bU9XUbq@!RubI7}|@O zea8A`ripZd8{bQN~l-GH8I|uQZtGT$X}*-f|lFTZ6}X0pTPS?Kir(joB#u_r>m{>ViH6r zmpYQ?S5OR1gJb{xkF_ttrFA+L4h(Ae>PUSA_GC}Nl(wTNMc_H0{NGtOWOF{cC94Sf z`~Vb(yuSZcw&~;CHQSR&B;Nl;&*PtS#Kr&1L;1&A>*ar80sh<7`K*GQf@XX?g6O_6NlNW}joxJ7oq(Zf_>pd%izD?XogvH!MOYCy41`1NX& z@Gyytma}*>hX;y%LBu_h|BD?|%E^=yNcFShok+`c4exTe$iHF~&#rK0KEEjyF70$A zl8L8dFuCxRGuzr|VBtr-JltdnXPNzt7WEg&ke?(KzEYzGt)qnD1oP&L{gez*h8B7_ z?s8qHKixOA|JE}Ev*9PXb^LDMmV{`Udfn#c^mK7)ZM+bim|F({Mq4}iTB z?_H*kC)-Rh9sgS8V20UgyH4+1v0$yd(yXjIQ#yIgvY~z#YSo6Ldxwn=ZPlxnMAtc~ zZv}u?)&n&5^rVR}+<9#&c_t@Dzao8j zGo}XU|MBEvl12m#Z<##H?)D<_@0{)LytI-WNFeOVn9cP;?$V7*HgE*fy=*TuXM%$j zMv(Sep%%|S$V@1*oF$w05Y|kAwFMCiN`%&?_!tUq5z8U9iIue&G9&0y3v6fj`IZ!v z21w$8q{*JvSOlFnkc5Ms-Lq2Ce{Py~_f+a|qm~O0fmH4ZB(MjymMXCZi1X64-nB3p)gLFI zk!Qct&(=-TCVTJ2?)Japl0bxFuz%zZ&sC1k>Jn)&&wGZK{?@P?S4-Zp-d80DU2o~) zdsdTTngI1}db^C4M%Bc-BqMFjTX1r9uS$Ho(nuRW5lE&3U9u<0B_SBiJqYkp7k({` z`EhZ@B;N!-dAwc(5FPvS^N#67fHZecSg;!y$}^s!Ghwi(Z4}uOnLv$zc30z@T@ht2*Nh%I{Ft&bW!y2h&>Ot#UYww`1o(>oNd$x8tYo{*O*{un$Nr(EtQkt z&f$)~e#c#|Nxcis9S+dfmKjs`q5(MNk-^kENw>ka^zLk?g9*F>)Qbvl&Vnw>R@-t< z0Mk|ovJl^!PwGqUiLOv+#S5W%%GK}7v0c?N_GJY676HX( z$tw|OLmOiT82Ak%&2>KV$thAzhV&ORUi^n*BI8D$!GvfJTz)$HXi7vPv};RFv{F=7 zV}m{W4&!~$e91wDE%TMUINDm-T)v-o5SNLBL3H_6tKIP<{$1;IiSw5+X^bj#2IV1K zG6fH+_i}CC{Z6QsG64quB)sk#rq6F@|4It?7ECeH{V7fo9R%L+$o$)&cqNL>D>N>R z?(HrggR>Sv#|JpYvU6eg!GdBZ6y%BBUf$b*e!UpG%^=NE{( z)NlExM9tHY@l2IzKT)6jNh9&AqGN;O<~4>F zMf5yFf+o7%#@oV(TGaRekzk*_k1iC`&{#;NBQ|G>otOWW zoo{Tp>F);{LaLhT`$sHTc0l^Tk*>}PD~DbYom?u;(A8_&5~_yp*J}8b3Dnsa*0*)o zmlSupeVQGwSwc=$vS1x$PhbiKuWf(Vrr_W$QlS@)$X+btB*FS%mPqW0vgqm)gYD9v zRxo(`cqhQ0FI-cLf95Tr95Bv@;-B1ZNGG6a&P~nXd47rj%UgK6XTGuNd}WIFy-K_9 z$dG*k7%XqbzPKfN**`E2BKg)}M4YBCQQlTq8VyS4=k4BTLc#i*yGB(@aiTU-Nn1zI z3I)icRPAJGXw-IsmT^mLTgBfgQ_9JgKyt))!rbLw+1}pqRi-KW$(1@>%@Zn0Q^^4* zDuxG%$;0`3EYdX`B?U%D80ZuK*UaM4*27XN!`$8e5Y=V3{?JQVQqpXJPyB<@JGAFC@`^lgD;&uWh@V;l z^7riBBk+4-@?~c#jel!~0+^`gYNaSqm+IaTItmf;0N~@5br|ufDWhwi=2bHef)~f; z?SPU>78fpO%Kfg60RpiG71*_20l>!JvJ6Q#ZQP;zx1RH6{?nE00&mG0!2Znv^qBBne@Qn!ijtCq{K7$dnqL6$fz(0b z8|mXUZ_st;d=BaCd8)XJ3n>CNs?ifBtA{CkPLVljF&dy~;g|beewzsSp$n-d-MzY{ z4u(F;v9;3ToqG)+GI`JiG{&>OF*=h<&k%2ky-ddd<481G_L<3K0V z?*cjpobH&vl5!nVO6yMDz<`x?rm#U57&3Ez)q4S}w#w1fiN^7a5S& zG$OVg)$+*>z9v#pKhJ5Y7hF^5rQKiV;nOaEf89rp2q7RcyXu1MQ?SU%rv{EC58Ieb zlmJE$j_ip5u{s}{L(<0*d2e$ZqH%;wy!B-q=}CSP9?Gl)uE~h-AousHZ&urLTFKKA zGb^D;s2&3=rdd+^<#kNZ4|d$XUwAjsbqX+xFu#V(Cbu0qKb?uch?D@-x|Lhdic$z_ ztfXw1rBk~(U}+YTgC^gXx{-at4_hi_E>*sN(T;Poo24glyzW&~5%PYqbN0L>xv1g@ zpAq8Rnh{G&{@C6J#8V&-Qrxd#fXZQ0wxDge{mNaz~nze z>qvG6W`Wm%q?Ey^HIEPr>C6!Hg?JSzBn+w3zv(~tTusT?3g%65Isc&TEq1D6eZj4ZX6qo9z5oYiv_Nif4KmzE z1+u=?r(2CA{5-n4JmqgI{o)ON#Wi9@ow7m6D_j>_K1mSVlTakeAg*bJ&ER-KNmU2S zQ@VazTMjPG?;uiyIJ(5?jBa_2z};+$Sx-q|eMa6rkh@7%yQ4dmSvVqQch1-8vbME{ z_chdc=6YHDT@ziA0%${p4vH3}h z7pdoH-;*1dZIsCx`W_3|tOjrQVh58$t`v59nfnUUGcFiHElX5~Qa%kIZDjZ~qY0Rh zI{-E?&id@_m`<8V8ym`gdoErvjAQnlgkYy9%qv-jF3tCjuYU+vy+#Zz&-008y0;#D zVDDua3QujG3?i#7kyKl=(=R>zHZvUv6(bJjPubbgH>4%Sg(jP4-8m_=Dph?=moG@ZV!jk{YqG5rHY)0t_8?TkvykA!jGzgkxz%-{80qWY$} zp+B?f`wGW3$iK<`MkTTLuw~)=O+T!Bf{T8f&RzXiVSre1+t&t|mcnR;q}1|G%s7$n z$=Lw+Ju5oXH8Eo!7*BaR;t8(BjSG=JyL17P?i5Yr!*2FubWnZ;TOqP(*_b>(#-vNF z^}>v-+|LJ$i=T-6C7_5?bHh{5KcPxf@!>80v?X~#4v%I2sz8;OkLYK_W66u!N*T#Q zm1s1f`t|45(zfxz8uPc=mO3pvp4Kfa-i zMWaW;V7~-!j?Hap-e0H=J^6_X`omjkO6@V7;7iTvg`W?NKeitUJbZIOK^~M}R9NOY&g6>=sX$*Enb6C|mqF z(!T66QIOP0*g^O4iqN0G5t%E2*BO+oikceQJy?yMcbsVo_GG9lQ+FE$oHtiP@j@~O zul}Q+LMh|t5wE)FRWd-MT}qiNWxpsS1LL|!`aJb`5xxy!xhT;b&QGRLSd2LgmDYbe z6@R8qw45A9DgMvnO{Ds~SVMs>g>K+yVzh;xM)+|rowF5k5elp4N+?885}kcL$LX$p z^z?`H^=G#>wpQ?;A_vKCa_PA?oy*Koe?SB2OK9lJA$aAYM^HEqry9UwEYt0ArcO`O zVWfyscWlxa%#6{k5{*e9qnE~Cx^E5_w<8+TNu_2$*m0TCn>+BAsGY~(h#z%P60Wv+Z=lDg8nkcWPJ;n+%fWcnW zrZ*N@0X*ri887MRi965B--w5oD48SiKj?bLD9NH`TeM4E>auOywr$(C-DTUhtIL&T z+qP}nukLrwxo^BX-i=?m^Izs(F=Ix=j97m`5MD{-5&F}MEKQxSLZ-iz=SbjrJm>n` z1Jc4yG77D+y_hWXxE$8E&BoK$(oY}$Fx%vsm_FOKWO4QsC-Rn)Qn4nK1r5zrFD9WL z$lL10Kq4ImuO1w*-c^vl?d$`smNHFTr^M010}kOTfWh`+N!W5uR2-uhIMjAOZNx+jl5>c98H<8~(kGVgI5Z~X(5J7S_tVWErOD3%?q{(W+IzQcs3ZKUC zq!$DT+B^1YAd|zSkM<{Cyqs4i3v;&ZQXePk)$nuh#V9c_uzY3h^AD3GLc-eb+>|S> z{|8#YG>iTNE!1lANoqiXBqXSJwx+{$BFc%i&stxNnS%o!ex-ie#PP#$PLv$m6`Sq> zokXx{Ws*D+549A3F!lQge7Qzdav)q`SD=|4f}B`9UW4IxVk#k4CBX?OQiD!5q`qaw zmk?HLV1X`m5a#aUl4DAj@BxKlwcsQSesLS(OVifxitvd^AR{EkrKRwNGWqq*JRIfW z!tp&twmFTny6FDoU|Q5Iqq`5K9xK5ghw&t?c6NklwWS-w##C^}VF)r#A+9{N4i z@9?B3G+$@ zXs_#sorB2do3#jNasC(Ea3DBX%QTQW^4P)N_n2DUe!(aTMjUMxNb6nS$uwMh6y?!9 zO`UJEo3mg)-Fj?Y-DhfnN+D_U&{Xlx@M*06Lr)9Or{YY4KpHNiOcZA5Dc?r7loosZ zimga{A?6(47fUGi^enCCIl{vP`~1HQo9p|<9MTR%k3^}53i2d7M%RDEArVamgtH6x z0JzTgYZ9KM?rxXCZmSl8NS3+>%(`EV^Q*h`uN*nK+H$fAf$}>c4S!X_eIv~u(S^&Z z8XpKLloA}{bSNe}hdw?B8HOd$xe~8*HaOgg@xZ@jU()tC(tZ|+7g0p66akC6KKE-O zNwgMlF3ec$DVjbzh)iwtp6z?6USV62rjBYC;CmP}*^Pi)DX{1%qyzSYo36HC{@(tdPhcZL-tvV?$j-V;v7iYabA zV2fW(9KRc>LA-Hr<}X6-75z-ReCtQGRwB{92W*JPt3~NSl}jGZl*F>{ZiVA^{$@A< zkYOaYX6pHF%P1y_y|N0|x((>Y(W{gqqF4>@2>>H(7t0^g`uD;HMmN_*!utQg<%sF> z?`m0tR8CgSpBKzmjr@Dae6^T&dnv-!Y&c~jSC8JWJ==vmKs*YA`wM@;8Y9KT_@@?7 z+vcM|1;>5RjZ5|F*K&B!6|Lxix&TYlY4W2yFv8^C1n!+ z^Y#`|9Vsc=N9!chYgCc2y29s~N}aY7;1i5w9p5GtWJ!)@R{Tj#&ufF9jAX{C_cn{M zsx837sA}SCh#4}f>OdfFp-5Qn?}8!p0X`}@Cg*J996M)H$6fpAPRqbAyH(uu*pa0?%)xocjx0H`dxNWrPtZJ}OBKKHUB88(_VyniR^jTKg z<6C-3yKE&)B}A*ee%C>+$s8ES!E?A`2c6E4Jg&0wzERukqo1{cchdEZ!Z2!|Thb|F zu0;0o@^|*?lsTLaaxRV}&5j12pBOR)#ED77kgMf$vxpyr_>7m*5VolPPipZ)UfFIb zv@)F&PA z5L9Zb@V6~ukgV2EOfE_-&VoEwsCb2L_Q_wiaAmFr;b-jT(C!4rAd`Imb%Y1YJ=+%B#~NHRte-i_PZMtTvPVz zlv)b4=(uBW2gt~Oo%WILO_CxRTTb~STV56w!O9R{e<#!+hA}bnDfDQWa`ZAJb=s2^dtIAO2@+5Cn2RaOB;Mq+ImowGf!DC;q**VMtRD!lb~*Mvd@-5q zOn$Ay-ozRY-~Vhey1JCc!yGqjrz6h@LVIvUr#DEDF2d37SDOU}ODC146$S0Yc_THM zn;qIjiIAh*Mxkn9(qHo?Z5yofdUhK)N+w!yGq{M#Ja^4poF8X3Dhhcb;IXF)-;>Tr zn7X@rGO>{+B#?*((^+byBVnY9{7jlnnphf{*RFQ;w!3qXF-peMtb8}Q4JoKk8XWqi zI7JbY(Z*mrKs|I&Wn|{qX(K5)hR7V2MlWGXB_3acc^`APUSSUFd?7Mj?Ssx?L~O+9 zdVe31O`O7(#;JNRO;d@-MAWb&VNXW)ur=@Dc-koCLTbrBKO4E3o}QX)knb~!f_zk@ zn3gE1sEGSE7>PdD4uX*Hkk%*=mCcCAaEIi}n-)DF0E-)Qkn)boVDf@t2*|Q(*1%sEWd2i>gO$EFcmS+)rOr8lv#be;0na5V{~ChFSyb@ z%TuH|D2&tI_K*J+({nCX6eeO0nh%5l2`P<-7E?1iw+TtSVp{Y#}GAq}8T~epbd&@*WPY_RY{Px8zf;ae+WB#K$5Mi@p zy-W!k=r8S1m=Ce3eQRgT!uN-PYi$%Aop}y~Y^U-|stUaS6@wmO()Sv?I2ep+K|oj_ zG-PyFypz_H&!x2FMq)+*)ppe5gzXzB-~J#s(tKknMwn|fC)4&iSP^rK=UuC&Kw6JR zfD1USsvpNn1Zc}NDg5U0y2yJqiWHOr4@$F~%^-9+nk)HN1`S!+{DJ(iMq*o6xms=4NC=+r&%~k9Y1r& z74t4!0WJg&82Vxtq1U&MH4H3{=bI8WaWz@yNtNZ*DF|8E4)ErN&yMqqdK2q^(r3Un zdmEzHM=?Dn4s4PR!azyxv;_x@UaYTh$sBL)AB9bumN2|gP*8HH6{-G53-I%Zf|5Mm zI);-=@{<$`dVk>JX515T{nm;t^USFIB0*@ z$m`t5H4fJ;vOKyW-B~oi_hN94xaGcU^kwpvlpW5w2G<^`%UNp~7()>c@;*K1GvUW0tBqtAJYYzM1&x8)tFT*edy8>#R$`-&qjD%Hh& z$U?}*wMw4wwxpL8;i!Evhx1xSSW+~C)z^JO`$Gr2BjYY%uvHF`FrVK{^EuFb+E=^CvNAmB`GZV0o7 zPc6_E*c`bQ?ble)Q_grCur+{DD%H_~6q}~~gUt#2-3L+?+bBpz z5&2*pezhsaME(ws7EuZR$Hrk%o5RD*fm>FUSKWmqG*qLL|5C)xud5m)pgFuhcu{;i zu)H%?ASaWcGS}S1-@#lfRCEV{$697pSBN8N|kK5auz|ef}01+CO!NGh~a@6CzSrSW`1I zqWdVNSRnE;}wQ~j(~Di>7U;rod| z5vIqAzofJDK4HXg{rr!&k;+yckTVg_HG0v*GajDKK9csUg;w|n9v#=df1ql6FbmCr%B^S^V=h%uGx7iNd6{Z#eYcaEnqCwAGJy4-+KY03OUhszB84u}+|B^s*v)jtrq|?%wz9qk8hv zgn`Y&FYLQ!L-uypP_WpXQXYZbs_`AyR!hjS zX42d7(wF-|y46MGY&jqlp*?HDev2Hgw$Q^45BMe93{d$Me;4+#e&Vx2yO$KMol; zXP}OYG77PS;kn!gikCycgXP(crQ6L2a@I?5E_NwH{GzmAV8(AI_B5rp(Rdqoaiy}H8=0VFqS=4 ze9MMJ-|>+J?8&U*5gsbh^_a|% zN8zyVBL9(~3|zvu0gDK(x3~<8$zDUior>1`hlpmnj7@;>Tm1Iw zz7LiV*@l#!5yB}MoBdtKU3N#xtYN|*(5KuXc{Z{?1>yAp%7)K1M#J%u?G0esUR#ra zMj*78%$RQ1NKuKjwLXW(LC51KkDT;6uR01E332rLGtm0f1a$3xt3i$%UP#05`u-#o zu-!cN?i;J$-?DL}3T1Zp+fJ~DY%%7ByZ+Ry9mk{;!)aQm>ui5uMO6%lVAFw?TR?ee zF(v^JA&~^23udIM$SwJl|5=Ed&pjC;m3?L&P!DEbddd;n1Oc+HboofvQ8Af(d^t{mi zJYDxg*QI?58&bSu+FxlVNV+Ln>k^S9v)+F;ABr(rhzfOB8dXL%i*rBl`r=+$AdD*+ zQj!A5TPjJBKOQfgV69wzN=fjU=2Oo)p*vk+1uhR78E#%3`MRg&-$!aZ_v*p8!Db@9`rab7bi9&3(|%`^Zak!oMUabgiKu@Z z9;0G#yzWX>UELBk@P1`!{4^e3-$VemF)Ar!@lqUjAKbW;o*`nWUy*LPE}}{r!m1Ah zfT5&?hsvc9BUZaYVzDCr9H|O2$AcPn(=8JCF19+1u8M;Hwz)a@A=pldDGr0p3RjfF zFra%44?~L9NV~o3?hGDmm0}=5+)p(7Z+1CDHos9jvwNFnkP}N1OQwY2`!|H|&C-jp zs%K5&76#)dMbebd5iKma)twemv)@~vfA*oecROvCsy>>4GsmmoWzb6axY3$FuWO&& zk!qgaT>oUYhTYP}S828;Ct=yVpa#SI0P>^tXh{H5FVgjKb(}ZgpVoLh@M0c}g+HrG zKmwp@j&sJ4~Y|-$DYK$=u`_OcPZ>srky{Q70O68lUX{!PbZf$w>XNi_?he z_O%9P70f~Y=k{x&qCM`>^y09qZ1k^_zz2$z-UhE~ZcLRyY{XVEzk_Ag$QGwNL(>tx z+x6A}t9u!7r<3)WsUI~9-&-t3q|-wz_C&{0&B5WpVX6Vdi9nbc>F6P;baHVLme$+V zzY_SFRSv@$^IE&@z+XZb;jhtGy~6vreQ`kj7zbKI;gj!+64O+*(RR$;IVfw&_QmIu z&@G+q;Uh47m5DjgbBfj#pocaZ#gBRS*CMq(w8I7SS~7~A#iQ0(9-!KR!we%Vu}K?@ z4_!j&J1P=$`HiSN2s3lm23U=a=e)>xMx-9a$q=r(&baJQ-4RY73D%PmAY4}F?Y^)3Tf^g;`olkwo8hZ;Ax9=?v}H^CTD;p!CmVQ(LB1rd5v+=gb2B= zbze;><|*fwq6H!nb=XRbx7xFHFa!sm^k{Q4N7`^6dd-oXPm9`uxc(w~f3A@)bI2G` zKi%+N+KDZ$;MTz=D^*vHnAB{NycCd=SFPhP!Znb7v6P@{XTi8Gi3<}Ws|)&sGr@1> z6%>JuNaU-3-kP%E24L1&f#{8A{pv(V;RZtEqH#1~;IJrNeYn@lvN&tMSCg@->R@fvVnekr&i_dL+q)=r!^lBIi6CdAHWsxFZW+T!QD5~ zEQ-S{hGGvWN53e8{dUaO4ey)XvzeTLXx%?^x1T>Q&|W-wrqIje6aY zu4(HoD3@*eqsKMi0L$?Ju8ib-^r_1p`x`B1IyvE)RaNCoBdc^9X5GdXha1fq#65Cw zoX#PR`G0k$WacL>?PN@$XV!Q%mKSLcwcv$qVKhq5s?mRsA?&u+(EFy!a z%v7Osm>oLGY51_OhblrZGncn)jFlShATZxH4hO3YSS`_(`{1+$-|JDn8!nXz0#bJA zz-~_yr<@;`0Rh6{5t2majn3G6V@TVew+j$h)FbfR7`In!RqCuc)dIF`B{~qa53Obd zOkzz9?5Jsx7GO2)8u56;qxBiN{oAr4v&vV|F$65Q&X^ula-Vf6rNd1YpGMLDIlyBs z1%t!)8Me>2LvGVd6t#nq6^39TQ}Z5=D-?h(o}1rf8ekV7oV&+Ivih}fGz4Z_72dvc zQs`5QnLTCSP8VRpTDHX0E0GRIf)yn^(WgPPzu_8%ydNBxZA0m&vSb#++RL`$wr>(> znPz-H$T7QB^&=x&q6OY}frC11(D6Jn&cwEdUiyI!u*>vqZ8T_m9u{9qx2LqQ?8xKvtv2yG0yz-Dj5xpF$zT~XQj=*%WRy7l@0K)cehploQr-_-zI6F0&FuUg0Wu0t|o>+p*ArsYlL!6Pyxcca`C4!l-%dkPS)w%;mk}%EE}|cB+W#}Hv5YPiLE{}KHd&1sINiiL8rie)EUE8 zTs$pqkLweE+K#k1|EpvEqv6WcS*e+0oOO3L+v|_L*)c+bbcP@K(>aScdfe zGTaxx*O2+$krJRs2Kp{tgmNAq!*VNl%!6==#1pT^>+hhp^_X14BA*>`s%J;|@xYo< zY!sKX7H%Cqt~2dw0CVKsr2;U)h`5UfdI}~$krlad0H&VjzFL8Ajc~U|4I1l>OrPos$ z%H+m-Fnp%q%FRm_SUBjCdi0erabeCgv=V_jyQP)Ey$m8y*}E(!bYT|$;nfZaZpS4` zb9?ye@;fbIB%~3GW>>{97%$`S_d{nvES4P4CI1lGZAU}D`z0w4(Cl>d`ddoVK_Y2|N@7T|uWg7(6tBfh2#A+Ni%q4sh3b&5B&`|sbX zT4SVY>N*2Tf-3-C7_VMROQS<6dEgq3ws?JO@#At)YlX+|US(DJKQZg@8;Q+0KD*?a zke3%x*8PgYRjhii7>@UFI0z{yTW5McvBNTG4rUi`_Lu|U28r`WfO^BL>q$HU{mGkk z>-DAXN(Q@?V8?qtu}X?>sLkZlq3wlwHoFV1v-cSM86gF0 zu&WHuW4{ilWsz`fs3;n64EuizQIY7R>&ClT)3iq2SmJ@$KY zfBbv`Tz%z&#vc(a{iLH`@&RybI$R7N%-{mou}7siP3_yR1%5vAlPuLpvpAp7pGv)t zw#@inULYZpkqFGHC&j|1(P(=@bOqZ@82zI%Cw}g@llC=KCSQlTpgvs8+utfwPULMO zk8U;L5Dm6sRa=1$D$YpBWkM~dvW-+~1$-Si{iinQPgR0!-3^W&plAbRU(vSra3@5~ zM(mwnpCN2BG56)|rgM?#Q&nQ?-M&%?2)M6^{kv!gbXV~5ihQ)2vmf^!&fUI&yoS&r zOnwIJcSvm>Ut>0!xqJI*yikD-Zuq~|Ig}|a(3aQzdOD_?j{txEf+8~Cijf2pNOxoG zj=&b_x#7Rc@y~Z|jZ_chCa)~0GEzmwQ%8fbVu2^?77~-zExr^J$J7Uv4K5c3+}Ohu zlAa*2aEeARJDR(Z*sLvbc`7Z;nfZF9*j$tS`(o8L#Szc$+wFJr{Ca3^qB?H?t?ZUmM&H zOqN%$(mo8~yKydi^PM(VZlck6Xz%%qe18ZY3bC>HNWHMCHCWwoC3#27h`^B#f=$zSa;1y0025Bjb_BK!ADsc{~hk&;U7At+vztwB9WvvaOv-_6jxIoe*>9 z>F?trou2ha{+?VO$SYFzF8WK*UsWr}6M#Y?AAcPguV?fA(qk_N>U?m&kJ;ep>sHpaEgMrMXith{aQ^xA#8{zr2mBIn zE;rEI3+eW@XZC0uDZm5q159Zfpvdj;@+8Nly&?%9K+=_;=R;9U{4e07%rWL_?6|PB zf&J)YV6yQ1(!3ErwxpEo`?G=Sefm*f%p9Mt1GC31)T`FbPb>$sQspWXP=z<~o=@n3 z{M3o+(*&r>k9}!y)qBRorb$jEEl$nfac_s^1Ix9W} z(Bq`6b@vNi$vwj@n&(+OssjQ+!^Cgok4argobQL!<3^^;?nqpcdbR2SCBkzqT- z@Ri^mjQ>cyV1M9<-JAnd^T}h*Qwj+Du}E}RF~B_~bet7X?ZAy-cOjVPDsSgi`A@K> zb};IK*#dBVau(lA+Su7&0lsgp0uZ~pyci(eK0?*DI&*Qjz9+-akh~dQQGKu6_;RbT zWe4`a)t%NrN7A4eYoGL?1k{FB>2wWTv{}BG@D6XM)pnJgspxEJ`F|e-Fj@PYd$5)s z$dfkapLa@m9PP(aMCMGqgI59})U0nCPF6^Wb1m1sjlzkKP>Wq}0u2WxiK+%iG-5O- z#V&0|-zJi9-ySJQ8vpY0q|xe+AW^aQeg#XxqsyNx1LOWYL+emrj2>1iC4l_@6ZyHD ze;#u&?liG@Km6^PV$&)iei7TJY^drnHq5-_-fCj%7+~^H|I|4Fr&amAI8GLJjF36e=0@0~d#ga)Zi#_-f&h6VIQSNPlmh2J4y5-2TBF*}$sU@%6u2g*jp;&})DFXt}%c>j(TWnBa?>c(LlrLIYifmsrnquEB>LR8m^cDoz4p(t0Bh<6X6UQGF`B zn3sWO=IRK`AED_+;kG0%(5vLFC7w!2vhT+-!G#I`W^=sijks z(fqr<)j(vmyF$iL$KzyIv@uQK{|rS_FF0PEO3AMGX! zv`j}HZbCqU(irJN?=5CVe|&9$ZONjZ&{x4ggEN=pVSvWN3T2(>ri|amq9oFppRz5d zg7e;y`AGtwBbo5$JuBg;vzABH=t&ssB?O&R5#jTbz8Wg2m9=ZDPvzzo)ssw$r{@)) zY~STf-4ySm?xm8cEzQ5EnKAgG(I9MLH{ ze&80MK69c2XebxyL>^Uor~Wt4>S9qP5qF!rH(7WMx~GoVKb}u&LX<(6j`9-nZ%vJ8 z#01d}bx<26L5zuucwOBa>lva3EE_CXf0T~nnMd4N+W~!*-Zdcu! z4II!S+xf)a^oeRU-)+QbBpt67HPUza+F((j*gp82Xy9tzv#tgn-wP!a(Ki?V3T7y> z*;6s`DB?7R0@xbsor*y4$ADOle(tA)Bx*~%{SCxA z#{iR?3vmySEVAnVsUm)B;IDx5ohHk*tJWw7Fk$K}+i-u4DD$qq(WE2FXAk7-CDQ8n z{eLu%M5pf!H%(d&Tla`=Q*SYiQUyUe8fp4@`##rqPwk%^-50t!dKf;ZX)wKBlyJOD zIX7gace&U1mbw7K!daM=VXOwSolaL8jmuJp>ulUC?^|45Ee9Y-h;15%on8TVUGJD_ zWKh~ROnI3EN{>ze9mt}HtDxtamQ%23K89uD>xI>i`ixl?qk+zsP{cEJY z9#o`ZTrn@TJZ&P)5MFA8#_EGY^6YhPpBCF_`N^KrzrQpa-QE*}NUXhqm$p>YOo}(e z#s`-&eD#58U8sh;P%+((x$vZtgbDMM|6elfKyNJ73@t_^g=1uWp(xLALHc#%!yI>% zAM(P-l`{8`E?+^r4SHWqH-X;5csrdT8HR_MC=pzRux!^aTM>LUAxlQb4--Hfbey1E zFLXcrx^FCn$nVH^QIG%80(5UdT#m=}Pg5+93;9pv7|m{M;=VS#GgSY3%nu`mctgBQ zU`T_t6HP%G`IC40cLnlbdk?clNJoy@4=)t{R<8p0t8>d(#(0xu3+kZN-NqAnZmW>x zJH}P`j}Nj#8C6M$K5q`O$cKiey$>P&8nl~M{o^+pmP}i??m~mwYH%Yfa5u*^WE*#W}qll$8p59(_ zd=|uL&c?u|V!>~*v`5MB*-QpUJ?E&s;I^})Um9^L0M@yN*UdZ)c$RB57Dx1uNJef; zFH13jqEk*lUv?LYLzG;>)D$GKCl>AB>9oOMksg=h!|WZ74t9FT`eqMN>#80A6~->( zU#^3G=DQd=3-mZ>ayiVg(3I^$nZ~lRBu#@268QFraaYfOC-G1mU+HXD&2hr~ePJ;k zW`Nlx#Y{-Fb5-siAQ=;q>cA=YdyE5PYX@@7nN8Kf+-b4+JBNpg-d^*U&-*!i?2^o{ zjAXInk&k0-2PiqCr%#VZ!$(d~Ig%$uWhT@|MC27Jv1%_}YjfUwFY8u%kt0DtmOgr( zF2Vt|%n@mNfAaYC159MBwS@#-(x2fdeeR6mQY&byt^GdA85hW-Suk5~0Fm16NmY+7 zlsD1%GT`!@@Qu|q%6BeNK&9BbvGgcSh2-u4NL+MJE~Rd}D{F2Q6Ol*?vN)-D9tM=X zhr)}**F5vhVgnWClfG9;fUhL)dXi1!Nu_r@IrtepOb=lkI#+EWO2(6hNd_0@vX}vW z17pv(IuD8oeO&!sniGWx-_S;5|GE^ldN!zqvCD_T>M1kMBy2@x$_ZKw z{|4J}MzvaTD~^+TycyGANQ1|G*V>zd*~qCMy=N2_U-hVvD*O*ueP|#Ju{P~GEYMB4 z6Mv-O8&7WlCh~L@4CK5m{}1{Z#$bI%%axmwqk;0L#KR*DX1G)@S5F|v;E0Pj*|Bk2 z5;Q;w*9c5_yfTtz#f{Qroqz5ZKt4be)Vb6Ox_w%=RgwrSz@zm?Zn$jXYUA<6d_}}W-;Zc>PErWN&4Z1kI0o{517rKjc+82t?b<)eNN_ii8505=$$#(d z6kxCVq#LLKd7D)qSW&;hXGZ1c2Cxj)uOPY*=k9o91J}89 z!rj}qIx&G5G?uGwmB-2nn?T##E#c`4AOU2;dXqTeX>Vvysl(;z-a%42fu7JXZ zl8EhC&Gel{!5sZ&Ob=t?K0e9gX}Y!BRbI0g8#?3l5O2U$WzoV|p9q|o>Gx*ly6DR8 zIh08&qU%$f?eas+$@n3{+Uk5IP1Z+I*{J%b=Gu8g{g2a^4zhTxuRdfDs5pIyqhxSh zK+MTE{9XJ_ejBZw383@Ow{s@hbD6OOlRh2vZY2)JvvQ-OiZAU|TPx0RP=E(&J=!iX z=i*Z)j7^X*|8a1t6R+f#3P2^u=AT9DRXvL@o+ra**80Fj4b3+Hy)bxQevdQpvpwjTqf)_ul|&59GhD@UsN`Qd}wP%Qs1 zBGrFdizHIwe~C3xhMY~Yx|O4kyN5nkX)OTJW-)<#j?Rw?3Dp}*VMqdS72NbY4 zMq0`jw9I3 z2qLeZUAZ_c9znns)A@O^xG9#RMu;{EehVj1wDiyIN}{FaS~*Q4`oL$;Ht)>@Cb0rp zYbb_I<|Xe$-?yxaBA44Xx_qiXgPVU7|9advIe)eBhJPI-qTxYs#)-WODH|fOvF=STtu%5lE+_Vq(loX1=fM~u zMy^oCDE>Vu>XnH5(K2$h6UKXF5?n?_)pbVgkT3c8DOMQvBT0B8`J{g>mQX#^Pw6s} zDqRYT(x9>=2nd!%@>xej5wj$fQu5{=+fkTas;k+yE=CVhaeJH?vrh9QNf>{j6tKjU znJtO=b7RQ&9e+!Za!&>ATn+S(`_Nnzv`go#%@@)PovE%=wlYtHWa_o~#lg-&VadW~ zSY5A-lq~5BaM2FFOuvT-UN%Evs(VCvox>_1dWPcmW01eTV1UfXQC@AbgZ4IJ8Iyb zItq(rWr2=0L@sXTQ2xhITXrLZD!w&`)UQKVHWr?lDJjd4u^k+n^2^%rL!xZu9!Hf1 zR6*L{-~QARxn6pCf1V3WiDna>^;pbNoe7dsJTjZju{@+RAm~AJ=ur@5$(d~oc7NHM zT*X>>czS0_(YX)JlX0jmrfhXyibuu}e8 z%!NsgwF$f6&Soy(?)J5(v0nVZG|vRmJus~oAZ+-?%l#nu9eu~=pU$wT3B_DxKm-&< z?;=?0_?{NYV-*A#uPRv#MsDie$Yi|z8FH8y6KMSyTSlZZ#4DrVU%fyyw#cw92oP@q z(&}B}Q0pMep5EOtxqXpbk?JSG3se3WIolVfd|d!7ABe|uP1;lo0W6%QX2=*NJa$hj zWbYwj(Y0W_5NRRCo8LvYn>(_~a*rQVL#B*4UC@>n0#!knzW*}qUMN%;ay%(MJ``ny zxROF$-ru)?xgxaF2>gJ?nIbZH#lOTsaa(R~rd@o35`tsa^+g?9+xHjuStk+bYQ}o@sqQ<;PQMS;+UCm3;$K*}%Ybt>|aUEg~sG*qQzOY40(a>M- z#Y`uuCI{cn{FGJyoJzRUzw@ejV;?5w$>rQcp7?^>xDST5qt_P52UnTdQbNR|oga4Y zW4ebSOb+?^jrbBJY8NMTo?GoZJy=_3o_Nu$OPM&# zx(VdgV^S1PuhR5gr>vuoGvkj>-WEuvnySaJ=vodQgA1J5TIWilg-`C(eb>^aPIstf zaV7|ddX0r6M-QB-8$J3WErTEQU{@=X(Ce1}30Ln6HOn#{14`iI&FaELRZ(|}cQjU7 zY%Ofg;C~;CMvn@%zw6XykKr%Cn!D~H^%QnV-L^U)Fk;=pi0zMm>+*$kolffnXVqn+ zn33#hqptI|j`RuBZDAgVHydl`Y^=D7&bF!$XU^T79){ti)cZmacH?2j4~wsR3Dx%b zs6>wtD!S!QW*-&!4VSNG!32xiY}6vF#O&E#=? z`&Q$R<$;Bu(&s7n5e)38mpHvjVg9*A%0EblvcgoW=H1@Vnz%j9Ou%R8Y>f%1RJ-j& ztJW~?*eMcgTL*hk&Txu7F(t@qJIW$;Yd!wC0IKZjkl&HC@m0kTwPr_bEjv|&8GaM{ zRK;Vy$KfdNbc%sbodxu(@h){ju5a=;?i7D_V-B8aci*$!R}XeBp2n9J1f(k)Fy=D8 z$GqVeZ%w}5TF*vn4WrqFwW&ChMk`l@_4IlO(HLZ^lLhK+LKU1mis zG|`oq&X$3=vI97jqD9}Su$6MBMtHqy@Gd5BMCu!69u_Fg2s=j@XXv4!|i8> zI@#^Mx1T9o`80J6yXwUVVJ|7t%#8FM-tEPd58+ENwV)CTE)M310Y^4yV)*YAYE|no zMq@f2*6`dE(3{PhHt9xPQzvrfE7TDynyRX+#H7S|U0K5|I-m(m&DiOp@haH*gv(t~ z7Y?$I8iPBReBHb#F&7MpfS#dhX4%;==;j3L6%vyxT;;%brg?VzG06xS(i>VfU>)Zv zGII~de}pb-4Un))r*ln|blI{yx#%#f^U0PmtSSI!RNr*T)KivMhia-wwPFO^nnL%q zG7-{BziCmI9^K->xX!F7H6Aw2Em&U}o)cc`zg)E11_N@hA5zN@Jjv1AJ+ZxXtE&if zN?N0ahbh#`W~&Pb`H@Lv;N_+$oZR|ki;KZto}hE38m!~*#7H$DE34fsIOm}%7~dgd z37xFUcx)9Y@Yf<^);(;_fwy3P%b|SuDo3dvcO;&r+hk}DZ2A9INgv5c9gKRgoOCm1 z*T7oAHqnTv)`0k`uf4U3#`~J{De*~ zX!^+Q&%9}QdOF7YSKrL}G9?#Qw&w*#kt3jI0Qj*lc6RSUjKmUa6z4a@6% zGh%!jgNQx-Pj}~<#tehEiVE({=J+{47MO_>pj0FX8`FRyroeIe$Vd^C9V2qO9VpOA zd+$~c*Tmji0h3=QJ4RqP=^hz$Oe8nod(RvR7z)G51@`C1hes9)&V3QbRw(uwSg!XWu)K<4C{P{^g8`cYb)WSwWL%CZNA^U&uy?@UM&1tfJlKw z4_|IH=-~FkE%ecrXHpc8;GMdvwHgMp6sE%+rAmEga9_CpleP-(=N3NEA}&jtGiHY+ z*lJ;})Hwi95PB`yl@z7Jma`9ie&u2aBtp{LZGtXD=e%LzJwf}MnId_Sj*oirb$Et0 z&++1m7oy{vx%}Qed~C#{P_NX2jW}@Bpmwc3W4Ov}{ObxYwsd`Z)$PFAKp zWe&qV^&rIXw&e1!M2Ge=7W67{E*!y0L;Qq##+7Foeyo4%;3 z=*o)9l`Hdo)|>Av{?TWXah9do@cP3;)ry5vRw7flV_N>G*w3fRojyYWDe=cU@-VSBWiFD8*Emk}I^5!T`ZbMXQ{PB+)<45w}7|237$(6tS#$>q7MFdumKqxA;> zol!Wo6Y~{{nlah~=QcAtjv%rTC^f&;XPcQJ9)v6cDYt6^h)wHNktFiXSdwj)_`$|wK~e`LV7X0`#Ac2wXK9BqDX zPBO~uxlR20#?DS?9Iu}+?eN2Z0{MqiJ&>#*f5=e*&5(WswZ#aQ<9=wuEjF7YytqBW zy9@J_3rg3dzj!jgfpga7*9syRH3ef(;>}MmgMH#Au!^)UREl0nemAlpS zC0wgiUz%by)!sfbg!+p_RCl#14byLhmz z4W1OuSijIlDrXuig_hTT_i&CL4m=5RyEnL*`e-i2nHf?{Ti{O_c(t21xJ!tPf-fDx z<+ZxL{|-${J#^F*Vudhl-OnC1fcd37Igq>lIHao)p~FzDK6ZoNpaxttw_cC_q~AYRsD!1L+$u(41DQQU zrHxM4+N`|GPRlqLxLcP<{9>3?jam;rY*?!3nXIkh+doo8J?6XhYBd|Mx7-A0vx6g3 zag#;xkQ&rl*PJtZL;b0>d9YM(*j4fwD!+pd5b=6%`Me|-AkZ+PDy#3e`lNd~!(D@2 z991ne9m{h0mmX>PYr~uL5KBYP4g2m*)p?Zchcekw{|{ zI8{N=B6h&#{pfh3q`O=Y#AQcW3z0gYl^+m#5dOHL9*@=O{n+I$4Xzr#HxFRCg_Zux zX5{6_rYRXqOG6EED4xO;^LTD=p)KwzCia;_U7LLh7~>ez75h0}+`>rg1Ti4N|MTV7 zKS5rZWnc5-?a9n#S3JiQIJo|tCDOrnpU&UYyUE_??x%B|xPbn(MNq|eb-V8)RVdvh zf146}q%T_SkVRjlN<{Y@rSDqHd)!rn$eCWqZKXd@3Fqc{PnGWsV`OwVf$mhHl-->m zvY1_es#~|c>CLbxED|KSU+(a*=2n zyvx`p>L%^^*zi}HkP+c<$Im6#uwK=~JU$hMpwc6d#+~xrFk0a&7co1uxLiX)37Mea zV>L#QxJ)6WN_Rc>=vqfSn_Qo7dEHxv>Wp7{MAZ&vTFFFrVf0`pWn+7AIo7aBJcaDg zTP3_p|4^`;=VBXTk(^u7T7tq5os&0870+8q1E1bbT=y8%$N;z-Zx}E+z_l|Dwol56 zo&>9|42($)E9x#VzXt7dsv#&UoG+^8mtkV&z$YvGP&?h}r?N^~ay7a6_M9cV-wPXM zH~)Kz&#GHo6AP6sY1@68N#Mpl&w~Veq#kBtv+7s(VC0))-M03m_Euu{JDB^mEBDz} z)SigRbD83IAvdtrF;jwU#T?41lS%^5conkjJ8=2%Hv_m!Hmz<Vd-l_(nrY+fj3h*W5|W(Q8=gfsoG0VN-2$Hodi=2^EVv zLKi~s1b-cIg^i(r4A1ndcdfL$^;GM~n##asbR>l-P~Yx5l?L4V4lqf#@dhH45jEUIqCC4DEhfG2X2s0H+^Yn z1c&-a0eQk1<$PzNyj+U@Zn^ebNfMvlOq{oTr*_QASBQlC>BA|34ZYz!&7!}l$WhDl z$m!HDVlRR-YB1p6{B!e(0Hte_fUmJ?1+7g%DW()}_Ap8>19cs_^WshQuLbsE{ovC- zrK|oVau=(9NCS@KWEKX);TfYcGldk1bgB+_goKyiOdnf&5BJK!hMZNUgDxPQmz3$U zB==rB9cZ}ZpWbsqW@@e;wK}j}9gw0YNtKuXd96~1vm;$;bN5~tK*`FxPMUwFn1UJ# z;jyhQMV9rDPg*c-719*1jvPCwb)(8u|MaiVEB@=xHqQry_=YJmrej9)4TkW)z2*l} z61SrV?0<9aJ9v@krg)U#uO@)uWBc7mUO%dCJi>+$CP z;_^`RM+#*WU9#9E_OyMjB;q#KNP&ijs~pmYkVQp$10tX|&REm6W^k>;PEeZz7FjPK zff&&2r0?=k({hC;Xd+rHnsa*Bs^p;i`q{Xn;iluw(cMZ4u8@7fGv7UTyQ{2&xyvN4 zPR-zyz^%grFCUQ{^3R3`{EAI`LVG)NxV%H4&JRcfsa8ZVmhS3Cvc>X?jD3l>JnR8c zjt{x%L$|y9)R*msdS8;H9s0zSb;zYhsFI@{-J1B< z8*1E$rnjL>U9r);Vj5*HLyQ2m6|!RLOfSAh`rC1kRYUc`;(28uR-@Mqh-{leS7Nv+yQ>P`|thqJ#JUypG+ul(RR48_(3S7B`^^+l#a0eNsd8KWb;b>D}e zLJzw4widA#{3xS)_?`3SYaKH?3i)~eW)=UU*`^4urCTqUPJC#zz%_L5W);*T0==OY zAgxIv@rYppFGR5oWqB8e57FSIoKUeB16b1){<-HPaO@#pZ^jW_G0w9i_8Eft6kznn zu1(Q)H-&RxMLAl`#|rLYqw}X;d!S+ocq9E-vwaw`cPG8aGF1!HFiWvVIEJFFg@~y-FaP3I7RymKLrT zGIoUh)ak`>x}Gl`v;8?EPL@|WTnGG7%3k^;0mQa&`-I`m(Ik1%$4a2ZVbyqNH}ZrU z_PtK{{f8$JK76#nI6=&3TS*u*x-%2%&?d~iMOKqB{;Ka>a}iZ*I5#8egc>HhMbfJB8C@hk^ElxUY5YWa-x9Q)>S-8~hVl(MiZO@{JZo6?=PfoK&L*;^}4>t7&s~eRXaW#p^q@s;?KdlW`FlQrLYFgjgApL?ansnV3 zMh6vkq&H0RkJ$fuRNGo;!UlT=nqE2ZGvmHK7_5dYf7|PwebOMu=bSt|c}%)B?8v`0 z<28UyT;=b%weh;``)U@YlZ1Q3hhkb=WM=Bza zlf&-eO-*4m+h(MrzV~njVu`&O5*T> z2rY)qB)z#f%yT^YGQ`~Q;<1ysA&5E8`k(@CIbVJY5Yk}CHJTPWEoLX~)_*X{M2DC6 zGp8o0wo>?);pC5#>J9S|IU<$YFLl2jnl)p8G;zlG6k{tbiN|Bmil|8u>H2eMm{N?8 zzVrymkIz@;6)ZgN=7~9E@`Aavxhv|;wJ5JeMd6pUNnkrz96)$pBwZhCClZSHAo3Qh z#OEVpst~lOABa}({VJeh^99kxY#}5YZKO-j+gu3EyX~~!H;MdJ&2DEXWaD>pHW8VU z!jtV#VDVk>@C<6!N1P}}y4Ic9r=(8u?DTHU$9NMD-)WrX`GU8h2|AbG8Cv#IE>jHs&UOa-^ zl7U#TRoPoa*!yG+cwdz9{N`vw*t!`%39KR2_1O%Cyrp-lu5;B!Aae^8n#+$PuNTT1HJs@!2~m@Y>+3BRd1}r zPqMW48LyQ;dpZQop5;gPd*ZXPbrz4Fiy-ykYm#EibVRIVtOLghmLKpn8(D5~w7yQD`~{rw0tC=e6MCX}=I|rX7=HE9 zlt0LDJalbGbqpclhEkZC{r$l`rOk_TK;t)YnpPhnZ!s9>(_{U5=#N_gLo0XXA;iFs zSmDE)OP0g!u`@^iY%s;vsLL_flZ1qzy*iP(guD6Ob0w8KMEcFSU3&-@=BBIgNs%Gi-*TF|^E(5+sbf=tCHHZfx z0$%*ottEZVPWZ7IJqL|DgwYN%j!5YTu?t?v+nTXIc<=CIgjduq5s)QZ?DO3-Vo*QJ7n>$mORMA`EKUR>z5#W4W-d#fK-LAb3p+py~r8J z=7quMZu>!P%Ef0%Y%b2`%j$&RK?!K?u%k7#vz?sWRs}AvgpJgwwh+aRWI1?gI$bT5 z&pXvlUoI^!xoOyf`kANbzRvk9X!!m?f1PUBeWbm2e~u$z*SG_^h3ygjaXTMHkGgT5)QCI9r_0egFHgmV1%V z#ZG8#I~g-q#fi2W!QaTb-GQlXw=V&C@HN~L5&WtrfZUodMd_lZL+0sJMQ!hash725 zQKiT~61JwYwOGyS5OMr(+u-kx&+;c{P%eTK8;i-tOV299>GCn<% zBWbV^f)VZvz3I@Tt(BqT^fKiaD;PEAi?d$mZYCwmX zR;`Z~*r2P^!HcfPK(Zh}v)-kAC|cd|+SF6x43|4AYA_dhivdY*fH?h_sTvZ$4h4H@ z?;A)bdGtev()x)gB=)c{(Wgq?!C{96uF+-S1@iaExKGWJLC>d}&&I!+xlnP&c-Q0Y z_T?AnMvm@$WsY(F-aa{yV>Ren2WHhn)4M%Y1OHHX4(4o=45#yW>b()|@GZ?_W04(` zTGoxkgcOJGNVqT8c#%%-P(p_=+cK-IkG4}M+1oqKp^mj;%XMtQc7*520~W!$N-R>V zHcIzc_r!TgY3|uq48}3_OLnzd!nUIMaueZs6(Ifn{`%XlP~P>nWRy2Ypx$fjVX9M+ z2AzKKYG2LGPHXfVfm1}EB*oJ!(9{&cF1^H#&CMI2f;_~T<{o7uxW)W-jl;f*UhrIx z47Xn)I}!~aqoE@#w=$JUsTH4dI#nOtU;Q7+p=^JQoM3wXnqv)CihYd+yxXe>G4U;} zAj?9D&C#IyYd0AFS%G!bP%W5{^g7=M_og3P0;rQWEbC}ip}j((sWbWZM9oBUxj*!g zpfF;03e^c?x~u!B+39ir*_&D3@O2(B&E*(I`F#A!qsHEj8S(HfK8Q{wPcRh9BDpi{ z@x_`#0cO9t$BxnbHliOM+DeHy8-E;}%C;m0nuh|@F1m0}0L+}+eMuq1ZXUUpya~6UI^@qK zT3`&%1M@h2d8wXngsZHOXEk9lXATLxJHD5^=M0lxi?=X)5&4ZU{IuJ+s1&97NpaBK zllr)xbI^vBJjH}?>{!5WP5YhM1t!S-`1QEe&A`lS>c$x_fgABtR;0P8$LY2hvFLgI zE!&2unBgp(-W_JOgNo=k37_S9aN3Qp0LXR!8%HR0PT1EaOd=TwIlZuUX=^4;^ zM9MXhV@Y`_tRd`AC7989KilO-`5ZTl1C+7f)v30-N*2Fw)Lrff5pyZhrV+aQZ`74I zd|yIz2R^?%0GE_$;Jaw8kAMA(ps-2QJ-6&h7pWe>#|=d>5AIYAVf5jgJ@Y-mY0vLh zyh<}Q)DHqKsHMUyzZy>kyF^)vEwmIn$Aj!p>1HYY1ru9bl^l%@mqj0ffy_i4-^XWW zPi-Iak{pwLCfp3(L_)7@lLecv-tR?Fuj1l$+s?9h<&=#)?`;@>K{@7x@JTiYubfl3 z3=IHvY;gS>@$YfoOSbI_0y5n{BhZNO&`4>Jd8&5uV%TnbhL!=g5E;fOB&|AnF(@A(s<#}tb>yDr&}x73;Gl7(Mp4Q2i%C$ zy#{&ocJRzN;!@eI@sB>fide5z4RTwR=zH@JW43Kv*$DcPp}--+1kT%Y^anC@-Rk9L zdyJ4(F0fAjah<)>TNk_Vb3IYU*HZj%?e6c?eis(`Kh8(3=q0~Z{bWVMyf`uMS;M{P znu0K?;Fa^z_zau|_=jRkusKi4+V0y5YKl!mb?il9W@rU%c}(Q6Svk%-(?DfK6w(hK z^+ey;HpVk^F!AUSKe1faohbU=cmG1pf9XD!Hpybhd6-STKu`cE39(mdQ z7k91Bp3D9DIe0vs8fC(v6d7@?&#(`qm}sd~M1)=4iF3}*b2jaa3-O^xV#TMoIZU9E zq3K5Y{UZfE6aqlFls@lzAN^sxM&#y1`-oO^+Cd+_@F0`%hw~>lKUOC(vCWb274EQN zawIxBT4O_i!s50D_zM&w3o4rom@ciuCq(%td+mk}3zrjYF|fVkK`nMAc6Iez3_6l3 zum)YtZO0CKix_S_isNP8fivvj9)m%1bF}Y$p{OqHc8EEo%P&UpVng3&*@fk7lqi27 ztwAW2dx|>30-4yy;9-0^E9w#b(9^i@MgDm6L0swfM1KL6%=-t zVDYqLc!|00Bo;w)-r&RZct6q=KO9SM2S8|3#g4PFf!kcFP@nTcjUDOZ+%cTy$qdHe zz`J7=*|{_zzUu4wxsaZ?V?C^;7+tIqSlnmeQjdXM3fLAF*mWs)2iDC@>3zvTo3=r6 zA>gISp}iw*s2bV{d>Cd;$=99FbHy#x#6I>4h(0^86FB@Gs-RmQ%y-5Lgspy=?vyaU zA}JO`%p+J1tHzDP7AOl{|9;1M1XC^SWR=uJAh`O-x|nz7D6@#ysz5&dn`dcsFjHjB zgX5@6cs;%eLAY}Ymz4=w5h3?tr1#A{6c<$#o;#04PCTog-^_Os8ITa&Xv4f(9e^Os ztj_Pzi4s9TMZ{hK#o@7APxC$$H^rS^7iIE=7Kc|+xbxAqkW34oKfiHBc$if#_d*s7 zzoBbV-}6AQutv7v=;C+8jhVQ$re!+*2|dguY~}(Y4%MD~ms0D;nPZii`!j@JjLh-C zJ}2h%Z(eSe3xf5B;?B08#{8(A39^TI#vh&qko5OU*DJo^Bqtc8b`W;h3Emyp9Nobk zds7TP=L@eVF{TE#b5SE3?ccSLr^sYTh?oGzgy?|kL~kNMm=gq407mxUVU>k37}ONi zdRsjt{u)5Nb{%2C`*>?En3;L%W~SX*S$}JjW*i=HwJJ~OyO|>(1aIfbYqYyuVj!KB z!`##0#=gEVgzpFC+(jp+49V@PolK#=bJ1lvo!8Nu@SU3rrSmHcsfm-j93%yb^N&O$ zuN$78viLYWn zlTy=&1t})|8tRwmqhMPZ<1@X<%&?gjBwJz2OCcBRUTdn_9jalR>)>iZhkTjHIVo0R zIfREkXz!*UoMxEQ7c}EC8BCQqi1m#FxWFv<<+0y(+SM1CPYG66cdw$a_uD zJ|DaQM3m=wEjJr1ExcS8ZG^iW-^C%09Z}@s4(CMwSaK!Rqdi!jWbKH+tsj*+w2;k8 zIi7AshB4e#`e*Gq8Ikdt@+aoH=Hp?a%AR?{884)tbKD)^mwErC?zkKKQ7+_&;Uh)j zO8XtjVWxLlA1>P;lM_oe8(5MF|EP)i7X(dpx>^gbQsKlJ&2^9bS<;M`*B|kTBBLs6 zvrfDA*x{2jX2h7*$ zxLTwyn`2kjPNZ|>Ms9FyR^j`H{&&|fG;@ycZH57n3yv}Ky-_?4$cLC^r$_K#{^ku5@49RN`} z8%|f?j3UH1S0VZhb(NS@;VI54unQ(9jFDJQa4R}iOb%*g2v!1|7sBRU_pXL3pCu>u zom4`OFUr}3Vs%SR3qqK!L)82vuAIc9bt@n=H4nrwar9Xk6?vd-URz2?FZOYuqm2MB zg8#wv&=n0C958?IwXmRKI>9l!{Bkpt?E*T<)TuUyh7{@MwZ4FjHhJ76G~Rbw9-HEx zTPz!OtCCYD3CnjwuXPTVln_-nA%>PYaSJ5>|0%JokkL9bJ}@}w(?SjdR7C*RGc^Ekqow24my-u0 zl3haa@tmCnvWxjRZ;s7@&*ULnzNI6TI@x&7yy>)7s4MN%0B0yYy4SY3y()oLSNoz? zcH@4l=2pQds~ztA+cA;_=e?nEv5i!G(QauI8P$e8s}#9a zrJZ&$I4Z&7J06K6zPkcQyQOz%Q~Zv9IV3z`#OaYAH^9#R_tyTP$v9weX?KQ>jwan~ zpeYMVrfyE7?sAb8M(8mig;IQjH985cbeEFbHz#EP%ndWEA*n&jgAU&DoV&RyWpw65 zE5e%nra*^JPB}w5Q9zFrC6l~hx6u1LX8jqBQ0a8eah9Nd#|G)(zNQxo2$1V9`D|K6T=0ggxD8ow^ zD{j{R@GsOgmH`RU@FcUtE>=nQb1aanUDh&lQ`KiXUB{B_p(KE>lFc(-=4DTGMRMGq z8fU_vpT(=YeGndE%jK=Np}bB4N;Yf8X9NP(k0YkjEv@NY0wLdW&Y%qW?5e;{UoOqX zg)Z(6U%Y9l>Td)nFTuTU1xZQiAdcNzQH2>pPB#z4{Z5_uJ~aa`H+0WJuzKiU`1`Uh zdCxz!W1?Fh1Q-H0YY{S|e(py?Jk|dT;%zau|5cdMNXVi%sPP_?cXVtnA>nue*=T|Y zfhaj-I>*UN5x|E_^pX_QkrJ~{x7AD{TVLmDOd2V4b7H4tH#^4Yby7+t(IN(jSdf5% zKN0)bQLZM~N;_Lw{3EjI$CC_!=ym%v#|M{&+8t-)5QU2}me&sup~hMXzSx>KQ{G_d z1nu*9O_oV?62N8YbfoDxKkv7AdN!!;ucUJr*UUOb)*#b7oV(zp5pi)|?&~XD$0S;x zejgZC`k}aaX@T>T+~}{)ep|#XJjFmNhSfoWPM(1h^G%Ep`NuCj%owAFD(XdcV`vC` zuXu+-CwDO{pO2fPf|_^tola`$RfPZ|Kay+Rkk6hWx%2VWmj$7#SIchTR!)QtyPkEn zQhq{QP^`^`yA%zRz`I{48H05NP4V`HA26=(Iou0G)d+5XJ#^KTlQwv7Je58>mDzxR zW;s`faPfnCO|(nCDiS%?d$|i{O=mhu8$`Z>^$Y(UXR;$%(|KuKU3l z=Ymt7$njtExkj;*fPwX>wM$nr=N`;)t&W^Iia^emM{q_Bh{90~4P&W`&!y%oF*;He;`E+vUQapfUR02G?Rp9iGrAi@8~C^Sss~ievERHYc`?M0I`~QYa3T4 zZ>n2@a8n?Bsp7$5o0E`i!CE|WmvI{!Bicx&IW^c&-kz}6R@S!THco(=RB5ZdWY9IhX`rBVT!+1$E1 z4l(QOOyCsv2LVY^GU&HM4t+!f*4UE#DaqDeaHB80<}le^B_I;**SYI(^ke}7e%Rl$ z-xWFj%c?^pE|y1W)iKKrEKGU@r60YuP|4;0qyNMo?I)EeeEIzUQS@8>VK zL~5H=i6FYzgOcczOoYkZ`5qcWJZZ6pG-=r(Apk{tsj39%Ag<$AII`EquYE2)=QC0# zTPql34~$52W0SACFxkP;e^VBV5ex*f*VAYk+O{Cp%HV{~f16V9edRfAL(9R%B&tW= zvS;ahySz6#27Ih1+a(udCk;*pa?yH$+c%jRcyb|Ow#F<#?(4?w2w;=gY(z&;t+UUE z(Ga^}uC80q-MA^a1|-+7?XEb__IDt*Vzi*N{o!IkxirxT^^m7opIhut|9cIDYfMOL z^HZzm#D$4oZ8o6XMV4>A5E~tK;W=660EpIP9TZVi`v`g11!svNddbh_t&5o~<&zp} zJ>9Io6F9dMB3Mh4U1Qizpthp9B})YtzKQ#VitxIUo_4u14YY70eP! ze{jNF9?D#IzF_j6qwE|h*;bM=ZwYo?Qp`3H@tiwkm<{1OUS-RvU0~nyoin?lIs=JE zRAK$kZAqrd0K-pr?bNsb4Ou=Pc2Bd$6IMnzzR!El-6)<`w(ucp%Pbs4vb!wI>C-Eg ziWxe6yNnVeV)DbQDL>=j5KbeSN5;IIKWnJHhsXmFy4aG%nRlX&m8jH{_>~Q>|K(Yq z&z(JdfZJ%s+pZ)dN_eEy*MgUS+&B1g>&>5&kH7A%2?+&g3(it4cvC#^=Cc`Y_3}Bv zv>DoY&rZ{o6NHZdyTpK?Z)?8AmhK>Hkl>y*>R|jU`I)JiZMMDLvD{5_xL;c&RbQKz zWlLm=ZeV;Uj@Z80gRtov;{beoYTS^Z`Q7Upey`8wu!?(jcLc_XAac&9{ zm-EOn4VKg2_%Sw<$^|nYFhlbt%ymco z52A&_%(c<02}^R(iWnbG_+Wf`Xy|MyUq>Kl;+TmwzC)}^30ZC-aGs+zb|4?O&|%EF zWrdWi5~=CL;2a6?V%L@_q(|lrfAq#fCD#$o!zDSWb{Xw+z7)ZaXT4H64UU_Jup}n8 z6%0SLXPq)39H3NRz(nW_q;#b#xUkemwZR^zCO$cI^4EvXo297XQM($)>nB2+o0C7s zy1O3nA-@S>miXKE}D@2cK92d;iU(FtHN!pDX}R`IrO> zIGbycLxh-ibFy(C9i`J_F3x&R`k=p(s-iWMzgr9PalR9u;8>y+dRbeBjQt6(#PCmoG;s^}|s{DD3ZS|*KmFSc^92P)bS;DK@b zY0J!w>y_21;G@CUb%z}o65r7yy;iZ~GVA@XzHq)HlQHh;tG_U)n~d|g#)@OU>lE^E zO0ZT;?|kJhtEO9@BZ=disMFY_NO#BYp8TjfQRlB5ePUNm0shTg6{3VQO7)Aua>aJ6|xlD zHeoi;P25)tn1YZ57ILvd;NvWEHc0p%aCdC8Bl60`rJJpc*I_tb4?55 z(0npC44z{$r8g$W4fyxi%b-xz5X62(?(VNcSL)#kl~*J4+R@lS$QtigC_O*m!)l5# zIS>iH5oBC+D^O$Or9Z*c=wJ;@``26xc70iIP&S+buhoB~AG~J@is=NRT7M;B>|lYs zJ;NCOCCKjvN6WO;;l}1x$XCFyRF|vA5%V(10$fwN;hp+Fh^|}xUqsij|1G-i`M-y* z|7W~pnOEbg*kZiNbVlgh3N=XGI=uh5yna@fO9G|4jse>uppU zi5Ew3#}!{sneD(uz!*zhei?jZ*eSNq=1ZLn$OR7@y+629?(||O=yLFtPSZl?oEQuz zQ?Oo#D+?i=<#=-7C`hzoc>BBReSmn~Q}%YJ33pq))l%eeoMIhtS_v|Efa?U}4F-GNe5%&wNR*L0|Co5f?3aT zwc*jJ4Y0Fs__?uQof6&od&x`X#t(1+L3(f?UJxvkqu5tRpjMWAYh^I<{Ri=eh64vmr((oA`{MD+^!BxCKKwl{=G~|6{WVvUvzZxCJ=NQ}{@AWd_9}wc zr}u~PYT`?4hgN9M1(!u?-FMA&N%WDK=4s9kVHVhoh3#jH`{Qhrz4-+5#<%#aF^u3; z#`(n}+$;Er!NXc)Wo*5`PP>Hz&d6W$*O`+lZcpYTyJvrv-Rb+dST1h44950WxN|Xt zLt5(1esMvVP7WOF8e>c5*_2S(Glg$^x5ymU!c9^wxNm<&rC!2H#+I+wVCGKWwAqgr!uwYYVzamT}YGa5qW9ym|P!7@S&2r~(ZDF?)$g zn;b7U0(~iti<{*&@Al5RbW3LbB9H%n>soZZ6@TE&M2g>x9{9o$-1C5$KJ;wq5Hr&y zBu19ptcA$qaSt?yea1!BWzL6%(6@EVZyRR5a*vf*vJ-&r!|1%i)5#GvyOHG>EvuI( zzi%dEZEv2cz{h{V(gz}*Po1xK)Z+N+4`fJhWASGXCKoe|{fNyuO{!b@ZJ-aE;RSgN zn%*RaXdwFs4Z!V}@6X$me37nL7jOBKY?UskLsp$>FT_D#rlptho9QtmnAx$M-5Xu} z1_zQ_n`Y(BZz?p#{QHPqy>H88K>E%=&1EW$Sn{9DUERu7%#9Rg1<7~Iy>&N)W-s0b zNvTtSHNzJxs#$j4TH0v(=BRq0Nl>qK1}+Jcn*9G{y#2MuatJ|{vTw(8Q5$L0ISJCv zddryx3h{4d+T}fxk-q{~5}{vD6|BS_|h;k|lLyavGw4(A$WCMBZ%L zP9(#j5&?NQ49(7YfT@NV2O%H^xE2N#0)O~WXcP5LUC(>sHD zerzErKiyFQv7>-mCWwAz9dhGpS2!{EC#RgI%UF%U0;uT`Fvl(6omBlyQ1 zaAhBV)wveQ-2udlQiffrEkbR%`nHG5x?Mkoa}|#<7z9toLC^pf$D?h&DIs=KDb(av zf%4~?xGR%Jo!ZGq&hr6rdtmLn(vLX?HTl3yf#~}N{4pKC!~K^}F;7Xo=dq}2vimHm z0NG6_WBZ!*SG{qKVdHa?O+9QKqP<-|oEENF?_1P4-MZTHrL{`5#O!~F#?t;zVb4GJ zQ?fbbdm;sEi^*?|gi#|`$cNU?YpWuDXQF* z_2d&&Ploxy)w=mdThn(LXYMwB7&8&+FVVg8Fx)LhmIqS}9v}0{Hkr#veZwP^?A661 zq5p3NKUn5PmEgZ1_}f9t3;#<1et;kQZmHHk`PRQd?d1&^KOO(QBM<)F$$#z$(2Dt7 zeEm0f{!9HJ_*6eWF9N9=+wH4C<+Hd@+DTF&RZGw#3bpiAv_l|&2cHzNkT|;DJ6T%6 zENeHR{QjSiJq)=SzgF$laVgH7&$Ke53+8ti;RK774jjA3PKZyCNW>eOin^(MD_&-u z^A8tjNn0c+w~lx&U(UsIql}|QC)KCLXwbERu$D?ZiE-@D$fTJEttr!6ev&$1L2lG z**<86`||2%dCk`@=6W)sj=1S>&>gGtf&v2I3JS=+xxarUBYexx4}b@U_^N>c(a z-}yuWoDzjgLhc6z7x0GtaKy61F zk{#k`#}?}LGmo#~0Q}kvi*4$;+Ea#`;E_^kP|V}Vp@+vq9}B>oZATh6&|-ePj}b&9pm4W&ol z^4==IDa0IENRH}3sR*kteRTw(Qhf97HxOb~I>T!=?e_z6`1tr)pzkm0$|Y!w@+T`h z(k~=IHVy`O?TtAuL`o+s?KEI9Izi?w9 zWG|Wx{qyxCh3ehv4?UL~G05JT1VMa0*KB(u{#((cEYu!qI?pyuAf)#8 ztDX{vj|#00C(k-|eK8y0!=k%$4*vU!Vix_@ZDGpUxk(lkpbTh`t0^NKzPs_M1}E`o zHZQ$RD-T5T&6U@m>VrLGR}O{T`?o)qGuz?}C?zI5xA$SU^5G7w`@uYL(wg9HELjK_ zK5Hc(AIFbU;P+ZQx?^2dH7K;k?+vrf#}vFoD_)x|47TnVGlZKZ}73~o7;isJWA(ftJxK=*^v^iK z$k63y<*N|hXgSeYiP8I&=cS>fBfnRFKgg0t49(AsruOi8gi*2l!J#JztsCXo92bh* z<%%W7B9g`aZA}llh){LvX^!>%U&-*#Le?7Q;Ld& zRaX5Dhb+oAM`F48jx7<$$hZh@@rt`QH@D7PM7yhvC{E(NnHROzQVL;h>Z@Z2kL-rO zQCUYdR^EK@{P1Y{8kw<5HJADaE(_d`?7o5^ZJqIzgL5>w7~5Y>5Q+{lqV*hNYB;FU#tWcFXwuMzYAe1SoV83ZOZDe3BpatonHeKR^CTz)FqzCO zL8BbJizp%#A>mjQRfdQmSP64Co4vqC8&j+eL+&;e*^`a{e#4*;kJ?1C=XX&QY|;%K zaGE_^MxI~LdVVuRm*e6~ngn8h^WpYFY1pQUf3y;6MAdmzoZYZAIBz>DYvKwsxNNES zac6>dL#j3$h>7}WW=u78HoF;YUi+*yDwAvL5ys-E#P>e{uMXl6c|nmMck<7a29_Bk zy@yM6J-*gx{ptT>x@~@X{uJIc$UoMYeRerq%=ORlhzKf!H{>#j?p;usOS?NzY_wgG zIW7;D&Tdb3o?4wp81{oNVXrc^);9x^54v2n6l4tsX!&sY-7;6?$2c;OMRnA+X*8XK zW$1M++#a&xpNKF2v7)s)!f=!c?a86oe9sW2K&<8G%zUrSd;3_z3u);1@cq)0Xi7MB za1_YB;`>Y*nhH z=j1;0r93_GxPI^Mo?2w`kB{nD=7y^xKIA#H(OA=jliOB$j*+Dh7C(HIBXcgqROkuU zv(;gcEz$m)0zzR>J?MrmAXIq@-g7206jEkCw1ayE-7g@EB8AC8-H7Mh0@zSdYL+iQ zB~qY0@`V+ZxWMUymI*;wpgiWz>wB>r?vd}*Jj~tHs&(E!#tWaK6J5-IoHEOEUufy4Ar+J|QB^D&pS z)F2Tnyymb-41`@8cG%KG+!7MuG zmv5NbnyQ8S{S5}~Mx2&Elq4+Wm_~^D+|GkGVjjO3V8Zc zp>%;IS-|sSduu&47Gu$HZzj`m&27n0OpoIHVYU~(V=;+h*myS@o&C;oIN-xWHqZP| z>b61*9U%l#do>&2^I7dggdgxK#y(3eT=`3j`i&jm^l`K>bPdg8rSK{a$GAJP9CzOb zU9uG5)!P~|DoW73bxQiIHKTMj55C3`T$z|k@Ve$$bT85hwl9r@1I!M{C?KuPC5AB{Ho6|>n!mF)@^aB_Bp<_Qd=XsNAA!|EjuppV?JMX2po)-V%~5qUNg!@ zZF*#Gcdui9RUTIJ6+ZL;4gQ@7=SKGbVePG=A`gN+&4$L^-QC^Yp>Y~_cW4@SXx!c1 z-Q6kNrEzz6mqM4GJ9F;c*}bzbn-6vB)TxK8{AWf*#_x*=0<@x9!w8`VbuYKRbdl@z zi3|ZR&$?Zk@}fwghZO#@ZpO5&6bi@Ia!P!hLwqT;97Zq&h2k8Gkule-yJMQ${dtdb z#G8`M4K9M*GoP)K>f++N{(H0mm+nr*3wa*+u+uJ&xrzF?l`TA4A_g)m1ib0c5*CdO zbF>IjnR?xx&`)J=pvj~Mhd$3}Ug6%Gw;3sG$O3>oJBohGRQ#S9fsDX(yqgf~(PnV{ z+-EB~He_|E4heoyp>6B(j!IfNEhO183y z+;R_P3}J5Oe!pWc?1{1gKLcBK?H{{WSevY8}Q1U7_#JfCueUCx}5AMWVndTK!{!_@UjWc+=4C+1DU@b+jabl_Z@RUMT#0EdG-IhRFT38zo zJkWfrpR0=eTq}t0Lvp^A?z#%@h*6)hK$xHxd#6qknN`ggA05UYQ6%OAtJpVm)gH|u z64cWr&!kbzMin0+tD8X|eSpIc>4jFh9iQXD8|tJ9qt*+f*^WUd9qyMNlA~saoSm@v zJ1rRb9_oPFK^&$Yb*?NlmFnkclWm^B71bD7GKebuN?!?D;Cwge>H2#qG(@7P9dT0) z{ILANscNR6bzND3$zm#F_#}ArSU~!i#W*j*sJ=g@f;WH8w0^kxgLy$Jo@t@h*{^$R zD|236>}z{f?pm6l&j;s{Rfr@tJ!-m*?SXP(~7*NrJ_?+pnykO*~(9i1d*3_(@aSB|&j9fr27NmJcmIV)(q# zrDrk2SpL%pTl<^B`=n1K*}jL z`prrtrY+5ekx+oDS+|iBcCi7SnKZakXfo8IFs#XPy{;~N&v30B-C$YgVHtnqp$idI zn`5krRD^QfNhk_{86dxa%LyJ?Wl5(CCCKSIioT?%AL8oSIfeN+I5&%8o;1@D zQoyMMO(p>U*kTUQa(_uocXVK19X_QLwq?lkoWUovHVjrXn~&|7AA&wrWPPto1Es(s zlW^|OkbSH1(A(7-|NnSN%C&%*8DvL_+5~+A*v^v5MizD7}}s`d>R#{+1_H!Ct5$j&lqo1TTK37 zEKLPzE|hV9m8AF6{pAss%<5%4@|}lH5}|=J>wxomQe92JHxmbiy zJxgxj-hUh~00SVv;i;wlMZS2xZ3)B!A_N=TbS8vttnRk_CQcbQ0&^qBGktQVy@(Dh z>cxWK3-bCM&|D$mog;Fp3UZK;y}YQG^Ml6)L%W~|i z4867veOI4fw2aa?;kvD6QxsnD!Rg0UAerC_O6LSYKbU@8ZX94)OJIbV{txrFoa8E< z45(J~iCb!a`F+1fAPXfB0~+{C*@Ex7Zq30}DRdK}6D(;WVARI9f~4kA>1N#{t^dRf z3=tX;Ilhzccx!s#vaTfU_VwXu4u7=oSqPuP$zj+|lP?85C@|F~p5L^xT>2JhlTQkZ z07gy|6-E4y>7SC~>n5g(SqW}x)B~7gi#+l zzMdN`z4rzTePrL_n4aBYZo|xQ%VT`*D% zlSebqI`F{2?TSIYbgg2IE-?7V_Ts}uOyqc}5f@2@fzr&FdJ;BPCE>sZwKQ5KmQa|R zpi5!<+_;uq52EDG%fVB$V%oT#RYY_ei)Y@uvu9p(k|BKwuLn2`L`q_t)Um$>t0k%o`DJbnCx|ZQ6L@w8)f}ff@bdL zGkbe897d;SSS3cSx#3XNbO6pSe@XPLrKl~}5F+~lTGfT#8t(;zNsJuQMuE3zY%f6 zpq7gAWG|WW6XX|JH;|T#oWD)uHSW9KOSa+%p^A$&Ywlg%ofZyn-datMCMW)2D)7_b zT1XYktO$%dnG5h;;TaL*GKbXi*(eq>wGb#TeI?P+(Nh3y?;Yimc_rnK{fNaZO~obx zdI&hvkyXMSVD^Myyen5J65tH88{l1897pHk&FvQj@Rqm^E8%MVg`eT0JsM1J{-eRz z#Mt5)>0u&erIx*~dQhGG5)X3@%<9#RB$jm-EaNHxx@X?3To|;W`b|8t(M6;`*m9SU-gA6;`D(Vebn1 zPK0SHgO(?i_j?rjCf)Ir6}{ zZ|x<$5d*7ODX>mWqZe1ZOK%5?%F;NYf@ zkA0X%jU&o`UH6QY=7jH^AW?+ngf%;Goxc(R+F@(6zU=@@kg6|)fUhqzHgX(YGE`rn zKS^ubdeE56j4$5do$H8h!~q|bdMw}>-2MhK72#2m;1Q=8&;$____vz7FAkmR2X|9> z$761!J3`qMpA97pbl}^EO^uuknd3;jMtu zP|LAayvoBSrtjH&u;3i9uTL6S>!WEn5K`4AVsfK>>7p+3Q!B&@w$pg-Drh2rt^q`#(jVVG*Z=K`EXhM8#E~Vb@EW7BLnLF{yJCy%8e-(#fC2_77 zU6_Jy^xcQ$WqGczgH>wn1p*zgRItD7oQAPp{`V2 z98PG_rv;783O=_-S=z%|YHD(!1^xBk76~hQ%Kdg^*X_FkH3tTvPRa&z`WPmPSI3~{ zZLZ`8`)|(WCx9ufS=3^Y55>+Kw3%FbBgOe^x0pJwq-oy9w0BOtsDKZP-ik2VrusRc z^Yq5qxzol~qb&AmHC17u5I5({$5UXMu03;EVNv16i`FO+DjgP4=Eq zx)i-2BNz%EMZtcq{d1N8FB%)BtF@oX&z~w|?yV`ruj231&hBxXz8BNm0S#^~uPt~ZuNaO^ zp+TrheSgq1CN~Xp!$|_%jLFC${yvm^?p@?=Ob|R_bo3=4d*_&p?9PCnd(c{uhzYCp z7*OrJLV+Re!&aWM{>f3;0|Doe>jLx~VWo+K4ZpMBzDEK%b0|iQz5+~^!>P4|d zwVdwZ%{&!;I|)U{`w?RA0a-YlV!Rfo*9s>d*9r@)>BZ8wiK?|=?@NF;_;f7=*CWm- znd|AM;(2; zbSn&vTE$q|wuMJ^$Z`Tr^B9ePMCxla*I?fjsgVX+wt{bkSBBo>ii~)&@CYlda-{?f z6;#_EVu-_gdOx4D_7nxn<%$Z1z(H>VquVUwa*D9z9v15{3?!-@YL$T)s%|&DiOB++ z&+{!_UL1>m3T=S7I@KYXu7G>>7C{x7yDNpuN=0Gf#ca5cP%N@hf}JiN9nGeFtlUK& zmOG4C>w{-FXuPntH6~Dd5ctn6Z!_Adm&^CCCEaZn!T7l3VmOzy)>p5z3k5}?X(ri| zI{16vaKC=pZ@%R=*L1=@WY-Z;W>0^k7jAp3h!>~a_!t82#q7J;7DjXn++=QkX9Z0a9NUTr~gWpN|_0iA^6})Kn6%?h}Z?T)!w=#mU zZc#D#DT!FlAe-wl?l9uXSjP$c1+6dQv@tH>sx&Z(&N98>r*B>oXNG+JD#&}fGgS_`!&iDiL&c3^?^(~a4wH?M}sOYqW|DSO1ad3>Q<1~h$ zDy%A$c&kwO)ra<~H#SDEq)s1teLh|2`6VZ9<_KNSh9hkx1d=u1e8DR^M_esAe0wL_ zv%4-04_mlVD-DvMLZjY*d2y$ANM4LCb0~)P$>CHCi_(Sg=%*Tp=vXCPJvgQgKg_c@ zx!nQdB@d0j5e>6s9+_x)DZ4_DcJ}c9JfCrkX z-B@EqQeg{?hHZepK$M57m{&2lh~b%bbRy*);eOH)lAagHNlS#sLuas5u&nt_(#y7~uS1+aYh&r$D)JQits&Y2rN_zzN z-@bJCuezW2!|f~td8(+dPEB7uc;kQ&;s>y=G&D*C^FN1_+$ClBxlL(i3T^2lan9(W zdgTVh73)@oteYWI>zwYG1i z@iot-1!_j}&CU-@UalDI))iA!mav319*o^-$T)^MwJ0#*mWb{GUBt-nd37l24c#|S zt=bnb!)Lz&0pAg!#OS*E-X2aeVa9j&{pc80xUrNNOdU>P_^vZ~KZ5odJI+-_8kf(0 zkwCc3mL*A{(3Qzuhy?JtGpAbK211{W!jahgF(MTJ(fKYn z7!wr{s~KUpjH@pYiZ9EK;^s?b)d9Ie4?J^KlZd3>A&S|o&Bq5`p zvZfgH8jP1E9%;4f7ZN2dz-daU!PP*X4NnQj_fe=E$6E($E&6o`(`}Eb@RF3-nqs=7 z%m}wBhDJ3Fw#= z3@KNu)LTzbG&m}usGqvu+@e-DdVxOHD3ISU5KPWF0}!1#d6il)GU3M=-n1_JYQ=x^ zz*!0W1T24L(2Wi5Wp!`*ZpXSYzO`*rl%rRuivM?nJvhG6O_+{~s1xO5-Qg-J?A7ibPKJmPl=)elDMCrCV})rW;C0GuH+(udMWnM3=oh2d$$#wOO)v}(W5IV>diMm=#(LEXCQyP z)$hTCfA!FfSZ23SW~?!tI_G@ex!?x#oH9Kj+#y@eys6TR+lL2SKXF4|dMhU=Hsb}@ zZ!p9C5Z=UuBzSy}QBDt*v{PIr8Ne{+BhOFuQGtW#wz5wsX zI-;FV{EPN>I*a)+CvfJSb06Ql_Ylm_r}n`$ak(?O<&dsDHKNt(Sp>`~9dq}OUg$0{ z(=$;jN!%w_hW263!pN}xz%`Hv>B)FQt;c01|4UrCTc{e5X*a@qgN7IfO_V~;GiHQy zHi~<-ZMgbfj2BVQ4r+1OIMkg5ufEUkX12z85zQ2W7Fr?kP0O>b3;wxdD zAbA%U8`*BdE`4Q)C4i#XB0J*FxeI?@Uk1^*jOKu5u?R84j^ci_;D6@PiaB(pC(Ol# zdNv~0@=~TI-S7J=zxOC&H#^Eyh{+Gz!Hf(xPskG*YBHUuIBTGt7l(a1>g;q%`EMSy z5wK{)-lZ`CBf~1G4J&*C^HD{mkFV5%YOnOmhb4Hpro@CEr7)h{njE1CR~i@uJ6#|eK zDxx!eASS{)jlC5|A6i|rZCa$cf)!ot7q&Ter>x2ys(lp^3DmawTv$3ajJ*;@JbOo> z)E)@BI|*ylJ0*RghWHzS{m^uw1A}mzX{Bo6r2~hdAAgI{j{ z0yD57iqGcFb2kUQas5DW2}$#Y5K0(ar{{~j6|CeiJAd+r8s-A-$c1Ryj8UK z=~DlUo(|bhD`m`&OzuXIx#`Z!RxmqBzLmvNO+wtSu;FJ^1Tr(?U~fOlJUU_^Ps$tZ zhz{kiD#~=G7~%3*-L4z{3;Qk8W)c~31PuQC%j-#hzM4K5E0EoIElpfl+Z86hq9TgD zh4E`%{;L*#_17OpAr=5ItC{ZD^eMMDI!Y}!l zXJ*f6#)BtaqrbIc4C|w*`b+QKU+8~T5GB11E6{WeU&o#y=jck#9E+ooRKdf})KLTk zu)8n1Ktw1TT<(?u4z~&NRisH)ZCBig!PJ7+-IDvOzpdR)+O|SkOFl(}&r1Ia7i@e6 zVG{RHcoTB8Cqt_yMNmUYHPOS#&>sTzFZMrS49|S99m)hUs{c!xNcy9lfoZYh5-v`h z@=r_o8lMjLK?+&3qEsQ;O(AaX26k3pZbYU-jyqCRjvd~U{Bq_-Y7y&R5+=#>;M?oZ>z z8g99AhamWXmz~=|4G&+QgsDR(27)Q~T@^Im!uSGU)fUt`CHz6tiXh9_TwrFL$j@_T zm#oPW&r>xsgm>txn~i5Dpip3AL}uy}v$ZcGrK)P^`wjo*&1)jEH|6cdt65^C$!_0M z9XH8FH0=)-uD281PIn)&fLFk%WS6(S+Rruz4?=B#-C;jg?4`53JkQgj!pN< zvwW|2j>TmGsSW+XLt$L&|I4?0!F zlp3)Uew_YX$XYLVvaK!E`K@8&s4M^Z8vfbmDX1wS5xTLfts^?=vgky}Ry6<4L-v7c zOR77OyNxPGN)CEZdJgd4thd^mQT zyAE{`jEgIUc7bsYBtLBL=d=Jk@1LR3${2T1FGsA;pQ-6NoYBM&VLfSR3<>5FY4MX> z8|M?I{OtL$Jsxy}?}4#pS8c?mCBNCd2yv$^e5ZeMt0U8K+-_}Gtg6_ACOU%5qJ02q z!a*?KvF_U$jXSV^tD#j(6gT|V2i2Ll`-lI35)oW6Z+h|7@EN5i?E&;m-X)|K$m06^ z81u1`bm9!{)C_|W2SSQiJ||_~-0n71U?2gJM$b(!aHnc<)S}{K5wbdBJQ-7m-JVq< zDc^kXwJ#6l-}1(-ndg&xmu#o&nY4^yp-JL@?A8$^9V?0VMM~cW(LMV5M%VaGbB1Xo zei=*|iLBxG{@eRhyb&ox5N=Tlpfp8?(RoIEd*d!%S2b|f_Yb8v;JSAhn`MtaQ^!7R zl6Zxz;>&0bSk$Ei)UI%0{>qcv62gEB+K1J|TX%~SXh?vX-g?MK;b2VWexRA#n=9{3 zv^|)&tFw6~jZnceQSM0pDr>J2l=}Td%=zA=PeCBnBxt#Sdg9bENdNJ$+3>su(`~v1 z#m>2Ag0mkyu@Wx_G3@90KPS*4g7KRPEToIc+zt71!V`bZ(x?EzaBa2P3*bVg1Q8VL zI6vNnJHmUMm6~!H7n5fAaG<_cUYtjvTq1>zmeM!MbS`Jb-A2g`Th3dz9b!wd(#DH1 z-NBf4_AINs(baL6W2l9h`TIM>yzAR#dt0XrsS8sNe6aX)RzI3;k4)WX zte_HN`c1TA$-lA02`-8+V_%8$VXoUi;7Xc%^(nv$6pv339>+U}B?$UV3_TzhmPccU zY$b8i9tGdyz2eOlBWid{2ws&H6cpUqNxGF=DnG0uf2Dt(9KQ`CS3DYR#?1^odHTDm zZ5FwHzAquY$6+w^#WIDRw|vlKZJ~9>`MkkXyWl0)y-KU88*oNetmbtKAgzV*y<&}v zFv>|m)yNIq8-5#0Mb=o-Ww7?Uq!+UrP%tyF@x@-Fy&}bI&X{k)n4Pf1qOluzd9z96 z?P{)Vd>f7v(8n@8xEmT~vdIa(wsoxp3}WgGDLu}L7;;QTV5D6sHSP=ZtDhc!jnE|T z)K_)zUCFH7t_SxaWq&^g!6nnNsZF z?5SLG)&hOr*9toVZoOJk*plWHyiSkN!qQ%9y5+Mf=Q2CuqmNYs4n-kv4Q4Sm*Rqus z`%V(U*wNW3!1F9+2DFq)!#L&}$*7M1r}52Dto zC)m+2B?uHYq+q7`YE9lo)u8hV#8e^kMQ>u#2CrfR`z;+xCuc=i=`yiS$fmtc`&~hq*wkfRjk$xxk0V|+x#WnaZ9l)JD=EpQ z*c1l&p8ZhbAMz*OjPz6yCg-WUZ*eUchX%4c!qk2NCQ?@!5|Q;B7k6lTFbH6Ml|8sU zaV^Jwtldzn4qYlX9bGuSKk<51m}0OUl;({6N~VEAkC%=){&~UBCsH1znFjU&^wj=hv(yY!Ab4?DCZB4gZ$^$Kh?*1(@rbUr=7yQy1gezXNL_3OH5! zAHVBAUxt5)d0D4kGh^zJyru^oSG+ice90pJG30;D)U+>r#cKcSXtSo$iI~{c(v0|b zvi|3AM6Ig?|NCFR_P@QSrv9ILi!b`j_c}Y@!qwvXWeGLv{N-`vA`uKf>hjBo6)Lp>St&URSb&2`uhK_s#w3p#$kl zqK~QZ#mx5$c~#%`b#}HsU9Sn9KhyLhK$M9-OZe)uL~w(ki-TsLbGIS(yU6g0*^BFX zr+yUNR3Mm6^4;(1c6Be?Bl|xlT=)G;4muKyk*Rpnbg?Id-L(8|CtKdCa_DESt-Pg9 zxVY)B02w*lY={8(06vz*FJUhu^ylj{;jj2}(}5XtUTJ?`p2sY=Q~N)?o4_LHt>9v% z&<{Deu^zsUGtm=7sA#u0rEUjtx$q3_Od^4Ph(>sQm2SV4@alvQCM+?sIMgi9Pl6c) zdRcClYG8E69tIg8togfNF=7eYh8Q?imLd$#u<%mpuRFS5|1v1;Zz$Nxb;eJr&tRi5 zFI2+iJVi%y@fRzY>UOfS5~JCkCKqwYe>Wi1T12^$+qkj@^U{81c6Y{a&bAo2Bz4xT zqx-|~be$hIj*6~%tH6JV=)$1c$vaXkK1M3V_blz+&gQ77vkosv*Z+*-)d~FgM~5GI zJx<*cebxsivqVcI#B4-(`y1o}DTTcRXQSFdAiVR%jy}X#6uZJ8Nio2g%Y(jim_Yrv z^nP1zDRY(LhpIm+DR+9@-O8s7r0u(*W(+Yop%otxv|y28|AJ%lW%r*}_cbK$Z#w1E z468qRSV`xN3ei|f}z&eNj~27|sqDZ~tL!P)Se=N5uW0mw`8 zB?|ph!_!Z`)gbF2kMZUaAc%A0s^7j`qLBKttO^oQ|bHNm85z1o+)(k+A zR_}fVITPPUq$H-x@MmjhA1kdYeQedS917w`JGgr?rK^aIPfZYYxfJaFk>DGaUO&Zg|zTHssP$ zooXYof#81K!o=3jH@*$m(AMUvf%Q115>L9dpJ=-b$rq*PKB92H^Ch@xGXODrF#)E_ zE)d^U&(d3;z0t1T=(o3eqN)SRua#vegIxg=dGqV}H?&hT^y`82Bxl*7Po8qwvm)h( zgZrjQ?rg+q;0e22E~>d+{FpeNVH%C?@pO(Sk#PBOk}Q{g2|Auh8H5u^(Kx8K$|%)wOM0?PgGsD%@so@|?`3k&f3^5^}-S!#ed7!m^Ir zQ{pexV6C_K6}lFL;XJ3aF8ULGv<9G&fAxDjj=^=blyIf6i=<^^xXft7=T2Kz9wmA~ zP4UTza;0Og#O6x(;Pu(9x|y!8I(p%)FZ+)z;0vwvmr?pJwJF%su)FmP)%p?KSI)=2 zKjg6gf92{)8PV5I=YSbGn&{4n0n~=)Q5%!(wuFD*UaGP=Yj}PL_66%&Yx1_lR>;GB z@ZDT4(eb#4zK`yl$G{&v0S=i8FCJX!pt9Ls-53N3*GC`6DY+A!5(S^$dajRXx9S^& zs6O#>&fhb#mimB{KrCPUk2HJ!Go{O5harkiLZ9@0UI@iWJh2@7k{bPfhv16_t;2JT z#8V~<1uX4u+w6{jnKwLT)V^m+i=+%*$^VwC-_WJ;xcQqK_Ezgk*|DoodpGct0>L`6 zgg+|F1^q!5?KI6Ikrq4GOvJ=#3Lrk@2P3r*=Y!aLZ@IUAm8NCLhvQ#&^gzGw&%0Yn zoyt+u=Vb$XNLs;6??HHS!`u2PeJ|wY%{H_k!$b)DewVXlc)Wl_4AAR)h|%dG&D;@& z^R*r9tmO5l{${z7d{(m6Z1nP0m&rnK!9?(aI#;^t-fbcAw(B}^_DU?}a$F>CR>rrR z8^iqGL~u9boSGadYNhw#lBV7nY)~F}zm~IPiqN~vd3C~$oF`*0cIIMp*AHtcM%ss} z%RG@0V)**rC3L^G&K4y;JrvdT2w%ixlixb^dN)apfX7BLtp9Cn8F&)yz&TEuke~ zYp~U9(@A^K=Aq1XZBR2dv8j=QZwSBKTmXUkl02R?yxo`jHyBStH`|JWwM(6|(|ye0 zEag{ArW6xfRnP2q_+`&_kPJ>$W2p=;buXMqK4!K*Ij#%lc5dE_H88nb3BWO~4EM`D zA~k%gp6*(Yx9(Ct_*-BO{fvl-8-`K}x{B(Kl^6zc2|d}?@;;Fi8!rBce{8!A|7PsF zInMQPuoaxGb*Hn<-tNG74rHB?@ApS>HnMvu{>932sRrEl7KsgTLPUX8jTGAw90#U+ z5Z_vGzv5WQu=T{e=1oFHPLZ$&Ik#MYjUwu)v#Y_b zzj<7KtsW;J zXX5ok)bF(B`8tq2XuBH8qjN^%P%6GF38$~|KyOQykP464Ji|>g4KIC4 zHxHI~F;TVdv&<4kWOoNE{>PAK<*Q#dwoA@{D<}1@s`!Q;Fh;!kdZjLuS;1ltdR>-g zD4z@igZ~{suaBeMJsy5+j3>o6mh3IabiNYa_U^aHT=fOe6XUb!EpWv+ z{l5Y9JUoMWInVRdx?+N>-mFP3l*4n5tIVJ!7(b_wMvQH(cXyyCwIN8VVuriNrQrCu z9zBP;(E2~Sc>mzXok$UUsK7V=X5cRgOt1A*AP^Md9t}DA%5p8~@zAm0GX&Xs#9ZdN zAvMGAgg97ZIUnY9>x`!D^`*#zIe@adA%maAyfK`vB7^baW4XqMpi7k&nOg!Lyr{|l z{|wRdvj2ZU^d)!yM~MD?_x~HBznQ=YUa!D(2>68su!_9BgdExVR|o&)M_b}|Do9(b zY>UY2i<8U*!=KIrk~&JRgi}CYrHHS9uZytP6>VhA$cVmzL>L;9-enZA8;uBnc=@pi zlm4!$qOJW$5|TD#-mi?1_@93Z%KluRdt#L|1?pDB4u6K?q=0uP?YteFnX-+La*^^q zQW*_dRThW{Uke}45cc0AD)B~(L@OCgGbBbPe{m=Md2>q#75>W2!tPnSVux6DM%)#OvC6l-w1Sv2ERliGecL zyJa;;cR-n&$`2c~gn~#F>y0QMxeFU}%I^ZzG>~dV*MROzm9^a2G1{Oa+)TH|CJ;;z zx^k(4IKK%GUDL_=2(TQ*>tCn+rObY^QiA=ViOwV^YvO46!7ne042YV zEr{4>FSLjPRiSM^r3I|D&Ayl%@dj22VJFs~P32>4xOvq|uI$_ig54b>!5X;O{(kMB zcjot)s-vNN{gI;O67IQPAv6=8L-_9Ie6RxTB2Jn9qf3F~ZMaKnR<$X~kJKVHmDX(l z)yl4Ky-PtUp6vw;Z3B`t;SaK9h|Eo8!mj2xUuXq#VI>$-UT8tS<+5+|M?Pq<_t zvR`RJH^F_z?K$Yx80BttYQ}Ss9v$gm^Ho5zpNx;y{aOpNcIJ>YwI$}r8X&fQD|$g^ zSM#lD?ccjDEhnhBwHfX82~%0JU}G_12{~~nb+G7gnF_N*m0EJlqS@Y-YT1+zVx+(5 z!FPv`kQ##m51sw0SQ%JeL*czPuvz*;B&~ly@z%V?{3kS@NP;8%e~0ZKj~aIW4-EYa z4?A}h{;R4Mn`_b7y{wT*&-MUm6gi5>Tv#VAr`u`!R!Tdit(=j%hpHC8slkjFZ;f3t zsKt#A{ZCGLQnt3(4~esN`MA);E50=FyR`zF_p}@dAG!G?Sy`Wj-(l^)BPOK(TW)@B z=Fbpwq47|~>S5gflbd(s_$QTp<$Pj)#Yg0+Np_#cN+R8zC<$BE#WG2dH8hmY1Sbfe zvD0k99^Za1Pl+ph`>yAc9a|eEy7uV!mJWxvC$&l9K%+kObs>@PB39>u{+OO?(I--x z(>FU`GURqh?K+_Y_}O*W-vayJnz$2f3_k7EaR>>=6RJ8j@Z9N-@>X&ByOWqX$8-41 z4(qB}-AjLn4|E+Vj*#q}YsPma)Rn8n8l&7E=m(eUu=~bTOF8dvJ)!D?!`mMmK+sJKvUw4+sdKejsn3z&W#SBkMzQTq|T+SV3F&DEM{qXbXT6-l%%=8btA49Tg zihJ7MA2Of$5-x^+X4IIQ*E4>{!cIMbI4#1=o6tP@L}|KN%57fu%T@%Xz_NZa>iCpS zM07D!BlzRhWbSLEb;R<@s;44|F~7uRVTlVE%4(0&U$Jj-xYA{|@iqOnS3)rGZLx;A zbwBR1Byyv>G|EPdmf}DgR`XTN@epR^z9EX)nP zxf5963~qmboM$!0EEV(4_(kF3Ew4G79A8rjtR+h7-AO^LTN%HuQFk&Ikn}bZgS5lL zkq^bR6EaTkuq;+`t7UKCq*U6<*JR^3&hn*Fi49vRZkw#Vu(<4}330D274R>Z0UBOeK=UtGF+NTZn%(u95?ViDndj>li7Gfyayk~?G(vS)v4{8o{Y?EFb;}z zMyFFgrgXC8Bw3%`^`RvMl>JJ~24Pbs9;`}8Di5%V)xKb-dcc}XKP}RxRsf%F<`YQQ z)UY2`^8fBREFbAc6h2zvKp#DYv1^)VYr%D64TK220D@ixQSC=TiDI&;H7eg3Lmb z;}vcwctO;hW`u#^{twNL3hrv3%(WZxT-<=+Es^I{^|p$uOlf1+KI)9q(KF5V{SxEh zucigO*+-|=`Kg?wh;x0Ji!e<^f{r6R*>=DJJ@zcg@=7mEURUb-_FR6P_-8Z`C}OX; zff6p`M2RkJ?n@(%nC_qlAbZv_^N3vPq-4pIE4@SP^`cwyw0Bh5kEMFsQ53Za-GO>l z>fsu#=02C+K__UwzWRkxA9DSu%9fW&xsdN@z@V1<_eS@0S#@_t@2K*iCe~VZeSwtb!bgu4?X=NZG{8 z4aF?L?&kg;oZfS#JdLp>@u9A&Rm0$of6WBRLqRL%FEf7dG z2^UFiU7;|bK^@cbbZBOf?2m&EvLB!sSoakWgH+FR&i12(PRytg=Wa*XJ2GxD(+)qe zG)fey3qv9Bz;W?`SX1a~0&4NtdbgpF!oE+glt=x3U=s6rvlIq zP?auMq}3uD-^*Ft%w3bI&LNjKoBJATm9XS+s|^Xutj*808k-8X-DI7Vn%+(pw^!-q zimiky%B~+Aczp&8nIraMx~3>EE{Z`fK`Hkvk_X-eu;*`{Obmr3ZB|g1LY_P0C`1)7 zAfyNlYrumeYLyu5eqX2CIOe(W;*E)z#MOlZkfuIzP@N-*K~4^~tiAE|o=Cd$h^rNrlbmfAmF<)0AuZqJVe~8t4Yjj=7f(&KQeJqIM3K=ouP6Ve_jXG#}6zVQooU zx?fU2iY*tpe9PnjrKMtc*b225Gqk(c}1vk(ZpN zNGv7UF$XelH((_Jv}E?OcuN~dJ((vA`JRC@Gn!+WQ6)76(i_$tlbV$shKed+#5?6% zG9{>35*B}c{7?#8^4ZyH%Q`KiXFTlV%G#o+E_FG792ybm*+EH)mdjTfsc6{>(BhvY zw&uNha3MI^hYzQwVS)Z?zbjb$b+86Lv3_`kvr^pGxZW(aYBZie>j-x3vJaXK7e^ZT ze8tUl?ulSy=JjFvn_Q8NJt_?k^*^WNyzNYnGefYtomr+IKV#{Wa=`3fq%uo!IM&_x zK9a`={tCYvbChQf>*l95=JfFG;0Q>=bNwfVp-ULrT4y=ZXgqDyKc z;2vkvg^@PF+9n4Ahi#pbe~sUzK=b6P!JJOhGC5QWzk8ZXHRo-(5FGA?VMQDJ);#U) zNp{U6K`ApKsx1ke9!EA2`mS~|9%<=UcY0%;<_eUS!P*0AkytlkZ2VmaP8X_j@2EhV zV_u9y3?T2y?}(_Y(m6dWr#}W78BD8Hf;~>kobe&6Y?Sr)qKn>Bv8br5HspsY3~>y@ z!`5u;t*vmrr-%&6qbg*Ep_FI4O=a)wp2eL^tST3M{CZFH0Aezt>OvOHrd8osoA~=V z7=0+!$inUI-?sj|bw9Bioap3;2W+4W#+KE=t!4Kg8*1823Km&xmVw;A*TNTQC0*ZW zLuvr@CBqPmpA*g4*TvO~_=eSDI3jVP>j`hlD?EJA3bt~5^t5?R(6Te}?iaSoN>X16 zf^Xf5GT#LHzfGE5SiHVfrmsryKQNMx4)u8q80QWKZsi2;blENFjYDa}j50 z<r~77=pV6)7b7f+Fk6T+Yb~kC{rHY7~07~_YpTuU-G@tU!g-O<_xpb zpPjgV)8qw;mkT8|yh;$XNy7K7wh&9`s-JbB@bNY`tUdTTUEqifsRPvM@Lk%mf@1CR z8?D0fDY85HiWsd}6J^qSyWc4(xn&Vs{bFOoJGyB&Gx$50HaPQLpa8rPM&LbIyxndj9o;b}A@?Oyqz zM)!&CJp<5z4B9z z&Apckov-0FRd@J;-~C_hTjx^~%(q4aktRs*s2~IgC3K_&g@6dsdk?)A5kn0YItaXi z^e!EI0qHe@0t&$x5GjGsiVrJ7>=B>^x`YoU_l_ZGyQ= zMOtmhwmASZIh0nDdJee%=vBt1HFjfOZp~TTKFb7ZI7IAHH7)2=u_9R1bUajgJ}hk% z-0>n>Xz2DnrA4e*=vuCX-mBe&1dmLKlP=7(=k!JfKQ2?;kso9aiXg%3d=1WB%GBP@ zQvM1Kr@$4d2uD(=t!>=5E}(Gn>2{sg&3oUqmrG2Hd%KTajcT<}AHUzt4VeaGOkxPl@=Z85C)vYY9)nu}QSW>09Sq+4 zy({MYX2D;G6}G=i8vIxw}gl=_rk)TEb)l^lS=EtMd@d53b`8nk*$Nsi5Sv3`g@b zcHU-c0RvBsgcEmD`|u49L@tenz*y+^IihrDfL9f zNSr)v(3@hrhyy*pme^=S`q?qjbUNKM>%*7r>v7$f z-H}XIpa-qTlNbSc8#?I;J zgXzZZ)Rq%AF8Z{7#$Hb5xE>IF4Vr?|6#OuZD3Cj1NNuZ~x#pP8)3$s%Q7n36?AamH z^gBbYq-Y=C!^W5aWT=)k16?FK=3$Gaz!>M(?OXz4AZ!WbCoPd%KWwGOi)Vrl{h*Wp z#U;}n?!M+}7d#i0rgeD^W*Q_n_L#IA4yyaa9ceqedzPQ`d6tcno_srY*Q9-}S)*KU z$JmHBHI6vel$=e9)Z+1|OyJ4`@4ZBvX!@%$sgIIE7SIO0OneRtkcnaJ5}Ab`z>W%+U3?GB2ko-|mm$b_|T zQP?V((E3)D1HM;iG4m%gJTY}(vkiD7o})HjI5ws~(>n?I%dS$d!#SVo@_DJ8VZQHK zLl4gFBp{QQD8Etc5dJ6MyEL8#{bR9F941zRa6&S&+Wp@5zK?XYe&F+W5{HLS+7piG zeeDw=$53}M$0J~QsLZAl)$4n4XqYyTQ&4SmwZzgpZ{o{M3JQk1dfNBRovQ917>aIZ z=%_tOPf+L*msFskw^aVVBT+?WoAV482S`5H3XMxxDWxD!G}2yOEkg2IKm40^2|77y zYTu~nfn>cGqS%Rk_tJa25mwvYlR8?o7CU)}{R zOE~k@hoeStMTngO6NhY|h3hbQHktF# z&r;AlaYtZ0PEpw#9kT!#^r1<=kydW?^rl4p{H&i?6-Rs09=l=nifomzvR=LX;ujmM zcv5?iRN#}Wvnd0WzF_|`SMsqALv)Wyvgsgiy}G0H54!E2v0^2kCzL^%(De0?8byLd zZRD$ZRn4$QKt$YIxyZ+o*_%I^4hYH7YKrAdR*JUcC*h(o$xfTkqAb1WOA!6Z2t)p{ z);RuLg-9VW6{*(WBA@Z9^qiv8i-pIhija!k(FeTq=?OG$gb*lO@UHXn8@I!Ve68YQ zf+XE?UtpQlsg%ciWR|MNzco;Snz!E>6bzex-voEazp!c%8}NGZ*Hb~+v`9M$OJe5W z)yRiYp2GX%?}-gmMGKjZ?mpAO^CvhrqO#oV*)Iv;`7`Mu9geYh#@mPEAT^n1EU;_Y zPfl}^TU_Fob!!jhH*Rq2eU803uj+o!NQb&KhutItuSzRskzELM9Iv=pg7&fp4@-qn zRj$eCO*%sH(~$s0M%gOxh}`30*@FS4(`mvBF88>#UA;BZnkhItRBSb;`9$t;2@nsK z&&m-@ygLIB@eRBq$J*cID6MaRlwKUo{3nK2MHh7LCD+)6%iMo93anj*6n zzjs8@b7NgLgtV->ub{^0o7AJ}k897%O@_SN)i$iSUcn&M%_28q8r;PnSER>Dqj zX9%gBvFqj~X8{<{bi6h^EodQJi6jD26JPn?KB=%x@pH(JY@dvj+yDT)YcNSF-O(S2&d!9{4lZQ8mrJ3Mc_i> zE}CDDYwlG|j&5np&WNr}l6N#-#qqS|-MzF|*~6nm9$3R}b(W+wqnq||i`ugYT)#oHrDy_`qKZbH#|moIY-S2a;@j6=aeY~E zKvrwX<-z})IbO3{e6)^#H_mnW#e-G1#!ITGaLv@RqA(2Z(9*vN@-(=9?0O;CTq-1_ z=H43qF&Y8un#iBmKs2^0bBFXJTCsZf#7Ii3RWCdaZXM z-sCU3h&)56r3&pYa?#H*`>^NtbzQ@^mH`4$a`Cy&090sm2vlUT@u{g7@n++_7M8|{ z6?~9$dJ2}Q9Xfm+ptk0`bMT3w*z{v@ol{@#ZgKX2sKW}EOgnpr%EE4b!N7-oJ~lHW z(9wbk(O;SML&GHX0vJ!RaLYCFW+sg%C~`8%H5atWj2dcRN+JjUAp}!@GC^-x zRq?>HW*=diej+HoJNVgbR-pWwnh!!8(}QtemYaL#)eg;pu-+hn^w|6?zTk|`(n98; zML6_;B+89U4g6NbYCAc-?6FzMVEJdMfsI}evlSql2B=ww{K>V)$FHqJ_@Y`tD%>M1 zBJk{LV(w2qdi}+|N{+d@WskEE@7PrDZ>65L!I7okmKPT`7YvbA3c*8OGmYsLcG?;Vb+g}2JoOfMZ5L?iHpnJ7YwDQ@{yzT0ibYx6CCy~K z5*OJX#!_r_h?U+(-f}&Dh(p1j>*_3g&Ct7=L~8*+>4|zD^G0-D0K4GL9L_p;E!S1(MLsO*xd|9qsreC z9aF-c-#|W?;izum$a6EyL72S;6E@@Pesy@>o*ZNw_YPP0TEZ!C z1WevoDN>!V$#BG^M9ABMWs7WBF(1ffS9+EAjbCkXUcCJ=Y=4BB?TsWsMI6<$&cs+~ zZ^trU^aS8w1(kbiX-2oB=+^#3;)JlxIah5eYtVaLE}w<$&6|=pe`Xi$4JLKVq<|(< zjHP<$+FZpw`e|A!DEK&HxZ>M1y?H5@@m?rw!iOlz;v&ZN>DIKfA>K|*UrO{~n4 z9+*k*OkvD>$+{+ZE-R^&Qppnzk zL0f&XWR2_p@@b{|$qd>UUT3bg9Z-g=EAn*;{It_i0OWxu# zmdIf{b8CTM_?K8L^8@QUNtb|{@S?Xz3&LwqfP4sb zIj3i4w>R}_Kun267o&pBE%8oJReu83N4cFk#DtPDwfSVJxWIF7Z}Vh8d5xC8?IygjEeI_k>vIp~l34HzXOz+P>2YrdZA+>4v<2mFsMm5N z)wfK*w`Vp`3Xj<4NQ5Sbvik$jRPOzR7o}4 zP4-FDP3kFzHXUn8<6_KH~kQ&jUH@IguNxrNKr`S(HUdEB>kmlJ(Gc<5|;>GznuRols#=6wk;J5 zx;*s%m;TZM-Bjz@L-DWWf3GpI`lmPXKQ>{0ARqSs%Kr->7ff5pNcXyW?(Y @code_typed applyf([100,200]) +julia> applyf([100]) +1 + +julia> @code_typed applyf([100]) CodeInfo( 1 ─ Base.arrayref(true, container, 1)::Int64 -│ Base.arrayref(true, container, 2)::Int64 -└── return 2 +└── return 1 ) => Int64 ``` -The compiler itself knows that the answer will be 2, as long as the input array -has elements indexable by 1 and 2. (Those `arrayref` statements enforce +The compiler knows that the answer will be 1, as long as the input array +has an element indexable by 1. (That `arrayref` statement enforces bounds-checking, and ensure that Julia will throw an appropriate error if you -call `applyf([100])`.) +call `applyf([])`.) -If you pass a `Vector{Bool}`, it will compile `applyf` again, this time specializing it for `Bool` elements: +For the purpose of this blog post, things start to get especially interesting if we use a container that can store elements with different types (here, type `Any`): ```julia-repl -julia> @code_typed applyf([true,false]) -CodeInfo( -1 ─ Base.arrayref(true, container, 1)::Bool -│ Base.arrayref(true, container, 2)::Bool -└── return 4 -) => Int64 -``` - -In this case, you can see that Julia knew those two `arrayref` statements would -return a `Bool`, and since it knows the value of `f(::Bool)` it just went -ahead and computed the result at compile time for you. - -After calling `applyf` with both sets of arguments, hidden away in Julia's "method cache" there will -be two `MethodInstance`s of `applyf`, one specialized for `Vector{Int}` and the other specialized for `Vector{Bool}`. -You don't normally see these, but Julia manages them for you; anytime you write -code that calls `applyf`, it checks to see if this previous compilation work can be reused. - -For the purpose of this blog post, things start to get especially interesting if use a container that can store elements with different types (here, type `Any`): - -```julia-repl -julia> c = Any[1, false]; +julia> c = Any[100]; julia> applyf(c) -3 +1 julia> @code_typed applyf(c) CodeInfo( -1 ── %1 = Base.arrayref(true, container, 1)::Any -│ %2 = (isa)(%1, Bool)::Bool -└─── goto #3 if not %2 -2 ── goto #6 -3 ── %5 = (isa)(%1, Int64)::Bool -└─── goto #5 if not %5 -4 ── goto #6 -5 ── %8 = Main.f(%1)::Int64 -└─── goto #6 -6 ┄─ %10 = φ (#2 => 2, #4 => 1, #5 => %8)::Int64 -│ %11 = Base.arrayref(true, container, 2)::Any -│ %12 = (isa)(%11, Bool)::Bool -└─── goto #8 if not %12 -7 ── goto #11 -8 ── %15 = (isa)(%11, Int64)::Bool -└─── goto #10 if not %15 -9 ── goto #11 -10 ─ %18 = Main.f(%11)::Int64 -└─── goto #11 -11 ┄ %20 = φ (#7 => 2, #9 => 1, #10 => %18)::Int64 -│ %21 = Base.add_int(%10, %20)::Int64 -└─── return %21 +1 ─ %1 = Base.arrayref(true, container, 1)::Any +│ %2 = (isa)(%1, Int64)::Bool +└── goto #3 if not %2 +2 ─ goto #4 +3 ─ %5 = Main.f(%1)::Core.Const(1, false) +└── goto #4 +4 ┄ %7 = φ (#2 => 1, #3 => %5)::Core.Const(1, false) +└── return %7 ) => Int64 ``` This may seem a lot more complicated, but the take-home message is actually -quite simple. First, look at those `arrayref` statements: they are annotated -`Any`, meaning that Julia can't predict in advance what they will return. -Immediately after each reference, notice that there are two `isa` statements +quite simple. First, look at that `arrayref` statement: it is annotated +`Any`, meaning that Julia can't predict in advance what it will return. +Immediately after the reference, notice the `isa` statement followed by a `ϕ`; all of this is essentially equivalent to -``` -if isa(x, Bool) - value = 2 -elseif isa(x, Int) +```julia +if isa(x, Int) value = 1 else value = f(x)::Int end ``` -This is [union-splitting], a performance optimization for cases where an object might be of one of several types. - -### Triggering method invalidation +The compiler might not know in advance what type `container[1]` will have, but it knows there's a method of `f` specialized for `Int`. +To improve performance, it checks (as efficiently as possible at runtime) whether that method might be applicable, +and if so calls the method it knows. +In this case, `f` just returns a constant, so when applicable the compiler even "hard-wires" the return value in for you. -One point in particular is worth noting: what's up with that final `f(x)::Int` call? -Didn't it already handle all the possibilities? -Well, all the possibilities *so far*. -But we might next define some new method of `f` for a different type: +However, the compiler also acknowledges the possiblity that `container[1]` *won't* be an `Int`. +That allows the above code to throw a MethodError: +```julia-repl +julia> applyf(Any[true]) +ERROR: MethodError: no method matching f(::Type{Bool}) +Closest candidates are: + f(::Int64) at REPL[1]:1 +[...] ``` -f(::String) = 3 + +That's similar to what happens if we see how a well-typed call gets inferred: + +```julia-repl +julia> @code_typed applyf([true]) +CodeInfo( +1 ─ %1 = Base.getindex(container, 1)::Bool +│ Main.f(%1)::Union{} +└── unreachable +) => Union{} ``` +where in this case the compiler knows that the call won't succeed. -and Julia has prepared the way to make sure it will still give the right answer even if we add new methods to `f`. +### Triggering method invalidation -However, if you try `@code_typed applyf(Any[1, false])` again, you'll notice something curious: -Julia has gone to the trouble to create a new-and-improved implementation of `applyf`, -one which also union-splits for `String`. -This brings us to the topic of this blog post: the old compiled method has been *invalidated*. -Given new information--which here comes from defining or loading new methods--Julia changes its mind about how things should be implemented, -and this forces Julia to recompile `applyf`. +**NOTE**: This demo is begin run on Julia's master branch (which will become 1.6). +Depending on your version of Julia, you might get different results and/or need to define more than one +additional method for `f` to see the outcome shown here. -If you add fourth and fifth methods, +Julia is interactive: we can define new methods on the fly and try them out. +Let's see what happens when we define a new method for `f`: -``` -f(::AbstractArray) = 4 -f(::Missing) = 5 +```julia +f(::Bool) = 2 ``` -then Julia produces +While the `Vector{Int}`-typed version of `applyf` is not changed by this new definition (try it and see), +both the `Bool`-typed version and the untyped version are different than previously: -```julia-repl -julia> @code_typed applyf(c) +```julia +julia> @code_typed applyf([true]) +CodeInfo( +1 ─ Base.arrayref(true, container, 1)::Bool +└── return 2 +) => Int64 + +julia> @code_typed applyf(Any[true]) CodeInfo( 1 ─ %1 = Base.arrayref(true, container, 1)::Any -│ %2 = Main.f(%1)::Any -│ %3 = Base.arrayref(true, container, 2)::Any -│ %4 = Main.f(%3)::Any -│ %5 = (%2 + %4)::Any -└── return %5 -) => Any +│ %2 = Main.f(%1)::Int64 +└── return %2 +) => Int64 ``` -There are now so many possibilities that Julia just gives up and -uses "runtime dispatch" to decide what method of `f` to call. -It doesn't even try to enforce the fact that `f` returns an `Int`, -in part because determining such facts takes time (adding to compiler latency) -and because functions with many methods typically tend to return multiple types -anyway. Adding further methods of `f` would no longer cause invalidations of this very generic implementation or any of its callers. +This shows that defining a new method for `f` forced *recompilation* of these `MethodInstance`s. -Compiling each of these new implementations takes JIT-time. -If Julia knew in advance that you'd arrive at this place, it would never have bothered to produce that first, heavily-optimized version of `applyf`. -But the performance benefits of such optimizations are so large that, when applicable, they can be well worth it. -For example, if you start a fresh Julia session and just define the `f(::Int)` -and `f(::Bool)` methods, then - -```julia-repl -julia> using BenchmarkTools - -julia> @btime applyf($c) - 4.659 ns (0 allocations: 0 bytes) -3 +You can see that Julia no longer uses that optimization for when `container[1]` happens to be an `Int`. +Experience has shown that once a method has a couple of specializations, it may accumulate yet more, +and to reduce the amount of recompilation needed Julia quickly abandons attempts to optimize the +case handling containers with unknown element types. +If you look carefully, you'll notice one remaining optimization: the call to `Main.f` is type-asserted as `Int`, +since both known methods of `f` return an `Int`. +This can be a very useful performance optimization for any code that uses the output of `applyf`. +But define two additional methods of `f`, +```julia julia> f(::String) = 3 -f (generic function with 3 methods) - -julia> f(::AbstractArray) = 4 f (generic function with 4 methods) -julia> f(::Missing) = 5 -f (generic function with 5 methods) +julia> f(::Dict) = 4 +f (generic function with 4 methods) -julia> @btime applyf($c) - 33.537 ns (0 allocations: 0 bytes) -3 +julia> @code_typed applyf(Any[true]) +CodeInfo( +1 ─ %1 = Base.arrayref(true, container, 1)::Any +│ %2 = Main.f(%1)::Any +└── return %2 +) => Any ``` -It's almost a tenfold difference. -If `applyf` is performance-critical, you'll be very happy that Julia tries to give you the best version it can, given the available information. -But this leaves the door open to invalidation, which means recompilation the next time you use `applyf`. -If method invalidation happens often, this might contribute to making Julia feel sluggish. + +and you'll see that even this optimization is now abandoned. +This is because checking lots of methods, to see if they all return `Int`, is too costly an operation to be worth doing in all cases. + +Altogether, `apply(::Vector{Any})` has been compiled three times: once when `f(::Int)` was the only method for `f`, +once where there were two methods, and once when there were four. +Had Julia known in advance that we'd end up here, it never would have bothered to compile those intermediate versions; +recompilation takes time, and we'd rather avoid spending that time if it's not absolutely necessary. +Fortunately, from where we are now Julia has stopped making assumptions about how many methods of `f` there will be, +so from this point forward we won't need to recompile `applyf` for a `Vector{Any}` even if we define further methods of `f`. + +Recompilation is triggered by two events: +- when a new method is defined, old compiled methods that made now-incorrect assumptions get *invalidated* +- when no valid compiled method can be found to handle a call, Julia (re)compiles the necessary code. + +It's the moment of method definition, triggering invalidations of other previously-compiled code, +that will be the focus of this blog post. ## How common is method invalidation? -Unfortunately, method invalidation is pretty common. +Unfortunately, method invalidation is (or rather, used to be) pretty common. First, let's get some baseline statistics. -Using the [MethodAnalysis] package (which is at a very early stage of development -at the time of this writing), you can find out that a fresh Julia session has almost 50,000 `MethodInstance`s tucked away in its cache. +Using the [MethodAnalysis] package, you can find out that a fresh Julia session has almost 50,000 `MethodInstance`s tucked away in its cache. These are mostly for `Base` and the standard libraries. (There are some additional `MethodInstance`s that get created to load the MethodAnalysis package and do this analysis, but these are surely a very small fraction of the total.) -Using some not-yet merged work in both Julia itself and [SnoopCompile], we can count the number of invalidations when we load various packages into a fresh Julia session: - -| Package | Version | # of unique invalidations | -|:------- | -------:| ------------------:| -| Example | 0.5.3 | 0 | -| Revise | 2.6.6 | 27 | -| FixedPointNumbers | 0.8.0 | 429 | -| SIMD | 2.8.0 | 2799 | -| StaticArrays | 0.12.3 | 2852 | -| Optim | 0.21.0 | 3171 | -| Images | 0.22.2 | 3638 | -| Flux | 0.10.4 | 3697 | -| Plots | 1.2.3 | 4002 | -| DataFrames | 0.21.0 | 4048 | -| JuMP | 0.21.2 | 4666 | -| Makie | 0.10.0 | 6118 | -| DifferentialEquations | 6.13.0 | 6777 | - -You can see that key packages used by large portions of the Julia ecosystem invalidate -hundreds or thousands of `MethodInstance`s, sometimes more than 10% of the total +Using [SnoopCompile], we can count the number of invalidations triggered by loading various packages into a fresh Julia session: + +| Package | Version | # invalidations (Julia 1.5) | # invalidations (Julia 1.6) | +|:------- | -------:| ---------:| ---------:| +| Example | 0.5.3 | 0 | 0 | +| Revise | 2.7.3 | 23 | 0 | +| FixedPointNumbers | 0.8.4 | 335 | 22 | +| StaticArrays | 0.12.4 | 2181 | 47 | +| Images | 0.22.4 | 2881 | 332 | +| Optim | 0.22.0 | 2902 | 155 | +| SIMD | 2.8.0 | 2949 | 13 | +| Plots | 1.5.8 | 3156 | 278 | +| Makie | 0.11.1 | 3273 | 306 | +| Flux | 0.10.4 | 3461 | x | +| DataFrames | 0.21.6 | 4126 | 2822 | +| JuMP | 0.21.3 | 4281 | 2215 | +| DifferentialEquations | 6.15.0 | 6373 | 3251 | + +('x' indicates that the package cannot be loaded) + +You can see that key packages used by large portions of the Julia ecosystem have traditionally +invalidated hundreds or thousands of `MethodInstance`s, sometimes more than 10% of the total number of `MethodInstance`s present before loading the package. +The situation has been dramatically improved on Julia 1.6, but there remains more +work left to do for particular packages. -## How serious is method invalidation? +## What are the impacts of method invalidation? The next time you want to call functionality that gets invalidated, you have to wait for recompilation. @@ -277,7 +265,7 @@ julia> @time display(plot(rand(5))) 0.305394 seconds (19.96 k allocations: 781.836 KiB) ``` -But if you load a package that does a lot of invalidation: +But if you load a package that does a lot of invalidation (on Julia versions 1.5 and below), ```julia-repl julia> using SIMD @@ -287,477 +275,80 @@ julia> @time display(plot(rand(5))) ``` Because so much got invalidated by loading SIMD, Julia had to recompile many methods before it could once again produce a plot, so that in terms of time it was almost as expensive as the first usage. -The size of the effect varies substantially depending on what task you are trying to achieve and what packages you load to do the invalidation. +The size of the effect is strongly dependent on the particular packages and tasks, +and this particular example is already fixed on Julia 1.6. + +This doesn't just affect plotting or packages. For example, loading packages could transiently make the REPL, the next Pkg update, or the next call to a distributed worker lag for several seconds. -It's worth noting that you can escape much of this cost by loading all -packages at the outset. -Loading time is somewhat increased by invalidation, but the cost of -first-time usage is typically larger. When possible, it's best to get -the invalidation out of the way before starting your work. +You can minimize some of the costs of invalidation by loading all packages at the outset--if all invalidations happen before you start compiling very much code, then the first compilation already takes the full suite of methods into account. + +On versions of Julia prior to 1.5 and 1.6, package load time was substantially increased by invalidation. +Especially on Julia 1.6, invalidation only rarely affects package load times, +largely because Pkg and the loading code have been made reasonably resistant to invalidation using some of the strategies described below. ## Can and should this be fixed? Invalidations affect latency on their own, but they also impact other potential strategies for reducing latency. For example, julia creates "precompile" (`*.ji`) files to speed up package usage. -Currently, it saves type-inferred but not "native" code to its precompile files. -In principle, saving native code would eliminate latency for the method and type combinations that have been precompiled, but this will be ineffective if most of this code ends up getting invalidated. -Indeed, it could make it even worse, because you're doing work (loading more stuff from disk) without much reward. -Consequently, reducing invalidation seems likely to be useful on its own, and a necessary prerequisite to other potential strategies for reducing latency. - -As to "can it be fixed?", that depends on the goal. +Currently, it saves type-inferred but not [native code](https://www.techopedia.com/definition/3846/native-code) to its precompile files. +In principle, we could greatly reduce latency by also saving native code, since that would eliminate the need to do any compilation at all. +However, this strategy would be ineffective if most of this code ends up getting invalidated. +Indeed, it could make latency even worse, because you're doing work (loading more stuff from disk) without much reward. +However, if we get rid of most invalidations, then we can expect to get more benefit from caching native code, +and so if the community eliminates most invalidations then it begins to make much more sense to work to develop this new capability. + +As to "can invalidations be fixed?", that depends on the goal. We will never get rid of invalidation altogether; as the `applyf` example above shows, invalidation is sometimes necessary if you want both good runtime performance and interactive usage, and this combination is one of the best things about Julia. The real question is whether there are *unnecessary* invalidations, or an unexploited strategy to limit their impact. -Determining the answer to that question requires that we develop an understanding the common reasons for the large number of invalidations listed in the table above. - -## An analysis of the causes of invalidation - -This section relies on a recent [pull request to Julia][PRJulia] and -the [invalidations branch][PRSC] of [SnoopCompile]. -If you try to replicate these results, remember that invalidations occur only -for methods that have been compiled, which generally means you have to -execute them first. - -As we analyze causes of invalidation, you'll note that in some cases we can begin to think about how they might be fixed. -However, we'll save more detailed recommendations for the final section. - -### New methods with greater specificity - -It will be simplest to start with a case we already understand, the `applyf` example above. In a fresh Julia session, - -``` -f(x::Int) = 1 -f(x::Bool) = 2 -function applyf(container) - x1 = f(container[1]) - x2 = f(container[2]) - return x1 + x2 -end -c = Any[1, false]; -applyf(c) -``` - -Then, - -```julia-repl -julia> using SnoopCompile - -julia> invalidation_trees(@snoopr f(x::String) = 3) -1-element Array{SnoopCompile.MethodInvalidations,1}: - insert f(x::String) in Main at REPL[7]:1 invalidated: - mt_backedges: signature Tuple{typeof(f),Any} triggered MethodInstance for applyf(::Array{Any,1}) (0 children) more specific -``` - -Let's walk through this output a bit. -`@snoopr` turns on some debugging code inside Julia, and then executes the supplied statment; -it returns a fairly opaque list that can be parsed by `invalidation_trees`. -Entries in the array returned by `invalidation_trees` correspond to method additions (or deletions, if relevant) that trigger one or more invalidations. -In this case, the output means that the new `f(x::String)` method triggered an invalidation of `applyf(::Array{Any,1})`, -due to intersection with the signature `f(::Any)`. -`(0 children)` means that `applyf(::Vector{Any})` does not yet have any methods that called it and which in turn need to be invalidated. -Finally, `more specific` (which is printed in cyan) indicate that the new method `f(::String)` was strictly more specific than the signature `f(::Any)` used by the `applyf` `MethodInstance` that got invalidated. - -As we mentioned above, there are good reasons to think this invalidation is "necessary," meaning that it is an unavoidable consequence of the choices made to optimize runtime performance while also allowing one to dynamically extend functions. -However, that doesn't mean there is nothing that you, as a developer, could do to eliminate this invalidation. -Perhaps there is no real need to ever call `applyf` with a `Vector{Any}`; -perhaps you can fix one of its upstream callers to supply a concretely-type vector. -Or perhaps you could define more `f` methods at the outset, so that Julia has a better understanding of the different types that `applyf` needs to handle. -In some cases, though, you might really need to call `applyf` with a `Vector{Any}`, in which case the best choice is to accept this invalidation as necessary and move on. - -### New methods with ambiguous specificity +Determining the answer to that question requires that we develop an understanding the common sources of invalidation and what can be done to fix them. -Now let's try a real-world case, where the outcomes are more complex. +## Changes in Julia that have reduced the number of invalidations -```julia-repl -julia> trees = invalidation_trees(@snoopr using FixedPointNumbers) -6-element Array{SnoopCompile.MethodInvalidations,1}: - insert one(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:94 invalidated: - mt_backedges: signature Tuple{typeof(one),Type{T} where T<:AbstractChar} triggered MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (0 children) ambiguous - 1 mt_cache - - insert oneunit(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:93 invalidated: - backedges: superseding oneunit(::Type{T}) where T in Base at number.jl:300 with MethodInstance for oneunit(::Type{T} where T<:AbstractChar) (1 children) more specific - 3 mt_cache - - insert promote_rule(::Type{T}, ::Type{Tf}) where {T<:Normed, Tf<:AbstractFloat} in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/normed.jl:310 invalidated: - backedges: superseding promote_rule(::Type{var"#s822"} where var"#s822"<:AbstractIrrational, ::Type{T}) where T<:Real in Base at irrationals.jl:42 with MethodInstance for promote_rule(::Type{Union{}}, ::Type{Float64}) (1 children) ambiguous - superseding promote_rule(::Type{var"#s92"} where var"#s92", ::Type{var"#s91"} where var"#s91") in Base at promotion.jl:235 with MethodInstance for promote_rule(::Type{S} where S<:Integer, ::Type{Float64}) (1 children) more specific - 6 mt_cache - - insert sizeof(::Type{X}) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:100 invalidated: - backedges: superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::DataType) (26 children) more specific - superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::Type) (4 children) more specific - superseding sizeof(x) in Base at essentials.jl:449 with MethodInstance for sizeof(::Type{T} where T) (1 children) more specific - 4 mt_cache - - insert reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 invalidated: - backedges: superseding reduce_empty(op, T) in Base at reduce.jl:309 with MethodInstance for reduce_empty(::Function, ::Type{T} where T) (137 children) more specific - - insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: - mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous - backedges: superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) (187 children) ambiguous - superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (198 children) ambiguous - 3 mt_cache -``` - -This list is ordered from least- to most-consequential in terms of total number of invalidations. -The final entry, for `(::Type{X})(x::Real) where X<:FixedPoint`, triggered the invalidation of what nominally appear to be more than 350 `MethodInstance`s. -(There is no guarantee that these methods are all disjoint from one another; -the results are represented as a tree, where each node links to its callers.) -In contrast, the first three entries are responsible for a tiny handful of invalidations. - -One does not have to look at this list for very long to see that the majority of the invalidated methods are due to [method ambiguity]. -Consider the line `...char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32)`. -We can see which method this is by the following: - -```julia-repl -julia> which(Char, (Int32,)) -(::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 -``` - -or directly as - -```julia-repl -julia> tree = trees[end] -insert (::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 invalidated: - mt_backedges: signature Tuple{Type{T} where T<:Int64,Int64} triggered MethodInstance for convert(::Type{T}, ::Int64) where T<:Int64 (1 children) ambiguous - backedges: superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) (187 children) ambiguous - superseding (::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 with MethodInstance for (::Type{T} where T<:AbstractChar)(::UInt32) (198 children) ambiguous - 3 mt_cache - -julia> tree.method -(::Type{X})(x::Real) where X<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:51 +Much of the progress in reducing invalidations in Julia 1.6 come from changes to generic mechanisms in the compiler. +A [crucial change](https://github.com/JuliaLang/julia/pull/36733) was the realization that some invalidations were formerly triggered by new methods of *ambiguous specificity* for particular argument combinations; since calling a function with this argument combination would have resulted in a `MethodError` anyway, it is unnecessary to invalidate their (nominal) dependents. -julia> node = tree[:backedges,1] -MethodInstance for (::Type{T} where T<:AbstractChar)(::Int32) at depth 0 with 187 children +Another [pair](https://github.com/JuliaLang/julia/pull/36208) of [fundamental](https://github.com/JuliaLang/julia/pull/35904) changes led to subtle but significant alterations in how much Julia specializes poorly-inferred code to reflect the current state of the world. +By making Julia less eager to specialize in such circumstances, huge swaths of the package ecosystem became more robust against invalidation. -julia> node.mi.def -(::Type{T})(x::Number) where T<:AbstractChar in Base at char.jl:48 -``` - -`trees[end]` selected the last (most consequential) method and the invalidations it triggered; indexing this with `:backedges` selected the category (`:mt_backedges`, `:backedges`, or `:mt_cache`), and the integer index selected the particular entry from that category. -This returns an `InstanceTree`, where the latter is a type encoding the tree. -(`:mt_backedges` will return a `sig=>node` pair, where `sig` is the invalidated signature.) - -You may find it surprising that this method signature is ambiguous with `(::Type{X})(x::Real) where X<:FixedPoint`: after all, an `AbstractChar` is quite different from a `FixedPoint` number. -We can discover why with - -```julia-repl -julia> tree.method.sig -Tuple{Type{X},Real} where X<:FixedPoint - -julia> node.mi.specTypes -Tuple{Type{T} where T<:AbstractChar,Int32} - -julia> typeintersect(tree.method.sig, node.mi.specTypes) -Tuple{Type{Union{}},Int32} -``` - -These two signatures have non-empty intersection. -The second parameter, `Int32`, makes sense as the intersection of `Int32` and `Real`. -The first arises from - -```julia-repl -julia> typeintersect(Type{<:FixedPoint}, Type{<:AbstractChar}) -Type{Union{}} -``` +Most of the remaining improvements stem from more directed changes that improved inferrability of dozens of specific methods. You find many examples by searching for the [latency tag](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+label%3Alatency+is%3Aclosed) among closed pull requests. -which shows that there is one Type, the "empty Type", that lies in their intersection. +Most of these changes were at least partially inspired by new tools to analyze the source of invalidations, and we briefly touch on this final topic below. -There are good reasons to believe that the right way to fix such methods is to exclude ambiguous pairs from invalidation--if it were to be called by the compiled code, it would trigger an error anyway. -If this gets changed in Julia, then all the ones marked "ambiguous" should magically disappear. -Consequently, we can turn our attention to other cases. +## Tools for analyzing and fixing invalidations -For now we'll skip `trees[end-1]`, and consider `tree[end-2]` which results from defining -`sizeof(::Type{X}) where X<:FixedPoint`. -There's a perfectly good default definition, and this looks like a method that we don't need; perhaps it dates from some confusion, or an era where perhaps it was necessary. -So we've discovered an easy place where a developer could do something to productively decrease the number of invalidations, in this case by just deleting the method. +Recently, the [SnoopCompile] package gained the ability to analyze invalidations and help developers fix them. +Because these tools will likely change over time, this blog post will only touch the surface; people who want to help fix invalidations are encouraged to read [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) for further detail. +There is also a [video]() available with a live-session fixing a real-world invalidation, which might serve as a useful example. -Rarely (in other packages) you'll notice cases where the new method is *less specific*. -It is not clear why such methods should be invalidating, and this may be either a SnoopCompile or Julia bug. +But to give you a taste of what this looks like, here are a couple of screenshots. +These were taken in Julia's REPL, but you can also use these tools in [vscode] or other environments. -### Partial specialization +First, let's look at a simple way (one that is not always precisely accurate, see SnoopCompile's documentation) to collect data on a package (in this case, loading the SIMD package): -Let's return now to +![snoopr_simd](/assets/blog/2020-invalidations/SIMD_invalidations.png) -```julia-repl -julia> tree = trees[end-1] -insert reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 invalidated: - backedges: superseding reduce_empty(op, T) in Base at reduce.jl:309 with MethodInstance for reduce_empty(::Function, ::Type{T} where T) (137 children) more specific - -julia> node = tree[:backedges, 1] -MethodInstance for reduce_empty(::Function, ::Type{T} where T) at depth 0 with 137 children -``` - -Our new method certainly looks more specific method than the one that triggered the invalidation, -so at face value this looks like one of those "necessary" invalidations we'd be hard-pressed to avoid. -However, appearances can be deceptive. -We can look at the callers of this `reduce_empty` method: - -```julia-repl -julia> node.children -5-element Array{SnoopCompile.InstanceTree,1}: - MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) at depth 1 with 39 children - MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{Int64}) at depth 1 with 39 children - MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Pkg.Resolve.FieldValue}) at depth 1 with 21 children - MethodInstance for mapreduce_empty(::typeof(identity), ::Pkg.Resolve.var"#132#134"{Pkg.Resolve.var"#smx#133"{Pkg.Resolve.Graph,Pkg.Resolve.Messages}}, ::Type{Int64}) at depth 1 with 10 children - MethodInstance for mapreduce_empty(::typeof(identity), ::typeof(max), ::Type{Int64}) at depth 1 with 23 children -``` - -If we look at the source for these definitions, we can figure out that they'd call `reduce_empty` with one of two functions, `max` and `identity`. -Neither of these is consistent with the method for `add_sum` we've defined: - -```julia-repl -julia> tree.method -reduce_empty(::typeof(Base.add_sum), ::Type{F}) where F<:FixedPoint in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/w2pxG/src/FixedPointNumbers.jl:222 -``` - -What's happening here is that we're running up against the compiler's heuristics for specialization: -it's not actually possible that any of these callers would end up calling our new method, -but because the compiler decides to create a "generic" version of the method, the signature gets flagged by the invalidation machinery as matching. - - -### Some summary statistics - -Let's go back to our table above, and count the number of invalidations in each of these categories: - -| Package | more specific | less specific | ambiguous | -|:------- | ------------------:| --------:| -----:| -| Example | 0 | 0 | 0 | 0 | -| Revise | 7 | 0 | 0 | -| FixedPointNumbers | 170 | 0 | 387 | -| SIMD | 3903 | 0 | 187 | -| StaticArrays | 989 | 0 | 3133 | -| Optim | 1643 | 0 | 2921 | -| Images | 1749 | 14 | 3671 | -| Flux | 1991 | 26 | 3460 | -| Plots | 1542 | 11 | 4302 | -| DataFrames | 4919 | 0 | 783 | -| JuMP | 2145 | 0 | 4670 | -| Makie | 6233 | 46 | 5526 | -| DifferentialEquations | 5152 | 18 | 6218 | - -The numbers in this table don't add up to those in the first, for a variety of reasons (here there is no attempt to remove duplicates, here we don't count "mt_cache" invalidations which were included in the first table, etc.). -In general terms, the last two columns should probably be fixed by changes in how Julia does invalidations; the first column is a mixture of ones that might be removed by changes in Julia (if they are due to partial specialization) or ones that should either be fixed in packages, Base and the standard libraries, or will need to remain unfixed. -The good news is that these counts reveal that more than half of all invalidations will likely be fixed by "automated" means. -However, it appears that there will need to be a second round in which package developers inspect individual invalidations to determine what, if anything, can be done to remediate them. - -## Fixing invalidations - -You may have noticed that two packages, `Example` and `Revise`, trigger far fewer invalidations that the rest of the packages in our analysis. -`Example` is quite trivial, but `Revise` and its dependencies are quite large. -How does it avoid this problem? -First, Revise does not extend very many Base methods; -most of its methods are for functions it "owns," and the same is true for its dependencies. -Second, in the closing days of Julia 1.5's merge window, -Revise (and Julia) underwent a process of tracking down invalidations and eliminating them; -for comparison, on Julia 1.4, Revise triggers more than a 1000 non-unique invalidations. -The success of this effort gives one hope that other packages too may one day have fewer invalidations. - -As stated above, there is reason to hope that most of the invalidations marked as "ambiguous" will be fixed by changes to Julia's compiler. -Here our focus is on those marked "more specific," since those are cases where it is harder to imagine a generic fix. - -### Fixing type instabilities - -Most of the time, Julia is very good at inferring a concrete type for each object, -and when successful this eliminates the risk of invalidations. -However, as illustrated by our `applyf` example, certain input types can make inference impossible. -Fortunately, it's possible to write your code in ways that eliminate or reduce the impact of such invalidations. - -An excellent resource for avoiding inference problems is Julia's [performance tips] page. -That these tips appear on a page about performance, rather than invalidation, is a happy state of affairs: -not only are you making your (or Julia's) code more robust against invalidation, you're almost certainly making it faster. - -Virtually all the type-related tips on that page can be used to reduce invalidations. -Here, we'll present a couple of examples and then focus on some of the more subtle issues. - -#### Add annotations for containers with abstractly-typed elements - -As the performance page indicates, when working with containers, concrete-typing is best: -`Vector{Int}` will typically be faster in usage and more robust to invalidation than `Vector{Any}`. -When possible, using concrete typing is highly recommended. -However, there are cases where you sometimes need elements to have an abstract type. -In such cases, one common fix is to annotate elements at the point of usage. - -For instance, Julia's [IOContext] structure is defined roughly as - -``` -struct IOContext{IO_t <: IO} <: AbstractPipe - io::IO_t - dict::ImmutableDict{Symbol, Any} -end -``` - -There are good reasons to use a value-type of `Any`, but that makes it impossible for the compiler to infer the type of any object looked up in an IOContext. -Fortunately, you can help! -For example, the documentation specifies that the `:color` setting should be a `Bool`, and since it appears in documentation it's something we can safely enforce. -Changing - -``` -iscolor = get(io, :color, false) -``` - -to either of - -``` -iscolor = get(io, :color, false)::Bool # the rhs is Bool-valued -iscolor::Bool = get(io, :color, false) # `iscolor` must be Bool throughout scope -``` - -makes computations performed with `iscolor` robust against invalidation. -For example, the SIMD package defines a new method for `!`, which is typically union-split on non-inferred arguments. -Julia's `with_output_color` function computes `!iscolor`, -and without the type annotation it becomes vulnerable to invalidation. -Adding the annotation fixed hundreds of invalidations in methods that directly or indirectly call `with_output_color`. -(Such annotations are not necessary for uses like `if iscolor...`, `iscolor ? a : b`, or `iscolor && return nothing` because these are built into the language and do not rely on dispatch.) - -#### Force runtime dispatch - -In some circumstances it may not be possible to add a type annotation: `f(x)::Bool` will throw an error if `f(x)` does not return a `Bool`. -To avoid breaking generic code, sometimes it's necessary to have an alternate strategy. - - -Above we looked at cases where Julia can't specialize the code due to containers with abstract elements. -There are also circumstances where specialization is undesirable because it would force Julia to compile too many variants. -For example, consider Julia's `methods(f::Function)`: -by default, Julia specializes `myfunc(f::Function)` for the particular function `f`, but for something like `methods` which might be called hundreds or thousands of times with different `f`s and which is not performance-critical, that amount of compilation would be counterproductive. -Consequently, Julia allows you to annotate an argument with `@nospecialize`, and so `methods` is defined as `function methods(@nospecialize(f), ...)`. - -`@nospecialize` is a very important and effective tool for reducing compiler latency, but ironically it can also make you more vulnerable to invalidation. -For example, consider the following definition: - - - -### Redirecting call chains - -Let's return to our FixedPointNumbers `reduce_empty` example above. -A little prodding as done above reveals that this corresponds to the definition - -```julia-repl -julia> node = tree[:backedges, 1] -MethodInstance for reduce_empty(::Function, ::Type{T} where T) at depth 0 with 137 children - -julia> node.mi.def -reduce_empty(op, T) in Base at reduce.jl:309 -``` - -If you look up this definition, you'll see it's - -``` -reduce_empty(op, T) = _empty_reduce_error() -``` - -which indicates that it is the fallback method for reducing over an empty collection, and as you might expect from the name, calling it results in an error: - -```julia-repl -julia> op = Base.BottomRF(Base.max) -Base.BottomRF{typeof(max)}(max) - -julia> Base.reduce_empty(op, VersionNumber) -ERROR: ArgumentError: reducing over an empty collection is not allowed -Stacktrace: - [1] _empty_reduce_error() at ./reduce.jl:299 - [2] reduce_empty(::Function, ::Type{T} where T) at ./reduce.jl:309 - [3] reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{T} where T) at ./reduce.jl:324 - [4] top-level scope at REPL[2]:1 -``` - -This essentially means that no "neutral element" has been defined for this operation and type. - -For the purposes of illustration, let's ignore the fact that this might be a case where the fix might in principle be made in the compiler. -Using ordinary Julia code, can we avoid this fallback? -One approach is to define the method directly: modify Julia to add - -``` -reduce_empty(::typeof(max), ::Type{VersionNumber}) = _empty_reduce_error() -``` - -so that we get the same result but don't rely on the fallback. - -Given our observation above that this *apparent* invalidating method is not actually reachable by this call chain, -another approach is to force the compiler to specialize the method by adding type parameters: - -``` -reduce_empty(op::F, ::Type{T}) where {F,T} = _empty_reduce_error() -``` - -While there's little actual reason to force specialization on a method that just issues an error, in this case it does have the effect of allowing the compiler to realize that our new method is not reachable from this call path. - -For addressing this purely at the level of Julia code, perhaps the best approach is to see who's calling it. We looked at `node.children` above, but now let's get a more expansive view: - -```julia-repl -julia> show(node) -MethodInstance for reduce_empty(::Function, ::Type{T} where T) - MethodInstance for reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) - MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}, ::Base.HasEltype) - MethodInstance for reduce_empty_iter(::Base.BottomRF{typeof(max)}, ::Set{VersionNumber}) - MethodInstance for foldl_impl(::Base.BottomRF{typeof(max)}, ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) - MethodInstance for mapfoldl_impl(::typeof(identity), ::typeof(max), ::NamedTuple{(),Tuple{}}, ::Set{VersionNumber}) - MethodInstance for #mapfoldl#201(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapfoldl), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for mapfoldl(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for #mapreduce#205(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(mapreduce), ::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for mapreduce(::typeof(identity), ::typeof(max), ::Set{VersionNumber}) - MethodInstance for maximum(::Set{VersionNumber}) - MethodInstance for set_maximum_version_registry!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec) - MethodInstance for collect_project!(::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::Dict{Base.UUID,Array{Pkg.Types.PackageSpec,1}}) - MethodInstance for collect_fixed!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Base.UUID,String}) - ⋮ -``` - -This indicates that this invalidation path resulted from a call to `maximum` from a function in `Pkg`, `set_maximum_version_registry!`. A little digging reveals that it is defined as - -``` -function set_maximum_version_registry!(ctx::Context, pkg::PackageSpec) - pkgversions = Set{VersionNumber}() - for path in registered_paths(ctx, pkg.uuid) - pathvers = keys(load_versions(ctx, path; include_yanked=false)) - union!(pkgversions, pathvers) - end - if length(pkgversions) == 0 - pkg.version = VersionNumber(0) - else - max_version = maximum(pkgversions) - pkg.version = VersionNumber(max_version.major, max_version.minor, max_version.patch, max_version.prerelease, ("",)) - end -end -``` - -From that error above, we know that this invalidation is produced by an error-path triggered by trying to reduce over an empty collection. -Interestingly, we can see that `set_maximum_version_registry!` handles an empty collection by other means: -there is, in fact, no chance that `maximum(pkgversions)` will ever reach the error case. -However, the compiler does not realize that, and therefore generates code that has been prepared to call `reduce_empty`, and that makes it vulnerable to invalidation. - -There are a couple of potential fixes. -From here, we see that a potentially better definition `reduce_empty` for `VersionNumber`s might be - -``` -reduce_empty(::Base.BottomRF{typeof(max)}, ::Type{VersionNumber}) = VersionNumber(0) -``` - -In that case we could delete the `length` check and rely on this to produce the desired outcome. -One should take this approach only if one can be convinced that version 0 can act as a universal "neutral element" for reductions involving `max` over `VersionNumber`s. -Adding this to `Pkg` would be type-piracy, since `max`, `BottomRF`, and `VersionNumber` are all defined in `Base`, so fixing it this way would best be done in Base. - -But there are other possible fixes. -A little higher up the tree we see a call to `mapreduce`, and this presents another opportunity because `mapreduce` allows you to supply an `init` value: - -```julia-repl -julia> mapreduce(identity, max, Set(VersionNumber[]); init=VersionNumber(0)) -v"0.0.0" -``` +This prints out a list of method definitions (shown in light purple) that triggered invalidations (shown in yellow). +In very rough terms, you can tell how "important" these are by the number of children; +the ones with more children are listed last. +You can gain more insight into exactly what happened, and what the consequences were, +with `ascend`: -Perhaps we could just call this instead of `maximum`. -However, it's a bit uglier than the original; -perhaps a nicer approach would be to allow one to supply `init` as a keyword argument to `maximum` itself. -While this is not supported on Julia versions up through 1.5, it's a feature that seems to make sense, and this analysis suggests that it might also allow developers to make code more robust against certain kinds of invalidation. +![snoopr_simd_ascend](/assets/blog/2020-invalidations/SIMD_ascend.png) -As this hopefully illustrates, there's often more than one way to "fix" an invalidation. -Finding the best approach may require some experimentation. +You're referred elsewhere for information about how to navigate this interactive menu, +and how to interpret the results to identify a fix, but in very rough terms +what's happening is that `deserialize_msg` is creating a [`Distributed.CallWaitMsg`](https://github.com/JuliaLang/julia/blob/5be3a544250a3c13de8d8ef2b434953005aee5c3/stdlib/Distributed/src/messages.jl#L26-L30) with very poor *a priori* type information (all arguments are inferred to be of type `Any`); +since the `args` field of a `CallWaitMsg` must be a `Tuple`, and because inference doesn't know that the supplied argument is already a `Tuple`, it calls `convert(Tuple, args)` so as to ensure it can construct the `CallWaitMsg` object. +But SIMD defines a new `convert(Tuple, v::Vec)` method which has greater specificity, +and so loading the SIMD package triggers invalidation of everything that depends on the less-specific method `convert(Tuple, ::Any)`. -## Notes +As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `CallWaitMsg` (it doesn't need to call `convert` if there are no restrictions on the type), or we could assert that `args` is *already* a tuple in `deserialize_msg`, thus informing Julia that it can afford to skip the call to `convert`. +This is intended only as a taste of what's involved in fixing invalidations; more extensive descriptions are available in [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) and the [video](). -MethodInstances with no backedges may be called by runtime dispatch. (Not sure how those get `::Any` type annotations, though.) +But it also conveys an important point: most invalidations come from poorly-inferred code, so by fixing invalidations you're often improving quality in other ways. Julia's [performance tips] page has a wealth of good advice about avoiding non-inferrable code, and in particular cases (where you might know more about the types than inference is able to determine on its own) you can help inference by adding type-assertions. +Again, some of the recently-merged `latency` pull requests to Julia might serve as instructive examples. ## Summary @@ -765,15 +356,12 @@ Julia's remarkable flexibility and outstanding code-generation open many new hor These advantages come with a few costs, and here we've explored one of them, method invalidation. While Julia's core developers have been aware of its cost for a long time, we're only now starting to get tools to analyze it in a manner suitable for a larger population of users and developers. -Because it's not been easy to measure previously, it would not be surprising if there are numerous opportunities for improvement waiting to be discovered. -One might hope that the next period of development might see significant improvement in getting packages to work together gracefully without stomping on each other's toes. +Because it's not been easy to measure previously, there are still numerous opportunities for improvement waiting to be discovered. +One might hope that the next period of Julia's development might see significant progress in getting packages to work together gracefully without inadvertently stomping on each other's toes. [Julia]: https://julialang.org/ [union-splitting]: https://julialang.org/blog/2018/08/union-splitting/ [MethodAnalysis]: https://github.com/timholy/MethodAnalysis.jl [SnoopCompile]: https://github.com/timholy/SnoopCompile.jl -[PRJulia]: https://github.com/JuliaLang/julia/pull/35768 -[PRSC]: https://github.com/timholy/SnoopCompile.jl/pull/79 -[method ambiguity]: https://docs.julialang.org/en/latest/manual/methods/#man-ambiguities-1 -[IOContext]: https://docs.julialang.org/en/latest/base/io-network/#Base.IOContext [performance tips]: https://docs.julialang.org/en/latest/manual/performance-tips/ +[vscode]: https://www.julia-vscode.org/ From 806a5c3d45f401fb06c1a11247000c8d7ed73bba Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 12 Aug 2020 17:40:43 -0500 Subject: [PATCH 16/21] Move invalidations blog to the publication month --- blog/2020/{05 => 08}/invalidations.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename blog/2020/{05 => 08}/invalidations.md (100%) diff --git a/blog/2020/05/invalidations.md b/blog/2020/08/invalidations.md similarity index 100% rename from blog/2020/05/invalidations.md rename to blog/2020/08/invalidations.md From e22433ae52326dd30d7b9d7e85062d329ec248d0 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 12 Aug 2020 21:05:16 -0500 Subject: [PATCH 17/21] Fixes in response to odow review --- blog/2020/08/invalidations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blog/2020/08/invalidations.md b/blog/2020/08/invalidations.md index d7364d1475..b87ddfaddb 100644 --- a/blog/2020/08/invalidations.md +++ b/blog/2020/08/invalidations.md @@ -90,7 +90,7 @@ This may seem a lot more complicated, but the take-home message is actually quite simple. First, look at that `arrayref` statement: it is annotated `Any`, meaning that Julia can't predict in advance what it will return. Immediately after the reference, notice the `isa` statement -followed by a `ϕ`; all of this is essentially equivalent to +followed by a `φ`; all of this is essentially equivalent to: ```julia if isa(x, Int) @@ -337,7 +337,7 @@ with `ascend`: ![snoopr_simd_ascend](/assets/blog/2020-invalidations/SIMD_ascend.png) -You're referred elsewhere for information about how to navigate this interactive menu, +The [SnoopCompile documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) has information about how to navigate this interactive menu, and how to interpret the results to identify a fix, but in very rough terms what's happening is that `deserialize_msg` is creating a [`Distributed.CallWaitMsg`](https://github.com/JuliaLang/julia/blob/5be3a544250a3c13de8d8ef2b434953005aee5c3/stdlib/Distributed/src/messages.jl#L26-L30) with very poor *a priori* type information (all arguments are inferred to be of type `Any`); since the `args` field of a `CallWaitMsg` must be a `Tuple`, and because inference doesn't know that the supplied argument is already a `Tuple`, it calls `convert(Tuple, args)` so as to ensure it can construct the `CallWaitMsg` object. From 9e86a063c767fda19cf21915e590d1f1b668401e Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Thu, 13 Aug 2020 10:21:22 -0500 Subject: [PATCH 18/21] Set the date and link to the video --- blog/2020/08/invalidations.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/blog/2020/08/invalidations.md b/blog/2020/08/invalidations.md index b87ddfaddb..98346f43da 100644 --- a/blog/2020/08/invalidations.md +++ b/blog/2020/08/invalidations.md @@ -1,7 +1,7 @@ @def authors = "Tim Holy, Jeff Bezanson, and Jameson Nash" -@def published = "15 August 2020" +@def published = "17 August 2020" @def title = "Analyzing sources of compiler latency in Julia: method invalidations" -@def rss_pubdate = Date(2020, 8, 15) +@def rss_pubdate = Date(2020, 8, 17) @def rss = """Julia runs fast, but suffers from latency due to compilation. This post analyzes one source of excess compilation and tools for detecting and eliminating its causes.""" \toc @@ -320,7 +320,7 @@ Most of these changes were at least partially inspired by new tools to analyze t Recently, the [SnoopCompile] package gained the ability to analyze invalidations and help developers fix them. Because these tools will likely change over time, this blog post will only touch the surface; people who want to help fix invalidations are encouraged to read [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) for further detail. -There is also a [video]() available with a live-session fixing a real-world invalidation, which might serve as a useful example. +There is also a [video](https://www.youtube.com/watch?v=7VbXbI6OmYo) available with a live-session fixing a real-world invalidation, which might serve as a useful example. But to give you a taste of what this looks like, here are a couple of screenshots. These were taken in Julia's REPL, but you can also use these tools in [vscode] or other environments. @@ -345,7 +345,7 @@ But SIMD defines a new `convert(Tuple, v::Vec)` method which has greater specifi and so loading the SIMD package triggers invalidation of everything that depends on the less-specific method `convert(Tuple, ::Any)`. As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `CallWaitMsg` (it doesn't need to call `convert` if there are no restrictions on the type), or we could assert that `args` is *already* a tuple in `deserialize_msg`, thus informing Julia that it can afford to skip the call to `convert`. -This is intended only as a taste of what's involved in fixing invalidations; more extensive descriptions are available in [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) and the [video](). +This is intended only as a taste of what's involved in fixing invalidations; more extensive descriptions are available in [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) and the [video](https://www.youtube.com/watch?v=7VbXbI6OmYo). But it also conveys an important point: most invalidations come from poorly-inferred code, so by fixing invalidations you're often improving quality in other ways. Julia's [performance tips] page has a wealth of good advice about avoiding non-inferrable code, and in particular cases (where you might know more about the types than inference is able to determine on its own) you can help inference by adding type-assertions. Again, some of the recently-merged `latency` pull requests to Julia might serve as instructive examples. From ab30096e44562bfcde70dd7ced046c9a8c6611f0 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 26 Aug 2020 13:49:54 -0500 Subject: [PATCH 19/21] Add last figure --- .../invalidation_backedge_analysis.png | Bin 0 -> 24429 bytes blog/2020/08/invalidations.md | 98 ++++++++++++++---- 2 files changed, 80 insertions(+), 18 deletions(-) create mode 100644 _assets/blog/2020-invalidations/invalidation_backedge_analysis.png diff --git a/_assets/blog/2020-invalidations/invalidation_backedge_analysis.png b/_assets/blog/2020-invalidations/invalidation_backedge_analysis.png new file mode 100644 index 0000000000000000000000000000000000000000..d4c4e1066ea6022b6aad7446a4614ad861219d46 GIT binary patch literal 24429 zcmeFZWmuMLyEXcN*a{Ykh@c1t2#A8xSfGGPgEWfLCEegOP!LfRQ6vmHRk~3m6b0#2 zK)OM?_qd#MzH`0nd)NAY>|_7i^EjOP)E(D##yH10&ik(1g){5dFs`9cDC?!po>rhx zs68kYs@7F2@D6`Pa|iySu|6rKv zrQ(x=q&U+K2fu5M*c)kOt%ra}8%F}1hRk?BR*?Wciu7-!@!lI>!j_ixo&E1kB ziQVn*+uvuD+3M7F|NOGO<$G)agB}eo-d8?4doBL<$XVrqznE`RR#7Oem;V13|KF@Z zQC9ZasFF2n*V=ZJY*da{SEluRon1;rq15FwysnCrhwf^ z9nbJ#3g!5LEBg8|+Br_0T9O&NC{Iq*#3-|C&9lg*7}tExV5CqixvZ?Ll=9qMyYt#Z z?ockS2%w!)iB<8zejZ=m{wdPJvFo}8c9>3*f_)!3M01?HIQ8cXO`1P0^D`#JRW~sQ zRaSkFhCS5OFZ*o?rfWSoRpP$gUcVzKvBjyM>$}_Bp!C_Z-#$IsH^{M!LdkD$*h&d? zn>Fg}=s4?l`0!yq&iZ2cpl;-P?X-fCGC#c{d~GfeWfizt%qf?G#;a zPsM9L?jOuu&nlSpW7EPxir-C6@n5Apob2opxw*MzlUc*L8qbeer2ch!4a%3261kC3l*!SGlLo~+#KgpAM+)dB zDrI=cCU6`)sN}tst2#_-a~8Ir;$gbRV`gT?xG_1}&(9Cr7GC7DqluRbA9-CW-L1>Q z!qVB*^|{Tmo_8nSOZbp#P4T2-Nr+aA?E7$Qt=;+6IL76A?D`Wfjq$>5I{O6$)p&V% z*Q{G-Y!WUC5(WX_hlJpay}JF(i!v)z~6vZS$fBZheI6?fksZ06~qk!;vzweb$GWJ5#4>}W9y4iKl{J6EHsWAr22(YQA5nT+%0%df=d z299uDd~03q_g#=)M;0> zzQ4hL?dNIbSe5v7vCFY5ZZJofti{YzU-89sr^$in`ud9jl&67#G4t~-g@uKJO&DFqjoY%6PPe|#0YoK@nIk&*HF>sQ;{vQMAhx}3-T z4CyxsdT48F3ptG&KAdbX_AMoW+US~i8}rY`!zP^J+M9dV|lr(o0}VZ3_WbnseO_npjWg` zYTH!Y*JnqHJ1rCOAM7Gb(7Nlf;gU3c?2;o?({1awROpbyBS+QAYGO-fBotuezoQroeUUP>X$ZjcWfc=hVl=nKI@eqkLQSKuZYG%~Q#9X?IVNMvS{ zvTt!s+sdP@*^!-H8Ks^Q^Xm0$tw!o)%jh?4`f2Ju@V1yl_FFe^MoIcGGKxASBM1hm zE-x-DJb)mxY$a`VY+`QA=6x3o$9pOm4_{fON?C@(2b!r;C2Hq1CPcGoHIMZS4#rb0 zThTK=GuGFb65CZCI^4Xhx3e?+YUvZB`Zt&QzP-FbO-=py!Gjk@RS^y9`*4Nd5hMI; zrbjMuX=N&T((dH=_0gxdDl))m0M3-2JvlX12Rl!6QtbXKqMsW>lu+VBMBvGtKIu9; zE`R#;&B=j=*F8OkJ4GDiiWpK%8!G<}nQ7Oz3){uvp2CBL?LW7)=pYO#=<7!a*$>67 zWOm5^`+HQDQ$ybw`3xVheQIjb=q?Xc_2*C?|LM(R+*ciK=TCyiD6SNdpsYCRmZzr) ze6non{!U?iRU{?~b2%a;LQkyEj$<5HFgSH}l>F<*41Kwl1L)3-G9QcugI@E**Q7>j~|B#+4hwn zvh(ws)*60!ci7C`Qd-uVWXQs|T$G5&NKSr!WdxXi1$mbIsHpuzQ!8;1JKu0oU}MVV zB4hVzu2X|PSYJc##wHL-(%1c<9%(x419 zXC!GUlI7&UeIniOPyOCzS%QsO&&c;6G&EGJ6W??=h~EvSwYb$dFU?F(_q<=pQVrP9 zIXrBksz&a(I5pNI;Go*8z2^fB5ox7mWmVPH1G~OHkBm6~yzjNISf%iA_G`p6qvrH$ zcK*Obtq)C`mI=U5au#R%y?lM`Q(<4{Ci-fsLKnyPnSIYx#@>?Xj05QZeG*ZR?c28- z!*U-^0wTdJPgNEce1!M?eaN*QSgF9nSG|7y`W0jbweL_y5@r6L z2D??$@o7I4Ld9H_617FiiLpPpdASoOt|I}nNf9{p=2F4HU*F$KK`>0bv68Ou2A$Zq zA34rf?^Rg1jqPA(V~aAbiSY}AOE)Uh!hMS0aZ#idV5@vKJ#xa4xY2x`bbbmqoauX( zeSoj8?-On&$u$FFEYmz`EArhIW)A>qq?pvH%Blye`j%OQ~s9j$WS+q@?LD4h#yc0TtYqF^5xjXGCGkLShFbe@0qi+{fU30 z!L5|;vf$X+iGCSB1qFp*a%8HdMIgcDs^h>^6xTdhENZRinuO=0+n(iKe$sdSm0IPD3_MW$y zpT1^0{6ov{aY#r40A0gBeqD5LyK!~Y!^208Onl}{jFR@}@Z)A??3&5L-A%&cXQ!u0 zOp|Ebym|BBZ=lH0Jx77@=DX1x$9Mil(SN?sYm(UIv6In9O(kNG?n1ATJUfM=nb*ur z*~7}(VJQ%lI5pfdJam9~EWJ6M$5?l{{nQp5Wj|Hg%KWlm;U(ixyvMm~SN?ah*vF4* z)BKul_ubayHaVcyt=ES4THO%a5X2D#%EAW+8j^tap08Q6X5cSCZbxrvW%W)7Ncq8o z4Wbj(ibAfl$v@lMI}v?5O9TD^MUtma_l%W`s|IjuDMc%=lN^gIq@HT}3~~SOy>ep7 zN8n^cyBdsco$w;4(sgb?vC7Z=6)2wdD)~NK~F>J z?X;xM`OA=!4a%7V#E^g)#az;Xexmd9k5BYgodI;T>$U#RV9N13Zn^GA+7Q)1U1F$4 zkrn_rv-|wD@7IcVW?HuW5#qk+2+Uyw*iu&T>F(XTvw+FTy6>rxs|*oi5nTcl6cvHX z!vFY#dSakK5j9gKyp@;)V#?VfhOa?_Ra~~XB~3b*wvI#F^0a$dFsOjK;oP|d!zT5gaFlviAl4^A6((`>CQZC{j=JCD;NW;5G7_@N zMW=$aT}BHPd`<3Epd71tt9M4?%o*Y0sfh_?OUpMhE#JP$T{SQep1XPbc4jsU6VpKe zD}zuSfHLI^7w(tR!2~7rnw0Z}?T3_}A2nl>VsW2;OA4TC?UDp7_I!IeI~Ka68FMYv z;pb{n3GFy)I$e`n#30B-+udL4o~IC6dOk!%ZS&r9`;~*!?R<(3at?0-pnBPmy7CZi z6DIe&+>4P774r)#DLD(E;&1$=t-#Zyw=!Hk$0-F{*lkL;cCDhecF2Vgk>1KBlxAf) zZkH}yl9(J17I9QSV3n9$r+wIN;9{_l?dKV+uUbqdob2->Gpdo)CiAKwLF;6<1*h<^ zuqcf*bH6ay>3%l0u+w_m^~OJLmNi68qMl;x&&tZG`NN_n#^1%o<*}dNPt(%RpDW>( z2$0UdkyNsE_l1CwXF)*+#l^*8Nip?~3-j{;===dJ2)ABEsi*Q4BQGeUxSQyUO++zv z!uUsa2AdJ?rR-EeT_37|x-%@UVeSk4Ix)am*0S1iau0xdwrtrFR?O^t?zNeVYg%huyLvwg8b*M8n4wcqYjw>J#H6W4!DrPPPZY}+Yp zcTP}Hkb9fK{NhxeYUgKv8NONsh+sYKq3`xBHGuN$Qb+W^JV1F&LMkFnERu1cxO;AP zb~c+7a5wROI7bBSPNDs~b{${6dUfDYGxgqW$WwAu^|Sj0cz6<7RlAPj@;`WEEREw6OGSF?5 z5gqgO9Ud4rlY9pYEXg%uDv1)6Bfn>D;hngg)OnV++?Q9EjsfW z_QkEbOvh=`9H-DC(ba+Pe}f;wD>vXfbt|cN;V7QFNyv9#`18fvLDmP(Lm6khxACX| zB4{;QTUk{n>WHIG`<$TQ!D-B=uNIvqDi9ud%O3$-EOn4dGfCeEAiyhm33Y*bo?Fge zNi1!cs`;&+=hzei8**;bOeShRa4vG7E52)?@}Ml14pNOwHdzD|v^^+j`zAWotk|*O z;Xxr`>-$xF0U4Q@LdPVqOYa`zG4r^1jkFk*c=fU=7gty1EITt`!i=nS`GcRW7GHc6 zyh)+VrHfLIn%2wtDP6qSGZc*@SgtqCS~Km3OX&Uf@{|PkDj;m%-wDWp4NHVIuE}$E zzk1~gfME&49w{pUCO>ZN4OkJ%RMOzlLwh9e$pK}%O}DL7PS8|qaT)Wejn|089}^(d zzoBi@vUJBWeXNl*97lnatH?UG1NHJKk!l~wibW?U^ZirLBU&p*D@gb$CF!5v2`&u9 zq;kRS^(n|9S)VE9XEm>{qwHUe>KCP@-9Y_IFcke|!o$^^iP@|OVqc#hlSPL7hjL*$ zW*$@)BFZV{G}9RX)`15Fo)ju>f>izn6x6yXP{*clC{hL!H~ZtpjbK&kWg&5Kp4X@- z=_;u$D@UvlrS}8vG*dor3KE>!=n-kJdBW(p`E#k91dx8b6CH=N+-u(K@mvi>Ah}j-GmUGPc zh^3@pLpq+F=??XO_ADADQV)W_=W?Kqah!X4IZy|})d4{I{<_4vpM~Bvug}okLQ#F} zm};o|V#e&$aN?UchwMR*9NQx6I?wD`Mr8u~>FVx|#f}(PMfhmD&0HlZi<494?c2Ar zO)ZNN2wbsXD;tvxe6wVzC^E{`#q}FsQSdvElIOY9ljTs!Jw|bC-CY)q>-#l1Sqt7s z8Q>RbtA<$h!IK^|9AarYG*;kBv5^;$CrG|WxgC!Z;{XqjEHE&9LazA!&c7Yy(mh_g z^IMsi;z0Ttps&q>-)vMmeRVS)6tvL{KSp^yJuGB-&(tkgO^u6N&l+?){xJB zNR!}N)c6WuhCXMBARByC3~jmB|L6=0b%>a2CbRPZ@51oXVxoD@gvvrEkGnjsir;y1 z;1~CLT#T8B9i1)`D=-qj{uhRTN#JZORdl@>A!?pK?{{$p5hAs1c>fUXt&DEz95Xc& zj?!id#U!J@WxK3x@32vO!M6lq<{cE@ox&0DT3Ao!U%X6Kz1d#bkhs;G_ii!Dn_s+L z=lvq@vHfxiC3a+~cj(&exDkp_fBWs7AM}>BveSANbf)k|oNZ#)E52U?xDuR*I>Ywo z$JH=n&1=sqKY#i}E*_TbSRfwx?NKeq+aEi2_UF9CUpIIqBV|KX92^`R0~E|fj#99q z01Vm^lao6^SCnYKmzkfR2Sn|zRLOC&O{>{21+FIRWv-2CyOfFl#bKZO6F=%NS`>Ci zJ1=sN)_WgE_{29j0U6ShBY_)`t2&jy(0I%_4lo~=VidFrTS3bdg&-G;^$$E|5rU){ z_VSeH6MG{VfO$K=+x+xsr`Bo5yZ7&_Af=-i4Fj*y3;M%pX<@1cxbGMDdA|-oQ`BQ6 zO>;%%y&=gecPD=I;BtH>eoD3;3UOJ_hf}WnRd4+}O3?p@D6xT3`YBE=Y0JiqyWQR0 z5nH9<%X!92NV$ZZf#Z02c~$zc%b3PgGqS4<8Icy z%7cw5;Gx6^8;J2=ydB1Mghwgxr?(V>zM@PmaSO#4IG+j zLbi=`rgra7t?s0E?=;=!7Z%LoGRG0*V^Qh%qYscBvAB|!NzRv96wb6SJUqNNqdTOE zq#CSEM#k?O3IISV?i@3#47>ZBBe@JAVgdzMKloB5X=yKR&2&dT^PB?V%D_6qRoLJSwqkdn|h%t-#?0TPB)R^rV$}H`-vJy*+ z+C%EK&U;+leR%f@2-Ig|w+eIZ_qe78Xqx35Yc%NZtzTxCl$|X?!ZtT|S-2=#EkO4i zcF&PExAlVFcQ0J5dsG>x>Q@#d_!@waG@nQ*RasfttYw^$QQ4hk)^xP9yE{Npd8qD{ z#M+u@#WQe&8hoiVD@Z{Qr4m~-cs11l#fN~1HeCJw=J$pz0KK*zy!ax?pgacOE}YDG zDAr`6mT05rlA3;pVweQaC#3hwmoK>o@xRYyW!!ep2L=Wuu9;{dk}^6v%{#%HwCEl(gJa7M;7M(Y zqXtmIcfnM<7P`^T%sFS6Xjx$O^sihYGyuriDxjg1E1xz=NJ!8(O-W9h%DnsGwb<~1 zzTEb}zQ4njhT5!-g|vb@?4TyqOryIWVIT^OW}u*T#1Dt|^=Y+g+Fo;WTk72Q7$UF& zPxrZehbvsw*GGfJD6>By>+{=pevUuy?n_2SF{^*mraGrGKAdPX*dO21(t_HLLsV1? zZuyAZ%845*jNodj>m=wGlDIxOan2g7RUM#8Rs+&Pm8;y5w*mI9-jT<~vwH1XPWh#N zDWlps8Bjo#JMEjV6)dWD`HSgTSXdyaK2}toG{$w`nsp)Vn{b%`W5C%M)x;mlb1EIf z!|aF_Jj%(#gn>5CozYJ*|E>>vSHy4xHB zSNwesRnZK(cBLk(*%c}$?z4#hh`adq!$4D7d02S(vop+&A03>XpL1!XKCas&-7(%< zwU>+QD?ZvTrH_^nc`@HDzQl`jAd!;u7%VN!O+EkY&-pwu#Od=uV@hfESebB9adEK= znoON3H#Z&Vbu#$!^wN`qYL_QBN3kSPD1T}JhQ6EF4K*8qo6=)@hFIlaw<)24Woi1% z<}F*UAhYSAMHnOw7R?$Sl$#f>W+Ab-2)4O4P9v?1Nx(9Hp#E)v$W*g+1aS6VLBR$F zYdzQ0qQ@>d^fO5k@-+EYziKrsNTtUJtX%c(s!Tu=hs{uP!#YO3lJF(*r5V|(WFy7) z_I5qiJ$pDmia!b9{Nw}%>^xA$Jy=7ua_p14-rh$@)bNav>8>=p;5K7zo4Sw0cU5xyt~{k(4#v8WBQ%W>9GgQ}6PKbsGD6Y$AUd9U*lKb4e<+ z?fG`Xk1!d;713L^d(u-un07>3f5Ybp>7pa~%VyhImK+Mj2&b=)>?W#Q#IOS7t^uY^v#3fK&O zFvKQ5Hjl}m_dE+Tj}~*C)qGoy;1on^*+-_vnP^Qe{;v&$Eg0!^s+x|QxI`jY%5#sy7B>ST{aB|8_ zDn}5gMp;A3N#qoQB;YDtE=vFc^5@RoBjn4bO|QV2r7^n&AG-8Mh*RHz!jB&{-^!vK zL>x#!VpSKjy9UIxW$V_i`DnC<@20MlJ4|!@1@(5ANvc^>1PPvqi-(J%&iFE;!U|wB zYfhI(aZ__=;~qjgI}hj7v5!2i#<|g(7LP`a!aWGKOb#Je8IaT!F~MnZrkng5jlVM% z*`v1+iW3efMrhg&lQM}k)Qfgl7$P)~He~Q{cV}mBcz||gosN4vu$vM9&o>xT z@US~DW|?KGaSaF3d>`?m)s_LkiIOBu-(0u?>8#KCq_ff86?Gy z4P&#S07zs3qGu5n`3=gLMCS&QxZ6=xsOLCOsbJ~A<|G7YJ4K%jjf*>IJJhU(o=g^5 z4SG(M9+=fP)K5Xf;?Pq#p*~J=o-&We1x$Un50zoyri1+wa;!E{$D-5)z)~K0&9;;) z&9LObZix?K6z}bnx_Gc}&o}OoWYD|U0HMdjD@V#pxq0S++}{WxQIcI*=6KJ zRg!^h0PkYA`(iN-iVEXJc9|!HHc|x1QR%ig?`YkcPnF~{W3`!y$#inK#R8Y94~+kn zgn#r)2$V|9Owh2%skz#05b9dn?9?Bx+N=+BEa@i*3Z1v;>07>1}KYsFoyDb2QG!szAo-Fzd9xT)ZSgW zivuw(!Se0pTTv= z@E`Ng>DCHq+nk@$l;fO+rk^bmrBKt!lP9YY@&|w9X!sGLeqy+#gXJuLw%xpgUtBvGBTc z=Nm!=LE!+w+Eq>nbNz?SQMQgZnj{j$2P-4Wjp@;LfA|U-$x(0K$obizDFEW6&-wk)(M0hSupJc8G_c{!ordWxWCYKEKlCja)iFlcIF~V zRH9KZ>aUJ|^n^#pt#B|=`S@`U*rjUMHMg(m!1c_F3JIx$cT*t*m1)Ca;89oeP?nL? z2oQf9QWan`ymxIyG|2EuJKR}yAioKp+q{E86KXOg$On8_#I^Tb2$DmuE!q6LPz@?q zq;^7gm)_bL@IlhtJb^hxwP~VuOt@!fHg(Xc{1+j&!1biZHol$a2$nSCCUuk z!f^HnfnB>=Rf|cQC7~^EVbG$lAt|zy5uIVg(3_W+-OkH%f9pQ{b1^kZRuCp1z`uchvUiUlT7gN`SMl-Ip`)d<&G zsX(DkiT86XTExCvm2&xop>5a$w%3o1NX|euezeROAGaa(7Kuk_ypxa$*KtJz0GPBF z3>cQqo&QzJn)sa0h!xJ}is*^`Av zWLj`<#xRpiC^x+4m12~n5L+KVd9oM4ic9TXhgW~5%e_q7{?AQKS!bWq1J*?Y#$P9n$#b(hZpvj=8`SZyd;s3ihe=QYq)>(U;wT-MWkpcB57_v@=p#szN(K=#5h<<@ zsm7{d@$SA5IrN9z=9*ea{V6R3PG}K-553J-;nGcErNW`r15MYDo$s8^p6KgKhWj69j2247(?S3+M+mP?u~} z#kSD$<3llvyC&QP8a>f4IGr)jiS&gFwdm@{AOx5-ypur3w-WY5q6bmwi7!sSK2n`k z0Pi4HgPcqvPH%5742LKzf?R2tMWTlDmb5`&KJneTeS7b^bV&Ou&}T{k=pwQalAhoq z${^lYJqM{SM$KOkrx`Kf&Rg}7LGMD4z$HLO!$1D`;~YZ6U0>g0uyET3eP|CNuyrI= zjrZ4;XTw1E9z6JEX0bmp&m88Zi>P}J`+Xi^^iz5HF<;Tir`sV)+I~paqaoWdDQO9f zt8-Xk@9LwDzpjjBwJek|=zYOr8eG=XlSxj1n?1vA1p**GgZhb79s%Ab#Xt&IMal1J z@9s7Lp<#e}xzFm_E#4xRg^9YosQ8ru_>Hsa_RKU z%rg!1jKVKLf@6V8Pr@@+T|pa7QsH4JlAiD!$*v!Zi?d9pKCI=L@6)V1oXR!^17y^@ zCy3BbsuCFBVd$YEkfQbR(=L?p?CP66{H`%5mQ68IT2)TnURMFAqS4pnNLVl;bObFS z*TK4K%LFX358A2<(60xDx0RctUu_I>XmQlnJ-xYZw9?E0!X+KCTU2k>ZM#l6_LNY1 zKb(h^p$tTNNFYEBkwzuDcnsPk>{Pc^`{nu2sUOayAEAP_&csBWPCsyM4ZsnH>GY$p zCDe@s`FS^2JHvg5qD)YEdA{@j6^;2JLU!SJHXut06Ne(B4!2X96}O`nt&Ue%X(H7? zb}1;$@CWE20NfMC_78}Rfy@1EM%|CYFgUx7QG2D7 z{K?P;FWq-$g@##EXg~uDJ87EH-`f|O+xcu|F6nhJ{0A1(a$|{Y+<(`4Vg<;xKcUG7 zpfn_9Mx8~@#1r$p1v_Y6dZ6-CIfnyMyf1G0-(<+~Pf4q4>)Q^dd`Q)ZEY>gCu!0r9 zTZV*+$-ySl_TY@bmpmj6HZ1D_je*vLlpoPW0uM+j_%QMZIj_K}EM4DRq0>bg;J4~H zOR6!@cs1xk5ZRarr7q_CQPS-O)ls7uA`Po&JG=!=w@0cGjYMrUnEt+z)gF39 z_bR|-#yx(#g$N^?P}-TB{m5?cB2pjSjWX1_l9IRKkvd^HplyWQ=F{mF@ku@_Rvfv3 zlan*{tTU8TV_v#=Ta)GBY7ex8N#6H|2or4ahYu%7ne+OLZ*2jcxM?BLvb?#&{(n&- z`#WT#`iZt5{;`IR4n3T2$n<@`7XM2P++Vp59Vi&PDl#=a149(b%^ozoxLpa!4#gAa zp+j;IN>vjw7etQV5-|gg4KhQ^deE<-6Q7sk$r&fz<^)_~iebe`WYz;U4q&5|342wt z$F~OwizihUW$`$ohmekr+vr_3_Pu-ilFG%ZfS?I*LZKI}ny4*({ye0d0caXkqsoA} z%b;%ihIEDmxof)~ALwy~8gx_SHa-^o-5NtoKt-ycr1TgaSarz`{-d-Xv{uLR#ZD4iQ@vgjebrR^d?V_yc#L96JDY({DDRPTToB1Yflu?{0>o zCgpG31;tw z-m{X>-q1!%DcISiqLiX%Vj6g-WoQ`nE-5M46-Yzt2OTZ#6VOFTx_O<=+>pMb1%m{w{B{hf4QamoHa)C0 z-Itx}n=yeIOLOi^{W} zW-TowtU_=Vxcu?XQld`})`>oz}n2FqId_c8vnn*N_GLzM4@hj8pS*V(%%PL7UbPDG;7mqmOi zdzggp1Uke{&hOHP5Wj$(|N7NcMNut&?_`HPzGI)3IPv(3`z0;Y7 zCZN(n5zPl1d8dqd9Eyzp0SdE#<#9h7P$(wQEUlrXO%2`|r{nI-TWbZ#I z1$sY(0ck(X2c)3V>vl#SO}hL$rG51KzX+uN*#h)=6EFaA*00EyB@zEHMEj7`ZNbO9 zadY_-@BJwC>DTnShe2I}8%>I1z(5AYT(`lM!cP1j{G9ToOD|=dK~~2F{az1U+yU4n z(klnT{S^M4VGoiUstL{rc&f*8c+LNzkiN)!5Ql9>E`Exl0^|&s4Ru~7@?MGG97R$> z;#!_Ke!S*`A4R~nPe~N5JawpmNE8VQVj_Amil+EMK^mqVJ2<5X0U;v{K#4cqaXPg3 zQZK_}DfKu(eTKCG=WTK&>+#WlP({-d$@GhR_z5dLZhHYPxSXF3IH%E8gkH|AUZfOJqCp~IQ#x(GLsaPH}Z*jiqhHF->l7nb<939^DaY4*c z5ZWGXQ!q6~t@+nU)Y6>2f-WBJo_v>JZdm(g2E{;etkdyZm!v2z{qbp!p>_wHCCXL= zyF>sU^!eGy(3oE`W{cMEI64qmBn*}XZN!9QEiOs*Q#u%_I3FseWsy4*kvrSRiIhUX ze+|m2!fM*^^7>5GE*B(kze zr)sElIV02?J}}PsmhCjmmo8vFv{Y`J+!cvj!qRVEi(5Zr2uLOc^1Tt(n1@Z z`!_^CdQ5Ba z#<^_!_MJI#;slYoBFn2p*lvydH$a~6Q}BJsO_k<}CJn0;IH(E~BJqqvUbIXCF9rro ze~jg-}ibi)CZPKVQaAdD$P&(jI31__~^VgaeVwD9?jn5*(r>Tvl08S4U? z64xf9ftenfLA^EDEQYcOxs>B@QTUYCS`Q*J12gkE8xHUBOR0}i53z#5w@e#{RDnOI zA;N>ZV%oLK7}_gUns+zrfI~5y_NQF}U$25zu?-ngBJvX%Z4uazAcrmGApV=Qw}(jO zkFygWSTdVue4`tMl|cSA#2e3Jp*=tB^(w{-`k zX}*<)b`GHMPr+$vX%$NLe+f=ep&uZcs0IVXWwJ9ep@7aA&6()xslgW{XxHOl&1d2I z$eE-P2~K2jXr)5XK7}vVfK$=>IZAAccza}|Or}HI8FWPequCJRFf|8YEG6^gU8v`7B+nZoG}_{eNSeM(87F0 zMmjr69@g0`M}j;qaaD>EBy68BrkRfNBVIE@fyf%MixX&n1UO_Pa}ybcamA}!Y~*L; zAB(7jW&A=MyS4*}lfeq(@0lsNON-9X8fP_Ibpe1Xq3!S&>xmv?FA$%7E(;I?&Os6$ z2%%cQ4?a|90oL>IF7wk*npYk|BMyG0Or%ejKdvG)aUd=Yd_*h;K@@=qlV8>L(@vq> zeDSkQ%wy{y)jC>Q6@C2(HY2z)nU%8{jZwG%JzU^XnbaZqjWjXf@MLC^Fq$B#0@$#= z{V4lV|4wPY2f4`90%$&h43&@-r4&?DYWc;Nw&B_)5efh6d_nx51)x=vD-ruxy8czc zQ}2Sw{<{M8K$UVna2$8}*Y8kfK3j|M?%@n{P!=lQ@`{Xu>INKM{%->XFAi4W?fjEZ zf|gDHw}}GOv}}~UP;+A#hTDvkNGOVwJkMAeF1)nJE!?yH1ZPvT;2bnbvLTo>k1Z2ga_fMRY*^Y@&)ZZ z+!vb>B-VY9%Ca3ed{_ZvKmTA9i>$$;q=^S5mMUaKDE{|K5w-<28`?%fVK3WiC#t~M}t)te_a%y#S0C1UhI#ys*+@LB8okDjd+I&WM}vya}INyA2y zGvM>#BTS{&F5ND?Rrtm`Y}aA06RR0t9_QXQGP6+OE38dL@6hU$Fa^YXaBhy zqWo7X5hZ52Ft=-MAC|M<#yjM#vsCG@zHBJsr$H~uD=FRl^~)+apc1o-myjXHtgcm{ z#O`5r7|xm6Ku>=QZOCO5@D{WN1_qdh+YB0G6N(~BWJhRC-j|otQQlzsoQ8&G7f!~^ z%xo687S}JYp%6w{w|ccaj)^kt6l_Y@+?*Cu3J|eSo12@Ll$W2mc8wQkA;s+L*RLBl zZTbTtGkd&J=Ir_Nx8mdDQ!&*8xgVBM$;hS6qVgll#N~(OWdQ z*0G2&p(;6n15)xE8q^|vlah{*Qi^gMZv_Pf_4)L7bc7AIJox-sT1iF4xFxM<#s{~4 zI{xdfwfWEoo48L-PD&m>PJwjh2HG$MEiEjRKGA1kLoOeM){ACz3>~%i_VJOxGRYL| z!`4)CN+eL%rXd@DXlmL4RYl_%^>E0iEUQj0JC$#Rh06oLtsN^P(8r={$h#&1}530PE}cXx1iv5#Eq2{EbNQ8xOJq# zkSZlBdjl)5@2^4cxT>yJ`zz}bwawg|2~bl|{hWo6}wCoP$_Yf1fbd9#q)Z128v!9w(y z&9bzyxtW(I4sqT$@Kcn0lf<0cm=?N>w4d%Y!GTC>i3!?4Q3_Qn$gKhhKvPuLqGV-8 zb6ZhF1!sBX>2vqaS!-c~t5>NgDCBNpnn>hV$(}}Aetv!dhml=q1mDECL#n_xPkBd2 zK~pm`w0>(jbXHEOzsqZIe~nl zcho~)9s8rKtV|LRGH2~cQu=l1lMkYnZ#h8W(MBo5#i8#JiV&w{GE{O7f8kkF?nDk?sF@PKmW%o#M0 zsL;_nE+@AE?G|45MHfRpM)sqp$Zf2VyRzaapDQGR)% znRS$7p|xe za_Am5&3Q6lVb|i#Sjqg_S=`;OLx(m1YMi`uX}8mO&vn?>PjKmy69liYZP^4DZ)?WVdlovth%+;&xfUh@y{=6LilSFOSTe#V5*SVu$ z`ffwJR*q1AQ}XqD6p=3#qDP=XAj3Z+BO`Jg#MW6Aj(g`)r@npr9)$Bo28KH^F&scX z%P`=3h@Za@YN2oO8pkk8bTaxPNl9m^xHu?0TA4Rd2YRwu24wb_jxH9hYYu?wuYt^|`6E-E!lS-%?XkSJBaV zK6|zuz!SjA7c|lSjMe0pjCU7?ED!SczlE!(rr@CrtB)KxlAfNfi^5vk?2o9L(d{MJ z)pc7qDOup67iJKJq}R**UT4KP3Y&tLSh=|A>JWD<{q2(#)|5YUaq;%!uRLkW#re2{ zc=xip9-gJP1caW(jK}R)uUJK9dtU!eTbQ^Qesd@Aqg>w6mE+E(tXt(zUx~YyNoh_o z*+wEC^lCIKR$NBa%OL8+2X+_zP%xOBoQ{35iOVSV6?J+zBalR);+H}$8X}k*M zWHDmH#vMDHTNbl7@@?GCTIS;Sw%zB2kHd;LF)?d^**`)*Ve4s|KT@OzQd5bZN|M?fZpkZrBLzJ=e1P!O1?iGmN85Pe-?U zAc7*vUr$O(Dzie?g`nbC{l23|x8jKd&2XEYY@#+<|R;Epb{eCCCB?jKhH0RDYxd)s7mn#XM%#RvgM!fgRy?c{P z$H}Sn3aKB!iAqY|P>8P{K78=EZQWb9_PD58o|`Di!zh=hr>9%Nq)wUUp|R}X==i?5 znF&gdJKo+Vuh05dJa$PlYg(~+^CPE3kWML=K7O@FZ6LBRUe!|>z7`sV6zOMWWi8rEo<~N{Q0KMS=0-)G?Y?^H zQpIigV=d4-@M2Y=+a0>zi0KqP#JE+ow6_tyK$G7Gm8u7JHl8W^aA2TCFq$pF3r*<9 zs(8r>e){yu0Y&A5Cr@r;$CXr7_wey;se_m<*>^Wf-q%mt4mG!@VYM!1Sg}L5`rzTi z>lh}Z#w2b&L=TLn7A13A4V`j5+J-cYJ>;IFrS%jlT@8b0k&%%H)!v1@pTwpA0yOD8 zNGOi(5a)!N%SsB0V_*eh0TWXt4C84%0BOD_eFfs!8xj&CRI2UN%RV|aW%2c!df|wdC<|(A=$(*JL(*{@>81V&OBCD0Ks3#+(*+BL5B?Obz!9(Bhw}5_*;=qoV;Qt)-#JRBqSs-2wJgi+PryU=Y=FlVW}w0jvWti zF~3Hg(^h`kX2?=(asK>zGO?$iC@bbi*v(U-cn5U{!^%kM4XlS zeKB=aRhv-|6hIa@0>Mp`&waTN@P}9o^s@CQBi5y^P+t1Ef_G3~E(hf`zxCxD~*ZrE3%8 z?C4q%huqt4^G*&9yN(>$41iJq(a|B-SsU<&$JDoMpUpLs&b^&$A+^nb(sc2!h_-Oc z0mZVDyHqXIc18+wo>JDoax^f!aR?yGZcbHSZ?3ModJ8QrEmp@2t%e=di^0b%n6ZJL z2(UK&wSysM>u09ps~)=y>l+p)s>e&BdTs0P_l9tH9h1NY=y%+1m^BrUv)dm@c9qGz zNJ-^QQ77O5D8K(8W&(=QLf$d3dBX;|#0B&>cbF|5GP;m!zj-5`ulAv)CWZfGbym?N z8kRg?zI++qqTEzz$hYFHy95o&^g>W+K7hsJL*M6eUDwS_n_b9V?{lf7IPuO4$noFX z-p;nzVV&r*!`P%`uC}MQ?Zf8RQ+W2(9gsgn1mlN}Iau9Fh-NYzMqk~zgD}~nx!W1f zA&`$&dWWZAz@Z#eu5BJ0(?4-7H%s94nc)6;eYax zl9JLL$=4fwB==CRfS7+(Vb}q6)Rv78!51S8-%MSACi^35F!H1{V8&mN0pBIpgIU8_ z>6cQ0fy1P&iY>7kY8E6aS2+G@N(nY}-2cNeXN<}g*4C;(ZW)H4HJ5p<{dVekXCb%e zj}j4LP-FRn89nRfmV4#u)lZ05+(63@P)2#5ooh^EJzrk;V2Z1zm|9MjsAtY1kV(X= zl!8RGF^}D_X%j1X)QwRdRKcSoBO3r%M#snb?&v(RH9*uhmP*E&+D@a!e}Fkl%8=ZX=y)H+1~2$PoVbvgxK9EHllBAoI9L7b{{C`43fde zA3wCA&ANh8Yj$<+UB=IwJxoClHANu)eEn4Sz_$ef(ReA;%Esm~u$r;=?uU~p77~FW ziMxk2V#-fa6`)CqE0TJc(;0Rf)yE~c$pw~qY!brO$=xG_V=$hV5To|q^^y9%t+9}ozbVPee` z&lM!K;U6GIUMJ1@xvz-6I%u7 zw0WU4C@jv^P*U20X8R5FVlIJZx3sn{K%b^Y;(m8J*u-;j_ZGUdL7+*i1_Fb zHa2V|r%khHSwVrilnH>R2uc>nw(jl=by0|HHHv`^;TeYOQNRe5VqkZLu&{7_Lqi+X z94B!$OxnG{lRaGUShzPyNd*{I85W#G2)<-&?1laTdEWuV_bAe}F=0*G&{rygP!8xz z8}J=+qwS3o;!E>MsRA-*&*p>6;P9*b#F;ZlV&v09O0la9%OWO0C56^70$O zryha^2hF}#6mz7qBE98)WJoV;H0!>7D*^rRD4N6L3s3D*)O{fI{|Rf3&kV+ih%*fh z4Xs+c_Qs<}o57MaE;v3p?X>Uo_&3b2JOp0=+zQL0@gP#&tlY4$(+9b@I?)J0ghlVH zgmkTn(B)d(1%LY5=!VC1L@9AZ$H&*Gxs7$5htwkgdb2xlK~1Qifx%A4tAI(rkEJSw z@ee;!MMSbK(t?0%e7GgALJX|a28x2bJa8ikt}rPXIN6Os@a^2dZ4WSZ;RQbcFQ{nX ziU+uW+dFB{fW2h}x?yFtg5VXoL;;+amOI8BsCFT;v$DR!vjeYLT0Q_JWZVSrlAw^J zvGE~Vnu_29ZBsELv7f?*wi|>n2Php-;rMj9@qj(AgazqO09o~mE}21iDFt(@;LxkX zT=6E#Wz^x2#ggm;-7LI>6>I&q@#x0yq0!7HvaH5`AlTrsfFJO9yR0wo-&29C1o=iq z>FDaZTp1=Mw19AN+ahm4WloHPV-1uQD@W$OFpSLAy#=N9-je z9C+O$5w~tFM+CZsZ*79QMH0H)hkky40ERm_If==MVJ1kZeFY6oD`sGXjg5_8C`1Dl zZrHig2hIs;_N&y?trR&qISORrNc^k<0t|T0pO5(-QPG`fpL2n6&>FTzW0@^?Zb&$I zdu3oCV>5_13=JN?aDkRk6$s9~9~~lr$Ja3&I^M0B0+mut(*>*nE?bEw7j{k1II`o- zw6tU3vNtd?dKrW`t^nBe>9Xy<2wII4t7>>!!9CarU*pmVHFfo$V1BrmyS4i!`_Zd> z1CcZ(|HX+L4{&!VW?1pW8s#+ED{&Voo3^hq>fk)Fv0|%&{bKXQJn*u6z!3s0G+xD? z_cW$(f(4gXSNFQRdEZaW()ToH95HWBzYeY=c&%`smk&Szp|^yBvNHEB8cHzke&bQu%hj+Rbqti{t#Kc5c6+KKD3JHRkVcolT1%P@& zUHOL(uilquy)2(}3$;fPOnjEiJWdo@+uGie4oA=w0g5MVB-CwNp;SU?bO!sk<4b?p z*xIV9s<#fKm(6(TMCuL3cAAPSWt<8wffh(T+xVZoh*8o(rux^2_xqI=zBDeyKa ze=sGIH2{=W2%r6iwiv}DG7=so@Un_$&z|;Ny-;y?X81eB1GAhoz^z~j&aDGwx6U4# zmeZ1wE0j3s7x6SB0&9^;rjgekLh)+Nt)mBOwJ&G5E|CRa!GJjiP&UP(qbVHgFtvOa z1Ym^HPU#ju8U~Gb6XR zpk@HNCdR;>H8;kLXD5QTy#Xu(djjiDHLrbf#flYVi5Y!X7^g5|dx-@xpwoRTI@&1b zs?sPJpXgRw+Sy5hBds>($HNT|!OT{2AIGH0TWDpvmhFUGgt;;CGe z;}}kUp%VKR@>Rq{Dm;9)0Kq)I${llO`$48#my3ELpW&ehZA99sdR9PC5Kn2LLRYd7 z*PDWI0zA&N5N!bB&rtvDLX3yP;RBw{Nkw7cH{K#5A~FJ0(TZ;i;8}>1)P6!Bo=(9U z7a>~NjjyBX_<)*}YThDs#9vkWgptvK2qX-Q=Cq=JF5ohK2v0P+3#Si@jMPK>x1^-x z_=yu#q)vl?pQrIvEq$+#d`M^U z*EWp*uEqSzd-tU|9aQ_Ycu3BAXfLXMUAY@|X;ygQ!JVP*GVkN3Ps8fjSy@klVJgIO zBXo3N#zV;UsKt+i*m8P6y=RyuJ&1u*O-;?@emqLh730lYR<2z646}=PSnnsaQ#yU! z=U*Sh6Wx#w$%s^w8)W%Ehlh1|Qc*5BV%H^3Ux9TU#`MbGPBPnv5i@*LPm&d>0lHs3 zz#OkM9^Uy8LGJ$C#=hqFh{n`zkbm-2ri+qk&b#L|`F#5;&B!wHBB@RW&p`u4HZ9v3vJv zVDr6Hk4aF_61e^tRGb2Zfx|&3faAxk4RdU(!)`wVhJaD#C6$evHwy!e0(KLo0FOif z*0WQA4J}}sNf=m683Jd+%FD}v2UomhuAg=mxM=`5v&6LEo;L8bl-p^*SQG+|onL+r z+;z0;{WiIj9EMG$ufyin$=80F_~q^GaA3I z&O*b#=YdHZG @time using Plots + 7.252560 seconds (13.27 M allocations: 792.701 MiB, 3.19% gc time) + +julia> @time using Example + 0.698413 seconds (1.60 M allocations: 79.200 MiB, 4.90% gc time) + +julia> @time display(plot(rand(5))) + 6.605240 seconds (11.02 M allocations: 568.698 MiB, 1.75% gc time) + +julia> using SIMD + +julia> @time display(plot(rand(5))) + 5.724359 seconds (16.36 M allocations: 825.491 MiB, 7.77% gc time) +``` + +In contrast, on the current development branch of Julia pre-1.6 (commit 1c9c24170c53273832088431de99fe19247af172), we have + +```julia-repl +julia> @time using Plots + 3.239665 seconds (7.80 M allocations: 543.796 MiB, 6.18% gc time) + +julia> @time using Example + 0.000742 seconds (2.91 k allocations: 185.703 KiB) + +julia> @time display(plot(rand(5))) + 5.428075 seconds (9.04 M allocations: 524.531 MiB, 2.59% gc time) + +julia> using SIMD + +julia> @time display(plot(rand(5))) + 0.114283 seconds (393.18 k allocations: 22.544 MiB) +``` + +You can see that loading is dramatically faster. +Why is usage faster? There likely to be several contributions: + +- improvements in compilation speed on Julia 1.6 (a topic worthy of a separate blog post) +- with less invalidation, some of the code needed for plotting may not need recompilation +- with less invalidation, more `precompile` statements deliver value + +As an example of the latter two, on Julia 1.5 the most expensive call to infer (which you can measure with SnoopCompile's `@snoopi` macro) is `recipe_pipeline!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{Array{Float64,1}}))`, which requires nearly 0.4s of inference time. +On Julia 1.6, this inference doesn't even need to happen, because the precompiled code is still valid. + +Since a lot of the reduction in invalidation comes from have less poorly-inferred code, we can get something of a comprehensive look by analyzing all the inferred `MethodInstance`s and determining which ones have been inferred for "problematic" signatures (e.g., `==(Int, Any)` when we'd prefer a concrete type for the second argument), and of those, how many other MethodInstances would be invalidated if those problematic instances were invalidated. Below is a histogram tallying the number of dependents for each problematic inferred signature: + +![backedges](/assets/blog/2020-invalidations/invalidation_backedge_analysis.png) + +The very large zero bin for the master branch indicates that many of the problematic signatures on Julia 1.5 are never needed by any of `master`'s precompiled code. This change is largely due to an extensive (though not exhaustive) audit of the code in Base and the standard libraries, fixing many places that were causing inference to return abstract types. These changes have made Base and the standard libraries less vulnerable to invalidation. + +### Outlook for the future + Julia's remarkable flexibility and outstanding code-generation open many new horizons. These advantages come with a few costs, and here we've explored one of them, method invalidation. While Julia's core developers have been aware of its cost for a long time, we're only now starting to get tools to analyze it in a manner suitable for a larger population of users and developers. Because it's not been easy to measure previously, there are still numerous opportunities for improvement waiting to be discovered. +In particular, while Base Julia and its standard libraries are well along the way to being "protected" from invalidation, the same may not be true of many packages. One might hope that the next period of Julia's development might see significant progress in getting packages to work together gracefully without inadvertently stomping on each other's toes. [Julia]: https://julialang.org/ From b6a72d2fb18ed5b795af2fdb5cbe03a8089c89b2 Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 26 Aug 2020 15:08:59 -0500 Subject: [PATCH 20/21] A bit more polish --- blog/2020/08/invalidations.md | 69 +++++++++++++++++++++-------------- 1 file changed, 41 insertions(+), 28 deletions(-) diff --git a/blog/2020/08/invalidations.md b/blog/2020/08/invalidations.md index 477391b670..b3e34cf16d 100644 --- a/blog/2020/08/invalidations.md +++ b/blog/2020/08/invalidations.md @@ -68,7 +68,7 @@ CodeInfo( The compiler knows that the answer will be 1, as long as the input array has an element indexable by 1. (That `arrayref` statement enforces bounds-checking, and ensure that Julia will throw an appropriate error if you -call `applyf([])`.) +pass `applyf` an empty container.) For the purpose of this blog post, things start to get especially interesting if we use a container that can store elements with different types (here, type `Any`): @@ -93,9 +93,9 @@ CodeInfo( This may seem a lot more complicated, but the take-home message is actually quite simple. First, look at that `arrayref` statement: it is annotated -`Any`, meaning that Julia can't predict in advance what it will return. +`::Any`, meaning that Julia can't predict in advance what it will return. Immediately after the reference, notice the `isa` statement -followed by a `φ`; all of this is essentially equivalent to: +followed by a `φ`; including the final return, all of this is essentially equivalent to: ```julia if isa(x, Int) @@ -103,12 +103,13 @@ if isa(x, Int) else value = f(x)::Int end +return value ``` The compiler might not know in advance what type `container[1]` will have, but it knows there's a method of `f` specialized for `Int`. To improve performance, it checks (as efficiently as possible at runtime) whether that method might be applicable, and if so calls the method it knows. -In this case, `f` just returns a constant, so when applicable the compiler even "hard-wires" the return value in for you. +In this case, `f` just returns a constant, so when applicable the compiler even hard-wires in the return value for you. However, the compiler also acknowledges the possiblity that `container[1]` *won't* be an `Int`. That allows the above code to throw a MethodError: @@ -132,6 +133,7 @@ CodeInfo( ) => Union{} ``` where in this case the compiler knows that the call won't succeed. +Calls annotated `::Union{}` do not return. ### Triggering method invalidation @@ -140,7 +142,7 @@ Depending on your version of Julia, you might get different results and/or need additional method for `f` to see the outcome shown here. Julia is interactive: we can define new methods on the fly and try them out. -Let's see what happens when we define a new method for `f`: +Having already run the above code in our session, let's see what happens when we define a new method for `f`: ```julia f(::Bool) = 2 @@ -166,7 +168,7 @@ CodeInfo( This shows that defining a new method for `f` forced *recompilation* of these `MethodInstance`s. -You can see that Julia no longer uses that optimization for when `container[1]` happens to be an `Int`. +You can see that Julia no longer uses that optimization checking whether `container[1]` happens to be an `Int`. Experience has shown that once a method has a couple of specializations, it may accumulate yet more, and to reduce the amount of recompilation needed Julia quickly abandons attempts to optimize the case handling containers with unknown element types. @@ -191,7 +193,7 @@ CodeInfo( ``` and you'll see that even this optimization is now abandoned. -This is because checking lots of methods, to see if they all return `Int`, is too costly an operation to be worth doing in all cases. +This is because checking lots of methods to see if they all return `Int` is too costly an operation to be worth doing in all cases. Altogether, `apply(::Vector{Any})` has been compiled three times: once when `f(::Int)` was the only method for `f`, once where there were two methods, and once when there were four. @@ -211,7 +213,7 @@ that will be the focus of this blog post. Unfortunately, method invalidation is (or rather, used to be) pretty common. First, let's get some baseline statistics. -Using the [MethodAnalysis] package, you can find out that a fresh Julia session has almost 50,000 `MethodInstance`s tucked away in its cache. +Using the [MethodAnalysis] package, you can find out that a fresh Julia session has approximately 50,000 `MethodInstance`s tucked away in its cache (more precisely, about 56,000 on Julia 1.5, about 45,000 on Julia 1.6). These are mostly for `Base` and the standard libraries. (There are some additional `MethodInstance`s that get created to load the MethodAnalysis package and do this analysis, but these are surely a very small fraction of the total.) @@ -245,7 +247,7 @@ work left to do, especially for particular packages. The next time you want to call functionality that gets invalidated, you have to wait for recompilation. -We can illustrate this using everyone's favorite example, plotting: +We can illustrate this on Julia 1.5 using everyone's favorite example, plotting: ```julia-repl julia> using Plots @@ -261,7 +263,7 @@ julia> @time display(plot(rand(5))) 0.311226 seconds (19.93 k allocations: 775.055 KiB) ``` -Moreover, if you decide you want some additional functionality and decide to load a new package, sometimes it's essentially as fast again: +Moreover, if you decide you want some additional functionality via ```julia-repl julia> using StaticArrays @@ -270,7 +272,12 @@ julia> @time display(plot(rand(5))) 0.305394 seconds (19.96 k allocations: 781.836 KiB) ``` -But if you load a package that does a lot of invalidation (on Julia versions 1.5 and below), +you can see it's essentially as fast again. However, StaticArrays was +already present because it gets loaded internally by Plots---all the +invalidations it causes have already happened before we issued that +second plotting command. Let's contrast that with what happens if you +load a package that does a lot of *new* invalidation (again, this is on Julia +versions 1.5 and below): ```julia-repl julia> using SIMD @@ -279,17 +286,15 @@ julia> @time display(plot(rand(5))) 7.238336 seconds (26.50 M allocations: 1.338 GiB, 7.88% gc time) ``` -Because so much got invalidated by loading SIMD, Julia had to recompile many methods before it could once again produce a plot, so that in terms of time it was almost as expensive as the first usage. -The size of the effect is strongly dependent on the particular packages and tasks, -and this particular example is already fixed on Julia 1.6. +Because so much got invalidated by loading SIMD, Julia had to recompile many methods before it could once again produce a plot, so that "time to second plot" was nearly as bad as "time to first plot." This doesn't just affect plotting or packages. For example, loading packages could transiently make the REPL, the next Pkg update, or the next call to a distributed worker lag for several seconds. You can minimize some of the costs of invalidation by loading all packages at the outset--if all invalidations happen before you start compiling very much code, then the first compilation already takes the full suite of methods into account. -On versions of Julia prior to 1.5 and 1.6, package load time was substantially increased by invalidation. +On versions of Julia prior to 1.5 and especially 1.6, package load time was substantially increased by invalidation. Especially on Julia 1.6, invalidation only rarely affects package load times, -largely because Pkg and the loading code have been made reasonably resistant to invalidation using some of the strategies described below. +largely because Pkg and the loading code have been made more resistant to invalidation. ## Can and should this be fixed? @@ -317,20 +322,28 @@ A [crucial change](https://github.com/JuliaLang/julia/pull/36733) was the realiz Another [pair](https://github.com/JuliaLang/julia/pull/36208) of [fundamental](https://github.com/JuliaLang/julia/pull/35904) changes led to subtle but significant alterations in how much Julia specializes poorly-inferred code to reflect the current state of the world. By making Julia less eager to specialize in such circumstances, huge swaths of the package ecosystem became more robust against invalidation. -Most of the remaining improvements stem from more directed changes that improved inferrability of dozens of specific methods. You find many examples by searching for the [latency tag](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+label%3Alatency+is%3Aclosed) among closed pull requests. +As demonstrated by the `applyf` example, the version that was well-inferred from the outset--when we passed it `container = [100]`, a `Vector{Int}`--never needed to be invalidated; it was only those `MethodInstance`s compiled for `Vector{Any}` that needed updating. So the last major component of how we've reduced invalidations is to improve the inferrability of Julia's own code. By recording and analyzing invalidations, we gained a much better understanding of where the weaknesses in Julia's own code were with respect to inferrability. We used this information to improve the implementations of dozens of methods, and added type annotations in many more, to help Julia generate better code. +You find many examples by searching for the [latency tag](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+label%3Alatency+is%3Aclosed) among closed pull requests. -Most of these changes were at least partially inspired by new tools to analyze the source of invalidations, and we briefly touch on this final topic below. +It's worth noting that the task of improving code is never finished, and this is true here as well. In tackling this problem we ended up developing a number of new tools to analyze the source of invalidations and fix the corresponding inference problems. We briefly touch on this final topic below. ## Tools for analyzing and fixing invalidations Recently, the [SnoopCompile] package gained the ability to analyze invalidations and help developers fix them. -Because these tools will likely change over time, this blog post will only touch the surface; people who want to help fix invalidations are encouraged to read [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) for further detail. +Because these tools will likely change over time, this blog post will only scratch the surface; people who want to help fix invalidations are encouraged to read [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) for further detail. There is also a [video](https://www.youtube.com/watch?v=7VbXbI6OmYo) available with a live-session fixing a real-world invalidation, which might serve as a useful example. But to give you a taste of what this looks like, here are a couple of screenshots. These were taken in Julia's REPL, but you can also use these tools in [vscode] or other environments. -First, let's look at a simple way (one that is not always precisely accurate, see SnoopCompile's documentation) to collect data on a package (in this case, loading the SIMD package): +First, let's look at a simple way (one that is not always precisely accurate, see SnoopCompile's documentation) to collect data on a package: + +``` +using SnoopCompile +trees = invalidation_trees(@snoopr using SIMD) +``` + +which in the REPL looks like this: ![snoopr_simd](/assets/blog/2020-invalidations/SIMD_invalidations.png) @@ -349,7 +362,7 @@ since the `args` field of a `CallWaitMsg` must be a `Tuple`, and because inferen But SIMD defines a new `convert(Tuple, v::Vec)` method which has greater specificity, and so loading the SIMD package triggers invalidation of everything that depends on the less-specific method `convert(Tuple, ::Any)`. -As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `CallWaitMsg` (it doesn't need to call `convert` if there are no restrictions on the type), or we could assert that `args` is *already* a tuple in `deserialize_msg`, thus informing Julia that it can afford to skip the call to `convert`. +As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `struct CallWaitMsg` (it doesn't need to call `convert` if there are no restrictions on the type), or we could assert that `args` is *already* a tuple in `deserialize_msg`, thus informing Julia that it can afford to skip the call to `convert`. This is intended only as a taste of what's involved in fixing invalidations; more extensive descriptions are available in [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) and the [video](https://www.youtube.com/watch?v=7VbXbI6OmYo). But it also conveys an important point: most invalidations come from poorly-inferred code, so by fixing invalidations you're often improving quality in other ways. Julia's [performance tips] page has a wealth of good advice about avoiding non-inferrable code, and in particular cases (where you might know more about the types than inference is able to determine on its own) you can help inference by adding type-assertions. @@ -381,18 +394,18 @@ In contrast, on the current development branch of Julia pre-1.6 (commit 1c9c2417 ```julia-repl julia> @time using Plots - 3.239665 seconds (7.80 M allocations: 543.796 MiB, 6.18% gc time) + 3.133783 seconds (7.49 M allocations: 526.797 MiB, 6.90% gc time) julia> @time using Example - 0.000742 seconds (2.91 k allocations: 185.703 KiB) + 0.002499 seconds (2.69 k allocations: 193.344 KiB) julia> @time display(plot(rand(5))) - 5.428075 seconds (9.04 M allocations: 524.531 MiB, 2.59% gc time) + 5.481113 seconds (9.71 M allocations: 562.839 MiB, 2.28% gc time) julia> using SIMD julia> @time display(plot(rand(5))) - 0.114283 seconds (393.18 k allocations: 22.544 MiB) + 0.115085 seconds (395.11 k allocations: 22.646 MiB) ``` You can see that loading is dramatically faster. @@ -403,13 +416,13 @@ Why is usage faster? There likely to be several contributions: - with less invalidation, more `precompile` statements deliver value As an example of the latter two, on Julia 1.5 the most expensive call to infer (which you can measure with SnoopCompile's `@snoopi` macro) is `recipe_pipeline!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{Array{Float64,1}}))`, which requires nearly 0.4s of inference time. -On Julia 1.6, this inference doesn't even need to happen, because the precompiled code is still valid. +On Julia's master branch, this inference doesn't even need to happen, because the precompiled code is still valid. -Since a lot of the reduction in invalidation comes from have less poorly-inferred code, we can get something of a comprehensive look by analyzing all the inferred `MethodInstance`s and determining which ones have been inferred for "problematic" signatures (e.g., `==(Int, Any)` when we'd prefer a concrete type for the second argument), and of those, how many other MethodInstances would be invalidated if those problematic instances were invalidated. Below is a histogram tallying the number of dependents for each problematic inferred signature: +There's another way to get a more comprehensive look at the state of Julia and the progress since 1.5: since most invalidations come from poorly-inferred code, we can analyze `MethodInstance`s and determine which ones have been inferred for "problematic" signatures (e.g., `==(Int, Any)` when we'd prefer a concrete type for the second argument). Aside from counting them, it's also worth noting how many *other* `MethodInstance`s depend on these "bad" `MethodInstance`s and would therefore be invalidated if we happened to define the right (or wrong) new method. Below is a histogram tallying the number of dependents for each problematic inferred signature, comparing Julia 1.5 (top) with the master branch (bottom): ![backedges](/assets/blog/2020-invalidations/invalidation_backedge_analysis.png) -The very large zero bin for the master branch indicates that many of the problematic signatures on Julia 1.5 are never needed by any of `master`'s precompiled code. This change is largely due to an extensive (though not exhaustive) audit of the code in Base and the standard libraries, fixing many places that were causing inference to return abstract types. These changes have made Base and the standard libraries less vulnerable to invalidation. +The very large zero bin for the master branch indicates that many of the problematic signatures on Julia 1.5 do not even appear in `master`'s precompiled code. This plot reveals extensive progress, but that tail suggests that the effort is not yet entirely done. ### Outlook for the future From 338e776e0df0d0ff150ca760c009d8eab1a63add Mon Sep 17 00:00:00 2001 From: Tim Holy Date: Wed, 26 Aug 2020 15:55:18 -0500 Subject: [PATCH 21/21] Yet more polish --- blog/2020/08/invalidations.md | 42 +++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/blog/2020/08/invalidations.md b/blog/2020/08/invalidations.md index b3e34cf16d..d10f6ee8f8 100644 --- a/blog/2020/08/invalidations.md +++ b/blog/2020/08/invalidations.md @@ -219,7 +219,7 @@ These are mostly for `Base` and the standard libraries. Using [SnoopCompile], we can count the number of invalidations triggered by loading various packages into a fresh Julia session: -| Package | Version | # invalidations (Julia 1.5) | # invalidations (master branch) | +| Package | Version | # inv., Julia 1.5 | # inv., master branch | |:------- | -------:| ---------:| ---------:| | Example | 0.5.3 | 0 | 0 | | Revise | 2.7.3 | 23 | 0 | @@ -235,9 +235,9 @@ Using [SnoopCompile], we can count the number of invalidations triggered by load | JuMP | 0.21.3 | 4281 | 1952 | | DifferentialEquations | 6.15.0 | 6373 | 2309 | -('x' indicates that the package cannot be loaded. Run when Julia's `master` was at commit 1c9c24170c53273832088431de99fe19247af172, pre-1.6.) +('x' indicates that the package cannot be loaded. Julia's master-branch results were at commit 1c9c24170c53273832088431de99fe19247af172, partway through the development of Julia 1.6.) -You can see that key packages used by large portions of the Julia ecosystem have traditionally +You can see that key packages used by large portions of the Julia ecosystem have historically invalidated hundreds or thousands of `MethodInstance`s, sometimes more than 10% of the total number of `MethodInstance`s present before loading the package. The situation has been dramatically improved on Julia 1.6, but there remains more @@ -292,8 +292,8 @@ This doesn't just affect plotting or packages. For example, loading packages cou You can minimize some of the costs of invalidation by loading all packages at the outset--if all invalidations happen before you start compiling very much code, then the first compilation already takes the full suite of methods into account. -On versions of Julia prior to 1.5 and especially 1.6, package load time was substantially increased by invalidation. -Especially on Julia 1.6, invalidation only rarely affects package load times, +On versions of Julia up to and including 1.5, package load time was substantially increased by invalidation. +On Julia 1.6, invalidation only rarely affects package load times, largely because Pkg and the loading code have been made more resistant to invalidation. ## Can and should this be fixed? @@ -312,7 +312,7 @@ We will never get rid of invalidation altogether; as the `applyf` example above invalidation is sometimes necessary if you want both good runtime performance and interactive usage, and this combination is one of the best things about Julia. The real question is whether there are *unnecessary* invalidations, or an unexploited strategy to limit their impact. -Determining the answer to that question requires that we develop an understanding the common sources of invalidation and what can be done to fix them. +While the table above resoundingly demonstrates the feasibility of reducing the frequency of invalidation, going any further requires that we develop an understanding the common sources of invalidation and what can be done to fix them. ## Changes in Julia that have reduced the number of invalidations @@ -322,7 +322,7 @@ A [crucial change](https://github.com/JuliaLang/julia/pull/36733) was the realiz Another [pair](https://github.com/JuliaLang/julia/pull/36208) of [fundamental](https://github.com/JuliaLang/julia/pull/35904) changes led to subtle but significant alterations in how much Julia specializes poorly-inferred code to reflect the current state of the world. By making Julia less eager to specialize in such circumstances, huge swaths of the package ecosystem became more robust against invalidation. -As demonstrated by the `applyf` example, the version that was well-inferred from the outset--when we passed it `container = [100]`, a `Vector{Int}`--never needed to be invalidated; it was only those `MethodInstance`s compiled for `Vector{Any}` that needed updating. So the last major component of how we've reduced invalidations is to improve the inferrability of Julia's own code. By recording and analyzing invalidations, we gained a much better understanding of where the weaknesses in Julia's own code were with respect to inferrability. We used this information to improve the implementations of dozens of methods, and added type annotations in many more, to help Julia generate better code. +Finally, as demonstrated by the `applyf` example, the version that was well-inferred from the outset--when we passed it `container = [100]`, a `Vector{Int}`--never needed to be invalidated; it was only those `MethodInstance`s compiled for `Vector{Any}` that needed updating. So the last major component of how we've reduced invalidations is to improve the inferrability of Julia's own code. By recording and analyzing invalidations, we gained a much better understanding of where the weaknesses in Julia's own code were with respect to inferrability. We used this information to improve the implementations of dozens of methods, and added type annotations in many more, to help Julia generate better code. You find many examples by searching for the [latency tag](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+label%3Alatency+is%3Aclosed) among closed pull requests. It's worth noting that the task of improving code is never finished, and this is true here as well. In tackling this problem we ended up developing a number of new tools to analyze the source of invalidations and fix the corresponding inference problems. We briefly touch on this final topic below. @@ -362,17 +362,17 @@ since the `args` field of a `CallWaitMsg` must be a `Tuple`, and because inferen But SIMD defines a new `convert(Tuple, v::Vec)` method which has greater specificity, and so loading the SIMD package triggers invalidation of everything that depends on the less-specific method `convert(Tuple, ::Any)`. -As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `struct CallWaitMsg` (it doesn't need to call `convert` if there are no restrictions on the type), or we could assert that `args` is *already* a tuple in `deserialize_msg`, thus informing Julia that it can afford to skip the call to `convert`. +As is usually the case, there are several ways to fix this: we could drop the `::Tuple` in the definition of `struct CallWaitMsg`, because there's no need to call `convert` if the object is already known to have the correct type (which would be `Any` if we dropped the field type-specification, and `Any` is not restrictive). Alternatively, we could keep the `::Tuple` in the type-specification, and use external knowledge that we have and assert that `args` is *already* a `Tuple` in `deserialize_msg` where it calls `CallWaitMsg`, thus informing Julia that it can afford to skip the call to `convert`. This is intended only as a taste of what's involved in fixing invalidations; more extensive descriptions are available in [SnoopCompile's documentation](https://timholy.github.io/SnoopCompile.jl/stable/snoopr/) and the [video](https://www.youtube.com/watch?v=7VbXbI6OmYo). But it also conveys an important point: most invalidations come from poorly-inferred code, so by fixing invalidations you're often improving quality in other ways. Julia's [performance tips] page has a wealth of good advice about avoiding non-inferrable code, and in particular cases (where you might know more about the types than inference is able to determine on its own) you can help inference by adding type-assertions. -Again, some of the recently-merged `latency` pull requests to Julia might serve as instructive examples. +Again, some of the recently-merged "latency" pull requests to Julia might serve as instructive examples. ## Summary ### Impacts of progress to date -The cumulative impact of these and other changes over the course of the development (so far) of Julia 1.6 is noticeable. For example, on Julia 1.5 we have extra latencies to load the next package (due to the loading code being invalidated) and to execute already-code if it gets invalidated by loading other packages: +The cumulative impact of these and other changes over the course of the development (so far) of Julia 1.6 is noticeable. For example, on Julia 1.5 we have extra latencies to load the next package (due to the loading code being invalidated) and to execute already-compiled code if it gets invalidated by loading other packages: ```julia-repl julia> @time using Plots @@ -408,21 +408,25 @@ julia> @time display(plot(rand(5))) 0.115085 seconds (395.11 k allocations: 22.646 MiB) ``` -You can see that loading is dramatically faster. -Why is usage faster? There likely to be several contributions: +You can see that the first load is dramatically faster, an effect that is partly but not completely due to reduction in invalidations (reducing invalidations helped by preventing sequential invalidation of the loading code itself as Julia loads the sequence of packages needed for Plots). This effect is illustrated clearly by the load of `Example`, which on Julia 1.5 led to another 0.7s worth of recompilation due to invalidations triggered by loading Plots; on Julia's master branch, this cost is gone because Plots does not invalidate the loading code. -- improvements in compilation speed on Julia 1.6 (a topic worthy of a separate blog post) -- with less invalidation, some of the code needed for plotting may not need recompilation -- with less invalidation, more `precompile` statements deliver value +Likewise, the second call to `display(plot(...))` is much faster, because loading SIMD did not extensively invalidate crucial code compiled for plotting. -As an example of the latter two, on Julia 1.5 the most expensive call to infer (which you can measure with SnoopCompile's `@snoopi` macro) is `recipe_pipeline!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{Array{Float64,1}}))`, which requires nearly 0.4s of inference time. -On Julia's master branch, this inference doesn't even need to happen, because the precompiled code is still valid. +Perhaps the biggest mystery is, why is the first usage faster? There likely to be several contributions: -There's another way to get a more comprehensive look at the state of Julia and the progress since 1.5: since most invalidations come from poorly-inferred code, we can analyze `MethodInstance`s and determine which ones have been inferred for "problematic" signatures (e.g., `==(Int, Any)` when we'd prefer a concrete type for the second argument). Aside from counting them, it's also worth noting how many *other* `MethodInstance`s depend on these "bad" `MethodInstance`s and would therefore be invalidated if we happened to define the right (or wrong) new method. Below is a histogram tallying the number of dependents for each problematic inferred signature, comparing Julia 1.5 (top) with the master branch (bottom): +- improvements in compilation speed on Julia 1.6 (a separate line of work worthy of its own blog post) +- with less invalidation, some Base code exploited by plotting may not need recompilation +- with less invalidation, more `precompile` statements for Plots' own internal methods can be used + +As an example of the latter two, on Julia 1.5 the most expensive call to infer during the `display(plot(...))` call (which you can measure with SnoopCompile's `@snoopi` macro) is `recipe_pipeline!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{Array{Float64,1}}))`, which by itself requires nearly 0.4s of inference time. +Plot actually attempts to precompile this statement, but it can't benefit from the result because some of the code that this statement depends on gets invalidated by loading Plots. +On Julia's master branch, this inference doesn't need to happen because the precompiled code is still valid. + +There's another way to get a more comprehensive look at the state of Julia and the progress since 1.5: since most invalidations come from poorly-inferred code, we can analyze all extant `MethodInstance`s and determine which ones have been inferred for "problematic" signatures (e.g., `isequal(::Int, ::Any)` when we'd prefer a concrete type for the second argument). Aside from counting them, it's also worth noting how many *other* `MethodInstance`s depend on these "bad" `MethodInstance`s and would therefore be invalidated if we happened to define the right (or wrong) new method. Below is a histogram tallying the number of dependents for each problematic inferred signature, comparing Julia 1.5 (top) with the master branch (bottom): ![backedges](/assets/blog/2020-invalidations/invalidation_backedge_analysis.png) -The very large zero bin for the master branch indicates that many of the problematic signatures on Julia 1.5 do not even appear in `master`'s precompiled code. This plot reveals extensive progress, but that tail suggests that the effort is not yet entirely done. +The very large zero bin for the master branch indicates that many of the problematic signatures on Julia 1.5 do not even appear in `master`'s precompiled code. This plot reveals extensive progress, but that tail suggests that the effort is not yet done. ### Outlook for the future