Updating dev branches with latest version from upstream #85
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uliaLang#56668) Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
This makes them usable for external consumers like GPUCompiler.jl.
I'm not sure what `` `cmd` `` could refer to, but it would make sense to refer to `` `str` `` in this case. I'm assuming it's a typo.
…56702) These are not user-visible, so this makes the compiler faster and more efficient with no effort on our part, and avoids duplicating the debug_level parameter.
This helps when profiling remotely since VTunes doesn't support setting environment variables on remote systems. Will still respect `ENABLE_JITPROFILING=0`.
A `TAGGED_RELEASE_BANNER` with spaces such as `Official https://julialang.org release` produces the error `/cache/build/builder-amdci4-5/julialang/julia-master/deps/tools/jlchecksum: 66: [: Official: unexpected operator`.
…tation (JuliaLang#56727) Fixes JuliaLang#56680. This PR updates the documentation for the ispunct function in Julia to explicitly note its differing behavior from the similarly named function in C. --------- Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com>
- support gc running - don't duplicate field 4 - remove some unused code only previously needed for handling cycles (which are not valid in IR)
Since most ssavalue are used just after their def, this gives a small memory savings on compressed IR (a fraction of a percent).
When we declare inner methods, e.g. the `f` in
```
function fs()
f(lhs::Integer) = 1
f(lhs::Integer, rhs::(local x=Integer; x)) = 2
return f
end
```
we must hoist the definition of the (appropriately mangled) generic
function `f` to top-level, including all variables that were used in the
signature definition of `f`. This situation is a bit unique in the
language because it uses inner function scope, but gets executed in
toplevel scope. For example, you're not allowed to use a local of the
inner function in the signature definition:
```
julia> function fs()
local x=Integer
f(lhs::Integer, rhs::x) = 2
return f
end
ERROR: syntax: local variable x cannot be used in closure declaration
Stacktrace:
[1] top-level scope
@ REPL[3]:1
```
In particular, the restriction is signature-local:
```
julia> function fs()
f(rhs::(local x=Integer; x)) = 1
f(lhs::Integer, rhs::x) = 2
return f
end
ERROR: syntax: local variable x cannot be used in closure declaration
Stacktrace:
[1] top-level scope
@ REPL[4]:1
```
There's a special intermediate form `moved-local` that gets generated
for this definition. In c6c3d72, this
form stopped getting generated for certain inner methods. I suspect this
happened because of the incorrect assumption that the set of moved
locals is being computed over all signatures, rather than being a
per-signature property.
The result of all of this was that this is one of the few places where
lowering still generated a symbol as the lhs of an assignment for a
global (instead of globalref), because the code that generates the
assignment assumes it's a local, but the later pass doesn't know this.
Because we still retain the code for this from before we started using
globalref consistently, this wasn't generally causing a problems, except
possibly leaking a global (or potentially assigning to a global when
this wasn't intended). However, in follow on work, I want to make use of
knowing whether the LHS is a global or local in lowering, so this was
causing me trouble.
Fix all of this by putting back the `moved-local` where it was dropped.
Fixes JuliaLang#56711
…ll zero When not all-zero, run-length encoding would also probably be great here for lowered code (before inference).
This is an alternative mechanism to JuliaLang#56650 that largely achieves the same result, but by hooking into `invoke` rather than a generated function. They are orthogonal mechanisms, and its possible we want both. However, in JuliaLang#56650, both Jameson and Valentin were skeptical of the generated function signature bottleneck. This PR is sort of a hybrid of mechanism in JuliaLang#52964 and what I proposed in JuliaLang#56650 (comment). In particular, this PR: 1. Extends `invoke` to support a CodeInstance in place of its usual `types` argument. 2. Adds a new `typeinf` optimized generic. The semantics of this optimized generic allow the compiler to instead call a companion `typeinf_edge` function, allowing a mid-inference interpreter switch (like JuliaLang#52964), without being forced through a concrete signature bottleneck. However, if calling `typeinf_edge` does not work (e.g. because the compiler version is mismatched), this still has well defined semantics, you just don't get inference support. The additional benefit of the `typeinf` optimized generic is that it lets custom cache owners tell the runtime how to "cure" code instances that have lost their native code. Currently the runtime only knows how to do that for `owner == nothing` CodeInstances (by re-running inference). This extension is not implemented, but the idea is that the runtime would be permitted to call the `typeinf` optimized generic on the dead CodeInstance's `owner` and `def` fields to obtain a cured CodeInstance (or a user-actionable error from the plugin). This PR includes an implementation of `with_new_compiler` from JuliaLang#56650. This PR includes just enough compiler support to make the compiler optimize this to the same code that JuliaLang#56650 produced: ``` julia> @code_typed with_new_compiler(sin, 1.0) CodeInfo( 1 ─ $(Expr(:foreigncall, :(:jl_get_tls_world_age), UInt64, svec(), 0, :(:ccall)))::UInt64 │ %2 = builtin Core.getfield(args, 1)::Float64 │ %3 = invoke sin(%2::Float64)::Float64 └── return %3 ) => Float64 ``` However, the implementation here is extremely incomplete. I'm putting it up only as a directional sketch to see if people prefer it over JuliaLang#56650. If so, I would prepare a cleaned up version of this PR that has the optimized generics as well as the curing support, but not the full inference integration (which needs a fair bit more work).
…6713) This adjusts lowering to emit `setglobal!` for assignment to globals, thus making the `=` expr head used only for slots in `CodeInfo` and entirely absent in `IRCode`. The primary reason for this is just to reduce the number of special cases that compiler passes have to reason about. In IRCode, `=` was already essentially equivalent to `setglobal!`, so there's no good reason not to canonicalize. Finally, the `=` syntax form for globals already gets recognized specially to insert `convert` calls to their declared binding type, so this doesn't impose any additional requirements on lowering to distinguish local from global assignments. In general, I'd also like to separate syntax and intermediate forms as much as possible where their semantics differ, which this accomplises by just using the builtin. This change is mostly semantically invisible, except that spliced-in GlobalRefs now declare their binding because they are indistinguishable from ordinary assignments at the stage where I inserted the lowering. If we want to, we can preserve the difference, but it'd be a bit more annoying for not much benefit, because: 1. The spliced in version was only recently made to work anyway, and 2. The semantics of when exactly bindings are declared is still messy on master anyway and will get tweaked shortly in further binding partitions work.
… the REPL (JuliaLang#54800) When a user requests a completion for a backslash shortcode, this PR adds the glyphs for all the suggestions to the output. This makes it much easier to find the result one is looking for, especially if the user doesn't know all latex and emoji specifiers by heart. Before: <img width="813" alt="image" src="https://github.com/JuliaLang/julia/assets/22495855/bf651399-85a6-4677-abdc-c66a104e3b89"> After: <img width="977" alt="image" src="https://github.com/JuliaLang/julia/assets/22495855/04c53ea2-318f-4888-96eb-0215b49c10f3"> --------- Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
…g#56736) Annotations now use a NamedTuple
There is no reason to preserve duplicates of the bits for the value before and after inference, and many of the numbers in the comments had gotten incorrect. Now we are able to pack all 16 bits of currently defined bitflags into 20 bits, instead of 25 bits (although either case still rounds up to 32). There was also no reason for InferenceState to be mutating of CodeInfo during execution, so remove that mutation.
…#56743) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: 7b759d7f0 New commit: d84a1a38b Julia version: 1.12.0-DEV Pkg version: 1.12.0 Bump invoked by: @KristofferC Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@7b759d7...d84a1a3 ``` $ git log --oneline 7b759d7f0..d84a1a38b d84a1a38b Allow use of a url and subdir in [sources] (JuliaLang#4039) cd75456a8 Fix heading (JuliaLang#4102) b61066120 rename FORMER_STDLIBS -> UPGRADABLE_STDLIBS (JuliaLang#4070) 814949ed2 Increase version of `StaticArrays` in `why` tests (JuliaLang#4077) 83e13631e Run CI on backport branch too (JuliaLang#4094) ``` Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
- support gc running - don't duplicate field 4 - remove some unused code only previously needed for handling cycles (which are not valid in IR) - avoid serializing ssaflags, which are very large (relative to the rest of the IR) and usually all zeros sysimg : 159 MB => 157 MB => 154 MB du -sh usr/share/julia/compiled : 260M => 256 MB => 254 MB
Revise in theory wants to re-evaluate this include, but it fails at doing so, because the include call no longer works after bootstrap. It happens to work right now on master, because the lowering of `Compiler.include` happens to hide the include call from Revise, but that's a Revise bug I'm about to fix. Address this by moving the include call into the package and using an absolute include if necessary.
We need to quote it, otherwise it would result in `UnderVarError`.
This PR generalizes the `reshape` methods to accept `Integer`s instead
of `Int`s, and adds a `_reshape_uncolon` method for `Integer` arguments.
The current `_reshape_uncolon` method that accepts `Int`s is left
unchanged to ensure that the inferred types are not impacted. I've also
tried to ensure that most `Integer` subtypes in `Base` that may be
safely converted to `Int`s pass through that method.
The call sequence would now go like this:
```julia
reshape(A, ::Tuple{Vararg{Union{Integer, Colon}}}) -> reshape(A, ::Tuple{Vararg{Integer}}) -> reshape(A, ::Tuple{Vararg{Int}}) (fallback)
```
This lets packages define `reshape(A::CustomArray, ::Tuple{Integer,
Vararg{Integer}})` without having to implement `_reshape_uncolon` by
themselves (or having to call internal `Base` functions, as in
JuliaArrays/FillArrays.jl#373). `reshape`
calls involving a `Colon` would convert this to an `Integer` in `Base`,
and then pass the `Integer` sizes to the custom method defined in the
package.
This PR does not resolve issues like
JuliaLang#40076 because this still
converts `Integer`s to `Int`s in the actual reshaping step. However,
`BigInt` sizes that may be converted to `Int`s will work now:
```julia
julia> reshape(1:4, big(2), big(2))
2×2 reshape(::UnitRange{Int64}, 2, 2) with eltype Int64:
1 3
2 4
julia> reshape(1:4, big(1), :)
1×4 reshape(::UnitRange{Int64}, 1, 4) with eltype Int64:
1 2 3 4
```
Note that the reshape method with `Integer` sizes explicitly converts
these to `Int`s to avoid self-recursion (as opposed to calling
`to_shape` to carry out the conversion implicitly). In the future, we
may want to decide what to do with types or values that can't be
converted to an `Int`.
---------
Co-authored-by: Neven Sajko <s@purelymail.com>
Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
Some more questions still to be answered, but this should address the immediate actionable review items.
Close JuliaLang#47351 (builds on top of JuliaLang#48416) Adds two per-task metrics: - running time = amount of time the task was actually running (according to our scheduler). Note: currently inclusive of GC time, but would be good to be able to separate that out (in a future PR) - wall time = amount of time between the scheduler becoming aware of this task and the task entering a terminal state (i.e. done or failed). We record running time in `wait()`, where the scheduler stops running the task as well as in `yield(t)`, `yieldto(t)` and `throwto(t)`, which bypass the scheduler. Other places where a task stops running (for `Channel`, `ReentrantLock`, `Event`, `Timer` and `Semaphore` are all implemented in terms of `wait(Condition)`, which in turn calls `wait()`. `LibuvStream` similarly calls `wait()`. This should capture everything (albeit, slightly over-counting task CPU time by including any enqueuing work done before we hit `wait()`). The various metrics counters could be a separate inlined struct if we think that's a useful abstraction, but for now i've just put them directly in `jl_task_t`. They are all atomic, except the `metrics_enabled` flag itself (which we now have to check on task start/switch/done even if metrics are not enabled) which is set on task construction and marked `const` on the julia side. In future PRs we could add more per-task metrics, e.g. compilation time, GC time, allocations, potentially a wait-time breakdown (time waiting on locks, channels, in the scheduler run queue, etc.), potentially the number of yields. Perhaps in future there could be ways to enable this on a per-thread and per-task basis. And potentially in future these same timings could be used by `@time` (e.g. writing this same timing data to a ScopedValue like in JuliaLang#55103 but only for tasks lexically scoped to inside the `@time` block). Timings are off by default but can be turned on globally via starting Julia with `--task-metrics=yes` or calling `Base.Experimental.task_metrics(true)`. Metrics are collected for all tasks created when metrics are enabled. In other words, enabling/disabling timings via `Base.Experimental.task_metrics` does not affect existing `Task`s, only new `Task`s. The other new APIs are `Base.Experimental.task_running_time_ns(::Task)` and `Base.Experimental.task_wall_time_ns(::Task)` for retrieving the new metrics. These are safe to call on any task (including the current task, or a task running on another thread). All these are in `Base.Experimental` to give us room to change up the APIs as we add more metrics in future PRs (without worrying about release timelines). cc @NHDaly @kpamnany @d-netto --------- Co-authored-by: Pete Vilter <pete.vilter@gmail.com> Co-authored-by: K Pamnany <kpamnany@users.noreply.github.com> Co-authored-by: Nathan Daly <nathan.daly@relational.ai> Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
This is an alternative to JuliaLang#56714 that goes in the opposite direction - just outline all GlobalRefs during lowering. It is a lot simpler that JuliaLang#56714 at the cost of some size increase. Numbers: sys.so .ldata size: This PR: 159.8 MB Master: 158.9 MB I don't have numbers of JuliaLang#56714, because it's not fully complete. Additionally, it's possible that restricting GlobalRefs from arguments position would let us use a more efficient encoding in the future.
Show glyphs for latex or emoji shortcodes being suggested in the REPL
…Lang#57169) Co-authored-by: IanButterworth <1694067+IanButterworth@users.noreply.github.com>
This is another step in the long-term direction to have type-inference fully-explore the code graph we want to emit, largely side-stepping the vestigial `jl_type_infer` paths that we still have in type-inference for ccallable (and OpaqueClosure) This is needed for `juliac`, since the trim checks in codegen check if code was enqueued in the workqueue as expected. --------- Co-authored-by: Jameson Nash <vtjnash@gmail.com>
`Ryu.writeshortest` is documented to have a method that "allows passing
in a byte buffer", just like `Ryu.writefixed` and `Ryu.writeexp`, but
unlike those functions `writeshortest` is type constrained to
`::Vector{UInt8}`. This PR loosens that to `::AbstractVector{UInt8}`, to
allow the "byte buffer" to e.g. be a `Memory` rather than a `Vector`.
I've added tests and updated the docstrings for all three functions to
ensure that they're not just restricted to `Vector{UInt8}`.
This change was prompted by our private codebase hitting `MethodError:
no method matching writeshortest(::Memory{UInt8}, ::Int64, ::Float64,
...)` when trying it out with Julia v1.12.0-DEV.
(cc @quinnj -- i think you added this method originally, but i couldn't
see any reason why e.g. Memory shouldn't be allowed now we have it)
…iaLang#57150) This is the analog of JuliaLang#57102 for global variables. Unlike for consants, there is no automatic global backdate mechanism. The reasoning for this is that global variables can be declared at any time, unlike constants which can only be decalared once their value is available. As a result code patterns using `Core.eval` to declare globals are rarer and likely incorrect.
udesou
force-pushed
the
merging-master-into-dev
branch
4 times, most recently
from
c80b388 to
66b1fa9
Compare
udesou
changed the title
udesou
force-pushed
the
merging-master-into-dev
branch
2 times, most recently
from
8669faa to
ac06dbb
Compare
udesou
force-pushed
the
merging-master-into-dev
branch
from
ac06dbb to
bda40bd
Compare
…#57194) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: bc9fb21b1 New commit: 6091533bc Julia version: 1.12.0-DEV Pkg version: 1.12.0 Bump invoked by: @KristofferC Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@bc9fb21...6091533 ``` $ git log --oneline bc9fb21b1..6091533bc 6091533bc fix ambiguity in apps `rm` (JuliaLang#4144) ecdf6aa38 rename rot13 app in test to avoid name conflicts on systems where such a binary is already installed (JuliaLang#4143) a3626bf29 add `PREV_ENV_PATH` to globals getting reset and move the reset to before precompilation is finished instead of `__init__` (JuliaLang#4142) 938e9b24e app support in Pkg (JuliaLang#3772) df1931a93 Use copy_test_packages in some places, and ensure that the copied test package is user writable. This allows running the Pkg tests as part of Base.runtests of an installed Julia installation, where sources might be readonly. (JuliaLang#4136) 2609a9411 don't set up callbacks if using cli git (JuliaLang#4128) ``` Co-authored-by: KristofferC <1282691+KristofferC@users.noreply.github.com>
qinsoon
approved these changes
Jan 29, 2025
udesou
force-pushed
the
merging-master-into-dev
branch
from
f47523c to
259f6cc
Compare
udesou
force-pushed
the
merging-master-into-dev
branch
from
259f6cc to
263269c
Compare
udesou
added a commit
to mmtk/mmtk-julia
that referenced
this pull request
Jan 31, 2025
Updating `dev`. Merge with mmtk/julia#85. --------- Co-authored-by: Stephen Kell <srk31-github@srcf.ucam.org> Co-authored-by: Yi Lin <qinsoon@gmail.com> Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@rat.moma>
qinsoon
added a commit
to qinsoon/mmtk-julia
that referenced
this pull request
Feb 3, 2025
Updating `dev`. Merge with mmtk/julia#85. --------- Co-authored-by: Stephen Kell <srk31-github@srcf.ucam.org> Co-authored-by: Yi Lin <qinsoon@gmail.com> Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@rat.moma>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR merges the latest changes from upstream into our dev branch.