Loading external StyledString package removes markdown highlighting

[KristofferC]

This is when having this package as the active project and loading it.

[topolarity]

Do we have a root cause for this yet?

[tecosaur]

I don't, but I have noticed that with StyledStrings as the active project you can see this in a Julia REPL session:

help?> thing
# Observe working syntax highlighting

julia> using StyledStrings

help?> thing
# Syntax highlighting breaks

julia> using Markdown

help?> thing
# Syntax highlighting works again

Checking Base.loaded_modules_order it seems like two versions of StyledStrings, JuliaSyntaxHighlighting, and Markdown are loaded by doing this:

julia> Base.loaded_modules_order
22-element Vector{Module}:
 Core
 Base
 Main
 FileWatching
 Libdl
 Artifacts
 SHA
 Sockets
 LinearAlgebra
 OpenBLAS_jll
 libblastrampoline_jll
 Random
 Base64
 StyledStrings
 JuliaSyntaxHighlighting
 Markdown
 InteractiveUtils
 Unicode
 REPL
 StyledStrings
 JuliaSyntaxHighlighting
 Markdown
 
julia> length(unique(Base.loaded_modules_order))
22

[topolarity]

If I had to guess, StyledStrings has some code that is embedding Face types:

StyledStrings.jl/src/styledmarkup.jl

Line 725 in da41b6a

face = Expr(:call, Face, kwargs...)

but then getface here is effectively doing a isa Face check:

StyledStrings.jl/src/io.jl

Line 237 in da41b6a

face = getface(styles)

which runs into an issue almost exactly like the one I accidentally created with TOML recently (JuliaLang/Pkg.jl#4017 (comment))

Basically require_stdlib means that multiple StyledStrings can exist, which means that the _ansi_writer code has to support the Face type for all of the loaded StyledStrings.

[topolarity]

I think I found the issue - The second copy of StyledStrings contains a default "FACES" dictionary (w/o the Markdown faces) and then overrides Base.write(::IO, ::Base.AnnotatedString) so that any old AnnotatedStrings floating about start seeing the "FACES" only from the second loaded copy of StyledStrings. In particular, the AnnotatedString's created by Markdown are suddenly missing their Face definitions.

Mis-behavior like this is really easy to get with require_stdlib + type-piracy, so I've started to document the rules of the game in JuliaLang/julia#56005

The part that matters is:

A stdlib must not access any global state that may differ between stdlib copies in type-pirated methods

Given that guideline, I think we only have a few options here:

• Eliminate the type piracy: Make AnnotatedString a different type for each of the StyledStrings copies (move it out of Base)
• Avoid FACES duplication: StyledStrings needs to move into the sysimage, or require_stdlib needs to be banned
• Include a unique, type-based identifier for Annotations so that StyledStrings can resolve FACES in a non-type-pirated method
• Maintain a reference to the relevant FACES dict in the AnnotatedString

The basic problem is that a type-pirated method must be simultaneously correct for all copies of a stdlib.