Very bad performance on ARM, nginx, Julia 1.10

[fonsp]

I switched the featured notebook server to Julia 1.10 and disaster! All HTTP requests took >30 secs to complete.

[fonsp]

Might be caused by #146 ? total guess

[fonsp]

Also related is that I recently added more uptime checks, and the server is reporting lots of errors like this:

Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: ┌ Error: handle_connection handler error.
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ =========================== HTTP Error message:
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ERROR: IOError: write: broken pipe (EPIPE) Stacktrace: [1]
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ uvwrite(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) @ Base
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ./stream.jl:1066 [2] unsafewrite(s::Sockets.TCPSocket, p::Ptr{UInt8},
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ n::UInt64) @ Base ./stream.jl:1120 [3] unsafewrite @
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ~/.julia/packages/HTTP/sJD5V/src/Connections.jl:132 [inlined] [4]
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ unsafewrite(http::HTTP.Streams.Stream{HTTP.Messages.Request,
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64)
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ @ HTTP.Streams ~/.julia/packages/HTTP/sJD5V/src/Streams.jl:93 [5]
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ unsafewrite @ ./io.jl:698 [inlined] [6]
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ write(s::HTTP.Streams.Stream{HTTP.Messages.Request,
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ HTTP.Connections.Connection{Sockets.TCPSocket}}, a::Vector{UInt8}) @ Base
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ./io.jl:721 7
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ (::HTTP.Handlers.var"#1#2"{PlutoSliderServer.var"../HTTPRouter.jl".var"#29#30"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404),
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ typeof(HTTP.Handlers.default405),
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ Nothing}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request,
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ HTTP.Connections.Connection{Sockets.TCPSocket}}) @ HTTP.Handlers
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ~/.julia/packages/HTTP/sJD5V/src/Handlers.jl:61 [8] #invokelatest#2 @
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ./essentials.jl:892 [inlined] [9] invokelatest @ ./essentials.jl:889
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ [inlined] [10] handleconnection(f::Function,
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ c::HTTP.Connections.Connection{Sockets.TCPSocket},
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer},
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ readtimeout::Int64, access_log::Nothing) @ HTTP.Servers
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ ~/.julia/packages/HTTP/sJD5V/src/Servers.jl:469 11
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ (::HTTP.Servers.var"#16#17"{HTTP.Handlers.var"#1#2"{PlutoSliderServer.var"../HTTPRouter.jl".var"#29#30"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404),
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ typeof(HTTP.Handlers.default405), Nothing}}},
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ HTTP.Servers.Listener{Nothing, Sockets.TCPServer},
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock,
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() @
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │ HTTP.Servers ~/.julia/packages/HTTP/sJD5V/src/Servers.jl:401
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │   request =
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    HTTP.Messages.Request:
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    """
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    GET / HTTP/1.1
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    Host: fonsi.armada.silentech.gr
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    X-Real-IP: 143.110.192.36
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    X-Forwarded-For: 143.110.192.36
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    X-Forwarded-Proto: https
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    Connection: upgrade
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    User-Agent: DigitalOcean Uptime Probe 0.22.0 (https://digitalocean.com)
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: │    """
Oct 11 14:43:21 georgettarm pluto-slider-server.sh[2965931]: └ @ HTTP.Servers /home/fons/.julia/packages/HTTP/sJD5V/src/Servers.jl:483

After switching to Julia 1.6, the errors are gone.

[fonsp]

We tried to run Julia with threads: julia threads=2,2 but this fixed the problem for one person visiting the site, but for another it is still broken.

[fonsp]

I tried to switch everything to @spawn and julia threads=8,8 but still broken

[fonsp]

Hmmm I noticed the following:

• If I do wget http://fonsi.armada.silentech.gr:8555/ it always responds instantly
• If I forward the port using VS Code, then http://localhost:8555/ works on safari and chrome
• But visiting http://fonsi.armada.silentech.gr:8555 in a browser does not work
• On firefox I could visit http://fonsi.armada.silentech.gr:8555 a couple times, but http://fonsi.armada.silentech.gr:8555/pluto_export.json in a new tab does not work

I changed to Julia 1.6 with an old manifest and it's still broken!

[fonsp]

Right now I have:

• http://fonsi.armada.silentech.gr:8555/ is not loading in any browser
• http://142.132.155.215:8555/ works fine on all browsers