KTOR-5199 Support WebSockets in Curl engine

[dtretyakov]

Subsystem
Client, Curl engine

Motivation
Curl 7.86.0 added experimental support for WebSockets.

Solution
This PR brings support of experimental WebSockets in libcurl KTOR-5199.

To verify WebSockets availability we're using curl_version_info which returns list of enabled protocols.

The CurlResponseBodyData become interface with CurlHttpResponseBody/CurlWebSocketResponseBody implementations.

The bodyStartedReceiving is used to detect the end of headers section.

We're using WebSocket with callbacks approach where:

• curl_ws_send used to send outgoing frames
• onBodyChunkReceived with curl_ws_meta used to receive incoming frames

Environment
Since WebSockets feature is experimental we need to enable them during curl compliation.

macOS

curl https://raw.githubusercontent.com/dtretyakov/homebrew-cloudflare/master/curl.rb -o curl.rb
brew install -s curl.rb 

linux

• Reference
• Commit
• Workflow file

[e5l]

Hey @dtretyakov, thanks for the PR.

Please let me know when you have an update about the Curl binary version. I will run tests on CI

[DRSchlaubi]

Trying to connect to wss://gateway.discord.gg/?v=10&encoding=json&compress=zlib-stream is returning 400 Bad request so there are some issues with this implementation`

After some further debugging the Sec-WebSocket-Key and Sec-WebSocket-Version headers are set twice, once by curl and once by ktor, removing them form ktor solves the 400 Bad request issue

New issue curl_multi_perform seems to block until the connection is done, making any additional requests impossible

[dtretyakov]

@DRSchlaubi thanks a lot for verification and review.

New issue curl_multi_perform seems to block until the connection is done, making any additional requests impossible

Maybe you could share a test case when it could be reproduced?

Trying to connect to wss://gateway.discord.gg/?v=10&encoding=json&compress=zlib-stream
Possible fix for duplicated header issue

Thanks, I added a list of WebSocket headers which are handled by cURL itself, so connection to this service should work.

[DRSchlaubi]

Maybe you could share a test case when it could be reproduced?

Sure

fun main() = runBlocking { 
    val client = HttpClient { 
        install(WebSockets)
    }
    
    launch { 
        client.webSocket("wss://echo.websocket.org") {
            while(!incoming.isClosedForReceive) {
              // Keep connection alive
                incoming.receive()
            }
        }
    }
    
    println("Wait for websocket to connect")
    delay(10.seconds)
    println("Requesting now!")
    
    println(client.get("https://httpbin.org/200"))
}

[DRSchlaubi]

After the latest changes most headers are not set, including "Authorization", which is not great

Fix in review comment

[DRSchlaubi]

After more debugging it seems like the issue is this loop

    @OptIn(ExperimentalForeignApi::class)
    internal fun perform() {
        if (activeHandles.isEmpty()) return

        memScoped {
            val transfersRunning = alloc<IntVar>()
            do {
                println("Transfers running:${transfersRunning.value}")
                synchronized(easyHandlesToUnpauseLock) {
                    var handle = easyHandlesToUnpause.removeFirstOrNull()
                    while (handle != null) {
                        curl_easy_pause(handle, CURLPAUSE_CONT)
                        handle = easyHandlesToUnpause.removeFirstOrNull()
                    }
                }
                println("Running curl_multi_perform")
                curl_multi_perform(multiHandle, transfersRunning.ptr).verify()
                println("Ran curl_multi_perform")
                if (transfersRunning.value != 0) {
                    curl_multi_poll(multiHandle, null, 0.toUInt(), 10000, null).verify()
                    println("Ran curl_multi_poll")
                }
                if (transfersRunning.value < activeHandles.size) {
                    handleCompleted()
                    println("Ran complete")
                }
                println("Loop done")
            } while (transfersRunning.value != 0)
        }
    }

Because while (transfersRunning.value != 0) will remain true as long as the ws connection is still active

[dtretyakov]

After more debugging it seems like the issue is this loop
Because while (transfersRunning.value != 0) will remain true as long as the ws connection is still active

@DRSchlaubi hopefully it was also addressed in the last commit, so you could try updating.

[DRSchlaubi]

At first glance your new test case should cover this, will test myself tmr 👍

[DRSchlaubi]

That indeed fixed it

[dtretyakov]

Please let me know when you have an update about the Curl binary version. I will run tests on CI

Currently we're waiting when static linking of libcurl with it's dependencies will be in the repository for all targets and after that could merge the changes.

• Prototype: dtretyakov@3c4dcd6
• Build scripts which could be used to create static libraries for linux/macOS: https://github.com/stunnel/static-curl

[DRSchlaubi]

Any updates

[git-bruh]

Hey @dtretyakov , thanks a lot for your work on this PR. I noticed some strange behaviour when receiving payloads greater than 1024 bytes and it seems to be a libcurl quirk, have raised a PR here: dtretyakov#1

[e5l]

Hey, @dtretyakov! Sorry for the long delay.
Could you rebase it on 3.1.0-eap branch before merging?

[DRSchlaubi]

I actually have a branch which is based on 3.0.0 for kord, I could push that as it might be easier to migrate to 3.1.0 as 2.x

[DRSchlaubi]

I pushed my work to curl/ws

[dtretyakov]

Could you rebase it on 3.1.0-eap branch before merging?

It was rebased, but now we need to have a libcurl with websockets enabled:
#4445 (comment)

[dtretyakov]

It looks that Conan libcurl receipt does no support option to enable WebSockets: conan-io/conan-center-index#26221