Add SSE support to Core HTTP APIs

[pgayvallet]

Right now, if SSE between the Kibana server and browser can be technically achieved, implementing it is very tedious and "manual".

On the server-side, it was basically only made possible by the fact that we can send a stream as response's body. The conversion to any kind of "observable" or "event emitter" to an SSE-compatible ("event source") stream must be implemented by the endpoint owner.

E.g.

kibana/x-pack/plugins/inference/server/routes/chat_complete.ts

Lines 141 to 146 in 253cf6e

	async (context, request, response) => {
	const chatCompleteResponse = await callChatComplete({ request, stream: true });
	return response.ok({
	body: observableIntoEventSourceStream(chatCompleteResponse, logger),
	});
	}

Same goes for the browser side: the SSE stream can be handled due to the asResponse and rawResponse parameter of our fetch service, but there's no tool out of the box to convert the stream back to some kind of typed observable

E.g.

kibana/x-pack/plugins/inference/public/chat_complete.ts

Lines 43 to 49 in 1bd42ad

	return from(
	http.post('/internal/inference/chat_complete/stream', {
	asResponse: true,
	rawResponse: true,
	body: JSON.stringify(body),
	})
	).pipe(httpResponseIntoObservable());

Ideally, Core HTTP APIs should have better / native support for SSE endpoints, which would make DX around SSE better, in addition to allow the encode/decode logic to be factorized in a single, platform, place.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[jloleysens]

As we have also discussed offline, one risk here is when not using HTTP/2 we have a limited number of connections to also manage which a Core owned implementation could help manage:

When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6). The issue has been marked as "Won't fix" in Chrome and Firefox. This limit is per browser + domain, which means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com (per StackOverflow). When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100).

--- Excerpt from MDN

[rudolf]

I'm not 100% sure if we've stayed consistent in this. But we have traditionally tried to keep the API surface of Core as small as possible by avoiding leaking large API surfaces like an Observable (not holding my breath that it would ever get added to ECMAScript). Another way to achieve this could be to expose a stream that can then be very easily turned into an observable with Observable.from.

[afharo]

Looking at the code examples, it feels like a helper package to convert the response to SSE would suffice. WDYT?

[pgayvallet]

it feels like a helper package to convert the response to SSE would suffice. WDYT?

If we just want to factorize the tooling, then an helper package could be sufficient.

If we want to go further and create some kind of "SSE event bus" so that with http1, only one connection is kept open when multiple "endpoints" want to stream SSE (basically what bfetch was doing, and what @jloleysens is suggesting - I think - in his prior comment), then it's more work, and we'll for sure need some better integration with our APIs.

I'd also like to mention that "helper packages" have a very bad discoverability. We already have some sse-* utility packages, and yet every team needing sse are reimplementing it mostly because they're not aware that such utilities already exist. Exposing things from Core API always have a better discoverability, which is why I was thinking that it might make sense to do so.

[jloleysens]

then it's more work

I'm not too concerned about having more adoption (even though it could be a problem) bc I doubt users will use these features simultaneously. It seems more realistic that a single feature could be opened in many tabs. But I take your point, perhaps this is premature, I just don't want to be caught off guard by it.

The "right" solution will require a service worker... but I feel like even if we can have a simple limit mechanism in place it could help. For example: when we open an SSE connection we set a counter in localStorage. If we reach, e.g., 3, opening another connection will throw and calling code needs to handle that, the UX kinda sucks, but Kibana won't get (fully) choked when users have 8 tabs open which IMO is a worse UX.

[rudolf]

Now that http2 will be on by default and http1 is deprecated for 9.x #204384 I think the risk of exceeding the available sockets goes down a lot. Ideally we should still fail gracefully before allowing all available sockets to be consumed by SSE. But unless we know of a short term need to keep many SSE sockets open in the background, doing an SSE "bus" seems unnecessary.

[tsullivan]

Possibly related: #206352