Stop generation button (closes #86)

[Grsmto]

Need to see if we want to have it for the release.
It's using @gary149 design instead of the one in the issue!

closes #86

[coyotte508]

Code LGTM, but I think we should also not save the message in the backend in this case? Or should we?

[Grsmto]

Code LGTM, but I think we should also not save the message in the backend in this case? Or should we?

Yeah so that's why I mentioned the AbortController, I suppose that way we could avoid saving/save only the portion the user saw. I think it's OK for now tho because the main point of this feature is to allow users to quickly send a new request without having to wait until the end.

I suppose ideally also we would save the aborted message + not count it for further generations. But I think we would need to mark it as aborted or something. I'll create an issue to discuss this maybe.

[gary149]

Miss alignement on mobile:

[coyotte508]

ok then, ok for me to merge and create new issue (after mobile UI is fixed)

[julien-c]

no strong opinion, but imo let's only launch after launch (can be later today...)

[coyotte508]

yea let's make it 100%, stopping on frontend = stopping on backend

[Grsmto]

Thinking more about this and the backend part, the only way to "catch" the abort afaik is to send an extra API call from the client. However even with that, since the server is stateless I don't see how you could cancel the on-going request. Any idea @coyotte508?

[coyotte508]

Thinking more about this and the backend part, the only way to "catch" the abort afaik is to send an extra API call from the client.

Ok I looked more into it and at first glance this is right: https://www.reddit.com/r/SvelteKit/comments/yjdx13/detect_aborted_request_on_the_server/

But then I looked at websocket implementations:

• https://joyofcode.xyz/using-websockets-with-sveltekit
• https://github.com/suhaildawood/SvelteKit-integrated-WebSocket

And so using a similar approach - having our own custom server file that reuses' the adapter's handler function - it should be possible for us to monitor request.on('close') for each request and put that info in a global WeakMap for example.

Though this is advanced stuff, not sure we want to dip our toes yet into the internals

However even with that, since the server is stateless I don't see how you could cancel the on-going request. Any idea @coyotte508?

We could... add an "abortedRequests" function in a mongodb collection, listen to it across all instances with a changestream (or just a periodic db.abortedReqs.find().toArray() to be simpler to update a global list of aborted reqs)

We could base the abort on the convo id

new commit

[coyotte508]

Should remove this, it reappeared in the last commit but not valid anymore