Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract over Stream type in the outbound path #1056

Merged
merged 1 commit into from
Jan 9, 2019
Merged

Conversation

vkostyukov
Copy link
Collaborator

@vkostyukov vkostyukov commented Dec 21, 2018

This to address #1038 as well as make it more pluggable to adopt new streaming libraries (eg FS2). The type-class name is inspired by what Sergey is doing in #1042. The new machinery allows us to reuse the same instances between ToResponse (when serving streaming content) and Input.withStream allowing users to created chunked requests for testing.

Here is how it looks and feels:

import io.finch._, io.finch.iteratee._, io.interatee.Enumerator, cats.effect.IO

val i = Input
  .post("/")
  .withStream[Text.Plain](Enumerator.enumerate[IO, String]("foo", "bar", "baz"))

Also possible to encode JSON (as you would imagine). This does the right thing and encodes ND (newline-delimited) JSON stream, reusing the same code that we use on the outbound path:

import io.finch._, io.finch.iteratee._, io.interatee.Enumerator, cats.effect.IO
import io.finch.circe._, io.circe.generic.auto._

case class Foo(s: String)

val i = Input
  .post("/")
  .withBody[Application.Json](Enumerator.enumerate[IO, Foo](Foo("foo"), Foo("bar"), Foo("baz")))

As a bonus, this addresses #1040.

/**
*
*/
trait EncodeStreamToReader[S[_[_], _], F[_], A] {
Copy link
Collaborator Author

@vkostyukov vkostyukov Dec 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is essentially a slam of @sergeykolbasov 's DecodeStream and FromReader as we don't really need both in the outbound path (it's way simpler).


trait IterateeInstances extends LowPriorityIterateeInstances {

implicit def encodeJsonEnumeratorToReader[F[_]: Effect, A](implicit
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I was also thinking is that streaming libraries doesn't all have to do that. We can somewhat define this "interleave new lines" logic for JSON generally in finch-core (we have access to Encode.Json there). I'll keep this in mind and maybe revisit later.

@codecov-io
Copy link

codecov-io commented Dec 22, 2018

Codecov Report

❗ No coverage uploaded for pull request base (master@66f15e2). Click here to learn what that means.
The diff coverage is 93.75%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #1056   +/-   ##
=========================================
  Coverage          ?   84.53%           
=========================================
  Files             ?       50           
  Lines             ?      957           
  Branches          ?       61           
=========================================
  Hits              ?      809           
  Misses            ?      148           
  Partials          ?        0
Impacted Files Coverage Δ
core/src/main/scala/io/finch/Input.scala 100% <100%> (ø)
core/src/main/scala/io/finch/ToResponse.scala 97.14% <100%> (ø)
...tee/src/main/scala/io/finch/iteratee/package.scala 88.88% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66f15e2...2e841ff. Read the comment docs.

@vkostyukov vkostyukov changed the title Abstract over Stream type in the outbound path [WIP] Abstract over Stream type in the outbound path Dec 22, 2018
@vkostyukov vkostyukov force-pushed the vk/chunked-input branch 2 times, most recently from ab0c26b to fb0bfa0 Compare December 23, 2018 16:56
@vkostyukov
Copy link
Collaborator Author

@sergeykolbasov Do you prefer this or #1042 being merged this?

@vkostyukov vkostyukov force-pushed the vk/chunked-input branch 2 times, most recently from c0296f0 to 5ca183b Compare January 3, 2019 06:46
Copy link
Collaborator

@sergeykolbasov sergeykolbasov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it.
Do you think it's okay if I merge this one into #1042 so we can have one PR about streaming?

@@ -85,6 +85,17 @@ trait HighPriorityToResponseInstances extends LowPriorityToResponseInstances {

rep
}

implicit def streamToResponse[S[_[_], _], F[_], A, CT <: String](implicit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@vkostyukov
Copy link
Collaborator Author

@sergeykolbasov I'm thinking we should probably go the opposite way - merge smaller pieces when possible. I was worried you'd need to resolve a lot of conflicts in your PR hence kept it open. If you're up for a challenge (and it sounds like you're), I'd say we merge this first and rebase your work.

I wouldn't even mind if we split your PR into couple of independent changes if possible.

@vkostyukov vkostyukov mentioned this pull request Jan 3, 2019
2 tasks
@vkostyukov
Copy link
Collaborator Author

Given #1042 is blocked on a new circe-fs2 release (am I right, @sergeykolbasov ?), should we merge this first and, perhaps, cut a release?

@travisbrown
Copy link
Collaborator

@BenFradet Now that circe/circe-fs2#47 and circe/circe-fs2#50 are merged do you want to run a circe-fs2 release? I'm also happy to do it later this evening.

@BenFradet
Copy link
Contributor

I don't think I can do so myself but I'm all for it 👍

@travisbrown
Copy link
Collaborator

Okay, circe-fs2 is ready: https://github.com/circe/circe-fs2/releases/tag/v0.11.0

@vkostyukov
Copy link
Collaborator Author

Thank you, Travis!

@sergeykolbasov
Copy link
Collaborator

@vkostyukov FYI: just merged #1042

@vkostyukov
Copy link
Collaborator Author

Awesome @sergeykolbasov! I will rebase this tonight.

@vkostyukov
Copy link
Collaborator Author

Rebased. Some pieces feel repetitive but it's workable. I think we should merge this in and keep iterating.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants