Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import scodec-stream and scodec-protocols in to fs2 #2588

Merged
merged 36 commits into from
Oct 23, 2021

Conversation

mpilquist
Copy link
Member

@mpilquist mpilquist commented Sep 3, 2021

This PR imports the entirety of scodec-stream and scodec-protocols in to FS2. The motivation for this PR are (1) reducing maintenance burden and (2) improving discoverability of various generic combinators.

More specifically, this PR:

  • Adds the fs2.Scan type, which provides a new primitive for building pure, stateful transformations of stream elements. A Scan can be converted to a pipe and applied to a stream. However, it supports various forms of composition that streams/pipe cannot express, like step-wise input/output transformations (i.e., feeding a single input to a scan and then transforming the values received in response).
  • Adds the fs2.timeseries package to fs2-core module. This provides the TimeStamped and TimeSeries objects.
  • Adds the fs2-scodec module containing fs2.interop.scodec package with types StreamDecoder and StreamEncoder. This is a port of scodec-stream.
  • Adds the fs2-protocols module, primarily as an example of how to use fs2-scodec.

@mpilquist mpilquist marked this pull request as ready for review September 6, 2021 12:12
build.sbt Outdated Show resolved Hide resolved
core/shared/src/main/scala/fs2/Scan.scala Show resolved Hide resolved
core/shared/src/main/scala/fs2/Scan.scala Outdated Show resolved Hide resolved
core/shared/src/main/scala/fs2/Scan.scala Outdated Show resolved Hide resolved
core/shared/src/main/scala/fs2/timeseries/TimeSeries.scala Outdated Show resolved Hide resolved
core/shared/src/main/scala/fs2/timeseries/TimeSeries.scala Outdated Show resolved Hide resolved
Copy link
Collaborator

@SystemFw SystemFw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loving the TimeSeries stuff in its current form, sooo much easier to understand, even though it's just naming changes

@mpilquist mpilquist changed the title Initial import of time series from scodec-protocols Import scodec-stream and scodec-protocols in to fs2 Oct 23, 2021
Comment on lines +44 to +48
final class Scan[S, -I, +O](
val initial: S,
private val transform_ : AndThen[(S, I), (S, Chunk[O])],
private val onComplete_ : AndThen[S, Chunk[O]]
) {
Copy link
Contributor

@diesalbla diesalbla Oct 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be more flexible to replace this class definition with a Scan trait, that declared the few main methods initial, transform and onComplete, and included any definitions that can be derived from those?

trait Scan[S, -I, +O](
    def initial: S
    
    def transform(s: S, i: I): (S, Chunk[O])
    
    def onComplete(s: S): Chunk[O]
    
    /** Chunk form of [[transform]]. */
  def transformAccumulate(s: S, c: Chunk[I]): (S, Chunk[O]) =
    c.foldLeft(s -> Chunk.empty[O]) { case ((s, acc), i) =>
      val (s2, os) = transform(s, i)
      (s2, acc ++ os)
    }
    
      /** Converts this scan to a pipe. */
  def toPipe[F[_]]: Stream[F, I] => Stream[F, O] =
    _.pull
      .scanChunks(initial)(transformAccumulate)
      .flatMap(state => Pull.output(onComplete(state)))
      .stream
  }

Even some of the combinators, such as map and contramap, could be defined on this trait:

  /** Returns a new scan which transforms output values using the supplied function. */
  def map[O2](f: O => O2): Scan[S, I, O2] = new Scan[S, I, O2] {
      def initial = self.initial
      def transform(s: S, i: I): (S, Chunk[O2]) = {
         val (s1, os) = self.transform(s, i)
         s1 -> os.map(f)
      }
      def onComplete(s: S): Chunk[O] = 
        self.onComplete(s).map(f)
  }

This design would be similar to that of other libraries, such as the entity encoders and decoders of Http4S.
The form that passes the transform_ and onComplete_ would then just be a specific implementation. It may be, of course, that other basic primitives for the Scan can be found.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s basically what we used to have: https://github.com/scodec/scodec-protocols/blob/main/src/main/scala/scodec/protocols/Transform.scala#L38

I switched to the new version as part of adding stack safety for various operations.

@mpilquist mpilquist merged commit 18e0eb9 into typelevel:main Oct 23, 2021
@mpilquist mpilquist deleted the topic/timeseries branch February 18, 2024 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants