Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] Adds a "dynamic" module to dynamically load services from smithy models #17

Merged
merged 41 commits into from
Feb 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
e20d7e4
Adds a "dynamic" module to dynamically load services from smithy modules
Baccata Dec 27, 2021
217be26
Towards cross-compilable dynamic
Baccata Dec 28, 2021
98cfbfe
Merge remote-tracking branch 'origin/main' into dynamic-schemas
Baccata Dec 30, 2021
5db12e3
Fix model schema (multiple lists), remove old dependency and unused code
kubukoz Jan 4, 2022
ea4d5ff
Merge branch 'main' into dynamic-schemas
kubukoz Jan 4, 2022
a4a1cc8
Update after merge
kubukoz Jan 4, 2022
41e5d1f
Add I/O hints
kubukoz Jan 4, 2022
7c89069
Merge branch 'dynamic-schemas' of github.com:disneystreaming/smithy4s…
Baccata Jan 5, 2022
a581687
Merge remote-tracking branch 'origin/main' into dynamic-schemas
Baccata Jan 5, 2022
a83ae61
Update dynamic with new ShapeId model
Baccata Jan 5, 2022
676fa7a
do not render operation names
Baccata Jan 5, 2022
f82a7b1
Merge remote-tracking branch 'origin/main' into dynamic-schemas
Baccata Jan 11, 2022
1d08fb2
Wire discriminated trait in dynamic model
Baccata Jan 11, 2022
bd36e90
Merge branch 'main' into dynamic-schemas
kubukoz Jan 14, 2022
eeef6b8
Add failing test for extracting something from the service
kubukoz Jan 15, 2022
99357f0
Add dynamic to allModules
kubukoz Jan 16, 2022
78e9f50
Visit prelude shapes - quick and dirty
kubukoz Jan 16, 2022
d24710c
Support Float
kubukoz Jan 16, 2022
fd35be6
Add new failing spec
kubukoz Jan 16, 2022
32bfcae
Fix loading of specs in tests
Baccata Jan 16, 2022
eb05278
Implement rest of metadamodel in dynamic compiler
Baccata Jan 16, 2022
40216d3
Prepare functions for testing purposes
Baccata Jan 16, 2022
c473e46
Merge branch 'main' into dynamic-schemas
kubukoz Jan 17, 2022
64ba24d
Merge branch 'main' into dynamic-schemas
kubukoz Jan 19, 2022
99ed96d
Various fixes to the dynamic compiler (#55)
Jan 20, 2022
ba66a54
wip
Baccata Jan 21, 2022
374ecbb
Merge branch 'main' into dynamic-schemas
kubukoz Jan 21, 2022
faa231a
Move dynamic tests to shared sources
kubukoz Jan 21, 2022
62fe0a2
Merge branch 'dynamic-schemas' of github.com:disneystreaming/smithy4s…
Baccata Jan 22, 2022
f80372a
More tests
Baccata Jan 22, 2022
e44dfaa
2.12
Baccata Jan 22, 2022
0a2c361
Merge branch 'main' into dynamic-schemas
kubukoz Jan 26, 2022
56cf259
Post-rebase fixes
kubukoz Jan 26, 2022
f02e914
Merge remote-tracking branch 'origin/main' into dynamic-schemas
Baccata Jan 28, 2022
a68a4a2
Update dynamic with SchemaIndex construct
Baccata Jan 28, 2022
2e6d756
Merge remote-tracking branch 'origin/main' into dynamic-schemas
Baccata Feb 9, 2022
705b5f3
Cleanup
Baccata Feb 9, 2022
97b7d03
Headers
Baccata Feb 9, 2022
6c23f9b
Fmt
Baccata Feb 9, 2022
4d9dde3
Fix 2.12
Baccata Feb 9, 2022
0a5860a
Fix 2.12 AND 2.13
Baccata Feb 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ lazy val allModules = Seq(
`aws-kernel`.projectRefs,
aws.projectRefs,
`aws-http4s`.projectRefs,
`codegen-cli`.projectRefs
`codegen-cli`.projectRefs,
dynamic.projectRefs
).flatten

lazy val docs =
Expand Down Expand Up @@ -409,6 +410,39 @@ lazy val protocol = projectMatrix
)
)

/**
* This modules contains utilities to dynamically instantiate
* the interfaces provide by smithy4s, based on data from dynamic
* Model instances.
*/
lazy val dynamic = projectMatrix
.in(file("modules/dynamic"))
.dependsOn(core)
.settings(
isCE3 := true,
libraryDependencies ++= Seq(
"org.scala-lang.modules" %%% "scala-collection-compat" % "2.6.0",
Dependencies.Cats.core.value,
Dependencies.Weaver.cats.value % Test
),
Test / fork := true,
// Forwarding cwd to tests
Test / javaOptions += s"-Duser.dir=${sys.props("user.dir")}",
testFrameworks += new TestFramework("weaver.framework.CatsEffect"),
Compile / allowedNamespaces := Seq("smithy4s.dynamic.model"),
Compile / smithySpecs := Seq(
(ThisBuild / baseDirectory).value / "modules" / "dynamic" / "smithy" / "dynamic.smithy"
),
(Compile / sourceGenerators) := Seq(genSmithyScala(Compile).taskValue)
)
.jvmPlatform(
allJvmScalaVersions,
jvmDimSettings ++ Seq(
libraryDependencies += Dependencies.Smithy.model % Test
)
)
.jsPlatform(allJsScalaVersions, jsDimSettings)

/**
* Module that contains the logic for generating "openapi views" of the
* services that abide by some custom protocols provided by this library.
Expand Down
24 changes: 18 additions & 6 deletions modules/codegen/src/smithy4s/codegen/CollisionAvoidance.scala
Original file line number Diff line number Diff line change
Expand Up @@ -59,27 +59,39 @@ object CollisionAvoidance {
hints.map(modHint),
version
)
case Product(name, fields, recursive, hints) =>
case Product(name, originalName, fields, recursive, hints) =>
Product(
protect(name.capitalize),
originalName,
fields.map(modField),
recursive,
hints.map(modHint)
)
case Union(name, alts, recursive, hints) =>
case Union(name, originalName, alts, recursive, hints) =>
Union(
protect(name.capitalize),
originalName,
alts.map(modAlt),
recursive,
hints.map(modHint)
)
case TypeAlias(name, tpe, hints) =>
TypeAlias(protect(name.capitalize), modType(tpe), hints.map(modHint))
case Enumeration(name, values, hints) =>
case TypeAlias(name, originalName, tpe, hints) =>
TypeAlias(
protect(name.capitalize),
originalName,
modType(tpe),
hints.map(modHint)
)
case Enumeration(name, originalName, values, hints) =>
val newValues = values.map { case EnumValue(value, name, hints) =>
EnumValue(value, name.map(protect), hints.map(modHint))
}
Enumeration(protect(name.capitalize), newValues, hints.map(modHint))
Enumeration(
protect(name.capitalize),
originalName,
newValues,
hints.map(modHint)
)
}
CompilationUnit(compilationUnit.namespace, declarations)
}
Expand Down
11 changes: 9 additions & 2 deletions modules/codegen/src/smithy4s/codegen/IR.scala
Original file line number Diff line number Diff line change
Expand Up @@ -54,23 +54,30 @@ case class Operation(

case class Product(
name: String,
originalName: String,
fields: List[Field],
recursive: Boolean = false,
hints: List[Hint] = Nil
) extends Decl

case class Union(
name: String,
originalName: String,
alts: NonEmptyList[Alt],
recursive: Boolean = false,
hints: List[Hint] = Nil
) extends Decl

case class TypeAlias(name: String, tpe: Type, hints: List[Hint] = Nil)
extends Decl
case class TypeAlias(
name: String,
originalName: String,
tpe: Type,
hints: List[Hint] = Nil
) extends Decl

case class Enumeration(
name: String,
originalName: String,
values: List[EnumValue],
hints: List[Hint] = Nil
) extends Decl
Expand Down
37 changes: 22 additions & 15 deletions modules/codegen/src/smithy4s/codegen/Renderer.scala
Original file line number Diff line number Diff line change
Expand Up @@ -81,18 +81,20 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>
def renderDecl(decl: Decl): RenderResult = decl match {
case Service(name, originalName, ops, hints, version) =>
renderService(name, originalName, ops, hints, version)
case Product(name, fields, recursive, hints) =>
renderProduct(name, fields, recursive, hints)
case Union(name, alts, recursive, hints) =>
renderUnion(name, alts, recursive, hints)
case TypeAlias(name, tpe, hints) => renderTypeAlias(name, tpe, hints)
case Enumeration(name, values, hints) => renderEnum(name, values, hints)
case _ => RenderResult.empty
case Product(name, originalName, fields, recursive, hints) =>
renderProduct(name, originalName, fields, recursive, hints)
case Union(name, originalName, alts, recursive, hints) =>
renderUnion(name, originalName, alts, recursive, hints)
case TypeAlias(name, originalName, tpe, hints) =>
renderTypeAlias(name, originalName, tpe, hints)
case Enumeration(name, originalName, values, hints) =>
renderEnum(name, originalName, values, hints)
case _ => RenderResult.empty
}

def renderPackageContents: RenderResult = {
val typeAliases = compilationUnit.declarations.collect {
case TypeAlias(name, _, _) =>
case TypeAlias(name, _, _, _) =>
s"type $name = ${compilationUnit.namespace}.${name}.Type"
}

Expand Down Expand Up @@ -267,11 +269,12 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>
alts <- errorNel.traverse { t =>
t.name.map(n => Alt(n, t))
}
} yield Union(opName + "Error", alts)
name = opName + "Error"
} yield Union(name, name, alts)

val renderedErrorUnion = errorUnion.foldMap {
case Union(name, alts, recursive, hints) =>
renderUnion(name, alts, recursive, hints, error = true)
case Union(name, originalName, alts, recursive, hints) =>
renderUnion(name, originalName, alts, recursive, hints, error = true)
}

lines(
Expand Down Expand Up @@ -326,6 +329,7 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>

private def renderProduct(
name: String,
originalName: String,
fields: List[Field],
recursive: Boolean,
hints: List[Hint]
Expand Down Expand Up @@ -356,7 +360,7 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>
}
} else line(decl),
obj(name, ext = shapeTag(name))(
renderId(name),
renderId(originalName),
newline,
renderHintsValWithId(hints),
renderProtocol(name, hints),
Expand Down Expand Up @@ -415,6 +419,7 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>

private def renderUnion(
name: String,
originalName: String,
alts: NonEmptyList[Alt],
recursive: Boolean,
hints: List[Hint],
Expand All @@ -428,7 +433,7 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>
lines(
s"sealed trait $name extends scala.Product with scala.Serializable",
obj(name, ext = shapeTag(name))(
renderId(name),
renderId(originalName),
newline,
renderHintsValWithId(hints),
newline,
Expand Down Expand Up @@ -482,12 +487,13 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>

private def renderEnum(
name: String,
originalName: String,
values: List[EnumValue],
hints: List[Hint]
): RenderResult = lines(
s"sealed abstract class $name(val value: String, val ordinal: Int) extends scala.Product with scala.Serializable",
obj(name, ext = s"$Enumeration_[$name]", w = shapeTag(name))(
renderId(name),
renderId(originalName),
newline,
renderHintsValWithId(hints),
newline,
Expand All @@ -511,14 +517,15 @@ private[codegen] class Renderer(compilationUnit: CompilationUnit) { self =>

private def renderTypeAlias(
name: String,
originalName: String,
tpe: Type,
hints: List[Hint]
): RenderResult = {
val imports = tpe.imports ++ Set("smithy4s.Newtype") ++ syntaxImport

lines(
obj(name, extensions = List(s"Newtype[${tpe.render}]"))(
renderId(name),
renderId(originalName),
renderHintsValWithId(hints),
s"val underlyingSchema : $Schema_[${tpe.render}] = ${tpe.schemaRef}.withHints(hints)",
lines(
Expand Down
13 changes: 7 additions & 6 deletions modules/codegen/src/smithy4s/codegen/SmithyToIR.scala
Original file line number Diff line number Diff line change
Expand Up @@ -70,25 +70,26 @@ private[codegen] class SmithyToIR(model: Model, namespace: String) {
if (shape.isMemberShape()) None
else
shape.tpe.flatMap {
case Type.Alias(_, name, tpe) => TypeAlias(name, tpe, hints).some
case Type.PrimitiveType(_) => None
case other => TypeAlias(shape.name, other, hints).some
case Type.Alias(_, name, tpe) =>
TypeAlias(name, name, tpe, hints).some
case Type.PrimitiveType(_) => None
case other => TypeAlias(shape.name, shape.name, other, hints).some
}
}

override def structureShape(shape: StructureShape): Option[Decl] = {
val rec = isRecursive(shape.getId(), Set.empty)

val hints = traitsToHints(shape.getAllTraits().asScala.values.toList)
Product(shape.name, shape.fields, rec, hints).some
Product(shape.name, shape.name, shape.fields, rec, hints).some
}

override def unionShape(shape: UnionShape): Option[Decl] = {
val rec = isRecursive(shape.getId(), Set.empty)

val hints = traitsToHints(shape.getAllTraits().asScala.values.toList)
NonEmptyList.fromList(shape.alts).map { case alts =>
Union(shape.name, alts, rec, hints)
Union(shape.name, shape.name, alts, rec, hints)
}
}

Expand All @@ -101,7 +102,7 @@ private[codegen] class SmithyToIR(model: Model, namespace: String) {
EnumValue(value.getValue(), value.getName().asScala)
}
.toList
Enumeration(shape.name, values).some
Enumeration(shape.name, shape.name, values).some
case _ => super.stringShape(shape)
}

Expand Down
21 changes: 21 additions & 0 deletions modules/core/src/smithy4s/Document.scala
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package smithy4s
import smithy4s.http.PayloadError
import smithy4s.internals.SchematicDocumentDecoder
import smithy4s.internals.SchematicDocumentEncoder
import smithy4s.Document._

/**
* A json-like free-form structure serving as a model for
Expand All @@ -31,6 +32,26 @@ sealed trait Document extends Product with Serializable {
): Either[PayloadError, A] =
decoder.decode(this)

override def toString(): String = this.show

/**
* Toy renderer that does not comply the json specification :
* strings aren't escaped and keys aren't quoted.
* Do not use for any real purpose other than debugging.
*/
def show: String = this match {
case DNumber(value) => {
if (value.isValidLong) value.toLong.toString()
else value.toString()
}
case DBoolean(value) => value.toString
case DString(value) => s""""$value""""
case DNull => "null"
case DArray(value) => value.map(_.show).mkString("[", ", ", "]")
case DObject(value) =>
value.map { case (k, v) => k + "=" + v.show }.mkString("{", ", ", "}")
}

}

object Document {
Expand Down
2 changes: 2 additions & 0 deletions modules/core/src/smithy4s/SchemaIndex.scala
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ sealed trait SchemaIndex {

object SchemaIndex {

val empty: SchemaIndex = new Impl(Map.empty)

def apply[S](bindings: Binding[_]*): SchemaIndex = {
new Impl(bindings.map(b => b.tuple: (ShapeTag[_], Schema[_])).toMap)
}
Expand Down
3 changes: 3 additions & 0 deletions modules/core/src/smithy4s/ShapeId.scala
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,12 @@ package smithy4s

case class ShapeId(namespace: String, name: String) {
def show = s"$namespace#$name"
def withMember(member: String): ShapeId.Member = ShapeId.Member(this, member)
override def toString = show
}

object ShapeId extends ShapeTag.Companion[ShapeId] {
def id: ShapeId = ShapeId("smithy4s", "ShapeId")

case class Member(shapeId: ShapeId, member: String)
}
2 changes: 2 additions & 0 deletions modules/core/src/smithy4s/http/HttpContractError.scala
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ case class PayloadError(
expected: String,
message: String
) extends HttpContractError {
override def toString(): String =
s"PayloadError($path, expected = $expected, message=$message)"
override def getMessage(): String = message
}

Expand Down
1 change: 1 addition & 0 deletions modules/core/src/smithy4s/http/internals/PathEncode.scala
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ object PathEncode {
def fromToString[A]: Make[A] = from(_.toString)

def noop[A]: Make[A] = Hinted.static[MaybePathEncode, A](None)

}

}
Loading