Python: Type-stub generation for SSDKs #2149

unexge · 2022-12-30T12:19:43Z

Motivation and Context

What is a Python type-stub?

Type-stubs allows us to define type information for Python modules. They usually used to define type information for untyped modules, and it is the only way to define type information for the native extensions which is the case for Smithy Python Server SDKs. That’s why we need to provide type-stubs alongside our SSDKs to allow our users to get a good IDE integration, static type checking and better documentation.

Type-stubs uses same syntax with regular Python code except they just use ... for function or class bodies, and they’ve .pyi extensions. An example type-stub:

# foo.pyi

class Foo:
    ...

class Bar:
   baz: int

def qux(foobar: str) -> Bar:
    ...

For more details about Python type-stubs: https://typing.readthedocs.io/en/latest/source/stubs.html

Implementation

There is some ongoing work on PyO3 side to handle this, but it is still in early stages. Since we have most of the information needed in our code generator we thought we can roll our custom solution for that until we have a solution on PyO3 side.

Implementation is mostly based on the idea shared by Tpt: PyO3/pyo3#2454 (comment), that is:

Annotate Rust types with Python type information using Sphinx docstring format (We don't accurately follow the spec though):
For code generated types: this PR introduces PythonType to represent and render Python types.
For runtime types: we manually add type information to the comments of Rust types (as you can see in PR diff for most of the Rust files)
Build Python module from Rust crate
Inspect Python module, parse doc strings of elements and generate type-stubs based on them. This is done in stubgen.py
Build Python module from Rust crate again but this time Rust crate also includes Python type-stubs

Testing

You can checkout this branch and run make build in rust-runtime/aws-smithy-http-server-python/examples, that command will generate and install a Python wheel for Pokémon Service containing Python type-stubs. You should then open rust-runtime/aws-smithy-http-server-python/examples/pokemon_service.py in your text editor, and you should be able to use go to definition, autocomplation etc. in your editor. Alternatively you can also see the generated type-stubs in rust-runtime/aws-smithy-http-server-python/examples/pokemon-service-server-sdk/python/pokemon_service_server_sdk.

TODOs - Questions

As you can see in the steps we need to do double build. Is there a way to prevent that? Is it okay to build it twice?
Use generics for Context class, currently we're using Any for Context (Done in 541df2d)
Handle default values, currently we are using ... in place of default values, but we shouldn't emit ... if there is no default value for that type (Done in 5e56f45)
Omit __init__ for classes that doesn't have constructors (Done in fb5bcd7)
How to handle getters and setters? Currently, we only annotate getters and in type-stubs we generate them as regular fields. Should we generate two methods with @property and @T.setter with correct type information?
How to integrate it with CI? Should we generate diffs for generated stubs similar to generated SDKs? Or should we just check out generated type-stubs to Git? Should we run mypy to verify we generate correct type-stubs? (Done in cde0276 and df4a079)
Should we preserve comments for types? Is this going to help documentation tools and IDEs? (Done in d05d3a0)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

unexge · 2022-12-30T12:21:01Z

codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/rustlang/RustWriter.kt

@@ -387,6 +387,7 @@ class RustWriter private constructor(
        fun factory(debugMode: Boolean): Factory<RustWriter> = Factory { fileName: String, namespace: String ->
            when {
                fileName.endsWith(".toml") -> RustWriter(fileName, namespace, "#", debugMode = debugMode)
+                fileName.endsWith(".py") -> RustWriter(fileName, namespace, "#", debugMode = debugMode)


Can we do this customization in Python namespace?

I don't think so (well, not easily).. This is a foundational module that is hard to move around or inherit from. We could have a PyRustWriter that adds this functionality, but the real RustWriter is so embedded into the whole codebase that it's going to be really hard to just use something that inherits from it everywhere.

To me it is reasonable to have it here, same goes for toml for example, but I would like to hear the SDK team thoughts..

github-actions · 2022-12-30T12:31:39Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

github-actions · 2023-01-03T16:15:29Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

github-actions · 2023-01-04T16:47:00Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

github-actions · 2023-01-05T13:18:33Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

crisidev · 2023-01-05T15:40:56Z

rust-runtime/aws-smithy-http-server-python/examples/stubgen.py

@@ -0,0 +1,391 @@
+#  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.


Is the example folder the right place for this file? We need to be able to vend it to customers somehow, right?

Yes we need to vend it to customers somehow, but I'm not sure how yet. I think we'll need a build system eventually but currently the closest thing we have is the Makefile in the examples folder, so that's why I've put this file here.

I think it is fine for this PR as we are just adding the functionality. We need to start thinking about how to vend this file. We should probably cut an issue about it.

crisidev · 2023-01-05T15:43:40Z

rust-runtime/aws-smithy-http-server-python/examples/stubgen.py

+def is_fn_like(obj: Any) -> bool:
+    return (
+        inspect.isbuiltin(obj)
+        or inspect.ismethod(obj)
+        or inspect.isfunction(obj)
+        or inspect.ismethoddescriptor(obj)
+    )


Don't you need also inspect.iscoroutinefunction ?

This is what inspect gives you:

'isabstract', 'isasyncgen', 'isasyncgenfunction', 'isawaitable', 'isbuiltin', 'isclass', 'iscode', 'iscoroutine', 'iscoroutinefunction', 'isdatadescriptor', 'isframe', 'isfunction', 'isgenerator', 'isgeneratorfunction', 'isgetsetdescriptor', 'ismemberdescriptor', 'ismethod', 'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback',

I am pretty sure you'll have to add iscoroutine and iscoroutinefunction. I am not sure about isawaitable, isgenerator, isgeneratorfunction, isroutine, isasyncgen and isasyncgenfunction.

This can help you understand what is what:

for x in dir(inspect): if x.startswith("is"): obj = getattr(inspect, x) print(f"============= {x} ============") print(f"{obj.__doc__}") print("==============================================\n")

I've added iscoroutine and iscoroutinefunction. Since async fns are not permitted by PyO3 and coroutines/generators are currently experimental we currently only expose regular functions and so the current check should be sufficient.

crisidev · 2023-01-05T15:46:47Z

rust-runtime/aws-smithy-http-server-python/examples/stubgen.py

+    parser = argparse.ArgumentParser()
+    parser.add_argument("module")
+    parser.add_argument("outdir")
+    args = parser.parse_args()


This code is quite complex and hard to review (it's a hard problem) :D I would like to see some tests in this file to ensure we are doing the right thing and we do not regress in the future.

I think you have several options:

Make a test suite using pytest with a standard python module

Add an option to the main, something like --run-tests, which will lazily import pytest and run something like pytest.run(), which will collect the test cases and run them

Delegate the testing to the github CI or to something written in kotlin in the code generator.

What do you think?

I don't think tests are a requirement for this PR, but we should work on them before releasing this feature as stable.

Yes you are definitely right. I've tried to keep this file easy to distribute, so I didn't create any submodules, or I didn't use any dependency other than standard library, but it certainly needs a refactor and a better testing story. I've made some improvements to our CI, it runs mypy to make sure we are generating valid type-stubs and the diff created by bot includes generated type-stubs to detect unintended changes, but I'll add more end-to-end tests.

github-actions · 2023-01-06T17:49:39Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

github-actions · 2023-01-09T11:31:07Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

crisidev · 2023-01-09T11:53:40Z

Looking at the diff, I noticed something weird. I think there is an off-by-one error somewhere since docstrings that should be above the attribute are actually below. Example:

This attribute taken one of the pyi

body: typing.Awaitable[bytes]
"""
Return the HTTP body of this request.
Note that this is a costly operation because the whole request body is cloned.
"""

should be

"""
Return the HTTP body of this request.
Note that this is a costly operation because the whole request body is cloned.
"""
body: typing.Awaitable[bytes]

Am I wrong?

The same applies to everything in the pyi files.

unexge · 2023-01-09T12:03:51Z

Yes it indeed seems a bit weird but as defined in PEP-257

String literals occurring immediately after a simple assignment at the top level of a module, class, or init method are called “attribute docstrings”.

That's the way to define docstring for items. See https://stackoverflow.com/a/6061254 for more detail

crisidev · 2023-01-09T13:59:18Z

Yes it indeed seems a bit weird but as defined in PEP-257

String literals occurring immediately after a simple assignment at the top level of a module, class, or init method are called “attribute docstrings”.

That's the way to define docstring for items. See https://stackoverflow.com/a/6061254 for more detail

Oh wow. TIL..

github-actions · 2023-01-09T14:38:12Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

crisidev · 2023-01-13T16:16:23Z

...ython/src/main/kotlin/software/amazon/smithy/rust/codegen/server/python/smithy/PythonType.kt

+        // TODO(Constraints): How to handle this?
+        // Revisit as part of https://github.com/awslabs/smithy-rs/issues/2114


I don't think you have to.. I was talking with @david-perez and right now Python only handles constraints during deserialization and hands you off the original structure, without newtypes if the validation succeeded or an error otherwise.

crisidev · 2023-01-13T16:18:21Z

rust-runtime/aws-smithy-http-server-python/examples/stubgen_test.py

+def create_module(name: str, code: str) -> ModuleType:
+    mod = ModuleType(name)
+    exec(dedent(code), mod.__dict__)
+    if not hasattr(mod, "__all__"):
+        # Manually populate `__all__` with all the members that doesn't start with `__`
+        mod.__all__ = [k for k in mod.__dict__.keys() if not k.startswith("__")]  # type: ignore
+    sys.modules[name] = mod
+    return mod


…xtended from `object`

…of a class

Python 3.7 doesn't support reading signatures from `__text_signature__` for non-builtin functions (i.e. C/Rust functions). For testing we're using regular Python syntax for defining signature.

github-actions · 2023-02-10T12:47:09Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

github-actions · 2023-02-10T13:10:15Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
Server Test Python (ignoring whitespace)

A new doc preview is ready to view.

crisidev · 2023-02-10T14:36:51Z

\0/ Nice!

* Initial Python stub generation * Handle default values correctly * Only generate `__init__` for classes that have constructor signatures * Preserve doc comments * Make context class generic * Put type hint into a string to fix runtime error * Run `mypy` on CI * Use `make` to build Python SSDKs while generating diffs * Escape Python types in Rust comments * Only mark class methods with * Sort imports to minimize diffs * Add type annotations for `PySocket` * Dont extend classes from `object` as every class already implicitly extended from `object` * Use `vars` instead of `inspect.getmembers` to skip inherited members of a class * Fix linting issues * Add some tests for stubgen and refactor it * Add type annotations to `PyMiddlewareException` * Fix tests on Python 3.7 Python 3.7 doesn't support reading signatures from `__text_signature__` for non-builtin functions (i.e. C/Rust functions). For testing we're using regular Python syntax for defining signature. * Provide default values for `typing.Optional[T]` types in type-stubs * Update `is_fn_like` to cover more cases * Remove `tools/smithy-rs-tool-common/` * Make `DECORATORS` an array instead of a list * Ignore missing type stub errors for `aiohttp`

@required

commit 7ce8032 Author: John DiSanti <jdisanti@amazon.com> Date: Fri Mar 3 09:12:45 2023 -0800 Make it possible to create `SdkError` context structs for tests (smithy-lang#2428) commit 818343e Author: Burak <unexge@gmail.com> Date: Thu Mar 2 17:43:42 2023 +0000 Make `tools/ci-scripts/codegen-diff-revisions.py` fault tolerant (smithy-lang#2426) commit 7ebfbcc Author: ysaito1001 <gperson22@gmail.com> Date: Thu Mar 2 11:25:34 2023 -0600 Move using-native-tls-instead-of-rustls to smithy-rs (smithy-lang#2423) * Move using-native-tls-instead-of-rustls to smithy-rs This commit adds the test `using-native-tls-instead-of-rustls` to `smithy-rs` that was originally in the `aws-doc-sdk-examples`. The test is more useful to be in `smithy-rs` because it can catch a test failure early prior to cutting a release. * Fix Copyright header * Update aws/sdk/integration-tests/using-native-tls-instead-of-rustls/tests/no-rustls-in-dependency.rs Co-authored-by: Zelda Hessler <zhessler@amazon.com> * Update aws/sdk/integration-tests/using-native-tls-instead-of-rustls/tests/no-rustls-in-dependency.rs Co-authored-by: Zelda Hessler <zhessler@amazon.com> * Update Cargo.toml This commit addresses smithy-lang#2423 (comment) smithy-lang#2423 (comment) --------- Co-authored-by: Yuki Saito <awsaito@amazon.com> Co-authored-by: Zelda Hessler <zhessler@amazon.com> commit 049287d Author: ysaito1001 <gperson22@gmail.com> Date: Tue Feb 28 14:35:08 2023 -0600 Fix Inaccurate documentation on ProfileFileCredentialsProvider (smithy-lang#2421) This commit addresses awslabs/aws-sdk-rust#746. Co-authored-by: Yuki Saito <awsaito@amazon.com> commit c3ae6f7 Author: Julian Antonielli <julianantonielli@gmail.com> Date: Tue Feb 28 20:26:20 2023 +0000 Refactor event stream tests with `{client,server}IntegrationTest`s (smithy-lang#2342) * Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use `clientIntegrationTest` (WIP) * Refactor `ClientEventStreamUnmarshallerGeneratorTest` with `clientIntegrationTest` * Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use generic test cases * Start refactoring `ServerEventStreamUnmarshallerGeneratorTest` * Make `ServerEventStreamUnmarshallerGeneratorTest` tests work * Uncomment other test models * Allow unused on `parse_generic_error` * Rename `ServerEventStreamUnmarshallerGeneratorTest` * Make `EventStreamUnmarshallTestCases` codegenTarget-agnostic * Refactor `ClientEventStreamMarshallerGeneratorTest`: Tests run but fail * Refactor `ServerEventStreamMarshallerGeneratorTest` * Move `.into()` calls to `conditionalBuilderInput` * Add "context" to TODO * Fix client unmarshall tests * Fix clippy lint * Fix more clippy lints * Add docs for `event_stream_serde` module * Fix client tests * Remove `#[allow(missing_docs)]` from event stream module * Remove unused `EventStreamTestTools` * Add `smithy-validation-model` test dep to `codegen-client` * Temporarily add docs to make tests compile * Undo change in model * Make event stream unmarshaller tests a unit test * Remove unused code * Make `ServerEventStreamUnmarshallerGeneratorTest` a unit test * Make `ServerEventStreamMarshallerGeneratorTest` a unit test * Make `ServerEventStreamMarshallerGeneratorTest` pass * Make remaining tests non-integration tests * Make event stream serde module private again * Remove unnecessary clippy allowances * Remove clippy allowance * Remove docs for `event_stream_serde` module * Remove docs for `$unmarshallerTypeName::new` * Remove more unnecessary docs * Remove more superfluous docs * Undo unnecessary diffs * Uncomment last test * Make `conditionalBuilderInput` internal commit 72df844 Author: Julian Antonielli <julianantonielli@gmail.com> Date: Tue Feb 28 07:47:21 2023 +0000 Correct outdated details (smithy-lang#2420) commit 9f7c49a Author: Julian Antonielli <julianantonielli@gmail.com> Date: Mon Feb 27 12:12:37 2023 +0000 Remove unused `wrk-api-bench` dependency (smithy-lang#2331) * Remove unused `wrk-api-bench` dependency * Remove benchmark file commit dbafd68 Author: ysaito1001 <gperson22@gmail.com> Date: Fri Feb 24 19:31:41 2023 -0600 Add crate versions in CHANGELOG (smithy-lang#2348) * Move rendering external contributors to a function This commit moves the code for rendering external contributors to its own function, in line with other render_* functions. * Add type alias `CrateVersionMetadataMap` This commit adds a type alias `CrateVersionMetadataMap` for a field in `VersionsManifest`. In a subsequent commit, the `changelogger` crate will refer to this field, and it's better for that crate to not use the field's bare type `BTreeMap<String, CrateVersion>`. * Render crate versions in CHANGELOG This commit addresses a pain point brought up in awslabs/aws-sdk-rust#731. A binary `changelogger` now has a new command line option `--current-release-versions-manifest` that points to the latest `versions.toml` in the `aws-sdk-rust` repository. The program will generate a markdown table showing crate versions from that manifest and include it in an expand/collapse section. * Add a leading pipe when rendering markdown table This commit addresses smithy-lang#2348 (comment) --------- Co-authored-by: Yuki Saito <awsaito@amazon.com> Co-authored-by: Zelda Hessler <zhessler@amazon.com> commit 5183ccb Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 24 11:53:09 2023 -0800 Split the `types` and `builders` modules into per-shape private modules (smithy-lang#2415) commit 4c76847 Author: david-perez <d@vidp.dev> Date: Fri Feb 24 13:22:51 2023 +0100 Enclose missing Rust template parameters in backticks in the exception message (smithy-lang#1492) * Enclose missing Rust template parameters in backticks in the exception message * appease ktlint * ./gradlew ktlintFormat commit e23790e Author: ysaito1001 <gperson22@gmail.com> Date: Thu Feb 23 18:29:53 2023 -0600 Remove InvalidUri from external-types.toml (smithy-lang#2414) Co-authored-by: Yuki Saito <awsaito@amazon.com> commit 38665d2 Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 18:03:25 2023 -0800 Fix lint commit 3221246 Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 17 10:23:57 2023 -0800 Reorganize the types/models/primitives modules commit 0da92a4 Author: John DiSanti <jdisanti@amazon.com> Date: Thu Feb 23 12:12:11 2023 -0800 Move fluent builders and hide the parse strict response impl struct (smithy-lang#2401) * Hide the operation parse impl structs * Move fluent builders into per-operation modules commit 3d40835 Author: Russell Cohen <rcoh@amazon.com> Date: Thu Feb 23 14:56:24 2023 -0500 Use rustls for reqwest (smithy-lang#2403) * Use rustls for reqwest This also adds a CI check to ensure that we can build the SDK on MUSL * Fix MUSL build in image * Fix build image commit 530386e Author: Harry Barber <106155934+hlbarber@users.noreply.github.com> Date: Thu Feb 23 19:48:37 2023 +0000 Update changelog to reflect 0.54.x branch releases (smithy-lang#2411) commit 14bd809 Author: John DiSanti <jdisanti@amazon.com> Date: Thu Feb 23 09:19:27 2023 -0800 Add builder symbol/module resolution to symbol providers (smithy-lang#2395) * Add builder symbol/module resolution to symbol providers * Back out breaking changes * Reduce codegen diff * Fix server example clippy lints * Associate smithy-lang#2396 with TODO comments * Improve doc comment * Revert doc hidden change commit 76582ba Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 16:03:54 2023 -0800 Move `AppName` and `PKG_VERSION` (smithy-lang#2389) * Move `AppName` and `PKG_VERSION` * Fix merge issue commit 08c16d3 Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 15:37:32 2023 -0800 Fix escaping of `Self` in symbol providers (smithy-lang#2381) * Fix escaping of `Self` in symbol providers * Clean up an old hack commit 6dd7bc7 Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 15:38:03 2023 -0800 Reorganize more modules and re-exports (smithy-lang#2400) * Conditionally re-export the Smithy Client Builder in clients * Reorganize error type re-export * Reorganize the `paginator` module * Flatten the `presigning` module * Hide the `http_body_checksum` module commit 7cffe14 Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 13:20:43 2023 -0800 Remove `toEnumVariantName` from `RustSymbolProvider` (smithy-lang#2377) The `toEnumVariantName` function existed on symbol provider to work around enum definitions not being shapes. In the future when we refactor to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s for each enum member. This change incrementally moves us to that future by creating fake `MemberShape`s in the enum generator from the enum definition. commit da26405 Author: ysaito1001 <gperson22@gmail.com> Date: Wed Feb 22 15:04:48 2023 -0600 Add support for the awsQueryCompatible trait (smithy-lang#2398) * Add support for the awsQueryCompatible trait This commit adds support for the awsQueryCompatible trait. This allows services already supporting custom error codes through the AWS Query protocol with the awsQueryError trait to continue supporting them after the services switch to the AWS JSON 1.0 protocol. * Add copyright header * Fix clippy warning for clippy::manual-map * Update CHANGELOG.next.toml * Update CHANGELOG.next.toml * Update CHANGELOG.next.toml * Remove unused variables from `errorScope` This commit addresses smithy-lang#2398 (comment) * Reorder arguments for test verification This commit addresses smithy-lang#2398 (comment) --------- Co-authored-by: Yuki Saito <awsaito@amazon.com> commit d579c56 Author: Russell Cohen <rcoh@amazon.com> Date: Wed Feb 22 14:25:52 2023 -0500 Fix docker image hash script to be contained to ci-build (smithy-lang#2404) commit 85d2ace Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 22 10:21:06 2023 -0800 Upgrade Kotlin and Ktlint (smithy-lang#2392) * Upgrade Kotlin to 1.7.21 * Upgrade Ktlint to 0.48.2 * Run `pre-commit run --all-files` to fix broken Ktlints * Fix string comparison broken by code formatting commit ec1a551 Author: ysaito1001 <gperson22@gmail.com> Date: Tue Feb 21 14:44:15 2023 -0600 Address clippy warnings in pokemon-service (smithy-lang#2399) Co-authored-by: Yuki Saito <awsaito@amazon.com> commit a40dbfc Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 17 13:20:11 2023 -0800 Reorganize credentials, endpoints, region, and op customization (smithy-lang#2393) * Move `Endpoint` re-export into `config` * Move `Credentials` and `Region` * Move `customize` into `crate::client` * Clean up `RustModule` commit 86bddca Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 17 13:19:45 2023 -0800 Move inputs, outputs, and op errors into operation modules (smithy-lang#2394) commit afb1f16 Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 17 12:17:59 2023 -0800 Move `RustSymbolProvider` and related types out of `SymbolVisitor` (smithy-lang#2380) * Move base `RustSymbolProvider` types out of `SymbolVisitor` * Rename `SymbolVisitorConfig` to `RustSymbolProviderConfig` commit 3d00767 Author: Fahad Zubair <fahadzubair@gmail.com> Date: Fri Feb 17 19:21:01 2023 +0000 Constraint member types are refactored as standalone shapes. (smithy-lang#2256) * Constraint member types are refactored as standalone shapes. * ModelModule to ServerRustModule.model * Constraints are written to the correct module * Code generates for non-public constrained types. * Removed a comment * Using ConcurrentHashmap just to be on the safe side * Clippy warnings removed on constraints, k.into() if gated * Wordings for some of the checks changed * Test need to call rustCrate.renderInlineMemoryModules * ktlintFormat related changes * RustCrate need to be passed for server builder * Param renamed in getParentAndInlineModuleForConstrainedMember * pubCrate to publicConstrainedType rename * PythonServer symbol builder needed to pass publicConstrainedTypes * @required still remains on the member shape after transformation * ConcurrentLinkedQueue used for root RustWriters * runTestCase does not run the tests but just sets them up, hence has been renamed * CHANGELOG added --------- Co-authored-by: Fahad Zubair <fahadzub@amazon.com> commit 5a5a7c4 Author: 82marbag <69267416+82marbag@users.noreply.github.com> Date: Fri Feb 17 13:19:34 2023 -0500 Nested server structure member shapes targeting simple shapes with default trait (smithy-lang#2352) * Nested server structure member shapes targeting simple shapes with `@default` Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Add changelog entry Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Update CHANGELOG Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Add unit test Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Add integration test Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Change method Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> * Include comment to describe the test Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> --------- Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> commit 198c500 Author: david-perez <d@vidp.dev> Date: Fri Feb 17 18:24:25 2023 +0100 Disallow `@uniqueItems`-constrained list shapes that reach a map shape (smithy-lang#2375) * Disallow `@uniqueItems`-constrained list shapes that reach a map shape The server SDK codegen generates Rust code that does not compile when a `@uniqueItems`-constrained list shape reaches a map shape, essentially because `std::collections::HashMap` does not implement `std::hash::Hash`. A ticket with the Smithy team was opened in smithy-lang/smithy#1567 to disallow such models. This commit makes it so that codegen aborts with an informative message when such models are encountered, instead of going ahead and producing code that does not compile. This commit also slightly adjusts the error messages produced when unsupported constraint traits are found. * ./gradlew ktlintFormat commit b9f8090 Author: ysaito1001 <gperson22@gmail.com> Date: Fri Feb 17 10:21:24 2023 -0600 Ensure cache miss logging works in concurrent setting (smithy-lang#2391) This commit ensures that logging for cache miss should be displayed properly when multiple threads execute `get_or_load` concurrently. Specifically, the log should be displayed only once for the first thread that succeeds in populating a cache value. Co-authored-by: Yuki Saito <awsaito@amazon.com> commit 52b07c2 Author: John DiSanti <jdisanti@amazon.com> Date: Thu Feb 16 11:21:24 2023 -0800 Add feature flag for the crate reorganization (smithy-lang#2388) commit ba3b937 Author: Russell Cohen <rcoh@amazon.com> Date: Thu Feb 16 14:11:36 2023 -0500 Add presigned URL canary & ep2 test (smithy-lang#2379) * Add presigned URL canary & ep2 test * fix canary tests commit d54c005 Author: Zelda Hessler <zhessler@amazon.com> Date: Thu Feb 16 11:50:42 2023 -0600 add: deprecation notice to deprecated fluent builder methods (smithy-lang#2386) * add: deprecation notice to deprecated fluent builder methods * add: CHANGELOG.next.toml notes commit 42671dd Author: Nugine <nugine@foxmail.com> Date: Thu Feb 16 23:53:19 2023 +0800 Upgrade `base64-simd` to 0.8 (smithy-lang#2384) commit f7ac844 Author: John DiSanti <jdisanti@amazon.com> Date: Thu Feb 16 07:52:40 2023 -0800 Add generic client codegen diff to PR bot (smithy-lang#2383) commit 2473c5c Author: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com> Date: Thu Feb 16 02:15:40 2023 +0100 feat(codegen): support for api key auth trait (smithy-lang#2154) * feat(codegen): support for api key auth trait * chore: update to new codegen decorator interface * chore: include basic test * chore: set api key into rest xml extras model * chore: update test * chore: refactor api key definition map * feat(codegen): add api key decorator by default * chore: add smithy-http-auth to runtime type * chore: reference new smithy-http-auth crate * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Revert "chore: set api key into rest xml extras model" This reverts commit 93b99c8. * chore: moved api key re-export to extras customization * chore: include test for auth in query and header * chore: fix linting * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt Co-authored-by: John DiSanti <johndisanti@gmail.com> * chore: add doc hidden to re-export * chore: ensure extras are added only if it applies * Revert "chore: add doc hidden to re-export" This reverts commit 8a49e2b. --------- Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com> Co-authored-by: John DiSanti <jdisanti@amazon.com> Co-authored-by: John DiSanti <johndisanti@gmail.com> commit f9fb9e6 Author: david-perez <d@vidp.dev> Date: Wed Feb 15 23:09:37 2023 +0100 Fix recursive constraint violations with paths over list and map shapes (smithy-lang#2371) * Fix recursive constraint violations with paths over list and map shapes There is a widespread assumption throughout the generation of constraint violations that does not hold true all the time, namely, that a recursive constraint violation graph has the same requirements with regards to boxing as the regular shape graph. Some types corresponding to recursive shapes are boxed to introduce indirection and thus not generate an infinitely recursive type. The algorithm however does not superfluously introduce boxes when the cycle goes through a list shape or a map shape. Why list shapes and map shapes? List shapes and map shapes get rendered in Rust as `Vec<T>` and `HashMap<K, V>`, respectively, they're the only Smithy shapes that "organically" introduce indirection (via a pointer to the heap) in the recursive path. For other recursive paths, we thus have to introduce the indirection artificially ourselves using `Box`. This is done in the `RecursiveShapeBoxer` model transform. However, the constraint violation graph needs to box types in recursive paths more often. Since we don't collect constraint violations (yet, see smithy-lang#2040), the constraint violation graph never holds `Vec<T>`s or `HashMap<K, V>`s, only simple types. Indeed, the following simple recursive model: ```smithy union Recursive { list: List } @Length(min: 69) list List { member: Recursive } ``` has a cycle that goes through a list shape, so no shapes in it need boxing in the regular shape graph. However, the constraint violation graph is infinitely recursive if we don't introduce boxing somewhere: ```rust pub mod model { pub mod list { pub enum ConstraintViolation { Length(usize), Member( usize, crate::model::recursive::ConstraintViolation, ), } } pub mod recursive { pub enum ConstraintViolation { List(crate::model::list::ConstraintViolation), } } } ``` This commit fixes things by making the `RecursiveShapeBoxer` model transform configurable so that the "cycles through lists and maps introduce indirection" assumption can be lifted. This allows a server model transform, `RecursiveConstraintViolationBoxer`, to tag member shapes along recursive paths with a new trait, `ConstraintViolationRustBoxTrait`, that the constraint violation type generation then utilizes to ensure that no infinitely recursive constraint violation types get generated. For example, for the above model, the generated Rust code would now look like: ```rust pub mod model { pub mod list { pub enum ConstraintViolation { Length(usize), Member( usize, std::boxed::Box(crate::model::recursive::ConstraintViolation), ), } } pub mod recursive { pub enum ConstraintViolation { List(crate::model::list::ConstraintViolation), } } } ``` Likewise, places where constraint violations are handled (like where unconstrained types are converted to constrained types) have been updated to account for the scenario where they now are or need to be boxed. Parametrized tests have been added to exhaustively test combinations of models exercising recursive paths going through (sparse and non-sparse) list and map shapes, as well as union and structure shapes (`RecursiveConstraintViolationsTest`). These tests even assert that the specific member shapes along the cycles are tagged as expected (`RecursiveConstraintViolationBoxerTest`). * Address comments commit 6cade90 Author: John DiSanti <jdisanti@amazon.com> Date: Wed Feb 15 13:36:09 2023 -0800 Move symbol extension functions into `SymbolExt` (smithy-lang#2378) commit 1c1a3ef Author: david-perez <d@vidp.dev> Date: Wed Feb 15 16:09:42 2023 +0100 Allow server decorators to postprocess `ValidationException` not attached error messages (smithy-lang#2338) Should they want to, a server decorator can now postprocess the error message that arises when a constrained operation does not have the `ValidationException` shape attached to its errors. This commit adds a test to ensure that when such a decorator is registered, the `ValidationResult` can indeed be altered, but no such decorator is added to the `rust-server-codegen` plugin. commit d7f8130 Author: Harry Barber <106155934+hlbarber@users.noreply.github.com> Date: Wed Feb 15 11:45:56 2023 +0000 Update the service builder RFCs (smithy-lang#2374) * Complete service_builder.md * Complete refine_builder.md commit 181889d Author: Julian Antonielli <julianantonielli@gmail.com> Date: Tue Feb 14 16:23:31 2023 +0000 Fix manifest path comma (smithy-lang#2369) commit 18a26b6 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Tue Feb 14 15:36:28 2023 +0000 Add quotes to get a valid Javascript string (smithy-lang#2367) commit 10bb26c Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Tue Feb 14 15:36:14 2023 +0000 Add tests to make sure the publisher works as expected. (smithy-lang#2365) * Add tests to make sure the publisher works as expected. * Fix lint. commit ca3ef20 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Tue Feb 14 13:27:37 2023 +0000 Fix version regex. (smithy-lang#2364) commit aa07854 Author: Zelda Hessler <zhessler@amazon.com> Date: Mon Feb 13 17:32:59 2023 -0600 Fix S3 canary match statement (smithy-lang#2358) * fix: s3 canary error match * update: use Error.is_ to check for "no such key" error in canary * remove: leftover import commit 1d1e68a Author: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com> Date: Tue Feb 14 00:07:06 2023 +0100 feat(smithy-http-auth): add api key auth types (smithy-lang#2153) * feat(aws-types): add api key to configuration * chore: set package version to 0.52.0 * feat(aws-smithy-types): create auth types * chore: use auth from smithy types * chore: fix return self type * chore: create http auth definition type * chore: add constructor for http auth definition * chore: ensure properties are not moved * chore: create convenience constructors * chore: add some todo comments * feat: move query writer to aws-smithy-http crate * chore(codegen): expose smithy http tower dependency * chore: remove setters for auth definition * chore: fix logical error for scheme not allowed * chore: add constructor for basic and digest auth * chore: allow equality comparision for api key * Revert "chore: set package version to 0.52.0" This reverts commit da660fc. * chore: fix additional references to querywriter * chore: implement from string for api key struct * chore: disallow none api key in sdk config * chore: fix formatting * chore: add unit tests for auth types * chore: add auth api key to external types * chore: make query writer doc hidden * feat: create aws-smithy-http-auth crate * chore: use zeroing for auth api key * chore: use builder pattern for auth definition * chore: restructure http auth package * chore: define default lint warning * chore: include http auth in runtime common list * chore: define setter for optional scheme * chore: should panic while building auth definition * chore: return missing required field error * chore: make few code simplications for api key * Revert "chore: add auth api key to external types" This reverts commit b2318b0. * chore: revert api key from sdk config * chore: panic on missing required field * Opt out of `clippy::derive_partial_eq_without_eq` --------- Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com> Co-authored-by: John DiSanti <jdisanti@amazon.com> commit 10520d7 Author: Zelda Hessler <zhessler@amazon.com> Date: Mon Feb 13 16:45:50 2023 -0600 Fix: native-tls client creation to support HTTPS again (smithy-lang#2360) * update: use `enforce_http = false` when creating native-tls hyper connector refactor: move smithy conns module to its own file add: sanity tests ensuring we can make HTTP and HTTPS requests with the rustls and native-tls connectors remove: `use crate::*` imports in favor of explicit imports * update: CHANGELOG.next.toml * add: feature gate to conns tests commit f7c550e Author: John DiSanti <jdisanti@amazon.com> Date: Mon Feb 13 11:38:57 2023 -0800 Fix CI for forks by moving merge queue runs into a separate workflow (smithy-lang#2359) commit 9ecd7f0 Author: Harry Barber <106155934+hlbarber@users.noreply.github.com> Date: Mon Feb 13 18:25:56 2023 +0000 Fix consistent service naming and add test coverage (smithy-lang#2349) * Make service name casing consistent * Add naming-obstacle-course-casing.smithy * Add missing import * Relax obstacle course even further * Better TODO * Add CHANGELOG.next.toml commit f705bde Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 13 17:33:14 2023 +0000 Idempotent release action (smithy-lang#2353) * Do not require the released commit to be at the tip of the branch. * Update comment. * If you release an old commit, it must be ready to go as is. * Tag the commit that updates gradle.properties. * Check out the specific commit instead of the branch. * Simplify. * We need to set the release branch. * You can't refer to outputs directly. * Make sure to tag the right commit. * Pass the commit SHA explicitly. commit c060263 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 13 17:32:26 2023 +0000 Do not run the PR bot on the merge queue (smithy-lang#2357) * Do not run the PR bot on the merge queue * Update ci-pr.yml commit bfbe25e Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 13 17:25:48 2023 +0000 Do not skip CI jobs for merge queue events. (smithy-lang#2356) * Do not skip for merge queue events. * Fix quotes. commit b3b3393 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 13 17:01:01 2023 +0000 Run CI on merge groups. (smithy-lang#2355) commit e63f3ee Author: david-perez <d@vidp.dev> Date: Mon Feb 13 12:41:43 2023 +0100 Simplify `simple.smithy` (smithy-lang#2339) `simple.smithy` is currently not that simple. Besides, it serves no effective purpose: everything that the model exercises is already exercised in a clearer and more orderly manner in other integration tests. This PR makes the model minimal, so that it recovers its purpose as a simple smoke test. Contributors can also use the model as a scratchpad for features or bugfixs they're working on. commit d48878e Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 10 17:36:48 2023 -0800 Implement request ID access for SDK clients RFC (smithy-lang#2129) * Add `RequestId` trait * Implement `RequestId` for generated AWS client errors * Move `RustWriter.implBlock` out of `StructureGenerator` * Create structure/builder customization hooks * Customize `_request_id` into AWS outputs * Set request ID on outputs * Refactor SDK service decorators * Refactor S3's extended request ID implementation * Combine `Error` and `ErrorKind` * Add test for service error conversion * Move error generators into `codegen-client` and fix tests * Re-export `ErrorMetadata` * Add request IDs to trace logs * Simplify some error trait handling * Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization` * Add deprecated alias to guide customers through upgrading * Rename the `ErrorMetadata` trait to `ProvideErrorMetadata` * Rename `aws_smithy_types::Error` to `ErrorMetadata` commit 0a11d51 Author: ysaito1001 <gperson22@gmail.com> Date: Fri Feb 10 17:03:38 2023 -0600 Add jitter to `LazyCredentialsCache` (smithy-lang#2335) * Add jitter to `LazyCredentialsCache` This commit adds jitter to `LazyCredentialsCache`. A jitter provides a mechanism for randomizing the buffer time for credentials. This allows credentials with the same expiry to expire at slightly different times, thereby preventing thundering herds. * Update CHANGELOG.next.toml --------- Co-authored-by: Yuki Saito <awsaito@amazon.com> commit cdc710d Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 10 14:21:16 2023 -0800 Make modules in `codegen-core` configurable (smithy-lang#2336) * Refactor modules to be configurable in `codegen-core` * Remove panicking default test symbol provider * Remove as many references to Error/Types as possible * Rename module constants commit de18667 Author: Nugine <nugine@foxmail.com> Date: Sat Feb 11 04:59:31 2023 +0800 Bump Rust MSRV to 1.63.0 (smithy-lang#2222) Co-authored-by: Zelda Hessler <zhessler@amazon.com> Co-authored-by: John DiSanti <jdisanti@amazon.com> commit 13b10d5 Author: John DiSanti <jdisanti@amazon.com> Date: Fri Feb 10 11:07:56 2023 -0800 Add RFC for improving request ID access in SDK clients (smithy-lang#2083) commit e84ef6c Author: Burak <unexge@gmail.com> Date: Fri Feb 10 14:00:53 2023 +0000 Python: Type-stub generation for SSDKs (smithy-lang#2149) * Initial Python stub generation * Handle default values correctly * Only generate `__init__` for classes that have constructor signatures * Preserve doc comments * Make context class generic * Put type hint into a string to fix runtime error * Run `mypy` on CI * Use `make` to build Python SSDKs while generating diffs * Escape Python types in Rust comments * Only mark class methods with * Sort imports to minimize diffs * Add type annotations for `PySocket` * Dont extend classes from `object` as every class already implicitly extended from `object` * Use `vars` instead of `inspect.getmembers` to skip inherited members of a class * Fix linting issues * Add some tests for stubgen and refactor it * Add type annotations to `PyMiddlewareException` * Fix tests on Python 3.7 Python 3.7 doesn't support reading signatures from `__text_signature__` for non-builtin functions (i.e. C/Rust functions). For testing we're using regular Python syntax for defining signature. * Provide default values for `typing.Optional[T]` types in type-stubs * Update `is_fn_like` to cover more cases * Remove `tools/smithy-rs-tool-common/` * Make `DECORATORS` an array instead of a list * Ignore missing type stub errors for `aiohttp` commit 086d965 Author: John DiSanti <jdisanti@amazon.com> Date: Thu Feb 9 12:10:31 2023 -0800 Refactor client/server logic out of `EnumGenerator` (smithy-lang#2334) * Refactor client/server enum generation logic into new `EnumType` * Move client enum logic into `codegen-client` commit bae9380 Author: Zelda Hessler <zhessler@amazon.com> Date: Thu Feb 9 11:02:19 2023 -0600 Update pretty_assertions (smithy-lang#2332) * update: pretty_assertions * update: just use v1.3 commit 78203c0 Author: 82marbag <69267416+82marbag@users.noreply.github.com> Date: Thu Feb 9 09:43:03 2023 -0500 RestJson with body expects application/json content type (smithy-lang#2330) Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de> commit bf678fd Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Thu Feb 9 13:16:41 2023 +0000 [Release workflow] Workaround for the ownership bug (smithy-lang#2318) * First publish everything, then try to correct ownership. * Avoid trying to remove teams, since we know it won't succeed. * Fix warning and update dependencies. * Avoid allocation unless necessary. * Get full error representation for retryable failures. * We want to hash the file content, not the file path. commit a3075ea Author: david-perez <d@vidp.dev> Date: Wed Feb 8 23:14:31 2023 +0100 Move converters from constraint violations into `ValidationException` to decorators (smithy-lang#2302) This allows "easily" converting to other custom validation exception shapes by implementing a decorator. As a simple example, this PR adds a `CustomValidationExceptionWithReasonDecorator` decorator that is able to convert into a shape that is very similar to `smithy.framework#ValidationException`, but that has an additional `reason` field. The decorator can be enabled via the newly added `experimentalCustomValidationExceptionWithReasonPleaseDoNotUse` codegen config flag. This effectively provides a way for users to use custom validation exceptions without having to wait for the full implementation of smithy-lang#2053, provided they're interested enough to write a decorator in a JVM language. This mechanism is _experimental_ and will be removed once full support for custom validation exceptions as described in smithy-lang#2053 lands, hence why the configuration key is strongly worded in this respect. This commit also ports the mechanism to run codegen integration tests within Kotlin unit tests for client SDKs to the server. See smithy-lang#1956 for details. The custom validation exception decorator is tested this way. commit bb6155a Author: John DiSanti <jdisanti@amazon.com> Date: Tue Feb 7 16:16:11 2023 -0800 Remove `codegen-client` dependency from Python server (smithy-lang#2307) commit b1ee45a Author: Julian Antonielli <julianantonielli@gmail.com> Date: Tue Feb 7 16:45:57 2023 +0000 Fix protocol test: `RestJsonMalformedPatternSensitiveString` (smithy-lang#2321) * Make `RestJsonMalformedPatternSensitiveString` pass * Use `AwsJson11` variable instead of hardcoded string * Refactor `errorMessage` into private function * Add comment about remaining `@range` on floats tests * Use `AwsJson11` variable instead of hardcoded string (missed one) * Use `hasTrait` over `getTrait() != null` commit 12a66c2 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Tue Feb 7 16:20:03 2023 +0000 Instruct the release script to tag the commit on the release branch. It was previously left unspecified, resulting on a tag on the `main` branch. (smithy-lang#2322) commit 18bfa20 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Tue Feb 7 15:25:37 2023 +0000 Remove entries that have been released in 0.54.2. (smithy-lang#2323) commit 3ee62e8 Author: david-perez <d@vidp.dev> Date: Tue Feb 7 13:51:35 2023 +0100 RFC: Better Constraint Violations (smithy-lang#2040) commit 0226446 Author: david-perez <d@vidp.dev> Date: Mon Feb 6 19:16:43 2023 +0100 Rename Rust server codegen plugins (smithy-lang#2306) Rename `RustCodegenServerPlugin` to `RustServerCodegenPlugin`, for consistency with `RustClientCodegenPlugin`. This is a better name, since the plugin is named `rust-server-codegen`. This commit also renames `PythonCodegenServerPlugin` to `RustServerCodegenPythonPlugin` for the same reasons. This commit also contains other drive-by improvements made while working on smithy-lang#2302. commit 7fdb5c9 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 6 16:39:38 2023 +0000 Fetch history when cloning, otherwise git commands such as `rev-parse` could fail. (smithy-lang#2314) commit a389ea2 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 6 16:30:33 2023 +0000 Enforce the same minimum TLS version (1.2) for both TLS backends (smithy-lang#2312) * Enforce the same minimum TLS version (1.2) for both TLS backends * Add CHANGELOG entry * Add documentation for both `https` and `native_tls`. * Remove unnecessary mut commit 51047b1 Author: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Date: Mon Feb 6 16:03:43 2023 +0000 Use `refs/remote/<remote-name>/<branch-name>` to avoid a "Fatal: needed a single revision" error, as we have seen in https://github.com/awslabs/smithy-rs/actions/runs/4105169302/jobs/7082226485 (smithy-lang#2313) commit c9275fb Author: Russell Cohen <rcoh@amazon.com> Date: Mon Feb 6 10:12:17 2023 -0500 Prevent test dependencies from leaking into production (smithy-lang#2264) * Prevent test dependencies from leaking into production * refactor & fix tests * fix tests take two * fix more tests * Fix missed called to mergeDependencyFeatures * Add test * fix glacier compilation * fix more tests * fix one more test

unexge added the python-server Python server SDK label Dec 30, 2022

unexge commented Dec 30, 2022

View reviewed changes

unexge force-pushed the unexge/python-type-stub-generation branch from 55a10ec to 541df2d Compare January 3, 2023 16:04

unexge force-pushed the unexge/python-type-stub-generation branch from 541df2d to df4a079 Compare January 4, 2023 15:53

unexge force-pushed the unexge/python-type-stub-generation branch from df4a079 to a3b7d82 Compare January 5, 2023 12:18

unexge force-pushed the unexge/python-type-stub-generation branch from a3b7d82 to 333bbcf Compare January 5, 2023 14:50

crisidev reviewed Jan 5, 2023

View reviewed changes

unexge force-pushed the unexge/python-type-stub-generation branch from 25c9701 to f67a787 Compare January 6, 2023 16:53

unexge marked this pull request as ready for review January 6, 2023 18:06

unexge requested review from a team as code owners January 6, 2023 18:06

unexge requested review from jdisanti and hlbarber January 6, 2023 18:07

unexge force-pushed the unexge/python-type-stub-generation branch from f67a787 to 580ae56 Compare January 9, 2023 10:44

unexge force-pushed the unexge/python-type-stub-generation branch from 580ae56 to 884aeeb Compare January 9, 2023 13:50

crisidev reviewed Jan 13, 2023

View reviewed changes

unexge added 15 commits February 10, 2023 10:54

Put type hint into a string to fix runtime error

5adc034

Run mypy on CI

bc17873

Use make to build Python SSDKs while generating diffs

8a20a4f

Escape Python types in Rust comments

ac02352

Only mark class methods with

6b77a10

Sort imports to minimize diffs

9bf1b17

Add type annotations for PySocket

022835e

Dont extend classes from object as every class already implicitly e…

c8201ab

…xtended from `object`

Use vars instead of inspect.getmembers to skip inherited members …

9afcfbb

…of a class

Fix linting issues

d081d8a

Add some tests for stubgen and refactor it

f7155fc

Add type annotations to PyMiddlewareException

9b29bd1

Fix tests on Python 3.7

086370c

Python 3.7 doesn't support reading signatures from `__text_signature__` for non-builtin functions (i.e. C/Rust functions). For testing we're using regular Python syntax for defining signature.

Provide default values for typing.Optional[T] types in type-stubs

d72fff9

Update is_fn_like to cover more cases

0ddba29

unexge force-pushed the unexge/python-type-stub-generation branch from 5f355d1 to 0ddba29 Compare February 10, 2023 11:03

unexge added 2 commits February 10, 2023 11:15

Remove tools/smithy-rs-tool-common/

7ad167d

Make DECORATORS an array instead of a list

7e9c9cb

Ignore missing type stub errors for aiohttp

cd00b0b

unexge merged commit e84ef6c into main Feb 10, 2023

unexge deleted the unexge/python-type-stub-generation branch February 10, 2023 14:00

unexge mentioned this pull request Mar 2, 2023

Make codegen diff fault tolerant #2426

Merged

crisidev mentioned this pull request Mar 2, 2023

[WIP] Generate server implementation for pure Python business logic #1367

Open

34 tasks

unexge mentioned this pull request Mar 22, 2023

[Python] Generate codegen diff for every test package #2483

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Type-stub generation for SSDKs #2149

Python: Type-stub generation for SSDKs #2149

unexge commented Dec 30, 2022 •

edited

Loading

unexge Dec 30, 2022

crisidev Jan 13, 2023 •

edited

Loading

github-actions bot commented Dec 30, 2022

github-actions bot commented Jan 3, 2023

github-actions bot commented Jan 4, 2023

github-actions bot commented Jan 5, 2023

crisidev Jan 5, 2023

unexge Jan 5, 2023

crisidev Jan 13, 2023

crisidev Jan 5, 2023

crisidev Jan 13, 2023

unexge Jan 23, 2023

crisidev Jan 5, 2023

crisidev Jan 5, 2023

unexge Jan 5, 2023

github-actions bot commented Jan 6, 2023

github-actions bot commented Jan 9, 2023

crisidev commented Jan 9, 2023

unexge commented Jan 9, 2023

crisidev commented Jan 9, 2023

github-actions bot commented Jan 9, 2023

crisidev Jan 13, 2023

crisidev Jan 13, 2023

github-actions bot commented Feb 10, 2023

github-actions bot commented Feb 10, 2023

crisidev commented Feb 10, 2023

		@@ -0,0 +1,391 @@
		# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

		// TODO(Constraints): How to handle this?
		// Revisit as part of https://github.com/awslabs/smithy-rs/issues/2114

Python: Type-stub generation for SSDKs #2149

Python: Type-stub generation for SSDKs #2149

Conversation

unexge commented Dec 30, 2022 • edited Loading

Motivation and Context

What is a Python type-stub?

Implementation

Testing

TODOs - Questions

Choose a reason for hiding this comment

crisidev Jan 13, 2023 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Dec 30, 2022

github-actions bot commented Jan 3, 2023

github-actions bot commented Jan 4, 2023

github-actions bot commented Jan 5, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jan 6, 2023

github-actions bot commented Jan 9, 2023

crisidev commented Jan 9, 2023

unexge commented Jan 9, 2023

crisidev commented Jan 9, 2023

github-actions bot commented Jan 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 10, 2023

github-actions bot commented Feb 10, 2023

crisidev commented Feb 10, 2023

unexge commented Dec 30, 2022 •

edited

Loading

crisidev Jan 13, 2023 •

edited

Loading