Skip to content

Commit

Permalink
feat(libs,metagen): draft python
Browse files Browse the repository at this point in the history
wip: python mdk

feat: represent union, either with Union

feat: solve cycles, order python types by priority

fix: self-reference

fix: premature checks

docs: add a comparison b/n metatype and other similar solutions/products. (#697)

<!--
Pull requests are squash merged using:
- their title as the commit message
- their description as the commit body

Having a good title and description is important for the users to get
readable changelog.
-->

<!-- 1. Explain below WHAT the change is -->
- Adds a comparison table between metatype and other similar services.
- Add artifact upload protocol to `Architecture` section in docs.

<!-- 2. Explain below WHY the change cannot be made simpler -->

<!-- 3. Explain below WHY the was made or link an issue number -->

[MET-443](https://linear.app/metatypedev/issue/MET-443/include-comparisons-with-other-products-similar-to-metatype)

<!-- 4. Explain HOW users should update their code or remove that
section -->

_No Migration Needed_

<!-- 5. Readiness checklist
- [ ] The change come with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change
-->

refactor(gate): wasi 0.2 pyrt (#687)

- Rewrites the PythonRuntime host using a `componentize-py` based
component.
- Leaf through this
[memo](https://hackmd.io/@SC-qT-WXTROceKYdNA-Lpg/ryyAXiQlC/edit) for a
mental model.

Todo:
- [x] `PythonRuntime.import_` support #699
- [x] ~~Add `pyrt.wasm` to release job~~ obviated by `build.rs`

Items for other PRs:
- Implemen `hostcall`

MET-404.

_No end-user changes required_

- [x] The change come with new or modified tests
- [x] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change

---------

Co-authored-by: afmika <afmichael73@gmail.com>

cleanups

fix: pre-commit

fix: forward-refs

fix: top level simple types

test(metagen): python mdk

cleanups and small fixes

feat(metagen): codegen decorator and some cleanups

feat(metagen): always merge defs if refered file is the same

fix(metagen): prioritize relto typegraph path only if base.path is empty

feat: Artifact removal (#668)

- Add GC: remove artifacts when unreferenced by any deployed typegraph
- Improve resource management: use `AsyncDisposable` and
`AsyncDisposableStack`
- Improve testability (for parallel testing): always read the tmpDir
config from the `Typegate` object

[MET-433](https://linear.app/metatypedev/issue/MET-433/file-removal)

_N/A_

- [x] The change come with new or modified tests
- [x] Hard-to-understand functions have explanatory comments
- [x] End-user documentation is updated to reflect the change

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **New Features**
  - Enhanced search functionality with the addition of a new search bar.
  - Introduced new test configurations to improve script execution.
- Updated artifact storage documentation to clarify management
processes.
  - Added new extensions to support improved code commenting.

- **Bug Fixes**
- Removed outdated Deno import mapping settings to streamline
development environment setup.

- **Documentation**
- Expanded documentation on artifact tracking and management, including
reference counting and garbage collection mechanisms.

- **Refactor**
- Implemented interface changes in `QueryEngine` for better async
disposal management.
- Code restructuring in artifact management for enhanced performance and
maintainability.

- **Chores**
- Adjusted settings and configurations in the development environment to
align with current best practices.

- **Tests**
- Introduced new test cases for artifact upload and management
functionalities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Natoandro <anatoandro@hotmail.com>
Co-authored-by: destifo <estifanosbireda@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

chore(release): prepare 0.4.0 (#710)

Bumps version to release 0.4.0.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **New Features**
- Updated the software across various components to version 0.4.0,
enhancing functionality and potentially introducing new features or
fixes.
- **Documentation**
- Updated version documentation in multiple configuration files to
reflect new version 0.4.0.
- **Bug Fixes**
- Adjusted version constants and dependencies to ensure compatibility
and stability with the new software version 0.4.0.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Natoandro <anatoandro@hotmail.com>

chore(docs): final polish to comparison table. (#709)

some changes to comparison table(docs)

_No Migrations Needed_

<!-- 5. Readiness checklist
- [ ] The change come with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change
-->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **Documentation**
- Introduced a new section on Artifact Tracking Protocol in the
architecture documentation, explaining artifact classification and
tracking modes in Metatype.
- Updated comparisons documentation with additional platforms, criteria
for choosing Metatype, and detailed feature comparison tables.
- Renamed project directory for clarity and consistency in project setup
documentation.
- **Bug Fixes**
  - Removed outdated `TODO` comment in installation documentation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

chore: bump to version 0.4.1-0 (#713)

- Bumps version to 0.4.1-0.
- Fixes broken release CI.
- #719
- Adds 20 minutes to test-full timeout.

<!-- 5. Readiness checklist
- [ ] The change come with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change
-->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **New Features**
- Updated platform support for better compatibility with "x86_64-linux".

- **Bug Fixes**
- Minor version updates across multiple configurations to enhance
stability.

- **Chores**
- Updated version numbers from "0.4.0" to "0.4.1-0" across various files
and configurations.

- **Refactor**
- Adjusted build and test scripts for improved efficiency and
compatibility.

- **Documentation**
- Enhanced internal documentation to reflect version and platform
changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com>

feat: polish documentation and project (#696)

<!--
Pull requests are squash merged using:
- their title as the commit message
- their description as the commit body

Having a good title and description is important for the users to get
readable changelog.
-->

<!-- 1. Explain below WHAT the change is -->

- update the headline, the overviews and many other documentation areas
- upgrades the dependencies.

<!-- 2. Explain below WHY the change cannot be made simpler -->

<!-- 4. Explain HOW users should update their code or remove that
section -->

- [ ] The change come with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [x] End-user documentation is updated to reflect the change

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **Bug Fixes**
- Updated Docker image version for the `typegate` service to ensure
stability and compatibility.

- **Documentation**
- Revised `TAGLINE` for better clarity on supported languages: WASM,
Typescript, and Python.
- Updated version declarations for improved consistency and
functionality across multiple files.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

feat(sdk): expose metagen to `typegraph/sdk` (#718)

Expose metagen features to `typegraph/sdk`

Depends on #707 and #696

None

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

- **New Features**
- Introduced new `Metagen` class for code generation tasks in both
TypeScript and Python SDKs.
- Added functionality for defining policies and structures for
deployment examples using Node.js.

- **Improvements**
- Enhanced `Metagen` class with methods for simulating and executing
code generation tasks.
- Simplified file reading and writing functions for better performance
and maintainability.

- **Bug Fixes**
- Refined `compress_and_encode` function to streamline file handling
processes.

- **Tests**
- Added comprehensive tests for `Metagen` functionality in both
TypeScript and Python SDKs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com>
Co-authored-by: Teo Stocco <teo@zifeo.com>
Co-authored-by: Teo Stocco <zifeo@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Estifanos Bireda <77430541+destifo@users.noreply.github.com>
Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com>
  • Loading branch information
6 people committed May 22, 2024
1 parent 3145fb6 commit 9ffe44d
Show file tree
Hide file tree
Showing 243 changed files with 13,793 additions and 5,148 deletions.
260 changes: 260 additions & 0 deletions .ghjk/deno.lock

Large diffs are not rendered by default.

431 changes: 291 additions & 140 deletions .ghjk/lock.json

Large diffs are not rendered by default.

20 changes: 7 additions & 13 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,25 @@
<!--
Pull requests are squash merged using:
Pull requests are squashed and merged using:
- their title as the commit message
- their description as the commit body
Having a good title and description is important for the users to get readable changelog.
-->

<!-- 1. Explain below WHAT the change is -->
<!-- 1. Explain WHAT the change is about -->

...

<!-- 2. Explain below WHY the change cannot be made simpler -->

...
-

<!-- 3. Explain below WHY the was made or link an issue number -->
<!-- 2. Explain WHY the change cannot be made simpler -->

...
-

<!-- 4. Explain HOW users should update their code or remove that section -->
<!-- 3. Explain HOW users should update their code -->

#### Migration notes

...

<!-- 5. Readiness checklist
- [ ] The change come with new or modified tests
- [ ] The change comes with new or modified tests
- [ ] Hard-to-understand functions have explanatory comments
- [ ] End-user documentation is updated to reflect the change
-->
2 changes: 1 addition & 1 deletion .github/workflows/publish-website.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
- main

env:
GHJK_VERSION: "423d38e"
GHJK_VERSION: "2725af8"

jobs:
changes:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ on:
- v*

env:
GHJK_VERSION: "423d38e"
GHJK_VERSION: "2725af8"
DENO_VERSION: "1.41.0"
REGISTRY_IMAGE: ghcr.io/${{ github.repository_owner }}/typegate

Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: macos-latest
- os: macos-13
target: x86_64-apple-darwin
suffix: ""
- os: macos-14
Expand Down
21 changes: 14 additions & 7 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:
- ready_for_review

env:
GHJK_VERSION: "423d38e"
GHJK_VERSION: "2725af8"
DENO_VERSION: "1.41.0"
RUST_BACKTRACE: "full"
RUST_LOG: "info,swc_ecma_codegen=off,tracing::span=off"
Expand Down Expand Up @@ -123,7 +123,7 @@ jobs:
strategy:
matrix:
include:
- os: macos-latest
- os: macos-13
activate: "source .venv/bin/activate"
- os: macos-14
activate: "source .venv/bin/activate"
Expand Down Expand Up @@ -188,8 +188,10 @@ jobs:
runner: ubuntu-latest
# FIXME: try macos-14 runner once all actions support it
# docker buildx action broken as of 2024-02-09
- platform: linux/arm64
runner: custom-macos
# - platform: linux/arm64
# runner: custom-macos
# runner: macos-14
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
Expand Down Expand Up @@ -221,7 +223,7 @@ jobs:
if: "! steps.check.outputs.has-permission"
with:
file: dev/Dockerfile
platforms: linux/amd64
platforms: ${{ matrix.platform }}
push: false
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/typegate:latest
target: check
Expand All @@ -232,7 +234,7 @@ jobs:
runs-on:
- self-hosted # https://check-jsonschema.readthedocs.io/en/latest/faq.html#using-self-hosted-runners
- custom-ubuntu-large
timeout-minutes: 60
timeout-minutes: 80
steps:
- uses: actions/checkout@v4
- uses: dsherret/rust-toolchain-file@v1
Expand Down Expand Up @@ -299,6 +301,8 @@ jobs:
npm publish --registry $NPM_CONFIG_REGISTRY
cd ../../../..
ghjk x clean-deno-lock
# install packages inside metatype examples
pnpm install -C examples/typegraphs
Expand All @@ -317,13 +321,16 @@ jobs:
deno run -A dev/update.ts --cache-only || deno run -A dev/update.ts --cache-only
cargo build -p meta-cli
cargo build -p xtask # xtask and meta-cli are used by the test suite
ghjk x clean-deno-lock
deno run -A dev/test.ts --threads 1 -- --coverage=coverage
# FIXME: required due to https://github.com/denoland/deno/issues/21621#issuecomment-1887870712
deno cache --import-map typegate/import_map.json \
typegate/src/main.ts \
typegate/tests/utils/*.ts \
typegate/tests/runtimes/wasm/*.ts \
typegate/tests/runtimes/python_wasi/*.ts \
typegate/tests/runtimes/wasm_wire/*.ts \
typegate/tests/runtimes/wasm_reflected/*.ts \
typegate/tests/runtimes/python/*.ts \
dev/deps.ts \
dev/utils.ts
deno --unstable-worker-options --unstable-net coverage ./coverage --lcov > coverage.lcov
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ typegate/codegen
typegate/tests/prisma-migrations
typegate/tests/e2e/cli/prisma-migrations
typegate/tests/importers/copy/
typegate/tests/**/*.wasm
.env
typegate/native/bindings.json
bindings
Expand All @@ -47,3 +48,4 @@ typegraph/python/typegraph/gen
*.egg-info/

examples/typegraphs/migrations
libs/pyrt_wit_wire/wit_wire
8 changes: 7 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@ repos:
- id: check-merge-conflict
- id: end-of-file-fixer
# exclude all generated files
exclude: (.ghjk/.*|typegate/deno.lock|.*\.snap$|typegate/src/typegraphs/.*\.json|website/docs/reference/)
exclude: (?x)(
.ghjk/.*
|.*deno.lock
|.*\.snap$|typegate/src/typegraphs/.*\.json
|website/docs/reference/
|libs/pyrt_wit_wire/pyrt)
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.27.2
hooks:
Expand Down Expand Up @@ -113,6 +118,7 @@ repos:
- --license-filepath=dev/license-header-Elastic-2.0.txt
- "--comment-style=//"
- "--skip-license-insertion-comment=no-auto-license-header"
- "--skip-license-insertion-comment=@generated"
types_or:
- rust
files: ^(typegate|libs!(/metagen/.*))/
Expand Down
3 changes: 2 additions & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"chrischinchilla.vale-vscode",
"hediet.vscode-drawio",
"davidlday.languagetool-linter",
"dtsvet.vscode-wasm"
"dtsvet.vscode-wasm",
"bierner.comment-tagged-templates"
]
}
1 change: 0 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
"website/.docusaurus": true,
"website/node_modules": true
},
"deno.importMap": "typegate/import_map.json",
"deno.enablePaths": [
"typegate",
"dev",
Expand Down
Loading

0 comments on commit 9ffe44d

Please sign in to comment.