-
Notifications
You must be signed in to change notification settings - Fork 57
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
split emitter into autorest.python and pygen (#2588)
* initial moving over into pygen core * split package into pygen/core and autorest.python * black * pylint and mypy * add editable install of pygen into autorest.python * move files over to pygen * basic generaiton possible from autorest with script files moved to pygen * autorest generation working from scripts in pygen file * require changelog check * add changeset * add pcgc to changelog * comment out pylinting * add back linting and check * list installed packages * list installed packages * activate env in pipeline * add pygen to common reqs * change path to be relative from autorest.python * pylint * pyright * add pygen to setup.py, remove from requirements.txt * Revert "add pygen to setup.py, remove from requirements.txt" This reverts commit ea73b14. * add pygen to unittests requirements.txt * update unittests * hopefully fix multiapi * keep it as just one venv in pygen * typespec compiling with pygen inside dist * can copy pygen into dist when building and generate all typespec * trying to remove packaging files from pygen * typespec and autorest.python working with vendored pygen * update chronus * fix version mismatch * lockfile * add venv folder path * fix template * add venv folder path with $ sign * don't put venv in the venvFolderPath * add fi to venv script * add tilde to semver * rmeove venv from pipelines, too complicated * udpate lockfile * revert dev_requirements.txt back to current in main * remove pygen from reqs list * add pygen to dev reqs * add pygen back to unittests reqs * temp * continue moving pygen to into typespec-python * add dep on typespec-python in autorest.python * can generate autorest * can generate typespec locally * update reqs * duplicate runpython3 script in autorest, fix dev reqs path * move scripts for typespec out of pygen folder * add scripts to incldued files * copyfiles venv into autorest.python * duplicate venv * move venv into pygen folder * can generate typespec * fix installation for tsp and try to copy pygen into autorest.python * use own script for copying pygen * fix readme and setup.py of autorest.pyton * copy pygen into autorest.python * try moving pygen up a folder * make pygen a package again * keep trying with setup.py file * remove generator from autorest.python committed code, can generate typespec * move venv into top level of package * get autorest.python running * add generator to dev reqs * fix venvtools path * update dev reqs * add generator to published files * update script * update * update --------- Co-authored-by: iscai-msft <isabellavcai@gmail.com> Co-authored-by: Chenjie Shi <tadelesh.shi@live.cn>
- Loading branch information
1 parent
ad59817
commit 48d1426
Showing
147 changed files
with
949 additions
and
347 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
changeKind: feature | ||
packages: | ||
- "@autorest/python" | ||
- "@azure-tools/typespec-python" | ||
--- | ||
|
||
add package pygen that both autorest.python and typespec-python will rely on |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -125,3 +125,4 @@ node_modules/ | |
|
||
# Generated test folders | ||
test/services/*/_generated | ||
**/autorest.python/generator |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# ------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for | ||
# license information. | ||
# -------------------------------------------------------------------------- | ||
from typing import Dict, Any | ||
from pygen.black import BlackScriptPlugin | ||
from . import PluginAutorest | ||
|
||
|
||
class BlackScriptPluginAutorest(BlackScriptPlugin, PluginAutorest): | ||
def get_options(self) -> Dict[str, Any]: | ||
return {"output_folder": self._autorestapi.get_value("outputFolderUri")} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
# ------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for | ||
# license information. | ||
# -------------------------------------------------------------------------- | ||
import logging | ||
from typing import Dict, Any, Union | ||
from pathlib import Path | ||
import yaml | ||
|
||
from pygen.codegen import CodeGenerator | ||
from pygen.codegen.models import CodeModel | ||
from pygen.codegen.serializers import JinjaSerializer | ||
|
||
from . import ReaderAndWriterAutorest | ||
from .jsonrpc import AutorestAPI | ||
from . import PluginAutorest | ||
|
||
|
||
_LOGGER = logging.getLogger(__name__) | ||
|
||
|
||
class JinjaSerializerAutorest(JinjaSerializer, ReaderAndWriterAutorest): | ||
def __init__( | ||
self, | ||
autorestapi: AutorestAPI, | ||
code_model: CodeModel, | ||
*, | ||
output_folder: Union[str, Path], | ||
**kwargs: Any, | ||
) -> None: | ||
super().__init__( # type: ignore | ||
autorestapi=autorestapi, | ||
code_model=code_model, | ||
output_folder=output_folder, | ||
**kwargs, | ||
) | ||
|
||
|
||
class CodeGeneratorAutorest(CodeGenerator, PluginAutorest): | ||
def get_options(self) -> Dict[str, Any]: | ||
if self._autorestapi.get_boolean_value("python3-only") is False: | ||
_LOGGER.warning("You have passed in --python3-only=False. We have force overriden this to True.") | ||
if self._autorestapi.get_boolean_value("add-python3-operation-files"): | ||
_LOGGER.warning( | ||
"You have passed in --add-python3-operation-files. " | ||
"This flag no longer has an effect bc all SDKs are now Python3 only." | ||
) | ||
if self._autorestapi.get_boolean_value("reformat-next-link"): | ||
_LOGGER.warning( | ||
"You have passed in --reformat-next-link. We have force overriden " | ||
"this to False because we no longer reformat initial query parameters into next " | ||
"calls unless explicitly defined in the service definition." | ||
) | ||
options = { | ||
"azure-arm": self._autorestapi.get_boolean_value("azure-arm"), | ||
"header-text": self._autorestapi.get_value("header-text"), | ||
"low-level-client": self._autorestapi.get_boolean_value("low-level-client", False), | ||
"version-tolerant": self._autorestapi.get_boolean_value("version-tolerant", True), | ||
"show-operations": self._autorestapi.get_boolean_value("show-operations"), | ||
"python3-only": self._autorestapi.get_boolean_value("python3-only"), | ||
"head-as-boolean": self._autorestapi.get_boolean_value("head-as-boolean", False), | ||
"keep-version-file": self._autorestapi.get_boolean_value("keep-version-file"), | ||
"no-async": self._autorestapi.get_boolean_value("no-async"), | ||
"no-namespace-folders": self._autorestapi.get_boolean_value("no-namespace-folders"), | ||
"basic-setup-py": self._autorestapi.get_boolean_value("basic-setup-py"), | ||
"package-name": self._autorestapi.get_value("package-name"), | ||
"package-version": self._autorestapi.get_value("package-version"), | ||
"client-side-validation": self._autorestapi.get_boolean_value("client-side-validation"), | ||
"tracing": self._autorestapi.get_boolean_value("trace"), | ||
"multiapi": self._autorestapi.get_boolean_value("multiapi", False), | ||
"polymorphic-examples": self._autorestapi.get_value("polymorphic-examples"), | ||
"models-mode": self._autorestapi.get_value("models-mode"), | ||
"builders-visibility": self._autorestapi.get_value("builders-visibility"), | ||
"show-send-request": self._autorestapi.get_boolean_value("show-send-request"), | ||
"only-path-and-body-params-positional": self._autorestapi.get_boolean_value( | ||
"only-path-and-body-params-positional" | ||
), | ||
"combine-operation-files": self._autorestapi.get_boolean_value("combine-operation-files"), | ||
"package-mode": self._autorestapi.get_value("package-mode"), | ||
"package-pprint-name": self._autorestapi.get_value("package-pprint-name"), | ||
"packaging-files-config": self._autorestapi.get_value("package-configuration"), | ||
"default-optional-constants-to-none": self._autorestapi.get_boolean_value( | ||
"default-optional-constants-to-none" | ||
), | ||
"generate-sample": self._autorestapi.get_boolean_value("generate-sample"), | ||
"generate-test": self._autorestapi.get_boolean_value("generate-test"), | ||
"default-api-version": self._autorestapi.get_value("default-api-version"), | ||
} | ||
return {k: v for k, v in options.items() if v is not None} | ||
|
||
def get_yaml(self) -> Dict[str, Any]: | ||
inputs = self._autorestapi.list_inputs() | ||
_LOGGER.debug("Possible Inputs: %s", inputs) | ||
if "code-model-v4-no-tags.yaml" not in inputs: | ||
raise ValueError("code-model-v4-no-tags.yaml must be a possible input") | ||
|
||
if self._autorestapi.get_value("input-yaml"): | ||
input_yaml = self._autorestapi.get_value("input-yaml") | ||
file_content = self._autorestapi.read_file(input_yaml) | ||
else: | ||
inputs = self._autorestapi.list_inputs() | ||
_LOGGER.debug("Possible Inputs: %s", inputs) | ||
if "code-model-v4-no-tags.yaml" not in inputs: | ||
raise ValueError("code-model-v4-no-tags.yaml must be a possible input") | ||
|
||
file_content = self._autorestapi.read_file("code-model-v4-no-tags.yaml") | ||
|
||
# Parse the received YAML | ||
return yaml.safe_load(file_content) | ||
|
||
def get_serializer(self, code_model: CodeModel): | ||
return JinjaSerializerAutorest( | ||
self._autorestapi, | ||
code_model, | ||
output_folder=self.output_folder, | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# ------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for | ||
# license information. | ||
# -------------------------------------------------------------------------- | ||
"""An autorest MD to RST plugin. | ||
""" | ||
from typing import Any, Dict | ||
|
||
from pygen.m2r import M2R | ||
from . import YamlUpdatePluginAutorest | ||
|
||
|
||
class M2RAutorest(YamlUpdatePluginAutorest, M2R): | ||
def get_options(self) -> Dict[str, Any]: | ||
return {} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# ------------------------------------------------------------------------- | ||
# Copyright (c) Microsoft Corporation. All rights reserved. | ||
# Licensed under the MIT License. See License.txt in the project root for | ||
# license information. | ||
# -------------------------------------------------------------------------- | ||
from typing import Any, Dict | ||
|
||
from pygen.postprocess import PostProcessPlugin | ||
from . import PluginAutorest | ||
|
||
|
||
class PostProcessPluginAutorest(PostProcessPlugin, PluginAutorest): | ||
def get_options(self) -> Dict[str, Any]: | ||
return {"outputFolderUri": self._autorestapi.get_value("outputFolderUri")} |
Oops, something went wrong.