Skip to content

Commit

Permalink
Update changes from public (aws#61)
Browse files Browse the repository at this point in the history
* docs: fix link to all_policy_templates example file (aws#1682)

* fix: Enhance CompatibleRuntimes property of AWS::Serverless::LayerVersion (aws#1683)

* Adding Tracing property to State Machine resource (aws#1697)

* Step Functions: Adding support for X-Ray tracing

* Renaming TracingConfiguration to Tracing

Co-authored-by: Vaib Suri <surivaib@amazon.com>

* add new arguments to allow vpc/vpce endpoints as intrinsics (aws#1689)

* add new arguments to allow vpc/vpce endpoints as intrinsics

* Add more unit tests

* Add more test cases, and code changes according to comments.

* Fix: Updated Slack Invite Link (aws#1712)

* Updated Slack Invite Link

* Restricted jsonschema to Python 2

* Forced pyrsistent to 0.16 in Python 2

* Reverted Changes to enum34

* Merge master back to develop (aws#1734)

* Release v1.26.0 (aws#1680)

* feat: add support for VPCEndpointIds in EndpointConfiguration

* fix: update formatting with black

* docs: update 2016-10-31.md

* docs: added api endpointconfiguration example

* docs: make example more generic

* fix: remove nested EndpointConfiguration types from output

* fix: only allow one EndpointConfiguration Type

* doc: update example to reflect only allowing one EndpointConfiguration
Type

* fix : missing UserPool properties (aws#1506) (aws#1581)

* fix: resource policy generation for {path+} (aws#1580)

* refactor: Remove 2016-10-31 examples

* update PR template

* adjust pr template

* Adding authorization scopes as list validation in ApiGatewayAuthorizer (v1 and v2). (aws#1670)

* Adding authorization scopes as list validation in ApiGatewayAuthorizer and ApiGatewayV2Authorizer.

* make black.

* Adding functional test for invalid auth scope.

* adding error condition for invalid test.

* removing test template file.

* feat: MSK event type support for AWS::Serverless::Function (aws#52)

Co-authored-by: Steve Brown <steve@fabric.com>
Co-authored-by: jtaylor00 <joetaylor00@gmail.com>
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Alex Wood <awood45@gmail.com>
Co-authored-by: Tarun <c2tarun@users.noreply.github.com>

* Update __init__.py (aws#1704)

* Release/v1.27.0 resolveconflict (aws#1717)

* Release v1.26.0 (aws#1680)

* feat: add support for VPCEndpointIds in EndpointConfiguration

* fix: update formatting with black

* docs: update 2016-10-31.md

* docs: added api endpointconfiguration example

* docs: make example more generic

* fix: remove nested EndpointConfiguration types from output

* fix: only allow one EndpointConfiguration Type

* doc: update example to reflect only allowing one EndpointConfiguration
Type

* fix : missing UserPool properties (aws#1506) (aws#1581)

* fix: resource policy generation for {path+} (aws#1580)

* refactor: Remove 2016-10-31 examples

* update PR template

* adjust pr template

* Adding authorization scopes as list validation in ApiGatewayAuthorizer (v1 and v2). (aws#1670)

* Adding authorization scopes as list validation in ApiGatewayAuthorizer and ApiGatewayV2Authorizer.

* make black.

* Adding functional test for invalid auth scope.

* adding error condition for invalid test.

* removing test template file.

* feat: MSK event type support for AWS::Serverless::Function (aws#52)

Co-authored-by: Steve Brown <steve@fabric.com>
Co-authored-by: jtaylor00 <joetaylor00@gmail.com>
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Alex Wood <awood45@gmail.com>
Co-authored-by: Tarun <c2tarun@users.noreply.github.com>

* Fix: Updated Slack Invite Link (aws#1712)

* Updated Slack Invite Link

* Restricted jsonschema to Python 2

* Forced pyrsistent to 0.16 in Python 2

* Reverted Changes to enum34

Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com>
Co-authored-by: Steve Brown <steve@fabric.com>
Co-authored-by: jtaylor00 <joetaylor00@gmail.com>
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Alex Wood <awood45@gmail.com>
Co-authored-by: Tarun <c2tarun@users.noreply.github.com>
Co-authored-by: Cosh_ <CoshUS@users.noreply.github.com>

Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com>
Co-authored-by: Steve Brown <steve@fabric.com>
Co-authored-by: jtaylor00 <joetaylor00@gmail.com>
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Alex Wood <awood45@gmail.com>
Co-authored-by: Tarun <c2tarun@users.noreply.github.com>
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
Co-authored-by: Cosh_ <CoshUS@users.noreply.github.com>

* Lambdaauth (aws#1733)

* Add support for Lambda Authorizers in HttpAPI

* Address comments and fix formatting

* fix version

* Validate input parameters. Update tests

* black reformat

Co-authored-by: Raymond Wang <14915548+wchengru@users.noreply.github.com>

* Feature toggle (aws#1737)

* Adding logic to pipe app config providers. Unit test pending

* Adding some documentation to config providers.

* Adding some unit tests and making black ignore json files.

* minor cleanup.

* Addressing PR comments.

* feature: Support MTLS auth properties in REST and HTTP API domain names (aws#1725)

* feature: Support MTLS auth properties in REST and HTTP API domain names

* fix unicode != str issue in py2.7

* add SecurityPolicy because default RESTAPI is using TLS1.0

* Add new property DisableExecuteApiEndpoint

* black reformat

* Address comments

* Add tests on invalid templates

* address test failures in py2.7

* restart travis tests

* fix: adding support for passing target id to EventBridgeRule (aws#1747)

* Manage black version using requirement file (aws#1748)

* chore: Manage black version in dev.txt

- config pre-commit
- config development guide
- config travis

Refer to the commit below in aws-sam-cli
aws/aws-sam-cli@d725db5fbfc698a9f0c7582

* Format using black 20.8b1

* Opt-out black fron dev.txt for Python 2

Co-authored-by: Patrick Greenwell <patrickgreenwell@users.noreply.github.com>
Co-authored-by: Raymond Wang <14915548+wchengru@users.noreply.github.com>
Co-authored-by: vaib-amz <55562387+vaib-amz@users.noreply.github.com>
Co-authored-by: Vaib Suri <surivaib@amazon.com>
Co-authored-by: Cosh_ <CoshUS@users.noreply.github.com>
Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com>
Co-authored-by: Steve Brown <steve@fabric.com>
Co-authored-by: jtaylor00 <joetaylor00@gmail.com>
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Alex Wood <awood45@gmail.com>
Co-authored-by: Tarun <c2tarun@users.noreply.github.com>
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
Co-authored-by: Tolledo <ps.tolledo@gmail.com>
Co-authored-by: _sam <3804518+aahung@users.noreply.github.com>
  • Loading branch information
15 people authored Oct 27, 2020
1 parent 90a2f7d commit 4bec81e
Show file tree
Hide file tree
Showing 83 changed files with 3,368 additions and 184 deletions.
15 changes: 9 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# https://pre-commit.com/#repository-local-hooks
repos:
- repo: https://github.com/python/black
rev: 19.3b0
hooks:
- id: black
language_version: python3
exclude_types: ['markdown', 'ini', 'toml', 'rst']
- repo: local
hooks:
- id: black
name: black
entry: black
language: system
types: [python]

6 changes: 0 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,6 @@ matrix:

install:
# Install the code requirements
- mkdir $HOME/bin-black
- wget -O $HOME/bin-black/black https://github.com/python/black/releases/download/19.10b0/black
- chmod +x $HOME/bin-black/black
- export PATH=$PATH:$HOME/bin-black
- black --version

- make init

# Install Docs requirements
Expand Down
40 changes: 34 additions & 6 deletions DEVELOPMENT_GUIDE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,44 @@ Setup Python locally using `pyenv`_

2. Install Additional Tooling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Black
~~~~~~~~
Black
'''''
We format our code using `Black`_ and verify the source code is black compliant
in Appveyor during PRs. You can find installation instructions on `Black's docs`_.
in Appveyor during PRs. Black will be installed automatically with ``make init``.

After installing, you can check your formatting through our Makefile by running `make black-check`. To automatically update your code to match our formatting, please run `make black`. You can also integrate Black directly in your favorite IDE (instructions
can be found `here`_)
After installing, you can run our formatting through our Makefile by
``make black`` or integrating Black directly in your favorite IDE
(instructions can be found `here <https://black.readthedocs.io/en/stable/editor_integration.html>`__)

(workaround) Integrating Black directly in your favorite IDE
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

Since black is installed in virtualenv, when you follow `this
instruction <https://black.readthedocs.io/en/stable/editor_integration.html>`__,
``which black`` might give you this

::

(samtranslator37) $ where black
/Users/<username>/.pyenv/shims/black

However, IDEs such as PyCharm (using FileWatcher) will have a hard time
invoking ``/Users/<username>/.pyenv/shims/black`` and this will happen:

::

pyenv: black: command not found

The `black' command exists in these Python versions:
3.7.2/envs/samtranslator37
samtranslator37

A simple workaround is to use
``/Users/<username>/.pyenv/versions/samtranslator37/bin/black`` instead of
``/Users/<username>/.pyenv/shims/black``.

Pre-commit
~~~~~~~~~~
''''''''''
If you don't wish to manually run black on each pr or install black manually, we have integrated black into git hooks through `pre-commit`_.
After installing pre-commit, run `pre-commit install` in the root of the project. This will install black for you and run the black formatting on
commit.
Expand Down
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,20 @@ test:
pytest --cov samtranslator --cov-report term-missing --cov-fail-under 95 tests

black:
black setup.py samtranslator/* tests/* bin/*
black setup.py samtranslator/* tests/* bin/*.py

black-check:
black --check setup.py samtranslator/* tests/* bin/*
black --check setup.py samtranslator/* tests/* bin/*.py

# Command to run everytime you make changes to verify everything works
dev: test

# Verifications to run before sending a pull request
pr: black-check init dev

# Verifications to run before sending a pull request, skipping black check because black requires Python 3.6+
pr2.7: init dev

define HELP_MESSAGE

Usage: $ make [TARGETS]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Read the [SAM Documentation Contribution Guide](https://github.com/awsdocs/aws-s
started.

### Join the SAM Community on Slack
[Join the SAM developers channel (#samdev)](https://join.slack.com/t/awsdevelopers/shared_invite/enQtMzg3NTc5OTM2MzcxLTIxNjc0ZTJkNmYyNWY3OWE4NTFiNzU1ZTM2Y2VkNmFlNjQ2YjI3YTE1ZDA5YjE5NDE2MjVmYWFlYWIxNjE2NjU) on Slack to collaborate with fellow community members and the AWS SAM team.
[Join the SAM developers channel (#samdev)](https://join.slack.com/t/awsdevelopers/shared_invite/zt-h82odes6-qYN2Cxit7hBGIvC6oMjGpg) on Slack to collaborate with fellow community members and the AWS SAM team.



8 changes: 7 additions & 1 deletion bin/sam-translate.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
from samtranslator.translator.transform import transform
from samtranslator.yaml_helper import yaml_parse
from samtranslator.model.exceptions import InvalidDocumentException
from samtranslator.feature_toggle.feature_toggle import FeatureToggleLocalConfigProvider, FeatureToggle

LOG = logging.getLogger(__name__)
cli_options = docopt(__doc__)
Expand Down Expand Up @@ -95,7 +96,12 @@ def transform_template(input_file_path, output_file_path):
sam_template = yaml_parse(f)

try:
cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client))
feature_toggle = FeatureToggle(
FeatureToggleLocalConfigProvider(
os.path.join(my_path, "..", "tests", "feature_toggle", "input", "feature_toggle_config.json")
)
)
cloud_formation_template = transform(sam_template, {}, ManagedPolicyLoader(iam_client), feature_toggle)
cloud_formation_template_prettified = json.dumps(cloud_formation_template, indent=2)

with open(output_file_path, "w") as f:
Expand Down
2 changes: 1 addition & 1 deletion docs/policy_templates.rst
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,4 @@ folder.
- CloudWatchPutMetricPolicy: {}
.. _policy_templates.json: https://github.com/awslabs/serverless-application-model/blob/develop/samtranslator/policy_templates_data/policy_templates.json
.. _all_policy_templates.yaml: https://github.com/awslabs/serverless-application-model/blob/develop/examples/2016-10-31/policy_templates/all_policy_templates.yaml
.. _all_policy_templates.yaml: https://github.com/awslabs/serverless-application-model/blob/develop/tests/translator/input/all_policy_templates.yaml
1 change: 1 addition & 0 deletions requirements/base.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
pyrsistent~=0.16.0; python_version<"3"
boto3~=1.5
enum34~=1.1; python_version<"3.4"
jsonschema~=3.0
Expand Down
3 changes: 3 additions & 0 deletions requirements/dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@ requests>=2.20.0

# CLI requirements
docopt>=0.6.2

# formatter
black==20.8b1; python_version >= '3.6'
2 changes: 1 addition & 1 deletion samtranslator/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.26.0"
__version__ = "1.27.0"
Empty file.
125 changes: 125 additions & 0 deletions samtranslator/feature_toggle/feature_toggle.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
import os
import sys
import json
import boto3
import logging

my_path = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, my_path + "/..")

LOG = logging.getLogger(__name__)


class FeatureToggle:
"""
FeatureToggle is the class which will provide methods to query and decide if a feature is enabled based on where
SAM is executing or not.
"""

def __init__(self, config_provider):
self.feature_config = config_provider.config

def is_enabled_for_stage_in_region(self, feature_name, stage, region="default"):
"""
To check if feature is available for a particular stage or not.
:param feature_name: name of feature
:param stage: stage where SAM is running
:param region: region in which SAM is running
:return:
"""
if feature_name not in self.feature_config:
LOG.warning("Feature '{}' not available in Feature Toggle Config.".format(feature_name))
return False
stage_config = self.feature_config.get(feature_name, {}).get(stage, {})
if not stage_config:
LOG.info("Stage '{}' not enabled for Feature '{}'.".format(stage, feature_name))
return False
region_config = stage_config.get(region, {}) if region in stage_config else stage_config.get("default", {})
is_enabled = region_config.get("enabled", False)
LOG.info("Feature '{}' is enabled: '{}'".format(feature_name, is_enabled))
return is_enabled

def is_enabled_for_account_in_region(self, feature_name, stage, account_id, region="default"):
"""
To check if feature is available for a particular account or not.
:param feature_name: name of feature
:param stage: stage where SAM is running
:param account_id: account_id who is executing SAM template
:param region: region in which SAM is running
:return:
"""
if feature_name not in self.feature_config:
LOG.warning("Feature '{}' not available in Feature Toggle Config.".format(feature_name))
return False
stage_config = self.feature_config.get(feature_name, {}).get(stage, {})
if not stage_config:
LOG.info("Stage '{}' not enabled for Feature '{}'.".format(stage, feature_name))
return False
account_config = stage_config.get(account_id) if account_id in stage_config else stage_config.get("default", {})
region_config = (
account_config.get(region, {}) if region in account_config else account_config.get("default", {})
)
is_enabled = region_config.get("enabled", False)
LOG.info("Feature '{}' is enabled: '{}'".format(feature_name, is_enabled))
return is_enabled


class FeatureToggleConfigProvider:
"""Interface for all FeatureToggle config providers"""

def __init__(self):
pass

@property
def config(self):
raise NotImplementedError


class FeatureToggleDefaultConfigProvider(FeatureToggleConfigProvider):
"""Default config provider, always return False for every query."""

def __init__(self):
FeatureToggleConfigProvider.__init__(self)

@property
def config(self):
return {}


class FeatureToggleLocalConfigProvider(FeatureToggleConfigProvider):
"""Feature toggle config provider which uses a local file. This is to facilitate local testing."""

def __init__(self, local_config_path):
FeatureToggleConfigProvider.__init__(self)
with open(local_config_path, "r") as f:
config_json = f.read()
self.feature_toggle_config = json.loads(config_json)

@property
def config(self):
return self.feature_toggle_config


class FeatureToggleAppConfigConfigProvider(FeatureToggleConfigProvider):
"""Feature toggle config provider which loads config from AppConfig."""

def __init__(self, application_id, environment_id, configuration_profile_id):
FeatureToggleConfigProvider.__init__(self)
self.app_config_client = boto3.client("appconfig")
try:
response = self.app_config_client.get_configuration(
Application=application_id,
Environment=environment_id,
Configuration=configuration_profile_id,
ClientId="FeatureToggleAppConfigConfigProvider",
)
binary_config_string = response["Content"].read()
self.feature_toggle_config = json.loads(binary_config_string.decode("utf-8"))
except Exception as ex:
LOG.error("Failed to load config from AppConfig: {}. Using empty config.".format(ex))
# There is chance that AppConfig is not available in a particular region.
self.feature_toggle_config = json.loads("{}")

@property
def config(self):
return self.feature_toggle_config
3 changes: 1 addition & 2 deletions samtranslator/model/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,8 +355,7 @@ def to_cloudformation(self, **kwargs):


class SamResourceMacro(ResourceMacro):
"""ResourceMacro that specifically refers to SAM (AWS::Serverless::*) resources.
"""
"""ResourceMacro that specifically refers to SAM (AWS::Serverless::*) resources."""

# SAM resources can provide a list of properties that they expose. These properties usually resolve to
# CFN resources that this SAM resource generates. This is provided as a map with the following format:
Expand Down
31 changes: 31 additions & 0 deletions samtranslator/model/api/api_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,37 @@ def _construct_api_domain(self, rest_api):

domain.EndpointConfiguration = {"Types": [endpoint]}

mutual_tls_auth = self.domain.get("MutualTlsAuthentication", None)
if mutual_tls_auth:
if isinstance(mutual_tls_auth, dict):
if not set(mutual_tls_auth.keys()).issubset({"TruststoreUri", "TruststoreVersion"}):
invalid_keys = list()
for key in mutual_tls_auth.keys():
if not key in {"TruststoreUri", "TruststoreVersion"}:
invalid_keys.append(key)
invalid_keys.sort()
raise InvalidResourceException(
",".join(invalid_keys),
"Available MutualTlsAuthentication fields are {}.".format(
["TruststoreUri", "TruststoreVersion"]
),
)
domain.MutualTlsAuthentication = {}
if mutual_tls_auth.get("TruststoreUri", None):
domain.MutualTlsAuthentication["TruststoreUri"] = mutual_tls_auth["TruststoreUri"]
if mutual_tls_auth.get("TruststoreVersion", None):
domain.MutualTlsAuthentication["TruststoreVersion"] = mutual_tls_auth["TruststoreVersion"]
else:
raise InvalidResourceException(
mutual_tls_auth,
"MutualTlsAuthentication must be a map with at least one of the following fields {}.".format(
["TruststoreUri", "TruststoreVersion"]
),
)

if self.domain.get("SecurityPolicy", None):
domain.SecurityPolicy = self.domain["SecurityPolicy"]

# Create BasepathMappings
if self.domain.get("BasePath") and isinstance(self.domain.get("BasePath"), string_types):
basepaths = [self.domain.get("BasePath")]
Expand Down
Loading

0 comments on commit 4bec81e

Please sign in to comment.