Feature/aws improvements #336
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nd easy to update
Lambda node by default uses "invoke" instead of S3. Includes handler improvements to consolidate the "pipeline" data model that is passed amongst the various triggers. Data is passed back and forth via netcdf. API Gateway and S3 triggers still needs testing. In the future, I think we should deprecate the "eval_s3" and "eval_api" methods since these are no longer relevant. We should keep the ability to add extra triggers for S3 and APIGateway since there may be other use cases that need this, but the PODPAC only evaluation will be done through invoke.
dsully-dev
reviewed
Nov 15, 2019
| @@ -173,6 +181,13 @@ def _function_name_default(self): | |||
|
|
|||
| return settings["FUNCTION_NAME"] | |||
|
|
|||
| @tl.default("function_triggers") | |||
There was a problem hiding this comment.
@mlshapiro This block seems to indicate I must set function_eval_trigger to 'APIGateway' in order to build the api gateway. More generally, it is not obvious to me how to specify that I want optional resources built (e.g. API Gateway).
dsully-dev
reviewed
Nov 18, 2019
dist/aws/handler.py
Outdated
|
|
||
| pipeline = default_pipeline() | ||
| pipeline["url"] = event["queryStringParameters"] | ||
| pipeline["params"] = urllib.parse_qs(urllib.urlparse(pipeline["url"]).query) |
There was a problem hiding this comment.
We actually don't need to do any URL parsing, because event['queryStringParameters'] is a dict with all of the params already parsed.
dsully-dev
reviewed
Nov 18, 2019
|
|
||
| url = event["queryStringParameters"] |
There was a problem hiding this comment.
It looks like it was in the porting of this block that we lost the isinstance check.
The `queryStringParameters` are already a dict, so they don't need parsed with `urllib`. This also merges any settings passed in the event with the default pipeline, then again with the settings on the AWS instance.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
function_restrict_pipelines. This uses an environment variable (PODPAC_RESTRICT_PIPELINES) in the lambda function to store the hashes of acceptable Nodes. The handler checks to see if the node hash is in the environment variable and raises a ValueError if not.API Gateway and S3 triggers still needs testing. Handler may need to be updated for these as well.
In the future, I think we should deprecate the "eval_s3" and "eval_api" methods on the Lambda node since these are no longer relevant. We should keep the ability to add extra triggers for S3 and APIGateway since there are other use cases that need these.
To Do:
response["LogResult"]to get the "REPORT" which contains billing informationrequestslibrary.