Removes a managed scaling policy from a specified Amazon EMR cluster.
", "RemoveTags": "Removes tags from an Amazon EMR resource, such as a cluster or Amazon EMR Studio. Tags make it easier to associate resources in various ways, such as grouping clusters to track your Amazon EMR resource allocation costs. For more information, see Tag Clusters.
The following example removes the stack tag with value Prod from a cluster:
", "RunJobFlow": "RunJobFlow creates and starts running a new cluster (job flow). The cluster runs the steps specified. After the steps complete, the cluster stops and the HDFS partition is lost. To prevent loss of data, configure the last step of the job flow to store results in Amazon S3. If the JobFlowInstancesConfig KeepJobFlowAliveWhenNoSteps parameter is set to TRUE, the cluster transitions to the WAITING state rather than shutting down after the steps have completed.
For additional protection, you can set the JobFlowInstancesConfig TerminationProtected parameter to TRUE to lock the cluster and prevent it from being terminated by API call, user intervention, or in the event of a job flow error.
A maximum of 256 steps are allowed in each job flow.
If your cluster is long-running (such as a Hive data warehouse) or complex, you may require more than 256 steps to process your data. You can bypass the 256-step limitation in various ways, including using the SSH shell to connect to the master node and submitting queries directly to the software running on the master node, such as Hive and Hadoop.
For long-running clusters, we recommend that you periodically store your results.
The instance fleets configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions. The RunJobFlow request can contain InstanceFleets parameters or InstanceGroups parameters, but not both.
You can use the SetKeepJobFlowAliveWhenNoSteps to configure a cluster (job flow) to terminate after the step execution, i.e., all your steps are executed. If you want a transient cluster that shuts down after the last of the current executing steps are completed, you can configure SetKeepJobFlowAliveWhenNoSteps to false. If you want a long running cluster, configure SetKeepJobFlowAliveWhenNoSteps to true.
For more information, see Managing Cluster Termination in the Amazon EMR Management Guide.
", "SetTerminationProtection": "SetTerminationProtection locks a cluster (job flow) so the Amazon EC2 instances in the cluster cannot be terminated by user intervention, an API call, or in the event of a job-flow error. The cluster still terminates upon successful completion of the job flow. Calling SetTerminationProtection on a cluster is similar to calling the Amazon EC2 DisableAPITermination API on all Amazon EC2 instances in a cluster.
SetTerminationProtection is used to prevent accidental termination of a cluster and to ensure that in the event of an error, the instances persist so that you can recover any data stored in their ephemeral instance storage.
To terminate a cluster that has been locked by setting SetTerminationProtection to true, you must first unlock the job flow by a subsequent call to SetTerminationProtection in which you set the value to false.
For more information, seeManaging Cluster Termination in the Amazon EMR Management Guide.
", "SetVisibleToAllUsers": "The SetVisibleToAllUsers parameter is no longer supported. Your cluster may be visible to all users in your account. To restrict cluster access using an IAM policy, see Identity and Access Management for Amazon EMR.
Sets the Cluster$VisibleToAllUsers value for an Amazon EMR cluster. When true, IAM principals in the Amazon Web Services account can perform Amazon EMR cluster actions that their IAM policies allow. When false, only the IAM principal that created the cluster and the Amazon Web Services account root user can perform Amazon EMR actions on the cluster, regardless of IAM permissions policies attached to other IAM principals.
This action works on running clusters. When you create a cluster, use the RunJobFlowInput$VisibleToAllUsers parameter.
For more information, see Understanding the Amazon EMR Cluster VisibleToAllUsers Setting in the Amazon EMR Management Guide.
", "StartNotebookExecution": "Starts a notebook execution.
", @@ -221,6 +222,7 @@ "JobFlowInstancesDetail$KeepJobFlowAliveWhenNoSteps": "Specifies whether the cluster should remain available after completing all steps.
", "JobFlowInstancesDetail$TerminationProtected": "Specifies whether the Amazon EC2 instances in the cluster are protected from termination by API calls, user intervention, or in the event of a job-flow error.
", "RunJobFlowInput$VisibleToAllUsers": "The VisibleToAllUsers parameter is no longer supported. By default, the value is set to true. Setting it to false now has no effect.
Set this value to true so that IAM principals in the Amazon Web Services account associated with the cluster can perform Amazon EMR actions on the cluster that their IAM policies allow. This value defaults to true for clusters created using the Amazon EMR API or the CLI create-cluster command.
When set to false, only the IAM principal that created the cluster and the Amazon Web Services account root user can perform Amazon EMR actions for the cluster, regardless of the IAM permissions policies attached to other IAM principals. For more information, see Understanding the Amazon EMR cluster VisibleToAllUsers setting in the Amazon EMR Management Guide.
A Boolean that indicates whether to terminate the cluster after all steps are executed.
", "SetTerminationProtectionInput$TerminationProtected": "A Boolean that indicates whether to protect the cluster and prevent the Amazon EC2 instances in the cluster from shutting down due to API calls, user intervention, or job-flow error.
", "SetVisibleToAllUsersInput$VisibleToAllUsers": "A value of true indicates that an IAM principal in the Amazon Web Services account can perform Amazon EMR actions on the cluster that the IAM policies attached to the principal allow. A value of false indicates that only the IAM principal that created the cluster and the Amazon Web Services root user can perform Amazon EMR actions on the cluster.
Indicates whether the SupportedInstanceType only supports 64-bit architecture.
A list of session mapping summary objects. Each object includes session mapping details such as creation time, identity type (user or group), and Amazon EMR Studio ID.
" } }, + "SetKeepJobFlowAliveWhenNoStepsInput": { + "base": null, + "refs": { + } + }, "SetTerminationProtectionInput": { "base": "The input argument to the TerminationProtection operation.
", "refs": { @@ -2345,6 +2352,7 @@ "HadoopJarStepConfig$Args": "A list of command line arguments passed to the JAR file's main function when executed.
", "ListStepsInput$StepIds": "The filter to limit the step list based on the identifier of the steps. You can specify a maximum of ten Step IDs. The character constraint applies to the overall length of the array.
", "ScriptBootstrapActionConfig$Args": "A list of command line arguments to pass to the bootstrap action script.
", + "SetKeepJobFlowAliveWhenNoStepsInput$JobFlowIds": "A list of strings that uniquely identify the clusters to protect. This identifier is returned by RunJobFlow and can also be obtained from DescribeJobFlows.
", "SetTerminationProtectionInput$JobFlowIds": "A list of strings that uniquely identify the clusters to protect. This identifier is returned by RunJobFlow and can also be obtained from DescribeJobFlows .
", "SetVisibleToAllUsersInput$JobFlowIds": "The unique identifier of the job flow (cluster).
", "SupportedProductConfig$Args": "The list of user-supplied arguments.
", diff --git a/models/endpoints/endpoints.json b/models/endpoints/endpoints.json index 18e48ec0050..6d994032418 100644 --- a/models/endpoints/endpoints.json +++ b/models/endpoints/endpoints.json @@ -5467,7 +5467,12 @@ "tags" : [ "fips" ] } ] }, - "ca-west-1" : { }, + "ca-west-1" : { + "variants" : [ { + "hostname" : "ds-fips.ca-west-1.amazonaws.com", + "tags" : [ "fips" ] + } ] + }, "eu-central-1" : { }, "eu-central-2" : { }, "eu-north-1" : { }, @@ -5483,6 +5488,13 @@ "deprecated" : true, "hostname" : "ds-fips.ca-central-1.amazonaws.com" }, + "fips-ca-west-1" : { + "credentialScope" : { + "region" : "ca-west-1" + }, + "deprecated" : true, + "hostname" : "ds-fips.ca-west-1.amazonaws.com" + }, "fips-us-east-1" : { "credentialScope" : { "region" : "us-east-1" @@ -13746,6 +13758,19 @@ "deprecated" : true, "hostname" : "rds-fips.ca-central-1.amazonaws.com" }, + "ca-west-1" : { + "variants" : [ { + "hostname" : "rds-fips.ca-west-1.amazonaws.com", + "tags" : [ "fips" ] + } ] + }, + "ca-west-1-fips" : { + "credentialScope" : { + "region" : "ca-west-1" + }, + "deprecated" : true, + "hostname" : "rds-fips.ca-west-1.amazonaws.com" + }, "eu-central-1" : { }, "eu-central-2" : { }, "eu-north-1" : { }, @@ -13764,6 +13789,13 @@ "deprecated" : true, "hostname" : "rds-fips.ca-central-1.amazonaws.com" }, + "rds-fips.ca-west-1" : { + "credentialScope" : { + "region" : "ca-west-1" + }, + "deprecated" : true, + "hostname" : "rds-fips.ca-west-1.amazonaws.com" + }, "rds-fips.us-east-1" : { "credentialScope" : { "region" : "us-east-1" @@ -13802,6 +13834,16 @@ "tags" : [ "fips" ] } ] }, + "rds.ca-west-1" : { + "credentialScope" : { + "region" : "ca-west-1" + }, + "deprecated" : true, + "variants" : [ { + "hostname" : "rds-fips.ca-west-1.amazonaws.com", + "tags" : [ "fips" ] + } ] + }, "rds.us-east-1" : { "credentialScope" : { "region" : "us-east-1" diff --git a/service/emr/api.go b/service/emr/api.go index 61b2b1ba12d..94f21be74bb 100644 --- a/service/emr/api.go +++ b/service/emr/api.go @@ -4675,6 +4675,94 @@ func (c *EMR) RunJobFlowWithContext(ctx aws.Context, input *RunJobFlowInput, opt return out, req.Send() } +const opSetKeepJobFlowAliveWhenNoSteps = "SetKeepJobFlowAliveWhenNoSteps" + +// SetKeepJobFlowAliveWhenNoStepsRequest generates a "aws/request.Request" representing the +// client's request for the SetKeepJobFlowAliveWhenNoSteps operation. The "output" return +// value will be populated with the request's response once the request completes +// successfully. +// +// Use "Send" method on the returned Request to send the API call to the service. +// the "output" return value is not valid until after Send returns without error. +// +// See SetKeepJobFlowAliveWhenNoSteps for more information on using the SetKeepJobFlowAliveWhenNoSteps +// API call, and error handling. +// +// This method is useful when you want to inject custom logic or configuration +// into the SDK's request lifecycle. Such as custom headers, or retry logic. +// +// // Example sending a request using the SetKeepJobFlowAliveWhenNoStepsRequest method. +// req, resp := client.SetKeepJobFlowAliveWhenNoStepsRequest(params) +// +// err := req.Send() +// if err == nil { // resp is now filled +// fmt.Println(resp) +// } +// +// See also, https://docs.aws.amazon.com/goto/WebAPI/elasticmapreduce-2009-03-31/SetKeepJobFlowAliveWhenNoSteps +func (c *EMR) SetKeepJobFlowAliveWhenNoStepsRequest(input *SetKeepJobFlowAliveWhenNoStepsInput) (req *request.Request, output *SetKeepJobFlowAliveWhenNoStepsOutput) { + op := &request.Operation{ + Name: opSetKeepJobFlowAliveWhenNoSteps, + HTTPMethod: "POST", + HTTPPath: "/", + } + + if input == nil { + input = &SetKeepJobFlowAliveWhenNoStepsInput{} + } + + output = &SetKeepJobFlowAliveWhenNoStepsOutput{} + req = c.newRequest(op, input, output) + req.Handlers.Unmarshal.Swap(jsonrpc.UnmarshalHandler.Name, protocol.UnmarshalDiscardBodyHandler) + return +} + +// SetKeepJobFlowAliveWhenNoSteps API operation for Amazon EMR. +// +// You can use the SetKeepJobFlowAliveWhenNoSteps to configure a cluster (job +// flow) to terminate after the step execution, i.e., all your steps are executed. +// If you want a transient cluster that shuts down after the last of the current +// executing steps are completed, you can configure SetKeepJobFlowAliveWhenNoSteps +// to false. If you want a long running cluster, configure SetKeepJobFlowAliveWhenNoSteps +// to true. +// +// For more information, see Managing Cluster Termination (https://docs.aws.amazon.com/emr/latest/ManagementGuide/UsingEMR_TerminationProtection.html) +// in the Amazon EMR Management Guide. +// +// Returns awserr.Error for service API and SDK errors. Use runtime type assertions +// with awserr.Error's Code and Message methods to get detailed information about +// the error. +// +// See the AWS API reference guide for Amazon EMR's +// API operation SetKeepJobFlowAliveWhenNoSteps for usage and error information. +// +// Returned Error Types: +// - InternalServerError +// Indicates that an error occurred while processing the request and that the +// request was not completed. +// +// See also, https://docs.aws.amazon.com/goto/WebAPI/elasticmapreduce-2009-03-31/SetKeepJobFlowAliveWhenNoSteps +func (c *EMR) SetKeepJobFlowAliveWhenNoSteps(input *SetKeepJobFlowAliveWhenNoStepsInput) (*SetKeepJobFlowAliveWhenNoStepsOutput, error) { + req, out := c.SetKeepJobFlowAliveWhenNoStepsRequest(input) + return out, req.Send() +} + +// SetKeepJobFlowAliveWhenNoStepsWithContext is the same as SetKeepJobFlowAliveWhenNoSteps with the addition of +// the ability to pass a context and additional request options. +// +// See SetKeepJobFlowAliveWhenNoSteps for details on how to use this API operation. +// +// The context must be non-nil and will be used for request cancellation. If +// the context is nil a panic will occur. In the future the SDK may create +// sub-contexts for http.Requests. See https://golang.org/pkg/context/ +// for more information on using Contexts. +func (c *EMR) SetKeepJobFlowAliveWhenNoStepsWithContext(ctx aws.Context, input *SetKeepJobFlowAliveWhenNoStepsInput, opts ...request.Option) (*SetKeepJobFlowAliveWhenNoStepsOutput, error) { + req, out := c.SetKeepJobFlowAliveWhenNoStepsRequest(input) + req.SetContext(ctx) + req.ApplyOptions(opts...) + return out, req.Send() +} + const opSetTerminationProtection = "SetTerminationProtection" // SetTerminationProtectionRequest generates a "aws/request.Request" representing the @@ -17444,6 +17532,91 @@ func (s *SessionMappingSummary) SetStudioId(v string) *SessionMappingSummary { return s } +type SetKeepJobFlowAliveWhenNoStepsInput struct { + _ struct{} `type:"structure"` + + // A list of strings that uniquely identify the clusters to protect. This identifier + // is returned by RunJobFlow (https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html) + // and can also be obtained from DescribeJobFlows (https://docs.aws.amazon.com/emr/latest/APIReference/API_DescribeJobFlows.html). + // + // JobFlowIds is a required field + JobFlowIds []*string `type:"list" required:"true"` + + // A Boolean that indicates whether to terminate the cluster after all steps + // are executed. + // + // KeepJobFlowAliveWhenNoSteps is a required field + KeepJobFlowAliveWhenNoSteps *bool `type:"boolean" required:"true"` +} + +// String returns the string representation. +// +// API parameter values that are decorated as "sensitive" in the API will not +// be included in the string output. The member name will be present, but the +// value will be replaced with "sensitive". +func (s SetKeepJobFlowAliveWhenNoStepsInput) String() string { + return awsutil.Prettify(s) +} + +// GoString returns the string representation. +// +// API parameter values that are decorated as "sensitive" in the API will not +// be included in the string output. The member name will be present, but the +// value will be replaced with "sensitive". +func (s SetKeepJobFlowAliveWhenNoStepsInput) GoString() string { + return s.String() +} + +// Validate inspects the fields of the type to determine if they are valid. +func (s *SetKeepJobFlowAliveWhenNoStepsInput) Validate() error { + invalidParams := request.ErrInvalidParams{Context: "SetKeepJobFlowAliveWhenNoStepsInput"} + if s.JobFlowIds == nil { + invalidParams.Add(request.NewErrParamRequired("JobFlowIds")) + } + if s.KeepJobFlowAliveWhenNoSteps == nil { + invalidParams.Add(request.NewErrParamRequired("KeepJobFlowAliveWhenNoSteps")) + } + + if invalidParams.Len() > 0 { + return invalidParams + } + return nil +} + +// SetJobFlowIds sets the JobFlowIds field's value. +func (s *SetKeepJobFlowAliveWhenNoStepsInput) SetJobFlowIds(v []*string) *SetKeepJobFlowAliveWhenNoStepsInput { + s.JobFlowIds = v + return s +} + +// SetKeepJobFlowAliveWhenNoSteps sets the KeepJobFlowAliveWhenNoSteps field's value. +func (s *SetKeepJobFlowAliveWhenNoStepsInput) SetKeepJobFlowAliveWhenNoSteps(v bool) *SetKeepJobFlowAliveWhenNoStepsInput { + s.KeepJobFlowAliveWhenNoSteps = &v + return s +} + +type SetKeepJobFlowAliveWhenNoStepsOutput struct { + _ struct{} `type:"structure"` +} + +// String returns the string representation. +// +// API parameter values that are decorated as "sensitive" in the API will not +// be included in the string output. The member name will be present, but the +// value will be replaced with "sensitive". +func (s SetKeepJobFlowAliveWhenNoStepsOutput) String() string { + return awsutil.Prettify(s) +} + +// GoString returns the string representation. +// +// API parameter values that are decorated as "sensitive" in the API will not +// be included in the string output. The member name will be present, but the +// value will be replaced with "sensitive". +func (s SetKeepJobFlowAliveWhenNoStepsOutput) GoString() string { + return s.String() +} + // The input argument to the TerminationProtection operation. type SetTerminationProtectionInput struct { _ struct{} `type:"structure"` diff --git a/service/emr/emriface/interface.go b/service/emr/emriface/interface.go index cbc96ed4e4f..83f837afd10 100644 --- a/service/emr/emriface/interface.go +++ b/service/emr/emriface/interface.go @@ -284,6 +284,10 @@ type EMRAPI interface { RunJobFlowWithContext(aws.Context, *emr.RunJobFlowInput, ...request.Option) (*emr.RunJobFlowOutput, error) RunJobFlowRequest(*emr.RunJobFlowInput) (*request.Request, *emr.RunJobFlowOutput) + SetKeepJobFlowAliveWhenNoSteps(*emr.SetKeepJobFlowAliveWhenNoStepsInput) (*emr.SetKeepJobFlowAliveWhenNoStepsOutput, error) + SetKeepJobFlowAliveWhenNoStepsWithContext(aws.Context, *emr.SetKeepJobFlowAliveWhenNoStepsInput, ...request.Option) (*emr.SetKeepJobFlowAliveWhenNoStepsOutput, error) + SetKeepJobFlowAliveWhenNoStepsRequest(*emr.SetKeepJobFlowAliveWhenNoStepsInput) (*request.Request, *emr.SetKeepJobFlowAliveWhenNoStepsOutput) + SetTerminationProtection(*emr.SetTerminationProtectionInput) (*emr.SetTerminationProtectionOutput, error) SetTerminationProtectionWithContext(aws.Context, *emr.SetTerminationProtectionInput, ...request.Option) (*emr.SetTerminationProtectionOutput, error) SetTerminationProtectionRequest(*emr.SetTerminationProtectionInput) (*request.Request, *emr.SetTerminationProtectionOutput)