Glue_version and python_version not working in terraform

[vickymca2005]

Hellow everyone,
I am using terraform to create the glue job. Now AWS Glue now supports the ability to run ETL jobs on Apache Spark 2.4.3 (with Python 3).
I want to use this feature. but whenever i am making changes it is throwing error.
I am using
aws-cli/1.16.184.
Terraform v0.12.6
aws provider 2.29

I want to use below terraform configuration.

resource "aws_glue_job" "aws_glue_job_foo" {
glue_version = "1"
name = "job-name"
description = "job-desc"
role_arn = data.aws_iam_role.aws_glue_iam_role.arn
max_capacity = 1
max_retries = 1
connections = [aws_glue_connection.connection.name]
timeout = 5

command {
name = "pythonshell"
script_location = "s3://bucket/script.py"
python_version = "3"
}

default_arguments = {
"--job-language" = "python"
"--ENV" = "env"
"--ROLE_ARN" = data.aws_iam_role.aws_glue_iam_role.arn
}

execution_property {
max_concurrent_runs = 1
}
}

But it is throwing error to me,

Error: Unsupported argument
An argument named "glue_version" is not expected here.

[vickymca2005]

If I remove glue_version it update the Python 3 but whenever ETL jobs run it throw error that Python 3 can not run with glue .9. I am not sure is there any way to update glue version?

[ewbankkit]

I think this is the same as #9524 which is addressed via #10237.

[vickymca2005]

@ewbankkit it seems it is resolved but why I am getting error? Do I need to update my provider or anything I am missing here.

[ewbankkit]

@vickymca2005 That PR is not yet merged.

[vickymca2005]

@vickymca2005 That PR is not yet merged.

@ewbankkit Thanks for the reply. When should i expect this to be available?

[bflad]

Hi folks 👋 As mentioned before, this issue appears to be the same as being reported in #9524 with the associated new argument being added in #10237. To consolidate efforts and discussions, I'm going to close this issue in preference of the earlier ones.

When should i expect this to be available?

I just provided a review on that pull request and it has one item of feedback before it can be merged so it doesn't break existing configurations. I would anticipate it will get merged before our next release, version 2.33.0 on Thursday next week, but please follow #9524 and #10237 for updates (I will not be posting more here).

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!