Bgd 5677 spark operator security patching #25
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@sigmarkarl, please, add screenshots from the integration test results |
There was a problem hiding this comment.
- what about this finding https://netappcloud.armorcode.com/#/u/389/2075/findings/389/2075/877512950
- please, add screenshots from integration tests
This was referenced Aug 11, 2024
alextarasov-spot
approved these changes
Aug 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira ticket
https://spotinst.atlassian.net/browse/BGD-5677
Description
Update libraries to fix security vulnerabilities
Demo
Not relevant, no functional change
Checklist
How to test
No functional change, run integration tests
Test plan and results
The integration tests ran successfully
4:11PM INF Checking cluster availability clusterId=osc-739db584 4:11PM INF Cluster is available, tests are ready to run clusterId=osc-739db584 4:11PM INF Start running tests 4:11PM INF Optimizer Url: https://api.spotinst.io 4:11PM INF accountId: act-12f6b1b9 4:11PM INF clusterId: osc-739db584 4:11PM INF sparkProps: [{Version:3.5.0 Image:}] 4:11PM INF 20 tests selected: [list_applications, kill_application_pending_3.5.0, kill_application_running_3.5.0, kill_application_completed_3.5.0, list_jobs, get_job, put_job, list_clusters, get_config_template, create_valid_config_template, create_invalid_config_template, list_config_templates, put_config_templates, delete_config_template, run_workspace, run_spark_connect_application-3.5.0, run_spark_connect_application_code-3.5.0, run_notebook_application-Python, run_notebook_application-Scala, run_application-3.5.0] 4:11PM INF Starting suite=list_jobs 4:11PM INF Starting suite=list_applications 4:11PM INF Starting suite=put_job 4:11PM INF Starting suite=get_job 4:11PM INF Starting suite=run_workspace 4:11PM INF Starting suite=list_config_templates 4:11PM INF Workspace creation test suite=run_workspace 4:11PM INF Starting suite=run_spark_connect_application_code-3.5.0 4:11PM INF Testing spark connect app submit with properties {3.5.0 } suite=run_spark_connect_application_code-3.5.0 4:11PM INF Starting suite=get_config_template 4:11PM INF Starting suite=run_notebook_application-Scala 4:11PM INF Starting suite=run_spark_connect_application-3.5.0 4:11PM INF Starting suite=list_clusters 4:11PM INF Starting suite=kill_application_completed_3.5.0 4:11PM INF Starting suite=create_valid_config_template 4:11PM INF Trying to create config template integration-tests-config-template-create-ffwdnv suite=create_valid_config_template 4:11PM INF Starting suite=delete_config_template 4:11PM INF Starting suite=run_notebook_application-Python 4:11PM INF Creating config Scala template. suite=run_notebook_application-Scala 4:11PM INF Starting suite=kill_application_running_3.5.0 4:11PM INF Starting suite=put_config_templates 4:11PM INF Starting suite=create_invalid_config_template 4:11PM INF Testing spark connect app submit with properties {3.5.0 } suite=run_spark_connect_application-3.5.0 4:11PM INF Creating config Python template. suite=run_notebook_application-Python 4:11PM INF Starting suite=run_application-3.5.0 4:11PM INF Testing Spark submit with properties {3.5.0 } suite=run_application-3.5.0 4:11PM INF Trying to create invalid config template integration-tests-config-template-create-oytejp suite=create_invalid_config_template 4:11PM INF Successfully received unprocessable entity response for integration-tests-config-template-create-oytejp suite=create_invalid_config_template 4:11PM INF Success suite=create_invalid_config_template 4:11PM INF Config template created successfully suite=get_config_template 4:11PM INF Config template created successfully suite=put_config_templates 4:11PM INF Config template created successfully suite=delete_config_template 4:11PM INF Success suite=list_clusters 4:11PM INF Config template created successfully suite=list_config_templates 4:11PM INF Testing Spark submit with properties Python suite=run_notebook_application-Python 4:11PM INF Testing Spark submit with properties Scala suite=run_notebook_application-Scala 4:11PM INF Workspace created: wksp-170654-9ef01fbb suite=run_workspace 4:11PM INF Success suite=list_jobs 4:11PM INF Config template successfully updated suite=put_config_templates 4:11PM INF Submitted app integration-tests-8d6aa-smear suite=kill_application_running_3.5.0 4:11PM INF Submitted spark connect app suite=run_spark_connect_application-3.5.0 4:11PM INF Submitted app integration-tests-18d47-grate suite=kill_application_completed_3.5.0 4:11PM INF Success suite=list_applications 4:11PM INF Submitted app suite=run_application-3.5.0 4:11PM INF Submitted app integration-tests-job-put-xfxcsa-cdba3-bales suite=put_job 4:11PM INF App is processed suite=kill_application_running_3.5.0 4:11PM INF App is processed suite=kill_application_completed_3.5.0 4:11PM INF Created config template integration-tests-config-template-create-ffwdnv suite=create_valid_config_template 4:11PM INF Trying to create config template with same name integration-tests-config-template-create-ffwdnv suite=create_valid_config_template 4:11PM INF Submitted app integration-tests-job-get-pikkxo-effc6-lauds suite=get_job 4:11PM INF App is processed suite=run_application-3.5.0 4:11PM INF Job config has updated suite=put_job 4:11PM INF Updated job labels are correct suite=put_job 4:11PM INF Job fetched successfully suite=get_job 4:11PM INF Got a correct job suite=get_job 4:11PM INF Successfully received bad request response for integration-tests-config-template-create-ffwdnv suite=create_valid_config_template 4:11PM INF Success suite=create_valid_config_template 4:11PM INF Trying to list the config templates suite=list_config_templates 4:11PM INF Config template fetched successfully suite=get_config_template 4:11PM INF Got a correct config template suite=get_config_template 4:11PM INF Success suite=get_config_template 4:11PM INF Config template successfully deleted suite=delete_config_template 4:11PM INF Success suite=delete_config_template 4:11PM INF Config template fetched successfully suite=put_config_templates 4:11PM INF Got a correct config template suite=put_config_templates 4:11PM INF Config template fetched successfully suite=list_config_templates 4:11PM INF Got a correct config template list suite=list_config_templates 4:11PM INF Success suite=put_config_templates 4:11PM INF Success suite=list_config_templates 4:11PM INF Live kubernetes logs are available suite=run_application-3.5.0 4:11PM INF Submitted spark connect app suite=run_spark_connect_application_code-3.5.0 4:12PM INF Workspace is running: wksp-170654-9ef01fbb suite=run_workspace 4:12PM INF Workspace killed: wksp-170654-9ef01fbb suite=run_workspace 4:12PM INF Success suite=run_workspace 4:12PM INF Success suite=get_job 4:12PM INF Success suite=put_job 4:14PM INF App is now running. suite=kill_application_running_3.5.0 4:14PM INF App killed suite=kill_application_running_3.5.0 4:14PM INF Success suite=kill_application_running_3.5.0 4:14PM INF App is now running. suite=run_application-3.5.0 4:14PM INF Live driver logs are available suite=run_application-3.5.0 4:14PM INF Listing applications for Python test suite=run_notebook_application-Python 4:14PM INF Submitted app suite=run_notebook_application-Python 4:14PM INF App is processed suite=run_notebook_application-Python 4:14PM INF App is now running. suite=run_notebook_application-Python 4:14PM INF Get App status. suite=run_notebook_application-Python 4:14PM INF Cleaning up config templates. suite=run_notebook_application-Python 4:14PM INF Success suite=run_notebook_application-Python 4:15PM INF Spark connect app is now running suite=run_spark_connect_application-3.5.0 4:15PM INF Spark connect app killed suite=run_spark_connect_application-3.5.0 4:15PM INF Success suite=run_spark_connect_application-3.5.0 4:15PM INF Listing applications for Scala test suite=run_notebook_application-Scala 4:15PM INF Submitted app suite=run_notebook_application-Scala 4:15PM INF App is processed suite=run_notebook_application-Scala 4:15PM INF App is now running. suite=run_notebook_application-Scala 4:15PM INF Get App status. suite=run_notebook_application-Scala 4:15PM INF Cleaning up config templates. suite=run_notebook_application-Scala 4:15PM INF Success suite=run_notebook_application-Scala 4:15PM INF App is completed suite=kill_application_completed_3.5.0 4:15PM INF Success suite=kill_application_completed_3.5.0 4:15PM INF Spark connect app is now running suite=run_spark_connect_application_code-3.5.0 4:16PM INF App is completed suite=run_application-3.5.0 4:17PM INF Spark connect app executed sql suite=run_spark_connect_application_code-3.5.0 4:17PM INF Spark connect app killed suite=run_spark_connect_application_code-3.5.0 4:17PM INF Success suite=run_spark_connect_application_code-3.5.0 4:21PM INF Logs are processed suite=run_application-3.5.0 4:21PM INF Driver log archive is available suite=run_application-3.5.0 4:21PM INF Success suite=run_application-3.5.0 4:21PM INF Starting sequential tests 4:21PM INF Starting suite=kill_application_pending_3.5.0 4:21PM INF Submitted app integration-tests-96afe-reach suite=kill_application_pending_3.5.0 4:21PM INF App is processed suite=kill_application_pending_3.5.0 4:21PM INF App killed suite=kill_application_pending_3.5.0 4:21PM INF Success suite=kill_application_pending_3.5.0 4:21PM INF All tests are successful