Skip to content

[SPARK-45423][SQL] Lower ParquetWriteSupport log level to debug #145

[SPARK-45423][SQL] Lower ParquetWriteSupport log level to debug

[SPARK-45423][SQL] Lower ParquetWriteSupport log level to debug #145

Triggered via push October 5, 2023 16:28
Status Failure
Total duration 3h 37m 47s
Artifacts 21

build_main.yml

on: push
Run  /  Check changes
40s
Run / Check changes
Run  /  Base image build
58s
Run / Base image build
Run  /  Breaking change detection with Buf (branch-3.5)
1m 1s
Run / Breaking change detection with Buf (branch-3.5)
Run  /  Run TPC-DS queries with SF=1
1h 36m
Run / Run TPC-DS queries with SF=1
Run  /  Run Docker integration tests
43m 50s
Run / Run Docker integration tests
Run  /  Run Spark on Kubernetes Integration test
1h 35m
Run / Run Spark on Kubernetes Integration test
Matrix: Run / build
Matrix: Run / java-other-versions
Run  /  Build modules: sparkr
50m 0s
Run / Build modules: sparkr
Run  /  Linters, licenses, dependencies and documentation generation
1h 48m
Run / Linters, licenses, dependencies and documentation generation
Matrix: Run / pyspark
Fit to window
Zoom out
Zoom in

Annotations

29 errors and 3 warnings
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-17de168b00d7ddae-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-8057318b00d92d03-exec-1".
Run / Run Spark on Kubernetes Integration test
sleep interrupted
Run / Run Spark on Kubernetes Integration test
sleep interrupted
Run / Run Spark on Kubernetes Integration test
Task io.fabric8.kubernetes.client.utils.internal.SerialExecutor$$Lambda$676/0x00007f70085898c0@75122c51 rejected from java.util.concurrent.ThreadPoolExecutor@2adf91c[Shutting down, pool size = 4, active threads = 2, queued tasks = 0, completed tasks = 327]
Run / Run Spark on Kubernetes Integration test
Task io.fabric8.kubernetes.client.utils.internal.SerialExecutor$$Lambda$676/0x00007f70085898c0@280ef78b rejected from java.util.concurrent.ThreadPoolExecutor@2adf91c[Shutting down, pool size = 4, active threads = 2, queued tasks = 0, completed tasks = 327]
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-c68a5c8b00f397d8-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-af46ce8b00f4ec40-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-55d0a38b00f9878a-exec-1".
Run / Run Spark on Kubernetes Integration test
Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-test-app-adabdac5be5b4da3b20b0d9c40066136-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-test-app-adabdac5be5b4da3b20b0d9c40066136-driver" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={})..
Run / Build modules: pyspark-connect
Process completed with exit code 19.
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_dataframes_with_less_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T17:22:35.635310981+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_dataframes_with_more_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T17:32:46.482627797+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_dataframes_with_other_column_names: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T17:42:58.866315679+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_different_output_length: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T17:53:10.593573359+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_dataframes: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T18:03:22.87467842+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_dataframes_with_less_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T18:13:34.11864901+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_dataframes_with_more_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T18:23:45.835116145+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_dataframes_with_other_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T18:33:57.802628537+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_dataframes_without_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T18:44:09.762662109+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_empty_iterator: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T18:54:21.043642043+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_large_variable_types: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T19:04:32.964320108+00:00"}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_map_in_pandas: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T19:14:45.137732909+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_map_in_pandas_with_column_vector: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
[Errno 2] No such file or directory: '/__w/spark/spark/python/target/c53726b1-945e-4e32-9bcc-456bf15b2009/tmpo6ipao21'
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_multiple_columns: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T19:35:09.086863347+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_no_column_names: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T19:45:20.114886023+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_other_than_dataframe_iter: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {created_time:"2023-10-05T19:55:32.394607698+00:00", grpc_status:14}" >
python/pyspark/sql/tests/connect/test_parity_pandas_map.py.test_self_join: python/pyspark/sql/tests/connect/test_parity_pandas_map.py#L1
<_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:36001: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-10-05T20:05:44.070439807+00:00"}" >
Run / Build modules: pyspark-core, pyspark-streaming
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
Run / Build modules: pyspark-errors
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
Run / Build modules: pyspark-sql, pyspark-resource, pyspark-testing
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size
site Expired
59.2 MB
test-results-catalyst, hive-thriftserver--17-hadoop3-hive2.3 Expired
199 KB
test-results-core, unsafe, kvstore, avro, network-common, network-shuffle, repl, launcher, examples, sketch, graphx--17-hadoop3-hive2.3 Expired
132 KB
test-results-docker-integration--8-hadoop3-hive2.3 Expired
119 KB
test-results-hive-- other tests-17-hadoop3-hive2.3 Expired
911 KB
test-results-hive-- slow tests-17-hadoop3-hive2.3 Expired
853 KB
test-results-pyspark-connect--8-hadoop3-hive2.3 Expired
306 KB
test-results-pyspark-mllib, pyspark-ml, pyspark-ml-connect--8-hadoop3-hive2.3 Expired
1.23 MB
test-results-pyspark-pandas--8-hadoop3-hive2.3 Expired
1.14 MB
test-results-pyspark-pandas-connect-part0--8-hadoop3-hive2.3 Expired
1.06 MB
test-results-pyspark-pandas-connect-part1--8-hadoop3-hive2.3 Expired
972 KB
test-results-pyspark-pandas-connect-part2--8-hadoop3-hive2.3 Expired
637 KB
test-results-pyspark-pandas-connect-part3--8-hadoop3-hive2.3 Expired
326 KB
test-results-pyspark-pandas-slow--8-hadoop3-hive2.3 Expired
1.85 MB
test-results-sparkr--8-hadoop3-hive2.3 Expired
279 KB
test-results-sql-- extended tests-17-hadoop3-hive2.3 Expired
2.96 MB
test-results-sql-- other tests-17-hadoop3-hive2.3 Expired
4.24 MB
test-results-sql-- slow tests-17-hadoop3-hive2.3 Expired
2.76 MB
test-results-streaming, sql-kafka-0-10, streaming-kafka-0-10, mllib-local, mllib, yarn, kubernetes, hadoop-cloud, spark-ganglia-lgpl, connect, protobuf--17-hadoop3-hive2.3 Expired
2.12 MB
test-results-tpcds--8-hadoop3-hive2.3 Expired
21.9 KB
unit-tests-log-pyspark-connect--8-hadoop3-hive2.3 Expired
887 MB