Pin update March 2024 #6677
+74
−85
Merged
Pin update March 2024 #6677
Google Cloud Build / ci-tpu-test-trigger (tpu-pytorch)
succeeded
Mar 13, 2024 in 50m 0s
Summary
Build Information
Trigger | ci-tpu-test-trigger |
Build | 5845e302-3247-4462-b05a-5fb23f9bb08c |
Start | 2024-03-12T17:40:56-07:00 |
Duration | 49m16.296s |
Status | SUCCESS |
Steps
Step | Status | Duration |
---|---|---|
git_fetch | SUCCESS | 9.308s |
git_checkout | SUCCESS | 584ms |
build_pytorch-xla-test_docker_image | SUCCESS | 39m44.79s |
push_pytorch-xla-test_docker_image | SUCCESS | 1m1.978s |
run_e2e_tests | SUCCESS | 8m16.148s |
Details
starting build "5845e302-3247-4462-b05a-5fb23f9bb08c"
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/pytorch/xla
* branch 9003ef75d60b9c4736392a037c298411b4099021 -> FETCH_HEAD
HEAD is now at 9003ef7 recover local xla repo path
BUILD
Starting Step #0 - "git_fetch"
Step #0 - "git_fetch": Already have image (with digest): gcr.io/cloud-builders/git
Step #0 - "git_fetch": From https://github.com/pytorch/xla
Step #0 - "git_fetch": * [new branch] 01172023_pin_update -> origin/01172023_pin_update
Step #0 - "git_fetch": * [new branch] 2-0-python-3-10 -> origin/2-0-python-3-10
Step #0 - "git_fetch": * [new branch] 4483-improve-resnet50-data-loading-performance -> origin/4483-improve-resnet50-data-loading-performance
Step #0 - "git_fetch": * [new branch] 5d-unit-test -> origin/5d-unit-test
Step #0 - "git_fetch": * [new branch] AddVimToReleasePackage -> origin/AddVimToReleasePackage
Step #0 - "git_fetch": * [new branch] FixTransferToServer -> origin/FixTransferToServer
Step #0 - "git_fetch": * [new branch] JackCaaoG/spmd_device_python_test -> origin/JackCaaoG/spmd_device_python_test
Step #0 - "git_fetch": * [new branch] JackCao/Disable_hlo_dump_test_on_tpu -> origin/JackCao/Disable_hlo_dump_test_on_tpu
Step #0 - "git_fetch": * [new branch] JackCao/cherry_pick_dynamo_1 -> origin/JackCao/cherry_pick_dynamo_1
Step #0 - "git_fetch": * [new branch] JackCao/cherry_pick_dynamo_2 -> origin/JackCao/cherry_pick_dynamo_2
Step #0 - "git_fetch": * [new branch] JackCao/cherry_pick_dynamo_3 -> origin/JackCao/cherry_pick_dynamo_3
Step #0 - "git_fetch": * [new branch] JackCao/spmd_python_device -> origin/JackCao/spmd_python_device
Step #0 - "git_fetch": * [new branch] JackCao/unordered_sharding_spec -> origin/JackCao/unordered_sharding_spec
Step #0 - "git_fetch": * [new branch] JackCao/update_to_2.0 -> origin/JackCao/update_to_2.0
Step #0 - "git_fetch": * [new branch] JackCao/update_to_223 -> origin/JackCao/update_to_223
Step #0 - "git_fetch": * [new branch] JackCao/workaround_einsum_faketensor -> origin/JackCao/workaround_einsum_faketensor
Step #0 - "git_fetch": * [new branch] JackCaoG-patch-1 -> origin/JackCaoG-patch-1
Step #0 - "git_fetch": * [new branch] JackCaoG-patch-2 -> origin/JackCaoG-patch-2
Step #0 - "git_fetch": * [new branch] JackCaoG-patch-3 -> origin/JackCaoG-patch-3
Step #0 - "git_fetch": * [new branch] JackCaoG-patch-4 -> origin/JackCaoG-patch-4
Step #0 - "git_fetch": * [new branch] JackCaoG-patch-5 -> origin/JackCaoG-patch-5
Step #0 - "git_fetch": * [new branch] JackCaoG-remove-pin -> origin/JackCaoG-remove-pin
Step #0 - "git_fetch": * [new branch] JackCaoG/1_13_revert_addcmul -> origin/JackCaoG/1_13_revert_addcmul
Step #0 - "git_fetch": * [new branch] JackCaoG/2.0_patch -> origin/JackCaoG/2.0_patch
Step #0 - "git_fetch": * [new branch] JackCaoG/AddDynamoSpmdMetrics -> origin/JackCaoG/AddDynamoSpmdMetrics
Step #0 - "git_fetch": * [new branch] JackCaoG/CI_build_cpp_test -> origin/JackCaoG/CI_build_cpp_test
Step #0 - "git_fetch": * [new branch] JackCaoG/Dyanmo_execute_async -> origin/JackCaoG/Dyanmo_execute_async
Step #0 - "git_fetch": * [new branch] JackCaoG/Dynamo_training_test -> origin/JackCaoG/Dynamo_training_test
Step #0 - "git_fetch": * [new branch] JackCaoG/PJRT_wait_buffer_sync -> origin/JackCaoG/PJRT_wait_buffer_sync
Step #0 - "git_fetch": * [new branch] JackCaoG/SPMD_execute_metrics -> origin/JackCaoG/SPMD_execute_metrics
Step #0 - "git_fetch": * [new branch] JackCaoG/SPMD_int_to_cpu -> origin/JackCaoG/SPMD_int_to_cpu
Step #0 - "git_fetch": * [new branch] JackCaoG/Update_short_metrics -> origin/JackCaoG/Update_short_metrics
Step #0 - "git_fetch": * [new branch] JackCaoG/Update_wheel_docker_to_table -> origin/JackCaoG/Update_wheel_docker_to_table
Step #0 - "git_fetch": * [new branch] JackCaoG/XLAShardedTensor_torch_dispatch -> origin/JackCaoG/XLAShardedTensor_torch_dispatch
Step #0 - "git_fetch": * [new branch] JackCaoG/add_2_1_py311_build -> origin/JackCaoG/add_2_1_py311_build
Step #0 - "git_fetch": * [new branch] JackCaoG/add_dynamo_test_counter_check -> origin/JackCaoG/add_dynamo_test_counter_check
Step #0 - "git_fetch": * [new branch] JackCaoG/add_dynamo_test_to_tpu_ci -> origin/JackCaoG/add_dynamo_test_to_tpu_ci
Step #0 - "git_fetch": * [new branch] JackCaoG/add_gpu_doc -> origin/JackCaoG/add_gpu_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/add_gpu_doc_readme -> origin/JackCaoG/add_gpu_doc_readme
Step #0 - "git_fetch": * [new branch] JackCaoG/add_test_dynamo_graph_dump -> origin/JackCaoG/add_test_dynamo_graph_dump
Step #0 - "git_fetch": * [new branch] JackCaoG/add_test_print -> origin/JackCaoG/add_test_print
Step #0 - "git_fetch": * [new branch] JackCaoG/api_init_runtime -> origin/JackCaoG/api_init_runtime
Step #0 - "git_fetch": * [new branch] JackCaoG/backout_22_mesh_spmd -> origin/JackCaoG/backout_22_mesh_spmd
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_allgather_out -> origin/JackCaoG/backport_22_allgather_out
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_diagonal_scatter -> origin/JackCaoG/backport_22_diagonal_scatter
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_doc -> origin/JackCaoG/backport_22_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_reduce_scatter -> origin/JackCaoG/backport_22_reduce_scatter
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_reducescatter_out -> origin/JackCaoG/backport_22_reducescatter_out
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_scalar -> origin/JackCaoG/backport_22_scalar
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_22_transfer_gpu_scalar -> origin/JackCaoG/backport_22_transfer_gpu_scalar
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_dynamo_cpu_move -> origin/JackCaoG/backport_dynamo_cpu_move
Step #0 - "git_fetch": * [new branch] JackCaoG/backport_frame_count -> origin/JackCaoG/backport_frame_count
Step #0 - "git_fetch": * [new branch] JackCaoG/barrier_comment -> origin/JackCaoG/barrier_comment
Step #0 - "git_fetch": * [new branch] JackCaoG/better_dynamo_error_message -> origin/JackCaoG/better_dynamo_error_message
Step #0 - "git_fetch": * [new branch] JackCaoG/buffer_donor_custom_op -> origin/JackCaoG/buffer_donor_custom_op
Step #0 - "git_fetch": * [new branch] JackCaoG/build_audio_whl -> origin/JackCaoG/build_audio_whl
Step #0 - "git_fetch": * [new branch] JackCaoG/bump+gpu_coverage_timeout -> origin/JackCaoG/bump+gpu_coverage_timeout
Step #0 - "git_fetch": * [new branch] JackCaoG/bump_build_timeout -> origin/JackCaoG/bump_build_timeout
Step #0 - "git_fetch": * [new branch] JackCaoG/cache_sharded_placeholder_dynamo -> origin/JackCaoG/cache_sharded_placeholder_dynamo
Step #0 - "git_fetch": * [new branch] JackCaoG/cherry_pick_dynamo_test -> origin/JackCaoG/cherry_pick_dynamo_test
Step #0 - "git_fetch": * [new branch] JackCaoG/cherry_pick_frame_count -> origin/JackCaoG/cherry_pick_frame_count
Step #0 - "git_fetch": * [new branch] JackCaoG/cherry_picl_xrt_crash_2.1 -> origin/JackCaoG/cherry_picl_xrt_crash_2.1
Step #0 - "git_fetch": * [new branch] JackCaoG/clang-format-11 -> origin/JackCaoG/clang-format-11
Step #0 - "git_fetch": * [new branch] JackCaoG/clang-format-11-2 -> origin/JackCaoG/clang-format-11-2
Step #0 - "git_fetch": * [new branch] JackCaoG/clean_up_gettensor -> origin/JackCaoG/clean_up_gettensor
Step #0 - "git_fetch": * [new branch] JackCaoG/clean_up_use_virtual_device -> origin/JackCaoG/clean_up_use_virtual_device
Step #0 - "git_fetch": * [new branch] JackCaoG/cleanup_docs -> origin/JackCaoG/cleanup_docs
Step #0 - "git_fetch": * [new branch] JackCaoG/colab_whl_abi -> origin/JackCaoG/colab_whl_abi
Step #0 - "git_fetch": * [new branch] JackCaoG/colab_whl_abi_2 -> origin/JackCaoG/colab_whl_abi_2
Step #0 - "git_fetch": * [new branch] JackCaoG/data_to_string -> origin/JackCaoG/data_to_string
Step #0 - "git_fetch": * [new branch] JackCaoG/debug_fix -> origin/JackCaoG/debug_fix
Step #0 - "git_fetch": * [new branch] JackCaoG/debug_tool -> origin/JackCaoG/debug_tool
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_coverage -> origin/JackCaoG/disable_coverage
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_dynamo_different_shape_test -> origin/JackCaoG/disable_dynamo_different_shape_test
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_flaky_test -> origin/JackCaoG/disable_flaky_test
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_flaky_test_2 -> origin/JackCaoG/disable_flaky_test_2
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_flaky_test_3 -> origin/JackCaoG/disable_flaky_test_3
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_more_dynamic -> origin/JackCaoG/disable_more_dynamic
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_more_dynamic_test -> origin/JackCaoG/disable_more_dynamic_test
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_pjrt_computation_client_test -> origin/JackCaoG/disable_pjrt_computation_client_test
Step #0 - "git_fetch": * [new branch] JackCaoG/disable_test_xla_sharding -> origin/JackCaoG/disable_test_xla_sharding
Step #0 - "git_fetch": * [new branch] JackCaoG/disbale_docker_sandbox -> origin/JackCaoG/disbale_docker_sandbox
Step #0 - "git_fetch": * [new branch] JackCaoG/dump_output_sharding -> origin/JackCaoG/dump_output_sharding
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_2.0_cherry_pick -> origin/JackCaoG/dynamo_2.0_cherry_pick
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_aliasing_2 -> origin/JackCaoG/dynamo_aliasing_2
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_debug_message -> origin/JackCaoG/dynamo_debug_message
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_different_graph -> origin/JackCaoG/dynamo_different_graph
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_do_not_markstep_to_sync_input -> origin/JackCaoG/dynamo_do_not_markstep_to_sync_input
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_doc -> origin/JackCaoG/dynamo_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_dump_tensor_file -> origin/JackCaoG/dynamo_dump_tensor_file
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_note_backport_2.1 -> origin/JackCaoG/dynamo_note_backport_2.1
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_only_warm_up_cache -> origin/JackCaoG/dynamo_only_warm_up_cache
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_optimizer -> origin/JackCaoG/dynamo_optimizer
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_reduce_execute -> origin/JackCaoG/dynamo_reduce_execute
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_rng_seed_without_mark_step -> origin/JackCaoG/dynamo_rng_seed_without_mark_step
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_spmd -> origin/JackCaoG/dynamo_spmd
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_spmd_input_sharding_change -> origin/JackCaoG/dynamo_spmd_input_sharding_change
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_spmd_optimization -> origin/JackCaoG/dynamo_spmd_optimization
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_sync_input -> origin/JackCaoG/dynamo_sync_input
Step #0 - "git_fetch": * [new branch] JackCaoG/dynamo_warning -> origin/JackCaoG/dynamo_warning
Step #0 - "git_fetch": * [new branch] JackCaoG/enable_cumsum_long -> origin/JackCaoG/enable_cumsum_long
Step #0 - "git_fetch": * [new branch] JackCaoG/enable_dynamo_upstream_test -> origin/JackCaoG/enable_dynamo_upstream_test
Step #0 - "git_fetch": * [new branch] JackCaoG/enable_fsdp_gpu_test -> origin/JackCaoG/enable_fsdp_gpu_test
Step #0 - "git_fetch": * [new branch] JackCaoG/enable_mp_test_on_tpuci -> origin/JackCaoG/enable_mp_test_on_tpuci
Step #0 - "git_fetch": * [new branch] JackCaoG/enable_reduce_scatter_test_gpu -> origin/JackCaoG/enable_reduce_scatter_test_gpu
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_cc_linter_once -> origin/JackCaoG/fix_cc_linter_once
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_deprecated_test_oepration_api -> origin/JackCaoG/fix_deprecated_test_oepration_api
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_diagonal_scatter_dim -> origin/JackCaoG/fix_diagonal_scatter_dim
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dyanmo_additional_tracing -> origin/JackCaoG/fix_dyanmo_additional_tracing
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dynamo -> origin/JackCaoG/fix_dynamo
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dynamo_crash -> origin/JackCaoG/fix_dynamo_crash
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dynamo_inplace -> origin/JackCaoG/fix_dynamo_inplace
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dynamo_input_sharding_test -> origin/JackCaoG/fix_dynamo_input_sharding_test
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_dynamo_text_allclose -> origin/JackCaoG/fix_dynamo_text_allclose
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_hlo_dump -> origin/JackCaoG/fix_hlo_dump
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_hlo_dump_spmd -> origin/JackCaoG/fix_hlo_dump_spmd
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_mark_step_on_spmd_device -> origin/JackCaoG/fix_mark_step_on_spmd_device
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_mp_save_test -> origin/JackCaoG/fix_mp_save_test
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_send_cpu_data -> origin/JackCaoG/fix_send_cpu_data
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_spmd_buff_is_null -> origin/JackCaoG/fix_spmd_buff_is_null
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_tpu_ci_accuracy -> origin/JackCaoG/fix_tpu_ci_accuracy
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_typo -> origin/JackCaoG/fix_typo
Step #0 - "git_fetch": * [new branch] JackCaoG/fix_vision_build -> origin/JackCaoG/fix_vision_build
Step #0 - "git_fetch": * [new branch] JackCaoG/fsdp_autowrap_test -> origin/JackCaoG/fsdp_autowrap_test
Step #0 - "git_fetch": * [new branch] JackCaoG/gpu_build_pytorch_cuda -> origin/JackCaoG/gpu_build_pytorch_cuda
Step #0 - "git_fetch": * [new branch] JackCaoG/graph_hash_debug -> origin/JackCaoG/graph_hash_debug
Step #0 - "git_fetch": * [new branch] JackCaoG/handle_dynamo_fn_without_arg -> origin/JackCaoG/handle_dynamo_fn_without_arg
Step #0 - "git_fetch": * [new branch] JackCaoG/improve_data_tostring -> origin/JackCaoG/improve_data_tostring
Step #0 - "git_fetch": * [new branch] JackCaoG/increase_timeout_docker -> origin/JackCaoG/increase_timeout_docker
Step #0 - "git_fetch": * [new branch] JackCaoG/ir_value_sharding -> origin/JackCaoG/ir_value_sharding
Step #0 - "git_fetch": * [new branch] JackCaoG/lower_dropout_backward -> origin/JackCaoG/lower_dropout_backward
Step #0 - "git_fetch": * [new branch] JackCaoG/move_op_lowering_guide -> origin/JackCaoG/move_op_lowering_guide
Step #0 - "git_fetch": * [new branch] JackCaoG/move_pin_2_0 -> origin/JackCaoG/move_pin_2_0
Step #0 - "git_fetch": * [new branch] JackCaoG/move_pjrt_lock_location -> origin/JackCaoG/move_pjrt_lock_location
Step #0 - "git_fetch": * [new branch] JackCaoG/move_return_future -> origin/JackCaoG/move_return_future
Step #0 - "git_fetch": * [new branch] JackCaoG/new_alias_api -> origin/JackCaoG/new_alias_api
Step #0 - "git_fetch": * [new branch] JackCaoG/nightly_build_python -> origin/JackCaoG/nightly_build_python
Step #0 - "git_fetch": * [new branch] JackCaoG/parallel_spmd -> origin/JackCaoG/parallel_spmd
Step #0 - "git_fetch": * [new branch] JackCaoG/pin_expecttest -> origin/JackCaoG/pin_expecttest
Step #0 - "git_fetch": * [new branch] JackCaoG/pin_update_20230808 -> origin/JackCaoG/pin_update_20230808
Step #0 - "git_fetch": * [new branch] JackCaoG/pt_xla_debug_5 -> origin/JackCaoG/pt_xla_debug_5
Step #0 - "git_fetch": * [new branch] JackCaoG/pt_xla_debug_dynamo_execute -> origin/JackCaoG/pt_xla_debug_dynamo_execute
Step #0 - "git_fetch": * [new branch] JackCaoG/pt_xla_debug_max_frame -> origin/JackCaoG/pt_xla_debug_max_frame
Step #0 - "git_fetch": * [new branch] JackCaoG/r20_backportdoc -> origin/JackCaoG/r20_backportdoc
Step #0 - "git_fetch": * [new branch] JackCaoG/r2_0_ABI_0 -> origin/JackCaoG/r2_0_ABI_0
Step #0 - "git_fetch": * [new branch] JackCaoG/r2_0_back_port_profile_flag -> origin/JackCaoG/r2_0_back_port_profile_flag
Step #0 - "git_fetch": * [new branch] JackCaoG/r2_0_backport_loader_thread -> origin/JackCaoG/r2_0_backport_loader_thread
Step #0 - "git_fetch": * [new branch] JackCaoG/r2_0_backport_loader_thread_2 -> origin/JackCaoG/r2_0_backport_loader_thread_2
Step #0 - "git_fetch": * [new branch] JackCaoG/r2_0_cherry_pick_defer_init -> origin/JackCaoG/r2_0_cherry_pick_defer_init
Step #0 - "git_fetch": * [new branch] JackCaoG/remove_pin -> origin/JackCaoG/remove_pin
Step #0 - "git_fetch": * [new branch] JackCaoG/remove_reshape_again -> origin/JackCaoG/remove_reshape_again
Step #0 - "git_fetch": * [new branch] JackCaoG/remove_reshape_lowering -> origin/JackCaoG/remove_reshape_lowering
Step #0 - "git_fetch": * [new branch] JackCaoG/restrict_mark_sharding -> origin/JackCaoG/restrict_mark_sharding
Step #0 - "git_fetch": * [new branch] JackCaoG/revert_addcmul -> origin/JackCaoG/revert_addcmul
Step #0 - "git_fetch": * [new branch] JackCaoG/revert_protobuf_tf_patch -> origin/JackCaoG/revert_protobuf_tf_patch
Step #0 - "git_fetch": * [new branch] JackCaoG/shard_device_data_ir -> origin/JackCaoG/shard_device_data_ir
Step #0 - "git_fetch": * [new branch] JackCaoG/skip_partitioner -> origin/JackCaoG/skip_partitioner
Step #0 - "git_fetch": * [new branch] JackCaoG/skip_tpu_test_precision -> origin/JackCaoG/skip_tpu_test_precision
Step #0 - "git_fetch": * [new branch] JackCaoG/spmd_aliasing -> origin/JackCaoG/spmd_aliasing
Step #0 - "git_fetch": * [new branch] JackCaoG/spmd_dynamo_sharding_changed -> origin/JackCaoG/spmd_dynamo_sharding_changed
Step #0 - "git_fetch": * [new branch] JackCaoG/spmd_incorrect_result -> origin/JackCaoG/spmd_incorrect_result
Step #0 - "git_fetch": * [new branch] JackCaoG/tensor_debugging_api -> origin/JackCaoG/tensor_debugging_api
Step #0 - "git_fetch": * [new branch] JackCaoG/tensor_with_xla_data_sharding -> origin/JackCaoG/tensor_with_xla_data_sharding
Step #0 - "git_fetch": * [new branch] JackCaoG/test_torch_compile_usage -> origin/JackCaoG/test_torch_compile_usage
Step #0 - "git_fetch": * [new branch] JackCaoG/tf_update_20230201 -> origin/JackCaoG/tf_update_20230201
Step #0 - "git_fetch": * [new branch] JackCaoG/tracing_timer -> origin/JackCaoG/tracing_timer
Step #0 - "git_fetch": * [new branch] JackCaoG/trouble_shooting_update -> origin/JackCaoG/trouble_shooting_update
Step #0 - "git_fetch": * [new branch] JackCaoG/truncate_python_stack -> origin/JackCaoG/truncate_python_stack
Step #0 - "git_fetch": * [new branch] JackCaoG/unordered_partial_sharding_spec -> origin/JackCaoG/unordered_partial_sharding_spec
Step #0 - "git_fetch": * [new branch] JackCaoG/updateDocker -> origin/JackCaoG/updateDocker
Step #0 - "git_fetch": * [new branch] JackCaoG/update_2_0_ABI -> origin/JackCaoG/update_2_0_ABI
Step #0 - "git_fetch": * [new branch] JackCaoG/update_API_GUIDE -> origin/JackCaoG/update_API_GUIDE
Step #0 - "git_fetch": * [new branch] JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE -> origin/JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE
Step #0 - "git_fetch": * [new branch] JackCaoG/update_build_guide -> origin/JackCaoG/update_build_guide
Step #0 - "git_fetch": * [new branch] JackCaoG/update_colab_2_10_whl -> origin/JackCaoG/update_colab_2_10_whl
Step #0 - "git_fetch": * [new branch] JackCaoG/update_doc1 -> origin/JackCaoG/update_doc1
Step #0 - "git_fetch": * [new branch] JackCaoG/update_doc_remove_xrt -> origin/JackCaoG/update_doc_remove_xrt
Step #0 - "git_fetch": * [new branch] JackCaoG/update_docker_instruction -> origin/JackCaoG/update_docker_instruction
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_backend_name -> origin/JackCaoG/update_dynamo_backend_name
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_backend_nograd -> origin/JackCaoG/update_dynamo_backend_nograd
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_doc -> origin/JackCaoG/update_dynamo_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_number_2.1 -> origin/JackCaoG/update_dynamo_number_2.1
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_readme -> origin/JackCaoG/update_dynamo_readme
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_save_graph_path -> origin/JackCaoG/update_dynamo_save_graph_path
Step #0 - "git_fetch": * [new branch] JackCaoG/update_dynamo_torch_equal -> origin/JackCaoG/update_dynamo_torch_equal
Step #0 - "git_fetch": * [new branch] JackCaoG/update_gpu_doc -> origin/JackCaoG/update_gpu_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/update_gpu_wheel_link -> origin/JackCaoG/update_gpu_wheel_link
Step #0 - "git_fetch": * [new branch] JackCaoG/update_gpu_wheellink -> origin/JackCaoG/update_gpu_wheellink
Step #0 - "git_fetch": * [new branch] JackCaoG/update_pjrt_future -> origin/JackCaoG/update_pjrt_future
Step #0 - "git_fetch": * [new branch] JackCaoG/update_readme_2 -> origin/JackCaoG/update_readme_2
Step #0 - "git_fetch": * [new branch] JackCaoG/update_set_default_tensor_type -> origin/JackCaoG/update_set_default_tensor_type
Step #0 - "git_fetch": * [new branch] JackCaoG/update_tf_2023018 -> origin/JackCaoG/update_tf_2023018
Step #0 - "git_fetch": * [new branch] JackCaoG/update_to_2.2 -> origin/JackCaoG/update_to_2.2
Step #0 - "git_fetch": * [new branch] JackCaoG/update_to_223 -> origin/JackCaoG/update_to_223
Step #0 - "git_fetch": * [new branch] JackCaoG/update_torch_compile_usage -> origin/JackCaoG/update_torch_compile_usage
Step #0 - "git_fetch": * [new branch] JackCaoG/update_trouble_shotting_doc -> origin/JackCaoG/update_trouble_shotting_doc
Step #0 - "git_fetch": * [new branch] JackCaoG/update_troubleshotting_with_sanitycheck -> origin/JackCaoG/update_troubleshotting_with_sanitycheck
Step #0 - "git_fetch": * [new branch] JackCaoG/update_wheel_docker_format -> origin/JackCaoG/update_wheel_docker_format
Step #0 - "git_fetch": * [new branch] JackCaoG/use_new_alias_api -> origin/JackCaoG/use_new_alias_api
Step #0 - "git_fetch": * [new branch] JackCaoG/weak_ref_dynamo -> origin/JackCaoG/weak_ref_dynamo
Step #0 - "git_fetch": * [new branch] JackCaoG/xrt_cherry_pick_07_10 -> origin/JackCaoG/xrt_cherry_pick_07_10
Step #0 - "git_fetch": * [new branch] JackCaoG/xrt_cherrypick_0721 -> origin/JackCaoG/xrt_cherrypick_0721
Step #0 - "git_fetch": * [new branch] JackCaog/dynamo_input_sharding -> origin/JackCaog/dynamo_input_sharding
Step #0 - "git_fetch": * [new branch] JackCaog/quick_spmd_fix -> origin/JackCaog/quick_spmd_fix
Step #0 - "git_fetch": * [new branch] JackCaog/spmd_device_default -> origin/JackCaog/spmd_device_default
Step #0 - "git_fetch": * [new branch] Liyang90-patch-1 -> origin/Liyang90-patch-1
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-1 -> origin/ManfeiBai-patch-1
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-10 -> origin/ManfeiBai-patch-10
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-11 -> origin/ManfeiBai-patch-11
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-12 -> origin/ManfeiBai-patch-12
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-13 -> origin/ManfeiBai-patch-13
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-14 -> origin/ManfeiBai-patch-14
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-15 -> origin/ManfeiBai-patch-15
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-16 -> origin/ManfeiBai-patch-16
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-17 -> origin/ManfeiBai-patch-17
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-18 -> origin/ManfeiBai-patch-18
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-19 -> origin/ManfeiBai-patch-19
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-2 -> origin/ManfeiBai-patch-2
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-20 -> origin/ManfeiBai-patch-20
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-21 -> origin/ManfeiBai-patch-21
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-22 -> origin/ManfeiBai-patch-22
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-23 -> origin/ManfeiBai-patch-23
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-24 -> origin/ManfeiBai-patch-24
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-25 -> origin/ManfeiBai-patch-25
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-26 -> origin/ManfeiBai-patch-26
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-27 -> origin/ManfeiBai-patch-27
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-28 -> origin/ManfeiBai-patch-28
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-29 -> origin/ManfeiBai-patch-29
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-3 -> origin/ManfeiBai-patch-3
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-30 -> origin/ManfeiBai-patch-30
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-31 -> origin/ManfeiBai-patch-31
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-32 -> origin/ManfeiBai-patch-32
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-33 -> origin/ManfeiBai-patch-33
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-34 -> origin/ManfeiBai-patch-34
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-35 -> origin/ManfeiBai-patch-35
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-36 -> origin/ManfeiBai-patch-36
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-37 -> origin/ManfeiBai-patch-37
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-38 -> origin/ManfeiBai-patch-38
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-39 -> origin/ManfeiBai-patch-39
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-4 -> origin/ManfeiBai-patch-4
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-40 -> origin/ManfeiBai-patch-40
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-41 -> origin/ManfeiBai-patch-41
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-41-test -> origin/ManfeiBai-patch-41-test
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-42 -> origin/ManfeiBai-patch-42
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-43 -> origin/ManfeiBai-patch-43
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-44 -> origin/ManfeiBai-patch-44
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-45 -> origin/ManfeiBai-patch-45
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-46 -> origin/ManfeiBai-patch-46
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-47 -> origin/ManfeiBai-patch-47
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-48 -> origin/ManfeiBai-patch-48
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-49 -> origin/ManfeiBai-patch-49
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-5 -> origin/ManfeiBai-patch-5
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-50 -> origin/ManfeiBai-patch-50
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-51 -> origin/ManfeiBai-patch-51
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-52 -> origin/ManfeiBai-patch-52
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-53 -> origin/ManfeiBai-patch-53
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-54 -> origin/ManfeiBai-patch-54
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-55 -> origin/ManfeiBai-patch-55
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-56 -> origin/ManfeiBai-patch-56
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-57 -> origin/ManfeiBai-patch-57
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-58 -> origin/ManfeiBai-patch-58
Step #0 - "git_fetch": * [new branch] ManfeiBai-patch-59
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds;region=us-central1/5845e302-3247-4462-b05a-5fb23f9bb08c?project=164006649440.]
...
.0 torch-xla-2.3.0 typing-extensions-4.10.0 uritemplate-3.0.1 urllib3-2.2.1
Step #2 - "build_pytorch-xla-test_docker_image": �[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #2 - "build_pytorch-xla-test_docker_image": �[0m�[91m
Step #2 - "build_pytorch-xla-test_docker_image": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #2 - "build_pytorch-xla-test_docker_image": [notice] To update, run: pip install --upgrade pip
Step #2 - "build_pytorch-xla-test_docker_image": �[0mRemoving intermediate container bf593dc52c90
Step #2 - "build_pytorch-xla-test_docker_image": ---> 3d38cf2da9c6
Step #2 - "build_pytorch-xla-test_docker_image": Step 24/25 : WORKDIR /
Step #2 - "build_pytorch-xla-test_docker_image": ---> Running in b3fee7c84e93
Step #2 - "build_pytorch-xla-test_docker_image": Removing intermediate container b3fee7c84e93
Step #2 - "build_pytorch-xla-test_docker_image": ---> 079a8b0552d0
Step #2 - "build_pytorch-xla-test_docker_image": Step 25/25 : RUN rm -rf /ansible /tmp/wheels
Step #2 - "build_pytorch-xla-test_docker_image": ---> Running in f37cb91268ee
Step #2 - "build_pytorch-xla-test_docker_image": Removing intermediate container f37cb91268ee
Step #2 - "build_pytorch-xla-test_docker_image": ---> f837a35cb485
Step #2 - "build_pytorch-xla-test_docker_image": Successfully built f837a35cb485
Step #2 - "build_pytorch-xla-test_docker_image": Successfully tagged us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #2 - "build_pytorch-xla-test_docker_image": Successfully tagged local_image:latest
Finished Step #2 - "build_pytorch-xla-test_docker_image"
Starting Step #3 - "push_pytorch-xla-test_docker_image"
Step #3 - "push_pytorch-xla-test_docker_image": Already have image (with digest): gcr.io/cloud-builders/docker
Step #3 - "push_pytorch-xla-test_docker_image": The push refers to repository [us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test]
Step #3 - "push_pytorch-xla-test_docker_image": 09fb5087fa97: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 911661ce5cae: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8cef9b706f27: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8e983065a15a: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 705501a374bd: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 0e5768b110ff: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 705501a374bd: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 09fb5087fa97: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 8e983065a15a: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 0e5768b110ff: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 8cef9b706f27: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 911661ce5cae: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 5845e302-3247-4462-b05a-5fb23f9bb08c: digest: sha256:c2637622d0c7bbccb63dd2dbaf3989865c14cabfeca09236d1eb3cd404f6a426 size: 4733
Finished Step #3 - "push_pytorch-xla-test_docker_image"
Starting Step #4 - "run_e2e_tests"
Step #4 - "run_e2e_tests": Pulling image: google/cloud-sdk
Step #4 - "run_e2e_tests": Using default tag: latest
Step #4 - "run_e2e_tests": latest: Pulling from google/cloud-sdk
Step #4 - "run_e2e_tests": ec335f17d0c7: Already exists
Step #4 - "run_e2e_tests": 8998ef4d29e2: Pulling fs layer
Step #4 - "run_e2e_tests": 47fcb7f183ee: Pulling fs layer
Step #4 - "run_e2e_tests": fc58c263007f: Pulling fs layer
Step #4 - "run_e2e_tests": e5c2b4203b75: Pulling fs layer
Step #4 - "run_e2e_tests": 0894066a283a: Pulling fs layer
Step #4 - "run_e2e_tests": 574b19029fbb: Pulling fs layer
Step #4 - "run_e2e_tests": e5c2b4203b75: Waiting
Step #4 - "run_e2e_tests": 0894066a283a: Waiting
Step #4 - "run_e2e_tests": 574b19029fbb: Waiting
Step #4 - "run_e2e_tests": fc58c263007f: Verifying Checksum
Step #4 - "run_e2e_tests": fc58c263007f: Download complete
Step #4 - "run_e2e_tests": 8998ef4d29e2: Verifying Checksum
Step #4 - "run_e2e_tests": 8998ef4d29e2: Download complete
Step #4 - "run_e2e_tests": 47fcb7f183ee: Verifying Checksum
Step #4 - "run_e2e_tests": 47fcb7f183ee: Download complete
Step #4 - "run_e2e_tests": 574b19029fbb: Verifying Checksum
Step #4 - "run_e2e_tests": 574b19029fbb: Download complete
Step #4 - "run_e2e_tests": 8998ef4d29e2: Pull complete
Step #4 - "run_e2e_tests": 0894066a283a: Verifying Checksum
Step #4 - "run_e2e_tests": 0894066a283a: Download complete
Step #4 - "run_e2e_tests": 47fcb7f183ee: Pull complete
Step #4 - "run_e2e_tests": fc58c263007f: Pull complete
Step #4 - "run_e2e_tests": e5c2b4203b75: Verifying Checksum
Step #4 - "run_e2e_tests": e5c2b4203b75: Download complete
Step #4 - "run_e2e_tests": e5c2b4203b75: Pull complete
Step #4 - "run_e2e_tests": 0894066a283a: Pull complete
Step #4 - "run_e2e_tests": 574b19029fbb: Pull complete
Step #4 - "run_e2e_tests": Digest: sha256:4a4884995f6734698aa910e43c0966502a8e980fdaee3ac04bf956b912921ecd
Step #4 - "run_e2e_tests": Status: Downloaded newer image for google/cloud-sdk:latest
Step #4 - "run_e2e_tests": docker.io/google/cloud-sdk:latest
Step #4 - "run_e2e_tests": + apt-get update
Step #4 - "run_e2e_tests": Get:1 https://packages.cloud.google.com/apt cloud-sdk-bullseye InRelease [6406 B]
Step #4 - "run_e2e_tests": Hit:2 http://deb.debian.org/debian bullseye InRelease
Step #4 - "run_e2e_tests": Hit:3 http://deb.debian.org/debian-security bullseye-security InRelease
Step #4 - "run_e2e_tests": Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Step #4 - "run_e2e_tests": Get:5 https://packages.cloud.google.com/apt cloud-sdk-bullseye/main amd64 Packages [468 kB]
Step #4 - "run_e2e_tests": Fetched 519 kB in 1s (1023 kB/s)
Step #4 - "run_e2e_tests": Reading package lists...
Step #4 - "run_e2e_tests": + apt-get -y install gettext
Step #4 - "run_e2e_tests": Reading package lists...
Step #4 - "run_e2e_tests": Building dependency tree...
Step #4 - "run_e2e_tests": Reading state information...
Step #4 - "run_e2e_tests": The following additional packages will be installed:
Step #4 - "run_e2e_tests": gettext-base
Step #4 - "run_e2e_tests": Suggested packages:
Step #4 - "run_e2e_tests": gettext-doc autopoint libasprintf-dev libgettextpo-dev
Step #4 - "run_e2e_tests": The following NEW packages will be installed:
Step #4 - "run_e2e_tests": gettext gettext-base
Step #4 - "run_e2e_tests": 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Step #4 - "run_e2e_tests": Need to get 1486 kB of archives.
Step #4 - "run_e2e_tests": After this operation, 6658 kB of additional disk space will be used.
Step #4 - "run_e2e_tests": Get:1 http://deb.debian.org/debian bullseye/main amd64 gettext-base amd64 0.21-4 [175 kB]
Step #4 - "run_e2e_tests": Get:2 http://deb.debian.org/debian bullseye/main amd64 gettext amd64 0.21-4 [1311 kB]
Step #4 - "run_e2e_tests": debconf: delaying package configuration, since apt-utils is not installed
Step #4 - "run_e2e_tests": Fetched 1486 kB in 0s (5250 kB/s)
Step #4 - "run_e2e_tests": Selecting previously unselected package gettext-base.
Step #4 - "run_e2e_tests": (Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 70391 files and directories currently installed.)
Step #4 - "run_e2e_tests": Preparing to unpack .../gettext-base_0.21-4_amd64.deb ...
Step #4 - "run_e2e_tests": Unpacking gettext-base (0.21-4) ...
Step #4 - "run_e2e_tests": Selecting previously unselected package gettext.
Step #4 - "run_e2e_tests": Preparing to unpack .../gettext_0.21-4_amd64.deb ...
Step #4 - "run_e2e_tests": Unpacking gettext (0.21-4) ...
Step #4 - "run_e2e_tests": Setting up gettext-base (0.21-4) ...
Step #4 - "run_e2e_tests": Setting up gettext (0.21-4) ...
Step #4 - "run_e2e_tests": Processing triggers for libc-bin (2.31-13+deb11u8) ...
Step #4 - "run_e2e_tests": + gcloud container clusters get-credentials tpu-cluster --zone europe-west4-a
Step #4 - "run_e2e_tests": Fetching cluster endpoint and auth data.
Step #4 - "run_e2e_tests": kubeconfig entry generated for tpu-cluster.
Step #4 - "run_e2e_tests": + export IMAGE=us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #4 - "run_e2e_tests": + IMAGE=us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #4 - "run_e2e_tests": + export PROJECT_ID=tpu-pytorch
Step #4 - "run_e2e_tests": + PROJECT_ID=tpu-pytorch
Step #4 - "run_e2e_tests": ++ envsubst
Step #4 - "run_e2e_tests": ++ kubectl create -f - -o name
Step #4 - "run_e2e_tests": + pod_name=pod/xla-test-job-nkt9n
Step #4 - "run_e2e_tests": ++ kubectl wait --for condition=ready --timeout=60m pod/xla-test-job-nkt9n -o name
Step #4 - "run_e2e_tests": + pod_name=pod/xla-test-job-nkt9n
Step #4 - "run_e2e_tests": + kubectl logs -f pod/xla-test-job-nkt9n --container=xla-test
Step #4 - "run_e2e_tests": ++ cat /src/pytorch/.github/ci_commit_pins/vision.txt
Step #4 - "run_e2e_tests": + pip install --no-use-pep517 git+https://github.com/pytorch/vision.git@2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Collecting git+https://github.com/pytorch/vision.git@2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Cloning https://github.com/pytorch/vision.git (to revision 2c127da8b5e2e8f44b50994c6cb931bcca267cfe) to /tmp/pip-req-build-g277s747
Step #4 - "run_e2e_tests": Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-g277s747
Step #4 - "run_e2e_tests": Running command git rev-parse -q --verify 'sha^2c127da8b5e2e8f44b50994c6cb931bcca267cfe'
Step #4 - "run_e2e_tests": Running command git fetch -q https://github.com/pytorch/vision.git 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Running command git checkout -q 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Resolved https://github.com/pytorch/vision.git to commit 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Preparing metadata (setup.py): started
Step #4 - "run_e2e_tests": Preparing metadata (setup.py): finished with status 'done'
Step #4 - "run_e2e_tests": Requirement already satisfied: numpy in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (1.26.4)
Step #4 - "run_e2e_tests": Requirement already satisfied: torch in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (2.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (10.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: filelock in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.13.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: fsspec in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (2024.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.1.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: sympy in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (1.12)
Step #4 - "run_e2e_tests": Requirement already satisfied: networkx in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.2.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (4.10.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.18.0a0+2c127da) (2.1.5)
Step #4 - "run_e2e_tests": Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/site-packages (from sympy->torch->torchvision==0.18.0a0+2c127da) (1.3.0)
Step #4 - "run_e2e_tests": Building wheels for collected packages: torchvision
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): started
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests": Building wheel for torchvision (setup.py): finished with status 'done'
Step #4 - "run_e2e_tests": Created wheel for torchvision: filename=torchvision-0.18.0a0+2c127da-cp310-cp310-linux_x86_64.whl size=1155955 sha256=c0d63207ed227ed9421b2eb781fc957a6410d6ec00197f2070e48a6aba1baf99
Step #4 - "run_e2e_tests": Stored in directory: /root/.cache/pip/wheels/df/22/ac/a8966fc02b0debfa66de2104c7994b5cc6745074c57f2f695f
Step #4 - "run_e2e_tests": Successfully built torchvision
Step #4 - "run_e2e_tests": Installing collected packages: torchvision
Step #4 - "run_e2e_tests": Attempting uninstall: torchvision
Step #4 - "run_e2e_tests": Found existing installation: torchvision 0.17.1
Step #4 - "run_e2e_tests": Uninstalling torchvision-0.17.1:
Step #4 - "run_e2e_tests": Successfully uninstalled torchvision-0.17.1
Step #4 - "run_e2e_tests": Successfully installed torchvision-0.18.0a0+2c127da
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests":
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install expecttest==0.1.6
Step #4 - "run_e2e_tests": Collecting expecttest==0.1.6
Step #4 - "run_e2e_tests": Downloading expecttest-0.1.6-py3-none-any.whl (6.5 kB)
Step #4 - "run_e2e_tests": Installing collected packages: expecttest
Step #4 - "run_e2e_tests": Successfully installed expecttest-0.1.6
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests":
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install rich
Step #4 - "run_e2e_tests": Collecting rich
Step #4 - "run_e2e_tests": Downloading rich-13.7.1-py3-none-any.whl (240 kB)
Step #4 - "run_e2e_tests": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 12.9 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting markdown-it-py>=2.2.0
Step #4 - "run_e2e_tests": Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Step #4 - "run_e2e_tests": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 14.3 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting pygments<3.0.0,>=2.13.0
Step #4 - "run_e2e_tests": Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
Step #4 - "run_e2e_tests": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 61.9 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting mdurl~=0.1
Step #4 - "run_e2e_tests": Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Step #4 - "run_e2e_tests": Installing collected packages: pygments, mdurl, markdown-it-py, rich
Step #4 - "run_e2e_tests": Successfully installed markdown-it-py-3.0.0 mdurl-0.1.2 pygments-2.17.2 rich-13.7.1
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests":
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install 'torch_xla[pallas]' -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html -f https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html
Step #4 - "run_e2e_tests": Looking in links: https://storage.googleapis.com/jax-releases/jax_nightly_releases.html, https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html
Step #4 - "run_e2e_tests": Requirement already satisfied: torch_xla[pallas] in /usr/local/lib/python3.10/site-packages (2.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (6.0.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (2.1.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: cloud-tpu-client>=0.10.0 in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (0.10)
Step #4 - "run_e2e_tests": Collecting torch_xla[pallas]
Step #4 - "run_e2e_tests": Downloading torch_xla-2.2.0-cp310-cp310-manylinux_2_28_x86_64.whl (85.5 MB)
Step #4 - "run_e2e_tests": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.5/85.5 MB 11.3 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": WARNING: torch-xla 2.2.0 does not provide the extra 'pallas'
Step #4 - "run_e2e_tests": Requirement already satisfied: oauth2client in /usr/local/lib/python3.10/site-packages (from cloud-tpu-client>=0.10.0->torch_xla[pallas]) (4.1.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-api-python-client==1.8.0 in /usr/local/lib/python3.10/site-packages (from cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.8.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: httplib2<1dev,>=0.9.2 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.22.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-auth>=1.4.1 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.28.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: six<2dev,>=1.6.1 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.16.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.0.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-api-core<2dev,>=1.13.0 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.34.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (4.9)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.5.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.31.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<4.0.0dev,>=3.19.5 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.20.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.63.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/site-packages (from google-auth>=1.4.1->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (5.3.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /usr/local/lib/python3.10/site-packages (from httplib2<1dev,>=0.9.2->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.1.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.2.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.6)
Step #4 - "run_e2e_tests": Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2024.2.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.3.2)
Step #4 - "run_e2e_tests": Installing collected packages: torch_xla
Step #4 - "run_e2e_tests": Attempting uninstall: torch_xla
Step #4 - "run_e2e_tests": Found existing installation: torch-xla 2.3.0
Step #4 - "run_e2e_tests": Uninstalling torch-xla-2.3.0:
Step #4 - "run_e2e_tests": Successfully uninstalled torch-xla-2.3.0
Step #4 - "run_e2e_tests": Successfully installed torch_xla-2.2.0
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests":
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + cd /src/pytorch/xla
Step #4 - "run_e2e_tests": + TPU_LIBRARY_PATH=/usr/local/lib/python3.10/site-packages/torch_xla/lib/libtpu.so
Step #4 - "run_e2e_tests": + test/tpu/run_tests.sh
Step #4 - "run_e2e_tests": + python3 test/test_operations.py -v
Step #4 - "run_e2e_tests": terminate called after throwing an instance of 'c10::Error'
Step #4 - "run_e2e_tests": what(): Tried to register multiple backend fallbacks for the same dispatch key AutocastCUDA; previous registration registered at /src/pytorch/aten/src/ATen/autocast_mode.cpp:244, new registration registered at torch_xla/csrc/autocast_mode.cpp:25
Step #4 - "run_e2e_tests": Exception raised from registerFallback at /src/pytorch/aten/src/ATen/core/dispatch/Dispatcher.cpp:419 (most recent call first):
Step #4 - "run_e2e_tests": frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f2322046f97 in /usr/local/lib/python3.10/site-packages/torch/lib/libc10.so)
Step #4 - "run_e2e_tests": frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x65 (0x7f2321ff583c in /usr/local/lib/python3.10/site-packages/torch/lib/libc10.so)
Step #4 - "run_e2e_tests": frame #2: c10::Dispatcher::registerFallback(c10::DispatchKey, c10::KernelFunction, std::string) + 0x443 (0x7f23235232a3 in /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
Step #4 - "run_e2e_tests": frame #3: torch::Library::_fallback(torch::CppFunction&&) & + 0x264 (0x7f2323562f04 in /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
Step #4 - "run_e2e_tests": frame #4: <unknown function> + 0x51bd9b6 (0x7f20f91ce9b6 in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #5: torch::detail::TorchLibraryInit::TorchLibraryInit(torch::Library::Kind, void (*)(torch::Library&), char const*, std::optional<c10::DispatchKey>, char const*, unsigned int) + 0x62 (0x7f20f91c5ee2 in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #6: <unknown function> + 0x5022ced (0x7f20f9033ced in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #7: <unknown function> + 0xffe2 (0x7f23d0545fe2 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #8: <unknown function> + 0x100e9 (0x7f23d05460e9 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #9: _dl_catch_exception + 0xdd (0x7f23d015fb2d in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #10: <unknown function> + 0x14058 (0x7f23d054a058 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #11: _dl_catch_exception + 0x80 (0x7f23d015fad0 in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #12: <unknown function> + 0x138fa (0x7f23d05498fa in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #13: <unknown function> + 0x1258 (0x7f23d0003258 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": frame #14: _dl_catch_exception + 0x80 (0x7f23d015fad0 in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #15: _dl_catch_error + 0x2f (0x7f23d015fb8f in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #16: <unknown function> + 0x1a65 (0x7f23d0003a65 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": frame #17: dlopen + 0x44 (0x7f23d00032e4 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": <omitting python frames>
Step #4 - "run_e2e_tests":
Step #4 - "run_e2e_tests": test/tpu/run_tests.sh: line 5: 499 Aborted (core dumped) python3 test/test_operations.py -v
Step #4 - "run_e2e_tests": ++ kubectl get pod/xla-test-job-nkt9n -o 'jsonpath={.status.containerStatuses[?(@.name=="xla-test")].state.terminated.exitCode}'
Step #4 - "run_e2e_tests": + exit
Finished Step #4 - "run_e2e_tests"
PUSH
DONE
Loading