Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin update March 2024 #6677

Merged
merged 11 commits into from
Mar 13, 2024

recover local xla repo path

9003ef7
Select commit
Loading
Failed to load commit list.
Merged

Pin update March 2024 #6677

recover local xla repo path
9003ef7
Select commit
Loading
Failed to load commit list.
Google Cloud Build / ci-tpu-test-trigger (tpu-pytorch) succeeded Mar 13, 2024 in 50m 0s

Summary

Build Information

Trigger ci-tpu-test-trigger
Build 5845e302-3247-4462-b05a-5fb23f9bb08c
Start 2024-03-12T17:40:56-07:00
Duration 49m16.296s
Status SUCCESS

Steps

Step Status Duration
git_fetch SUCCESS 9.308s
git_checkout SUCCESS 584ms
build_pytorch-xla-test_docker_image SUCCESS 39m44.79s
push_pytorch-xla-test_docker_image SUCCESS 1m1.978s
run_e2e_tests SUCCESS 8m16.148s

Details

starting build "5845e302-3247-4462-b05a-5fb23f9bb08c"

FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/pytorch/xla
 * branch            9003ef75d60b9c4736392a037c298411b4099021 -> FETCH_HEAD
HEAD is now at 9003ef7 recover local xla repo path
BUILD
Starting Step #0 - "git_fetch"
Step #0 - "git_fetch": Already have image (with digest): gcr.io/cloud-builders/git
Step #0 - "git_fetch": From https://github.com/pytorch/xla
Step #0 - "git_fetch":  * [new branch]          01172023_pin_update     -> origin/01172023_pin_update
Step #0 - "git_fetch":  * [new branch]          2-0-python-3-10         -> origin/2-0-python-3-10
Step #0 - "git_fetch":  * [new branch]          4483-improve-resnet50-data-loading-performance -> origin/4483-improve-resnet50-data-loading-performance
Step #0 - "git_fetch":  * [new branch]          5d-unit-test            -> origin/5d-unit-test
Step #0 - "git_fetch":  * [new branch]          AddVimToReleasePackage  -> origin/AddVimToReleasePackage
Step #0 - "git_fetch":  * [new branch]          FixTransferToServer     -> origin/FixTransferToServer
Step #0 - "git_fetch":  * [new branch]          JackCaaoG/spmd_device_python_test -> origin/JackCaaoG/spmd_device_python_test
Step #0 - "git_fetch":  * [new branch]          JackCao/Disable_hlo_dump_test_on_tpu -> origin/JackCao/Disable_hlo_dump_test_on_tpu
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_1 -> origin/JackCao/cherry_pick_dynamo_1
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_2 -> origin/JackCao/cherry_pick_dynamo_2
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_3 -> origin/JackCao/cherry_pick_dynamo_3
Step #0 - "git_fetch":  * [new branch]          JackCao/spmd_python_device -> origin/JackCao/spmd_python_device
Step #0 - "git_fetch":  * [new branch]          JackCao/unordered_sharding_spec -> origin/JackCao/unordered_sharding_spec
Step #0 - "git_fetch":  * [new branch]          JackCao/update_to_2.0   -> origin/JackCao/update_to_2.0
Step #0 - "git_fetch":  * [new branch]          JackCao/update_to_223   -> origin/JackCao/update_to_223
Step #0 - "git_fetch":  * [new branch]          JackCao/workaround_einsum_faketensor -> origin/JackCao/workaround_einsum_faketensor
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-1        -> origin/JackCaoG-patch-1
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-2        -> origin/JackCaoG-patch-2
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-3        -> origin/JackCaoG-patch-3
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-4        -> origin/JackCaoG-patch-4
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-5        -> origin/JackCaoG-patch-5
Step #0 - "git_fetch":  * [new branch]          JackCaoG-remove-pin     -> origin/JackCaoG-remove-pin
Step #0 - "git_fetch":  * [new branch]          JackCaoG/1_13_revert_addcmul -> origin/JackCaoG/1_13_revert_addcmul
Step #0 - "git_fetch":  * [new branch]          JackCaoG/2.0_patch      -> origin/JackCaoG/2.0_patch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/AddDynamoSpmdMetrics -> origin/JackCaoG/AddDynamoSpmdMetrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/CI_build_cpp_test -> origin/JackCaoG/CI_build_cpp_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Dyanmo_execute_async -> origin/JackCaoG/Dyanmo_execute_async
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Dynamo_training_test -> origin/JackCaoG/Dynamo_training_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/PJRT_wait_buffer_sync -> origin/JackCaoG/PJRT_wait_buffer_sync
Step #0 - "git_fetch":  * [new branch]          JackCaoG/SPMD_execute_metrics -> origin/JackCaoG/SPMD_execute_metrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/SPMD_int_to_cpu -> origin/JackCaoG/SPMD_int_to_cpu
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Update_short_metrics -> origin/JackCaoG/Update_short_metrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Update_wheel_docker_to_table -> origin/JackCaoG/Update_wheel_docker_to_table
Step #0 - "git_fetch":  * [new branch]          JackCaoG/XLAShardedTensor_torch_dispatch -> origin/JackCaoG/XLAShardedTensor_torch_dispatch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_2_1_py311_build -> origin/JackCaoG/add_2_1_py311_build
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_dynamo_test_counter_check -> origin/JackCaoG/add_dynamo_test_counter_check
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_dynamo_test_to_tpu_ci -> origin/JackCaoG/add_dynamo_test_to_tpu_ci
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_gpu_doc    -> origin/JackCaoG/add_gpu_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_gpu_doc_readme -> origin/JackCaoG/add_gpu_doc_readme
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_test_dynamo_graph_dump -> origin/JackCaoG/add_test_dynamo_graph_dump
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_test_print -> origin/JackCaoG/add_test_print
Step #0 - "git_fetch":  * [new branch]          JackCaoG/api_init_runtime -> origin/JackCaoG/api_init_runtime
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backout_22_mesh_spmd -> origin/JackCaoG/backout_22_mesh_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_allgather_out -> origin/JackCaoG/backport_22_allgather_out
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_diagonal_scatter -> origin/JackCaoG/backport_22_diagonal_scatter
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_doc -> origin/JackCaoG/backport_22_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_reduce_scatter -> origin/JackCaoG/backport_22_reduce_scatter
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_reducescatter_out -> origin/JackCaoG/backport_22_reducescatter_out
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_scalar -> origin/JackCaoG/backport_22_scalar
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_transfer_gpu_scalar -> origin/JackCaoG/backport_22_transfer_gpu_scalar
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_dynamo_cpu_move -> origin/JackCaoG/backport_dynamo_cpu_move
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_frame_count -> origin/JackCaoG/backport_frame_count
Step #0 - "git_fetch":  * [new branch]          JackCaoG/barrier_comment -> origin/JackCaoG/barrier_comment
Step #0 - "git_fetch":  * [new branch]          JackCaoG/better_dynamo_error_message -> origin/JackCaoG/better_dynamo_error_message
Step #0 - "git_fetch":  * [new branch]          JackCaoG/buffer_donor_custom_op -> origin/JackCaoG/buffer_donor_custom_op
Step #0 - "git_fetch":  * [new branch]          JackCaoG/build_audio_whl -> origin/JackCaoG/build_audio_whl
Step #0 - "git_fetch":  * [new branch]          JackCaoG/bump+gpu_coverage_timeout -> origin/JackCaoG/bump+gpu_coverage_timeout
Step #0 - "git_fetch":  * [new branch]          JackCaoG/bump_build_timeout -> origin/JackCaoG/bump_build_timeout
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cache_sharded_placeholder_dynamo -> origin/JackCaoG/cache_sharded_placeholder_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_pick_dynamo_test -> origin/JackCaoG/cherry_pick_dynamo_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_pick_frame_count -> origin/JackCaoG/cherry_pick_frame_count
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_picl_xrt_crash_2.1 -> origin/JackCaoG/cherry_picl_xrt_crash_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clang-format-11 -> origin/JackCaoG/clang-format-11
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clang-format-11-2 -> origin/JackCaoG/clang-format-11-2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clean_up_gettensor -> origin/JackCaoG/clean_up_gettensor
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clean_up_use_virtual_device -> origin/JackCaoG/clean_up_use_virtual_device
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cleanup_docs   -> origin/JackCaoG/cleanup_docs
Step #0 - "git_fetch":  * [new branch]          JackCaoG/colab_whl_abi  -> origin/JackCaoG/colab_whl_abi
Step #0 - "git_fetch":  * [new branch]          JackCaoG/colab_whl_abi_2 -> origin/JackCaoG/colab_whl_abi_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/data_to_string -> origin/JackCaoG/data_to_string
Step #0 - "git_fetch":  * [new branch]          JackCaoG/debug_fix      -> origin/JackCaoG/debug_fix
Step #0 - "git_fetch":  * [new branch]          JackCaoG/debug_tool     -> origin/JackCaoG/debug_tool
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_coverage -> origin/JackCaoG/disable_coverage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_dynamo_different_shape_test -> origin/JackCaoG/disable_dynamo_different_shape_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test -> origin/JackCaoG/disable_flaky_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test_2 -> origin/JackCaoG/disable_flaky_test_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test_3 -> origin/JackCaoG/disable_flaky_test_3
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_more_dynamic -> origin/JackCaoG/disable_more_dynamic
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_more_dynamic_test -> origin/JackCaoG/disable_more_dynamic_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_pjrt_computation_client_test -> origin/JackCaoG/disable_pjrt_computation_client_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_test_xla_sharding -> origin/JackCaoG/disable_test_xla_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disbale_docker_sandbox -> origin/JackCaoG/disbale_docker_sandbox
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dump_output_sharding -> origin/JackCaoG/dump_output_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_2.0_cherry_pick -> origin/JackCaoG/dynamo_2.0_cherry_pick
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_aliasing_2 -> origin/JackCaoG/dynamo_aliasing_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_debug_message -> origin/JackCaoG/dynamo_debug_message
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_different_graph -> origin/JackCaoG/dynamo_different_graph
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_do_not_markstep_to_sync_input -> origin/JackCaoG/dynamo_do_not_markstep_to_sync_input
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_doc     -> origin/JackCaoG/dynamo_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_dump_tensor_file -> origin/JackCaoG/dynamo_dump_tensor_file
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_note_backport_2.1 -> origin/JackCaoG/dynamo_note_backport_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_only_warm_up_cache -> origin/JackCaoG/dynamo_only_warm_up_cache
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_optimizer -> origin/JackCaoG/dynamo_optimizer
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_reduce_execute -> origin/JackCaoG/dynamo_reduce_execute
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_rng_seed_without_mark_step -> origin/JackCaoG/dynamo_rng_seed_without_mark_step
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd    -> origin/JackCaoG/dynamo_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd_input_sharding_change -> origin/JackCaoG/dynamo_spmd_input_sharding_change
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd_optimization -> origin/JackCaoG/dynamo_spmd_optimization
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_sync_input -> origin/JackCaoG/dynamo_sync_input
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_warning -> origin/JackCaoG/dynamo_warning
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_cumsum_long -> origin/JackCaoG/enable_cumsum_long
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_dynamo_upstream_test -> origin/JackCaoG/enable_dynamo_upstream_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_fsdp_gpu_test -> origin/JackCaoG/enable_fsdp_gpu_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_mp_test_on_tpuci -> origin/JackCaoG/enable_mp_test_on_tpuci
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_reduce_scatter_test_gpu -> origin/JackCaoG/enable_reduce_scatter_test_gpu
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_cc_linter_once -> origin/JackCaoG/fix_cc_linter_once
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_deprecated_test_oepration_api -> origin/JackCaoG/fix_deprecated_test_oepration_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_diagonal_scatter_dim -> origin/JackCaoG/fix_diagonal_scatter_dim
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dyanmo_additional_tracing -> origin/JackCaoG/fix_dyanmo_additional_tracing
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo     -> origin/JackCaoG/fix_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_crash -> origin/JackCaoG/fix_dynamo_crash
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_inplace -> origin/JackCaoG/fix_dynamo_inplace
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_input_sharding_test -> origin/JackCaoG/fix_dynamo_input_sharding_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_text_allclose -> origin/JackCaoG/fix_dynamo_text_allclose
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_hlo_dump   -> origin/JackCaoG/fix_hlo_dump
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_hlo_dump_spmd -> origin/JackCaoG/fix_hlo_dump_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_mark_step_on_spmd_device -> origin/JackCaoG/fix_mark_step_on_spmd_device
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_mp_save_test -> origin/JackCaoG/fix_mp_save_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_send_cpu_data -> origin/JackCaoG/fix_send_cpu_data
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_spmd_buff_is_null -> origin/JackCaoG/fix_spmd_buff_is_null
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_tpu_ci_accuracy -> origin/JackCaoG/fix_tpu_ci_accuracy
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_typo       -> origin/JackCaoG/fix_typo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_vision_build -> origin/JackCaoG/fix_vision_build
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fsdp_autowrap_test -> origin/JackCaoG/fsdp_autowrap_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/gpu_build_pytorch_cuda -> origin/JackCaoG/gpu_build_pytorch_cuda
Step #0 - "git_fetch":  * [new branch]          JackCaoG/graph_hash_debug -> origin/JackCaoG/graph_hash_debug
Step #0 - "git_fetch":  * [new branch]          JackCaoG/handle_dynamo_fn_without_arg -> origin/JackCaoG/handle_dynamo_fn_without_arg
Step #0 - "git_fetch":  * [new branch]          JackCaoG/improve_data_tostring -> origin/JackCaoG/improve_data_tostring
Step #0 - "git_fetch":  * [new branch]          JackCaoG/increase_timeout_docker -> origin/JackCaoG/increase_timeout_docker
Step #0 - "git_fetch":  * [new branch]          JackCaoG/ir_value_sharding -> origin/JackCaoG/ir_value_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/lower_dropout_backward -> origin/JackCaoG/lower_dropout_backward
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_op_lowering_guide -> origin/JackCaoG/move_op_lowering_guide
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_pin_2_0   -> origin/JackCaoG/move_pin_2_0
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_pjrt_lock_location -> origin/JackCaoG/move_pjrt_lock_location
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_return_future -> origin/JackCaoG/move_return_future
Step #0 - "git_fetch":  * [new branch]          JackCaoG/new_alias_api  -> origin/JackCaoG/new_alias_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/nightly_build_python -> origin/JackCaoG/nightly_build_python
Step #0 - "git_fetch":  * [new branch]          JackCaoG/parallel_spmd  -> origin/JackCaoG/parallel_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pin_expecttest -> origin/JackCaoG/pin_expecttest
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pin_update_20230808 -> origin/JackCaoG/pin_update_20230808
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_5 -> origin/JackCaoG/pt_xla_debug_5
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_dynamo_execute -> origin/JackCaoG/pt_xla_debug_dynamo_execute
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_max_frame -> origin/JackCaoG/pt_xla_debug_max_frame
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r20_backportdoc -> origin/JackCaoG/r20_backportdoc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_ABI_0     -> origin/JackCaoG/r2_0_ABI_0
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_back_port_profile_flag -> origin/JackCaoG/r2_0_back_port_profile_flag
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_backport_loader_thread -> origin/JackCaoG/r2_0_backport_loader_thread
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_backport_loader_thread_2 -> origin/JackCaoG/r2_0_backport_loader_thread_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_cherry_pick_defer_init -> origin/JackCaoG/r2_0_cherry_pick_defer_init
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_pin     -> origin/JackCaoG/remove_pin
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_reshape_again -> origin/JackCaoG/remove_reshape_again
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_reshape_lowering -> origin/JackCaoG/remove_reshape_lowering
Step #0 - "git_fetch":  * [new branch]          JackCaoG/restrict_mark_sharding -> origin/JackCaoG/restrict_mark_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/revert_addcmul -> origin/JackCaoG/revert_addcmul
Step #0 - "git_fetch":  * [new branch]          JackCaoG/revert_protobuf_tf_patch -> origin/JackCaoG/revert_protobuf_tf_patch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/shard_device_data_ir -> origin/JackCaoG/shard_device_data_ir
Step #0 - "git_fetch":  * [new branch]          JackCaoG/skip_partitioner -> origin/JackCaoG/skip_partitioner
Step #0 - "git_fetch":  * [new branch]          JackCaoG/skip_tpu_test_precision -> origin/JackCaoG/skip_tpu_test_precision
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_aliasing  -> origin/JackCaoG/spmd_aliasing
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_dynamo_sharding_changed -> origin/JackCaoG/spmd_dynamo_sharding_changed
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_incorrect_result -> origin/JackCaoG/spmd_incorrect_result
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tensor_debugging_api -> origin/JackCaoG/tensor_debugging_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tensor_with_xla_data_sharding -> origin/JackCaoG/tensor_with_xla_data_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/test_torch_compile_usage -> origin/JackCaoG/test_torch_compile_usage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tf_update_20230201 -> origin/JackCaoG/tf_update_20230201
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tracing_timer  -> origin/JackCaoG/tracing_timer
Step #0 - "git_fetch":  * [new branch]          JackCaoG/trouble_shooting_update -> origin/JackCaoG/trouble_shooting_update
Step #0 - "git_fetch":  * [new branch]          JackCaoG/truncate_python_stack -> origin/JackCaoG/truncate_python_stack
Step #0 - "git_fetch":  * [new branch]          JackCaoG/unordered_partial_sharding_spec -> origin/JackCaoG/unordered_partial_sharding_spec
Step #0 - "git_fetch":  * [new branch]          JackCaoG/updateDocker   -> origin/JackCaoG/updateDocker
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_2_0_ABI -> origin/JackCaoG/update_2_0_ABI
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_API_GUIDE -> origin/JackCaoG/update_API_GUIDE
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE -> origin/JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_build_guide -> origin/JackCaoG/update_build_guide
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_colab_2_10_whl -> origin/JackCaoG/update_colab_2_10_whl
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_doc1    -> origin/JackCaoG/update_doc1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_doc_remove_xrt -> origin/JackCaoG/update_doc_remove_xrt
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_docker_instruction -> origin/JackCaoG/update_docker_instruction
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_backend_name -> origin/JackCaoG/update_dynamo_backend_name
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_backend_nograd -> origin/JackCaoG/update_dynamo_backend_nograd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_doc -> origin/JackCaoG/update_dynamo_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_number_2.1 -> origin/JackCaoG/update_dynamo_number_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_readme -> origin/JackCaoG/update_dynamo_readme
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_save_graph_path -> origin/JackCaoG/update_dynamo_save_graph_path
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_torch_equal -> origin/JackCaoG/update_dynamo_torch_equal
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_doc -> origin/JackCaoG/update_gpu_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_wheel_link -> origin/JackCaoG/update_gpu_wheel_link
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_wheellink -> origin/JackCaoG/update_gpu_wheellink
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_pjrt_future -> origin/JackCaoG/update_pjrt_future
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_readme_2 -> origin/JackCaoG/update_readme_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_set_default_tensor_type -> origin/JackCaoG/update_set_default_tensor_type
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_tf_2023018 -> origin/JackCaoG/update_tf_2023018
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_to_2.2  -> origin/JackCaoG/update_to_2.2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_to_223  -> origin/JackCaoG/update_to_223
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_torch_compile_usage -> origin/JackCaoG/update_torch_compile_usage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_trouble_shotting_doc -> origin/JackCaoG/update_trouble_shotting_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_troubleshotting_with_sanitycheck -> origin/JackCaoG/update_troubleshotting_with_sanitycheck
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_wheel_docker_format -> origin/JackCaoG/update_wheel_docker_format
Step #0 - "git_fetch":  * [new branch]          JackCaoG/use_new_alias_api -> origin/JackCaoG/use_new_alias_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/weak_ref_dynamo -> origin/JackCaoG/weak_ref_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/xrt_cherry_pick_07_10 -> origin/JackCaoG/xrt_cherry_pick_07_10
Step #0 - "git_fetch":  * [new branch]          JackCaoG/xrt_cherrypick_0721 -> origin/JackCaoG/xrt_cherrypick_0721
Step #0 - "git_fetch":  * [new branch]          JackCaog/dynamo_input_sharding -> origin/JackCaog/dynamo_input_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaog/quick_spmd_fix -> origin/JackCaog/quick_spmd_fix
Step #0 - "git_fetch":  * [new branch]          JackCaog/spmd_device_default -> origin/JackCaog/spmd_device_default
Step #0 - "git_fetch":  * [new branch]          Liyang90-patch-1        -> origin/Liyang90-patch-1
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-1       -> origin/ManfeiBai-patch-1
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-10      -> origin/ManfeiBai-patch-10
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-11      -> origin/ManfeiBai-patch-11
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-12      -> origin/ManfeiBai-patch-12
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-13      -> origin/ManfeiBai-patch-13
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-14      -> origin/ManfeiBai-patch-14
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-15      -> origin/ManfeiBai-patch-15
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-16      -> origin/ManfeiBai-patch-16
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-17      -> origin/ManfeiBai-patch-17
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-18      -> origin/ManfeiBai-patch-18
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-19      -> origin/ManfeiBai-patch-19
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-2       -> origin/ManfeiBai-patch-2
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-20      -> origin/ManfeiBai-patch-20
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-21      -> origin/ManfeiBai-patch-21
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-22      -> origin/ManfeiBai-patch-22
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-23      -> origin/ManfeiBai-patch-23
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-24      -> origin/ManfeiBai-patch-24
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-25      -> origin/ManfeiBai-patch-25
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-26      -> origin/ManfeiBai-patch-26
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-27      -> origin/ManfeiBai-patch-27
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-28      -> origin/ManfeiBai-patch-28
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-29      -> origin/ManfeiBai-patch-29
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-3       -> origin/ManfeiBai-patch-3
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-30      -> origin/ManfeiBai-patch-30
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-31      -> origin/ManfeiBai-patch-31
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-32      -> origin/ManfeiBai-patch-32
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-33      -> origin/ManfeiBai-patch-33
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-34      -> origin/ManfeiBai-patch-34
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-35      -> origin/ManfeiBai-patch-35
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-36      -> origin/ManfeiBai-patch-36
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-37      -> origin/ManfeiBai-patch-37
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-38      -> origin/ManfeiBai-patch-38
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-39      -> origin/ManfeiBai-patch-39
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-4       -> origin/ManfeiBai-patch-4
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-40      -> origin/ManfeiBai-patch-40
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-41      -> origin/ManfeiBai-patch-41
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-41-test -> origin/ManfeiBai-patch-41-test
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-42      -> origin/ManfeiBai-patch-42
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-43      -> origin/ManfeiBai-patch-43
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-44      -> origin/ManfeiBai-patch-44
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-45      -> origin/ManfeiBai-patch-45
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-46      -> origin/ManfeiBai-patch-46
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-47      -> origin/ManfeiBai-patch-47
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-48      -> origin/ManfeiBai-patch-48
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-49      -> origin/ManfeiBai-patch-49
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-5       -> origin/ManfeiBai-patch-5
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-50      -> origin/ManfeiBai-patch-50
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-51      -> origin/ManfeiBai-patch-51
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-52      -> origin/ManfeiBai-patch-52
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-53      -> origin/ManfeiBai-patch-53
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-54      -> origin/ManfeiBai-patch-54
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-55      -> origin/ManfeiBai-patch-55
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-56      -> origin/ManfeiBai-patch-56
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-57      -> origin/ManfeiBai-patch-57
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-58      -> origin/ManfeiBai-patch-58
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-59    
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds;region=us-central1/5845e302-3247-4462-b05a-5fb23f9bb08c?project=164006649440.]
...
.0 torch-xla-2.3.0 typing-extensions-4.10.0 uritemplate-3.0.1 urllib3-2.2.1
Step #2 - "build_pytorch-xla-test_docker_image": �[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #2 - "build_pytorch-xla-test_docker_image": �[0m�[91m
Step #2 - "build_pytorch-xla-test_docker_image": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #2 - "build_pytorch-xla-test_docker_image": [notice] To update, run: pip install --upgrade pip
Step #2 - "build_pytorch-xla-test_docker_image": �[0mRemoving intermediate container bf593dc52c90
Step #2 - "build_pytorch-xla-test_docker_image":  ---> 3d38cf2da9c6
Step #2 - "build_pytorch-xla-test_docker_image": Step 24/25 : WORKDIR /
Step #2 - "build_pytorch-xla-test_docker_image":  ---> Running in b3fee7c84e93
Step #2 - "build_pytorch-xla-test_docker_image": Removing intermediate container b3fee7c84e93
Step #2 - "build_pytorch-xla-test_docker_image":  ---> 079a8b0552d0
Step #2 - "build_pytorch-xla-test_docker_image": Step 25/25 : RUN rm -rf /ansible /tmp/wheels
Step #2 - "build_pytorch-xla-test_docker_image":  ---> Running in f37cb91268ee
Step #2 - "build_pytorch-xla-test_docker_image": Removing intermediate container f37cb91268ee
Step #2 - "build_pytorch-xla-test_docker_image":  ---> f837a35cb485
Step #2 - "build_pytorch-xla-test_docker_image": Successfully built f837a35cb485
Step #2 - "build_pytorch-xla-test_docker_image": Successfully tagged us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #2 - "build_pytorch-xla-test_docker_image": Successfully tagged local_image:latest
Finished Step #2 - "build_pytorch-xla-test_docker_image"
Starting Step #3 - "push_pytorch-xla-test_docker_image"
Step #3 - "push_pytorch-xla-test_docker_image": Already have image (with digest): gcr.io/cloud-builders/docker
Step #3 - "push_pytorch-xla-test_docker_image": The push refers to repository [us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test]
Step #3 - "push_pytorch-xla-test_docker_image": 09fb5087fa97: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 911661ce5cae: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8cef9b706f27: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8e983065a15a: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 705501a374bd: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 0e5768b110ff: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Preparing
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Waiting
Step #3 - "push_pytorch-xla-test_docker_image": 705501a374bd: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 09fb5087fa97: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": b669c623e79c: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 9a0dbac5d018: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": c58c23d5cc09: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 596ba7929e20: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 8d684d540706: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 8e983065a15a: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": c2ccc8984c53: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 5856b6e60177: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": b6258dcae7d4: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": a3f42debabf6: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 61dd36fe7bf5: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 0e5768b110ff: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": b04cebd30172: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 42a291ef6d1f: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": ac2949c69757: Layer already exists
Step #3 - "push_pytorch-xla-test_docker_image": 8cef9b706f27: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 7a939bdfe999: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 911661ce5cae: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 1d83e433402f: Pushed
Step #3 - "push_pytorch-xla-test_docker_image": 5845e302-3247-4462-b05a-5fb23f9bb08c: digest: sha256:c2637622d0c7bbccb63dd2dbaf3989865c14cabfeca09236d1eb3cd404f6a426 size: 4733
Finished Step #3 - "push_pytorch-xla-test_docker_image"
Starting Step #4 - "run_e2e_tests"
Step #4 - "run_e2e_tests": Pulling image: google/cloud-sdk
Step #4 - "run_e2e_tests": Using default tag: latest
Step #4 - "run_e2e_tests": latest: Pulling from google/cloud-sdk
Step #4 - "run_e2e_tests": ec335f17d0c7: Already exists
Step #4 - "run_e2e_tests": 8998ef4d29e2: Pulling fs layer
Step #4 - "run_e2e_tests": 47fcb7f183ee: Pulling fs layer
Step #4 - "run_e2e_tests": fc58c263007f: Pulling fs layer
Step #4 - "run_e2e_tests": e5c2b4203b75: Pulling fs layer
Step #4 - "run_e2e_tests": 0894066a283a: Pulling fs layer
Step #4 - "run_e2e_tests": 574b19029fbb: Pulling fs layer
Step #4 - "run_e2e_tests": e5c2b4203b75: Waiting
Step #4 - "run_e2e_tests": 0894066a283a: Waiting
Step #4 - "run_e2e_tests": 574b19029fbb: Waiting
Step #4 - "run_e2e_tests": fc58c263007f: Verifying Checksum
Step #4 - "run_e2e_tests": fc58c263007f: Download complete
Step #4 - "run_e2e_tests": 8998ef4d29e2: Verifying Checksum
Step #4 - "run_e2e_tests": 8998ef4d29e2: Download complete
Step #4 - "run_e2e_tests": 47fcb7f183ee: Verifying Checksum
Step #4 - "run_e2e_tests": 47fcb7f183ee: Download complete
Step #4 - "run_e2e_tests": 574b19029fbb: Verifying Checksum
Step #4 - "run_e2e_tests": 574b19029fbb: Download complete
Step #4 - "run_e2e_tests": 8998ef4d29e2: Pull complete
Step #4 - "run_e2e_tests": 0894066a283a: Verifying Checksum
Step #4 - "run_e2e_tests": 0894066a283a: Download complete
Step #4 - "run_e2e_tests": 47fcb7f183ee: Pull complete
Step #4 - "run_e2e_tests": fc58c263007f: Pull complete
Step #4 - "run_e2e_tests": e5c2b4203b75: Verifying Checksum
Step #4 - "run_e2e_tests": e5c2b4203b75: Download complete
Step #4 - "run_e2e_tests": e5c2b4203b75: Pull complete
Step #4 - "run_e2e_tests": 0894066a283a: Pull complete
Step #4 - "run_e2e_tests": 574b19029fbb: Pull complete
Step #4 - "run_e2e_tests": Digest: sha256:4a4884995f6734698aa910e43c0966502a8e980fdaee3ac04bf956b912921ecd
Step #4 - "run_e2e_tests": Status: Downloaded newer image for google/cloud-sdk:latest
Step #4 - "run_e2e_tests": docker.io/google/cloud-sdk:latest
Step #4 - "run_e2e_tests": + apt-get update
Step #4 - "run_e2e_tests": Get:1 https://packages.cloud.google.com/apt cloud-sdk-bullseye InRelease [6406 B]
Step #4 - "run_e2e_tests": Hit:2 http://deb.debian.org/debian bullseye InRelease
Step #4 - "run_e2e_tests": Hit:3 http://deb.debian.org/debian-security bullseye-security InRelease
Step #4 - "run_e2e_tests": Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Step #4 - "run_e2e_tests": Get:5 https://packages.cloud.google.com/apt cloud-sdk-bullseye/main amd64 Packages [468 kB]
Step #4 - "run_e2e_tests": Fetched 519 kB in 1s (1023 kB/s)
Step #4 - "run_e2e_tests": Reading package lists...
Step #4 - "run_e2e_tests": + apt-get -y install gettext
Step #4 - "run_e2e_tests": Reading package lists...
Step #4 - "run_e2e_tests": Building dependency tree...
Step #4 - "run_e2e_tests": Reading state information...
Step #4 - "run_e2e_tests": The following additional packages will be installed:
Step #4 - "run_e2e_tests":   gettext-base
Step #4 - "run_e2e_tests": Suggested packages:
Step #4 - "run_e2e_tests":   gettext-doc autopoint libasprintf-dev libgettextpo-dev
Step #4 - "run_e2e_tests": The following NEW packages will be installed:
Step #4 - "run_e2e_tests":   gettext gettext-base
Step #4 - "run_e2e_tests": 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Step #4 - "run_e2e_tests": Need to get 1486 kB of archives.
Step #4 - "run_e2e_tests": After this operation, 6658 kB of additional disk space will be used.
Step #4 - "run_e2e_tests": Get:1 http://deb.debian.org/debian bullseye/main amd64 gettext-base amd64 0.21-4 [175 kB]
Step #4 - "run_e2e_tests": Get:2 http://deb.debian.org/debian bullseye/main amd64 gettext amd64 0.21-4 [1311 kB]
Step #4 - "run_e2e_tests": debconf: delaying package configuration, since apt-utils is not installed
Step #4 - "run_e2e_tests": Fetched 1486 kB in 0s (5250 kB/s)
Step #4 - "run_e2e_tests": Selecting previously unselected package gettext-base.
Step #4 - "run_e2e_tests": (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 70391 files and directories currently installed.)
Step #4 - "run_e2e_tests": Preparing to unpack .../gettext-base_0.21-4_amd64.deb ...
Step #4 - "run_e2e_tests": Unpacking gettext-base (0.21-4) ...
Step #4 - "run_e2e_tests": Selecting previously unselected package gettext.
Step #4 - "run_e2e_tests": Preparing to unpack .../gettext_0.21-4_amd64.deb ...
Step #4 - "run_e2e_tests": Unpacking gettext (0.21-4) ...
Step #4 - "run_e2e_tests": Setting up gettext-base (0.21-4) ...
Step #4 - "run_e2e_tests": Setting up gettext (0.21-4) ...
Step #4 - "run_e2e_tests": Processing triggers for libc-bin (2.31-13+deb11u8) ...
Step #4 - "run_e2e_tests": + gcloud container clusters get-credentials tpu-cluster --zone europe-west4-a
Step #4 - "run_e2e_tests": Fetching cluster endpoint and auth data.
Step #4 - "run_e2e_tests": kubeconfig entry generated for tpu-cluster.
Step #4 - "run_e2e_tests": + export IMAGE=us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #4 - "run_e2e_tests": + IMAGE=us-central1-docker.pkg.dev/tpu-pytorch/docker/pytorch-xla-test:5845e302-3247-4462-b05a-5fb23f9bb08c
Step #4 - "run_e2e_tests": + export PROJECT_ID=tpu-pytorch
Step #4 - "run_e2e_tests": + PROJECT_ID=tpu-pytorch
Step #4 - "run_e2e_tests": ++ envsubst
Step #4 - "run_e2e_tests": ++ kubectl create -f - -o name
Step #4 - "run_e2e_tests": + pod_name=pod/xla-test-job-nkt9n
Step #4 - "run_e2e_tests": ++ kubectl wait --for condition=ready --timeout=60m pod/xla-test-job-nkt9n -o name
Step #4 - "run_e2e_tests": + pod_name=pod/xla-test-job-nkt9n
Step #4 - "run_e2e_tests": + kubectl logs -f pod/xla-test-job-nkt9n --container=xla-test
Step #4 - "run_e2e_tests": ++ cat /src/pytorch/.github/ci_commit_pins/vision.txt
Step #4 - "run_e2e_tests": + pip install --no-use-pep517 git+https://github.com/pytorch/vision.git@2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests": Collecting git+https://github.com/pytorch/vision.git@2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests":   Cloning https://github.com/pytorch/vision.git (to revision 2c127da8b5e2e8f44b50994c6cb931bcca267cfe) to /tmp/pip-req-build-g277s747
Step #4 - "run_e2e_tests":   Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-g277s747
Step #4 - "run_e2e_tests":   Running command git rev-parse -q --verify 'sha^2c127da8b5e2e8f44b50994c6cb931bcca267cfe'
Step #4 - "run_e2e_tests":   Running command git fetch -q https://github.com/pytorch/vision.git 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests":   Running command git checkout -q 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests":   Resolved https://github.com/pytorch/vision.git to commit 2c127da8b5e2e8f44b50994c6cb931bcca267cfe
Step #4 - "run_e2e_tests":   Preparing metadata (setup.py): started
Step #4 - "run_e2e_tests":   Preparing metadata (setup.py): finished with status 'done'
Step #4 - "run_e2e_tests": Requirement already satisfied: numpy in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (1.26.4)
Step #4 - "run_e2e_tests": Requirement already satisfied: torch in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (2.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/site-packages (from torchvision==0.18.0a0+2c127da) (10.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: filelock in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.13.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: fsspec in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (2024.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.1.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: sympy in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (1.12)
Step #4 - "run_e2e_tests": Requirement already satisfied: networkx in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (3.2.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/site-packages (from torch->torchvision==0.18.0a0+2c127da) (4.10.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.18.0a0+2c127da) (2.1.5)
Step #4 - "run_e2e_tests": Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/site-packages (from sympy->torch->torchvision==0.18.0a0+2c127da) (1.3.0)
Step #4 - "run_e2e_tests": Building wheels for collected packages: torchvision
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): started
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): still running...
Step #4 - "run_e2e_tests":   Building wheel for torchvision (setup.py): finished with status 'done'
Step #4 - "run_e2e_tests":   Created wheel for torchvision: filename=torchvision-0.18.0a0+2c127da-cp310-cp310-linux_x86_64.whl size=1155955 sha256=c0d63207ed227ed9421b2eb781fc957a6410d6ec00197f2070e48a6aba1baf99
Step #4 - "run_e2e_tests":   Stored in directory: /root/.cache/pip/wheels/df/22/ac/a8966fc02b0debfa66de2104c7994b5cc6745074c57f2f695f
Step #4 - "run_e2e_tests": Successfully built torchvision
Step #4 - "run_e2e_tests": Installing collected packages: torchvision
Step #4 - "run_e2e_tests":   Attempting uninstall: torchvision
Step #4 - "run_e2e_tests":     Found existing installation: torchvision 0.17.1
Step #4 - "run_e2e_tests":     Uninstalling torchvision-0.17.1:
Step #4 - "run_e2e_tests":       Successfully uninstalled torchvision-0.17.1
Step #4 - "run_e2e_tests": Successfully installed torchvision-0.18.0a0+2c127da
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests": 
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install expecttest==0.1.6
Step #4 - "run_e2e_tests": Collecting expecttest==0.1.6
Step #4 - "run_e2e_tests":   Downloading expecttest-0.1.6-py3-none-any.whl (6.5 kB)
Step #4 - "run_e2e_tests": Installing collected packages: expecttest
Step #4 - "run_e2e_tests": Successfully installed expecttest-0.1.6
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests": 
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install rich
Step #4 - "run_e2e_tests": Collecting rich
Step #4 - "run_e2e_tests":   Downloading rich-13.7.1-py3-none-any.whl (240 kB)
Step #4 - "run_e2e_tests":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 12.9 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting markdown-it-py>=2.2.0
Step #4 - "run_e2e_tests":   Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Step #4 - "run_e2e_tests":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 14.3 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting pygments<3.0.0,>=2.13.0
Step #4 - "run_e2e_tests":   Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
Step #4 - "run_e2e_tests":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 61.9 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": Collecting mdurl~=0.1
Step #4 - "run_e2e_tests":   Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Step #4 - "run_e2e_tests": Installing collected packages: pygments, mdurl, markdown-it-py, rich
Step #4 - "run_e2e_tests": Successfully installed markdown-it-py-3.0.0 mdurl-0.1.2 pygments-2.17.2 rich-13.7.1
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests": 
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + pip install 'torch_xla[pallas]' -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html -f https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html
Step #4 - "run_e2e_tests": Looking in links: https://storage.googleapis.com/jax-releases/jax_nightly_releases.html, https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html
Step #4 - "run_e2e_tests": Requirement already satisfied: torch_xla[pallas] in /usr/local/lib/python3.10/site-packages (2.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (6.0.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (2.1.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: cloud-tpu-client>=0.10.0 in /usr/local/lib/python3.10/site-packages (from torch_xla[pallas]) (0.10)
Step #4 - "run_e2e_tests": Collecting torch_xla[pallas]
Step #4 - "run_e2e_tests":   Downloading torch_xla-2.2.0-cp310-cp310-manylinux_2_28_x86_64.whl (85.5 MB)
Step #4 - "run_e2e_tests":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.5/85.5 MB 11.3 MB/s eta 0:00:00
Step #4 - "run_e2e_tests": WARNING: torch-xla 2.2.0 does not provide the extra 'pallas'
Step #4 - "run_e2e_tests": Requirement already satisfied: oauth2client in /usr/local/lib/python3.10/site-packages (from cloud-tpu-client>=0.10.0->torch_xla[pallas]) (4.1.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-api-python-client==1.8.0 in /usr/local/lib/python3.10/site-packages (from cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.8.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: httplib2<1dev,>=0.9.2 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.22.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-auth>=1.4.1 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.28.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: six<2dev,>=1.6.1 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.16.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.0.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-api-core<2dev,>=1.13.0 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.34.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.10/site-packages (from google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.2.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (4.9)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.5.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python3.10/site-packages (from oauth2client->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (0.3.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.31.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<4.0.0dev,>=3.19.5 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.20.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /usr/local/lib/python3.10/site-packages (from google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (1.63.0)
Step #4 - "run_e2e_tests": Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/site-packages (from google-auth>=1.4.1->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (5.3.3)
Step #4 - "run_e2e_tests": Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /usr/local/lib/python3.10/site-packages (from httplib2<1dev,>=0.9.2->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.1.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2.2.1)
Step #4 - "run_e2e_tests": Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.6)
Step #4 - "run_e2e_tests": Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (2024.2.2)
Step #4 - "run_e2e_tests": Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2dev,>=1.13.0->google-api-python-client==1.8.0->cloud-tpu-client>=0.10.0->torch_xla[pallas]) (3.3.2)
Step #4 - "run_e2e_tests": Installing collected packages: torch_xla
Step #4 - "run_e2e_tests":   Attempting uninstall: torch_xla
Step #4 - "run_e2e_tests":     Found existing installation: torch-xla 2.3.0
Step #4 - "run_e2e_tests":     Uninstalling torch-xla-2.3.0:
Step #4 - "run_e2e_tests":       Successfully uninstalled torch-xla-2.3.0
Step #4 - "run_e2e_tests": Successfully installed torch_xla-2.2.0
Step #4 - "run_e2e_tests": WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #4 - "run_e2e_tests": 
Step #4 - "run_e2e_tests": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #4 - "run_e2e_tests": [notice] To update, run: pip install --upgrade pip
Step #4 - "run_e2e_tests": + cd /src/pytorch/xla
Step #4 - "run_e2e_tests": + TPU_LIBRARY_PATH=/usr/local/lib/python3.10/site-packages/torch_xla/lib/libtpu.so
Step #4 - "run_e2e_tests": + test/tpu/run_tests.sh
Step #4 - "run_e2e_tests": + python3 test/test_operations.py -v
Step #4 - "run_e2e_tests": terminate called after throwing an instance of 'c10::Error'
Step #4 - "run_e2e_tests":   what():  Tried to register multiple backend fallbacks for the same dispatch key AutocastCUDA; previous registration registered at /src/pytorch/aten/src/ATen/autocast_mode.cpp:244, new registration registered at torch_xla/csrc/autocast_mode.cpp:25
Step #4 - "run_e2e_tests": Exception raised from registerFallback at /src/pytorch/aten/src/ATen/core/dispatch/Dispatcher.cpp:419 (most recent call first):
Step #4 - "run_e2e_tests": frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f2322046f97 in /usr/local/lib/python3.10/site-packages/torch/lib/libc10.so)
Step #4 - "run_e2e_tests": frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x65 (0x7f2321ff583c in /usr/local/lib/python3.10/site-packages/torch/lib/libc10.so)
Step #4 - "run_e2e_tests": frame #2: c10::Dispatcher::registerFallback(c10::DispatchKey, c10::KernelFunction, std::string) + 0x443 (0x7f23235232a3 in /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
Step #4 - "run_e2e_tests": frame #3: torch::Library::_fallback(torch::CppFunction&&) & + 0x264 (0x7f2323562f04 in /usr/local/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so)
Step #4 - "run_e2e_tests": frame #4: <unknown function> + 0x51bd9b6 (0x7f20f91ce9b6 in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #5: torch::detail::TorchLibraryInit::TorchLibraryInit(torch::Library::Kind, void (*)(torch::Library&), char const*, std::optional<c10::DispatchKey>, char const*, unsigned int) + 0x62 (0x7f20f91c5ee2 in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #6: <unknown function> + 0x5022ced (0x7f20f9033ced in /usr/local/lib/python3.10/site-packages/_XLAC.cpython-310-x86_64-linux-gnu.so)
Step #4 - "run_e2e_tests": frame #7: <unknown function> + 0xffe2 (0x7f23d0545fe2 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #8: <unknown function> + 0x100e9 (0x7f23d05460e9 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #9: _dl_catch_exception + 0xdd (0x7f23d015fb2d in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #10: <unknown function> + 0x14058 (0x7f23d054a058 in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #11: _dl_catch_exception + 0x80 (0x7f23d015fad0 in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #12: <unknown function> + 0x138fa (0x7f23d05498fa in /lib64/ld-linux-x86-64.so.2)
Step #4 - "run_e2e_tests": frame #13: <unknown function> + 0x1258 (0x7f23d0003258 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": frame #14: _dl_catch_exception + 0x80 (0x7f23d015fad0 in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #15: _dl_catch_error + 0x2f (0x7f23d015fb8f in /lib/x86_64-linux-gnu/libc.so.6)
Step #4 - "run_e2e_tests": frame #16: <unknown function> + 0x1a65 (0x7f23d0003a65 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": frame #17: dlopen + 0x44 (0x7f23d00032e4 in /lib/x86_64-linux-gnu/libdl.so.2)
Step #4 - "run_e2e_tests": <omitting python frames>
Step #4 - "run_e2e_tests": 
Step #4 - "run_e2e_tests": test/tpu/run_tests.sh: line 5:   499 Aborted                 (core dumped) python3 test/test_operations.py -v
Step #4 - "run_e2e_tests": ++ kubectl get pod/xla-test-job-nkt9n -o 'jsonpath={.status.containerStatuses[?(@.name=="xla-test")].state.terminated.exitCode}'
Step #4 - "run_e2e_tests": + exit
Finished Step #4 - "run_e2e_tests"
PUSH
DONE

Build Log: https://console.cloud.google.com/cloud-build/builds;region=us-central1/5845e302-3247-4462-b05a-5fb23f9bb08c?project=164006649440