Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

disable some models for torchbench #6352

Merged
merged 16 commits into from
Jan 24, 2024
Merged

disable some models for torchbench #6352

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
206cf25
disable some models for torchbench
zpcore Jan 22, 2024
f8f06df
stricktly remove some models for benchmarking
zpcore Jan 22, 2024
3173e75
Merge remote-tracking branch 'origin' into piz/skipmodel
zpcore Jan 22, 2024
0e87dae
nit update
zpcore Jan 22, 2024
a318157
syntax update
zpcore Jan 23, 2024
1bbef43
syntax update
zpcore Jan 23, 2024
d2216eb
Merge remote-tracking branch 'origin' into piz/skipmodel
zpcore Jan 23, 2024
22185da
Merge remote-tracking branch 'origin' into piz/skipmodel
zpcore Jan 23, 2024
9c95e4d
looses the strict_deny_list for uninstalled models
zpcore Jan 23, 2024
3f3d52b
nit update
zpcore Jan 23, 2024
23550ad
skip pjrt for eval
zpcore Jan 23, 2024
4f7a3a4
move some strict rules to general rule
zpcore Jan 23, 2024
31bf854
Merge remote-tracking branch 'origin' into piz/skipmodel
zpcore Jan 23, 2024
5ad5f2e
add pytorch_stargan to strict deny list for TPU
zpcore Jan 23, 2024
ab0ad8a
add soft_actor_critic to strict deny list for TPU
zpcore Jan 24, 2024
363cd4d
add speech_transformer to strict deny list for TPU
zpcore Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions benchmarks/torchbench_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@

# Skip the experiment of a model if any of the experiment configs in the list is fully matched
DENY_LIST = {
"cm3leon_generate": [{}], # no install.py
zpcore marked this conversation as resolved.
Show resolved Hide resolved
"doctr_det_predictor": [{
"test": "train"
},], # not implemented
Expand All @@ -71,6 +72,7 @@
},], # not implemented
# https://github.com/pytorch/torchdynamo/issues/145
"fambench_xlmr": [{}],
"hf_T5_generate": [{}], # no install.py
"llama": [{
"test": "train"
},], # not implemented
Expand All @@ -84,6 +86,9 @@
"accelerator": "tpu"
},
], # not implemented
"opacus_cifar10": [{
"accelerator": "tpu"
},], # stackdump issus in TPU
# self.load_benchmark() exits the main process. See issue #6207.
"pytorch_CycleGAN_and_pix2pix": [{}],
"pyhpc_equation_of_state": [{
Expand Down
Loading