Skip to content

Commit

Permalink
Fix audio test CI by checking out PR HEAD commit instead of merge commit
Browse files Browse the repository at this point in the history
  • Loading branch information
@albertvillanova
albertvillanova committed Nov 16, 2021
1 parent 2fc997a commit 416d1bf
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions .github/workflows/test-audio.yml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
name: Test audio

on:
push:
branches:
- master
pull_request:
branches:
- master
Expand All @@ -18,7 +15,7 @@ jobs:
sudo apt-get install libsndfile1 sox
- uses: actions/checkout@v2
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha }}
- name: Set up Python
uses: actions/setup-python@v2
with:
Expand Down

2 comments on commit 416d1bf

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Show benchmarks

PyArrow==3.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.072379 / 0.011353 (0.061026) 0.004192 / 0.011008 (-0.006816) 0.031687 / 0.038508 (-0.006821) 0.037088 / 0.023109 (0.013979) 0.313530 / 0.275898 (0.037632) 0.343952 / 0.323480 (0.020472) 0.087718 / 0.007986 (0.079732) 0.005086 / 0.004328 (0.000758) 0.009183 / 0.004250 (0.004932) 0.046627 / 0.037052 (0.009575) 0.314060 / 0.258489 (0.055571) 0.339650 / 0.293841 (0.045809) 0.087210 / 0.128546 (-0.041336) 0.009159 / 0.075646 (-0.066487) 0.254994 / 0.419271 (-0.164277) 0.046643 / 0.043533 (0.003110) 0.315755 / 0.255139 (0.060616) 0.332378 / 0.283200 (0.049179) 0.088270 / 0.141683 (-0.053413) 1.675700 / 1.452155 (0.223546) 1.699399 / 1.492716 (0.206682)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.376351 / 0.018006 (0.358344) 0.557934 / 0.000490 (0.557445) 0.027136 / 0.000200 (0.026936) 0.000516 / 0.000054 (0.000462)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.037649 / 0.037411 (0.000238) 0.023186 / 0.014526 (0.008661) 0.030811 / 0.176557 (-0.145746) 0.198211 / 0.737135 (-0.538924) 0.031830 / 0.296338 (-0.264508)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.431460 / 0.215209 (0.216251) 4.314090 / 2.077655 (2.236435) 1.874387 / 1.504120 (0.370267) 1.637115 / 1.541195 (0.095920) 1.765311 / 1.468490 (0.296821) 0.422690 / 4.584777 (-4.162087) 4.761815 / 3.745712 (1.016103) 3.864789 / 5.269862 (-1.405073) 0.899256 / 4.565676 (-3.666421) 0.051494 / 0.424275 (-0.372781) 0.011347 / 0.007607 (0.003740) 0.541876 / 0.226044 (0.315831) 5.399492 / 2.268929 (3.130563) 2.338959 / 55.444624 (-53.105666) 1.943735 / 6.876477 (-4.932741) 2.261382 / 2.142072 (0.119310) 0.542826 / 4.805227 (-4.262401) 0.116918 / 6.500664 (-6.383746) 0.058204 / 0.075469 (-0.017265)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 1.562897 / 1.841788 (-0.278891) 12.950901 / 8.074308 (4.876592) 27.088913 / 10.191392 (16.897521) 0.809662 / 0.680424 (0.129238) 0.536048 / 0.534201 (0.001847) 0.378609 / 0.579283 (-0.200674) 0.516146 / 0.434364 (0.081782) 0.258390 / 0.540337 (-0.281947) 0.271182 / 1.386936 (-1.115754)
PyArrow==latest
Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.069361 / 0.011353 (0.058008) 0.004069 / 0.011008 (-0.006939) 0.029662 / 0.038508 (-0.008846) 0.034445 / 0.023109 (0.011335) 0.302353 / 0.275898 (0.026455) 0.365114 / 0.323480 (0.041634) 0.092206 / 0.007986 (0.084220) 0.004391 / 0.004328 (0.000062) 0.007331 / 0.004250 (0.003081) 0.039325 / 0.037052 (0.002273) 0.302468 / 0.258489 (0.043979) 0.372388 / 0.293841 (0.078547) 0.084360 / 0.128546 (-0.044186) 0.008867 / 0.075646 (-0.066779) 0.251263 / 0.419271 (-0.168008) 0.045198 / 0.043533 (0.001665) 0.302829 / 0.255139 (0.047690) 0.355583 / 0.283200 (0.072384) 0.085064 / 0.141683 (-0.056619) 1.697565 / 1.452155 (0.245411) 1.764920 / 1.492716 (0.272204)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.368191 / 0.018006 (0.350185) 0.531494 / 0.000490 (0.531005) 0.000987 / 0.000200 (0.000787) 0.000086 / 0.000054 (0.000032)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.034211 / 0.037411 (-0.003200) 0.021761 / 0.014526 (0.007235) 0.028370 / 0.176557 (-0.148186) 0.198375 / 0.737135 (-0.538760) 0.030107 / 0.296338 (-0.266231)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.427400 / 0.215209 (0.212191) 4.294384 / 2.077655 (2.216729) 1.836768 / 1.504120 (0.332648) 1.612459 / 1.541195 (0.071264) 1.710224 / 1.468490 (0.241734) 0.423097 / 4.584777 (-4.161680) 4.668943 / 3.745712 (0.923231) 2.105634 / 5.269862 (-3.164227) 0.863887 / 4.565676 (-3.701790) 0.050871 / 0.424275 (-0.373404) 0.010905 / 0.007607 (0.003298) 0.536863 / 0.226044 (0.310818) 5.347948 / 2.268929 (3.079019) 2.287955 / 55.444624 (-53.156669) 1.901718 / 6.876477 (-4.974759) 2.034224 / 2.142072 (-0.107848) 0.540170 / 4.805227 (-4.265057) 0.115177 / 6.500664 (-6.385487) 0.056074 / 0.075469 (-0.019395)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 1.553229 / 1.841788 (-0.288559) 12.182251 / 8.074308 (4.107943) 27.276210 / 10.191392 (17.084818) 0.713769 / 0.680424 (0.033345) 0.513940 / 0.534201 (-0.020261) 0.370730 / 0.579283 (-0.208553) 0.507022 / 0.434364 (0.072658) 0.257058 / 0.540337 (-0.283280) 0.266216 / 1.386936 (-1.120720)

CML watermark

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Show benchmarks

PyArrow==3.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.087072 / 0.011353 (0.075719) 0.004467 / 0.011008 (-0.006541) 0.038587 / 0.038508 (0.000079) 0.042905 / 0.023109 (0.019796) 0.375534 / 0.275898 (0.099636) 0.407517 / 0.323480 (0.084037) 0.097568 / 0.007986 (0.089582) 0.004678 / 0.004328 (0.000349) 0.010975 / 0.004250 (0.006724) 0.047596 / 0.037052 (0.010543) 0.370738 / 0.258489 (0.112249) 0.399901 / 0.293841 (0.106060) 0.104871 / 0.128546 (-0.023675) 0.010640 / 0.075646 (-0.065007) 0.314817 / 0.419271 (-0.104455) 0.056148 / 0.043533 (0.012615) 0.380458 / 0.255139 (0.125319) 0.401621 / 0.283200 (0.118421) 0.096799 / 0.141683 (-0.044884) 2.111622 / 1.452155 (0.659467) 2.160863 / 1.492716 (0.668147)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.359495 / 0.018006 (0.341489) 0.461943 / 0.000490 (0.461453) 0.034769 / 0.000200 (0.034570) 0.000430 / 0.000054 (0.000375)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.044791 / 0.037411 (0.007380) 0.026112 / 0.014526 (0.011586) 0.033271 / 0.176557 (-0.143285) 0.245153 / 0.737135 (-0.491982) 0.032273 / 0.296338 (-0.264065)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.518894 / 0.215209 (0.303685) 5.188256 / 2.077655 (3.110602) 2.218291 / 1.504120 (0.714172) 1.947354 / 1.541195 (0.406159) 2.001914 / 1.468490 (0.533424) 0.507723 / 4.584777 (-4.077054) 6.109277 / 3.745712 (2.363565) 2.726904 / 5.269862 (-2.542958) 1.046892 / 4.565676 (-3.518784) 0.061656 / 0.424275 (-0.362620) 0.013405 / 0.007607 (0.005798) 0.650289 / 0.226044 (0.424245) 6.445199 / 2.268929 (4.176271) 2.810857 / 55.444624 (-52.633767) 2.329761 / 6.876477 (-4.546716) 2.517509 / 2.142072 (0.375436) 0.658540 / 4.805227 (-4.146688) 0.141563 / 6.500664 (-6.359101) 0.069423 / 0.075469 (-0.006046)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 1.926439 / 1.841788 (0.084652) 14.430210 / 8.074308 (6.355902) 32.146643 / 10.191392 (21.955251) 0.920706 / 0.680424 (0.240282) 0.632350 / 0.534201 (0.098149) 0.454307 / 0.579283 (-0.124976) 0.650876 / 0.434364 (0.216512) 0.324699 / 0.540337 (-0.215638) 0.349357 / 1.386936 (-1.037579)
PyArrow==latest
Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.084652 / 0.011353 (0.073299) 0.004333 / 0.011008 (-0.006676) 0.036291 / 0.038508 (-0.002217) 0.039944 / 0.023109 (0.016834) 0.364705 / 0.275898 (0.088806) 0.396665 / 0.323480 (0.073185) 0.098914 / 0.007986 (0.090928) 0.005472 / 0.004328 (0.001144) 0.008618 / 0.004250 (0.004368) 0.042779 / 0.037052 (0.005726) 0.359776 / 0.258489 (0.101287) 0.410898 / 0.293841 (0.117057) 0.104240 / 0.128546 (-0.024306) 0.010552 / 0.075646 (-0.065094) 0.312616 / 0.419271 (-0.106656) 0.054487 / 0.043533 (0.010954) 0.368227 / 0.255139 (0.113088) 0.399086 / 0.283200 (0.115887) 0.091818 / 0.141683 (-0.049865) 2.127840 / 1.452155 (0.675685) 2.171869 / 1.492716 (0.679152)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.301122 / 0.018006 (0.283116) 0.453623 / 0.000490 (0.453134) 0.005693 / 0.000200 (0.005493) 0.000286 / 0.000054 (0.000232)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.042041 / 0.037411 (0.004629) 0.025133 / 0.014526 (0.010608) 0.029288 / 0.176557 (-0.147269) 0.237830 / 0.737135 (-0.499305) 0.031009 / 0.296338 (-0.265329)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.540203 / 0.215209 (0.324994) 5.322974 / 2.077655 (3.245319) 2.219983 / 1.504120 (0.715863) 1.963648 / 1.541195 (0.422453) 2.016031 / 1.468490 (0.547541) 0.517371 / 4.584777 (-4.067406) 6.138089 / 3.745712 (2.392377) 5.283542 / 5.269862 (0.013681) 1.066231 / 4.565676 (-3.499446) 0.061482 / 0.424275 (-0.362793) 0.013269 / 0.007607 (0.005662) 0.650369 / 0.226044 (0.424325) 6.485560 / 2.268929 (4.216632) 2.779543 / 55.444624 (-52.665082) 2.319327 / 6.876477 (-4.557150) 2.410655 / 2.142072 (0.268583) 0.660223 / 4.805227 (-4.145005) 0.140873 / 6.500664 (-6.359791) 0.069344 / 0.075469 (-0.006125)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 1.929829 / 1.841788 (0.088042) 14.317415 / 8.074308 (6.243107) 33.559469 / 10.191392 (23.368077) 0.952338 / 0.680424 (0.271914) 0.648609 / 0.534201 (0.114408) 0.462686 / 0.579283 (-0.116597) 0.643908 / 0.434364 (0.209544) 0.312573 / 0.540337 (-0.227764) 0.324697 / 1.386936 (-1.062239)

CML watermark

Please sign in to comment.