Remove instructions for ffmpeg system package installation on Colab

[polinaeterna]

Colab now has Ubuntu 20.04 which already has ffmpeg of required (>4) version.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[lhoestq]

thanks !

[github-actions]

Show benchmarks

PyArrow==6.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.014525 / 0.011353 (0.003172)	0.006871 / 0.011008 (-0.004137)	0.135577 / 0.038508 (0.097069)	0.039620 / 0.023109 (0.016511)	0.499829 / 0.275898 (0.223931)	0.571000 / 0.323480 (0.247520)	0.009726 / 0.007986 (0.001740)	0.005654 / 0.004328 (0.001325)	0.104732 / 0.004250 (0.100482)	0.046849 / 0.037052 (0.009796)	0.486667 / 0.258489 (0.228178)	0.543611 / 0.293841 (0.249770)	0.056414 / 0.128546 (-0.072133)	0.019974 / 0.075646 (-0.055672)	0.484878 / 0.419271 (0.065606)	0.059244 / 0.043533 (0.015711)	0.490046 / 0.255139 (0.234907)	0.517427 / 0.283200 (0.234227)	0.114692 / 0.141683 (-0.026991)	1.935935 / 1.452155 (0.483780)	1.990253 / 1.492716 (0.497537)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.271008 / 0.018006 (0.253002)	0.610964 / 0.000490 (0.610474)	0.013423 / 0.000200 (0.013223)	0.000523 / 0.000054 (0.000468)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.031940 / 0.037411 (-0.005472)	0.130755 / 0.014526 (0.116229)	0.146616 / 0.176557 (-0.029941)	0.239386 / 0.737135 (-0.497749)	0.146612 / 0.296338 (-0.149726)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.675383 / 0.215209 (0.460174)	6.656828 / 2.077655 (4.579174)	2.741231 / 1.504120 (1.237111)	2.232921 / 1.541195 (0.691726)	2.172116 / 1.468490 (0.703626)	1.221623 / 4.584777 (-3.363154)	5.683653 / 3.745712 (1.937941)	5.344137 / 5.269862 (0.074275)	2.969670 / 4.565676 (-1.596006)	0.142107 / 0.424275 (-0.282168)	0.015808 / 0.007607 (0.008201)	0.767366 / 0.226044 (0.541321)	8.059605 / 2.268929 (5.790676)	3.333535 / 55.444624 (-52.111089)	2.669619 / 6.876477 (-4.206857)	2.652989 / 2.142072 (0.510917)	1.526397 / 4.805227 (-3.278830)	0.265609 / 6.500664 (-6.235055)	0.082759 / 0.075469 (0.007290)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.631086 / 1.841788 (-0.210701)	18.701351 / 8.074308 (10.627043)	22.843802 / 10.191392 (12.652410)	0.240134 / 0.680424 (-0.440290)	0.046683 / 0.534201 (-0.487518)	0.576488 / 0.579283 (-0.002795)	0.650123 / 0.434364 (0.215759)	0.661190 / 0.540337 (0.120853)	0.759563 / 1.386936 (-0.627373)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.009883 / 0.011353 (-0.001470)	0.006692 / 0.011008 (-0.004316)	0.098550 / 0.038508 (0.060042)	0.035188 / 0.023109 (0.012078)	0.463535 / 0.275898 (0.187637)	0.472762 / 0.323480 (0.149282)	0.007199 / 0.007986 (-0.000787)	0.007961 / 0.004328 (0.003632)	0.093140 / 0.004250 (0.088890)	0.051752 / 0.037052 (0.014700)	0.453412 / 0.258489 (0.194922)	0.502741 / 0.293841 (0.208900)	0.056006 / 0.128546 (-0.072540)	0.020164 / 0.075646 (-0.055482)	0.116828 / 0.419271 (-0.302444)	0.067205 / 0.043533 (0.023672)	0.442715 / 0.255139 (0.187576)	0.472525 / 0.283200 (0.189326)	0.122767 / 0.141683 (-0.018915)	1.881366 / 1.452155 (0.429212)	1.978786 / 1.492716 (0.486069)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.284180 / 0.018006 (0.266174)	0.601556 / 0.000490 (0.601067)	0.008455 / 0.000200 (0.008255)	0.000112 / 0.000054 (0.000057)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.033515 / 0.037411 (-0.003896)	0.136407 / 0.014526 (0.121881)	0.143341 / 0.176557 (-0.033215)	0.225394 / 0.737135 (-0.511741)	0.153343 / 0.296338 (-0.142995)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.688202 / 0.215209 (0.472993)	6.576502 / 2.077655 (4.498847)	2.839175 / 1.504120 (1.335055)	2.481152 / 1.541195 (0.939957)	2.617227 / 1.468490 (1.148736)	1.314854 / 4.584777 (-3.269922)	5.805950 / 3.745712 (2.060238)	3.188930 / 5.269862 (-2.080932)	2.141719 / 4.565676 (-2.423957)	0.145069 / 0.424275 (-0.279206)	0.014567 / 0.007607 (0.006960)	0.780000 / 0.226044 (0.553955)	7.898016 / 2.268929 (5.629088)	3.549060 / 55.444624 (-51.895564)	2.856569 / 6.876477 (-4.019907)	3.117719 / 2.142072 (0.975647)	1.512560 / 4.805227 (-3.292668)	0.262689 / 6.500664 (-6.237975)	0.085979 / 0.075469 (0.010509)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.623550 / 1.841788 (-0.218238)	19.597063 / 8.074308 (11.522755)	21.293369 / 10.191392 (11.101977)	0.263780 / 0.680424 (-0.416643)	0.027289 / 0.534201 (-0.506912)	0.560361 / 0.579283 (-0.018922)	0.646288 / 0.434364 (0.211924)	0.712699 / 0.540337 (0.172361)	0.818332 / 1.386936 (-0.568604)