Temporarily pin pydantic test dependency

[albertvillanova]

Temporarily pin pydantic until a permanent solution is found.

Fix #5394.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[github-actions]

Show benchmarks

PyArrow==6.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.012220 / 0.011353 (0.000867)	0.005943 / 0.011008 (-0.005065)	0.128223 / 0.038508 (0.089715)	0.037352 / 0.023109 (0.014242)	0.397143 / 0.275898 (0.121245)	0.483935 / 0.323480 (0.160455)	0.010279 / 0.007986 (0.002293)	0.004842 / 0.004328 (0.000513)	0.101403 / 0.004250 (0.097153)	0.042935 / 0.037052 (0.005883)	0.421642 / 0.258489 (0.163153)	0.456328 / 0.293841 (0.162487)	0.065639 / 0.128546 (-0.062907)	0.019820 / 0.075646 (-0.055826)	0.426090 / 0.419271 (0.006818)	0.069583 / 0.043533 (0.026051)	0.402662 / 0.255139 (0.147523)	0.428826 / 0.283200 (0.145626)	0.116760 / 0.141683 (-0.024923)	1.806216 / 1.452155 (0.354061)	1.852629 / 1.492716 (0.359913)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.226555 / 0.018006 (0.208548)	0.584693 / 0.000490 (0.584203)	0.008612 / 0.000200 (0.008412)	0.000205 / 0.000054 (0.000150)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.028393 / 0.037411 (-0.009018)	0.123355 / 0.014526 (0.108829)	0.134423 / 0.176557 (-0.042133)	0.188536 / 0.737135 (-0.548600)	0.141595 / 0.296338 (-0.154743)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.589359 / 0.215209 (0.374150)	5.974655 / 2.077655 (3.897001)	2.465580 / 1.504120 (0.961460)	2.007618 / 1.541195 (0.466424)	2.078788 / 1.468490 (0.610298)	1.216646 / 4.584777 (-3.368131)	5.217516 / 3.745712 (1.471804)	3.107188 / 5.269862 (-2.162674)	2.251641 / 4.565676 (-2.314036)	0.138640 / 0.424275 (-0.285635)	0.015046 / 0.007607 (0.007439)	0.780092 / 0.226044 (0.554048)	7.749564 / 2.268929 (5.480635)	3.080708 / 55.444624 (-52.363917)	2.393897 / 6.876477 (-4.482579)	2.387738 / 2.142072 (0.245665)	1.458844 / 4.805227 (-3.346384)	0.252476 / 6.500664 (-6.248188)	0.076594 / 0.075469 (0.001125)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.540868 / 1.841788 (-0.300919)	17.295684 / 8.074308 (9.221376)	19.669300 / 10.191392 (9.477908)	0.250315 / 0.680424 (-0.430109)	0.045068 / 0.534201 (-0.489133)	0.538840 / 0.579283 (-0.040443)	0.584443 / 0.434364 (0.150079)	0.614476 / 0.540337 (0.074138)	0.729928 / 1.386936 (-0.657008)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.009218 / 0.011353 (-0.002135)	0.006261 / 0.011008 (-0.004747)	0.125541 / 0.038508 (0.087033)	0.034405 / 0.023109 (0.011296)	0.468381 / 0.275898 (0.192483)	0.503336 / 0.323480 (0.179856)	0.006839 / 0.007986 (-0.001146)	0.004724 / 0.004328 (0.000396)	0.097875 / 0.004250 (0.093625)	0.051278 / 0.037052 (0.014225)	0.473323 / 0.258489 (0.214834)	0.537392 / 0.293841 (0.243551)	0.055588 / 0.128546 (-0.072958)	0.021041 / 0.075646 (-0.054605)	0.416952 / 0.419271 (-0.002320)	0.070128 / 0.043533 (0.026595)	0.465224 / 0.255139 (0.210085)	0.504678 / 0.283200 (0.221478)	0.112504 / 0.141683 (-0.029179)	1.865865 / 1.452155 (0.413710)	1.988296 / 1.492716 (0.495580)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.314170 / 0.018006 (0.296164)	0.526726 / 0.000490 (0.526236)	0.018691 / 0.000200 (0.018491)	0.000128 / 0.000054 (0.000073)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.033772 / 0.037411 (-0.003639)	0.124796 / 0.014526 (0.110270)	0.134700 / 0.176557 (-0.041856)	0.190595 / 0.737135 (-0.546541)	0.143205 / 0.296338 (-0.153133)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.656708 / 0.215209 (0.441499)	6.470503 / 2.077655 (4.392848)	2.866430 / 1.504120 (1.362310)	2.506846 / 1.541195 (0.965651)	2.548669 / 1.468490 (1.080179)	1.226695 / 4.584777 (-3.358082)	5.117866 / 3.745712 (1.372153)	3.032822 / 5.269862 (-2.237040)	1.999152 / 4.565676 (-2.566524)	0.142974 / 0.424275 (-0.281301)	0.015011 / 0.007607 (0.007404)	0.799729 / 0.226044 (0.573684)	8.286313 / 2.268929 (6.017385)	3.636482 / 55.444624 (-51.808142)	2.888038 / 6.876477 (-3.988439)	2.924982 / 2.142072 (0.782910)	1.471996 / 4.805227 (-3.333231)	0.257119 / 6.500664 (-6.243545)	0.077294 / 0.075469 (0.001825)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.608290 / 1.841788 (-0.233497)	17.599119 / 8.074308 (9.524811)	18.917086 / 10.191392 (8.725694)	0.236237 / 0.680424 (-0.444187)	0.026061 / 0.534201 (-0.508140)	0.527359 / 0.579283 (-0.051925)	0.589176 / 0.434364 (0.154812)	0.602310 / 0.540337 (0.061973)	0.726756 / 1.386936 (-0.660180)

[albertvillanova]

Issue reported to pydantic:

• Pydantic 1.10.3 incompatible with typing-extensions 4.1.1 pydantic/pydantic#4885

Fixing PR at pydantic:

• Typing extensions min version pydantic/pydantic#4886