Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to limit export of (large) arrays (to speed up PHPUnit) #59

Merged
merged 8 commits into from
Jun 18, 2024

Conversation

staabm
Copy link
Contributor

@staabm staabm commented Jun 17, 2024

goal is to speedup running tests in phpunit with dataproviders with loooots of elements.


repro from sebastianbergmann/phpunit#5774 (comment)

PHPUnit 10.5.21 on PHP 8.3.8 (on m1 pro)

vendor/bin/phpunit tests/TokenizerTest.php 
...
Time: 00:31.553, Memory: 374.09 MB

with this PR

Time: 00:01.964, Memory: 293.88 MB

@staabm
Copy link
Contributor Author

staabm commented Jun 17, 2024

//cc @mvorisek

src/Exporter.php Outdated
foreach ($array as $key => $value) {
if ($count > self::MAX_SHORTENED_ITEMS && $i > self::MAX_SHORTENED_ITEMS) {
Copy link
Contributor Author

@staabm staabm Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we are in the shortenedRecursiveExport method, I think its acceptable to shorten these arrays more agressively.

before this PR running phpunit on a dataprovider with lots of elements resulted in a big wall-of-text description like

PHPUnit 11.2.2 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.8
Configuration: /Users/staabm/workspace/exporter/phpunit.xml

................................................................. 65 / 74 ( 87%)
.....F...                                                         74 / 74 (100%)

Time: 00:00.042, Memory: 20.00 MB

There was 1 failure:

1) SebastianBergmann\Exporter\ExporterTest::testShortenedRecursiveExport with data set "big array" (['cast('foo0' as blob)', 'cast('foo1' as blob)', 'cast('foo2' as blob)', 'cast('foo3' as blob)', 'cast('foo4' as blob)', 'cast('foo5' as blob)', 'cast('foo6' as blob)', 'cast('foo7' as blob)', 'cast('foo8' as blob)', 'cast('foo9' as blob)', 'cast('foo10' as blob)', 'cast('foo11' as blob)', 'cast('foo12' as blob)', 'cast('foo13' as blob)', 'cast('foo14' as blob)', 'cast('foo15' as blob)', 'cast('foo16' as blob)', 'cast('foo17' as blob)', 'cast('foo18' as blob)', 'cast('foo19' as blob)', 'cast('foo20' as blob)', 'cast('foo21' as blob)', 'cast('foo22' as blob)', 'cast('foo23' as blob)', 'cast('foo24' as blob)', 'cast('foo25' as blob)', 'cast('foo26' as blob)', 'cast('foo27' as blob)', 'cast('foo28' as blob)', 'cast('foo29' as blob)', 'cast('foo30' as blob)', 'cast('foo31' as blob)', 'cast('foo32' as blob)', 'cast('foo33' as blob)', 'cast('foo34' as blob)', 'cast('foo35' as blob)', 'cast('foo36' as blob)', 'cast('foo37' as blob)', 'cast('foo38' as blob)', 'cast('foo39' as blob)', 'cast('foo40' as blob)', 'cast('foo41' as blob)', 'cast('foo42' as blob)', 'cast('foo43' as blob)', 'cast('foo44' as blob)', 'cast('foo45' as blob)', 'cast('foo46' as blob)', 'cast('foo47' as blob)', 'cast('foo48' as blob)', 'cast('foo49' as blob)', 'cast('foo50' as blob)', 'cast('foo51' as blob)', 'cast('foo52' as blob)', 'cast('foo53' as blob)', 'cast('foo54' as blob)', 'cast('foo55' as blob)', 'cast('foo56' as blob)', 'cast('foo57' as blob)', 'cast('foo58' as blob)', 'cast('foo59' as blob)', 'cast('foo60' as blob)', 'cast('foo61' as blob)', 'cast('foo62' as blob)', 'cast('foo63' as blob)', 'cast('foo64' as blob)', 'cast('foo65' as blob)', 'cast('foo66' as blob)', 'cast('foo67' as blob)', 'cast('foo68' as blob)', 'cast('foo69' as blob)', 'cast('foo70' as blob)', 'cast('foo71' as blob)', 'cast('foo72' as blob)', 'cast('foo73' as blob)', 'cast('foo74' as blob)', 'cast('foo75' as blob)', 'cast('foo76' as blob)', 'cast('foo77' as blob)', 'cast('foo78' as blob)', 'cast('foo79' as blob)', 'cast('foo80' as blob)', 'cast('foo81' as blob)', 'cast('foo82' as blob)', 'cast('foo83' as blob)', 'cast('foo84' as blob)', 'cast('foo85' as blob)', 'cast('foo86' as blob)', 'cast('foo87' as blob)', 'cast('foo88' as blob)', 'cast('foo89' as blob)', 'cast('foo90' as blob)', 'cast('foo91' as blob)', 'cast('foo92' as blob)', 'cast('foo93' as blob)', 'cast('foo94' as blob)', 'cast('foo95' as blob)', 'cast('foo96' as blob)', 'cast('foo97' as blob)', 'cast('foo98' as blob)', 'cast('foo99' as blob)', 'cast('foo100' as blob)', 'cast('foo101' as blob)', 'cast('foo102' as blob)', 'cast('foo103' as blob)', 'cast('foo104' as blob)', 'cast('foo105' as blob)', 'cast('foo106' as blob)', 'cast('foo107' as blob)', 'cast('foo108' as blob)', 'cast('foo109' as blob)', 'cast('foo110' as blob)', 'cast('foo111' as blob)', 'cast('foo112' as blob)', 'cast('foo113' as blob)', 'cast('foo114' as blob)', 'cast('foo115' as blob)', 'cast('foo116' as blob)', 'cast('foo117' as blob)', 'cast('foo118' as blob)', 'cast('foo119' as blob)', 'cast('foo120' as blob)', 'cast('foo121' as blob)', 'cast('foo122' as blob)', 'cast('foo123' as blob)', 'cast('foo124' as blob)', 'cast('foo125' as blob)', 'cast('foo126' as blob)', 'cast('foo127' as blob)', 'cast('foo128' as blob)', 'cast('foo129' as blob)', 'cast('foo130' as blob)', 'cast('foo131' as blob)', 'cast('foo132' as blob)', 'cast('foo133' as blob)', 'cast('foo134' as blob)', 'cast('foo135' as blob)', 'cast('foo136' as blob)', 'cast('foo137' as blob)', 'cast('foo138' as blob)', 'cast('foo139' as blob)', 'cast('foo140' as blob)', 'cast('foo141' as blob)', 'cast('foo142' as blob)', 'cast('foo143' as blob)', 'cast('foo144' as blob)', 'cast('foo145' as blob)', 'cast('foo146' as blob)', 'cast('foo147' as blob)', 'cast('foo148' as blob)', 'cast('foo149' as blob)', 'cast('foo150' as blob)', 'cast('foo151' as blob)', 'cast('foo152' as blob)', 'cast('foo153' as blob)', 'cast('foo154' as blob)', 'cast('foo155' as blob)', 'cast('foo156' as blob)', 'cast('foo157' as blob)', 'cast('foo158' as blob)', 'cast('foo159' as blob)', 'cast('foo160' as blob)', 'cast('foo161' as blob)', 'cast('foo162' as blob)', 'cast('foo163' as blob)', 'cast('foo164' as blob)', 'cast('foo165' as blob)', 'cast('foo166' as blob)', 'cast('foo167' as blob)', 'cast('foo168' as blob)', 'cast('foo169' as blob)', 'cast('foo170' as blob)', 'cast('foo171' as blob)', 'cast('foo172' as blob)', 'cast('foo173' as blob)', 'cast('foo174' as blob)', 'cast('foo175' as blob)', 'cast('foo176' as blob)', 'cast('foo177' as blob)', 'cast('foo178' as blob)', 'cast('foo179' as blob)', 'cast('foo180' as blob)', 'cast('foo181' as blob)', 'cast('foo182' as blob)', 'cast('foo183' as blob)', 'cast('foo184' as blob)', 'cast('foo185' as blob)', 'cast('foo186' as blob)', 'cast('foo187' as blob)', 'cast('foo188' as blob)', 'cast('foo189' as blob)', 'cast('foo190' as blob)', 'cast('foo191' as blob)', 'cast('foo192' as blob)', 'cast('foo193' as blob)', 'cast('foo194' as blob)', 'cast('foo195' as blob)', 'cast('foo196' as blob)', 'cast('foo197' as blob)', 'cast('foo198' as blob)', 'cast('foo199' as blob)', 'cast('foo200' as blob)', 'cast('foo201' as blob)', 'cast('foo202' as blob)', 'cast('foo203' as blob)', 'cast('foo204' as blob)', 'cast('foo205' as blob)', 'cast('foo206' as blob)', 'cast('foo207' as blob)', 'cast('foo208' as blob)', 'cast('foo209' as blob)', 'cast('foo210' as blob)', 'cast('foo211' as blob)', 'cast('foo212' as blob)', 'cast('foo213' as blob)', 'cast('foo214' as blob)', 'cast('foo215' as blob)', 'cast('foo216' as blob)', 'cast('foo217' as blob)', 'cast('foo218' as blob)', 'cast('foo219' as blob)', 'cast('foo220' as blob)', 'cast('foo221' as blob)', 'cast('foo222' as blob)', 'cast('foo223' as blob)', 'cast('foo224' as blob)', 'cast('foo225' as blob)', 'cast('foo226' as blob)', 'cast('foo227' as blob)', 'cast('foo228' as blob)', 'cast('foo229' as blob)', 'cast('foo230' as blob)', 'cast('foo231' as blob)', 'cast('foo232' as blob)', 'cast('foo233' as blob)', 'cast('foo234' as blob)', 'cast('foo235' as blob)', 'cast('foo236' as blob)', 'cast('foo237' as blob)', 'cast('foo238' as blob)', 'cast('foo239' as blob)', 'cast('foo240' as blob)', 'cast('foo241' as blob)', 'cast('foo242' as blob)', 'cast('foo243' as blob)', 'cast('foo244' as blob)', 'cast('foo245' as blob)', 'cast('foo246' as blob)', 'cast('foo247' as blob)', 'cast('foo248' as blob)', 'cast('foo249' as blob)', 'cast('foo250' as blob)', 'cast('foo251' as blob)', 'cast('foo252' as blob)', 'cast('foo253' as blob)', 'cast('foo254' as blob)', 'cast('foo255' as blob)', 'cast('foo256' as blob)', 'cast('foo257' as blob)', 'cast('foo258' as blob)', 'cast('foo259' as blob)', 'cast('foo260' as blob)', 'cast('foo261' as blob)', 'cast('foo262' as blob)', 'cast('foo263' as blob)', 'cast('foo264' as blob)', 'cast('foo265' as blob)', 'cast('foo266' as blob)', 'cast('foo267' as blob)', 'cast('foo268' as blob)', 'cast('foo269' as blob)', 'cast('foo270' as blob)', 'cast('foo271' as blob)', 'cast('foo272' as blob)', 'cast('foo273' as blob)', 'cast('foo274' as blob)', 'cast('foo275' as blob)', 'cast('foo276' as blob)', 'cast('foo277' as blob)', 'cast('foo278' as blob)', 'cast('foo279' as blob)', 'cast('foo280' as blob)', 'cast('foo281' as blob)', 'cast('foo282' as blob)', 'cast('foo283' as blob)', 'cast('foo284' as blob)', 'cast('foo285' as blob)', 'cast('foo286' as blob)', 'cast('foo287' as blob)', 'cast('foo

so I think this is also a DX improvement

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely agree and thank you for this PR!

src/Exporter.php Outdated Show resolved Hide resolved
Copy link

codecov bot commented Jun 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (b9c7cad) to head (deb644f).
Report is 8 commits behind head on main.

Current head deb644f differs from pull request most recent head 4c25986

Please upload reports for the commit 4c25986 to get more accurate results.

Additional details and impacted files
@@             Coverage Diff             @@
##                main       #59   +/-   ##
===========================================
  Coverage     100.00%   100.00%           
+ Complexity        60        54    -6     
===========================================
  Files              2         1    -1     
  Lines            177       169    -8     
===========================================
- Hits             177       169    -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/Exporter.php Outdated Show resolved Hide resolved
@sebastianbergmann sebastianbergmann merged commit 085f691 into sebastianbergmann:main Jun 18, 2024
5 checks passed
@sebastianbergmann sebastianbergmann changed the title Shorten big arrays to speedup phpunit Add option to limit export of (large) arrays (to speed up PHPUnit) Jun 18, 2024
@staabm staabm deleted the short branch June 18, 2024 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants