Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Env-specific tests fail depending on random execution order #224

Closed
Ocramius opened this issue Dec 18, 2022 · 1 comment · Fixed by #225
Closed

Env-specific tests fail depending on random execution order #224

Ocramius opened this issue Dec 18, 2022 · 1 comment · Fixed by #225
Assignees
Labels
Bug Something isn't working
Milestone

Comments

@Ocramius
Copy link
Member

Bug Report

Q A
Version(s) 1.20.0

#220 introduced new environment variables, but the tests around it fail due to missing isolation of global state during test execution.

See for example #223 failure @ https://github.com/laminas/automatic-releases/actions/runs/3716382625/jobs/6302643279#step:3:512

Error: ] Project tests must be in a passing state before running Infection.     
         Infection runs the test suite in a RANDOM order. Make sure your tests  
         do not have hidden dependencies.                                       
                                                                                
         You can add these attributes to `phpunit.xml` to check it: <phpunit    
         executionOrder="random" resolveDependencies="true" ...                 
                                                                                
         If you don't want to let Infection run tests in a random order, set the
         `executionOrder` to some value, for example <phpunit                   
         executionOrder="default"                                               
                                                                                
         Check the executed command to identify the problem:                    
         '/usr/bin/phpdbg8.1' '-qrr' '/github/workspace/vendor/bin/phpunit'     
         '--configuration'                                                      
         '/tmp/infection/phpunitConfiguration.initial.infection.xml'            
         '--coverage-xml=/tmp/infection/coverage-xml'                           
         '--log-junit=/tmp/infection/junit.xml'                                 
         PHPUnit reported an exit code of 1.                                    
         Refer to the PHPUnit's output below:                                   
         STDOUT:                                                                
         PHPUnit 9.5.27 by Sebastian Bergmann and contributors.                 
                                                                                
         Runtime:       PHPDBG 8.1.13                                           
         Configuration:                                                         
         /tmp/infection/phpunitConfiguration.initial.infection.xml              
         Random Seed:   1671223208                                              
                                                                                
         ...............................................................  63 /  
         229 ( 27%)                                                             
         ........................F                                              
                                                                                
         Time: 00:02.168, Memory: 46.00 MB                                      
                                                                                
         There was 1 failure:                                                   
                                                                                
         1)                                                                     
         Laminas\AutomaticReleases\Test\Unit\Environment\EnvironmentVariablesTes
         t::testDebugModeOffEnvironmentVariables                                
         Failed asserting that two strings are equal.                           
         --- Expected                                                           
         +++ Actual                                                             
         @@ @@                                                                  
         -'INFO'                                                                
         +'DEBUG'                                                               
                                                                                
         /github/workspace/test/unit/Environment/EnvironmentVariablesTest.php:13
         1                                                                      
                                                                                
         FAILURES!                                                              
         Tests: 88, Assertions: 296, Failures: 1.                               
                                                                                
         Generating code coverage report in PHPUnit XML format ... done         
         [00:00.288]                                                            \

Should be reproducible with the same random seed.

@Ocramius Ocramius added Bug Something isn't working Help Wanted labels Dec 18, 2022
@Ocramius Ocramius added this to the 1.20.1 milestone Dec 18, 2022
@Ocramius Ocramius self-assigned this Dec 18, 2022
@Ocramius
Copy link
Member Author

Handled in #225

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant