Error: EACCES: permission denied, scandir .pytest_cache with 8.2.0

[danyeaw]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

When upgrading to Pytest 8.2.0, I was getting a Error: The template is not valid. .github/workflows/full-build.yml (Line: 92, Col: 16): hashFiles('**/poetry.lock') failed. Fail to hash files under directory '/home/runner/work/gaphor/gaphor'. When I turned on debug mode on the runner, it looks like the error is caused by an access issue with .pytest_cache which could be a regression caused by #12168.

OS: Fedora 39 Docker running on Ubuntu 22.04 VM
Python: Python 3.11.9
Pytest: 8.2.0

The error happens during the Post Use Dependency Cache step with actions/cache@0c45773.

The larger traceback is:

##[debug]..Evaluating hashFiles:
##[debug]....Evaluating String:
##[debug]....=> '**/poetry.lock'
##[debug]Search root directory: '/home/runner/work/gaphor/gaphor'
##[debug]Search pattern: '**/poetry.lock'
##[debug]Starting process:
##[debug]  File name: '/home/runner/runners/2.316.1/externals/node16/bin/node'
##[debug]  Arguments: '"/home/runner/runners/2.316.1/bin/hashFiles"'
##[debug]  Working directory: '/home/runner/work/gaphor/gaphor'
##[debug]  Require exit code zero: 'False'
##[debug]  Encoding web name:  ; code page: ''
##[debug]  Force kill process on cancellation: 'False'
##[debug]  Redirected STDIN: 'False'
##[debug]  Persist current code page: 'False'
##[debug]  Keep redirected STDIN open: 'False'
##[debug]  High priority process: 'False'
##[debug]Updated oom_score_adj to 500 for PID: 4319.
##[debug]Process started with process id 4319, waiting for process exit.
##[debug]Match Pattern: **/poetry.lock
##[debug]::debug::followSymbolicLinks 'false'
##[debug]::debug::followSymbolicLinks 'false'
##[debug]::debug::implicitDescendants 'true'
##[debug]::debug::matchDirectories 'true'
##[debug]::debug::omitBrokenSymbolicLinks 'true'
##[debug]::debug::Search path '/home/runner/work/gaphor/gaphor'
##[debug][Error: EACCES: permission denied, scandir '/home/runner/work/gaphor/gaphor/.pytest_cache'] {
##[debug]  errno: -13,
##[debug]  code: 'EACCES',
##[debug]  syscall: 'scandir',
##[debug]  path: '/home/runner/work/gaphor/gaphor/.pytest_cache'
##[debug]}
##[debug]STDOUT/STDERR stream read finished.
##[debug]STDOUT/STDERR stream read finished.
##[debug]Finished process 4319 with exit code 1, and elapsed time 00:00:00.0692324.
##[debug].github/workflows/full-build.yml (Line: 92, Col: 16):
Error: The template is not valid. .github/workflows/full-build.yml (Line: 92, Col: 16): hashFiles('**/poetry.lock') failed. Fail to hash files under directory '/home/runner/work/gaphor/gaphor'

The full run is here: https://github.com/gaphor/gaphor/actions/runs/8976797623/job/24855725418

The output of pip list for this environment is:

Package                       Version   Editable project location
----------------------------- --------- -------------------------
alabaster                     0.7.16
asttokens                     2.4.1
attrs                         23.2.0
Babel                         2.15.0
BabelGladeExtractor           0.7.0
beautifulsoup4                4.12.3
better-exceptions             0.3.3
certifi                       2024.2.2
cfgv                          3.4.0
charset-normalizer            3.3.2
click                         8.1.7
comm                          0.2.1
coverage                      7.4.3
debugpy                       1.8.1
decorator                     5.1.1
defusedxml                    0.7.1
distlib                       0.3.8
docutils                      0.20.1
dulwich                       0.22.1
exceptiongroup                1.2.0
executing                     2.0.1
fastjsonschema                2.19.1
filelock                      3.13.1
furo                          2024.1.29
gaphas                        4.0.0
gaphor                        2.25.1    /home/dan/Projects/gaphor
generic                       1.1.2
greenlet                      3.0.3
hypothesis                    6.100.5
identify                      2.5.35
idna                          3.7
imagesize                     1.4.1
importlib-metadata            7.0.1
iniconfig                     2.0.0
ipykernel                     6.29.3
ipython                       8.24.0
jedi                          0.19.1
Jinja2                        3.1.3
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
jupyter-cache                 1.0.0
jupyter_client                8.6.0
jupyter_core                  5.7.1
markdown-it-py                3.0.0
MarkupSafe                    2.1.5
matplotlib-inline             0.1.6
mdit-py-plugins               0.4.0
mdurl                         0.1.2
myst-nb                       1.1.0
myst-parser                   2.0.0
nbclient                      0.9.0
nbformat                      5.9.2
nest-asyncio                  1.6.0
nodeenv                       1.8.0
packaging                     23.2
parso                         0.8.3
pastel                        0.2.1
pexpect                       4.9.0
pillow                        10.3.0
pip                           24.0
platformdirs                  4.2.0
pluggy                        1.5.0
poethepoet                    0.26.1
pre-commit                    3.7.0
prompt-toolkit                3.0.43
psutil                        5.9.8
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycairo                       1.26.0
pydot                         2.0.0
Pygments                      2.17.2
PyGObject                     3.48.2
pyparsing                     3.1.1
pytest                        8.2.0
pytest-archon                 0.0.6
pytest-cov                    5.0.0
pytest-randomly               3.15.0
python-dateutil               2.8.2
PyYAML                        6.0.1
pyzmq                         25.1.2
referencing                   0.33.0
requests                      2.31.0
rpds-py                       0.18.0
setuptools                    69.1.1
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.5
Sphinx                        7.3.7
sphinx-basic-ng               1.0.0b2
sphinx-copybutton             0.5.2
sphinx-intl                   2.2.0
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
SQLAlchemy                    2.0.27
stack-data                    0.6.3
tabulate                      0.9.0
tinycss2                      1.3.0
tomli                         2.0.1
tornado                       6.4
traitlets                     5.14.1
typing_extensions             4.10.0
urllib3                       2.2.1
virtualenv                    20.25.1
wcwidth                       0.2.13
webencodings                  0.5.1
xdoctest                      1.1.3
zipp                          3.17.0

[bluetech]

For understanding better, can you add a step ls -l after the pytest run but before the cache teardown step? Actually, also before the pytest run would be helpful.

[bluetech]

Specifically ls -l /home/runner/work/gaphor/gaphor/.

[danyeaw]

Hi @bluetech yup, let me try to build up a quick minimum reproducible repo and I'll add that as a step.

[danyeaw]

I created a minimum reproducible example with a single test, no dependencies except for Pytest 8.2.0, on a Fedora 39 image is here:
https://github.com/danyeaw/pytest-issue-reproduction

The list of files is:

Run ls -l -a
total 60
drwxr-xr-x 7 root root  4096 May 14 01:56 .
drwxr-xr-x 3 1001  127  4096 May 14 01:55 ..
drwxr-xr-x 8 root root  4096 May 14 01:56 .git
drwxr-xr-x 3 root root  4096 May 14 01:56 .github
-rw-r--r-- 1 root root  3078 May 14 01:56 .gitignore
drwx------ 3 root root  4096 May 14 01:56 .pytest_cache
drwxr-xr-x 5 root root  4096 May 14 01:56 .venv
-rw-r--r-- 1 root root 11357 May 14 01:56 LICENSE
-rw-r--r-- 1 root root   180 May 14 01:56 README.md
drwxr-xr-x 2 root root  4096 May 14 01:56 __pycache__
-rw-r--r-- 1 root root  2714 May 14 01:56 poetry.lock
-rw-r--r-- 1 root root   349 May 14 01:56 pyproject.toml
-rw-r--r-- 1 root root   103 May 14 01:56 test_example.py

The test run with debugging enabled is here:
https://github.com/danyeaw/pytest-issue-reproduction/actions/runs/9072535335

[bluetech]

Makes sense:

drwx------ 3 root root  4096 May 14 01:56 .pytest_cache

I assume that the TemporaryDirectory we're using since #12168 is creating a directory with 700 permissions, while the github cache action wants to access the directory as another user so wants 755 permissions.

The fix would be to add path.chmod(0o755) after this line. Let me see.

[danyeaw]

Thanks so much for the fix @bluetech!