Functions Framework buildpack fails when .venv folder is present in the repo

[xSAVIKx]

Describe the bug

While using a local .venv folder the buildpack fails to complete the python3 -m compileall -f -q . command with the following errors:

[builder] === Python - Functions Framework (google.python.functions-framework@0.9.6) ===
[builder] --------------------------------------------------------------------------------
[builder] Running "python3 -m compileall -f -q ."
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/_pytest/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/_pytest/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/mypy/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/mypy/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/mypyc/irbuild/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/mypyc/irbuild/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pip/_internal/cli/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pip/_internal/cli/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pip/_internal/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pip/_internal/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pre_commit/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pre_commit/__pycache__'
[builder] *** Error compiling './python_functions_cloud_run_buildpacks/__init__.py'...
[builder] PermissionError: [Errno 13] Permission denied: './python_functions_cloud_run_buildpacks/__pycache__'
[builder] *** Error compiling './python_functions_cloud_run_buildpacks/example.py'...
[builder] PermissionError: [Errno 13] Permission denied: './python_functions_cloud_run_buildpacks/__pycache__'
[builder] Done "python3 -m compileall -f -q ." (159.000402ms)
[builder] Failure: (ID: 49c34848) *** Error compiling './.venv/lib/python3.11/site-packages/_pytest/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/_pytest/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/mypy/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/mypy/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/mypyc/irbuild/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/mypyc/irbuild/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pip/_internal/cli/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pip/_internal/cli/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pip/_internal/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pip/_internal/__pycache__'
[builder] *** Error compiling './.venv/lib/python3.11/site-packages/pre_commit/main.py'...
[builder] PermissionError: [Errno 13] Permission denied: './.venv/lib/python3.11/site-packages/pre_commit/__pycache__'
[builder] *** Error compiling './python_functions_cloud_run_buildpacks/__init__.py'...
[builder] PermissionError: [Errno 13] Permission denied: './python_functions_cloud_run_buildpacks/__pycache__'
[builder] *** Error compiling './python_functions_cloud_run_buildpacks/example.py'...
[builder] PermissionError: [Errno 13] Permission denied: './python_functions_cloud_run_buildpacks/__pycache__'
[builder] --------------------------------------------------------------------------------
[builder] Sorry your project couldn't be built.
[builder] Our documentation explains ways to configure Buildpacks to better recognise your project:
[builder]  -> https://cloud.google.com/docs/buildpacks/overview
[builder] If you think you've found an issue, please report it:
[builder]  -> https://github.com/GoogleCloudPlatform/buildpacks/issues/new
[builder] --------------------------------------------------------------------------------
[builder] ERROR: failed to build: exit status 1

Additional context
How are you using GCP buildpacks?

• pack and the gcr.io/buildpacks/builder
• Cloud Functions
• Cloud Run
• Cloud Build
• App Engine Standard
• App Engine Flex

What language is your project primarily written in?
Python

Steps To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/xSAVIKx/python-functions-cloud-run-buildpacks.git
2. cd python-functions-cloud-run-buildpacks
3. poetry install
4. ./build.sh

Or just see the GH build here: https://github.com/xSAVIKx/python-functions-cloud-run-buildpacks/actions/runs/4957968805/jobs/8870285151

Expected behavior
The build completes successfully.

Actual behavior
The buildpack fails to compile files that do not belong to the build scope and are not accessible.

Similar problem happens when e.g. you have a source package with some on-purpose invalid Python code in the tests. The compileall tries to compile an invalid Python file from the tests and this makes the whole build fail as well.

[kennethye1]

#301 (reply in thread)
Please see if this fixes it for you.

[xSAVIKx]

But how is that thread related to this one? Is it a requirement to always exclude local virtual env?
I believe I can come up with a project that'd fail even if it just has an invalid .py file in the report (that is there for a test scenario).

The problem is that there is a compileall with a . in it IMO. It should be somehow restricted only to the actual sources.

[jama22]

Sorry for the categorization whiplash, if its an issue with the functions framework buildpack our team might have to dive deeper into it

[xSAVIKx]

The one described in this issue probably is. The exclude/include thing we're discussing here #301 is not directly related.

[xSAVIKx]

any updates?

[josephlewis42]

Hi @xSAVIKx,

I just looked through the buildpack and it seems like this behavior is intentional. We're compiling all the source code that's provided to ensure that any code that's potentially reachable at runtime is syntactically valid. We currently expect that you will only provide the bulidpack with source code that's relevant to the running of your function and would exclude the rest.

Can you help us understand more about your use-case here so we can try to come up with an appropriate solution?

Thanks!

[xSAVIKx]

@josephlewis42, thx for checking this. The problem is that libraries sometimes bring their test code with them and test code may intentionally have invalid code that is run and tested in a controlled environment. This particular problem we've seen with the Functions Framework. The one that was built by the Google teams for cloud functions and it was failing to compile while they had intentionally wrong code in the tests.