Providing guidance for __app__ in the face of unit testing

[brettcannon]

If you look at the unit testing example, you will notice it uses relative imports. If you look earlier in that same page you will notice that the suggested folder structure has the folder containing the function files be named FunctionApp. The problem is that if the unit testing code were to switch to using absolute imports and thus import from __app__, then the unit testing example will fail with an ImportError.

I think there are two ways to solve this. One is to not encourage people to use __app__. If you do this then the current recommended practice of making FunctionApp the folder you commit to git and naming it appropriately for your project can continue. That does also mean, though, that beyond discouraging absolute imports via __app__ you will also need to document how run test runners like pytest such that they know where the root of the project is. This is important because if the unit testing example were to suddenly include an import for shared code -- from ..SharedCode import myFirstHelperFunction as listed in the folder structure example -- it would fail as pytest . from within your project/git checkout wouldn't know where the module root was (it works now because pytest just thinks myapp is the top-level package thanks to it only referencing code from within that folder). And you can't do pytest .. as that would cause pytest to search all folders in the parent folder for tests which could be a lot if you keep all of your git clones in a single directory (e.g. all of my GitHub clones are in a Repositories/ folder, so pytest .. would search 5 other repositories on the machine I'm typing from). And so some guidance for how people should tell pytest how to run their tests appropriately would be required (on top of advising people not to use __app__).

The other option is to change the folder structure guidance to be more like the following (see the functions version of pvscbot to see this working in a deployed function):

FunctionApp
 | - __app__
 | | - __init__.py
 | | - MyFirstFunction
 | | | - __init__.py
 | | | - function.json
 | | - MySecondFunction
 | | | - __init__.py
 | | | - function.json
 | | - SharedCode
 | | | - myFirstHelperFunction.py
 | | | - mySecondHelperFunction.py
 | | - host.json
 | | - requirements.txt

By embracing the fact that the package needs to be named __app__ you avoid all of the above mentioned issues with pytest and breaking unit tests if you start using absolute imports without naming your git repository appropriately. It also has a nice side-effect of letting people keep their tests and related requirements specifications outside of the folder that gets deployed to Azure (which should speed up deployment; see the sample repo I linked to above to see how to make a dev-requirements.txt refer to __app__/requirements.txt to practice DRY with dependencies).

The problem is this solution obviously goes against the guidance to have the folder that func init creates for you be what becomes your git repository (e.g. the example on how to use Azure Pipelines for CD won't work out-of-the-box as you now need to deploy a sub-folder of your repository). It also means that func new is creating a .vscode folder in the wrong location.

Anyway, it would be great to solve this somehow as unit testing stateless functions on Azure is really nice and simple thanks to HttpRequest having a constructor that is easy to work with.

[bigdatamoore]

We are facing this same challenge with our functions. Thanks for posting this solution. @asavaritayal It would nice if Microsoft could tackle this as I think it impacts sdlc for many functions users that are trying to write automated unit tests.

[mrossi6]

+1, @brettcannon summarized perfectly. It would be great to have a way to unit test function modules without running the host or bloating code (eg, this stack answer)

[jeffhollan]

Thank you very much for this @brettcannon - was able to dig into this tonight and build an app with SharedCode and ran into the issues mentioned. I really like the structure of the sample project you have provided (and recommend others reference in meantime). Team is looking into right ways that we can surface this. Will circle back in next few days once I hear back.

Only quirkiness I ran into in the interim is when using VS Code to debug, the only way I got it to work where VS Code was happy, the local debugging was happy, and pytest was happy was creating a virtual environment at the root of the project, and a virtual environment in the __app__. Whenever we sort out cleanest ways to support this pattern that's the type of thing we should be able to clean up.

[dxkaufman]

I have wound up using a workaround to allow me to continue to test locally with my functions, but I am concerned that the workaround isn't going to function in Azure. I will have to set up a new appservice plan to test that, which I haven't had time to do yet. At the moment, when I tried creating an __app__ directory and putting all the function code under it, Azure was unable to see the functions. If I can stick with the workaround until this is resolved I will. (see both #8592, mentioned above, and #577)

[nateinindy]

@jeffhollan is there an update yet on this? Calling the project dir "app" is not a good workaround.

[jeffhollan]

Thanks - I know this was being designed a few weeks back and had some related issues that popped up. But adding @anirudhgarg and @kulkarnisonia16 to help track the ask and provide an update

[TSeverinDSE]

A third alternative to the versions above that mitigates the problems:
Create another folder in your root directory called __app__ and make it a package. In this folder you can then create a (relative) symlink to your actual SharedCode folder.

This should give you the following folder structure

FunctionApp
| - __app__
| | - __init__.py
| | - SharedCode [=../SharedCode]
| - MyFirstFunction
| | - __init__.py
| | - function.json
| - MySecondFunction
| | - __init__.py
| | - function.json
| - SharedCode
| | - myFirstHelperFunction.py
| | - mySecondHelperFunction.py
| - host.json
| - requirements.txt

For me (Win, Python 3.7, PyCharm IDE) this works.

[gaurcs]

The above folder structure works all nice until you deploy to Azure as the Azure copies everything under __app__ to /home/site/wwwroot/ folder. So if you use the absolute import method, it fails as it is not able to find the app_ folder.

[dan-osull]

This is my workaround, for what it's worth.

Folder structure:

FunctionApp
| - function1
|| - __init__.py
| - function2
|| - __init__.py
| - SharedCode
|| - settings.py

Each __init__.py begins with:

# Add parent folder to sys.path
# Allows "from SharedCode import x" instead of "from __app__.SharedCode import x"
# This works in both Azure Functions runtime and local debugger
from pathlib import Path
import sys
sys.path.append( (Path(__file__).parent.parent).as_posix() )

# SharedCode
from SharedCode import settings

[gaurcs]

Actually, the folder structure suggested by @brettcannon works . However, if I try to coverage run any files, it gives me import errors. I have to try something to workaround that.

[brettcannon]

@gaurcs how are you running coverage? And what are the exact import failures? I have coverage running on https://github.com/microsoft/pvscbot using this exact directory structure so it might simply require a tweak (you can look at the GitHub Action to see how I'm doing it).

[ryanshepherd]

For newbies like myself, there are some additional configurations that might be confusing when refactoring from the default folder structure to this new structure. I found this comment to be really helpful.

Here's a summary of what I configured to get the VS Code debugging and deploy features working with the new structure.

1. Make sure .vscode is in the root folder (FunctionApp)
2. Make sure .venv is also in the root folder
3. In .vscode\settings.json:
   3a. add the line: "azureFunctions.deploySubpath": "__app__"
   3b. update the line: "azureFunctions.pythonVenv": "..\\.venv"
4. In .vscode\tasks.json:
   4a. Update each task to include an "options" object that specifies the working directory:

    "tasks": [
        {
            "type": "func",
            "command": "host start",
            "problemMatcher": "$func-watch",
            "isBackground": true,
            "dependsOn": "pipInstall",
            "options": {
                "cwd": "${workspaceFolder}\\${config:azureFunctions.deploySubpath}"
            }
        },
        {
            "label": "pipInstall",
            "type": "shell",
            "osx": {
                "command": "${config:azureFunctions.pythonVenv}/bin/python -m pip install -r requirements.txt"
            },
            "windows": {
                "command": "${config:azureFunctions.pythonVenv}\\Scripts\\python -m pip install -r requirements.txt"
            },
            "linux": {
                "command": "${config:azureFunctions.pythonVenv}/bin/python -m pip install -r requirements.txt"
            },
            "problemMatcher": [],
            "options": {
                "cwd": "${workspaceRoot}\\${config:azureFunctions.deploySubpath}"
            }
        }
    ]

Suggestions are welcome.

[Hazhzeng]

Sorry for the long waiting, we updated the Python worker to accept import statements without using __app__ namespace, which should be much cleaner and intuitive. I've updated the folder structure, import behavior and unit testing guide in our official docs: https://docs.microsoft.com/en-us/azure/azure-functions/functions-reference-python#folder-structure