add localmod storage

[jacksund]

Summary

This is the start to a LocalModule storage class. It is currently working with Prefect Server along with a LocalAgent, but full testing and features have not been added yet. Thus, this is a draft PR for discussion of code. Prior discussion of this topic is within #3304 and #3335 threads.

Changes

• adds localmod.py (with LocalModule class) to prefect.environments.storage
• adds LocalModule storage schema to prefect.serialization.storage
• adds LocalModule storage as a default label and support for LocalAgent
• adds tests for the above changes [incomplete at the moment]

Importance

Local module based storage allows for easier sharing of workflow libraries, where flows are accessible in the python path. LocalModule will serve just like Local with stored_as_script=True, where instead of a path='/path/to/my/flows/exampleflow.py', you could use module_path='from myrepo.flows import exampleflow' or 'myrepo.flows.exampleflow'. Thus the flow is accessible in any environment where that module import works, not just the environments that share the same path structure.

Checklist

NOTE -- this is a draft PR so the tasks below are still ongoing.

This PR:

• adds new tests (if appropriate)
• adds a change file in the changes/ directory (if appropriate)
• updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

[marvin-robot]

Here I am, brain the size of a planet and they ask me to welcome you to Prefect.

So, welcome to the community @jacksund! 🎉 🎉

[cicdw]

Hi @jacksund - first off, apologies for the extended delay on reviewing. Now that I've seen your implementation (which is great!) I realize that we don't need to create an entirely new class for this storage pattern.

In particular, I think we can expand the types of location strings we accept on the pre-existing Local storage class to allow for module imports instead of file paths. And note that we will only use this functionality when store_as_script=True. In that case we can do something like

module_path, class_str = location.split(".")[:-1], location.split(".")[-1]

and recreate your functionality here.

Let me know what you think about that, and apologies for not identifying this earlier!

[jacksund]

No worries on the slow reply :). And yeah, I can definitely merge the functionality, so I'll give this a go.

Thanks for taking a look!

[jacksund]

Alright so I have a working version if you take a look at my branch again. It ended up being extremely little code, so it's nice and easy to read through. There are a couple things to note though: (1) I haven't updated the doc-strings to describe the new functionality, (2) I didn't change the 'path' parameter to 'location' as you suggest because I'm scared that will cause breaks in the code elsewhere.

While everything looks to be running correctly on my end, I'm completely new to circleci, so I don't understand the build errors that I'm getting. Would you be willing to walk me through these?

[marvin-robot]

Here I am, brain the size of a planet and they ask me to welcome you to Prefect.

So, welcome to the community @jacksund! 🎉 🎉

[codecov]

Codecov Report

Merging #3351 into master will decrease coverage by 0.06%.
The diff coverage is 46.66%.

[cicdw]

Two minor requests.

Your CircleCI job is faliing because of our formatting rules - if you install and run black on the codebase you should be good; here's how to do that from the CLI:

pip install -U black
black . # note the period

Otherwise, if you could add a changelog entry to the changes/ directory with your contributor info I think we're good to go!

[jacksund]

I think that covers the docs and typos. Let me know if I need to do anything else.

[cicdw]

LGTM thank you for the cool contribution @jacksund ! Hope it's your first of many :D