Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use GitHub Actions for CI #26

Open
NickleDave opened this issue Mar 29, 2021 · 9 comments
Open

use GitHub Actions for CI #26

NickleDave opened this issue Mar 29, 2021 · 9 comments

Comments

@NickleDave
Copy link

as mentioned here the UCB-MDS cookiecutter does this so we could adapt / integrate with theirs
#25 (comment)

@xmnlab and @mbjoseph discussed Azure Pipelines in #13 but my impression is that GitHub Actions is a million times easier to use, especially for newbies (and people like me who aren't so newbie but do not want to spent their whole lives thinking about CI pipelines)

@mbjoseph
Copy link
Member

Thanks @NickleDave I agree GitHub actions > Azure at present!

@xmnlab
Copy link
Collaborator

xmnlab commented Mar 29, 2021

@NickleDave you're right. nowadays muchos han migrado de azure pipelines para github actions (including me :) )

@lwasser
Copy link
Member

lwasser commented Mar 29, 2021

oh yea. actions are so nice for windows-mac-linux. i'm using them now on all of our packages. including earthpy and abc-classroom.

just a note that i have a few reservations about poetry. it seems to be very pip focused. i need to revisit our cookie cutter again but pip and conda do not place nicely together. And for some applications (SPATIAL!!) pip is not optimal given GDAL conflicts. so we need to think carefully about what best practice recommendations look like. i may also be wrong about poetry but that was my first take when i looked at it. i've slowly been moving towards conda and conda-lock to lock environment builds . very open to any and all feedback here and to build upon the great work that @ttimbers has done!!

@ttimbers
Copy link

Tagging @TomasBeuzen here because he has more experience than I do mixing poetry with conda. AFAIK from my discussions with him, the two do play well together, and we plan to better explain that in our book.

@TomasBeuzen - can you elaborate a bit here on your experiences mixing poetry with conda? And any cons/challenges in doing so?

@TomasBeuzen
Copy link

TomasBeuzen commented Mar 29, 2021

I'm a bit convoluted in my workflow. I use conda exclusively for environment management and for installing packages into those environments where possible. I use poetry to make pure Python packages, but I don't use it for dependency management/installing packages for a project 😬 . So, I don't have much experience installing packages into an environment using conda and poetry, but I suspect you'll run into the same kinds of problems you get when mixing conda and pip.

@ttimbers
Copy link

Ah, I see. I think I misunderstood what you told me before Tom!

@TomasBeuzen
Copy link

TomasBeuzen commented Mar 29, 2021

Just to be clear - conda virtual environments work great with poetry and I use the two together like that all the time, I'm just not too sure about how well the two mix together when installing things :)

@lwasser
Copy link
Member

lwasser commented Mar 29, 2021

thank you all for this. it will be great to sort this all out! even mixing conda forge and defaults can cause conflicts so i'm just sensitive to those pain points! i do hope to find some time to play with poetry to better understand what it offers! (and to revisit our cookie cutter as well! And i seriously appreciate everyone's input here - thank you!! keep the discussion and suggestions coming!

@NickleDave
Copy link
Author

NickleDave commented Mar 31, 2021

My usage lines up with @TomasBeuzen -- I use conda as a quick and dirty way to manage environments, e.g. for a project that's mainly experiments + analysis, not a library

I use poetry for developing pure Python packages, and I also try to use it to "pin" the development environment for myself.

I for sure agree that there are both pros and cons to every tool and I definitely don't mean to suggest we should impose poetry as the one true way.

I just really like how easy it can be for pure Python packages and I bet that captures a lot of use cases, especially for devs that we want to support. Like, if I'm writing a wrapper for an API, I can just use requests and I don't have some painful process that involves scikit-build or whatever.

That's why I like the book from @ttimbers and @TomasBeuzen ... an advantage might be making it as easy as possible to get started for as many people as possible, without having to worry about the 12 different ways to package a Python library.

As for cons ... witness some of my interactions with the poetry maintainers here:
python-poetry/poetry#3757 (reply in thread)
😬 Not my proudest moment but I think the discussion highlights a lot of the pain points in packaging.
I am still myself fighting to build up a packaging "concept map" in my head, and understand how all the moving parts relate to each other.

I was excited to learn from one of the maintainers that we will soon be able to declare metadata in a "tool neutral" section in pyrproject.toml files:
python-poetry/poetry#3757 (reply in thread)
so there will be less 'lock-in' and tools like conda-build will be able to leverage it as well

flit just added experimental support for this:
pypa/flit#393

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants