Locking keripy dependency

[iFergal]

Opening a new issue for this more generally but #152 already mentions this. Would it be OK if we pin the githash of keripy used? e.g. keri @ git+https://git@github.com/WebOfTrust/keripy.git@2d530601d3463d820229de0676cda519523b73ca

Right now we are lacking determinism for building KERIA - building older versions typically break, and even the latest can break if it falls slightly behind keripy.

This in general causes problems with deployment to environments with multiple team members, git bisect, CICD determinism etc.

[psteniusubi]

What's a working strategy for this? Can it be automated?

Right now integration tests in signify-ts are failing if running with the latest development branches of keria and keripy. I have to roll back both keria and keripy to make the tests running.

[iFergal]

I don't think it needs to be automated though, or if it even could - it's just a dependency version. development would point to a certain githash that works, and then if a feature branch of keria requires an update of keripy, that PR should also update the githash. (this unfortunately requires some coordination between branches - a branch that's open for longer shouldn't roll back to an older githash but I think that should be a merge conflict to resolve anyway)

Automatically updating the githash would mean automatically making the code changes in KERIA if there were breaking changes.

My current solution when tests fail is to find the githash on keripy that occured right before the latest commit to KERIA. :P

[lenkan]

I agree that we need a solution to ensure deterministic builds. From my point of view, just pinning the dependency to a commit seems reasonable and simple enough. The workflow to manually update the dependency does not seem to add any significant overhead.

[iFergal]

@m00sey Bump =)

Right now in my team we are sharing a manually deployed KERIA instance (/pointing to local KERIAs in dev) so I can work around this, but we are soon changing to infra as code so deterministic builds would be awesome!

[lenkan]

@iFergal can this be closed now since Keria is pointing to a published pypi package of keripy?

keria/setup.py

Line 79 in 4217166

'keri>=1.2.0-dev0',

[iFergal]

@lenkan I'm not super familiar with Python deps but I think this means it would also use version 1.3.0 or even 2.0.0 if available - it seems there's no lockfile concept here. Or does the dev tag affect that?

It's probably OK since releases will be coordinated, but it would be a headache if we were trying to find the root cause of a regression using git bisect or something.

[iFergal]

This still applies; at time of writing setup.py relies on >=1.2.0-dev0 but in a fresh venv it is pulling 1.2.0-dev2 which is not compatible.

@pfeairheller OK if we use == instead? Or use poetry/pyproject.toml instead so we have a lockfile? Not sure on what's involved, or if we should, but happy to give a try.

[pfeairheller]

Interesting that pip will roll up to a "dev2", I assumed it wouldn't do that. Switching to an "==" is the right fix here.