Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repository size #21

Open
blazk opened this issue Sep 27, 2019 · 7 comments
Open

repository size #21

blazk opened this issue Sep 27, 2019 · 7 comments
Assignees
Labels
enhancement New feature or request

Comments

@blazk
Copy link

blazk commented Sep 27, 2019

Hi, I've just tried to clone Lisvap repository and the size is 2.3 GB. it's a bit unmanageable. Could it be possibly reduced?

@domeniconappo
Copy link
Contributor

@blazk
I struggled to make it "uploadable" on github so I've already reduced at minimum the size of input files for tests. That's the source of the problem. Let us discuss to find an alternative. At the moment, we only have that kind of black box tests and we need a minimum "full dataset". Since tests are for developers, they must be included in the repository.
Same problem on lisflood-code repo but here we have a GLOFAS test so here it's more problematic.

What do you need to do with it? Which kind of managing problems do you have in particular?
Are you going to clone the repo from scratch so often?
I mean: the typical use case is to clone it once so it's not a real problem.

@blazk
Copy link
Author

blazk commented Sep 30, 2019

Thanks for looking into it!

It will be cloned often, we have automated deployment procedures where our systems are assembled from many packages and these packages are cloned from git repos.

Do developers actually need to use realistic maps to run tests? If these tests only check if results stay the same after the code change, then maybe you could use small e.g. 3x2 pixel maps (attached)
3x2_map.zip

@domeniconappo
Copy link
Contributor

Hi,
ideally, we shouldn't have those real case tests but rather a battery of unit tests and some full tests on smaller maps. For the moment, we need it as we feel safer in that way :) Code is far away to be well designed to easily write unit tests.
Anyway, lisvap can be used as a pypi package. The package does not include tests and it's very quick to install. Is it an option to install it with the pip tool of your Python env?

pip install lisflood-lisvap

@domeniconappo
Copy link
Contributor

@blazk
Hi Blazej,
any opinion on this?

@blazk
Copy link
Author

blazk commented Oct 3, 2019

I suppose we could use "pip install lisflood-lisvap". How much up-to-date do you keep the pip package?

When we install from git, we have a custom installation procedure. Part of this procedure is to configure the package, e.g. in case of LISVAP we set the "PathBaseMapsIn" variable in the XML settings template file - this variable does not change in our case so it can be set to a fixed value during package's installation. It may sound like a small detail but actually it's important from architecture point of view (encapsulation!) - it means that the client code that calls LISVAP doesn't need to know how to set "PathBaseMapsIn", which means the client code can be simpler.

But anyway, sorry for digression, I think we can do it, eg. install LISVAP from pip and then have an auxiliary package with the XML settings template only.

@domeniconappo
Copy link
Contributor

Pip package is in sync with git version, as soon as there is a new tag/release.

Maybe we can use pip with --install-option arg, to pass extra arguments to the setup.py script. So you can pass the path and I can produce the XML template for you during installation. I will look into it in a few days.

https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-install-option

@blazk
Copy link
Author

blazk commented Oct 3, 2019

sounds good, thanks!

@domeniconappo domeniconappo self-assigned this Nov 4, 2019
@domeniconappo domeniconappo added the enhancement New feature or request label Nov 4, 2019
@domeniconappo domeniconappo pinned this issue Nov 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants