Smoke testing

[skshetry]

It'd be great if we could do a smoke testing before releases.

See: #4318

[efiop]

@skshetry Could you elaborate on how you see it applied in dvc?

[skshetry]

1. Build package.
2. Run tests (e2e types) that are critical for the release.
3. Deploy package if it passes.

We could create a separate repository for this and run those scripts before "deploying".

[skshetry]

And, on smoke tests, we should test something that's very simple (dvc add directory/file, import/get, etc) and/or something that differs between different packages (eg: related to dependencies, etc). Any other kinds of tests can be added as we see fit later on.

And, other specialized tests can be easily covered with our func/unit tests (even if it is not, smoketest won't help here).

[skshetry]

Closing as stale.