Test Coverage: End to End Tests for Dataflow Runner

[mrzzy]

Is your feature request related to a problem? Please describe.

Currently, Feast's end to end tests only verify correctness with Feast Core's Direct Runner, leaving a gap in test coverage for the Dataflow runner. This is an issue a we are unable to verify that new additions to the codebase work well with the Dataflow runner, requiring PR authors to do error prone manual testing.

Case study: PR #575

• I submitted small PR Fix runner to string inconsistency #575 which updated how Runners are rendered.
• Since the PR is relatively small and all the tests pass, I assumed that the PR did not introduce any bugs and didn't do any independent testing on my own.
• I only discovered that the PR broke the Dataflow runner after it was merged, leaving a known bug on the master branch.

This bug in the PR would have been caught by Feast's test suite, had it tested with the Dataflow runner in addition to the Direct runner.

Describe the solution you'd like
Add End to End Tests using the Dataflow Runner, in additional to the Direct Runner.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[woop]

I believe this is now resolved: https://github.com/feast-dev/feast/blob/master/.prow/config.yaml#L263