Feast plan clean up #2256

felixwang9817 · 2022-01-29T00:43:07Z

What this PR does / why we need it: This PR cleans up feast plan. Specifically:

Fco is renamed to FeastObject.
Unchanged objects are no longer logged.
first feast apply:

then feast plan:
Validation and inference are run during feast plan.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

feast plan output only includes changed objects

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

codecov-commenter · 2022-01-29T01:22:48Z

Codecov Report

Merging #2256 (335e8db) into master (12e2130) will decrease coverage by 0.00%.
The diff coverage is 55.17%.

@@            Coverage Diff             @@
##           master    #2256      +/-   ##
==========================================
- Coverage   85.31%   85.30%   -0.01%     
==========================================
  Files         108      108              
  Lines        9127     9108      -19     
==========================================
- Hits         7787     7770      -17     
+ Misses       1340     1338       -2

Flag	Coverage Δ
integrationtests	`73.24% <31.03%> (+0.17%)`	⬆️
unittests	`59.66% <50.00%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdk/python/feast/diff/infra_diff.py	`75.30% <0.00%> (-2.90%)`	⬇️
sdk/python/feast/inference.py	`87.50% <ø> (ø)`
sdk/python/feast/diff/registry_diff.py	`72.03% <37.83%> (ø)`
sdk/python/feast/feature_store.py	`90.46% <100.00%> (+0.93%)`	⬆️
sdk/python/feast/repo_operations.py	`54.87% <100.00%> (+0.27%)`	⬆️
sdk/python/tests/integration/e2e/test_usage_e2e.py	`100.00% <100.00%> (ø)`
sdk/python/tests/unit/diff/test_registry_diff.py	`100.00% <100.00%> (ø)`
.../integration/online_store/test_universal_online.py	`97.91% <0.00%> (+0.41%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 12e2130...335e8db. Read the comment docs.

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

adchia

/lgtm

adchia · 2022-01-30T17:01:49Z

sdk/python/feast/feature_store.py

-        )
-        self._make_inferences(entities_to_update, views_to_update, odfvs_to_update)
-
-        # Apply infra and registry changes.
        infra_diff.update()


non-blocking, but another cleanup would be to have InfraObject#update -> InfraObject#create or something since it only right now creates the object

feast-ci-bot · 2022-01-30T17:01:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adchia, felixwang9817

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [adchia,felixwang9817]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Add backticks to left_table_query_string (#2250) Signed-off-by: david <davidmiller252@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Delete entity key from Redis only when all attached feature views are gone (#2240) * Delete entity from redis when the last attached feature view is deleted Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * Delete entity key from Redis only when all attached feature views are gone Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * make lint happy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * make lint happy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * one more try with mypy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * historical_field_mappings2 merge for one sign off commit (#2252) Signed-off-by: Michelle Rascati <michelle.rascati@sailpoint.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Correct inconsistent dependency (#2255) Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Add snowflake environment variables to allow testing on snowflake infra (#2258) * add snowflake environment vars to test framework Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * add snowflake environment vars to test framework Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Return `UNIX_TIMESTAMP` as Python `datetime` (#2244) * Refactor `UNIX_TIMESTAMP` conversion Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Return `UNIX_TIMESTAMP` types as `datetime` to user Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Fix linting errors Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Rename variable to something more sensible Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Feast plan clean up (#2256) * Run validation and inference on views and entities during plan Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Do not log objects that are unchanged Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Rename Fco to FeastObject Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Remove useless method Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Lint Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Always initialize registry during feature store initialization Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Fix usage test Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Remove print statements Signed-off-by: Felix Wang <wangfelix98@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Squash commits Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Add error type and refactor query execution to have retries Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Handle more snowflake errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix wrong import Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * modify registry.db s3 object initialization to work in S3 subdirectory with Java Feast Server (#2259) Signed-off-by: NalinGHub <nalinm01@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * clean up docs Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * lint-python Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * fixed historical test Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * fixed historical test Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> Co-authored-by: David Miller <david@patagona.ca> Co-authored-by: Oleksii Moskalenko <moskalenko.alexey@gmail.com> Co-authored-by: Michelle Rascati <44408275+michelle-rascati-sp@users.noreply.github.com> Co-authored-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Co-authored-by: Felix Wang <wangfelix98@gmail.com> Co-authored-by: Danny Chiao <danny@tecton.ai> Co-authored-by: Nalin Mehra <37969183+NalinGHub@users.noreply.github.com>

felixwang9817 added 4 commits January 28, 2022 16:19

Run validation and inference on views and entities during plan

817a8da

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

Do not log objects that are unchanged

22e3eec

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

Rename Fco to FeastObject

6afdeba

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

Remove useless method

8f5d9ef

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

felixwang9817 requested a review from a team as a code owner January 29, 2022 00:43

felixwang9817 requested review from woop and removed request for a team January 29, 2022 00:43

feast-ci-bot added release-note approved needs-kind labels Jan 29, 2022

felixwang9817 requested review from adchia and removed request for woop January 29, 2022 00:43

felixwang9817 assigned adchia Jan 29, 2022

feast-ci-bot added the size/L label Jan 29, 2022

github-actions bot assigned tsotnet Jan 29, 2022

felixwang9817 added kind/feature New feature or request and removed size/L labels Jan 29, 2022

feast-ci-bot removed the needs-kind label Jan 29, 2022

felixwang9817 added needs-kind ok-to-test labels Jan 29, 2022

Lint

4d5eb6b

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

feast-ci-bot added the size/L label Jan 29, 2022

felixwang9817 added ok-to-test and removed ok-to-test needs-kind labels Jan 29, 2022

felixwang9817 unassigned tsotnet Jan 29, 2022

Always initialize registry during feature store initialization

bc720f5

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

felixwang9817 added 2 commits January 28, 2022 17:37

Fix usage test

4e93615

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

Remove print statements

335e8db

Signed-off-by: Felix Wang <wangfelix98@gmail.com>

adchia approved these changes Jan 30, 2022

View reviewed changes

feast-ci-bot added the lgtm label Jan 30, 2022

feast-ci-bot merged commit 895589a into feast-dev:master Jan 30, 2022

tpvasconcelos mentioned this pull request May 15, 2022

Performing I/O on instantiation #2697

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feast plan clean up #2256

Feast plan clean up #2256

felixwang9817 commented Jan 29, 2022 •

edited by adchia

Loading

codecov-commenter commented Jan 29, 2022 •

edited

Loading

adchia left a comment

adchia Jan 30, 2022

feast-ci-bot commented Jan 30, 2022

Feast plan clean up #2256

Feast plan clean up #2256

Conversation

felixwang9817 commented Jan 29, 2022 • edited by adchia Loading

codecov-commenter commented Jan 29, 2022 • edited Loading

Codecov Report

adchia left a comment

Choose a reason for hiding this comment

adchia Jan 30, 2022

Choose a reason for hiding this comment

feast-ci-bot commented Jan 30, 2022

felixwang9817 commented Jan 29, 2022 •

edited by adchia

Loading

codecov-commenter commented Jan 29, 2022 •

edited

Loading