Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support adding request data in on demand transforms #1851

Merged
merged 11 commits into from
Sep 13, 2021

Conversation

adchia
Copy link
Collaborator

@adchia adchia commented Sep 10, 2021

What this PR does / why we need it:
Supports adding request data as a dependency in on demand transforms. This validates when if the entity_df or entity_rows don't have the required request data features.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I think we should add some tests to assert proper handling of the error cases, but looks good otherwise.

sdk/python/feast/data_source.py Outdated Show resolved Hide resolved
sdk/python/feast/feature_store.py Outdated Show resolved Hide resolved
sdk/python/feast/feature_store.py Show resolved Hide resolved
sdk/python/feast/feature_store.py Show resolved Hide resolved
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
@codecov-commenter
Copy link

codecov-commenter commented Sep 10, 2021

Codecov Report

Merging #1851 (3a83dff) into master (1b809a7) will increase coverage by 0.21%.
The diff coverage is 97.46%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1851      +/-   ##
==========================================
+ Coverage   84.87%   85.08%   +0.21%     
==========================================
  Files          91       91              
  Lines        6961     7089     +128     
==========================================
+ Hits         5908     6032     +124     
- Misses       1053     1057       +4     
Flag Coverage Δ
integrationtests 85.01% <97.46%> (+0.21%) ⬆️
unittests 62.66% <35.44%> (-0.54%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdk/python/feast/infra/offline_stores/bigquery.py 81.94% <ø> (ø)
sdk/python/feast/infra/offline_stores/file.py 97.43% <ø> (ø)
sdk/python/feast/infra/offline_stores/redshift.py 90.99% <ø> (ø)
...ts/integration/feature_repos/repo_configuration.py 100.00% <ø> (ø)
sdk/python/feast/data_source.py 61.29% <88.23%> (+4.28%) ⬆️
sdk/python/feast/errors.py 70.79% <100.00%> (+1.63%) ⬆️
sdk/python/feast/feature_store.py 94.34% <100.00%> (+0.58%) ⬆️
sdk/python/feast/on_demand_feature_view.py 89.09% <100.00%> (+3.90%) ⬆️
...tegration/feature_repos/universal/feature_views.py 96.42% <100.00%> (+0.42%) ⬆️
...fline_store/test_universal_historical_retrieval.py 99.18% <100.00%> (+0.05%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b809a7...3a83dff. Read the comment docs.

Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Signed-off-by: Danny Chiao <danny@tecton.ai>
Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Comment on lines +524 to +526
@staticmethod
def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
raise NotImplementedError
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if this method makes sense for this type of data source. We should probably refactor the DataSource interface to make it simpler and introduce a BatchDataSource somewhere in the type hierarchy.

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: achals, adchia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot feast-ci-bot merged commit 101e014 into feast-dev:master Sep 13, 2021
@adchia adchia deleted the request_data branch September 15, 2021 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants