Skip to content

Commit

Permalink
Sync master to release (#20)
Browse files Browse the repository at this point in the history
* chore: Update language.md (feast-dev#4412)

Update language.md

* feat: Create ADOPTERS.md (feast-dev#4410)

* Create ADOPTERS.md

* Update ADOPTERS.md

* fix: Using repo_config parameter in teardown to allow for feature-store-yaml overrides (feast-dev#4413)

* fix: using repo_config parameter in teardown to allow for feature-store-yaml overrides

Signed-off-by: Dan Baron <dan.baron@starlingbank.com>

* fix: fixing linting and formatting issues in tests

Signed-off-by: Dan Baron <dan.baron@starlingbank.com>

* fix: removing unnecessary Path object construction

Signed-off-by: Dan Baron <dan.baron@starlingbank.com>

---------

Signed-off-by: Dan Baron <dan.baron@starlingbank.com>

* feat: Updating docs to include model inference guidelines (feast-dev#4416)

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

* fix: Retire pytz library (feast-dev#4406)

* fix: Remove pytz.

Signed-off-by: Shuchu Han <shuchu.han@gmail.com>

* fix: Keep the pytz.UTC part in dask.py

Signed-off-by: Shuchu Han <shuchu.han@gmail.com>

---------

Signed-off-by: Shuchu Han <shuchu.han@gmail.com>

* Update model-inference.md

* chore: Auto-detect python version in Makefile (feast-dev#4419)

* fix: Default to pandas mode if not specified in ODFV proto in database (feast-dev#4420)

* chore: Update SUMMARY.md (feast-dev#4422)

Update SUMMARY.md

* docs: Updated README template and fixed links to be consistent for HTML (feast-dev#4423)

* chore: fixed README template to be consistent with current README

Signed-off-by: dandawg <12484302+dandawg@users.noreply.github.com>

* docs: markdown links consistency with html

Signed-off-by: dandawg <12484302+dandawg@users.noreply.github.com>

---------

Signed-off-by: dandawg <12484302+dandawg@users.noreply.github.com>

* fix: Add feast-operator Makefile to semantic-release script (feast-dev#4424)

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

* feat: Add health check service to registry server (feast-dev#4421)

Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Co-authored-by: Bhargav Dodla <bdodla@expediagroup.com>

* feat: Feast Security Model (aka RBAC) (feast-dev#4380)

* initial commit

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed linting issues (but 1)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* deleted AuthzedResource and moved types to the Permission class

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* using pytest.mark.parametrize tests

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* moved decorator to decorator module

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* parametrized decision tests

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added matcher and action modules. Added global assert_permissions function

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed linting error

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Managing with_subclasses flag and overriding it in case it's an abstract class like DataSource

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Permission includes a single Policy

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* completed docstrings for permissions package

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed inter issues

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Changed roles matching rule from "all" to "any"

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* removed test code

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* hiding sensitive data (false positive, anyway)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added filter_only flag to assert_permissions and returning a list of filtered resources instead of PermissionError

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added the option to return the single resource, or None

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* separate validating functions: assert_permission and filtered_resources

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Applied review comments

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Fixes to code
- Made test case broader

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Fixed incorrectly recognized linter error

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Added test
- Fixed missing property to permission
- Changed code following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Fixes to code
- Made test case broader

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Fixed incorrectly recognized linter error

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Added test
- Fixed delete and apply permission

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* replaced aggregated actions with aliases for QUERY and WRITE and ALL

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Updated user guide

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Updated enum in proto

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Fixed test errors following refactor
- Added test

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Store and Manage permissions in the Registry
- Removed redundant property
- Added tags filter option to list_permissions

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added permission assert check for registry server, offline server, online server functions

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix linter after rebase

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* CLI command "feast permissions list"
Added cli command permissions
Added tags parameter to list_validation_references and list_saved_datasets in registry
Added list_validation_references and list_saved_datasets apis to feature_store
Added missing tags parameters to registry_server methods

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* CLI command "feast permissions list"
- Changes following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* CLI command "feast permissions list"
- Changes following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added the documents reference for permissions for online, offline, registry server endpoints.

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating code review comments to parse the auth block from the f… (feast-dev#36)

* Incorporating code review comments to parse the auth block from the feature_store.yaml file.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Incorporating code review comments - renaming type from k8 to kubernetes.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

---------

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* definition and integration of auth manager in feast offline and online servers

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* typo

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* duplicated if

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* renamed functions with long name

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* using User class instead of RoleManager (completely removed)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Feed SecurityManager with Registry instance to fetch the actual permissions

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed linter

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* review comments

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed broken IT

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding registry server (UT to be completed)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix linter

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* passing auth manager type from config

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* used auth config to set auth manager type

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* inject the user details

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* created decorator function and applied to arrow function for injecting the user detail:wq

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* code review fixes including the unit test and integration test as suggested

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Implementation of oidc client authentication. (feast-dev#40)

* Adding initial draft code to manage the oidc client authentication.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Adding initial draft code to manage the oidc client authentication.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Incorporating code review comments.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

---------

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Added authentication header for client grpc calls

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added auth configuration for arrow flight client

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Made changes following code review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix linter

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Propagating auth config to token parser in server init

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* adding headers and client_secret to token request

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* working E2E test of authenticated registy server

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* renamed test

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fixed broken test

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix rebase issues

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix rebase issues

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding the auth client documentations and unit testing for auth client code.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding the auth client documentations and unit testing for auth client code.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating code review comments.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating code review comments.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* CLI command "feast permissions list"
- Added missing dependency

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Client module-grpc
- Added missing auth header for calls to remote registry

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix auth tests with permissions
- Made changes to enforcer ana security manager permission checking logic

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix auth tests with permissions
- Made changes following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix auth tests with permissions
- Made changes following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix auth tests with permissions
- Made changes following review

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py  (feast-dev#54)

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Adding missed dependency and regenerated the requirements files.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Addinig missing changes from the original PR.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

---------

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added check and list-roles subcommands

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* typo

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added comment in cli_utils to remind the original function from which this logic was derived

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* 1) Updating the existing integration test with auth permissions configurations.
2) Refactored the common code and moved to the util class and common conftest.py file.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding missed dependency and regenerated the requirements files.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* 1) Updating the existing integration test with auth permissions configurations.
2) Refactored the common code and moved to the util class and common conftest.py file.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* 1) Fixing an issue with the way getting markers after changing the fixture scope to module. Now looking up the markers coming from the entire module run.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fixed bug in GetPermission API

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Permission CRUD test

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added feast-rbac example

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added support to read the token from enviroment variable to run from local

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fix the header for arrow fligth

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix the header issue

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added permissions apply file

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* set the user in the grpc server

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* added roles and updated permission with all roles

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* updated chart to include the service account

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* created client example with roles and updated installation/cleanup script

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* rebased with master

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py  (feast-dev#54)

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Adding missed dependency and regenerated the requirements files.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Addinig missing changes from the original PR.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

---------

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fixed DecisionStrategy not persisted

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fixed DecisionStrategy not persisted

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fixed DecisionStrategy not persisted
- Implemented review comments

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Revert "Fix decision strategy not saved"

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Dropped global decision strategy

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* updated rbac demo example

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding permissions directly instead of from the common place for the online read integration tests.
Cleaned up some minor changes to fix the unpredictable issue with the feature server process.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Initial Draft version to the tests with remote offline server with OIDC authentication permissions. Happy path only.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Abstracting the specific code for Offline Permissions by creating new class for PermissionsEnvironment.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Formatting the python files using make format-python.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Separated the permissions for online, offline and registry servers. moved the fixtures scope accordingly as we can't reuse the permissions for all the test cases.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Separated the permissions for online, offline and registry servers. moved the fixtures scope accordingly as we can't reuse the permissions for all the test cases.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Created the grpc client auth header interceptor and removed the manual injection of the header.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Created the grpc client auth header interceptor and removed the manual injection of the header.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix: java to proto failing
- changed java_outer_classname for Permission.proto and Policy.proto
- removed experimental optional from permission proto

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* CLI command "feast permissions list"
Added cli command permissions
Added tags parameter to list_validation_references and list_saved_datasets in registry
Added list_validation_references and list_saved_datasets apis to feature_store
Added missing tags parameters to registry_server methods

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py  (feast-dev#54)

* Moved the common fixtures to the root conftest.py or auth_permissions_util.py

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Adding missed dependency and regenerated the requirements files.
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

* Addinig missing changes from the original PR.

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>

---------

Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix: java to proto failing
- changed java_outer_classname for Permission.proto and Policy.proto
- removed experimental optional from permission proto

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding the extra writer permission to fix the integration test issue with offline server.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Try to fix java integration test - ModuleNotFoundError: No module named 'feast.permissions.server'

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix java integration test - ModuleNotFoundError: No module named 'jwt'

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix java integration test - ModuleNotFoundError: No module named 'kubernetes'

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Adding missing permissions for offline store test cases - classes FileSource, FeatureService classes. (feast-dev#64)

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Updating the offline integration test permissions.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* updated test.py file for rbac-example

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix the DeleteFeatureView function to handle stream feature view type

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Updating permissions of the integration test cases to address code review comments and also check if the online_read integration test fixes.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating the code review comments from Francisco on upstream PR.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Update docs/getting-started/concepts/permission.md

Co-authored-by: Francisco Arceo <farceo@redhat.com>
Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Update docs/getting-started/concepts/permission.md

Co-authored-by: Francisco Arceo <farceo@redhat.com>
Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Update docs/getting-started/concepts/permission.md

Co-authored-by: Francisco Arceo <farceo@redhat.com>
Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Small fixes (feast-dev#71)

* Improved permission denial log

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>

* Added leeway option to accept tokens released in the past (up to 10")

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>

---------

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* commented/removed oidc tests to verify  integration test
commented/removed test_auth_permission.py file

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Enabling the keycloak related integration tests and also initializing the keycloak only once in the entire run.
Reduced the number of works and increased the duration as well.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Making number of workers back to 8 and enabled the test_remote_online_store_read

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Making number of workers to 4.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating the code review comments from Tornike to use @pytest.mark.xdist_group(name="keycloak").

Reverting number of markers from 4 to 8 for the make file target test-python-integration-local.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Reverting number of workers from 8 to 4.
Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Reverting number of workers from 8 to 4. Reverting the marker @pytest.mark.xdist_group(name="keycloak")
Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Reverting number of workers from 8 to 4 for make target test-python-integration-local

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Added the arrow flight interceptor to inject the auth header. (feast-dev#68)

* * Added the arrow flight interceptor to inject the auth header.
* Injecting grpc interceptor if it is needed when auth type is not NO_AUTH.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* Fixing the failing integration test cases by setting the header in binary format.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* Refactored method and moved to factory class to incorporate code review comment.
Fixed lint error by removing the type of port. and other minor changes.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* Incorproating code review comments from Daniel.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

---------

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* removed with_subclasses option (it's the default and unique behavior)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* a full, minimal, reproducible example of the RBAC feature

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Add missing required_tags to permission object and cli info
- Add missing required_tags to permission object
- added required_tags to cli info

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Fixed the registry apply function assertation

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* removed the examples

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Integrated comment

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>

* removed the firebase depdency and fix the doc conflicts

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Introducing permission framework and authorization manager in user guide (to be continued after the code is consolidated)

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Permission resources miss the created_timestamp and last_updated_timestamp fields

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* remove error incase if user has no roles assinged incase unthorized user

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* renamed READ action to DESCRIBE

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>

* Specified authorization manager and authorization configuration

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>

* fix the linter and remove subclass from doc

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* addressed the pr reivew comments

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Incorporating code review comment and this file is not needed.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* Addressed the review comments on the PR

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* Reducing the markers from 8 to 4 to see if it fixes the issues with memory.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* addresses feedback on rbac doc

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* rename action name from QUERY to READ

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix the doc to replace query with read

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

---------

Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>
Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Co-authored-by: Theodor Mihalache <tmihalac@redhat.com>
Co-authored-by: Abdul Hameed <ahameed@redhat.com>
Co-authored-by: lokeshrangineni <lokeshforjava@gmail.com>
Co-authored-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Co-authored-by: Francisco Arceo <farceo@redhat.com>

* chore: Fix rbac url.

* fix: Links to the RBAC documentation under Concepts and Components (feast-dev#4430)

* fix the rbac docs links

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* fix: links to the RBAC documentation under Concepts and Components sections

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

---------

Signed-off-by: Abdul Hameed <ahameed@redhat.com>

* docs: Reorganize registry docs (feast-dev#4407)

* reorganize registry docs

Signed-off-by: tokoko <togurgenidze@gmail.com>

* remove commented out text

Signed-off-by: tokoko <togurgenidze@gmail.com>

* changes in registry.md

Signed-off-by: tokoko <togurgenidze@gmail.com>

---------

Signed-off-by: tokoko <togurgenidze@gmail.com>
Co-authored-by: tokoko <togurgenidze@gmail.com>

* chore: Update Slack link

* build: Set a proper build-system protobuf version (feast-dev#4438)

build: force the protobuf version in the build system so that it is compatible with the runtime dependency

Signed-off-by: Yang, Bo <bo.yang@protonbase.io>

* Update README.md

* fix: Typos related to k8s (feast-dev#4442)

fix typos

Signed-off-by: Brijesh Vora <brijesh.vora@sailpoint.com>

* feat: Refactoring code to get oidc end points from discovery URL. (feast-dev#4429)

* refactoring the permissions side server side code to get the OIDC end points from the discovery URL. Also removing the auth_server_url config from oidc auth config.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* refactoring the permissions side server side code to get the OIDC end points from the discovery URL. Also removing the auth_server_url config from oidc auth config.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* refactoring the permissions side server side code to get the OIDC end points from the discovery URL. Also removing the auth_server_url config from oidc auth config.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* refactoring the permissions side server side code to get the OIDC end points from the discovery URL. Also removing the auth_server_url config from oidc auth config.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* Fixing the issue with pre-commit hook template. Accidentally this was reverted in previous rebase and reverting it now.

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

---------

Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>

* chore: Mark tests using keycloak with xdist_group (feast-dev#4436)

* mark keycloak tests with xdist_group

Signed-off-by: tokoko <togurgenidze@gmail.com>

* apply changes to test-python-integration

Signed-off-by: tokoko <togurgenidze@gmail.com>

---------

Signed-off-by: tokoko <togurgenidze@gmail.com>
Co-authored-by: tokoko <togurgenidze@gmail.com>

* fix: Locate feature_store.yaml from __file__ (feast-dev#4443)

fix: locate feature_store.yaml from __file__

Signed-off-by: Yang, Bo <bo.yang@protonbase.io>

* feat: Update roadmap.md (feast-dev#4445)

* chore: Remove Rockset from feast (feast-dev#4434)

---------

Signed-off-by: Dan Baron <dan.baron@starlingbank.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Shuchu Han <shuchu.han@gmail.com>
Signed-off-by: dandawg <12484302+dandawg@users.noreply.github.com>
Signed-off-by: Tommy Hughes <tohughes@redhat.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Signed-off-by: Abdul Hameed <ahameed@redhat.com>
Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
Signed-off-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Signed-off-by: tokoko <togurgenidze@gmail.com>
Signed-off-by: Yang, Bo <bo.yang@protonbase.io>
Signed-off-by: Brijesh Vora <brijesh.vora@sailpoint.com>
Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>
Co-authored-by: Dan Baron <84331438+danbaron63@users.noreply.github.com>
Co-authored-by: Shuchu Han <shuchu.han@gmail.com>
Co-authored-by: Francisco Arceo <farceo@redhat.com>
Co-authored-by: Tornike Gurgenidze <togurg14@freeuni.edu.ge>
Co-authored-by: bdodla@expedia.com <13788369+EXPEbdodla@users.noreply.github.com>
Co-authored-by: Daniel Dowler <12484302+dandawg@users.noreply.github.com>
Co-authored-by: Tommy Hughes IV <tohughes@redhat.com>
Co-authored-by: Bhargav Dodla <bdodla@expediagroup.com>
Co-authored-by: Daniele Martinoli <86618610+dmartinol@users.noreply.github.com>
Co-authored-by: Theodor Mihalache <tmihalac@redhat.com>
Co-authored-by: Abdul Hameed <ahameed@redhat.com>
Co-authored-by: lokeshrangineni <lokeshforjava@gmail.com>
Co-authored-by: Lokesh Rangineni <19699092+lokeshrangineni@users.noreply.github.com>
Co-authored-by: tokoko <togurgenidze@gmail.com>
Co-authored-by: Yang, Bo <pop.atry@gmail.com>
Co-authored-by: Yang, Bo <bo.yang@protonbase.io>
  • Loading branch information
18 people authored Aug 27, 2024
1 parent e523c87 commit 44baad9
Show file tree
Hide file tree
Showing 186 changed files with 7,913 additions and 2,055 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!-- Thanks for sending a pull request! Here are some tips for you:
1. Ensure that your code follows our code conventions: https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md#code-style--linting
1. Ensure that your code follows our code conventions: https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md#code-style-and-linting
2. Run unit tests and ensure that they are passing: https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md#unit-tests
3. If your change introduces any API changes, make sure to update the integration tests here: https://github.com/feast-dev/feast/tree/master/sdk/python/tests
4. Make sure documentation is updated for your PR!
Expand Down
2 changes: 1 addition & 1 deletion .releaserc.js
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ module.exports = {
"CHANGELOG.md",
"java/pom.xml",
"infra/charts/**/*.*",
"infra/feast-operator/**/*.*",
"infra/feast-operator/**/*",
"ui/package.json",
"sdk/python/feast/ui/package.json",
"sdk/python/feast/ui/yarn.lock"
Expand Down
17 changes: 9 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ ifeq ($(shell uname -s), Darwin)
OS = osx
endif
TRINO_VERSION ?= 376
PYTHON_VERSION = ${shell python --version | grep -Eo '[0-9]\.[0-9]+'}

# General

Expand All @@ -37,22 +38,22 @@ build: protos build-java build-docker
# Python SDK

install-python-ci-dependencies:
python -m piptools sync sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
python -m piptools sync sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
pip install --no-deps -e .
python setup.py build_python_protos --inplace

install-python-ci-dependencies-uv:
uv pip sync --system sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip sync --system sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
uv pip install --system --no-deps -e .
python setup.py build_python_protos --inplace

install-python-ci-dependencies-uv-venv:
uv pip sync sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip sync sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
uv pip install --no-deps -e .
python setup.py build_python_protos --inplace

lock-python-ci-dependencies:
uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt

package-protos:
cp -r ${ROOT_DIR}/protos ${ROOT_DIR}/sdk/python/feast/protos
Expand All @@ -61,11 +62,11 @@ compile-protos-python:
python setup.py build_python_protos --inplace

install-python:
python -m piptools sync sdk/python/requirements/py$(PYTHON)-requirements.txt
python -m piptools sync sdk/python/requirements/py$(PYTHON_VERSION)-requirements.txt
python setup.py develop

lock-python-dependencies:
uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py$(PYTHON)-requirements.txt
uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py$(PYTHON_VERSION)-requirements.txt

lock-python-dependencies-all:
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py3.9-requirements.txt"
Expand All @@ -85,14 +86,14 @@ test-python-unit:
python -m pytest -n 8 --color=yes sdk/python/tests

test-python-integration:
python -m pytest -n 8 --integration --color=yes --durations=10 --timeout=1200 --timeout_method=thread \
python -m pytest -n 8 --integration --color=yes --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \
-k "(not snowflake or not test_historical_features_main)" \
sdk/python/tests

test-python-integration-local:
FEAST_IS_LOCAL_TEST=True \
FEAST_LOCAL_ONLINE_CONTAINER=True \
python -m pytest -n 8 --color=yes --integration --durations=5 --dist loadgroup \
python -m pytest -n 8 --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \
-k "not test_lambda_materialization and not test_snowflake_materialization" \
sdk/python/tests

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
[![GitHub Release](https://img.shields.io/github/v/release/feast-dev/feast.svg?style=flat&sort=semver&color=blue)](https://github.com/feast-dev/feast/releases)

## Join us on Slack!
👋👋👋 [Come say hi on Slack!](https://join.slack.com/t/feastopensource/signup)
👋👋👋 [Come say hi on Slack!](https://communityinviter.com/apps/feastopensource/feast-the-open-source-feature-store)

## Overview

Expand Down Expand Up @@ -230,4 +230,4 @@ Thanks goes to these incredible people:

<a href="https://github.com/feast-dev/feast/graphs/contributors">
<img src="https://contrib.rocks/image?repo=feast-dev/feast" />
</a>
</a>
15 changes: 15 additions & 0 deletions community/ADOPTERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Adopters of Feast

Below are the adopters of Feast. If you are using Feast please add
yourself into the following list by a pull request. Please keep the list in
alphabetical order.

| Organization | Contact | GitHub Username |
| ------------ | ------- | ------- |
| Affirm | Francisco Javier Arceo | franciscojavierarceo |
| Bank of Georgia | Tornike Gurgenidze | tokoko |
| Get Ground | Zhiling Chen | zhilingc |
| Gojek | Pradithya Aria Pura | pradithya |
| Twitter | David Liu | mavysavydav|
| Shopify | Matt Delacour | MattDelac |
| Snowflake | Miles Adkins | sfc-gh-madkins |
25 changes: 16 additions & 9 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,28 +8,31 @@
## Getting started

* [Quickstart](getting-started/quickstart.md)
* [Architecture](getting-started/architecture/README.md)
* [Overview](getting-started/architecture/overview.md)
* [Language](getting-started/architecture/language.md)
* [Push vs Pull Model](getting-started/architecture/push-vs-pull-model.md)
* [Write Patterns](getting-started/architecture/write-patterns.md)
* [Feature Transformation](getting-started/architecture/feature-transformation.md)
* [Feature Serving and Model Inference](getting-started/architecture/model-inference.md)
* [Role-Based Access Control (RBAC)](getting-started/architecture/rbac.md)
* [Concepts](getting-started/concepts/README.md)
* [Overview](getting-started/concepts/overview.md)
* [Data ingestion](getting-started/concepts/data-ingestion.md)
* [Entity](getting-started/concepts/entity.md)
* [Feature view](getting-started/concepts/feature-view.md)
* [Feature retrieval](getting-started/concepts/feature-retrieval.md)
* [Point-in-time joins](getting-started/concepts/point-in-time-joins.md)
* [Registry](getting-started/concepts/registry.md)
* [Permission](getting-started/concepts/permission.md)
* [\[Alpha\] Saved dataset](getting-started/concepts/dataset.md)
* [Architecture](getting-started/architecture/README.md)
* [Overview](getting-started/architecture/overview.md)
* [Language](getting-started/architecture/language.md)
* [Push vs Pull Model](getting-started/architecture/push-vs-pull-model.md)
* [Write Patterns](getting-started/architecture/write-patterns.md)
* [Feature Transformation](getting-started/architecture/feature-transformation.md)
* [Components](getting-started/components/README.md)
* [Overview](getting-started/components/overview.md)
* [Registry](getting-started/components/registry.md)
* [Offline store](getting-started/components/offline-store.md)
* [Online store](getting-started/components/online-store.md)
* [Batch Materialization Engine](getting-started/components/batch-materialization-engine.md)
* [Provider](getting-started/components/provider.md)
* [Authorization Manager](getting-started/components/authz_manager.md)
* [Third party integrations](getting-started/third-party-integrations.md)
* [FAQ](getting-started/faq.md)

Expand All @@ -41,7 +44,6 @@
* [Real-time credit scoring on AWS](tutorials/tutorials-overview/real-time-credit-scoring-on-aws.md)
* [Driver stats on Snowflake](tutorials/tutorials-overview/driver-stats-on-snowflake.md)
* [Validating historical features with Great Expectations](tutorials/validating-historical-features.md)
* [Using Scalable Registry](tutorials/using-scalable-registry.md)
* [Building streaming features](tutorials/building-streaming-features.md)

## How-to Guides
Expand Down Expand Up @@ -106,10 +108,15 @@
* [PostgreSQL (contrib)](reference/online-stores/postgres.md)
* [Cassandra + Astra DB (contrib)](reference/online-stores/cassandra.md)
* [MySQL (contrib)](reference/online-stores/mysql.md)
* [Rockset (contrib)](reference/online-stores/rockset.md)
* [Hazelcast (contrib)](reference/online-stores/hazelcast.md)
* [ScyllaDB (contrib)](reference/online-stores/scylladb.md)
* [SingleStore (contrib)](reference/online-stores/singlestore.md)
* [Registries](reference/registries/README.md)
* [Local](reference/registries/local.md)
* [S3](reference/registries/s3.md)
* [GCS](reference/registries/gcs.md)
* [SQL](reference/registries/sql.md)
* [Snowflake](reference/registries/snowflake.md)
* [Providers](reference/providers/README.md)
* [Local](reference/providers/local.md)
* [Google Cloud Platform](reference/providers/google-cloud-platform.md)
Expand Down
8 changes: 8 additions & 0 deletions docs/getting-started/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,11 @@
{% content-ref url="feature-transformation.md" %}
[feature-transformation.md](feature-transformation.md)
{% endcontent-ref %}

{% content-ref url="model-inference.md" %}
[model-inference.md](model-inference.md)
{% endcontent-ref %}

{% content-ref url="rbac.md" %}
[rbac.md](rbac.md)
{% endcontent-ref %}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
A *feature transformation* is a function that takes some set of input data and
returns some set of output data. Feature transformations can happen on either raw data or derived data.

## Feature Transformation Engines
Feature transformations can be executed by three types of "transformation engines":

1. The Feast Feature Server
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/architecture/language.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Python: The Language of Production Machine Learning

Use Python to serve your features online.
Use Python to serve your features.


## Why should you use Python to Serve features for Machine Learning?
Python has emerged as the primary language for machine learning, and this extends to feature serving and there are five main reasons Feast recommends using a microservice in Feast.
Python has emerged as the primary language for machine learning, and this extends to feature serving and there are five main reasons Feast recommends using a microservice written in Python.

## 1. Python is the language of Machine Learning

Expand Down
97 changes: 97 additions & 0 deletions docs/getting-started/architecture/model-inference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Feature Serving and Model Inference

Production machine learning systems can choose from four approaches to serving machine learning predictions (the output
of model inference):
1. Online model inference with online features
2. Offline mode inference without online features
3. Online model inference with online features and cached predictions
4. Online model inference without features

*Note: online features can be sourced from batch, streaming, or request data sources.*

These three approaches have different tradeoffs but, in general, have significant implementation differences.

## 1. Online Model Inference with Online Features
Online model inference with online features is a powerful approach to serving data-driven machine learning applications.
This requires a feature store to serve online features and a model server to serve model predictions (e.g., KServe).
This approach is particularly useful for applications where request-time data is required to run inference.
```python
features = store.get_online_features(
feature_refs=[
"user_data:click_through_rate",
"user_data:number_of_clicks",
"user_data:average_page_duration",
],
entity_rows=[{"user_id": 1}],
)
model_predictions = model_server.predict(features)
```

## 2. Offline Model Inference without Online Features
Typically, Machine Learning teams find serving precomputed model predictions to be the most straightforward to implement.
This approach simply treats the model predictions as a feature and serves them from the feature store using the standard
Feast sdk. These model predictions are typically generated through some batch process where the model scores are precomputed.
As a concrete example, the batch process can be as simple as a script that runs model inference locally for a set of users that
can output a CSV. This output file could be used for materialization so that the model could be served online as shown in the
code below.
```python
model_predictions = store.get_online_features(
feature_refs=[
"user_data:model_predictions",
],
entity_rows=[{"user_id": 1}],
)
```
Notice that the model server is not involved in this approach. Instead, the model predictions are precomputed and
materialized to the online store.

While this approach can lead to quick impact for different business use cases, it suffers from stale data as well
as only serving users/entities that were available at the time of the batch computation. In some cases, this tradeoff
may be tolerable.

## 3. Online Model Inference with Online Features and Cached Predictions
This approach is the most sophisticated where inference is optimized for low-latency by caching predictions and running
model inference when data producers write features to the online store. This approach is particularly useful for
applications where features are coming from multiple data sources, the model is computationally expensive to run, or
latency is a significant constraint.

```python
# Client Reads
features = store.get_online_features(
feature_refs=[
"user_data:click_through_rate",
"user_data:number_of_clicks",
"user_data:average_page_duration",
"user_data:model_predictions",
],
entity_rows=[{"user_id": 1}],
)
if features.to_dict().get('user_data:model_predictions') is None:
model_predictions = model_server.predict(features)
store.write_to_online_store(feature_view_name="user_data", df=pd.DataFrame(model_predictions))
```
Note that in this case a seperate call to `write_to_online_store` is required when the underlying data changes and
predictions change along with it.

```python
# Client Writes from the Data Producer
user_data = request.POST.get('user_data')
model_predictions = model_server.predict(user_data) # assume this includes `user_data` in the Data Frame
store.write_to_online_store(feature_view_name="user_data", df=pd.DataFrame(model_predictions))
```
While this requires additional writes for every data producer, this approach will result in the lowest latency for
model inference.

## 4. Online Model Inference without Features
This approach does not require Feast. The model server can directly serve predictions without any features. This
approach is common in Large Language Models (LLMs) and other models that do not require features to make predictions.

Note that generative models using Retrieval Augmented Generation (RAG) do require features where the
[document embeddings](../../reference/alpha-vector-database.md) are treated as features, which Feast supports
(this would fall under "Online Model Inference with Online Features").

### Client Orchestration
Implicit in the code examples above is a design choice about how clients orchestrate calls to get features and run model inference.
The examples had a Feast-centric pattern because they are inputs to the model, so the sequencing is fairly obvious.
An alternative approach can be Inference-centric where a client would call an inference endpoint and the inference
service would be responsible for orchestration.
11 changes: 8 additions & 3 deletions docs/getting-started/architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,16 @@ Feast's architecture is designed to be flexible and scalable. It is composed of
online store.
This allows Feast to serve features in real-time with low latency.

* Feast supports On Demand and Streaming Transformations for [feature computation](feature-transformation.md) and
will support Batch transformations in the future. For Streaming and Batch, Feast requires a separate Feature Transformation
Engine (in the batch case, this is typically your Offline Store). We are exploring adding a default streaming engine to Feast.
* Feast supports [feature transformation](feature-transformation.md) for On Demand and Streaming data sources and
will support Batch transformations in the future. For Streaming and Batch data sources, Feast requires a separate
[Feature Transformation Engine](feature-transformation.md#feature-transformation-engines) (in the batch case, this is
typically your Offline Store). We are exploring adding a default streaming engine to Feast.

* Domain expertise is recommended when integrating a data source with Feast understand the [tradeoffs from different
write patterns](write-patterns.md) to your application

* We recommend [using Python](language.md) for your Feature Store microservice. As mentioned in the document, precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable. Because of this we believe the pros of Python outweigh the costs, as reimplementing feature logic is undesirable. Java and Go Clients are also available for online feature retrieval.

* [Role-Based Access Control (RBAC)](rbac.md) is a security mechanism that restricts access to resources based on the roles of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups can access or modify specific resources, thereby maintaining data security and operational integrity.


Binary file added docs/getting-started/architecture/rbac.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 44baad9

Please sign in to comment.