Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Bumping fastapi + starlette #3938

Merged

Conversation

bushwhackr
Copy link
Contributor

@bushwhackr bushwhackr commented Feb 7, 2024

What this PR does / why we need it:
Bumping vuln fastapi version. https://nvd.nist.gov/vuln/detail/CVE-2024-24762

Changes

  • cicd: mypy now only checks sdk/python/feast directory
  • cicd: sdk/python/requirements/*.txt are all bumped wrt. the version constraint in setup.py
  • lint: many base class with abstract methods have be altered to throw NotImplementedError instead due to missing implementation in some subclass.

Which issue(s) this PR fixes:

Fixes #3931 #3930

Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.99.1 to 0.109.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](fastapi/fastapi@0.99.1...0.109.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@bushwhackr bushwhackr changed the title chore: bumping fastapi + starlette chore: Bumping fastapi + starlette Feb 7, 2024
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
@sudohainguyen
Copy link
Collaborator

Linting failed, could you check @bushwhackr ?

@bushwhackr
Copy link
Contributor Author

bushwhackr commented Feb 8, 2024

image

The issue is mypy version I think

@sudohainguyen
Copy link
Collaborator

Considering upgrading mypy too? 👀

@bushwhackr
Copy link
Contributor Author

I'm looking to pin the mypy version to 1.4.1 but like the #3942 there are quite a number of linting failures. Let me see if I can do one with minimum changes.

bushwhackr and others added 3 commits February 8, 2024 14:03
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
…tedError

Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
@bushwhackr
Copy link
Contributor Author

bushwhackr commented Feb 8, 2024

I'm of the opinion to skip tests mypy type errors till a later PR. Seems like alot of classes in sdk/python/tests/integration/feature_repos/universal/**/* have classes that do not confirm to the BaseClass abstract method definition.

For example:
baseclass:

@abstractmethod
def create_data_source(
self,
df: pd.DataFrame,
destination_name: str,
event_timestamp_column="ts",
created_timestamp_column="created_ts",
field_mapping: Dict[str, str] = None,
timestamp_field: Optional[str] = None,
) -> DataSource:

subclass:

def create_data_source(
self,
df: pd.DataFrame,
destination_name: str,
timestamp_field="ts",
created_timestamp_column="created_ts",
field_mapping: Dict[str, str] = None,
**kwargs,
) -> DataSource:

Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
@bushwhackr
Copy link
Contributor Author

@sudohainguyen could you rerun the tests. Linting should pass now

   ~/repo/feast    bushwhackr/fastapi-starlette ?1 ❯ make lint-python                                                                                                                                                                                                                                                                      30s   feast   00:14:46
cd /home/chester/repo/feast/sdk/python; python -m mypy --exclude=/tests/ --follow-imports=skip feast
feast/on_demand_feature_view.py:345: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/on_demand_feature_view.py:368: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/on_demand_feature_view.py:403: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/feature_store.py:1009: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/repo_operations.py:249: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/repo_operations.py:263: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/utils/snowflake/snowflake_utils.py:78: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/materialization/contrib/bytewax/bytewax_materialization_dataflow.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Success: no issues found in 293 source files
cd /home/chester/repo/feast/sdk/python; python -m isort feast/ tests/ --check-only
Skipped 1 files
cd /home/chester/repo/feast/sdk/python; python -m flake8 feast/ tests/
cd /home/chester/repo/feast/sdk/python; python -m black --check feast tests
All done! ✨ 🍰 ✨
365 files would be left unchanged.

@sudohainguyen
Copy link
Collaborator

great! re-running

@sudohainguyen
Copy link
Collaborator

oh no 😢 @bushwhackr fail again

Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
@bushwhackr
Copy link
Contributor Author

hopefully it works now, just tested against python 3.8.

image

@sudohainguyen
Copy link
Collaborator

great, now unit tests 😓

…concrete

Signed-off-by: Chester Ong <chester.ong.ch@gmail.com>
@bushwhackr
Copy link
Contributor Author

bushwhackr commented Feb 9, 2024

@sudohainguyen my bad, please try again. Hopefully I can get this in before lunar new year

@sudohainguyen
Copy link
Collaborator

Perfect! All checks passed

Copy link
Collaborator

@sudohainguyen sudohainguyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! all problem solved
lgtm
is there any issue in terms of performance we should know? @bushwhackr

@bushwhackr
Copy link
Contributor Author

great work! all problem solved lgtm is there any issue in terms of performance we should know? @bushwhackr

None at all, all changes are to fix linting issues.

cd ${ROOT_DIR}/sdk/python; python -m mypy
cd ${ROOT_DIR}/sdk/python; python -m mypy --exclude=/tests/ --follow-imports=skip feast
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would feel this is the biggest change, is that I only limit the linting checks to sdk/python/feast directory

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if we still check tests? a lot of changes needed right?

@bushwhackr
Copy link
Contributor Author

to revert to the old mypy settings:

feast/on_demand_feature_view.py:345: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/on_demand_feature_view.py:368: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/on_demand_feature_view.py:403: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/offline_stores/snowflake_source.py:222: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
feast/infra/offline_stores/snowflake_source.py:253: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
feast/feature_store.py:1009: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/repo_operations.py:249: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/repo_operations.py:263: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/utils/snowflake/snowflake_utils.py:78: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/offline_stores/snowflake.py:189: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
feast/infra/offline_stores/snowflake.py:240: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
feast/infra/offline_stores/snowflake.py:274: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
feast/infra/offline_stores/snowflake.py:395: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
tests/utils/feature_records.py:335: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/unit/test_registry_server.py:22: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/unit/test_registry_server.py:33: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/unit/test_registry_server.py:51: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/materialization/contrib/bytewax/bytewax_materialization_dataflow.py:56: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/feature_repos/universal/online_store/redis.py:16: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/mysql.py:21: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/hbase.py:16: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/hazelcast.py:32: error: Return type "Dict[str, Any]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/dynamodb.py:18: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/datastore.py:26: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/cassandra.py:35: error: Return type "Dict[str, object]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/online_store/bigtable.py:31: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/feature_repos/universal/data_sources/snowflake.py:54: error: Argument 6 of "create_data_source" is incompatible with supertype "DataSourceCreator"; supertype defines the argument type as "Optional[str]"  [override]
tests/integration/feature_repos/universal/data_sources/snowflake.py:54: note: This violates the Liskov substitution principle
tests/integration/feature_repos/universal/data_sources/snowflake.py:54: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
tests/integration/feature_repos/universal/data_sources/snowflake.py:59: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOfflineStoreConfig"; expected "str"  [arg-type]
tests/integration/feature_repos/universal/data_sources/redshift.py:54: error: Argument 6 of "create_data_source" is incompatible with supertype "DataSourceCreator"; supertype defines the argument type as "Optional[str]"  [override]
tests/integration/feature_repos/universal/data_sources/redshift.py:54: note: This violates the Liskov substitution principle
tests/integration/feature_repos/universal/data_sources/redshift.py:54: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
tests/integration/feature_repos/universal/data_sources/file.py:38: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
tests/integration/feature_repos/universal/data_sources/file.py:38: note:      Superclass:
tests/integration/feature_repos/universal/data_sources/file.py:38: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/file.py:38: note:      Subclass:
tests/integration/feature_repos/universal/data_sources/file.py:38: note:          def create_data_source(self, df: Any, destination_name: str, timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/file.py:93: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
tests/integration/feature_repos/universal/data_sources/file.py:93: note:      Superclass:
tests/integration/feature_repos/universal/data_sources/file.py:93: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/file.py:93: note:      Subclass:
tests/integration/feature_repos/universal/data_sources/file.py:93: note:          def create_data_source(self, df: Any, destination_name: str, timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/file.py:167: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
tests/integration/feature_repos/universal/data_sources/file.py:167: note:      Superclass:
tests/integration/feature_repos/universal/data_sources/file.py:167: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/file.py:167: note:      Subclass:
tests/integration/feature_repos/universal/data_sources/file.py:167: note:          def create_data_source(self, df: Any, destination_name: Optional[str] = ..., suffix: Optional[str] = ..., timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/bigquery.py:63: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
tests/integration/feature_repos/universal/data_sources/bigquery.py:63: note:      Superclass:
tests/integration/feature_repos/universal/data_sources/bigquery.py:63: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
tests/integration/feature_repos/universal/data_sources/bigquery.py:63: note:      Subclass:
tests/integration/feature_repos/universal/data_sources/bigquery.py:63: note:          def create_data_source(self, df: Any, destination_name: str, timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., **kwargs: Any) -> DataSource
feast/infra/online_stores/snowflake.py:117: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOnlineStoreConfig"; expected "str"  [arg-type]
feast/infra/online_stores/snowflake.py:181: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOnlineStoreConfig"; expected "str"  [arg-type]
feast/infra/online_stores/snowflake.py:223: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOnlineStoreConfig"; expected "str"  [arg-type]
feast/infra/online_stores/snowflake.py:251: error: Argument 1 to "GetSnowflakeConnection" has incompatible type "SnowflakeOnlineStoreConfig"; expected "str"  [arg-type]
feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py:67: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py:67: note:      Superclass:
feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py:67: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py:67: note:      Subclass:
feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py:67: note:          def create_data_source(self, df: Any, destination_name: str, timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., **kwargs: Any) -> DataSource
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:58: error: Return type "MsSqlServerOfflineStoreConfig" of "create_offline_store_config" incompatible with return type "FeastConfigBaseModel" in supertype "DataSourceCreator"  [override]
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:63: error: Signature of "create_data_source" incompatible with supertype "DataSourceCreator"  [override]
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:63: note:      Superclass:
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:63: note:          def create_data_source(self, df: Any, destination_name: str, event_timestamp_column: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., timestamp_field: Optional[str] = ...) -> DataSource
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:63: note:      Subclass:
feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py:63: note:          def create_data_source(self, df: Any, destination_name: str, timestamp_field: Any = ..., created_timestamp_column: Any = ..., field_mapping: Optional[Dict[str, str]] = ..., **kwargs: Any) -> DataSource
feast/infra/offline_stores/contrib/athena_offline_store/tests/data_source.py:54: error: Argument 6 of "create_data_source" is incompatible with supertype "DataSourceCreator"; supertype defines the argument type as "Optional[str]"  [override]
feast/infra/offline_stores/contrib/athena_offline_store/tests/data_source.py:54: note: This violates the Liskov substitution principle
feast/infra/offline_stores/contrib/athena_offline_store/tests/data_source.py:54: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
feast/infra/contrib/spark_kafka_processor.py:71: error: Return type "StreamingQuery" of "ingest_stream_feature_view" incompatible with return type "None" in supertype "StreamProcessor"  [override]
feast/infra/contrib/spark_kafka_processor.py:132: error: Return type "StreamingQuery" of "_write_stream_data" incompatible with return type "None" in supertype "StreamProcessor"  [override]
tests/integration/registration/test_universal_types.py:337: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/registration/test_universal_types.py:356: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/feature_repos/repo_configuration.py:103: error: Dict entry 0 has incompatible type "str": "Tuple[str, Type[FileDataSourceCreator]]"; expected "str": "DataSourceCreator"  [dict-item]
tests/integration/feature_repos/repo_configuration.py:104: error: Dict entry 1 has incompatible type "str": "Tuple[str, Type[BigQueryDataSourceCreator]]"; expected "str": "DataSourceCreator"  [dict-item]
tests/integration/feature_repos/repo_configuration.py:105: error: Dict entry 2 has incompatible type "str": "Tuple[str, Type[RedshiftDataSourceCreator]]"; expected "str": "DataSourceCreator"  [dict-item]
tests/integration/feature_repos/repo_configuration.py:106: error: Dict entry 3 has incompatible type "str": "Tuple[str, Type[SnowflakeDataSourceCreator]]"; expected "str": "DataSourceCreator"  [dict-item]
tests/integration/feature_repos/repo_configuration.py:172: error: Value expression in dictionary comprehension has incompatible type "Tuple[Union[str, Dict[Any, Any], None], Optional[Type[OnlineStoreCreator]]]"; expected type "Tuple[Union[str, Dict[str, str]], Optional[Type[OnlineStoreCreator]]]"  [misc]
tests/integration/feature_repos/repo_configuration.py:353: error: Argument "driver_odfv" to "UniversalFeatureViews" has incompatible type "Optional[OnDemandFeatureView]"; expected "OnDemandFeatureView"  [arg-type]
tests/integration/feature_repos/repo_configuration.py:354: error: Argument 1 to "conv_rate_plus_100_feature_view" has incompatible type "List[Any]"; expected "Dict[str, Union[RequestSource, FeatureView]]"  [arg-type]
tests/integration/feature_repos/repo_configuration.py:378: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/feature_repos/repo_configuration.py:415: error: Incompatible types in assignment (expression has type "FeastConfigBaseModel", variable has type "Union[str, Dict[Any, Any], None]")  [assignment]
tests/integration/feature_repos/repo_configuration.py:418: error: Incompatible types in assignment (expression has type "Union[str, Dict[Any, Any], None]", variable has type "FeastConfigBaseModel")  [assignment]
tests/integration/feature_repos/repo_configuration.py:425: error: Missing named argument "feature_logging" for "AwsLambdaFeatureServerConfig"  [call-arg]
tests/integration/feature_repos/repo_configuration.py:434: error: Incompatible types in assignment (expression has type "LocalFeatureServerConfig", variable has type "AwsLambdaFeatureServerConfig")  [assignment]
tests/integration/feature_repos/repo_configuration.py:449: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/integration/feature_repos/repo_configuration.py:449: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/conftest.py:222: error: Incompatible types in assignment (expression has type "List[Tuple[str, Type[DataSourceCreator]]]", variable has type "Optional[List[Optional[DataSourceCreator]]]")  [assignment]
tests/conftest.py:225: error: List item 0 has incompatible type "Tuple[str, Type[FileDataSourceCreator]]"; expected "Optional[DataSourceCreator]"  [list-item]
tests/conftest.py:237: error: Incompatible types in assignment (expression has type "dict_values[str, Tuple[Union[str, Dict[str, str]], Optional[Type[OnlineStoreCreator]]]]", variable has type "Optional[List[Optional[Tuple[Union[str, Dict[str, str]], Optional[Type[OnlineStoreCreator]]]]]]")  [assignment]
tests/conftest.py:262: error: "DataSourceCreator" object is not iterable  [misc]
tests/conftest.py:262: error: "None" object is not iterable  [misc]
tests/conftest.py:263: error: "None" object is not iterable  [misc]
tests/conftest.py:292: error: List item 0 has incompatible type "Tuple[str, Type[FileDataSourceCreator]]"; expected "Optional[DataSourceCreator]"  [list-item]
tests/unit/online_store/test_online_retrieval.py:139: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/unit/online_store/test_online_retrieval.py:139: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/unit/online_store/test_online_retrieval.py:202: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/unit/online_store/test_online_retrieval.py:202: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/unit/infra/test_local_registry.py:41: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/unit/infra/test_local_registry.py:41: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/unit/cli/test_cli_chdir.py:21: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:24: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:28: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:37: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:47: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:52: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/unit/cli/test_cli_chdir.py:55: error: List item 1 has incompatible type "Path"; expected "str"  [list-item]
tests/integration/registration/test_registry.py:45: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/integration/registration/test_registry.py:45: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/integration/registration/test_registry.py:56: error: Missing named argument "registry_store_type" for "RegistryConfig"  [call-arg]
tests/integration/registration/test_registry.py:56: error: Missing named argument "s3_additional_kwargs" for "RegistryConfig"  [call-arg]
tests/integration/offline_store/test_universal_historical_retrieval.py:170: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/offline_store/test_s3_custom_endpoint.py:43: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
tests/integration/offline_store/test_offline_write.py:134: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
feast/infra/offline_stores/contrib/trino_offline_store/tests/data_source.py:87: error: Argument 6 of "create_data_source" is incompatible with supertype "DataSourceCreator"; supertype defines the argument type as "Optional[str]"  [override]
feast/infra/offline_stores/contrib/trino_offline_store/tests/data_source.py:87: note: This violates the Liskov substitution principle
feast/infra/offline_stores/contrib/trino_offline_store/tests/data_source.py:87: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
feast/infra/offline_stores/contrib/trino_offline_store/tests/data_source.py:125: error: Missing named argument "user" for "TrinoOfflineStoreConfig"  [call-arg]
feast/infra/offline_stores/contrib/trino_offline_store/tests/data_source.py:125: error: Missing named argument "auth" for "TrinoOfflineStoreConfig"  [call-arg]
feast/infra/offline_stores/contrib/postgres_offline_store/tests/data_source.py:88: error: Argument 6 of "create_data_source" is incompatible with supertype "DataSourceCreator"; supertype defines the argument type as "Optional[str]"  [override]
feast/infra/offline_stores/contrib/postgres_offline_store/tests/data_source.py:88: note: This violates the Liskov substitution principle
feast/infra/offline_stores/contrib/postgres_offline_store/tests/data_source.py:88: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
feast/infra/offline_stores/contrib/postgres_offline_store/tests/data_source.py:109: error: Return type "Dict[str, str]" of "create_online_store" incompatible with return type "FeastConfigBaseModel" in supertype "OnlineStoreCreator"  [override]
tests/integration/e2e/test_python_feature_server.py:131: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Found 73 errors in 27 files (checked 427 source files)

@sudohainguyen
Copy link
Collaborator

alright, let's get rid of it in another PR

@@ -418,7 +418,7 @@ def _delete_object(
"""
cursor = execute_snowflake_statement(conn, query)

if cursor.rowcount < 1 and not_found_exception:
if cursor.rowcount < 1 and not_found_exception: # type: ignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this and can pass the mypy check:
if cursor.rowcount and (cursor.rowcount < 1) and not_found_exception:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, this is due to another change I did to the mypy config: --follow-imports=skip.

run the below command to see the error:
python -m mypy --exclude=/tests/ sdk/python/feast/infra/registry/snowflake.py you will see an error.

`sdk/python/feast/infra/registry/snowflake.py:421: error: Unsupported operand types for > ("int" and "None")  [operator]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

essentially cursor.rowcount could return none from snowflake's signature

    @property
    def rowcount(self) -> int | None:
        return self._total_rowcount if self._total_rowcount >= 0 else None

@bushwhackr
Copy link
Contributor Author

any blockers?

@sudohainguyen
Copy link
Collaborator

Need your merge actions here 🙏 @franciscojavierarceo

@franciscojavierarceo franciscojavierarceo merged commit 4e450ad into feast-dev:master Feb 10, 2024
18 checks passed
@shuchu
Copy link
Collaborator

shuchu commented Feb 11, 2024

BaseClass

I will fix (have fixed) this in my PR. #3942

@sudohainguyen sudohainguyen added the dependencies Pull requests that update a dependency file label Feb 11, 2024
@bushwhackr bushwhackr deleted the bushwhackr/fastapi-starlette branch February 11, 2024 16:46
tqtensor pushed a commit to tqtensor/feast that referenced this pull request Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved dependencies Pull requests that update a dependency file ok-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants