refactor(all): unify method signatures across backends #9383
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
ACTION NEEDED Ibis follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
gforsyth
commented
Jun 13, 2024
Comment on lines
18
to
26
| @pytest.mark.parametrize("base_cls, method", params) | ||
| def test_signatures(base_cls, method, backend_cls): | ||
| if not hasattr(backend_cls, method): | ||
| pytest.skip(f"Method {method} not present in {backend_cls}, skipping...") | ||
|
|
||
| base_sig = inspect.signature(getattr(base_cls, method)) | ||
| backend_sig = inspect.signature(getattr(backend_cls, method)) | ||
|
|
||
| assert compatible(base_sig, backend_sig, check_annotations=False) |
There was a problem hiding this comment.
I imagine we could break up the base classes into groups where we think type annotations should match vs. those where we expect some variation.
|
xref #8994 |
gforsyth
changed the title
gforsyth
force-pushed
the
sigcheck
branch
2 times, most recently
from
44a4bb4
to
a9cb5be
Compare
|
xref: #9125 |
13 tasks
This is a fork of some of the utilities in Scott Sanderson's `python-interface` project (https://github.com/ssanderson/python-interface). While the upstream is no longer maintained, the goal of that project aligns quite well with some of the issues we face with maintaining consistent interfaces across backends. I'm grabbing a few of these utilities and ripping out Python 2 support, then I'm going to work on setting up tests for making sure that backend methods match the signatures in the parent `BaseBackend` class.
Should be `implementation, interface_def`, not the other way around
BREAKING CHANGE: The first argument to `read_csv` is renamed to `path` and is now a positional-only argument. All other arguments to `read_csv` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `read_parquet` is renamed to `path` and is now a positional-only argument. All other arguments to `read_parquet` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `read_json` is renamed to `path` and is now a positional-only argument. All other arguments to `read_json` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `read_delta` is renamed to `path` and is now a positional-only argument. All other arguments to `read_delta` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `create_catalog` is now a positional-only argument. All other arguments to `create_catalog` are now keyword-only arguments. BREAKING CHANGE: The first argument to `drop_catalog` is now a positional-only argument. All other arguments to `drop_catalog` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `create_database` is now a positional-only argument. All other arguments to `create_database` are now keyword-only arguments. BREAKING CHANGE: The first argument to `drop_database` is now a positional-only argument. All other arguments to `drop_database` are now keyword-only arguments. BREAKING CHANGE: All arguments to `list_databases` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `table` is now a positional-only argument. All other arguments to `table` are now keyword-only arguments.
BREAKING CHANGE: all arguments to `list_tables` are now keyword-only arguments. BREAKING CHANGE: The first argument to `create_table` is now a positional-only argument. BREAKING CHANGE: The first argument to `drop_table` is now a positional-only argument. BREAKING CHANGE: The first argument to `truncate_table` is now a positional-only argument. All other arguments are now keyword-only arguments.
BREAKING CHANGE: The first argument to `compile` is now a positional-only argument. All other arguments to `compile` are now keyword-only arguments.
BREAKING CHANGE: The first argument to `execute` is now a positional-only argument. All other arguments to `execute` are now keyword-only arguments.
gforsyth
changed the title
BREAKING CHANGE: The first argument to `insert` is now a positional-only argument. The second argument, `obj`, is keyword or positional. All other arguments to `insert` are now keyword-only arguments.
We can revisit this, but for now, these can be different enough that I'm ok not enforcing strict compatibility.
BREAKING CHANGES: The first argument to `create_view` is now a positional-only argument. The second argument, `obj`, is keyword or positional. All other arguments to `create_view` are now keyword-only arguments. The first argument to `drop_view` is now a postional-only argument. All other arguments to `drop_view` are now keyword-only arguments.
ncclementi
reviewed
Aug 20, 2024
|
Closing in favor of #10008 |
gforsyth
added a commit
that referenced
this pull request
Sep 13, 2024
Opening this in favor of #9383 -- that PR also included all of the breaking changes to unify function signatures and it was too much at once. This PR adds only the signature checking mechanism, plus the requisite xfails to lay out which inconsistencies are currently in Ibis. ## Motivation We want to ensure that, for a given backend, that the argument names, plus usage of positional, positional-only, keyword, and keyword-only arguments match, so that there is API consistency when moving between backends. I've grabbed a few small parts of some of the utilities in Scott Sanderson's `python-interface` project (https://github.com/ssanderson/python-interface). While the upstream is no longer maintained, the goal of that project aligns quite well with some of the issues we face with maintaining consistent interfaces across backends. Note that while the upstream project focused on _runtime_ enforcement of these signatures matching, here it is only run in the test suite. ## Rough procedure Any method that doesn't match can be skipped entirely (this is useful for things like `do_connect`, which cannot reasonably be assumed to match) or individually (by specifying a `pytest.param` and marking the failing backends). Then we scrape across the common parent classes and add any methods that are NOT currently specified in the pre-existing xfailed ones. It's a bit of a nuisance, but it's done, and ideally the manual listing of the inconsistent methods goes away as we unify things. I've opted for not checking that type annotations match, because that seems... unreasonable. This would satisfy #9125 once all of the xfail markers are removed, e.g., it checks that all keyword and positional arguments are standardized.
ncclementi
pushed a commit
to ncclementi/ibis
that referenced
this pull request
Sep 24, 2024
…-project#10008) Opening this in favor of ibis-project#9383 -- that PR also included all of the breaking changes to unify function signatures and it was too much at once. This PR adds only the signature checking mechanism, plus the requisite xfails to lay out which inconsistencies are currently in Ibis. ## Motivation We want to ensure that, for a given backend, that the argument names, plus usage of positional, positional-only, keyword, and keyword-only arguments match, so that there is API consistency when moving between backends. I've grabbed a few small parts of some of the utilities in Scott Sanderson's `python-interface` project (https://github.com/ssanderson/python-interface). While the upstream is no longer maintained, the goal of that project aligns quite well with some of the issues we face with maintaining consistent interfaces across backends. Note that while the upstream project focused on _runtime_ enforcement of these signatures matching, here it is only run in the test suite. ## Rough procedure Any method that doesn't match can be skipped entirely (this is useful for things like `do_connect`, which cannot reasonably be assumed to match) or individually (by specifying a `pytest.param` and marking the failing backends). Then we scrape across the common parent classes and add any methods that are NOT currently specified in the pre-existing xfailed ones. It's a bit of a nuisance, but it's done, and ideally the manual listing of the inconsistent methods goes away as we unify things. I've opted for not checking that type annotations match, because that seems... unreasonable. This would satisfy ibis-project#9125 once all of the xfail markers are removed, e.g., it checks that all keyword and positional arguments are standardized.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We want to ensure that, for a given backend, that the argument names, plus usage of positional, positional-only, keyword, and keyword-only arguments match, so that there is API consistency when moving between backends.
I've grabbed a few small parts of some of the utilities in Scott Sanderson's
python-interfaceproject (https://github.com/ssanderson/python-interface).While the upstream is no longer maintained, the goal of that project
aligns quite well with some of the issues we face with maintaining
consistent interfaces across backends.
Whether we keep the
pytestapproach for testing the signatures or not, this has proven reasonably effective at finding spots where our argument names aren't matching. (I think I'm in favor of keeping it, since it flagged up some inconsistencies into_parquet_dirthat were added after I opened this PR initially)I've tried to keep the various refactor commits reasonably atomic so that this isn't an absolute nightmare to review.