Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_tap_class, and inspect fields instead of signature for data models #128

Merged
merged 40 commits into from
Mar 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7c454c7
[WIP] Tap class from Pydantic model or dataclass
kddubey Jan 2, 2024
420c1ff
use str for Any, fix docstring
kddubey Jan 3, 2024
9ace3f6
Remove comments about refactor
kddubey Jan 3, 2024
ea6c046
add pydantic as a dependency so tests run
martinjm97 Jan 13, 2024
d44f48e
use older verions
martinjm97 Jan 13, 2024
2d92219
use typing type
martinjm97 Jan 13, 2024
603c6ad
convert_to_tap_class
kddubey Jan 18, 2024
27df903
add dev extra
kddubey Jan 18, 2024
89123b7
fix desc for known_only
kddubey Jan 18, 2024
0b4f202
dont require pydantic
kddubey Jan 19, 2024
a11a088
fix docstring extraction
kddubey Jan 19, 2024
c0d226f
test pydantic BaseModel
kddubey Jan 19, 2024
0f4df97
test pydantic dataclass
kddubey Jan 19, 2024
53eb203
basic test convert_to_tap_class
kddubey Jan 19, 2024
9163e5c
add test todos
kddubey Jan 19, 2024
c8ff11d
dict -> Dict
kddubey Jan 19, 2024
64820af
rename convert_to_tap_class -> to_tap_class
kddubey Jan 19, 2024
99ea551
lingering pipe
kddubey Jan 19, 2024
1b28e87
fix comment
kddubey Jan 19, 2024
0dcb864
test more complex subclasser
kddubey Jan 20, 2024
d399370
update demo
kddubey Jan 20, 2024
43d0d4f
std docstrings
kddubey Jan 20, 2024
368dcf7
test help message
kddubey Jan 20, 2024
0efc236
test arg_list optional
kddubey Jan 20, 2024
5dbe6dd
no func_kwargs for to_tap_class
kddubey Jan 20, 2024
e46120c
fix for python<=3.10 methinks
kddubey Jan 20, 2024
bda73c0
pydantic v1 wackiness
kddubey Jan 20, 2024
978b97d
fix for py39 argparse
kddubey Jan 20, 2024
97f1b6c
add to readme
kddubey Jan 21, 2024
c2c7087
test subparsing
kddubey Jan 23, 2024
76a68f7
test SystemExit error message
kddubey Jan 23, 2024
15eaf3b
stdout -> stderr for non-help
kddubey Jan 23, 2024
f38fba7
dont require pydantic for tests to run
kddubey Jan 23, 2024
878162a
dont require pydantic for test workflow
kddubey Jan 23, 2024
48d237c
unused field
kddubey Jan 23, 2024
0c439f6
use type_to_str instead
kddubey Jan 23, 2024
d83bc1d
little things
kddubey Jan 23, 2024
3523da2
add general pattern
kddubey Feb 1, 2024
28e3c19
littler things
kddubey Feb 19, 2024
7461ce7
allow extra args for Pydantic BaseModels with extra=allow
kddubey Feb 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,14 @@ jobs:
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
- name: Test without pydantic
run: |
pytest
- name: Test with pydantic v1
run: |
python -m pip install "pydantic < 2"
pytest
- name: Test with pydantic v2
run: |
python -m pip install "pydantic >= 2"
pytest
143 changes: 142 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -666,7 +666,7 @@ from tap import tapify
class Squarer:
"""Squarer with a number to square.

:param num: The number to square.
:param num: The number to square.
"""
num: float

Expand All @@ -681,6 +681,94 @@ if __name__ == '__main__':

Running `python square_dataclass.py --num -1` prints `The square of your number is 1.0.`.

<details>
<summary>Argument descriptions</summary>

For dataclasses, the argument's description (which is displayed in the `-h` help message) can either be specified in the
class docstring or the field's description in `metadata`. If both are specified, the description from the docstring is
used. In the example below, the description is provided in `metadata`.

```python
# square_dataclass.py
from dataclasses import dataclass, field

from tap import tapify

@dataclass
class Squarer:
"""Squarer with a number to square.
"""
num: float = field(metadata={"description": "The number to square."})

def get_square(self) -> float:
"""Get the square of the number."""
return self.num ** 2

if __name__ == '__main__':
squarer = tapify(Squarer)
print(f'The square of your number is {squarer.get_square()}.')
```

</details>

#### Pydantic

Pydantic [Models](https://docs.pydantic.dev/latest/concepts/models/) and
[dataclasses](https://docs.pydantic.dev/latest/concepts/dataclasses/) can be `tapify`d.

```python
# square_pydantic.py
from pydantic import BaseModel, Field

from tap import tapify

class Squarer(BaseModel):
"""Squarer with a number to square.
"""
num: float = Field(description="The number to square.")

def get_square(self) -> float:
"""Get the square of the number."""
return self.num ** 2

if __name__ == '__main__':
squarer = tapify(Squarer)
print(f'The square of your number is {squarer.get_square()}.')
```

<details>
<summary>Argument descriptions</summary>

For Pydantic v2 models and dataclasses, the argument's description (which is displayed in the `-h` help message) can
either be specified in the class docstring or the field's `description`. If both are specified, the description from the
docstring is used. In the example below, the description is provided in the docstring.

For Pydantic v1 models and dataclasses, the argument's description must be provided in the class docstring:

```python
# square_pydantic.py
from pydantic import BaseModel

from tap import tapify

class Squarer(BaseModel):
"""Squarer with a number to square.

:param num: The number to square.
"""
num: float

def get_square(self) -> float:
"""Get the square of the number."""
return self.num ** 2

if __name__ == '__main__':
squarer = tapify(Squarer)
print(f'The square of your number is {squarer.get_square()}.')
```

</details>

### tapify help

The help string on the command line is set based on the docstring for the function or class. For example, running `python square_function.py -h` will print:
Expand Down Expand Up @@ -751,4 +839,57 @@ if __name__ == '__main__':
Running `python person.py --name Jesse --age 1` prints `My name is Jesse.` followed by `My age is 1.`. Without `known_only=True`, the `tapify` calls would raise an error due to the extra argument.

### Explicit boolean arguments

Tapify supports explicit specification of boolean arguments (see [bool](#bool) for more details). By default, `explicit_bool=False` and it can be set with `tapify(..., explicit_bool=True)`.

## Convert to a `Tap` class

`to_tap_class` turns a function or class into a `Tap` class. The returned class can be [subclassed](#subclassing) to add
special argument behavior. For example, you can override [`configure`](#configuring-arguments) and
[`process_args`](#argument-processing).

If the object can be `tapify`d, then it can be `to_tap_class`d, and vice-versa. `to_tap_class` provides full control
over argument parsing.

### Examples

#### Simple

```python
# main.py
"""
My script description
"""

from pydantic import BaseModel

from tap import to_tap_class

class Project(BaseModel):
package: str
is_cool: bool = True
stars: int = 5

if __name__ == "__main__":
ProjectTap = to_tap_class(Project)
tap = ProjectTap(description=__doc__) # from the top of this script
args = tap.parse_args()
project = Project(**args.as_dict())
print(f"Project instance: {project}")
```

Running `python main.py --package tap` will print `Project instance: package='tap' is_cool=True stars=5`.

### Complex

The general pattern is:

```python
from tap import to_tap_class

class MyCustomTap(to_tap_class(my_class_or_function)):
# Special argument behavior, e.g., override configure and/or process_args
```

Please see `demo_data_model.py` for an example of overriding [`configure`](#configuring-arguments) and
[`process_args`](#argument-processing).
96 changes: 96 additions & 0 deletions demo_data_model.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
"""
Works for Pydantic v1 and v2.

Example commands:

python demo_data_model.py -h

python demo_data_model.py \
--arg_int 1 \
--arg_list x y z \
--argument_with_really_long_name 3

python demo_data_model.py \
--arg_int 1 \
--arg_list x y z \
--arg_bool \
-arg 3.14
"""
from typing import List, Literal, Optional, Union

from pydantic import BaseModel, Field
from tap import tapify, to_tap_class, Tap


class Model(BaseModel):
"""
My Pydantic Model which contains script args.
"""

arg_int: int = Field(description="some integer")
arg_bool: bool = Field(default=True)
arg_list: Optional[List[str]] = Field(default=None, description="some list of strings")


def main(model: Model) -> None:
print("Parsed args into Model:")
print(model)


def to_number(string: str) -> Union[float, int]:
return float(string) if "." in string else int(string)


class ModelTap(to_tap_class(Model)):
# You can supply additional arguments here
argument_with_really_long_name: Union[float, int] = 3
"This argument has a long name and will be aliased with a short one"

def configure(self) -> None:
# You can still add special argument behavior
self.add_argument("-arg", "--argument_with_really_long_name", type=to_number)

def process_args(self) -> None:
# You can still validate and modify arguments
# (You should do this in the Pydantic Model. I'm just demonstrating that this functionality is still possible)
if self.argument_with_really_long_name > 4:
raise ValueError("argument_with_really_long_name cannot be > 4")

# No auto-complete (and other niceties) for the super class attributes b/c this is a dynamic subclass. Sorry
if self.arg_bool and self.arg_list is not None:
self.arg_list.append("processed")


# class SubparserA(Tap):
# bar: int # bar help


# class SubparserB(Tap):
# baz: Literal["X", "Y", "Z"] # baz help


# class ModelTapWithSubparsing(to_tap_class(Model)):
# foo: bool = False # foo help

# def configure(self):
# self.add_subparsers(help="sub-command help")
# self.add_subparser("a", SubparserA, help="a help", description="Description (a)")
# self.add_subparser("b", SubparserB, help="b help")


if __name__ == "__main__":
# You don't have to subclass tap_class_from_data_model(Model) if you just want a plain argument parser:
# ModelTap = to_tap_class(Model)
args = ModelTap(description="Script description").parse_args()
# args = ModelTapWithSubparsing(description="Script description").parse_args()
print("Parsed args:")
print(args)
# Run the main function
model = Model(**args.as_dict())
main(model)


# tapify works with Model. It immediately returns a Model instance instead of a Tap class
# if __name__ == "__main__":
# model = tapify(Model)
# print(model)
8 changes: 7 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@
with open("README.md", encoding="utf-8") as f:
long_description = f.read()

test_requirements = [
"pydantic >= 2.5.0",
"pytest",
]

setup(
name="typed-argument-parser",
version=__version__,
Expand All @@ -26,7 +31,8 @@
packages=find_packages(),
package_data={"tap": ["py.typed"]},
install_requires=["typing-inspect >= 0.7.1", "docstring-parser >= 0.15"],
tests_require=["pytest"],
tests_require=test_requirements,
kddubey marked this conversation as resolved.
Show resolved Hide resolved
extras_require={"dev": test_requirements},
python_requires=">=3.8",
classifiers=[
"Programming Language :: Python :: 3",
Expand Down
11 changes: 9 additions & 2 deletions tap/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
from argparse import ArgumentError, ArgumentTypeError
from tap._version import __version__
from tap.tap import Tap
from tap.tapify import tapify
from tap.tapify import tapify, to_tap_class

__all__ = ["ArgumentError", "ArgumentTypeError", "Tap", "tapify", "__version__"]
__all__ = [
"ArgumentError",
"ArgumentTypeError",
"Tap",
"tapify",
"to_tap_class",
"__version__",
]
Loading