Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] dataclass has no type validation so errors end up being hard to debug #3495

Closed
alvarobartt opened this issue Aug 2, 2023 · 0 comments · Fixed by #3497
Closed

[BUG] dataclass has no type validation so errors end up being hard to debug #3495

alvarobartt opened this issue Aug 2, 2023 · 0 comments · Fixed by #3497
Assignees
Labels
type: bug Indicates an unexpected problem or unintended behavior
Milestone

Comments

@alvarobartt
Copy link
Member

Describe the bug

This is more an improvement than a bug, because some errors end up being cryptic and hard to debug, because the client is just throwing an HTTP 500 error code while the actual traceback is in the server.

This is due to some missing validation when dataclass is used with no __post_init__ validation, instead of pydantic.BaseModel.

Stacktrace and Code to create the bug

There are more situations when this can break, but this is one of those when using TextClassificationSettings

import argilla as rg

rg.init(api_url="...", api_key="...")

rg.configure_dataset("my-dataset", workspace=rg.get_workspace(), settings=rg.TextClassificationSettings(label_schema=["A", "B"]))

Fails because it's a List[str] instead of Set[str], but the error displayed in the Python client is AttributeError: 'str' object has no attribute 'name', but the traceback cannot be followed from the Python client, just from the server.

Expected behavior

A validation layer to make sure that everything's right before calling the API, to be able to show a nice error message to the user, and better error handling, to avoid showing API exceptions straight away in the client, but a formatted version of those.

Environment:

  • Argilla Version [e.g. 1.0.0]: 1.13.0 (but it's N/A)
  • ElasticSearch Version [e.g. 7.10.2]: N/A
  • Docker Image (optional) [e.g. argilla:v1.0.0]: argilla:v1.13.0

Additional context

This ends up with users asking for help on issues they cannot easily debug, so this would also avoid people from running into these issues or just having a simple way to solve it themselves.

@alvarobartt alvarobartt added the type: bug Indicates an unexpected problem or unintended behavior label Aug 2, 2023
@alvarobartt alvarobartt added this to the v1.14.0 milestone Aug 2, 2023
frascuchon added a commit that referenced this issue Aug 2, 2023
…on tasks (#3497)

<!-- Thanks for your contribution! As part of our Community Growers
initiative 🌱, we're donating Justdiggit bunds in your name to reforest
sub-Saharan Africa. To claim your Community Growers certificate, please
contact David Berenstein in our Slack community or fill in this form
https://tally.so/r/n9XrxK once your PR has been merged. -->

# Description

This PR prevents errors described in #3495 by parsing labels into
strings. The backend parsing is also changed to apply label
normalization after the type conversion.

Closes #3495

**Type of change**

(Please delete options that are not relevant. Remember to title the PR
according to the type of change)

- [X] Bug fix (non-breaking change which fixes an issue)

**How Has This Been Tested**

(Please describe the tests that you ran to verify your changes. And
ideally, reference `tests`)

The issue code snippet has been launched locally and verified that it
works with new changes.

**Checklist**

- [X] follows the style guidelines of this project
- [X] I did a self-review of my code
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] I have added relevant notes to the CHANGELOG.md file (See
https://keepachangelog.com/)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
keithCuniah pushed a commit that referenced this issue Aug 3, 2023
…on tasks (#3497)

<!-- Thanks for your contribution! As part of our Community Growers
initiative 🌱, we're donating Justdiggit bunds in your name to reforest
sub-Saharan Africa. To claim your Community Growers certificate, please
contact David Berenstein in our Slack community or fill in this form
https://tally.so/r/n9XrxK once your PR has been merged. -->

# Description

This PR prevents errors described in #3495 by parsing labels into
strings. The backend parsing is also changed to apply label
normalization after the type conversion.

Closes #3495

**Type of change**

(Please delete options that are not relevant. Remember to title the PR
according to the type of change)

- [X] Bug fix (non-breaking change which fixes an issue)

**How Has This Been Tested**

(Please describe the tests that you ran to verify your changes. And
ideally, reference `tests`)

The issue code snippet has been launched locally and verified that it
works with new changes.

**Checklist**

- [X] follows the style guidelines of this project
- [X] I did a self-review of my code
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] I have added relevant notes to the CHANGELOG.md file (See
https://keepachangelog.com/)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants