feat(instructor): introduce ANTHROPIC_JSON mode #542
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Deploying instructor with
|
| Latest commit: |
abfded0
|
| Status: | 🚫 Build failed. |
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 Looks good to me!
- Reviewed the entire pull request up to ab1b10b
- Looked at
394lines of code in8files - Took 2 minutes and 11 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
0additional comments because they didn't meet confidence threshold of85%.
Workflow ID: wflow_fftB6sZVemh5d4kU
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
|
why not just use MD_JSON vs adding a new mode |
There was a problem hiding this comment.
❌ Changes requested.
- Performed an incremental review on ddcf7bf
- Looked at
98lines of code in3files - Took 1 minute and 17 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
0additional comments because they didn't meet confidence threshold of85%.
Workflow ID: wflow_VtT8Hg3zt5QJPejC
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. We'll respond in a few minutes. Learn more here.
| @@ -20,15 +20,15 @@ jobs: | |||
| uses: actions/setup-python@v4 | |||
| with: | |||
| python-version: 3.11 | |||
There was a problem hiding this comment.
Python 3.11 is not yet released. Please use a stable version of Python.
There was a problem hiding this comment.
Thanks, @jxnl - great example of how the training data extension stuff in GPT4 isn't perfect yet.
| @@ -22,8 +22,8 @@ jobs: | |||
| uses: actions/setup-python@v4 | |||
| with: | |||
| python-version: ${{ matrix.python-version }} | |||
There was a problem hiding this comment.
Python 3.11 is not yet released. Please use a stable version of Python.
There was a problem hiding this comment.
👍 Looks good to me!
- Performed an incremental review on d4a3fdb
- Looked at
191lines of code in11files - Took 3 minutes and 7 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
7additional comments because they didn't meet confidence threshold of85%.
1. .github/workflows/test.yml:36:
- Assessed confidence :
50% - Comment:
Thepoetry installcommand has been updated to include theanthropicoption. Please ensure that this option is necessary and that it does not introduce any potential issues or conflicts with other dependencies. - Reasoning:
The PR introduces a new modeANTHROPIC_JSONininstructor/mode.pyand updates theinstructor/function_calls.pyandinstructor/process_response.pyto handle the new mode. The PR also updates the documentation and tests to reflect these changes. However, the PR does not provide the actual changes in the mentioned files. It would be necessary to review these changes to ensure that the new mode is implemented correctly and that the tests cover the new functionality. Additionally, the PR removes some files, but it does not provide a reason for these removals. It would be important to understand why these files were removed and whether their removal could impact the functionality of the software.
2. tests/llm/test_openai/evals/test_classification_enums.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
3. tests/llm/test_openai/evals/test_classification_literals.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
4. tests/llm/test_openai/evals/test_entities.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
5. tests/llm/test_openai/test_modes.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
6. tests/llm/test_openai/test_stream.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
7. tests/llm/test_openai/test_validators.py:11:
- Assessed confidence :
50% - Comment:
The import statement has been updated. Please ensure that this change does not break the tests and that all necessary modules are imported correctly. - Reasoning:
The PR includes changes to the test files, renaming them and updating the imports. However, without the actual changes in the test files, it's hard to determine if the tests are updated correctly to cover the newANTHROPIC_JSONmode. It would be necessary to review these changes to ensure that the tests are updated correctly.
Workflow ID: wflow_70cupd6tCyCrKMl6
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
i dont have polyglot, so i have to handle system message, and getting the text out |
There was a problem hiding this comment.
👍 Looks good to me!
- Performed an incremental review on 31810fa
- Looked at
301lines of code in1files - Took 3 minutes and 28 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
0additional comments because they didn't meet confidence threshold of85%.
Workflow ID: wflow_YzRJWAj2fQyXhRZ3
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
There was a problem hiding this comment.
👍 Looks good to me!
- Performed an incremental review on b887b2e
- Looked at
12lines of code in1files - Took 1 minute and 29 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
1additional comments because they didn't meet confidence threshold of85%.
1. .github/workflows/test_docs.yml:39:
- Assessed confidence :
80% - Comment:
Please ensure that the test path 'tests/llm/test_openai/docs' exists and contains the relevant tests for the documentation. - Reasoning:
The test path in the 'Run tests' step of the GitHub workflow has been changed. I need to check if the new path 'tests/llm/test_openai/docs' exists in the codebase and contains the relevant tests.
Workflow ID: wflow_R8kDzfj7nyTmJckF
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
There was a problem hiding this comment.
👍 Looks good to me!
- Performed an incremental review on 492f671
- Looked at
124lines of code in4files - Took 2 minutes and 11 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
1additional comments because they didn't meet confidence threshold of85%.
1. docs/concepts/patching.md:87:
- Assessed confidence :
100% - Grade:
0% - Comment:
The PR description mentions the introduction of a newANTHROPIC_JSONmode, but this is not visible in the diff. Please ensure that this new mode has been implemented correctly ininstructor/mode.py,instructor/function_calls.py, andinstructor/process_response.py. - Reasoning:
The PR introduces a newANTHROPIC_JSONmode, but I don't see any changes in the codebase that reflect this. The PR description mentions updates toinstructor/mode.py,instructor/function_calls.py, andinstructor/process_response.py, but these are not visible in the diff. I need to check these files in the codebase to ensure the new mode has been implemented correctly.
Workflow ID: wflow_CUbyr40lQp17KkQO
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
There was a problem hiding this comment.
👍 Looks good to me!
- Performed an incremental review on abfded0
- Looked at
22lines of code in1files - Took 2 minutes and 8 seconds to review
More info
- Skipped
0files when reviewing. - Skipped posting
1additional comments because they didn't meet confidence threshold of85%.
1. .github/workflows/test.yml:39:
- Assessed confidence :
100% - Grade:
0% - Comment:
The pytest command is excluding tests related toanthropic. If there are new tests related to theANTHROPIC_JSONmode, they might not be executed. Please ensure that the new tests are included in the CI/CD pipeline. - Reasoning:
The PR mentions that a new modeANTHROPIC_JSONis introduced and the tests are updated. However, in the GitHub workflow file, the pytest command is excluding tests related toanthropic. This might lead to the new tests not being executed as part of the CI/CD pipeline.
Workflow ID: wflow_O2yxfoFByzpF4jM4
Not what you expected? You can customize the content of the reviews using rules. Learn more here.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This PR introduces a new
ANTHROPIC_JSONmode in the Instructor library, updates relevant files and tests, modifies GitHub workflows in.github/workflows/test.yml, and updates thepoetry.lockfile.Key points:
ANTHROPIC_JSONmode in the Instructor library ininstructor/mode.pyinstructor/function_calls.pyandinstructor/process_response.pyto handle the new modedocs/blog/posts/anthropic.mdanddocs/concepts/patching.mdtests/anthropic/test_simple.pyexamples/classification/test_run.pyandtests/anthropic/test_anthropic.py.github/workflows/test.ymlto includeanthropicdependencypoetry.lockfileGenerated with ❤️ by ellipsis.dev