Migrate to pydantic v2 #78
Conversation
Signed-off-by: Yu Ishikawa <yu-iskw@users.noreply.github.com>
Signed-off-by: Yu Ishikawa <yu-iskw@users.noreply.github.com>
Signed-off-by: Yu Ishikawa <yu-iskw@users.noreply.github.com>
WalkthroughThe recent updates primarily focus on enhancing compatibility with Pydantic v2 for Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Outside diff range and nitpick comments (4)
README.md (2)
Line range hint
34-34: Consider adding blank lines around headings for better readability and to adhere to Markdown best practices.+ ## Installation + ## Python classes + ### Catalog + ### Run Results + ### Sources +Also applies to: 37-37, 40-40, 54-54, 62-62
Line range hint
70-70: Ensure that fenced code blocks are surrounded by blank lines to comply with Markdown formatting standards.+ ### Parse run-results.json +dbt_artifacts_parser/parsers/manifest/manifest_v1.py (1)
70-70: Consider the implications of allowing extra fields inColumnInfo.Allowing extra fields in
ColumnInfowithConfigDict(extra='allow')could lead to unintended data being stored in the model. This might be necessary for flexibility but could also introduce data consistency issues. Evaluate the need for this configuration based on the use case.dbt_artifacts_parser/parsers/manifest/manifest_v2.py (1)
9-9: Ensure consistent import organization.Consider grouping imports from the same library together for better readability.
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files ignored due to path filters (2)
dbt_artifacts_parser/resources/manifest/manifest_v10.jsonis excluded by!**/*.jsonpyproject.tomlis excluded by!**/*.toml
Files selected for processing (11)
- README.md (1 hunks)
- dbt_artifacts_parser/parsers/catalog/catalog_v1.py (1 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v1.py (38 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v10.py (70 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v11.py (7 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v2.py (33 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v3.py (33 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v4.py (36 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v5.py (38 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v6.py (36 hunks)
- dbt_artifacts_parser/parsers/manifest/manifest_v7.py (35 hunks)
Files not summarized due to errors (8)
- dbt_artifacts_parser/parsers/manifest/manifest_v1.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v11.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v2.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v3.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v4.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v5.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v6.py: Error: Message exceeds token limit
- dbt_artifacts_parser/parsers/manifest/manifest_v7.py: Error: Message exceeds token limit
Additional Context Used
LanguageTool (3)
README.md (3)
Near line 48: A punctuation mark might be missing here.
Context: ...est/manifest_v8.py) for manifest.json v8 - [ManifestV9](dbt_artifacts_parser/parsers...
Near line 54: A punctuation mark might be missing here.
Context: ...) for manifest.json v12 ### Run Results - [RunResultsV1](dbt_artifacts_parser/parse...
Near line 62: A punctuation mark might be missing here.
Context: ...py) for run_results.json v6 ### Sources - [SourcesV1](dbt_artifacts_parser/parsers/...
Additional comments not posted (65)
dbt_artifacts_parser/parsers/catalog/catalog_v1.py (3)
8-8: Ensure thatAwareDatetimeandConfigDictare correctly imported frompydantic.
14-16: Consistent use ofConfigDict(extra='forbid')across all models enhances attribute management by forbidding unexpected fields.Also applies to: 25-27, 37-39, 47-49, 58-60, 68-70
19-19: The use ofAwareDatetimeforgenerated_atensures proper timezone-aware datetime handling, aligning with best practices for datetime management in Python.README.md (2)
16-20: The warning about Pydantic v1 not being supported for dbt 1.9 or later is clear and well-placed, helping users understand the migration necessity.
22-27: The version compatibility table is informative and helps users quickly identify which versions ofdbtandpydanticare compatible with each release ofdbt-artifacts-parser.dbt_artifacts_parser/parsers/manifest/manifest_v11.py (7)
16-18: Ensure consistent use ofConfigDictacross all models.The use of
ConfigDict(extra='forbid')is consistent with the PR's objective to enforce strict attribute handling. This is a good practice to prevent unexpected attributes from being processed.
41-43: Consistent application ofConfigDict(extra='forbid')across models.The consistent enforcement of
extra='forbid'across various models aligns with the goal of strict attribute management, which is crucial for maintaining data integrity and predictability in data models.Also applies to: 49-51, 58-60, 66-68, 80-82, 118-120, 129-131, 143-145, 152-154, 160-162, 168-170, 177-179
1144-1145: Consider the implications of allowing extra fields inSavedQueryConfig.Allowing extra fields in
SavedQueryConfigcan introduce variability that might be undesirable. Ensure that this flexibility is necessary and properly documented.
1332-1334: Assess the appropriateness of allowing extra fields inSemanticModelConfig.The allowance of extra fields in
SemanticModelConfigshould be scrutinized to ensure it does not compromise the model's integrity and aligns with the system's architectural requirements.Verification successful
Allowing extra fields in
SemanticModelConfigappears to be justified based on its usage in the codebase. The presence ofextra='allow'in bothmanifest_v10.pyandmanifest_v11.py, along with thefield_extraattribute inmanifest_v11.py, indicates that additional fields are expected and dynamically handled.
manifest_v10.py:SemanticModelConfigincludesextra='allow'and is used with a default factory that validates a dictionary with anenabledfield.manifest_v11.py:SemanticModelConfigincludesextra='allow'and has an additionalfield_extraattribute, suggesting that extra fields are anticipated.These patterns suggest that allowing extra fields is a deliberate design choice to accommodate dynamic configurations.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the necessity of allowing extra fields in SemanticModelConfig and assess potential impacts on system behavior. # Test: Check for any dynamic usage of SemanticModelConfig that might justify the allowance of extra fields. rg --type python 'SemanticModelConfig'Length of output: 69
Script:
#!/bin/bash # Description: Verify the necessity of allowing extra fields in SemanticModelConfig and assess potential impacts on system behavior. # Test: Check for any dynamic usage of SemanticModelConfig that might justify the allowance of extra fields. rg --type py 'SemanticModelConfig'Length of output: 834
Script:
#!/bin/bash # Description: Extract relevant lines from files where SemanticModelConfig is used to assess the necessity of allowing extra fields. # Extract lines around the usage of SemanticModelConfig in manifest_v10.py rg --context 5 'SemanticModelConfig' dbt_artifacts_parser/parsers/manifest/manifest_v10.py # Extract lines around the usage of SemanticModelConfig in manifest_v11.py rg --context 5 'SemanticModelConfig' dbt_artifacts_parser/parsers/manifest/manifest_v11.py # Extract lines around the usage of SemanticModelConfig in manifest_v12.py rg --context 5 'SemanticModelConfig' dbt_artifacts_parser/parsers/manifest/manifest_v12.pyLength of output: 2584
222-224: Consider the implications of allowing extra fields inTestConfig.Allowing extra fields in configurations like
TestConfigcan introduce variability that might be undesirable. Ensure that this flexibility is necessary and properly documented.
595-597: Review the decision to allow extra fields inSnapshotConfig.Similar to
TestConfig, the decision to allow extra fields inSnapshotConfigshould be carefully considered to ensure it aligns with the overall system design and requirements.Verification successful
The
SnapshotConfigclass inmanifest_v11.pyis defined withextra='allow', which permits additional fields not explicitly defined in the model. This is similar to theTestConfigclass, which also allows extra fields.Given the structure and usage of
SnapshotConfig, it appears that allowing extra fields is consistent with the design of other configuration classes in the same file. This flexibility might be necessary to accommodate various configurations that are not predefined.
- Location in Codebase:
dbt_artifacts_parser/parsers/manifest/manifest_v11.py(lines definingSnapshotConfig)Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the necessity of allowing extra fields in SnapshotConfig and assess potential impacts on system behavior. # Test: Check for any dynamic usage of SnapshotConfig that might justify the allowance of extra fields. rg --type python 'SnapshotConfig'Length of output: 64
Script:
#!/bin/bash # Description: Verify the necessity of allowing extra fields in SnapshotConfig and assess potential impacts on system behavior. # Test: Check for any dynamic usage of SnapshotConfig that might justify the allowance of extra fields. rg --type py 'SnapshotConfig'Length of output: 2413
Script:
#!/bin/bash # Description: Extract the definition of SnapshotConfig from manifest_v11.py to understand its structure and usage. # Extract the class definition of SnapshotConfig from manifest_v11.py ast-grep --lang python --pattern $'class SnapshotConfig(BaseParserModel) { $$$ }' dbt_artifacts_parser/parsers/manifest/manifest_v11.pyLength of output: 608
Script:
#!/bin/bash # Description: Extract the definition of SnapshotConfig from manifest_v11.py to understand its structure and usage. # Extract the class definition of SnapshotConfig from manifest_v11.py ast-grep --lang python --pattern 'class SnapshotConfig(BaseParserModel):' dbt_artifacts_parser/parsers/manifest/manifest_v11.pyLength of output: 129
Script:
#!/bin/bash # Description: Extract the definition of SnapshotConfig from manifest_v11.py to understand its structure and usage. # Extract lines around the class definition of SnapshotConfig from manifest_v11.py rg -A 20 -B 5 'class SnapshotConfig(BaseParserModel):' dbt_artifacts_parser/parsers/manifest/manifest_v11.pyLength of output: 110
Script:
#!/bin/bash # Description: List the files and manually inspect the manifest_v11.py to confirm the presence and structure of SnapshotConfig. # List the files in the directory fd manifest_v11.py # Display the content of manifest_v11.py cat dbt_artifacts_parser/parsers/manifest/manifest_v11.pyLength of output: 43412
675-677: Evaluate the need for flexibility inSeedConfig.Allowing extra fields in
SeedConfigincreases flexibility but also adds complexity. Ensure that this is justified by actual use cases and documented.dbt_artifacts_parser/parsers/manifest/manifest_v1.py (5)
15-17: Review and verify the consistent use ofConfigDictacross all models.The use of
ConfigDictwithextra='forbid'orextra='allow'is consistent with the PR's objective to enforce stricter attribute handling in Pydantic models. This is a crucial part of the migration to Pydantic v2. Ensure that all models have the correct configuration as per their requirements.Also applies to: 44-46, 52-54, 61-63, 69-71, 81-83, 88-90, 100-102, 138-140, 151-153, 183-185, 254-256, 315-317, 378-380, 410-412, 438-440, 465-467, 475-477, 490-492, 512-514, 520-522, 530-532, 541-543, 548-550, 557-559, 598-600, 606-608, 625-627, 685-687, 744-746, 804-806, 863-865, 925-927, 1044-1046, 1098-1100, 1153-1155, 1207-1209, 1261-1263, 1379-1381, 1438-1440, 1297-1299, 1306-1308, 1319-1321, 1330-1332, 1355-1357
22-22: Use ofAwareDatetimefor datetime fields is appropriate.The migration to
AwareDatetimefromdatetimeis in line with Pydantic v2's enhanced support for timezone-aware datetime objects, which is a good practice for ensuring consistency in time-related data handling.Also applies to: 480-480
100-102: Ensure appropriate use ofextra='allow'in configuration.The use of
extra='allow'in configurations likeTestConfig,SeedConfig,TimestampSnapshotConfig,CheckSnapshotConfig,GenericSnapshotConfig,ExternalPartition,SourceConfig,NodeConfig,CompiledSeedNode, andParsedSnapshotNodeshould be carefully reviewed to ensure it aligns with the intended flexibility of these models without compromising the integrity of the data structure.Also applies to: 151-153, 378-380, 410-412, 438-440, 520-522, 530-532, 606-608, 925-927, 1261-1263
1477-1547: Review the comprehensive changes inManifestV1.The changes in
ManifestV1to include various node types and configurations appear well-structured and align with the objectives of the PR. The use ofField(...)for required fields and the detailed descriptions enhance the readability and maintainability of the model.
29-30: Validate regex patterns in string constraints.The regex patterns used in
user_idandseverityfields should be verified against Pydantic v2 documentation to ensure they are correctly implemented and will not cause runtime errors.Also applies to: 117-118
Verification successful
The regex patterns used in the
constrfunction for theuser_idandseverityfields have been identified across multiple versions of the manifest files. The patterns are as follows:
user_id:r'[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}'severity:r'^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$'These patterns appear to be correctly implemented according to Pydantic v2 documentation, which supports regex patterns for string constraints.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check the regex patterns for user_id and severity fields. # Test: Search for the regex usage in the codebase. Expect: Correct regex patterns as per Pydantic v2 documentation. rg --type python $'constr(pattern=)'Length of output: 68
Script:
#!/bin/bash # Description: Check the regex patterns for user_id and severity fields. # Test: Search for the regex usage in the codebase. Expect: Correct regex patterns as per Pydantic v2 documentation. rg --type py 'constr(pattern=)'Length of output: 33
Script:
#!/bin/bash # Description: Check the usage of the `constr` function in the codebase. # Test: Search for the `constr` function usage in the codebase. rg --type py 'constr('Length of output: 89
Script:
#!/bin/bash # Description: Check the usage of the `constr` function in the codebase. # Test: Search for the `constr` function usage in the codebase. rg --type py 'constr\('Length of output: 3272
dbt_artifacts_parser/parsers/manifest/manifest_v2.py (5)
15-17: Correct use ofConfigDictto enforce attribute restrictions.This configuration helps prevent unexpected attributes from being processed, aligning with the principles of explicit and strict data handling in Pydantic v2.
44-46: Consistent application ofConfigDictacross models.The consistent use of
ConfigDictwithextra='forbid'orextra='allow'across various models ensures that the models are either strictly adhering to the schema or are flexible where explicitly desired.Also applies to: 52-54, 61-63, 81-83, 88-90, 144-146, 157-159, 398-400, 427-429, 437-439, 452-454, 474-476, 482-484, 492-494, 503-505, 510-512, 519-521, 560-562, 568-570, 587-589, 655-657, 716-718, 778-780, 839-841, 909-911, 971-973, 1032-1034, 1088-1090, 1145-1147, 1201-1203, 1257-1259, 1295-1297, 1304-1306, 1317-1319, 1328-1330, 1354-1356, 1381-1383, 1442-1444
1482-1552: Review the structure ofManifestV2.The
ManifestV2class is well-structured, providing clear and detailed descriptions for each field. This structure aids in understanding the relationships and data flow within the dbt project.
69-71: Review the use ofConfigDict(extra='allow').This is to ensure that the flexibility provided by
extra='allow'is intentional and documented, as it can lead to less strict data validation which might not be desirable in all cases.Also applies to: 100-102, 158-159, 399-400, 483-484, 493-494
22-22: Validate the default value forgenerated_at.dbt_artifacts_parser/parsers/manifest/manifest_v10.py (12)
15-17: ConfigDict usage withextra='forbid'is appropriate for strict attribute management.
22-22: Correct use ofAwareDatetimeforgenerated_atto ensure timezone-aware datetime handling.
31-31: Updated regex pattern foruser_idis consistent with typical UUID formats and Pydantic v2 requirements.
46-48: Consistent application ofConfigDictwithextra='forbid'across various models to prevent unexpected fields, enhancing robustness and maintainability.Also applies to: 60-62, 69-71, 77-79, 93-95, 104-106, 113-115, 121-123, 129-131, 141-143, 178-180, 190-192, 212-214, 225-227, 266-268, 299-301, 310-312, 326-328, 334-336, 344-346, 360-362, 373-375, 404-406, 412-414, 438-440, 461-463, 469-471, 479-481, 491-493, 522-524, 532-534, 546-548, 569-571, 590-592, 612-614, 620-622, 627-629, 659-661, 672-674, 743-745, 822-824, 906-908, 984-986, 1062-1064, 1137-1139, 1189-1191, 1256-1258, 1265-1267, 1276-1278, 1301-1303, 1333-1335, 1342-1344, 1353-1355, 1361-1363, 1375-1377, 1383-1385, 1461-1463, 1500-1502, 1514-1516, 1527-1529, 1562-1564, 1592-1594
249-252: Usage ofdefault_factoryinDocsandContractConfigfields to ensure proper instantiation of default values.Also applies to: 290-293, 651-654, 791-791, 870-870, 1032-1032, 1104-1104, 1158-1158, 1238-1238
687-687: Usage ofdefault_factoryinTestConfigto provide a clean way to set default configurations.Also applies to: 1078-1078
728-728: Proper use ofdefault_factoryinDependsOnto initialize lists of dependencies.Also applies to: 806-806, 885-885, 969-969, 1119-1119, 1173-1173, 1250-1250, 1324-1324, 1446-1446, 1581-1581
736-736: Consistent use ofdefault_factoryinContractto ensure default values are correctly set.Also applies to: 814-814, 893-893, 977-977, 1127-1127, 1181-1181
153-153: Ensure the regex pattern forseveritycorrectly captures only 'WARN' or 'ERROR' in appropriate cases.Verification successful
The regex pattern for
severitycorrectly captures only 'WARN' or 'ERROR' in appropriate cases.
- The pattern
^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$successfully matches 'WARN' and 'ERROR' while not matching 'info'.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the regex pattern for `severity` field. # Test: Check if the regex pattern correctly matches only 'WARN' or 'ERROR'. echo "WARN" | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' echo "ERROR" | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' echo "info" | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' # Should not matchLength of output: 209
9-9: Ensure all necessary imports from Pydantic are included for the migration to v2.Verification successful
The necessary Pydantic imports for v2 compatibility are present in the file.
AwareDatetimeConfigDictFieldconstrScripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that all necessary Pydantic imports are present for v2 compatibility. # Test: Search for Pydantic imports in the file. Expect: All necessary imports for v2 should be present. rg --type python $'from pydantic import' dbt_artifacts_parser/parsers/manifest/manifest_v10.pyLength of output: 126
Script:
#!/bin/bash # Description: Verify that all necessary Pydantic imports are present for v2 compatibility. # Test: Search for Pydantic imports in the file. Expect: All necessary imports for v2 should be present. rg --type py $'from pydantic import' dbt_artifacts_parser/parsers/manifest/manifest_v10.pyLength of output: 153
758-758: Review the use ofNodeConfigin various node types to ensure it aligns with the intended configurations and constraints.Also applies to: 837-837, 921-921, 999-999
Verification successful
The extracted context shows that
NodeConfigis used consistently withFieldanddefault_factoryto provide default values for configurations. Themodel_validatemethod is used to set default values for various attributes likeenabled,alias,schema, anddatabase. This usage appears to align with the intended configurations and constraints.
NodeConfigis defined as a subclass ofBaseParserModel.- It uses
ConfigDictto allow extra fields.- The
Fieldwithdefault_factoryis used to provide default values forNodeConfig.Based on the provided context, the use of
NodeConfigin various node types seems appropriate and consistent with the intended configurations.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the use of `NodeConfig` in various node types for appropriate configurations. # Test: Check the configuration settings in `NodeConfig` usage across different node types. rg --type python $'NodeConfig' dbt_artifacts_parser/parsers/manifest/manifest_v10.pyLength of output: 114
Script:
#!/bin/bash # Description: Verify the use of `NodeConfig` in various node types for appropriate configurations. # Test: Check the configuration settings in `NodeConfig` usage across different node types. rg 'NodeConfig' dbt_artifacts_parser/parsers/manifest/manifest_v10.pyLength of output: 609
Script:
#!/bin/bash # Description: Extract surrounding lines of code for each instance of `NodeConfig` usage to verify configurations. # Test: Extract 5 lines before and after each instance of `NodeConfig` usage. rg 'NodeConfig' -C 5 dbt_artifacts_parser/parsers/manifest/manifest_v10.pyLength of output: 2062
141-143: Review the use ofConfigDict(extra='allow')to ensure it is justified by the need to accept additional attributes in these specific models.Also applies to: 225-227, 266-268, 344-346, 620-622, 627-629, 659-661
dbt_artifacts_parser/parsers/manifest/manifest_v3.py (5)
15-17: Review the consistent use ofConfigDictwith appropriate settings (extra='forbid'orextra='allow') across all models.Also applies to: 44-46, 52-54, 61-63, 69-71, 81-83, 88-90, 100-102, 138-140, 151-153, 184-186, 258-260, 322-324, 385-387, 415-417, 425-427, 440-442, 462-464, 470-472, 479-481, 489-491, 500-502, 507-509, 516-518, 557-559, 565-567, 585-587, 648-650, 711-713, 775-777, 838-840, 903-905, 967-969, 1030-1032, 1088-1090, 1147-1149, 1205-1207, 1263-1265, 1338-1340, 1364-1366, 1391-1393, 1454-1456, 1494-1496, 1302-1304, 1311-1313, 1327-1329
22-22: Use ofAwareDatetimefor datetime fields aligns with Pydantic v2 requirements for timezone-aware datetimes.Also applies to: 430-430
110-112: The regex pattern for severity levels inTestConfigis a good practice to ensure data integrity.Also applies to: 111-112, 275-291, 602-618, 856-872
9-9: Ensure consistent import ofBaseModelfrompydantic.
25-36: Ensure that all fields are properly annotated withField(...)where necessary, especially where defaults and descriptions are provided.Also applies to: 44-54, 61-63, 69-71, 81-83, 88-90, 100-102, 138-140, 151-153, 184-186, 258-260, 322-324, 385-387, 415-417, 425-427, 440-442, 462-464, 470-472, 479-481, 489-491, 500-502, 507-509, 516-518, 557-559, 565-567, 585-587, 648-650, 711-713, 775-777, 838-840, 903-905, 967-969, 1030-1032, 1088-1090, 1147-1149, 1205-1207, 1263-1265, 1338-1340, 1364-1366, 1391-1393, 1454-1456, 1494-1496, 1302-1304, 1311-1313, 1327-1329
Verification successful
The search results indicate that the
Fieldfunction is used consistently in the specified file, including for the fields mentioned in the review comment. This confirms that the fields are properly annotated withField(...)where necessary, especially where defaults and descriptions are provided.
project_iduser_idsend_anonymous_usage_statsadapter_typeThese fields are correctly using
Fieldwith appropriate descriptions and default values.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for proper use of Field in model definitions. rg --type python "Field\("Length of output: 59
Script:
#!/bin/bash # Description: Check for proper use of Field in model definitions in the specified file. rg 'Field\(' dbt_artifacts_parser/parsers/manifest/manifest_v3.pyLength of output: 4385
dbt_artifacts_parser/parsers/manifest/manifest_v4.py (6)
15-17: Correct use ofConfigDictto enforce attribute restrictions inManifestMetadata.
25-36: Good documentation and use ofFieldfor additional metadata inManifestMetadata.
44-46: Consistent application ofConfigDictacross various models to manage attribute behavior. Ensure this aligns with the overall data handling strategy.Also applies to: 52-54, 61-63, 69-71, 81-83, 88-90, 100-102, 142-144, 155-157, 188-190, 266-268, 331-333, 394-396, 424-426, 434-436, 449-451, 471-473, 479-481, 488-490, 498-500, 509-511, 516-518, 525-527, 568-570, 576-578, 585-587, 605-607, 668-670, 731-733, 795-797, 858-860, 921-923, 987-989, 1051-1053, 1114-1116, 1172-1174, 1231-1233, 1289-1291, 1347-1349, 1405-1407, 1564-1566, 1627-1629
1667-1747: Review theManifestV4class for correct configuration and data structure definitions. Ensure that the use ofField(...)and the descriptions provided align with the project's documentation standards.
9-9: Ensure consistent import ofBaseModelfrompydantic.
22-22: EnsureAwareDatetimeis correctly used and compatible with all datetime fields across the project.Verification successful
Ensure
AwareDatetimeis correctly used and compatible with all datetime fields across the project.The search results indicate that
AwareDatetimeis consistently used for datetime fields across various files in the project. Here are some of the relevant locations:
dbt_artifacts_parser/parsers/catalog/catalog_v1.pydbt_artifacts_parser/parsers/run_results/run_results_v4.pydbt_artifacts_parser/parsers/manifest/manifest_v9.pydbt_artifacts_parser/parsers/sources/sources_v3.pyThis confirms that
AwareDatetimeis being used correctly and consistently for datetime fields.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that all datetime fields across the project use AwareDatetime. rg --type python "Optional\[datetime\]"Length of output: 72
Script:
#!/bin/bash # Description: Verify that all datetime fields across the project use AwareDatetime. rg --type py "Optional\[datetime\]"Length of output: 37
Script:
#!/bin/bash # Description: Search for all datetime fields and verify their types. rg --type py "datetime"Length of output: 23
Script:
#!/bin/bash # Description: Search for the usage of AwareDatetime across the project. rg --type py "AwareDatetime"Length of output: 10921
dbt_artifacts_parser/parsers/manifest/manifest_v5.py (7)
15-17: Correct use ofConfigDictto enforce attribute restrictions inManifestMetadata.
44-46: Consistent application ofConfigDictwithextra='forbid'orextra='allow'across various models to manage attribute behavior correctly.Also applies to: 52-54, 61-63, 81-83, 88-90, 142-144, 155-157, 189-191, 267-269, 332-334, 396-398, 426-428, 436-438, 451-453, 473-475, 481-483, 490-492, 500-502, 511-513, 518-520, 527-529, 570-572, 578-580, 587-589, 608-610, 671-673, 735-737, 800-802, 864-866, 928-930, 994-996, 1059-1061, 1123-1125, 1182-1184, 1242-1244, 1301-1303, 1360-1362, 1419-1421, 1458-1460, 1467-1469, 1483-1485, 1494-1496, 1520-1522, 1578-1580
29-29: Validate the updated regex pattern for UUID inuser_idto ensure it matches the intended format.Verification successful
The regex pattern for UUID in the
user_idfield has been validated successfully. The pattern correctly matches the intended UUID format.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Validate the regex pattern for UUID. echo 'Testing regex pattern for UUID' echo '643c124a-0b61-4f90-90a5-a2decad774d0' | grep -P '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'Length of output: 226
111-111: Check the regex pattern for severity levels to ensure it correctly captures only 'WARN' or 'ERROR' in various cases.Also applies to: 295-295, 956-956, 1604-1604
Verification successful
The regex pattern correctly captures 'WARN' and 'ERROR' in various cases as intended.
- The pattern
^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$successfully matches 'WARN', 'ERROR', 'warn', and 'error'.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Test the regex pattern for severity levels. echo 'WARN' | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' echo 'ERROR' | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' echo 'warn' | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$' echo 'error' | grep -P '^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$'Length of output: 287
9-9: Ensure consistent import ofBaseModelfrompydantic.
22-22: Ensure thatAwareDatetimeis correctly used and compatible with all datetime fields across the project.Verification successful
Ensure that
AwareDatetimeis correctly used and compatible with all datetime fields across the project.The search results indicate that
AwareDatetimeis consistently used for datetime fields across various files in the project. Here are some of the files whereAwareDatetimeis used:
dbt_artifacts_parser/parsers/manifest/manifest_v1.pydbt_artifacts_parser/parsers/manifest/manifest_v2.pydbt_artifacts_parser/parsers/manifest/manifest_v3.pydbt_artifacts_parser/parsers/manifest/manifest_v4.pydbt_artifacts_parser/parsers/manifest/manifest_v5.pydbt_artifacts_parser/parsers/manifest/manifest_v6.pydbt_artifacts_parser/parsers/manifest/manifest_v7.pydbt_artifacts_parser/parsers/manifest/manifest_v8.pydbt_artifacts_parser/parsers/manifest/manifest_v9.pydbt_artifacts_parser/parsers/manifest/manifest_v10.pydbt_artifacts_parser/parsers/run_results/run_results_v1.pydbt_artifacts_parser/parsers/run_results/run_results_v2.pydbt_artifacts_parser/parsers/run_results/run_results_v3.pydbt_artifacts_parser/parsers/run_results/run_results_v4.pydbt_artifacts_parser/parsers/sources/sources_v1.pydbt_artifacts_parser/parsers/sources/sources_v2.pydbt_artifacts_parser/parsers/sources/sources_v3.pydbt_artifacts_parser/parsers/catalog/catalog_v1.pyThis confirms that
AwareDatetimeis being used consistently for datetime fields across the project.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that AwareDatetime is used for all datetime fields in the project. rg --type python "Optional\[AwareDatetime\]"Length of output: 77
Script:
#!/bin/bash # Description: Verify that AwareDatetime is used for all datetime fields in the project. rg "Optional\[AwareDatetime\]" --glob "*.py"Length of output: 6531
70-71: Review the use ofConfigDictwithextra='allow'to ensure that allowing additional fields is intentional and secure.Also applies to: 100-102, 156-157, 190-191, 268-269, 333-334, 397-398, 491-492, 501-502, 588-589, 609-610, 672-673, 736-737, 801-802, 865-866, 929-930, 995-996, 1060-1061, 1124-1125, 1183-1184, 1243-1244, 1302-1303, 1361-1362, 1420-1421, 1459-1460, 1468-1469, 1484-1485, 1495-1496, 1521-1522, 1579-1580
dbt_artifacts_parser/parsers/manifest/manifest_v6.py (6)
15-17: Review the consistent use ofConfigDictwith appropriate settings across all models.Also applies to: 44-46, 52-54, 61-63, 69-71, 81-83, 88-90, 100-102, 142-144, 155-157, 190-192, 269-271, 335-337, 401-403, 432-434, 442-444, 457-459, 479-481, 487-489, 496-498, 506-508, 517-519, 524-526, 533-535, 576-578, 584-586, 593-595, 615-617, 679-681, 745-747, 812-814, 878-880, 944-946, 1011-1013, 1078-1080, 1144-1146, 1205-1207, 1267-1269, 1328-1330, 1389-1391, 1450-1452, 1490-1492, 1499-1501, 1515-1517, 1526-1528, 1552-1554, 1579-1581, 1612-1614
29-30: Ensure the updated regex pattern inconstris correct and matches the intended format.Verification successful
The regex pattern used in
constrcorrectly matches the intended UUID format.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correctness of the regex pattern used in `constr`. # Test: Check the regex pattern against expected UUID formats. echo 'b98791cb-6931-421b-aeb5-945dc062e419' | grep -P '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'Length of output: 155
9-9: Ensure all necessary imports from Pydantic are included for v2 compatibility.Verification successful
Ensure all necessary imports from Pydantic are included for v2 compatibility.
The current imports in
manifest_v6.pyare:
- AwareDatetime
- ConfigDict
- Field
- constr
These imports are consistent with Pydantic v2. No issues found.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that all necessary Pydantic imports are present for v2 compatibility. # Test: Check for missing imports in the file. rg --type python 'from pydantic import' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 125
Script:
#!/bin/bash # Description: Verify that all necessary Pydantic imports are present for v2 compatibility. # Test: Check for missing imports in the file. rg 'from pydantic import' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 142
206-224: Review the default factory methods for configurations to ensure they are correctly setting default values and behaviors.Also applies to: 286-304, 352-372, 633-650, 962-980, 1029-1048, 1161-1180, 1222-1241, 1345-1364, 1406-1425, 1630-1649, 286-304, 633-650, 962-980, 1029-1048, 1161-1180, 1222-1241, 1345-1364, 1406-1425, 1630-1649
Verification successful
The
default_factorymethods in themanifest_v6.pyfile appear to be correctly setting default values and behaviors for various configurations. The output shows multiple instances wheredefault_factoryis used to initialize configurations with specific default values.
TestConfig.model_validateDependsOn.model_validateDocs.model_validateSeedConfig.model_validateNodeConfig.model_validateMacroDependsOn.model_validateQuoting.model_validateSourceConfig.model_validateThese instances indicate that the default values are being set appropriately using the
model_validatemethod for different configuration classes.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correctness of default factory methods in configurations. # Test: Check for consistency and correctness in the default values set by factory methods. rg --type python 'default_factory' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 118
Script:
#!/bin/bash # Description: Verify the correctness of default factory methods in configurations. # Test: Check for consistency and correctness in the default values set by factory methods. rg 'default_factory' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 4301
22-22: Verify the correct usage ofAwareDatetimefor datetime fields to ensure timezone awareness.Also applies to: 447-447
Verification successful
The usage of
AwareDatetimefor datetime fields indbt_artifacts_parser/parsers/manifest/manifest_v6.pyappears to be correct. The fieldsgenerated_at,started_at,completed_at,max_loaded_at, andsnapshotted_atare all usingAwareDatetime.
generated_at: Optional[AwareDatetime] = '2022-07-26T13:02:44.173047Z'generated_at: Optional[AwareDatetime] = '2022-07-26T13:02:44.168999Z'started_at: Optional[AwareDatetime] = Nonecompleted_at: Optional[AwareDatetime] = Nonemax_loaded_at: AwareDatetimesnapshotted_at: AwareDatetimeScripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that `AwareDatetime` is used correctly across all datetime fields. # Test: Search for datetime field definitions not using `AwareDatetime`. rg --type python 'datetime' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 111
Script:
#!/bin/bash # Description: Verify that `AwareDatetime` is used correctly across all datetime fields. # Test: Search for datetime field definitions and check their types. rg --type py 'datetime' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 76
Script:
#!/bin/bash # Description: Verify the usage of `AwareDatetime` in the specified file. # Test: Search for `AwareDatetime` usage and check its context. rg --type py 'AwareDatetime' dbt_artifacts_parser/parsers/manifest/manifest_v6.pyLength of output: 454
231-232: Ensure that theDependsOnmodel's default factory is correctly populating themacrosandnodeslists.Also applies to: 311-312, 379-380, 656-657, 855-857, 1121-1123, 1186-1188, 1247-1249, 1370-1372, 1431-1433, 1556-1557, 1655-1657
dbt_artifacts_parser/parsers/manifest/manifest_v7.py (7)
15-17: Correct implementation ofConfigDictto enforce attribute restrictions inManifestMetadata.
44-46: Consistent use ofConfigDictacross various models to enforce attribute restrictions. Good adherence to the new Pydantic v2 configurations.Also applies to: 52-54, 61-63, 69-71, 77-79, 89-91, 101-103, 143-145, 156-158, 196-198, 276-278, 343-345, 413-415, 459-461, 474-476, 496-498, 504-506, 513-515, 523-525, 539-541, 546-548, 555-557, 598-600, 606-608, 613-615, 629-631, 637-639, 644-646, 671-673, 736-738, 806-808, 877-879, 947-949, 1017-1019, 1085-1087, 1156-1158, 1226-1228, 1291-1293, 1357-1359, 1422-1424, 1487-1489, 1593-1595, 1602-1604, 1618-1620, 1629-1631, 1656-1658, 1688-1690, 1726-1728
9-9: Ensure consistent import ofBaseModelfrompydantic.Verification successful
Ensure consistent import of
BaseModelfrompydantic.
- The import statement
from pydantic import BaseModelis found indbt_artifacts_parser/parsers/base.py.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that BaseModel is consistently imported from pydantic across all files. rg --type python "from pydantic import BaseModel"Length of output: 82
Script:
#!/bin/bash # Description: Verify that BaseModel is consistently imported from pydantic across all files. rg "from pydantic import BaseModel" --glob "*.py"Length of output: 119
22-22: EnsureAwareDatetimeis correctly used and compatible with the expected date format.Verification successful
Ensure
AwareDatetimeis correctly used and compatible with the expected date format.The search results show that
AwareDatetimeis used across multiple files in the codebase, including the file in question (manifest_v7.py). The usage pattern is consistent, withAwareDatetimebeing assigned ISO 8601 formatted date strings.
dbt_artifacts_parser/parsers/manifest/manifest_v7.py:22:generated_at: Optional[AwareDatetime] = '2022-09-14T20:35:15.346636Z'Given the consistent usage and the correct format of the date strings, it appears that
AwareDatetimeis being used correctly.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that AwareDatetime is correctly handled in the application. rg --type python "AwareDatetime"Length of output: 63
Script:
#!/bin/bash # Description: Verify that AwareDatetime is correctly handled in the application. rg --type py "AwareDatetime"Length of output: 10921
112-112: Check the regex pattern forseverityto ensure it correctly matches 'WARN' or 'ERROR' in various cases.Verification successful
The regex pattern for
severityis correctly defined and used in multiple versions of themanifestparser files, includingmanifest_v7.py. The patternr'^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$'is designed to match 'WARN' or 'ERROR' in various cases, which aligns with the intended functionality.
- Locations where the pattern is used:
dbt_artifacts_parser/parsers/manifest/manifest_v1.pydbt_artifacts_parser/parsers/manifest/manifest_v2.pydbt_artifacts_parser/parsers/manifest/manifest_v3.pydbt_artifacts_parser/parsers/manifest/manifest_v4.pydbt_artifacts_parser/parsers/manifest/manifest_v5.pydbt_artifacts_parser/parsers/manifest/manifest_v6.pydbt_artifacts_parser/parsers/manifest/manifest_v7.pydbt_artifacts_parser/parsers/manifest/manifest_v8.pydbt_artifacts_parser/parsers/manifest/manifest_v9.pydbt_artifacts_parser/parsers/manifest/manifest_v10.pydbt_artifacts_parser/parsers/manifest/manifest_v11.pydbt_artifacts_parser/parsers/manifest/manifest_v12.pyScripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that the regex pattern for severity is correctly validating the strings. rg --type python "constr(pattern=r'^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$')"Length of output: 118
Script:
#!/bin/bash # Description: Verify that the regex pattern for severity is correctly validating the strings. rg --type py "constr(pattern=r'^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$')"Length of output: 83
Script:
#!/bin/bash # Description: Search for the `constr` function and inspect for the specific pattern. rg --type py "constr"Length of output: 7433
Line range hint
1836-1914: Review theManifestV7class to ensure that all fields are correctly typed and default values are set appropriately.Verification successful
The
ManifestV7class fields appear to be correctly typed and have appropriate default values set. The fields include various types such asDict,Optional, andUnion, and they useFieldto provide descriptions and default values where necessary.
model_configis set withextra='forbid'.metadata,nodes,sources,macros,docs,exposures,metrics, andselectorsare all required fields with descriptions.disabled,parent_map, andchild_mapare optional fields with descriptions.The fields are well-defined and follow the expected structure.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the correct implementation of ManifestV7 class fields. rg --type python "ManifestV7"Length of output: 60
Script:
#!/bin/bash # Description: Verify the correct implementation of ManifestV7 class fields. rg --type py "class ManifestV7"Length of output: 121
Script:
#!/bin/bash # Description: Extract the full definition of the ManifestV7 class. ast-grep --lang python --pattern $'class ManifestV7(BaseParserModel):\n $$$' dbt_artifacts_parser/parsers/manifest/manifest_v7.pyLength of output: 7872
29-29: Ensure the regex pattern foruser_idis thoroughly tested to match the intended format.
| extra='allow', | ||
| ) |
There was a problem hiding this comment.
Allowing extra fields in ExternalTable might lead to unintended data entries. Consider if this is necessary or if it should be more restrictive.
- model_config = ConfigDict(extra='allow')
+ model_config = ConfigDict(extra='forbid')Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| extra='allow', | |
| ) | |
| extra='forbid', |
| severity: Optional[ | ||
| constr(pattern=r'^([Ww][Aa][Rr][Nn]|[Ee][Rr][Rr][Oo][Rr])$') | ||
| ] = 'ERROR' |
There was a problem hiding this comment.
Consider adding more specific validation or constraints to other fields where applicable to enhance data integrity and error handling.
Also applies to: 111-112, 275-291, 602-618, 856-872
| from enum import Enum | ||
| from typing import Any, Dict, List, Optional, Union | ||
|
|
||
| # pylint: disable=no-name-in-module | ||
| from pydantic import Extra, Field, constr | ||
| from pydantic import AwareDatetime, ConfigDict, Field, constr |
There was a problem hiding this comment.
Ensure consistent import of BaseModel from pydantic.
It appears that BaseModel is not explicitly imported, yet it is used throughout the file. This could lead to a NameError at runtime. Consider adding the following import statement:
from pydantic import BaseModel| config: Optional[TestConfig] = Field( | ||
| default_factory=lambda: TestConfig.model_validate( | ||
| { | ||
| 'enabled': True, | ||
| 'alias': None, | ||
| 'schema': 'dbt_test__audit', | ||
| 'database': None, | ||
| 'tags': [], | ||
| 'meta': {}, | ||
| 'materialized': 'test', | ||
| 'severity': 'ERROR', | ||
| 'store_failures': None, | ||
| 'where': None, | ||
| 'limit': None, | ||
| 'fail_calc': 'count(*)', | ||
| 'warn_if': '!= 0', | ||
| 'error_if': '!= 0', | ||
| } | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Tip
Codebase Verification
The recent script execution did not find any instances of default_factory in the test files. This indicates that the default_factory usage in the models might not be covered by unit tests.
- Ensure that the
default_factoryusage in the models is covered by unit tests to avoid regression issues.
Analysis chain
The use of default_factory in Field for complex defaults is correctly implemented. Ensure that these defaults are covered by unit tests to avoid regression issues.
Also applies to: 275-294, 602-621, 856-875
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify unit tests cover default_factory usage in models.
rg --type python "default_factory" -A 3 | rg "test"
Length of output: 80
Script:
#!/bin/bash
# Description: Verify unit tests cover default_factory usage in models.
rg --type py "default_factory" -A 3 | rg "test"
Length of output: 789
Script:
#!/bin/bash
# Description: Verify unit tests cover default_factory usage in models.
# Searching for test files
test_files=$(fd --type f --name '*test*.py')
# Searching for default_factory usage in test files
for file in $test_files; do
echo "Checking $file"
rg "default_factory" "$file"
done
Length of output: 298
Script:
#!/bin/bash
# Description: Verify unit tests cover default_factory usage in models.
# Searching for test files
test_files=$(fd 'test.*\.py')
# Searching for default_factory usage in test files
for file in $test_files; do
echo "Checking $file"
rg "default_factory" "$file"
done
Length of output: 685
| dbt_schema_version: Optional[str] = None | ||
| dbt_version: Optional[str] = '1.8.0a1' | ||
| generated_at: Optional[str] = None | ||
| invocation_id: Optional[Optional[str]] = None | ||
| invocation_id: Optional[str] = None |
There was a problem hiding this comment.
Tip
Codebase Verification
The search results indicate that the fields invocation_id, project_name, user_id, send_anonymous_usage_stats, and adapter_type are consistently defined as optional across multiple versions and files. However, the context provided does not show any specific handling or checks for these fields when they are not set, which raises concerns about their necessity and potential impact on the application's functionality.
Consider reviewing the necessity of these fields being optional. If they are critical for the application's operation, it might be more appropriate to make them mandatory.
invocation_id: Found in multiple files and versions, often set toNone.project_name: Found inmanifest_v11.py,manifest_v10.py, andmanifest_v12.py, set toNone.user_id: Found in multiple files and versions, often set toNone.send_anonymous_usage_stats: Found in multiple files and versions, often set toNone.adapter_type: Found in multiple files and versions, often set toNone.
Please review these fields and consider making them mandatory if they are essential for the application's operation.
Analysis chain
Review the use of optional fields for critical metadata.
Consider making critical fields like project_name and user_id non-optional if they are essential for the operation of the application.
Also applies to: 24-25, 29-29, 32-32, 35-35
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that the use of Optional fields is justified and that there are no instances where these fields should be mandatory.
# Test: Search for usage of these fields in the codebase to ensure they are handled properly.
rg --type python 'invocation_id|project_name|user_id|send_anonymous_usage_stats|adapter_type'
Length of output: 126
Script:
#!/bin/bash
# Description: Verify that the use of Optional fields is justified and that there are no instances where these fields should be mandatory.
# Test: Search for usage of these fields in the codebase to ensure they are handled properly.
rg --type py 'invocation_id|project_name|user_id|send_anonymous_usage_stats|adapter_type'
Length of output: 7450
Script:
#!/bin/bash
# Description: Verify how the optional fields are used in the codebase to determine if they should be mandatory.
# Test: Search for usage of these fields in the codebase to ensure they are handled properly.
rg 'invocation_id|project_name|user_id|send_anonymous_usage_stats|adapter_type' --type py
Length of output: 7450
Script:
#!/bin/bash
# Description: Verify how the optional fields are used in the codebase to determine if they should be mandatory.
# Test: Search for usage of these fields in the codebase to ensure they are handled properly.
rg 'invocation_id|project_name|user_id|send_anonymous_usage_stats|adapter_type' --type py -A 5 -B 5
Length of output: 45641
|
@OnkarVO7 I haven't caught up with updates of OpenMetadata for a while. But, if it still use the package, please make sure the compatibility in pydantic. We won't support pydantic v1 any more after v0.7.0. https://github.com/yu-iskw/dbt-artifacts-parser/releases/tag/v0.7.0 |
|
Thanks for informing @yu-iskw we'll be updating to pydantic v2 by next release. |
|
@OnkarVO7 Sounds awesome. Please let me know, if there is anything I can help. |
Overview
We migrate pydantic from v1 to v2.
Supported Versions and Compatibility