You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
dbt currently accepts a command line flag, --partial-parse, which enables partial parsing of a dbt project. Two changes are required to make this flag ready for primetime:
The flag should be moved to the config: block in the profiles.yml file
The partial parse .pickle file should record which version of dbt it was produced with, and dbt should not try to re-use a partial_parse.pickle file across versions of dbt.
Moving --partial-parse to the profile
The big idea here is that --partial-parse is cumbersome to apply in repeated invocations of dbt run. By supporting this flag in the config: block of the profiles.yml file, users can opt-in to partial parsing without needing to specify the flag in every invocation of dbt. Partial parsing should remain an opt-in functionality.
dbt may preserve the --partial-parse flag (and optionally support a --no-partial-parse flag) which overrides the behavior defined in the profiles.yml config. This flag is not strictly necessary to support, but may be useful especially in debugging contexts.
Recording dbt versions in the partial_parse.pickle file
The partial_parse.pickle file should be written with a dbt_version: key which provides the dbt version that the pickle file was written with. If the version recorded in the .pickle file does not match the version of dbt that is running, then dbt should not attempt to partial-parse the project.
The text was updated successfully, but these errors were encountered:
Describe the feature
dbt currently accepts a command line flag,
--partial-parse
, which enables partial parsing of a dbt project. Two changes are required to make this flag ready for primetime:config:
block in theprofiles.yml
filepartial_parse.pickle
file across versions of dbt.Moving --partial-parse to the profile
The big idea here is that
--partial-parse
is cumbersome to apply in repeated invocations ofdbt run
. By supporting this flag in theconfig:
block of theprofiles.yml
file, users can opt-in to partial parsing without needing to specify the flag in every invocation of dbt. Partial parsing should remain an opt-in functionality.dbt may preserve the
--partial-parse
flag (and optionally support a--no-partial-parse
flag) which overrides the behavior defined in theprofiles.yml
config. This flag is not strictly necessary to support, but may be useful especially in debugging contexts.Recording dbt versions in the
partial_parse.pickle
fileThe
partial_parse.pickle
file should be written with adbt_version:
key which provides the dbt version that the pickle file was written with. If the version recorded in the.pickle
file does not match the version of dbt that is running, then dbt should not attempt to partial-parse the project.The text was updated successfully, but these errors were encountered: