[CT-2721] [Feature] support specify partial parsing file location

[ChenyuLInx]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

All inputs for dbt-core should be configurable independently, with sane default when necessary. This makes running dbt-core on cloud easier to reason about. Because you can clearly configure all of your input for a dbt invocation.

Currently dbt-core will try to load up partial_parse.msgpack from the target dir(link). There's no way to configure it without configure target dir to the same place.

We should provide a parameter(for all commands that does parsing) to being able to specify location to load partial_parse.msgpack, and if not specified dbt-core will fall back to use the current target dir.

Describe alternatives you've considered

Not do anything.

Who will this benefit?

No response

Are you interested in contributing this feature?

No response

Anything else?

No response

[jtcohen6]

@ChenyuLInx Given how internal this is to dbt-core, I'd love if there were a way to support configuring this without another flag/parameter. Ideas:

• Support passing this into dbtRunner → @requires.manifest programmatically
• Support this as an env var (without associated CLI flag) - similar to configurable location for hub / package registry

[ChenyuLInx]

@jtcohen6 How does create hidden parameter like what we do for deprecated params sounds?
User don't get it from the help text(they could use it if they really want). And it follows our existing patterns.

[jtcohen6]

@ChenyuLInx Adding this as a parameter with hidden=true feels like a reasonable compromise. Let's do it