-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source TikTok Marketing : Access token is null #14299 #14461
Source TikTok Marketing : Access token is null #14299 #14461
Conversation
… when we have multiple options with the same key on spec
…ave a unique path values
because this change affects many connectors, please test that it does not break any GA and Beta connectors and also verify that it does not break jobs after a user upgrades to a platform version |
@@ -178,6 +178,7 @@ public static List<String> getSortedSecretPaths(final JsonNode spec) { | |||
.anyMatch(field -> field.getKey().equals(JsonSecretsProcessor.AIRBYTE_SECRET_FIELD))) | |||
.stream() | |||
.map(JsonPaths::mapJsonSchemaPathToJsonPath) | |||
.distinct() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you add a unit test for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added unit tests. will attach screenshots with failed tests without this change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you add in the javadoc comment that it will not return duplicates?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added javadoc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need tests here that show the problem and then how this fixes it.
added unit tests
…acement' into osheg/bug-with-json-secrets-replacement
hi @cgardens, I have added unit tests for the described case, and also updated the description with failed unit tests without the fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks! the test is great.
can you point to what line of code causes things to work in an unexpected way when there are duplicates? this is still not intuitive to me, and i want to understand where the brittleness is coming from.
@@ -178,6 +178,7 @@ public static List<String> getSortedSecretPaths(final JsonNode spec) { | |||
.anyMatch(field -> field.getKey().equals(JsonSecretsProcessor.AIRBYTE_SECRET_FIELD))) | |||
.stream() | |||
.map(JsonPaths::mapJsonSchemaPathToJsonPath) | |||
.distinct() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you add in the javadoc comment that it will not return duplicates?
@cgardens in case |
updated javadoc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on local testing and my understanding of the problem space this looks good to go.
Tik Tok access token are broken so we hope to see a more different error here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank, @alexandr-shegeda ! That explanation makes sense to me too. I think we are good to go on this.
@cgardens The issue is because this duplicated are secrets. Here is what happens:
Edit: Sorry I didn't saw that the explaination was post already |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
triple approve?
What
After recent changes introduced in this PR logic related to path recalculation for JSON nodes we faced an issue for connectors that contain oneOf elements that have the same field names. So, for described case
getSortedSecretPaths()
returned duplicates, which lead to incorrect postprocessing during secret replacement.How
Made
getSortedSecretPaths()
return a unique values.Recommended reading order
SecretsHelpers.java
🚨 User Impact 🚨
Without these changes, users are not able to configure a proper connection using multiple auth options on spec.
Pre-merge Checklist
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereTests
Before fix:
After fix:
Integration
Put your integration tests output here.
Acceptance
Put your acceptance tests output here.