Source TikTok Marketing : Access token is null

[marcosmarxm]

This Github issue is synchronized with Zendesk:

Ticket ID: #1385
Priority: normal
Group: User Success Engineer
Assignee: Marcos Marx

Original ticket description:

• Is this your first time deploying Airbyte?: Yes
• OS Version / Instance: Amazon Linux 2
• Memory / Disk: 8Gb / 40Gb
• Deployment: Docker
• Airbyte Version: v0.39.28-alpha
• Source name/version: TikTok Marketing API (0.1.14)
• Destination name/version: None
• Step: The issue is happening creating new TikTok Marketing API source
• Description: While trying to save new data source (TikTok Marketing API) this error is returned :

TiktokException({'code': 40104, 'message': 'Access token is null, you should set it in http header with key Access-Token.', 'request_id': '202206301058430102452480431E019A05', 'data': {}})

The access key is however well informed in the associated field, with you an idea to solve this problem?

Have a nice day !

[Discourse post]

[marcosmarxm]

I was able to reproduce issue using Airbyte version 0.39.29 and Tiktok Marketing 0.1.14

[marcosmarxm]

Comment made from Zendesk by Marcos Marx on 2022-06-30 at 14:30:

I tried with our integration account and got same issue Jules. 
I created the issue #14299 any update here I'll return to you

[marcosmarxm]

The request from the web UI sent the acess token, but looking at the spec.json the access token is null. I didn't found the reason why this is happening.

[midavadim]

I have confirmed the same issue in my local OSS deployment and on CLOUD.

Important: issue also happens with old versions of tiktok connector:
latest - 0.1.14, tried 0.1.12 and 0.1.10

However, connector is working when to run commands in CLI:

(.venv) vratniuk@KBP1-LHP-A10380 ~/air/airbyte/airbyte-integrations/connectors/source-tiktok-marketing (master)$ python main.py check --config secrets/prod_config.json
{"type": "LOG", "log": {"level": "INFO", "message": "Check succeeded"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "SUCCEEDED"}}

(.venv) vratniuk@KBP1-LHP-A10380 ~/air/airbyte/airbyte-integrations/connectors/source-tiktok-marketing (master)$ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-tiktok-marketing:dev check --config /secrets/prod_config.json
{"type": "LOG", "log": {"level": "INFO", "message": "Check succeeded"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "SUCCEEDED"}}

THE PROBLEM WITH SANDBOX account (from API side)
(.venv) vratniuk@KBP1-LHP-A10380 ~/air/airbyte/airbyte-integrations/connectors/source-tiktok-marketing (master)$ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-tiktok-marketing:dev check --config /secrets/config.json
{"type": "LOG", "log": {"level": "ERROR", "message": "Check failed"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "FAILED", "message": "TiktokException({'code': 51000, 'message': 'Service error: error_code=1601 cds_key=EGRESS|ad.open.open_api_i18n:sandbox:sg1:|ad.platform.i18n_bc_rpc:default::|CheckUserAssetOwner|canary| onAccessDeny, authentication error=, tag=, reason=, authorization error=AUTHORIZATION_DENY, rip=/opt/tiger/toutiao/var/service/ad.open.open_api_i18n.mesh/rpc.egress.sock', 'request_id': '2022070213144201024524414421069D0C', 'data': {}})"}}

[midavadim]

The problem indeed happens because source container receives config file with empty access_token in it, check example:

{"credentials":{"access_token":"","auth_type":"oauth2.0","secret":"","app_id":""},"start_date":"2021-04-01"}

as for me it looks like a problem with UI.

Edit benmoriceau: sanitize secrets

[marcosmarxm]

@midavadim I was able to get the access token if I use order in the spec class.
Adding the order for both OAuth and Sandbox:

airbyte/airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/spec.py

Lines 16 to 36 in 5cdf2f4

	class OauthCredSpec(BaseModel):
	class Config:
	title = "OAuth2.0"
	auth_type: str = Field(default="oauth2.0", const=True, order=0)
	app_id: str = Field(title="App ID", description="The Developer Application App ID.", airbyte_secret=True)
	secret: str = Field(title="Secret", description="The Developer Application Secret.", airbyte_secret=True)
	access_token: str = Field(title="Access Token", description="Long-term Authorized Access Token.", airbyte_secret=True)
	class SandboxEnvSpec(BaseModel):
	class Config:
	title = "Sandbox Access Token"
	auth_type: str = Field(default="sandbox_access_token", const=True, order=0)
	# it is string because UI has the bug https://github.com/airbytehq/airbyte/issues/6875
	advertiser_id: str = Field(
	title="Advertiser ID", description="The Advertiser ID which generated for the developer's Sandbox application."
	)
	access_token: str = Field(title="Access Token", description="The long-term authorized access token.", airbyte_secret=True)

[alexandr-shegeda]

I will take a look at the issue from the Java side

[alexandr-shegeda]

Found the root cause of the issue, looks like it was introduced in the and of June in the scope of this PR

[misteryeo]

Oncall team is investigating this issue.

[alexandr-shegeda]

FYI @misteryeo the fix is on the airbyte review
cc @grishick @sherifnada