Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1568 Google Ads best practices #1726

Merged
merged 19 commits into from
Jan 23, 2021

Conversation

vitaliizazmic
Copy link
Contributor

What

Apply connector best practices for Salesforce source

How

  • Updating check_config method: receiving accounts directly from Ads API instead using read_cmd method of source.
  • Checking Customer IDs format, adding pattern for customer_ids field in spec.json.
  • Checking syncing REPORTS. Creating check_internal method to verify to prevent trying to run a report against a manager account
  • Disabling syncing for incompatible fields in REPORT streams. You can find detail information about reports here.
  • Checking hitting rate limits. Singer tap doesn't use backs off, but it has custom solution
  • Implementing incremental mode

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jan 20, 2021

/test connector=source-google-adwords-singer

🕑 source-google-adwords-singer https://github.com/airbytehq/airbyte/actions/runs/497309687
❌ source-google-adwords-singer https://github.com/airbytehq/airbyte/actions/runs/497309687

from base_python import AirbyteLogger
from base_singer import SingerSource, SyncMode, SyncModeInfo
from googleads import adwords, oauth2
from tap_adwords import VERSION


class SourceGoogleAdwordsSinger(SingerSource):
Copy link
Contributor

@yevhenii-ldv yevhenii-ldv Jan 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use BaseSingerSourse, this will slightly reduce the amount of code, for example, it will remove the disсover_сmd and the check_config functions, WDYT?

accounts = self._get_accounts(logger, sdk_client, selector)
if not accounts:
return AirbyteConnectionStatus(status=Status.FAILED)
return AirbyteConnectionStatus(status=Status.SUCCEEDED)
except Exception as e:
return AirbyteConnectionStatus(status=Status.FAILED, message=f"{str(e)}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of BaseSingerSource will display a universal and understandable wording in UI in case of an error.

@sherifnada sherifnada linked an issue Jan 21, 2021 that may be closed by this pull request
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with two comments on messages displayed to the user. once those are fixed we should be ready to go

}
accounts = self._get_accounts(logger, sdk_client, selector)
if not accounts:
return AirbyteConnectionStatus(status=Status.FAILED)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should always have a message set if the connection check fails. What should the error message here say?

logger.log_by_prefix(f"Unable to sync {stream} with the manager account {customer_id}", "ERROR")
sys.exit(1)
else:
logger.log_by_prefix(f"Unable to sync with the provided customer id {customer_id}", "ERROR")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this log also specify a reason why it's failing? What should the user who sees this message do?

@sherifnada sherifnada linked an issue Jan 23, 2021 that may be closed by this pull request
@sherifnada
Copy link
Contributor

created #1805 with follow up actions for this PR

@sherifnada sherifnada merged commit 01866fb into master Jan 23, 2021
@sherifnada sherifnada deleted the vitalii/1568_google_ads_best_practices branch January 23, 2021 20:08
@sherifnada sherifnada linked an issue Jan 23, 2021 that may be closed by this pull request
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants