-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New Source: Display&Video 360 #11828
New Source: Display&Video 360 #11828
Conversation
Thanks for creating this connector @iberchid did you have the chance to run the integration test? |
2255499
to
ca2dfe6
Compare
Hi @marcosmarxm , I added the results of the integrations tests to the description. The docker image gets built without name: Do you have any suggestions? |
What command are you using to build the image? You can run |
Hi @marcosmarxm The docker image was not tagged with the connector's name, so I did it manually and it worked. I am using I still have 4 failing acceptance tests, could you give me some guidance here ? Here is the output of my check method, I do not understand why the check test fails Here are more details about the errors: |
Sorry the long delay in reply here @iberchid did you solve the problem with acceptance tests? do you need any help? |
Hello @marcosmarxm , not yet, I still have these 4 failing tests. |
Hi @marcosmarxm , any hints? |
sorry @iberchid can you send me dm in slack so I can heplp you? |
Thanks, I wrote you on slack @marcosmarxm |
@iberchid let me know when you submit the new code with the changes! |
@iberchid please share unit and integration tests |
Hi @marcosmarxm below are the results: I still have 2 tests still failing, attached is the detailed output: Do these tests only take the name of the stream and try to read all the available fields in it? Because DV360 in a bit special, not all fields within the same stream can be fetched at the same time, many are not compatible with each other. |
1st error: there is a stream the connector is not getting any record |
def chunk_date_range( | ||
start_date: str, | ||
field: str, | ||
end_date: str = None, | ||
range_days: int = None, | ||
) -> Iterable[Mapping[str, any]]: | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please run ./gradlew format
and commit only your files this is not the default formating style.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am still facing some issues with blackFormat
. I have python v3.8
and installed sudo pip install --upgrade click==8.0.1
Attached are the errors I am getting, do you have any hints for me? Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marcosmarxm any hints?
Delete old folder
… metrics + add required fields in unique_reach_audience stream
9962c55
to
be65546
Compare
/publish connector=connectors/source-dv-360 run-tests=false
if you have connectors that successfully published but failed definition generation, follow step 4 here |
Sorry the long delay to merge this @iberchid thanks for the contribution |
* Add dv360 connector source * update query methods * sanitize fields and fetch only fields in config_catalog * Read and check methods + rmv extra spaces from schema fields * add timezone in query + log error message in Read method * start incr streamand tests * Add unit tests and documentation * rmv chunck date method * Add incremental stream * Add dv360 connector * Add dv360 connector * rmv .hbs from unit test file * Add BOOTSTRAP.md file * Delete airbyte-integrations/connectors/source-dv360 directory Delete old folder * Add Docs * update config_Catalog * filter last row in case of an additional empty row for the sum of the metrics + add required fields in unique_reach_audience stream * Update read methode by removing summary row in the case of the standard report * Update invalid_config and spec files * Add first state msg in Read method * Code format * format connector * add to seed file * auto-bump connector version [ci skip] Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
* Add dv360 connector source * update query methods * sanitize fields and fetch only fields in config_catalog * Read and check methods + rmv extra spaces from schema fields * add timezone in query + log error message in Read method * start incr streamand tests * Add unit tests and documentation * rmv chunck date method * Add incremental stream * Add dv360 connector * Add dv360 connector * rmv .hbs from unit test file * Add BOOTSTRAP.md file * Delete airbyte-integrations/connectors/source-dv360 directory Delete old folder * Add Docs * update config_Catalog * filter last row in case of an additional empty row for the sum of the metrics + add required fields in unique_reach_audience stream * Update read methode by removing summary row in the case of the standard report * Update invalid_config and spec files * Add first state msg in Read method * Code format * format connector * add to seed file * auto-bump connector version [ci skip] Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
What
Relates to #10850
Closed #2984
How
A connector integration for a new source (DV360).
Recommended reading order
spec.py
source.py
streams.py
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/SUMMARY.md
docs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereTests
Unit
Integration
Acceptance
Put your acceptance tests output here.
┆Issue is synchronized with this Monday item by Unito