Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TDL-20804 Refactor the tap code #55

Merged
merged 20 commits into from
May 18, 2023
Merged

TDL-20804 Refactor the tap code #55

merged 20 commits into from
May 18, 2023

Conversation

prijendev
Copy link
Contributor

@prijendev prijendev commented Oct 7, 2022 •
edited
Loading

Description of change

  • Reduced record duplication.
  • Added integration test for interrupted sync.
  • Optimized parent-child stream, by making common parent API calls.
  • Handled 402 error (Payment require) in discover mode for a free plan Mixpanel account.

Manual QA steps

  • Run and verified the parent-child stream independently and by selecting both.
  • For windowing streams, no duplication records are written.
  • Run tap with state and without state for incremental streams.
  • For free account tap does not break while generating catalog.

Risks

Rollback steps

  • revert this branch

dbshah1212
dbshah1212 reviewed Oct 11, 2022
View reviewed changes
tap_mixpanel/streams.py Outdated Show resolved Hide resolved
dbshah1212
dbshah1212 reviewed Oct 11, 2022
View reviewed changes
tap_mixpanel/streams.py Outdated Show resolved Hide resolved
@kethan1122 kethan1122 changed the base branch from TDL-20804-format-code-with-standard-formatters to master May 4, 2023 07:07
@kethan1122 kethan1122 removed the request for review from bryantgray May 4, 2023 09:28
RushiT0122
RushiT0122 reviewed May 8, 2023
View reviewed changes
tap_mixpanel/schema.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 reviewed May 8, 2023
View reviewed changes
tap_mixpanel/schema.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 reviewed May 8, 2023
View reviewed changes
tests/tap_tester/base.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 reviewed May 8, 2023
View reviewed changes
tests/tap_tester/base.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 reviewed May 8, 2023
View reviewed changes
tap_mixpanel/streams.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 requested changes May 10, 2023
View reviewed changes
Copy link

@RushiT0122 RushiT0122 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in-line.

tests/unittests/test_error_handling.py Outdated Show resolved Hide resolved
tests/unittests/test_error_handling.py Outdated Show resolved Hide resolved
tests/unittests/test_main.py Outdated Show resolved Hide resolved
tests/unittests/test_medium_client.py Outdated Show resolved Hide resolved
tests/unittests/test_transform.py Outdated Show resolved Hide resolved
tests/unittests/test_transform_event_times.py Outdated Show resolved Hide resolved
RushiT0122
RushiT0122 approved these changes May 15, 2023
View reviewed changes
Copy link

@RushiT0122 RushiT0122 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested in-line.

@kethan1122 kethan1122 merged commit f97ce37 into master May 18, 2023
@kethan1122 kethan1122 deleted the TDL-20805-refactor-tap-code branch May 18, 2023 10:55
This was referenced May 19, 2023
Closed
Closed
@kethan1122 kethan1122 restored the TDL-20805-refactor-tap-code branch May 22, 2023 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kethan1122 kethan1122 kethan1122 left review comments

@RushiT0122 RushiT0122 RushiT0122 approved these changes

@savan-chovatiya savan-chovatiya savan-chovatiya approved these changes

@dbshah1212 dbshah1212 dbshah1212 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@prijendev @dbshah1212 @savan-chovatiya @RushiT0122 @kethan1122 @NevilParikh14