-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source Chargebee: cannot sync transaction
objects
#17657
Comments
Hey @kevinyeung-3p, could you provide the complete logs from when this happened in case there is useful information before or after this snippet? |
I was not able to reproduce this issue - can you confirm that your other connections are syncing successfully? |
Hi @sh4sh , thanks for your reply. I've added more log lines surround the error. Meanwhile, I'll also try syncing
|
MySQL to Postgres, also Salesforce to Postgres sync beautifully. |
Just an update. I changed my Chargebee source setting start date to
|
@kevinyeung-3p Very weird, I tried with those dates and earlier, all the way back to 2021 but it seems to work fine for my Chargebee data. It sounds like their database is not providing this data for some reason, could you open a support ticket with them to see if they can take a look? |
transaction
objects
@kevinyeung-3p We are seeing the same, but in our case not even 2022-12-01 works. We'll also open a support ticket with them, but I would appreciate if any known workaround was shared here. |
I wrote a quick Python program to test the transaction/ endpoint and see if the problem reproduces but it doesn't. This makes me think that the connector is sending something that makes the Chargebee server trip over. |
I've been able to reproduce the issue. It happens because the connector sends: 'sort_by[asc]': 'date' instead of: 'sort_by[asc]': 'updated_at'. Line 310 in streams.py harcodes the sort_field to date: airbyte/airbyte-integrations/connectors/source-chargebee/source_chargebee/streams.py Line 310 in a3c00fa
It was introduced in this PR by @mustangJaro: Why do we need that particular line? Let me explain: By default sort_field is assigned here: airbyte/airbyte-integrations/connectors/source-chargebee/source_chargebee/streams.py Line 171 in a3c00fa
And this assignment seems to me the only thing we should do. The default cursor_field for Transactions is 'updated_at', which works well (doesn't make Chargebee's server to freak out) for the sort[asc] parameter. |
@sh4sh I don't understand why in your case it's working. If I might ask, what version of the connector are you using? |
I have no issues with changing this again. I struggled to find the proper fields in Chargebees documentation. If you found the 'updated-at' field then that's fantastic news! |
Hi @mustangJaro I think the right thing to do is to let line 171 do its job and remove line 310. What do you think? |
I removed line 310 and rebuilt the container in our self-hosted deployment, and the connector has been working for > 1 week with no issues. I wonder what the intention of the original author with that line was. I bet that the issue is that there's no "updated_at" field for dates before 2016, and that's why the sort field had to be hardcoded to "created_at" or "date". |
Fixed in #21688 |
Environment
Current Behavior
Keep getting
chargebee.api_error.InvalidRequestError
and backoffs and never succeeded.Expected Behavior
Syncs Chargebee transactions to a table.
Logs
Steps to Reproduce
Are you willing to submit a PR?
Yes, if there's guidance given.
The text was updated successfully, but these errors were encountered: