🐛 Source Cart: fixing of testing for a lot of data

[antixar]

What

This source generates new data constantly. And the full_refresh test can catch new changes between the first and second syncs.
We added the config field 'end_date' for loading not all data from some date but and for some necessary period. This new field is optional and by default the connector will load all.
for CI the following period settings were set:

    "start_date": "2021-07-01T00:00:00Z",
    "end_date": "2021-07-20T00:00:00Z",

Dockerfile was migrated to Alpine base image too.

Recommended reading order

1. source_cart/streams.py
2. Dockerfile

Pre-merge Checklist

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions
• Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1140656694
✅ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1140656694

[midavadim]

I only have the last question:

If 'current' users are be able to use their 'old' spec files without end_date attr and is not a problem, then I 100% fine with this approach.

Another words - will the customer be able to run syncs with old specs on this updated source?

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1158656179
❌ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1158656179

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169773448
❌ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169773448

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169890716
❌ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169890716

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169919610
❌ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1169919610

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1170135933
✅ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1170135933

[antixar]

@sherifnada , about this test option 'end_date'. I added to its spec because in my opinion using the flag "additionalProperties"=True is a wrong way. All input parameters must be validated strictly and it will reduce number of failures in the future.
And we want to discuss some alternative approach for similar test cases in the future when we need to add specific input attributes for tests without 'spec' changes.

Now I see that Marcos Marx set this flag to 'True' thus I can remove the 'end_date' description from the spec.

[antixar]

/test connector=connectors/source-cart

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1170284924
✅ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1170284924

[antixar]

/publish connector=connectors/source-cart run-tests=false

Error: No ref found for: antixar/5346-cart-com-ci-failure

[antixar]

/publish connector=connectors/source-cart run-tests=false

🕑 connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1171570460
✅ connectors/source-cart https://github.com/airbytehq/airbyte/actions/runs/1171570460