🎉 New Source: Tyntec SMS [low-code cdk] #18883

michizhou · 2022-11-03T00:58:32Z

What

Developing new connector for the source: Tyntec SMS
Documentation: https://api.tyntec.com/reference/sms/current.html#sms-api

Resolves:
🎉 New Source: Tyntec SMS

How

Developed using Low-Code CDK

🚨 User Impact 🚨

No breaking changes. Just the addition of a new source.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

marcosmarxm · 2022-11-04T01:16:28Z

Hello @michizhou, Marcos from Airbyte here 👋 . We received more than 25 new contributions along the weekend. One is yours 🎉 thank so much for! Our team is limited and maybe the review process can take longer than expected. As described in the Airbyte's Hacktoberfest your contribution was submitted before November 2nd and it is eligible to win the prize. The review process will validate other requirements. I ask to you patience until someone from the team review it.

Because I reviewed some contributions for Hacktoberfest so far I saw some common patterns you can check in advance:

Make sure you have added connector documentation to /docs/integrations/
Remove the file catalog from /integration_tests
Edit the sample_config.json inside /integration_tests
For the configured_catalog you can use only json_schema: {}
Add title to all properties in the spec.yaml
Make sure the documentationUrl in the spec.yaml redirect to Airbyte's future connector page, eg: connector Airtable the documentationUrl: https://docs.airbyte.com/integrations/sources/airtable
Review now new line at EOF (end-of-file) for all files.

If possible send to me a DM in Slack with the tests credentials, this process will make easier to us run integration tests and publish your connector. If you only has production keys, make sure to create a bootstrap.md explaining how to get the keys.

marcosmarxm · 2022-11-04T08:52:50Z

Hello! I'm going to be out of the office this Friday and won't be able to review your contribution again today, I return next Monday. So far, most contributions look solid and are almost done to be approved. As said in Chris' comment all contributions made before 2-November are eligible to receive the prize and have 2 weeks to merge the contributions. But I ensure next week we're going to have your contribution merged. If you have questions about the implementation you can send them in #hacktoberfest-2022 in Airbyte's Slack.

Sorry the inconvenience and see you again next week, thank you so much for your contribution!

michizhou · 2022-11-07T09:35:46Z

@marcosmarxm Thanks for taking your time to review this. I have DM'ed you on slack with test credentials and to the best of my knowledge I believe I have fulfilled all the checklist items:

Make sure you have added connector documentation to /docs/integrations/
Remove the file catalog from /integration_tests
Edit the sample_config.json inside /integration_tests
For the configured_catalog you can use only json_schema: {}
Add title to all properties in the spec.yaml
Make sure the documentationUrl in the spec.yaml redirect to Airbyte's future connector page, eg: connector Airtable the documentationUrl: https://docs.airbyte.com/integrations/sources/airtable
Review now new line at EOF (end-of-file) for all files.

marcosmarxm

Some comments.

marcosmarxm · 2022-11-08T17:20:14Z

airbyte-integrations/connectors/source-tyntec-sms/source_tyntec_sms/tyntec_sms.yaml

+  messages_stream:
+    $ref: "*ref(definitions.base_stream)"
+    $options:
+      name: "messages"
+      primary_key: "doneDate"
+      path: "/messaging/v1/messages/{{ config['request_id'] }}"


Why you don't use the other endpoint to collect all request_id messgages?
https://api.tyntec.com/reference/sms/current.html#sms-api-Send%20SMS%20(GET)

Are you sure the primary key is DoneDate?

@marcosmarxm Are you suggesting the linked endpoint is to be used to create a new stream to get request_id, which would then be passed onto the messages stream as a parameter? If so, how could I set up such a stream and the messages stream so they link up properly? The linked endpoint also seems to be designed to send a single SMS message and return the request_id of the sent message, so I don't know if it can fetch request_id values of previously sent messages. I suppose we can make it work if we configure it so every SMS message is sent via this new stream and the request_id is stored or immediately used to trigger the messages stream.

As for the primary key being doneDate, the response doesn't have a designated key so I picked it as a reasonable choice. Let me know if you think a different field (or even no field) is more suitable from the response fields: https://api.tyntec.com/reference/sms/current.html#sms-api-restmessageresponse

Exactly. Please read the docs about StreamSlicer: https://docs.airbyte.com/connector-development/config-based/understanding-the-yaml-file/stream-slicers check other connectors implementation using ListStreamSlicer.

Check Survey Sparrow connector, it uses a List StreamSlicer can help you.

marcosmarxm · 2022-11-08T17:20:55Z

airbyte-integrations/connectors/source-tyntec-sms/source_tyntec_sms/spec.yaml

+    request_id:
+      type: string
+      title: SMS Message Request ID
+      description: The unique identifier provided for each SMS messaging request.
+      order: 1


I think this should be removed and create a new stream to retrieve all requests id/messages and use a stream slices with the message stream.

@marcosmarxm The new stream you suggested to retrieve all request IDs/messages will likely require the addition of new fields to, from, and message, which are required for the linked endpoint. Is this acceptable?

You can implement to from but check my previous comment.

michizhou · 2022-11-10T18:50:42Z

@marcosmarxm I have addressed your comments and sent you a DM on Slack with updated test credentials. Please let me know if any additional changes are needed.

marcosmarxm · 2022-11-14T13:42:59Z

Sorry the delay @michizhou I'll review today!

marcosmarxm · 2022-11-16T13:24:18Z

/publish connector=connectors/source-tyntec-sms run-tests=false

🕑 Publishing the following connectors:
connectors/source-tyntec-sms
https://github.com/airbytehq/airbyte/actions/runs/3479732515

Connector	Did it publish?	Were definitions generated?
connectors/source-tyntec-sms	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

marcosmarxm

Thanks @michizhou I tested locally

* Initial implementation: Support for SMS API streams * Add connector documentation and icon * Add new SMS stream with associated stream slicer * add tyntec sms to source def * format files * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

github-actions bot added the area/connectors Connector related issues label Nov 3, 2022

octavia-squidington-iv added bounty community connectors/source/tyntec-sms labels Nov 3, 2022

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 3, 2022

marcosmarxm added the hacktober label Nov 3, 2022

marcosmarxm requested changes Nov 8, 2022

View reviewed changes

michizhou added 2 commits November 9, 2022 21:06

Initial implementation: Support for SMS API streams

cca482f

Add connector documentation and icon

34a5f3a

michizhou requested a review from marcosmarxm November 10, 2022 18:49

Add new SMS stream with associated stream slicer

b3477dc

marcosmarxm added 2 commits November 16, 2022 10:15

add tyntec sms to source def

7baa1c8

format files

5ee5c9b

marcosmarxm requested a review from a team November 16, 2022 13:21

octavia-squidington-iv added the connectors/source/tplcentral label Nov 16, 2022

auto-bump connector version

9381f72

marcosmarxm removed the request for review from a team November 16, 2022 13:41

Merge branch 'master' into source-tyntec-sms

a536628

marcosmarxm approved these changes Nov 16, 2022

View reviewed changes

marcosmarxm merged commit 4858fba into airbytehq:master Nov 16, 2022

github-actions bot mentioned this pull request Nov 17, 2022

Bump helm chart version reference to 0.40.44 #19546

Merged

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

sajarin removed the bounty label Dec 5, 2022

sajarin added the internal label Dec 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source: Tyntec SMS [low-code cdk] #18883

🎉 New Source: Tyntec SMS [low-code cdk] #18883

michizhou commented Nov 3, 2022 •

edited

Loading

marcosmarxm commented Nov 4, 2022

marcosmarxm commented Nov 4, 2022

michizhou commented Nov 7, 2022

marcosmarxm left a comment

marcosmarxm Nov 8, 2022

michizhou Nov 8, 2022

marcosmarxm Nov 8, 2022

marcosmarxm Nov 8, 2022

marcosmarxm Nov 8, 2022

michizhou Nov 8, 2022

marcosmarxm Nov 8, 2022

michizhou commented Nov 10, 2022

marcosmarxm commented Nov 14, 2022

marcosmarxm commented Nov 16, 2022 •

edited by github-actions bot

Loading

marcosmarxm left a comment

🎉 New Source: Tyntec SMS [low-code cdk] #18883

🎉 New Source: Tyntec SMS [low-code cdk] #18883

Conversation

michizhou commented Nov 3, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

marcosmarxm commented Nov 4, 2022

marcosmarxm commented Nov 4, 2022

michizhou commented Nov 7, 2022

marcosmarxm left a comment

Choose a reason for hiding this comment

marcosmarxm Nov 8, 2022

Choose a reason for hiding this comment

michizhou Nov 8, 2022

Choose a reason for hiding this comment

marcosmarxm Nov 8, 2022

Choose a reason for hiding this comment

marcosmarxm Nov 8, 2022

Choose a reason for hiding this comment

marcosmarxm Nov 8, 2022

Choose a reason for hiding this comment

michizhou Nov 8, 2022

Choose a reason for hiding this comment

marcosmarxm Nov 8, 2022

Choose a reason for hiding this comment

michizhou commented Nov 10, 2022

marcosmarxm commented Nov 14, 2022

marcosmarxm commented Nov 16, 2022 • edited by github-actions bot Loading

marcosmarxm left a comment

Choose a reason for hiding this comment

michizhou commented Nov 3, 2022 •

edited

Loading

marcosmarxm commented Nov 16, 2022 •

edited by github-actions bot

Loading