[DEVPLAT-1850] Add SQS support to PubSub

[Neurostep]

Description

The MLD team is building the asynchronous architecture based on the AWS SQS and wants to integrate it into the Doc Chat service. We have to add the SQS PubSub interface to the go-sdk to support that.

In this PR we add support for SQS transport in our PubSub system:

• Add SQS PubSub client wrappers
• Add go-kit SQS transport
• Update PubSub configuration documentation

NOTE

I stumbled into the problem with the AWS GO SDK v2 issue described here: aws/aws-sdk-go-v2#2370

The solution is to upgrade the aws-sdk-go-v2 and all dependent packages to the latest version. I didn't notice any other breaking changes that we should take care of. Eventually, we will upgrade Chassis services using Parity workflow

Testing considerations

• Create SQS service in Go Chassis

Checklist

• Prefixed the PR title with the JIRA ticket code
• Performed simple, atomic commits with good commit messages
• Verified that the commit history is linear and commits are squashed as necessary
• Thoroughly tested the changes in development and/or staging
• Updated the README.md as necessary

Related links

• DEVPLAT-1850

[laynax]

LGTM 🚀 📜

I still think in case of an interrupt we will lose some messages mid way, but we can improve it in the future

[Neurostep]

@laynax

still think in case of an interrupt we will lose some messages mid way, but we can improve it in the future

Do you have something in particular in mind? I tested it with the service restart, and all looks 👌

In AWS SQS, if the message is not deleted from the queue, it will be re-queued again after the timeout (default 30 seconds).

[laynax]

@laynax

still think in case of an interrupt we will lose some messages mid way, but we can improve it in the future

Do you have something in particular in mind? I tested it with the service restart, and all looks 👌

In AWS SQS, if the message is not deleted from the queue, it will be re-queued again after the timeout (default 30 seconds).

Yeah but we better not to rely on the retention config. What I had in mind was something simple like having a waitgroup and returning from Unsubscribe() when the waitgroup is done.
Or a more complex, wait for the channel with tasks in the pool to be emptied and then return from Unsubscribe()
Again, we can leave it for later :) just wanted to have it here for posterity

[Neurostep]

@laynax

Yeah but we better not to rely on the retention config. What I had in mind was something simple like having a waitgroup and returning from Unsubscribe() when the waitgroup is done.

This is make sense and I think we can easily achieve this 👍 Please, check out the new version 🙏

[laynax]

LGTM