CamelAwssqsSinkConnector does not work when specifying AWS credentials in camel.sink.endpoint

[scholzj]

I tried to configure CamelAwssqsSinkConnector connector with my Kafka Connect. To pass the AWS credentials, I first used the camel.sink.endpoint.accessKey, camel.sink.endpoint.secretKey and camel.sink.endpoint.region. But that resulted into following error: The request signature we calculated does not match the signature you provided.. Following is the exact error in the Connect log.

org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[aws-sqs:my-queue?accessKey= AKIAIOSFODNN7EXAMPLE&region=US_EAST_1&secretKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY] <<< in route: Route(route1)[From[direct:start] -> [To[aws-sqs:my-queue?acc... because of Failed to resolve endpoint: aws-sqs://my-queue?accessKey= AKIAIOSFODNN7EXAMPLE&region=US_EAST_1&secretKey=xxxxxx due to: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'POST
/

amz-sdk-invocation-id:829aeb9c-19a8-3f8e-b679-95de536cf129
amz-sdk-retry:3/283/485
host:sqs.us-east-1.amazonaws.com
user-agent:aws-sdk-java/1.11.714 Linux/4.18.0-147.20.1.el8_1.x86_64 OpenJDK_64-Bit_Server_VM/25.252-b09 java/1.8.0_252 scala/2.12.10 kotlin/1.3.50 vendor/Oracle_Corporation
x-amz-date:20200714T111213Z

amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date
48a38266faf90970d6c7fea9b15e6ba366e5f6397c2970fc893f8a7b5e207bd0'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20200714T111213Z
20200714/us-east-1/sqs/aws4_request
50180b44f389851b5d93fae95a084b4006e7495374055ed8eebfc3b68d38994e' (Service: AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: cc58b611-ca29-5d47-b673-e1965f64a5c2)
	at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:393)
	at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:117)
	at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:374)
	at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:348)
	at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2617)
	at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
	at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2450)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
	at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2467)
	at org.apache.camel.main.Main.doStart(Main.java:118)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:115)
	at org.apache.camel.main.MainSupport.run(MainSupport.java:83)
	at org.apache.camel.kafkaconnector.utils.CamelMainSupport$CamelContextStarter.run(CamelMainSupport.java:234)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

When I changed to using the camel.component.aws-sqs.* options, it started to work fine, so it seems to be connected specifically to the options for passing the credentials.

[oscerd]

We need to find a way to specify RAW for these particular options.

[davsclaus]

Okay so camel.component are component specific options and they have no problems with uri encoding (usually passwords etc are more problematic with uri encoding due to using special chars).

So for camel.endpoint as they become part of endpoint uri, you need to specify RAW(xxx) syntax.
Maybe you can try testing this?

camel.sink.endpoint.secretKey=RAW(myKeyGoesHere)

There is a JIRA ticket at Camel to see if we can come up with a smarter out of the box solution
https://issues.apache.org/jira/browse/CAMEL-15368

[lburgazzoli]

in camel 3.4 we also have #property:xyz right as possible solution right ?

[davsclaus]

@lburgazzoli ah yeah that would work too

[oscerd]

We need to add documentation about this

[davsclaus]

Here is a PR with a a bit of rushed fix, would be good to give that a test too. And maybe improve it.
#366

But it shows how we can find out about secret options and auto RAW their values.

[oscerd]

This should have been fixed through #366