Error while registering a new kinesis stream as a scalable target

[pprandive]

Hi,

My application makes use of kinesis streams. In order to cope up with the changing load pattern, I wish to use application autoscaling service to scale up/down kinesis shards. To achieve this, I followed steps mentioned here.
After following all the mentioned steps, I could see the kinesis stream getting scaled based on the load.

Now here is the problem. Our application ends up creating new kinesis streams during its lifetime. So, the requirement is that as soon as the new kinesis stream gets created it should get registered as a scalable target so that shards can be scaled up/down based on the load on the newly created stream. To achieve this, I used the following command, however, it resulted into an error given below:

aws application-autoscaling register-scalable-target --service-namespace custom-resource --resource-id https://<API ID>.execute-api.ap-south-1.amazonaws.com/prod/scalableTargetDimensions/POC-KinesisAutoShardTest-2 --scalable-dimension 'custom-resource:ResourceType:Property' --min-capacity 1 --max-capacity 8 --role-arn arn:aws:iam::<ACC ID>:role/aws-service-role/custom-resource.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_CustomResource

and the error I got is:
Error parsing parameter '--resource-id': Unable to retrieve https://<API ID>.execute-api.ap-south-1.amazonaws.com/prod/scalableTargetDimensions/POC-KinesisAutoShardTest-2: received non 200 status code of 403

Could you please help me achieve this? How can we let the pre-created solution (consisting of API gateway, Lambda, Cloudwatch) know that new kinesis streams have arrived and it should be scaled on shards based on the load?

Thanks and Regards,
Prafulla

[Reneehi]

Thanks for the question. I hope you were able to find the solution to your problem. For future reference, the documentation in this GitHub shows the right way to specify the --resource-id in the CLI command. See Step 1: Register Your Scalable Target.