unable to setup secure tunneling while using HTTP proxy

[brnaba-aws]

Describe the bug

Using the latest tag (1.8), I'm not able to create a secure tunneling connection. The aws-iot-device-client is able to receive MQTT notifications, but nothing more. Opening the tunnel doesn't work.

To Reproduce

Steps to reproduce the behavior:

1. Setup an EC2 with a squid proxy (allow all traffic) within a security group
2. Setup greengrass on EC2 with security group to have all tcp traffic from this EC2 to the squid EC
3. Follow the readme instructions on github
4. Try to setup aws-iot-device-client with a proxy configuration.
5. Go into AWS Console and open a tunnel.
6. The tunnel will never open

Expected behavior

I'm expecting the secure tunneling to work with a proxy.

Actual behavior

A clear and concise description of what actually happened.

Logs
with a restrictive security group that allow all tcp to EC2 squid security group:

2023-12-04T15:54:21.254Z [INFO]  {FileUtils.cpp}: Successfully create directory /root/.aws-iot-device-client/sample-shadow/ with required permissions 700
2023-12-04T15:54:21.254Z [INFO]  {Config.cpp}: ~/.aws-iot-device-client/sample-shadow/default-sample-shadow-document
2023-12-04T15:54:21.254Z [INFO]  {Config.cpp}: Succesfully create default file: /root/.aws-iot-device-client/sample-shadow/default-sample-shadow-document required for storage of shadow document
2023-12-04T15:54:21.254Z [DEBUG] {Config.cpp}: Did not find a runtime configuration file, assuming Fleet Provisioning has not run for this device
2023-12-04T15:54:21.254Z [WARN]  {FileUtils.cpp}: Permissions to given file/dir path '../../' is not set to recommended value... {Permissions: {desired: 745, actual: 700}}
2023-12-04T15:54:21.254Z [WARN]  {FileUtils.cpp}: Permissions to given file/dir path '../../proxy.conf' is not set to recommended value... {Permissions: {desired: 600, actual: 644}}
2023-12-04T15:54:21.254Z [INFO]  {Config.cpp}: Successfully fetched JSON config file: {
    "http-proxy-enabled": true,
    "http-proxy-host": "10.0.0.160",
    "http-proxy-port": "3128",
    "http-proxy-auth-method": "None"
}

2023-12-04T15:54:21.254Z [INFO]  {Config.cpp}: Successfully fetched http proxy config file '../../proxy.conf' and validated its content.
2023-12-04T15:54:21.254Z [DEBUG] {EnvUtils.cpp}: Updated PATH environment variable to: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/root/.aws-iot-device-client:/root/.aws-iot-device-client/jobs:/home/ubuntu/aws-iot-device-client-1.8/build:/home/ubuntu/aws-iot-device-client-1.8/build/jobs
2023-12-04T15:54:21.254Z [DEBUG] {LockFile.cpp}: creating lockfile
2023-12-04T15:54:21.254Z [INFO]  {Main.cpp}: Now running AWS IoT Device Client version v1.8.0-bb7ff67
2023-12-04T15:54:21.255Z [INFO]  {SharedCrtResourceManager.cpp}: SDK logging is enabled. Check /var/log/aws-iot-device-client/sdk.log for SDK logs.
2023-12-04T15:54:21.256Z [DEBUG] {Retry.cpp}: Retryable function starting, it will retry until success
2023-12-04T15:54:21.256Z [INFO]  {SharedCrtResourceManager.cpp}: Attempting to establish MQTT connection with proxy: 10.0.0.160:3128
2023-12-04T15:54:21.256Z [INFO]  {SharedCrtResourceManager.cpp}: Proxy Authentication is disabled
2023-12-04T15:54:21.258Z [INFO]  {SharedCrtResourceManager.cpp}: Establishing MQTT connection with client id test-proxy...
2023-12-04T15:54:21.425Z [INFO]  {SharedCrtResourceManager.cpp}: MQTT connection established with return code: 0
2023-12-04T15:54:21.425Z [INFO]  {SharedCrtResourceManager.cpp}: Shared MQTT connection is ready!
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Provisioning with Secure Elements is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Config shadow is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Jobs is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Secure Tunneling is enabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Device Defender is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Sample shadow is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Pub Sub is disabled
2023-12-04T15:54:21.425Z [INFO]  {Main.cpp}: Sensor Publish is disabled
2023-12-04T15:54:21.425Z [INFO]  {SharedCrtResourceManager.cpp}: Starting Device Client features.
2023-12-04T15:54:21.425Z [DEBUG] {FeatureRegistry.cpp}: Attempting to start Secure Tunneling
2023-12-04T15:54:21.425Z [INFO]  {SecureTunnelingFeature.cpp}: Running Secure Tunneling!
2023-12-04T15:54:21.426Z [INFO]  {Main.cpp}: Client base has been notified that Secure Tunneling has started
2023-12-04T15:54:21.455Z [DEBUG] {SecureTunnelingFeature.cpp}: Subscribed to tunnel notification topic

Environment (please complete the following information):

• OS: Ubuntu
• Version: 22.04
• Architecture: x86-64
• Device Client commit: tag 1.8
  Additional context

Add any other context about the problem here.

[RogerZhongAWS]

HTTP proxy support has been added under: #416

We are now just working on fixing some tests and cleaning up some things, then we will make the release later this week.

[HarshGandhi-AWS]

Hello @brnaba-aws , this issue is been resolved in Device Clients latest commit. Closing this issue now. Please let us know if you see any other issue using device client.

Regards,
Harsh Gandhi