SetChecksumAlgorithm (Aws::S3::Model::ChecksumAlgorithm::NOT_SET) Error

[zhangc1210]

Describe the bug

if call SetChecksumAlgorithm (Aws::S3::Model::ChecksumAlgorithm::NOT_SET) on request ,the http Authorization header will as like SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-md5;content-type;host;x-amz-content-sha256;x-amz-date;**x-amz-sdk-checksum-algorithm,**
but x-amz-sdk-checksum-algorithm is not exists in http header.

Expected Behavior

remove x-amz-sdk-checksum-algorithm from Authorization if not set x-amz-sdk-checksum-algorithm header

Current Behavior

PUT /test/obj.rar HTTP/1.1
Cache-Control: no-cache
Connection: Keep-Alive
Pragma: no-cache
Content-Type: binary/octet-stream
Content-MD5: /Rtc6T8sitAOPBjRsYvfcg==
Accept: */*
Authorization: AWS4-HMAC-SHA256 Credential=AKIAQJ3NF3UI6RHBFJ73/20220615/us-west-r2/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-md5;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-sdk-checksum-algorithm, Signature=a0767a0444d3b2af587761b0454f2807d99adbf7ea9c1e33fe32a2608724e27e
Host: 192.168.235.115:10000
User-Agent: aws-sdk-cpp/1.9.276 Windows/10.0.19041.1706 AMD64 MSVC/1929
amz-sdk-invocation-id: C2E67A4A-3DDB-4C64-A961-525C231AA6E7
amz-sdk-request: attempt=1
x-amz-content-sha256: 8aa5f7c43c720b2da30b8b0bd4f4913abdc10f35aec22a75bf566f9af82c678f
x-amz-date: 20220615T071604Z
Content-Length: 682231

Reproduction Steps

        Aws::S3::S3Client s3_client(config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never,false);
	Aws::S3::Model::PutObjectRequest request;
	request.SetBucket("test");
	string objectName = "obj.rar";
	request.SetKey(objectName);

	std::shared_ptr<Aws::IOStream> input_data =
		Aws::MakeShared<Aws::FStream>("FileReadTag","D:\\FarFieldWavelet_RTQC_146.rar",
			std::ios_base::in | std::ios_base::binary);
	request.SetBody(input_data);
	request.SetChecksumAlgorithm(Aws::S3::Model::ChecksumAlgorithm::NOT_SET);
	Aws::S3::Model::PutObjectOutcome outcome =
		s3_client.PutObject(request);

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.9.277

Compiler and Version used

Visual Studio 2019

Operating System and version

Windows 10 x64

[yasminetalby]

Hello @zhangc1210 ,

My sincere apologies for the long delay of answer. We really appreciate and value your time and feedback to the AWS CPP SDK.

I have tested the reported behavior and was able to reproduce it. You are right x-amz-sdk-checksum-algorithm should not be present in the Authorization SignedHeaders if not present in the request header names per the Signature Calculations for the Authorization Header: Transferring Payload in a Single Chunk (AWS Signature Version 4) documentation within the S3 API Reference.

Thank you very much for reporting this.

Best regards,

Yasmine

[ghost]

Any progress on this?

I believe i'm facing the same issue. I set the
Aws::Transfer::TransferManagerConfiguration tr_config; tr_config..checksumAlgorithm = Aws::S3::Model::ChecksumAlgorithm::NOT_SET;

and when i try to upload a file using the transfer manager i'm getting the error:
header 'x-amz-sdk-checksum-algorithm' is listed in singed headers, but not present

[thierryba]

I am having the same issue than @CrackedDS .... but it is when I try to upload to Cloudflare R2 (to test the API) ;)

[sbiscigl]

Im currently unable to reproduce with the following code, one just using the plain s3 client, and the other using the transfer manger

normal client:

#include <aws/core/Aws.h>
#include <aws/core/utils/HashingUtils.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/PutObjectRequest.h>

using namespace Aws;
using namespace Aws::Utils;
using namespace Aws::S3;
using namespace Aws::S3::Model;

static const char* BUCKET_NAME = "your-bucket-name";

auto main() -> int {
  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Trace;
  InitAPI(options); {
    Client::ClientConfiguration config;
    const auto client = Aws::MakeUnique<S3Client>("test",
      config,
      Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never,
      false);

    auto request = PutObjectRequest().WithBucket(BUCKET_NAME)
        .WithKey("test-no-checksum")
        .WithChecksumAlgorithm(ChecksumAlgorithm::NOT_SET);

    std::shared_ptr<IOStream> body = Aws::MakeShared<StringStream>("alloc-tag",
      "test body",
      std::ios_base::in | std::ios_base::binary);
    request.SetBody(body);
    const auto response = client->PutObject(request);
    assert(response.IsSuccess());
  }
  ShutdownAPI(options);
  return 0;
}

transfer manager:

#include <aws/core/Aws.h>
#include <aws/core/utils/threading/Executor.h>
#include <aws/transfer/TransferManager.h>
#include <aws/s3/S3Client.h>
#include <iostream>
#include <thread>

using namespace std;
using namespace Aws;
using namespace Aws::Utils;
using namespace Aws::S3;
using namespace Aws::Transfer;

int main() {
  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Debug;

  InitAPI(options); {
    S3ClientConfiguration config({}, Client::AWSAuthV4Signer::PayloadSigningPolicy::Never);
    auto s3Client = Aws::MakeShared<S3Client>("S3Client", config);
    auto executor = Aws::MakeShared<Threading::PooledThreadExecutor>("executor",
      std::thread::hardware_concurrency() - 1);
    TransferManagerConfiguration managerConfiguration(executor.get());
    managerConfiguration.s3Client = s3Client;
    managerConfiguration.checksumAlgorithm = Model::ChecksumAlgorithm::NOT_SET;
    managerConfiguration.transferInitiatedCallback = [](const TransferManager*,
      const std::shared_ptr<const TransferHandle>&) -> void {
        };
    managerConfiguration.errorCallback = [](const TransferManager*,
      const std::shared_ptr<const TransferHandle>&,
      const Client::AWSError<S3Errors>&error) -> void {
          std::cout << "Transfer failed!\n";
          std::cout << "Error: " << error.GetMessage() << "\n";
        };
    auto transferManager = TransferManager::Create(managerConfiguration);

    transferManager->UploadDirectory("/path/to/the/aws-sdk-cpp",
      "your-bucket",
      "your prefix",
      {});

    transferManager->WaitUntilAllFinished();
  }
  ShutdownAPI(options);
  return 0;
}

so just trying to narrow down the problem, are you able to replicate with AWS S3 or a only API compatible backend?

[sbiscigl]

created pull/2875 that wont serialize any enum headers that use the value NOT_SET unless they are defined in the service model. NOT_SET is something we define and results in a header without a value. This is against the smithy spec that says we should not be sending empty strings on the wire.

[sbiscigl]

pull request merged, give a shout if it gives you anymore trouble

[thierryba]

Will do thanks a lot Envoyé à partir de Outlook pour Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Sam Bisciglia ***@***.***> Sent: Tuesday, February 27, 2024 9:38:19 PM To: aws/aws-sdk-cpp ***@***.***> Cc: Thierry Bastian ***@***.***>; Comment ***@***.***> Subject: Re: [aws/aws-sdk-cpp] SetChecksumAlgorithm (Aws::S3::Model::ChecksumAlgorithm::NOT_SET) Error (Issue #1961) pull request merged, give a shout if it gives you anymore trouble — Reply to this email directly, view it on GitHub<#1961 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAKRRARN5M43VJSLMLLDIXDYVY73XAVCNFSM5Y2M2CR2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJWG42TIOJXGA3Q>. You are receiving this because you commented.Message ID: ***@***.***> Thierry Bastian Head of Architect FileWave Take 10 seconds and let us know what you think about FileWave.<https://survey.hsforms.com/1wqQqyBrSQNqIFHDV8oqndw5ecbv> www.filewave.com<https://www.filewave.com/> ***@***.***

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.