Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to push blobs to docker registry #132

Closed
1 task done
daniel-pebble opened this issue Aug 7, 2024 · 1 comment · Fixed by #136
Closed
1 task done

Unable to push blobs to docker registry #132

daniel-pebble opened this issue Aug 7, 2024 · 1 comment · Fixed by #136
Labels
bug Something isn't working triage New issues or PRs to be acknowledged by maintainers

Comments

@daniel-pebble
Copy link
Contributor

daniel-pebble commented Aug 7, 2024

What happened in your code base?

Hi,

I've attempted to upload a blob using the following code however it fails with the following error

Error response from registry: {"errors":[{"code":"BLOB_UPLOAD_INVALID","message":"blob upload invalid","detail":{}}]}

Registry Log Error: 2024-08-07 21:18:46 time="2024-08-07T20:18:46.952219413Z" level=error msg="response completed with error" err.code="blob upload invalid" err.detail="invalid secret" err.message="blob upload invalid" go.version=go1.20.8 http.request.contenttype="application/octet-stream" http.request.host=localhost http.request.id=dc571b23-ea56-4b20-b670-1874999be808 http.request.method=PUT http.request.remoteaddr="172.17.0.1:57354" http.request.uri="/v2/installationbundle/blobs/uploads/45ce63a0-a429-4185-b5ac-23874cb9d2c9?digest=sha256%3a44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.160542ms http.response.status=404 http.response.written=88 vars.name=installationbundle vars.uuid=45ce63a0-a429-4185-b5ac-23874cb9d2c9
2024-08-07 21:18:46 172.17.0.1 - - [07/Aug/2024:20:18:46 +0000] "PUT /v2/installationbundle/blobs/uploads/45ce63a0-a429-4185-b5ac-23874cb9d2c9?digest=sha256%3a44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a HTTP/1.1" 404 88 "" ""

I believe the issue is due to the fact that the _state query parameter from the registry response is being discarded with the current PushAsync method. I modified the code below and after this I was able to push blobs into the registry.

    public async Task PushAsync(Descriptor expected, Stream content, CancellationToken cancellationToken = default)
    {
        var url = new UriFactory(Repository.Options).BuildRepositoryBlobUpload();
        using (var response = await Repository.Options.HttpClient.PostAsync(url, null, cancellationToken).ConfigureAwait(false))
        {
            if (response.StatusCode != HttpStatusCode.Accepted)
            {
                throw await response.ParseErrorResponseAsync(cancellationToken).ConfigureAwait(false);
            }

            var location = response.Headers.Location ?? throw new HttpRequestException("missing location header");
            url = location.IsAbsoluteUri ? location : new Uri(url, location);
        }

        // monolithic upload
        // add digest key to query string with expected digest value
        var req = new HttpRequestMessage(HttpMethod.Put, new UriBuilder(url)
        {
            //Query = $"digest={HttpUtility.UrlEncode(expected.Digest)}"
            Query = url.Query + "&" + $"digest={HttpUtility.UrlEncode(expected.Digest)}"
        }.Uri);
        req.Content = new StreamContent(content);
        req.Content.Headers.ContentLength = expected.Size;

What did you expect to happen?

Blobs can be pushed

How can we reproduce it?

My example code

var repository = await _registry.GetRepositoryAsync(repo, CancellationToken.None);

var emptyJsonContent = "{}";
var emptyJsonBytes = Encoding.UTF8.GetBytes(emptyJsonContent);
var emptyJsonDescriptor = new Descriptor
{
    MediaType = "application/vnd.unknown.config.v1+json",
    Size = emptyJsonBytes.Length,
    Digest = "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
};

using (var emptyJsonStream = new MemoryStream(emptyJsonBytes))
{
    await repository.Blobs.PushAsync(emptyJsonDescriptor, emptyJsonStream, CancellationToken.None);
}

What is the version or commit of the ORAS .NET library?

Commit: c58adc9

What are your OS and Runtime environments?

Windows 11 .net 8

Are you willing to submit PRs to fix it?

  • Yes, I am willing to fix it.
@daniel-pebble daniel-pebble added bug Something isn't working triage New issues or PRs to be acknowledged by maintainers labels Aug 7, 2024
@Wwwsylvia
Copy link
Member

Hi @daniel-pebble , looks like the URI parameter got overwritten. We should have re-build the query and append the digest parameter. Would you like to open a PR for this?

daniel-pebble added a commit to daniel-pebble/oras-dotnet that referenced this issue Aug 9, 2024
Use existing query parameter string from blobstore response and then append digest.  Test updated to ensure existing query parameters are maintained

Signed-off-by: Daniel Robinson <daniel.robinson@pebble.tv>
daniel-pebble added a commit to daniel-pebble/oras-dotnet that referenced this issue Aug 9, 2024
Use existing query parameter string from blobstore response and then append digest.  Test updated to ensure existing query parameters are maintained

Signed-off-by: Daniel Robinson <daniel.robinson@pebble.tv>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage New issues or PRs to be acknowledged by maintainers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants