Creation of Zstd:chunked layers seems racy

[mtrmac]

Reading the code (not testing this),

image/storage/storage_dest.go

Line 604 in 32e1469

layer, err := s.imageRef.transport.store.CreateLayer(id, lastLayer, nil, "", false, nil)

creates a layer, and

image/storage/storage_dest.go

Line 612 in 32e1469

if err := s.imageRef.transport.store.ApplyDiffFromStagingDirectory(layer.ID, diffOutput.Target, diffOutput, nil); err != nil {

later fills the contents.

AFAICS the layer is, in the meantime, visible to other processes, who can find it based on the deterministic “chain ID”, and try to use it, while the contents are missing.

Cc: @giuseppe

[giuseppe]

this was added before we had something like PutLayer that combines CreateLayer and ApplyDiff.

We may need something like CreateLayerFromStagingDirectory to solve the problem in a similar way

[mtrmac]

Yes — now that we are looking into this area, I just wanted to record that we will need a c/storage API update.

That might interact with other changes we may need to make…

[mtrmac]

@giuseppe this seems to be a blocker for production use.

[giuseppe]

opened a PR for c/storage: containers/storage#1826

[mtrmac]

#2301 has completed the fix.