fix(index): close file before sign #1221
Conversation
lpcalisi
force-pushed
the
fix/index-file-condition-race
branch
from
b0a5321
to
0dba6da
Compare
Signed-off-by: Lucas Pablo Calisi <calisi.lucas@gmail.com>
lpcalisi
force-pushed
the
fix/index-file-condition-race
branch
from
0dba6da
to
6748410
Compare
Signed-off-by: Lucas Pablo Calisi <calisi.lucas@gmail.com>
| @@ -239,10 +239,11 @@ func (idx *Index) WriteArchiveIndex(ctx context.Context, destinationFile string) | |||
| if err != nil { | |||
| return fmt.Errorf("failed to create archive file: %w", err) | |||
| } | |||
| defer outFile.Close() | |||
|
|
|||
| if _, err = io.Copy(outFile, archive); err != nil { | |||
| return fmt.Errorf("failed to write contents to archive file: %w", err) | |||
There was a problem hiding this comment.
Its not a super big deal, but you would leak a file descriptor here, wouldn't you?
before returning you shouldn't you outfile.Close().
And "saying" that out loud, it looks like we must also be leaking archive file descriptor, aren't we? should we defer archive.Close() above? or does that not need a close for some reason?
There was a problem hiding this comment.
scratch that. it is just a io.Reader, which doens't have Close. So we do not need to close archive.
it's generally not advisable to write to storage locations like these. melange does a lot of in-place changes, and any network i/o failures with networked filesystems will result in non-atomic updates of .apk and/or APKINDEX.tar.gz. Meaning it might be old, partially written file, or new one unsigned, or fully updated and signed. In our actions, we typically use bucket as input, but use a temporary output. Once the command is successful we upload the result out of the action (sign elsewhere) and then push to the bucket. Similar strategy is advisable for any other buckets / publication implmeentations. Complete the build, then upload new .apk, then upload new apkindex.tar.gz last. Such that any new files references by the index are available ahead of the index update. |
ok, but that doesn't mean we should sign files what are currently open and have not been flushed. The fix is still valid, no? |
sure, the fix is valid. but likely there is more to do still; and it still is not safe. If we want to make this safe, it would be better to store the apk in memory; generate update in memory; sign it in memory; write out to a tmp file; atomically rename to target file; whilst holding a lock on it. |
| @@ -239,10 +239,11 @@ func (idx *Index) WriteArchiveIndex(ctx context.Context, destinationFile string) | |||
| if err != nil { | |||
| return fmt.Errorf("failed to create archive file: %w", err) | |||
| } | |||
There was a problem hiding this comment.
I can't comment on lines that weren't changed, so I'm commenting here.
This function WriteArchiveIndex takes a destinationFile , writes to that destinationFile and then signs the idx.IndexFile. So if they're not the same thing, then destinationFile won't actually get signed.
There is only one caller of WriteArchiveIndex (GenerateIndex). So it feels like to me we should change WriteArchiveIndex to not take a destinationFile but rather just operate on idx.IndexFile rather than pretending it can do something else.
I admit I could be missing something here.
We are delegate the sync index data with the fs to linux, we have more guarantees with this change
The
outFilefile should closed before opening it again for signing, this probably causes condition races sometimesI discovered it testing mountpoint-s3 and get
We saw some prs to fix
BAD signature, maybe this can fix?cc @mesaglio