Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update the ESP by creating a tmpdir and RENAME_EXCHANGE #454

Open
martinezjavier opened this issue Mar 30, 2023 · 4 comments · May be fixed by #669
Open

Update the ESP by creating a tmpdir and RENAME_EXCHANGE #454

martinezjavier opened this issue Mar 30, 2023 · 4 comments · May be fixed by #669
Assignees
@travier
Labels
enhancement New feature or request jira For syncing with Jira triaged This issue was triaged

Comments

@martinezjavier
Copy link

Since Linux v6.0 the vfat filesystem has support for renameat2(..., RENAME_EXCHANGE):

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=da87e1725ae2

So bootupd could instead of applying the diff files one by one to the destination dir, it could create a temporary dir that is a copy of the existing ESP, apply the diff to that temp dir and finally do an atomic rename exchange of the two directories.

That way, the update mechanism will be safer since it would only require a single renameat2() system call.
@martinezjavier martinezjavier mentioned this issue Mar 30, 2023
Open
@martinezjavier martinezjavier changed the title Make update the ESP by creating a tmpdir and renameat2(..., RENAME_EXCHANGE) Update the ESP by creating a tmpdir and RENAME_EXCHANGE to make it atomically Mar 30, 2023
@martinezjavier martinezjavier changed the title Update the ESP by creating a tmpdir and RENAME_EXCHANGE to make it atomically Update the ESP by creating a tmpdir and RENAME_EXCHANGE Mar 30, 2023
@martinezjavier
Copy link
Author

openat::Dir already has a local_rename that seems to support this: https://docs.rs/openat/latest/openat/struct.Dir.html#method.local_rename

@jlebon jlebon mentioned this issue Apr 14, 2023
Open
@cgwalters
Copy link
Member

The cost here is write amplification; because FAT doesn't have reflinks, every update to the ESP would require rewriting every file, and a transient doubling of disk space.

@cgwalters cgwalters added the triaged This issue was triaged label May 2, 2023
@travier travier added the jira For syncing with Jira label Feb 12, 2024
@travier travier self-assigned this Feb 15, 2024
@travier travier added the enhancement New feature or request label Feb 15, 2024
@cgwalters cgwalters mentioned this issue Apr 11, 2024
Open
@travier
Copy link
Member

travier commented Jun 19, 2024 •
edited
Loading

It looks like local_exchange does what we want in Rust:

It calls renameat2 with RENAME_EXCHANGE.

@travier
Copy link
Member

travier commented Jun 19, 2024 •
edited
Loading

So the logic would be:

  • cp -a fedora .fedora.tmp
    • We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
  • Update the content of .fedora.tmp with the new binaries
  • Exchange .fedora.tmp -> fedora
  • Remove now "old" .fedora.tmp

HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 19, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
5311abb 
Fixes coreos#454
See Timothée's comment coreos#454 (comment)
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 19, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
a53c172 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 19, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
d824131 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
@HuijingHei HuijingHei linked a pull request Jun 19, 2024 that will close this issue
Open
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 19, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
3a7cd83 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 19, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
f3bb105 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
b635467 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
e91cab0 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
366b8eb 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
a6c8400 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
d033a4e 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
ad56520 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 20, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
6bf2e6f 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 21, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
b8fd9a9 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 21, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
e7f4e69 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 21, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
79cb3ad 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 21, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
3cf8636 
See Timothée's comment coreos#454 (comment)

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 21, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
5891859 
See Timothée's comment coreos#454 (comment)
Also make some tunes for the logic:
- `cp -a EFI .EFI.tmp`
- We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `.EFI.tmp` with the new binaries
- Exchange `.EFI.tmp -> EFI`
- Remove now "old" .EFI.tmp

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 25, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
f308add 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 25, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
751dbe8 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jun 25, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
3717639 
See Timothée's comment coreos#454 (comment)
logic is like this, also need to consider `BOOT` update:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 1, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
a0215c0 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 1, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
7100781 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 1, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
d73ed88 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 1, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
39cd6aa 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 2, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
3f460cc 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files
that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

If we have a file not in a directory in `EFI`, then we can copy
it to `foo.tmp` and then act on it and finally rename it. No
need to copy the entire `EFI`.
Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 2, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
445d0a4 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files
that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

If we have a file not in a directory in `EFI`, then we can copy
it to `foo.tmp` and then act on it and finally rename it. No
need to copy the entire `EFI`.
Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 2, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
e663d4d 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files
that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

If we have a file not in a directory in `EFI`, then we can copy
it to `foo.tmp` and then act on it and finally rename it. No
need to copy the entire `EFI`.
Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 2, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
7413a99 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files
that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

If we have a file not in a directory in `EFI`, then we can copy
it to `foo.tmp` and then act on it and finally rename it. No
need to copy the entire `EFI`.
Fixes coreos#454
HuijingHei added a commit to HuijingHei/bootupd that referenced this issue Jul 4, 2024
@HuijingHei
efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE
6d9f6d1 
See Timothée's comment coreos#454 (comment)
logic is like this:
- `cp -a fedora fedora.tmp`
  - We start with a copy to make sure to keep all other files
that we do not explicitly track in bootupd
- Update the content of `fedora.tmp` with the new binaries
- Exchange `fedora.tmp` -> `fedora`
- Remove now "old" `fedora.tmp`

If we have a file not in a directory in `EFI`, then we can copy
it to `foo.tmp` and then act on it and finally rename it. No
need to copy the entire `EFI`.
Fixes coreos#454
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@travier travier

Labels
enhancement New feature or request jira For syncing with Jira triaged This issue was triaged
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

efi: update the ESP by creating a tmpdir and RENAME_EXCHANGE HuijingHei/bootupd
3 participants
@cgwalters @martinezjavier @travier