Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase to Git for Windows v2.45.2 #654

Merged
merged 204 commits into from
Jun 17, 2024
Merged

Rebase to Git for Windows v2.45.2 #654

merged 204 commits into from
Jun 17, 2024

Conversation

dscho
Copy link
Member

@dscho dscho commented Jun 3, 2024

Range-diff relative to clean/vfs-2.45.1
  • 1: 9d4453e (upstream: b6b9faf) = 1: 9d4453e ci: drop mention of BREW_INSTALL_PACKAGES variable

  • 2: 11c7001 (upstream: 93ec0a7) < -: ----------- ci: avoid bare "gcc" for osx-gcc job

  • 3: 7df2405 (upstream: d17d18f) < -: ----------- ci: stop installing "gcc-13" for osx-gcc

  • 4: 22c9468 = 2: c9f8124 for-each-repo: optionally keep going on an error

  • 5: 2e13c37 = 3: 363fcf8 maintenance: running maintenance should not stop on errors

  • 6: 13fd78b = 4: b1a4e29 reset --stdin: trim carriage return from the paths

  • 7: 257787b ! 5: c3e4b17 Identify microsoft/git via a distinct version suffix

    @@ Commit message
      ## GIT-VERSION-GEN ##
     @@
      GVF=GIT-VERSION-FILE
    - DEF_VER=v2.45.1
    + DEF_VER=v2.45.2
      
     +# Identify microsoft/git via a distinct version suffix
     +DEF_VER=$DEF_VER.vfs.0.0
  • 8: 6c6a882 = 6: 5f73c4b gvfs: ensure that the version is based on a GVFS tag

  • 9: 8642241 = 7: d100b33 gvfs: add a GVFS-specific header file

  • 10: 4d672df = 8: 8545025 gvfs: add the core.gvfs config setting

  • 11: a4b9a71 = 9: df35325 gvfs: add the feature to skip writing the index' SHA-1

  • 12: b9f275b = 10: 3ad4d9b gvfs: add the feature that blobs may be missing

  • 13: c4f195b = 11: 6f4087b gvfs: prevent files to be deleted outside the sparse checkout

  • 14: 526552c = 12: aa9410f gvfs: optionally skip reachability checks/upload pack during fetch

  • 15: 1c196d7 = 13: aaed1c8 gvfs: ensure all filters and EOL conversions are blocked

  • 16: 22b726b = 14: 6add08a gvfs: allow "virtualizing" objects

  • 24: 2e30eb6 = 15: 9938de9 Hydrate missing loose objects in check_and_freshen()

  • 25: 24bcae8 = 16: f5be3c3 sha1_file: when writing objects, skip the read_object_hook

  • 26: 51870be ! 17: 3bc6f98 gvfs: add global command pre and post hook procs

    @@ hook.c
      #include "advice.h"
      #include "gettext.h"
      #include "hook.h"
    -@@ hook.c: static int identical_to_template_hook(const char *name, const char *path)
    - 	return ret;
    - }
    +@@
    + #include "environment.h"
    + #include "setup.h"
      
     +static int early_hooks_path_config(const char *var, const char *value,
     +				   const struct config_context *ctx, void *cb)
  • 27: e6ff17f = 18: 01df7d1 t0400: verify that the hook is called correctly from a subdirectory

  • 28: 42b51e2 = 19: c6b53f9 Pass PID of git process to hooks.

  • 29: 31ed9bc = 20: 2927634 pre-command: always respect core.hooksPath

  • 30: 56a24e4 = 21: 6d01e9c sparse-checkout: update files with a modify/delete conflict

  • 31: 019a683 = 22: 36bea5a sparse-checkout: avoid writing entries with the skip-worktree bit

  • 32: 4f7cfe3 = 23: d0f43c5 Do not remove files outside the sparse-checkout

  • 33: 6fc4113 = 24: 1d61d9e send-pack: do not check for sha1 file when GVFS_MISSING_OK set

  • 34: 118a527 = 25: 5fa02b1 cache-tree: remove use of strbuf_addf in update_one

  • 35: 1777a92 = 26: 14634cb gvfs: block unsupported commands when running in a GVFS repo

  • 36: db31b45 = 27: bf99c7d worktree: allow in Scalar repositories

  • 37: d38541d = 28: 70c71d5 gvfs: allow overriding core.gvfs

  • 38: 180746b = 29: 0cfb4ed BRANCHES.md: Add explanation of branches and using forks

  • 39: d29907e = 30: 5feb663 Add virtual file system settings and hook proc

  • 40: 06228cd = 31: a51fc4a virtualfilesystem: don't run the virtual file system hook if the index has been redirected

  • 41: 3445aa0 = 32: 23e0312 virtualfilesystem: check if directory is included

  • 42: b56d9d4 = 33: 21546cc backwards-compatibility: support the post-indexchanged hook

  • 43: ec31fd5 = 34: da51b7f gvfs: verify that the built-in FSMonitor is disabled

  • 44: 91546cb = 35: ee91f7b status: add status serialization mechanism

  • 45: 5883f07 = 36: 7161326 Teach ahead-behind and serialized status to play nicely together

  • 46: e6ee448 = 37: 46e4d2b status: serialize to path

  • 47: 8ccf9fb = 38: 430f931 status: reject deserialize in V2 and conflicts

  • 48: 82efde9 = 39: a25585a serialize-status: serialize global and repo-local exclude file metadata

  • 49: 4830a2e = 40: 2f1678a status: deserialization wait

  • 50: a9c962c = 41: 4d2686e merge-recursive: avoid confusing logic in was_dirty()

  • 51: 9802bfd = 42: acfa9dd merge-recursive: add some defensive coding to was_dirty()

  • 52: 31f1d9c = 43: f353922 merge-recursive: teach was_dirty() about the virtualfilesystem

  • 53: 705e175 = 44: 5b2983d status: deserialize with -uno does not print correct hint

  • 54: 45e4ed8 = 45: 0a3cb9b fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate

  • 55: daed3ee = 46: 39fc367 fsmonitor: add script for debugging and update script for tests

  • 56: be8e145 = 47: 6aa8a77 status: disable deserialize when verbose output requested.

  • 57: 16993e8 = 48: 062196e t7524: add test for verbose status deserialzation

  • 58: 297161e = 49: 54b0899 deserialize-status: silently fallback if we cannot read cache file

  • 59: af290cd = 50: 35d034d gvfs:trace2:data: add trace2 tracing around read_object_process

  • 60: a64d037 = 51: e096a30 gvfs:trace2:data: status deserialization information

  • 61: d9948c9 = 52: 6b6c6fe gvfs:trace2:data: status serialization

  • 62: 3d9f4cd = 53: 0bb194d gvfs:trace2:data: add vfs stats

  • 63: 5fefbad = 54: ffa48df trace2: refactor setting process starting time

  • 64: 207a15e = 55: 5bd219f trace2:gvfs:experiment: clear_ce_flags_1

  • 65: 83055a8 = 56: bfb46cb trace2:gvfs:experiment: report_tracking

  • 66: f42f31e = 57: 8d041c0 trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache

  • 67: fe738aa = 58: e3fba71 trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension

  • 68: 3a8eaf0 = 59: 73970d7 trace2:gvfs:experiment: add region to apply_virtualfilesystem()

  • 69: 960cd8a = 60: 54d6dbb trace2:gvfs:experiment: add region around unpack_trees()

  • 70: 5b2520f = 61: 7d85f72 trace2:gvfs:experiment: add region to cache_tree_fully_valid()

  • 71: 23f9c10 = 62: a0fa905 trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()

  • 72: 3a70cd1 = 63: aba0497 trace2:gvfs:experiment: increase default event depth for unpack-tree data

  • 73: 27a974b = 64: ad9f6fb trace2:gvfs:experiment: add data for check_updates() in unpack_trees()

  • 74: c494bda = 65: a81af9d Trace2:gvfs:experiment: capture more 'tracking' details

  • 75: 2924e38 = 66: eb6901f credential: set trace2_child_class for credential manager children

  • 76: bbff5ff = 67: b6e14a4 sub-process: do not borrow cmd pointer from caller

  • 77: 344654c = 68: 4ee6a3c sub-process: add subprocess_start_argv()

  • 78: 216562b = 69: dd0ba04 sha1-file: add function to update existing loose object cache

  • 79: c0bdffc = 70: f5e04fb packfile: add install_packed_git_and_mru()

  • 80: f7068d0 = 71: 58ccf77 index-pack: avoid immediate object fetch while parsing packfile

  • 81: e6f6455 = 72: 1b2ae4e gvfs-helper: create tool to fetch objects using the GVFS Protocol

  • 82: 336904a = 73: b697893 sha1-file: create shared-cache directory if it doesn't exist

  • 83: ff9c896 = 74: 35bdb65 gvfs-helper: better handling of network errors

  • 84: 799562e = 75: aa9823b gvfs-helper-client: properly update loose cache with fetched OID

  • 85: b53a311 = 76: db5d65a gvfs-helper: V2 robust retry and throttling

  • 86: da85141 = 77: fc7e510 gvfs-helper: expose gvfs/objects GET and POST semantics

  • 87: 0b64a68 = 78: 5cf2d8b gvfs-helper: dramatically reduce progress noise

  • 88: 57dea35 = 79: 7e4b11f gvfs-helper-client.h: define struct object_id

  • 89: ea95531 = 80: b9f27b6 gvfs-helper: handle pack-file after single POST request

  • 90: b3ec480 = 81: 876f1a7 test-gvfs-prococol, t5799: tests for gvfs-helper

  • 91: 7c82e54 = 82: d6313df gvfs-helper: move result-list construction into install functions

  • 92: ad217d4 = 83: cd1958b t5799: add support for POST to return either a loose object or packfile

  • 93: e4cba83 = 84: 60577c5 t5799: cleanup wc-l and grep-c lines

  • 94: 1ce899d = 85: aff1ce5 gvfs-helper: verify loose objects after write

  • 95: 41ad104 = 86: 88ae689 t7599: create corrupt blob test

  • 96: 6243647 = 87: b2be88c gvfs-helper: add prefetch support

  • 97: 651d8cb = 88: 67bd02b gvfs-helper: add prefetch .keep file for last packfile

  • 98: ff6c545 = 89: f1ec908 gvfs-helper: do one read in my_copy_fd_len_tail()

  • 99: c6515af = 90: 2e9328b gvfs-helper: move content-type warning for prefetch packs

  • 100: 5305659 = 91: bb2d109 fetch: use gvfs-helper prefetch under config

  • 101: 3a933f4 = 92: 49b99dd gvfs-helper: better support for concurrent packfile fetches

  • 102: 968175d = 93: 99690f0 remote-curl: do not call fetch-pack when using gvfs-helper

  • 103: d314f2d = 94: 3520875 fetch: reprepare packs before checking connectivity

  • 104: 545e939 = 95: 5a57919 gvfs-helper: retry when creating temp files

  • 105: ee2b8b4 = 96: b6b8bf2 sparse: avoid warnings about known cURL issues in gvfs-helper.c

  • 108: bf9234b = 97: ac88d2c maintenance: care about gvfs.sharedCache config

  • 109: 820079f = 98: 3b34d78 unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags

  • 106: 5e1af6c = 99: 2c25272 gvfs-helper: add --max-retries to prefetch verb

  • 110: 3492d82 = 100: 564cbd7 homebrew: add GitHub workflow to release Cask

  • 107: d6d818b = 101: c7644ea t5799: add tests to detect corrupt pack/idx files in prefetch

  • 111: 3dfe79a = 102: 18773d6 Adding winget workflows

  • 117: 5a08847 = 103: 2043a78 gvfs-helper: ignore .idx files in prefetch multi-part responses

  • 112: f3fecaa ! 104: af92330 Disable the monitor-components workflow in msft-git

    @@ .github/workflows/monitor-components.yml (deleted)
     -            title-pattern: ^(?!.*(5\.[0-9]+[13579]|RC))
     -          - label: pcre2
     -            feed: https://github.com/PCRE2Project/pcre2/tags.atom
    --          - label: mingw-w64-clang
    --            feed: https://github.com/msys2/MINGW-packages/commits/master/mingw-w64-clang.atom
    +-          - label: mingw-w64-llvm
    +-            feed: https://github.com/msys2/MINGW-packages/commits/master/mingw-w64-llvm.atom
     -      fail-fast: false
     -    steps:
     -      - uses: git-for-windows/rss-to-issues@v0
  • 113: e0bd85c = 105: cbacdfc .github: enable windows builds on microsoft fork

  • 114: 25f21e3 = 106: 9fe4f1d release: create initial Windows installer build workflow

  • 115: 48dc640 = 107: 46aae33 help: special-case HOST_CPU universal

  • 116: af15e3b = 108: 98467b0 release: add Mac OSX installer build

  • 118: f2a141d = 109: a646aa1 release: build unsigned Ubuntu .deb package

  • 119: cb824c4 = 110: 9a94045 release: add signing step for .deb package

  • 120: 4cab821 = 111: d0d8eeb release: create draft GitHub release with packages & installers

  • 121: 91c9159 = 112: 3229478 build-git-installers: publish gpg public key

  • 122: 9c5d54a = 113: c3bf3de release: continue pestering until user upgrades

  • 123: b347098 = 114: 621fd85 update-microsoft-git: create barebones builtin

  • 124: f8b462c = 115: 46fce0c Makefile: allow specifying GIT_BUILT_FROM_COMMIT

  • 125: f3ef0b8 = 116: 636b6ea update-microsoft-git: Windows implementation

  • 126: 8bfcd80 = 117: efaad38 dist: archive HEAD instead of HEAD^{tree}

  • 127: 810b7d9 = 118: 62e0e1c update-microsoft-git: use brew on macOS

  • 128: 368e4c1 = 119: 09214e5 release: include GIT_BUILT_FROM_COMMIT in MacOS build

  • 129: a05be3e = 120: f9654fa .github: update ISSUE_TEMPLATE.md for microsoft/git

  • 130: 3e282e7 = 121: cbe4036 release: add installer validation

  • 131: a7cbc7c = 122: 97308f0 .github: update PULL_REQUEST_TEMPLATE.md

  • 17: af7f27f = 123: b232088 git_config_set_multivar_in_file_gently(): add a lock timeout

  • 18: 9bfd2ed = 124: 9532286 scalar: set the config write-lock timeout to 150ms

  • 19: de1765e = 125: 7e2e07e scalar: add docs from microsoft/scalar

  • 20: e64cc9c = 126: 39bc9f4 scalar (Windows): use forward slashes as directory separators

  • 21: ad5658f = 127: 16310d3 scalar: add retry logic to run_git()

  • 22: 9fdf70d = 128: 4d1c500 scalar: support the config command for backwards compatibility

  • 23: b944f0a = 129: 9cf5289 sequencer: avoid progress when stderr is redirected

  • 132: 0ab5831 ! 130: 91323a2 Adjust README.md for microsoft/git

    @@ README.md
     -Git for Windows
     -===============
     -
    +-[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)
     -[![Open in Visual Studio Code](https://img.shields.io/static/v1?logo=visualstudiocode&label=&message=Open%20in%20Visual%20Studio%20Code&labelColor=2c2c32&color=007acc&logoColor=007acc)](https://open.vscode.dev/git-for-windows/git)
     -[![Build status](https://github.com/git-for-windows/git/workflows/CI/badge.svg)](https://github.com/git-for-windows/git/actions?query=branch%3Amain+event%3Apush)
     -[![Join the chat at https://gitter.im/git-for-windows/git](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/git-for-windows/git?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
  • 133: fb70d6b = 131: 72cb263 scalar: implement a minimal JSON parser

  • 134: a2b0d56 = 132: 094d84e scalar clone: support GVFS-enabled remote repositories

  • 135: ce509f6 = 133: 680bf81 test-gvfs-protocol: also serve smart protocol

  • 136: 2d3cff3 = 134: 351dfd5 gvfs-helper: add the endpoint command

  • 137: 0be0f3f = 135: 20f9f2c dir_inside_of(): handle directory separators correctly

  • 138: 4543f54 = 136: ceeb92b scalar: disable authentication in unattended mode

  • 139: bb54b06 = 137: a36d18e scalar: do initialize gvfs.sharedCache

  • 140: af5bbd9 = 138: c79e191 scalar diagnose: include shared cache info

  • 141: 67ed368 = 139: 5a516d0 scalar: only try GVFS protocol on https:// URLs

  • 142: 5192e70 = 140: 9985047 scalar: verify that we can use a GVFS-enabled repository

  • 151: 9de21e8 = 141: 1e6434e scalar: add the cache-server command

  • 152: 6f81798 = 142: 02cfd7b scalar: add a test toggle to skip accessing the vsts/info endpoint

  • 153: c0b0b70 = 143: cfbc0d8 scalar: adjust documentation to the microsoft/git fork

  • 154: a4a5c1a = 144: 97cec47 scalar: enable untracked cache unconditionally

  • 155: 0d14225 = 145: 14eb55f scalar: parse clone --no-fetch-commits-and-trees for backwards compatibility

  • 156: 3133ca8 = 146: b582621 scalar diagnose: accommodate Scalar's Functional Tests

  • 157: 3dc508a = 147: a5913c3 ci: run Scalar's Functional Tests

  • 158: 1b24886 = 148: dcd18d5 scalar: upgrade to newest FSMonitor config setting

  • 159: c30dbba = 149: ff81430 abspath: make strip_last_path_component() global

  • 160: 5dd0b9e = 150: e01e132 scalar: .scalarCache should live above enlistment

  • 143: 42149f3 = 151: d7b5f75 add/rm: allow adding sparse entries when virtual

  • 144: 837faa6 = 152: 1719a1b sparse-checkout: add config to disable deleting dirs

  • 145: fc4ffa9 = 153: af4111b diff: ignore sparse paths in diffstat

  • 146: fdc1eaa = 154: 6eacaee repo-settings: enable sparse index by default

  • 147: b20e275 = 155: dc289a5 diff(sparse-index): verify with partially-sparse

  • 148: efb2ce1 = 156: 1efaae3 stash: expand testing for git stash -u

  • 149: 3a70279 = 157: 1d56c43 sparse: add vfs-specific precautions

  • 150: 7e0d519 ! 158: d01f498 reset: fix mixed reset when using virtual filesystem

    @@ builtin/reset.c: static void update_index_from_diff(struct diff_queue_struct *q,
     +		 */
     +		if (core_virtualfilesystem && !file_exists(two->path))
     +		{
    ++			respect_skip_worktree = 0;
     +			pos = index_name_pos(&the_index, two->path, strlen(two->path));
    ++
     +			if ((pos >= 0 && ce_skip_worktree(the_index.cache[pos])) &&
     +			    (is_missing || !was_missing))
     +			{
    @@ builtin/reset.c: static void update_index_from_diff(struct diff_queue_struct *q,
     +						two->path);
     +
     +				checkout_entry(ceBefore, &state, NULL, NULL);
    -+				respect_skip_worktree = 0;
     +			}
     +		}
      
  • 161: c138a91 = 159: 258d325 credential: add new interactive config option

  • 162: fdba939 = 160: 0689571 maintenance: add custom config to background jobs

  • 163: f8edebb = 161: 89fd5b8 scalar: configure maintenance during 'reconfigure'

  • 164: cbc232e = 162: 700c97c scalar: avoid segfault in reconfigure --all

  • 165: 0cbacc1 = 163: 9c910a1 scalar: make GVFS Protocol a forced choice

  • 166: 3a27091 = 164: 023c72e t5300: confirm failure of git index-pack when non-idx suffix requested

  • 167: 0e1a1e3 = 165: 3f3b1cf index-pack: disable rev-index if index file has non .idx suffix

  • 168: 6662919 = 166: 62d6bf6 sparse-index.c: fix use of index hashes in expand_index

  • 169: b7b93e5 = 167: 012b7e7 t1092: add test for untracked files and directories

  • 170: d1840ac < -: ----------- fixup! reset: fix mixed reset when using virtual filesystem

peff and others added 30 commits May 9, 2024 09:57
The last user of this variable went away in 4a6e4b9 (CI: remove
Travis CI support, 2021-11-23), so it's doing nothing except making it
more confusing to find out which packages _are_ installed.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In #623, it was reported that
the regularly scheduled maintenance stops if one repo in the middle of
the list was found to be missing.

This is undesirable, and points out a gap in the design of `git
for-each-repo`: We need a mode where that command does not stop on an
error, but continues to try running the specified command with the other
repositories.

Imitating the `--keep-going` option of GNU make, this commit teaches
`for-each-repo` the same trick: to continue with the operation on all
the remaining repositories in case there was a problem with one
repository, still setting the exit code to indicate an error occurred.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In #623, it was reported that
maintenance stops on a missing repository, omitting the remaining
repositories that were scheduled for maintenance.

This is undesirable, as it should be a best effort type of operation.

It should still fail due to the missing repository, of course, but not
leave the non-missing repositories in unmaintained shapes.

Let's use `for-each-repo`'s shiny new `--keep-going` option that we just
introduced for that very purpose.

This change will be picked up when running `git maintenance start`,
which is run implicitly by `scalar reconfigure`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While using the reset --stdin feature on windows path added may have a
\r at the end of the path that wasn't getting removed so didn't match
the path in the index and wasn't reset.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
It has been a long-standing practice in Git for Windows to append
`.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep
doing that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since we really want to be based on a `.vfs.*` tag, let's make sure that
there was a new-enough one, i.e. one that agrees with the first three
version numbers of the recorded default version.

This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being
used when the current release train was not yet tagged.

It is important to get the first three numbers of the version right
because e.g. Scalar makes decisions depending on those (such as assuming
that the `git maintenance` built-in is not available, even though it
actually _is_ available).

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
This header file will accumulate GVFS-specific definitions.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
This does not do anything yet. The next patches will add various values
for that config setting that correspond to the various features
offered/required by GVFS.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>

gvfs: refactor loading the core.gvfs config value

This code change makes sure that the config value for core_gvfs
is always loaded before checking it.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
This takes a substantial amount of time, and if the user is reasonably
sure that the files' integrity is not compromised, that time can be saved.

Git no longer verifies the SHA-1 by default, anyway.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>

Update for 2023-02-27: This feature was upstreamed as the index.skipHash
config option. This resulted in some changes to the struct and some of
the setup code. In particular, the config reading was moved to
prepare_repo_settings(), so the core.gvfs bit check was moved there,
too.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Prevent the sparse checkout to delete files that were marked with
skip-worktree bit and are not in the sparse-checkout file.

This is because everything with the skip-worktree bit turned on is being
virtualized and will be removed with the change of HEAD.

There was only one failing test when running with these changes that was
checking to make sure the worktree narrows on checkout which was
expected since we would no longer be narrowing the worktree.

Update 2022-04-05: temporarily set 'sparse.expectfilesoutsideofpatterns' in
test (until we start disabling the "remove present-despite-SKIP_WORKTREE"
behavior with 'core.virtualfilesystem' in a later commit).

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
While performing a fetch with a virtual file system we know that there
will be missing objects and we don't want to download them just because
of the reachability of the commits.  We also don't want to download a
pack file with commits, trees, and blobs since these will be downloaded
on demand.

This flag will skip the first connectivity check and by returning zero
will skip the upload pack. It will also skip the second connectivity
check but continue to update the branches to the latest commit ids.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Ensure all filters and EOL conversions are blocked when running under
GVFS so that our projected file sizes will match the actual file size
when it is hydrated on the local machine.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
The idea is to allow blob objects to be missing from the local repository,
and to load them lazily on demand.

After discussing this idea on the mailing list, we will rename the feature
to "lazy clone" and work more on this.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
This is an early version of patches I am about to send upstream:
gitgitgadget#1719.

This addresses #623.
This adds hard-coded call to GVFS.hooks.exe before and after each Git
command runs.

To make sure that this is only called on repositories cloned with GVFS, we
test for the tell-tale .gvfs.

2021-10-30: Recent movement of find_hook() to hook.c required moving these
changes out of run-command.c to hook.c.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Suggested by Ben Peart.

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Signed-off-by: Alejandro Pauly <alpauly@microsoft.com>
We need to respect that config setting even if we already know that we
have a repository, but have not yet read the config.

The regression test was written by Alejandro Pauly.

2021-10-30: Recent movement of find_hook() into hook.c required moving this
change from run-command.c.

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
When using the sparse-checkout feature, the file might not be on disk
because the skip-worktree bit is on.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
When using the sparse-checkout feature git should not write to the working
directory for files with the skip-worktree bit on.  With the skip-worktree
bit on the file may or may not be in the working directory and if it is
not we don't want or need to create it by calling checkout_entry.

There are two callers of checkout_target.  Both of which check that the
file does not exist before calling checkout_target.  load_current which
make a call to lstat right before calling checkout_target and
check_preimage which will only run checkout_taret it stat_ret is less than
zero.  It sets stat_ret to zero and only if !stat->cached will it lstat
the file and set stat_ret to something other than zero.

This patch checks if skip-worktree bit is on in checkout_target and just
returns so that the entry doesn't not end up in the working directory.
This is so that apply will not create a file in the working directory,
then update the index but not keep the working directory up to date with
the changes that happened in the index.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
String formatting can be a performance issue when there are
hundreds of thousands of trees.

Change to stop using the strbuf_addf and just add the strings
or characters individually.

There are a limited number of modes so added a switch for the
known ones and a default case if something comes through that
are not a known one for git.

In one scenario regarding a huge worktree, this reduces the
time required for a `git checkout <branch>` from 44 seconds
to 38 seconds, i.e. it is a non-negligible performance
improvement.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
The following commands and options are not currently supported when working
in a GVFS repo.  Add code to detect and block these commands from executing.

1) fsck
2) gc
4) prune
5) repack
6) submodule
8) update-index --split-index
9) update-index --index-version (other than 4)
10) update-index --[no-]skip-worktree
11) worktree

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Hydrate missing loose objects in check_and_freshen() when running
virtualized. Add test cases to verify read-object hook works when
running virtualized.

This hook is called in check_and_freshen() rather than
check_and_freshen_local() to make the hook work also with alternates.

Helped-by: Kevin Willford <kewillf@microsoft.com>
Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
The 'git worktree' command was marked as BLOCK_ON_GVFS_REPO because it
does not interact well with the virtual filesystem of VFS for Git. When
a Scalar clone uses the GVFS protocol, it enables the
GVFS_BLOCK_COMMANDS flag, since commands like 'git gc' do not work well
with the GVFS protocol.

However, 'git worktree' works just fine with the GVFS protocol since it
isn't doing anything special. It copies the sparse-checkout from the
current worktree, so it does not have performance issues.

This is a highly requested option.

The solution is to stop using the BLOCK_ON_GVFS_REPO option and instead
add a special-case check in cmd_worktree() specifically for a particular
bit of the 'core_gvfs' global variable (loaded by very early config
reading) that corresponds to the virtual filesystem. The bit that most
closely resembled this behavior was non-obviously named, but does
provide a signal that we are in a Scalar clone and not a VFS for Git
clone. The error message is copied from git.c, so it will have the same
output as before if a user runs this in a VFS for Git clone.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
If we are going to write an object there is no use in calling
the read object hook to get an object from a potentially remote
source.  We would rather just write out the object and avoid the
potential round trip for an object that doesn't exist.

This change adds a flag to the check_and_freshen() and
freshen_loose_object() functions' signatures so that the hook
is bypassed when the functions are called before writing loose
objects. The check for a local object is still performed so we
don't overwrite something that has already been written to one
of the objects directories.

Based on a patch by Kevin Willford.

Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Teach STATUS to optionally serialize the results of a
status computation to a file.

Teach STATUS to optionally read an existing serialization
file and simply print the results, rather than actually
scanning.

This is intended for immediate status results on extremely
large repos and assumes the use of a service/daemon to
maintain a fresh current status snapshot.

2021-10-30: packet_read() changed its prototype in ec9a37d (pkt-line.[ch]:
remove unused packet_read_line_buf(), 2021-10-14).

2021-10-30: sscanf() now does an extra check that "%d" goes into an "int"
and complains about "uint32_t". Replacing with "%u" fixes the compile-time
error.

2021-10-30: string_list_init() was removed by abf897b (string-list.[ch]:
remove string_list_init() compatibility function, 2021-09-28), so we need to
initialize manually.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
On index load, clear/set the skip worktree bits based on the virtual
file system data. Use virtual file system data to update skip-worktree
bit in unpack-trees. Use virtual file system data to exclude files and
folders not explicitly requested.

Update 2022-04-05: disable the "present-despite-SKIP_WORKTREE" file removal
behavior when 'core.virtualfilesystem' is enabled.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Copy link
Collaborator

@derrickstolee derrickstolee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update. Range-diff looks good.

@dscho dscho merged commit 1748df7 into vfs-2.45.2 Jun 17, 2024
143 checks passed
@dscho dscho deleted the tentative/vfs-2.45.2 branch June 17, 2024 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.