-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rebase to v2.43.0 #615
Rebase to v2.43.0 #615
Commits on Nov 20, 2023
-
reset --stdin: trim carriage return from the paths
While using the reset --stdin feature on windows path added may have a \r at the end of the path that wasn't getting removed so didn't match the path in the index and wasn't reset. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a974b14 - Browse repository at this point
Copy the full SHA a974b14View commit details -
gvfs: start by adding the -gvfs suffix to the version
Signed-off-by: Saeed Noursalehi <sanoursa@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ad8d0b9 - Browse repository at this point
Copy the full SHA ad8d0b9View commit details -
gvfs: ensure that the version is based on a GVFS tag
Since we really want to be based on a `.vfs.*` tag, let's make sure that there was a new-enough one, i.e. one that agrees with the first three version numbers of the recorded default version. This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being used when the current release train was not yet tagged. It is important to get the first three numbers of the version right because e.g. Scalar makes decisions depending on those (such as assuming that the `git maintenance` built-in is not available, even though it actually _is_ available). Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for fca083a - Browse repository at this point
Copy the full SHA fca083aView commit details -
gvfs: add a GVFS-specific header file
This header file will accumulate GVFS-specific definitions. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d78de08 - Browse repository at this point
Copy the full SHA d78de08View commit details -
gvfs: add the core.gvfs config setting
This does not do anything yet. The next patches will add various values for that config setting that correspond to the various features offered/required by GVFS. Signed-off-by: Kevin Willford <kewillf@microsoft.com> gvfs: refactor loading the core.gvfs config value This code change makes sure that the config value for core_gvfs is always loaded before checking it. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 3556ea0 - Browse repository at this point
Copy the full SHA 3556ea0View commit details -
gvfs: add the feature to skip writing the index' SHA-1
This takes a substantial amount of time, and if the user is reasonably sure that the files' integrity is not compromised, that time can be saved. Git no longer verifies the SHA-1 by default, anyway. Signed-off-by: Kevin Willford <kewillf@microsoft.com> Update for 2023-02-27: This feature was upstreamed as the index.skipHash config option. This resulted in some changes to the struct and some of the setup code. In particular, the config reading was moved to prepare_repo_settings(), so the core.gvfs bit check was moved there, too. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for ccf7f70 - Browse repository at this point
Copy the full SHA ccf7f70View commit details -
gvfs: add the feature that blobs may be missing
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 1aa1248 - Browse repository at this point
Copy the full SHA 1aa1248View commit details -
gvfs: prevent files to be deleted outside the sparse checkout
Prevent the sparse checkout to delete files that were marked with skip-worktree bit and are not in the sparse-checkout file. This is because everything with the skip-worktree bit turned on is being virtualized and will be removed with the change of HEAD. There was only one failing test when running with these changes that was checking to make sure the worktree narrows on checkout which was expected since we would no longer be narrowing the worktree. Update 2022-04-05: temporarily set 'sparse.expectfilesoutsideofpatterns' in test (until we start disabling the "remove present-despite-SKIP_WORKTREE" behavior with 'core.virtualfilesystem' in a later commit). Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for e44e823 - Browse repository at this point
Copy the full SHA e44e823View commit details -
gvfs: optionally skip reachability checks/upload pack during fetch
While performing a fetch with a virtual file system we know that there will be missing objects and we don't want to download them just because of the reachability of the commits. We also don't want to download a pack file with commits, trees, and blobs since these will be downloaded on demand. This flag will skip the first connectivity check and by returning zero will skip the upload pack. It will also skip the second connectivity check but continue to update the branches to the latest commit ids. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for f4de9c9 - Browse repository at this point
Copy the full SHA f4de9c9View commit details -
gvfs: ensure all filters and EOL conversions are blocked
Ensure all filters and EOL conversions are blocked when running under GVFS so that our projected file sizes will match the actual file size when it is hydrated on the local machine. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4728ebc - Browse repository at this point
Copy the full SHA 4728ebcView commit details -
gvfs: allow "virtualizing" objects
The idea is to allow blob objects to be missing from the local repository, and to load them lazily on demand. After discussing this idea on the mailing list, we will rename the feature to "lazy clone" and work more on this. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 9c2a07a - Browse repository at this point
Copy the full SHA 9c2a07aView commit details -
Hydrate missing loose objects in check_and_freshen()
Hydrate missing loose objects in check_and_freshen() when running virtualized. Add test cases to verify read-object hook works when running virtualized. This hook is called in check_and_freshen() rather than check_and_freshen_local() to make the hook work also with alternates. Helped-by: Kevin Willford <kewillf@microsoft.com> Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2206633 - Browse repository at this point
Copy the full SHA 2206633View commit details -
sha1_file: when writing objects, skip the read_object_hook
If we are going to write an object there is no use in calling the read object hook to get an object from a potentially remote source. We would rather just write out the object and avoid the potential round trip for an object that doesn't exist. This change adds a flag to the check_and_freshen() and freshen_loose_object() functions' signatures so that the hook is bypassed when the functions are called before writing loose objects. The check for a local object is still performed so we don't overwrite something that has already been written to one of the objects directories. Based on a patch by Kevin Willford. Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for b7c8f6e - Browse repository at this point
Copy the full SHA b7c8f6eView commit details -
gvfs: add global command pre and post hook procs
This adds hard-coded call to GVFS.hooks.exe before and after each Git command runs. To make sure that this is only called on repositories cloned with GVFS, we test for the tell-tale .gvfs. 2021-10-30: Recent movement of find_hook() to hook.c required moving these changes out of run-command.c to hook.c. Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 841fbcf - Browse repository at this point
Copy the full SHA 841fbcfView commit details -
t0400: verify that the hook is called correctly from a subdirectory
Suggested by Ben Peart. Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 7d495f8 - Browse repository at this point
Copy the full SHA 7d495f8View commit details -
Pass PID of git process to hooks.
Signed-off-by: Alejandro Pauly <alpauly@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d589761 - Browse repository at this point
Copy the full SHA d589761View commit details -
pre-command: always respect core.hooksPath
We need to respect that config setting even if we already know that we have a repository, but have not yet read the config. The regression test was written by Alejandro Pauly. 2021-10-30: Recent movement of find_hook() into hook.c required moving this change from run-command.c. Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 82294a9 - Browse repository at this point
Copy the full SHA 82294a9View commit details -
sparse-checkout: update files with a modify/delete conflict
When using the sparse-checkout feature, the file might not be on disk because the skip-worktree bit is on. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for e53d8d9 - Browse repository at this point
Copy the full SHA e53d8d9View commit details -
sparse-checkout: avoid writing entries with the skip-worktree bit
When using the sparse-checkout feature git should not write to the working directory for files with the skip-worktree bit on. With the skip-worktree bit on the file may or may not be in the working directory and if it is not we don't want or need to create it by calling checkout_entry. There are two callers of checkout_target. Both of which check that the file does not exist before calling checkout_target. load_current which make a call to lstat right before calling checkout_target and check_preimage which will only run checkout_taret it stat_ret is less than zero. It sets stat_ret to zero and only if !stat->cached will it lstat the file and set stat_ret to something other than zero. This patch checks if skip-worktree bit is on in checkout_target and just returns so that the entry doesn't not end up in the working directory. This is so that apply will not create a file in the working directory, then update the index but not keep the working directory up to date with the changes that happened in the index. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a3c5312 - Browse repository at this point
Copy the full SHA a3c5312View commit details -
Do not remove files outside the sparse-checkout
Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 31ec876 - Browse repository at this point
Copy the full SHA 31ec876View commit details -
Configuration menu - View commit details
-
Copy full SHA for a147bf0 - Browse repository at this point
Copy the full SHA a147bf0View commit details -
cache-tree: remove use of strbuf_addf in update_one
String formatting can be a performance issue when there are hundreds of thousands of trees. Change to stop using the strbuf_addf and just add the strings or characters individually. There are a limited number of modes so added a switch for the known ones and a default case if something comes through that are not a known one for git. In one scenario regarding a huge worktree, this reduces the time required for a `git checkout <branch>` from 44 seconds to 38 seconds, i.e. it is a non-negligible performance improvement. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a25fe51 - Browse repository at this point
Copy the full SHA a25fe51View commit details -
Add virtual file system settings and hook proc
On index load, clear/set the skip worktree bits based on the virtual file system data. Use virtual file system data to update skip-worktree bit in unpack-trees. Use virtual file system data to exclude files and folders not explicitly requested. Update 2022-04-05: disable the "present-despite-SKIP_WORKTREE" file removal behavior when 'core.virtualfilesystem' is enabled. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ff2514c - Browse repository at this point
Copy the full SHA ff2514cView commit details -
gvfs: block unsupported commands when running in a GVFS repo
The following commands and options are not currently supported when working in a GVFS repo. Add code to detect and block these commands from executing. 1) fsck 2) gc 4) prune 5) repack 6) submodule 8) update-index --split-index 9) update-index --index-version (other than 4) 10) update-index --[no-]skip-worktree 11) worktree Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 30a0f59 - Browse repository at this point
Copy the full SHA 30a0f59View commit details -
virtualfilesystem: don't run the virtual file system hook if the inde…
…x has been redirected Fixes #13 Some git commands spawn helpers and redirect the index to a different location. These include "difftool -d" and the sequencer (i.e. `git rebase -i`, `git cherry-pick` and `git revert`) and others. In those instances we don't want to update their temporary index with our virtualization data. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 23e0749 - Browse repository at this point
Copy the full SHA 23e0749View commit details -
worktree: allow in Scalar repositories
The 'git worktree' command was marked as BLOCK_ON_GVFS_REPO because it does not interact well with the virtual filesystem of VFS for Git. When a Scalar clone uses the GVFS protocol, it enables the GVFS_BLOCK_COMMANDS flag, since commands like 'git gc' do not work well with the GVFS protocol. However, 'git worktree' works just fine with the GVFS protocol since it isn't doing anything special. It copies the sparse-checkout from the current worktree, so it does not have performance issues. This is a highly requested option. The solution is to stop using the BLOCK_ON_GVFS_REPO option and instead add a special-case check in cmd_worktree() specifically for a particular bit of the 'core_gvfs' global variable (loaded by very early config reading) that corresponds to the virtual filesystem. The bit that most closely resembled this behavior was non-obviously named, but does provide a signal that we are in a Scalar clone and not a VFS for Git clone. The error message is copied from git.c, so it will have the same output as before if a user runs this in a VFS for Git clone. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 094e64e - Browse repository at this point
Copy the full SHA 094e64eView commit details -
virtualfilesystem: check if directory is included
Add check to see if a directory is included in the virtualfilesystem before checking the directory hashmap. This allows a directory entry like foo/ to find all untracked files in subdirectories.
Configuration menu - View commit details
-
Copy full SHA for 4a58f5a - Browse repository at this point
Copy the full SHA 4a58f5aView commit details -
gvfs: allow overriding core.gvfs
We found a user who had set "core.gvfs = false" in their global config. This should not have been necessary, but it also should not have caused a problem. However, it did. The reason is that gvfs_load_config_value() is called from config.c when reading config key/value pairs from all the config files. The local config should override the global config, and this is done by config.c reading the global config first then reading the local config. However, our logic only allowed writing the core_gvfs variable once. Put the guards against multiple assignments of core_gvfs into gvfs_config_is_set() instead, because that will fix the problem _and_ keep multiple calls to gvfs_config_is_set() from slowing down. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a7dea34 - Browse repository at this point
Copy the full SHA a7dea34View commit details -
backwards-compatibility: support the post-indexchanged hook
When our patches to support that hook were upstreamed, the hook's name was eliciting some reviewer suggestions, and it was renamed to `post-index-change`. These patches (with the new name) made it into v2.22.0. However, VFSforGit users may very well have checkouts with that hook installed under the original name. To support this, let's just introduce a hack where we look a bit more closely when we just failed to find the `post-index-change` hook, and allow any `post-indexchanged` hook to run instead (if it exists).
Configuration menu - View commit details
-
Copy full SHA for 9fda9ad - Browse repository at this point
Copy the full SHA 9fda9adView commit details -
status: add status serialization mechanism
Teach STATUS to optionally serialize the results of a status computation to a file. Teach STATUS to optionally read an existing serialization file and simply print the results, rather than actually scanning. This is intended for immediate status results on extremely large repos and assumes the use of a service/daemon to maintain a fresh current status snapshot. 2021-10-30: packet_read() changed its prototype in ec9a37d (pkt-line.[ch]: remove unused packet_read_line_buf(), 2021-10-14). 2021-10-30: sscanf() now does an extra check that "%d" goes into an "int" and complains about "uint32_t". Replacing with "%u" fixes the compile-time error. 2021-10-30: string_list_init() was removed by abf897b (string-list.[ch]: remove string_list_init() compatibility function, 2021-09-28), so we need to initialize manually. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 112fa44 - Browse repository at this point
Copy the full SHA 112fa44View commit details -
Configuration menu - View commit details
-
Copy full SHA for 543775e - Browse repository at this point
Copy the full SHA 543775eView commit details -
gvfs: verify that the built-in FSMonitor is disabled
When using a virtual file system layer, the FSMonitor does not make sense. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 457e6a4 - Browse repository at this point
Copy the full SHA 457e6a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for e8e29af - Browse repository at this point
Copy the full SHA e8e29afView commit details -
Teach status serialization to take an optional pathname on the command line to direct that cache data be written there rather than to stdout. When used this way, normal status results will still be written to stdout. When no path is given, only binary serialization data is written to stdout. Usage: git status --serialize[=<path>] Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 5779892 - Browse repository at this point
Copy the full SHA 5779892View commit details -
status: reject deserialize in V2 and conflicts
Teach status deserialize code to reject status cache when printing in porcelain V2 and there are unresolved conflicts in the cache file. A follow-on task might extend the cache format to include this additiona data. See code for longer explanation. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 631b365 - Browse repository at this point
Copy the full SHA 631b365View commit details -
serialize-status: serialize global and repo-local exclude file metadata
Changes to the global or repo-local excludes files can change the results returned by "git status" for untracked files. Therefore, it is important that the exclude-file values used during serialization are still current at the time of deserialization. Teach "git status --serialize" to report metadata on the user's global exclude file (which defaults to "$XDG_HOME/git/ignore") and for the repo-local excludes file (which is in ".git/info/excludes"). Serialize will record the pathnames and mtimes for these files in the serialization header (next to the mtime data for the .git/index file). Teach "git status --deserialize" to validate this new metadata. If either exclude file has changed since the serialization-cache-file was written, then deserialize will reject the cache file and force a full/normal status run. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 69f2880 - Browse repository at this point
Copy the full SHA 69f2880View commit details -
Teach `git status --deserialize` to either wait indefintely or immediately fail if the status serialization cache file is stale. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 46da1e9 - Browse repository at this point
Copy the full SHA 46da1e9View commit details -
merge-recursive: avoid confusing logic in was_dirty()
It took this developer more than a moment to verify that was_dirty() really returns 0 (i.e. "false") if the file was not even tracked. In other words, the `dirty` variable that was initialized to 1 (i.e. "true") and then negated to be returned was not helping readability. The same holds for the final return: rather than assigning the value to return to `dirty` and then *immediately* returning that, we can simplify it to a single statement.
Configuration menu - View commit details
-
Copy full SHA for dc54eb4 - Browse repository at this point
Copy the full SHA dc54eb4View commit details -
gvfs:trace2:data: add trace2 tracing around read_object_process
Add trace2 region around read_object_process to collect time spent waiting for missing objects to be dynamically fetched. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 56c9f6e - Browse repository at this point
Copy the full SHA 56c9f6eView commit details -
merge-recursive: add some defensive coding to was_dirty()
It took this developer quite a good while to understand why the current code cannot get a `NULL` returned by `index_file_exists()`. To un-confuse readers (and future-proof the code), let's just be safe and check before we dereference the returned pointer.
Configuration menu - View commit details
-
Copy full SHA for 8e00850 - Browse repository at this point
Copy the full SHA 8e00850View commit details -
gvfs:trace2:data: status deserialization information
Add trace2 region and data events describing attempts to deserialize status data using a status cache. A category:status, label:deserialize region is pushed around the deserialize code. Deserialization results when reading from a file are: category:status, path = <path> category:status, polled = <number_of_attempts> category:status, result = "ok" | "reject" When reading from STDIN are: category:status, path = "STDIN" category:status, result = "ok" | "reject" Status will fallback and run a normal status scan when a "reject" is reported (unless "--deserialize-wait=fail"). If "ok" is reported, status was able to use the status cache and avoid scanning the workdir. Additionally, a cmd_mode is emitted for each step: collection, deserialization, and serialization. For example, if deserialization is attempted and fails and status falls back to actually computing the status, a cmd_mode message containing "deserialize" is issued and then a cmd_mode for "collect" is issued. Also, if deserialization fails, a data message containing the rejection reason is emitted. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c400ad0 - Browse repository at this point
Copy the full SHA c400ad0View commit details -
merge-recursive: teach was_dirty() about the virtualfilesystem
The idea of the virtual file system really is to tell Git to avoid accessing certain paths. This fixes the case where a given path is not yet included in the virtual file system and we are about to write a conflicted version of it.
Configuration menu - View commit details
-
Copy full SHA for a31e2c5 - Browse repository at this point
Copy the full SHA a31e2c5View commit details -
gvfs:trace2:data: status serialization
Add trace information around status serialization. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2bd8a8b - Browse repository at this point
Copy the full SHA 2bd8a8bView commit details -
status: deserialize with -uno does not print correct hint
With the "--untracked-files=complete" option status computes a superset of the untracked files. We use this when writing the status cache. If subsequent deserialize commands ask for either the complete set or one of the "no", "normal", or "all" subsets, it can still use the cache file because of filtering in the deserialize parser. When running status with the "-uno" option, the long format status would print a "(use -u to show untracked files)" hint. When deserializing with the "-uno" option and using a cache computed with "-ucomplete", the "nothing to commit, working tree clean" message would be printed instead of the hint. It was easy to miss because the correct hint message was printed if the cache was rejected for any reason (and status did the full fallback). The "struct wt_status des" structure was initialized with the content of the status cache (and thus defaulted to "complete"). This change sets "des.show_untracked_files" to the requested subset from the command-line or config. This allows the long format to print the hint. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for daa14a8 - Browse repository at this point
Copy the full SHA daa14a8View commit details -
gvfs:trace2:data: add vfs stats
Report virtual filesystem summary data. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 5f31f1b - Browse repository at this point
Copy the full SHA 5f31f1bView commit details -
fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
When using fsmonitor the CE_FSMONITOR_VALID flag should be checked when wanting to know if the entry has been updated. If the flag is set the entry should be considered up to date and the same as if the CE_UPTODATE is set. In order to trust the CE_FSMONITOR_VALID flag, the fsmonitor data needs to be refreshed when the fsmonitor bitmap is applied to the index in tweak_fsmonitor. Since the fsmonitor data is kept up to date for every command, some tests needed to be updated to take that into account. istate->untracked->use_fsmonitor was set in tweak_fsmonitor when the fsmonitor bitmap data was loaded and is now in refresh_fsmonitor since that is being called in tweak_fsmonitor. refresh_fsmonitor will only be called once and any other callers should be setting it when refreshing the fsmonitor data so that code can use the fsmonitor data when checking untracked files. When writing the index, fsmonitor_last_update is used to determine if the fsmonitor bitmap should be created and the extension data written to the index. When running through unpack-trees this is not copied to the result index. This makes the next time a git command is ran do all the work of lstating all files to determine what is clean since all entries in the index are marked as dirty since there wasn't any fsmonitor data saved in the index extension. Copying the fsmonitor_last_update to the result index will cause the extension data for fsmonitor to be in the index for the next git command to use. Signed-off-by: Kevin Willford <Kevin.Willford@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a6bf6f1 - Browse repository at this point
Copy the full SHA a6bf6f1View commit details -
trace2: refactor setting process starting time
Create trace2_initialize_clock() and call from main() to capture process start time in isolation and before other sub-systems are ready. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Configuration menu - View commit details
-
Copy full SHA for 2bd6834 - Browse repository at this point
Copy the full SHA 2bd6834View commit details -
fsmonitor: add script for debugging and update script for tests
The fsmonitor script that can be used for running all the git tests using watchman was causing some of the tests to fail because it wrote to stderr and created some files for debugging purposes. Add a new debug script to use with debugging and modify the other script to remove the code that would cause tests to fail. Signed-off-by: Kevin Willford <Kevin.Willford@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for e8099de - Browse repository at this point
Copy the full SHA e8099deView commit details -
trace2:gvfs:experiment: clear_ce_flags_1
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d51366d - Browse repository at this point
Copy the full SHA d51366dView commit details -
status: disable deserialize when verbose output requested.
Disable deserialization when verbose output requested. Verbose mode causes Git to print diffs for modified files. This requires the index to be loaded to have the currently staged OID values. Without loading the index, verbose output make it look like everything was deleted. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 10bf9c1 - Browse repository at this point
Copy the full SHA 10bf9c1View commit details -
trace2:gvfs:experiment: report_tracking
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a1f300e - Browse repository at this point
Copy the full SHA a1f300eView commit details -
t7524: add test for verbose status deserialzation
Verify that `git status --deserialize=x -v` does not crash and generates the same output as a normal (scanning) status command. These issues are described in the previous 2 commits. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c48f9a9 - Browse repository at this point
Copy the full SHA c48f9a9View commit details -
trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
Add trace2_thread_start() and trace2_thread_exit() events to the worker threads used to read the index. This gives per-thread perf data. These workers were introduced in: abb4bb8 read-cache: load cache extensions on a worker thread 77ff112 read-cache: load cache entries on worker threads Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 33229f6 - Browse repository at this point
Copy the full SHA 33229f6View commit details -
deserialize-status: silently fallback if we cannot read cache file
Teach Git to not throw a fatal error when an explicitly-specified status-cache file (`git status --deserialize=<foo>`) could not be found or opened for reading and silently fallback to a traditional scan. This matches the behavior when the status-cache file is implicitly given via a config setting. Note: the current version causes a test to start failing. Mark this as an expected result for now. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ca53dad - Browse repository at this point
Copy the full SHA ca53dadView commit details -
credential: set trace2_child_class for credential manager children
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 93ed390 - Browse repository at this point
Copy the full SHA 93ed390View commit details -
sub-process: do not borrow cmd pointer from caller
Teach subprocess_start() to use a copy of the passed `cmd` string rather than borrowing the buffer from the caller. Some callers of subprocess_start() pass the value returned from find_hook() which points to a static buffer and therefore is only good until the next call to find_hook(). This could cause problems for the long-running background processes managed by sub-process.c where later calls to subprocess_find_entry() to get an existing process will fail. This could cause more than 1 long-running process to be created. TODO Need to confirm, but if only read_object_hook() uses TODO subprocess_start() in this manner, we could drop this TODO commit when we drop support for read_object_hook(). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for f9ff180 - Browse repository at this point
Copy the full SHA f9ff180View commit details -
sub-process: add subprocess_start_argv()
Add function to start a subprocess with an argv. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2224c09 - Browse repository at this point
Copy the full SHA 2224c09View commit details -
sha1-file: add function to update existing loose object cache
Create a function to add a new object to the loose object cache after the existing odb/xx/ directory was scanned. This will be used in a later commit to keep the loose object cache fresh after dynamically fetching an individual object and without requiring the odb/xx/ directory to be rescanned. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for b621659 - Browse repository at this point
Copy the full SHA b621659View commit details -
packfile: add install_packed_git_and_mru()
Create a function to install a new packfile into the packed-git list and add it to the head of the MRU list. This function will be used in a later commit to install packfiles created by dynamic object fetching. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c9a3d28 - Browse repository at this point
Copy the full SHA c9a3d28View commit details -
index-pack: avoid immediate object fetch while parsing packfile
Prevent packfile parsing from accidentally dynamically fetching each individual object found in the packfile. When index-pack parses the input packfile, it does a lookup in the ODB to test for conflicts/collisions. This can accidentally cause the object to be individually fetched when gvfs-helper (or read-object-hook or partial-clone) is enabled. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4fb7e70 - Browse repository at this point
Copy the full SHA 4fb7e70View commit details -
trace2:gvfs:experiment: read-cache: time read/write of cache-tree ext…
…ension Add regions around code to read and write the cache-tree extension when the index is read or written. This is an experiment and may be dropped in future releases if we don't need it anymore. This experiment demonstrates that it takes more time to parse and deserialize the cache-tree extension than it does to read the cache-entries. Commits [1] and [2] spreads cache-entry reading across N-1 cores and dedicates a single core to simultaneously read the index extensions. Local testing (on my machine) shows that reading the cache-tree extension takes ~0.28 seconds. The 11 cache-entry threads take ~0.08 seconds. The main thread is blocked for 0.15 to 0.20 seconds waiting for the extension thread to finish. Let's use this commit to gather some telemetry and confirm this. My point is that improvements, such as index V5 which makes the cache entries smaller, may improve performance, but the gains may be limited because of this extension. And that we may need to look inside the cache-tree extension to truly improve do_read_index() performance. [1] abb4bb8 read-cache: load cache extensions on a worker thread [2] 77ff112 read-cache: load cache entries on worker threads Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ed99b1b - Browse repository at this point
Copy the full SHA ed99b1bView commit details -
gvfs-helper: create tool to fetch objects using the GVFS Protocol
Create gvfs-helper. This is a helper tool to use the GVFS Protocol REST API to fetch objects and configuration data from a GVFS cache-server or Git server. This tool uses libcurl to send object requests to either server. This tool creates loose objects and/or packfiles. Create gvfs-helper-client. This code resides within git proper and uses the sub-process API to manage gvfs-helper as a long-running background process. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ac8e9f9 - Browse repository at this point
Copy the full SHA ac8e9f9View commit details -
trace2:gvfs:experiment: add region to apply_virtualfilesystem()
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d68452c - Browse repository at this point
Copy the full SHA d68452cView commit details -
sha1-file: create shared-cache directory if it doesn't exist
The config variable `gvfs.sharedCache` contains the pathname to an alternate <odb> that will be used by `gvfs-helper` to store dynamically-fetched missing objects. If this directory does not exist on disk, `prepare_alt_odb()` omits this directory from the in-memory list of alternates. This causes `git` commands (and `gvfs-helper` in particular) to fall-back to `.git/objects` for storage of these objects. This disables the shared-cache and leads to poorer performance. Teach `alt_obj_usable()` and `prepare_alt_odb()`, match up the directory named in `gvfs.sharedCache` with an entry in `.git/objects/info/alternates` and force-create the `<odb>` root directory (and the associated `<odb>/pack` directory) if necessary. If the value of `gvfs.sharedCache` refers to a directory that is NOT listed as an alternate, create an in-memory alternate entry in the odb-list. (This is similar to how GIT_ALTERNATE_OBJECT_DIRECTORIES works.) This work happens the first time that `prepare_alt_odb()` is called. Furthermore, teach the `--shared-cache=<odb>` command line option in `gvfs-helper` (which is runs after the first call to `prepare_alt_odb()`) to override the inherited shared-cache (and again, create the ODB directory if necessary). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 0d67b3b - Browse repository at this point
Copy the full SHA 0d67b3bView commit details -
trace2:gvfs:experiment: add region around unpack_trees()
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 025fd6a - Browse repository at this point
Copy the full SHA 025fd6aView commit details -
gvfs-helper: better handling of network errors
Add trace2 message for CURL and HTTP errors. Fix typo reporting network error code back to gvfs-helper-client. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 412fe9c - Browse repository at this point
Copy the full SHA 412fe9cView commit details -
trace2:gvfs:experiment: add region to cache_tree_fully_valid()
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a90ad70 - Browse repository at this point
Copy the full SHA a90ad70View commit details -
gvfs-helper-client: properly update loose cache with fetched OID
Fix parsing of the "loose <odb>" response from `gvfs-helper` and use the actually parsed OID when updating the loose oid cache. Previously, an uninitialized "struct oid" was used to update the cache. This did not cause any corruption, but could cause extra fetches for objects visited multiple times. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4512d14 - Browse repository at this point
Copy the full SHA 4512d14View commit details -
trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() …
…and report_tracking() Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a768214 - Browse repository at this point
Copy the full SHA a768214View commit details -
gvfs-helper: V2 robust retry and throttling
Add robust-retry mechanism to automatically retry a request after network errors. This includes retry after: [] transient network problems reported by CURL. [] http 429 throttling (with associated Retry-After) [] http 503 server unavailable (with associated Retry-After) Add voluntary throttling using Azure X-RateLimit-* hints to avoid being soft-throttled (tarpitted) or hard-throttled (429) on later requests. Add global (outside of a single request) azure-throttle data to track the rate limit hints from the cache-server and main Git server independently. Add exponential retry backoff. This is used for transient network problems when we don't have a Retry-After hint. Move the call to index-pack earlier in the response/error handling sequence so that if we receive a 200 but yet the packfile is truncated/corrupted, we can use the regular retry logic to get it again. Refactor the way we create tempfiles for packfiles to use <odb>/pack/tempPacks/ rather than working directly in the <odb>/pack/ directory. Move the code to create a new tempfile to the start of a single request attempt (initial and retry attempts), rather than at the overall start of a request. This gives us a fresh tempfile for each network request attempt. This simplifies the retry mechanism and isolates us from the file ownership issues hidden within the tempfile class. And avoids the need to truncate previous incomplete results. This was necessary because index-pack was pulled into the retry loop. Minor: Add support for logging X-VSS-E2EID to telemetry on network errors. Minor: rename variable: params.b_no_cache_server --> params.b_permit_cache_server_if_defined. This variable is used to indicate whether we should try to use the cache-server when it is defined. Got rid of double-negative logic. Minor: rename variable: params.label --> params.tr2_label Clarify that this variable is only used with trace2 logging. Minor: Move the code to automatically map cache-server 400 responses to normal 401 response earlier in the response/error handling sequence to simplify later retry logic. Minor: Decorate trace2 messages with "(cs)" or "(main)" to identify the server in log messages. Add params->server_type to simplify this. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2f700e6 - Browse repository at this point
Copy the full SHA 2f700e6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f23c0f - Browse repository at this point
Copy the full SHA 9f23c0fView commit details -
gvfs-helper: expose gvfs/objects GET and POST semantics
Expose the differences in the semantics of GET and POST for the "gvfs/objects" API: HTTP GET: fetches a single loose object over the network. When a commit object is requested, it just returns the single object. HTTP POST: fetches a batch of objects over the network. When the oid-set contains a commit object, all referenced trees are also included in the response. gvfs-helper is updated to take "get" and "post" command line options. the gvfs-helper "server" mode is updated to take "objects.get" and "objects.post" verbs. For convenience, the "get" option and the "objects.get" verb do allow more than one object to be requested. gvfs-helper will automatically issue a series of (single object) HTTP GET requests and creating a series of loose objects. The "post" option and the "objects.post" verb will perform bulk object fetching using the batch-size chunking. Individual HTTP POST requests containing more than one object will be created as a packfile. A HTTP POST for a single object will create a loose object. This commit also contains some refactoring to eliminate the assumption that POST is always associated with packfiles. In gvfs-helper-client.c, gh_client__get_immediate() now uses the "objects.get" verb and ignores any currently queued objects. In gvfs-helper-client.c, the OIDSET built by gh_client__queue_oid() is only processed when gh_client__drain_queue() is called. The queue is processed using the "object.post" verb. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a6e9d9e - Browse repository at this point
Copy the full SHA a6e9d9eView commit details -
trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
Add data for the number of files created/overwritten and deleted during the checkout. Give proper category name to all events in unpack-trees.c and eliminate "exp". This is modified slightly from the original version due to interactions with 26f924d (unpack-trees: exit check_updates() early if updates are not wanted, 2020-01-07). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for af18d6b - Browse repository at this point
Copy the full SHA af18d6bView commit details -
gvfs-helper: dramatically reduce progress noise
During development, it was very helpful to see the gvfs-helper do its work to request a pack-file or download a loose object. When these messages appear during normal use, it leads to a very noisy terminal output. Remove all progress indicators when downloading loose objects. We know that these can be numbered in the thousands in certain kinds of history calls, and would litter the terminal output with noise. This happens during 'git fetch' or 'git pull' as well when the tip commits are checked for the new refs. Remove the "Requesting packfile with %ld objects" message, as this operation is very fast. We quickly follow up with the more valuable "Receiving packfile %ld%ld with %ld objects". When a large "git checkout" causes many pack-file downloads, it is good to know that Git is asking for data from the server. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 5d60e52 - Browse repository at this point
Copy the full SHA 5d60e52View commit details -
Trace2:gvfs:experiment: capture more 'tracking' details
Update tracing around report_tracking() to use 'tracking' category rather than 'exp' category. Add ahead/behind results from stat_tracking_info(). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c9403b8 - Browse repository at this point
Copy the full SHA c9403b8View commit details -
gvfs-helper: add prefetch support
Teach gvfs-helper to support "/gvfs/prefetch" REST API. This includes a new `gvfs-helper prefetch --since=<t>` command line option. And a new `objects.prefetch` verb in `gvfs-helper server` mode. If `since` argument is omitted, `gvfs-helper` will search the local shared-cache for the most recent prefetch packfile and start from there. The <t> is usually a seconds-since-epoch, but may also be a "friendly" date -- such as "midnight", "yesterday" and etc. using the existing date selection mechanism. Add `gh_client__prefetch()` API to allow `git.exe` to easily call prefetch (and using the same long-running process as immediate and queued object fetches). Expanded t5799 unit tests to include prefetch tests. Test setup now also builds some commits-and-trees packfiles for testing purposes with well-known timestamps. Expanded t/helper/test-gvfs-protocol.exe to support "/gvfs/prefetch" REST API. Massive refactor of existing packfile handling in gvfs-helper.c to reuse more code between "/gvfs/objects POST" and "/gvfs/prefetch". With this we now properly name packfiles with the checksum SHA1 rather than a date string. Refactor also addresses some of the confusing tempfile setup and install_<result> code processing (introduced to handle the ambiguity of how POST works with commit objects). Update 2023-05-22 (v2.41.0): add '--no-rev-index' to 'index-pack' to avoid writing the extra (unused) file. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 69a82ce - Browse repository at this point
Copy the full SHA 69a82ceView commit details -
gvfs-helper: add prefetch .keep file for last packfile
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 303a008 - Browse repository at this point
Copy the full SHA 303a008View commit details -
gvfs-helper-client.h: define struct object_id
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 49dce2c - Browse repository at this point
Copy the full SHA 49dce2cView commit details -
gvfs-helper: do one read in my_copy_fd_len_tail()
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 8168ff2 - Browse repository at this point
Copy the full SHA 8168ff2View commit details -
gvfs-helper: handle pack-file after single POST request
If our POST request includes a commit ID, then the the remote will send a pack-file containing the commit and all trees reachable from its root tree. With the current implementation, this causes a failure since we call install_loose() when asking for one object. Modify the condition to check for install_pack() when the response type changes. Also, create a tempfile for the pack-file download or else we will have problems! Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4a8e740 - Browse repository at this point
Copy the full SHA 4a8e740View commit details -
gvfs-helper: move content-type warning for prefetch packs
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c03629f - Browse repository at this point
Copy the full SHA c03629fView commit details -
test-gvfs-prococol, t5799: tests for gvfs-helper
Create t/helper/test-gvfs-protocol.c and t/t5799-gvfs-helper.sh to test gvfs-helper. Create t/helper/test-gvfs-protocol.c as a stand-alone web server that speaks the GVFS Protocol [1] and serves loose objects and packfiles to clients. It is borrows heavily from the code in daemon.c. It includes a "mayhem" mode to cause various network and HTTP errors to test the retry/recovery ability of gvfs-helper. Create t/t5799-gvfs-helper.sh to test gvfs-helper. [1] https://github.com/microsoft/VFSForGit/blob/master/Protocol.md Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a037644 - Browse repository at this point
Copy the full SHA a037644View commit details -
fetch: use gvfs-helper prefetch under config
The gvfs-helper allows us to download prefetch packs using a simple subprocess call. The gvfs-helper-client.h method will automatically compute the timestamp if passing 0, and passing NULL for the number of downloaded packs is valid. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 1538ce2 - Browse repository at this point
Copy the full SHA 1538ce2View commit details -
gvfs-helper: move result-list construction into install functions
gvfs-helper prints a "loose <oid>" or "packfile <name>" messages after they are received to help invokers update their in-memory caches. Move the code to accumulate these messages in the result_list into the install_* functions rather than waiting until the end. POST requests containing 1 object may return a loose object or a packfile depending on whether the object is a commit or non-commit. Delaying the message generation just complicated the caller. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for cb005db - Browse repository at this point
Copy the full SHA cb005dbView commit details -
gvfs-helper: better support for concurrent packfile fetches
Teach gvfs-helper to better support the concurrent fetching of the same packfile by multiple instances. If 2 instances of gvfs-helper did a POST and requested the same set of OIDs, they might receive the exact same packfile (same checksum SHA). Both processes would then race to install their copy of the .pack and .idx files into the ODB/pack directory. This is not a problem on Unix (because of filesystem semantics). On Windows, this can cause an EBUSY/EPERM problem for the loser while the winner is holding a handle to the target files. (The existing packfile code already handled simple the existence and/or replacement case.) The solution presented here is to silently let the loser claim victory IIF the .pack and .idx are already present in the ODB. (We can't check this in advance because we don't know the packfile SHA checksum until after we receive it and run index-pack.) We avoid using a per-packfile lockfile (or a single lockfile for the `vfs-` prefix) to avoid the usual issues with stale lockfiles. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ccb306f - Browse repository at this point
Copy the full SHA ccb306fView commit details -
t5799: add support for POST to return either a loose object or packfile
Earlier versions of the test always returned a packfile in response to a POST. Now we look at the number of objects in the POST request. If > 1, always send a packfile. If = 1 and it is a commit, send a packfile. Otherwise, send a loose object. This is to better model the behavior of the GVFS server/protocol which treats commits differently. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 34c9ff0 - Browse repository at this point
Copy the full SHA 34c9ff0View commit details -
remote-curl: do not call fetch-pack when using gvfs-helper
When using the GVFS protocol, we should _never_ call "git fetch-pack" to attempt downloading a pack-file via the regular Git protocol. It appears that the mechanism that prevented this in the VFS for Git world is due to the read-object hook populating the commits at the new ref tips in a different way than the gvfs-helper does. By acting as if the fetch-pack succeeds here in remote-curl, we prevent a failed fetch. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a2156c9 - Browse repository at this point
Copy the full SHA a2156c9View commit details -
t5799: cleanup wc-l and grep-c lines
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for b83f2f2 - Browse repository at this point
Copy the full SHA b83f2f2View commit details -
fetch: reprepare packs before checking connectivity
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for dfb5b01 - Browse repository at this point
Copy the full SHA dfb5b01View commit details -
gvfs-helper: add --max-retries to prefetch verb
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for ff3e440 - Browse repository at this point
Copy the full SHA ff3e440View commit details -
gvfs-helper: verify loose objects after write
It is possible that a loose object that is written from a GVFS protocol "get object" request does not match the expected hash. Error out in this case. 2021-10-30: The prototype for read_loose_object() changed in 31deb28 (fsck: don't hard die on invalid object types, 2021-10-01) and 96e41f5 (fsck: report invalid object type-path combinations, 2021-10-01). Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a2ffa64 - Browse repository at this point
Copy the full SHA a2ffa64View commit details -
gvfs-helper: retry when creating temp files
When we create temp files for downloading packs, we use a name based on the current timestamp. There is no randomness in the name, so we can have collisions in the same second. Retry the temp pack names using a new "-<retry>" suffix to the name before the ".temp". Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for b4526b7 - Browse repository at this point
Copy the full SHA b4526b7View commit details -
t5799: add tests to detect corrupt pack/idx files in prefetch
Add "mayhem" keys to generate corrupt packfiles and/or corrupt idx files in prefetch by trashing the trailing checksum SHA. Add unit tests to t5799 to verify that `gvfs-helper` detects these corrupt pack/idx files. Currently, only the (bad-pack, no-idx) case is correctly detected, Because `gvfs-helper` needs to locally compute the idx file itself. A test for the (bad-pack, any-idx) case was also added (as a known breakage) because `gvfs-helper` assumes that when the cache server provides both, it doesn't need to verify them. We will fix that assumption in the next commit. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for 99bd372 - Browse repository at this point
Copy the full SHA 99bd372View commit details -
t7599: create corrupt blob test
Teach helper/test-gvfs-protocol to be able to send corrupted loose blobs. Add unit test for gvfs-helper to detect receipt of a corrupted loose blob. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 9e32438 - Browse repository at this point
Copy the full SHA 9e32438View commit details -
sparse: avoid warnings about known cURL issues in gvfs-helper.c
`sparse` complains with an error message like this: gvfs-helper.c:2912:17: error: expression using sizeof on a function The culprit is this line: curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite); Similar lines exist in `http-push.c` and other files that are in upstream Git, and to avoid these bogus warnings, they are already exempted from `sparse`'s tender, loving care. We simply add `gvfs-helper.c` to that list. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 91cdad9 - Browse repository at this point
Copy the full SHA 91cdad9View commit details -
gvfs-helper: ignore .idx files in prefetch multi-part responses
The GVFS cache server can return multiple pairs of (.pack, .idx) files. If both are provided, `gvfs-helper` assumes that they are valid without any validation. This might cause problems if the .pack file is corrupt inside the data stream. (This might happen if the cache server sends extra unexpected STDERR data or if the .pack file is corrupt on the cache server's disk.) All of the .pack file verification logic is already contained within `git index-pack`, so let's ignore the .idx from the data stream and force compute it. This defeats the purpose of some of the data cacheing on the cache server, but safety is more important. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for a107698 - Browse repository at this point
Copy the full SHA a107698View commit details -
release: create initial Windows installer build workflow
- trigger on tag matching basic "vfs" version pattern - validate tag is annotated & matches stricter checks - include `scalar` - build x86_64 & portable git installers, upload artifacts to workflow Update Apr 18, 2022: these steps are built explicitly on 'windows-2019' agents (rather than 'windows-latest') to ensure the correct version of Visual Studio is used (verified in the pipeline via 'type -p mspdb140.dll'). Additionally, due to a known (but not-yet-fixed) issue downloading the 'build-installers' flavor of the Git for Windows SDK with the 'git-for-windows/setup-git-for-windows-sdk' Action, the SDK used is the 'full' flavor.
Configuration menu - View commit details
-
Copy full SHA for 2dc2755 - Browse repository at this point
Copy the full SHA 2dc2755View commit details -
help: special-case HOST_CPU
universal
When building Git as a universal binary on macOS, the binary supports more than one target architecture. This is a bit of a problem for the `HOST_CPU` setting that is woefully unprepared for such a situation, as it wants to show architecture hard-coded at build time. In preparation for releasing universal builds, work around this by special-casing `universal` and replacing it at run-time with the known values `x86_64` or `arm64`. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c68b447 - Browse repository at this point
Copy the full SHA c68b447View commit details -
release: add Mac OSX installer build
- include `scalar` - build signed .dmg & .pkg for target OS version 10.6 - upload artifacts to workflow Co-authored-by: Lessley Dennington <ldennington@github.com>
Configuration menu - View commit details
-
Copy full SHA for dbbb38b - Browse repository at this point
Copy the full SHA dbbb38bView commit details -
release: build unsigned Ubuntu .deb package
- include `scalar` - build & upload unsigned .deb package Co-authored-by: Lessley Dennington <ldennington@github.com>
Configuration menu - View commit details
-
Copy full SHA for 7fa18cd - Browse repository at this point
Copy the full SHA 7fa18cdView commit details -
release: add signing step for .deb package
- sign using Azure-stored certificates & client - sign on Windows agent via python script - job skipped if credentials for accessing certificate aren't present Co-authored-by: Lessley Dennington <ldennington@github.com>
Configuration menu - View commit details
-
Copy full SHA for 9fa8d8e - Browse repository at this point
Copy the full SHA 9fa8d8eView commit details -
Makefile: allow specifying GIT_BUILT_FROM_COMMIT
Allow specification of a custom `GIT_BUILT_FROM_COMMIT` string to replace the output of `git rev-parse HEAD`. This allows a build of `git` from somewhere other than an active clone of `git` (e.g. from the archive created with `make dist`) to include commit information in `git version --build-options`. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for 7046a8e - Browse repository at this point
Copy the full SHA 7046a8eView commit details -
release: create draft GitHub release with packages & installers
- create release & uploads artifact using Octokit - use job "if" condition to handle uploading signed *or* unsigned .deb Co-authored-by: Lessley Dennington <ldennington@github.com>
Configuration menu - View commit details
-
Copy full SHA for 7563d72 - Browse repository at this point
Copy the full SHA 7563d72View commit details -
dist: archive HEAD instead of HEAD^{tree}
Update `git archive` tree-ish argument from `HEAD^{tree}` to `HEAD`. By using a commit (rather than tree) reference, the commit hash will be stored as an extended pax header, extractable git `git get-tar-commit-id`. The intended use-case for this change is building `git` from the output of `make dist` - in combination with the ability to specify a fallback `GIT_BUILT_FROM_COMMIT`, a user can extract the commit ID used to build the archive and set it as `GIT_BUILT_FROM_COMMIT`. The result is fully-populated information for the commit hash in `git version --build-options`. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for aff19d5 - Browse repository at this point
Copy the full SHA aff19d5View commit details -
update-microsoft-git: create barebones builtin
Just do the boilerplate stuff of making a new builtin, including documentation and integration with git.c. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a12eb00 - Browse repository at this point
Copy the full SHA a12eb00View commit details -
homebrew: add GitHub workflow to release Cask
Add a GitHub workflow that is triggered on the `release` event to automatically update the `microsoft-git` Homebrew Cask on the `microsoft/git` Tap. A secret `HOMEBREW_TOKEN` with push permissions to the `microsoft/homebrew-git` repository must exist. A pull request will be created at the moment to allow for last minute manual verification. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Configuration menu - View commit details
-
Copy full SHA for 29ffc42 - Browse repository at this point
Copy the full SHA 29ffc42View commit details -
Disable the
monitor-components
workflow in msft-gitIt really does not make sense to run that workflow in any fork of git-for-windows/git. Typically, it is enough to simply disable it (since it is a scheduled workflow, it is disabled by default in any new fork). However, in microsoft/git, we switch the default branch whenever we rebase to a new upstream version, and every time we do so, this scheduled workflow gets re-enabled. Let's just delete it in microsoft/git and never be bothered by it again. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f627d1c - Browse repository at this point
Copy the full SHA f627d1cView commit details -
build-git-installers: publish gpg public key
Update build-git-installers workflow to publish `microsoft/git`'s GPG public key as part of each release. Add explanation for how to use this key to verify the Debian package's signature to the README.
Configuration menu - View commit details
-
Copy full SHA for 6872758 - Browse repository at this point
Copy the full SHA 6872758View commit details -
release: include GIT_BUILT_FROM_COMMIT in MacOS build
Set the `GIT_BUILT_FROM_COMMIT` based on the version specified in the `make dist` output archive header. This ensures the commit hash is shown in `git version --build-options`. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for 03c9452 - Browse repository at this point
Copy the full SHA 03c9452View commit details -
update-microsoft-git: Windows implementation
On Windows, we have the 'git update-git-for-windows' command. It is poorly named within the microsoft/git fork, because the script has been updated to look at the GitHub releases of microsoft/git, not git-for-windows/git. Still, it handles all the complicated details about downloading, verifying, and running the installer. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 476e968 - Browse repository at this point
Copy the full SHA 476e968View commit details -
maintenance: care about gvfs.sharedCache config
For Scalar and VFS for Git, we use an alternate as a shared object cache. We need to enable the maintenance builtin to work on that shared object cache, especially in the background. 'scalar run <task>' would set GIT_OBJECT_DIRECTORY to handle this. We set GIT_OBJECT_DIRECTORY based on the gvfs.sharedCache config, but we also need the checks in pack_loose() to look at that object directory instead of the current ODB's. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 3347e58 - Browse repository at this point
Copy the full SHA 3347e58View commit details -
unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags
When the virtualfilesystem is enabled the previous implementation of clear_ce_flags would iterate all of the cache entries and query whether each one is in the virtual filesystem to determine whether to clear one of the SKIP_WORKTREE bits. For each cache entry, we would do a hash lookup for each parent directory in the is_included_in_virtualfilesystem function. The former approach is slow for a typical Windows OS enlistment with 3 million files where only a small percentage is in the virtual filesystem. The cost is O(n_index_entries * n_chars_per_path * n_parent_directories_per_path). In this change, we use the same approach as apply_virtualfilesystem, which iterates the set of entries in the virtualfilesystem and searches in the cache for the corresponding entries in order to clear their flags. This approach has a cost of O(n_virtual_filesystem_entries * n_chars_per_path * log(n_index_entries)). The apply_virtualfilesystem code was refactored a bit and modified to clear flags for all names that 'alias' a given virtual filesystem name when ignore_case is set. n_virtual_filesystem_entries is typically much less than n_index_entries, in which case the new approach is much faster. We wind up building the name hash for the index, but this occurs quickly thanks to the multi-threading. Signed-off-by: Neeraj Singh <neerajsi@ntdev.microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for a6b2e7d - Browse repository at this point
Copy the full SHA a6b2e7dView commit details -
Configuration menu - View commit details
-
Copy full SHA for fa5fa71 - Browse repository at this point
Copy the full SHA fa5fa71View commit details -
.github: enable windows builds on microsoft fork
This was disabled by a0da6de (ci: only run win+VS build & tests in Git for Windows' fork, 2022-12-19) to avoid other forks doing too many builds. But we want to keep these builds for the microsoft/git fork. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for e4fb0a6 - Browse repository at this point
Copy the full SHA e4fb0a6View commit details -
release: continue pestering until user upgrades
In 'git-update-git-for-windows', there is a recently_seen variable that is loaded from Git config. This is intended to allow users to say "No, I don't want that version of Git for Windows." If users say no, then they are not reminded. Ever. We want users of microsoft/git to be notified repeately until they upgrade. The first notification might be dismissed because they don't want to interrupt their work. They should get the picture within a few reminders and upgrade in a timely fashion. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for d6e810c - Browse repository at this point
Copy the full SHA d6e810cView commit details -
release: add installer validation
Add basic installer validation to release pipeline for Windows, macOS, and Linux (Debian package only). Validation runs the installers/any necessary setup and checks that the installed version matches the expected version.
Configuration menu - View commit details
-
Copy full SHA for ab2a0e5 - Browse repository at this point
Copy the full SHA ab2a0e5View commit details -
git_config_set_multivar_in_file_gently(): add a lock timeout
In particular when multiple processes want to write to the config simultaneously, it would come in handy to not fail immediately when another process locked the config, but to gently try again. This will help with Scalar's functional test suite which wants to register multiple repositories for maintenance semi-simultaneously. As not all code paths calling this function read the config (e.g. `git config`), we have to read the config setting via `git_config_get_ulong()`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 38fb624 - Browse repository at this point
Copy the full SHA 38fb624View commit details -
scalar: set the config write-lock timeout to 150ms
By default, Git fails immediately when locking a config file for writing fails due to an existing lock. With this change, Scalar-registered repositories will fall back to trying a couple times within a 150ms timeout. Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 200a0e0 - Browse repository at this point
Copy the full SHA 200a0e0View commit details -
scalar: add docs from microsoft/scalar
These docs have been altered to fit the version implemented in C within microsoft/git. This means in particular that the advanced.md file no longer applied at all. Some other areas were removed or significantly edited. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for ed3b89a - Browse repository at this point
Copy the full SHA ed3b89aView commit details -
Merge branch 'scalar-gentler-config-locking'
Allow concurrent `scalar register` and `scalar unregister` calls to be more collaborative when trying to lock the global Git config at the very same time. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e12cc68 - Browse repository at this point
Copy the full SHA e12cc68View commit details -
.github: update ISSUE_TEMPLATE.md for microsoft/git
We have been using the default issue template from git-for-windows/git, but we should ask different questions than Git for Windows. Update the issue template to ask these helpful questions. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for c812236 - Browse repository at this point
Copy the full SHA c812236View commit details -
Merge branch 'scalar-extra-docs'
Adding the extra documentation from the Scalar project. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7e7ae70 - Browse repository at this point
Copy the full SHA 7e7ae70View commit details -
.github: update PULL_REQUEST_TEMPLATE.md
We have long inherited the pull request template from git-for-windows/git, but we should probably do a better job of specifying the need for why a PR in microsoft/git exists instead of an upstream contribution. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for f3f3e30 - Browse repository at this point
Copy the full SHA f3f3e30View commit details -
scalar (Windows): use forward slashes as directory separators
Git traditionally uses those, not backslashes, ever. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for fd18ac9 - Browse repository at this point
Copy the full SHA fd18ac9View commit details -
update-microsoft-git: use brew on macOS
The steps to update the microsoft-git cask are: 1. brew update 2. brew upgrade --cask microsoft-git This is adapted from the UpgradeVerb within microsoft/scalar. There is one important simplification: Scalar needed to check 'brew list --cask' to find out if the 'scalar' cask or the 'scalar-azrepos' cask was installed (which determined if the 'microsoft-git' cask was a necessary dependency). We do not need that here, since we are already in the microsoft-git cask. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 56ca1c2 - Browse repository at this point
Copy the full SHA 56ca1c2View commit details -
Adjust README.md for microsoft/git
Microsoft's fork of Git is not quite Git for Windows, therefore we want to tell the keen reader all about it. :-) Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 8ee8ea0 - Browse repository at this point
Copy the full SHA 8ee8ea0View commit details -
scalar: implement a minimal JSON parser
No grown-up C project comes without their own JSON parser. Just kidding! We need to parse a JSON result when determining which cache server to use. It would appear that searching for needles `"CacheServers":[`, `"Url":"` and `"GlobalDefault":true` _happens_ to work right now, it is fragile as it depends on no whitespace padding and on the order of the fields remaining as-is. Let's implement a super simple JSON parser (at the cost of being slightly inefficient) for that purpose. To avoid allocating a ton of memory, we implement a callback-based one. And to save on complexity, let's not even bother validating the input properly (we will just go ahead and instead rely on Azure Repos to produce correct JSON). Note: An alternative would have been to use existing solutions such as JSON-C, CentiJSON or JSMN. However, they are all a lot larger than the current solution; The smallest, JSMN, which does not even provide parsed string values (something we actually need) weighs in with 471 lines, while we get away with 182 + 29 lines for the C and the header file, respectively. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for aa6747a - Browse repository at this point
Copy the full SHA aa6747aView commit details -
scalar clone: support GVFS-enabled remote repositories
With this change, we come a big step closer to feature parity with Scalar: this allows cloning from Azure Repos (which do not support partial clones at time of writing). We use the just-implemented JSON parser to parse the response we got from the `gvfs/config` endpoint; Please note that this response might, or might not, contain information about a cache server. The presence or absence of said cache server, however, has nothing to do with the ability to speak the GVFS protocol (but the presence of the `gvfs/config` endpoint does that). An alternative considered during the development of this patch was to perform simple string matching instead of parsing the JSON-formatted data; However, this would have been fragile, as the response contains free-form text (e.g. the repository's description) which might contain parts that would confuse a simple string matcher (but not a proper JSON parser). Note: we need to limit the re-try logic in `git clone` to handle only the non-GVFS case: the call to `set_config()` to un-set the partial clone settings would otherwise fail because those settings would not exist in the GVFS protocol case. This will at least give us a clearer reason why such a fetch fails. Co-authored-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 7d6c79d - Browse repository at this point
Copy the full SHA 7d6c79dView commit details -
test-gvfs-protocol: also serve smart protocol
This comes in handy, as we want to verify that `scalar clone` also works against a GVFS-enabled remote repository. Note that we have to set `MSYS2_ENV_CONV_EXCL` to prevent MSYS2 from mangling `PATH_TRANSLATED`: The value _does_ look like a Unix-style path, but no, MSYS2 must not be allowed to convert that into a Windows path: `http-backend` needs it in the unmodified form. (The MSYS2 runtime comes in when `git` is run via `bin-wrappers/git`, which is a shell script.) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 836ea67 - Browse repository at this point
Copy the full SHA 836ea67View commit details -
gvfs-helper: add the
endpoint
commandWe already have the `config` command that accesses the `gvfs/config` endpoint. To implement `scalar`, we also need to be able to access the `vsts/info` endpoint. Let's add a command to do precisely that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7ab1b72 - Browse repository at this point
Copy the full SHA 7ab1b72View commit details -
scalar: add retry logic to run_git()
Use a fixed 3 tries total to see how that increases our chances of success for subcommands such as 'git fetch'. We special-case the `diagnose` command here: When 672196a (scalar-diagnose: use 'git diagnose --mode=all', 2022-08-12) updated 'scalar diagnose' to run 'git diagnose' as a subprocess, it was passed through the run_git() caller. We need to avoid repeating the call when the underlying 'git diagnose' command fails. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 037027e - Browse repository at this point
Copy the full SHA 037027eView commit details -
dir_inside_of(): handle directory separators correctly
On Windows, both the forward slash and the backslash are directory separators. Which means that `a\b\c` really is inside `a/b`. Therefore, we need to special-case the directory separators in the helper function `cmp_icase()` that is used in the loop in `dir_inside_of()`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 2355926 - Browse repository at this point
Copy the full SHA 2355926View commit details -
scalar: support the
config
command for backwards compatibilityThe .NET version supported running `scalar config` to reconfigure the current enlistment, and now the C port does, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 392824f - Browse repository at this point
Copy the full SHA 392824fView commit details -
Merge branch 'microsoft/vfs-2.35.0'
We actually only need the FSMonitor... Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4776a36 - Browse repository at this point
Copy the full SHA 4776a36View commit details -
scalar: disable authentication in unattended mode
Modified to remove call to is_unattended() that has not been implemented yet. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8724aa9 - Browse repository at this point
Copy the full SHA 8724aa9View commit details -
This merges the upstreamable part of the Scalar patches. Minor merge conflicts (caused by the gvfs-helper) were resolved trivially. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9898612 - Browse repository at this point
Copy the full SHA 9898612View commit details -
scalar: do initialize
gvfs.sharedCache
This finalizes the port of the `QueryVstsInfo()` function: we already taught `gvfs-helper` to access the `vsts/info` endpoint on demand, we implemented proper JSON parsing, and now it is time to hook it all up. To that end, we also provide a default local cache root directory. It works the same way as the .NET version of Scalar: it uses C:\scalarCache on Windows, ~/.scalarCache/ on macOS and ~/.cache/scalar on Linux Modified to include call to is_unattended() that was removed from a previous commit. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 501d792 - Browse repository at this point
Copy the full SHA 501d792View commit details -
scalar diagnose: include shared cache info
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f70311e - Browse repository at this point
Copy the full SHA f70311eView commit details -
scalar: only try GVFS protocol on https:// URLs
Well, technically also the http:// protocol is allowed _when testing_... Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4e91983 - Browse repository at this point
Copy the full SHA 4e91983View commit details -
scalar: verify that we can use a GVFS-enabled repository
Azure Repos does not support partial clones at the moment, but it does support the GVFS protocol. To that end, the Microsoft fork of Git has a `gvfs-helper` command that is optionally used to perform essentially the same functionality as partial clone. Let's verify that `scalar clone` detects that situation and enables the GVFS helper. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 68220bc - Browse repository at this point
Copy the full SHA 68220bcView commit details -
scalar: add the
cache-server
commandThis allows setting the GVFS-enabled cache server, or listing the one(s) associated with the remote repository. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for aec7f5e - Browse repository at this point
Copy the full SHA aec7f5eView commit details -
scalar: add a test toggle to skip accessing the vsts/info endpoint
In Scalar's functional tests, we do not do anything with authentication. Therefore, we do want to avoid accessing the `vsts/info` endpoint because it requires authentication even on otherwise public repositories. Let's introduce the environment variable `SCALAR_TEST_SKIP_VSTS_INFO` which can be set to `true` to simply skip that step (and force the `url_*` style repository IDs instead of `id_*` whenever possible). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e7f533d - Browse repository at this point
Copy the full SHA e7f533dView commit details -
scalar: adjust documentation to the microsoft/git fork
Scalar in Microsoft's Git fork can do a little more than Scalar in upstream Git: in Microsoft's Git, it supports the GVFS protocol so that Scalar can clone from Azure DevOps. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4f9b060 - Browse repository at this point
Copy the full SHA 4f9b060View commit details -
scalar: enable untracked cache unconditionally
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for fd68d27 - Browse repository at this point
Copy the full SHA fd68d27View commit details -
scalar diagnose: accommodate Scalar's Functional Tests
Those tests specifically verify that the `.zip` file path is shown on `stdout`. Let's do that again, under the assumption that there are scripts out there that rely on this behavior. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 1339fd6 - Browse repository at this point
Copy the full SHA 1339fd6View commit details -
scalar: parse
clone --no-fetch-commits-and-trees
for backwards comp……atibility This option does not do anything anymore, though. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ab3184f - Browse repository at this point
Copy the full SHA ab3184fView commit details -
ci: run Scalar's Functional Tests
Scalar's Functional Test suite is pretty comprehensive, and caught more than just one bug in the built-in FSMonitor that was missed by Git's own test suite. To benefit from this test suite, automatically run it on the `vfs-*` and `features/*` branches. Note: for simplicity, we're building Git from scratch in all matrix jobs. Also note: for speed, we are using `git-sdk-64-minimal`, even if it lacks the `/bin/install` that we need to install Git's files; We're providing a minimal shell script shim instead. Also, we do not need to bother with the Tcl/Tk parts, therefore we're skipping them, too. Finally, we use GIT_FORCE_UNTRACKED_CACHE in the functional tests, to give the untracked cache a thorough work-out. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ee54857 - Browse repository at this point
Copy the full SHA ee54857View commit details -
Merge branch 'scalar-with-gvfs'
Prepare `scalar` to use the GVFS protocol instead of partial clone (required to support Azure Repos). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d9b8fc0 - Browse repository at this point
Copy the full SHA d9b8fc0View commit details -
abspath: make strip_last_path_component() global
The strip_last_component() method is helpful for finding the parent directory of a path stored in a strbuf. Extract it to a global method advertised in abspath.h. With that additional visibility, it is helpful to rename it to be more specific to paths. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 92c86aa - Browse repository at this point
Copy the full SHA 92c86aaView commit details -
scalar: upgrade to newest FSMonitor config setting
When FSMonitor was upstreamed, the 'core.useBuiltinFSMonitor' config was deprecated and replaced with an overload of the 'core.fsmonitor' config (i.e., if a boolean value was specified in 'core.fsmonitor', it is treated the way 'core.useBuiltinFSMonitor' originally was). Because 'scalar register' actively sets that config, use it to upgrade the deprecated config setting. Co-authored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for 34c210a - Browse repository at this point
Copy the full SHA 34c210aView commit details -
Merge branch 'run-scalar-functional-tests'
Run the comprehensive Scalar Functional Tests as part of CI and PR builds. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for eddc9fd - Browse repository at this point
Copy the full SHA eddc9fdView commit details -
scalar: .scalarCache should live above enlistment
We should not be putting the .scalarCache inside the enlistment as a sibling to the 'src' directory. This only happens in "unattended" mode, but it also negates any benefit of a shared object cache because each enlistment absolutely does not share any objects with others. Move the shared object cache in this case to a level above the enlistment, so at least there is some hope that it can be reused. This is also critical to the upcoming --no-src option, since the shared object cache cannot be located within the Git repository. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 8cc95bc - Browse repository at this point
Copy the full SHA 8cc95bcView commit details -
sparse-checkout: add config to disable deleting dirs
The clean_tracked_sparse_directories() method deletes the tracked directories that go out of scope when the sparse-checkout cone changes, at least in cone mode. This is new behavior, but is recommended based on our understanding of how users are interacting with the feature in most cases. It is possible that some users will object to the new behavior, so create a new configuration option 'index.deleteSparseDirectories' that can be set to 'false' to make clean_tracked_sparse_directories() do nothing. This will keep all untracked files in the working tree and cause performance problems with the sparse index, but those trade-offs are for the user to decide. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for c147171 - Browse repository at this point
Copy the full SHA c147171View commit details -
add/rm: allow adding sparse entries when virtual
Upstream, a20f704 (add: warn when asked to update SKIP_WORKTREE entries, 2021-04-08) modified how 'git add <pathspec>' works with cache entries marked with the SKIP_WORKTREE bit. The intention is to prevent a user from accidentally adding a path that is outside their sparse-checkout definition but somehow matches an existing index entry. A similar change for 'git rm' happened in d5f4b82 (rm: honor sparse checkout patterns, 2021-04-08). This breaks when using the virtual filesystem in VFS for Git. It is rare, but we could be in a scenario where the user has staged a change and then the file is projected away. If the user re-adds the file, then this warning causes the command to fail with the advise message. Disable this logic when core_virtualfilesystem is enabled. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 146c851 - Browse repository at this point
Copy the full SHA 146c851View commit details -
diff: ignore sparse paths in diffstat
The diff_populate_filespec() method is used to describe the diff after a merge operation is complete, especially when a conflict appears. In order to avoid expanding a sparse index, the reuse_worktree_file() needs to be adapted to ignore files that are outside of the sparse-checkout cone. The file names and OIDs used for this check come from the merged tree in the case of the ORT strategy, not the index, hence the ability to look into these paths without having already expanded the index. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 4832a77 - Browse repository at this point
Copy the full SHA 4832a77View commit details -
Merge pull request #392: add: allow adding sparse entries when virtual
Upstream, a20f704 (add: warn when asked to update SKIP_WORKTREE entries, 04-08-2021) modified how 'git add <pathspec>' works with cache entries marked with the SKIP_WORKTREE bit. The intention is to prevent a user from accidentally adding a path that is outside their sparse-checkout definition but somehow matches an existing index entry. This breaks when using the virtual filesystem in VFS for Git. It is rare, but we could be in a scenario where the user has staged a change and then the file is projected away. If the user re-adds the file, then this warning causes the command to fail with the advise message. Disable this logic when core_virtualfilesystem is enabled. This should allow the VFS for Git functional tests to pass (at least the ones in the default run). I'll create a `-pr` installer build to check before merging this.
Configuration menu - View commit details
-
Copy full SHA for 9e2b398 - Browse repository at this point
Copy the full SHA 9e2b398View commit details -
repo-settings: enable sparse index by default
There is some strangeness when expanding a sparse-index that exists within a submodule. We will need to resolve that later, but for now, let's do a better job of explicitly disabling the sparse-index when requested, and do so in t7817. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 7d11caf - Browse repository at this point
Copy the full SHA 7d11cafView commit details -
Merge pull request #410: Sparse Index: latest integrations
``` 6e74958 p2000: add 'git checkout -' test and decrease depth 3e1d03c p2000: compress repo names cd94f82 commit: integrate with sparse-index 65e79b8 sparse-index: recompute cache-tree e9a9981 checkout: stop expanding sparse indexes 4b801c8 t1092: document bad 'git checkout' behavior 71e3015 unpack-trees: resolve sparse-directory/file conflicts 5e96df4 t1092: test merge conflicts outside cone defab1b add: allow operating on a sparse-only index 9fc4313 pathspec: stop calling ensure_full_index 0ec03ab add: ignore outside the sparse-checkout in refresh() adf5b15 add: remove ensure_full_index() with --renormalize ``` These commits are equivalent to those already in `next` via gitgitgadget#999. ``` 80b8d6c Merge branch 'sparse-index/add' into stolee/sparse-index/add ``` This merge resolves conflicts with some work that happened in parallel, but is already in upstream `master`. ``` c407b2c t7519: rewrite sparse index test 9dad0d2 sparse-index: silently return when not using cone-mode patterns 2974920 sparse-index: silently return when cache tree fails e7cdaa0 unpack-trees: fix nested sparse-dir search 347410c sparse-checkout: create helper methods 4537233 attr: be careful about sparse directories 5282a86 sparse-index: add SPARSE_INDEX_MEMORY_ONLY flag 3a2f316 sparse-checkout: clear tracked sparse dirs fb47b56 sparse-checkout: add config to disable deleting dirs ``` These commits are the ones under review as of gitgitgadget#1009. Recent review made this less stable. It's a slightly different and more robust version of #396. > Note: I'm still not done with the feedback for upstream, but the remaining feedback is "can we add tests that cover these tricky technical bits?" and in `microsoft/git` these are already covered by the Scalar functional tests (since that's how they were found). ``` 080b02c diff: ignore sparse paths in diffstat d91a647 merge: make sparse-aware with ORT df49b5f merge-ort: expand only for out-of-cone conflicts cdecb85 t1092: add cherry-pick, rebase tests 0c1ecfb sequencer: ensure full index if not ORT strategy 406dfbe sparse-index: integrate with cherry-pick and rebase ``` These commits integrate with `git merge`, `git cherry-pick`, `git revert`, and `git rebase` as of gitgitgadget#1019. This got some feedback that changed how the tests were working so they are more robust. This led to a new commit (0c1ecfb). ``` cbb0ab3 Merge branch 'sparse-index/merge' into vfs-2.33.0 acb8623 t7524: test no longer fails ``` Finally, the commits are merged into `vfs-2.33.0` and also we include a fix to a `microsoft/git` test that is no longer broken.
Configuration menu - View commit details
-
Copy full SHA for f6b0002 - Browse repository at this point
Copy the full SHA f6b0002View commit details -
Merge branch 'scalar-reconfigure'
This allows fixing settings after a Scalar upgrade, or after botching the enlistments configuration. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9704f54 - Browse repository at this point
Copy the full SHA 9704f54View commit details -
diff(sparse-index): verify with partially-sparse
This verifies that `diff` and `diff --staged` behave the same in sparse index repositories in the following partially-staged scenarios (i.e. the index, HEAD, and working directory differ at a given path): 1. Path is within sparse-checkout cone. 2. Path is outside sparse-checkout cone. 3. A merge conflict exists for paths outside sparse-checkout cone. Signed-off-by: Lessley Dennington <lessleydennington@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 7ee263f - Browse repository at this point
Copy the full SHA 7ee263fView commit details -
Merge pull request #414: Make sparse index the default
This branch is exactly #410, but with one more commit: enabling the sparse index by default in d59110a. Having this in the `vfs-2.33.0` branch helps build confidence that the sparse index is doing what it should be doing by running in the Scalar functional tests and in our test branches. If we want to cut a new `microsoft/git` release without enabling the sparse index, we can simply revert this commit.
Configuration menu - View commit details
-
Copy full SHA for 2d3da85 - Browse repository at this point
Copy the full SHA 2d3da85View commit details -
Merge pull request #536: Allow --no-src during clones and git worktre…
…e after clones These are two highly-requested items from an internal team considering a move to Scalar using Azure Repos. 1. Remove the requirement that we create a `src` directory at clone time. 2. Allow `git worktree` even when using the GVFS protocol. These are not difficult to implement. The `--no-src` option could even be submitted upstream (though the commit will need to drop one bit about an interaction with the local cache path).
Configuration menu - View commit details
-
Copy full SHA for d7141fa - Browse repository at this point
Copy the full SHA d7141faView commit details -
stash: expand testing for
git stash -u
Test cases specific to handling untracked files in `git stash` a) ensure that files outside the sparse checkout definition are handled as-expected and b) document the index expansion inside of `git stash -u`. Note that, in b), it is not the full repository index that is expanded - it is the temporary, standalone index containing the stashed untracked files only. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for b5689de - Browse repository at this point
Copy the full SHA b5689deView commit details -
Merge pull request #419 from ldennington/sparse-index-diff
diff: enable and test the sparse index
Configuration menu - View commit details
-
Copy full SHA for b4c88e6 - Browse repository at this point
Copy the full SHA b4c88e6View commit details -
sequencer: avoid progress when stderr is redirected
During a run of the Scalar functional tests, we hit a case where the inexact rename detection of a 'git cherry-pick' command slowed to the point of writing its delayed progress, failing the test because stderr differed from the control case. Showing progress like this when stderr is not a terminal is non-standard for Git, so inject an isatty(2) when initializing the progress option in sequencer.c. Unfortunately, there is no '--quiet' option in 'git cherry-pick' currently wired up. This could be considered in the future, and the isatty(2) could be moved to that position. This would also be needed for commands like 'git rebase', so we leave that for another time. Reported-by: Victoria Dye <vdye@github.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for b2b0d53 - Browse repository at this point
Copy the full SHA b2b0d53View commit details -
Merge pull request #430 from vdye/sparse-index/clean
Sparse index: integrate with `clean` and `stash -u`
Configuration menu - View commit details
-
Copy full SHA for b131d21 - Browse repository at this point
Copy the full SHA b131d21View commit details -
Merge pull request #432: sequencer: avoid progress when stderr is red…
…irected During a run of the Scalar functional tests, we hit a case where the inexact rename detection of a 'git cherry-pick' command slowed to the point of writing its delayed progress, failing the test because stderr differed from the control case. Showing progress like this when stderr is not a terminal is non-standard for Git, so inject an isatty(2) when initializing the progress option in sequencer.c. Unfortunately, there is no '--quiet' option in 'git cherry-pick' currently wired up. This could be considered in the future, and the isatty(2) could be moved to that position. This would also be needed for commands like 'git rebase', so we leave that for another time.
Configuration menu - View commit details
-
Copy full SHA for f868440 - Browse repository at this point
Copy the full SHA f868440View commit details -
reset: fix mixed reset when using virtual filesystem
During the 2.35.0 rebase, we ejected 570f64b (Fix reset when using the sparse-checkout feature., 2017-03-15) because of a similar change upstream that actually works with the expected behavior of sparse-checkout. That commit only ever existed in microsoft/git, but when it was considered for upstream we realized that it behaved strangely for a sparse-checkout scenario. The root problem is that during a mixed reset, 'git reset <commit>' updates the index to aggree with <commit> but leaves the worktree the same as it was before. The issue with sparse-checkout is that some files might not be in the worktree and thus the information from those files would be "lost". The upstream decision was to leave these files as ignored, because that's what the SKIP_WORKTREE bit means: don't put these files in the worktree and ignore their contents. If there already were files in the worktree, then Git does not change them. The case for "losing" data is if a committed change outside of the sparse-checkout was in the previous HEAD position. However, this information could be recovered from the reflog. The case where this is different is in a virtualized filesystem. The virtualization is projecting the index contents onto the filesystem, so we need to do something different here. In a virtual environment, every file is considered "important" and we abuse the SKIP_WORKTREE bit to indicate that Git does not need to process a projected file. When a file is populated, the virtual filesystem hook provides the information for removing the SKIP_WORKTREE bit. In the case of these mixed resets, we have the issue where we change the projection of the worktree for these cache entries that change. If a file is populated in the worktree, then the populated file will persist and appear in a follow-up 'git status'. However, if the file is not populated and only projected, we change the projection from the current value to the new value, leaving a clean 'git status'. The previous version of this commit includes a call to checkout_entry(), which populates the file. This causes the file to be actually in the working tree and no longer projected. To make this work with the upstream changes, stop setting the skip-worktree bit for the new cache entry. This seemed to work fine without this change, but it's likely due to some indirection with the virtual filesystem. Better to do the best-possible thing here so we don't hide a corner-case bug by accident. Helped-by: Victoria Dye <vdye@github.com> Signed-off-by: Kevin Willford <kewillf@microsoft.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for c9ceb95 - Browse repository at this point
Copy the full SHA c9ceb95View commit details -
sparse: add vfs-specific precautions
* t1092: remove the 'git update-index' test that currently fails because the command ignores the bad path, but doesn't return a failure. * dir.c: prevent matching against sparse-checkout patterns when the virtual filesystem is enabled. Should prevent some corner case issues. * t1092: add quiet mode for some rebase tests because the stderr output can change in some of the modes. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 9229e50 - Browse repository at this point
Copy the full SHA 9229e50View commit details -
These were done in private, before microsoft/git. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 0fbf6f8 - Browse repository at this point
Copy the full SHA 0fbf6f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2a8ec98 - Browse repository at this point
Copy the full SHA 2a8ec98View commit details -
Add virtual file system settings and hook proc. On index load, clear/set the skip worktree bits based on the virtual file system data. Use virtual file system data to update skip-worktree bit in unpack-trees. Use virtual file system data to exclude files and folders not explicitly requested. The hook was first contributed in private, but was extended via the following pull requests: #15 #27 #33 #70 Signed-off-by: Ben Peart <Ben.Peart@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 3adaa15 - Browse repository at this point
Copy the full SHA 3adaa15View commit details -
Configuration menu - View commit details
-
Copy full SHA for ef3d676 - Browse repository at this point
Copy the full SHA ef3d676View commit details -
Merge pull request #494: reset: fix mixed reset when using virtual fi…
…lesystem This replaces #493 (can't reopen a PR after a force-push...). I updated this commit with a more firm version of the fix. This hopefully answers Victoria's excellent concerns with the previous approach. I did not manage to get an automated test for this, but I did carefully verify this manually with a few commits in a VFS for Git enlistment (with different files every time). I updated the commit message with more details about why this works. --- This fork contains changes specific to monorepo scenarios. If you are an external contributor, then please detail your reason for submitting to this fork: * [X] This change only applies to the virtualization hook and VFS for Git. Resolves #490.
Configuration menu - View commit details
-
Copy full SHA for b2fea5d - Browse repository at this point
Copy the full SHA b2fea5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1d0b9cf - Browse repository at this point
Copy the full SHA 1d0b9cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1dfe6d7 - Browse repository at this point
Copy the full SHA 1dfe6d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 564da1d - Browse repository at this point
Copy the full SHA 564da1dView commit details -
Harden gvfs-helper to validate the packfiles in a multipart prefetch …
…response (#571) Teach `gvfs-helper` to ignore the optional `.idx` files that may be included in a `prefetch` response and always use `git index-pack` to create them from the `.pack` files received in the data stream. This is a little wasteful in terms of client-side compute and of the network bandwidth, but allows us to use the full packfile verification code contained within `git index-pack` to ensure that the received packfiles are valid.
Configuration menu - View commit details
-
Copy full SHA for 0c96727 - Browse repository at this point
Copy the full SHA 0c96727View commit details -
Merge pull request #301: Update 'git maintenance' to match upstream
This PR updates our `vfs-2.29.0` branch's version of `git maintenance` to match the latest in upstream. Unfortunately, not all of these commits made it to the `2.30` release candidate, but there are more commits from the series making it in. They will cause a conflict in the `vfs-2.30.0` rebase, so merge them in here. This also includes the `fixup!` reverts of the earlier versions. Finally, I also noticed that we started depending on `git maintenance start` in Scalar for macOS, but we never checked that this worked with the shared object cache. It doesn't! 😨 The very tip commit of this PR includes logic to make `git maintenance run` care about `gvfs.sharedCache`. Functional test updates in Scalar will follow.
Configuration menu - View commit details
-
Copy full SHA for e5936d5 - Browse repository at this point
Copy the full SHA e5936d5View commit details -
Merge pull request #315: unpack-trees:virtualfilesystem: Improve effi…
…ciency of clear_ce_flags When the virtualfilesystem is enabled the previous implementation of clear_ce_flags would iterate all of the cache entries and query whether each one is in the virtual filesystem to determine whether to clear one of the SKIP_WORKTREE bits. For each cache entry, we would do a hash lookup for each parent directory in the is_included_in_virtualfilesystem function. The former approach is slow for a typical Windows OS enlistment with 3 million files where only a small percentage is in the virtual filesystem. The cost is O(n_index_entries * n_chars_per_path * n_parent_directories_per_path). In this change, we use the same approach as apply_virtualfilesystem, which iterates the set of entries in the virtualfilesystem and searches in the cache for the corresponding entries in order to clear their flags. This approach has a cost of O(n_virtual_filesystem_entries * n_chars_per_path * log(n_index_entries)). The apply_virtualfilesystem code was refactored a bit and modified to clear flags for all names that 'alias' a given virtual filesystem name when ignore_case is set. n_virtual_filesystem_entries is typically much less than n_index_entries, in which case the new approach is much faster. We wind up building the name hash for the index, but this occurs quickly thanks to the multi-threading.
Configuration menu - View commit details
-
Copy full SHA for be2a47b - Browse repository at this point
Copy the full SHA be2a47bView commit details -
Adding a few workflows to publish releases! 🥳
Configuration menu - View commit details
-
Copy full SHA for ce46bd8 - Browse repository at this point
Copy the full SHA ce46bd8View commit details -
Merge branch 'adjust-g4w-workflows'
This disables the `monitor-components` workflow in msft-git, and re-enables the `win+VS` tests in this repository, too.
Configuration menu - View commit details
-
Copy full SHA for 106e466 - Browse repository at this point
Copy the full SHA 106e466View commit details -
Merge pull request #399 from vdye/feature/build-installers
Implement workflow to create GitHub release with attached `git` installers
Configuration menu - View commit details
-
Copy full SHA for e7b5b7d - Browse repository at this point
Copy the full SHA e7b5b7dView commit details -
Merge pull request #472 from vdye/ms/macos-build-options
Fixes for MacOS release build & build options
Configuration menu - View commit details
-
Copy full SHA for 5efa558 - Browse repository at this point
Copy the full SHA 5efa558View commit details -
Merge pull request #329: Add
git update-microsoft-git
This adds a new builtin, `git update-microsoft-git`, that executes the platform-specific upgrade steps to get the latest version of `microsoft-git`. On Windows, this means running `git update-git-for-windows` which was updated to use the `microsoft/git` releases page, when appropriate. See #321 for details. On macOS, this means running a sequence of `brew` commands. These are adapted from the `UpgradeVerb` in `microsoft/scalar`, with an important simplification: we don't need to differentiate between the `scalar` and `scalar-azrepos` cask.
Configuration menu - View commit details
-
Copy full SHA for c5a06e4 - Browse repository at this point
Copy the full SHA c5a06e4View commit details -
Merge pull request #333: update microsoft/git README
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 5fab868 - Browse repository at this point
Copy the full SHA 5fab868View commit details -
Merge pull request #371 from dscho/run-scalar-functional-tests-and-fi…
…x-built-in-fsmonitor Fix the built-in FSMonitor, and run Scalar's Functional Tests as part of the automated builds
Configuration menu - View commit details
-
Copy full SHA for 997e354 - Browse repository at this point
Copy the full SHA 997e354View commit details -
Merge branch 'sparse-index-stuff'
This is random stuff that probably all got upstream in the meantime.
Configuration menu - View commit details
-
Copy full SHA for 55a59e8 - Browse repository at this point
Copy the full SHA 55a59e8View commit details -
credential: add new interactive config option
When scripts or background maintenance wish to perform HTTP(S) requests, there is a risk that our stored credentials might be invalid. At the moment, this causes the credential helper to ping the user and block the process. Even if the credential helper does not ping the user, Git falls back to the 'askpass' method, which includes a direct ping to the user via the terminal. Even setting the 'core.askPass' config as something like 'echo' will causes Git to fallback to a terminal prompt. It uses git_terminal_prompt(), which finds the terminal from the environment and ignores whether stdin has been redirected. This can also block the process awaiting input. Create a new config option to prevent user interaction, favoring a failure to a blocked process. The chosen name, 'credential.interactive', is taken from the config option used by Git Credential Manager to already avoid user interactivity, so there is already one credential helper that integrates with this option. However, older versions of Git Credential Manager also accepted other string values, including 'auto', 'never', and 'always'. The modern use is to use a boolean value, but we should still be careful that some users could have these non-booleans. Further, we should respect 'never' the same as 'false'. This is respected by the implementation and test, but not mentioned in the documentation. The implementation for the Git interactions takes place within credential_getpass(). The method prototype is modified to return an 'int' instead of 'void'. This allows us to detect that no attempt was made to fill the given credential, changing the single caller slightly. Also, a new trace2 region is added around the interactive portion of the credential request. This provides a way to measure the amount of time spent in that region for commands that _are_ interactive. It also makes a conventient way to test that the config option works with 'test_region'. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 2a05783 - Browse repository at this point
Copy the full SHA 2a05783View commit details -
t5300: confirm failure of git index-pack when non-idx suffix requested
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path` fails if <idx-path> does not end in ".idx" when `--rev-index` is enabled. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. The test here demonstrates the failure. (The test forces `--rev-index` to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.) Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for 5cadc4c - Browse repository at this point
Copy the full SHA 5cadc4cView commit details -
maintenance: add custom config to background jobs
At the moment, some background jobs are getting blocked on credentials during the 'prefetch' task. This leads to other tasks, such as incremental repacks, getting blocked. Further, if a user manages to fix their credentials, then they still need to cancel the background process before their background maintenance can continue working. Update the background schedules for our four scheduler integrations to include these config options via '-c' options: * 'credential.interactive=false' will stop Git and some credential helpers from prompting in the UI (assuming the '-c' parameters are carried through and respected by GCM). * 'core.askPass=true' will replace the text fallback for a username and password into the 'true' command, which will return a success in its exit code, but Git will treat the empty string returned as an invalid password and move on. We can do some testing that the credentials are passed, at least in the systemd case due to writing the service files. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 0ae4ac7 - Browse repository at this point
Copy the full SHA 0ae4ac7View commit details -
index-pack: disable rev-index if index file has non .idx suffix
Teach index-pack to silently omit the reverse index if the index file does not have the standard ".idx" suffix. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. Since we do not know why the user requested a non-standard suffix for the index, we cannot guess what the proper corresponding suffix should be for the reverse index. So we disable it. The t5300 test has been updated to verify that we no longer error out and that the .rev file is not created. TODO We could warn the user that we skipped it (perhaps only if they TODO explicitly requested `--rev-index` on the command line). TODO TODO Ideally, we should add an `--rev-index-path=<path>` argument TODO or change `--rev-index` to take a pathname. TODO TODO I'll leave these questions for a future series. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for ae6e1c9 - Browse repository at this point
Copy the full SHA ae6e1c9View commit details -
scalar: configure maintenance during 'reconfigure'
The 'scalar reconfigure' command is intended to update registered repos with the latest settings available. However, up to now we were not reregistering the repos with background maintenance. In particular, this meant that the background maintenance schedule would not be updated if there are improvements between versions. Be sure to register repos for maintenance during the reconfigure step. Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Configuration menu - View commit details
-
Copy full SHA for 8336d80 - Browse repository at this point
Copy the full SHA 8336d80View commit details -
trace2: redact passwords from https:// URLs by default
It is an unsafe practice to call something like git clone https://user:password@example.com/ This not only risks leaking the password "over the shoulder" or into the readline history of the current Unix shell, it also gets logged via Trace2 if enabled. Let's at least avoid logging such secrets via Trace2, much like we avoid logging secrets in `http.c`. Much like the code in `http.c` is guarded via `GIT_TRACE_REDACT` (defaulting to `true`), we guard the new code via `GIT_TRACE2_REDACT` (also defaulting to `true`). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7229680 - Browse repository at this point
Copy the full SHA 7229680View commit details -
t0211: test URL redacting in PERF format
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for e8eec34 - Browse repository at this point
Copy the full SHA e8eec34View commit details -
trace2: fix signature of trace2_def_param() macro
Add `struct key_value_info` argument to `trace2_def_param()`. In dc90208 (trace2: plumb config kvi, 2023-06-28) a `kvi` argument was added to `trace2_def_param_fl()` but the macro was not up updated. Let's fix that. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for ff13c04 - Browse repository at this point
Copy the full SHA ff13c04View commit details -
sparse-index.c: fix use of index hashes in expand_index
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23), 'expand_index()' was updated to expand the index to a given pathspec. However, the 'path_matches_pattern_list()' method used to facilitate this has the side effect of initializing or updating the index hash variables ('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is performed on 'istate', though, not 'full'; as a result, the initialized hashes are later overwritten when copied from 'full'. To ensure the correct hashes are in 'istate' after the index expansion, change the arg used in 'path_matches_pattern_list()' from 'istate' to 'full'. Note that this does not fully solve the problem. If 'istate' does not have an initialized 'name_hash' when its contents are copied to 'full', initialized hashes will be copied back into 'istate' but 'name_hash_initialized' will be 0. Therefore, we also need to copy 'full->name_hash_initialized' back to 'istate' after the index expansion is complete. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for b216189 - Browse repository at this point
Copy the full SHA b216189View commit details -
t0212: test URL redacting in EVENT format
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Configuration menu - View commit details
-
Copy full SHA for 8548168 - Browse repository at this point
Copy the full SHA 8548168View commit details -
t1092: add test for untracked files and directories
Add a test verifying that sparse-checkout (with and without sparse index enabled) treat untracked files & directories correctly when changing sparse patterns. Specifically, it ensures that 'git sparse-checkout set' * deletes empty directories outside the sparse cone * does _not_ delete untracked files outside the sparse cone Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for 18c885d - Browse repository at this point
Copy the full SHA 18c885dView commit details -
Fix rev index in 2.42.0 series (#600)
Cherry-pick rev-index fixes from v2.41.0.vfs.0.5 into v2.42.0.*
Configuration menu - View commit details
-
Copy full SHA for fcf67c9 - Browse repository at this point
Copy the full SHA fcf67c9View commit details -
t0210,t0211: unset SANITIZE_LEAK
Turn off TEST_PASSES_SANITIZE_LEAK in t0210 and t0211 tests. The tests added in a previous commit to confirm that we redact URLs in the Trace2 output uncovered leaks in `builtin/clone.c` and `remote.c`. We observed that (1) `the_repository->remote_status` is not released properly. And (2) that we are using `url...insteadOf` and that runs into a code path where an allocated URL is replaced with another URL. And (3) `remote_states` contains plenty of `struct remote`s whose refspecs seem to be usually allocated by never released. More investigation is needed here to identify the exact cause and proper fixes these leaks / bugs. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 52c75cc - Browse repository at this point
Copy the full SHA 52c75ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7a3b441 - Browse repository at this point
Copy the full SHA 7a3b441View commit details -
Redact unsafe URLs in the Trace2 output (#616)
The Trace2 output can contain secrets when a user issues a Git command with sensitive information in the command-line. A typical (if highly discouraged) example is: `git clone https://user:password@host.com/`. With this PR, the Trace2 output redacts passwords in such URLs by default. This series also includes a commit to temporarily disable leak checking on t0210,t0211 because the tests uncover other unrelated bugs in Git.
Configuration menu - View commit details
-
Copy full SHA for b7a6ed7 - Browse repository at this point
Copy the full SHA b7a6ed7View commit details