Merge branch 'jh/experimental-survey'

This topic branch brings in a new, experimental built-in command to
assess the dimensions of a local repository.

It is experimental and subject to change! It might grow new options,
change its output, or even be moved into `git diagnose --analyze` or
something like that.

The hope is that this command, which was inspired by `git sizer`
(https://github.com/github/git-sizer), will be helpful not only in
diagnosing issues with large repositories, but also in modeling what
shapes and sizes of repositories can be handled by Git (and as a
corollary: where Git needs to improve to be able to accommodate the
natural growth of repositories).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

.gitignore

@@ -164,6 +164,7 @@
 /git-submodule
 /git-submodule--helper
 /git-subtree
+/git-survey
 /git-svn
 /git-switch
 /git-symbolic-ref

Documentation/config.txt

@@ -536,6 +536,8 @@ include::config/status.txt[]
 
 include::config/submodule.txt[]
 
+include::config/survey.txt[]
+
 include::config/tag.txt[]
 
 include::config/tar.txt[]

Documentation/config/survey.txt

@@ -0,0 +1,41 @@
+survey.namerev::
+	Boolean to show/hide `git name-rev` information for
+	each reported commit and the containing commit of each
+	reported tree and blob.
+
+survey.progress::
+	Boolean to show/hide progress information.  Defaults to
+	true when interactive (stderr is bound to a TTY).
+
+survey.showBlobSizes::
+	A non-negative integer value.  Requests details on the <n>
+	largest file blobs by size in bytes.  Provides a default
+	value for `--blob-sizes=<n>` in linkgit:git-survey[1].
+
+survey.showCommitParents::
+	A non-negative integer value.  Requests details on the <n>
+	commits with the most number of parents.  Provides a default
+	value for `--commit-parents=<n>` in linkgit:git-survey[1].
+
+survey.showCommitSizes::
+	A non-negative integer value.  Requests details on the <n>
+	largest commits by size in bytes.  Generally, these are the
+	commits with the largest commit messages.  Provides a default
+	value for `--commit-sizes=<n>` in linkgit:git-survey[1].
+
+survey.showTreeEntries::
+	A non-negative integer value.  Requests details on the <n>
+	trees (directories) with the most number of entries (files
+	and subdirectories).  Provides a default value for
+	`--tree-entries=<n>` in linkgit:git-survey[1].
+
+survey.showTreeSizes::
+	A non-negative integer value.  Requests details on the <n>
+	largest trees (directories) by size in bytes.  This will
+	set will usually be equal to the `survey.showTreeEntries`
+	set, but may be skewed by very long file or subdirectory
+	entry names.  Provides a default value for
+	`--tree-sizes=<n>` in linkgit:git-survey[1].
+
+survey.verbose::
+	Boolean to show/hide verbose output.  Default to false.

Documentation/git-survey.txt

@@ -0,0 +1,108 @@
+git-survey(1)
+=============
+
+NAME
+----
+git-survey - EXPERIMENTAL: Measure various repository dimensions of scale
+
+SYNOPSIS
+--------
+[verse]
+(EXPERIMENTAL!) `git survey` <options>
+
+DESCRIPTION
+-----------
+
+Survey the repository and measure various dimensions of scale.
+
+As repositories grow to "monorepo" size, certain data shapes can cause
+performance problems.  `git-survey` attempts to measure and report on
+known problem areas.
+
+Ref Selection and Reachable Objects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In this first analysis phase, `git survey` will iterate over the set of
+requested branches, tags, and other refs and treewalk over all of the
+reachable commits, trees, and blobs and generate various statistics.
+
+OPTIONS
+-------
+
+--progress::
+	Show progress.  This is automatically enabled when interactive.
+
+--json::
+	Print results in JSON rather than in a human-friendly format.
+
+--[no-]name-rev::
+	Print `git name-rev` output for each commit, tree, and blob.
+	Defaults to true.
+
+Ref Selection
+~~~~~~~~~~~~~
+
+The following options control the set of refs that `git survey` will examine.
+By default, `git survey` will look at tags, local branches, and remote refs.
+If any of the following options are given, the default set is cleared and
+only refs for the given options are added.
+
+--all-refs::
+	Use all refs.  This includes local branches, tags, remote refs,
+	notes, and stashes.  This option overrides all of the following.
+
+--branches::
+	Add local branches (`refs/heads/`) to the set.
+
+--tags::
+	Add tags (`refs/tags/`) to the set.
+
+--remotes::
+	Add remote branches (`refs/remote/`) to the set.
+
+--detached::
+	Add HEAD to the set.
+
+--other::
+	Add notes (`refs/notes/`) and stashes (`refs/stash/`) to the set.
+
+Large Item Selection
+~~~~~~~~~~~~~~~~~~~~
+
+The following options control the optional display of large items under
+various dimensions of scale.  The OID of the largest `n` objects will be
+displayed in reverse sorted order.  For each, `n` defaults to 10.
+
+--commit-parents::
+	Shows the OIDs of the commits with the most parent commits.
+
+--commit-sizes::
+	Shows the OIDs of the largest commits by size in bytes.  This is
+	usually the ones with the largest commit messages.
+
+--tree-entries::
+	Shows the OIDs of the trees with the most number of entries.  These
+	are the directories with the most number of files or subdirectories.
+
+--tree-sizes::
+	Shows the OIDs of the largest trees by size in bytes.  This set
+	will usually be the same as the vector of number of entries unless
+	skewed by very long entry names.
+
+--blob-sizes::
+	Shows the OIDs of the largest blobs by size in bytes.
+
+OUTPUT
+------
+
+By default, `git survey` will print information about the repository in a
+human-readable format that includes overviews and tables.
+
+CONFIGURATION
+-------------
+
+include::config/survey.txt[]
+
+GIT
+---
+Part of the linkgit:git[1] suite

Makefile

@@ -1309,6 +1309,7 @@ BUILTIN_OBJS += builtin/sparse-checkout.o
 BUILTIN_OBJS += builtin/stash.o
 BUILTIN_OBJS += builtin/stripspace.o
 BUILTIN_OBJS += builtin/submodule--helper.o
+BUILTIN_OBJS += builtin/survey.o
 BUILTIN_OBJS += builtin/symbolic-ref.o
 BUILTIN_OBJS += builtin/tag.o
 BUILTIN_OBJS += builtin/unpack-file.o

builtin.h

@@ -238,6 +238,7 @@ int cmd_status(int argc, const char **argv, const char *prefix);
 int cmd_stash(int argc, const char **argv, const char *prefix);
 int cmd_stripspace(int argc, const char **argv, const char *prefix);
 int cmd_submodule__helper(int argc, const char **argv, const char *prefix);
+int cmd_survey(int argc, const char **argv, const char *prefix);
 int cmd_switch(int argc, const char **argv, const char *prefix);
 int cmd_symbolic_ref(int argc, const char **argv, const char *prefix);
 int cmd_tag(int argc, const char **argv, const char *prefix);