Manage the aggregation of git branches from different remotes to build a consolidated one.
Create a repos.yaml
or repos.yml
file:
./product_attribute:
remotes:
oca: https://github.com/OCA/product-attribute.git
acsone: git+ssh://git@github.com/acsone/product-attribute.git
merges:
- oca 8.0
- oca refs/pull/105/head
- oca refs/pull/106/head
./connector-interfaces:
remotes:
oca: https://github.com/OCA/connector-interfaces.git
acsone: https://github.com/acsone/connector-interfaces.git
merges:
- oca 6054de2c4e669f85cec380da90d746061967dc83
- acsone 8.0-connector_flow
- acsone 80_connector_flow_ir_cron_able-lmi
- acsone 8.0_connector_flow_improve_eval_config
target: acsone aggregated_branch_name
fetch_all:
- oca
Environment variables inside of this file will be expanded if the proper option is selected.
All the merges
are combined into a single branch. By default this branch is called _git_aggregated
but another name may be given in the target
section.
If any of your merges refer to a specific commit, you will probably need to fetch all remotes from the corresponding remote or use any other strategy to get that fetch working, but we recommend to simply add this like in the example above:
fetch_all:
- oca
- other-remote
You can specify that you want to fetch all references from all remotes you have defined with:
fetch_all: true
To save big amounts of bandwidth and disk space, you can use shallow clones. These download only a restricted amount of commits depending on some criteria. Available options are depth, shallow-since and shallow-exclude.
Warning
Available options depend on server and client Git version, be sure to use options available for your environment.
You can use those in the defaults
sections to apply them everywhere, or
specifying them in the corresponding merges
section, for which you must use
the dict
alternate construction. If you need to disable a default in
merges
, set it to false
:
./odoo:
defaults:
depth: 20
remotes:
odoo: https://github.com/odoo/odoo.git
ocb: https://github.com/OCA/OCB.git
acsone: https://github.com/acsone/odoo.git
merges:
-
remote: ocb
ref: "9.0"
depth: 1000
-
remote: odoo
ref: refs/pull/14859/head
target: acsone 9.0
Remember that you need to fetch at least the common ancestor of all merges for it to succeed.
It's also possible to specify a command or a list of shell commands to execute after the aggregation (and before the push). The commands are executed into the aggregated directory.
./product_attribute:
remotes:
oca: https://github.com/OCA/product-attribute.git
acsone: git+ssh://git@github.com/acsone/product-attribute.git
merges:
- oca 8.0
target: acsone aggregated_branch_name
shell_command_after: echo 'my command'
./connector-interfaces:
remotes:
oca: https://github.com/OCA/connector-interfaces.git
acsone: https://github.com/acsone/connector-interfaces.git
merges:
- oca 9.0
target: acsone aggregated_branch_name
shell_command_after:
- echo 'a first command'
- echo 'a second command'
A real life example: applying a patch
./odoo:
remotes:
oca: https://github.com/OCA/OCB.git
acsone: git@github.com/acsone/OCB.git
merges:
- oca 9.0
target: acsone aggregated_branch_name
shell_command_after:
- git am "$(git format-patch -1 XXXXXX -o ../patches)"
Following the example repos.yaml
file from above, aggregate your
repositories at any time:
$ gitaggregate -c repos.yaml
Expand environment variables inside of the configuration file when loading:
$ gitaggregate -c repos.yaml --expand-env
The variables in the configuration file can be specified in one of the following ways:
- $VARIABLE
- ${VARIABLE}
For more information, see the Python's string.Template documentation.
Use additional variables from file while expanding:
$ gitaggregate -c repos.yaml --expand-env --env-file .env
The env file should contain VAR=value lines. Lines starting with # are ignored.
You can also aggregate and automatically push the result to the target, if the
target
option is configured:
$ gitaggregate -c repos.yaml -p
Only aggregate a specific repository using fnmatch:
$ gitaggregate -c repos.yaml -p -d connector-interfaces
gitaggregate has a mechanism to identify merges that correpond to merged or closed Github pull requests.
Such merges are of the form refs/pull/NNN/head where NNN is the pull request number, with a https://github.com or git@github.com remote.
To work around API limitation, you must first generate a Github API token.
$ export GITHUB_TOKEN=...
$ gitaggregate -c repos.yaml show-all-prs
$ gitaggregate -c repos.yaml show-closed-prs
- [BREAKING] drop support for other configuration file formats than yaml
- Ensure git pull is always done in fast-forward mode
- Drop support for python 3.6, test with python 3.11, stop testing with pypy
- Fix git clone issue with git < 2.17
- When updating remotes the log message now states
Updating remote
instead ofRemote remote
- Add
--no-color
option to disable colored output - Use git clone --filter=blob:none + fetch strategy to improve performance and benefit from
git-autoshare
if installed
- Migrate Github API authentication to new spec (https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/)
- Drop support for python < 3.6
- Do not exit with success on KeyboardInterrupt
- Make
target
optional.
- Support environment variables in the configuration file.
- If an error happens, log in which repo it happens. Helpful when running in parallel.
- Fix a bug in
--show-all-prs
, which was printing a wrong PR URL. - Display PR labels too in
--show-all-prs
.
- Add --show-all-prs command to list all GitHub pull requests used in merge sections.
- Add --force. If set, dirty repositories will fail to aggregate.
- Add --jobs option for multi-process operation.
- Improve configuration file parsing by mimicing Kaptan's behavior of resolving handler by extension (#22)
- show-closed-prs now displays merge status
- some documentation improvements
- support .yml config file extension
- add a show-closed-prs command to display github pull requests that are not open anymore; github pull requests must be referenced as refs/pull/NNN/head in the merges section
- Use setuptools_scm for the release process (#10)
- Expand env vars in config. (#8)
- Shallow repositories. (#7)
- Fetch only required remotes. (#6)
- Display readable error if config file not found. (#2)
- First release
- Laurent Mignon (ACSONE)
- Andrei Boyanov
- Cyril Gaudin (camptocamp)
- Jairo Llopis (Tecnativa)
- Stéphane Bidoul (ACSONE)
- Dave Lasley (LasLabs)
- Patric Tombez
- Cristian Moncho
- Simone Orsi (camptocamp)
- Artem Kostyuk
- Jan Verbeek
This project is maintained by ACSONE SA/NV.