Criteria for a repo to be included in an O3DE release?

[tonybalandiuk]

Currently (as of O3DE release 22.10.0, Oct 2022), the release process includes the following O3DE Repositories ("repos"):
- o3de
- o3de-multiplayersample
- o3de-netsoaktest

This means that during the release cycle:
(1) each of these repositories will have a "Stabilization branch" created where big fixing occurs leading up to the release. If any of these repositories are not declared "stable", the release will be held.
(2) when we are ready to release, the code from the stabilization branch for each repository is merged to main is updated
(3) a release tag is applied upon updating the code in these repositories.

Purpose of this discussion:
Determine the specific criteria we use to determine which O3DE repositories to include in the O3DE release process. This discussion may be as simple as declaring "all canonical repos are included in the O3DE release process". Regardless, we want to open this discussion to the O3DE community so everyone can voice their opinion.

Background:
During the 1/17 #sig-release meeting we discussed the possibility of including the O3DE Extras Repo since it is now considered "canonical". See notes at https://github.com/o3de/sig-release/blob/main/meetings/notes/2023%20-%20January%2017th.md

This raised a more general question of "what should be the criteria for a repository to be included in a release?" which will be formally discussed under this discussion.
The O3DE Extras repo, specifically, will be requested to sig-release AFTER this discussion. This discussion is about the broader discussion for any repo, not just O3DE extras.

Why are we talking about this now:
We're talking about this because the O3DE extras repo was brought up as a candidate for inclusion, noting that it is a "canonical repo" as of November 2022.

Considerations:

1. Canonical repos have to follow the rule that they have to use Automated Review (AR). Canonical repos MUST ALL reference ALL other canonical repos in their AR. See https://discord.com/channels/805939474655346758/852574203437121586/1042253958342787142
2. Any repository that is included in an O3DE Release can block a release if the repository is not declared stable.

Related:

1. Discord announcement of O3DE Extras as canonical https://discord.com/channels/805939474655346758/852574203437121586/1042253958342787142

[byrcolin]

We want there to be as few canonical repos as possible because this is a multiplication point for SIG load. For all practical purposes we should think of all canonical repos conceptually as 1 repo, or conceptually as submodules of a master O3DE repo. All canonical repos are therefore considered part of the engine. All canonical repos are tested as working as part of a release of the engine. A breakage in any canonical repo is considered a break in the engine as a whole, just as if it was in the core repo, and thus would halt a release. We should refer to o3de/o3de now as the "core engine repo", it is a special canonical repo as it must be able to be built and run in isolation and therefore must not have any dependencies on any other repo.

Since we consider all canonical repos conceptually as one repo:

• All canonical repos are owned by LF/o3de.org (no exceptions), but NOT all LF/o3de.org repos are canonical.
• All canonical repos must maintain at least 2 branches, the main branch and the development branch so that the main branch can be updated and tagged for a release.
• All canonical repos must have a CODEOWNERS file and all objects/parts of objects in a canonical repo must be owned/managed by a SIG.
• All SIG's must include all canonical repos in their processes, such as triage.
• All issues for an object should reside in its repo, so we should not have issues for extras in the core engine repo, part of triage will be moving the issue to the correct repo.
• All canonical repos must all use the same tags so that there is consistency across canonical repos for how SIG's and the community interact with them. Use of identical tags will also allow issues to be portable between canonical repos, as when objects potentially move from one canonical repo to another the issues associated with that object move with it.

Other considerations:

• There is a object size consideration for all objects and therefore all repos. If an object is large it should not be included in the core engine repo unless absolutely necessary. The extras repo is appropriate for all non-core canonical objects.
• We should not consider any new project object as core. In my opinion we should have a discussion about the inclusion of the automatedtesting project in the core; this seems like this could/should be external.
• Only projects which are intended to demonstrate extras or core engine functions should be considered for extras, and they should all be minimalistic.
• All tutorial/sample/impressive/beautiful projects should neither be in the core nor extras, but in separate non-canonical repos.
• No partner objects should be in either the core or extras.

Future thoughts:

• Ideally, the core engine repo would largely be a collection of interface gems and all implementation gems could be in the extras, but if they are needed by the engine in isolation or would be used by just about everyone in their projects, then the implementation should be in the core engine repo.
• As the core shrinks in size and hardens, the number if issues in the core should decrease and alteration of the core more rare. We could at some point have multiple releases that have no core engine changes. Also the bar for inclusion/alteration of the core engine repo should go up and motivation to move out what can be moved out of the core should increase. It also follows that the default for all new object development should NOT be the core engine repo unless predetermined that it should be by the SIG which owns it. Once the core hardens, the SIG should be prepared to backup a decision to include/alter the core engine objects to the TSC.

[vincent6767]

A question. Can you share how's a repo considered a canonical repo? @byrcolin

[byrcolin]

The process by which a non canonical repo becomes a canonical repo:

Someone believes a repo/object(s) should be canonical and creates a GHI detailing their argument of why it should be so. A sample CODEOWNERS file detailing SIG(s) which would own the object(s) is proposed. If the repo objects originate out of org, i.e. from a community member, then we must NOT trust that community member and we MUST do our DUE DILIGENCE first. Once we have satisfied our due diligence we continue as if it originated in org.

The repo/object(s) originate out of Org else skip to 6.

1. Does this person understand that in order to become canonical, ownership MUST transfer to LF and that the license will be changed to Apache2/MIT?
2. Does the person offering the repo/object(s) actually own the repo/object?
3. Would the SIG want this repo/object(s) if it could have them? if not early exit before potentially spending any resources on 4.
4. LF Due diligence; make sure this is not ripping off someone else's code.

The repo/object(s) originate in Org:

5. The SIG(s) agrees/disagrees with the issue and either dies here or moves forward. (This is mostly done in 3. for out of org)
6. The SIG(s) which agree decides where the repo/object(s) should ultimately reside. Most likely, the object(s) inside the repo will be moved from that current repo into the extras repo. If so, then the extras CODEOWNERS file is updated and this is treated as normal PR. Very rarely should an object be promoted directly to the core engine repo, this should be a red flag if suggested and he TSC should be consulted well before inclusion is allowed. If included into an existing canonical repo in this manner, issues pertaining to the new objects are transferred and done.

If the object is a project or is large the SIG(s) could also decide that the repo should not be integrated into extras but remain as it is, as a separate canonical repo. This must also be approved by the TSC as this would represent a multiplication point for SIG load as this new separate repo would have to be included in every SIG's processes. This should also be a red flag for non-large/non-project objects, as we want to keep the number of canonical repos to a minimal number. If the SIG and TSC agree this should be separate a canonical repo:
7. A new repo must be created in the LF/o3de github org.
8. Two branches must be added if not present, "main" and a branch of main called "development".
9. Main must be empty at first and locked out for submissions.
10. All the standard o3de tags must be replicated into the new canonical repo.
11. All objects must be transferred into the new repos development branch.
12. The agreed upon CODEOWNERS file is added.
13. Standard o3de README is added.
14. Standard o3de LICENSE files are added.
15. A branch of o3de/o3de development is made adding the new repo to o3de/o3de repos .automatedtesting.json REPO section so it is included in o3de AR.
16. A new .automatedtesting.json is added to the new repo and all objects are registered and gems enabled for testing. SIG(s) can determine which gems are enabled for testing.
17. The new repo passes o3de AR (NOT its own AR, but o3de AR i.e. with step 15/16.)
18. LF/Infra stands up a jenkins project which uses the o3de/o3de Jenkinsfile to run AR on the new repo.
19. The new repos code must be conformant to the o3de coding standard as it is a canonical repo.
20. The appropriate headers in all applicable files are present.
21. A rep from the owning SIG(s) must all sign off.
22. The new repo has gating turned on to gate submissions to the new canonical repo development branch along with the standard DCO and 2 reviews requirement.
23. The o3de PR in 15 is committed to o3de/o3de and this new repo officially becomes part of o3de AR.
24. The issues associated with the new repo/objects are imported into the new repo.
25. SIG release must update the release process to include the new canonical repo for the next release.
26. On next release the main branch of the new repo is updated from the stabilization, merge and is tagged for the release.

[vincent6767]

@tonybalandiuk Few questions

1. Do we need to involve TSC to decide on these criteria?
2. Once it's decided, where should we document this criterion? In the SIG release repo?

[tonybalandiuk]

@vincent6767

1. TSC should be made aware. I'll post in Discord.
2. Our Major release process mentions the included repos. We should include it there.

[tonybalandiuk]

@byrcolin is there an easy way to see "The list of all Canonical repos"?

[byrcolin]

The list of canonical repos is the core engine repo (o3de/o3de) and any repo mentioned in the core engine .automatedtesting.json REPO section as these are part of o3de AR. However, there may be other canonical repos which are not currently a part of AR, like MultiplayerSample, NetSoakTest and AtomSampleViewer.
Should these project be canonical? Not sure.

[tonybalandiuk]

Does anyone disagree with the statement "All canonical repos should be included in the O3DE release process" ?

[byrcolin]

Should ALL canonical repos be part of a release including the projects... If they are canonical projects then yes I think so. If they are not canonical and just sample projects then no.
There is no question for me that the core engine repo and extras are canonical and must be part of a release, but we can be more flexible on these projects, at least for now.

[tonybalandiuk]

Decided on 2/28

1. "All canonical repos should be included in the O3DE release". The O3DE packaged installer will not contain all canonical repos, for example it will not contain O3DE extras, which seems ok still.
2. We will open a separate issue to request o3de-extras to be added for the 23.05 release
3. we will re-evaluate after the 23.05 release to confirm that we made the right decision.

[tonybalandiuk]

@tonybalandiuk to document in our release process documentation
@tonybalandiuk to inform SIGs on the O3DE-extras decision when we open the other issue for O3DE extras

[tonybalandiuk]

1. Handed by this issue requesting O3DE extras be added. Add o3de-extras repository to release process and release documentation #142
2. sigs informed at https://discord.com/channels/805939474655346758/821692567355523152/1082846770029142056
   "All SIGs - The next release (23.05.0) will include the O3DE-extras Repo as part of our release stabilization process. It's this repo: https://github.com/o3de/o3de-extras This means that (a) we will create a stabilization branch, and bugs discovered could potentially hold up the release (b) this repo will have a release tag applied when the release goes out. Making everyone aware so that you can plan appropriately for any testing and bug fixing."