GitHub Issues Processing Workflow

Intent

In order to keep the Community backlog up-to-date and complete with clear and reproducible tickets, we have added a verification procedure. It should include ways to supply all information required for development to fix the issue. Additionally, all required information, to add the ticket to Magento's internal issue tracker must be present.

Verification Workflow

The verification workflow was designed to keep the procedure as simple as possible, while still maintaining high quality of reports.

Glossary

The following terms help define the process:

• Reporter – User who filed initial issue report.
• Contributor – User who is working on the issue report to update and confirm report in accordance with all requirements.
• Automated Contributor Assistant – Non-human user that performs automatic checks and provides assistance to human users.
• Label – GitHub label applied to the ticket.

Label descriptions:

• Issue: Format is valid – label assigned when the issue report content has all required fields. This label is controlled by Automated Contributor Assistant.
• Issue: Format is not valid – Label assigned when the issue report content does not match the format or content required for issues. This label is controlled by Automated Contributor Assistant.
• Issue: Clear Description – Label assigned when human proof-reads the issue report and confirms that it is meaningful and may be sufficient to reproduce the issue.
• Issue: Confirmed – Label assigned when the Contributor reproduced the issue with at least one of active Magento release lines.
• Issue: Ready for Work – Label assigned by the Automated Contributor Assistant, when ticket was added to Magento's internal issue tracker.
• Issue: Cannot reproduce – Label assigned when the described issue is not reproducible by the described scenario or reproduction steps.
• non-issue – Label assigned when the described behavior in the report is correct, expected behavior and not an issue.
• Component: xxx – Multitude of labels indicating Magento components that may be the origin of the issue. There is a specific label for each major component. For edge cases, Component: Other and Component: Multiple may be used. See Magento Components Assignment for a list.
• Reproduced on 2.x - Label indicating the release line where the contributor can reproduce the issue.
• Progress: needs update – Label indicating that the issue is pending a response from the reporter.

For additional labels, see the Contributor's Guide.

Ticket Verification Workflow

This workflow verifies a submitted issue is ready for development. By the end of the process, it has been vetted for development including all labels for components, affected versions, and so on, reproduction steps are correct, and all the format passes automated review.

1. Preparation - Steps for reviewing the issue, verifying reproduction steps, and assigning a contributor to work it.
2. Validation - Steps for validating the issue format and all information provided checks out: described steps to reproduce are valid, expected behavior is valid, configuration described in preconditions is valid.
3. Verification - Steps for verifying if the issue exists in all supported development branches.
4. Finalization - Steps for finalizing all content for the issue for developers to begin work.

Supporting the Process: Working on an issue report as a reporter, contributor, or developer is always a commitment. It is beneficial for every participating party to monitor activity and comments on the ticket during it's lifetime, and provide necessary information or insights.

Preparation

Steps for reviewing the issue, verifying reproduction steps, and assigning a contributor to work it.

1. Contributor picks a ticket from the GitHub tracker which is not yet processed. The recomended tool is Community Backlog Dashboard.
2. A contributor reviews the list from "Ready for QA" column and selects an issue to begin processing.
3. After selecting the ticket, the contributor checks the description and reproduction steps.
4. When the contributor is ready to start processing the issue, the contributor should assign the ticket to himself. This indicates someone is actively working on the issue.

Validation

Steps for validating the issue format and all information provided checks out: described steps to reproduce are valid, expected behavior is valid, configuration described in preconditions is valid.

1. When the issue is entered, the Automated Contributor Assistant automatically checks the format and assigns one of the following labels after review: Issue: Format is valid or Issue: Format is not valid.

2. If the format is not valid (receives Issue: Format is not valid), the contributor should read the report carefully and edit the issue to better match one of the required templates. The contributor edits the report content to comply with requirements.

3. After saving any changes, the Automated Contributor Assistant runs again and updates the format label (usually less than one minute).

4. A contributor can select the issue and review all information, reproduction steps, etc. If the information is incomplete, the contributor requests more information from the reporter and applies the label Progress: needs update. All work pauses on this ticket until the reporter provides more information.

5. If the ticket has enough information, the contributor analyzes the problem described in the ticket: described steps to reproduce are valid, expected behavior is valid, configuration described in preconditions is valid.

6. Is it validated?

   • If all provided information is clear and sufficient, it is validated. The contributor applies the Issue: Clear description label to indicate that ticket is ready for manual testing.
   • If it is not validated, the contributor adds the label Progress: needs update and requests more information from the reporter.

Verification

Steps for verifying if the issue exists in all supported development branches: 2.3-develop and 2.2-develop and apply all correct labels. Be advised, we only accept pull requests for 2.3-develop and 2.2-develop (if the fix is applied to 2.3-develop first).

1. Contributor checks if the issue exists on the 2.3-develop branch with a clean installation, configured by described preconditions, following the exact reproduction steps.

   • If the described behavior was reproduced, the contributor should apply the Reproduced on 2.3 label to the ticket, indicating that issue was reproduced and specific version.
   • If the issue was not reproducible with 2.3-develop, the contributor should test on the 2.2-develop branch on a clean installation.
   • If the issue was not reproduced with both versions, add the label Issue: Cannot reproduce to the ticket and add a comment describing the verification process should be added. Ticket may then be closed or assigned as Progress: needs update depending on the expected result.

2. If steps required to reproduce were different from the initially described reproduction steps, update the ticket description with the actual information.

3. Based on the verification flow, add at least one or more Component: xxx labels to the issue. See Magento Components Assignment for a list. Use your best judgment to determine the components affected.

Finalization

Steps for final review of issue for contributors/developers to work the issue.

1. Please make sure that all required conditions are met:
   • Issue format is considered valid by automatic system.
   • Issue is reproducible at least with one of the supported versions and labeled appropriately.
   • At least one Component label(s) applied to the ticket.
2. Add the label Issue: Confirmed to the ticket.
3. Wait for the response from the Automated Contributor Assistant, which normally takes 30-60 seconds.
4. If all required information was provided Automated Contributor Assistant will apply label Issue: Ready for work and comment with reference ticket numbers. Otherwise, label Issue: Confirmed will be removed and information on what's missing in the report will be provided to the contributor.
5. Unassign the ticket from yourself so that developers can claim the issue and start development.

Tips and tricks

• If you need a proper testing environment, a verification instance with limited capabilities may be requested from the Automated Contributor Assistant.
• In most of the cases, we don't recommend verifying issues on older patch versions, even if it was supplied in the preconditions. Generally, fixes can only be provided with the next patch version, and not with any of the older patches.
• Always follow the Code of Conduct in issue comments and discussions.

Questions and Discussion

If you have any questions, feedback, or proposals for this workflow, please join the Community Engineering Slack Workspace. We have a #backlog-maintainers channel specifically for these discussions.