21. Suggestions for CIP review

For reviewers: Subject matter experts of any field are eagerly sought by CIP editors to submit reviews on CIP/CPS pull requests in their fields of expertise.  Spontaneous feedback is welcome but please also consider reviewing according to the stringent criteria below.

For authors: Especially when authoring a CIP for the first time, write as if a reviewer were working with you... expecting answers to the questions below.

Read first for essential CIP elements

CIP editors (rather than community reviewers) will generally scan a CIP / CPS submission as part of a "Triage" process which will check for:

• a sensible PR title, generally matching the title in the header, beginning with CIP-???? | or CPS-???? | before a candidate number is assigned
• a complete & syntactically correct header, with all information required by CIP-0001 or CIP-9999, including:
  • at least one Discussions: link (since it should always include the current PR URL)
• correct top-level (H2 or ##) sections matching the document template files (with HTML comments removed from that template scaffolding)
• a link in the Original Posting (OP) of the pull request pointing to the README.md in the submitter's branch, to allow easy reading of the document
• appropriate choice of open-source License (CC for mainly text, and Apache if it contains code sections)

Third party reviewers might also spot CIP/CPS format, grammar, spelling, and syntax mistakes and are also welcome to post these as GitHub corrections rather than waiting for authors or editors.

Here also can be applied some common-sense criteria, such as not making the Abstract too long (or including detail more properly contained in the latter sections).

Identify & tag the relevant people

Once a submission is free from glaring problems and confirmed readable, editors and those familiar with CIP subject matter should begin tagging authorities in that field: after first checking if they haven't already been tagged.

Often this will happen naturally in a review conversation… though in the original posting and/or early postings, authors & reviewers are also welcome to "cc" others by GitHub ID if they have done work in that field or have the potential to submit a qualified review themselves.

CIP editors will provide technical reviews whenever possible, but with the current number of active editors (3 at the time of this writing) we will generally not be able to cover enough topics in enough detail: so a fan-out of links to interested parties, stakeholders, and working groups should begin early in the review process.  Afterward, CIP editors will monitor all these posted comments & conversations to ensure progress is being made and/or try to recruit additional review help.

Detailed review of document

Feel free to use the checklists in the following sections when getting ready to post your CIP or CPS draft or reviewing someone else's work, even if only in an attempt to confirm its readiness for progressing through the CIP process:

CIP Sections

Motivation

• Does it provide the context of why a CIP is needed?
• Does it identify who/what will be affected if the CIP is not implemented?
• If it changes an established design, does it identify specific problems with that design?
• Does it identify use cases for the new method(s)?

Specification

• Does it cover the new method(s) in sufficient, unambiguous technical detail?
• Does the CIP stand alone (in addition to any resources externally linked, such as libraries or external repositories) to explain how to implement the new design?
• (if not a separate Versioning section) Does it cover how CIP versioning will be handled?
• If it characterises on-chain data, does it include a CDDL schema?

Rationale

• Does it identify previous and contemporary approaches to solving the same identified problem (or prove that there are no approaches yet)?
• Does it explain why the proposed method was chosen over other possible alternatives?
• Does it link to any such alternatives and/or quote enough detail from them for readers to understand the differences & validate the chosen method?
• Does it document any community consensus over these choices (or state/prove that the community has had not such opportunity)?
• Does it address issues of backward compatibility?
• If claiming to solve any CPS, does it address that CPS's Use cases, Goals, or Open Questions?

Path to Active

• Does it include the two vital sections Acceptance Criteria and Implementation Plan?
• Does the Acceptance Criteria read like a list of items to confirm that a CIP is of Active status (i.e. confirmed, standardised implementation)?
• Does the Implementation Plan indicate a list of steps taken (or to be taken) to confirm that a CIP is valid by those who will be implementing it?
• Are lists of to-do items (generally not done when a CIP is formulated, to be done before a CIP is declared Active) formatted as markdown Task lists (checkboxes / tickboxes)?

CPS Sections

Problem

• Does it explain why the CPS was written?
• If extended from a CIP, does it refer to that CIP explicitly & the background discussion from which the Problem Statement evolved?

Use cases

• Does it indicate to what designs potential solutions for this problem would be applied?
• Does it illustrate any current alternative methods and explain why they are unsuitable according to the problem definition?

Goals

• What criteria would any proposed solutions (CIPs) for this problem have for success?
• (if any) What projects have attempted to implement these goals, and what progress (if any) have they made?
• Does it rank these goals by relative importance (most significant goals first)?
• What metrics (if any) might be used to indicate whether the goals above are successful?

Open Questions

• Is the list of proposed questions useful to continue identifying possible solutions to the problem?
• Does the list of questions reflect knowledge & experience of possible vulnerabilities or design flaws?