RFC: +roles as verified expertise and contribution areas

[sockdrawermoney]

Problem/Opportunity:

• Sponsors mention an interest in audit contests beyond evm chains, or looking to add frontend to their scope, and even mechanism design and tokenomics review from a security perspective.
• Our experience with Cosmos contests has produced inconsistent participation quality ratios; sponsors have expressed value in submissions but the open nature of contests has meant greenfield contests have higher noise > signal ratio compared to other contests.
• There are Code4rena wardens who have capability and experience in fields beyond solidity.
• The greater attention Code4rena gains, the more the noise ratio has also tended to increase in general contests.
• The more noise, the greater inefficiency of the overall model.
• 'certified' is overly formal, dated, and implies centralization; current implementation is munging a lot of unrelated purposes
• It would be valuable to have the ability to identify and vet talent in a variety of areas for purposes of

Desired outcomes

• Code4rena is able to expand audit contests to other chains and technologies while feeling confident in our ability to bootstrap quality audits of those domains.
• Sponsors feel like the ratio of signal to noise in all contests is good.
• DAO members can see in someone's profile roles what skills and talents they have and can contribute in order to better empower people to team up together.

Proposed solution:

• Rename 'certified' to + (as in +v or +o in irc, to date myself)
• current ‘certified’ role becomes +id (completed kyc, etc)
• current ‘certified+’ role becomes +id, +w-evm, and `+w-review’ (completed kyc, demonstrated warden skill in evm, and approved by C4 team for involvement in review process in evm contests based on past contributions, community citizenship, and communication approach)
• Gradually expand the use of the + model and role as a method for vetting capability and to bootstrap a healthy talent ratio
• • Use a process similar to the judge application working group for reviewing warden capabilities like +w-cairo, +w-cosmos, +w-solana, +w-frontend, +w-tokenomics, etc
• This also applies to other areas outside of auditing: +build-data, +build-js, +build-ux, +illustration, +biz-ops, +biz-bd, +mechanism, etc
• Limit total number of wardens who are not +d on a contest’s relevant tech from competing in some kinds of contests to control signal-to-noise ratio.

Note that I don't think we need all of the examples provided here and we'd need to have a process for how these get added, how talent is vetted for being sufficient to review someone's capability, and how people apply to gain these additional roles. In many cases, self-identification is probably sufficient.

[youngDuckling]

Suggestion of a potential vetting mechanism:

Use finished contests for which results have not yet been published as a test.
Compare actual findings to findings found by applicant and see if applicant passes the established threshold.

[sockdrawermoney]

Oh that's an interesting idea

[Picodes]

To me, rather than limiting the number of wardens who are not +w on a contest’s relevant tech to participate, a good way to preserve signal and limit noise would be to only allow submissions for medium and high findings for newcomers, and grant the +w-x role once a participant has met a given threshold, e.g. 2 more valid medium findings than invalid ones in a single contest.

This way we don't prevent anyone from participating to maximize signaling, we limit the number of gas and low/NC submissions to limit noise and we incite new wardens to send only valid, high quality findings.

[sockdrawermoney]

Ideal in theory but in actual experience having a lot of people new to a tech is burdensome to sponsors and not a great experience to them when a lot of people are DM'ing with questions that indicate that unfamiliarity

[Picodes]

I see ! If DMs are the main source of noise why don't we ask someone from C4, maybe the judge or a trusted certified person to filter questions ?

[sockdrawermoney]

Usually sponsors are the right people to ask, and DM's with wardens are a pretty fundamental part of the process. Adding an intermediary is unworkable and unsustainable, unfortunately.

[CloudEllie]

Noting here that we are going to start rolling out some of these changes, starting with:

• +certified (completed KYC; formerly certified)
• +backstage (approved by C4 team for involvement in review process; formerly certified+)

We will not be attempting to vet wardens' specific skillsets at this time, and are tabling that idea for the time being.