Retrospective of the Security Self-Assessment Subproject Pilot

[PushkarJ]

Description: With kubernetes/sig-security#8 completed it is a good idea to perform a retrospective on what we learnt from the the pilot and how the lessons learnt could improve the overall security assessments process for CNCF TAG Security. This is also one of the pending tasks from #603

Impact: Idea would be to meet and document lessons learnt from participants in the pilot and then update the relevant documentation found here: https://github.com/cncf/tag-security/tree/main/assessments/guide

Scope: Assignee of this issue could do the following:

• Setup a meeting or async communication feedback loop from participants in the pilot across Kubernetes community
• Gather and compile the feedback for following sections: "What worked?", "What didn't work well?", "What could be improved?", "What should we do more?", "What should we do less?"
• Present the results to CNCF TAG Security in one of our meetings
• Evaluate potential changes to be made to self-assessment guidance for CNCF TAG Security
• Make a PR to add those changes
• Address PR feedback and then merge

Optionally, make retrospective as one of the last step post any completion of security self-assessment

[aladewberry]

I volunteer to do this! Sounds like a perfect learning opportunity as I get up to speed. Assign me at your leisure!

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[aladewberry]

In progress! Rounding up folks in Slack to see what we can get done at Kubecon

[aladewberry]

Retro done! https://kubernetes.slack.com/archives/C022K4F2W4W/p1667582978251009

Now to present at a TAG meeting!

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[PushkarJ]

@aladewberry can I help move the retrospective from slack to markdown page in this repo? If you have presented this already in a TAG meeting I may have missed it. So happy to reuse that content as well if needed

[aladewberry]

@PushkarJ Sorry for the slow reply here! Yes, I was able to present the CAPI retro highlights at a TAG security meeting a few months back. When you say "move the retrospective from slack to markdown page in this repo", are you referring to the TAG Security/Assessments repo?

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[PushkarJ]

Moving the retro highlights from slack compiled by @aladewberry as a comment so it is not lost somewhere in slack.

Highlights

• What went well - The overall process is sustainable - we got some actionable results without a ton of effort. Also, the fuzzing report we did was well scoped, and was useful as well.

• What was difficult - While some results were actionable at the code level, the core CAPI team is small, so items like documenting threat models and other larger tasks are difficult to accomplish. Priorities can be unclear because understanding what things are actually exposing users can be hard when you don't have security expertise. Likewise, when no one is specifically asking for the security issues to be addressed, it is hard to make priority calls. Setting up the process for reporting a security issue and also for monitoring that channel is time consuming.

• What can we do differently going forward/wishlist - We need more resources (or to become familiar with existing resources) for initial triage and monitoring of security issues. Then we need people like @PushkarJ who can help a team evaluate their security posture and prioritize things in a way that works from a security perspective and from a feature priority/normal work perspective - aka if there's something on fire we should drop everything, but otherwise how can we weave security improvements into normal ebbs and flows of work. For that matter, it sounds like there's also an opportunity to socialize findings (as is appropriate given sensitivity) to get community input/build consensus on priorities with the findings to help guide the team.