Revitalize OQS Demos Project

[ajbozarth]

OQS Demos as a project has gotten out of date due to lack of contribution. In order to bring the demos project back up to date and reinvigorate it we had a planning meeting on June 27th 2024. The minutes for that meeting will be posted below and a more in depth planning proposal will be shared and discussed here in the coming weeks.

[ajbozarth]

June 27th Meeting Notes

Attendees (based on the meeting transcript and my memory)

• @ajbozarth
• @baentsch
• @dstebila
• @maximilien
• @planetf1
• @Martyrshot
• @ydoroz
• @praveksharma
• @johnma14

History (provided by @baentsch )

• Started out as a collection of READMEs and blog posts provided by various authors
• Demos became stale over time and thus were updated to Dockerfiles, many of the existing demos Dockerfiles were created by @baentsch

Purpose of the demos

1. Teach users how to use OQS
2. Aid the OQS project, such as use in CI
3. Prover users a stable starting point for using OQS

Current State of the project (provided by @baentsch )

• All demos have a Dockerfile except the chromium demo since it's build can take up to 15 hours and would be unusable in CI
• curl and nginx demos are the most important and maintained, and are used by the test server
• httpd is supported because it is simple

Future

• @baentsch : get all the current demos running and up to date for use in CI

• @ajbozarth : can we update the Dockerfiles to leverage a base image with liboqs/provider?

  • @baentsch : We intentionally don't use a base image because most demos use different configurations for liboqs and the provider. To some degree this is "intentionally diverse" for CI

• @ajbozarth : should we keep the current demos in separate directories in one repo format?

  • separate directories allow easy removal of deprecated demos (will remain in git history)
  • separate repos is overkill for most demos, but could be useful if a specific demo becomes "large" enough

• Core purpose: CI and user tutorials

• Project Goal: Move the demo contents to their upstream projects. This would mean meeting with those project teams to coordinate once demos are back up to date

• Long-term Goal (ie 2-5 years): Complete deprecation of the demos repo as all PQ support exists out of the box in the upstream projects

Next Steps

@ajbozarth will draft a proposal based on the above discussion and present it at a future OQS Status meeting in addition to posting it here for discussion. Aiming for July 9th meeting, but may take longer due to holidays.

Once the proposal is generally accepted, @ajbozarth and any other volunteers will start executing on it.

[ajbozarth]

Now that meeting minutes are posted feel free to add any clarifications or corrections.

Also I have no permissions on this repo and can't update the labels or assignee on this issue.

[baentsch]

Now that meeting minutes are posted feel free to add any clarifications or corrections.

Thanks for the summary @ajbozarth !

Just adding/reiterating my proposal from the meeting: Contributions to #182 are the most urgently needed and "lowest hanging fruit". But of course any contributions are welcome.

I will (continue to) intentionally refrain from contributing too much to gauge whether there's true "community interest" in this or whether this sub project should be left to wither away. For the avoidance of doubt, I'd find the latter wrong as it will weaken OQS -- but I simply don't have the power any more to keep maintaining this single-handedly. Also my motivation is not increased witnessing things like the below:

Adding/documenting in writing here your verbal statement from the meeting @ajbozarth that IBM has "many OQS-based demos" in-house that it may consider contributing -- and I have a hunch that other companies have/do the same.

I heard this comment with great sorrow: Isn't FOSS most powerful when people (and companies) work together? The community could grow and learn; OQS in this case could benefit; lots of duplicate effort could be avoided.

This comment goes to all corporate members of PQCA (@dstebila @thb-sb please carry this plea to the TAC and GB): Please seriously consider contributing integrations of an OSS library (OQS) into other OSS libraries to the OSS community earlier than later. Proprietary code integrations may of course stay proprietary to afford "competitive advantage". I'd still personally consider the latter wrong as this wastes resources (many companies paying many people to do the same thing), introduces risks (proprietary integrations not gaining enough scrutiny to assure proper security standing) and doesn't strengthen a commonly used OSS component (OQS in this case not getting feedback from secret integrations). Allow me to tag @bencemali and his team and company as a great (counter)example: They seem to use OQS, contribute their findings and motivate changes to OQS based on their use -- all the while pursuing their own product(s). Big Thanks! This is how I understand FOSS to work for everyone and which motivates me to keep contributing.

Also I have no permissions on this repo and can't update the labels or assignee on this issue.

OQS originally had the common FOSS "meritocracy" principle: People that contribute & maintain got GH management rights. The LF take-over did away with this, though: Even I as "maintainer" (call me MINO :-) can't give you those permissions. Hence tagging @ryjones to give @ajbozarth or anyone else any GH permissions they want on this project: IMO this sub project will never be considered sensitive for productive use so I see no risk in this.

[ajbozarth]

Plan to revitalize OQS Demos

This is a proposed road map to revitalize the OQS demos project, the phases do not need to be executed sequentially and can handled in parallel for any given demo. Once I have received general approval for the road map I will begin work on Phase 1. If Phase 1 begins to take too long, we will start Phase 2 early and work on them in parallel. We can define "too long" as a community prior to starting Phase 1 (examples: 1 month, 3 months, 6 months)

Phase 1: Creating the project board

1. Create a GitHub project board for tracking the revitalization issues (both existing and yet to be opened).

2. Go through each existing demo and create a tracking issue. In each tracking issue answer the following questions. There are some existing issues that may be repurposed and added to the project board to handle these items (eg Update test server build script: liboqs-0.10.1 & oqs-provider-0.6.1-rc1 #272 for tracking HAPRoxy)

   1. What is the current state of the demo? Is it working, out of date, or broken?
   2. What version of code is the demo running on? (eg OpenSSL 1.1.1 vs 3)
   3. What changes does the demo make to the various projects it installs? What configurations are made?
   4. What elements of the demo distinguish it from other demos or the default installation? (eg non-default liboqs configs)

3. Open actionable sub-issues for updating demos based on the information gathered above.

4. Prioritize the project board with an aim to focus on higher priority demos first, even starting phase 2 on higher priority demos before finishing phase 1 if necessary.

Phase 2: Executing the project board

1. Start work on the actionable issues created in Phase 1. This step in Phase 2 can start separately for each demo as its action items are ready, allowing for Phase II to run in parallel with parts of Phase I if there is interest in the project.

2. Update and create documentation. In addition to documentation for each demo we will want to create project documentation.

   1. Online demo lifecycle and contribution guidance
   2. Project purpose and goals

Phase 3: The future

1. Open issues and PRs on upstream projects based on the changes demos make to enable PQ. Our goal for any given demo is for its content to eventually exist natively in the various upstream projects it uses.

2. Reach out to the PQCA member companies to donate internal demos they have created

[ajbozarth]

Sorry for the delay, I've shared my proposed plan above and expect further discussion here and in this week's OQS call. Depending on feedback I expect well will leave it open to discussion for at least a week (if not longer)

[baentsch]

Thanks for this proposal, there's quite a few good suggestions in here and some warranting further conversation. And you state correctly that this issue shall be left for further discussion, also as as there probably will hardly be a single PR closing this. Thus, converting it into a full-fledged discussion, out of which actionable issues can be created.

[baentsch]

I very much like the idea of creating separate issues for each integration (where not already available) and a project board to track this: This would nicely complement the current approach of tracking more general issues across projects, such as #182 . Cross-referencing then would allow closing out general issues if all single integrations (that are still deemed relevant) have them covered. Also, new integrations can cross-reference "general" issues/guidance such as the OpenSSL111->3 switchover to heed any lessons documented there.

One change of order proposal: 3.2 seems to be inefficient: If that were done not "in the future" but immediately, it may help close out issues right away: For example, members of your own organization (that I know is large and you may not know each other) publicly state to basically have #273 done; if it were contributed, statements of intent like this or this could be redirected. But we could also keep 3.2 rephrased to include the word "new" or "further":

Reach out to the PQCA member companies to donate further internal demos they have created

Another change of order suggestion: Item 2.2.ii states "Project purpose and goals". Wouldn't it be better to make this step 0? If we'd first agree on project purpose and goals, the questions (currently preceding this) could be guided by this, no? For example, should single integrations be used in experimentation or actual deployments and by whom? In each case at least one entity could step forward and "champion" the respective integration. My suggestion would be even to create an issue as per your Phase 1, step 2 only if there's someone championing it.

This would also address the more basic question that I always grappled with and that I'm not sure the proposal above addresses as-is: What integrations are important/worth while working on to boot? So far we accepted any integration as the premise was that they help at least "research experimentation". But we never followed up as to whether they were actually used. Creating issues for each for no good reason beyond "they're there" seems a waste of time and energy, no?

[ajbozarth]

As discussed in todays meeting, if you feel a current demo is worth putting effort into updating/fixing/maintaining please upvote the comment with that demo name below (this will aid in prioritization)

each comment includes a link to that demo's directory in the repo, to see the list of current states by @baentsch check https://github.com/open-quantum-safe/oqs-demos/blob/main/README.md

Voting will be tallied at the start of the OQS call on Tuesday July 23rd

[ajbozarth]

chromium

[ajbozarth]

curl

[baentsch]

Rationale for support: client-side openssl (integration) testbed for oqsprovider

[ajbozarth]

envoy

[ajbozarth]

epiphany

[ajbozarth]

h2load

[ajbozarth]

haproxy

[ajbozarth]

httpd

[ajbozarth]

mosquitto

[ajbozarth]

nginx

[baentsch]

Rationale for support: server-side openssl (integration) testbed for oqsprovider

[ajbozarth]

ngtcp2

[ajbozarth]

openlitespeed

[ajbozarth]

openssh

[ajbozarth]

openssl3

[ajbozarth]

openvpn

[ajbozarth]

quic

[ajbozarth]

unbound

[ajbozarth]

wireshark

[ajbozarth]

Final vote tally:

8 - curl
7 - nginx
6 - openssl3

3 - http
3 - openssh
2 - chromium
2 - epiphany
2 - haproxy
2 - wireshark
1 - envoy

0:
h2load
mosquito
ngtcp2
openlitespeed
openvpn
unbound
quic

[ajbozarth]

Based on today's call I will start work by going through the phases of my plan on the high priority demos, which should not need as much, if any, work. This will give me experience going through the process and seeing what working demos look like. I will then give a walkthrough of the process at a future call once I've completed it a couple times to gather feedback on my process.

This discussion will remain open through the entire process and I will share overall project updates here

[baentsch]

@ajbozarth Any updates to share on the above? Do you understand how to manually run the "high prio demos" now? If so, there's currently a good opportunity to contribute something to get your feet wet with a concrete contribution if you like: #291 upgrades the QUIC demos to nginx and curl but it's still missing a CI build-and-test run along the lines referenced here. If you'd want to contribute that, I'd suggest to sync with @pi-314159 on that PR.