Kubeflow User Survey - KubeCon Paris

[StefanoFioravanzo]

This is Phase 1 of our extended user research proposal. The aim is to understand and document what end user’s goals are in using Kubeflow and what they are trying to achieve. A focus will be on recruiting folks for interviews.

NOTE: The survey does not ask questions regarding specific components use, or deficiencies found in the platform. This is intentional. In the following months the plan is to publish component-specific surveys design with WG leads. For more info see the proposed roadmap in the doc linked here.

Timeline: Publish this survey in concomitance with KubeCon Paris
Running time: 4 weeks

Survey (draft): https://docs.google.com/document/d/1H0fehjYwaUHyVomFxve_dCGQj9c5SDnod1cc4WaCIuc/edit?usp=sharing

Roadmap

• Review survey with community. Sign-off by Wednesday, March 13th
• Legal advice from CNCF on data protection policy and survey disclaimer.
• Move survey to Google Form by Friday, March 15th
• Print QR codes for Kubeflow booth and Kubeflow summit
• Send mail to Kubeflow discuss
• Send message to Kubeflow slack and CNCF Slack (kubeflow channel)
• Kubeflow Social media post (dependent on getting access to LinkedIn account)

[StefanoFioravanzo]

@jbottum @andreyvelich @johnugeorge @james-jwu @terrytangyuan @akgraner @rimolive Please review.

[rimolive]

@StefanoFioravanzo Maybe you should check the items we already finished. I believe only the last 3 items are missing?

[StefanoFioravanzo]

Today we closed the survey. We got a total of 17 responses, 11 of which agreed to be contacted for an interview. Given that the focus of this survey was to gather contacts for interviews, I declare this a success! We have more volunteers than we can manage.

Couple of notes:

1. Last survey had a lot more respondents (more than 100). This is because it was run for 3 whole months and it was pushed quite a lot by the community. Given that we advertised our survey mainly at KubeCon, it's not surprising that we got just a few responses.
2. The feedback of this survey may be a little bit biased towards the European user base, since most people who responded are from Europe (again, because we advertised it at KubeCon Paris)

This is what's going to happen next. I will

• Share a summary of the questionnaire below shortly. I'll strip all PII and provide a summary of the open eneded questions.
• Share the raw data, stripped of all the PIIs.
• Reach out to via email to all the respondent who have provided their email. There will need to be some selection process to pick 4-5 people to interview. That's TBD.
• Start working on a interview template for the upcoming interview - will follow up with this in a separate issue. (EDIT: Kubeflow User Research Interview Template #719)

[StefanoFioravanzo]

Kubeflow Survey Summary

Appreciation

To the question "On a scale from 0-10, how likely are you to recommend Kubeflow to a friend or colleague?", responded answered with an average of 6.9, with median value of 7. There seem to be a split between people with good appreciation (7-8-9 score) and people with low appreciation (5-6 score).

Paraphrasing the open ended question that asked for clarification, it seems like the major friction points still are: documentation, onboarding, installation, maintainability and stability. Most of the respondent declared themselves as "builders" of an internal platform. Their pains are all related to getting Kubeflow up and running and providing value to their internal users. This theme will be the key focus of the follow up interviews.

The answers to the question "Thinking about your work or projects, could you describe a specific situation or task where you 'used' Kubeflow to achieve your goal? Please explain what you were trying to achieve, why it was important, and how Kubeflow helped in this context." did not provide much insight. This question was places in an effort to uncover some of the "jobs" that Kubeflow solves for people. Many respondents didn't answer and were very brief. The interview will collect much more insightful data.

[StefanoFioravanzo]

Anonymized response data here https://docs.google.com/spreadsheets/d/1-UTNKiLsAmXSkXi-_HwzPlvvU0vkBUuyoQS9n5lZIdU/edit?usp=sharing or here
[Public] Kubeflow User Survey 2024 (Anonymized Responses) - Form Responses 1.csv

[StefanoFioravanzo]

Reached out to all respondent via email (https://docs.google.com/document/d/1CAS7_WxXlqaWaIl4jcL8G0sOKxqDhpM6LyO7Q2s23_k/edit?usp=sharing)

Closing this issue in favour of a new one to track the interview process.