At the time of this experiment, Udacity courses currently have two options on the course overview page: "start free trial", and "access course materials".
If the student clicks "start free trial", they will be asked to enter their credit card information, and then they will be enrolled in a free trial for the paid version of the course. After 14 days, they will automatically be charged unless they cancel first.
If the student clicks "access course materials", they will be able to view the videos and take the quizzes for free, but they will not receive coaching support or a verified certificate, and they will not submit their final project for feedback.
In the experiment, Udacity tested a change where if the student clicked "start free trial", they were asked how much time they had available to devote to the course. If the student indicated 5 or more hours per week, they would be taken through the checkout process as usual. If they indicated fewer than 5 hours per week, a message would appear indicating that Udacity courses usually require a greater time commitment for successful completion, and suggesting that the student might like to access the course materials for free. At this point, the student would have the option to continue enrolling in the free trial, or access the course materials for free instead.
Hypothesis is free trial screener sets clearer expectations for students upfront and thus reduce the number of frustrated students who left the free trial because they didn't have enough time— without significantly reducing the number of students to continue pass the free trial and eventually complete the course.
Test results show with the introduction of free trial screener button there is negligible fall in net conversion rate which is not statistically significant & also practially significant (prac significance assumed as minimum 1% decrease in conversion rate).
I expect filtering students by setting minimum time expectations will not impact net conversions thus will not impact company's revenue and at the same time will reduce costs due to less tutor hours on non paying students & also lead to increased focus on serious students.