-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Changed the way reasons are being returned from decide APIs #279
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good!
It'll be great if we add a few more tests validating reasons collecting works ok for several hierarchical paths. No need for validating all messages.
|
||
whitelisted_variation_id = project_config.get_variation_id_from_key(experiment_key, whitelisted_variation_key) | ||
|
||
unless whitelisted_variation_id | ||
message = "User '#{user_id}' is whitelisted into variation '#{whitelisted_variation_key}', which is not in the datafile." | ||
@logger.log(Logger::INFO, message) | ||
decide_reasons&.push(message) | ||
return nil | ||
return nil, message |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this ok to return a string instead of a string array?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, if you notice, i am using splat operator *
when pushing into the array. it flattens the messages if its an array or uses it as is if its a single message.
lib/optimizely/decision_service.rb
Outdated
decide_reasons&.push(message) | ||
return saved_variation_id | ||
decide_reasons.push(message) | ||
return saved_variation_id, decide_reasons | ||
end | ||
end | ||
|
||
# Check audience conditions | ||
unless Audience.user_meets_audience_conditions?(project_config, experiment, attributes, @logger) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We miss reasons from "user_meets_audience_conditions". Can you change it to merge the reasons?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a lot of works for validating collections of reasons! They look good!
Suggesting 2-3 types of messages removed from reasons. See my comments.
lib/optimizely/audience.rb
Outdated
) | ||
message = format(logs_hash['EVALUATING_AUDIENCES_COMBINED'], logging_key, audience_conditions) | ||
logger.log(Logger::DEBUG, message) | ||
decide_reasons.push(message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This message should not be in reasons - no helpful info.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
) | ||
message = format(logs_hash['EVALUATING_AUDIENCE'], audience_id, audience_conditions) | ||
logger.log(Logger::DEBUG, message) | ||
decide_reasons.push(message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good! I think this message should BE in reasons since it shows "audience_conditions".
spec/audience_spec.rb
Outdated
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger) | ||
expect(user_meets_audience_conditions).to be false | ||
expect(reasons).to eq([ | ||
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\"].", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This debugging message should be removed from reasons.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
spec/decision_service_spec.rb
Outdated
"User 'test_user' is not in the forced variation map.", | ||
"Evaluating audiences for experiment 'test_experiment': [].", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we remove these two messages from reasons?
The first one (forced variation) should be added into reasons only when "is" in the forced variation map.
The 2nd one is not helpful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
spec/decision_service_spec.rb
Outdated
"User 'test_user' is not in the forced variation map.", | ||
"Evaluating audiences for experiment 'test_experiment': [].", | ||
"Audiences for experiment 'test_experiment' collectively evaluated to TRUE.", | ||
"Assigned bucket 4577 to user 'test_user' with bucketing ID: 'test_user'.", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This message is also not needed for reasons - bucket id assignment info is not helpful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
spec/decision_service_spec.rb
Outdated
expect(variation_received).to eq('111128') | ||
expect(reasons).to eq([ | ||
"User 'test_user' is not in the forced variation map.", | ||
"User '' was previously bucketed into variation ID '111111' for experiment '111127', but no matching variation was found. Re-bucketing user.", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
UserID is missing in the message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looked like an old bug. Fixed the log
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Summary
Reasons were previously being collected by sending a reasons object down the stack to all the related functions which would mutate it by pushing their own reasons. This is now changed. Every related function now returns the reasons array along with the actual value.
Test plan