Skip to content

nyuvis/rule_empirical_study

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rule Empirical Study

This repo contains the study materials we used for the empirical study of rule understanding.

  • 01_rule_visualization contains the code of generating rule visualizations
  • 02_study_analysis contains the analysis code of study results.

Rule Generation & Visualization

For the rule generation, we apply the algorithm proposed by Wang et al.[1].

We use the home equity line of credit (HELOC) dataset [2] provided by FICO for our training stages (tutorial, concept verification, task introduction, task verification).

We generate rules based on PIMA Indian Diabetes dataset[3] for our actual test. To avoid the influence of prior knowledge in the task performance, we tell the participants we are using a fictitious data set. We change the features names into mineral names as shown in the table below:

Feature nams in diabetes data Feature names in test
Pregnancies Iron
Glucose Magnesium
BloodPressure Sodium
SkinThickness Zinc
Insulin Potassium
BMI Vitamin A
DiabetesPedigreeFunction Calcium
Age Copper
Target names in diabetes data Target names in test
non-diabetic Low Risk
diabetic High Risk

Study Analysis

We follow the steps we stated in the pre-registration form.

Performance Overview (absolute effect size):

image

Reference:

[1] Wang, T., Rudin, C., Doshi-Velez, F., Liu, Y., Klampfl, E. and MacNeille, P., 2017. A bayesian framework for learning rule sets for interpretable classification. The Journal of Machine Learning Research, 18(1), pp.2357-2393.

[2] Explainable Machine Learning Challenge - FICO Community.

[3] Smith, J.W., Everhart, J.E., Dickson, W.C., Knowler, W.C. and Johannes, R.S., 1988, November. Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 261). American Medical Informatics Association.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published