-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
~4000 0bservations-- tibble giving one row of outputs #14
Comments
Hi @jg3405, in order to help, I’d need to see a reproducible example. Can you try posting one here? |
Hi Brian, Here is an example of a RF model I generated using the ranger package in R. The data used for the model is also included. As you can see, the response variable is factorial (only 0s and 1s). I would like to get shapley values for each observation, but when using the code:
I only get one row of data as output as though I had one observation. Thanks for trying to help. Julia |
Thanks for the example code @jg3405. The issue is that your prediction wrapper is returning probability estimates for both classes, but pfun <- function(object, newdata, type) {
# Return probability estimate for Y = 1 (i.e., the second column)
predict(object, data = newdata, type = "response")$predictions[, 2L]
}
shap_new <- explain(model_prob, X = all_data_no_nan[,-1], nsim = 10, pred_wrapper = pfun, .progress = "text") |
I'll try to add an informative error message for this in the next release! |
Amazing! Thank you for getting back to me so fast! It works like a charm :) |
The same error comes when using Full MWE below: library(fastshap)
library(AmesHousing)
library(glmnet)
data <- AmesHousing::make_ames()
dim(data)
#> [1] 2930 81
y <- data$Sale_Price
X <- as.matrix(data[, c("Lot_Frontage", "Lot_Area", "Year_Sold", "Year_Built")])
glm <- cv.glmnet(X, y)
expl_glm <- fastshap::explain(
glm, X = X,
pred_wrapper = function(mdl, newdata) as.numeric(predict(mdl, newdata, s = glm$lambda.min))
)
sapply(expl_glm, function(x) mean(abs(x)))
#> Lot_Frontage Lot_Area Year_Sold Year_Built
#> 81.60870 31.03433 3.05142 908.30222 |
Question related to the above
|
Hi @trilliumtechnical, it is difficult to provide a good answer without a reproducible example, but in general, the reference class for the Shapley values is dictated by the probability estimates output by the prediction wrapper (as supplied to |
@bgreenwell
Follow up question, is it possible to write a prediction function with fastshap that uses a workflow. My attempt failed, possibly due to the problem with tibbles? #20
|
@trilliumtechnical thanks for including an example. I see two issues with your code:
Adding tibble support isn't out of the question, but I'm not sure how difficult that change would be to implement (fastshapp uses logical subsetting and some other operations that may not be allowed on tibbles). For now, it's easy enough to drop the tibble class before passing it in. Also, XGBoost has "exact" Shapley values built-in (so-called TreeSHAP), which can be obtained even faster by setting |
Hello everyone, I am also experiencing difficulties on how to implement the explain() function to a fitted xgboost model from the mlr package. The error reads: Error in UseMethod("explain") : Is it not possible to apply SHAP values to a fitted xgboost model from the mlr package? For a simplified version of the fitted model, the code looks accordingly: train_df <- as.data.frame(train_df)
pfunWF <-function(object, newdata) { shap <- explain(xgb_model, Any help on how to fix this error would be much appreciated! |
Hi @Thomassimancik, not sure why you'd be getting this error message since |
Hi @bgreenwell, thanks for your swift reply! I am sending a small reproducible example below: `data(mtcars) pfunWF <-function(object, newdata) { I suspect that it has something to do with the definition of traintask object with the makeClassifTask function. |
Two issues in the above code: 1) data(mtcars)
mtcars$vs <- as.factor(mtcars$vs)
mtcars <- as.data.frame(mtcars)
traintask <- makeClassifTask(data = mtcars, target = "vs")
xgb_learner <- makeLearner(
"classif.xgboost",
predict.type = "prob",
par.vals = list(
objective = "binary:logistic",
eval_metric = "error",
nrounds = 50
)
)
model <- mlr::train(xgb_learner, task = traintask)
pfunWF <-function(object, newdata) {
# predict(object, newdata, type = 'prob')$data$prob.1
predict(object, newdata = newdata, type = 'prob')$data$prob.1
}
shap1 <- fastshap::explain(model,
#X = mtcars[, -9],
X = mtcars[, -8],
nsim = 1,
pred_wrapper = pfunWF) |
Hi I'm trying to generate the shapley values for a dataset with 14 features and around 4000 observations (dataframe with 14 columns and 4000 rows). I create a model using ranger, but then when I go to generate my shapley values I only end up receiving a one row output.... I get different values depending on how many observations I include... but still just one row output. Please help.
The text was updated successfully, but these errors were encountered: