Skip to content

Commit

Permalink
small improvements to technical solutions
Browse files Browse the repository at this point in the history
  • Loading branch information
sebffischer committed Oct 31, 2023
1 parent a901eba commit 19d6742
Showing 1 changed file with 7 additions and 16 deletions.
23 changes: 7 additions & 16 deletions book/chapters/appendices/solutions.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -857,8 +857,8 @@ bmr$aggregate(msr("classif.bbrier"))[, c(4, 7)]

## Solutions to @sec-technical

1. Consider the following example where you resample a learner (debug learner, sleeps for 3 seconds during train) on 4 workers using the multisession backend:
```{r technical-050, eval = FALSE}
1. Consider the following example where you resample a learner (debug learner, sleeps for 3 seconds during `$train()) on 4 workers using the multisession backend:
```{r technical-050}
task = tsk("penguins")
learner = lrn("classif.debug", sleep_train = function() 3)
resampling = rsmp("cv", folds = 6)
Expand All @@ -871,12 +871,12 @@ i. Assuming that the learner would actually calculate something and not just sle
ii. Prove your point by measuring the elapsed time, e.g., using `r ref("system.time()")`.
iii. What would you change in the setup and why?

Not all CPUs would be utilized in the example.
All 4 of them are occupied for the first 4 iterations of the cross validation.
Not all CPUs would be utilized for the whole duration.
All 4 of them are occupied for the first 4 iterations of the cross-validation.
The 5th iteration, however, only runs in parallel to the 6th fold, leaving 2 cores idle.
This is supported by the elapsed time of roughly 6 seconds for 6 jobs compared to also roughly 6 seconds for 8 jobs:

```{r solutions-022, eval = FALSE}
```{r solutions-022}
task = tsk("penguins")
learner = lrn("classif.debug", sleep_train = function() 3)
Expand Down Expand Up @@ -918,28 +918,19 @@ MeasureCustom = R6::R6Class("MeasureCustom",
inherit = mlr3::MeasureClassif, # classification measure
public = list(
initialize = function() { # initialize class
super$initialize(
super$initialize( # initialize method of parent class
id = "custom", # unique ID
packages = character(), # no dependencies
properties = character(), # no special properties
predict_type = "response", # measures response prediction
range = c(0, Inf), # results in values between (0, 1)
range = c(0, Inf), # results in values between [0, Inf)
minimize = TRUE # smaller values are better
)
}
),
private = list(
.score = function(prediction, ...) { # define score as private method
# define loss
costsens = function(truth, prediction) {
score = numeric(length(truth))
score[truth == "A" & prediction == "B"] = 10
score[truth == "B" & prediction == "A"] = 1
mean(score)
}
# call loss function
costsens(prediction$truth, prediction$response)
}
Expand Down

0 comments on commit 19d6742

Please sign in to comment.