Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate OnCall schedule quality #118

Closed
Matvey-Kuk opened this issue Jun 21, 2022 · 7 comments
Closed

Calculate OnCall schedule quality #118

Matvey-Kuk opened this issue Jun 21, 2022 · 7 comments
Assignees
Labels

Comments

@Matvey-Kuk
Copy link
Contributor

Matvey-Kuk commented Jun 21, 2022

Alice, Bob, Sam are on call. I think it would be nice to show some "schedule quality" to help users keep schedules in a good shape.

100% = ((50% * balance) + (50% * working_hours_balance)) * ( 1 / gaps_rate)

We're trying to balance on call hours between engineers on call. Balance = 1 is good. Balance = 0 is bad.

balance = abs (min(Alice, Bob) hours / max(Alice, Bob) hours) * .... all pairs

We're trying to avoid people being on call outside of working hours. At least their on call time outside of working hours should be equal.

working_hours_balance = abs (min(Alice, Bob) hours outside of working hours / max (Alice, Bob) hours outside of working hours) * .... all pairs

We want to avoid gaps in on call schedule. Shift with no users is equal to gap:

gaps_rate = gaps hours / total observed period hours 
@Matvey-Kuk Matvey-Kuk added feature request New feature or request RFC labels Jun 21, 2022
@Konstantinov-Innokentii
Copy link
Member

@Matvey-Kuk you want to calculate schedule's score or per-user score?

@Matvey-Kuk
Copy link
Contributor Author

@Konstantinov-Innokentii schedule's score

@raphael-batte
Copy link

@Konstantinov-Innokentii We want to provide overall schedule quality.
But if we have overloaded users in the schedule, we also focus attention on each of them.

@vadimkerr
Copy link
Member

@Matvey-Kuk I propose to use a slightly different formula for balance score calculation:

balance = (min(Alice, Bob) hours / max(Alice, Bob) hours + .... all pairs) / number of pairs

It should eliminate the issue with hours distributions like [7, 8, 8, 8, 8, 8], where the formula you provided returns around 50% score, but [7, 8, 8, 8, 8, 8] distribution is pretty balanced and should score more than 50% (returns around 95% with the formula I proposed).

@maskin25
Copy link
Contributor

@teodosii design here

@maskin25 maskin25 assigned teodosii and unassigned maskin25 Nov 10, 2022
vadimkerr added a commit that referenced this issue Jan 4, 2023
This PR adds an endpoint returning a schedule quality score, overloaded
users and comments on the existing issues (e.g. balance issues or gaps).

## Limitations
- Since working hours editor is not implemented yet, there are only two
scores taken into account: balance score and a score representing the
ratio of time when someone is on-call to the whole time period.
- Time period is now set to be constant (90 days from today), so **in
some cases the results will be inaccurate** (when rotations don't align
with the time period)
- It only takes primary rotations into account (overrides are ignored)

## Usage
`GET /api/internal/v1/schedules/<pk>/quality?date=<TOMORROW_DATE>`

Note that `date` should be tomorrow date, because we can only be sure
about changing tomorrow's shifts (some of the shifts for current day
could be "deleted" but still show up in the UI).

## Example response
```json
{
  "total_score": 90,
  "comments": ["Schedule has no gaps", "Schedule is well-balanced, but still can be improved"],
  "overloaded_users": ["USSZ5WRH2CUA9", "U74XJZSSQGBIH"]
}
```

Issue: #118
@vadimkerr
Copy link
Member

The backend part is now merged in #338, @teodosii will be working on the frontend part in a separate PR.

@teodosii teodosii mentioned this issue Feb 15, 2023
3 tasks
@teodosii
Copy link
Member

teodosii commented Feb 20, 2023

Frontend is implemented but waiting for a minor backend change to differentiate good remarks from warnings within the api response. Once that is done we can unblock the frontend, because right now on frontend we cannot differentiate info messages (like 'Good schedule') from warnings ('Gaps') as described in the mockups.

PR for it #1324

vadimkerr added a commit that referenced this issue Mar 6, 2023
# What this PR does

Changes the schedule quality API so it also returns types of comments
(this is needed to address
#118 (comment)).

## Which issue(s) this PR fixes
Related to #118

## Checklist

- [x] Tests updated
teodosii added a commit that referenced this issue Mar 13, 2023
# What this PR does

#118 

## Checklist

- [x] Tests updated
- [ ] Documentation added
- [x] `CHANGELOG.md` updated
brojd pushed a commit that referenced this issue Sep 18, 2024
# What this PR does

#118 

## Checklist

- [x] Tests updated
- [ ] Documentation added
- [x] `CHANGELOG.md` updated
brojd pushed a commit that referenced this issue Sep 18, 2024
# What this PR does

Related to grafana/irm#99

## Which issue(s) this PR closes

Closes #4991
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants