Skip to content
This repository has been archived by the owner on Oct 31, 2023. It is now read-only.

About computing FPR at 95% TPR used in calMetric.py #21

Open
W-XuWX opened this issue Jul 23, 2023 · 0 comments
Open

About computing FPR at 95% TPR used in calMetric.py #21

W-XuWX opened this issue Jul 23, 2023 · 0 comments

Comments

@W-XuWX
Copy link

W-XuWX commented Jul 23, 2023

Hi, thank you so much for your work. I would like to understand better on your tpr95 method.

Q1. For obtaining the results of FPR at 95% TPR, I would like to understand why the values of start and end were different when comparing your algorithm vs the baseline method, which uses an end of 0.1 and 0.01 for CIFAR10 and CIFAR100 respectively?

def tpr95(name):
...
    # calculate baseline
    if name == "CIFAR-10": 
	start = 0.1
	end = 1 
    if name == "CIFAR-100": 
	start = 0.01
	end = 1    
...
    # calculate our algorithm
...
    if name == "CIFAR-10": 
	start = 0.1
	end = 0.12 
    if name == "CIFAR-100": 
	start = 0.01
	end = 0.0104    

Q2. For the same function, how are the values "if tpr <= 0.9505 and tpr >= 0.9495:" chosen for 95% TPR? As in is it based on 4 decimal places or some other approach. I experimented with other datasets and models, and found that there were zerodivision errors errors if the tpr was slightly above 0.9505 (like 0.9545 for example)

Thank you so much!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant