How To Find TPR And TPR From Its Labels And Probablites To Evaluate My Model?

I need help regarding the data frame which I have asked in this question Link. Now I want to evaluate the model to find the total; like finding True Positive Rate and False Negativ

Solution 1:

From your new DataFrame :

>>> import pandas as pd

>>> df
                file    set     label                                        bbx    Atelectasis Cardiomegaly    Consolidation   Edema   Effusion    Emphysema   Fibrosis    Hernia  Infiltration    Mass    Nodule  Pleural_Thickening  Pneumonia   Pneumothorax
0   00000003_000.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False   0.145712    0.028958    0.205006    0.055228    0.115680    0.376638    0.349124    0.357694    0.122496    0.202218    0.075018    0.118994    0.195345    0.215577
1   00000003_001.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False   0.132639    0.046136    0.169713    0.092743    0.285383    0.614464    0.311035    0.344040    0.117032    0.447748    0.152327    0.094364    0.174125    0.316022
2   00000003_002.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False   0.233026    0.042541    0.227911    0.047988    0.116835    0.595102    0.330304    0.367272    0.117985    0.298624    0.109354    0.133473    0.185444    0.379627
3   00000003_003.png    Test    [[[0.0, 0.0, 1024.0, 1024.0], [0.0, 0.0, 1024....   False   0.298693    0.022646    0.237977    0.035348    0.143645    0.487804    0.384509    0.379062    0.083205    0.625744    0.102377    0.207353    0.184517    0.354402
4   00000003_004.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False   0.522152    0.052897    0.237475    0.082139    0.200029    0.473421    0.377468    0.336104    0.106339    0.488078    0.088047    0.146686    0.200919    0.313684

We can use idxmax to get the column name of the max value on each row like so :

>>> df["predicted_class"] = df.drop(['file', 'set', 'label', 'bbx'], axis=1).idxmax(axis=1)
>>> df["predicted_class"].head()
0      Emphysema
1      Emphysema
2      Emphysema
3           Mass
4    Atelectasis
Name: predicted_class, dtype: object

Then we check if this column name appears in the labels column using a lambda to get a boolean True Positive or True Negative value :

>>> df['evaluation'] = df.apply(lambda x: x["predicted_class"] in x["label"], axis=1)
>>> df['evaluation'].head()
0    False
1    False
2    False
3    False
4    False
Name: evaluation, dtype: bool

To finish, we can have the TPR for each class doing :

>>> df.groupby('predicted_class')['evaluation'].mean()
predicted_class
Atelectasis    0.000000
Emphysema      0.285714
Mass           0.000000
Name: evaluation, dtype: float64