How To Find Tpr And Tpr From Its Labels And Probablites To Evaluate My Model?

I need help regarding the data frame which I have asked in this question Link. Now I want to evaluate the model to find the total; like finding True Positive Rate and False Negativ

Solution 1:

From your new DataFrame :

>>> import pandas as pd

>>> df
                file    set     label                                        bbx    Atelectasis Cardiomegaly    Consolidation   Edema   Effusion    Emphysema   Fibrosis    Hernia  Infiltration    Mass    Nodule  Pleural_Thickening  Pneumonia   Pneumothorax
000000003_000.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False0.1457120.0289580.2050060.0552280.1156800.3766380.3491240.3576940.1224960.2022180.0750180.1189940.1953450.215577100000003_001.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False0.1326390.0461360.1697130.0927430.2853830.6144640.3110350.3440400.1170320.4477480.1523270.0943640.1741250.316022200000003_002.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False0.2330260.0425410.2279110.0479880.1168350.5951020.3303040.3672720.1179850.2986240.1093540.1334730.1854440.379627300000003_003.png    Test    [[[0.0, 0.0, 1024.0, 1024.0], [0.0, 0.0, 1024....   False0.2986930.0226460.2379770.0353480.1436450.4878040.3845090.3790620.0832050.6257440.1023770.2073530.1845170.354402400000003_004.png    Test    [[[0.0, 0.0, 1024.0, 1024.0]], ['Hernia']]  False0.5221520.0528970.2374750.0821390.2000290.4734210.3774680.3361040.1063390.4880780.0880470.1466860.2009190.313684

We can use idxmax to get the column name of the max value on each row like so :

>>> df["predicted_class"] = df.drop(['file', 'set', 'label', 'bbx'], axis=1).idxmax(axis=1)
>>> df["predicted_class"].head()
0      Emphysema
1      Emphysema
2      Emphysema
3           Mass
4    Atelectasis
Name: predicted_class, dtype: object

Then we check if this column name appears in the labels column using a lambda to get a boolean True Positive or True Negative value :

>>> df['evaluation'] = df.apply(lambda x: x["predicted_class"] in x["label"], axis=1)
>>> df['evaluation'].head()
0False1False2False3False4False
Name: evaluation, dtype: bool

To finish, we can have the TPR for each class doing :

>>> df.groupby('predicted_class')['evaluation'].mean()
predicted_classAtelectasis0.000000
Emphysema      0.285714
Mass           0.000000
Name: evaluation, dtype: float64