The decision which probability to use as realistic cutoff is far from trivial. It depends to a large degree on the preferences and demands of the user, since any cutoff represents a compromise between true and false positives. To assist the user making this decision we make the dependence between the score and the ratio TP/FP transparent.
Below you can see for both methods a colored ROC plot for overview, a download for detailed consideration of the underlying data and finetuning of the cutoff and finally a comparison of different statistical quality measures. In the colored ROC plots, the color scale represents the choice of cutoff. The plots wer created using the prettyPROC R-package.
