Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It's really amazing!! Why are the calculation results of the AUC (recall, precision) function and the average precision score function significantly different? #29051

Closed
kourenke opened this issue May 20, 2024 · 2 comments
Labels
Bug Needs Triage Issue requires triage

Comments

@kourenke
Copy link

Describe the bug

Method 1:
Precision, recall, _=Precision-Recall_curve()
A=auc (recall, precision)
Method 2:
B=average precision score ()
Method 1 and Method 2 both calculate the area under the PR curve, but why are the results significantly different, i.e. a ≠ b

Steps/Code to Reproduce

Method 1:
Precision, recall, _=Precision-Recall_curve()
A=auc (recall, precision)
Method 2:
B=average precision score ()
Method 1 and Method 2 both calculate the area under the PR curve, but why are the results significantly different, i.e. A ≠ B

Expected Results

A = B

Actual Results

A ≠ B

Versions

sklearn: 1.0.2
@kourenke kourenke added Bug Needs Triage Issue requires triage labels May 20, 2024
@MichaelSkowronek
Copy link

AUC uses the trapezoidal rule for interpolation between the points. The average precision score utilizes an interpolation scheme that can be described as a step function. Therefore the difference. For more information, see the user guide: https://scikit-learn.org/stable/modules/model_evaluation.html#precision-recall-and-f-measures .

@glemaitre
Copy link
Member

Indeed this is linked with the following comment:

The average precision (cf. average_precision_score) in scikit-learn is computed without any interpolation. To be consistent with this metric, the precision-recall curve is plotted without any interpolation as well (step-wise style).

You can change this style by passing the keyword argument drawstyle="default". However, the curve will not be strictly consistent with the reported average precision.

The computation of average precision is therefore not trivial :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Triage Issue requires triage
Projects
None yet
Development

No branches or pull requests

3 participants