@lorentzenchr I'm thinking about taking a stab at this. I have a two questions about d2_log_loss_score.

Q1: I can't seem to locate literature that specifically uses log_loss in computation. Any pointers here for reading on how D2 and log_loss are used together? I am not too familiar with D2 overall so anything is welcome. Already checked out the Hastie & Tibshirani reference in the docstring. Also found this paper by Guisan and Zimmermann that has an explanation closer to the "meta-score" explanation in the RFC on eq 1, pg 166.

Q2, on file structure: Given that Tweedie applies to regression models and d2_tweedie_score goes into sklearn.metrics._regression. In the case of log_loss it is a classification metric (source), does this mean d2_log_loss_score should go under sklearn.metrics._classification? Thanks.

Just realized that it's the same as McFadden's R^2 which was much easier to search than D^2. Thanks for the doc!

e.g.

• https://thestatsgeek.com/2014/02/08/r-squared-in-logistic-regression/
• https://stats.stackexchange.com/a/463029

I think I'll take on d2_log_loss_score. Thanks. Do I need to wait for progress on #21718 ?

@changhsinlee A McFadden R2 version for log loss is not controversial, IMO, so go ahead. A PR is most welcome.

@lorentzenchr I started working on implementing d2_absolute_error_score, I'll add a PR when it's ready for review.

Working on d2_log_loss_score

Hi @MaxwellLZH
I just wanted to confirm that are you still working on d2_log_loss_score?

Hi @MaxwellLZH I just wanted to confirm that are you still working on d2_log_loss_score?

Hi @OmarManzoor, I'm not working on it at the moment, you can go ahead :)

Hi @MaxwellLZH I just wanted to confirm that are you still working on d2_log_loss_score?

Hi @OmarManzoor, I'm not working on it at the moment, you can go ahead :)

Thank you for the clarification.

Hi @lorentzenchr

Is the d2_log_loss_score the D^2 version for the mean_squared_log_error?

@OmarManzoor No, d2_log_loss_score is for log_loss and not yet implemented.

@OmarManzoor No, d2_log_loss_score is for log_loss and not yet implemented.

I see. Could you kindly provide some reference to the specification of log_loss so that I can get some guidance on how to implement it? We have a log_loss defined in classification but this should be a bit different than that?

I would still add a D2 Brier score and then we can finally close here.

I would still add a D2 Brier score and then we can finally close here.

Since the approach for this seems similar, would it be okay if I work on this too?