Missing imports in Quickstart Tutorial

[eroell]

Describe the issue linked to the documentation

Thanks for all your efforts on this nice package!

When checking out the quickstart tutorial, it appears that in the last section after

Fairlearn also allows us to quickly plot these metrics from the fairlearn.metrics.MetricFrame

some imports here

metrics = {
    "accuracy": accuracy_score,
    "precision": precision_score,
    "false positive rate": false_positive_rate,
    "false negative rate": false_negative_rate,
    "selection rate": selection_rate,
    "count": count,
}

are missing.

Suggest a potential alternative/fix

Including the missing imports that made it working for me would then look like this:

from fairlearn.metrics import false_negative_rate, false_positive_rate, count
from sklearn.metrics import precision_score

metrics = {
    "accuracy": accuracy_score,
    "precision": precision_score,
    "false positive rate": false_positive_rate,
    "false negative rate": false_negative_rate,
    "selection rate": selection_rate,
    "count": count,
}
metric_frame = MetricFrame(
    metrics=metrics, y_true=y_true, y_pred=y_pred, sensitive_features=sex
)
metric_frame.by_group.plot.bar(
    subplots=True,
    layout=[3, 3],
    legend=False,
    figsize=[12, 8],
    title="Show all metrics",
)

Can make the PR fixing this example, but wanted to check here first for someone to agree (that is, not that I just missed something)

Best

[romanlutz]

From taking a Quick Look it seems you're right. What surprises me is that the PR checks didn't catch it.

I don't think it needs to be addressed since #1310 is about the rewrite the entire quickstart page.

[eroell]

Ah yes I see, thanks for the pointer - if I see that correctly, the missing imports persist there in the screenshot of the fist comment, and the unaltered use of plot_quickstart.py in line 192 without the imports having been added I think?

I dare to quickly comment there to prompt a quick check, but maybe @alliesaizan already got it.

[eroell]

What surprises me is that the PR checks didn't catch it.

I'm not heavily routined with all effects and traps of using .. literalinclude::, so my first guess would be that since the imports are made in plot_quickstart.py this passes through the checks...

In that case, only if someone comes along copy-pasting the snippets to try on their own it'd become apparent I guess.

[romanlutz]

I believe the reason for using literalinclude is the plots. If you run normal code blocks and generate plots they won't show. There may be other ways of doing that, of course. Feel free to suggest a fix if you find one.