fairlearn.metrics.plot_model_comparison#

fairlearn.metrics.plot_model_comparison(*, y_preds, y_true=None, sensitive_features=None, x_axis_metric=None, y_axis_metric=None, ax=None, axis_labels=True, point_labels=False, point_labels_position=(0, 0), legend=False, show_plot=False, **kwargs)[source]#

Create a scatter plot comparing multiple models along two metrics.

A typical use case is when one of the metrics is a performance metric (e.g., balanced_accuracy) and the other is a fairness metric (e.g., false_negative_rate_difference).

Parameters:
y_predsarray-like, dict of array-like

An array-like containing predictions per model. Hence, predictions of model i should be in y_preds[i].

y_truelist, pandas.Series, numpy.ndarray, pandas.DataFrame

The ground-truth labels (for classification) or target values (for regression).

sensitive_featureslist, pandas.Series, dict of 1d arrays, numpy.ndarray, pandas.DataFrame, None

Sensitive features for the fairness metrics (if a fairness metric is specified for the x-axis or the y-axis).

x_axis_metricCallable | None

The metric function for the x-axis. The metric function must take y_true, y_pred, and optionally sensitive_features as arguments, and return a scalar value.

y_axis_metricCallable | None

The metric function for the y-axis, similar to x_axis_metric. The metric function must take y_true, y_pred, and optionally sensitive_features as arguments, and return a scalar value.

axmatplotlib.axes.Axes, optional

If supplied, the scatter plot is drawn on this Axes object. Else, a new figure with Axes is created.

axis_labelsbool, list

If true, add the names of x and y axis metrics. You can also pass a list of size two (or a two-tuple) of strings to use as axis labels instead.

point_labelsbool, list

If true, annotate text with inferred point labels. These labels are the keys of y_preds if y_preds is a dictionary, else simply the integers 0…number of points - 1. You can specify point_labels as a list of labels as well.

point_labels_positionlist

a list (or a two-tuple) containing precisely two numbers that define the offset of the point labels in the x and y direction respectively. The offset value is in data coordinates, not pixels.

legendbool

If True, add a legend. Legend entries are created by passing the key word argument label to calls to this function. If you want to customize the legend, you should manually call ax.legend (where ax is the Axes object) with your customization params

show_plotbool

If true, call pyplot.plot.

Returns:
axmatplotlib.axes.Axes

The Axes object that was drawn on.

Notes

To offer flexibility in stylistic features besides the aforementioned API options, one has at least three options: 1) supply matplotlib arguments to plot_model_comparison as you normally would to matplotlib.axes.Axes.scatter 2) change the style of the returned Axes 3) supply an Axes with your own style already applied

In case no Axes object is supplied, axis labels are automatically inferred from their class name.