API Docs#

fairlearn.datasets#

This module contains datasets that can be used for benchmarking and education.

datasets.fetch_acs_income

Load the ACS Income dataset (regression).

datasets.fetch_adult

Load the UCI Adult dataset (binary classification).

datasets.fetch_bank_marketing

Load the UCI bank marketing dataset (binary classification).

datasets.fetch_boston

Load the boston housing dataset (regression).

datasets.fetch_credit_card

Load the 'Default of Credit Card clients' dataset (binary classification).

datasets.fetch_diabetes_hospital

Load the preprocessed Diabetes 130-Hospitals dataset (binary classification).

fairlearn.metrics#

Functionality for computing metrics, with a particular focus on disaggregated metrics.

For our purpose, a metric is a function with signature f(y_true, y_pred, ....) where y_true are the set of true values and y_pred are values predicted by a machine learning algorithm. Other arguments may be present (most often sample weights), which will affect how the metric is calculated.

This module provides the concept of a disaggregated metric. This is a metric where in addition to y_true and y_pred values, the user provides information about group membership for each sample. For example, a user could provide a ‘Gender’ column, and the disaggregated metric would contain separate results for the subgroups ‘male’, ‘female’ and ‘nonbinary’ indicated by that column. The underlying metric function is evaluated for each of these three subgroups. This extends to multiple grouping columns, calculating the metric for each combination of subgroups.

metrics.count

Calculate the number of data points in each group when working with MetricFrame.

metrics.demographic_parity_difference

Calculate the demographic parity difference.

metrics.demographic_parity_ratio

Calculate the demographic parity ratio.

metrics.equalized_odds_difference

Calculate the equalized odds difference.

metrics.equalized_odds_ratio

Calculate the equalized odds ratio.

metrics.false_negative_rate

Calculate the false negative rate (also called miss rate).

metrics.false_positive_rate

Calculate the false positive rate (also called fall-out).

metrics.make_derived_metric

Create a scalar returning metric function based on aggregation of a disaggregated metric.

metrics.mean_prediction

Calculate the (weighted) mean prediction.

metrics.plot_model_comparison

Create a scatter plot comparing multiple models along two metrics.

metrics.selection_rate

Calculate the fraction of predicted labels matching the 'good' outcome.

metrics.true_negative_rate

Calculate the true negative rate (also called specificity or selectivity).

metrics.true_positive_rate

Calculate the true positive rate (also called sensitivity, recall, or hit rate).

metrics.MetricFrame

Collection of disaggregated metric values.

fairlearn.postprocessing#

This module contains methods which operate on a predictor, rather than an estimator.

The predictor’s output is adjusted to fulfill specified parity constraints. The postprocessors learn how to adjust the predictor’s output from the training data.

postprocessing.ThresholdOptimizer

A classifier based on the threshold optimization approach.

postprocessing.plot_threshold_optimizer

Plot the chosen solution of the threshold optimizer.

fairlearn.preprocessing#

Preprocessing tools to help deal with sensitive attributes.

preprocessing.CorrelationRemover

A component that filters out sensitive correlations in a dataset.

fairlearn.reductions#

This module contains algorithms implementing the reductions approach to disparity mitigation.

In this approach, disparity constraints are cast as Lagrange multipliers, which cause the reweighting and relabelling of the input data. This reduces the problem back to standard machine learning training.

reductions.AbsoluteLoss

Class to evaluate absolute loss.

reductions.BoundedGroupLoss

Moment for constraining the worst-case loss by a group.

reductions.ClassificationMoment

Moment that can be expressed as weighted classification error.

reductions.DemographicParity

Implementation of demographic parity as a moment.

reductions.EqualizedOdds

Implementation of equalized odds as a moment.

reductions.ErrorRate

Misclassification error as a moment.

reductions.ErrorRateParity

Implementation of error rate parity as a moment.

reductions.ExponentiatedGradient

An Estimator which implements the exponentiated gradient reduction.

reductions.TruePositiveRateParity

Implementation of true positive rate parity as a moment.

reductions.FalsePositiveRateParity

Implementation of false positive rate parity as a moment.

reductions.UtilityParity

A generic moment for parity in utilities (or costs) under classification.

reductions.GridSearch

Estimator to perform a grid search given a blackbox estimator algorithm.

reductions.LossMoment

Moment that can be expressed as weighted loss.

reductions.Moment

Generic moment.

reductions.BoundedGroupLoss

Moment for constraining the worst-case loss by a group.

reductions.SquareLoss

Class to evaluate the square loss.

reductions.ZeroOneLoss

Class to evaluate a zero-one loss.

fairlearn.adversarial#

Adversarial techniques to help mitigate unfairness.

adversarial.AdversarialFairnessClassifier

Train PyTorch or TensorFlow classifiers while mitigating unfairness.

adversarial.AdversarialFairnessRegressor

Train PyTorch or TensorFlow regressors while mitigating unfairness.

fairlearn.experimental#

Enables experimental functionality that may be migrated to other modules at a later point.

Warning

Anything can break from version to version without further warning.

metrics._plotter.plot_metric_frame

Visualization for metrics with and without confidence intervals.