API Docs

fairlearn.datasets

This module contains datasets that can be used for benchmarking and education.

datasets.fetch_acs_income	Load the ACS Income dataset (regression).
datasets.fetch_adult	Load the UCI Adult dataset (binary classification).
datasets.fetch_bank_marketing	Load the UCI bank marketing dataset (binary classification).
datasets.fetch_boston	Load the boston housing dataset (regression).
datasets.fetch_credit_card	Load the 'Default of Credit Card clients' dataset (binary classification).
datasets.fetch_diabetes_hospital	Load the preprocessed Diabetes 130-Hospitals dataset (binary classification).

fairlearn.metrics

Functionality for computing metrics, with a particular focus on disaggregated metrics.

For our purpose, a metric is a function with signature f(y_true, y_pred, ....) where y_true are the set of true values and y_pred are values predicted by a machine learning algorithm. Other arguments may be present (most often sample weights), which will affect how the metric is calculated.

This module provides the concept of a disaggregated metric. This is a metric where in addition to y_true and y_pred values, the user provides information about group membership for each sample. For example, a user could provide a ‘Gender’ column, and the disaggregated metric would contain separate results for the subgroups ‘male’, ‘female’ and ‘nonbinary’ indicated by that column. The underlying metric function is evaluated for each of these three subgroups. This extends to multiple grouping columns, calculating the metric for each combination of subgroups.

metrics.count	Calculate the number of data points in each group when working with MetricFrame.
metrics.demographic_parity_difference	Calculate the demographic parity difference.
metrics.demographic_parity_ratio	Calculate the demographic parity ratio.
metrics.equalized_odds_difference	Calculate the equalized odds difference.
metrics.equalized_odds_ratio	Calculate the equalized odds ratio.
metrics.equal_opportunity_difference	Calculate the equal opportunity difference.
metrics.equal_opportunity_ratio	Calculate the equal opportunity ratio.
metrics.false_negative_rate	Calculate the false negative rate (also called miss rate).
metrics.false_positive_rate	Calculate the false positive rate (also called fall-out).
metrics.make_derived_metric	Create a scalar returning metric function based on aggregation of a disaggregated metric.
metrics.mean_prediction	Calculate the (weighted) mean prediction.
metrics.plot_model_comparison	Create a scatter plot comparing multiple models along two metrics.
metrics.selection_rate	Calculate the fraction of predicted labels matching the 'good' outcome.
metrics.true_negative_rate	Calculate the true negative rate (also called specificity or selectivity).
metrics.true_positive_rate	Calculate the true positive rate (also called sensitivity, recall, or hit rate).

metrics.MetricFrame

Collection of disaggregated metric values.

fairlearn.postprocessing

This module contains methods which operate on a predictor, rather than an estimator.

The predictor’s output is adjusted to fulfill specified parity constraints. The postprocessors learn how to adjust the predictor’s output from the training data.

postprocessing.ThresholdOptimizer

A classifier based on the threshold optimization approach.

postprocessing.plot_threshold_optimizer

Plot the chosen solution of the threshold optimizer.

fairlearn.preprocessing

Preprocessing tools to help deal with sensitive attributes.

preprocessing.CorrelationRemover

A component that filters out sensitive correlations in a dataset.

fairlearn.reductions

This module contains algorithms implementing the reductions approach to disparity mitigation.

In this approach, disparity constraints are cast as Lagrange multipliers, which cause the reweighting and relabelling of the input data. This reduces the problem back to standard machine learning training.

reductions.AbsoluteLoss	Class to evaluate absolute loss.
reductions.BoundedGroupLoss	Moment for constraining the worst-case loss by a group.
reductions.ClassificationMoment	Moment that can be expressed as weighted classification error.
reductions.DemographicParity	Implementation of demographic parity as a moment.
reductions.EqualizedOdds	Implementation of equalized odds as a moment.
reductions.ErrorRate	Misclassification error as a moment.
reductions.ErrorRateParity	Implementation of error rate parity as a moment.
reductions.ExponentiatedGradient	An Estimator which implements the exponentiated gradient reduction.
reductions.TruePositiveRateParity	Implementation of true positive rate parity as a moment.
reductions.FalsePositiveRateParity	Implementation of false positive rate parity as a moment.
reductions.UtilityParity	A generic moment for parity in utilities (or costs) under classification.
reductions.GridSearch	Estimator to perform a grid search given a blackbox estimator algorithm.
reductions.LossMoment	Moment that can be expressed as weighted loss.
reductions.Moment	Generic moment.
reductions.BoundedGroupLoss	Moment for constraining the worst-case loss by a group.
reductions.SquareLoss	Class to evaluate the square loss.
reductions.ZeroOneLoss	Class to evaluate a zero-one loss.

fairlearn.adversarial

Adversarial techniques to help mitigate unfairness.

adversarial.AdversarialFairnessClassifier	Train PyTorch or TensorFlow classifiers while mitigating unfairness.
adversarial.AdversarialFairnessRegressor	Train PyTorch or TensorFlow regressors while mitigating unfairness.

fairlearn.experimental

Enables experimental functionality that may be migrated to other modules at a later point.

Warning

Anything can break from version to version without further warning.

metrics._plotter.plot_metric_frame

Visualization for metrics with and without confidence intervals.