v0.5.0#
Adjust classes to abide by naming conventions for attributes.
Change
ExponentiatedGradient
’s signature by renaming argumentT
tomax_iter
,eta_mul
toeta0
, and by addingrun_linprog_step
.API refactoring to separate out different uses of
eps
withinExponentiatedGradient
. It is now solely responsible for setting the L1 norm bound in the optimization (which controls the excess constraint violation beyond what is allowed by theconstraints
object). The other usage ofeps
as the right-hand side of constraints is now captured directly in the moment classes as follows:Classification moments:
ConditionalSelectionRate
renamed tofairlearn.reductions.UtilityParity
and its subclasses have new arguments on the constructor:difference_bound
- for difference-based constraints such as demographic parity differenceratio_bound_slack
- for ratio-based constraints such as demographic parity ratioAdditionally, there’s a
ratio_bound
argument which represents the argument previously calledratio
.
Regression moments:
ConditionalLossMoment
and its subclasses have a new argumentupper_bound
with the same purpose for newly enabled regression scenarios onExponentiatedGradient
.
For a comprehensive overview of available constraints refer to the new user guide on fairness constraints for reductions methods.
Renamed several constraints to create a uniform naming convention according to the accepted metric harmonization proposal:
ErrorRateRatio
renamed toErrorRateParity
, andTruePositiveRateDifference
renamed toTruePositiveRateParity
since the desired pattern is<metric name>Parity
with the exception ofEqualizedOdds
andDemographicParity
.ConditionalSelectionRate
renamed toUtilityParity
.GroupLossMoment
renamed toBoundedGroupLoss
in order to have a descriptive name and for consistency with the paper. Similarly,AverageLossMoment
renamed toMeanLoss
.For a comprehensive overview of available constraints refer to the new user guide on fairness constraints for reductions methods.
Added
TrueNegativeRateParity
to provide the opposite constraint ofTruePositiveRateParity
to be used with reductions techniques.Add new constraints and objectives in
ThresholdOptimizer
Add class
InterpolatedThresholder
to represent the fittedThresholdOptimizer
Add
fairlearn.datasets
module.Change the method to make copies of the estimator in
ExponentiatedGradient
frompickle.dump
tosklearn.clone
.Add an argument
sample_weight_name
toGridSearch
andExponentiatedGradient
to control howsample_weight
is supplied toestimator.fit
.Large changes to the metrics API. A new class
MetricFrame
has been introduced, andmake_group_summary()
removed (along with related functions). Please see the documentation and examples for more information.
Migrating to v0.5.0 from v0.4.6#
The update from v0.4.6 to v0.5.0 of Fairlearn has brought some major changes. This section goes through the adjustments required.
Metrics#
We have substantially altered the fairlearn.metrics
module.
In place of calling group_summary()
to produce a
sklearn.utils.Bunch
containing the disaggregated metrics, we have a
new class, MetricFrame
. The key advantages of the new API are:
Support for evaluating multiple metric functions at once
Support for multiple sensitive features
Support for control features
The MetricFrame
class has a constructor similar to
group_summary()
.
In v0.4.6, one would write
gs = group_summary(metric_func, y_true, y_pred, sensitive_features=A_col)
With the new API, this becomes
mf = MetricFrame(metrics=metric_func, y_true=y_true, y_pred=y_pred, sensitive_features=A_col)
The new object has MetricFrame.overall
and
MetricFrame.by_group
properties, to access the metric evaluated on
the entire dataset, and the metric evaluated on the subgroups of
A_col
.
In v0.4.6, we provided the following aggregator functions to compute a single
scalar from the result of group_summary()
.
group_min_from_summary()
group_max_from_summary()
difference_from_summary()
ratio_from_summary()
With MetricFrame
these become methods:
Before, one might write:
min_by_group = group_min_from_summary(gs)
Now, one can write:
min_by_group = mf.group_min()
The make_derived_metric()
function has been removed, but will be
reintroduced in a future release. The predefined convenience functions such as
accuracy_score_group_min()
and precision_score_difference()
remain.
For an introduction to all the new features, see the Metrics with Multiple Features example in Example Notebooks.
Renamed object attributes#
Some of the object attributes have been renamed from _<name>
to
<name>_
.
For example in both ExponentiatedGradient
and GridSearch
,
the _predictors
attribute is now called predictors_
.
Exponentiated Gradient and Moments#
In addition to the trailing underscore change mentioned above, several
adjustments have been made to the constructor arguments of
ExponentiatedGradient
.
The T
argument has been renamed to max_iter
, and the
eta_mul
argument to eta0
.
Furthermore, the eps
argument was previously used for two
different purposes, and these two uses have now been separated.
The use of eps
as the righthand side of the constraints
has now been moved to the Moment
classes.
The only remaining use of the eps
argument
is to control the optimality requirements for the optimization
algorithm in ExponentiatedGradient
.
For classification moments, ConditionalSelectionRate
has been
renamed to UtilityParity
, and there are three new
constructor arguments: difference_bound
, ratio_bound
(which
replaces ratio
) and ratio_bound_slack
.
For regression moments, BoundedGroupLoss
and its
subclasses have gained a new argument upper_bound
to serve as
the righthand side of the constraints.
Several Moment
objects have also been renamed in an effort
to improve consistency:
ErrorRateRatio
has becomeErrorRateParity
(when used with theratio_bound
andratio_bound_slack
arguments)TruePositiveRateDifference
has becomeTruePositiveRateParity
(when used with thedifference_bound
argument)ConditionalSelectionRate
has becomeUtilityParity
GroupLossMoment
has becomeBoundedGroupLoss