- fairlearn.metrics.make_derived_metric(*, metric, transform, sample_param_names=['sample_weight'])[source]#
Create a scalar returning metric function based on aggregation of a disaggregated metric.
Many higher order machine learning operations (such as hyperparameter tuning) make use of functions which return scalar metrics. We can create such a function for our disaggregated metrics with this function.
This function takes a metric function, a string to specify the desired aggregation transform (matching the methods
MetricFrame.ratio()), and a list of parameter names to treat as sample parameters.
The result is a callable object which has the same signature as the original function, with a
sensitive_features=parameter added. If the chosen aggregation transform accepts parameters (currently only
method=is supported), these can also be given when invoking the callable object. The result of this function is identical to creating a
MetricFrameobject, and then calling the method specified by the
transform=argument (with the
method=argument, if required).
See the Defining custom fairness metrics section in the User Guide for more details. A sample notebook is also available.
metric (callable) – The metric function from which the new function should be derived
transform (str) – Selects the transformation aggregation the resultant function should use. The list of possible options is: [‘difference’, ‘group_min’, ‘group_max’, ‘ratio’].
sample_param_names (List[str]) – A list of parameters names of the underlying
metricwhich should be treated as sample parameters (i.e. the same leading dimension as the
y_predparameters). This defaults to a list with a single entry of
sample_weight(as used by many SciKit-Learn metrics). If
Noneor an empty list is supplied, then no parameters will be treated as sample parameters.
Function with the same signature as the
metricbut with additional
method=arguments, to enable the required computation
- Return type