v0.15.0#

Summary#

This release adds ROC curves by sensitive group, automatic plotting of bootstrap confidence intervals, and output feature names for CorrelationRemover. It fixes incorrect fairness results for weighted single-sample groups, centering of multiple sensitive features, and reuse of reductions constraints across fits.

Compatibility changes: Python 3.11 or newer and scikit-learn 1.6.0 or newer are now required, and Python 3.14 is supported. The bank marketing and credit card dataset loaders now use the authoritative UCI column names; code selecting columns by the previous OpenML names needs updating. MetricFrame now rejects missing sensitive or control feature values, and CorrelationRemover rejects reordered or renamed columns in named transform inputs. The fitted reductions constraint is now available as constraints_, while the user-supplied constraints object remains unmodified.

New features#

  • Added plot_roc_curve_by_group() to plot ROC curves disaggregated by sensitive feature, together with the overall curve and a chance-level baseline. The corresponding AUC scores can be obtained with MetricFrame and sklearn.metrics.roc_auc_score: #869 by kstohr and Roman Lutz.

  • When bootstrap confidence intervals are available on a MetricFrame, plot_metric_frame() now auto-detects them from by_group_ci when the conf_intervals argument is not supplied: #1674 by Dresden.

  • Added get_feature_names_out to CorrelationRemover, so it can be used with set_output and reports its output feature names in pipelines and ColumnTransformer: #1745 by Raashish Aggarwal.

Bug fixes#

  • CorrelationRemover rejects reordered or renamed columns in named transform inputs instead of silently applying the fitted sensitive-column positions to different features: #1739 by @nawaaaaaAaar.

  • MetricFrame now raises a ValueError when a sensitive or control feature contains missing values, naming the offending feature. Previously a missing value was silently dropped from by_group while still counting towards overall, and a feature passed as a list instead of a pandas.Series turned the missing value into a category named 'nan': #1698 by Edward Yi.

  • Fixed AssertionError: data can be loaded only once raised when a Moment instance is passed as constraints to more than one call of ExponentiatedGradient.fit or GridSearch.fit, or shared between estimators. The fitted estimators now deep-copy the user-supplied constraints before loading data, leave the original object unmodified, and expose the loaded copy as the fitted attribute constraints_: #1632 by BALOGUN DAVID TAIWO.

  • Calling Moment.load_data more than once on the same instance now raises a ValueError instead of an AssertionError: #1705 by @SpiliosDimakopoulos.

  • Fixed fetch_diabetes_hospital so that as_frame=False returns numpy arrays instead of raising. The data is now always fetched as a pandas DataFrame internally and converted to numpy when needed. The skipped tests and the obsolete test_fetch_diabetes_hospital_as_ndarray_raises_value_error test were removed: #1636 by Richard Ogundele.

  • BackendEngine used to read X.shape[1] without checking how many dimensions X had, so passing data with more than two dimensions (for example an array of images) would silently build a predictor model with the wrong input size instead of failing. It now raises a ValueError up front when X is not two-dimensional: #1671 by Arnav.

  • Fixed FloatTransformer.transform and FloatTransformer.inverse_transform so that they raise NotFittedError instead of AttributeError when called before fit: #1709 by @SpiliosDimakopoulos.

  • Fixed plot_roc_curve_by_group() so that rows with missing sensitive feature values are included in the overall curve but omitted from per-group curves, instead of raising a ValueError: #1713 by @SpiliosDimakopoulos.

  • Already-fitted adversarial estimators now enforce the same two-dimensional input requirement in predict and partial_fit, including when input validation is skipped: #1711 by Roman Lutz.

  • Fixed CorrelationRemover so that each sensitive feature is centered by its own mean. With more than one sensitive feature, fit used a single mean over all sensitive columns, so the transformed features stayed correlated with the sensitive features whenever those columns had different means (for example several one-hot encoded groups). sensitive_mean_ now holds one mean per sensitive feature: #1732 by @breken-ai.

  • MetricFrame now accepts NumPy integers (for example np.int64) for n_boot and random_state. Previously they raised a ValueError: #1737 by @Arthur031221.

  • Fixed selection_rate() returning a one-element array instead of a scalar when sample_weight is passed for a single sample. In a weighted MetricFrame this happens for any group with one member, and demographic_parity_difference() and demographic_parity_ratio() then reported 0 and 1 (no disparity) regardless of the actual selection rates: #1734 by @breken-ai.

Documentation#

  • Removed stale deprecation notices in docstrings for changes that completed in v0.10 (ThresholdOptimizer.predict_method default, InterpolatedThresholder.predict_method default, and MetricFrame constructor positional-argument deprecation).

  • Added versionadded/versionchanged directives to the CorrelationRemover API reference, documenting when the estimator was introduced (v0.6.0) and subsequently changed (v0.8.0, v0.12.0): #1748 by Peace Amhanesi.

Other improvements#

  • Tightened the internal projection of Lagrange multipliers for difference-based reductions constraints by removing the redundant per-event group-membership component while preserving the classifier-dependent Lagrangian term: #1714 by quinnarnold.

  • Behavior change. fetch_bank_marketing() and fetch_credit_card() now report the authoritative UCI column names (age, job, … and LIMIT_BAL, SEX, …) instead of passing through the placeholder names carried by the underlying OpenML records (V1 … V16 and x1 … x23), so a sensitive feature can be selected by its documented name. The names are applied consistently to feature_names, the data and frame DataFrame columns, and the return_X_y=True result; categorical metadata keys in non-frame Bunches are remapped to match. Each returned Bunch has its own feature_names list, so modifying it does not affect other results or later loads. Code that selects columns by the previous OpenML names will need updating; positional access, OpenML data IDs, targets, arrays, and dtypes are unchanged. The loaders now raise a ValueError rather than relabelling if a record’s column count ever stops matching: #1728 by @breken-ai.

  • Preserved multi-column sensitive feature data in fairlearn.postprocessing.ThresholdOptimizer data reformatting helpers: #1679 by BALOGUN-DAVID.

  • Replaced the PyTorch adversarial engine’s hard-coded activation string checks with a lookup table and added support for Tanh, GELU, ELU, and SELU activation strings. Activation strings are now matched case-insensitively on both the PyTorch and TensorFlow backends (the TensorFlow backend previously failed on capitalized strings such as Sigmoid): #1665 by Aryan.

  • Added support for Python 3.14: #1649 by Roman Lutz.

  • Bumped the minimum sphinx version from 7.3.7 to 8.3.0 (and bumped sphinx-prompt from 1.8.0 to 1.9.0 and sphinx-autodoc-typehints from 2.0.1 to 3.5.2 in the minimum-version pins for compatibility) so the documentation build picks up newer sphinx theme and extension improvements.

  • Modernized internal type annotations to PEP 585 / PEP 604 syntax (list[T], tuple[T, ...], T | U) where the imports from typing were no longer necessary.

  • Removed support for Python 3.10. The minimum supported Python version is now 3.11: #1653 by Roman Lutz.

  • Raised the minimum supported scikit-learn version to 1.6.0 and removed the fairlearn.utils._fixes compatibility shim, which existed to bridge the scikit-learn 1.5 ↔ 1.6 API change (validate_data import location, force_all_finite → ensure_all_finite rename, and the wrapped parametrize_with_checks). Internal callers now import directly from sklearn.utils.validation and sklearn.utils.estimator_checks. Also added a second set of pytest-mpl baseline snapshots under test/unit/plot_snapshots_mpl310/ for matplotlib==3.10.x, alongside the matplotlib>=3.11 snapshots, since the two render-engine versions produce noticeably different output: #1654 by Roman Lutz.

  • Moved package metadata and runtime dependencies from setup.py and requirements.txt into pyproject.toml. Builds continue to use setuptools and the version in fairlearn/__init__.py: #1629 by Christian Heimes.

  • Removed dead # noqa comments throughout fairlearn/ whose suppressed rule codes are not enabled in the project’s ruff configuration, plus the invalid # noqa : G003 (extra space) typo variants.

  • Worked around a pytest.PytestRemovedIn10Warning collection failure on pytest>=9 by adding a thin parametrize_with_checks wrapper in test/_sklearn_compat.py that materializes the generator returned by sklearn.utils.estimator_checks.parametrize_with_checks into a list before handing it to pytest.mark.parametrize: #1666 by Roman Lutz.

  • Removed the fairlearn.utils._compatibility shim, which only existed to bridge the OneHotEncoder(sparse=...) → sparse_output=... rename completed in scikit-learn 1.4. No longer needed now that the minimum supported scikit-learn is ≥ 1.6: #1664 by Roman Lutz.

  • Enabled the pydocstyle (D) ruff rule set with the numpy convention. Added docstrings where missing for public APIs and per-file ignores for tests, examples, and scripts.

  • Enabled the flake8-logging-format (G) ruff rule set and converted existing logging calls to use lazy %s formatting.

  • Clarified the fitted-state check in _AdversarialFairness._validate_input by replacing a try/except NotFittedError block used purely for control flow with a direct hasattr(self, "_is_setup") probe against the class’s canonical fitted flag, and removed the now-unused NotFittedError import: #1673 by Dresden.

  • Updated Ruff to 0.15.20, enabled the UP, B, SIM, and C4 rule families, and applied the resulting lint cleanups across the codebase: #1675 by Roman Lutz.