Contributing code

General advice

The field of ML fairness is nascent and developing, and while there are many emerging methods in the fairness literature, the Fairlearn team is discerning when it comes to adding new methods to the library. We often get requests to add emerging methods as features, but if you want to suggest including new features in the future, please keep in mind the guidance in this section. For algorithms, we require all methods to be described in a peer-reviewed paper; the Fairlean team specifies this requirement as a quality check, so we do not need to complete the peer reviewing ourselves. We have a preference for algorithms that are intuitive, easy to understand, and make explicit the underlying empirical and normative assumptions (for example, algorithms that are designed to address a specific type of measurement bias). For metrics, Fairlearn currently only supports disaggregated methods, so any proposed metrics that do not fall into the group fairness metric paradigm would first require thorough discussion with maintainers.

API conventions

This section relies on the definitions from our Fairness in Machine Learning guide, including the definitions of “estimator”, “reduction”, “sensitive features”, “moment”, and “parity”.

Unfairness mitigation algorithms

Unfairness mitigation algorithms take form of scikit-learn-style estimators. Any algorithm-specific parameters are passed to the constructor. The resulting instance of the algorithm should support methods fit and predict with APIs resembling those of scikit-learn as much as possible. Any deviations are noted below.

Reductions

Reduction constructors require a parameter corresponding to an estimator that implements the fit method with the sample_weight argument. Parity constraints for reductions are expressed via instances of various subclasses of the class fairlearn.reductions.Moment. Formally, instances of the class Moment implement vector-valued random variables whose sample averages over the data are required to be bounded (above and/or below).

constraints = Moment()
reduction = Reduction(estimator, constraints)

Reductions provide fit and predict methods with the following signatures:

reduction.fit(X, y, **kwargs)
reduction.predict(X)

All of the currently supported parity constraints (subclasses of Moment) are based on sensitive features that need to be provided to fit as a keyword argument sensitive_features. In the future, it will also be possible to provide sensitive features as columns of X.

Postprocessing algorithms

The constructors of postprocessing algorithms require either an already trained predictor or an estimator (which is trained on the data when executing fit). For postprocessing algorithms, the constraints argument is provided as a string.

postprocessor = PostProcessing(estimator=estimator, constraints=constraints)

Post-processing algorithms (such as the ones under fairlearn.postprocessing) provide the same functions as the reductions above albeit with sensitive_features as a required argument for predict. In the future, we will make sensitive_features optional if the sensitive features are already provided through X.

postprocessor.fit(X, y, sensitive_features=sensitive_features)
postprocessor.predict(X, sensitive_features=sensitive_features)

Code Style

We use flake8 to check for PEP8 compatibility issues. You can either follow the guidelines, or you could run black on your code. The generated formatting by black is compatible with the requirements we have. You can configure your IDE to use black to format your code. Please refer to your IDE’s instructions for further details.