Development process#

Development happens against the main branch following the GitHub flow model. Contributors should use their own forks of the repository. In their fork, they create feature branches off of main, and their pull requests should target the main branch. Maintainers are responsible for prompt review of pull requests.

Pull requests against main trigger automated tests that are run through Azure DevOps, GitHub Actions, and CircleCI. Additional test suites are run periodically. When adding new code paths or features, tests are a requirement to complete a pull request. They should be added in the test directory.

Documentation should be provided with pull requests that add or change functionality. This includes comments in the code itself, docstrings, and user guides. For exceptions to this rule the pull request author should coordinate with a maintainer. For changes that fix bugs, add new features, change APIs, etc., i.e., for changes that are relevant to developers and/or users please also add an entry in Version guide in the section corresponding to the next release, since that’s where your change will be included. If you’re a new contributor please also add yourself to AUTHORS.md.

Docstrings should follow numpydoc format. This is a recent decision by the community. The new policy is to update docstrings that a PR touches, as opposed to changing all the docstrings in one PR.

Advanced installation instructions#

While working on Fairlearn itself you may want to install it in editable mode. This allows you to test the changed functionality. First, clone the repository locally via:

$ git clone git@github.com:fairlearn/fairlearn.git

To install in editable mode, from the repository root path run:

$ pip install -e .

To verify that the code works as expected run:

$ python ./scripts/install_requirements.py --pinned False
$ python -m pytest -s ./test/unit

Fairlearn currently includes plotting functionality provided by matplotlib. This is for a niche use case, so it isn’t a default requirement. To install run:

$ pip install -e .
$ pip install matplotlib

The Requirements Files#

The prerequisites for Fairlearn are split between three separate files:

requirements.txt contains the prerequisites for the core Fairlearn package

requirements-dev.txt contains the prerequisites for Fairlearn development (such as ruff and pytest)

The requirements.txt file is consumed by setup.py to specify the dependencies to be documented in the wheel files. To help simplify installation of the prerequisites, we have the install_requirements.py script which runs pip install on both the above files. This script will also optionally pin the requirements to any lower bound specified (by changing any occurrences of >= to == in each file).

Contributing a pull request#

Follow the steps below to create a pull request.

Get a GitHub account.
Install GIT.
Look at Fairlearn’s issues on GitHub, specifically the ones marked “help wanted”. Within this category we’ve marked issues with labels:
- “good first issue”: issues suitable for first time contributors, including people with no prior experience with coding or GitHub. This is an excellent way to get started!
- “easy”: issues suitable for folks with at least a bit of experience and/or able to allocate some time to look for a solution.
- Neither of the two above: issues that are demanding or awaiting scope. Likely to take more than a day or two. If you think this is something for you, please:
  - Identify an issue that you would like to work on.
  - Leave a comment on the issue indicating interest and outlining possible questions.
  - Once we know you are working on it, we will support you on your contribution journey!
The communication channels are outlined here: Real-time communication.
Fork the project repository by clicking on the ‘Fork’ button near the top of the page. This creates a copy of the code on your GitHub user account. For more details on how to fork a repository see this guide.

Clone your fork of the fairlern repo from your GitHub account to your local machine:

git clone git@github.com:YourLogin/fairlearn.git  # add --depth 1 if your connection is slow
cd fairlearn

Add the upstream remote. This saves a reference to the main fairlearn repository, which you can use to keep your repository synchronized with the latest changes:
```
$ git remote add upstream git@github.com:fairlearn/fairlearn.git
```

Check that the upstream and origin remote aliases are configured correctly by running

git remote -v which should display:

origin  git@github.com:YourLogin/fairlearn.git (fetch)
origin  git@github.com:YourLogin/fairlearn.git (push)
upstream        git@github.com:fairlearn/fairlearn.git (fetch)
upstream        git@github.com:fairlearn/fairlearn.git (push)

(Optional) Install pre-commit to run code style checks before each commit:
$ pip install pre-commit $ pre-commit install

Pre-commit checks can be disabled for a particular commit with git commit -n.
To contribute, you will need to create a branch on your forked repository and make a pull request to the original fairlearn repository. Detailed description of this process you can find here:
- Create a branch.
- Commit and push changes.
- Make a pull request.
  - Build the website following the guidelines in Contributing documentation and run the tests if necessary.
  - Opening a pull request comes with filling up an already provided description template. Please fill it up! If you created the pull request in response to an issue add #<issue-number> for reference.
  - If the PR introduces something that will affect the users, please add a changelog entry in the docs/user_guide/installation_and_version_guide directory.
Celebration time! We would like to encourage you to become a part of our Fairlearn community. To do so, join our communication channels: Real-time communication.

Investigating automated test failures#

For every pull request to main with automated tests, you can check the logs of the tests to find the root cause of failures. Our tests currently run through Azure Pipelines with steps for setup, testing, and teardown. The Checks tab of a pull request contains a link to the Azure Pipelines page), where you can review the logs by clicking on a specific step in the automated test sequence. If you encounter problems with this workflow, please reach out through GitHub issues.

To run the same tests locally, find the corresponding pipeline definition (a yml file) in the devops directory. It either directly contains the command to execute the tests (usually starting with python -m pytest) or it refers to a template file with the command.