Development process#

Development happens against the main branch following the GitHub flow model. Contributors should use their own forks of the repository. In their fork, they create feature branches off of main, and their pull requests should target the main branch. Maintainers are responsible for prompt review of pull requests.

Pull requests against main trigger automated tests that are run through Azure DevOps, GitHub Actions, and CircleCI. Additional test suites are run periodically. When adding new code paths or features, tests are a requirement to complete a pull request. They should be added in the test directory.

Documentation should be provided with pull requests that add or change functionality. This includes comments in the code itself, docstrings, and user guides. For exceptions to this rule the pull request author should coordinate with a maintainer. For changes that fix bugs, add new features, change APIs, etc., i.e., for changes that are relevant to developers and/or users please also add an entry in CHANGES.md in the section corresponding to the next release, since that’s where your change will be included. If you’re a new contributor please also add yourself to AUTHORS.md.

Docstrings should follow numpydoc format. This is a recent decision by the community. The new policy is to update docstrings that a PR touches, as opposed to changing all the docstrings in one PR.

Advanced installation instructions#

While working on Fairlearn itself you may want to install it in editable mode. This allows you to test the changed functionality. First, clone the repository locally via

git clone git@github.com:fairlearn/fairlearn.git

To install in editable mode using pip run

pip install -e .

from the repository root path.

To verify that the code works as expected run

python ./scripts/install_requirements.py --pinned False
python -m pytest -s ./test/unit

Fairlearn currently includes plotting functionality that requires the matplotlib package to be installed. Since this is for a niche use case Fairlearn comes without matplotlib by default. To install Fairlearn with its full feature set simply append customplots to the install command

pip install -e .[customplots]

The Requirements Files#

The prerequisites for Fairlearn are split between three separate files:

requirements.txt contains the prerequisites for the core Fairlearn package

requirements-customplots.txt contains additional prerequisites for the [customplots] extension for Fairlearn

requirements-dev.txt contains the prerequisites for Fairlearn development (such as flake8 and pytest)

The requirements.txt and requirements-customplots.txt files are consumed by setup.py to specify the dependencies to be documented in the wheel files. To help simplify installation of the prerequisites, we have the install_requirements.py script which runs pip install on all three of the above files. This script will also optionally pin the requirements to any lower bound specified (by changing any occurrences of >= to == in each file).

Contributing a pull request#

Follow the steps below to create a pull request.

Get a GitHub account
Install git
Look at Fairlearn’s issues on GitHub, specifically the ones marked “help wanted”. Within this category we’ve marked issues with labels:
- “good first issue” means issues are a good fit for first time contributors including people with no prior experience with coding or GitHub. This is an excellent way to get started!
- “easy” means the issue is somewhat harder than a “good first issue”, but still quite doable if you have prior experience or you’re willing to spend a little bit of time.
- Neither of the two above: this means issues haven’t been scoped out properly yet or are more difficult and likely won’t be doable within a day or two. If you’re still interested just let us know and we’ll figure out a way! Once you find an issue you’re interested in comment on the issue with any questions you may have and let us know that you’d like to do it. This helps us avoid duplication and we can help you quicker.
Whenever questions come up don’t hesitate to reach out (see Real-time communication). We’re here to help!
Clone the Fairlearn repository onto your machine using git clone https://github.com/fairlearn/fairlearn.git
Fork the Fairlearn repository (using the “Fork” button on the Fairlearn repo), then run git remote add <your-alias> https://github.com/<your-alias>/fairlearn.git while replacing <your-alias> with your actual alias. To check whether it worked run git remote -v and it should show both origin pointing to the original fairlearn repo and <your-alias> pointing to your fork. Now you can create a new branch and start changing the world!
To check your branch run git status. Initially it will point to main which is the default. Create a new branch for yourself by running git checkout -b <branch-name>. git checkout is your way of switching branches, while -b creates a new branch and should only be added the first time you check out a (new) branch. Whenever you are ready to commit your changes run git add --all and git commit --all or use the version control functionality of your IDE (e.g., Visual Studio Code). To push the changes to your fork run git push <your-alias>. Note that you cannot push to origin (the main fairlean repository) because it is access-restricted.
Build the website following Contributing documentation
To create a pull request go to the Fairlearn repo and select “New Pull Request”. Click “compare across forks” and subsequently configure the “compare” branch to be the one you pushed your changes to. Briefly check the file changes in the resulting view and click “create pull request” when you’re confident about your changes. The following view will ask you to add a pull request title and description, and if you created the pull request in response to an issue add #<issue-number> for reference.
Celebrate! You did great by participating. If you would like to be a part of the Fairlearn community we’d be thrilled to discuss ways for you to get involved! Check out our communication channels, Real-time communication, for more information.

Investigating automated test failures#

For every pull request to main with automated tests, you can check the logs of the tests to find the root cause of failures. Our tests currently run through Azure Pipelines with steps for setup, testing, and teardown. The Checks tab of a pull request contains a link to the Azure Pipelines page), where you can review the logs by clicking on a specific step in the automated test sequence. If you encounter problems with this workflow, please reach out through GitHub issues.

To run the same tests locally, find the corresponding pipeline definition (a yml file) in the devops directory. It either directly contains the command to execute the tests (usually starting with python -m pytest) or it refers to a template file with the command.