Aequitas is an open source bias and fairness audit toolkit that was released in 2018. It is designed to enable developers to seamlessly test models for a series of bias and fairness metrics in relation to multiple population sub-groups.
As part of Microsoft Fair Learn, this is a general-purpose methodology for approaching fairness. Using binary classification, the method applies constraints to reduce fair classification to a sequence of cost-sensitive classification problems. Whose solutions yield a randomized classifier with the lowest (empirical) error subject to the desired constraints.
The What-if Tool from Google is an open-source TensorBoard web application which lets users analyse an ML model without writing code. It visualises counterfactuals so that users can compare a data-point to the most similar point where the model predicts a different result. In addition, users can explore the effects of different classification thresholds, taking into account constraints such as different numerical fairness criteria. There are a number of demos available – showing how the different functions work on pre-trained models.
IBM 360 degree toolkit contains a comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets at the pre-processing and model training stages.
