StereoSet is a dataset that measures stereotype bias in language models. StereoSet consists of 17,000 sentences that measures model preferences across gender, race, religion, and profession.
StereoSet measures model preferences for stereotypical conditions across race, gender, religion, and profession, while also ensuring that debiasing techniques do not affect underlying model performance.
Their site includes a leaderboard where researchers test different ML models such as BERT, RoBERTA, XLNET, and GPT2 against StereoSet.
Company Type: Academia
Region: US & Canada
Industry Category: Data Science
Fighting Type of Bias: Racial bias Gender bias Cultural bias
Research: StereoSet: Measuring stereotypical bias in pretrained language models
Data set: StereoSet database (distributed under the CC BY-SA 4.0 license)
