The things we choose to mention when describing other people can say a lot about what we consider to be ”normal”. For example, have you ever noticed how female sci-fi authors are often described in precisely that way — as ”female sci-fi authors” — while male sci-fi authors are simply considered to be ”sci-fi authors”? Have you ever heard someone tell a story about someone they met and mentioning that person’s disability, even when it had no importance for the story? Or heard someone talk about their ”gay friend”, without describing any of their other friends as ”straight”?
This selectivity in descriptions is a necessary feature of human language: if we tried to mention everything about the thing we are describing, the resulting message would be too long to communicate. Therefore, we have to restrict ourselves to the information that we consider most relevant, and leave out things that are obvious or can be inferred from context. This leads to something known as reporting bias: a systematic discrepancy between reality and our descriptions of it. Reporting bias explains why, for example, the bigram ”green banana” is a lot more common
in text data than ”yellow banana”, even though yellow is the more prototypical color for bananas. In descriptions of people, reporting bias often manifests as a tendency to over report on attributes that deviate from the norm, such as queerness, disability or being non-white. This makes reporting bias a useful lens through which to study social norms and attitudes in both human utterances and language models.
We developed the MARB (Marked Attribute Reporting Bias) dataset as a way to study language models’ reporting bias with regard to sensitive human attributes such as race, queerness and disability. Unlike many existing benchmark datasets, MARB does not rely on artificially constructed templates or crowdworkers to create contrasting examples. Instead, the templates used in MARB are based on naturally occurring written language from the 2021 version of the enTenTen corpus (Jakubíček et al., 2013).
The dataset consists of 28.5K template sequences – 9.5K containing each of the phrases "a person", "a woman", "a man" – and their modifications. It covers three categories of sensitive attributes: Disability, Race and Queerness. Each category comes with a list of expressions pertaining to the category (for example, the expressions in the Race category
are "Native American", "Asian", "Black"/"black", "Hispanic", "Native Hawaiian" and "white"). Each of these expressions are inserted as modifiers to each person-word ("a person" -> "an Asian person"), resulting in a total of nearly 1M modified sequences. We also provide code and templates for expanding the dataset with your own categories and descriptors. By comparing the probabilities a language model assigns to different versions of the same original sequence, we can find out whether the model expects to see certain attributes mentioned more than others. For example, our experiments showed that models are noticeably more likely to mention sensitive attributes (such as race, queerness or disability) when describing women.
If you are curious to know more about the dataset or our experiments, you can read our paper, which won the Best Paper Award at the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP 2025). Feel free to also explore the dataset for your own research questions!