An article written by Aleksandra Plewinska
«Data are just summaries of thousands of stories—tell a few of those stories to help make the data meaningful.» – Chip and Dan Heath.
In today’s corporate world, “data-driven decisions” have become a modern buzzword, advertised as the key to effective management. But are we making sure that all stories are being told equally through existing data? The reality is often disappointing: the gender data gap—a lack of comprehensive, differentiated data about women—results in skewed decision-making that disproportionately favours men.
The standard «human» taken into account in decisions which are supported by data is predominantly male. Why is femininity often viewed as inferior? The roots of this issue are complex. The concept of the «Default Male» is deeply embedded in patriarchal structures, shaping everything from evolution and history to language and design. Society has long operated on the assumption that men are the norm, often overlooking female needs and experiences. Caroline Criado Perez’s book Invisible Women delves into this bias, uncovering how it influences daily life in ways that are often surprising and not immediately obvious.
For example, mobile phone keyboards are typically designed to fit larger male hands, making them less ergonomic for many women. Office temperatures are calibrated based on male metabolic rates, leaving women frequently uncomfortably cold. In terms of safety standards, car crash-test dummies are modelled on male bodies, which makes women 71% more likely to sustain injuries in accidents. Similarly, the first artificial heart was designed too large to fit most women’s bodies. The issue extends further into medical research. Nearly half of the diseases that disproportionately affect women are studied using only male animals, resulting in significant gaps in healthcare knowledge and treatment options for women. These examples underscore the widespread and systemic nature of the «Default Male» bias and its far-reaching impact on women’s lives.
These examples reflect a systemic bias where “male” is assumed to be the default, and women’s needs are often overlooked or ignored. Data biases stem from how data was being collected and how research was being conducted through the centuries. This deficiency not only hampers the ability to accurately capture diversity of behaviours and traits but also jeopardises efforts to create an inclusive world for everyone to live and thrive in.
As of today not a single country in the world has achieved gender equality.
Amazon’s AI Hiring Algorithm and its issues: a case study
While things are definitely slowly improving, the existing data gap still has a big impact on how biases shape our future. It’s not just some outdated issue from the past—it’s something we’re facing head-on with today’s most advanced technologies. The development of AI and the data it is trained on carry significant gender implications for AI-driven solutions. These systems often reflect societal patriarchal structures and inherent biases, largely influenced by their predominantly male creators.
Think about hiring algorithms. Instead of being fair, some of them actually penalize resumes with female-associated terms or those that show career breaks—like for caregiving—basically punishing people for their life choices.
One example of this can be a currently defunct AI recruiting tool from Amazon. In 2014 the company started developing an automated tool that, in ideal circumstances, would help with the recruiting process by pre-filtering the candidates. A year later its developers noticed that the tool was not rating the female candidates as well as their male counterparts.
They have soon discovered the root of the problem was the data source. The computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. Which simply meant that the gender imbalance in tech was only being perpetuated by favouring hiring male job applicants. At the end the tool had to be scrapped, as it was not possible to find a straightforward solution to fix it.
Dr. Boulamwini study on Facial Recognition Biases
Joy Buolamwini in her book Unmasking AI adds more context for how biases in Machine Learning start. Each of us has a unique perspective shaped by our social and cultural experiences—our personal lens through which we see and understand the world. This is our own ground truth.
In machine learning, however, ground truth isn’t something absolute; it’s determined by those who decide what the «correct» labels should be and by the annotators applying those labels. Their choices, influenced by their own experiences and biases, shape the data that trains AI models. And because AI is becoming such a big part of our everyday lives, those biases can stick around or even get worse.Take facial recognition, for example. It often struggles to identify women, especially women of colour, which can lead to all kinds of issues, from inconvenience to serious discrimination.
AI reflects and reinforces the data gaps we have failed to address.
In her research, Dr. Buolamwini decided to focus particularly on people with darker skin tones and immediately encountered obstacles as there were not many datasets available for those populations. She had to assemble the datasets herself, classifying the skintones based on a scale used by dermatologists to assess the risk of sunburn.
Then she applied three existing models designed and trained by leading companies in the industry to her dataset in order to assert the quality of their facial recognition. Across all three, the error rates for gender classification were consistently higher for females than they were for males, and for darker-skinned subjects than for lighter-skinned subjects.
Faces of women with darker skin error rates were 20.8, 34.5 and 34.7 consecutively for skin tones rated on the highest end of this scale – IV, V and VI. These numbers mean that the models might as well have guessed their gender at random. This is especially alarming considering that many governments and governmental agencies use facial recognition models for surveillance and policing: such a high error rate directly translates to risk of false arrest or wrongful accusations, disproportionately impacting already marginalized communities.
The problem is, the more we rely on AI to make decisions or shape policies, the more these biases and blind spots sneak into systems that affect everyone. If we don’t fix the data gaps now, we’re setting ourselves up for a future where inequality is baked into the very tools meant to make life better. Addressing this isn’t just about being fair—it’s about making sure AI works for everyone, not just a select group. To address the well-documented challenges with data quality, various strategies are being developed to help close the gender data gap.
New Strategies Aiming to Close the Gender Data Gap
1. Inclusive Data Collection:
Collecting gender-balanced data is critical for creating fair and inclusive solutions across various sectors such as research, technology, and urban planning. By ensuring that data reflects the experiences and needs of all genders, we can design systems and policies that better serve diverse populations and reduce existing inequalities.
2. Synthetic Data:
On 14th of February 2024 UNU – United Nations University (a think tank and academic arm of the United Nations) published a policy guideline: Recommendations on the Use of Synthetic Data to Train AI Models. It recommends, whenever possible, the use of synthetic data in training models as a bridge filling missing data; citing synthetic data’s capacity to “address imbalanced training datasets that lead to AI bias, as in the case of gender, or racial bias” as being one of its main advantages. This type of data can help to fill out the missing gaps in the knowledge, making the results more balanced. The obvious disadvantage is artificially simulated figures can’t be used to distinguish emerging patterns or recognize new relationships between data.
3. Diverse Development Teams:
The composition of development teams significantly impacts projects’ outcomes. Teams which consist of diverse perspectives, including a balance of genders, are better equipped to identify and address gender biases during the development process. This diversity promotes innovative solutions that consider a wider range of user needs and experiences.
4. Policy and Regulation:
Government policies and organizational regulations can help to create more inclusive data practices by setting standards for data collection, analysis, and usage. These frameworks can ensure accountability and encourage stakeholders to prioritize gender equity in data-driven initiatives.
5. Education and Awareness:
Educating data scientists, engineers, and decision-makers about the presence and impact of gender biases in data is essential. Awareness and training programmes can empower professionals to recognize and correct biases, ensuring that their work contributes to equitable and inclusive advancements in technology and policy.
The gender data gap is not just a minor oversight; it is a systemic issue with consequences in real-world situations. Closing this gap requires recognizing and addressing the underlying biases in how we collect, analyze, and apply data. Only then can we ensure that data-driven decisions truly reflect the diversity of human experience and create a world designed for everyone—not just the «Default Male.»
Love these topics? Stay connected! Follow us on Eventbrite, explore our upcoming events, and be part of our next exciting session!
References
Buolamwini, J. (2023). Unmasking AI: My Mission to Protect What Is Human in a World of Machines. Random House.
Criado-Perez, C. (2019). Invisible Women: Exposing Data Bias in a World Designed for Men. Chatto & Windus.
Reuters. (2018, October 10). Insight: Amazon scraps secret AI recruiting tool that showed bias against women. Retrieved from https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG/
United Nations university (2024). Recommendations on the Use of Synthetic Data to Train AI Models. [Policy guideline].
