Mitigating AI Harms on Marginalized Populations: Key Takeaways from our AI and Marginalized Communities Discussion

On Thursday, December 19, All Tech Is Human hosted its fourth and final livestream in celebration of Mozilla’s 2025 Rise25 honorees, led by affiliate Lama Mohammed, in conversation with honoree Dr. Desmond Patton, on AI and marginalized communities. Desmond is the Brian and Randi Schwartz University Professor and the thirty-first Penn Integrates Knowledge University Professor. He holds joint appointments in the School of Social Policy & Practice and the Annenberg School for Communication, with secondary appointments in the Department of Psychiatry at the Perelman School of Medicine and at the Children's Hospital of Philadelphia (CHOP).

The conversation began with an overview of the sources of bias in AI. Bias in AI often stems from multiple sources, all of which reflect and perpetuate existing societal inequalities:

• Bias in Data: Traditional data systems are often built on datasets that reflect historical prejudices. These biases can be inadvertently integrated into AI algorithms, resulting in outputs that skew towards favoring dominant groups, while marginalizing others.

• Lack of Diverse Perspectives in Design: A significant challenge in AI development is the absence of diverse voices at the design table. Without input from BIPOC communities, AI systems fail to account for the unique experiences and needs of these groups. This leads to systems that do not fully serve or represent the communities they impact.

…My starting point has been to reimagine even how I think about [young people’s] work. A big shift…has been to work with young folks as domain or community experts, as opposed to research participants. That redefining of the role hopefully puts power back into young people and communities, and honors their lived experience as expertise and a major contributor to how we can develop algorithmic systems…

Desmond shared that his own journey in AI research didn’t start with these realizations. He actively sought out spaces where others challenged his assumptions and pushed him to confront the biases in his professional journey. For AI to evolve in a way that benefits everyone, it must center BIPOC perspectives and needs.

Mitigating Bias in AI Development

Despite these challenges, there are efforts to mitigate bias and improve the impact of AI on marginalized communities. One example is a project in collaboration with Columbia University, the Contextual Analysis of Social Media (CASM), focused on developing AI systems to identify and predict emotional states, including different stages of grief, to better understand and predict violent behavior online.

Initially, this approach failed to capture the full spectrum of emotional expression online. By questioning what "accuracy" truly means, particularly in the context of African American end users, Desmond and his colleagues worked to design models that are more empathetic and accurate. They found that AI development requires an interdisciplinary approach, where teams slow down the process to ask the right questions and, most importantly, engage in honest dialogue about what AI can and can’t do. Creating trust in marginalized communities is essential. Desmond emphasized that transparency and empathy in AI are vital for fostering trust and ensuring that these technologies serve everyone fairly.

Accountability in Data Practices

One of the most pressing questions in AI development is how to give marginalized communities power over their data. Desmond discussed the importance of accountability measures and ethical practices when collecting and analyzing data, particularly when it comes to vulnerable groups.

His project CASM is a prime example of how data analysis can be more ethically conducted. CASM looks at social media through a culturally informed lens, considering the emotional and cultural context behind online behaviors. For instance, it encourages researchers not to automatically interpret anger as aggression but to also consider it as a form of grief or emotional distress, a perspective informed by young people of color on the project. Patton's work with CASM is also a reminder of the ethical risks inherent in data scraping. It can be tempting to extract large amounts of data for analysis, but doing so without proper consent, particularly from young people, can cause significant harm. His research colleagues embraced a grassroots, on-the-ground approach, engaging directly with the communities they study to gain consent in real time and receive feedback.

Youth as Experts in AI Development

Young people of color, according to Desmond, play a vital role in shaping the future of AI. Rather than viewing them as research participants, he suggests that we reframe them as community experts. This shift in perspective not only empowers young people but also helps center their lived experiences as critical knowledge in AI development.

…[Young people of color] have shown us social media isn’t just a communication tool, it’s a neighborhood where community and identity and resilience are built.

Social media, often seen as a mere communication tool, is more than that for many young people of color; it’s a space for community-building, identity development, and resilience. By involving young people in the development process, we can shift the narrative around how Black and brown communities are represented and engaged within digital spaces.

The Role of Families and Parents in Digital Spaces

The role of families, especially parents, is another essential component in shaping a healthy digital experience for young people. Desmond noted that parents are often an untapped resource when it comes to supporting their children's digital lives. Many parents are overwhelmed by mixed messaging about the dangers of technology, but they remain critical educators in helping their children navigate digital spaces.

Patton advises parents to have open conversations with their children about their online experiences, asking them what they enjoy, what they see, and how they can make informed decisions about their digital lives. Pediatricians should also be consulted to understand the mental and physical impacts of social media on children and adolescents. Parents must assess their child's readiness and maturity to engage with social media.

Integrating Social Science into Policy

Desmond’s approach to working with tech companies is grounded in reflexivity and a commitment to transformational justice. He encourages tech companies to be more thoughtful and self-reflective when developing products and policies. In particular, he pointed to content moderation policies that often disproportionately silence Black and brown users. This reflects a broader issue in tech: ethical considerations need to be integrated at the beginning of the development process, rather than as a reactive response to harm.

AI as a Tool for Joy and Connection

A key theme of the discussion was that AI has the potential to be a force for joy and connection if we ask better questions and listen to the communities it impacts. Patton’s upcoming project, JoyNet, seeks to harness machine learning to create healthier digital spaces where young people can thrive. The idea behind JoyNet is to amplify moments of joy and resilience, shifting the narrative away from harm and focusing on hope. It’s about embedding joy as resistance into technology, allowing people to have thriving, positive experiences online.

Social media can serve as a third space for young people, a social realm separate from home and work, where they build connections that can strengthen their offline lives. This integration of online and offline experiences is essential for fostering resilience and community-building in digital spaces.

Policymakers: What Needs to Change?

In terms of policy, there is an urgent need for funding and opportunities to research social media's impact on young people. Patton calls for young people to be at the table when policies are crafted, ensuring their perspectives are considered in the development of digital literacy tools and platform initiatives.

Additionally, policy development should adopt a multistakeholder approach, bringing together tech companies, researchers, young people, and policymakers to create solutions that prioritize healthy, thriving online communities.

Desmond’s Vision for the Future

Looking ahead, Desmond is deeply committed to integrating joy into AI practices. He envisions a future where AI is not just a tool for solving problems but also a means of creating spaces of resilience and hope. This vision, he believes, requires more thinkers and funders to work alongside stakeholders committed to building healthy, thriving communities in digital spaces.

In conclusion, the AI and Marginalized Communities livestream underscored the importance of centering BIPOC voices in AI development, from data practices to product design. By addressing biases and including marginalized communities as collaborators and experts, we can ensure that AI is a tool for empowerment, connection, and joy for all.

Join us Thursday, January 16, 2025 for our next livestream, the first of 2025, and stay engaged with us on Slack, LinkedIn, and via our newsletter using our Linktree.