AI Assessments, Audits, and Public Oversight Methods: An All Tech Is Human and IBM Collaborative Workshop
All Tech Is Human hosted a curated workshop focused on AI Assessments, Audits, and Public Oversight Methods in collaboration with IBM Research on Friday, September 20th at Betaworks in New York City.
We convened 75 Responsible Tech professionals doing the crucial work of operationalizing AI Governance within organizations, alongside researchers, policy advisors, and civil society leaders to talk about how things are actually unfolding in practice throughout each stage of the AI product development lifecycle – pre-development, post-development/pre-deployment, and post-deployment – to uncover emerging best practices and to discover the pain points that are most common across our collective varied experiences, and to identify ways to engage communities in public accountability mechanisms that can shape the development of AI to be more aligned with the public interest.
The gathering began with a presentation about building an AI evaluation system by Juana Catalina Becerra Sandoval and a panel discussion about AI assessments, audits, and public oversight methods. You can discover videos of the presentations and join the conversation in our global Slack community.
Presentation: Building an AI Evaluation Ecosystem
Juana Catalina Becerra Sandoval (Responsible Tech Research, IBM) discusses building an AI evaluation ecosystem in a keynote presentation. In Building an AI Evaluation Ecosystem Becerra Sandoval discusses…
The current challenges, evolution, and future of AI governance.
The importance of AI governance and audits shape in growing a Responsible AI future.
The ways AI ethics boards and participatory methods empower the field.
AI Governance Panel: AI Assessments, Audits, and Public Oversight Methods Workshop
Dr. Rumman Chowdhury (CEO & Co-Founder, Humane Intelligence), Gemma Galdon Clavell (CEO, Eticas AI), and Noah Broestl (Partner & Associate Director of Responsible AI, Boston Consulting Group) joined moderator Renée Cummings for an AI governance panel on Friday, September 20, at All Tech Is Human and IBM Research's workshop in New York City. The panel discuss…
The complexities of integrating Responsible AI practices.
How to adapt robust AI regulatory frameworks.
Why multistakeholder participation is essential to address AI bias and sustainability.
The ways global disparities in AI governance present themselves.
How we can rethink traditional test and evaluation methods to bolster public confidence in AI.
During breakout sessions, participants engaged in a deep dive into their experiences doing this work, comparing perspectives to identify what is working well and what common pain points are emerging.
Participants were directed to choose their primary area of expertise – Planning & Design (involving Data Decisions like sourcing, cleaning) → Model Development → Testing, Evaluation, Validation, and Verification → Red-Teaming & Public Oversight → and Audit and Assessment – for the breakout group discussion, describing their experiences, evaluating the challenges and constraints, and laying the groundwork to explore potential solutions.
Following this workshop, we are releasing a summary report to share with the larger responsible tech community what we have identified as common pain points and hurdles, with initial recommendations as we plan a deeper engagement into a select subset of these issues throughout the coming year.
Look for our forthcoming report, and stay tuned for how to get involved — we are looking forward to exploring ways that we can collaborate to solve the problems that we are collectively uncovering. Email Executive Director Rebekah Tweed if you missed this workshop and would like to learn more about collaborating in future workshops!
