Co-design Inclusive and Empathetic Videoconferencing Experience with People Who Stutter
March 13, 2024Tech4ALL Digest, March 26
March 26, 2024
AImpower.org was founded on a fundamental belief that marginalized communities – underserved and underrepresented by technologies – have the authority and the rights to take charge of their own technological experiences. As a tech nonprofit, our role is to amplify their authority and rights, co-creating fair and equitable technologies for, with, and by marginalized communities.
As we entered our second year of operation, 2023 is a year of growth and expansion. Our team grew exponentially, now with 9 committed volunteer staff members leading various programs and operations. With the support of our talented volunteers, we conducted and published even more community and technical research on the intersection of stuttering and technology; and embarked the design and development of a few tangible technical products in this space. We greatly expanded our outreach and advocacy for fair and just technologies in online and offline spaces. Driven by our belief and commitment to build capacities, we also spent a great amount of time and resources mentoring and supporting students and communities organizations in designing and developing technical solutions with their own hands.
Let’s dive into our achievements and progress in 2023 in the aforementioned areas.
1. Team
Our team has grown 8x in size! We now have a full startup team of PM (Ran Wei), research (Jingjin Li, Qisheng Li), engineering (Kexin Feng, Ben Lickly, Shaomei Wu), design (Shawna Zhao), and Marketing (Rongcheng Zhang). We are also lucky to have people with a diverse set of backgrounds and experiences – ranging from seasoned tech veterans, newly minted PhDs, to engineering students, providing us a perfect blend of expertise and enthusiasm.
While everyone works for AImpower.org as volunteers with varying capacities, I haven’t seen a team that is so committed and self-motivated in my entire career, and have been constantly impressed by everyone’s ownership and drive over the past few months. I am confident that our team will achieve even greater impact in 2024, and would prioritize our fundraising to compensate key staff members, retaining the talents and the momentum.
2. Research
Following the research theme in 2022, we significantly expanded our community-centered research to uncover existing biases in videoconferencing and speech AI technologies, resulting in multiple publications submitted to and accepted by top-tier conferences in 2024.
- In the upcoming CHI 2024 paper “Finding My Voice over Zoom: An Autoethnography of Videoconferencing Experience for a Person Who Stutters” by Shaomei Wu, Jingjin Li, and Gilly Leshed (Cornell University), Shaomei reported her own videoconferencing experiences as a a person who stutters, advocating for videoconferencing technologies to cater to the “soft” side of meetings, such as empathy, authenticity, and belonging.
- In our CSCW 2024 submission “’I Want to Publicize My Stutter’: Community-led Collection and Curation of Chinese Stuttered Speech Data”, Qisheng Li and Shaomei Wu documented the process undertaken by StammerTalk – our partnering community organization of Chinese speaking people who stutter, to autonomously collect and curate stuttered speech data for more inclusive speech AI services. We found that, community-led data collection not only produced data critically needed to fairly represent marginalized communities in AI systems, but also empower and community and its members by embracing their marginalized identities and building long lasting social bonds.
- In our upcoming CHI 2024 paper “Towards Fair and Inclusive Speech Recognition for Stuttering: Community-led Chinese Stuttered Speech Dataset Creation and Benchmarking”, Qisheng Li and Shaomei Wu performed a technical evaluation of the stuttered speech data collected by StammerTalk, highlighting the diversity and variability of stutters captured, and its unique value in authentically representing the stuttering community in AI data. We also leveraged this dataset to conduct initial benchmarking of popular ASR models from OpenAI and Meta, identifying potential issues in model architecture that contribute the performance disparity for stuttered speech.
- In our DIS 2024 submission “Co-designing Inclusive and Empowering Videoconferencing Technologies with People Who Stutter” by Jingjin Li, Shaomei Wu, and Gilly Leshed (Cornell University), we presented our co-design process with the stuttering community to explore and reflect the design space for inclusive videoconferencing. Our co-design studies produced a large set of tools that support meeting participants before, during, and after VC, showing both universal value to all meeting participants and the desire to redistribute the power between stuttering and non-stuttering speakers in meetings.
3. Products
As our community-driven research uncovered biases in videoconferencing and AI speech technologies, we using this knowledge to reshape the design of videoconferencing and speech AI products that ensure the inclusion and respect of all voices.
Working closely with the stuttering community in 2023, we have:
- Co-designed and prototyped a videoconferencing companion App that enhances the socio-emotional experiences of virtual meetings and builds a more supportive meeting environment for all participants.
- Supported StammerTalk in their collection and curation of the first and largest Chinese stuttered speech dataset, with over 50 hours of conversational and command reading speech from 72 speakers who stutter.
In 2024, we will expand our current technical work to land even bigger, more tangible impact to videoconferencing and speech AI technologies:
- We will further develop and polish our videoconferencing companion App and launch it publicly in the first half of 2024 – stay tuned for our public release!
- We will work with StammerTalk to release the collected stuttered dataset, also build a platform and public dashboard for other researchers to evaluate their ASR models with this dataset.
- Leveraging the collected stuttered speech dataset, we will work with StammerTalk to evaluate, benchmark, and improve popular ASR models for disfluent speech. We will share our benchmarking results publicly to raise public awareness on this issue.
4. Outreach & Advocacy
Despite our background as technologists, we at AImpower.org understand the inherent limitations of technology, and firmly believe in the power of collective actions. We have always been committed to engage and drive public discussion and community advocacy. Ultimately, we strive to foster solidarity and raise public awareness on the unseen barriers created by technology, as a foundation for lasting, structural changes in the tech ecosystem.
With the support of community and student volunteers, we have multiplied our outreach and advocacy efforts in 2023, extending our presence in both online and offline spaces to engage with larger, more diverse audiences, including students, academic researchers, tech developers, stuttering community members, and the broader public.
- Community outreach: we are closely embedded within the stuttering community, speaking and organizing in different conferences, workshops, and community events for and with people who stutter across the globe. Here are just a few highlighted events:
- Claim Your Virtual Presence workshop at National Stuttering Associate Annual Conference on July 6, 2023. With Gary Goldsmith (PWS) from Unthinkable, we co-organized and co-moderated the first workshop for communication over videoconferences, in which the participants and us co-created a set of strategies and recommendations for inclusive videconconferencing for people who stutter. In our post-workshop surveys, 84% of the participants found the workshop “very” or “extremely” informative, and 100% of the participants found themselves “very” or “extremely” likely to applied the strategies and recommendations from the workshop in their future video calls!
- We joined the Stammering Pride against Prejudice event in London 🇬🇧 on September 15, 2023. It was a great experience celebrating Stammer Pride with the stuttering community in the UK. Shaomei spoke about AImpower.org’s work on combatting technological prejudice against stuttering and hanged out with several of our friends, supporters, and research participants for the first time in person.
- Shaomei Wu – AImpower’s founder and CEO – gave the keynote speech at StammerTalk (口吃说) International Stuttering Awareness Day virtual conference on Oct 22, 2023. As the largest community event for Chinese speaking people who stutter, StammerTalk’s ISAD event gathered nearly one hundred participants from across the world!
- Educational outreach: we believe in the talent and drive of the next generation of tech workforce and are always excited to share our knowledge and mission with students and educators. A few examples of our educational outreach from 2023:
- AI and the Systematic Marginalization of People with Disabilities. Guest Lecture by Shaomei at University of Ottawa CSI5195/ELG5295/IAI5130 Ethics for Artificial Intelligence (winter 2023). Mar 8, 2023. [Video, Slides]
- Designing Empowering Technologies for and with Marginalized Communities. Shaomei presented at Computational Media Department Seminar at University of California, Santa Cruz. Mar 20, 2023. [Slides]
- Advocacy with academia & tech industry: to create the movement within the tech community, we are always seeking out opportunities to advocate the needs and demands of marginalized user groups. We presented challenges with videoconferencing and speech AI in prominent HCI conferences and academia-industry convergence events last year, including:
- “The World is Designed for Fluent People”:Benefits and Challenges of Videoconferencing Technologies for People Who Stutter. Presentation at CHI ’23. Apr 26, 2023. [Video, Slides]
- Sociotechnical Challenges in Voice-Activated AI. Panelist at Michigan State University, Voice-Activated AI for Stuttered Speech Convergence Symposium. Apr 14, 2023.
- Public outreach. Technology does not exist in vacuums and we need to public awareness and support to generate fundamental, socio-structural changes for marginalized groups. We have greatly scaled our public outreach effort in 2023, raising awareness about the (often invisible) challenges faced by women, people with disabilities, and the stuttering community in various offline and online channels.
- Stuttering community can create their own technological future. This is a part of online exhibition Multiverse: Disability, Technology, and Co-created Futures curated by Tsinghua University Science Museum & MIT Museum, co-created by AImpower and the StammerTalk community.
- 人工智能数据标注行业扫描~(AI Data Annotation Industry) (in Chinese). Shaomei was invited to speak at 奇途无障碍 – 数字经济与残障女性就业分享第05期~(Digital Economy and Employment Opportunities for Women with Disabilities Forum Vol. 05). May 6, 2023. [Text summary]
- Break the Invisible Wall: Challenges and Opportunities for People Who Stutter to Participate in In-person and Virtual Conferences. Shaomei spoke to academic conference organizers about the participation challenges for people who stutter at the Organizer Seminar Series hosted by Virtual Chair. Jan 24, 2023. [Video, slides]
- AI and Accessibility (in Chinese). Shaomei was featured at podcast by 扩博智聊 (Clobotics), in which she talked about intersectionality, accessibility, and her vision for AImpower.
- Website: we have been diligently sharing our progress and thoughts on our website, creating 12 blogposts in 2023. Our posts attracted nearly 3000 visitors across the world, and were viewed almost 7000 times in a year! We have been receiving comments and contact constantly through our website, with people showing interests in our work and desire to collaborate!
- Social media: We have created our organizational accounts on LinkedIn and Facebook, and our content gathered thousands of organic impressions, hundreds of clicks and likes in the social media channels.
- Newsletter: We also started a newsletter from AImpower.org since August 2023, in which we share our progress, research findings, and advocacy work. Our newsletter organically grew to 2x subscribers over a few months, and were viewed almost 1000 times by our subscribers in 2023! Inspired by our success with the newsletter, we are re-structuring it in 2024 as the “Tech4ALL Digest by AImpower”, and will be sent out at a bi-weekly frequency!
5. Technical Advisory and Mentorship
As we research and develop technologies with and by the communities, we strive to share not only the final products but also the method, knowledge, and expertise with communities we encounter and partner with.
Back In 2022, Shaomei volunteered at MySpeech – a community of people who stutter – as their inaugural CTO to support their technological initiative and explorations. We expanded our advisory roles to several other community and educational organizations in 2023, including:
- Technical advisory for StammerTalk to support their speech data and speech AI initiatives.
- Industry advisory board at University of California, Santa Cruz, Computational Media Department, to support the curriculum and student career development, especially for underrepresented populations in STEM.
We also offered direct technical and product mentorship to more than 20 students from multiple universities in the US and Canada, either through Master Project Programs or volunteer internships. For example:
- We mentored a group of master students from the University of Ottawa to design and develop a prototype Zoom app that empowers marginalized voices in virtual meetings. See the students presentation video here.
- We hosted a Master of Professional Study Project with a group of HCI Master program students from Cornell University, guiding the students to engage and learn from the stuttering community and turning their insights into tangible design prototypes for videoconferencing add-ons.
Check out the list of our talented students and interns here!
We are proud of all we have achieved in 2023, from conducting cutting-edge research to co-designing real world product interventions. Our capabilities and outputs have multiplied with an extreme lean budget.
We would like to thank all our funders, partners, community supporters, and volunteers for their ongoing support of our work in research, technology, community engagement, advocacy and global outreach. Working together, we will continue to have a lasting and positive impact on the technological experiences of marginalized communities – something we believe is more important than ever before.
