Recall: GASSS Framework

The GASSS framework helps structure product evaluations by considering:

• Goal – What are we trying to improve?
• Assumptions – What do we believe to be true about the problem or users?
• Structure – What are the key areas to evaluate success?
• Solution – What measurements or experiments will validate the decision?
• Synthesis – What’s the final judgment and what else should be done?

Step 1: Goal

The primary goal is to make Facebook Live and video content more accessible and inclusive for users with hearing disabilities. This supports Facebook’s broader mission to foster connection and provide equitable experiences for all users.

Secondary goals include:

• Improving satisfaction among the hearing-impaired community
• Meeting regulatory and accessibility standards (e.g. ADA, WCAG)
• Increasing engagement with video by making it more consumable across demographics

Step 2: Assumptions

Key assumptions include:

• A significant portion of users face hearing-related challenges when consuming video content
• Adding captions, transcripts, or visual cues would make content more accessible
• Accessibility features could improve engagement metrics among affected users
• Caption accuracy and customization options influence satisfaction and retention

These assumptions can be validated through usage data, user studies, or advocacy group feedback.

Step 3: Structure

To evaluate the initiative, consider several dimensions:

• Adoption and usage: Are accessibility features (e.g. captions, transcripts) being turned on and used?
• Engagement impact: Are users with accessibility features more engaged—watching longer, interacting more?
• Feature customization: Are users personalizing settings like font size or visual alerts?
• Quality and satisfaction: Are captions accurate? Are users satisfied with accessibility improvements?
• Compliance and standards: Does this help Facebook meet external accessibility guidelines and benchmarks?

Step 4: Solution

Metrics to track include:

• % of Live and recorded videos with captions or subtitles enabled
• Caption error rate or word accuracy
• Watch time, clicks, shares, or comments segmented by accessibility feature usage
• % of users who customize subtitle settings or enable visual/audio alerts
• Number of accessibility-related bug reports or complaints
• Survey scores or direct feedback from hearing-impaired users

To validate impact:

• Run A/B tests comparing different caption configurations
• Conduct interviews or surveys with users who rely on accessibility tools
• Partner with advocacy groups for usability testing and input

Step 5: Synthesis

Improving accessibility for Facebook Live is aligned with user trust, inclusivity, and regulatory compliance. But success depends not just on feature availability, but also on quality and usability.

Next steps:

• Ensure caption accuracy through regular audits and testing
• Implement reporting tools for inaccessible or low-quality content
• Continue collaborating with users and advocacy organizations to refine the experience

If done well, this investment can meaningfully improve the experience for a large underserved group, while enhancing Facebook’s reputation for accessibility and inclusion.