publications

Conference Papers

2024

1. COLM

   Long-Form Answers to Visual Question from Blind and Low Vision People

   Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, and Amy Pavel

   Conference on Language Modeling, 2024

   💡 Oral Spotlight (Top 2%)

   Abs PDF Website

   Vision language models can now generate long-form answers to questions about images – long-form visual question answers (LFVQA). We contribute VizWiz-LF, a dataset of long-form answers to visual questions posed by blind and low vision (BLV) users. VizWiz-LF contains 4.2k long-form answers to visual questions, collected from human expert describers and six VQA models. We develop and annotate functional roles of sentences in LFVQA and demonstrate that long-form answers contain information beyond the answer such as explanations and suggestions. We further evaluate 360 long-form answers with BLV and sighted people to understand their preferences and assess the correctness of the long-form answers. Long-form answers generated by models often hallucinate with incorrect information to unanswerable questions (e.g., blurry images), yet BLV people often perceive these long-form answers as correct. To reduce hallucinations, we evaluate VQA models on their ability to correctly abstain from answering unanswerable questions.

2. UIST

   DesignChecker: Visual Design Support for BLV Web Developers

   Mina Huh, and Amy Pavel

   Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology 2024, 2024

   Abs PDF Website

   Blind and low vision (BLV) developers create websites to share knowledge and showcase their work. A well-designed website can engage the audience and deliver information effectively, yet it remains challenging for BLV developers to review their web designs. We conducted interviews with BLV developers (N=12) and analyzed 20 websites created by BLV developers. BLV developers created highly accessible websites but wanted to assess the usability of their website for sighted users and follow the design standards of other websites. They also encountered challenges using screen readers to identify illegible text, misaligned elements, and inharmonious colors. We present DesignChecker, a browser extension that helps BLV developers to improve their web design. With DesignChecker, users can assess their current design by comparing it to visual design guidelines, a reference website of their choice, or a trend of similar websites. DesignChecker also identifies the specific HTML elements that violate web design guidelines and suggests CSS changes for improvements. Our user study participants (N=8) all expressed enthusiasm about using DesignChecker in the future to improve their web design.

3. CHI

   ShortScribe: Making Short-Form Videos Accessible with Hierarchical Video Summaries

   Tess Van Daele, Akhil Iyer, Yuning Zhang, Jayln Derry, Mina Huh, and Amy Pavel

   Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, 2024

   Abs PDF Website

   Short videos on platforms such as TikTok, Instagram Reels, and YouTube Shorts (i.e. short-form videos) have become a primary source of information and entertainment. Many short-form videos are inaccessible to blind and low vision (BLV) viewers due to their rapid visual changes, on-screen text, and music or meme-audio overlays. In our formative study, 7 BLV viewers who regularly watched short-form videos reported frequently skipping such inaccessible content. We present ShortScribe, a system that provides hierarchical visual summaries of short-form videos at three levels of detail to support BLV viewers in selecting and understanding short-form videos. ShortScribe allows BLV users to navigate between video descriptions based on their level of interest. To evaluate ShortScribe, we assessed description accuracy and conducted a user study with 10 BLV participants comparing ShortScribe to a baseline interface. When using ShortScribe, participants reported higher comprehension and provided more accurate summaries of video content.

2023

1. UIST

   GenAssist: Making Image Generation Accessible

   Mina Huh, Yi-Hao Peng, and Amy Pavel

   Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology 2023, 2023

   🏆 Best Paper Award (Top 1%)

   Abs PDF Website

   Blind and low vision (BLV) creators use images to communicate with sighted audiences. However, creating or retrieving images is challenging for BLV creators as it is difficult to use authoring tools or assess image search results. Thus, creators limit the types of images they create or recruit sighted collaborators. While text-to-image generation models let creators generate high-fidelity images based on a text description (i.e. prompt), it is difficult to assess the content and quality of generated images. We present GenAssist, a system to make text-to-image generation accessible. Using our interface, creators can verify whether generated image candidates followed the prompt, access additional details in the image not specified in the prompt, and skim a summary of similarities and differences between image candidates. To power the interface, GenAssist uses a large language model to generate visual questions, vision-language models to extract answers, and a large language model to summarize the results. Our study with 12 BLV creators demonstrated that GenAssist enables and simplifies the process of image selection and generation, making visual authoring more accessible to all.

2. CHI

   AVscript: Accessible Video Editing with Audio-Visual Scripts

   Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang ’Anthony’ Chen, Young-Ho Kim, and Amy Pavel

   Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023

   Abs PDF Website

   Sighted and blind and low vision (BLV) creators alike use videos to communicate with broad audiences. Yet, video editing remains inaccessible to BLV creators. Our formative study revealed that current video editing tools make it difficult to access the visual content, assess the visual quality, and efficiently navigate the timeline. We present AVscript, an accessible text-based video editor. AVscript enables BLV creators to edit their video using a script that embeds the video’s visual content, visual errors (e.g., dark or blurred footage), and speech. BLV creators can use AVscript to efficiently navigate between scenes and visual errors or to locate objects in the frame or spoken words of interest. A comparison study (N=12) showed that AVscript significantly lowered BLV creators’ mental demands while increasing confidence and independence in video editing. We further demonstrate the potential of AVscript through an exploratory study (N=3) where BLV creators edited their own footage.

2022

1. CHI

   Cocomix: Utilizing Comments to Improve Non-Visual Webtoon Accessibility

   Mina Huh, Yunjung Lee, Dasom Choi, Haesoo Kim, Uran Oh, and Juho Kim

   Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022

   Abs HTML PDF

   Webtoon is a type of digital comics read online where readers can leave comments to share their thoughts on the story. While it has experienced a surge in popularity internationally, people with visual impairments cannot enjoy webtoon with the lack of an accessible format. While traditional image description practices can be adopted, resulting descriptions cannot preserve webtoons’ unique values such as control over the reading pace and social engagement through comments. To improve the webtoon reading experience for BLV users, we propose Cocomix, an interactive webtoon-reader that leverages comments into the design of novel webtoon interactions. Since comments can identify story highlights and provide additional context, we designed a system that provides 1) comments-based adaptive description with selective access to details and 2) panel-anchored comments for easy access to relevant descriptive comments. Our evaluation (N=12) showed that Cocomix users could adapt the description for various needs and better utilize comments.

2. CHI

   “It Feels Like Taking a Gamble”: Exploring Perceptions, Practices, and Challenges of Using Makeup and Cosmetics for People with Visual Impairments

   Mingzhe Li*, Francheska Spektor*, Meng Xia*, Mina Huh*, Peter Cederberg, Yuqi Gong, Kristen Shinohara, and Patrick Carrington

   Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022

   Abs PDF

   Makeup and cosmetics offer the potential for self-reinvention and reshaping of social roles for visually impaired people. However, there exist barriers to conducting a beauty regime because of the reliance on visual information and color variances in makeup. We present a content analysis of 145 YouTube videos to demonstrate visually impaired individuals’ unique practices before, during, and after doing makeup. We then conducted semi-structured interviews with 12 visually impaired people to uncover their perceptions of and challenges with the makeup process. Overall, our findings on unique practices of how visually impaired people do makeup (e.g., how to differentiate makeup products) illuminate challenges in the learning process, task completion, and acquisition of feedback. The existing challenges provide opportunities for future research to address learning barriers, the lack of sufficient feedback, and inaccessible product details, making the experience of doing makeup more accessible to people with visual impairments.

3. CSCL

   A Duoethnographic Study of a Mixed-Ability Team In a Collaborative Group Programming Project

   Mina Huh, and Jooyoung Seo

   Proceedings of the 15th International Conference on Computer-Supported Collaborative Learning-CSCL, 2022

   Abs PDF

   As diversity efforts in computer science education strive to make programming more accessible, we take a step forward by exploring collaborative context. We present a duoethnography of a mixed-ability team collaborating in a group programming project. We (one sighted; one blind) immersed ourselves into a collaborative programming project using a popular social coding platform (i.e., GitHub) and triangulated our self-reflexivity through cross- checking reviews and interviews. By analyzing interaction logs on GitHub, code writing in integrated development environments, and interview notes, we show distinct approaches adopted by the sighted programmer and the programmer with visual impairments, tools and technologies used, and social implications of visual impairment. This research offers reflections on accommodations and technologies provided to support mixed-ability teams collaborating in a group programming project.

2021

1. CHI

   RubySlippers: Supporting Content-based Voice Navigation for How-to Videos

   Minsuk Chang, Mina Huh, and Juho Kim

   Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

   Abs HTML PDF

   Directly manipulating the timeline, such as scrubbing for thumbnails, is the standard way of controlling how-to videos. However, when how-to videos involve physical activities, people inconveniently alternate between controlling the video and performing the tasks. Adopting a voice user interface allows people to control the video with voice while performing the tasks with hands. However, naively translating timeline manipulation into voice user interfaces (VUI) results in temporal referencing (e.g. “rewind 20 seconds”), which requires a different mental model for navigation and thereby limiting users’ ability to peek into the content. We present RubySlippers, a system that supports efficient content-based voice navigation through keyword-based queries. Our computational pipeline automatically detects referenceable elements in the video, and finds the video segmentation that minimizes the number of needed navigational commands. Our evaluation (N=12) shows that participants could perform three representative navigation tasks with fewer commands and less frustration using RubySlippers than the conventional voice-enabled video interface.

Others

2021

1. report

   Reimagining Accessibility and Inclusion in K-12 CS Education through Curriculum and Professional Development

   Microsoft Accessible Computer Science Education Fall Workshop, 2021

   PDF