Amy Pavel

I am currently a postdoctoral fellow at Carnegie Mellon University (supervised by Jeff Bigham) and a Research Scientist at Apple. I recently graduated with my PhD from the department of Electrical Engineering and Computer Science at UC Berkeley. I was advised by professors Björn Hartmann at UC Berkeley and Maneesh Agrawala at Stanford, and my doctoral work was supported by an NDSEG fellowship. I've previously been supported by a department EECS Excellence Award and a Sandisk Gold Fellowship.

In the summer of 2018, I served as the lecturer for CS160, the UC Berkeley user interface design and development course. Here is the course website.

I'm on the job market this year! Materials: CV · Research · Teaching · Diversity


Research Highlights

Rescribe

UIST 2020

On one side, a set of extended audio description sentences with their corresponding frames, and in the other column a set of inline audio description sentences and the same frames. The shortened descriptions are as follows: `Shots of lavender in a farmers market' is shortened to `Shots of lavender', `Red flowers against a white house and blue sky' is shortened to `Red flowers', `a courtyard and a pool' is not shortened, `Gaby bikes along a path' is shortened to `Close up of french fries', and `Close up of tater tots and french fries' is shortened to `Close up of french fries'

PDF | Video | Talk

360° Video

UIST 2017

A thumbnail of a paper figure indicating three different editing techniques for 360 video: traditional, viewpoint-oriented cuts and active reorientation.

PDF | Code | Video

SceneSkim

UIST 2015

PDF | Video

Video Digests

UIST 2014

A thumbnail of the Video Digests interface. Video thumbnails displayed alongside short summaries of the video content.

PDF | Demo | Code

Research Summary

My research leverages the connections between communication mediums (e.g., spoken audio, multiple video sources, text transcripts) to enable people to fully participate in and better work with digital communication. As a systems researcher in Human-Computer Interaction, I embed machine learning technologies (e.g., NLU, NLG) into new human interactions that I then deploy to test. Using my systems, remote content creators more effectively collaborate, video authors efficiently create accessible descriptions for blind users, and instructors help students to learn and retain key points. To inform future systems that capture what is important to domain experts and people with disabilities, I conduct and collaborate on in-depth qualitative (e.g., AAC communication, memes) and quantitative studies (e.g., 360° Video, VR Saliency). My long term research goal is to make communication more effective and accessible to all.

Conference Papers

A slide from a lecture where most text is in black but some of the text has been colored green to represent that it has been spoken by the presenter. There is an image with a squiggly circle brush on the slide, and it has a green border to indicate the presenter described the image.
Say It All: Feedback for Improving Non-Visual Presentation Accessibility
Yi-Hao Peng, JiWoong Jang, Jeffrey P. Bigham, Amy Pavel
The ACM Conference on Computer Human Interaction (Accepted to CHI 2021).

A YouTube search result augmented with accessibility information. A thumbnail of the video shows a woman in Rome. The rest of the video information reads as follows: Rome, Italy! Tips & Photo Spots by Ruby Keyvani. 5/7 Somewhat Accessible. 76% of the video is speech. The speech is descriptive but contains many visual references (3 per minute). Visual changes occur infrequently (5 shots per minute); few on-screen objects are described (20%).
What Makes a Video Accessible to Blind and Visually Impaired People?
Xingyu Liu, Patrick Carrington, Xiang "Anthony" Chen, Amy Pavel
The ACM Conference on Computer Human Interaction (Accepted to CHI 2021).


A photograph taken at a workshop with a person using an AAC device, her close conversational partner, and puppeteers. All participants stand around a table full of craft supplies and are engaged in coversation.
Co-designing Socially Assistive Sidekicks for Motion-based AAC
Stephanie Valencia, Michal Luria, Amy Pavel, Jeffrey P. Bigham, Henny Admoni
The ACM Conference on Human Robot Interaction (Accepted to HRI 2021).

On one side, a set of extended audio description sentences with their corresponding frames, and in the other column a set of inline audio description sentences and the same frames. The shortened descriptions are as follows: `Shots of lavender in a farmers market' is shortened to `Shots of lavender', `Red flowers against a white house and blue sky' is shortened to `Red flowers', `a courtyard and a pool' is not shortened, `Gaby bikes along a path' is shortened to `Close up of french fries', and `Close up of tater tots and french fries' is shortened to `Close up of french fries'
Rescribe: Authoring and Automatically Editing Audio Descriptions
Amy Pavel, Gabriel Reyes, Jeffrey P. Bigham
ACM Symposium on User Interface Software and Technology (UIST 2020).

PDF | Video | Talk



Two example prototypes for making AR apps accessible. A: Foundational Accessibility. Screenshot of a virtual chair with a voice over target around it, a speech bubble shows the app announcing 'Back of chair with blue cushion'. B: Scanning. Screenshot of AR grid overlaid on a coffee table. Speech bubbles show the app announcing 'Found a new horizontal surface' and 'Scanned 2 surfaces totaling 2.3 square meters'.
Making Mobile Augmented Reality Applications Accessible
Jaylin Herskovitz, Jason Wu, Samuel White, Amy Pavel, Gabriel Reyes, Anhong Guo, Jeffrey P. Bigham
ACM Conference on Computers and Accessibility (ASSETS 2020).

PDF | Video


A cartoon stylized version of a popular reaction GIF of Oprah Winfrey shrugging. She turns to look to the camera, glances to the side, stares at the camera, then shrugs with her palms up.
Making GIFs Accessible
Cole Gleason, Amy Pavel, Himalini Gururaj, Kris M. Kitani, Jeffrey P. Bigham
ACM Conference on Computers and Accessibility (ASSETS 2020).

Paper | Link to Thumbnail GIF


A thumbnail with a conversation between a person using an AAC device and two other people.
Conversational Agency in Augmentative and Alternative Communication
Stephanie Valencia, Amy Pavel, Jared Santa Maria, Seunga (Gloria) Yu, Jeffrey P. Bigham, Henny Admoni
The ACM Conference on Computer Human Interaction (CHI 2020).

PDF



A thumbnail of a paper figure showing three memes with descriptions.
Making Memes Accessible
Cole Gleason, Amy Pavel, Xingyu Liu, Patrick Carrington, Lydia Chilton, Jeffrey P. Bigham
ACM Conference on Computers and Accessibility (ASSETS 2019).

PDF | Time article




A thumbnail of a paper figure indicating three different editing techniques for 360 video: traditional, viewpoint-oriented cuts and active reorientation.
Shot Orientation Controls for Interactive Cinematography with 360 Video
Amy Pavel, Björn Hartmann, Maneesh Agrawala
ACM 30th Symposium on User Interface Software and Technology (UIST 2017).

PDF | Code | Video


A thumbnail of the VidCrit interface with a video on the left and text critiques of the video on the right.
VidCrit: Video-Based Asynchronous Video Review
Amy Pavel, Dan B Goldman, Björn Hartmann, Maneesh Agrawala
ACM 29th Symposium on User Interface Software and Technology (UIST 2016).

PDF | Video



A thumbnail indicating an overview of the CrowdCrit critique process.
Structuring, Aggregating, and Evaluating Crowdsourced Design Critique
Kurt Luther, Jari-lee Tolentino, Wei Wu, Amy Pavel, Maneesh Agrawala, Brian P Bailey, Björn Hartmann, Steven Dow
The 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2015).

A thumbnail of the Video Digests interface. Video thumbnails displayed alongside short summaries of the video content.
Video Digests: A Browsable, Skimmable Format for Informational Lecture Videos
Amy Pavel, Colorado Reed, Björn Hartmann, Maneesh Agrawala
ACM 27th Symposium on User Interface Software and Technology (UIST 2014).

PDF | Demo | Code

Thesis and Technical Reports

Title page of dissertation
Navigating Video Using Structured Text
Amy Pavel
PhD in Computer Science, University of California, Berkeley
Advisors: Bjoern Hartmann and Maneesh Agrawala
Additional committee members: Eric Paulos, Abigail De Kosnik

Posters and workshops


A thumbnail with an illegible system diagram of Twitter A11y.
Twitter A11y: A Browser Extension to Make Twitter Images Accessible
Christina Low*, Emma McCamey*, Cole Gleason, Patrick Carrington, Jeffrey P. Bigham, Amy Pavel
The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019 Poster).

A thumbnail indicating an overview of the CrowdCrit critique process.
CrowdCrit: crowdsourcing and aggregating visual design critique
Kurt Luther, Wei Wu, Amy Pavel, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, Steven Dow
The 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2014 Demo).

Work

Postdoctoral Fellow — Carnegie Mellon University

Advisor: Jeffrey P. Bigham January 2019 - Present

Research Scientist — Apple

Group: Accessibility/ML Research
July 2019 - Present

Graduate student researcher — UC Berkeley

Advisors: Björn Hartmann, Maneesh Agrawala Committee: Eric Paulos, Björn Hartmann, Maneesh Agrawala, Abigail de Kosnik
August 2013 - December 2018

Undergraduate research assistant — UC Berkeley

Advisors: Maneesh Agrawala, Björn Hartmann, Floraine Berthouzoz
May 2011 - May 2013

Research intern — Adobe Research, Seattle

Advisor: Dan Goldman
May 2014 - August 2014, May 2015 - August 2015

Teaching

Instructor — UC Berkeley

CS 160: User interface design and development
Summer 2018

Graduate Student Instructor (TA) — UC Berkeley

CS 160: User interface design and development
Summer 2017

Project Advisor — UC Berkeley

NWMEDIA 190: Making Sense of Cultural Data
Fall 2017

Instructor (Workshop) — UC Berkeley

CS Kickstart, intro CS for incoming freshmen women
Summer 2012