Amy Pavel

apavel at cs.cmu.edu | twitter | scholar

I am currently a postdoctoral fellow at Carnegie Mellon University (supervised by Jeff Bigham) and a Research Scientist at Apple. I recently graduated with my PhD from the department of Electrical Engineering and Computer Science at UC Berkeley. I was advised by professors Björn Hartmann at UC Berkeley and Maneesh Agrawala at Stanford, and my doctoral work was supported by an NDSEG fellowship. I've previously been supported by a department EECS Excellence Award and a Sandisk Gold Fellowship.

In the summer of 2018, I served as the lecturer for CS160, the UC Berkeley user interface design and development course. Here is the course website.

Research

My dissertation research focuses on developing new algorithms and interfaces for searching, browsing and skimming videos. My projects include new text-based interfaces for exploring educational lecture videos, films, and video critiques. In addition, I've worked to understand how people view and interact with 360° videos. Though I've used video as a medium, I'm broadly interested in how to support users in benefiting from imperfect computational results in domains such as search, collaboration, and content generation.

As an undergraduate, I worked on a system for searching and browsing photoshop tutorials, and tools for helping crowdworkers provide useful design critiques.

Papers

On one side, a set of extended audio description sentences with their corresponding frames, and in the other column a set of inline audio description sentences and the same frames. The shortened descriptions are as follows: `Shots of lavender in a farmers market' is shortened to `Shots of lavender', `Red flowers against a white house and blue sky' is shortened to `Red flowers', `a courtyard and a pool' is not shortened, `Gaby bikes along a path' is shortened to `Close up of french fries', and `Close up of tater tots and french fries' is shortened to `Close up of french fries'

Rescribe: Authoring and Automatically Editing Audio Descriptions
Amy Pavel, Gabriel Reyes, Jeffrey P. Bigham
ACM Symposium on User Interface Software and Technology (UIST 2020).

PDF | 5 Minute Talk | 15 Minute Talk

Screenshot of a tweet by @CDCgov from April 1, 2020 3:55pm: Actions to reduce spread of the virus, such as social distancing, are key to #FlattenTheCurve. 2 of 3 (original tweet link: https://twitter.com/CDCgov/status/1245439600472084486) The tweet contains an image of the common public health infographic about “flattening the curve”, but the tweet did not include alt text for the image. The image shows an example of a common flatten the curve info-graphic. A tall peak indicates the height of the pandemic if left unchecked, and a shorter spread out curve depicts the effects of social distancing efforts.

Disability and the COVID-19 Pandemic: Using Twitter to Understand Accessibility during Rapid Societal Transition
Cole Gleason, Stephanie Valencia, Lynn Kirabo, Jason Wu, Anhong Guo, Elizabeth J. Carter, Jeffrey P. Bigham, Cynthia L. Bennett, Amy Pavel
ACM Conference on Computers and Accessibility (ASSETS 2020).

PDF (coming soon)

Two example prototypes for making AR apps accessible. A: Foundational Accessibility. Screenshot of a virtual chair with a voice over target around it, a speech bubble shows the app announcing 'Back of chair with blue cushion'. B: Scanning. Screenshot of AR grid overlaid on a coffee table. Speech bubbles show the app announcing 'Found a new horizontal surface' and 'Scanned 2 surfaces totaling 2.3 square meters'.

Making Mobile Augmented Reality Applications Accessible
Jaylin Herskovitz, Jason Wu, Samuel White, Amy Pavel, Gabriel Reyes, Anhong Guo, and Jeffrey P. Bigham
ACM Conference on Computers and Accessibility (ASSETS 2020).

PDF | Video (coming soon)

A thumbnail with a conversation between a person using an AAC device and two other people.

Conversational Agency in Augmentative and Alternative Communication
Stephanie Valencia, Amy Pavel, Jared Santa Maria, Seunga (Gloria) Yu, Jeffrey P. Bigham, Henny Admoni
The ACM Conference on Computer Human Interaction (CHI 2020).

PDF

A thumbnail with illegible examples of tweet images and their corresponding alt text.

Twitter A11y: A Browser Extension to Make Twitter Images Accessible
Cole Gleason, Amy Pavel, Emma McCamey, Christina Low, Patrick Carrington, Kris M. Kitani, Jeffrey P. Bigham
The ACM Conference on Computer Human Interaction (CHI 2020).

PDF

A thumbnail of a paper figure showing three memes with descriptions.

Making Memes Accessible
Cole Gleason, Amy Pavel, Xingyu Liu, Patrick Carrington, Lydia B. Chilton, and Jeffrey P. Bigham
ACM Conference on Computers and Accessibility (ASSETS 2019).

PDF | Time article

Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References
Prakhar Gupta, Shikib Mehri, Tiancheng Zhao, Amy Pavel, Maxine Eskenazi and Jeffrey P. Bigham
SIGDIAL Meeting on Discourse and Dialogue (SIGDIAL 2019).

PDF | Code

A thumbnail of a paper figure including nine images with heat maps indicating visual saliency in VR.

Saliency in VR: How do people explore virtual environments?
Vincent Sitzmann*, Ana Serrano*, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein
IEEE Transactions on Visualization and Computer Graphics (VR 2018).

Project Page | Code | Video

A thumbnail of a paper figure indicating three different editing techniques for 360 video: traditional, viewpoint-oriented cuts and active reorientation.

Shot Orientation Controls for Interactive Cinematography with 360 Video
Amy Pavel, Björn Hartmann, Maneesh Agrawala
ACM 30th Symposium on User Interface Software and Technology (UIST 2017).

PDF | Interactive Demo | Code | Video

A thumbnail of the VidCrit interface with a video on the left and text critiques of the video on the right.

VidCrit: Video-Based Asynchronous Video Review
Amy Pavel, Dan B Goldman, Björn Hartmann, Maneesh Agrawala
ACM 29th Symposium on User Interface Software and Technology (UIST 2016).

PDF | Video

SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries
Amy Pavel, Dan B Goldman, Björn Hartmann, Maneesh Agrawala
ACM 28th Symposium on User Interface Software and Technology (UIST 2015).

PDF | Video

A thumbnail indicating an overview of the CrowdCrit critique process.

Structuring, Aggregating, and Evaluating Crowdsourced Design Critique
Kurt Luther, Jari-lee Tolentino, Wei Wu, Amy Pavel, Maneesh Agrawala, Brian Bailey, Björn Hartmann, Steven Dow
The 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2015).

A thumbnail of the Video Digests interface. Video thumbnails displayed alongside short summaries of the video content.

Video Digests: A Browsable, Skimmable Format for Informational Lecture Videos
Amy Pavel, Colorado Reed, Björn Hartmann, Maneesh Agrawala
ACM 27th Symposium on User Interface Software and Technology (UIST 2014).

Demo | Code for interface

A thumbnail of the Sifter interface for browsing common sets of image editing commands.

Browsing and Analyzing the Command-Level Structure of Large Collections of Image Manipulation Tutorials
Amy Pavel, Floraine Berthouzoz, Björn Hartmann, Maneesh Agrawala
Berkeley EECS Technical Report and Poster Presentation (TR 2013, TECHCON 2013).

Posters and workshops

A thumbnail showing a line graph of the precision-at-one of an algorithm going up as the number of noteworthy sentences considered rises. After a certain point, the number of noteworthy sentences decreases the precision at one -- indicating that lower quality noteworthy sentences add noise rather than value to the prediction.

Extracting Structured Data from Doctor-Patient Conversations By Predicting Noteworthy Utterances
Kundan Krishna, Amy Pavel, Benjamin Schloss, Jeffrey Bigham and Zachary Lipton
AAAI Workshop on Health Intelligence (W3PHIAI 2020 Workshop Paper).

A thumbnail with an illegible system diagram of Twitter A11y.

Twitter A11y: A Browser Extension to Make Twitter Images Accessible
Christina Low*, Emma McCamey*, Cole Gleason, Patrick Carrington, Jeffrey P. Bigham, Amy Pavel
The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019 Poster).

CrowdCrit: crowdsourcing and aggregating visual design critique
Kurt Luther, Wei Wu, Amy Pavel, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, Steven Dow
The 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2014 Demo).

Teaching

Instructor — UC Berkeley

CS 160: User interface design and development
June 2018 - August 2018

Graduate student instructor — UC Berkeley

CS 160: User interface design and development
June 2017 - August 2017

Student project advisor — UC Berkeley

NWMEDIA 190: Making Sense of Cultural Data
August 2017 - December 2017

Instructor — UC Berkeley

CS Kickstart, intro CS for incoming freshmen women
August 2012

Work

Postdoctoral Fellow — Carnegie Mellon University

Advisor: Jeffrey P. Bigham January 2019 - Present

Research Scientist — Apple

Group: Accessibility/ML Research
July 2019 - Present

Graduate student researcher — UC Berkeley

Advisors: Björn Hartmann, Maneesh Agrawala Committee: Eric Paulos, Björn Hartmann, Maneesh Agrawala, Abigail de Kosnik
August 2013 - December 2018

Undergraduate research assistant — UC Berkeley

Advisors: Maneesh Agrawala, Björn Hartmann, Floraine Berthouzoz
May 2011 - May 2013

Research intern — Adobe Research, Seattle

Advisor: Dan Goldman
May 2014 - August 2014, May 2015 - August 2015

Classes

Grad classes

INFO C265 — Interface Aesthetics (Ryokai)
CS 260B — Human Computer Interaction Research (Paulos)
INFO 271B — Quantitative Researh Methods (Cheshire)
CS 294-115 — Algorithmic Human Robot Interaction (Dragan)
CS 280 — Computer Vision (Malik)
CS 262a — Advanced Topics in Computer Systems (Kubiatowicz)
CS 270 — Combinatorial Algorithms and Data Structures (Rao)
CS 294-10 — Visualization (Agrawala)
CS 294-84 — Interactive Device Design (Hartmann)

Class projects

Vidistrator: Illustrating text stories with video
Amy Pavel, Philippe Laban, Mike Laielli, James Smith
HCI Research (CS 260B 2017).

Towards automated feedback for learning physical activities
Amy Pavel
Algorithmic Human Robot Interaction (CS 294-115 2015).

Drinke: Hydration tracking smart cup
Amy Pavel, Sean Chen, Steve Rubin, Elliot Nahman
Interactive Device Design (CS 294-84 2012).

Project Page | Video