T08: Optimizing survey research for HCI purposes

Full Day Tutorial

Hendrik Müller (short bio)
Senior User Experience Researcher, Google, Inc., Sydney, Australia

Aaron Sedley (short bio)
Senior User Experience Researcher, Google, Inc., Mountain View, USA

Objectives:

Online surveys are widely used in HCI to gather feedback and measure satisfaction; at a glance there are many available tools, and the cost of conducting surveys appears low. However, there is a wide gap between quick-and-dirty surveys and surveys that are properly planned, constructed, and analyzed. This full-day tutorial will examine survey research approaches that meet HCI goals, selecting the appropriate sampling method, questionnaire design best practices, identifying and avoiding common survey biases, and considerations for survey implementation, fielding, and analysis. Interactive exercises as well as numerous examples will be used throughout the entire tutorial to engage the attendees with the material and to make it immediately applicable to their work. The audience will gain an appreciation for the breadth and depth of surveys in HCI, combined with keys to conducting valid, reliable, and impactful survey research themselves.

In particular, attendees will learn about:

• The role of survey research to measure attitudes and to gather feedback
• When it is and when it is not appropriate to use survey methods
• The entire survey lifecycle, from goals to project planning, sampling considerations, questionnaire design, choosing the right tool, fielding, and analysis
• How to design surveys, with an understanding of sources of survey error and questionnaire biases

Content:

Below is a detailed outline of the topics for the morning and afternoon session:

• Part 1: Surveys fundamentals and survey use in HCI research
  • Introduction to survey research
  • Exercise: Attendees’ experience with survey research
  • Survey types and examples
  • A short history of survey research
  • Surveys appropriateness in the context of HCI
  • Exercise: Identifying survey appropriateness in proposed scenarios
  • Overview of the survey life cycle
  • Research goals & constructs
  • Exercise: Turning goals into constructs
  • Population & sampling (sample methods, modes, sample sizes)
  • Open Q&A
• Part 2: Questionnaire design and survey biases
  • Overview of the survey life cycle
  • Question types & when to use each
  • Questionnaire biases
  • Other question types to be avoided
  • Visual design considerations
  • Exercise: Identifying biases from example surveys
  • Testing and optimizing your survey
  • Implementation considerations for online surveys
  • Maximizing response rates
  • Data analysis fundamentals (closed & open-ended data)
  • Open Q&A

Target Audience:

People from academia, industry, and government with a common desire to further their knowledge of survey research, i.e., including but not limited to user experience researchers, other user experience practitioners, product managers, and analysts. The tutorial targets both those wanting to use surveys and other feedback-gathering methods as a quick and simple tool, as well as those looking to refine their approach to survey research.

Bio Sketch of Presenters:

Hendrik Müller (Mueller) is a senior user experience researcher at Google currently in Sydney, Australia. He leads user research for Google Drive, supports Google Docs, and previously worked on Google Health and several other products. His research interests focus on file management, cloud storage, collaboration, mobile devices, and survey methodology among other methods. Together with other researchers, he leads survey efforts within Google’s user experience team to measure user happiness. Hendrik received his master’s degree in Human-Computer Interaction from the Georgia Institute of Technology in Atlanta, USA.

Aaron Sedley is a senior user experience researcher at Google, Inc. in Mountain View, focused on tracking and analyzing user attitudes via surveys. He currently leads survey research within Search, and consults with teams across Google on survey methodology, planning and implementation. Aaron initiated Happiness Tracking Surveys (HaTS) at Google in 2006, an attitudinal measurement platform that is now deployed across over 15 products. Prior to joining Google in 2003, Aaron held research positions with New York Times Digital, Young & Rubicam, and the Carnegie Endowment for International Peace. He earned a bachelor’s degree in Government from Wesleyan University.