uxanthros

[uxanthros]

SfAA 2019 UX Workshop

Geek Out Vol. 4: User Experience (UX) Research Methods Jamboree

Overview

a strategy to research people’s experiences with a product or service online and offline

a process to analyze the data 

tools to visualize and communicate the findings to teammates or clients 

templates and other resources to keep practicing UX research 

“Standup”

Purpose: Used daily on teams 

to keep everyone on the same page

aligned with completing two-week chunks of projects 

and it allows for team members to help each other with their obstacles

How to: Everyone stands in a circle, and shares the following in 30 seconds (while passing something amusing from one speaker to the next):

*Name?

What are you working on / studying?

What are the obstacles you’re facing at work / school?

It’s not a conversation. 

Say, “parking lot” to end current speaker and move to the next.

If you can help someone with their obstacles then say, “I can help.”

After standup have follow up conversations (3-5 minutes).

For later:

Other ways of doing a standup.

Broader context for doing standups when building digital products.

Who is this guy? What’s his UX research journey?

Marc has been doing user research and service design for eight years. He currently leads a team of UX researchers and service and visual designers inside San Francisco government. He has facilitated numerous workshops on this topic for practitioners in industry and government, including at three previous SfAAs. He likes comic books and struggles to cook fish. Pointers welcome.

Why is anthropology a good fit for UX research?

Reflection & Curiosity: “Anthropological thinking is a habit of mind that begins by questioning fundamental categories of meaning.”  (Aaron Podolefsky, 2011:187)

Holistic & Critical: Lived experiences unfold in a historical context shaped by structures of power and privilege that impact personal agency. 

Methods Geeks: Blend qualitative and quantitative methods and anything else to be more effective.

Expectations?

What would you like to get out of our time together?

Marc’s expectations: have fun & deepen your understanding of how to be a more effective practitioner of design & UX research.

Guardrails?

How should we act / treat each other in order to be successful today? 

Example: At any time ask questions, leave for the room to use the bathroom or take a phone call.

Why UX research?

Source: Nick Bowmast, Userpalooza, Pg. 86

What is UX research?

To understand people’s needs are:

Functional (seeking out a company or org. to complete a task or job),

Emotional or context dependent (nostalgic, anxious, hungry, sick, etc.) (Clayton Christensen et al, HBR, 2016). 

We should also consider people’s needs within societal, environmental, historic and systemic contexts (social impact UX research or “design anthropology”).

Examples from the services you receive?

What about the services you provide others?

What is service design?

End to end journey or experience of the person/resident/customer/user. 

Face-to-face, online, email, SMS, mail and other “touchpoints.” 

From this perspective, product design done holistically is service design. 

Example: (SEE TAB)

Source: Adaptive Path & UIE

What is systems design? 

consider integration of services 

the whole organization/company 

society / other stakeholders 

the environment

Example: (SEE TAB)

Source: U.S. Veterans’ Affairs

Design Research (vs User Research):

Wireframes & other design outputs / deliverables (Sarah’s website).

Often tech-oriented.

Part of on an Agile or iterative product development team. 

Lets Do This!

57% of young Portlanders qualify for free or reduced-price lunch during the school year. 

Thought Experiment: 

The City of Portland wants to make it easier for the kindergarten-12th grade population to access these lunches over the summer when most students are not in school. 

The City plans to use their recreation, parks and libraries to provide these lunchtime meals. 

They hired your UX research company to understand how to redesign some pages on its website (PortlandOregon.gov) to help in this effort. 

Let’s understand the problem space: Grab a marker and easel paper, and answer the first six questions only (7-10 minutes). We’re creating a design brief / project plan. (No need to write the question.)

Who may use the website?

What are their needs? (e.g., Role of equity? Historical context? Accessibility & usability?)

Intended outcomes / results / future state?

Outputs / deliverables?

Success Metrics (How many? How well? How is anyone better off?)

Assumptions about the problem & what UX research will uncover?

Current state / Project description? Historical context?  Existing data? (Critical for projects but we’re constrained by this thought experiment.)

If we___, then we____ (”Hypotheses” to test through research by bridging the current and future states. We plan to use this soon, but not now). 

57% of young Portlanders qualify for free or reduced-price lunch during the school year. Thought Experiment: The City of Portland wants to make it easier for the kindergarten-12th grade population to access these lunches over the summer when most students are not in school. The City plans to use their recreation, parks and libraries to provide these lunchtime meals. They hired your UX research company to understand how to redesign some pages on its website (PortlandOregon.gov) to help in this effort.

[Break?] Now, let’s create a team strategy by placing our distilled answers on David Sibbet’s Game Plan template:

Who may use the website?

What are their needs? (e.g., Historical context? Equity? Accessibility & Usability?)

Intended outcomes / results / future state?

Outputs / deliverables?

Success Metrics (How many? How well? How is anyone better off?)

Assumptions about the problem & what UX research will uncover?

[We’re missing the proposed steps to achieve our future state.]

Time to do a premortem as a group to create some action items. Write your responses on sticky notes.

Thought experiment set in the future about the past.

Use the past tense of verbs. 

Imagine this project is done and...

Now imagine...

What can we do to...

Use “if we___, then we___”

Let’s add these in the middle.

Now that we understand the problem space we can introduce:

Source: U.K. Design Council & Cargo Collective

Pre-Double Diamond

What we did above was pre-DD phase.

Here is another pre-DD activity.

Top-Down (deductive analysis) Activity

It works against “confirmation bias” when analyzing our findings.

It also serves as a benchmark with clients or teams to help demonstrate the value of doing UX research.

Instructions:

Write one idea per sticky note. 

What you think Portland residents likely experience on the City of Portland website today when trying to find information?

3 minutes, go!

Affinity Cluster

Put the sticky notes on the Affinity Cluster of Assumptions/Top-Down Analysis space.

Next analyze the notes, looking for patterns.

Create column headings by writing on a sticky a note a pattern or theme.

Arrange the notes into vertical columns under the appropriate heading. 

There is no one right way. 

Rearrange the work of others or to write new column headings if you wish.

We will use these headings as part of the analysis later (see below).

Photograph your work for reference, data back ups, and portfolio building.

5-7 minutes, go!

Pause for reflection time.

Discover users’ “pain points”/needs.

Source: UK British Design Council (origin story), CargoCollective Visual

Usability Test

Form groups of three and decide 

who will be the local resident (sitting between)

the interviewer asking questions

the other interviewer taking notes to document their “pain points”, needs and overall experiences on the website.

Everyone in groups?

Before starting we will do this on the screen in front of the group for a couple of minutes with an example. A volunteer please?

Write one thing (observation or verbal feedback) per sticky note.

Be sure to keep your feedback in sequential order as you blow through the sticky notes with the first sticky note on top.

Starting prompt “Please speak aloud whatever you’re thinking“

Let’s try this for 3 – 5 minutes. [Set timer, go!]

Pause for reflections and questions before working in teams of three?

Your Turn

The user / local resident should be sitting between two researchers, and everyone should see the screen. 

Researchers: 

“Please speak aloud whatever you’re thinking. There are no wrong answers here. We’re trying to figure this out together.”

Researchers aren’t saying anything else besides the occasional “Please speak aloud whatever you’re thinking” and the occasional “a-huh” and “mmmm.”

Keep your notes in sequential order as to write them, so that the first note is on top of your pile.

Questions? 5 minutes. Go!

Define the problem(s)

Source: UK British Design Council (origin story), CargoCollective Visual

Experience Map

We’re now going to put all these notes on  the wall.

“Start” is written on the top far left and “Finish” on the top far right.

A horizontal line goes across the middle of the paper.

On far left side a + (positive experience) is above the line, – (negative experience) is below.

The middle area is a neutral experience.

Researchers work with the user to create the map.

Once done, put the sticky notes on the wall between Start & Finish, and either in the + or middle ground or – feeling expressed by or observed of the user.

Add additional notes underneath for how they were 

feeling

thinking

anything else that’s noteworthy

7 - 10 minutes. Go!

Now let’s add the column headings

Aware, Access, Receive, Finish/Next Steps

Bottom-Up (inductive analysis)

Next analyze the notes, looking for patterns.

Next compare your analysis with the Top-Down (deductive analysis). 

Similarities? 

Differences?

Create an experiment or prototype (e.g., draft, mock up, simulation) & Test it!

Source: UK British Design Council (origin story), CargoCollective Visual

Creating the prototype

Working individually, 

redesign 3 - 5 pages on the City of Portland website 

to make finding information about free and reduced lunches 

for K-12th graders & their families

to access in local parks, recreation centers and libraries

easy to learn about on their mobile phone. 

Start by drawing a simple outline of a mobile phone. (See Marc’s attempt).

Feel free to see other websites on your mobile device for inspiration.

Purpose is to work quickly and cheaply in order to get feedback early.

Feedback will be your teammates clicking on the paper and talking aloud what they expect to see and why.

Have 10 – 15 minutes. Go!

Testing the prototype with actual users / people.

Gather around me to demo.

Need two volunteers.

Testing prototypes is goal directed.

There are no wrong answers.

This isn’t a test.

We just want your help to make the website better.

Please talk aloud whatever you’re thinking or feeling. 

Let’s say you are interested to find free or reduced lunches this summer for you and your kids.

You are on the City of Portland website.

What you do to find the nearest location to your home for these lunches?

Can you touch the paper and tell me what you what expect to see on the next webpage?

Talk aloud whatever you’re thinking or feeling.

Questions from anyone? Your turn?

Return to your group of three.

Everyone switch roles and test your prototypes.

Three minutes each before switching roles.

Questions. Set timer Go.

Switch.

Switch.

Prototyping Guidelines

5 – 15 (or even 30) at a time until saturation.

Edge cases can be gold!

Pill bottle example.

Source: Jakob Nielsen, 2000 (read this article and keep in mind Nielsen’s advice: “You need to test additional users when a website has several highly distinct groups of users. The formula only holds for comparable users who will be using the site in fairly similar ways.”)

Deliver 

We can’t deliver a final product today.

Some UX researchers hand insights over to clients after doing a presentation and never know the impact they had.

Other UX researchers provide their clients with:

personas or profiles (see pages 15-20) 

video or audio clips

or transcriptions 

and ask them what patterns they see in the data.

Doing so brings in the decision-makers into the discovery and definition phases. 

Some UX researchers ask their clients to accompany them to do fieldwork as well.  

Take Home Messages

1) What was the big idea you plan to take home?

2) What’s your biggest unanswered question?

3) What’s going to be your next step?

Rest Assure: Basically Everyone’s Winging It (so don’t feel like you need to know all the answers).

Source: Lauren Ancona

Additional Resources

Community

Follow up with me to get on the Design Research listserv

EpicPeople.org

AnthroDesign listserv

Various Slack channels

UX Research Meetups

Code for America Brigades, Civic UX Research

One Team Gov

Service Design Network

Templates & Toolkits

Steve Krug, Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems

Usability.gov 

Service Design Toolkit

NYC Civic Service Design Tool & Tactics

Global Database of Innovation Toolkits

Authoritative References & Training

UIE (Jared Spool) | CenterCentre

Nielsen Norman Group

UX Booth

A List Apart

Center for Civic Design

Public Policy Lab

Google Analytics, Free Online Courses

Books

Userpalooza, Nick Bowmast

Just Enough Research, Erika Hall

Don’t Make Me Think, Steve Krug

Rosenfeld Media

Design Research Apps

PopApp.in (Paper prototypes)

SketchApp (Wireframes)

Podcasts

Dollars to Donuts (UX research)

99% Invisible (Design of everyday life)

Social Impact & Gov Jobs

Design Gigs for Good

Code for America Jobs Board

U.S. Digital Service

18F (Federal Gov)

Joining Your First Digital Product Team? (linguistic variation, nuance & meaning)

Agile: Sprints, Scrum, Kanban Board

Lean Startup: Test Hypothesis, Pivot

Design Thinking: ideate, Prototype, Iteration

Waterfall: Requirements, User Acceptance Testing

AI, ML, DL: Artificial Intelligence, Machine Learning, Deep Learning

Data: Big Data, Thick Data

Misc.: Front End & Back End Development, DevOps, API, Full Stack Developer, Pair Programming (GitHub, Trello, Slack, AirTable, Twilio, Asana, StackOverflow, Medium, GIPHY)