Interaction10 – Livia Labate – Ceci n’est pas une KPI
Written by Russ on February 7, 2010 – 9:27 pmI’m proud to call Livia Labate a good friend and a fellow board member for the IA Institute, so I’ll admit having bias here. That aside, she gave a great presentation and everyone learned a bit more about Key Performance Indicators, as well as the challenges and struggles that Livia went through in her journey to be able to share this information with us.
Ceci n’est pas une KPI Session Description:
“What are the Key Performance Indicators of a succesful user experience?” asked the business stakeholder, “It depends” answered the wise interaction designer…
Inquiries about measuring or managing anything seek to answer an underlying question: How do we know we are doing a good job?
(So, how DO you know you are doing a good job in the work that you do?)
While many UX practitioners like to answer this question with ‘when we achieve our goals’, we have struggled as a community to identify and articulate approaches and measures to assess how successful our solutions are.
More at the Interaction10 Website >
Livia Labate’s Bio:
Livia Labate is a user experience designer practicing in Philadelphia at Comcast Interactive Media. She also currenty on the IA Institute Board of Directors and co-chairing the 2010 and 2011 IA Summits. In other words, she loves the UX world and enjoys giving back to the community.
Here are my notes from Livia’s awesome session–I hope she’ll continue her exploration on the topic so we can learn more!
How do we value success in UX?
- Livia promises us she has more questions than answers
How do we KNOW we are doing a good job?
- Livia asked all the designers at Comcast “How do you make decisions?”
- The theme was that people were wanting to know how they were doing a good job throughout the interviews.
How do we MEASURE UX?
- She had to put the question aside; couldn’t get a definitive answer.
- Felt it was a lot like asking someone to measure love.
- Do I have to deal with all those NUMBERS?
- We’re really consumers of data, but we’re not analysts
NUMBERS
- When people hand numbers that were meaningful for them, they really didn’t want to investigate <the problem> further.
- Numbers by themselves don’t express meaning–context really help us understand the way to answer a question.
- Jared Spool made her read “Moneyball: The Art of Winning an Unfair Game.
- “Statistics do not have the power of language, it’s just numbers.”
METRICS
- “You can’t manage what you don’t measure” – W. Edwards Deming (or Peter Drucker)
- We need to focus on the outcome
- “You can’t improve what you don’t understand.”
- A metric is a unit of measure. It’s a quantitative measurement that describes events or trends.
- A KPI is a metric that helps you understand HOW you are doing against your objectives.
- Context really sets the frame for what KPIs make sense. A competitor will most likely have different KPIs.
- KPIlibrary.com is a good resource for finding–big surprise–KPIs
- KPIs will help us play the game, as soon as we know what they are?
- He who knows the KPIs can direct the conversation(s)
USER EXPERIENCE
- How can we measure the undefinable?
- Focus on the outcome
- The task completion rate
- The time a task requires
- The error rate
- Users’ subjective satisfaction
- Morville created the honeycomb to help clients move beyond just usability.
- Livia talks about “Linking Elephants”; references Adaptive Path’s 2005 report on Leveraging Business Value: How ROI Changes User Experience
- Livia tried to do this, but it was difficult/impossible for her in the projects she tried it with.
- It’s not a KPI in it’s own right, but it can be useful for framing a conversation.
Why KPIs?
- KPIs are concrete metrics
–quantifiable and measurable
-KPIs are relative measures
–to predefined objectives
-KPIs are understood in context
–circumstances in which they are measured
-KPIs are about behaviors
–and this is why we like them!
Posted in Information Architecture, User Experience Design | 2 Comments »
Hey Russ, thanks for the recap! For folks who end up here and are interested in more, the slides are up on slideshare: http://j.mp/kpisixd10
Thanks, Liv! I’ve now embedded!