UX Tea Break: Eye tracking in usability testing is almost never worth it
Вставка
- Опубліковано 5 лип 2024
- Subscribe to my channel: bit.ly/2NBKaOe
I answer the question, "I have the opportunity to conduct an eye-tracking study before my PhD contract is up and I was pondering on how valuable this technique is for a UX practitioner."
Want to see other questions I've answered? Try the video finder: www.notion.so/userfocus/UX-Te...
- My Udemy class: uxtraining.net/index-udemy.html
- My book, 'Think Like a UX Researcher': uxresearchbook.com
_______________________________________________________________________________________________
I can bring live, instructor-led training to you via Zoom. All you have to do is provide the delegates:
- BCS Foundation Certificate in User Experience bit.ly/2XQTaTA
- Introduction to User Experience bit.ly/2NAeHMd
- User Research Fundamentals bit.ly/2LBfh9Q
- Design ethnography: Take control of customer visits and interviews bit.ly/2RYfqFy
- Usability testing: Take control of testing your product with users bit.ly/30el89e
- How to design and implement web surveys bit.ly/2YBEQeF
- Improving your impact as a user researcher bit.ly/2XQU5U2
Contact me about UX training: bit.ly/2XqtXjk
_______________________________________________________________________________________________
Let's connect:
- Twitter: / userfocus
- Facebook: / userfocus
- LinkedIn: / davidtravis
- Instagram: / uxresearchbook
--
Spanish captions (when present) provided by the LabX of Sperientia [studio+lab]® (México)
Very much agree with this generally - the most valuable user research data almost always comes from traditional low-tech social science based techniques like think-alouds, surveys, diary studies, and so on. However, in my particular space (video game user research) we've often found eye tracking very useful in figuring out general trends among our core user demographics in non-text scanning tasks. For instance, if you're testing players of a first person shooter, knowing where users are likely to look in the environment under different lighting or briefing conditions, or how UI elements affect their scanning strategies is really helpful. So there are specific cases (as Dr. Travis points out), where this can be really helpful.
That's a good point and a really good example of where eye tracking makes sense. Thank you.
Thanks for this video! I really love your content ; I'd love to learn more about this subject. Would you recommend any articles that can explain/describe the disadvantages of using this kind of tool?
I've not read much that goes into more depth than this video to be honest. My suggestion is instead to read a good book on eye tracking and then you can make an informed decision. I'd suggest 'Eye Tracking the User Experience: A Practical Guide to Research' by Aga Bojko.
Thanks for your insights, David! Very informative as usual.
I heard that usability tests are mostly about testing working / semi-working prototypes. But some books I've read tell that we can get useful information by showing people static wireframes, user flows, concept maps (on early stages, especially). It would be interesting to learn about this process. I know about 5 sec test, first click test, A/B. Are there any other techniques we can apply to test static representations?
The methods you're describing aren't usability tests, though they provide interesting data points. Usability testing is all about tasks -- if your participants aren't doing tasks, you're not running a usability test. These other methods work well so long as you don't solicit opinions and ask people what they like/dislike. For example, with a 5-second test, you show a participant a screen and ask them what they remember. That gives you a useful pointer as to what stands out in the design.
im sorry but i dont admit. ofc its also important to understand labels but are you walking streets by pointing your finger in the direction you want to go? no you dont, your eyes steer your movement and its the same with the mouse. you dont "search" with your cursor... you search with your eyes and only if you have found what you were looking for you do start moving your mouse. eye tracking is essential for finding ux flaws in your design BUT it shouldnt be your first testing method.
That's a good analogy. When you are navigating the streets you don't just pay attention to what's in your foveal vision. Your 'attentional spotlight' scans the periphery too, which is what stops you getting hit by the approaching car. Eye tracking in UI design may help you optimise very specific UI elements that you are wrangling with. But that's rarely the problem with the vast majority of user interfaces which suffer from UI and UX problems that are much more easily detected with a thinking aloud usability test. If you think I'm wrong, provide an example of where eye tracking has found an important user experience problem that was missed by conventional usability testing.
Excellent presentation!
Thank you kindly!
Thank you so much, David, my company was actually reflecting on this and this was my take on it too!
At the same time, what do you think about services that follow/record user sessions and track the mouse cursor?
All these tools are useful when used in combination with other, in-person, methods.
How can mouse cursor tracking be useful; at time with eye tracking you might notice something without moving your cursor - you move your cursor when you've already made a decision to click/progress. This information is missed whilst not using eye tracking. Potentially it can be used in combination, but I wouldn't rely to much on cursor tracking.
Thank you for this video David.
There is a case study where Uber's homepage redesign was (supposedly) guided by eye tracking findings.
This does not necessarily mean there was no simpler way to get to the same findings, but what I found is that, in less mature companies and teams, a Ux designer's findings through qualitative, in depth research, is less convincing to non experienced stakeholders or decision makers. The wow effect of the heat maps speaks to them the way market research speaks to them.
So, in some casesw eye tracking is useful, not only to gather information but to gather inside approval. And is at times a way to get a budget for a in-person interviews in places where only remote non moderated tests are conducted, if any.
I agree that the wow factor of eye tracking makes research interesting to people who might otherwise ignore it. But user research is ignored only in teams with low UX maturity (by definition). If the org invests £20k+ on an eye tracker (and the training that needs to go with it), the user researcher will be under pressure to do a lot of eye tracking studies to get a return on the investment. And that will condemn the org to spending longer in the UX wilderness because the user researcher will be doing faux-quantitative studies when the org would benefit much more from a low-tech, thinking aloud, usability test.
@@DavidTravis Thank you for your reply. Agreed
This was super interesting. I've never done eye-tracking and now I don't feel bad. :D Thanks for sharing.
Glad it was helpful.
Thanks, Travis I feel supported on the research part of your view. I would really appreciate if you could make a detailed explanation on two separate pressing issue i feel in this industry, 1. How in agile delivery environment sprint0:initial research--->sprint 1 validation usability testing-->sprint2 validation--->and so on to feed previous sprint outcome and feed in next sprint works. 2. In the world of design thinking, validation of journey & quick prototype ideas will never be with real users, how we can overcome or your thoughts on this
If you say you're doing design thinking, and you're not validating your ideas with users, you're not doing design thinking.
Very informative David!
Glad it was helpful!
Thanks for these videos David, very insightful.
I've a question for another video possibly: how do you make clear if the design, or part of it, works well, very well or doesn't work at all? How do you remove ambiguity and walk away from a test with confident decisions?
Great suggestion!
@@DavidTravis I would find this super helpful too. I confess I've done research and been unsure what the takeaway was.
Hi David, do you coach reserachers? I want to get some hands on advanced training for solving large ux problems? Advice ?
Sorry, I can't offer that service at the moment as I'm swamped with client work. Perhaps look for relevant Slack or Facebook communities?
Just finished reading Think Like a UX researcher and it’s an amazing resource. Are you planning to upload more in the future or are you too swamped. Hope all is well.
Thanks for the comment. I've paused my tea breaks as I decided I had answered every possible question you could pose in UX. :-)
There are tools that record anonymous user sessions on your website and can build click and scroll heat maps. These occure during real use context. They are anonymous and don't have the eye tracking installation issues. We see what users do but not why they do it. We see users behavior when it is initiated by them. They use the site because the wanted or needed to, not because they were asked to during a test or an interview sessions.
I have sometimes identified useful insights through this, for example, how far users would scroll on a given page this suggesting how engaging the content of the page relative to it's length or where users hesitate among other things.
Some of this information can be gathered through statistics but there is a qualitative aspect that is sometimes useful.
Any thoughts on those ? Thank you.
I think those tools are useful when used in combination with qualitative, in person research (so you can probe the 'why' behind behaviour). Unlike eye tracking, they are also very cheap.
I am not even sure about the wow factor anymore. Did someone actually impress management with the results? In my limited experience simply plotting were people look is not delivering actionable results and the presentation crumbles as soon Someone asks „so... what does this mean for our product?“
Good point, well made.
This makes so much sense, I am conducting a user research and they asked me if eye tracking was useful to be honest Not at all.
LOL.
I agree that eye-tracking is not worth it for testing web pages. Usability testing is highly valuable on the other hand and not expensive. Besides, eye tracking technology is expensive! The low-end trackers may not even do a good enough job.
Good point, you need to spend big on an eye tracker to get accuracy.