User Experience for a Better World
Remote Usability Testing
Replies to This Discussion
-
Permalink Reply by Nancy Tomaro on -
Consider this very high level and stream of consciousness just based on some experience.
Remote Usability Testing Pros
1. Cheaper (unless you have an on-site lab at your place of work)
2. You can recruit nationally/internationally relatively easily (and depending on your site, analytics and goals if some of your questions/goals in testing are around regional differences this can be informative)
3. May actually be easier to recruit since your pool is that much larger.
4. May be easier for quick, down and dirty testing.
Remote Usability Testing Cons
1. There's nothing like sitting there and watching the participant. Granted, not too quantitative, but body language can tell you a lot.
2. Depending on your test group, if they're not too tech savvy, trying to get them to get set up, dialed in, while you're remote can be challenging.
Lab Based Usability Testing Pros (assumption you do not have on-site lab)
1. Okay, it has to be said, it's just *funner*. (but that may not fly with your boss/client when it comes to the higher cost). But it is. heh :)
2. More ways to collect data: interacting one-on-one, videotaping their behavior as well as what they're doing on screen, etc.
3. If you contract out with a lab they can handle set up, comfortable space, video tape, computer set-up, snacks, beverages...all those minor details that can be a pain.
4. Again, you provide the lab the screener, they can do the recruiting. Typically when I remote test I'm doing the screener and recruiting.....ugh.
Lab Based Usability Testing Cons
1. More expensive.
2. Just not as flexible. You typically schedule a day, two day block of time and have to squeeze participants into those slots. Remote you have more flexibility. One tester can do it at night. The other early morning? No problem.
3. A con related to the screener is based on the lab you contract with their pool may not be as broad or deep as you need and the participants you get may not be as ideal. (ran into this before). Word of advice: work with a lab that's been recommended by others/has a good rep, etc.
Sure others have some other great insight, but that's just off the top of my head.
-
Permalink Reply by santhosh on -
in addition to the above
In remote usability testing it is harder to judge people as you don't get many channels of communication, especially non-verbal channels. In some studies you might get participants who is not good at verbalizing, so in that case you need to use your observation skills and your expertise to get more details from those kinds of participants. But while doing the remote usability study you lose quite a number of important communication channels.
Also not always the lab based test will give you the rich data. It all depends on what do you test,and how do you test: whether you are asking the participants to think-aloud or think-after or just observe by giving them a particular task or ask them to just explore. Importantly the facilitator skills and expertise plays an important role in both the studies because the preparation of the facilitator before the study and their behavior with participants during the study might have a huge impact on the results.
I am sure there are other factors as well...
-
Permalink Reply by Warren Anthony on -
I am a big fan of remote testing. The comfort level of users is so much higher and this leads to more authentic responses. It's easier and easier to do with web tools, and increasingly savvy users (generally speaking).
Here's a tip for a formal remote lab: We used WebEx and upgraded to their toll-free conference number option for users to dial into. WebEx is dead easy to setup and walk users through (works on a Mac too), and records the whole session without you needing any extra tools (we still record with Morae as a backup anyway).
Another bonus of remote testing is you see people's screen sizes, browser type and all those horrible extra bars and things they have installed! It's like being in their house. That is a quantative and measurable addition to your test results.
Lab tests in my opinion are way more stress and less fun!
This is a great book on the topic:
http://www.rosenfeldmedia.com/books/remote-research/
Optimalsort, the 5 second test, Usabilla, Loop11 and OpenHallway are other great tools too.
Finally - you can stealthily eat a banana during a remote test (see pic attached). - Attachments:
-
-
FCV user test.jpg, 24 KB
-
-
Permalink Reply by Anindita Paul on -
We, in our lab, has always avoided remote testing as we have always had issues with technology, sometimes does not give the quality data that we would prefer to have; Audio is not clear either.
Warren, thanks for the WebEx info, I would really like to try that out. As for the Bandwidth issue, I believe that is something which cannot be helped. And I would like to take a look at the book on remote testing too.
-
Permalink Reply by Ayush Jain on -
I believe this has also got to do a lot with the kind of user profile you are testing for. If its for a software product whose users are easier to catch hold of, a lab test might be just the best idea. For a web UI, which is to be used by a variety of people from their own browsers, a remote test might give the best results.
-
Permalink Reply by Christopher DeFields on
-
Consider the third option, field testing in the user's own environment. You have to travel, but every cost/benefit analysis I've done supports this approach. Unless you need to collect statistical data and prove high level of data confidence, behavior based feedback is best collected live in the user's world. And, since behavior has far fewer variations to opinion, you don't need the variance in test subjects as often expected...
Videos
© 2012 Created by Diane Chojnowski.
