Update: In October of 2022, ThomaBravo acquired UserTesting with the stated aim of combining it with UserZoom, which they'd acquired earlier that year. In April of 2023, UserTesting announced the completion of the integration, at least on corporate terms.
While that eliminates the "which is better" aspect of this article, I think the broader and final point I made some 7 years ago still stands: one shouldn't have to choose, but should in fact look to combine the best of all available tools whenever and wherever one can.
Another note: the pricing of both platforms has long since changed, and will likely continue to do so, and the acquisition by private equity doesn't necessarily bode well for non-enterprise users who want economize.
As always, I highly recommend developing, building and maintaining your own panel of users (it's usually more cost-effective, faster, and cheaper to organize and care for your own panel).
And maybe you should try Paixon.io if you want to have an amazing place to organize your own research around the entirety of the Customer Journey.
Both UserTesting.com and UserZoom.com offer a fairly comprehensive suite of usability testing and other research tools, and work remarkably well as platforms for conducting remote research. Why would you want to use a remote research platform? Three reasons:
- Without the geographic constraint of needing to find test participants who are near to where you are, you have a much larger pool of participants to pull from.
- Depending on when you need feedback, you can often get results in less than an hour (Though a day or two is more reasonable)
- Some of the templates for testing formats available let you script a test fairly quickly, using a wide variety of artifacts (sketches, clickable prototypes, live sites, or even just pointed, well-formed questions.)
The one major drawback of remote usability testing should be obvious: you're not in the room with the participant, so you can't always gauge non-verbal feedback, and you can't ask followup questions.
COSTS: REASONABLE VS. "ENTERPRISE"
The major difference between UserZoom and UserTesting.com comes down to the cost of using the platform (since technical capabilities are, if not equal, at least equivalent): UserTesting.com costs "only" $50/participant; an Enterprise-wide site-license for UserZoom starts at $100,000. So, if you're not running a research program for a company that has that kind of research budget, you can stop reading now. Go with UserTesting.com.
Participant Recruiting and Panel Management
A note about that $50/user for UserTesting.com: many people complain that this is "too expensive", but in my opinion they are focusing mostly on the cost of providing the technology for gathering feedback. The reason you're paying $50/user with UserTesting.com is because they provide a panel of participants that you can recruit from. They build that panel, screen participants, manage invitations, followup when participants don't complete a test, and provide feedback and other "care and feeding" to their panel.
Anyone who's worked with meatspace recruiters knows that $50/person is cheap. And if you decide to do all this yourself, and manage a panel of your own customers, you could easily find yourself spending 1/2 your time each week just dealing with panel management. Unless you have a very specific, very hard-to-find subgroup of customers, it makes little sense to do this yourself (and even then, it might make more sense to use UserTesting.com's Private Panel feature.)
With UserZoom, you need to bring your own panel. The platform will manage things like making sure participants only take a test once, and providing automated feedback to a panel recruitment provider when a participant has completed a test (so you know who to compensate), but they don't provide either a panel of participants to recruit from, or a recruitment tool.
Qualitative vs. Quantitative
UserZoom offers a fairly comprehensive array of question types, question styles, script "pathing", conditional questions, randomization, and even segment-based surveying (where you define a sub-segment of participants, and serve them different questions/tasks.)
UserTesting.com offers far fewer question styles, and focuses on giving participants tasks to accomplish with interactive artifacts, and recording their feedback. (UserZoom also offers participant recording and screen recording.) UserTesting.com also allows you to recruit participants on mobile devices, to test either mobile versions of websites or mobile apps, and offers panelists who will either submit screen recordings of their devices, or who use a webcam to capture their devices and their hands, so you can see what true "interaction" with a mobile app looks like. (This is as close to being in the room with a participant as you can get, I think.)
So, generally speaking, UserZoom is a more powerful and flexible quantitative tool, while UserZoom is a more flexible qualitative tool.
Ease vs. Flexibility
A drawback of having a very flexible tool for scripting tests is that you need to have a much better sense of what you're doing, otherwise you can write scripts that go awry, or don't ask the right questions at the right time. Because you're not in the room with the participant, it's especially important to try to anticipate, in the way in which you write a script, the problems participants might encounter.
UserZoom provides the ability to preview an evaluation as a participant would see it (whereas UserTesting.com only lets you see the questions and tasks being presented in a somewhat artificial "preview" state). But with UserZoom, adding conditions or randomization to a test puts the burden on you as the researcher to QA the evaluation: you need to make sure that participants are seeing what they're supposed to see, when they're supposed to see it.
Bonus: Why Choose if You Don't Have To?
One of the great things about using modern remote research tools these days is that you can combined multiple tools to create an evaluation which is both quantitative and qualitative, which combines a rigorous survey methodology with deep observational insights. You can certainly combine a UserZoom study into a UserTesting.com evaluation; you can also use a tool like SurveyMonkey or even Google Forms to embed a series of survey questions into a UserZoom study.
Budgets are an obvious limitation, but if you have some flexibility and some ingenuity, you can accomplish almost anything within a remote usability study that you can within a lab.