A Comparative Study of Synchronous and Asynchronous Remote Usability Testing Methods
Authors
AHMED S. ALGHAMDI, ALI H. AL-BADI, ROOBAEA ALROOBAEA and PAM J. MAYHEW
Abstract
Traditional in-lab usability testing has long been the standard method for evaluating and improving the usability of software interfaces. In-lab testing, though effective, has its drawbacks such as unavailability of representative end-users, high testing costs, and the difficulty of reproducing a user‟s everyday environment. To overcome these issues, various alternative usability evaluation methods (UEMs) have been developed over the past two decades. One of the most commonly used is the remote usability testing method. Ever since remote usability testing was introduced fourteen years ago, its effectiveness has been judged and evaluated against that of traditional in-lab testing. However, there is a distinct lack of research exploring the effectiveness of the various modes of remote usability testing. This research aimed to conduct a comparative study of two types of remote usability testing methods namely: synchronous and asynchronous remote usability testing. These two methods were compared through an evaluation of a website, which involved three points of comparison: number and type of problems discovered, overall task performance, and test participants‟ satisfaction. The results of the study showed that the synchronous testing method performed better than the asynchronous testing method in terms of the number and types of usability problems discovered, although no statistical significant differences were found. The participants in the synchronous test were notably more successful than the participants in the asynchronous test in completing the test tasks. However, the asynchronous test participants were significantly quicker than synchronous participants in performing those tasks. Participants in the synchronous tests also scored slightly higher satisfaction rate with regards to the targeted website. However, asynchronous participants were considerably more satisfied with the remote method that they had participated in. The paper concludes with a set of recommendations for conducting such research.
