Design and Evaluation of a User Experience Questionnaire for Remote Labs

Abstract

Remote laboratories have been in use for 25 years now. Whereas several learning oriented meta-analyses exist, validated and agreed-upon tools for assessing the user experience are not readily available. The present paper fills this gap by designing and evaluating a questionnaire focused on the needs of remote labs developers and educators using them. Building from pre-existing tools, a first version of the User eXperience Questionnaire (UXQ version 20190308) was designed to contain four scales, usability, utility, satisfaction and immersion. A total of 180 completed responses were collected from two different remote labs (VISIR and FPGA), in different campuses and in different courses to evaluate the questionnaire. The questionnaire was analyzed in terms of reliability, using Cronbach’s alpha and McDonalds’ omega, and validity of construct, through factor analysis. The reliability of the questionnaire and its four subscales is acceptable, but its validity should be improved. Accordingly, the questionnaire was redesigned to obtain the User eXperience Questionnaire (UXQ version 20191126), which includes three scales: usability, utility and immersion, and three questions per scale. This questionnaire was assessed using the same data. Reliability coefficients are above 0.7 and construct validity is satisfactory. A new questionnaire to evaluate the user experience in remote laboratories has been designed and validated. The questionnaire, now renamed as UXQ4RL v. 1.0, is presented and made available in this paper.