Evaluating a blended course for Japanese learners of English: why Quality Matters

Abstract With the increase in the number of blended courses in higher education over the past few years, it is now increasingly important to assure the quality of online education by evaluating blended (or hybrid) courses from various aspects such as consistency of learning objectives with assessment, learning materials and technology used, and pedagogical and technical support among others. This article describes the evaluation phase of a blended course of English for general academic purposes (EGAP) targeting second-year undergraduate Japanese students mostly from the Faculties of Law, Letters, Economics, and Human Sciences at Osaka University. In this study, the researchers adopted an inclusive approach to blended course evaluation. In order to ensure the course quality from the outset, the Fifth Edition of Quality Matters Higher Education Course Design Rubric was utilized as the major reference. As part of the evaluation process, students’ perception on the usefulness of the course was also measured quantitatively and qualitatively through an attitudinal survey instrument and open ended reflection questions. Eventually, to add an outsider positionality, the blended course was peer-reviewed by a certified reviewer from Quality Matters (QM) after having been self-reviewed by the researchers. The findings of the evaluation survey demonstrated students’ overall satisfaction with the course, and their responses to the open-ended questions provided further insight into the educational and technical difficulties they encountered. The QM peer review also yielded a score of 70 out of 99, resulting in failure to meet the essential standards. However, comments from the peer reviewer guided the refinements and improvement of the course design, and the course currently meets all the requirements of the Higher Education Course Design Rubric (Fifth Edition) upon amendment. This study discusses the implications for design, development, and evaluation of English-as-a-foreign-language blended courses and provides practical tips for online/blended course designers. Further refinements to the existing course can be made in the future by implementing it several times with various groups of students.