peer review

Some aspects of what we mean by eLearning quality can be captured in a reasonably objective manner (e.g. are learning objectives stated) but most of what we mean by quality (e.g. student engagement) can only be captured through more subjective measures. However, once we start to use subjective measures then the results begin to depend on who is doing the measuring, and, crucially, the results vary depending on the positioning of the reviewers with respect to the courseware.

So an eLearning producer may have one view (and within the company, the coders may have different views from the graphic designers), but the learners and teachers who will use the courseware, the employers who will employ those who have used the course, maybe the company that has commissioned the courseware for its employees, national government agencies and other social agencies may all have different perspectives on what is important in judging the quality of the courseware.

None of these perspectives have a monopoly on truth, and so the new international quality initiative ‘epprobate’ is using an approach that calls on views from a range of perspectives and stakeholders in order to develop its quality reviews.

Mere popularity is no guarantee of quality – one only has to look at the most popular TV programs, newspapers and YouTube videos to be convinced that popularity is not necessarily the same as quality!

On the other hand the traditional approach to quality assurance also has its problems. In education, the traditional approach has been for a small team of educational experts to come to a consensus view as to whether a journal article, a course, a programme of courses or an educational organization meets an established set of criteria. Such experts typically have knowledge of education and the quality evaluation processes and call on content experts if this is appropriate.

Such quality assurance systems have been criticised for being overly controlling, dominated by one particular perspective, and stifling initiative. So these approaches to quality assurance are giving way to quality enhancement approaches, and at the same time much more emphasis has begun to be put on student involvement in the quality process.

However these general quality schemes even in their most recent formulations are not ideally suited to the demands of an educational system subject to rapid change and growth and in particular those demands that arise from the use of eLearning. Many quality schemes for eLearning have been developed but most are somewhat tied to the limiting aspects of traditional quality approaches.

The solution that epprobate is proposing is to carry out reviews from a range of perspectives, in terms of a published set of quality criteria (http://epprobate.com/index.php/en/epprobate-quality-grid), and to involve the courseware producer with a learning community based around this review process. The production by the eLearning courseware producer of a self assessment is a vital part in encouraging the development of eLearning quality through self evaluation. A typical review panel would consist of representatives of the target group for the course, a pedagogical and quality expert, another eLearning courseware producer, a content expert and the eLearning courseware producer. This panel would produce a report examining the courseware in terms of the published criteria, and would award the epprobate label where the courseware was found to be of high quality.

Rather than simply a process of providing a label, the core of the epprobate process is the promotion of a community of peers working together to improve eLearning quality. We will achieve our goal of supporting the development of high quality eLearning courseware through a combination of consulting with a range of perspectives and multiple stakeholders, reviewing against a published set of criteria, producing detailed evaluative reports, and involving eLearning producers within our learning community.