Background: The digital revolution has led to a boom in the number of online healthcare resources available. To navigate these resources successfully, digital literacy education is required. Learners who can evaluate the reliability and validity of online healthcare information are likely to be more effective at avoiding potentially dangerous misinformation. In addition to providing healthcare education, massive open online courses (MOOCs) are well positioned to play a role in providing digital literacy education in this context.

Objective: This study focused on learners enrolled on a massive open online course on cancer genomics. The purpose of this study was to evaluate the efficacy of a series of digital literacy-related activities within this course. This was an iterative study, with changes made to digital literacy-related activities in four of the eight runs of the course.

Methods: This mixed-methods study focused on learner engagement with the digital literacy-related activities, including the final course written assignment. Quantitative data including the number of references listed in each written assignment were compared between runs. Qualitative data in the form of learner comments on discussion forums for digital literacy-related tasks were evaluated to determine the impact of these educational activities.

Results: Using the number of references included for each final course assignment as an indicator of digital literacy skills, the digital literacy-related activities in the final two runs were judged to be the most successful. We found a statistically significant increase in the number of references cited by learners in their final written assignments. The average number of references cited in Run 8 was significantly higher (3.5) than in Run 1 (1.8) of the MOOC (P = .001). Learner comments in Runs 7 and 8 showed that a poll in which learners were asked to select which of four online resources was reliable, was effective in stimulating learner discussion about how to evaluate resource reliability.

Conclusions: Like many healthcare MOOCs, the course studied here had a heterogeneous group of learners, including patients (and their families), the public, healthcare students and practitioners. Carefully designing a range of digital literacy-related activities that would be beneficial to this heterogenous group of learners, enabled learners to become more effective at evaluating and citing appropriate online resources within their written assignments.