# data[].attributes.body
Hello everybody, it is now time to choose 3 sources that you want others to have a look at. Any suggestions?

# data[].attributes.body
This paper was referenced in the HEA document below and I noticed that Simone Buitendijk (our new Vice Provost (Education)) was one of the authors. This paper sets out specific recommendations for Universities in terms of strategies to approach online education.

# data[].attributes.body
I completed a MOOC last year, it was an edX course called "Teams 101". I throughly enjoyed the experience and learned a lot. I was highly motivated to complete the MOOC as I believed that the subject was important to me in my job and in general, it was developed by the university where I did my PhD and we are currently collaborating with the developers on another project. It took me longer than the expected time to complete but I did it over a much shorter time period as I was enjoying it and wanted to see it through. It involved two small assignments which were peer marked. Once I had submitted my assignments, I was given others' assignments to evaluate. They were of poor quality; the English/grammar/content was poor; one was pulled straight from a book available online; and one was a sentence about spinach (yes, spinach). At the time I found this very disappointing and obviously this did not aid my learning at all. But the feedback I received on my assignments was constructive and useful and so I was able to learn a lot from this aspect. I have started two other MOOCs, and have not got past the first lesson on either. As much as I would love to do many MOOCs, the reality is that I don't have time. I'm sure I will do more in the future though, and I look forward to these experiences.

# data[].attributes.attachment_link.description
Learn how to build effective teams, be a great team player and manage team conflict.

# data[].attributes.body
This is more of a reflection on Open Education that occurred to me when I started reading one of the other papers posted here on MOOCs.
At primary school I wanted to learn French, but this subject wasn't offered (it was a small school in New Zealand). So my amazing teacher arranged for me to study French by correspondence. This was an incredible opportunity and I continued with this for a couple of years before starting high school and continuing learning French through to first year at university. I learned some French during my correspondence course, but there were certainly a lot of educational features that this process lacked. Today I brush up on my French using Duolingo, an Open Education website (and app) that I believe to be an amazing educational resource. The learning style; repetition, immersion, etc. is highly relevant; it is fun and addictive and has competitive elements (sometimes my husband and I compete against each other). I suspect my (5yr old) son will start to use this programme in the next couple of years and this excites me immensely. Open Education is a resource, that for me as a child was unimagined, but is now a reality and I believe it will further the education of millions around the world.

# data[].attributes.body
This a really engaging talk by Sal Khan, founder of Khan Academy. He tells the story of how he developed this amazing program starting with helping tutor his cousins. It summarises some of the core principles of MOOCs but is perhaps actually a more suitable resource for Flipped Learning.

# data[].attributes.attachment_link.title
Let's use video to reinvent education | Salman Khan

# data[].attributes.headline
A Systematic Review of the Socio-Ethical Aspects of Massive Online Open Courses

# data[].attributes.subject
A Systematic Review of the Socio-Ethical Aspects of Massive Online Open Courses

# data[].attributes.body
http://www.eurodl.org/?p=archives&sp=full&article=670
The previous article I posted touched on the ethical dimension of MOOCs and I was curious to know what this meant. I found this article, which describes aspects such as equality and diversity, and also the potential commercial motivations for setting up MOOCs by institutions, and collaboration with the private / commercial sectors.

# data[].attributes.headline
MOOCs: A Systematic Study of the Published Literature 2008-2012

# data[].attributes.subject
MOOCs: A Systematic Study of the Published Literature 2008-2012

# data[].attributes.body
http://www.irrodl.org/index.php/irrodl/article/view/1455/2531
I really liked this article which I found on-line, as it has a clear and concise summary of the history of MOOCs and the categorisations of MOOCs. It reviews the development of literature around MOOCs and how this has been rapidly increasing. It also presents themes identified in the literature, such as connectivism, dangers, learning experience and ethics,

# data[].attributes.body
https://www.mooc-list.com/

I have been searching up on the different types of MOOCs available so I came across this rather too comprehensive list.
I am very impressed by the number of prestigious universities offering MOOCs.

# data[].attributes.body
Being a novice to Digital learning I am taking my first exciting step into the world of MOOCs. I know about this as my daughter's teacher was talking to the class about signing up for a MOOC. The teacher said it may look good on their 'CV'. This gave the whole thing a feeling of tick boxing for the sake of it. My daughter engaged in a MOOC she felt would be interesting - she participated reasonably well, and got something out of it - at the end of the course, she realised she had not signed off the sections as she went through. In the end, she just clicked back and signed them all off again.
Then I was thinking...call me cynical, but what would stop anyone just singing on, clicking through, then printing out a nice new certificate at the end?

# data[].attributes.body
MOOCs boast to reach large auditoriums across the world, but the problem is that very few actually complete the courses and it questionable what practical implications the completion of course has on student's future employment. I am starting to think of MOOCs as an example of 'ecological fallacy' - it appears beneficial on the population level, but might have little use at the individual level.

# data[].attributes.attachment_link.title
The Opportunities-and Risks-in the MOOC Business Model

# data[].attributes.attachment_link.description
Advoctes of MOOCs claim they have the potential to transform higher education by expanding academic access.But it remains to be seen whether this promise can be realized. We asked the WSJ Experts to weigh in.

# data[].attributes.body
The article explored the role of MOOCs in medical education. Slightly outdated article judging by the tone of optimism it has. I read with interest a post by KG on obstacles MOOC is facing. I think particularly in medicine, where quality assurance is paramount and face-to-face sessions are key for becoming a doctor, MOOCs will serve mainly as a good source of learning for pre-med/pre-clinical studies only. (posted by K. Milinis)

# data[].attributes.attachment_link.readable_text
Are MOOCs the future of medical education?

Authors: Ben Harder

Publication date: 22 May 2013

Ben Harder looks at Massive Open Online Courses and asks whether they will change the future of medical education

Carol Aschenbrener recently took Clinical Problem Solving, a course taught by a highly regarded professor at the University of California-San Francisco (UCSF) School of Medicine. Aschenbrener, who lives in the Washington, DC, area, didn't attend lectures in person. In fact, the class had no lecture hall, even though some 28 000 students had enrolled. The teeming course was taught entirely online.

Clinical Problem Solving is one of a new breed of classes known as "massive open online courses" or MOOCs, which simultaneously promise and threaten to upend the way lecture courses are conducted across a range of academic disciplines. In the past few years, these free courses have attracted hundreds of thousands of students from every corner of the world and on nearly every conceivable subject. (The first MOOC that Aschenbrener took was on literature.) Online educational platforms such as Coursera,[1] which offers Clinical Problem Solving, and EdX, an undertaking launched jointly by Harvard and Massachusetts Institute of Technology (MIT), have teamed up with leading universities to develop MOOCs. Enrolled students watch prerecorded lectures and use online quizzes and chat forums to assess their progress and collaborate remotely with classmates.

Game changer

For future doctors, the rise of MOOCs could be an educational game changer-or not. Medical educators are actively exploring how MOOCs and other online courseware could be incorporated into medical training. They agree that lecture based courses, whether online or in person, can provide only part of a doctor's education. Yet some foresee an important role for online lectures within that context. "Online content delivery will be commonplace within about five years in medical school," predicts Catherine R Lucey, who taught the 28 000-student MOOC and who, as vice dean of education at UCSF's medical school, oversees the university's undergraduate, graduate, and continuing medical education (CME). Medical students, she says, might never be able to tick off a credit for taking an exclusively web based course, but before and after medical school they may find they can increasingly complete requirements online.

To date, no medical school or premed program seems to have offered academic credit for a MOOC. But in early February, the American Council on Education certified four Coursera courses, meaning that the council now encourages universities to grant undergraduate credit to students who complete them. Just days later, Coursera announced that two other courses, including Clinical Problem Solving, could be counted as CME, as long as students seeking CME pay a modest fee and submit to an optional process for identity verification. These two courses seem to be the first MOOCs eligible for CME credit.

In Aschenbrener's case, getting credit is beside the point. She's gainfully employed as the chief medical education officer of the Association of American Medical Colleges. The association represents 141 accredited US medical schools and about 400 affiliated teaching hospitals, the very institutions where US medical students and residents learn their art. Yet like her UCSF professor, Aschenbrener believes that parts of their medical education may be poised to migrate to the web. Medical schools, she notes, are increasingly shifting away from students attending lecture-format classes, opting instead to steer classroom time toward interaction driven learning. Some schools have introduced an "inverted" or "flipped classroom" model in which students watch prerecorded lectures online in their own time before showing up in class.

Why waste time at a lecture?

A 2012 essay in the New England Journal of Medicine heartily endorsed the use of the flipped classroom model in medical education. "In an era with a perfect video-delivery platform [online], why would anyone waste precious class time on a lecture?" wrote Charles G Prober, Stanford School of Medicine's senior associate dean of education, and Chip Heath, business professor and one of the authors of "Made to Stick."[2] The pair cited some evidence, including unpublished Stanford data, suggesting that flipped classes engender greater student engagement and better absorption of material.

But a flipped classroom is not the same thing as a course conducted exclusively online, such as a MOOC. Medical educators are quick to emphasize that nearly all courses in modern medical school curriculums are built around interactive learning experiences, some of which-such as mock-patient encounters-are impossible to fully reproduce online. For that reason, experts generally don't foresee a departure from today's campus based training of medical students. Some students, Prober says, might be able to power through medical school in fewer than four years, thanks to the easy availability of core knowledge online. But they'll still need the practical experience they can only get in person.

In the long run, Lucey predicts, "medical schools will either develop courses to be taken online or license courses from other schools." But either way, she says, these flipped classroom modules will be augmented with in-class interactions at the institution where each student is enrolled.

Johns Hopkins University long ago began experimenting with the flipped classroom approach. It's been 15 years since molecular biophysicist Harry Goldberg and a team of his faculty colleagues made taped lectures available to medical students taking a course on cardiovascular physiology. Now, the university offers one of academia's more comprehensive online course catalogs in the health sciences. Through the university's Bloomberg School of Public Health alone, more than 200 000 students have enrolled in online classes. Several thousand have taken them for credit, in some cases while living and working in farflung corners of the globe.

How big will MOOCs get?

Although Goldberg, now assistant dean at the university's School of Medicine, sees promise in online learning, he's skeptical it will revolutionize how tomorrow's doctors learn. "MOOCs have a role in medical education," he says. "I think that role is a lot smaller than people hope it will be." That's mainly because lecture-style didactics-or "content delivery," in the parlance of the web-is a small piece of the overall learning experience in medicine. MOOCs permit interaction but not necessarily to the extent that medical education requires, he says.

One place where MOOCs might come into their own is in preparing undergraduates for medical school. Premed courses are often lecture based and information dense, two attributes that play to MOOCs' strengths, as students can replay a lecture until they understand it. Aschenbrener envisions students being able to satisfy some premed competency requirements through MOOCs instead of classroom time. An undergraduate who completed enough mathematics MOOCs, for example, might be able to earn his or her BA in three years, saving himself or herself time and tuition dollars. (MOOCs also might help weed out weaker would-be doctors before they spend money on tuition for premed classes that they might fail to complete.) "The ultimate goal is not to replace [classroom] learning with this style of learning, but to create greater flexibility," Aschenbrener says. She hopes MOOCs will help draw more low income students to medicine and perhaps ease the shortage of doctors.

Continuing medical education

Once doctors are in clinical practice, online courses (massive or otherwise) could also help them sharpen their skills and deepen their knowledge. With limitations on working hours for medical residents, Prober says, it can be hard for clinical instructors to assemble all their trainees in the same place at the same time-and they have to repeat the exercise year after year. If their talks were online, instructors could simply have residents watch them and complete a quiz to demonstrate comprehension.

Trainees aren't the only ones who stand to gain. MOOCs are potentially superior to existing forms of distance CME, such as podcasts of grand rounds, because they enable interaction, including quiz taking, which assists students in mastering the material, and online discussions with fellow course takers. The lack of interaction with patients "isn't as critical because they are getting the interaction every day in their clinics," Prober says.

"There's a huge range in the quality of CME that's available," says Andrew Ng, cofounder of Coursera. MOOCs developed by medical schools and offered as CME to their alumni and other doctors are likely to be of high quality, he says. This is particularly the case where a faculty member may have unique and important findings to impart to fellow practitioners-but not enough opportunities at professional meetings or grand rounds to share them. In delivering such CME online, Ng says, "there's potential for much more rapid dissemination of medical innovation."

MOOCs could also benefit medical pedagogy. Lucey of UCSF sees them as a highly efficient way for faculty to share teaching techniques directly with one other. "We do that now by writing a paper or going to a meeting," she says. About 200 of her MOOC students have indicated they are faculty members at other US medical schools. That, she says, "suggests [faculty members] are taking this course to either (a) see how a MOOC works, or (b) see how other people teach clinical problem solving."

Obstacles to growth

Several hurdles stand in the way of wider use of MOOCs in medicine. "The greatest obstacle, of course, is tradition," says Aschenbrener. Although some faculty and schools will be open to experimenting, others may fear the challenge of adapting to new teaching methods, not to mention possible new financial pressures. Importantly, there are limited data on how well MOOCs educate students. Prober, despite his enthusiasm for online lectures, says it's appropriate for skeptics to demand more evidence of their effectiveness. He also notes that different faculty members and schools have divergent ideas about the best way to teach any given medical topic. Bringing them to a consensus that enables them to share courseware across institutions will be a challenge.

Nevertheless, online courses shouldn't be ignored. "To working professionals, there's something appealing about the convenience of an online course," says Ng. A busy clinician, for example, might catch a 20 minute lecture in the window created by an unexpected cancellation. MOOCs, he says, "are bringing a lot of working adults back into the education system." That's certainly true of Aschenbrener. "I'm a big fan of MOOCs," she says. "I'm signed up for four more in the next 12 months."

Competing interests: I have read and understood the BMJ Group policy on declaration of interests and declare the following interests: BH's employer, US News & World Report, publishes rankings of medical schools; hospitals, including academic medical centers; and online degree programs. It does not evaluate MOOCs. This article represents the author's views, not those of his employer.

Provenance and peer review: Commissioned; not externally peer reviewed.

References

1. Lucey CR. Clinical problem solving. Coursera. [Link] .

2. Prober CG, Heath C. Lecture halls without lectures-a proposal for medical education. N Engl J Med 2012;366:1657-9.

Ben Harder General manager, Health Rankings, US News & World Report 1050 Thomas Jefferson Street NW, Washington DC 20007, US

benharder@gmail.com

Cite this as BMJ Careers ; doi:

# data[].attributes.attachment_link.title
BMJ Careers - Are MOOCs the future of medical education?

# data[].attributes.attachment_link.description
Authors: Ben Harder Publication date: 22 May 2013 Ben Harder looks at Massive Open Online Courses and asks whether they will change the future of medical education Carol Aschenbrener recently took Clinical Problem Solving, a course taught by a highly regarded professor at the University of California-San Francisco (UCSF) School of Medicine.

# data[].attributes.headline
Prof Christina Wassen argues that key benefits of online learning are likely to be lost in MOOCs

# data[].attributes.subject
Prof Christina Wassen argues that key benefits of online learning are likely to be lost in MOOCs

# data[].attributes.body
Prof Wassen points out how education literature stresses that learning is 'social' and that this can take place through three kinds of presence:-

Cognitive presence, which is the collaborative exploration and construction of ideas;

Social presence, the cohesion, trust, and open communication that develop within a community of learners; and

Teaching presence, the instructor’s facilitation of the group’s learning process

She argues that only cognitive presence is facilitated through a MOOC.  I would agree with her, but the HEA paper I just uploaded suggests otherwise. See Wassen's blog here:-
http://ethnographymatters.net/blog/2014/02/18/demystifying-moocs-an-eye-opening-ethnographic-study-of-online-education/ (Posted by Mark Pope)

# data[].attributes.headline
HEA research suggests satisfied MOOC students

# data[].attributes.subject
HEA research suggests satisfied MOOC students

# data[].attributes.body
This paper by the Higher Education Authority reports on in-depth interviews conducted with students of various ages at the University of Southampton who completed MOOC courses. Students praised the flexibility, the lack of fees, the social aspects and variety of tasks used in MOOC learning. This was interesting but I think it would be interesting to hear from students who dropped out of MOOC courses too. (Posted by Mark Pope)

# data[].attributes.body
The former CEO of Microsoft on the lack of success of MOOCs, but also on the hidden potential that lies within peer collaboration and assessment.

Gates argues that the interactivity online environments offer can be taken advantage of, in order to take conventional learning to new directions.

# data[].attributes.attachment_link.entities[].name
Berkman Klein Center for Internet & Society

# data[].attributes.attachment_link.title
MOOCs - Bill Gates (former Ceo Microsoft)

# data[].attributes.attachment_link.description
Eerste Duin & Kruidbergconferentie VSNU, fragment uit de Kick off over MOOCs.

# data[].attributes.headline
CodeCombat - Learn how to code by playing a game (KG)

# data[].attributes.subject
CodeCombat - Learn how to code by playing a game (KG)

# data[].attributes.body
A different approach to massive online learning: "gamified" lessons. Students are asked to play a game which requires programming skills in order to advance to the next stage.

These "gamifications" techniques are very convenient when applied to online systems, since most young students are probably already accustomed to playing games on their computer and/or other smart devices.

Monika: This is interesting since gamification is another trend in TEL. How appropriate do you think it is for your subject and Higher Education in general?

# data[].attributes.attachment_link.title
CodeCombat: Learn to Code by Playing a Game

# data[].attributes.attachment_link.description
Learn typed code through a programming game. Learn Python, JavaScript, and HTML as you solve puzzles and learn to make your own coding games and websites.

# data[].attributes.headline
Who’s Benefiting from MOOCs, and Why (KG)

# data[].attributes.subject
Who’s Benefiting from MOOCs, and Why (KG)

# data[].attributes.body
A really interesting article from the Harvard Business Review on MOOCs and their pros & cons.

It offers useful data/figures, as well as primary and secondary information which provide a positive view on the discussion regarding the benefits of these systems.

# data[].attributes.attachment_link.readable_text
In the last three years, over 25 million people from around the world have enrolled in Massive Open Online Courses (MOOCs) offered by Coursera, EdX, and other platforms. Initially heralded as a revolution in higher education access, expectations have been tempered as research revealed that only a small percentage of these millions were completing the courses, approximately 80% already had at least a bachelor's degree, nearly 60% were employed full-time, and 60% came from developed countries (defined as members of the OECD). MOOCs seemed to be serving the most advantaged, the headlines blared, and most people weren't even completing them.

Are MOOCs merely an intellectual diversion for the well educated and well-off? Do they provide any tangible benefits? We are not neutral parties. Three of us work at Coursera, and the rest of us are from the University of Pennsylvania and the University of Washington, institutions that have offered MOOCs on the Coursera platform. Two of us have taught Coursera courses, and we have analyzed Coursera's data for our research. Nonetheless, we believe we have some evidence that the MOOC skeptics are overly pessimistic. Our latest research demonstrates that among learners who complete courses, MOOCs do have a real impact: 72% of survey respondents reported career benefits and 61% reported educational benefits.

Furthermore, our findings suggest that people from developing countries more frequently report benefits from taking MOOCs and, also in developing countries, people with lower socioeconomic status and with less education are more likely to report benefits. It appears that MOOCs are tangibly helping people who take the time and effort to complete courses.
Who Takes MOOCs?
In December 2014 we sent a survey to 780,000 people, from 212 countries and territories, all of whom had completed a Coursera MOOC prior to September 1, 2014. (For more on the data, see the sidebar.)

Of the nearly 52,000 respondents, 58% were male, 58% were employed full-time, 22% were full-time or part-time students in a traditional academic setting, and 83% had at least a bachelor's degree. Thirty-four percent were from the U.S., 39% from other OECD countries, and 26% were from non-OECD countries. The median age of respondents was 41, with a range of 31 to 55 years of age for the 25th and 75th percentiles, respectively. This cohort is similar to previous studies of Coursera users except they are somewhat more educated, less frequently male, and significantly older. They also are somewhat more educated and less male than a recent study of EdX users. (Some people were more likely to respond to the survey than others; for instance, those from OECD countries and those who had completed more than one MOOC filled out our survey in greater numbers. We addressed this potential bias in our survey sample, using iterative proportional fitting to obtain reweighted estimates of reported career and education benefits.) All differences based on survey results included in this paper are statistically significant at p<.01.

The critics are right that most people who start a MOOC don't finish: just 4% of Coursera users who watch at least one course lecture go on to complete the course and receive a credential. However, given the large number of users involved, the absolute reach of MOOCs is still significant. For instance, more than one million people have completed a Coursera course since its inception in 2012, with over 2.1 million course completions as of April 2015.
Career Builders
We asked Coursera course completers about two types of outcomes: career benefits and educational benefits. Career benefits are the more common reason for taking a MOOC. Fifty-two percent of the people surveyed report a primary goal of improving their current job or finding a new job - they are "career builders."

Of these career builders, 87% report a career benefit of some kind. We asked about tangible career benefits - getting a raise, finding a new job, or starting a new business - as well as intangible ones such as enhancing skills for a current job or improving their ability to get a new job. Tangible career benefits are more difficult to achieve than more abstract outcomes such as being better equipped to do your current work. Thirty-three percent of career builders report a tangible benefit as a result of completing a MOOC, with the majority of those respondents finding a new job or starting their own business. These data indicate that the vast majority of those who complete MOOCs specifically for career benefits find those benefits, with approximately one-third of course completers reporting a more direct, tangible benefit such as getting a new job.

But which users are receiving these benefits?

Since previous research has shown that MOOC enrollees predominantly are well-educated residents of developed countries, it would not be surprising if advantaged populations from developed countries were deriving the most benefits from completing MOOCs. However, whereas our research confirms the demographics of users taking these courses, the analysis on who is benefiting suggests a very different conclusion. Among all career builders, we find that general career benefits (both tangible and intangible) are more likely to be reported by people with higher socioeconomic status and higher levels of education. The story is different, however, when you look at tangible career benefits specifically. In developed countries, career builders with low socioeconomic status and lower levels of education report tangible career benefits at about the same rate as those with high status and lots of education. And in developing countries, those with lower levels of socioeconomic status and education are significantly more likely to report tangible career benefits.

The differences between general career benefits and tangible career benefits may be attributable to different needs among different populations. People who already have a high-skill job are likely to benefit most from up-skilling to improve at their current job (general career benefits), whereas people who do not have such a job are more likely to benefit from re-skilling to transition to a new job (tangible career benefits).
Education Seekers
Twenty-eight percent of all people who completed a MOOC enroll primarily to achieve an academic goal - they are "education seekers." Of education seekers, 88% report an educational benefit of some kind. Eighty-seven percent report an intangible educational benefit (such as gaining knowledge in their field), and 18% report a tangible educational benefit - either gaining credit toward an academic degree or completing prerequisites for an academic program. People from developing countries are more likely to be education seekers, as are people with lower socioeconomic status.

The chart above shows the breakdown of tangible and intangible education benefits from MOOCs. Our research revealed even more insights, including that approximately half (47%) of education seekers are students in a traditional academic setting. Among these traditional students, 94% report some educational benefits from taking a MOOC. Twenty-four percent specifically report a tangible educational benefit. The most common educational benefits are gaining knowledge essential to their field of study (76.6%) and deciding on a field of study (40.3%). Among these traditional students, the findings are consistent across region, education, and socioeconomic status.

Among the education seekers who are not in a traditional academic setting, disadvantaged populations are more likely to report educational benefits. Education seekers from developing countries were more likely to report educational benefits; those with low socioeconomic status were more likely to report benefits than those with higher status; and those without a postgraduate degree were more likely to report benefits than those with one.

In addition, older non-students are using MOOCs to prepare for an eventual return to school. For instance, people aged 25 to 40 who are full-time homemakers or caregivers are more likely to report using MOOCs to refresh key concepts prior to returning to school.

Overall

Seventy-two percent of the Coursera participants we surveyed reported career benefits and 61% reported educational benefits.
Career Builders
Fifty-two percent of the people surveyed reported a primary goal of improving their current job or finding a new job-they are "career builders". Of these career builders, 87% reported a career benefit of some kind. 33% of career-builders reported a tangible career benefit, like finding a new job or starting a business.

Among career builders, general career benefits are more likely to be reported by people with higher socioeconomic status (90% for the top 30% by SES vs. 81% for the bottom 30%) and higher levels of education (88% for learners with a college degree or higher vs. 86% for learners with no post-secondary degree).

But for tangible career benefits things look different. Career builders from non-OECD countries are more likely to report tangible career benefits (36% vs. 32%).

Of career builders from OECD countries, we see no statistically significant difference in tangible career benefit between the top and bottom 30% by socioeconomic status (34% vs 32%). Strikingly, among career builders from non-OECD countries, learners with low socioeconomic status are actually more likely to report tangible career benefit (39% vs 35%).

Furthermore, career builders with lower levels of education were also more likely to report tangible career benefits (33% for learners with a college degree or higher vs 38% for learners with no post-secondary degree). In particular, among career builders from non-OECD countries, career builders with lower levels of educational attainment are more likely to report career benefits (35% for learners with a college degree or higher vs 42% for learners with no post-secondary degree).
Education Seekers
Twenty-eight percent of all people who completed a MOOC enrolled primarily to achieve an academic goal-they are "education seekers". People from non-OECD countries are more likely to be education seekers (36% vs. 24% from OECD countries), as are people from the bottom 30% by socioeconomic status (37% vs. 22% from the top 30%).

Of education seekers, 88% report any educational benefit and 18% report a tangible educational benefit, like gaining knowledge essential to their field of study.

Approximately half (47%) of education seekers are students in a traditional academic setting. Ninety-four percent of these traditional students report some educational benefits. Twenty-four percent of these students in a traditional academic setting report a tangible educational benefit.

Less-advantaged groups are more likely to report educational benefits. Eighty-seven percent of non-student education seekers from non-OECD countries report educational benefits compared to 80% from OECD countries; 91% with low socioeconomic status report educational benefits, compared to 86% with high socioeconomic status; and 92% without a post-secondary degree report educational benefits, compared to 86% with a post-secondary degree.

Education seekers who were older or full-time homemakers or caregivers were also more likely to report using MOOCs to refresh key concepts prior to returning to school. Thirty-eight percent of education seekers between 25 and 40 report using MOOCs to refresh concepts, compared to 35% for education seekers younger than 25; 53% of full-time homemakers or caretakers reported using MOOCs to refresh key concepts, compared to 36% for other education seekers.
Overall findings

In our survey, the overwhelming majority of people who complete MOOCs report career or educational benefits, and a substantial proportion report tangible benefits such as getting a new job, starting a business, or completing prerequisites for an academic program. Both career and educational benefits are more likely reported by people from developing countries. Economically and academically disadvantaged populations are taking particular advantage of MOOCs. Within developing countries, tangible career benefits are most likely to be reported by people with lower levels of education and lower socioeconomic status. Among non-student MOOC completers, people with lower socioeconomic status, people with lower levels of education, and people from developing countries are all more likely to report educational benefits.

These findings support some of the early hopes that MOOCs would provide a life-changing opportunity for those who are less advantaged and have limited access to education. Of course, MOOCs are still available only to people who have access to the internet, and completion rates remain low. However, there are now over 1 million people who have completed courses from Coursera alone, more than 100,000 people have certified completion from HarvardX and MITx courses, and our findings suggest that many of them derived career or educational benefits from the opportunity.

Continued innovation and research needs to focus on two main issues: First, how should people be supported to complete MOOCs so as to secure desired career and educational benefits? Second, are people who engage with some MOOC content without completing the course also realizing career and educational benefits?

This type of research illustrates the possibilities MOOCs offer to change the educational landscape. The courses are reaching large numbers of people, and disadvantaged learners are more likely to report tangible benefits. Of course they aren't a cure-all for the myriad problems of global education. But they're a step in the right direction, providing open access to a learning experience that many find beneficial for furthering their education and careers.

# data[].attributes.attachment_link.description
In the last three years, over 25 million people from around the world have enrolled in Massive Open Online Courses (MOOCs) offered by Coursera, EdX, and other platforms. Initially heralded as a revolution in higher education access, expectations have been tempered as research revealed that only a small percentage of these millions were completing the courses, approximately 80% already had at least a bachelor's degree, nearly 60% were employed full-time, and 60% came from developed countries (defined as members of the OECD).

# data[].attributes.headline
Oxford announces its partnership with edX and its first MOOC (KG)

# data[].attributes.subject
Oxford announces its partnership with edX and its first MOOC (KG)

# data[].attributes.body
A few days ago, Oxford University announced that they will be partnering with edX in order to offer certain courses online.

This could potentially urge more UK universities to engage with this new, digital and globalised, approach to teaching and learning.

# data[].attributes.attachment_link.readable_text
As part of the agreement between the University of Oxford and edX, a leading nonprofit online learning destination, Oxford will offer a massive open online course (MOOC) from February 2017.
Professor Sarah Whatmore, Pro-Vice-Chancellor for Education, said: 'This collaboration will build capacity for the design and delivery of a range of other online education experiences as the University progresses its Digital Education Strategy. Oxford already has a strong reputation for delivering open educational resources that are used by students, academics and the public worldwide.'

This collaboration will build capacity for the design and delivery of a range of other online education experiences as the University progresses its Digital Education Strategy Professor Sarah Whatmore, Pro-Vice-Chancellor for Education

Professor Anne Trefethen, the University Chief Information Officer and Pro-Vice-Chancellor for Gardens, Libraries and Museums, said: 'We are delighted to partner with edX and believe we have much to learn from each other as we explore further the potential of technology-enhanced learning.'

The first MOOC course, ' From Poverty to Prosperity: Understanding Economic Development', is freely available for enrolment on edx.org. It will be led by Sir Paul Collier, Professor of Economics and Public Policy at the Blavatnik School of Government. The course will examine the vital role that governments play in boosting economic development through video lectures, case studies, comparative analysis and a range of other learning activities. The MOOC is aimed at anyone with an interest in economic development, from university students and people working in government and NGOs, through to education and business professionals, and active and engaged citizens who are passionate about the development of their countries. Enrolment opens today, and participants need no specific academic qualifications.

Professor Ngaire Woods, Dean at the Blavatnik School of Government, said: 'The School's mission is to improve government around the world through research and education, and this requires better educated public officials, teachers, entrepreneurs, journalists and citizens. We see this first massive open online course as an effective way to expand access to knowledge beyond the classrooms of Oxford, and to help people understand how their community and country can flourish wherever they are in the world.'

Anant Agarwal, CEO of edX and Professor at MIT, said: 'We are honoured to welcome the University of Oxford, a world-leading centre of learning, teaching and research and the oldest university in the English-speaking world, to the edX global consortium. Our collaboration begins with content from the Blavatnik School of Government to provide our nine million learners with access to an innovative programme that offer insights from a range of academic disciplines and a strong global outlook. We are excited to partner with Oxford to further our shared mission to improve lives and increase access to high quality education for learners everywhere.'

The University of Oxford already has a varied and extensive online presence, offering podcasts which are freely available from both the University's website and iTunes U. The University has also played a significant role in the development and use of digital resources for research in the humanities. Oxford University Department for Continuing Education, which has been delivering adult and professional education for 135 years, also offers a wide range of certified online courses for flexible and part-time study. Oxford University Press is a University department with a vast online presence and a range of platforms that include scholarly journals and academic reference works; interactive online teaching and homework resources for teachers such as MyMaths and Oxford Learner's Bookshelf; and the leading dictionary site oxforddictionaries.com
The MOOC enrolment to 'From Poverty to Prosperity: Understanding Economic Development' is open now. The course will last for six weeks, with learners expected to spend two to three hours a week on the course. After the offering period, the course will remain open and accessible to the public to view the contents (video lectures, case studies, readings) and to take the multiple choice questions but no interaction with peers or instructional staff will be possible.

The goal of the Digital Education Strategy is to ensure that in 2020 Oxford remains a premier institution for teaching, adopting the very best of teaching innovations that are made possible by digital technology.

# data[].attributes.attachment_link.title
Oxford announces its partnership with edX and its first MOOC

# data[].attributes.attachment_link.description
As part of the agreement between the University of Oxford and edX, a leading nonprofit online learning destination, Oxford will offer a massive open online course (MOOC) from February 2017.Professor Sarah Whatmore, Pro-Vice-Chancellor for Education, said: 'This collaboration will build capacity for the design and delivery of a range of other online education experiences as the University progresses its Digital Education Strategy.

# data[].attributes.headline
MOOCs by The Complete University Guide (KG)

# data[].attributes.subject
MOOCs by The Complete University Guide (KG)

# data[].attributes.body
A comparison of UK, and global, MOOCS, with useful information about e-learning, as well as any universities providing the content (and/or certificates) for each system.

# data[].attributes.attachment_link.readable_text
MOOCs offer university-level courses without the need to complete an entire programme of studies, and are becoming increasingly popular.

They offer a large number of students the opportunity to study high quality courses online with prestigious universities, often at no cost.

They are ideal for independent study and users can select courses from any institution offering them.

MOOCs do not always lead to formal qualifications. There are no entry requirements.

Video-based, they offer interaction either through peer review and group collaboration or automated feedback through objective, online assessments (including quizzes and exams).

What you need to know

While most courses are free, some are fee-paying. Videos are normally short and much activity takes place on online discussion groups and forums.

Drop-out rates are high - up to 90%. Rates are marginally lower for paid-for courses.

A reasonable degree of computer literacy is needed.

Many MOOC users are graduates seeking to top up their skills and competences.

Why you might consider a MOOC

They are low cost routes to accessing quality courses.

They can be combined with other study or work.

They are accessed from any computer at any location including your home.

Why a MOOC may not be suitable for you

MOOCs do not feed into a degree or other qualification but are self-contained.

Few students complete the courses.

Content from a MOOC offered by a university outside your home country may not match cultural and other conditions with which you are familiar.

MOOCs in the UK

In 2012 the Open University launched FutureLearn, partnering with more than 20 UK and international universities and other institutions such as the British Council, the British Library and the British Museum.

A browse through the courses available reveals diverse subjects including How to Succeed at Interviews, Inside Cancer, Understanding Modern Business and Organisations, Election 2015 for AS Level Politics, Dysphagia, and an Introduction to Dutch, offered by UK and overseas universities.

The following UK universities are offering MOOCs via FutureLearn:

And also the British Council, the British Film Institute, the British Library, the British Museum, the London School of Hygiene and Tropical Medicine, and the National Film and Television School.

Overseas universities offering courses via Futurelearn include:

University of Auckland, University of Cape Town, Fudan University, University of Groningen, Monash University, University of Oslo, Shanghai International Studies University, Shanghai Jiao Tong University, Sung Kyun Kwan University, Tel Aviv University, Trinity College Dublin, and Yonsei University.

Who else offers MOOCs?

Universities and other institutions worldwide partner with external providers, the largest including US-based Coursera, Udemy, Udacity, and EdX, and Australian MOOEC. Some providers specialise in particular areas of study.

Coursera features 500+ courses, from more than 100 partners worldwide, including Duke University in the US, Peking University (China) and the University of Amsterdam in the Netherlands.

Udemy offers free and paid-for courses, many from non-academic instructors.

Udacity focuses on free courses, categorised into beginner-, intermediate- and advanced-level. Students may also enrol in a number of paid-for courses in order to gain California State University system credits. Recently Udacity announced a partnership with Georgia Institute of Technology and AT&T to provide a Master of Science in Computer Science MOOC course at a cost of $6,600.

EdX is a non-for-profit provider, created by Harvard and MIT (Massachusetts Institute of Technology) universities. Today partners include the Australian National University, TU Delft (the Netherlands), and Rice, Berkeley and Georgetown universities in the US. EdX also offers ID verified certificates of achievement, for a minimum fee.

MOOEC, launched in November 2013, offers free courses and lessons in English supported by Pier online, the University of Queensland ITCE, Griffith University GELI, Queensland University of Technology, Goldcoast TAFE and many non-government provider. The courses offer English language learning all levels.

Around the world, other MOOC providers include EduKart in India, ALISON in Ireland, and Aprentica in Latin America.

# data[].attributes.attachment_link.description
MOOCs offer university-level courses without the need to complete an entire programme of studies, and are becoming increasingly popular. They offer a large number of students the opportunity to study high quality courses online with prestigious universities, often at no cost. They are ideal for independent study and users can select courses from any institution offering them.

# data[].attributes.headline
Massive Open Online Course - Notable Providers (KG)

# data[].attributes.subject
Massive Open Online Course - Notable Providers (KG)

# data[].attributes.body
A user-maintained list of notable MOOCs (from Wikipedia). This link should be used for reference only.

# data[].attributes.attachment_link.description
A massive open online course ( MOOC ) is an online course aimed at unlimited participation and open access via the web. In addition to traditional course materials such as filmed lectures, readings, and problem sets, many MOOCs provide interactive user forums to support community interactions among students, professors, and teaching assistants (TAs).

# data[].attributes.headline
Tuned models of peer assessment in MOOCs (KG)

# data[].attributes.subject
Tuned models of peer assessment in MOOCs (KG)

# data[].attributes.body
A paper that aims to address the issue of students being disappointed by the feedback they are receiving from peer markers. Even partially resolving this problem, could possibly encourage more students to continue their online courses, hopefully until the end.

# data[].attributes.headline
MOOCs and the Funnel of Participation (KG)

# data[].attributes.subject
MOOCs and the Funnel of Participation (KG)

# data[].attributes.body
A paper on the "elephant in the room", i.e. the participation and completion rates of MOOCs. The embedded data can be used to offer counterarguments on the benefits of online learning, focusing on the general lack of personalised tutoring (due to class volume) which makes some students give up.

# data[].attributes.headline
The pedagogical foundations of massive open online courses (KG)

# data[].attributes.subject
The pedagogical foundations of massive open online courses (KG)

# data[].attributes.body
A positive review of MOOCs and their potential, which tries to view these systems, and their students, from different perspectives.

Monika: This is a good article treating on the subject of MOOC pedagogy. I like it that the authors are questioning the innovative aspect of MOOC. Is the pedagogy really that innovative? It's interesting to look into the original MOOC designed by Siemens and Downes. It was Connectivist, and had somehow different pedagogical characteristics than what we see today.

# data[].attributes.attachment_link.readable_text
The pedagogical foundations of massive open online courses

In 2011, the respective roles of higher education institutions and students worldwide were brought into question by the rise of the massive open online course (MOOC). MOOCs are defined by signature characteristics that include: lectures formatted as short videos combined with formative quizzes; automated assessment and/or peer and self-assessment and an online forum for peer support and discussion. Although not specifically designed to optimise learning, claims have been made that MOOCs are based on sound pedagogical foundations that are at the very least comparable with courses offered by universities in face-to-face mode. To validate this, we examined the literature for empirical evidence substantiating such claims. Although empirical evidence directly related to MOOCs was difficult to find, the evidence suggests that there is no reason to believe that MOOCs are any less effective a learning experience than their face-to-face counterparts. Indeed, in some aspects, they may actually improve learning outcomes.
Contents
Introduction
Methodology
The efficacy of online learning
The importance of retrieval and testing for learning
Mastery learning
Peer and self-assessment
Short format videos
Online forums and video discussions
Conclusion
Introduction
In 2011, the respective roles of higher education institutions and students worldwide were brought into question by the rise of the massive open online course (MOOC). MOOC platforms Coursera (2012a), edX (2012) and Udacity (2012) have partnered with 33 universities, offering more than 200 courses to over two million students in 196 countries (Coursera, 2012b). Courses offered have attracted enrolments of up to 160,000 students (Fazackerley, 2012) lending the "massive" portion to the name MOOC. These courses are also free or "open". Given that these courses are being offered by some of the most prestigious of universities, the potential disruptive nature of MOOCs was recognised early on. After all, if a student could take a course from Princeton University for free, why would they pay for an identical course given by their local (less famous) institution? Given the growth in availability of MOOCs, the question could be extended to why someone wouldn't do an entire degree programme in this way. Of course, there are a number of practical issues that need to be resolved before this happens. Providing proctored examinations to students who have had their identities verified being the most salient. However, a more fundamental question has been raised on both sides of the argument. Namely, do MOOCs represent a pedagogically sound format for learning at a tertiary level? Claims for and against the pedagogical foundations of MOOCs have been made by a variety of interested parties (Association for Learning Technology, 2012; Baker, 2012; Moe, 2012) but these claims have been backed with only a scant amount of evidence or indeed agreement as to the defining characteristics of a MOOC and the pedagogical foundations it rests upon.

For the purposes of our study, we have taken the representative format of MOOCs as they exist on sites such as Udacity (2012), Coursera (2012a) and edX (2012). These courses exhibit common defining characteristics that include: massive participation; online and open access; lectures formatted as short videos combined with formative quizzes; automated assessment and/or peer and self-assessment and online fora for peer support and discussion.

There is no absolute definition of each of these characteristics, however. Even the concept of massive is open to interpretation. Although claims have been made to large registrations of up to 160,000 participants (Fazackerley, 2012), the number who complete the course is typically much lower, of the order of 5-15 percent of initial enrolees (Korn and Levitz, 2013). Realistically, in order to qualify as massive, the participation at any point during the running of the course should be large enough that it couldn't be run in a conventional face-to-face manner.

The pedagogical foundations claimed for MOOCs follow on from their attributes and in part are justifications for those attributes. So it has been argued that online learning is particularly effective, formative quizzes enhance learning through the mechanism of retrieval practice, short video formats with quizzes allow for mastery learning and peer and self-assessment enhance learning. Further claims have been made that short videos complement the optimal attention span of students (Khan, 2012) and that discussion forums provide an adequate replacement of direct teacher-student interactions that would be considered normal for a class delivered on campus.

The justification of pedagogical benefits of MOOCs is in all likelihood teleological. The benefits have been retrofitted after the fact to a course format pioneered by Sebastian Thrun and Peter Norvig (2012). The fact that their original course and others that have followed have proved so popular, however, would suggest that there are positive aspects to the way they have been presented. The structure and format of MOOCs is being adapted as more experience is gained with their delivery and so it is important to understand in a systematic manner their benefits and shortfalls.

The purpose of this review is to examine the evidence regarding the pedagogical foundations of MOOCs and indeed validate that these foundations actually relate to the attributes of MOOCs as they are currently envisioned. These attributes and their pedagogical consequences are shown in Table 1.

Table 1: Characteristics of MOOCs and their related pedagogical benefits.

MOOC characteristic Pedagogical benefits

Online mode of delivery

Efficacy of online learning

Online quizzes and assessments

Retrieval learning

Short videos and quizzes

Mastery learning

Peer and self-assessment

Enhanced learning through this assessment

Short videos

Enhanced attention and focus

Online forums

Peer assistance, out-of-band learning

A difficulty with the analysis of MOOC structure and its pedagogical foundations is the question of how similar a MOOC is to existing online courses offered for distance learning or as an extension of face-to-face delivery of courses as part of a so-called blended delivery. In some ways they are not and so the analysis of MOOCs is inherently not that different from research examining the benefits of online delivery of courses generally. The difference lies in the particular combination of the underlying characteristic components of MOOCs, their massive participation and the fact that they are open. The subtlety in the novelty of MOOCs is not the point of this paper, however, and will be left for exploration in future work.
Methodology
A narrative analysis of research related to the proposed pedagogical foundations of MOOCs was conducted using Google Scholar (2012), Web of Knowledge (2012), Education Resources Information Center (ERIC) (2012) and PsycINFO (2012). The keywords used when searching were as follows: "online learning", "retrieval learning", "mastery learning", "peer assessment", "short video AND lectures", "short videos AND education" and "online forums AND learning". There was no specific date period attached to the keyword searches.

Studies were only included in the analysis if they provided empirical evidence on the impact of the characteristics and pedagogical foundations under study. We excluded research that used only self-reported data from student surveys as a means of determining effective learning outcomes as this was considered a highly subjective and unreliable measure of whether a particular learning strategy is effective. Inclusion focused on research aimed at higher education environments. Studies from other age groups and environments were included in the case that there was insufficient research within the tertiary education sector.

There were limitations to using ERIC due to access restrictions: many full-text documents were inaccessible due to the discovery of sensitive information being published in some documents. Many articles were, therefore, taken off-line whilst under review. As a result, ERIC was unavailable as a source for research in the online learning section of this paper. With all the elements combined there was a total of 8,614 keyword finds, 138 articles met the criteria and were deemed suitable for this analysis.
The efficacy of online learning
The principle feature of MOOCs is they largely take place online. The prevailing argument is that online courses are at least as effective as face-to-face courses. In many ways the comparison is fraught. Online learning offers flexibility of access to course materials from anywhere at any time (Allen and Seaman, 2005; Means, et al., 2010) which is not possible in a solely face-to-face environment. Face-to-face courses also become largely impractical when class sizes exceed available physical room capacities. There are few face-to-face courses that do not include the flexibility of online access to lecture materials and recordings. It has also been shown that if lecture videos and material are provided to students, attendance at the actual lectures declines (Traphagan, et al., 2010). One reason for this is that the students see there being an equivalence between the recorded and live experiences.

Several meta-analyses found no significant differences in student achievement when university students accessed content via online means or through face-to-face (Bernard, et al., 2004; Cavanaugh, et al., 2004). Cavanaugh and colleagues determined that achievement in distance education for high school students is comparable to traditional instruction and concluded that educators should not anticipate any significant differences in performance as a result of online learning. These finding were supported by comparative studies that also found no difference in academic achievement (Barker and Wendel, 2001; Kozma, et al., 2000; Summers, et al., 2005).

Lapsley, et al. (2008) found that online learning pedagogy may even be superior in the overall effect on student performance. Indeed, this is supported by a meta-analysis conducted by Shachar and Neumann (2003) who found that distance education actually surpasses the more traditional teaching format. In two-thirds of the studies reviewed by Shacher and Neumann, students taking courses in distance education outperformed their face-to-face counterparts. In a meta-analysis of online learning studies prepared by the U.S. Department of Education, Means and associates (2010) found that students who engaged in online learning performed modestly better on average than students engaged in face-to-face instruction. Maki, et al. (2000) also found that Web-based students outperformed students enrolled in face-to-face classes.

Online learning is not without its disadvantages, however. Some researchers argue that interaction and timely feedback, are quite often absent in online instruction (El-Tigi and Branch, 1997; Olson and Wisher, 2002). It has also been widely recognized that online courses experience much higher attrition rates than classroom based courses (El-Tigi and Branch, 1997; Olson and Wisher, 2002; Merisotis and Phipps, 1999). In addition, specialised skills are required to work with the technology often resulting in sound and video production that is less than broadcast quality (Kerka, 1996). Students must also display greater learner initiative as there is less supervision than in a classroom environment and there is also the potential for online students to experience social isolation (Kerka, 1996).

Despite these criticisms, the majority of literature does support the notion that online learning is as effective, if not more so, than traditional classroom teachings (Gagné, 1985; Joy and Garcia, 2000; McDonald, 2002; McKissack, 1997; Russell, 1999; Wegner, et al., 1999). With class sizes increasing as universities try to rationalise the number of courses offered, online delivery is the only way of maintaining learning outcome quality with the available resources of space and teaching staff (Means, et al., 2010).
The importance of retrieval and testing for learning
A common format for MOOCs is the short video interspersed or associated with multiple-choice quizzes (Orn, 2012). The argument made is that the quizzes provide students with an opportunity for retrieval learning (Agarwal, et al., 2012; Karpicke and Roediger, 2007). Retrieval practice is the act of enhancing long-term memory of facts through recalling information from short-term memory. There is a belief here that it will also enhance learning (Karpicke and Blunt, 2011). According to Karpicke and Grimaldi (2012), retrieval is not just a neutral assessment of a learner's knowledge; learning occurs through the act of retrieval. Every time we retrieve knowledge, that knowledge is altered, and the ability to reconstruct that knowledge again in the future is strengthened. Recent studies have shown retrieval practice to also enhance meaningful learning (producing organised, coherent, and integrated mental models that allow people to make inferences and apply knowledge) (see Karpicke and Roediger, 2008).

There is much evidence for the benefit of retrieval practice or learning. In one study involving university students in the humanities (Karpicke and Grimalidi, 2012), retrieval practice improved students' abilities to memorise lists of words over repeated study alone. Retrieval practice resulted in a 50 percent improvement in long-term retention scores. In another study, retrieval practice produced more learning than elaborative studying with concept mapping (Karpicke and Blunt, 2011). Students in the first group studied the text in a single period while students in the second group studied the text in four consecutive study periods. Short answer tests conducted one week later found that retrieval practice produced the best learning. Agarwal and colleagues (2012) have also shown that retrieval practice implemented through quizzes improves long-term retention, of information over just listening to the teacher and completing standard homework assignments.

A study conducted by Karpicke and Roedieger (2008) demonstrates the fundamental role of retrieval practice in enhancing learning and shows that university students remain largely unaware of this fact. Students in the first group learned foreign language vocabulary words via the classic model of repeated study-test trials. Under the other three conditions the following occurred: once the vocabulary item was successfully produced by the student it was (1) studied repeatedly and then dropped from further testing; (2) tested repeatedly but was no longer studied; and, (3) was dropped from both study and test. Repeated studying after learning produced no significant effect on ability for recall. Repeated testing resulted in a large positive effect. Students' predictions regarding their performance proved to be uncorrelated with actual achievement.

In a study conducted by Karpicke and Roediger (2007), participants learned lists of words and took a recall test one week after learning. Repeated study of previously recalled items did not result in increased retention compared to dropping those items from additional study. Repeated recall of items previously recalled, however, improved retention by more than 100 percent compared to dropping those items from further testing. Similarly, Roediger and Butler (2011) concluded that retrieval practice serves as a powerful mnemonic enhancer, often leading to significant improvements in long-term retention relative to repeated studying. They argue that retrieval practice is often effective even without feedback, however, feedback enhances the benefits of testing.

While the studies discussed thus far established the effectiveness of retrieval learning in enhancing long term memory and deepening understanding, Storm, et al. (2010) argue that tests given immediately after exposure to content (as they are in many MOOCs at the end of videos) are nowhere near as effective as when tests are delayed. However, with delay comes the risk that information will be forgotten before the tests take place. The authors, therefore, suggest that initial testing should take place once information has been disseminated and should then be followed up with a series of delayed tests. Testing and spacing have the potential to enhance long-term retention of information (Bjork, 1999; Roediger and Karpicke, 2006) what is not clear is how the two manipulations can be most effectively combined (Storm, et al., 2010).
Mastery learning
It has been claimed that the online open course format provides an opportunity for students to engage in "Mastery Learning (ML)" (Koller, 2012). Mastery learning as first envisaged by Bloom (1968) allows students to achieve mastery of a concept before moving on to the next. This contrasts a more traditional approach of presenting material and concepts and moving everyone at the same pace regardless of their understanding. Bloom (1984) went further and argued that the introduction of ML could result in improvements of one standard deviation from the conventional group. Students who were tutored individually achieved a two standard deviation improvement in their performance on summative achievement scores. Bloom framed this as the 2 sigma problem, posing the challenge of making group teaching approach the outcomes of individual tutoring.

One approach to this is the provision of short videos that cover a concept in conjunction with quizzes that provide formative assessment. This method was adopted by Peter Norvig (2012) from Stanford University. Norvig developed his artificial intelligence class into a MOOC and within two weeks, 50,000 people had signed up. One student went on to comment: "This class felt like sitting in a bar with a really smart friend who's explaining something you haven't grasped, but are about to." Norvig was inspired by the work of Salman Khan of the Khan Academy (2012) whose videos emulate the one-on-one tutoring experience. Khan reports that improvements of 10-40 percent were achieved in a K-12 class in Los Altos after using the Khan Academy for mathematics instruction. There is unfortunately no formally published evidence for the effectiveness of the Khan Academy, or the use of short videos in enhancing student learning.

Other studies on ML have produced mainly positive results. A meta-analysis of 108 controlled evaluations showed mastery learning programs to have positive effects on examination performances of students in university, high school, and the upper grades in primary school (Kulik, et al., 1990). Indeed, the average effect of mastery learning was to raise student achievement scores by 0.52 standard deviations. Even larger improvements were reported by Guskey and Gates (1986) who claim effect sizes 0.65 at a college level. Walberg (1984) reports a mean effect size of 0.81 for "science mastery learning". In addition, Burrows and Okey (2006) found that when low-aptitude fourth graders were taught using mastery, they achieved results as high as the fifth graders who were taught in a more conventional manner.

The possible levels of engagement is also a feature of ML. Research conducted by Clark and colleagues (1983) found that the mastery group demonstrated higher levels of achievement than their peers trained in a typical lecture approach. The significantly fewer absences in the mastery learning group also suggested that, concomitant with their superior achievement, these students were more interested in their coursework. Aviles (2001) reported that while students in a junior-level introductory social work course did not perform better, they did prefer ML over non-ML.
Peer and self-assessment
A prominent feature of running a course with large numbers of students is the impossibility of providing marking and feedback that is not either automated or peer assessed. Automated marking provides instant feedback that can be combined with formative quizzes that enhance learning. There is also the possibility that peer and self-assessment may lead to enhanced learning outcomes. The principle concern with peer assessment is whether this can be adopted in the MOOC environment in a reliable and accurate way that approaches the accuracy of tutor or instructor marking. Initial data from a peer-assessed exam that was conducted for an introduction to sociology course on Coursera (Lewin, 2012) indicated a high degree of correlation between the average of five peer-assessed marks for a final exam and marks of the teaching staff.

Accuracy aside, there is less consensus regarding the learning benefits for students engaged in peer assessment. Sluijsmans and colleagues (2004) found there to be no difference in performance between two groups of students in the same college course, one who adopted peer assessment and the other that did not. In addition, Topping (1998) came to the rather vague conclusion that "peer assessment seems equally likely to contribute to or not contribute to the assessee's final official grade." [ 1] Similarly, Bloxham and West (2004) found no evidence of a relationship between a college student's ability to grade their peers and their own performance on an assessment. The vast majority of literature, however, does support the premise that there are additional learning benefits to the peer assessment process (Crooks, 1988; Falchikov, 2001; Lu and Law, 2012; Stiggins, 2002; Strijbos, et al., 2010; Topping, 1998).

Indeed, the literature reported many additional learning benefits for the peer assessor. Learning benefits can occur from exposure to other students' approaches, not to mention through access to the assessment marking criteria (Topping, 1998). Peer assessment is also said to develop a student's ability for self-learning, help them to recognise strengths and weaknesses, assist in developing professional skills and enhances reflective and critical thinking abilities (Sluijsmans, et al., 1998; Smith, et al., 2002; Topping, 1998). Nelson and Schunn (2009) argue that potential cognitive benefits of peer assessment arrive through summarisation, the identification of problems, localisation, and the provision of solutions. While Sadler and Good (2006) found no evidence of improved learning as a result of being a peer assessor, he did conclude that students who corrected their own work improved dramatically.

Self-assessment is said to facilitate greater autonomy in learning and is particularly effective in developing the self-learning skills (Boud and Falchikov, 1989) required for achievement in an online learning environment (Garrison, 2003). The cognitive benefits of self-assessment include improved understanding, performance, and ability for self-analysis (Gordon, 1992). There are also the long-term benefits of self-assessment. According to Oscarson (1989), we must not underestimate the importance of students being able to monitor and assess their own progress. The ability to self-assess is considered one of the most important skills that students require for effective and lifelong learning and for future professional development (Stefani, 1998; Taras, 2010).
Short format videos
Peter Norvig's (2012) decision to use short videos was inspired by Salman Khan's Khan Academy (2012). As mentioned earlier, the short video, interspersed with quizzes, emulates one-on-one tutoring. This draws on Bloom's (1984) findings that tutoring resulted in an average improvement of two standard deviations over teaching using standard lectures. With short videos, the students have the ability to control the pace, pause, rewind, explore and return to the content. They are unable to do this with standard lectures or with video recordings which may be one to two hours long. Videos are kept deliberately short following Khan's (2012) claim that videos of 10-15 minutes fit into an optimal period of time that students can maintain attention.
Online forums and video discussions
Online forums serve a number of roles. The first is as a mechanism for obtaining direct help with a problem, assessment or understanding of a concept (Darabi, et al., 2011). The second is as another mode of teaching to replace the face-to-face tutorial (Walker, 2007) and finally the online forum creates a space for exploring the subject matter, forming relationships and collaborating for project work and other assignments (Graham and Misanchuk, 2005). Forums play a vital role in online courses as they help establish a learning community through which learners generate knowledge (Li, 2004). Indeed, Thomas (2002) argues that students learn just as much from their interactions with each other as they do from the course materials.

Whether online forums and video discussions work in lieu of direct contact depends on which aspect is being considered. It is probably beyond question that the Internet provides an effective means of providing resources for problem solving and assistance. MOOC forums have illustrated (Young, 2012) that peer assistance is an effective way of dealing with student questions and issues. Koller (2012) discovered that more often than not students were responding to each other's posts before a moderator was able to. Due to the high volume of students enrolled in the course, the median response time for a forum question was 22 minutes. A service, Koller said, her Stanford students certainly do not receive. Enhancements of this process could be made in terms of promoting trusted students who show ability in assisting others.

Whether forums and online videoconferencing are an effective substitute for face-to-face tutorials is unclear. On the affirmative side, Cartwright (2000) found that the online format promoted excellent content-discussion and reflection in an online nursing course. In addition, the researchers/instructors found that the online discussion showed more student-initiated activity, higher quality, and better application of concepts than the face-to-face discussion. Indeed, their results indicated that the online discussion was so effective that subsequent offerings of this course used online discussion as the only method for case discussion. Jeong (2003) concluded that online discussion board postings at a major Midwestern university did exhibit strong evidence of the critical thinking necessary for higher learning and the creation of new knowledge (Walker, 2007) and Han and Hill (2007) concluded that online forums, when properly designed, can be a significant facilitator of collaborative discourse that leads to higher-level learning (Darabi, et al., 2011).

Against online discussion, Kanuka and Anderson (2007) found that most of the online interaction they reviewed from 25 corporate managers invited to participate proved merely to be an acquisition of information already compatible with existing knowledge. While the student's overall knowledge base increased, there was little evidence of new knowledge creation. Gustafson and Gibbs (2000) found that while individual student posts were of high quality and exhibited reflective thinking about the content, the students failed to actively engage one another's ideas (Walker, 2007). Criticisms regarding the ability of online forums to serve as a sufficient substitute for face-to-face discussion appear to be based on assumptions that higher learning occurs in face-to-face tutorials which may not always be the case.

While it was clear that the instructor plays an integral role in ensuring successful forum outcomes, there was less consensus regarding just how much of an online presence they should maintain. According to Mazzolini and Maddison (2007), the role of the instructor as moderator can vary from being the 'sage on the stage', to the 'guide on the side', to the 'ghost in the wings' [ 2]. Dysthe (2002) argues that the role of the instructor should be to intervene to motivate discussion and keep it on track. Guldberg and Pilkington (2007) argue, however, that if the tutor structures discussion and chooses questions carefully there may be less need to intervene to stimulate discussion or keep it on track than is sometimes assumed. This shifts the role of the tutor somewhat toward more preparatory and plenary work with less direct participation required to support the development of discussion skills amongst students, particularly in the later stages of the course.

As with many things related to online and face-to-face learning, successful outcomes of high levels of student engagement do not just happen by accident. Knowledge construction only occurs as a result of careful planning: clear, well-defined, well-crafted questions and discussion topics. Without such planning and subsequent guidance, only lower levels of cognitive engagement will occur (Andresen, 2009). This is the case in any format of teaching and is not restricted to online learning environments alone.
Conclusion
MOOCs are in essence a restatement of online learning environments that have been in use for some time. What is new is the numbers of participants, and the fact that the format concentrates on short form videos, automated or peer/self-assessment, forums and ultimately open content from a representation of the world's leading higher educational institutions. This review has demonstrated that MOOCs have a sound pedagogical basis for their formats. What we have not addressed however are the larger questions around whether taking a collection of MOOCs could replace obtaining an education on campus at a university in all of its facets of personal development and education. To many people taking MOOCs, this point is moot. They simply do not have the opportunity to attend a university in person. Either this is because of a lack of access to necessary prerequisite qualifications, geographical access, or financial means. To those that have the choice of either attending universities or undertaking a series of MOOCs that could one day represent a degree equivalence, the evidence suggests that their experience will not necessarily be any less rich in either case. What MOOCs present, however, is an opportunity to conduct educational research and examine the potential for use of its elements in on campus settings as a form of flipped classroom or blended learning approach. Whatever the outcomes, the nature of higher education will have changed as a result of this phenomenon.
About the authors
Associate Professor David Glance is director of the University of Western Australia (UWA) Centre for Software Practice, a UWA research and development centre. His research interests include open source software, technology and society and health informatics.
E-mail: david [dot] glance [at] uwa [dot] edu [dot] au

Associate Professor Martin Forsey teaches at the University of Western Australia and has written about neoliberal reform of schooling, school choice and supplementary education.
E-mail: martin [dot] forsey [at] uwa [dot] edu [dot] au

Myles Riley is a Research Assistant at the University of Western Australia.
Acknowledgements
The authors recognise the assistance given by the University of Western Australia in providing funding for this research.
Notes
1. Topping cited in Aoun, 2008, p. 1.

2. Mazzolini and Maddison, 2007, p. 194.
References
P.K. Agarwal, P.M. Bain, and R.W. Chamberlain, 2012. "The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist," Educational Psychology Review, volume 24, number 3, pp. 437-448. http://dx.doi.org/10.1007/s10648-012-9210-2

I.E. Allen and J. Seaman, 2005. Growing by degrees: Online education in the United States, 2005. Newburyport, Mass.: Sloan Consortium, at http://sloanconsortium.org/resources/growing_by_degrees.pdf, accessed 3 May 2013.

M.A. Andresen, 2009. "Asynchronous discussion forums: Success factors, outcomes, assessments, and limitations," Educational Technology & Society, volume 12, number 1, pp. 249-257, and at http://www.ifets.info/journals/12_1/19.pdf, accessed 3 May 2013.

C. Aoun, 2008. "Peer-assessment and learning outcomes: Product deficiency or process defectiveness?" Proceedings of the 34th International Association for Educational Assessment (IAEA) Conference, at http://www.cambridgeassessment.org.uk/ca/digitalAssets/154288_Aoun.pdf, accessed 3 May 2013.

Association for Learning Technology, 2012. "MOOC pedagogy: The challenges of developing for Coursera," Association for Learning Technology Newsletter, number 28, at http://newsletter.alt.ac.uk/2012/08/mooc-pedagogy-the-challenges-of-developing-for-coursera/, accessed 3 May 2013.

C.B. Aviles, 2001. "A study of mastery learning versus non-mastery learning instruction in an undergraduate social work policy class," Washington, D.C.: ERIC (Educational Resources Information Center), at http://www.eric.ed.gov/, accessed 3 May 2013.

T.J. Baker, 2012. "MOOC pedagogy: Theory & practice," Profesorbaker's ELT Blog (1 October), accessed at http://profesorbaker.com/2012/10/01/mooc-pedagogy-theory-practice/, accessed 3 May 2013.

K. Barker and T. Wendel, 2001. E-learning: Studying Canada's virtual secondary schools. Kelowna, B.C.: Society for the Advancement of Excellence in Education.

R.M. Bernard, P.C. Abrami, Y. Lou, E. Borokhovski, A. Wade, L. Wozney, P.A. Wallet, M. Fiset, and B. Huang, 2004. "How does distance education compare with classroom instruction? A meta-analysis of the empirical literature," Review of Educational Research, volume 74, number 3, pp. 379-439. http://dx.doi.org/10.3102/00346543074003379

R.A. Bjork, 1999. "Assessing your own competence: Heuristics and illusions," In: D. Gopher and A. Koriat (editors). Attention and performance XVII: Cognitive regulation of performance, interaction of theory and application. Cambridge, Mass.: MIT Press, pp. 435-459.

B.S. Bloom, 1984. "The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring," Educational Researcher, volume 13, number 6, pp. 4-16. http://dx.doi.org/10.3102/0013189X013006004

B.S. Bloom, 1968. Learning for mastery. Durham, N.C.: Regional Education Laboratory for the Carolinas and Virginia.

S. Bloxham and A. West, 2004. "Understanding the rules of the game: Marking peer assessment as a medium for developing students' conceptions of assessment," Assessment & Evaluation in Higher Education, volume 29, number 6, pp. 721-733. http://dx.doi.org/10.1080/0260293042000227254

D. Boud and N. Falchikov, 1989. "Quantitative studies of student self-assessment in higher education: A critical analysis of findings," Higher Education, volume 18, number 5, pp. 529-549. http://dx.doi.org/10.1007/BF00138746

C. Burrows and J.R. Okey, 2006. "The effects of a mastery learning strategy on achievement," Journal of Research in Science Teaching, volume 16, number 1, pp. 33-37. http://dx.doi.org/10.1002/tea.3660160106

J. Cartwright, 2000. "Lessons learned: Using asynchronous computer-mediated conferencing to facilitate group discussion," Journal of Nursing Education, volume 39, number 2, pp. 87-90.

C. Cavanaugh, K.J. Gillan, J. Kromrey, M. Hess, and R. Blomeyer, 2004. "The effects of distance education on K-12 student outcomes a meta-analysis," at http://faculty.education.ufl.edu/cathycavanaugh/docs/EffectsDLonK-12Students1.pdf, accessed 3 May 2013.

K. Cho, C.D. Schunn, and R.W. Wilson, 2006. "Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives," Journal of Educational Psychology, volume 98, number 4, pp. 891-901. http://dx.doi.org/10.1037/0022-0663.98.4.891

C.R. Clark, T.R. Guskey, and J.S. Benninga, 1983. "The effectiveness of mastery learning strategies in undergraduate education courses," Journal of Educational Research, volume 76, number 4, pp. 210-214.

Coursera, 2012a. "Course Explorer," at https://www.coursera.org/, accessed 19 November 2012.

Coursera, 2012b. "Coursera hits 1 million students across 196 countries," at http://blog.coursera.org/post/29062736760/coursera-hits-1-million-students-across-196-countries, accessed 3 May 2013.

T.J. Crooks, 1988. "The impact of classroom evaluation practices on students," Review of Educational Research, volume 58, number 4, pp. 438-481. http://dx.doi.org/10.3102/00346543058004438

A. Darabi, M.C. Arrastia, D.W. Nelson, T. Cornille, and X. Liang, 2011. "Cognitive presence in asynchrnous online learning: A comparison of four discussion strategies," Journal of Computer Assisted Learning, volume 27, number 3, pp. 216-227. http://dx.doi.org/10.1111/j.1365-2729.2010.00392.x

C. Doyle, 2012. "Professor Keith Devlin on teaching his first MOOC," Technapex (27 November), at http://www.technapex.com/2012/11/professor-keith-devlin-on-teaching-his-first-mooc/, accessed 3 May 2013.

O. Dysthe, 2002. "The learning potential of a Web-mediated discussion in a university course," Studies in Higher Education, volume 27, number 3, pp. 339-352. http://dx.doi.org/10.1080/03075070220000716

M. El-Tigi and R.M. Branch, 1997. "Designing for interaction, learner control, and feedback during Web-based learning," Educational Technology, volume 37, number 3, pp. 23-29.

ERIC, 2012. "Education Resources Information Center," at http://www.eric.ed.gov/, accessed 3 May 2013.

N. Falchikov, 2001. Learning together: Peer tutoring in higher education. New York: RoutledgeFalmer.

A. Fazackerley, 2012. "UK universities are wary of getting on board the mooc train," Guardian (3 December), at http://www.guardian.co.uk/education/2012/dec/03/massive-online-open-courses-universities, accessed 3 May 2013.

R. Gagné, 1985. The conditions of learning and theory of instruction. Fourth edition. New York: Holt, Rinehart and Winston.

D.R. Garrison, 2003. "Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction and metacognition," Elements of quality online education: Practice and direction, volume 4, pp. 47-58.

Google Scholar, 2012. "Google Scholar," at http://scholar.google.com/, accessed 9 December 2012.

M.J. Gordon, 1992. "Self-assessment programs and their implications for health professions training," Academic Medicine, volume 67, number 10, pp. 672-679. http://dx.doi.org/10.1097/00001888-199210000-00012

C.R. Graham and M. Misanchuk, 2005. "Computer-mediated learning groups," In: M. Khosrow-Pour (editor). Encyclopedia of Information Science and Technology, pp. 502-507.

K. Guldberg and R. Pilkington, 2007. "Tutor roles in facilitating reflection on practice through online discussion," Educational Technology & Society, volume 10, number 1, pp. 61-72, and at http://www.ifets.info/journals/10_1/ets_10_1.pdf, accessed 3 May 2013.

T.R. Guskey and S.L. Gates, 1986. "Synthesis of research on the effects of mastery learning in elementary and secondary classrooms," Educational Leadership, volume 43, number 8, pp. 73-81.

P. Gustafson and D. Gibbs, 2000. "Guiding or hiring? The role of the facilitator in online teaching and learning," Teaching Education, volume 11, number 2, pp. 195-210. http://dx.doi.org/10.1080/713698967

S.Y. Han and J.R. Hill, 2007. "Collaborate to learn, learn to collaborate: Examining the roles of context, community, and cognition in asynchronous discussion," Journal of Educational Computing Research, volume 36, number 1, pp. 89-123. http://dx.doi.org/10.2190/A138-6K63-7432-HL10

A.C. Jeong, 2003. "The sequential analysis of group interaction and critical thinking in online threaded discussions," American Journal of Distance Education, volume 17, number 1, pp. 25-43. http://dx.doi.org/10.1207/S15389286AJDE1701_3

E.H. Joy and F.E. Garcia, 2000. "Measuring learning effectiveness: A new look at no-significant-difference findings," Journal of Asynchronous Learning Networks, volume 4, number 1, pp. 33-39.

H. Kanuka and T. Anderson, 2007. "Online social interchange, discord, and knowledge construction," Journal of Distance Education, volume 13, number 1, pp. 57-74.

J.D. Karpicke and P.J. Grimaldi, 2012. "Retrieval-based learning: A perspective for enhancing meaningful learning," Educational Psychology Review, volume 24, number 3, pp. 401-418. http://dx.doi.org/10.1007/s10648-012-9202-2

J.D. Karpicke and J.R. Blunt, 2011. "Retrieval practice produces more learning than elaborative studying with concept mapping," Science, volume 331, number 6018 (20 January), pp. 772-775.

J.D. Karpicke and H.L. Roediger, 2008. "The critical importance of retrieval for learning," Science, volume 319, number 5865 (15 February), pp. 966-968.

J.D. Karpicke and H.L. Roediger, 2007. "Repeated retrieval during learning is the key to long-term retention," Journal of Memory and Language, volume 57, number 2, pp. 151-162. http://dx.doi.org/10.1016/j.jml.2006.09.004

S. Kerka, 1996. "Distance learning, the Internet, and the World Wide Web," at http://www.ericdigests.org/1997-1/distance.html, accessed 3 May 2013.

Khan Academy, 2012. "Khan Academy," at http://www.khanacademy.org/, accessed 3 May 2013.

S. Khan, 2012. The one world schoolhouse: Education reimagined. London: Hodder & Stoughton.

D. Koller, 2012. "What we're learning from online education," at http://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education.html, accessed 3 May 2013.

M. Korn and J. Levitz, 2013. "Online courses look for a business model," Wall Street Journal (1 January), at http://online.wsj.com/article/SB10001424127887324339204578173421673664106.html?mod=googlenews_wsj, accessed 3 May 2013.

R. Kozma, A. Zucker, C. Espinoza, R. McGhee, L. Yarnall, D. Zalles, and A. Lewis, 2000. "The online course experience: Evaluation of the Virtual High School's third year of implementation, 1999-2000," Menlo Park, Calif.: SRI International, at http://www.govhs.org/Images/SRIEvals/$file/SRIAnnualReport2000.pdf, accessed 3 May 2013.

C.-L.C. Kulik, J.A. Kulik, and R.L. Bangert-Drowns, 1990. "Effectiveness of mastery learning programs: A meta-analysis," Review of Educational Research, volume 60, number 2, pp. 265-299. http://dx.doi.org/10.3102/00346543060002265

R. Lapsley, B. Kulik, R. Moody, and J.B. Arbaugh, 2008. "Is identical really identical? An investigation of equivalency theory and online learning," Journal of Educators Online, volume 5, number 1, at http://www.thejeo.com/Archives/Volume5Number1/LapsleyetalPaper.pdf, accessed 3 May 2013.

T. Lewin, 2012. "College of future could be come one, come all," New York Times (19 November), at http://www.nytimes.com/2012/11/20/education/colleges-turn-to-crowd-sourcing-courses.html, accessed 3 May 2013.

Q. Li, 2004. "Knowledge building community: Keys for using online forums," TechTrends, volume 48, number 4, pp. 24-29. http://dx.doi.org/10.1007/BF02763441

J. Lu and N. Law, 2012. "Online peer assessment: Effects of cognitive and affective feedback," Instructional Science, volume 40, number 2, pp. 257-275. http://dx.doi.org/10.1007/s11251-011-9177-2

R.H. Maki, W.S. Maki, M. Patterson, and P.D. Whittaker, 2000. "Evaluation of a Web-based introductory psychology course: I. Learning and satisfaction in on-line versus lecture courses," Behavior Research Methods, Instruments, & Computers, volume 32, number 2, pp. 230-239. http://dx.doi.org/10.3758/BF03207788

M. Mazzolini and S. Maddison, 2007. "When to jump in: The role of the instructor in online discussion forums," Computers & Education, volume 49, number 2, pp. 193-213. http://dx.doi.org/10.1016/j.compedu.2005.06.011

J. McDonald, 2002. "Is 'as good as face-to-face' as good as it gets?" Journal of Asynchronous Learning Networks, volume 6, number 2, pp. 10-23, and at http://sloanconsortium.org/jaln/v6n2/quotas-good-face-facequot-good-it-gets, accessed 3 May 2013.

C.E. McKissack, 1997. "A comparative study of grade point average (GPA) between the students in traditional classroom setting and the distance learning classroom setting in selected colleges and universities," ETD Collection for Tennessee State University, paper AAI9806343, at http://digitalscholarship.tnstate.edu/dissertations/AAI9806343/, accessed 3 May 2013.

B. Means, Y. Toyama, R. Murphy, M. Bakia, and K. Jones, 2010. "Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies," Washington, D.C.: U.S. Department of Education, at http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf, accessed 3 May 2013.

J.P. Merisotis and R.A. Phipps, 1999. "What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education," Washington D.C.: Institute for Higher Education Policy, at http://www.ihep.org/Publications/publications-detail.cfm?id=88, accessed 3 May 2013.

R. Moe, 2012. "MOOC pedagogy - Waiting for big data?", at http://allmoocs.wordpress.com/2012/10/30/mooc-pedagogy-waiting-for-big-data/, accessed 3 May 2013.

M.M. Nelson and C.D. Schunn, 2009. "The nature of feedback: How different types of peer feedback affect writing performance," Instructional Science, volume 37, number 4, pp. 375-401. http://dx.doi.org/10.1007/s11251-008-9053-x

P. Norvig, 2012. "Peter Norvig: The 100,000-student classroom," at http://www.ted.com/talks/peter_norvig_the_100_000_student_classroom.html, accessed 3 May 2013.

T. Olson and R.A. Wisher, 2002. "The effectiveness of Web-based instruction: An initial inquiry," International Review of Research in Open and Distance Learning, volume 3, number 2, at http://www.irrodl.org/index.php/irrodl/article/view/103/182, accessed 3 May 2013.

S. Orn, 2012. "Napster, Udacity, and the Academy Clay Shirky," at http://www.kennykellogg.com/2012/11/napster-udacity-and-academy-clay-shirky.html, accessed 3 May 2013.

M. Oscarson, 1989. "Self-assessment of language proficiency: Rationale and applications," Language Testing, volume 6, number 1, pp. 1-13. http://dx.doi.org/10.1177/026553228900600103

PsycINFO, 2012. "PsycINFO," at http://www.apa.org/pubs/databases/psycinfo/index.aspx, accessed 3 May 2013.

H.L. Roediger and A.C. Butler, 2011. "The critical role of retrieval practice in long-term retention," Trends in Cognitive Sciences, volume 15, number 1, pp. 20-27. http://dx.doi.org/10.1016/j.tics.2010.09.003

H.L. Roediger and J.D. Karpicke, 2006. "The power of testing memory: Basic research and implications for educational practice," Perspectives on Psychological Science, volume 1, number 3, pp. 181-210. http://dx.doi.org/10.1111/j.1745-6916.2006.00012.x

T.L. Russell, 1999. The no significant difference phenomenon: As reported in 355 research reports, summaries and papers. Raleigh, N.C.: North Carolina State University.

P.M. Sadler and E. Good, 2006. "The impact of self- and peer-grading on student learning," Educational Assessment, volume 11, number 1, pp. 1-31. http://dx.doi.org/10.1207/s15326977ea1101_1

M. Shachar and Y. Neumann, 2003. "Differences between traditional and distance education academic performances: A meta-analytic approach," International Review of Research in Open and Distance Learning, volume 4, number 2, at http://www.irrodl.org/index.php/irrodl/article/view/153, accessed 3 May 2013.

D. Sluijsmans, F. Dochy, and G. Moerkerke, 1998. "Creating a learning environment by using self-, peer- and co-assessment," Learning Environments Research, volume 1, number 3, pp. 293-319. http://dx.doi.org/10.1023/A:1009932704458

D. Sluijsmans, S. Brand-Gruwel, J. van Merriënboer, and R.L. Martens, 2004. "Training teachers in peer-assessment skills: Effects on performance and perceptions," Innovations in Education and Teaching International, volume 41, number 1, pp. 59-78. http://dx.doi.org/10.1080/1470329032000172720

H. Smith, A. Cooper, and L. Lancaster, 2002. "Improving the quality of undergraduate peer assessment: A case for student and staff development," Innovations in Education and Teaching International, volume 39, number 1, pp. 71-81. http://dx.doi.org/10.1080/13558000110102904

L.A.J. Stefani, 1998. "Assessment in partnership with learners," Assessment & Evaluation in Higher Education, volume 23, number 4, pp. 339-350. http://dx.doi.org/10.1080/0260293980230402

R.J. Stiggins, 2002. "Assessment crisis: The absence of assessment for learning," Phi Delta Kappan, volume 83, number 10, pp. 758-765.

B.C. Storm, R.A. Bjork, and J.C. Storm, 2010. "Optimizing retrieval as a learning event: When and why expanding retrieval practice enhances long-term retention," Memory & Cognition, volume 38, number 2, pp. 244-253. http://dx.doi.org/10.3758/MC.38.2.244

J.-W. Strijbos, S. Narciss, and K. Dünnebier, 2010. "Peer feedback content and sender's competence level in academic writing revision tasks: Are they critical for feedback perceptions and efficiency?" Learning and Instruction, volume 20, number 4, pp. 291-303. http://dx.doi.org/10.1016/j.learninstruc.2009.08.008

J.J. Summers, A. Waigandt, and T.A. Whittaker, 2005. "A comparison of student achievement and satisfaction in an online versus a traditional face-to-face statistics class," Innovative Higher Education, volume 29, number 3, pp. 233-250. http://dx.doi.org/10.1007/s10755-005-1938-x

M. Taras, 2010. "Student self-assessment: Processes and consequences," Teaching in Higher Education, volume 15, number 2, pp. 199-209. http://dx.doi.org/10.1080/13562511003620027

M.J.W. Thomas, 2002. "Learning within incoherent structures: The space of online discussion forums," Journal of Computer Assisted Learning, volume 18, number 3, pp. 351-366. http://dx.doi.org/10.1046/j.0266-4909.2002.03800.x

K. Topping, 1998. "Peer assessment between students in colleges and universities," Review of Educational Research, volume 68, number 3, pp. 249-276. http://dx.doi.org/10.3102/00346543068003249

T. Traphagan, J.V. Kucsera, and K. Kishi, 2010. "Impact of class lecture webcasting on attendance and learning," Educational Technology Research and Development, volume 58, number 1, pp. 19-37. http://dx.doi.org/10.1007/s11423-009-9128-7

Udacity, 2012. "Udacity," at http://www.udacity.com, accessed 3 May 2013.

H.J. Walberg, 1984. "Improving the productivity of America's schools," Educational Leadership, volume 41, number 8, pp. 19-27, and at http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_198405_walberg.pdf, accessed 3 May 2013.

B.K. Walker, 2007. "Bridging the distance: How social interaction, presence, social presence, and sense of community influence student learning experiences in an online virtual environment," unpublished Ph.D. dissertation, University of North Carolina, at http://libres.uncg.edu/ir/uncg/f/umi-uncg-1472.pdf, accessed 3 May 2013.

Web of Knowledge, 2012. "Web of Knowledge," at http://apps.webofknowledge.com/, accessed 3 May 2013.

S.B. Wegner, K.C. Holloway, and E.M. Garton, 1999. "The effects of Internet-based instruction on student learning," Journal of Asynchronous Learning Networks, volume 3, number 2, pp. 98-106, and at http://sloanconsortium.org/jaln/v3n2/effects-internet-based-instruction-student-learning, accessed 3 May 2013.

J.R. Young, 2012. "Providers of free MOOC's now charge employers for access to student data," Chronicle of Higher Education (4 December), at http://chronicle.com/article/Providers-of-Free-MOOCs-Now/136117/, accessed 3 May 2013.
Editorial history
Received 27 January 2013; accepted 19 February 2013.

To the extent possible under law, David Glance has waived all copyright and related or neighboring rights to "The pedagogical foundations of massive open online courses". This work is published from Australia.

The pedagogical foundations of massive open online courses
by David George Glance, Martin Forsey, and Myles Riley.
First Monday, Volume 18, Number 5 - 6 May 2013
http://firstmonday.org/ojs/index.php/fm/article/viewArticle/4350/3673
doi:10.5210/fm.v18i5.4350

A Great Cities Initiative of the University of Illinois at Chicago University Library.

© First Monday, 1995-2018. ISSN 1396-0466.

# data[].attributes.attachment_link.title
The pedagogical foundations of massive open online courses | Glance | First Monday

# data[].attributes.attachment_link.description
The pedagogical foundations of massive open online courses In 2011, the respective roles of higher education institutions and students worldwide were brought into question by the rise of the massive open online course (MOOC). MOOCs are defined by signature characteristics that include: lectures formatted as short videos combined with formative quizzes; automated assessment and/or peer and self-assessment and an online forum for peer support and discussion.

# data[].attributes.headline
Active learning increases student performance in science, engineering, and mathematics (KG)

# data[].attributes.subject
Active learning increases student performance in science, engineering, and mathematics (KG)

# data[].attributes.body
A recent study has found that conventional lectures can be ineffective because they do not meet the expectations of modern students.

Even though it is not explored in this paper, this deficiency could potentially be addressed by online platforms, such as MOOCs.