New Study Questions Effectiveness of Online Learning

Is online education as good as traditional, face-to-face education?

. . . Since a Department of Education meta-analysis last summer concluded that “on average, students in online learning conditions performed better than those receiving face-to-face instruction,” many advocates now consider the matter closed.

Not so fast, say researchers at the National Bureau of Economic Research.

The Education Department’s study was deeply flawed and its implications have been overblown, say the authors of a working paper released this month by the bureau.

“None of the studies cited in the widely-publicized meta-analysis released by the U.S. Department of Education included randomly-assigned students taking a full-term course, with live versus online delivery mechanisms, in settings that could be directly compared (i.e., similar instructional materials delivered by the same instructor),” they write. “The evidence base on the relative benefits of live versus online education is therefore tenuous at best.”

In spring 2007, they randomly assigned 327 volunteers enrolled in an introductory microeconomics course to either attend the class lectures live or watch them online. Both groups would have access to the same ancillary materials and access to office hours and graduate assistants; the only difference would be the mode of lecture delivery.

They found no statistically significant differences between the academic performances of the two groups generally. However, they did find that Hispanic students, male students, and low-achieving students in the online group fared significantly worse than their counterparts in the live-attendance group.

These findings do not exactly refute the conclusions of the Education Department’s meta-analysis. Nor is the new study without flaws of its own, which the authors enumerate in detail — though not the most obvious, which is that videotaped lectures are a relatively primitive form of online teaching, and, where they are used, are usually only part of the package.

But Rush says the main takeaway of the bureau’s experiment is not that he and his co-authors are right or that the Education Department’s study was wrong; just that there is much more work — much more precise work — to be done before any firm pronouncements can be made on the merits of online education relative to the face-to-face kind.

Read the full article.