Determining what works in the classroom
By Paul Clopton and Bill Evers, The San Diego Union-Tribune, September 8, 2004

LINK

Teachers would like to know what teaching practices and instructional materials would work best for their students. Every teacher has heard the phrases "studies show that ... " and "research-based program," but all too often teachers can't find usable research for their K-12 classrooms.

Advances based on the scientific method abound today – everywhere but in the K-12 classroom. Teachers need high-quality research into what works and what doesn't work, but that need is largely unfulfilled. For example, a 1998 report to the California Board of Education examined 8,727 published studies of mathematics education and found that only 110 met minimal validity criteria.

The importance of high-quality research – and the difficulty of finding the useful research needle in the mostly straw haystack of current studies – has prompted the U.S. Department of Education to provide a screening service in the form of the new What Works Clearinghouse Web site (see www.whatworks. ed.gov). The Clearinghouse chose peer-assisted learning (also known as "group learning") and middle-school mathematics as the first topics to investigate.

So far, the results confirm the perception that we have a long way to go in the scientific study of education. For the peer-learning category, 15 of 191 studies met Clearinghouse criteria. (Some 109 studies have yet to be screened.) For the mathematics category, one met Clearinghouse criteria, one was close (met with reservations), and 48 did not meet the standards (20 are yet to be screened). Worse yet, neither of the two acceptable mathematics studies showed any significant difference between methods.

Some argue that the standards used to evaluate studies are too stringent. A review of the reports for the 17 accepted studies quickly dispels this notion. They are small studies with a host of methodological weaknesses. If anything, the Clearinghouse may have engaged in a bit of grade inflation so that it didn't end up rejecting all the studies. Those who want the Clearinghouse to set lower standards are most likely the producers of poor-quality research or those who support weak programs for philosophical rather than evidence-based reasons.

Others fear that the Clearinghouse will go beyond the Department of Education's mandate by endorsing particular programs. However, the Clearinghouse only looks at the quality of studies, not the quality of the programs they address. Maybe this is a moot point; so far it looks as if the Clearinghouse won't be perceived as offering a strong endorsement for any program in the foreseeable future. What may be the strongest of the 17 accepted studies show no added benefit of the program being evaluated, and none of the studies comes close to providing any strong supporting evidence.

It is hoped that the Clearinghouse will encourage better research, which is one of the goals of the project. Teachers need educational research that provides good advice on which materials and methods are best for classroom use for specified academic objectives with various groups of students. Clearly, the message from the What Works Clearinghouse is that educational research has a long way to go.

What Works Clearinghouse

--------------------------------------------------------------------------------
Clopton is a medical research statistician for the U.S. Department of Veterans Affairs in San Diego. Evers is a research fellow at the Hoover Institution and a member of Hoover's Koret Task Force on K-12 Education.