What Do You Think?
The Dangerous Consequences of High-Stakes Testing, and Other Concerns
The American School Board Journal looks at the issue of testing and assessment:
As most of us know by now, the No Child Left Behind Act -- the 2001 reauthorization of the federal Elementary and Secondary Education Act -- mandates a massive increase in state assessments, which mostly means standardized tests. We also know that the stakes attached to those tests are high. In many cases, the new federal law will seriously disrupt state and local testing programs and cause similar disruptions in schools and classrooms as they try to comply with the regulations.
It would be worth enduring these difficulties if we could be reasonably sure that the test-driven changes would produce improved learning opportunities and outcomes, particularly for the low-income students. Unfortunately, evidence and reason argue they will not. Indeed, the law misdirects the reform process while taking the power to make educationally useful changes away from school board members and other local and state elected officials.
So far, attention has been focused on the requirement that districts provide transfers, with transportation, from schools that failed to post required annual score gains (called "adequate yearly progress" or AYP) to other schools that have made the required gains. Soon, however, many additional schools will face more severe sanctions. Some researchers, including Robert Linn and Eva Baker, conclude that over the next decade the vast majority of schools receiving Title I money will see the most severe penalties: wholesale staff firings, state takeovers, private management takeovers, or transformation into public charter schools.
What is it about adequate yearly progress that makes so many schools, and even whole districts, likely to suffer these draconian penalties? More important, what are the likely consequences for students in those schools -- children who are mostly low income, disproportionately African American or Latino, or have limited English proficiency or disabilities? Finally, what can be done about the harmful consequences that will flow from the federally mandated emphasis on testing?
The problem with AYP
Under No Child Left Behind, states are required by 2006 to assess all students annually in reading and math in grades three through eight and once in high school. The law does not specifically mandate standardized tests, so a few states plan to use local assessments, including classroom-based information, rather than state exams. This opens up the possibilities of strengthening teachers' assessment capabilities and ensuring far richer information than can be obtained through one-time tests. However, the complexities of ensuring that a variety of local and classroom assessments all accurately measure student progress in light of state standards, as required, will be difficult.
Although building an accountability system based on classroom assessments makes more educational sense, most states will find it easier and less expensive to rely on standardized tests to meet the law's requirement. These tests are mostly multiple-choice in format, though many states include some short answer or brief essay items.
Test scores are the near-exclusive means for determining adequate yearly progress. According to law, almost all students must reach the proficient level on state assessments by 2014. Adequate yearly progress is determined by a formula intended to ensure steady advancement toward the goal. States typically will need to move an additional 4 to 6 percent of their students into the proficient category every year for the next 12 years. Yet, using the National Assessment of Educational Progress tests as the measure, only three of 33 states made even 1 percent gains in reading per year from 1992 to 1998.
The law also requires racial minorities, low-income students, and students with limited English proficiency or disabilities to close the gap with their higher-scoring peers. This is a laudable aim, but studies such as the one by Thomas Kane and Douglas Staiger have found that racially integrated schools will be more likely not to make adequate yearly progress simply because they have more groups of students who can fall short.
Schools serving mostly low-income communities start with students who are less academically ready, have greater social needs, and receive less academic support at home. Typically, these schools face a number of challenges. They have fewer qualified teachers, larger class sizes, fewer library books, little technology, and dilapidated buildings. With most states cutting budgets and President Bush proposing no increase in federal assistance, support for low-income families and the schools they attend is likely to decrease as the demands for score gains intensify.
An overemphasis on testing will undermine, not strengthen, the ability of schools to ensure a high-quality academic and social experience for all their students. That's what happens when you combine the limitations of standardized tests with unreasonable pressures for schools without the resources to solve long-term societal inequities.
Surveys conducted by Public Agenda, the American Association of School Administrators, and others show that parents want far more than just solid academics for their children. They want a supportive and engaging social and educational environment. They want their children to have opportunities to learn many things, including, but not limited to, the "basics." They prize having students learn to participate in a democracy. Yet, under the pressure of high-stakes testing, school boards and administrators are scrapping recess, cutting art, short-changing history, and otherwise narrowing their students' school experience to make room for more test prep in math or reading.
Even in the tested subjects, curriculum and instruction are often limited to fit the exam. In some cases, children are taught to read by learning to look at the answer options to multiple-choice questions and then search the short reading passage to find the clue to selecting the wanted answer. Independent evaluators find that these children cannot explain what they just read, though they got the test item right. Parents and teachers report that many children don't understand that text might be longer than a paragraph or two, because that's the length of passages on tests and in the test prep that passes for a curriculum in their schools.
Many things students learn simply cannot be tested with a paper-and-pencil test. In a high-quality education, students conduct science experiments, solve real-world math problems, write research papers, read and analyze novels and stories, deliver oral presentations, evaluate and synthesize information from a variety of fields, and apply their learning to new situations. In fact, standardized tests largely cannot evaluate these important kinds of learning. If instruction focuses on the test, students will not learn these skills, which are needed for success in college and in life.
Diagnosis or snapshot?
The No Child Left Behind Act requires assessments to provide diagnostic information -- another laudable goal that will be difficult to meet. Assessments of educational strengths and weaknesses are valuable at the individual, classroom, school, or district level. However, information needs to be sufficiently timely for the kind of diagnosis being made. The lengthy turnaround time for scoring most standardized tests makes them nearly useless for helping a particular individual, though the information might be of use for long-range planning.
In addition, standardized tests include only a few questions on any particular topic. They provide too little information to produce accurate, comprehensive, or detailed results or analysis. Since many topics in state standards are not addressed in state exams, the tests provide no diagnostic information about them.
Diagnosis suggests the use of assessment for learning -- assessment that can help a teacher and student know what to do next. In a major review of the research, Paul Black and Dylan William found that well-crafted assessments can make a larger contribution to student achievement than any other commonly measured factor. Low-achieving students benefit most, thereby closing learning gaps. Standardized tests administered at the end of the year cannot possibly meet this need.
Sound diagnostic practices also include determining why a student is succeeding or having difficulty and taking appropriate action. As snapshots with limited information, standardized tests provide neither an answer to "why" nor meaningful guidance for successful instruction. Worse, under the pressure to raise scores classroom assessment tends to simply resemble standardized tests, leaving teachers with too little information to respond to individual needs.
Test validity resides in the inferences drawn from assessment results and the consequences of their uses. Relying solely on scores from one test to determine success or progress in broad areas such as reading or math is likely to lead to incorrect inferences and then to actions that are ineffective or even harmful. For this reason, the standards for the measurement profession warn against using the results of any single test as the basis for making major decisions at the classroom, school, and district level.
Consequences for poor students
As educators respond to the pressure to raise scores, schooling for many children will be reduced to preparation for narrow, limited, mostly low-level tests. Already, many schools -- particularly low-scoring ones -- are teaching only those subjects that appear on the test and dropping untested ones. Children from poor families will receive this highly limited schooling. The children of the affluent who attend higher-scoring schools will continue to receive an education that prepares them for college, high-paying careers, and significant social and political influence. Sadly, when score gaps between poor and affluent schools close, some people will be fooled into thinking that education is being equalized. In reality, lower-income students will rarely be taught the many attributes for success in college and life that cannot be measured by tests.
Some test proponents argue that teaching to the test will ensure that children from low-income families learn at least the basics. Unfortunately, that is not always the case. Students who learn to fill in the correct bubble but cannot really read, or who have been trained to write canned responses to a prompt but cannot write on their own, have not learned the basics. This argument means giving up on many children -- in the name of higher standards. It says this society will not provide all children with an adequate, never mind equal, opportunity to learn.
Another documented consequence of high-stakes testing is that more students will be retained a grade or drop out. Compared with similar students who are promoted, those who are held back do not improve academically, are emotionally damaged by retention, suffer a loss of interest in school and self-esteem, and are more likely to drop out. Promotion with intensive support and academic acceleration is what most low-performing students need.
A comprehensive national study by Brian Jacob finds that graduation tests lead to a higher dropout rate for students who are relatively low achievers and do not produce improved learning for those who stay in school. Other studies find high-stakes testing does not lead to improved learning, even as measured by other tests.
In addition, good teachers are often discouraged by the overemphasis on testing and leave. Many flee from low-scoring schools, which especially need strong teachers. The "best and brightest" are not likely to become teachers when teaching is reduced to test prep and when schools are continually attacked by politicians, business leaders, and the media.
We know it is not necessary to focus on tests in order to provide the basics and beyond. Deborah Meier, who has founded schools that successfully serve inner-city students, points out that low-income students from her schools have learned to succeed in college -- even if their test scores do not go up dramatically. Leaders of similar schools, such as Ann Cook of New York's Urban Academy and Linda Nathan of the Boston Arts Academy, strongly affirm this point. Independent studies of networks of such schools in New York City and Boston found that these schools, which do not teach to standardized tests, succeed better on a range of measures (including test scores) than regular schools serving similar students.
What does this mean? First, standardized tests fail to measure many important academic and nonacademic attributes that students need to succeed. Second, children from low-income backgrounds can succeed academically at far more than the basics if they attend schools that help them do so. Third, avoiding a drill-and-kill focus on testable basics allows time for deep learning that provides a solid foundation for success in academics and in life. As Meier writes in her new book, In Schools We Trust, "Creating a culture in which all kids use their minds powerfully is well within our reach. ... Resorting to flawed standardized testing ... is both unnecessary and counterproductive to such ends."
What can be done?
My organization, the National Center for Fair & Open Testing, recommends several steps to pave the way to changing both the law and educational practices in schools serving poor kids:
* First, districts and states must emphasize assessment for learning. This requires professional development for teachers. It also means time for teachers to share knowledge and explore questions together -- precisely the kind of professional development research says is most productive. Schools and districts will have to work hard to overcome the tendency to let standardized tests crowd out better assessments. The payoff will be truly improved learning.
* Second, districts need to implement new forms of accountability that include far richer measures, both academic and social, than test scores. Teachers and parents should come together at the school and district levels to talk about what they want their schools to do, how to ensure it is done, and what steps to take if it is not. This presumes collaboration, support, and openness -- a far cry from test-based accountability. Some of this can be done under No Child Left Behind, and some states are taking good steps toward more authentic accountability. But the weight of No Child Left Behind pushes heavily in the wrong direction.
* Finally, if change is to happen, people must organize. In Massachusetts, an overwhelming majority of school committees voted for a moratorium on the state graduation testing requirement until it includes multiple measures, not just one test. Some school boards are voting to confer diplomas on students who meet their local requirements, regardless of state test scores -- a position the state Department of Education opposes. The teachers unions publicly oppose high-stakes testing and support richer accountability practices. Parents have organized the statewide Coalition for Authentic Reform in Education (CARE), which has worked with the unions and school committees to educate the public and change policy. Among other things, CARE has crafted a model accountability system.
Most of all, if the nation is to leave no child behind, we must comprehensively address poverty and its consequences and support higher-quality educational practices. We must use far more than standardized test scores to determine whether schools are improving and students are learning. Continuing the course of high-stakes testing will only deepen the crisis in schools serving our most vulnerable children and wreak terrible consequences on their communities.
Monty Neill is the executive director of the National Center for Fair & Open Testing (FairTest) in Cambridge, Mass.