How Schools Cheat
From underreporting violence to inflating graduation rates to fudging testscores, educators are lying to the American public.
Lisa Snell, the Reason Foundation
LINKOn March 17, 2005, 15-year-old Delusa Allen was shot in the head while leaving Locke High School in Los Angeles, sending her into intensive care and eventually killing her. Four months before that several kids were injured in a riot at the same school, and last year the district had to settle a lawsuit by a student who required eye surgery after he was beaten there. In 2000, 17-year-old Deangelo Anderson was shot just across the street from Locke; he lay dead on the sidewalk for hours before the coroner came to collect his body.
Violent crime is common at Locke. According to the Los Angeles Police Department, in the 2003-–04 school year its students suffered three sex offenses, 17 robberies, 25 batteries, and 11 assaults with a deadly weapon. And that's actually an improvement over some past years: In 2000–01 the school had 13 sex offenses, 43 robberies, 57 batteries, and 19 assaults with a deadly weapon.
Sounds unsafe, doesn't it? Not in the skewed world of official education statistics. Under the federal No Child Left Behind Act, states are supposed to designate hazardous schools as "persistently dangerous" and allow their students to transfer to safer institutions. But despite Locke's grim record, the state didn't think it qualified for the label.
Locke is not unique. In the 2003–04 school year only 26 of the nation's 91,000 public schools were labeled persistently dangerous. Forty-seven states and the District of Columbia proudly reported that they were home to not a single unsafe school. That would be news to the parents of James Richardson, a 17-year-old football player at Ballou Senior High in Southeast Washington, D.C., who was shot inside the school that very year. It would be news to quite a few people: The D.C. Office of the Inspector General reports that during that school year there were more than 1,700 "serious security incidents" in city schools, including 464 weapons offenses.
Most American schools are fairly safe, it's true, and the overall risk of being killed in one is less than one in 1.7 million. The data show a general decline in violence in American public schools: The National Center for Education Statistics' 2004 Indicators of School Crime and Safety shows that the crime victimization rate has been cut in half, declining from 48 violent victimizations per 1,000 students in 1992 to 24 in 2002, the last year for which there are complete statistics.
But that doesn't mean there has been a decline at every school. Most of the violence is concentrated in a few institutions. According to the National Center for Education Statistics, during the 1999–2000 school year 2 percent of U.S. schools (1,600) accounted for about 50 percent of serious violent incidents-and 7 percent of public schools (5,400) accounted for 75 percent of serious violent incidents. The "persistently dangerous" label exists to identify such institutions.
So why are only 26 schools in the country tagged with it?
The underreporting of dangerous schools is only a subset of a larger problem. The amount of information about schools presented to the general public is at an all-time high, but the information isn't always useful or accurate.
Thanks to the No Child Left Behind Act, now three years old, parents are seeing more and more data about school performance. Each school now has to give itself an annual report card, with assessment results broken down by poverty, race, ethnicity, disability, and English-language proficiency. Schools also are supposed to accurately and completely report dropout rates and teacher qualifications. The quest for more and better information about school performance has been used as a justification to increase education spending at the local, state, and national levels, with the federal Department of Education alone jacking up spending to nearly $60 billion for fiscal year 2005, up more than $7 billion since 2003.
But while federal and state legislators congratulate themselves for their newfound focus on school accountability, scant attention is being paid to the quality of the data they're using. Whether the topic is violence, test scores, or dropout rates, school officials have found myriad methods to paint a prettier picture of their performance. These distortions hide the extent of schools' failures, deceive taxpayers about what our ever-increasing education budgets are buying, and keep kids locked in failing institutions. Meanwhile, Washington-which has set national standards requiring 100 percent of school children to reach proficiency in math and reading by 2014-has been complicit in letting states avoid sanctions by fiddling with their definitions of proficiency.
The federal government is spending billions to improve student achievement while simultaneously granting states license to game the system. As a result, schools have learned to lie with statistics.
Prospering Cheaters
Under No Child Left Behind, if schools fail to make adequate yearly progress on state tests for three consecutive years, students can use federal funds to transfer to higher-performing public or private schools, or to obtain supplemental education services from providers of their choice. In addition, schools that fail for four to five consecutive years may face state takeovers, have their staffs replaced, or be bid out to private management.
Wesley Elementary in Houston isn't a school you'd expect to be worried about those threats. From 1994 to 2003, Wesley won national accolades for teaching a majority of its low-income students how to read. Oprah Winfrey once featured it in a special segment on schools that "defy the odds," and in 2002 the Broad Foundation awarded the Houston Independent School District a $1 million prize for being the best urban school district in America, largely based on the performance of schools like Wesley.
It turned out that Oprah was righter than she realized: Wesley was defying the odds. A December 31, 2004, exposé by The Dallas Morning News found that in 2003 Wesley's fifth-graders performed in the top 10 percent in the state on the Texas Assessment of Knowledge and Skills (TAKS) reading exams. The very next year, as sixth-graders at Houston's M.C. Williams Middle School, the same students fell to the bottom 10 percent.
The newspaper obtained raw testing data for 7,700 Texas public schools for 2003 and 2004. It found severe statistical anomalies in nearly 400 of them. The Houston, Dallas, and Fort Worth districts are now investigating dozens of their schools for possible cheating on the TAKS test. Fort Worth's most suspicious case was at A.M. Pate Elementary. In 2004, Pate fifth-graders finished in the top 5 percent of Texas students. In 2003, when those same students were fourth-graders, they had finished in the bottom 3 percent.
In the Winter 2004 issue of Education Next, University of Chicago economist Steven D. Levitt and Brian A. Jacob of Harvard's Kennedy School of Government explored the prevalence of cheating in public schools. Using data on test scores and student records from the Chicago public schools, Jacob and Levitt developed a statistical algorithm to identify classrooms where cheating was suspected. Their sample included all student test scores in grades 3–7 for the years 1993 to 2000. The final data set contained more than 40,000 "classroom years" of data and more than 700,000 "student year" observations. Jacob and Levitt's analysis looked for unexpected fluctuations in students' test scores and unusual patterns of answers for students within a classroom that might indicate skullduggery.
They found that on any given test the scores of students in 3 percent to 6 percent of classrooms are doctored by teachers or administrators. They also found some evidence of a correlation of cheating within schools, suggesting some centralized effort by a counselor, test coordinator, or principal. Jacob and Levitt argue that with the implementation of the No Child Left Behind Act, the incentives for teachers and administrators to manipulate the results from high-stakes tests will increase as schools begin to feel the consequences of low scores.
Texas' widespread cheating likely was a response both to high-stakes testing and to financial incentives for raising test scores. The Houston school district, for example, spends more than $7 million a year on performance bonuses that are largely tied to test scores. Those bonuses include up to $800 for teachers, $5,000 for principals, and $20,000 for higher-level administrators.
Texas is not the only state where schools have cheated on standardized tests. Teachers provided testing materials to students nearly a dozen times in 2003 in Nevada, for example. And Indiana has seen a raft of problems, including three Gary schools that were stripped of their accreditation in 2002 after hundreds of 10th-graders received answers for the Indiana Statewide Testing for Education Progress–Plus in advance. A teacher in Fort Wayne took a somewhat subtler approach in 2004, when school officials had to throw out her third-grade class's scores after she gave away answers by emphasizing certain words on oral test questions. In January 2005 another Fort Wayne third-grade teacher was suspended for tapping children on the shoulder to indicate a wrong answer.
Phantom Dropouts
If you want to make a school's performance look more impressive than it really is, you don't have to abet cheating on standardized tests. Instead you can misrepresent the dropout rate.
In 2003 The New York Times described an egregious example of this scam in Houston. Jerroll Tyler was severely truant from Houston's Sharpstown High School. When he showed up to take a math exam required for graduation, he was told he was no longer enrolled. He never returned.
So Tyler was surprised to learn, when the state audited his high school, that Sharpstown High had zero dropouts in 2002. According to the state audit of Houston's dropout data, Sharpstown reported that Tyler had enrolled in a charter school-an institution he had never visited, much less attended. The 2003 state audit of the Houston district examined records from 16 middle and high schools, and found that more than half of the 5,500 students who left in the 2002 school year should have been declared dropouts but were not.
The Manhattan Institute's Jay P. Greene argues, in his 2004 paper "Public School Graduation Rates in the United States," that "this problem is neither recent nor confined to the Houston school district....Official graduation rates going back many years have been highly misleading in New York City, Dallas, the state of California, the state of Washington, several Ohio school districts, and many other jurisdictions." Administrators, he explains, have strong incentives to count students who leave as anything other than dropouts. Next to test scores, graduation rates are an important measure of a school's performance: If parents and policy makers believe a school is producing a high number of graduates, they may not think reform is necessary. Greene writes that "when information on a student is ambiguous or missing, school and government officials are inclined to say that students moved away rather than say that they dropped out."
Greene and his associates have devised a more accurate method for calculating graduation rates. Simplifying a bit, it essentially counts the number of students enrolled in the ninth grade in a particular school or jurisdiction, makes adjustments for changes in the student population, and then counts the number of diplomas awarded when those same students leave high school. The percentage of original students who receive a diploma is the true graduation rate.
Using Greene's methodology, the national high school graduation rate for 2002 was 71 percent. Yet according to the National Center for Education Statistics, in 2002 the national high school "completion rate," defined as the percentage of adults 25 and older who had completed high school, was 85 percent. As Greene notes, "There were a total of 3,852,077 public school ninth-graders during the 1998–99 school year. In 2001–02, when that class was graduating, only 2,632,182 regular high school diplomas were distributed. Simply dividing these numbers produces a (very rough) graduation rate estimate of 68%." The states show similar discrepancies between their reported graduation rates and the number of students who actually receive diplomas.
As Sharpstown High School's former assistant principal, Robert Kimball, told The New York Times, "We go from 1,000 Freshman [sic] to less than 300 Seniors with no dropouts. Amazing!"
The problem isn't limited to Texas. In March researchers at Harvard's Civil Rights Project released an analysis of state graduation rates for 2002, in which they derived their figures by counting the number of students who move from one grade to the next and then on to graduation. The report found serious discrepancies between the rates calculated by the Civil Rights Project and those offered by education departments in all 50 states. In California, for example, the state reported an 83 percent graduation rate, but the Harvard report found that only 71 percent of students made it through high school.
The Civil Rights Project's paper also found a high dropout rate among minorities, which California officials hides behind state averages. Almost half of the Latino and African-American students who should have graduated from California high schools in 2002 failed to complete their education. In the Los Angeles Unified School District, just 39 percent of Latinos and 47 percent of African Americans graduated, compared with 67 percent of whites and 77 percent of Asians.
Moving the Goalposts on Proficiency
A subtler way to distort data is to report test scores as increasing when in fact more students have been excluded from taking the test. One egregious example of this practice took place in Florida, which grades schools from F to A based on their standardized test scores. Oak Ridge High School in Orlando boosted its test scores from an F to a D in 2004 after purging its attendance rolls of 126 low-performing students.
The students were cut from school enrollment records without their parents' permission, a violation of state law. According to the Orlando Sentinel, about three-quarters of the students had at least one F in their classes, and 80 percent were ninth- or 10th-graders-a key group, because Florida counts only the scores of freshmen and sophomores for school grades. More than half of the students returned to Oak Ridge a few weeks after state testing.
The Sentinel also reported that in 2004 some 160 Florida schools assigned students to new schools just before standardized testing in a shell game to raise school grades. In Polk County, for example, 70 percent of the students who were reassigned to new schools scored poorly on Florida's Comprehensive Assessment Test, suggesting they were moved to avoid giving their old schools a bad grade.
Florida is not alone. In a third of Houston's 30 high schools, scores on standardized exams have risen as enrollment has shrunk. At Austin High, for example, 2,757 students were enrolled in the 1997–98 school year, when only 65 percent passed the 10th-grade math test. Three years later, 99 percent of students passed the math exam, but enrollment had shrunk to 2,215 students. The school also reported that dropout figures had plummeted from 4.1 percent to 0.3 percent. Rather than a sudden 20 percent drop in enrollment, the school had used a strategy of holding back low-scoring ninth-graders and then promoting them directly to 11th grade to avoid the 10th-grade exam.
States are also excluding a higher percentage of disabled students and students for whom English is a second language. (Needless to say, these exclusion rates are not reported with the test score data.) And states often report that their test scores are going up when they've merely dumbed-down their standards by changing the percentage of correct responses necessary to be labeled "proficient" or by changing the content of the tests to make them easier. Of the 41 states that have reported their 2004 No Child Left Behind test results so far, 35-including all of the states showing improvement-had schools meet the targets not by improving the schools but by amending the rules that determine which schools pass and which fail.
For example, the Philadelphia Inquirer reported last October that Pennsylvania's "improvements" were a result of lower standards, not improved performance. These changes, approved by the federal government, allowed schools with lower graduation rates, lower standardized test scores, or lower attendance than in previous years to win passing marks. In 2004, 81 percent of the state's schools met No Child Left Behind's adequate yearly progress benchmarks using the new standards. But the Inquirer analysis found that if the same rules used in 2003 had been used in 2004, the number of schools falling short of the yearly benchmark would have grown from 566 to 1,164. Instead of 81 percent meeting the benchmark, just 61 percent would have succeeded. When the Pennsylvania Education Department announced in August that only 566 of 3,009 public schools failed to meet federal standards, it neglected to mention the role the rule changes played in the "significant gains" made.
This sort of thing has been going on for a while. Back in 2002 Education Week reported that "a number of states appear to be easing their standards for what it means to be 'proficient' in reading and math because of pressures to comply with a new federal law requiring states to make sure all students are proficient on state tests in those subjects within 12 years. In Louisiana, for instance, students will be considered proficient for purposes of the federal law when they score at the 'basic' achievement level on their state's assessment. Connecticut schoolchildren will be deemed proficient even if they fall shy of the state's performance goals in reading and mathematics. And Colorado students who score in the 'partially proficient' level on their state test will be judged proficient."
The federal government actually gives a seal of approval to states that are lowering the standards they had before Bush's era of "accountability." For example, the U.S. Department of Education allowed Washington state to lower its high school graduation rate from 73 percent to 66 percent and still meet No Child Left Behind requirements-with the promise of an 85 percent graduation rate by 2014. Apparently, the feds are spending billions to compel states to reduce their academic standards.
Lying by Omission
But the most common way school data deceive people is through omission. State and local education officials simply do not define their terms for the media or the general public. As we've already seen, "persistently dangerous" doesn't mean the same thing to officials that it means to you and me.
Another example: My local newspaper lists area schools that have met No Child Left Behind goals and are compliant with federal law. The article will tell you that every subgroup, from low-income children and Hispanics to special education children, is proficient in reading and in math. It will not say that in California, in order for yearly progress for each subgroup to be considered adequate, only 13 percent of the children in each group must be proficient. Imagine the difference-and how much more helpful it would be to a concerned parent trying to decide what is best for her child-if the newspaper article said, "Here is a list of schools where at least 13 percent of children in each group are proficient."
The newspaper should also explain what it really means to be "proficient" in reading. To be considered proficient for the third grade in California, you must score at the 51st percentile in reading and the 63rd percentile in math on California's standardized STAR test. In other words, all it really means when my school is listed as meeting "adequate yearly progress" under No Child Left Behind is that at least 13 percent of third-graders in every subgroup scored at the 51st percentile on the reading test.
Most parents assume that "proficiency" means grade-level performance. But proficiency standards are so different from state to state that students with the same skills will have very different proficiency rates. In third-grade reading, for example, Texas sets its cut score-the correct number of responses or percentile ranking a student needs to be considered proficient-at the 13th percentile. Nevada sets its cut score at the 58th percentile.
All this only scratches the surface of the ways schools use statistics to mislead parents and the public. From reporting teachers' salaries without including benefits as part of their compensation to reporting per-pupil spending while excluding billions in spending on school buildings and infrastructure, the list of deceptions goes on and on.
The No Child Left Behind Act was supposed to let parents and policy makers identify and fix failing schools. More important, it was supposed to give kids a right of exit out of failing or dangerous institutions. But that's meaningless if "failing" and "dangerous" can be defined away. Despite the violence at Locke High School, the teaching failures at Wesley Elementary School, and the high dropout rates at Sharpstown High School, the average kid in those institutions is no closer to escaping now than before the law was passed. And despite the glut of information being offered to parents-and the glut of dollars being spent on education-most families rarely see the facts about their schools' performance.
No Child Left Behind was sold as a way to make the schools more accountable. Instead, it has encouraged and abetted them as they distort the data and game the system. That may be the worst deception of all.
Lisa Snell is director of the Reason Foundation's education program.
The Reason Foundation Without facts and oversight, money can disappear. We do not know what services are being provided our children on a daily basis, and school boards are elected - outside of New York City, where school boards no longer exist - based upon the individuals' political connections and ability to keep doors closed to public eyes, it seems. Who provides the oversight?
June 19, 2005
After the Roslyn Mess, Who Audits Schools?By FAIZA AKHTAR, NY TIMES
LINK IF Roslyn was, perhaps, the Enron of Long Island school districts, then Miller, Lilly & Pearce would seem to be the Arthur Andersen.
In the small world of accounting firms that specialize in auditing school districts, Miller, Lilly was tops on the Island, handling 50 of the 124 districts, including wealthy, high-achieving Roslyn. When the Roslyn district erupted in a financial scandal last year, the state comptroller's office investigated where the district's money had been going and how carefully Miller, Lilly had been checking the books. The results were devastating.
After the comptroller, Alan G. Hevesi, issued a report saying that shoddy, unprofessional auditing work by Miller, Lilly was a major reason that apparent embezzlement had gone undetected in Roslyn for years, the firm became radioactive. Other districts moved quickly to dismiss it and hire new auditors. Though Miller, Lilly was not charged with any wrongdoing, its business was rapidly drying up; in April, it shut down.
Fast-forward to this month. Indictments are filed against Roslyn's former superintendent, his roommate, the district's former business manager and her niece. Smaller cases of suspected fraud and embezzlement have come to light in three other districts. The comptroller's office is spot-checking the books of one Long Island district after another, and often finding sloppiness and lapses in procedure, though so far not any more criminal acts. School officials throughout the Island are moving to put in place the tighter checks and controls that the comptroller has recommended, including internal audits in addition to the external audits that state law requires.
Seen another way, though, little has changed. Most districts on the Island are still audited by the same few specialist accounting firms, one of which has hired two former Miller, Lilly employees to help with the work. Many districts still use financial software made by a company controlled by two Miller, Lilly partners that remains in business. And the internal auditing at many school districts is also being done by one of a small circle of specialist accounting firms.
Still, keeping a closer eye on finances is now topic A in many districts, and officials are moving to adopt the comptroller's recommendations, known as the five-point plan, even before the State Legislature acts to make them mandatory. A version of Mr. Hevesi's plan has passed the State Senate and is awaiting action in the Assembly.
"Whatever Hevesi recommended, we complied with, to maintain public trust and make sure we're properly scrutinized," said Charles Schlesinger, the business administrator for the Islip school district. "They are another layer of eyes."
The plan is meant to close some of the gaps in oversight that allowed the problems in Roslyn and elsewhere to develop unnoticed. It calls on school boards to train their members in their financial responsibilities; to select auditing firms by competitive bidding and switch auditors periodically; to set up audit committees to supervise internal and external auditors; to receive auditors' reports directly and respond to the issues they raise, rather than delegating that role solely to the administration; and to establish an internal audit function, a common practice in private industry.
Mr. Schlesinger said that doing all that in the Islip district has doubled its costs for auditing and oversight.
Still, officials at districts like Lawrence and Manhasset, where Mr. Hevesi's auditors identified weaknesses in financial management, say the additional cost will be money well spent if it can prevent problems like Roslyn's from developing. Especially after the wave of budget rejections on Long Island in May, school officials appear eager to reassure voters that their tax money is being handled with care.
The acidic public mood was a major reason the 12-person Miller, Lilly firm, based in East Setauket, decided to fold rather than to try to retain its market position. "While many clients remain loyal and support them, the negative publicity forced them to dissolve," said William Petrillo, the lawyer for Andrew Miller, a principal in the firm. Mr. Miller did not respond to requests for comment made through Mr. Petrillo, and Mr. Petrillo said that no one else connected with Miller, Lilly would be available for comment.
THE fall of Miller, Lilly left a hole in the school-auditing landscape that rivals are scrambling to fill, even as they anticipate a flood of new internal auditing and consulting work generated by the five-point plan.
Ken Cerini of Cerini & Associates in Hauppauge, who is the president of the Suffolk chapter of the New York State Society of Certified Public Accountants, noted that most of Miller, Lilly's former external-audit work had been won in competitive bidding by two firms with long-established school practices: Coughlin, Foundotos, Cullen & Danowski of Port Jefferson Station and R. S. Abrams & Company of Islandia and Baldwin. A third firm in the field, Callaghan Nawrocki of Melville, picked up many of the rest, including the Roslyn district.
"A year ago, there were only four firms on Long Island doing external school district auditing, and when Miller, Lilly & Pearce liquidated, the rest of us had to try and accommodate," said Michael Nawrocki of Callaghan Nawrocki. "It's not like many new players have emerged in the external auditing field. The shame of it is that there are a lot of good accounting firms out there, but with no school district experience."
External auditing firms are hired by districts to evaluate their financial reports at arms' length and perform spot tests on the information that went into them. Internal auditors, by contrast, work with the district to set up and regularly monitor the way money is handled and books are kept.
"There are only a small group of firms that are doing internal auditing for school districts," Mr. Cerini said. "I am sure that as they get inundated with work, it will open up the playing field for more firms."
The same firm may do both kinds of auditing, though not for the same district; with so few firms auditing most Long Island districts, that sometimes presents a problem. In the William Floyd school district, officials said they needed a new internal auditor because the R. S. Abrams firm was resigning the assignment, saying it had so much new external-audit business after Miller, Lilly's demise that it could no longer perform the internal auditing work. Before R. S. Abrams took the assignment, two former officials from the district were charged with embezzling more than $1 million.
In expanding to handle the flood of new work, R. S. Abrams has hired two of Miller, Lilly's former employees; its two main rivals, Callaghan Nawrocki and Coughlin, Foundotos, said they had decided not to hire any.
"We have to be concerned with not only what our standards are, but also about our perception," said Vincent Cullen, a principal in Coughlin, Foundotos.
Several officials at school districts that use R. S. Abrams said that they had not been informed that the firm was employing former Miller, Lilly workers. But they said it was not a cause for concern.
"I am sure they chose the cream of the crop from that firm," said Robert Clark, the assistant superintendent for finance and operations at the Smithtown school district, where the Abrams firm has been the external auditor for over 20 years and where the firm's founder, Robert S. Abrams, has served on the school board.
"I am sure they can use as much help as they can get," Mr. Clark said of the Abrams firm. "I trust the folks that I have worked with all these years."
Mr. Abrams confirmed in a brief interview that he had hired former Miller, Lilly workers but would not comment further for this article.
THE version of the five-point plan that has passed the State Senate does not include one of Mr. Hevesi's recommendations, that districts conduct new competitive bidding for their auditing contract every five years to encourage rotation of external auditors. Accountants question whether a rotation rule would be useful.
"I don't necessarily agree with auditor rotation for rotation's sake," Mr. Cerini said. "A district should go out to bid every five years to test and see what's out there, but the time to change should be when the firm ceases to be challenging and becomes complacent."
In the Three Village school district, where a former superintendent recently pleaded guilty to embezzling more than $40,000, the assistant superintendent for business services, Jay Scotto-Friedman, said the district decided to start rotating its external auditors only because of Mr. Hevesi's recommendations.
Mr. Hevesi's report on the Roslyn scandal also accused the Miller, Lilly firm of having a conflict of interest, because two of its principals, Mr. Miller and William Lilly, were also the owners of Finance Manager. The company makes the specialized accounting software of the same name, which it has sold to Roslyn and 251 other districts statewide, according to the Hevesi report.
Finance Manager is still in business and its software is still in wide use, though Anthony Annunziato, Roslyn's assistant superintendent for business, said his district would stop using it after its fiscal year ends on June 30. Mr. Hevesi's report said that the financial improprieties in Roslyn escaped notice in part because the software's security features were routinely being bypassed, and it recommended that districts using the software take steps to prevent those problems.
School officials and auditing professionals said that while the five-point plan had come too late for the districts that have already experienced scandals, it was nonetheless transforming the usually sleepy world of school financial management with surprising speed, raising the prospect that the new rigor will quickly head off future problems.
"Government is known for moving at a snail's pace," Mr. Nawrocki said. "You have to tip your hat to Hevesi for reacting so quickly."
