May 14, 2009

Does more money always equal a better education?
As school districts consider their budgets, we look at how dollars do and don’t translate into educational success
By Jeff Mucciarone jmucciarone@hippopress.com

When Kelleigh Murphy, Manchester’s Ward 12 alderman, recently announced her plans to resign and move to Bedford to start a family with her husband, she set off something of a firestorm.

The insinuation — that Bedford is a better place to raise a family, that Bedford might offer better schools — was controversial but not particularly new. The perception that better schools lay outside Manchester’s borders is a common one. But are schools in Bedford — and suburbs like it — really better than schools in the Merrimack Valley’s more urban cities?

Any examination of which town has better schools brings up its own questions, specifically: how should a community’s commitment to education be measured? There seems to be no easy way, but per-pupil expenditures  and standardized test results could provide a pretty good indicator.

Specifically looking at the Manchester vs. Bedford question, statistics from the 2007-2008 school year would seem to give the advantage to Bedford. Bedford’s students do better than Manchester’s on the state standardized tests, the New England Common Assessment Program or NECAP, which New Hampshire, Vermont, Maine and Rhode Island utilize. Students in grades 3 through 8 and in grade 11 are tested in math and reading. (Tests in writing are also given to grades 5, 8 and 11 and science tests are given to grades 4, 8 and 11.)

Bedford spends an average of $10,982 per student, while the Queen City spends $9,184 per student — both below the state average of $11,135. (Waterville Valley tops the list at a whopping $23,935 per student, though it only has 36 students total in its school, which covers kindergarten through eighth grade. It has no high school.)

Cities vs. towns
The state’s largest city is a bit of an anomaly among local schools — it has a diverse population, more than 20 schools and about 1,200 teachers. So comparing Manchester to Bedford — or comparing any local urban school system to its suburban neighbors — is a little like comparing apples to oranges. Regardless of test scores, per-pupil expenditures or any other statistic, the educational experience is likely to be very different in the two communities, officials said.

But that doesn’t change the fact that Bedford students are clearly scoring much higher than Manchester students. Ninety percent of Bedford’s students scored proficient or better on the NECAP in reading and 86 percent scored at least proficient in math. In Manchester, students came in at 56 percent and 45 percent respectively. In Nashua, 71 percent of students scored proficient or better in reading and 61 percent of students scored at least proficient in math. Conversely, 88 percent of students in Hollis scored at least proficient in both math and reading. Amherst students also scored well, 86 percent proficient in reading and 82 percent proficient in math. 

If districts can simply buy better test scores, then Manchester could be in trouble this year, as Superintendent of Schools Tom Brennan recently received the go-ahead to lay off 78 of the district’s approximately 1,200 teachers. If Manchester Mayor Frank Guinta’s $146 million school department budget gets approval, Manchester’s per pupil expenditures would decrease by about $55 per student to $9,129. In Nashua, a district more comparable to Manchester than most suburban districts, the community spends $9,119 per student on average.

But cities spending less per student isn’t just a product of being big. Officials say cities have to deal with issues surburban counterparts typically don’t have to, such as diversity and higher rates of low income among students.

If test scores and per-student dollars are the best way to measure education in the Granite State, then people who leave the cities for the suburbs seem to have made the right move. However, “[Standardized tests] can be a useful tool in helping schools determine direction, but they have to be seen as just that, not as an overall measure,” said Michael Middleton, associate professor of education at the University of New Hampshire.

More money always means better scores … or does it?
Manchester, Nashua and Concord (which spends $10,521 per student), all spend less per student than the state average of $11,135. Like Nashua and Manchester, Concord’s suburbs typically outspend it per student.

So if cities want to boast that their schools are as good as suburbs’ schools, couldn’t they just spend more?

Not necessarily, officials said. Academic success isn’t achieved with money alone.

Irv Richardson, coordinator of public education and school support with New Hampshire’s chapter of the National Education Association, said per-pupil expenditures, for example, could include renovations to a school building to bring it up to fire code. As a parent do you want the schools to spend the money? Of course, he said, but people aren’t likely to expect a jump in test scores because the school is up to safety code. Just tossing money at schools won’t necessarily raise test scores; it matters how schools spend that money, Richardson said.

Take Bow for instance. On average, the community spends $12,071 per student and 85 percent of its students scored proficient in reading while 77 percent did in math. Taking the low score of 77, Bow spent $157 per percentage point — that is, $157 to get one percent of its students proficient in math and reading. Bedford spends about $1,000 less per student on average than Bow, but its students scored considerably better, particularly on the math portion of the NECAP. Taking Bedford’s low score of 86, the community spent $128 per percentage point of students reaching proficiency. In terms of investment, Bedford is getting more of a return for fewer dollars.

In fact, of all the communities that offer all three levels of education, Bedford is getting the most bang for its buck. Salem, which spends much less per student than Bedford, also spends $128 per percentage point, but its students are only proficient at 68 percent.

And in addition to lower proficiency, Manchester has the area’s highest dropout rate — 17.9 percent at its three schools compared with 11 percent at Concord High School (the high school with the area’s next highest dropout rate) or 5.5 percent at Salem High School. In terms of the dropout rate, Salem, which spends a few hundred dollars more than Manchester per high school student, is getting the better return. (Bedford’s dropout rate is a very enviable zero but its high school is only in its second year of existence.)

In Nashua, where 61 percent of students scored proficient or better in math, compared with 71 percent in reading, the district spent about $150 per percentage point, when looking at the lowest score. Bow is spending $7 more than Nashua per percentage point, but Bow students are scoring substantially better than Nashua students — in this case, Bow is getting a much greater return on its investment. Every district examined in this article scored better on reading than math, except for Chester, 78-79, and Hollis, 88-88. Hollis, which only has an elementary school and spends $12,387 per student, spent about $140 per percentage point (its high school students go to Hollis-Brookline High School).

Mark Joyce, executive director of the New Hampshire School Administrators Association, said it’s important to look at what opportunities are available within schools. A school spending a certain amount per student could offer a variety of enriching opportunities that help stimulate students. Another school spending the same amount per student might not offer even close to the same opportunities, maybe because they’re isolated or for other reasons. That’s just one factor that complicates simply looking at per pupil expenditures, he said.

Statistics are further misleading simply because of how school districts are split up. While Nashua and Concord’s districts educate students from kindergarten through high school graduation, other towns run their own elementary schools but send their students to other districts or partner with other towns for higher grades. For example, the community of Hollis has only its own elementary school. Hollis and Brookline formed a cooperative for middle and high school students. The cooperative for the communities spends on average $10,401 per student and students there scored at least proficient at 87 percent in reading and at 65 percent for math. Each cooperative chooses its own formulas for determining how much a given community contributes financially, typically based on enrollment and a community’s wealth in terms of a tax base, Joyce said.

More than money
But are dollars and test scores all there is to a quality school?

Education experts say certainly not. Social development, citizenship, strong extracurricular activities and morale are also key parts of a quality education — parts that prove much harder to measure. There’s not a statistic at the state Web site that accounts for student happiness or student motivation. Other items, such as attendance and dropout rates, provide further insight. 

“[Standardized tests] certainly give us a sense of direction and movement,” Middleton said. “What it doesn’t ask is whether these should be standards we should be trying to meet.”

Also, test scores by themselves don’t account for subpopulations. Just at Manchester Central High School, the state recognized 100 economically disadvantaged students, whereas Bedford High School had seven such students. That doesn’t show up in a test score or in per-pupil spending. Officials say it is proven that students from low-income families don’t score as well on tests.

District-wide per-pupil dollars can be deceiving. Bedford spends more than $15,000 per student at the high school level, and spends nearly $10,200 per student at the elementary level. (Bedford recently built a new high school.) Manchester spends $8,253 per high school student, which according to state statistics is the least per-student dollar amount in the state at the high school level. But then again, no school district has more high school students than the Queen City. Conversely, Manchester is comparable to Bedford at the elementary school level, where it spends more than $10,000 per elementary school student. But again, Bedford spends $128 per point to get to 86 percent proficient. Manchester spends more than $200 per percentage point to get to 45 percent proficient.

There are still gaps. The ratio of dollars to test scores doesn’t explain where the money is spent: teachers, resources, administration, transportation or energy costs, or professional development.

Joyce said there are several variables that affect per pupil dollars, one of which is size. Small schools often need to spend more per student to account for programs all schools require. Bigger school districts need to spend money on those programs too, but the cost can be spread out among students, presumably with no detriment to the quality of education.

“It is typical that size of scale has a way to reduce per pupil costs,” Joyce said. “You have to offer the same basic program to a high school of 100 kids that you do at a high school of 1,500 kids — physics, foreign language. But if you spread it over 1,500 kids, it becomes more efficient. That’s where the size of scale is a major variable.”

Per pupil dollars sometimes have nothing to do with what’s happening inside a school building, Joyce said.

Student to teacher ratios can also serve as indicators for school systems. Perhaps suprisingly considering its size, Manchester was largely in the middle of the pack in terms of student to teacher ratios at 13.8 students per teacher. For comparison, Nashua sits at 13.7, Concord at 14.4 and Bedford at 13.8 as well.

Middleton suggested taking a peek at disciplinary reports, attendance rates and which students are scoring well on tests and participating in extracurricular activities. If the best test-takers are also the only ones participating in the school band or playing sports, then just sampling test scores won’t capture a school’s issues. Does the school participate in civic activities? Are students participating in a wide range of extracurricular activities? Middleton said, in talking to people, he believes communities do see the importance of developing well-rounded students, beyond success on tests.

“Just looking at the finish line doesn’t give us a sense of who’s really developing,” Middleton said.

Especially given that employers are more interested in employees who can work well with a variety of people, testing isn’t a ticket to success or failure.

“No one takes a job as a test-taker,” said Todd DeMitchell, chairman of the UNH education department.

Why test?
This whole business of comparing districts and schools is relatively new to New Hampshire. Until the late 1990s, New Hampshire communities were essentially left to do their own thing. That changed with the national movement to standardize education. Now communities across the country can be measured, compared, contrasted, chastised or lauded based largely on how students perform on rigid, standardized tests, said Kathy Staub, director of the Manchester Coalition for Quality Education.

“That really kind of opened the door,” Staub said of standardized testing. “Now anyone can go to [www.schoolmatters.com] and do a comparison, one district to another. If you don’t do well, you can’t keep it a secret.”

The move to standardize education was largely due to a governmental need for accountability. Since the state government provides money directly to school districts now, it needs to be able to hold schools accountable. Enter standardized tests that at least to some extent measure academic success.

No Child Left Behind is the spoon that stirs the drink when it comes to testing. Former President George W. Bush’s legislation calls for all schools nationwide to meet pre-determined standards through testing. That’s where the NECAP comes from. Each state, save the ones using NECAP, has its own standardized test. States can voluntarily take part in the federal testing program, and so far results have shown “incredible variability” in how state tests correlate with the national assessment. Still, the penalties at the federal level for not achieving Adequate Yearly Progress (AYP) are all the same, despite each state having its own measure, Richardson said.

Along with instituting standardized tests nationwide, No Child Left Behind requires states to establish timelines for AYP on tests. For school districts that don’t demonstrate AYP, there are governmental intervention programs in place to help create further progress, mostly in the form of remediation.

Depending on how they’re applied, officials say standardized tests can give teachers a solid look at how well lessons are being absorbed or how to plan curriculum for the future. But they’re still just a snapshot.

Districts and communities that have under-performing students are labeled “districts in need of improvement,” Staub said. That title carries with it some extra state funding to help schools turn things around. But sometimes, and this appears to be so in Manchester (which is a district in need of improvement), the state aid doesn’t become additional dollars for the school system. In past years city officials have used the extra funding to make up for money that otherwise would have come from the city’s general fund (and during one year used the extra dollars to decrease taxes). The result is that schools get level funded — and maintaining the status quo doesn’t present any opportunity for progress, Staub argued.

“It’s pretty disingenuous,” Staub said. “Especially when we’re a district in need of improvement. I think the state might be getting a little fed up with us.”

Level funding doesn’t appear to really keep things at the status quo anyway, since Manchester’s per-pupil expenditures seem to keep moving away from state averages. The district was about $900 short of the average a few years ago, then, with the help of state funding, it cut that to $700 short. Now, it’s about $2,000 short on average.

“There’s no return on the status quo,” Staub said.

School districts that spend near the $11,135 state average for per pupil dollars have varying test scores. Windham spends $10,917 per student and its students scored well, 85 percent proficient in reading and 83 percent in reading. Merrimack spends $11,182 per student and students came in at 77 percent proficient in reading and 68 percent proficient in math. Spending at $11,125 per student, Mont Vernon students scored at 76 percent and 71 percent. Derry, spending at $10,977 per student, saw its students score 74 percent and 71 percent. In Milford, which spends $10,932 per student, students scored 74 percent proficient in reading and 66 percent proficient in math. Bedford is slightly below the state average and its students have those 90 percent proficient in reading and 86 proficient in math scores.

What the tests don’t show
Middleton said it’s certainly possible to have a school with students scoring poorly on standardized tests but doing well in other important areas, such as social development or in extracurricular programs. In that sense, Middleton says it can be more important to see how students progress through school, rather than focus on end result test scores.

The state legislature is currently looking at a bill on educational adequacy, which would take more into account than just standardized tests, such as attendance, Richardson said.

Middleton said tests are a useful tool for planning future curricula and making sure a current curriculum is working. But it’s not as simple as just figuring out who scored well and who didn’t; it’s also about whose scores are improving over time. He said not everyone starts at the same point in terms of their education. Maybe some low scoring students didn’t have any preschool preparation, maybe their parents didn’t speak English or maybe they simply weren’t at the same reading level as higher scoring students when they entered school.

Staub said Manchester has started testing students for kindergarten readiness and they’re finding a gap between students who attended some form of preschool and students who don’t. That meshes with Middleton’s point that students don’t all start at the same point and don’t develop at the same rate.

“We can’t change out the kids in the classroom,” Staub said. “We can’t replace them with kids from Bedford. We need to have teachers who are trained to be able to work with those kids. It is possible.”

But if administrators could simply switch Manchester students with Bedford students or Concord students or Nashua students, would test scores remain the same from school to school? In other words, would the Bedford students, now in Manchester, score the same on tests in Manchester as they would have in Bedford?  There’s no easy answer to that, but officials say it gets to the idea that there’s a lot more going on at schools than a test can measure. 

So while Bedford’s high school students score better on the NECAP than Manchester high school students, that doesn’t fully measure students’ success during their entire academic career. Scores don’t reflect the cultural and language barriers, apparent in Manchester and Nashua, that likely impact test scores and academic success in general.

Beyond testing
“As a former principal, if everyone is at the 100th percentile but they’re all hating school, I’m not happy with that,” Richardson said. “There must be issues we need to look at.”

But test scores are measurable, and in terms of government and policy at the very least, that makes them important, even if it only provides one piece of information. Middleton likened test scores to speeding tickets. People wouldn’t solely base their impression of a community on how many speeding tickets are issued there, but it could still be useful information.

The well-known problem with test scores is that they measure only what they test, which in New Hampshire is math, reading and in the upper grades science. The state lists six curriculum frameworks on its Web site (math, English and language arts, science, arts, social studies and world languages) yet standardized tests cover only three to four of the areas (math, reading, writing and science).

DeMitchell added a saying that he said educators believe to be reality: “That which gets measured, gets done.” He says studies have proven that testing reduces curriculum. “The seven that are important don’t always manifest themselves in some kind of test.”

Tests can be designed in two ways, formative and summative. Summative tests measure what students know at the end of instruction, while formative tests are used to plan curriculum for the future. While there is merit in summative tests, they don’t help anyone learn. Formative assessments do, Richardson said.

“If you have cows and you want them to gain weight, you don’t weigh them, you feed them,” Richardson said.

So it’s finding the right combination of summative and formative that matters. Summative tests can help teachers be sure their students are learning what they’re supposed to.

Richardson said educators need to know the purpose of an assessment, the method for meeting the purpose, whether the method is appropriate and what the evaluation criteria are. A state recently wanted to use the SAT to evaluate teachers. He said it was never designed for that purpose. The only students taking the test are the ones planning to go to college, so right off the bat the test neglects a whole segment of the population.

“We never will come up with the perfect test with no error,” Richardson said. “They can help us make informed decisions, but they’re not the be-all end-all.”

Money — how it creates disparities
DeMitchell said a key piece of information is communities’ assessed value. Communities with higher property assessments generally have lower tax rates than communities with lower assessed values. That means wealthier communities can tax less and receive more, in most cases.

“Wealth always plays a part,” DeMitchell said. “Wealth means options.”

DeMitchell said lower-wealth communities may need to spend money on remediation that a higher wealth community wouldn’t need to. Wealthier communities can put more dollars into advanced courses or resources, he said.

“That’s where you really start to see the separation between high wealth and low wealth,” DeMitchell said.

 So before students even begin school, the overall value of students’ community almost predetermines their type of educational experience (such as that disparity between students who have had preschool and those who haven’t).

DeMitchell said that as long as educational systems run on property taxes, there will be plenty of disparity.

“It’s such a narrow base to support a fundamental service such as public education,” DeMitchell said. “It helps decide the kinds of educational experiences that you have.” 

Likewise, large school districts in the North Country may need to spend considerably more on transportation than communities close to Manchester. That doesn’t mean the educational experience is any better, DeMitchell said.

Communities that need to deal with language issues, like Manchester, need to devote money to those issues that other schools simply don’t.

Staub said there is a public perception in some circles that the Queen City school system is invested too heavily in administration. But she says there’s no evidence to back that up. With a dwindling budget, Manchester has trouble providing the kind of professional development and support that could help get its students over the hump, Staub said.

Joyce is most interested in what schools do with their money. Which programs are really working at a particular school? Are successful programs sustainable? But officials say money, no matter how you slice it, keeps the door open to new opportunities that other communities more strapped for cash may not have access to. The application of new programs or new spending makes all the difference.

Still, officials were clear that dollars and academic success don’t simply go hand in hand. Richardson said people assume there should be a linear relationship between how much a community spends and test scores, but where and how the money is spent can make all the difference — and many times a school district’s hands are tied on where the money can go.

“If you take a toll booth, if it takes $1 to get to Manchester, $1.25 isn’t going to get you through any more effectively,” Richardson said. “But if you only have 90 cents, they’re not going to let you through. It isn’t just the amount spent, it’s where we’re choosing to spend that money and how we’re looking at accountability.”

Statistics!

District Test Scores

Per Pupil Expenditures

Dropout Rates

Student to Teacher Ratios

Dollars per NECAP percentage point