Sunday, August 20, 2006

Nazareth PSSA vs. Similar Sized Schools - a Surprise to Me.

Following the review of Nazareth's test scores in and of themselves, I conducted a more in-depth analysis. To achieve this I identified by observation schools with similar testing sizes to Nazareth (I scanned through the PDF and tried to identify all of them). I then summed and averaged the data and narrowed the list to schools very close to Nazareth's size, there were 20 of them across the state. The schools range in size from no less than 2200 to no more than 2700, with Nazareth at 2453.

Of the 21 schools (20 plus Nazareth), Upper Saint Clair, Allegheny County, had the best score in every category, while Woodland Hills, also in Allegheny County, had the worst score in every category.

In this analysis, Nazareth did not fare well, finishing in the bottom third for percent of students scoring in advanced math level, the top half for number of students scoring in below basic level of math, the bottom half for percent of students scoring in advanced reading level, and they were nearly in the middle for below basic reading.

Compared to the schools often cited as the areas elite, Parkland and Emmaus, Nazareth scored worse in all four categories, and Parkland was the best in all four.

Finally, Council Rock, where their teachers were once highest paid (and may still be I am not sure), bested Parkland in all categories. Compared to the sample of schools in Nazareth's size range, Upper St. Clair bested Council Rock.

To me, this says that Upper St. Clair is worth some further investigation. I'll try to gather some information and post it in the future:

Of the 21 schools:
  • 14 schools had a higher % Advanced math than Nazareth, 6 had a lower rate.
  • 9 schools had a lower % below basic than Nazareth, 10 had a higher rate, 1 same rate.
  • 12 schools had a higher % advanced reading than Nazareth, 8 had a lower rate.
  • 11 schools had a lower % below basic reading than Nazareth, 8 had a higher rate, 1 same rate.
In my earlier post I noted that the 11th grade results dropped off significantly. This was observed by most every district. It would indicate to me that there is a problem with the state requirements, if most if not all districts see such a drop from 8th to 11th grade in math and reading.

I have to admit I was surprised that Nazareth did not do better against similarly sized school districts.

What do you think? Can these test scores be used as a benchmark for districts? What does it say about Nazareth, especially the perception of the school vs. the reality? Can these test scores indicate quality of education?

6 comments:

Anonymous said...

I believe your analysis is interesting, but somewhat incomplete. A more accurate analysis would look at the progress Nazareth schools have made over time; how Nazareth schools compare with schools statewide on per pupil spending; and how Nazareth schools compare statewide on student test scores, not only against a limited district sample. A free tool to complete such an analysis can be found online at http://www.schoolmatters.com.

RossRN said...

Thanks for the input, especially on a Sunday.

This was not by any means intented to be a complete analysis, but I would have thought Nazareth would have been in the top half, not bottom half of pretty much any group of schools I picked.

I selected those of similar size because in the Lehigh Valley, Nazareth is very much a 'tweener district. Not as big as the big schools, but too big for the small schools. I wanted to compare Nazareth to other schools its own size.

When you consider the number of districts in PA, it is surprising to me that only 20 others were found to be close in size to Nazareth.

Regarding your other points, for me, measuring progress is what you do when you haven't gotten to where you want to be. You find the positives and try to build on them. Any which way you cut these results, it is not good news for Nazareth. In short it conflicts with the perception of our schools being good.

I do believe these are fair comparisons as nearly every student in the selected grade is tested. In this sense it is not a sample. Moreover, it is required by the state and like the SATs out of the control of the local district.

I visited the school matters site and maybe didn't spend enought time, but it appears that most of the financial data is from 2004 and the Student Performance Page uses the same test results I was looking at, except they were from 2005 and the numbers I reviewed were the newly released data for 2006.

I think it would be useful to compare Upper St. Clair with Nazareth to get a gauge on how much more/less spending per student took place, and what they do different that generates these results, but I could also go to their web site and get the most current data.

None-the-less, any insight you've gained on the school and its progress from this site, please feel free to share it.

Have a great week and thanks for taking a sip!

Anonymous said...

You've mentioned Upper Saint Clair School District a couple times as being worth closer examination, so I did a comparison between Upper Saint Clair and Nazareth using the School Matters site (it allows you to do a side-by-side comparison of up to 5 schools at a time).

Some interesting points about Upper Saint Clair:

--In 2005 they spent $10,300 per student as compared to $7600 spent by Nazareth.

--The average household income in Upper Saint Clair is over $150,000, as compared to $93,000 in Nazareth.

--The percent of adults in the community who have completed at least a bachelor's degree is a very high 57.8% as compared to
25.8% in Nazareth.

You are correct in that this district outscored Nazareth in almost every area, except for one: the return on spending index (RoSI), which is... "A measure of the average return, in terms of student proficiency, on the money spent by a school district on core activities. This metric reveals the average level of student performance produced for a given level of spending, for example the percentage of students reaching proficiency in reading and math for each unit of core operating spending. Although the index is not specifically a measure of marginal return, it is a proxy for exploring the relationship between achievement and spending. While the index does not purport to predict performance based on spending, it becomes an especially valuable tool in the context of the proliferation of equity and adequacy issues being debated across the country. Moreover, the index provides a measure to be considered when evaluating comparative educational productivity of similar school districts. "

In this area, overall return on investment, Nazareth (10.6) outscored Upper Saint Clair (9.5).

I encourage those who want to do a real "apples to apples" comparison of schools to use the School Matters (http://www.schoolmatters.com) tool. Although it currently lists 2005 scores, it will soon have the recently released 2006 scores, as well as a lot of information - besides PSSA scores - that provides a fuller picture of a district's actual progress.

RossRN said...

I've found in sports that some teams respond to a loss by identifying weaknesses and improving them to get better, while others find excuses that "caused" the loss and continue as they were.

Guess which one does better over time?

The former is the approach I took after reviewing these scores, and I have to take exception with a bit of your interpretation of my post.

I'm not concerned with progress, but with standing (or ranking), and I don't see how a straight-up comparison of standardized test scores is not "apples to apples."

If Nazareth has "progressed" to this point, I'd be more concerned with what the results reveal to me. I fully anticipated that Nazareth would score at least in the top half of this group of like-sized schools spread across the state.

That is really all I'm looking at. I do think there is value in taking a look at high performing schools to see what they do that we don't and vice-versa, and identifying what we can incorporate that would boost our students' performance.

It's not a knock, it is a way to improve.

Regarding the statistics you site, the Upper St. Clair district appears to be as good as it gets. While it has a higher income than Nazareth, this does not in and of itself explain how they achieved such good scores. That is what I think would be of interest to learn.

The fact that they have a higher income and higher percent of adults who graduated from college is a correlation, but this does not make it a causation.

Both Greater Latrobe and Penn-Trafford, also in my comparison, outscored Nazareth in all four categories.

Unlike St. Clair, these schools average income is $64,718 and $80,661 respectively, with 27.5% and 33.3% parents having graduated college. Despite this they each had a higher RoSI than Nazareth.

Again,this correlation is not a cause.

The examination I refer to is not finding more and more statistics, but instead looking to see what these districts do, that Nazareth could consider doing, to improve our students' education.

As I noted earlier, based on my own experience and that of my daughter, I was simply surprised that our scores were not higher.

I hope you better understand where I'm coming from and thanks again for visiting newsovercoffee.com!

Anonymous said...

We send our children to school each day. During the day they have 30 minutes of math, 30 minutes of reading, some science, lunch, social studies, perhaps music, art or computers. Then they go off to band, football, etc. Not to mention the multiple field trips, assemblies, guest speakers, visiting authors,... So after the long day of a dozen or so activities we hold up the PSSA multiple-guess yardstick to measure math and reading.

Good thinking! Let's focus on getting good standardized test scores!! I can't tell you how many times in life I've been reminded to fill in the bubble completely!!

We're completely distracted by this silly testing. Anyone notice that the US has recently fallen to number 18 in the world of broadband availability to the home... just surpassed by Luxembourg (sp).

As we continue to focus on bringing the "basic" and "below basic" students up to proficient on standardized tests, the Chinese and Indians are kickin' our a$$.

In China, if you're one in a million... there are still 1300 other people just like you or better. China has more HONOR STUDENTS than we have students. Wake up. Dump the standardized test.

We need to totally revamp our educational system. Give our kids the 21st century learning tools they'll need to compete in the real world.

end rant

RossRN said...

Sometimes news over decaf is better after five;-)

While the rant may make you feel better, I couldn't see in your post how you would improve the current educational system or better measure student achievement without standardized testing.

The ability to read and do math seem fairly fundamental. Logic and reason wouldn't be bad either.

Given the increasing emphasis on specialization, it would seem to me that young students, now more than ever need a firm basic education with a heavy emphasis on math, science, and communications (reading and writing). Then when they reach the point in their life when they begin to specialize, they'll have a solid foundation to work from.

Contrary to throwing out the basics in favor of something new, I think a return and focus on the basics, especially in the pre-college years is critical.

Hope you feel better getting that off your chest, and I would love to hear what you think Nazareth could do to revamp its education and be more competitive in the future.