Bad Conclusions
This week in the Senate Education Committee, we heard a presentation from the Department of Education & Early Development about statewide testing.
As some Alaskans talk about "accountability" in public education it seemed like a good time to look at how we measure whether kids are making progress. We got a guide to the Alaska System of Academic Readiness (AK STAR,) a new statewide assessment. The results were... perplexing.
In the two main categories—English Language Arts and Math—it was hard to find patterns in the results. So-called "proficiency" fluctuated wildly by grade. In English, for instance, the percentage of 7th graders who scored as “needs support” was nearly double that of 6th graders. The percentage "approaching proficient" in math rocketed up 60% from 5th grade to 6th, but plummeted by more than half from 7th grade to 8th.
And there were wild swings like that all over the charts. There were years when a grade of kids tested as making all kinds of progress in math but losing a lot of ground in English. And vice versa. These are statewide numbers, so it's not like an extraordinary school here or a few troubled children there moved the needle up or down.
A statistician would point to the "null hypothesis" here. Oversimplified, that means if we don’t find a meaningful relationship between the variables we're looking at, they don't mean anything. That's as opposed to the "alternate hypothesis," which might explain the data.
When I asked the department how they explain the huge, wild swings, they assured me the test was well-designed. You don't need a statistics degree to know that's called a "non-answer." Asked to respond to the suggestion that maybe the standards we measured were off-base, they assured us the standards are right on the money.
How do students statewide score nearly twice as badly one year than they did the year before? Especially while another grade of kids swung, statewide, the other way?
I suspect they didn't.
And that's the problem with trying to use one test, built from a nationwide database of questions, to measure children and schools as different as the North Slope and Nikiski. You can't.
So what does that mean for accountability? The Alaska Constitution has a path for us to follow. Article X, Sec. 1 recommends "maximum local self-government" and requires that "A liberal construction shall be given to the powers of local government units." That's why school boards are elected locally. That in turn keeps schools accountable to Alaskans all across our state.
There are lots of ways for local districts and communities to check on how our kids are doing. The AK STAR test sure doesn't look like one of them.
