assessment, being a teacher, being me, testing

Dear STAR Test, We Need to Talk

Dear STAR,

We first met two years ago, I was fresh out of a relationship with MAP, that stalwart older brother of yours that had taken up hours of my 5th graders time.  They took their time and the results were ok; sometimes, at least we thought so but we were not sure.  But oh the time MAP and I spent together that could have been used for so many better things.

So when I heard about you, STAR, and how you would give me 46 reading skills in 11 different domains in just 30 or so questions, I was intrigued.  After all, 34 timed questions meant that most of my students would spend about 15 or so minutes with you.  You promised me flexibility and adaptation to my students with your fancy language where you said you “…combine computer-adaptive technology with a specialized psychometric test design.”  While I am not totally sure what psychometric means, I was always a sucker for fancy words.   Game on.

With your fast-paced questions, I thought of all the time we would save.  After all, tests should be quick and painless so we can get on with things, right?  Except giving my students only 120 seconds to read a question and answer it correctly meant they got awfully good at skimming, skipping lines, and in general being more worried about timing out than being able to read the whole text.  (Fun fact, a fellow teacher timed out of most of her questions when she took the test in training and still received above 11th-grade level).  For vocabulary, all they get is 60 seconds because either they know it or they don’t, never mind that some of my kids try to sound words out and double-check their answer all within those precious seconds, just like I have taught them to do.  I watched in horror as students’ anxiety grew.  In fact, your 120 second time limit on reading passages meant that students started to believe that being a great reader was all about speed.  Nevermind that Thomas Newkirk’s research into reading pace tells us that we should strive for a comfortable pace and not a fast one.  So yes, being a slow reader= bad reader.  Thanks, STAR.

And yet, maybe it was just my first year with you.  After all, we all have growing pains.  But this year, it didn’t get better, it just got worse.  Students whose scores dropped 4 grade levels and students whose scores jumped 4 grade levels.  Or how about those that made no growth at all.  I didn’t know what to take credit for.  Was it possible that I was the worst teacher ever to have taught 7th grade ELA or perhaps the best?  You confused me, STAR, on so many occasions.  So when students significantly dropped, they sometimes got to re-test, after all, perhaps they were just having a bad day?  And sure, sometimes they went up more than 250 points, all in the span of 24 hours, but other times they dropped that amount as well.  That is a lot of unmotivated or “bad day” students apparently.   And yet, you tell me that your scores are reliable.  Yet, I guess they aren’t always, after all, at the 7th-grade reading level you only got a score of .82 retest reliability which you say is really good but to me doesn’t sound that way.  0.82 – shouldn’t it be closer to 1.0?  In fact, when your company compared you to other recognized standardized tests it dropped to 0.70 for 7th grade, but perhaps it was because of the small sampling size, just 3, 924 students?  Who knows? I suppose I could email you to ask for more updated results like it says in the very small footnote.

Yet through all of this, you have dazzled me with your data.  With all of the reports that I could print out and pour over.  Perhaps you were not accurate for all of my students, but certainly, you had to be for some.  It wasn’t until a long night spent pondering why some of my students’ scores were so low that I realized that in your 0.81 reliability lies my 0.19 insecurity.  After all, who are those kids whose scores are not reliable?   I could certainly guess but the whole point of having an accurate assessment means that I shouldn’t have to.  So it doesn’t feel like you are keeping up your end of the deal anymore, STAR test.  In fact, I am pretty sure that my own child will never make your acquaintance, at least not if we, her parents, have anything to say about it.

So dear STAR test, I love data as much as the next person.  I love reliable, accurate data that doesn’t stress my students out.  That doesn’t make them really quiet when they realize that perhaps they didn’t make the growth.  I love data that I can rely on and it turns out STAR, I just don’t think you fit that description.  Perhaps I should have realized that sooner when I saw your familial relationship with Accelerated Reader.  Don’t even get me started on that killer of reading joy.  You even mention it yourself in your technical manual that there may be measurements errors.  You said,  Measurement error causes students’ scores to fluctuate around their “true scores”. About half of all observed scores are smaller than the students’ true scores; the result is that some students’ capabilities are underestimated to some extent.”  Granted it wasn’t until page 81.  So you can wow me with all of your data reports.  With all of your breakdowns and your fancy graphs.  You can even try to woo me with your trend scores, your anticipated rate of growth and your national percentile rankings.  But it is not enough, because none of that matters if I can’t count on you to provide me with accurate results. It doesn’t matter if I can’t trust what you tell me about my students.

So I wish  I could break up with you, but it seems we have been matched for the long run for now.  All I can be thankful for is that I work for a district that sees my students for more than just one test, for more than just their points because does anyone actually know what those points mean?  I can be so thankful that I work in a district that encourages us to use STAR as only one piece of the data puzzle, that chooses to see beyond it so we can actually figure out a child’s needs.   But I know I am lucky, not everyone that is with you has that same environment. So dear STAR, I wish you actually lived up to all of your fancy promises, but from this tired educator to you; it turns out I don’t need you to see if my students are reading better because I can just ask them, watch them, and see them grow as they pick up more and more books.  So that’s what I plan on doing rather than staring at your reports, because in the end, it’s not really you, it’s me.  I am only sorry it took me so long to realize it.

Best,

Pernille

PS:  I am grateful that Renaissance Learning did reach out to me to discuss my post, here is their response:

Renaissance Learning is deeply committed to teacher success in the classroom. I am the STAR Product Marketer and read your blog regarding our product. I welcome the opportunity to talk with you about your concerns and help you get the best experiences with Renaissance!

I captured two primary issues from your blog:

  1. STAR Reading Time Limits
  2. Reliability

STAR Reading Time Limits

I wanted to make sure you know that you can set an extended time preference in the software to help reduce students’ test anxiety and frustration. The instructions for doing so are on page 217 in our STAR Reading software manual.

On page 12 of our STAR Reading technical manual there’s an overview of testing time by grade that illustrates guidance for timing. This information can be used to assess what is the best time limits for your students (based on analysis of testing conducted in the fall of 2011).

Reliability

Reliability is a far more complex topic. There are three things to look at when discussing this topic: Reliability, Validity and Standard Error of Measurement (SEM).

Reliability is the extent to which a test yields consistent results from one test administration to another. Validity is the degree to which it measures what it is intended to measure and is often used to judge a test’s effectiveness. Standard error of measurement (SEM) measures the precision of a test score. It provides a means to gauge the extent to which scores would be expected to fluctuate because of imperfect reliability, which is a characteristic of all educational tests. These elements are described in detail in the Understanding Why STAR Test Scores Fluctuate.

STAR assessments have been independently reviewed and certified by the National Center on Response to Intervention www.rti4success.org and the National Center on Intensive Intervention http://www.intensiveintervention.org  and received high ratings as a screening and progress monitoring tool based on the criteria set forth to meet exceptional standards.

And my response:

Thank you for your response; the time limit is not something decided by me but by my district, but the fact that the product even comes with one should be debated further; what does time have to do with reading comprehension and vocabulary knowledge besides the selling point of being able to administer it quickly?
 The next point is the reliability; you seem to have missed the major point of the post, which is that when we do not know which child’s scores are reliable or not, then it becomes very hard to use the test for anything.  While I have read the document you linked again (I had read it before the post) it doesn’t yield any new information.   In fact,  it appears that teachers are expected to either assume it is because of something going on with the child or a measurement error.  The reliability for 7th grade as reported by STAR itself is 0.70 as referenced on page 25 in this manual.  According to the technical manual the SEM reported on page 41 in table 12 it is 71.74 for 7th grade.  That is incredibly high error measurement when it comes to kids’ scores, and yet that wouldn’t cover the fluctuation that we see in many students.
While I appreciate your response, I stand by the post; it is a travesty that teachers are being evaluated based upon tests like this, particularly when they are meant to be a diagnostic tool.  And while scores are probably accurate for some students it is hard to figure out who they are accurate for and who they are not.  My only wish for the future is that the test is either more accurate or somehow allows us to better decide which children’s scores are accurate.
being a teacher, being me, testing

The Test Does Not Care

When the test passes we will resume learning in all of its glorious messiness

We teach our students to ask questions, to share, to discuss.  We teach them to find help when they need it, take their time when they can, and to always use their tools.

They sit where they are comfortable in order to access the learning best.  They reach out to those they trust and they use us whenever they are lost or just want to make sure that the path they are headed down is, indeed, the right one.

We try to create learning environments where discovering facts is only the first step of the journey, using them as a way to further understanding is the next.  We use our shared ideas to further the knowledge of others.  Where mistakes happen and we try again.  We try to create learning environments where students have a voice, where they have choices, where we try to make it personal so that the experience they have makes sense for who they are.

We do not pride ourselves on the scores that they get but instead on the books that they read, on the aha moments they have, on the growth they show.  We pride ourselves on who they are as learners and not just what they produce.  Their value is bigger than a number.

And yet…

In two weeks, my students will sit in rows in a bare room and spend four days taking the state standardized test.  They will not be allowed to ask questions.  They will not be allowed to help each others.  They will not be allowed to use the very resources that I have taught them to use.

We will not reflect.  We will not discover.  We will not question.  We will not grow.  Not in a way that matters, anyway.  Instead, they will sit, they will read, and they will answer. I will sit, I will watch, and I will make sure no one cheats.   I will have a few scripted responses that I am allowed to say if a child asks a question.  Once done, someone hired by the state, who has no idea who my students are, will grade their answers as if a short response will ever give a proper window into what they really know.

Because let’s be honest, the test does not care that they have grown in ways that cannot be measured.

Because the test does not care how hard they have worked to get where they are.

The test does not care that they may finally see themselves as a reader.  Or a writer.  Or a learner.  Or even as someone who deserves to be a friend.

The test only cares for multiple choice.  For pick the right answer.  For write it right or it will be counted wrong.

So in two weeks when my students are reduced to nothing more than an entry ticket, I will hope that they know that they are bigger than that.  That they are worth more than that.  That everything they have done, how hard they have worked, how much they have grown may not be measured on the test, but I know.  And so do they.  And when the test passes, because this too shall pass, we will resume learning in all of its glorious messiness.  We will fill our walls with what we need and our voices will ring true again.

I hope I have taught them enough.  I guess the test will tell.  Or perhaps maybe it won’t.

If you like what you read here, consider reading my book Passionate Learners – How to Engage and Empower Your Students.  Also, if you are wondering where I will be in the coming year or would like to have me speak, please see this page.

 

 

aha moment, assessment, being a teacher, being me, MIEExpert15, Passion, Personalized Learning, student choice, student voice, testing

So It Turns Out I Am a Terrible Teacher

It turns out I owe everyone an apology.  Or at least a great big “I am sorry” to all of the people who have ever been inspired by this blog to change the way they teach.  It turns out I don’t know what I am doing, at least not if you look at our test scores.  You see, my students took our district standardized test, the one they take three times a year, and it turns out that at least for some all of my crazy ideas have apparently ruined their English skills.  It wasn’t that their scores dropped just a touch, no, some lost hundreds of points in their comprehension skills; whole grade level disseminated by this terrible teacher.  And there is no one to blame but me, after all, I am the one responsible for all of the teaching.

These tests are a funny thing really, they have a way of messing with even the most stoic of teachers.  We say we don’t care what the test scores are and yet we cannot help but feel fully responsible for the negative scores.   The positive ones, the ones that gained hundreds of points since January; those cannot possible be my doing, because I am teaching all of these kids.  And not all of these kids are improving by leaps and bounds.  So those great scores, they have to be a fluke, but those kids with the big fat minus next to their number, yup, I did that.

As I wrestle with my own feelings of ineptitude tonight, I have realized that who ever thought that teachers could be evaluated by scores that change so dramatically over a year, has never been a teacher.  I could re-test my students tomorrow and guarantee you that all of them would have different scores.  How a test like that can help me plan instruction is beyond me.  How a test like that can be used to evaluate teachers in some states is even further out of my understanding.  And yet it does, and we take it ever so personal because we care.  We think if we had just tried a little harder, worked a little more then maybe we could have reached all of our kids, and not just the “easy” ones.

So I am sorry for ever thinking I could help change education from within.  I am sorry that I have told others to give the classroom back to students, to create passionate learning environments where students not only have a choice, but they also have a voice.  The test told me today that I am doing something wrong for these kids, because there is no way a 34 question test can be wrong, right?  All I can say is that I am thankful to work in an incredible district with an amazing administration that sees beyond the test scores.  That has faith in us and in all we do.  That knows we are bigger than the test scores our students get, because if I didn’t, according to this test, I don’t have any business teaching some of them, or blogging about what I do.

I am a passionate teacher in Oregon, Wisconsin, USA but originally from Denmark,  who has taught 4th, 5th, and 7th grade.  Proud techy geek, and mass consumer of incredible books. Creator of the Global Read Aloud Project, Co-founder of EdCamp MadWI, and believer in all children.  The second edition of my first book Passionate Learners – How to Engage and Empower Your Students” is available for pre-order now.   Second book“Empowered Schools, Empowered Students – Creating Connected and Invested Learners” is out now from Corwin Press.  Join our Passionate Learners community on Facebook and follow me on Twitter @PernilleRipp.

Be the change, being me, choices, student voice, testing

On Parenting and Standardized Tests

Much like the rest of North America, my students have been doing the BIG test these last few weeks.  Much like so many teachers, I have sat silently and hoped that I have been enough, that what we have done has adequately prepared them, that they do well.  And yet…

I have also refused to worry more about it.  I have refused to get further upset.  I have shaken my head, I have made my comments, but I have refused to take it home with me.  This test is so far out of my hands that me worrying more about it, losing sleep over it, getting my blood pressure up, is not going to do anything good.  I cannot change what I have taught.  I cannot help my students more than I have.  So by now as a teacher, I can only do my best and hope for the best.  I can raise my voice whenever I get a chance, and I can hope for change.  Because as teachers, our voices are being drowned.  Our voices are not being heard.  Proponents of the testing say teachers are too invested, too close to the situation to have an unbiased opinion.  We are afraid of what the tests may say about our teaching.  We do not want accountability.  And no matter how many times I argue about the fallacy of these statements, I am still lumped into a group that few want to listen to.  So as a teacher, I have had to find my peace within this testing obsessed nation, protect my students as best as I can and save my energy for the fight I will put up for my own children.

Because as a parent, I worry.  I worry about the massive amount of time these test are taking.  I worry about the developmentally appropriateness of questions.  I worry about how they don’t actually mimic the skills that we help our students develop such as arguing one’s opinion or noticing the different facets of an answers.   I worry about how these tests will be used to further rank our children as we rank their teachers, as we rank their schools, as we rank their districts.  I worry how these test will continue to perpetuate the myth that the American public education system is a broken one and it therefore needs to be all about choice and privatization.

So I already know all of my own children will be opted out when they get to that age.  I already know that my children will not be asked to sit through hours of testing to prove something that doesn’t benefit them or change their direct instruction.  They will not be asked to help rank their school through a computerized test.   As a parent, right now, I have a larger voice than I do as a teacher.   And I will keep using that voice whenever I can, even if it only means helping the four kids that I get to call my own.  The fight has to start somewhere.

I am a passionate teacher in Oregon, Wisconsin, USA but originally from Denmark,  who has taught 4th, 5th, and 7th grade.  Proud techy geek, and mass consumer of incredible books. Creator of the Global Read Aloud Project, Co-founder of EdCamp MadWI, and believer in all children.  The second edition of my first book Passionate Learners – How to Engage and Empower Your Students” will be published by Routledge in the fall.   Second book“Empowered Schools, Empowered Students – Creating Connected and Invested Learners” is out now from Corwin Press.  Join our Passionate Learners community on Facebook and follow me on Twitter @PernilleRipp.

Be the change, being a teacher, being me, reflection, Student-centered, testing

Why I Will Not Refuse to Give the Tests

image from icanread

I was told this week to just refuse to give standardized tests.  Just like that.  And while the person who told me probably meant well in their statement, I don’t think they realize how big of an action that would be.  I have long blogged about how standardized testing such as the WKCE here is Wisconsin is not an accurate measurement of what a student really knows, but rather a snapshot of that very moment they took the test.  I have also been vocal in my opposition to what that data sometime is used for and how we end up labeling students, teachers, schools, and entire districts on a meaningless measure that does little to emulate what we really do in our classroom.  And sure, I have dreamed of refusing to simple administer it in my room.  But that’s it, a dream, because in reality it probably wouldn’t do much for anyone but me.

If I were to refuse administering these state mandated tests, I would get in trouble.  That is an absolute guarantee.  And while I have never been one to shy away from too much controversy, the kind of trouble this time would be much bigger than a write up.  I could even lose my job for failing to do my duties.  To some that may not seem like a big deal, after all, I should be standing up for my students and their rights, my own opinions, I should protect those children that I teach from the tests.  But my job is vital to my own children.  My job is our health insurance.  My job gives us just enough money so that we can pay our bills.  I wish my husband had a huge paying job, he doesn’t, and so we are a very dependent two income family.  So losing my job refusing tests just isn’t something I can rationally do and in a sense, I am not sure I should be the one refusing the tests anyway.

Teachers can try to change education as much as we want.  Many do.  We write, we speak out, we try things in our own classrooms that we hope will spread to others.  We stand up for what we believe in, we spread our message.  But in the end we are just the teachers.  The real change must come from outside the classroom, from school administration, from school boards, from government, but they will not change until one group speaks out:  parents.  The real change must come from parents.  The real opposition must come from those who entrust their children to us.  They are the ones that can decide whether a test is harmful or inaccurate.  They are the ones that can choose to opt out.   I am not the one that decides whether testing  will harm a child or not, I can have my opinions sure, but in the end the decision does not rest with me and as long as parents willingly have their children tested then my job is to test them.

So while I can dream of refusing to test my students it will only stay a dream until the parents whose children I get to teach are the ones that decide that things should change.  We may think as teachers that it is only our responsibility to speak up and that if we don’t then no one else will.  This is not true anymore.  We may be the ones that start the conversation but others have to join the fight.  Whatever they believe in.  So when I am told I should simply refuse, I hope others see how it is not that easy.  How my refusal will do little for my students and only harm to my own family.  And while I would sacrifice my life for my students if I ever had to, I will not sacrifice my job in a non-lethal situation.  I will not sacrifice the life of my own children for something that many others do not see as a big deal.  Would you?

 

 

 

Be the change, reflection, students, testing

Testing Makes Me Feel Like a Bad Teacher

image from icanread

I know I should not care, I should go on my day like it is nothing, but the truth is; standardized testing makes me feel like a bad teacher.  It shouldn’t be a big deal but anyone who has had their students sit through a MAP test will tell you; printing out that report and seeing whether the students met their projected growth score is downright anxiety producing.

Once the test is over then we stand with the repercussions; scores that were not met because the kid was having a bad day, scores that were not met because they rushed, scores that were not met because they didn’t get that one question.  And yes, scores that were not met because I didn’t do my job well enough.  The problem is; I don’t know which category a score fits into.  I can certainly take a guess but that is all it would be; a guess.  So I base my teacher performance on a score that supposedly tells me everything without really teling me much.

I take their scores and try to let them be a guide merely, forget that they will go on with the students to middle school, forget that these scores will determine where on the data wall they sit.  Forget that as much as we pretend they don’t matter, that these scores will usually mean more to their future education than any of my feedback or summative assessment ever will.  And it makes me feel like a bad teacher.
I cannot protect my students from what I fight against; the standardization of their intellect.  The standardization of their knowledge, their creativity  and their aptitude.  I cannot protect them from being labeled due to test scores.  I can only do so much within my classroom to shield them from the test obsessed education policy that seems to be driving us.  I can downplay the test but the educational system does not let me downplay the result anymore.  So I feel like a bad teacher.
I became a teacher to make a difference, not to feel bad about the tests I have to put my students through in order that someone will believe me when I say that they grew as a reader, that they grew in their math knowledge, that they grew in their intellect.  Apparently my word is not enough anymore, perhaps it never has been, now the data is what guides us.  And the data makes me feel like a bad teacher.