Month: December 2013

Are international educational tests, like the PISA, asking the right questions (with the right answers)?

My friend Ruth Ackerman cross-posted a video from the AFT on her Facebook page today, which questions some of the conclusions that some groups make from the mediocre U.S. performance on the PISA. (And I’m not going to dive into that politically charged question right now.)

I think we need to step back further, and ask if the PISA is measuring the “right” knowledge.  To try and answer this for myself, I found a set of sample questions for the PISA.  And I started to go through and answer the questions.   And I found one question where the PISA is wrong.

The question basically is about a science article where it talks about scientists that cloned a cow into five calves.  And then asks if the calves have the same genes, the same gender, and the same color.  The question appeared to want to see if students could infer the answer, as it didn’t explicitly give this information in the article that it had the student read.  And the question wanted to have the answer being “yes” for each one.

BUT, while it may seem to be a reasonable assumption to make that the answer would be “yes” for each, it isn’t necessarily the case for color of the cows.  I say this, because people who have cloned their cats have found that they often turn out to be a different color than their original cat, so I suspect this same thing could happen with cows.  So what if a student knew had this advanced knowledge and marked “no”…  Well, they would have been said to be wrong, when in fact the PISA is likely wrong.  (I say likely, as I am not sure that cat and cow clones would behave similarly when it comes to color, but it is no more or less a reasonable or unreasonable assumption as the one the PISA wants the student to make.)

Update: I emailed PISA (edu.pisa@oecd.org) this blog entry, but never heard back from them.