Back in August, I wrote about my attempt to understand how English communication was getting in the way of measuring scientific reasoning skills. I assigned my students 20 of the questions from Lawson’s Classroom Test of Scientific Reasoning (CTSR), dropping the four linguistically toughest. Once they students had finished the test in the regular way, I demonstrated the scenarios one at a time to provide contextualization.
You can read more about the assessment and the results here.
There are two things that I’ve been meaning to adjust about my results. First, I forgot in my analysis that a correct response should only be noted as correct if the response and the follow-up “why” question are both correct. Thus, instead of a score out of 20, I should have a score out of 10. Second, I wanted to add a comparison to norms. In the graph below, the “Norm” line comes from a scaled (from 13 to 10) version of the results compiled by the Frameworks for Inquiry project. This line corresponds to the scores of 3800 American students from grades 10 to 12, and has meaningfully been connected with such things as Piagetian developmental stages.
I think it is clear that the blue line (my students in the normal testing situation) are pretty close to the norm data, while the post-demonstration results (in red) look to be quite different on this cumulative frequency graph. This should be taken as an indication that linguistic difficulties are an important factor in determining the score of the CTSR.