Latvian Physics Exam

Today was the pilot administration of Latvia’s new 12th-grade physics exam. Next year, students will be required to write either the physics or the chemistry exam as part of their secondary school leaving requirements (for those students who pursue the academic track of studies). In this post, I will take a close look at the exemplar, which can be found (in Latvian) here.

Part 1.1 (multiple choice): Assumptions and Memory

The exam opens with a question about the applicability of the accelerated motion model. It’s a promising sign, but all of the answers — a launched javelin, a parachutist in the first minute of fall, a person in an elevator, and a hammer dropped on the moon — are possible, and the question is really about the degree to which the model can be applied to these situations. For example, a javelin has a very narrow cross section, so it would be reasonable to assume it experiences very little drag force. In fact, the aerodynamics of javelins are interesting, and although their motion is non-parabolic, the reasons are not obvious (an example).

The fifth question is identical to one from an IB physics exam from a couple years ago, where the motion of a piston is presumed to be uniquely responsible for the temperature increase of a compressed ideal gas. The twelfth lists four core equations about electricity and asks which explains the need for high voltage in long-distance power lines. These types of questions are frustrating because multiple answers are reasonable, and the students are forced to try to guess what the exam-writer is thinking.

Another type of question that appears is simple recall. Question 6 requires that students remember that isochoric processes have constant volume. The 13th question seems to ask that students remember how the technology of electromagnetic inductive charging works. Question 20 is about the origin of the atoms that make up our bodies.

The third type of common question is about relationships between variables. Students are expected to recall how electric fields depend on the distances from point charges, how photon energy is related to wavelength, and how the energy of photoelectrons is related to the frequency of incident light.

These multiple choice questions are pretty standard, with questions similar to IB Physics and the GRE. The difficulty level is probably appropriate, in that students will show a nice Gaussian distribution of responses. But as a tool for educational assessment, I think these multiple-choice questions have only marginal utility.

Part 1.2 (short answer): Not For The Meticulous

The second part of the exam consists of 10 short-answer questions. In reality, it is a mix of question types that is sure to frustrate students. There are 6 multiple-choice style questions where students must choose all the correct answers, 2 ranking tasks, and 2 calculations. Problematically, some of the distractor options on the multiple-choice style problems could conceivably be relevant. For example, problem 27 shows possible light paths refracting and reflecting around a glass block. A line path that is not visibly refracted by the block is possible, depending on the index of refraction (and the refraction is exaggerated compared with what would be observed in the lab).

The biggest problem with questions like this is that, while they are accessible to very weak students and straightforward to students who have a teacher’s level of understanding of physics problems, they will not be very successful at differentiating between a student who is at 90% of a robust understanding, and a student who can intelligently guess 10%.

I’m happy that the test-writers are attempting to expand beyond multiple-choice questions, but this section will be hard for the students, and will provide little feedback about students’ ability to use their physics knowledge and skills to meaningfully create knowledge.

Part 2 (long answer): Physics Problems, With a Dash of Discussion

The long-answer section begins with the ominous note that this section will involve judging pace, but with 135 minutes to solve 5-7 problems, the question likely will not be pace so much as sustained concentration.

I want to like these problems: they combine calculations (kinetic and gravitational energies for the first, circular motion for the second) with diagrams and comparison of quantities. This fixes the problem with the first half of the exam, and allows students the opportunity to explain their answers.

However, the structures for the problems seem set up to greatly restrict the students from any sort of individual actions, knowledge construction, or creativity. The first problem requires that students draw the velocity and acceleration vectors for a pendulum — something that is usually recalled, rather than figured out.

The rest of the problems mix standard physics things-that-must-be-remembered with little calculations. There is a clear effort to make the problems meaningful and/or relevant: one concludes by asking students for a conclusion about heat loss in a house, and another is about the laser on a Mars rover. However, the overwhelming sense with these questions is that of standard physics regurgitation: draw a voltage divider, calculate the diffraction angle, identify this range of the EM spectrum, etc.

The final question (at last!) asks that students work with some data, but this is an illusion: after finding the slope from a graph, it’s the same deal as before: find the efficiency of the electric kettle.

By the time you get to the end of these problems, it is clear that they are merely standard physics problems, with just a few “explain why” questions. This is better than the mechanistic rigamarole of the IB physics exams, but not by much. Even here, there is no chance for students to construct their own meaning or use genuine critical thinking, and I suspect that most students will get most of their points by reiterating standard physics solutions to these standard physics problems.

Implications

There is a suspicion that the new physics and chemistry exams are an attempt by the Ministry of Education to add more rigour to a national curriculum that has recently come under fire after Oxford University decided not to recognize the Latvian grade 12 diploma as adequate preparation for undergraduate studies. Thus, this exam will probably give universities a better way to sort students. I’ve heard that, after promising they wouldn’t be required, many universities are asking to see students’ results for this pilot year for the exam.

However, like all standardized testing of this sort, what we will primarily find is that students attending better-funded schools, in Rīga, and with higher socioeconomic class will do better on these exams. Additionally, these exams will be felt backward into the 11th grade and earlier, as physics and chemistry teachers are increasingly under pressure to prepare students for the exam, and thus are forced to sacrifice good science teaching in favour of test preparation. The quality of meaningful science education will fall, resulting in weaker critical thinking and scientific reasoning skills across society, while universities will increasingly rely on these test scores to make admissions decisions, and disadvantaged students will be left behind.

If I’m wrong about some or all of this, or if you have perspective or insight to share, please reply or email me (danny.doucette at gmail).

Mastery vs IB Assessment

Exams make me depressed.

Every December and June (for grade 11) and December and March (for grade 12) I spend a lot of time creating an assessment that follows the objectives of my course and meaningfully assesses student proficiency in the required skills, and knowledge of the required information.

And every December and June, and December and March, the results are disappointing. It doesn’t seem to matter how much we review. I think the Modeling approach is helping with comprehension of ideas, but it isn’t translating into perfect test scores.

I’m clearly not alone in this. For IB science and math courses, the “excellent” grade of 7 is awarded to students who get more than ~70% of the questions correct on their exam (adjusted yearly, there is also the IA, etc). The passing grade (a 4) is ~40%. And according to the IB Statistical Bulletin, only 8% of students will get that elusive 7. And 39% of students won’t even get the 4.

The problem, I think, is an impedance mismatch between the IB grading system and my own mastery-oriented expectations.

The IB system is designed to assign a rank to students, for the purpose of university applications, mainly.  A 7 student needs to be clever, clever, clever. The average student, who can satisfy most of the curricular learning goals, is distinguished from the 7 student through trick questions, overly-specific grading rubrics, demanding that students memorize definitions, and even a few unsuitably-hard questions. Such a system is fine — we expect the SAT and IELTS to deliver such a ranking, for example.

My expectations as a teacher, however, and my sentiments as someone close to the learner, are a bit different. We see the examination as an opportunity to demonstrate the students’ mastery of the curricular content — not as a test to put the student in his/her place. For us, it is heartbreaking when students get less than 7 — and when the 7 they get is 70% instead of 95%.

I’m not an excellent teacher, but I’m getting better. Hopefully, soon, my students will go into an exam with full and deep understanding of everything they need to know. I hope that they achieve the scores they deserve, when they undertake that examination.

Assessment for Quick Feedback

This is a difficult time of year, as exams loom, the work piles up, and students need targeted help more than ever before. This post is to give a quick overview of a style of assessment I have been using lately. As I go along, I intend to write a follow-up describing how well it works.

Premise: students need to write summative assessments (ie: tests)
Problem: tests take a lot of time to grade, so feedback is usually too slow to be effective
Solution: a two-page test that separates easy-to-grade questions (multiple choice, calculate-a-number) from hard-to-grade questions (diagrams, solving problems, writing responses).

A two-page test is, as the name suggests, written on two pages of paper, stapled together. The first page consists of questions where:

(a) the answer can be instantly checked for correctness
(b) incorrect answers are anticipated, and understood to reflect some common, relevant misunderstandings

Critically, the first page needs to be something that can be checked for correctness in about 10 seconds. I design for this by having the students write their answers on blank lines on the right side of the page.

The second page consists of the regular sorts of questions where you need a few minutes to read, figure out what the student was thinking, and so forth.

When the student submits the test, the two pages get pulled apart. I quickly mark the first page. There is usually a 10-20 minute window between my first student handing in their work, and the last straggler wrapping up, so if I can mark while the tests are coming in, I can return the first page to the students about a minute after the last student finishes.

Now, the students get their near-instant feedback, and we have some time to go over the questions that gave most students problems. If I designed the test well, that means our post-test class time is spent directly addressing some of the major conceptual hurdles in the content.

As for the second page… I mark those in the regular way: when I can make time for it.

Exams Make Students Sick

Every March we run a two-week “Mock” exam session for our 12th-grade students. The goal is to simulate the IB DP exams in May, while also conducting leaving examinations for students who will not be writing the DP exams. It’s a stressful time. In some classes, time limitations have meant that the teachers were still teaching new material just last week. Given the stress and the time of year, it’s no surprise that the students are typically quite sick by the end of it.

Worse, perhaps, students come to school during the Mocks when they would normally stay home, which both exacerbates their own illness and spreads the contagion.

Today, more half of the students in the exam room were conspicuously ill. They took it in turns to sniffle, cough, sneeze, and blow their noses, creating a sort of auditory whack-a-mole of the common cold. One student spent a week in hospital prior to the exams, and will be returning as soon as they finish: such is the pressure we place on our teenage students.

Other than moving toward a holistic (ie: portfolios) or cumulative (ie: computer-based) assessment paradigm, I don’t see a solution. As long as there is high-stakes testing, there will be stress. And anywhere there is stress, there are physical repercussions.

Choice: Dan Schwartz’s Vision

When people find out that I teach high school physics or maths, the response is usually, “Oh, I was terrible at that.” There are two levels at which to analyze that statement. On the face, it’s a sort of mutual apology: “I’m sorry I didn’t learn the area of knowledge you care about” followed by my “I’m sorry that physics teachers like me are bad at their jobs.”

At a deeper level, one needs to recognize that most people don’t feel shame at their mathematical shortcomings; instead, they consider failure at mathematics to be an inevitable part of the educational process. In this light, the casual dinner party comment, “Oh, I was terrible at maths,” must be interpreted as “Oh, I was terrible at maths, but my life is alright, therefore maths isn’t important.”

I’ve seen this attitude, in varying shades of forthrightness, from community and business leaders, from education administrators, from fellow teachers and — of course — from parents. I’m sympathetic — it’s a rare day that I calculate an integral or use trigonometry explicitly — but there is a lot of value in learning mathematics (which I outline in this video).

This brings us to the point I’d like to make in this post: in math, physics, and most other subjects, the true goal of education is substantially different from the goals perceived by society. We don’t learn math so we can “solve for x” in a hundred different ways any more than we learn history so we can recount the denouement of the Vietnam War.

Stanford professor Dan Schwartz advocates a new approach to education, and particularly assessment, based not on knowledge or shallow domain-specific skills, but on the choices we make. After all,

It is our choices, Harry, that show what we truly are, far more than our abilities.
– Albus Dumbledore in JK Rowling’s Harry Potter and the Chamber of Secrets

I think Professor Schwartz does his thesis a disservice by identifying choice as the theme, as his programme seems more concerned with a holistic, process-based approach to assessment (which thus informs learning). I’ll try to identify the major salient points of his approach:

1. Assessment in the current education system, even at its best, is obsessed with knowledge. However, the end-goals of education are about much, much more than knowledge.
2. With new technologies, we have the ability to create activities that produce an “assessment of processes”.
3. These new assessments will allow us to (finally?) evaluate the effectiveness of instructional techniques and strategies, which will simplify the identification of best practices.

This should jive pretty well with the emerging trends (a) toward technology-based education systems (MOOCs, animations, etc), and (b) toward data-driven schools and data-mining, more generally.

Back to choice. Professor Schwartz correctly sees that the subplot in all this drama is the question of what the students are actually doing as they learn, and how they’ll take their learning into the world when they leave the classroom. Student autonomy seems to be increasingly important for educating participants in global democracies, in addition to the myriad of educational benefits that are experienced when students feel free to learn in a comfortable fashion.

However, there is a contradiction inherent in this argument: if a student is making poor choices, does the school not have a responsibility to correct those inappropriate actions? For example, if a student is writing with poor technique, one would expect a teacher to insist s/he correct her/his approach. On the other hand, if that same student organizes a Marxist student club, should the school intervene? And to what extent should teachers intervene: is classical or operant conditioning appropriate as a discipline-building strategy in a high school classroom?