Last spring, a visitor to our school introduced me to the Scrum framework for software development. The name “scrum” (borrowed from rugby) represents the goal of getting a group to push together for a short burst of time. Here are some highlights of the Scrum method:
- Goals are prioritized and assigned estimated completion times
- A handful of goals are chosen for a “sprint” that typically lasts two weeks
- The team meets daily to resolve issues
- Progress is tracked using a “burndown” chart (more below)
- Once the sprint has been completed, the process begins again
The most interesting part of this, from my perspective, is how the burndown chart provides a clear picture of the team’s progress. Could I do something similar with student assessment? It turns out that Google Apps for Education [GAFE] allows you to do this, although not easily.
I started by creating a Form containing questions lifted from, and inspired by, the IB Physics course guide (ie: my content standards). The example below is a knowledge question; other questions required that students applied their understanding of concepts or employed formulae.
The results from that Form are saved automatically on a Spreadsheet in Google Drive. I use the Spreadsheet to create the burndown chart using the following process:
- Copy all the data to a new sheet, so I can add new columns, ie:
- Create 10 new columns (one for each question) that compare the student’s answer with the first submitted answer (which are my own answers), printing a 1 if correct and a 0 otherwise, ie:
- Tally the score in an 11th new column, ie:
- Create a new sheet for each student. Cell A1 contains the student’s GAFE email address. The two columns we need are the test date and the test score. For the date, filter through all the submissions, looking for the correct student, ie:
- For the score, filter through the submissions, looking for the correct student, ie:
- To make the graph, we want the number of questions that were wrong, not the score. Also, to make the graph automatically, we need to make sure that there’s no number in this new column unless it corresponds to an actual score. Thus,
- The burndown chart is a column graph. Here’s what mine look like:
Notice that I put a “100% wrong” bar at the start of the chart, for the sake of comparison. The 18th of September is empty because the student got all the questions correct.
Now, in order to share this chart with the student, I created a page on our class GAFE Site that is visible only to the student and I. The chart is inserted into the student’s individual page. (I also use these pages for our gamified points system — more about that in a future post)
- Since I have to create a spreadsheet and a page for each student, this method doesn’t scale. I could use a script to automate the spreadsheet work, but (probably correctly!) Google doesn’t allow automated website creation on Sites.
- Once the students have seen the same question a couple of times, their understanding is no longer being evaluated effectively. Knowledge-based questions seem to hold up better (after all, the point is being able to answer such a question correctly) but have limited educational value.
- The Scrum method uses remaining man-hours as the value on the y-axis of these burndown charts. Knowledge/understanding, however, surely is not linearly related to the amount of time spent learning. Thus, a key insight to be gained from these charts — the ability to estimate a completion date — is lost.
- Perhaps unsurprisingly to those who are familiar with GAFE, this system hiccups. One day, the scores weren’t (automatically!) saved from the Form into the Spreadsheet for several hours. Students who are logged into personal Gmail accounts (in addition to their school GAFE accounts) experience bizarre and unpredictable errors.