Tag Archives: education

Are we using Study Halls effectively?

My school has an addiction to study halls. They are used to pad out the master schedule for students in our middle school. By the time they get to grade 11, our students might even have half a day in study hall.

Unfortunately, value of this self-directed time is unclear. In spite of a school policy that requires students either do classwork or read a book, students often spend their study hall time watching videos, chatting with friends, and neglecting their schoolwork. Policing study halls is difficult for the monitors because the students see study hall as “their” time, and resent being told what to do.

In order to gain a better understanding of how the students were using their study halls, I conducted a three-week study that visited 30 study hall classes. For as many study hall periods as possible, I dropped by and categorized each student accordingly:

Fully On-Task: The student is currently working on an assignment and clearly focused on his/her task. If the student is working with a peer, their discussion must be tightly bound to their assignment.

Distracted: The student has work out, but is not fully focused on the task. Distractions include chatting with peers or teachers, viewing non-task-related material on a computer, and using a phone.

No Work: There is no evidence of the student attempting to work on a school assignment during this period.

a

Figure 1: Student work in study halls over a three-week period.

The pie chart above depicts how students used the study halls during this investigation. Each student in each study hall is counted equally.

To compile and compare the data, I calculated a quantity called fractional studiousness, as follows. Fractional studiousness is a measure of the usefulness  of a study hall period.

g

My colleague suggested that the students often arrive on Monday with their homework completed and that they do little on Friday because due dates are several days away. I’d add that, on occasion, students have assignment deadlines extended until the end of the day on Friday — this might explain the large variability in studiousness on that day. Figure 2 shows a classic depiction of the “hump”-shaped workweek.

How studious are study halls over the course of a week?

Figure 2: How studious are study halls over the course of a week?

I was also curious about how students performed in their study halls over the course of a day. In figure 3, you can see lower performance in the afternoon, perhaps attributable to post-lunch sleepiness. There’s also a hint of lazy mornings.

Figure 4: Studiousness over the course of a school day. Breaks are indicated in red.

Figure 3: Studiousness over the course of a school day. Breaks are indicated in red.

Another way to slice these data is to isolate the worst study hall blocks. In this study, there were 12 worst study halls that have a studiousness of 0.4 or less. All the others have a studiousness of at least 0.5. Note that some of the data points in figures 3 and 4 coincide.

Figure 5: Distribution of the worst study hall periods.

Figure 4: Distribution of the worst study hall periods during the course of the week.

In the week-long view, the trend toward a mid-week efficiency “hump” is retained. The day-long graph shows ineffective mornings, and mid-afternoon blahs.

Figure 6:

Figure 5: Distribution of the worst study hall periods during the course of the day.

I was also curious about the role of technology in studiousness during study halls. Naively, one might suggest that computers offer students a plethora of ways to be distracted. For the purpose of this study, I counted laptop computers and tablets (iPad, etc) but not smartphones (iPhone). My belief is that the former is typically viewed as a tool (for writing essays, researching, reading electronic texts, etc) while the latter is typically viewed as a communication and entertainment device.

Figure 6 shows comparisons of the same study hall on two different weeks. This should offer a control against the studiousness of individual students. For example, the red line shows a particular group of students had a studiousness of 0.5 on a week when 20% had computers, and a lower studiousness of 0.2 on a week when 50% had computers. In this case, we can infer that more computers means more distraction.

Figure 6: How does computer use affect studiousness?

However, the overall trend is the opposite: of the 9 study halls for which I was able to obtain data, 6 show a positive correlation between computer use and studiousness (and 1 is agnostic). In other words, more computers means more students working. The two exceptional study halls are populated primarily by a small group of 11th-grade students new to our school.

This phenomenon can be explained by the fact that much of the students’ work requires the use of computers. When students don’t have computers available, they allow themselves to become distracted by other factors, such as their phones or chatting with others.

Conclusions:

  1. Study hall time is being used effectively only 30% of the time.
  2. 28% of study hall periods are used for non-academic purposes.
  3. Distractions are a key problem for study halls.
  4. The midweek is used more efficiently than Monday and Thursday. Friday is used effectively by some students, perhaps because of end-of-week deadlines.
  5. Study halls are more productive in the morning. Block 6 (also called block 3B at our school) is particularly ineffective.
  6. Generally, computer use in study halls correlates with improved studiousness. To improve the usefulness of study halls, the school should ensure students have computers available for use during study hall periods.
Advertisements

Scrum Education

Last spring, a visitor to our school introduced me to the Scrum framework for software development. The name “scrum” (borrowed from rugby) represents the goal of getting a group to push together for a short burst of time. Here are some highlights of the Scrum method:

  • Goals are prioritized and assigned estimated completion times
  • A handful of goals are chosen for a “sprint” that typically lasts two weeks
  • The team meets daily to resolve issues
  • Progress is tracked using a “burndown” chart (more below)
  • Once the sprint has been completed, the process begins again

The most interesting part of this, from my perspective, is how the burndown chart provides a clear picture of the team’s progress. Could I do something similar with student assessment? It turns out that Google Apps for Education [GAFE] allows you to do this, although not easily.

I started by creating a Form containing questions lifted from, and inspired by, the IB Physics course guide (ie: my content standards). The example below is a knowledge question; other questions required that students applied their understanding of concepts or employed formulae.

a

The results from that Form are saved automatically on a Spreadsheet in Google Drive. I use the Spreadsheet to create the burndown chart using the following process:

  1. Copy all the data to a new sheet, so I can add new columns, ie:
    =arrayformula(‘Sheet1’!A1:L99)
  2. Create 10 new columns (one for each question) that compare the student’s answer with the first submitted answer (which are my own answers), printing a 1 if correct and a 0 otherwise, ie:
    =if(C3=C$2,1,0)
  3. Tally the score in an 11th new column, ie:
    =sum(M3:V3)
  4. Create a new sheet for each student. Cell A1 contains the student’s GAFE email address. The two columns we need are the test date and the test score. For the date, filter through all the submissions, looking for the correct student, ie:
    =filter(‘Sheet2′!A1:A127,’Sheet2’!B1:B127=A1)
  5. For the score, filter through the submissions, looking for the correct student, ie:
    =filter(‘Sheet2′!W1:W127,’Sheet2’!B1:B127=A1)
  6. To make the graph, we want the number of questions that were wrong, not the score. Also, to make the graph automatically, we need to make sure that there’s no number in this new column unless it corresponds to an actual score. Thus,
    =if(D4<>””,10-D4,””)
  7. The burndown chart is a column graph. Here’s what mine look like:

b

 

Notice that I put a “100% wrong” bar at the start of the chart, for the sake of comparison. The 18th of September is empty because the student got all the questions correct.

Now, in order to share this chart with the student, I created a page on our class GAFE Site that is visible only to the student and I. The chart is inserted into the student’s individual page. (I also use these pages for our gamified points system — more about that in a future post)

Conclusions:

  1. Since I have to create a spreadsheet and a page for each student, this method doesn’t scale. I could use a script to automate the spreadsheet work, but (probably correctly!) Google doesn’t allow automated website creation on Sites.
  2. Once the students have seen the same question a couple of times, their understanding is no longer being evaluated effectively. Knowledge-based questions seem to hold up better (after all, the point is being able to answer such a question correctly) but have limited educational value.
  3. The Scrum method uses remaining man-hours as the value on the y-axis of these burndown charts. Knowledge/understanding, however, surely is not linearly related to the amount of time spent learning. Thus, a key insight to be gained from these charts — the ability to estimate a completion date — is lost.
  4. Perhaps unsurprisingly to those who are familiar with GAFE, this system hiccups. One day, the scores weren’t (automatically!) saved from the Form into the Spreadsheet for several hours. Students who are logged into personal Gmail accounts (in addition to their school GAFE accounts) experience bizarre and unpredictable errors.

To see my full burndown quizzes (if they’re still online), check out my physics and maths class web sites.

My Day as a Student

Lately, I’ve been concerned about my students’ engagement at school. To understand what their day is really like, I decided to attend school for a day as if I were a student.

I chose a Friday because my schedule is (anomalously) light. Aside from 2.25 hours of classes and a 15-minute stint monitoring the playground, I tried to follow a typical schedule, do the things students do, and think the way students think. Here are my conclusions:

1. Information Overload

As a teacher of two subjects (physics and math), I am free to spend most of my time thinking in the characteristic ways of those subjects. Students, on the other hand, are bombarded with ideas from diverse disciplines. I had classes in economics, philosophy, physics, and history. Each class had its own set of perspectives, contexts, background information, vocabulary, and procedures. In each classroom, I was expected to internalize a “chunk” of new ideas.

Unfortunately, the mind doesn’t work like that. For example, I was still ruminating on some ideas from economics when I was supposed to be working on an essay in philosophy. Not only did this disturb my focus, but I was unable to “go deep” with my thinking because I still had mental loose ends from the previous class.

There are four conclusions to be drawn from this. First, teachers shouldn’t be disheartened if they start a period and the class seems to have had their memories of earlier learning erased. In all likelihood, they are simply coming off a cognitively challenging lesson, and it will take a few minutes to re-orient.

Second, teachers should ensure they build off the students’ background knowledge. This will make it easier for students to build a robust gestalt in each subject area. Likewise, explicit references to background knowledge will help students locate themselves in the appropriate context as the lesson begins.

Third, collaboration between subject areas may help students transfer perspectives between disciplines, which would ease re-orientation.

Fourth, the administration should attempt to maximize the number of 1.5-hour “full blocks” when drawing up the schedule for high school students.

2. Sitting

In most classrooms, the teacher is walking around and the students are sitting. Here’s the timetable from my day as a student:

8:30 – sit down
9:15 – walk across hallway, then sit down
10:00 – walk down hallway, sit down in wrong room, walk to next room, sit down
10:50 – walk into hallway, back into classroom, sit down
11:35 – walk to cafeteria, get food, sit down and eat
12:05 – walk around outside for ten minutes, then walk to class and sit down
13:00 – walk down hallway, then sit down
13:45 – walk around outside for fifteen minutes
14:00 – walk to classroom, then sit down
15:30 – walk outside and leave

As you can see, I spent approximately 30 minutes (out of 7 hours) on my feet, and the rest of the time sitting. I sometimes found myself getting sleepy because I was stationary for so long (especially in the morning), and getting fidgety because the chairs were uncomfortable (especially in the afternoon). Physical growth during adolescence means more aches and discomfort as young bodies adapt to larger bones, so surely the students have it worse than I did.

Two years ago, I bought a set of seat cushions for my classroom, but due to some reorganization, I’m now floating between classrooms (and the cushions have magically vanished). The cushions aren’t significantly more comfortable, but I think they might have made a difference after a few hours.

It will be hard, but I’m going to try to incorporate more physical activity in my classes. I think that moving around and getting the blood flowing is the only way to keep student minds alert and primed.

3. Breaks

Our school has a 5 minute morning recess, a 40 minute lunch break, and a 15 minute afternoon recess. The lunch break seemed like it was long enough for me to eat, relax, catch my breath, and reflect just a bit on the day. The afternoon break allowed me enough time to get some fresh air and re-orient (see point 1 above).

Students often complain about being hungry during the morning or afternoon breaks. At both of those times, it seemed like there was enough time to eat a snack. I am usually a ravenous person (I haven’t yet lost my teenage hunger) but between a moderate breakfast and a solid lunch, I wasn’t hungry at any point during the day.

4. Listening versus Conversing

As a teacher, I’ve always known that lecturing is ineffective. My student day crystallized that conclusion. Although I tried my best to follow along and take notes, I could rarely get myself invested in what the teacher was saying when s/he was addressing the whole class. On the other hand, when I had to directly interact with one or two other people, there was no opportunity for “checking out” — if I was to be in the conversation, I would have to keep up and contribute.

The difficulty is getting groups to function effectively. I’m certainly going to try harder to make that happen.

5. Computers

There are three students in our 12th-grade class who don’t have functioning laptop computers. They are at a significant disadvantage because many of the classes required that students use a computer to do their work.

The school has a few (poorly functioning) laptops that can be borrowed, but these are in sufficiently high demand that they are difficult to get. As a fellow student, I was able to see the difficulty and exasperation involved in trying to get these things to work.

I don’t know how, but we’ve got to get computers into the hands of those students who don’t have them. For modern students in a contemporary school, it’s simply impossible to operate effectively without that invaluable tool.

6. Lateness

My last point is partly a reflection on life as a teacher. As I traveled from one classroom to another, I was frequently beset by fellow teachers and students who needed help with one thing or the other. Few of these discussions were overly deep or involved, but they involved the transfer of a small amount of vital information.

The result of these quick chats was that I was late for three of my eight classes. Embarrassing! At our school we have little tolerance for lateness of any form. I think that I’m going to be a little more willing to listen to my students’ excuses for being late in the future.

Thanks to M, G, & L for letting me join their (fantastic!) classes.

Khan Academy: A Misstep Forward

Popular accounts of education reformation are both ubiquitous and indistinguishable: an enthusiastic young person with little teaching experience tries something new, documents an increase in some particular metric (although this is often statistically insignificant or unquantifiable), and concludes that the entire education system doesn’t adopt their particular new approach because of the momentum of the entrenched orthodoxy. Massive Open Online Courses [MOOCs] such as Coursera and Udacity are providing a large-scale example of such a hollow revolution, while smaller versions run the gamut from SCF in my previous post to my own over-broad conclusions about how inspiration can provide the key foothold for students in developing nations.

I was disappointed to discover, when reading through Salman Khan’s new book, The One World School House [OWSH], that Khan Academy is founded on precisely such a misapprehension.

A bit of background: in 2004, Salman Khan agreed to help his 12-year-old cousin study math. The videos (hosted on YouTube) and Khan’s math problem-generating web software became popular. Before long, Khan quit his job as a financial analyst and began working on the eponymous Khan Academy full-time.

Khan’s vision of contemporary education is not flattering. In OWSH, he argues that American schools perpetuate a Prussian model of education — not only in the sense that school is a public institution, but also in the sense that the nefarious goal of public education is to grind the creativity out of the populace while instilling the sort of half-baked enlightenment that leads to productive workers who have little interest in rebelling against the existing social order (taking without a hint of salt the arguments of John Taylor Gatto). Khan also claims that most students receive a “Swiss Cheese” education: their learning is so full of gaps and holes that attempts to build on earlier studies will be unsuccessful.

More  importantly, Khan demonstrates throughout OWSH that his understanding of both modern schooling and the practice of education is desks-in-rows classrooms with stern  teachers who lecture the entire class period and give no consideration to any skills outside the purview of their narrow content area.

On the positive side (from the viewpoint of a progressive educator) Khan has little use for summative assessments, opposes burdensome homework loads, and strongly advocates for mastery learning (although his understanding of this seems basic or naive).

As a teacher, he first two-third of OWSH felt like running a gauntlet. Are teachers really destroying creativity, quashing adolescence, and stagnating the talent pool? Certainly not. Khan’s education diet doesn’t include such staples as instructional differentiation (which only appears later in the text as a selling-point of his new paradigm), discipline, practicing social skills, or even the notion of classroom activities.

That brings us to Khan Academy, in its current form. The site is heavily gamified. There are points, badges, avatars, goals, and other elements. Sub-topics have a few videos and maybe an automatic exercise. The exercises are built (rather cleverly) with jQuery and basically allow for random numbers to be put into the same problem framework. Here’s an example:

Daniel starts counting at 28, and he counts by fives.
If 28 is the 1st number that Daniel counts. what is the 14th number that he counts?

The numbers 28, “fives”, and 14 are picked randomly. The criteria for mastery have evolved, from 10-in-a-row, to a Bayesian estimate of the likelihood of the next problem being correct, and now back to 5-in-a-row but with four levels of reiteration before the concept is presumed to be fully mastered.

I used Khan Academy as an instructional and assessment tool with an 11th-grade math class for two months this past spring. Here were the highlights:

  1. Most students rapidly grokked the concept, but less than a tenth really bought into the gamification aspect.
  2. Many students became frustrated because mastery was highly contingent on getting a long “streak” of correct answers.
  3. The videos are boring and slow. I limit my lecturing to less than a quarter of class-time (and usually much less), and even those lectures are Socratic in nature, so students who had to sit through Khan’s long, drawn-out examples rapidly switched off — especially because they were using a computer capable of so much more interesting stuff!
  4. The Khan Academy approach is time-consuming, as it focuses on deep mastery of skills. While I fully support deep mastery of core concepts, I was disappointed with the slow progress we made. A fairly-intuitive unit on probability that normally takes two weeks was stretched out to a month and a half.
  5. The exercises are still too few and far between to constitute real practice. Important concepts had no exercises. Developing exercises is a slow process, granted, but it must surely be seen as a core competency of the Khan Academy, where only one 1 full-timer (out of 45) and 3 interns (out of 33) are indicated as exercise developers.

The Khan Academy’s user-base is broad, and thousands have attributed various kinds of success to the web site. While I don’t think it is ready for prime-time use as a major instructional tool in most classrooms (flipped or otherwise), that doesn’t mean I’m not optimistic about Khan Academy’s future.

Which brings us to the real reason to read OWSH: Khan’s considered vision of how education could look in a world that fully embraced today’s technological competencies.

  1. Khan suggested amalgamating 4-5 classrooms into a single large area. The 4-5 teachers would represent different backgrounds and expertises. Students would be self-motivated to do their learning with computers (à la Khan Academy) and would spend the rest of their day — Khan believes that individualized instruction will be nearly an order of magnitude more efficient — working on group projects, creating things, and tutoring younger students. The discipline issues are too huge for this to work. Students come to school with too much baggage: even the most optimistic educator cannot believe that a fairy-tale workspace would keep kids from unpacking all their troubles, emotions, and needs.
  2. Although he writes frequently in OWSH about the “artificial separation of traditional academic subjects”, it’s not clear to me that either Khan Academy or Khan’s vision do much to blur the boundaries. The academic subjects are divided traditionally on the Khan Academy web page. The current approach of interdisciplinary projects, as in the Group 4 Project in the International Baccalaureate programme, seems the best way forward short of major curricular revision.
  3. Khan advocates for classes that consist of students of mixed ages, so the elders can model behaviour for the younger members. It’s a good idea, and inter-age communities should be cultivated in schools, but this isn’t a behavioural panacea.
  4. There is the debate about flipping the classroom. I’ll return to this in a later post.
  5. Khan suggests eliminating the summer break and allowing students to take vacation time, as do many adults. Once we can get to the point where instruction is fully differentiated, this is workable — although the idea is not new.
  6. Khan has failed to convince me that his approach will reach our poorest students: those without computer access. His suggestion that middle-class kids would allow the poor to use their technology during the night shows a poor understanding of class dynamics and social realities. Instead, we should perhaps target the next wave of “African cell phones” — those electronic devices that spread because they are sufficiently affordable and valuable that even the poor farmer couldn’t afford not to have one.
  7. Perhaps one of the greatest strengths of computer-based, individualized instruction is how it allows for meaningful credentials. Perhaps in conjunction with portfolios, this could finally replace leaving exams, entrance exams, and other ineffective measures of competence for schools, universities, and employers.

This brings us full circle, as credentialing is also likely to be the defining issue of MOOCs. Like MOOCs, computer-mediated, individualized instruction is a trick snake that cannot be stuffed back into the can. In one form or another, it will persist. Whether in the form of good education, or mirroring a cartoonish vision of school learning cultivated by non-educators, computer-based instruction is here to stay.

I’d suggest the following should happen before we shift to full differentiation in our classrooms:

  1. High-quality videos. I think John & Hank Green have the right idea (although perhaps too much of their trademark manic energy) with Crash Course.
  2. Videos should be interspersed with comprehension quizzes, as offered in Coursera and elsewhere. Ultimately, videos should branch depending on the quizzes: obviously-wrong answers lead to a more fundamental approach, small-misunderstanding answers should include rectifications of those errors, and so forth.
  3. Textbook publishers should recognize that their greatest asset is not the ideas contained in their textbooks but the volume of assessment materials they possess: I’m talking about licensing data banks of questions.
  4. Figure out how to deal with motivation. Face-to-face contact provides a very strong motivational tool that hasn’t yet been replicated online. This is why the drop-out rate in MOOCs is so high, and why the vast majority of Khan Academy users don’t stick with it until they master calculus.