What have two years of interrupted schooling taught us about learning?


Covid-19 and the political efforts to contain it have radically disrupted the daily lives of children around the world. Student learning became a concern early on, with the World Bank, European Commission and others predicting a steep decline in progress (Azevedo et al, 2020; Di Pietro et al, 2020). This article highlights how these predictions have largely come true and shows that efforts to reverse the damage have had only partial success.

The effects of the pandemic on education are commonly referred to as “learning losses”, but here we call learning delays a “learning deficit”. This is more accurate given that in most cases students will not come back knowing less than before the pandemic. More generally, they will have made progress, but less than they would have in a normal year.

Although the data continues to come in, the evidence for a Covid-19 learning deficit is clear. Deficits opened at the start of the pandemic and they have remained largely constant. Children and young people from socio-economically disadvantaged backgrounds are the most vulnerable.

Urgent political action is needed to address these setbacks. This should target in particular, but not exclusively, disadvantaged students.

What is needed to assess the Covid-19 learning deficit?

To understand the full extent of the learning deficit, it is important to first ask what kind of evidence is needed and why it is hard to come by.

Frameworks for tracking student progress are underdeveloped and many existing tracking systems have been disrupted by the pandemic. A key lesson from the past two years is that we need to collect data on learning with the same commitment as we do on the economy or health care.

An early influential study from the Netherlands has become something of a benchmark (Engzell et al, 2021). The researchers studied learning during the first months of school closure, with national exams taking place twice a year.

In 2020, this happened just before schools closed in early spring and then when students returned to class in the summer. This allowed the researchers to assess the progress the students had made at home during this period and compare it with the same period in previous years, when the students were in school normally.

The study reports two main findings. First, despite tremendous efforts by parents and teachers, students learned very little during the eight weeks of homeschooling. Not only that, but students whose parents were less educated suffered the greatest setbacks.

Most other data on learning deficit are incomplete because they do not follow the same students over time. What if only some students returned to take tests after a period of school closures? How do we know how these students behaved before the pandemic? Were the types of tests used to assess students during the pandemic comparable to the tests used before the pandemic?

Many studies also use school samples that are not representative of the entire student population. Despite these limitations, most other studies tend to come to similar conclusions.

What is the evidence for the learning deficit of Covid-19?

Several other studies have highlighted the effects of Covid-19 on learning. A recent systematic review identifies over 30 studies in 12 countries that use standardized test scores from primary or secondary school students (Betthäuser et al, 2022). To arrive at this number, the authors reviewed more than 5,000 scientific studies and conducted a search for studies citing or being cited by the relevant sources initially identified.

Figure 1: Estimates of learning deficits by date and country

Source: data compiled by Betthäuser et al, 2022

Figure 1 shows the estimated learning deficit that each study arrived at, separately by country and date of measurement. The same study often contains more than one estimate (for example, for different subjects or for pupils of different ages). The horizontal axis shows the measurement time and the vertical axis shows the size of the estimated learning deficit. The size of the circles reflects the sample size of the study, with larger studies represented by larger circles.

On average throughout the period, the estimated learning deficit is 0.17 standard deviations (SD) – a statistical measure of effect size. Students generally improve their performance by 0.3 to 0.5 SD per year under normal circumstances.

Therefore, a learning deficit of 0.17 SD can be interpreted to mean that students have lost about 43% of a school year’s learning (0.17/0.4). That’s about double what the Dutch study found after eight weeks of distance learning. Most UK study estimates are close to this average (see list compiled by the Education Endowment Foundation).

The estimated learning deficit does not change over time: the mean in Figure 1 is constant throughout the observed period.

What should we make of this? One interpretation is that the worst learning deficit happened early. It was a time of chaos when no one knew how long school closures would last and there were few opportunities to prepare. After that, teachers, students, and parents became better able to cope with remote learning, but not enough to compensate for the initial setback.

What is the evidence of increased inequality?

One of the main conclusions of the study conducted in the Netherlands was that students from less educated homes suffered from the greatest learning deficit.

Several other studies have looked at how the pandemic has affected students from different backgrounds. This evidence is more difficult to summarize because different studies use different measures – such as eligibility for free school meals, family income or parental education – and the results cannot easily be compared on the same scale. Nevertheless, we can count the number of studies that have found evidence of increased inequality.

Figure 2: Evidence of the effects of Covid-19 on educational inequalities

Source: data compiled by Betthäuser et al, 2022

Figure 2 shows that there is strong evidence of increased socio-economic inequalities in learning as a result of Covid-19. Each circle or square refers to an estimate, and the results are separated by when the student’s progress was measured, in which subject, and in which grade.

The evidence is clear: the vast majority of studies find that socio-economic inequalities have widened. This holds true at every stage of the pandemic, for both math and reading, in primary and secondary education, and regardless of how socio-economic background has been measured.

What are the broader lessons and what needs to be done?

So what have two years of interrupted schooling taught us about learning? If anyone doubted the value of the hard work of teachers and young people in classrooms around the world, their doubts should have been dispelled.

Findings on the Covid-19 learning deficit underscore that, like so many things people often take for granted, the day-to-day functioning of our schools is fragile and dependent on continued effort, dedication and goodwill. of everyone involved.

There are three challenges to overcome. First, in the vast majority of countries around the world, we still lack data on the impact of the pandemic on student learning.

We can perhaps learn from the Netherlands in the way researchers were able to monitor and assess the situation as it unfolded.

We should also ask ourselves what structures could be put in place to ensure that we protect young people in times of crisis and give them the resources they need. If governments are serious about schooling, they must collect data on an ongoing basis and make it available to researchers for evaluation.

The second challenge is related to limiting the long-term consequences of the learning deficit caused by Covid-19. The picture of a stable learning deficit is not as dramatic as many had feared. Initially, it was feared that small setbacks would accumulate into large losses over time (Kaffenberger, 2021; Fuchs-Schündeln et al, 2021).

This fear is not borne out by the data. But if we are to repair the initial damage of the pandemic, we must step up our efforts. In the Netherlands, the government has earmarked €8.5 billion for an education recovery plan, of which €5.8 billion is earmarked for primary and secondary education. This package contains many things that can be emulated by other countries, such as more school staff and increased mental health support and summer schools.

The third challenge is whether we can learn anything from countries where the learning deficit was less pronounced or even absent.

A well-designed Danish study found surprising results (Birkelund and Karlson, 2021). Secondary students experienced a learning deficit approaching that of the Netherlands, but younger students improved their performance by an even greater margin. As older students stayed home longer, younger students benefited from increased teacher resources and density. Another factor may be that the Danish welfare state shielded parents from economic uncertainty.

Either way, these findings suggest that the learning deficit of Covid-19 can be prevented or reversed, if policy makers show sufficient determination.

Where can I find out more?

Who are the experts on this issue?

  • By Engzell
  • Bastian A. Betthauser
  • Simon Bourges
  • Sandra McNally
  • John Jerrim
  • Hans-Henrik Sievertsen
  • Lindsey Macmillan
  • Stephane Machin
Authors: Per Engzell, Bastian A. Betthäuser, Anders Bach-Mortensen
Photo by Chris Ryan on iStock.

About Author

Comments are closed.