Secondary Education: OECD PISA Questionnaires

Influence of socio-cultural-economic factors in 15-year-olds' aptitude for math and reading

The PISA questionnaires provide a vantage point on the educational situation of European adolescents over a period of more than two decades. Our analysis focuses on both cross-country comparisons and the role of social stratification within each country.

Variation of math and reading skills among school systems of EU countries

A comparison of math and reading scores on the PISA literacy scale reveals some relevant patterns across EU countries:

• Flattening to the mean: Average scores in math and reading have been decreasing over the years for the top-performing countries, while they tended to increase for the countries with the lowest scores. However, with COVID-19 there was a general regression and a sharp decline in scores in the last survey of 2022.

• Subject-based differences: Although math and reading scores are tightly correlated, there exist some local differences. In countries such as the Netherlands, students tend to perform better in math than in reading, whereas in others—such as Ireland and most Southern European countries—the opposite is true.

• Educational disparities: In Nordic and Baltic countries like Finland and Estonia the education system appears more equitable, as the gap between the lowest- and highest-performing students remains narrower. In contrast, this disparity has widened in many Central and especially Eastern European countries.

• Widening literacy gap: Notably, the gap between the strongest and weakest readers has widened over time, surpassing the variation in math scores in almost all countries.

Variation in math and reading skills by socio-economic status

The PISA Index of Economic, Social, and Cultural Status (ESCS) is a composite measure derived from three key indicators: parents’ employment status, their level of education, and possession of specific household resources—such as a room of one’s own, classical literature, and a computer—intended as proxies for an economic environment conducive to learning. These indicators are constructed separately based on students’ responses to background questions, then a principal component analysis (PCA) is performed to combine them into a single socio-economic index, weighting each component based on its contribution to overall variance.

To study the relationship between socio-economic background and academic performance, students were grouped into quartiles based on the EU-wide ESCS index for that year. Each quartile represents 25% of students, from the least affluent (first quartile) to the most privileged (fourth quartile). This classification allows us to compare the range of average scores across different socio-economic brackets.

Our analysis reveals that while average scores differ across countries, the gap between the least affluent and most privileged students remains remarkably stable. This underscores the persistent impact of socio-economic status on academic performance, which can only be partially mitigated by national education policies.

In some countries, such as Romania and Bulgaria, the performance distribution skews downward, with students in the third socio-economic quartile achieving scores comparable to those in the second quartile at the EU level. This highlights the role of personal privilege in shaping academic outcomes, suggesting that individual socio-economic advantage can partially offset broader geopolitical disadvantages.

Variation in math and reading skills by parents’ education

Because the ESCS index is a highly aggregate measure, we explored how its individual components contribute to variations in student performance. Among the available indicators, we considered parental education, as well as number of books, classical literature, and artworks in the household. Here, we present the results for the first two, as they provided layered rather than binary answers, making them more informative for analysis.

In the PISA dataset, students’ responses on the education level of both parents are converted into a single value, the higher education level of the two according to the International Standard Classification of Education (ISCED). To ensure comparability between 2000 and 2022 despite the 2011 ISCED revision, we grouped education levels into three broader categories, following Eurostat guidelines:

• Low education (ISCED 0-2): Completed primary education or lower secondary education (middle school)
• Medium education (ISCED 3-4): Completed upper secondary education (high school or vocational training)
• High education (ISCED 5-8): Tertiary education ranging from a bachelor’s degree to a PhD

The diagrams highlight that having low-educated parents is an even stronger predictor of poor academic performance than a low socio-economic index. This is because it captures a more extreme situation of disadvantage, with only a handful of parents not having completed upper secondary education in all of European countries.

However, at higher education levels, this pattern is less clear, suggesting that while low parental education is a strong disadvantage, additional academic benefits from higher parental education do not always scale proportionally. Indeed, in some countries such as Italy, the performance gap between students with middle- and highly educated parents was not statistically significant for years, suggesting that tertiary education does not always yield additional advantages over secondary education.

Variation in math and reading skills by number of books at home

The number of books in a household appears to have a similar influence on both math and reading scores, reinforcing the close relationship between these two skill sets. However, while book possession correlates with academic performance, it is not as strong a predictor as belonging to the highest socio-economic quartile. In contrast, having only up to a dozen books at home emerges as one of the strongest indicators of major learning struggles in school.

Interestingly, having more than 500 books does not provide an advantage over having between 200 and 500. In some cases, it even correlates with slightly lower performance. This could be due to estimation inaccuracies among respondents or response biases, leading to inconsistencies in reporting.

Variation in math and reading skills by frequency of computer use

Finally, we included in our analysis a factor not part of the ESCS index: not just the presence of a computer—whether a desktop or laptop—in the home, but also its availability for use and the frequency with which students actively engage with it.

Over the course of the survey period, we observe that what began as a tool for democratizing access to knowledge gradually evolved into a new source of inequality: daily access to a personal computer has increasingly become a prerequisite for maintaining average academic performance. Once again, this holds for both math and reading skills.

About the dataset

The dataset includes estimated mathematics, science and reading comprehension skills of 15-year-old students, derived from a progressively difficult test, along with responses to a general questionnaire on family and school background. The students were drawn from schools selected for statistical significance within the target country, and both students and schools were anonymized and assigned unique IDs. Since 2000, the survey has been repeated every three years—except for the gap between 2018 and 2022 due to COVID-19—and includes an increasing number of countries, also non-members of the Organization for Economic Cooperation and Development (OECD), which conducts the study.

Sampling and methodology

Our initial instinct was to visualize the data in a scatter plot to examine the relationship between mathematics performance, reading comprehension, and socio-economic status—a step we initially took. However, as we gained a deeper understanding of the structure of PISA questionnaires and their complex sampling methodology, we realized that this approach was fundamentally flawed.

Since schools are selected through a stratified sampling approach but may choose to opt out, sampling weights are assigned to each ID to correct for this variability. When applied, these weights ensure that each ID accurately represents its proportion within the total population of 15-year-olds in the reference country. Without them, students from oversampled schools would be overrepresented.^[1]

Moreover, students do not take a full test covering all subjects, which would take up to 10 hours. Instead, they are randomly assigned different test booklets, each containing only a subset of questions, and some students may not have answered any questions on mathematics or reading comprehension at all. To address this, the survey relies on multiple imputation, using responses from the background questionnaire to infer missing information and estimate proficiency levels.

Consequently, rather than reporting a single skill score per student, the dataset provides multiple plausible values—statistical estimates that account for the uncertainty of missing data. These values are not meant to measure an individual’s ability precisely but to approximate the performance of students with similar characteristics: they provide a statistical estimate of the skills of the stratum that student represents.

For this reason, linking questionnaire responses directly to a student’s plausible values in mathematics or reading leads to misleading conclusions: for meaningful analysis, plausible values cannot be treated as fixed individual scores. Instead, as recommended by the OECD guidelines, the correct approach involves:

1. Grouping students based on the variable of interest (e.g., socio-economic status).
2. Applying weights to account for sampling design.
3. Averaging each plausible value within the group.
4. Computing the final group-level estimate by taking the mean across all plausible values (five or ten, depending on the survey year).

To ensure that these steps were performed correctly, we used the R library Rrepest, also developed by OECD, which provides example analyses as well. The code for producing our dataset can be reviewed here.

Dataset Overview