Referring to the evidence presented by the SUN news article by blilley@postmedia.com and promoted by @LisaThompsonMPP @fordnation about class sizes: It is important to read the whole article and not (what some call) �cherry-pick� sentences from abstracts and conclusions. The details of the participants under study, and in a quantitative study such as those studies quoted, the variables NOT included or controlled for are very important to appreciating how generalizable the particular study is to other educational jurisdictions.
Researchers know that context matters. Context gives the possibility for generalizability of a study�s result to the greater population, but then it also inhibits the generalizability and transferability to OTHER populations. Context matters � the evidence Ontario uses for its decisions should be coming from Canadian (or at least North American) schools if it is being used for local effect.
For example, Project Star is a large scale experiment (that means �Gold Standard� research according to the USA government) in Tennessee schools �students in smaller classes have significantly higher achievement that students in larger classes� (Shen & Konstantopoulos, 2017, p. 99).
�Project Star is considered an exceptional study and perhaps the most impressive and powerful field experiment about class size effects in education (Mosteller, 1995)� (Shen & Konstantopoulos, 2017, p. 99).
This is important because it is a study about North American schools, and can be informative when thinking about North American schools. Context matters � the evidence should be coming from Canadian or at least North American schools if it is being used for local effect.
Cho, Glewwe, & Whitler (2012) completed a study with Minnesota�s elementary schools. This research on class size in Minnesota suggests that reducing class sizes by ten students in elementary schools in Minnesota increases students� test scores by 0.04-0.05 standard deviations.� But other variables such as family wealth have an impact, �most students in Minnesota (and Connecticut) may be too wealthy to benefit from small classes� (p. 93). Thus, the North American context matters.
In Kentucky schools, Barrett & Toma (2013) stated that the discussion about class sizes must take into account the local practices of principals, �in particular, principals place more students in the classroom of more effective teachers, controlling fo all other factors� (p. 51). �But if estimates from past models have been biased because class size is endogenous [that means affected by external influences], there is little policy advice that can be made based on prior estimates. The efficiency of public resources devoted to class size reduction should be further addressed� (p. 51).
Now, concerning the Shen & Konstantopoulos (2017) and Li and Konstantopoulos (2017) articles, these findings represent the schooling in eight to 14 European countries: Austria, Croatia, Czech Republic, Denmark, Germany, Hungary, Italy, Lithuania, Malta, Portugal, Romania, Slovak Republic, Slovenia, and Spain. I hazard a guess and suggest the cultural, economic, racial, etc. diversity of these European countries does not align with North American educational jurisdictions. We can learn some great things from these studies, but our job is then to think about how those findings are relevant and transferable to our jurisdictions. It is not a straight one-to-one match.
Some points from within those Shen & Konstantopoulos (2017) and Li and Konstantopoulos (2017) articles. Shen & Konstantopoulos (2017): �Variables such as family wealth, parental involvement in school or parental pressure regarding student assignment to classes, teacher quality, motivation, or commitment as well as relationships between principals, teachers, and parent may be critical factors regarding assigning students and teachers to smaller classes� (p. 99).
�The assignment of students and teachers to classrooms within grades and schools is rarely random� (p. 99).
Two of the eight countries [that�s 25% or one quarter of the countries in the study] involved in the Shen & Konstantopoulos European study showed smaller class size effects with student achievement (Romania and Germany) (see p. 109). So why do the other six of eight, given all the school systems in the world, make for a ringing endorsement for change in Ontario schools?
�These differences may indicate as a whole that class size effects are country and context specific. Unfortunately, classroom data (e.g., observations or videos) were not available and thus we had no knowledge of actual classroom practices� p. 109. The researchers themselves said be careful how one interprets their findings � class size effects are country and context specific.
A limitation to their study: �it is unclear that the [data] instrument was unrelated to unobserved variables� p. 112, meaning, context matters and context from unmeasured variables might likely change the results.
Li and Konstantopoulos (2017): �TIMSS does not provide data about classroom dynamics, instruction, and practices, it is difficult to know exactly why class-size effects were non-significant� (p. 308). �Perhaps teachers did not modify their instruction accordingly� for class size differences� (p. 308). �This kind of information was not available to us� (p. 309). This means, context matters, the variables not measured likely would change the results.
The most important thing I have learned as an education researcher, is that the context of the study matters. Generalizability is wonderful, but not always possible, and it is definitely not necessarily possible into other jurisdictions. Transferability of findings is wonderful, but for findings to be transferable to other jurisdictions, a lot of critical thinking and even more reading of relevant literature from research is necessary before making a research decision � and that is not even considering a social policy decision that has enormous impact on society and people directly. The other important thing I have appreciated � bias must be reduced. One cannot only look for the details that support one way of thinking. Combined with the first point, attention to bias and context are critical to evidence-based decision making.
References
Barrett, N, & Toma, E. F. (2013). Reward or punishment? Class size and teacher quality. Economics of Education review, 35, 41-52.
Cho, H., Glewwe, P. & Whitler, M. (2012). Do reductions in class size raise students� test scores? Evidence from population variation in Minnesota�s elementary school. Economics of Education Review, 31, 77-95.
Li, W. & Konstantopoulos, S. (2017. Does class-size reduction close the achievement gap? Evidence from TIMSS 2011. School Effectiveness and School Improvement, 28(2), 292-313.
Shen, T. & Konstantopoulos, S. (2017). Class size effects on reading achievement in Europe: Evidence from Pirls. Studies in Educational Evaluation, 53, 98-114.
