Using EQAO, which makes standardized content tests for students, for content tests and pedagogical tests for preservice secondary school mathematics teachers suggests to me that there are some questions that need to be answered.
1. To focus on the mathematics content:
A) p. 3 from EQAO technical reports (2017). “It should be noted that not all expectations can be measured in a largescale assessment. Measurable curriculum expectations are clustered by topic, and items are then mapped to these clusters. Not all of the measurable expectations in a cluster are measured in any one assessment; however, over a five-year cycle, all measurable expectations in a cluster are assessed.”
This means student achievement is an incomplete picture for each year because NOT all the curriculum expectations are tested each year. And, in my review and analysis of 5 years of sample tests, there is often NOT an equitable weighting of test items across all strands. What will be emphasized (read: particular nugget of math someone thinks is important) each year for the one-off test for graduation for preservice teachers?
For other expectations of the curriculum, such as how to learn and do mathematics, this also means that the curriculum document Program Planning (e.g., Gr 9-10, p. 23) statements, and any curriculum expectation that states by investigation, with manipulatives, with technology, are not reflected in the EQAO standardized test.
For other expectations of the curriculum, such as the mathematics processes (e.g., p. 29, Gr 9-10) of problem solving, reasoning and proving, reflecting, selecting tools and computational strategies, connecting, representing, and communicating, these are also, NOT, being assessed by EQAO, but are expected to be taught and practiced by students. To me, these key features of the mathematics curriculum, which do not appear in the EQAO test for which students’ achievement is being judged, indicates a problem with the intent of such a standardized test and its alignment with the curriculum it is purportedly measuring student achievement.
B) EQAO report p. 11, “Scorers are trained to refer constantly to the anchors to ensure consistent scoring. The rubric codes are related to, but do not correspond to, the levels of achievement outlined in the achievement charts in the Ministry of Education curriculum documents.”
This means how teachers are to teach and assess math according to Growing Success and the curriculum document is NOT how the curriculum is being assessed or items designed for the EQAO test. The reported EQAO student achievement levels may not match the stated achievement levels detailed in the curriculum documents. For me this calls into question the ability of teachers and schools to be able to make changes to learning (which is seamlessly linked to assessment) if there are different assessment perspectives in play with teaching and with standardized testing.
These problems and issues, 1A and 1B above, will be equally present for preservice teachers’ measurements of mathematics knowledge, as they are for students’ measurements of mathematics achievement. And for preservice teachers, they will be five (5) or more years AWAY from the last time they were learning secondary school grade 11 mathematics content, and what if the preservice teacher learned mathematics in a non-Ontario or non-Canadian secondary school!? (n.b. teacher preparation has mathematics content pre-requisites, and the program itself is about pedagogical knowledge learning and pedagogical-content knowledge learning (Shulman, 1986, 1987) NOT about learning mathematics content.)
2. To focus on pedagogy:
Since preservice teacher learning is at the university level, there is NOT a common curriculum, nor a common pedagogical strategy espoused by any governing body. The OCT and Ministry of Education conduct program certifications, but at no time has anyone watched me teach, interviewed me about my pedagogical approach, or judged my pedagogy against the artifacts and course syllabi I submit for certification evidence. And, I have never had a pedagogical discussion, workshop, conference with EQAO in the 16 years I have been a teacher educator, nor the 12 years I was an Associate Teacher with preservice teachers in my classroom on practica.
This means: I am not sure what pedagogy EQAO thinks should be practiced in teacher education programs, and what pedagogy teacher education programs should be teaching preservice teachers to use in their classrooms. How can a test be created and be valid and reliable, each year, given 11+ teacher education programs across Ontario, and the many teacher education programs that have taught non-Ontario preservice teachers who come to Ontario to teach?
If improved learning is truly desired by society, then let the experts do their work. Move the barriers, such as ineffective and/or misaligned testing, away. Stop looking back at the good old times and trying to replicate them for the new generations today, because one’s present success, while an outcome of that history is not relevant to the lives the new generations are living right now.
There are experts in education as a human endeavour, the teaching and learning of mathematics, the creation and implementation of educational policy, the organization and structure of curriculum, who are up to date with this century’s learning needs. Politicians, elected by the people, should be responsible to the people by recognizing their own knowledge limitations and taking advantage of others’ expert knowledge.
There is a risk of oversimplifying the issue of standardized tests and teacher testing that I put up front here; I welcome evidence- and literature-based discussion (and even correction) because I am really just another fallible human being trying to do the best for our shared human condition.
Reference: the Technical report, at the least the last available report from EQAO about test design, evaluation, and reporting, found online at http://www.eqao.com/en/assessments/Pages/Technical-Reports.aspx
Curriculum documents at http://www.edu.gov.on.ca/eng/curriculum/elementary/math18curr.pdf
http://www.edu.gov.on.ca/eng/curriculum/secondary/math910curr.pdf
http://www.edu.gov.on.ca/eng/curriculum/secondary/math1112currb.pdf
http://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf
http://www.edu.gov.on.ca/eng/literacynumeracy/index.html
Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14.
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22.
