Episode 19: Dr. Paula Winke
Photo of Dr. Winke courtesy of the College of Arts and Letters, Michigan State University.
In this episode, Andrew sits down with Michigan State University Associate Professor Dr. Paula Winke. Dr. Winke is a leading researcher on foreign and second language testing. At MSU, she researches if tests are valid, if tests measure what they should, methods for assessment, and differences among learners that affect assessment outcomes. She co-directs the Second Language Studies Eye-tracking Lab at MSU, and gave a plenary talk at the 2019 Task-based Language Teaching Conference in Ottawa, Canada called "We Need to Align Our Classroom Tasks with ACTFL & CEFR Can-Do Descriptors so that Our TBLT Programs are Chock-full of Proficiency Indicators."
In this episode, Dr. Winke provides insight on:
what proficiency is, the models usually used to measure it, and what problems exist in that model;
how proficiency progresses vertically, horizontally, and spherically, and questions why we usually only gauge progress on a vertical scale;
how programs can evaluate if they are assessing accurately;
the problem with measuring proficiency across different program levels;
how course grades do not correlate to proficiency and that most assessments don't test what is being learned;
ACTFL and CEFR scales and how tasks can be great assessment tools.
More About Dr. Winke:
See her list of publications here
Contact her: winke@msu.edu
As always, thank you for listening. Your support has been overwhelming and we couldn't do what we do without you. We hope this podcast serves as an effective CPD tool for you.
If you like the show, consider subscribing on Spotify, iTunes, or Google Podcasts, and sharing it with a friend. If you have a comment or question about today's show, we'd love to hear from you: info@learnyourenglish.com
Follow us Online:
Here’s further explanation from Dr. Winke on how language grows, or, at least how we think language grows:
“We know language has some sort of shape to it, because we say that we can measure it. It is a ghost, as we discussed, in the sense that we can't see it. It is a latent trait: that is, something that we can't directly measure. (Like motivation or anxiety or health, which too are latent traits: We measure them by measuring other things, which we call indicators. So, two indicators of health are one's height and weight. Two indicators of language knowledge are proficiency (on, say, the ACTFL scale) and vocabulary knowledge (through Paul Nation's vocabulary size test, for example).
We have married ourselves to the notion that we can measure language vertically on proficiency scales. So we know it grows taller, in a sense. We think we can measure its depth, especially in terms of vocabulary word knowledge or by counting up the number of tasks that a person can do in the language at any given proficiency level. So we have pretty good notions of how language grows both vertically and horizontally--we can measure these. So we can say language is at least two dimensional, from a measurement perspective.
But it is much richer: Language grows in ways we can't measure it. We just don't have the tools. We assume language is a single mass, but maybe the growth is spotty (here and there). If that is the case, that might explain why we have a hard time measuring language knowledge. We might have a hard time measuring or weighing language because it is in different places, and maybe in different places at different times. We don't know exactly how to measure second-language pragmatic skills development, cultural appropriateness in the language, or social skills. We don't know how to measure one's ability to use visual cues which they use to help parse language and assign meaning. Nor do we know how to measure a person's ability to use their personal charms and gestures to convey meaning beyond their linguistic skills. We aren't even sure if that counts as "language."
We don't know if "language" is a single mass, or if it is several different masses in various locations that sometimes touch and mush together. Some parts of human beings' repertoires for communication include things that are considered language by some, and not considered as language by others (like gestures), so we have these "Plutos" out there too: We are not sure if we should try to measure them or not, if they are part of language or not.
While I do believe language has depth and breadth, I don't think it is simply two-dimensional. It has to be more spherical. It might be like a cloud of stars, with brightness in certain areas, and not others. That is why we might be missing so much if we just measure the height of what we think is the single mass of language. It is probably right that as height goes up, horizontal depth goes out (they are related aspects, because language does have shape, and it is very unlikely to be super skinny), so proficiency measurements can be pretty good estimates of the overall shape of language ability. But, we as applied linguists cannot deny that we are only currently capable of measuring limited aspects of language and language growth.”