Maxwell: 6.1

This is Maxwell's chapter on validity. There are two questions: what threats to validity do I see in my research, and what strategies might I use to assess whether these threats are serious?

1. Potential Threats:

I think that my bias could be a threat to the validity of my study. I have very strong opinions about what I want to find and establish with my research. I believe that effective teachers in a EFL setting use certain strategies, and I want to document those strategies. Perhaps by believing those strategies exist, I will create a self-fulfilling prophecy and find them.

Another threat might be the limitations of my setting. I am semi-consciously expecting to find a phenomenon that is generalizable to international English as a medium of instruction settings everywhere, but so far I have only observed it in a couple of settings and am tentatively planning to study it in one setting. There is a threat that I will overgeneralize beyond my setting and overlook the ways that my setting is unique from other EMI settings--especially in other cultures.

There is the threat of reflexivity--that my interviewed subjects will try to give me what I want to here. That I might ask leading questions. That my familiarity with my interview subjects will cause them to neglect to mention pertinent facts (because they think I know them already).

2. Ways to Assess Seriousness of Threats

I think the best way to guard against bias is to make it explicit. Maxwell states that the researcher cannot hope to not influence the material they gather or what kind they get. All researchers are looking for specific answers, so if I make my bias explicit, I can also check to see if there are other interpretations to my findings. In the same way, I have to be careful about not over-generalizing. Perhaps, I can find people to interview who are not in my immediate setting, but equally legitimate would be to be careful not to apply my findings beyond my own setting. For the third threat, it may be a good idea to go and interview someone who does not know me or my context and find out what kind of answers they give me, and then go back and cross check these answers with the people I interview at Sias.

Maxwell mentions several strategies that I think are already incorporated into my research design. Intensive, long-term involvement: I am already deeply involved in this context and have a long history to inform my research question and the data I gather. Rich data: I think there is no limit to the amount of data I could collect on this topic. I could interview teachers, I could interview students, I could observe classrooms, I could video tape and record. I could collect sample assignments and student responses. Probably its a question of what I have time to do in the parameters of this dissertation.

Respondent validation: this is something I could implement fairly easily. I know that most of the teachers I interview will want to see my findings. I'm not so sure about intervention, however. I think I want to collect more information before I suggest changes to the way people teach. I may do second interviews, though, where I ask if our first interview influenced the way the teacher thought about their teaching methods. I hope that I will find a variety of answers to my questions about the best methods--because I know their is variation in our faculty, and so I am actually hoping to get opinions that disagree with my expected hypothesis. I think they will make the data richer and uncover assumptions that need to be addressed.

Triangulation would be, within my setting, choosing a diverse population (new teachers, old teachers, teachers who teach large lectures, teachers who teach break out sessions, MBA students, professors with Ph.D.s etc.). Triangulation outside my setting would be finding teachers in other schools here in China, in other countries in Asia, in countries in Europe and the Middle East and in Africa. There is a lot of potential for adding a quantitative element: conducting surveys of students here at Sias or in other EMI programs is the first one that comes to mind.

The question I think I will have to settle for my proposal is what is the scope of my study and what are the most serious threats, since I don't think I have time to implement all of Maxwell's suggestions. My strongest inclination is to explicitly limit myself to this setting. I think there is enough data here to illustrate significant findings that can be explored further in other settings at a later date.