Monday, May 16, 2016

Maxwell: 6.1

This is Maxwell's chapter on validity. There are two questions: what threats to validity do I see in my research, and what strategies might I use to assess whether these threats are serious?

1. Potential Threats:

I think that my bias could be a threat to the validity of my study. I have very strong opinions about what I want to find and establish with my research. I believe that effective teachers in a EFL setting use certain strategies, and I want to document those strategies. Perhaps by believing those strategies exist, I will create a self-fulfilling prophecy and find them.

Another threat might be the limitations of my setting. I am semi-consciously expecting to find a phenomenon that is generalizable to international English as a medium of instruction settings everywhere, but so far I have only observed it in a couple of settings and am tentatively planning to study it in one setting. There is a threat that I will overgeneralize beyond my setting and overlook the ways that my setting is unique from other EMI settings--especially in other cultures.

There is the threat of reflexivity--that my interviewed subjects will try to give me what I want to here. That I might ask leading questions. That my familiarity with my interview subjects will cause them to neglect to mention pertinent facts (because they think I know them already).

2. Ways to Assess Seriousness of Threats

I think the best way to guard against bias is to make it explicit. Maxwell states that the researcher cannot hope to not influence the material they gather or what kind they get. All researchers are looking for specific answers, so if I make my bias explicit, I can also check to see if there are other interpretations to my findings. In the same way, I have to be careful about not over-generalizing. Perhaps, I can find people to interview who are not in my immediate setting, but equally legitimate would be to be careful not to apply my findings beyond my own setting. For the third threat, it may be a good idea to go and interview someone who does not know me or my context and find out what kind of answers they give me, and then go back and cross check these answers with the people I interview at Sias.

Maxwell mentions several strategies that I think are already incorporated into my research design. Intensive, long-term involvement: I am already deeply involved in this context and have a long history to inform my research question and the data I gather. Rich data: I think there is no limit to the amount of data I could collect on this topic. I could interview teachers, I could interview students, I could observe classrooms, I could video tape and record. I could collect sample assignments and student responses. Probably its a question of what I have time to do in the parameters of this dissertation.

Respondent validation: this is something I could implement fairly easily. I know that most of the teachers I interview will want to see my findings. I'm not so sure about intervention, however. I think I want to collect more information before I suggest changes to the way people teach. I may do second interviews, though, where I ask if our first interview influenced the way the teacher thought about their teaching methods. I hope that I will find a variety of answers to my questions about the best methods--because I know their is variation in our faculty, and so I am actually hoping to get opinions that disagree with my expected hypothesis. I think they will make the data richer and uncover assumptions that need to be addressed.

Triangulation would be, within my setting, choosing a diverse population (new teachers, old teachers, teachers who teach large lectures, teachers who teach break out sessions, MBA students, professors with Ph.D.s etc.). Triangulation outside my setting would be finding teachers in other schools here in China, in other countries in Asia, in countries in Europe and the Middle East and in Africa. There is a lot of potential for adding a quantitative element: conducting surveys of students here at Sias or in other EMI programs is the first one that comes to mind.

The question I think I will have to settle for my proposal is what is the scope of my study and what are the most serious threats, since I don't think I have time to implement all of Maxwell's suggestions. My strongest inclination is to explicitly limit myself to this setting. I think there is enough data here to illustrate significant findings that can be explored further in other settings at a later date.

Saturday, May 14, 2016

Reflection on Observation

The observation was an interesting process. I think I do a lot of observing subconsciously, so when I sat down to write, it was hard for me to decide what to focus on first. For example, I have spent a lot of time in China watching the customs of people from the corner of my eye and I have spent the same amount of time forming theories about why they do things the way they do. It's a unique experience when you don't know the language because it forces you to focus on cues--what people wear, the expressions on their face, their body language--but after you've watched them for a while, you forget how they struck you on first impression. You forget that you used to think it was strange that that boy is carrying his girlfriend's purse, or that those schoolgirls are holding hands...

So it was really kind of hard to decide what to write down.

One of the comments that you made was that I should have spent a little more time interpreting what I saw. I wondered about that. The thing is, I have a lot of speculations that I could write down about the things I see, but usually if I get off speculating, I miss what is there, or at least I have the potential to miss it. I think as part of this exercise, I was trying to stay present and not evaluate. I think for me those two modes, observing versus interpreting, are very distinct and I would have liked a little more structure in the assignment as to how much of each I should be doing (for example 60% observation; 40% interpretation).

What do you think? Would it have been acceptable for me to fall back on my years of experience observing Chinese people and start with themes--for example: "Chinese people have a collective culture. Let me draw all of the aspects of this scene that illustrate the collective nature of their society..." Would that have been a legitimate approach? I often explain my insights to people who are here visiting or who are new to China. To what extent does my historical observations count as legitimate data in a qualitative study?

Reflection on Interview Assignment

I enjoyed working on my interview assignment, although I found it to be quite the extended process. I chose to interview my colleague, in part because she fit into the population that I wanted to examine for my dissertation, but also because I had helped her a little with her research recently, and so I thought she would be more interested in helping me (reciprocity). I may be asked to help more in the future, but that is fine with me. I need to get better at helping my colleagues with their research if I'm hoping for them to help me with mine.

It turns out that Mary doesn't really like to speculate and expand on a topic. She wanted me to be very specific in the way I asked questions. This was a little frustrating for me because I wanted to drop a word or sentence and get her stream of thought. On one hand, maybe I should have explained that to her at first (if I was even aware of it). On the other hand, I don't know if maybe that isn't a personality thing. I really like learning about personalities, and Myers Briggs personality typing has the iNtuitive versus Sensor category. Sensors are very practical and down to earth. They focus on the here and now and don't spend to much time speculating on abstract ideas or the future. iNtuitives, by contrast, spend a lot of time making abstract connections and creating paradigms as well as speculating on the future. Most people in higher education are on the iNtuitive side of things, but I think Mary tends to think more concretely. I don't really have a way to compensate for that except that maybe I just need to be ready to take a different approach if my interviewee has a different way of thinking about these things.

Anyway, it was an interesting experience.

By the way, I chose not to transcribe my interviews because I come from a linguistic background and I'm very aware of how much "noise" there is in a recording. I had to transcribe all of the small messy details of a five minute recording as a final project in one of my classes. It is a fascinating process, but it also demonstrated to me how bad I am at transcribing. I noticed that Rev does kind of a sloppy job, though. I'm not sure if that reflects their quality or my expectations. Maybe I just try too hard to get it right, and that is why I think I can't transcribe... I don't know.

Article Critique

The qualitative study that I read was "Positive Aspects of International Student Transitions: A Qualitative Inquiry" by Lisa Moores and Natalee Popadiuk (2011).

This article focused on the positive critical incidents reported by seven international students at a Canadian university. The researchers interviewed each student seven times (semi-structured). They used a technique called the Critical Incident Technique (CIT) and combined it with aspects of positive psychology. The authors state that the unique gap that their study fills in the literature is that it focuses on the positive aspects of international students' experiences rather than problematizing them. The researchers categorized the critical incidents reported into eight groups: Growth and/or change, Social support/building relationships with peers, learning to navigate the host culture, enjoyable activities outside of schoolwork, previous experience/preparation, supportive faculty and staff, persevering through hard times, and sense of belonging. The researchers further refined these categories into four themes: connecting with others, maintaining a foundation, embracing the process, and discovering strengths within.

I thought these researchers had an interesting and insightful approach to examining international students' experiences. The focus on the positive aspects of their experience does seem to be a unique approach to gaining understanding and insight. They did a thorough job of describing how they established the credibility of their methods, citing Guba and Lincoln's techniques which they used to test their own methods.

Perhaps because of the page limitations put on a journal article, the authors left me with some questions. For example, the article starts with statistics regarding foreign student populations in the U.S. but this study was conducted in Canada. I didn't figure that out until I was about halfway through the article. I think that authors could have explicitly acknowledged this and reflected on whether or not patterns and experiences for international students vary in Canada as opposed to the U.S. Perhaps they don't vary, but I would have liked some kind of explanation. Qualitative research is about focusing on singularity--not trying to create broad conclusions. The authors could have shed further light on their context by discussing the fact that it was Canadian.

Further, in the introduction, the authors mention, in passing, the applications of their research to college counselors--an application that they expand on in their implications. Since the article was published in the Journal of College Student Development (which is feasibly read by non-counselors), I would have liked some explanation as to how coming from a counseling perspective shaped this study--as opposed to if it had come from some other branch of student services.

Also, just because of my own recent experience in conducting interviews, I would have liked a more in-depth account of the seven interviews--what kind of questions the authors asked and whether they had any difficulties eliciting information (maybe they didn't). It just seems that these kind of interviews would require a certain understanding of researcher expectations (cultural norms) that the international students may not have had. The authors do mention briefly that all interviews were conducted in the students' second language which may have interfered with the material elicited.

Along these lines, the authors also mention that perhaps the students felt constrained by the request for only positive incidents (although some negative incidents were mentioned). I can see how this may have limited the authors' results. I think they could have cast their net wider and collected both positive and negative incidents and then interpreted them through the positive psychology lens, but I'm not sure if that approach would have served their aims. The focus solely on the positive aspects of the students' experiences does seem limiting to me, however. I suppose I just believe that positive incidents inform negative incidents and vice versa. It would have been nice to read a discussion about this at least.

Saturday, April 30, 2016

My Freshman Year: 4-7

I really enjoyed these chapters, perhaps because there was something I could personally identify with in every one of them.

The first chapter, chapter 4, was about international students. I suppose I can identify because I have been in international contexts a lot, and I've also been an American--and around Americans. I also have had experience with the shoe on the other foot, where I was expected to fit in with some of the norms of other countries (for example making friends with your friend's parents) and how hard that can be for an independent American who doesn't want intimacy too quickly because of the obligations that might be involved.

I like what Nathan said about the friendly veneer. It is so true that Americans use the veneer to grease any number of social wheels but that it can't be taken as genuine affection or interest. I have often observed that "We should get together some time" is a polite fiction unless it is followed up with an exact day and time. It has to be hard for non-Americans to get used to that. What was also funny about this chapter was Nathan's report of how shocking international students found the informality of the American classes. I had a colleague of mine read the excerpt and we both chuckled, because our classrooms are much the same. We even occasionally have students come to class in their pajamas, although probably not as frequently as in the U.S. I'm also not sure that they do all the same things in the classes they have with Chinese teachers. The teacher does a lot to set the tone of a classroom.

In chapter 5, Nathan points out that the focus of campus life is not academics--yet another observation that I could have told her. Still it is very nice to have it articulated the way that she does. I think in all of my school career, I tended to be the one labeled as a "witch" according to Nathan's experience. It's a very subtle thing, and I think her way of making it explicit was genius, because for many of us who experience it, it's like an itch somewhere which we can't quite put our finger on. I think this is why I hate being a student. I'm good at it, but I never fit into the ruling social paradigm and so I end up relatively isolated. I thought it would get less as I progressed further into graduate school, but that has not been my experience. Somehow loving ideas for their own sake is still "uncool" even sometimes at a doctorate level.

I suppose that is what comes of trying to make higher education accessible to all. Some people don't like ideas. They aren't good at generating or using them, but they are good at managing social situations. These people are encouraged and motivated, perhaps by desires for better jobs or career advancement, to engage in a pass-time that they naturally dislike and so they use their social skills to marginalize that pass-time in the lives of all of their social group. Those who do like to learn are shamed into silence in the very environment where they might ideally thrive. That, it appears to me, is one of the main problems of higher education. It tries to be all things to all people and ends up being a social finishing school for the elite--who aren't necessarily the smartest or the most motivated. Money, natural attractiveness, the right kind of cultural background, and social skills have a lot more to do with it.

Chapter 6 talks about the "art of college." I enjoyed Nathan's many insights on the way students negotiate college, because they were refreshingly realistic and reflective of my own experience. I sometimes think we get off track because most teachers and professors at universities were the overachieving types in their class (they were the "learners" from the Strengths Finder paradigm), so they end up measuring all their students by themselves. Yes, students make a lot of pragmatic decisions when it comes to their education--and most of them don't have enough energy left to really suck all the learning opportunities dry. I liked Nathan's account of the delusional advice given to students about how to plan their week. I have been present for such talks, and ignored them like most other students.

In Chapter 7, Nathan gives instructors some really good advice that I wish more teachers in my program would take to heart. She points out that when students start falling asleep in your class, "As hard as it may be to realize sometimes, it is really not personal." I think if more teachers came to terms with this--and realized that students are going to take the easy route if you give it to them, they would be much better teacher for it.

Thursday, April 14, 2016

Maxwell 5.1

Negotiating Relationships with participants in my study.

The participants in my study would most likely be the teachers at my university. Currently, Sias has about 120 foreign teachers, which I've heard is the largest foreign teacher population in China, although that could have changed since I heard it. A little over half of the teachers are language teachers, teaching oral English directly for Sias. Perhaps a third of the teachers work for an American university called Fort Hays State University which is based in Kansas. These instructors teach a range of subjects although mostly oriented around business courses. This population also includes about a dozen composition teachers who teach a typical first year composition class to sophomores in the Ft. Hays Program. The final sixth of the population are academic teachers that teach for Sias. Turnover is moderately high, with about a third of the teachers leaving every year. However Sias also has teachers who have been here for over ten years.

I intend to recruit volunteers from within this population. My criteria will be teacher who teach content course (not oral English) because I want to focus on how teachers reach non linguistic oriented learning objectives. I have at least an acquaintance with all of the foreign teachers at Sias because we are all required by university policy to live in the same building and free meals are offered in the community dining room. I have had the opportunity at many points to discuss my research interests with them and many of them are familiar with the proposed topic of my dissertation.

My tentative plan is to send out an email through the administrative office asking qualified teachers to volunteer. I may offer my baking skills as an inducement, since American style baking is a little harder to come by here in China. In my email, I will try to make if very clear to my participants that mine is an exploratory study--not evaluative. I think teachers will want to volunteer because they know me and are interested in my topic. If the email approach doesn't work, I may ask more directly, but I don't want to put too much pressure on potential participants, so I will have to think through how to manage that if things work out that way.

The people whom I'm most interested in interviewing are not in my department (the Fort Hays composition teachers) and therefore should not feel threatened or coerced by my inquiry. Teachers in the Fort Hays program have a lot of autonomy and as I am not their immediate supervisor, they should not perceive me as a threat to their job. Sias teachers are much more closely observed, but as I am not part of their leadership team, my position of not being an authority should be clear to them.

I have also chosen this population because I understand it quite well. These teachers will feel more confident opening up to me because we are from similar cultural backgrounds and work under the same kind of conditions. I can use my unique understanding of their situation to maintain our relationship.

One ethical consideration may arise. I am being considered for an assistant dean position within the Fort Hays program. If I get that position, I will be in a position with greater power difference between myself and the content teachers in the Fort Hays program (the bulk of the content teachers at Sias). If this should happen, I will have to reevaluate the situation and how my change in job title may affect my relationship and ability to collect data in this program. If things come to that, however, I may restrict my data gathering to the Sias content teachers, or search elsewhere in China for foreign content teachers to interview.

Friday, March 25, 2016

Reflections on Magolda & Gross's It's All About Jesus!

I have to make a confession. I did not attempt to buy this book before I got back to China and when I got here I realized that it did not have a Kindle edition. Shipping books to Zhengzhou can be a lengthy and uncertain process. I'm not sure I would even receive the book before the end of the semester. However, I did read the excerpts that were available on Amazon, and I do have some things to say about it.

First of all, I really wish I could buy it and read it this semester. Even the parts that I did get to read were fascinating. I love the way the authors describe a group that is very familiar to me from an outside perspective.

What struck me most was their generous treatment of their subject. I don't think I have seen a Evangelical Christian group portrayed with such sympathy from outsiders. To be honest, when I first started reading the book, I expected the researchers to be critical and even mocking. This is my experience of "non-believers'" observations of the Christian tradition that I come from. I think I have been trained to be rather stoic about this. Jesus did warn us that the world hated him and they will hate us. There is no way to say for certain, but maybe one reason that the researchers were fairly sympathetic is that they were Catholic, so that, while they may not agree with specific approaches or mentalities, they still agree and affirm the importance of Christ and the Bible.

It was very cool to have the "strange" aspects of that Christian group described from an outsider perspective because when one is in such a group, one forgets which parts of it are strange. Also, the uncritical nature of it gave me a new sympathy for my tradition. It made me think of those types of ministries as peopled, not by narrow minded zealots, but by sheep. Jesus compares us to sheep and tells us that he is sending us out among wolves, but I didn't really see that analogy until I read this study. From it, I got the impression that Christians really do love God and each other and are generally careful and inoffensive in the way they reach out to others.

I also thought it was cool that the researchers acknowledged that the members of the group failed in their mission where the researchers were concerned--that they failed to convert them. This made me smile because as an Evangelical Christian with perhaps a little less categorical beliefs, I do believe the group achieved their goal in that they were faithful to be who God had called them to be in front of these researchers and therefor showed them Christ. God's work isn't encapsulated in a two year research project. I like how the researchers maintained their autonomy, though, and never quite identified themselves with those they studied. I wonder if this would have been different if the researchers had just been their as participants instead of investigators.