Ep. 54: Researching How to Teach Research with Dr. Carolyn Forestiere

5:58PM Mar 27, 2024

Speakers:

Dr. Ian Anson

Dr. Carolyn Forestiere

Keywords:

students

umbc

data

research methods

research

political science

question

level

class

qualitative

methods

variables

teaching

undergraduate

learning

thinking

understand

paper

work

world value survey

Hello and welcome to retrieving the social sciences, a production of the Center for Social Science scholarship. I'm your host, Ian Anson, Associate Professor of Political Science here at UMBC.

On today's show, as always, we'll be hearing from UMBC faculty, students, visiting speakers and community partners about the social science research they've been performing in recent times. qualitative, quantitative, applied empirical normative, and retrieving the social sciences, we bring the best of UMBC social science community to you,

as any committed listener to the podcast will already know, research is hard. You know, I spend a lot of time thinking about research. While I'm not podcasting, of course, and one recurring theme in my work is that the path to effective inference in the social sciences requires a lot of patience and a lot of attention to detail.

Social Scientists often straddle the realms of science and public discourse around contemporary affairs in their work. I should know this because I study politics. And so we ought to have a very high standard when it comes to making claims about the social world. That is to say, if we make claims about evidence or interpret contemporary trends of that rigorous methodologies, we run the risk of making a very big mess.

It should come as no surprise then that we're also trying to inculcate the same ideals of rigor, skepticism and careful interpretation among our students. But teaching research can be just as hard if not harder than research itself. So much so that we would do well to research how to teach research using the tools and methods of our disciplines.

That's why I'm so delighted to bring you my recent conversation with Dr. Carolyn Forestier, Professor of Political Science here at UMBC. Dr. Forrest here is an expert on research on the teaching of research methods in political science. Her work on the subject has appeared in journals, like the Journal of Political Science, education, or forest here is also the author of a textbook on research methods, which is currently in its second edition with Oxford University Press. This book showcases a variety of innovative methods for teaching research to undergraduates. And it's especially notable for its strong emphasis on both qualitative as well as quantitative methods. Let's research teaching on teaching research together with Dr. Forrest here, right now.

Today, I am delighted to welcome Dr. Carolyn forest here to the podcast. Dr. forester, thank you so much for taking the time to talk to us today.

Sure, very happy to be here. Especially if we're talking about methods. Absolutely. Music to my ears. Yeah.

Would I like to say butterflies and birds and happy animals are gathering to hear this amazing discussion about methods in the classroom? Well, so that's such a great way to start us off, actually, for sure. Because, you know, I certainly share your vision of butterflies and sort of honey and rainbows and all these things, when I think about research methods, but I know that there are students out there and maybe some of them are listeners to the podcast, who are entering this discussion, much the way that they enter the research methods classroom with a lot of trepidation.

Why is it that you think the students encounter research methods and sometimes feel a little overwhelmed?

Oh, there are a number of reasons. One could just be a representative reputation residue of the course that maybe some students found it a little challenging, and then they talk in the department and things like that. But I think a lot of times, it's that fear of math issue, the fear of the quantitative gnus, because we have a very healthy dose of quantitative analysis now in our methods, modules. And I think that students come into it not quite understanding what they're going to be responsible for. And so I think there's a little anxiety about that. I mean, political science, traditionally, the mold has been to read materials, discuss them, right? They they write qualitative papers all the time, they just don't know that that's what they are. And so the methods course I think, does provide some anxiety, because it's not exactly clear what the topic of the class is. And it's a methods class, not a substantive class. So they're not learning about IR or

comparative politics or presidential elections. They're learning about how to structure political inquiry and how to ask the question, and to answer a question, and I think that that for some students, especially since maybe they have friends at other institutions that that aren't taking classes like that, especially in political science. So when they hear that, you know, oh, I have to do this methods requirement, it's so hard. But then when you look at the grades afterwards, of course, the modal grade is always a, you know, students are making A's, and then they're coming away with real hard skills that they can apply in various ways. So I think it's a very important course, I think it's one that we have to explain to students why it's important that they take it pedagogically, I think it's very well positioned. After the lower level courses that we, you know, we request that students will take 301, you know, we have our lower levels, or 200 levels, take the subfield classes, learn about how things work in those fields. And then you come into research methods. And then really the way that I teach it is very scripted. It's very instruction manual oriented. And in fact, the text that we use, some people have said to me, it feels like an instruction manual. And I'm like, great, because that way you know what to do, there's no guesswork. I don't want students thinking about what they have to do. I want them to just do it and to get involved and understand what a hypothesis is understand that behind every hypothesis is probably a good theory, or there should be a good theory explaining why we have a particular expectation, what the nature of data is how we test hypotheses with empirical data, but that data can be all different sorts of things from qualitative level data piece, what, what kinds of evidence do we use to substantiate a claim? We are evaluating claims, right? And so usually in the political science, we speak in terms of analytical research questions that we're trying to link to things or more things together, like how does this influence this across a number of cases. And I think that that type of mindset in the classroom really helps us ask all sorts of interesting questions about the political world. So I take this approach that's very scripted, very cookie cutter very instruction manual. So as I said before, where, you know, I provide concrete instructions of what to do. And I think that there also may be trepidation, because you actually have to do it. And every step follows. Like, if you don't do step one, step two, isn't going to make any sense. And if you don't do step two, certainly step three is not going to make any sense. And if you've finally woken up around step five, or six, you're totally lost. And it's really important that students understand from the first day that they're going to be required to do certain things and coming to class and being engaged, which should be for all their classes, it just shouldn't be for methods that we say come to class will remain engaged. But you know, maybe it's a good way to get them to come to class and remain engaged. And then when they approach their later classes, they really understand what they need to do. Now,

it's actually for us here. So first of all, I am on board with this class, it sounds great. I want to take it as well. But I want to also think about the unique approach that you take to this curriculum. And we were just talking a little bit a little bit about the analysis of evidence and about data. And about student engagement, the idea that students might actually come to this class and find that there's actually some interesting things going on here and beyond just, you know, learning about, you know, regression equation or something like that. How is it that you're able to maintain that engagement through the kinds of data that students are being exposed to, and the students are analyzing, and the course,

Truthfully, in my course, I give them the data. So I don't have them looking for data. Now in the advanced class that comes afterwards, everyone can answer their own question. And they're all over the place, which means that they have to either collect their own data, they have to go out there and get it somehow quantitative or qualitative, or they can use something that's already publicly available, that they can put together and for their purposes, but in the class in the class that we require of the majors, I give them the data. So they're using a publicly available database. It's very, it's a very nice database, because we're working with the American data. But then any student who's interested in any other area of the world can look to see that that type of data, public opinion, data exists for many different countries. So I think it's a very nice way to introduce students to data and I want to use survey data, because it's very easy to put your head around a question like how interested are you in politics on a scale from one to four? Everyone has answered questions like that before. So we can talk about how that's an ordinal level variable and how we can use that in data analysis. So I kind of get around the problem of the data by giving them the data. And then furthermore, we also collect qualitative level data by having students interview each other. Now, I will say that when I first started doing this, I actually took students there was a there was a pretty substantial civic engagement component to the class because I would take students to Charles Town retirement community down the street from UMBC, where they would sit one on one with people who are 60 and above, to ask them the same sorts of questions that they were analyzing in a quantitative sense. And so that was really wonderful that the students got a chance to sit down with people that were living right up the street, but get to talk to them about topics that they were interested in learning more about. And that was great, because also the research population change. So when they would think about their hypotheses, they couldn't think they weren't now now they think about it, because they interview each other. The research population now is 18 to 35. But when we were going to Charlestown, I encouraged them to think, Okay, what would a 60 and above year old think about this issue? And is it different than how say you and your cohort think about the issue. However, since COVID, I have been reluctant to take students to Charlestown. So it's possible that in the future will reintroduce that. But for right now, the qualitative element is having people interview each other to gather that kind of data, about the same types of questions that are asked that were asked in the survey that they're analyzing quantitatively. But you know, but I do think it's really important for students to see the data set, because I know that when I work with statistics, I can't do anything until I see the data, not just a description of the variables, I want to see the Excel spreadsheet, I want to see the columns, I want to know what they look like, I want to have some understanding of the variability that I see in the data. Of course, I can run all of this in my statistical program, but I got when I opened up the dataset with the students, I show them I'm like, this is the matrix, oh, these all these numbers just coming at you. And, you know, I'm like, Alright, now we need to be like Morpheus and understand what's going on here. But um, but But yeah, I think it's really important for students to work with data, because they see that every everything they read in a textbook, everything that they read in any kind of report is based on data analysis. And I want students to understand and believe that they have something to contribute, once they learn some just rudimentary skills. I will say that when I went through the review process of the book, there are professors out there that think that maybe those types of inquiries should be left for graduate students or more professional level people. But I don't take that point of view, I think that everybody has something to contribute. And I think that once you give people, a few, a few tools, let's see what happens next. And that's the approach I take in the classroom. So even if they never go on to do research, again, they at least understand how information is put together.

Yeah, that's fantastic. And something that I definitely aspire to also inculcate in my students, is that that sort of data literacy as well as the ability to ask and answer a question, it just seems so fundamental. And, you know, thinking about your approach, I really liked the idea of viewing these tools in ways that are very accessible to students. But then, you know, as you're describing in this sort of subsequent course, I guess the second stage as it were, in some students research journey, you really give them a lot more freedom and flexibility. I was wondering if you wouldn't mind telling us a little bit about the sort of the way that course works, and some of the data that the students are collecting, and kind of what the experience has been like thus far. Sure.

Um, so 409 is advanced research. And it's highly recommended that students take 301 so that they can have some research skills that before they come into the class, but we do have students that come from other majors, like for example, psychology has research methods as well. So we can have students you know, that bring those skills into the class. 409 is an opportunity for students to really think about what is what it is that they want to know more about that they learned something, I have one student, for example, who learned about things in another class. And they're now kind of pushing it a little further. And every student is bringing their unique interests. And so they can be I've got some comparative projects going on. I've got Americanist projects going on. I've got one student working with the counties of Maryland. So it's, it's, you know, very, every student brings what they want to the table. And basically, for the first week, we have to get started, because if they want to collect qualitative level data, they're going to have to get IRB approval. So it's like, we have to move quickly, when we're talking with our students with the research projects that they want to do. So 409 is much more open. It's usually a smaller environment. It's usually, you know, much more individualized in a way, but the students form a research cohort. They're in like this supportive environment they come in and it's wonderful to watch them helping each other. And it's really interesting to see how they're putting their literature reviews together and how many themes do you have versus you know, It's, it's really nice. But in many ways, it kind of does feel a little bit like we're putting multiple independent studies together because everyone is doing their own project. But we're all on the same timeline that maybe they're asking their question at the same time. They're, they're getting their literature together at the same time. And now, where we are right now is that this week, the papers that are being turned in it's the introduction, the statement of relevance for why that question is an important one, the literature review that they're starting, and then the Methods section just to basically say, what are they going to do after spring break? And what are they going to do? To answer their questions, either with qualitative level data or quantitative level data? So that's where we are. And I think they're, I honestly think that they're having a really nice experience. It's a very nice cohort, we teach it at 830 in the morning, and I did that on purpose so that nobody could tell me Well, I'm not free, because most horses are not at 830 in the morning. So it's, you know, but it's good. And like I said, at that point, what my job is, is to listen to the student, and to, you know, basically say, what energizes you like, what news stories? Have you read about that you want to know more about, like, where are you in, you're thinking about the things that interest you? Or what might you go on to do when you're finished at UMBC. And then what I do is like, I just kind of carefully steer to think, Okay, well, I work with the data that I know exists. But then on the other hand, we have Qualtrics, like, we have one student that's doing a project at UMBC. So that we can distribute the Qualtrics, you know, survey very easily, I have another student who's doing a survey of Peace Corps, former Peace Corps volunteers. And so Qualtrics is being used in that way as well. So we do have a very powerful survey tool at UMBC, to ask people sorts of different questions. But again, does take a good month to figure out what the question is, how to get it into IRB, how to make sure that everything is set there. But so it's a little bit more involved. But by spring break, they're going so it's like some of them are going to be using at least a day or two of spring break, I think to kind of make sure that everything is put together. And then they're going to present it the pricing to Alpha Research Conference that will have at the beginning of April and then one week, and that we use that as practice for one week later at OCAD that they can present to the bigger and much more varied audience.

Yeah, you've, you've preempted, my next question was, which was to ask about sort of what these projects are going to do in terms of presentation or in terms of sort of living a life beyond just a harddrive, right?

That's exactly what I call it the paper cemetery. Yeah. Yeah, that's true when I became faculty advisor, now co faculty advisor, because we share that responsibility. But when I first took that on in 2006, I sat down with the group of Pi Sigma Alpha students, I had this like, general meeting. And I said, Well, what do we want to do? You know, what are what are some ideas that you have about what you want to do with, you know, with this with this club, we can make it a social club, we can make it an academic club. And their idea was, you know, we write all of these papers, and they don't go anywhere. And they had seen like the idea of the posters and so we talked about that, and hence the first conference was born. And so this year is going to mark the 14th year that we've done it, which is very exciting. And except for maybe the first year, every every conference, we've used Pi Sigma Alpha national money, and to host it, which is really exciting for our chapter in our school. But I think it's really good for students to stand in front of their work. And even though it's we know that it's not complete, they've only been working on it for a couple of months. You know, it's like, we know that the literature review isn't complete, we know that their data analysis is rudimentary, and LM, you know, just at the beginning phases of what it could be if they were working on this much more. But I think there's so much power and standing in front of your ideas, and saying, This is what I'm thinking about this right now. And to have a friendly audience come in and say, Hey, I think about these ideas, too, let's engage in a conversation. And it just makes it makes it so much more rich. I think it's an incredible learning experience. I've never had a single student in the 14 years of doing this tell me that they've had a negative experience with it. And in fact, if anything, I just had students walk away from it. Like, that was fun, you know, that like, um, you know, sharing the ideas and, you know, talking to even professors and teaching them that's what I tell them. The students in the class, they are all teaching me about their topics, because I'm just sort of like the methods gatekeeper here. And I said, don't worry about grades, don't worry, but there are assignments that I need you to worry about. But just give it your best shot. Let me help you with like revising it. And today we even had a very important discussion about the comments that I'm going to give and I said, Listen, once what I'm going to promise you is like I'll do it with a smile, but you know, I, you know, I'm going to be completely honest with you about your work. I'm going to tell you if I think it's strong, and I'm going to tell you if I don't think it's strong, but I was like, please don't do this is really important. Because as we go on and become academics, or whatever it is that we do with our life, people are always going to criticize our work, they're going to make suggestions for how things can be better. And we have to just come off of our perfectionist, you know, a pedestal thinking that just because I wrote a paragraph, it's the most awesome paragraph ever written. That's not the case at all. And in fact, what I want students to do is become just, I want them to be neutral in reading the constructive comments that they receive about their work, rather than thinking, Oh, she hates it, she hates it, she hates it, it's more like no, she thought about it enough to give me suggestions on how to make it stronger, so that I can do more with this project. Because like, you know, we academics are a very insecure bunch. And I totally get that and, and I know I'm not gonna be able to fix 100%, that problem, but certainly I want them to read their comments on their papers with an eye of like, how do I make this better. And to understand that research is an iterative process, probably the first time you write something is not going to be the final word on it for and I told my students this morning, for every five pages you write, maybe one is going to make it into the paper. And maybe that's a little extreme for undergraduates. But I know that my dissertation ended up being 200 pages, but I think I wrote about 1000. So those were the pages that made it into the document, because like I was kind of, you know, working with different ideas, adding variables, and just, you know, changing things as I went along. And I think that that's what happens in research.

So speaking of dissertations, and speaking of students completing their projects, and finding success. And of course, the 14 years that you've been teaching research methods to students, can you tell me any stories about students in the past that have gone on to use research in their sort of futures?

Sure, I mean, I've had lots of students write me emails throughout the years, to tell me that they were in a competition for something like, you know, maybe for a graduate level position, or even in, you know, in a professional appointment in something. And it's very fun to get those emails, there was one student who was given an assignment with a data set. And they were asked to do, I think it was like a correlation coefficient. And one of the variables was binary. And so they pulled the person over and said, I don't think that this data is appropriate for what you're asking us to do. And that was the answer. The answer was to say that what you were looking for was not possible, given the data set that you gave us. And I think they got the job. And they wrote me, they said, I spoke so convincingly about what it is, and it was just like, I heard your voice in my head, like, you have to know what kind of variable you're working with in order to use it in a particular way, and data analysis. So, you know, we have those sorts of stories. Um, we also have a current story right now of a student who, you know, took 301 is also an economics major is probably getting some methods training over there, and wrote a very wonderful paper about economic outcomes in the 50 states, and gave it to me because he wanted to submit it for the Undergraduate Research Journal in political science. It's an undergraduate journal. And after I read that paper, I went straight to his advisor and said, I think my exact words were, this is restaurant quality. In other words, it's like, I think that this has the makings of a professional level paper. And so working with the adviser, they sent it, you know, wrote a cover letter and got it all sent, you know, sent it off to the journal, and he has a sitting on a revise and resubmit. And I think over in the past couple of months, he's revised a paper and he sent it out, to my knowledge that will be the first time an undergraduate has published in a professional level journal with, you know, in our, from our department, I mean, I think we've had success in students placing work and you know, the UNBC review, and perhaps other undergraduate outlets, but this this is like, you know, this, this is at the level of the professor, you know, also to that same student presented their work at the Southern Conference, this past January, the southern Political Science Association conference, and I was in the audience. And he did a fantastic job, he represented UNBC. So well, and it was really nice. And it was so his his analysis I, from what I understand, he's self taught, he taught himself our I use state in the classroom. That's another thing I have students who will use data with me, and then we'll go on and want to know our or wants to know other applications, like how do I do the same things with these other programs? That's great. And so he with the professor learned our learn the more advanced techniques, you know, regression to use and how to do the checks and controls and things like that. And it was obvious from the comments. In fact, one of the commentators said because like this students started off with a declaration like an undergraduate. Like, you don't need to say that, because it was so and then wait. And then they basically said something like, I wish all our undergraduates were like this. And I was like, Well, you got to come to our school and see what we're doing. Because we have lots of students who take their work to the next level and who aren't just excellent and have the capacity. That's what I say to students all the time. I'm like, you know, I know, you're smart. I know, you can do this. The question is, will you do this. And, you know, that's up to you. What we can do at UMBC is provide excellent instruction in the classroom, and give them the tools that they can answer their own questions, whether or not they do it and take it to the next level, that's completely up to them. And we know that we have a core group of highly committed students who want to, I know it's cliche to say, change the world in their own way. And they this is one way to do it by answering questions and revealing patterns have information that maybe we didn't understand before, or maybe even for themselves, and maybe like the exploration of a particular topic will lead to, like, I need to know more about that. And so the next thing, you know, they're applying for master's programs or PhD programs, or looking for internships in that field, and doing things like that. But I just think it's very important force, I think it's I know, your question was like, do we have examples of students? We do, and we have lots of students will take it to the next level. But again, I just think it's an important force for all students, just to understand like, how do we know how to evaluate we live in a time of such disinformation, it's really important to give students tools so that they can go out and answer their own questions. And understand that just because one dataset produced one result does not mean that that's the final word you should get. And at the end of the book, and in the end of my course, we introduce replication analysis, because we want to say, hey, if we, you know, did this with the World Value Survey, well, actually, I said something before, that wasn't true. In my course, we use something called the data prac. And then we do replication analysis with the World Value Survey. But it's really fun showing students the World Value Survey because they get to see the data that's out there, like for another example is today in one of my comparative classes I pulled up there's a data set about the level of debt that is held in African countries, and we were talking about how many countries are going into a debt crisis, because they're borrowing huge sums of money. And now a large portion of their government spending is servicing the budget. So we talked a lot about what that was about, and how that sometimes can cripple an economy that has other priorities, but they can't handle those priorities because they're servicing their debt. And so we pulled up data from the World Bank, we pulled up data from a number of sources. And you can see that the students are like, I didn't know this was here. Like I had no idea that I could just make a column about this or the Gini Coefficient, because then we were saying, Okay, once once a society has so much debt, what else does that impact are? So we looked at GDP per capita, we looked at the Gini index, we looked at government effectiveness, we look, you know, we're looking now very multiple sources to gather our data. And then now what I can do is throw that into Stata, the students open their state of program, I've given them the Excel file, they open it up. And next thing I know, they're all running correlations and looking to see if there are these connections among these between and among these variables.

That's awesome. It's, it sounds like such a fun experience to really dig into the data. And I think that the students at all have their successes in developing their own research speak to really that outcome. Dr. Forrester, I'm so grateful that you're able to share your experience with us teaching research methods, thinking about how to teach research methods and publishing textbooks to help other instructors. Before I let you go, normally, we have a question. On the podcast, I like to ask anybody who tends to have a role that teachers students, which is basically if you had advice for students, what would you give them? I want to turn the question on its head a little bit actually today, given your expertise in teaching research methods, and in helping students to understand these methods. I wanted to ask you, if you had any advice for instructors who are teaching research methods, were thinking about perhaps going pro and teaching research methods to students? What kind of advice would you give to those teachers in training,

I would say go slow, go slow, because you don't want to scare off the students who are a little bit math phobic, especially if you're doing quantitative stuff. But you also have to go slow with the qualitative stuff because you want to make sure the questions that you ask those are the questions that you want to ask. Right? So I would say go slow. I would say to the instructors, be prepared for a little bit of chaos in the classroom. Because students are going to be different in how capable they are working with a computer working with statistics. Some students come into the classroom already knowing how to code for example in Java, and so with that in mind, you know things like our that's much easier. Working with Stata is much easier. because they understand the nature of code, other students don't have that at all. So I would say go slow, have a goal, like, what is your goal at the end of this segment or this particular 75 minute period? Like, what is it that you want them to know? And just be clear, and the expectation, because I think that really what gets students is, you know, they know, like, when I show up on the first day, and I totally scare them, I just say, basically, say, you're gonna have a 25 page paper by the end of this class. And they look at me like with shock on their face. And I'm like, Really, you don't write these long papers, rather, glasses do. But okay. But I mean, what I would say to instructors is just be clear in your expectations. Go through the data yourself before you start introducing it to students so that you can find the pitfalls, but know that you have to put in a little bit of time to work with students who may not be as comfortable working with their computers. I would also say that if you're, if your class is around 30, people try to go into a lab, the engineering building has wonderful labs, that we are very lucky that they, if they have space, they let us use and then that way you can standardize the machines. So that way, you know, you're not working with an HP Mac, you know, you all these different machines, you know, so if you're going to do something, even on a qualitative level, if you're doing invivo, if you're doing something like that, you can make sure that everyone is on the same machine. So you can say click here, click here, click here. But I mean, I would say so the instructors, get your students excited about what they're doing, you know, find a way to bring it to real live, you know, events that are happening in the world. And so ask them, you know, to take a look at the data, because I know that so many students really begin to enjoy the data once they get into it. And the fact that this morning, I had a student, you know, we were talking about, like an example, talking about regression, we only cover OLS regression in the class, we don't we cover a lot. It's like a buffet. My class is like a buffet, we take like a little bit from each each segment. And some professors may want to do it that way. Some professors may want to go really deep into one thing, they just have to decide what they want to do. But we only do OLS regression. So this morning in class, I was telling a couple of the students that are working with quantitative level data, the saying, Well, what is your dependent variable? And what does it look like? Because if it's a binary level variable, I'm going to show you a different technique. And I was thinking I could introduce Logitech just them, you know, just say, Okay, there's this other technique that you could use to do your stats. And it was just like, they're like, what, you know, it's like, well, we talked about this, but now I'm reminding you, you know, and, and, you know, I would just go slow, go slow, and figure out what it is that you want them to know, and continue to tell them why you think it's important for them to learn these things. Because sometimes they do get frustrated. But I think that if you take them with you and you're honest about your pedagogical mission, they'll they'll come with you or at least some of them will, and the ones that really get the most out of it, they will definitely come with you.

Dr. Carolyn Forestiere, thank you so much for taking the time to talk to us for introducing us to this buffet of research methods. And I wish you all the best I wish the best to all of your students and hopefully, they will all be able to contribute to that buffet with their restaurant quality research. Yeah, thank you again.

Thank you for learning more about all the work that goes into ensuring our students become critical consumers of information, and maybe also become producers of fantastic social science research. Until next time, keep questioning.

Retrieving the Social Sciences is a production of the UMBC Center for Social Science Scholarship. Our director is Dr. Christine Mallinson, our Associate Director is Dr. Felipe Filomeno, and our undergraduate production assistant is Jean Kim. Our theme music was composed and recorded by D'Juan Moreland. Find out more about CS3 at socialscience@umbc.edu and make sure to follow us on Twitter, Facebook, Instagram, and YouTube, where you can find full video recordings of recent CS3 sponsored events. Until next time, keep questioning.