How to Publish series, Episode 11: Reviews, Reviewers, and Rejections

8:49PM May 4, 2019

Speakers:

STEMculture Podcast

Dani

All

Zach

Will

Moncie

Dani and Moncie

Dan Quintana

Joseph White

Keywords:

peer review process

reviewer

laughter

paper

review

editors

journal

people

science

publish

important

submit

peer review

field

hear

scientists

study

podcast

quintana

comments

[Intro music]

Welcome to STEMculture/Podcast!

Oh, I fucked up! [Laughter]

Alright, we have an intro

You guy's, it's really warm in here!

Welcome to STEMculture podcast, today we're talking about the peer review process. We'll talk about why peer review is important, its shortcomings and how to be successful with the system we have. Will and Dani have some experience to share and our guest host Dr. Moncie Wright has even more. But since this is a very important and complex topic, especially for scientists, we asked a few very experienced people to weigh in. Many of the scientists we asked were willing to participate, and because this topic is so important to STEM folks and grad students in particular, we are keep- we will keep collecting these tips and release bonus episodes as they come in.

This episode goes out to all the haters. We welcome your constructive criticism.

Today, we'll start by talking about the peer review process generally, you'll hear from our guest host Moncie Wright. Then you'll hear from to science heavy hitters with lots of experience who are going to weigh in, finally will require will reflect on those recordings. So first of all, Dr. Moncie Wright has a master's in environmental biology and recently finished her PhD in aquatic toxicology. She's published and has regularly reviewed papers, which is why we've invited her to join us. So hi, Moncie, welcome to STEMculture podcast

Yo, thanks for having me. [Laughter]

Is there anything else you'd like to add to that?

Certainly, I'd like to just briefly discuss what I'm currently doing with my life since post grad school seven years later, it's fine. No, wait seven and a half years. [Laughter]

Yeah, so currently, I help manage a $10 million Environmental Protection Agency contract.

Daaaaamn

We review pesticide toxicity studies for the EPA. And I helped for the day to day running of the contract, assigning work, authorizing work, strategizing for deliveries, training people, and all sorts of other fun stuff.

I also am a strongman competitor. So that's what I do with my free time, now. All that lovely free time. So thank you for having me. I'm excited.

Yaaaas

We're excited too, it's gonna be great.

Also, Moncie's one of our best friends so. [Laugther, then Will purring, and then much more laughter]

So I'm going to start by talking a little bit about peer review. And the peer review process in general, just for people who aren't yet in grad school haven't really talked to anybody about it. So effectively,

the peer review process is something that is much reviled by many people, but

but it's really important. And the reason for that is that it is the one sort of tool that we have to to approximate objectivity in science. So basically, what you do is you create a study of some kind, and you send it to a journal. And they, the editor, one of the editors of the journal looks at the paper and then picks out some reviewers who are other scientists that ideally know something about the topic of the paper, and send the paper to them, and then they anonymously review it. And based on those reviews, you may get to do revisions or not, if they decide that it's not relevant to the journal.

But basically through the back and forth with these the editor and then they anonymous reviewers, you get the opportunity to refine your study

and make it fit better into the grander body of knowledge that is your field. And so,

I mentioned that people revile it because nobody really likes being criticized.

But a couple of overly philosophical general points of wisdom.

Okay, so Rumi, because you know, I'm a hippie [laughter]

said, If you something I'm paraphrasing, but it's something like if you bristle with every rub, how will your mirror ever become polished? So,

basically, if you take offense, every time somebody criticizes you, then you're missing the opportunity to refine yourself.

Yeah, so like approach approach things with a growth mind that kind of idea.

Yeah. And so the last philosophical point that I have is actually from a science scientist. So Johan, US Kepler, who is an important part of the development of astronomy in the early days. So he said, I much prefer the sharpest criticism of a single intelligent person to the thoughtless approval of the masses. So criticism is actually a good thing, if what your goal is to become the best possible version of yourself.

And then Beyonce said, I ain't sorry. [laughter]

She's a great philosopher of our time.

She is

Yeah, not not not playing.

I would actually,

I would actually love to look through some of the things that she said and like, apply it to future episodes.

We could and we should and yes.

Okay, so we're going to do that.

Okay.

Taking that

It'll be a Beyonce episode. Philosopher Beyonce.

Yeah, I'm going to make an argument. I'm going to see what you guys think about it. I'm going to say that the peer review process is not just important, it is science. Without it, science couldn't exist as we know it today. So what do you think about that?

Okay, so I have I have a couple issues with it. Yeah, peer review is important. But it also costs money to be able to submit your paper to a journal who's going to publish it, but then you have to pay them to publish it. So there's many people that can't necessarily pay for that to happen.

So did you have to pay to submit your paper to the journal that you just submitted to?

No, sorry. So, clarification,

When they publish it? Sorry, not to submit it.

So I was not aware that that was even happening. So I didn't have to pay.

You didn't have to pay.

Yeah.

But did it come out of a grant?

No, there was no money offered.

Wow.

So is it some journals that do that?

Yeah.

I've only ever seen. I mean, every journal I've looked into, we had to pay them

Really!

money. Yeah.

Wow.

I think I think sometimes, sometimes it's as that that phenomenon, if there's a really steep price is associated with predatory journals,

I was about to say, yeah

Also, the top journals, I think, also charge a lot. So like hundreds and hundreds of dollars to get into Science or Nature.

We paid Nature, $5,200.

Wooooow

Excuse me?!

It's a flat rate.

Wow, okay, so

I had no idea.

I agree. There are big problems in the peer review process. And we are obviously going to get to that I think Moncie has some specific criticisms. And I completely agree with that. But I also think that without peer review, science would be effectively useless. The reason is that

without the peer review process, we would have no way to turn our disjointed ideas and studies into a body of knowledge that's useful.

I have a follow up question.

Yeah.

So you know, when you are, I don't know, actually, I know other people that have written book chapters, you write a book chapter, but that tends to be invited. Or you might offer and they might say yes, but that's not peer reviewed, the editors will edit it, but it's not necessarily going to go out and get peer reviewed by other people. So how does that fit in?

Well, that's not usually primary. Right. So it that's, that's like, secondary, or even tertiary literature, where you're taking a bunch of things that have already gone through the peer review process, and, and synthesizing those. And I think you could not do that in a reasonable way. If we didn't have the peer review process, imperfect, though it is.

And there might be a better way. But I don't think anybody knows of it. Because I think if there was a clear better way better system, we would know about it.

I'm not sure I agree with that.

Okay

I think there's too much money wrapped up in it now for there to be a better way forward. Yeah, I think there's been a lot of criticism about the peer review process. And I think it's pretty valid, I think first, okay, I'm going to jump the gun here. The first, if you think about it, even if you submit your raw data, that is not a guarantee that that's actually your raw data, nobody can confirm that your data wasn't made up, they just can't, they got to take your word for it that that Excel file that you sent, or whatever they you sent is, is the data. So we're kind of operating on the assumption that you're following good QA QC, and you're not being disingenuous or lying about your data. So you submit it right, but they still just are getting this paper to look out, and then they've got your data, if they're lucky, most of the time, you you don't have the person's data. And then you're, you know, you're probably not going to be directly involved in that person's field, there's going to be some gaps in your knowledge. And you're just going to do your best as a reviewer to make sure that you know, you do do the paper justice. But

so that I think that there's a problem on that side. But also, the other problem is that the process isn't double blind. So they know who you are, but you don't know who they are. And for all, you know, they're competing with you for a grant.

Yeah, which they should

Know

Yeah. And then there is so in a lot of journals when you when you submit, or your reviewer, if you feel like you have

a conflict of interest you're supposed to disclose, but there's plenty people that don't. Which is also problematic.

Yeah, absolutely. So you don't know what's going on the other side there. And I'm sure that there's a ton of politics that go on, especially at the bigger journals. And so I think that's a big problem. I I heard one scientist say recently that they feel that everything should get published, just publish everything, and let people sift through it. Yes, you know, and there's a lot of work out there where they weren't able to prove their hypothesis, they didn't, you know, get their nice beautiful p value. Yeah. And it got shelved, and there's what I may have read on somebody else's research for no reason. It didn't get published.

So So statistical testing and and and P values have become maybe stand in for actually having reviewers that are, that are in your field that actually know enough about your study to review it in a fair way. And, yeah, so I mean, I absolutely agree with you guys. I think that

that, like Moncie's said science or peer review has got a lot of money in it. And that creates conflict of interest, whether they're reported or you know, or not.

And then

well, science generally has become so diffuse, it's grown so fast, in terms of the variety of topics that are studied. That,

you know, we can't necessarily even have a conversation with the scientists in your building that that studies nominally the same thing as you.

So big problems affecting the peer review system, agreed.

But to stop being devil's advocate over here, [laughter]

I do think it's a really good idea because to have a peer review system in place, because, for instance, I was working on a paper for two years. And really the only people that saw it are my people that I'm working with directly the other co authors, my friends and family who I talked to about this thing all the time. And it's out for review now. And I'm really excited to get feedback from people that have never seen it before. And I really hope you know, when I've been writing it, I've really been trying to think about what is somebody that's never come across anything like this before, going to think and having that in my head, and really trying to make sure every part of what I'm doing is transparent, has been really helpful in creating, I think, a paper that is fairly well done. And I hope that the feedback I get from them is is reflective of that. But I'm excited to see how it can improve too. So I look forward to that. I'm scared, but I look forward to it. [chortle]

That's I mean, I don't have a lot of experience with with peer review. So you just quickly you have submitted one paper, is that right?

So I, I have I have three published papers.

I'm so sorry. I was way off base.

No, no, that's okay. Um, but the reason you thought I only had one is because of this. This particular one is because the first one I will be first author on for my PhD.

Got it

Yeah. So I think I have another paper for my PhD, I'm a co author on, but I was not involved in like, I did not submit it and deal directly with the reviews. That was my advisor. And then I have two papers prior, where I did deal directly with reviews. And one of those I'm first author on.

Awesome, [Dani burps - she's so rude but we love her] so

nice. Um, I'm not cutting that out.

You can't.

So yeah, so I have, I have one first author publication that well not publication, I've submitted one paper twice, and got rejected both times. So long story, which will probably come back to under another topic. But

I think one of the reasons why I have such a rosy view, or maybe not a rosy view, but a positive, basically positive view of the peer review system is because just like you're excited to get this feedback from people that you don't even know and you're hopeful that it will be constructive, at least in part, and it will help you improve your study. That's what you really care about. The

reviews that I got at the second submission of my one paper

is the first time that I've gotten substantive feedback from people in the field that I was studying, was the first time. And so for me, it was a really exciting moment because it was something I could really sink my teeth into. And, and so it's it's a, it's a social structure that exists for scientists, aside from the other ones that we depend on. That is a resource when it works. Right.

Yeah.

So that's my meager perspective. [laughter]

But, Dani, do you have any other experiences or thoughts to share about the peer review process?

Yeah, so like we just spoke about, I have gone through the roof, the review process a few times.

The kind of one of my main takeaways is that it's always good to know roughly when you should be hearing back about a decision on your manuscript. And the reason for that is, is because I have waited six months to hear back on the manuscript before, and it should not have taken that long. And I was inexperienced. And I didn't know, like one would be appropriate to contact the journal, I didn't want to bother them.

And I finally at six months was like, I feel like this is definitely too long to be waiting for this journal. And I emailed them, they're like, Oh,

yeah, so

we forgot about it. It was on somebody's desk, and it got buried. [laughter]

That sucks.

Yeah, yeah. And so that, that got it back on track, but I could have emailed earlier.

Also, I had an option at that point to if I didn't want to keep the paper with that journal, and the- you can always withdraw.

You can't have your manuscript at two different journals at the same time, it's a copyright issue. So.

So that's kind of one of my bad experiences with the peer review process.

But overall, like I've talked about, I do think the peer review peer reviews of really good idea to make sure because obviously, you and your team want it to be published. And you might, subconsciously or consciously, you know, speed by problematic areas of your paper. [laughter] And so it's really good to get other people's perspective, outside of your team.

Something else I think about. So obviously, I'm excited to see my review comment and the comments I get back on my paper, but also kind of anxious, because, you know, there might be things that they say that I totally disagree with, or it's absolutely, people talk about reviewer three being kind of the mean reviewer. And so some of the comments might hit kind of hard. So something I've learned over time is to basically just open the fucking document, read through the comments, and then walk away, don't work on them that day, give it a day or two, and then go back and read them again. And I found for me that when I do that, they are they seem much less harsh, and more reasonable, and something I can do. And then I can see them more as like constructive comments, instead of like, Oh, this person is trying to tank my paper,

I have actually reviewed a few articles as well that my advisor has given me, kind of he wanted to see how I thought about these papers. And then he would combine my thoughts with his and send those in to the paper to the journal again. And, you know, I try to give really good comments back.

So I really hope others are trying to do the same. But what I have found, and what I've heard from other grad students is, we all try really hard to give good comments back. And we try to do justice to the paper and to the scientist.

And I think grad students overall are just like under utilized as reviewers, for sure. And this is a an issue right now with peer review is that a editor will go out and ask people, scientists in this field to review and then they might ask 18, 19, 20 people to be reviewers, and none of them say yes. Then they have to ask more and more and more. And this is where grad students can come really in handy, because you can ask us and we're used to being overworked. So [laughter]

and we're, I don't know, I'm eager to kind of get that experience and see what it's like from the other side.

I think Yeah, and I think we all have read a lot in our science careers. But there's something more engaging about an interactive reading of a paper where you're in the process, that would be a great learning experience, if it was something that more grad students engaged in, and maybe a one possible avenue that that we can take as a community would be to create some official capacity for grad students to review papers. So there's a grad student reviewer slot on every paper submitted to our journal.

Oh, that would be cool. That'd be super cool. So um, so Dr. Dan Quintana is going to be on our as a guest on our on this podcast actually, a little bit later. And on his podcast with James Heather, Everything Hertz Podcast, they have talked about this before, about, you know, if you want to review for journals, and you're not getting pinged from certain journals, you can email the editors and tell them hey, you can ask me. Now I have actually done that. And I got several positive responses back. But that was a year ago, and I still haven't gotten any papers.

And I'm like, but give them to me, I will review them. And at this point, I'm a fourth year of my PhD. So now that that makes me. I mean, that makes me an expert in my very specific area. I think so even if I'm still young and early in my career, I'm still better than nothing. [awkward laughter]

Yeah, there's almost definitely something that you could contribute to almost almost any paper in your field.

Yeah. So editors out there. If you're perhaps listening, ping us grad students, or if we email you.

Please pick us to review stuff.

Yeah, I'll totally review papers.

Yeah. So Will does plant physiology. And Dani does animal physiology and endocrinology. So

I also do Earth system modeling, functional genomics and population genetics. So

Fine, I also do... stuff. [laughter]

Other other things. [laughter, Will purring, a cacophony of laughter, we're ridiculous]

So I'm

continuing on this path.

Moncie, as I mentioned earlier, has more experience than either Dani or I. And so we've asked her to

go through her experiences a little bit in parallel with the questions that we pose to the PIs that have responded and will play the recordings of in a few minutes. But, Moncie, can you tell us a little bit about your experience interacting with the peer review process?

Yeah, absolutely. So I think I'll just start talking about sort of my overall positives and the negatives of the peer review process. And I've already touched on some of them, so I won't repeat myself. But with regard to the positives, I agree with Dani, that it's definitely a lot process, I learned so much to the process of submitting my paper. There were things that they said that I was just like, oh, wow, I didn't even think about That's awesome. And I even found myself wishing some of them could be on my committee. That's such a good comments. So I think overall, was a very good experience. It did take me about a couple of weeks to respond to all of the comments. So it was a little frustrating at times. But it was fine. With regard to my other criticisms is I actually wish that it were more interactive. So they're formulating their comments, and they get sent to you in a bundle. And then you get those comments, and then you work on them. And then you send them back to them. And then they're working on their own time. But it just adds weeks to the process. And if there are a way to let us actually interacts on the online interface. I think that would be better for everybody. And I really think that that's I think that's something people are thinking about.

Yeah,

I hope to see more of that in the future. So that's actually more of a real time conversation.

I think there are some journals moving in that direction. There's also definitely some journals that are they are either making the the reviewers not anonymous anymore during that process, or they will de-anonymize them after.

And so this happened at at Nature, after the whole process was done, and our paper was published, they actually, you, they could also actually post the interaction between the two, like between the reviewers and the authors, kind of that back and forth. So you can see how the paper changed and all of that. So that's so if you ever see on any Nature papers, if you go to the actual Nature website and click on a paper, you can actually see that interaction, which is kind of interesting.

That's awesome.

I had no idea. That's really cool.

Yeah, that's really cool.

So it's happening, but slowly.

Yes. So my overall experience with regard to the comments part, and that's the part that I'm most interested in today talking about is that there's an art to addressing the comments that you get. So some of them are going to be rude. Because people are people and they're just typing away and they don't realize that they're being condescending,

or they're reviewing it at like midnight with a bourbon in hand. Yeah.

And we're responding. [laughter]

As they should [laughter]

So So in my field, aquatic toxicology, there's a heavy emphasis on chemistry, and especially engineering, because I work with nanoparticles. And so all of these reviewers, not a single one of them was an ecologist. And this is really an ecology paper. And in a sense, it's aquatic toxicology. But there's a huge ecology component. And they were all hyper focused on the chemistry. And I kept having to remind them that this isn't about the chemistry, we know a little bit about the behavior of these nanoparticles in the system. But what we're talking about is an applied thing, and applied experiment. And they kept asking for lab studies. And I was just like, there's a million lab studies. The whole point of this experiment was to take those and extrapolate it out to the field. Great. We know what happens in the lab that's been well characterized. Awesome. Now let's take us into the field. And let's find out what happens in a realistic field scenario, which is what we did. And we got dinged for that. They were like, Oh, you have too many environmental variables. It's literally what they said. I was like, yeah, that's because it's an outdoor mesocosm study.

That's the point! [laughter]

There's a reason why they're called environmental variables. [laughter]

The environment is complicated.

Can I ask what their reaction was? Like? Did they are you might not even heard back from them.

So I'm trying to remember, I think there was just one round of actual comments.

Okay.

So I actually have them up in front of me. So the way that I dealt with them all these comments was was by first thinking them being kind and thoughtful my responses to them saying, that's a great question. However, that's beyond the scope of what we did. This isn't supposed to be a lab study, it's not supposed to be controlled. And I would very professionally guide them through why what we did was important, getting them to look at the big picture, kind of redirecting them away from just like focusing on this tiny bit that they actually knew about. So I think they were very uncomfortable with the ecology part thing. That's what it really came down to was they didn't understand the ecology stuff. Yeah. So just like what about the nanoparticle behavior and the pH and this and that, and I'm just okay. I guess it is very important. This not the scope of this experiment.

Yeah.

So just taking the time to really, I mean, some of these are just, some of these are essays with citations. There's one response where I had like, 10 citations

Whoa

So it's like getting really deep into stuff. And I actually learned, I learned a lot through the process. One reviewer was like, there aren't enough controls. And I was like, What? There's, there's one control, it's that makes sense. There, there doesn't need to be another control. And so it was just, you know, being professional, and going through this thought process with them. And I think it I think I like what I like I like looking at the comments and my responses to them. I think that it came out really nicely. And it was also good to have the the feedback from my my co authors is very helpful.

Yeah.

And so yeah

I'm sorry,

oh, that's okay.

So it's not like you had to actually do every thing that the reviewer said.

Right. So that's, that's another point that I wanted to make, which is that you have to you have to figure out what battles you want to fight and which ones you don't want to fight. If they're like, I want to see this thing I italicized. Okay, fine. That's done. Yeah, no problem. So you have to pick your battles. Is it it? Is it something worth fighting over? So it could be like a really petty comment? Okay, fine. I'm going to fix that. And then it might be something else that's bigger. And you're just you just like, I can't do that. I can't go back in time. I'd literally tell a viewer, I can't do that. Because it's over now. [laughter]

In the nicest way possible.

Yeah.

So really just kind of like playing that that line of like showing respect, but also knowing when to stand up for yourself and say, No, I actually disagree with you, which I had to do. Now, I had a reviewer three who said they couldn't recommend my paper for publication. And I gave them a very long response. And I'm actually happy to put this up on the website or wherever is a good place for that. So people can see how you can you can address reviewer three.

Okay

I've also been reviewer three. Best thing ever. I had to be reviewer three recently, and it was really so traumatizing. [laughter] so bad.

I'm really excited to hear about your experience as reviewer three. I just want to interject real quick. I found out fairly recently that not everybody has reviewer three. [laughter] So

Four reviewers?

Yeah. Right. So there's the different journals.

Yeah.

Def have different numbers of reviewers and physics, I found out lately, they just have to. So I said to physicist, well, you know, reviewer number three? He sort of looked at me like I had three heads.

Or two heads.

Yeah, right. Two reviewers. Exactly.

That explains some of the memes I've seen where it's where they talk bad about reviewer number two.

I don't know why. But apparently the tough ones always get shuffled to the end of the deck.

Yeah. Well, I wonder if they do that on purpose. The editors they give you like the nice ones first to warm you up? And the slightly harder one.

Yeah.

And then the hardest one at the end. Just not to sideline?

Yeah I heard about that.

Yeah. I know. But if I had, I had some really intense comments. I would not want those ones. First, I would want the nice ones first. So I'm like, Okay, I'm not complete shit.

They like me, they really liked my brain. And they're like, oh, slightly less. Oh, fuck.

Okay, so, Dr. Monte, tell us about your experience as reviewer number three?

Oh, well, I just wanted to quickly wrap up my comments about what you do when you get a kind of a snarky reviewer who's like, just cannot be published. So. So I responded to them, I think pretty well. And I also decided to address it in my cover letter when I submitted the paper. So I pointed out that I did get her little I didn't say harsh criticism, but I did say that I received some comments regarding the lack of control over my environmental variables. And I, in addition, some of these reviewers suggest that I do additional lab studies. Haha. And so I actually addressed that like head on and my letter to the editor. And I explained, I kind of I didn't say I didn't like come out and say I completely disagree with that I explained to them the central theme, and the the purpose of a music awesome study. And I made a point to tie in the importance of having laboratory and semi field and outdoor music, awesome studies into our so that that integrates our understanding of science. And so I was able to professionally and kindly address it while also not attacking that reviewer. Nice. And I think that that actually might have played a role in the acceptance of the paper. Despite that one reviewers misgivings other three were good, but that one was the only one that was on the table. Okay, wait, see, you later changed their mind. So you're not really

powerless in the face of Yeah. Yeah. In fact, in that case, that reviewer wasn't in your field. Yeah. So it was appropriate for you to attempt an explanation. Yeah, the significance of that type of study? Because they may not have been familiar with it right. and point out that actually, it's appropriate for studies to conclude at a place that suggests more studies.

Right. Exactly. Yeah. And also, I think it's good to be honest with an editor. So I actually was like, Hey, I'm, uh, this is my first time. This is my first first author paper. And so you know, excuse me if I'm doing anything and appropriately but you know, here I am. I'm trying to do my best here. And I think it's good to be honest, and just be like, hey, first timer. Yeah, you know, I've got other papers published, but I wasn't first author. Yeah, I don't know what I'm doing. So please help me. Yeah. I think that's great. Yeah. And so then, when I got the final comments back, the reviewer three, I think, let me change their mind. It was like, this is good to go. And then it got published. And I was very happy. Victory, Victory is mine. So after publishing this paper, I started to get a lot of requests to to review other people's papers, which was a very good experience.

And it's really hard not to be snarky, because you're going to this long paper, and you're finding grammatical errors. And it, that's going to take me to another big point, I have to make actually my biggest recommendation for people when when publishing, but if somebody if you can tell that somebody did not take the time to edit their own paper, can be very frustrating. And then next thing, you know, you're just snark is coming out in your comments. And it's it's like, it's hard not to be frustrated, because you're like, Okay, your paper got this far. And you probably should have had somebody else look at it. And so it's, I see now why the Snark comes out, it's easy for it to happen. But I try to be I try to be positive, and try to give them positive feedback. And then also, hey, this isn't so great. I found myself doing a lot of editing. And I had to stop myself, there was one paper that I had already had, like three pages in a Word document of just editing, and I just covered the introduction. Yeah, I just thought myself.

Yeah, cuz the job as a reviewer is not to edit right, is to give feedback on the science. Is

that right? Um, I think it's both. I think that's what makes it kind of kind of tricky, is that language does matter. And if the sentence is actually grammatically wrong, and you don't understand it, yeah, you have to say something to so another thing, a lot of the papers I got were clearly, I think grad students from China, from other countries. And I could tell English wasn't their first language. And so that was like, I felt I felt like I needed to help them. So I would actually reformulate some of their sentences to give them examples. Yeah, so I would take more time with them, because I was like, Oh, these poor people, they, you know, they don't, they probably didn't have an English speaker, edit their work. And I think that that's a really big thing for for anybody to consider anybody publishing is make sure your grammar is right. So that is correct. And make sure that you've gone through and made sure that there are no spelling issues so that a scientist isn't getting misdirected by that, because I've been very distracted by that I can't even think about the science, if your paragraph structure is is poor, and you clearly did not have something somebody looked at your paper before you submitted

yeah, you don't want the language to get in the way of your science, you want the language to, to put your science on display, you want the language to do yeah, to really showcase your science and not obscure it. Because, you know, presumably you want people to read and understand and you want the ideas that are in it to get beyond the little bubble that that you're working in. That's that's the whole point.

And there's a quite a few journals will give guidance to so if you're thinking of a certain journal that you'd like to submit to, they will often give guidance to people like if you're not confident in your writing being good, and your English being good. If it's for an English speaking journal, then they might have guidance, they might have suggestions for you. So there's options.

Yeah, can you ask an editor, if they have a recommendation for a resource to go to if you're not a first language, English speaker, and you want someone to help you improve your your English science writing?

So I would first look at the journal author guidelines, because it's, it's probably a question they get all the time. And it'll likely be in there. But I mean, I don't see why you couldn't contact the editor. Just understand that they depending on what journal it is, they might be very busy.

But I don't know, I think it never hurts to ask. Because all they can ever say is no, yeah,

it's important enough that you might consider bringing on a new co author.

Mm hmm. I agree with that completely. Like I said, I've been getting I've gotten some really concerning papers and have had to say No way. I'm sorry. And I even got one back. I reviewed one a second time. And it was still poor. And I had to say no, because I was just I was like, this is a really good journal. I can't recommend this paper be put in this journal as it stands right now. And you clearly didn't do what we suggested, which was find an editor.

So this may be a question that gets us on a tangent, I don't know the answer. But when it comes to, to non native English speakers, trying to submit or submitting to journals written in English, and in their writing isn't good.

I feel like there's a better way for us to be inclusive, because the science is hopefully still good. But if we can't see that through the writing, yeah, it makes it really hard. Yeah.

So I guess I'm just kind of wondering out loud, what we can do. But the other are certain term. I mean, I think a lot of journals that I've looked at, have some guidelines, and suggestions if they need some help with their English, but I just haven't looked into it enough.

I think is is a really interesting conversation. And I agree, we should follow it up. It's a tangent, we should follow it up people out there in TV, land, aka Twitter, Instagram, Facebook, etc. Please get back to us if you know something that's a good resource for non native English speakers to get their science writing, checked and worked on.

You know, that's beyond just sort of a grassroots, getting a co author having a collaborator in the field, that's a primary English speaker. English is the I think, the most common international used language for science. And so I do think that we have some responsibility to, like Danny said, Make that inclusive for people because, you know, you know, turns out most of the people in the world are actually not English speakers.

Right.

An d I think we would all benefit by having access to their science.

Yeah, I agree. It's, like really been heartbreaking to see because I can tell that the research is good. I'm like, this is really cool. But then, um, yeah. So usually, I'll find a way to make sure you know what I mean,

and I'm not saying you're being

Yeah, I just hear it when I see just a complete lack of effort. Yeah, it's really hard. Yeah, it's really, really hard. And I agree, we have to got to find a way. I think some some journals actually let

the reviewers maybe get a place on the paper if they can contribute to it significantly. And maybe that's a way where we can help non English speakers is by being like, okay, reviewer, reviewer three, is willing to help you edit your paper, she he will do it, would you mind making them a co author or at least putting in the you know, the acknowledgments or something like that, but that's a viable way of, of helping them.

So we're going to move on now after that lively discussion. And we're going to hear the perspectives of two experienced academic scientists on the following four questions that we exactingly crafted, aka, we actually took a fair hour, I think, to kind of come up with these. For your edification, as Will likes to say.

Our two remote guests are Joseph White, and Dan Quintana. And the four questions are: How do you deal with reviews and reviewers? How do you manage editors? How do you participate in the peer review process? And how could the peer review process be improved?

roll that beautiful bean footage.

[Laughter]

Hi, this is my response to the request to talk a little bit about how to deal with the peer review process. From stemculture.

My name is Joseph White. I'm a professor at Baylor, been here for 21 years, been publishing since 92. So sometimes, I've dealt a lot with editors and the publication process. First of all, how do you deal with reviews and reviewers? as a reviewer of articles myself, one thing I just want to make sure is that the science is solid. So I look for all the elements: Do they propose a question? Do they answer with the appropriate methods, or the data there? If they say that there are significant results, do I find them quite easily. The other part of that is simply writing is important. So being able to tell a good stories good as a reviewer, I like to see things spelled out.

People reviewing my articles, goes all over the place, sometimes people give you good reviews, meaning not necessarily that they agree or that they want to publish your work. Rather, they give you a good honest interpretation of your, the things that you want to look for: Have you written things well? Are you clear as to the hypothesis? Are your data solid? All those things that you want to see, if someone looks at that and gives you an honest appraisal, then you can say you've gotten a good review. And if you also have done those things, and they say, go ahead and publish, that's good.

How do you manage editors? I don't think any of us manage editors, other than the fact that sometimes we write them a letter. And in the letter, you want to spell out what the paper is doing, how it is associated with the journal that the you want to publish in. If it's new, that's good.

The big thing, I think, or making sure that if you have reviewers that you know, are going to be intentionally negative, for a lot of different reasons, people are people. And as we go to meetings, as we interact with people, and because science is a social phenomenon, then we make people mad. And so they may not want to publish your work simply because they don't like you. And so one of the things in talking to editors is to make sure that the reviewers are going to operate and work as objectively as possible. And so trying to clear those paths is best.

How do you participate in the peer review process? Well, when you get the emails from various journals, you just say yes, if you have time, sometimes I have more time to review, I usually on a given month, review anywhere from two to five articles. That's a lot. And sometimes they have to slow down. But the process of participating is one having published work. And then as you publish, where people will get to know what you do, and editors get to know what you do. And then you may get put on a list and and then participate.

Generally editors then get to know which reviewers they can rely on both in terms of trusting that they're giving an objective opinion, and that they do things in a timely matter, these days, looking for, you know, two to three week turnaround on reviews. In the old days, you know, somewhere between four to nine months was common. But thankfully, people have gotten where they're reviewing faster.

Finally, how could the peer review process be improved? I think the big thing is double blind reviews. And that is because when you see someone else's name on a paper, again, Sciences is a social phenomenon. And you may have feelings that push you one way or another. And we want science to be again as objective as as possible. And so removing who the other person is on the other side, just like they take out your name as a reviewer is important. The alternative is what a few journals are doing is basically making the reviewers themselves non anonymous. And so it becomes a completely open process by which the review itself becomes part of the publication. And I think that's good to some degree.

But I don't know, I think the review process needs to take out a little bit of publishing work from people who have been successful at previously publishing it, rather making it more and honest, each each article has to stand on its own merits its own science, its own contribution, rather than on the merits of the individual, or the lab of the individual. So I find that to be kind of problematic. Anyway, good luck. Hope you guys are successful in your publishing. And hopefully, you will be asked to review a whole bunch of papers in the near future. Bye, bye.

I thought that was great.

Super. Yeah, I mean, I, I completely. I really enjoy hearing somebody talk about the fact that science is performed by human beings who are socially influenced and think socially, all the time, whether they acknowledge it or not. And you know, we're not objective. And you guys talked about this a little bit in your writing episode with passive versus active tense. And this idea that people had some time ago that taking yourself out of the language somehow made the study objective, and that's just not true. And I think that's great. And that's exactly why peer reviews important and why we need to work on improving it.

I was surprised Joseph said he reviews two to five papers a month. That is he's a hero. [laughter] I don't I don't know anybody that does that many, many. How about you Moncie?

Yeah, no. That's, that's awesome. Yeah.

And that's a big service to the community. So it is yeah.

Yeah. Yeah. I wonder how much time he takes for each one. Knowing him, it seems like probably quite a bit.

That's a good question. Right. I mean, it sort of gets into what he was saying later about letting each paper stand on its own merits. Because why wouldn't you? And I think that the clear answer is, well, one of the possible clear answers is that it's an easy way out, you know, maybe the person's lab, you know, the previous work they've done you say, Oh, well, you know, they always do good work. So I'm going to slap my stamp of approval on it and pass it along, or the converse, but one way or the other, you're not giving, as he said, an honest appraisal of that study on its merits.

Awesome. Let's hear the next one.

I'm excited.

My name is Dan Quintana, and I'm a researcher in biological Psychiatry at the University of Oslo. My main research interest is understanding how both neuro peptides and the autonomic nervous system impacts the way that we think and feel. But more recently, I've become interested in meta science, which is an emerging discipline whose goal is to evaluate and improve research practices. Along with my research, I also host two podcasts. The first is the Physiology and Behavior podcast, which is a daily update on the latest physiology and behavior research. And this is also available as an Amazon Alexa briefing if you have one of those devices. The second podcast is Everything Hertz, which I co host with James Heathers, we release episodes twice a month on methodology and scientific life in the biobehavioral Sciences. Now, when it comes to dealing with reviews, and reviewers, I had an interesting conversation with a PhD student in my lab this morning, actually, who was agonizing about what type of analyses to that we're going to include in their submitted paper wanting to make sure that any sort of analyses the reviewers wanted, that we're going to cover. Now, when I speaking to her, I let him know that well, you never going to be able to actually guess exactly what the reviewers want. So there's no point in an agonizing about this, you just need to submit what you think is, is going to address the research questions that you have. And then of course, the reviewer is going to let you know what they want to do. So it's important not to be stuck with this. And just to submit the best manuscript that you can.

When it comes to actually dealing with revisions. One thing that I always try and do is I try and reduce friction as much as possible. Being on the other end, being the reviewer of manuscripts. It's amazing how much work some authors put you through. It's really important, of course, and it goes without saying to address every single question, you don't necessarily have to agree with the reviewers. And if you don't explain why, but there are little things that you can do to make the reviewers life a little bit easier. Firstly, if you are editing some of the texts in response to some of the queries, paste in that text in your response to the reviewers, and also show them where to find the manuscript. If the journal doesn't already add continuous line numbers in your manuscript. So you can actually tell the reviewer exactly where to find these things. Now, when it comes to reviews, we'd like to think that review is evaluating these things objectively, but they're not they're human. So if you make them work harder, you are already on the back foot. So you want to reduce friction as much as possible.

When it comes to managing editors, it's important to always keep in mind that each editor is trying to solve a specific problem. And that problem is, how do I fill my journal with impactful articles that fit its scope. Your job as an author is to help them solve that problem. And that is where the cover letter comes into play. So many people treat the cover letter as an afterthought when they're submitting papers. When I think you need to spend a disproportionate amount of time on working on your cover letter and how your particular manuscript fits the scope of the journal. One concern that every journal editor has is that no one is going to sign a paper. While citation numbers aren't necessarily a perfect indicator of paper impact is certainly much better than going back back to this. One thing that I've started doing is pre printing all my papers or pre printing as many papers as I possibly can, at least a week or two before I plan to submit my paper to a journal. And then I include the download numbers of those pre prints to demonstrate to the editor, that there's already interest in the paper. And this helps head off any potential fears that no one's actually going to read or cite the paper.

When it comes to the peer review process, I try and review about two papers for every paper that I publish. Now in reality, I probably say yes to more papers in this because so many just seemed too interesting to pass up.

I always tell myself that I need to read more papers to keep on top of the literature. So by saying yes to these reviews, not only do I have had the chance to contribute to my field, but this also helps me keep up to date with the literature myself. Now one tool that I use to help me with the peer review process is Publons, which is a great online tool, which helps you track all the reviews that you've been doing. And you can even save the review contents. Now it's up to you as to how public you want this, this information to be you can just reveal the journal your you have reviewed for but you can also publish your reviews as well, assuming the paper eventually gets published. So it's a really good really cool tool to actually help you keep track of the papers that you're publishing and even your paper ratio, that is how many papers you review per papers that you publish. When it comes to actual review itself. I always try and review papers like I like mine to be reviewed. Now nothing is more frustrating, when you consider if you are hinting that they want you to cite a particular paper or use a particular method without actually telling you specifically what they want. Always try and include as many references as possible for my in my reviews to help the help the author out.

Now there's a few ways the peer review process could be improved and some more realistic than others. Right now the peer review process occurs behind closed doors, which increases gatekeeping both from editors and from reviewers. One thing that can bring reviews out into the open and to reduce gatekeeping is to make the review process transparent. And this can be done by publishing the reviews. And the response to response from authors. And some journals are doing this and it's fantastic. And I hope this becomes more widespread. It's really interesting as well, just seeing how a paper evolves from the original submission to the eventual publishedr papers and where these journals, you can see this process and you can see the requests made by the reviewers, and how the paper changes. And related to this, if you're a tenured professor, I don't see any reason why you shouldn't sign your reviews and that your review should be public. While whether the early career researchers should sign their reviews is a different story, I think and should be taken at a case by case basis. But if you're tenured, I don't see any reason why you shouldn't. And by doing this, and by signing reviews, tenured professors are unable to hide behind being an anonymous reviewer, which increases the risk of gatekeeping.

Can you can you just explain to me again what Dan Quintana means by gatekeeping?

Yeah, so by gatekeeping, he means there are barriers. So there's barriers to people being able to be successful.

So barriers to getting your publications accepted.

I think in this case, let me see me, let me look at my notes.

So the way I'm interpreting it, is it seems like he's saying that

because of the social imbalance between people submitting papers and people reviewing papers, because of the anonymity of reviewers. Reviewers are protected

by their anonymity, and so they can shut down papers without any consequences for less than laudable reasons. Is that it?

Yeah, I think so. That's how I took it.

Yeah.

Essentially. I mean, Joseph sounded like he was saying the same thing.

Right. They both mentioned the anonymity of reviewers, and the imbalance between that. And

yeah,

Dr. White said that there should be parity one way or the other.

And Dr. Quintana,

I guess proposed a sort of more intermediate solution?

Well, I think they were on, they were on different parts. So Joseph was suggesting that there be double blind during the review process. So no one knows who's reviewing and no one knows who's being reviewed. And then what Dan is suggesting is that we be able to see after the review process is done. We're able to see the names of people. And he's suggesting, like tenured professors can sign off on that, but also that some journals, like I'd mentioned earlier, are actually publishing the conversations around the review process. Yeah, so I think it's, it's just another step that could be taken. I think it's they're kind of separate.

Yeah, I agree.

I guess, though,

you could sign or review at any stage of the review process.

So

yeah, I mean, but the only chance you would have is to do it after you've actually submitted your review?

Of course.

Yeah.

Or you could do it after the second time. There's not always a guarantee there's going to be time.

Does he prefer Dan or Dr. Quintana? Dan,

I'm just calling everyone by their first names because I'm that way.

Okay, well, I'm gonna just be contrarian. Dr. Quintana.

Okay.

Dani and I have to do the opposite all the time, everything we do. So I thought he was a wealth of information and sort of specific insight into the way that things could be improved and things that you can do.

I am currently working on

what's going to be the first paper of my, my PhD and

his suggestion that it's not going to be perfect, you're not going to address all the possible reviewer criticisms, and so you just have to, you have to put something out there that you think is good and submit it.

Yeah.

It's definitely a good thing to hear for me.

Yeah, for sure. And I really liked how he also said something very similar to Joseph as well, where editors and reviewers are human. And and Moncie brought this up as well. But being kind in your responses, and clear is going to get you much further than responding in any kind of

adversarial way.

Right? I like, yeah, I like the way he put all that reduce friction, right? Is that what he kept saying?

Yeah.

I love that: Make things easy for them. I completely agree with that: page numbers!

Same thing with the numbers,

My numbers and all that fun stuff - I gave both.

Yeah.

So to make sure that

takes it beyond just the language. And as you know, the the structure of your submission, should ideally, really minimize the barriers between the people who are looking at it and your science.

Yeah, make it as easy as possible for them.

Yeah. Because I mean, I know when I review, I'm reviewing between, like class and teaching, and in the middle of like a lab experiment, or very late at night, or very early in the morning, right? I'm trying to squeeze it in where I can. So the easier it can all be, the better.

Absolutely.

We'll have to go back in and look up that system that Dr. Quintana was talking about.

The Publons?

Yeah. Tell us about that.

Yes. Okay. So Publons is great, because a lot of the time, well, every time when you review, it is this silent work that you do. And it's been really hard to get any kind of feedback on that, or any kind of

any kind of credit for that. Because it is work that we do. And, you know, like Dan was saying he really likes to do it, because he really feels like he can contribute to the field when he's peer reviewing. And I really feel the same way you get to see new exciting research that's coming out, you get to see the direction that the field is going. And so, but it'd be nice to get some credit for that work. And so Publons which is P, U, B, L as in Lagunitas Brewing Company, O, N, S as in salacious details, publons.com. [laughter] I'm being really irritating.

You mean, magical.

[laughter] Thank you,

is really great. We will link it in the show notes on the website, STEMculturepodcast.

com.

It is great. It's free to sign up. And you can add in your details. So basically, you just forward them, your email from the reviewer oh sorry, from the editor that says, hey, thanks for your review. And you send that to Publons. And then they acknowledge that you have done this review. And it doesn't, you can be as specific as you want, or as general as you want in terms of the review. So I have one review on there. And it just says like the journal and the month that I reviewed, and anybody that I reviewed for, they're not going to know it was me.

I don't think but also this is ages ago. So

You just reminded me actually that some of the publishers are doing something similar.

Oh, cool.

So I'm actually like, I actually have credit with one of the major publishers for something and I didn't, I haven't used it yet.

Credit for like... Yeah, go ahead.

I was gonna say what can you get with your credits?

Like a book, or a

Key chains?

[laughs] maybe...

t-shirts, hoodies

lighters

leggings

tumblers.

handcuffs! [laughter]

Excuse me. This is a family program.

Just in case...you never know when you're gonna need handcuffs.

I mean, that's never been truer.

And zip ties. [laughter]

We've been watching too many crime podcasts.

I'm sure people listening to this might listen to other podcasts. And I just want everyone to know that I refer to true crime podcasts as murder porn. [laughter]

That's really offensive. [sarcasm]

Moncie hates it. [laughter]

So those are our opinions from experts on the peer review process for now. Look forward to hearing more of those as they come in. We're really excited to hear more, I think this has been great.

Peer review is a central issue for the culture of science. It's one of the layers of glue that allows us to incorporate our ideas into the greater body of human knowledge. The peer review system isn't perfect. But while we work to change that we still want to be successful within the system that exists.

This is a topic that is really important to scientists. And we've got an amazing response from many of the professors and postdocs we've asked for input. And we're definitely planning to continue releasing these interviews and bonus episodes as they come in.

Ultimately, the peer review process...Ultimately, as the peer review process becomes better so does science. And I think that's good for everybody. We want to thank Dr. Moncie Wright for joining us today. We hope that you'll join us again someday soon.

Thanks for having me. This was awesome.

Yes, we're very happy to have you.

Meow

[whispers] In as many ways as possible. [more laughter]

Thanks so much for listening. Next time we'll be hearing from Dr. Brian Chad Starks, who has a PhD in criminology and our third In STEM episode, you'll hear about his story and his work on diversity and inclusion.

We are on Twitter, Instagram, and Facebook as STEMculture (one word) podcast

Search on Google and you will find us and when in doubt, visit our website, which I personally curate so you better visit it at STEMculturepodcast.com (sorry for threatening you), for Show Notes References. You can also find information on all of our guests today on our website and the guests tab.

Until next time, don't forget to consensually hug a graduate student or at least buy them a coffee or give them some substantive feedback on their work.

Fucking give them feedback! [end of episode]

[beginning of blooper] In the end of Candide, the novel by Voltaire, the my favorite part anyway [laughter]

Go ahead and express yourselves [laughter]

You're just so intellectual

I know. Well look, you know, I, this liberal arts education and all the bullshit that I read in high school. I get to use a little bit every now and again, so.