Hi everyone, we are just waiting a few minutes since we had a very big response for this webinar, and we want to give people a chance to join we can only push admit so fast. But we'll start in just a moment
I want to start by echoing Justin. It's so fantastic to see this crowd of people both wonderful familiar faces and also wonderful new ones. And names. So many of you popping up it's it's hard not to just continue to look and grin at you. But I will start us off. I'm Susan Benesch. I'm the founder and director of the dangerous speech project. This webinar is is ours together with Justin Hendrix and Dave Willner. Our co hosts and it's it's ours because of some fantastic work. pioneered by Kathy burger, our director of research we have done the most extensive body of work on Counterspeech which we define as discourse, human expression in response to hateful or dangerous speech designed to undermine that bad content. We have also more jokingly called this arguing online it's it's civic engagement. We're joking, but that that is often what it amounts to. And we have now become interested along with many other people in how AI might be used to scale this sort of effort to enhance it, but also how it must not interfere with it since there are certain things that that people can do better than computers and vice versa. That's as as much intro as all as I'll give since everybody else is going to speak very eloquently about all of this. So just a few quick guiding notes. Since so many people are joining us for this webinar will ask you to put questions and comments in the chat. Instead of speaking just to save time. We'll get to just as many of your questions as we can. We'll do the webinar for 45 minutes. And then at that point, we will switch over the q&a for at least 15 minutes more. Also, we are recording this webinar. So we'll be therefore able and more than willing to share it both with you and of course with other people who weren't able to attend because of conflicts. We want to thank the MacArthur and Ford Foundation's for supporting the work of the dangerous speech project that we will be describing in the next few minutes. So thank you so much for coming in with that I will turn it over to Kathy.
Hi, everyone. Let me just echo Susan's very warm welcome. I'm so pleased that you've all decided to come and listen to us for this hour. It's a topic that we have been thinking about and working on for a bit of time now. And so it's a great opportunity to kind of bring together lots of different voices that are thinking about this topic from different angles to engage in this conversation.
We find this topic extremely timely, right? It seems like that everybody is talking about generative AI right now, and finding new sometimes surprising things to do with It. People have used AI tools to construct and design parts for spaceships and predict earthquakes and even create strategies to help people win in the Olympic sport of curling. So it's only natural that there would be tremendous interest in using AI tools to help combat the huge amounts of troubling content that we see online every day. And we've had many people approach us saying that they're trying to develop tools to do just that. But few have taken the time and effort to involve counter speakers in their design process.
So that's a bit of what we're trying to do today, bringing together people who have extensive experience in challenging hateful speech online and doing peacebuilding work with people who have tremendous technical expertise and can help us really understand the possibilities and limits of AI tools as they stand today. So without further ado, I'm just going to very briefly introduce our speakers for today and then turn things over to them. I would encourage you to read their full bios. They're all very impressive people. So I'm going to drop that link in the chat now. If you haven't had a chance to look at those. So let me go through just who you're going to hear from.
Alena Helgason is our first person that I'll introduce. So in 2018, Alena founded a Canadian counter speaking group, under the umbrella of I am here International which is a very large collective counter speaking organization. The Canadian branch then registered as an independent nonprofit organization, which is called We Are Here Canada in 2021. Next is Martin SAP and Assistant Professor in Carnegie Mellon University's language technologies. department and a part time research scientist at the Allen Institute for AI. His research focuses on making NLP systems socially intelligent and understanding social inequality and bias in language. Justin Hendricks is CEO and editor of tech policy press a new nonprofit media venture that's concerned with the intersection of technology and democracy. Previously, he was executive director of NYC Media Lab, and he spent over a decade at the economist and roles including Vice President for Business and development and innovation, business development and innovation. He's an associate research scientist and adjunct professor at NYU Tandon School of Engineering. And the fourth presenter we have to talk about today is Dave Willner. Dave began working in trust and safety in 2008, where he joined Facebook's first team of moderators. He went on to write the company's first systematic content policies and to build the team that maintains those rules to this day. After leaving Facebook in 2013. He created and ran Airbnbs community policy team and then joined open AI as the company's first head of trust and safety. He now works as an advisor for AI companies and as a non resident Fellow at the Stanford cyber policy center. So as you can see, we have a tremendous and varied amount of expertise with us here today. Just to give you a sense of how this is going to go. We're going to start with some opening remarks from Dave Willner and Justin Hendrix, before moving into a conversation with Elena Ferguson and Martin Sapp. Then we'll conclude with some remarks from Dave Willner and turn it over to the audience for question and answer. So let's get started. Over to you, Dave.
Thanks for Thanks for having me. And thanks for organizing this. I think the topic couldn't possibly be more urgent. And frankly, I'm glad that the dangerous speech project is hosting my history with you all goes back a really long way. And in particular, Susan's thinking around how we can more more concretely fit rules to understanding when speech becomes extremely dangerous beyond simply true threats was really influential. On a lot of the work that I did. So just thank you for doing this up top. I don't want to take up too much time before we get into some of the perspectives from other folks. But just thought it was worth framing the conversation around a sort of metaphor of industrialization that I've been thinking about a lot and dealing with this a lot of my work right now deals with figuring out how to use language models specifically to do moderation, not Counterspeech. And I think in that context, that that sort of logic of industrialization basically works right like language models have, have enabled us to industrialize in a way that we've never seen before the production of speech of all kinds, and we are going to need tools that are capable of keeping up with them. But it seems to me that falls apart in the context of Counterspeech Simply because the bots on the other side can't be convinced themselves since they're not people. And because the sort of relational art of the thing is significantly beyond at least our current automation technology. And so I'm really interested in looking forward to hearing from the sort of other experts on the panelists we will dive in with that I'll turn it over to Justin.
Thank you, Dave, and I'll say I've been pleased to host some some of these thoughts about content moderation in large language models on tech policy press, and to learn from his recent events that he's done at Harvard and a couple of other locations where he's talked extensively about these things. And I would commend you all to that. I'll put a couple of links just in the chat to some things that we've published lately, including a transcript of that talk, but also a kind of syllabus. Cathy mentioned I also teach at NYU lately also at Cornell Tech. So sometimes when I'm trying to get my head around something, my instinct is make a syllabus or make a make a draft syllabus. So we've kind of done that, on what I'm thinking of is sort of large language models content, moderation and political communication, which may not hold together as a as a kind of category of thing. But I think it might, we'll see in the long run. If you have some ideas about what else should be on that syllabus, let us know – it's not complete. It's just a sort of first stab at it. So take a look at that. But you know, I think they've done a good job in some of the things that he's written and said about the opportunity, particularly around around content moderation, and I think Cathy and Susan are thinking a lot about this in the context of counterspeech. And I'm going to put a piece that they wrote for tech policy press also in here, as well. That's on the topic of kind of, you know, how we should think about the perhaps the limitations of chatbots for for counterspeech. And I guess, you know, without kind of going into too much length, with everything AI it seems like the right thing to do at the moment is, you know, take take one spoon of experimentation and about five spoons of caution with everything you do because, you know we're running essentially, or many people are running live experiments with no control group and trying to kind of see what happens I think we're seeing across the world right now, an enormous number of experiments playing out in election contexts. And we're seeing a lot of money being spent. And it certainly I think in India, we're already seeing lots of very interesting phenomenon emerge. There's going to be lots of science and study of these things going forward. And I think I think it will hold together as a kind of category of research that many of us who are concerned with issues of tech and democracy, tech and politics, tech in society will begin to sort of see as a kind of field, or an area that builds on a lot of the work that's happened over the last, you know, decade or so, around social media. And then you kind of begin to look at it through the prism of generative AI and its impact on political discourse. So we're excited about that. Study, I should say, not necessarily about all of the implications of of what's going to happen. Because I think, you know, on that will remain perhaps ambivalent. or neutral at the moment and see what happens. But I look forward to the conversation today. And to hearing a little bit more from from the other speakers about what it is they're seeing with regard to these phenomena. And what are the questions they think the community people on this call should be asking because I think that's the mode we're all in is trying to form hypotheses form questions, or form experiments in some cases, and think through possible implications of of all of that above. So with that, Kathy, I'll cut a stop and turn it back to you.
Thank you very much for for those thoughts. So keeping those thoughts in mind, I'd like to turn to Elena and Martin now. So let's start by giving you both just a few minutes to introduce your work in more detail. Tell us a little bit about what you're working on now and your background how you came to this conversation. Alena. Let's start with you.
Thanks, Cathy. So, as you had mentioned, we did start the grassroots initiative in 2018. And that happened after witnessing the public response to the killing of Colton Bushi a young indigenous man and the subsequent trial after that, and we found that social media was full of hateful racially charged posts and content, justifying his death, there was a lack of empathy, the lack of counter narratives, so that was really hard to see on social media. The second kind of factor that launched we are here Canada was a little bit more personal. I had a friend named James who lived in a very diverse neighborhood. In our city. He was friends with all sorts of different types of people and he was just a regular guy. He worked in trades. He also dabbled in photography and film, and during a conversation he was telling me how he was scared for him and his family, because he believed that at any given moment, should the religious leaders of some of his neighbors decide to take over the community. Because he is a white man he would be the first one targeted. And that was baffling to me because he's a normal person. And I wondered how if he could be so influenced by the these, this disinformation in these beliefs, who else was being influenced? So during that time, we were lucky to discover the dangerous speech project and, and to really use it as the foundation of of starting our counter speaking process. Counter speaking, involves, you know, finding articles with hateful comments online. Primarily, we used Facebook, which has gotten a little bit more difficult with the news media van and Canada, but we would find articles full of misinformation or hateful rhetoric, and then our members would go in and start posting a counter narrative usually facts, links to sources empathy, then other members in a coordinated kind of process would boost those comments. That way those comments rose to the top they were the first seen and it kind of buried all the hate. And we were quite successful in many articles. And it was really exciting to see when non members when just regular people in the public would get engaged with that as well. The goal was never about engaging, like the hateful protesters or the or the bots or anything like that. It really, it was was really the focus was on the audience as the casual readers who are just passively scrolling through and maybe they were members of marginalized communities who would feel less alone by seeing supportive comments. And it would also create, you know, spaces so that other diverse voices would be encouraged to speak up and the goal really of counter speaking is to slow the spread of rhetoric and misinformation, before it influences people in real life. Who passively accept harmful policies based on fear and ideology. So that's basically counter speaking in a nutshell
thanks so much, Alena. Maarten, you're gonna tell us about yourself. Yeah.
Hi, everyone. I'm really excited to be here. I'm a professor, assistant professor at Carnegie Mellon, as Cathy mentioned. So I am a little bit more in the academic ivory tower. So I'm really excited to get to connect with you all and sort of, you know, hear the actual things that are needed out there. I am particularly interested so I'm in kind of, like natural language processing, which is the field subfield of AI that deals with text as my field used to be completely unknown until the time CPT became a thing about everyone. Anyway, my nine year old grandmother are asking me about it, which is kind of wild. I liked it almost better before, like a known thing that I was doing. But yeah, so I'm really particularly interested in this in language. So what drives me in my research is kind of understanding like how humans are so expressive in the way that they talk and they're so able to communicate efficiently. And language is a beautiful medium, but it's also completely lossy, and it's such a like, you know, crapshoot of like misinterpretations and misunderstandings, and that's sort of like the benign side of things, but then hate speech and microaggressions and all that other stuff is sort of like the worst of how language can sort of function and reify existing inequality and sort of like hate and things like that. So that's kind of like what I'm really interested in. And I've done a lot of work related to kind of how AI systems can help us make large sense Oh, it makes sense of like large amounts of data and trying to understand particular social phenomena or things like that and more kind of studying social science questions. But then I've also worked on like, can we use language based technologies or language based AI to enhance or help with various socially important tasks? So I've done work on hate speech detection, which I'll talk a little bit more about, but also Counterspeech As we mentioned, and other things like improving human human connection with AI and things like that. So on the topic of hate speech detection and Counterspeech I was actually one of the first research papers to show that hate speech detection algorithms can backfire against racial minorities. Back in 2018, we found that there was like strong racial biases in sort of these pipelines. And that, you know, everyone had good intentions going in there, but like the way that they're designing things and thinking about this task was like sort of really backfire against minorities. I've also developed methods to sort of kind of help AI systems spell out the hidden, subtle, subtly prejudiced implications of language. So thinking a little bit more about the microaggressions sort of side of the spectrum. But you know, you can apply that to like fake news or like, kind of like presuppositions like things like that. I've also worked on like, sort of applying these explanations into improving content moderation pipelines. And so like showing that we can improve content moderators like flagging decisions if we add these like aI explanations. I've also recently worked on developing sort of like more Counterspeech related sort of things. So one work that one of my students was doing was related to developing countering strategy like rhetorical strategies to better address kind of the underlying stereotype that a post might be sort of implying, and then other work was with Kathy really excited to have worked with her on studying the needs of the AI needs and hesitancies of counter speakers. And sort of Yeah, the pitfalls and stuff like that. I've also done a bunch of other work on like aI ethics and like, yeah, how detoxifying your lunch language models will erase marginalized voices and like, there's a bunch of other things there. So, but yeah, I'm excited to talk. Great,
Thank you so much. Okay, so you mentioned that you and I have had the joy of being able to work together to study Counterspeech your needs, so let's get to that. Later. I have a question for you about those needs, right? So we know that if Counterspeech is going to have a real impact on online discourse, there needs to be right there is a lot of speech online and when we actually think about discourse, norm change, we're talking a lot of speech. And I think that that is generally one of the arguments that we hear when people come to us with these ideas of using more automated tools is the ability to scale up some of these efforts that are now happening on a relatively small scale. So I know that your group works very strategically to amplify your impact online and have created kind of a method of doing that. But can you tell us some more about the barriers that you and your members face that make it difficult to do kind of even more Counterspeech What what stands in the way from your perspective?
Thanks, Cathy. So there are quite a few different types of barriers that we face, even if it doesn't even involve AI or technology and I think one of the biggest ones is apathy. The only way to not the only way but to ensure that counter speaking is effective. We need the numbers we need the people to speak up and, you know, how do you inspire everyday citizens who are just going about their lives to speak up, especially when they aren't directly impacted? And we certainly see the response I guess, even with issues regarding the trans community because so few people are directly impacted. Few people have Trump's family members or friends. Harmful policies are being easily passed just based on ideology and misinformation. So apathy is huge. We don't know how to inspire people to engage. Lack of knowledge is something else that really impacts counter speakers. So even anecdotal ly with our own members, a lot of times they don't want to speak up because they don't know enough about a topic. And there's also several organizations that back that up on from media smart Stephen L'Oreal Paris. Who have studies that show that people don't speak up because they aren't sure what to say or what to do. And online, it's difficult to determine facts from misinformation. There are so many and you've all seen them so I don't even need to go into examples but they look real they look legitimate, they sound factual, and if they're kind of buying into our own biases and our own beliefs, it's really easy to just assume that they're truth. Now there's several excellent sites such as science up first, the Canadian anti hate net org or even Southern Poverty Law, who have a lot of information that people can use, but it's not always easy to find. We know that everybody's really busy with their lives and people don't fact check people don't look for information. So finding that current correct information quickly and easily can be really difficult. But I think the biggest barrier that we have is the scale of disinformation. And again, it's not a secret This is not a surprise. But because there's so much of it, it leads to the reiteration effect. And the reiteration effect is basically the tendency to believe false information after repeated exposure. The thinking sorry, the thinking is power website has an excellent article on that. So feel free to check that out. And certainly there is no better evidence about the redirection effect than just by looking at our own political arenas. In Canada in the US, for example, just to see, you know, what it's created. It's created the freedom convoys and the extra tax tours and the attacks on critical race theory or sexual orientation, identity education, and it's really difficult countering people online with the amount of disinformation it's even harder to, I don't know, engage Uncle Joe at Thanksgiving dinner, and kind of counter the disinformation that he's receiving when he's surrounded by it so often, we kind of think if the sheer volume of disinformation is overwhelming and if it discourages the plucky band of counter speakers from speaking up, how many other diverse voices are, are silent. So I think those are probably the biggest barriers that we're facing.
Thanks. Yeah. And hearing you talk about that it really resonates with I think what we found Martin in our research looking at, you know, we did in depth interviews with experienced counter speakers, and also ran a survey with the people who might have done Counterspeech or might not I like to think of them as potential counter speakers, right. And when we asked about barriers and needs, we found that there were some differences between the groups, that the people who had done Counterspeech for many more years and we're doing it kind of in a systematic fashion. We're asking some of these questions that Elena was asking, right, that thinking about strategy and how do we how do we kind of figure out which pieces of text to respond to that are going to have an impact? How do we do this work more quickly, more efficiently, whereas people who are maybe just getting started or like the idea of counter speaking but haven't done it, to have more of that pressure point in kind of in thinking about what to say, is this the right time to engage? What if I don't know all of the answers? Is it okay if I engage here can I do that right? So there are slightly different needs based off of your experience with Counterspeech which makes sense I think intuitively. So when I when I want to turn this question to you, Martin, from a technological perspective, where do you see possibilities for AI based tools to come in and maybe help address some of these barriers? And where are the limits for you, at least kind of where we stand now we know that this technology is changing and adapting very quickly. So maybe there maybe this will change but where are you seeing the range of possibilities at the moment? Yeah,
so I went back and kind of like refreshed my memory on the the needs that we the big themes that came out of our paper and tried to think a little bit about like, is this easily sort of something that we can address versus is actually more challenging? So there's kind of four things that we noted and it was AI for training counter speakers, and AI for identifying hate speech and for amplifying Counterspeech And then collaborative AI tools for factual suggestions. Were kind of the four main ones. And I think that there's some levels to which AI can help in each of these. So for the training counter speakers piece that was a thing that was kind of related to this, like, oh, I don't know what to do. Exactly. So it's kind of like onboarding people to to help them but I think that would require kind of like, developing curriculums for like, what good Counterspeech looks like. And then porting that over to like maybe AI systems like you could imagine like simulations like someone like there's a social social network simulation, and someone comes in and sort of sees the effects of it. So like that would maybe be a possibility that we that we could imagine. It's going to raise a broader question, which I don't want to get into breasts right now. But that raises the question of effectiveness, right? And like, how do you measure that in this training sort of setup? So that's kind of One Direction AI for identifying hate speech we've like basically already talked about so there's tons of work on hate speech detection. But the biggest challenge there other than these tools are not always perfect, and they can backfire. But I don't think that I never thought of them and they should never be thought of as like the decider of whether something should be moderated. They should just be like, a suggestion to denture to a human right. But the big challenge there is like access to the content in a way that like an AI system could sift through and I think I'm not sure how easy it is to like, you know, have your own bot or your own AI system filtering Twitter or like Facebook or Tik Tok or something like that. Like I know that companies could do that, but they're probably not going to do that. Right. So how do we how do you get access to that? I mean, there's there are other ways like you know, using hashtags or specific keywords and sort of looking for that you could think of like more bootstrapping approaches, like you sort of start with like some things that you notice like manually and then try to like look for similar content online, but even that is going to be challenging. So that seems like the technology might be usable, but the system and framework is not in place for that, basically. AI for amplifying Counterspeech I'm really interested in because that was the one that I was the most puzzled by because I I think that is almost more of a platform design decision again. So like yeah, I was like, I'm not sure how that would work. I mean, because we don't want bots of voting Counterspeech necessarily, either. Right. I'm seeing some people laughing. I'm
gonna pop into this is the thing we were referring to as the cheerleader bot, right?
Yeah, yeah. Yeah, that seems interesting. Yeah. Like if do we want that? I don't know. Um, so that one yeah. Like, I'm imagining someone could create a social media platform where like Counterspeech could be sort of like inherently more voted or something like that. But then presumably, that platform would be nice enough that you wouldn't need Counterspeech But yeah, so and then the last one I think is really interesting is collaborative AI tool for factual suggestion. So this is actually really related Elena to what you were saying about like, people not knowing the facts. So I will say I think there's challenges there because general AI suggestions are very doable. You we can we can absolutely. I mean, it already exists. Like people can do that kind of stuff. The connecting with relevant facts is more challenging. Because it's hard to know like sort of Yeah, how to sift through a large amount of data, especially once you sift through large amount of data. A lot of it might have like the misinformation that you're trying to counter like as like the data in itself. So that is kind of challenging. There are some like, real sort of approaches like Chad GPT, like kind of like the if you've ever used the like Bing AI, like, or Google's AI like they have like sort of sources that they cite. So there's like this called retrieval augmented language models. And so that could possibly help like if you sort of have your database of like trusted facts, you could potentially have like, an AI system, like go into that database and sort of retrieve facts that are relevant. But yeah, I think that would be kind of hard. And again, this makes me wonder because there's been a lot of work out there. Um, like, I think people's initial reaction is that they want facts to back up what they're saying. But I feel like there's been a lot of psychology work that has shown that like facts are actually not always going to help bridge the divide. So obviously, we're not trying to convince the party you know, who's posting potentially the hateful stuff. But I wonder and I think we need more work on like, sort of like, is factuality really the sort of goal or is that just what people think that they want? But what we want is just more general suggestions that have like the right sort of like rhetorical strategies or the right emotional strategies or things like that? Probably. It's both. But yeah, so that's all kind of stop there. Yeah,
no, that's I think that that's kind of where my thinking is on this too. And I do want to kind of plant a flag in this idea of the cheerleader, but I'm gonna explain a little bit more what it is, since we just kind of mentioned it here. There was an idea that came up both in our conversations with counter speakers. I think that's kind of where it emerged initially. For this, a way that could there be a tool that would help us find the good speech out there instead of just trying to find the bad speech and responding to it but helping us find the good speech and engage with that instead, since so many platforms kind of reward engagement, right. So how can we find the things that we like to see online and make those more prominent, basically based off of the I am here model right of amplifying the Counterspeech that is there. So I'd love to hear your thoughts on that Elena, and then the other the other question that I want to ask just for sake of time I'm going to ask all of this together, is that we know there's a lot of interest in building these tools, right, varying varying models, some of them to kind of create suggestions that counter speakers can adapt and use on their own, some are a bit more automated in their design. And we were there's a lot that needs to happen before I think we launch our full support into this. So I'm curious kind of what your just initial gut reaction. Are you excited about these tools? Are you are you nervous about them? What What questions do you think you want answers to before throwing your support behind a rollout of an AI based Counterspeech tool?
Thanks, Kathy. So for me, I'm kind of a tech optimist, so I might not be the best person to ask about what I would need as reassurance so I took um, we had actually been talking a lot about AI in our group with our leadership team and and things like that. And there were some patterns that came out or some, some topics that that came up quite a bit and so one of them is just that reassurance. I think in the understanding that AI is more than just bought armies. It seemed to be that overwhelming perception that that's what it is and and they were worried about losing that human element to everything and I think it's probably the same article that was referenced that you and Susan dead about losing the sense of humor and the nuance and everything by using AI. And then the second thing that had come up, which kind of relates to what Martin was talking about is the language learning model, whereas AI getting its information, if there is so much hate and disinformation and rhetoric online, is that what AI is going to be learning. We had discussed, I think, in a conversation previously about something, a tool that might really help counter speakers. And I kind of saw it as sort of a drop box kind of thing where you'd have a topic and then you get to see different sides of the argument. And I think Cathy there is a website called all sides, something that will show articles on the same topic from more of a right wing perspective, a centrist perspective and a left wing kind of perspective. And that would be really helpful to counter speakers and and then lists some different suggestions on how to approach that topic. Whether it's by using humor or sarcasm or questioning things, some emotional based kind of questions to engage with some of the posters. So that was kind of discussed and circling back. How can we be assured of what AI is learning in terms of how it will help us right like Martin was saying, it's kind of tricky to sift through all that information. The second thing or the third thing, because I have now lost count is how will AI recognize nuance, sarcasm micro aggressions or even the use of emojis and we see that in terms of hate speech, we see a lot of emojis that look innocent, but really, it's kind of promoting a hateful or bigoted kind. of belief to it. And how will ai ai be able to identify dangerous speech, which incites violence, but can appear less threatening, but it's more insidious, which I think are almost exact words that Susan uses in some of her writing. Hate speech is easy for AI probably to identify but dangerous speech. It's just so much more subtle. So my leadership team are is mostly full of techno skeptics, I guess. And so they really worry about that slippery slope. They worry about how AI might appear human but not come from a human perspective. And honestly, we can talk about this and debated and discussed and the ethics and the abilities to use AI and counter speaking but literally as we speak, millions of dollars are being spent to spread disinformation and rhetoric and fear and it's the marginalized groups that are feeling the impact of that. So I think there are tools that need to be used in the sphere of counter speaking and because talking about it isn't stopping the harm and it's not stopping the policies from being passed. We're just sort of prolonging it. Personally speaking, I kind of a tech optimist and as some of you know, it was recently May the fourth which is Star Wars Day. I'm not as big a Star Wars fan. I'm kind of a Star Trek fan. But I guess we're kind of looking to use AI AI technology a little bit differently. We're not using. We're not wanting to use it to create that propaganda Deathstar that's already existing. We're just kind of asking for better shields and maneuverability over X Wing fighter jets. I don't know that sounds Star Wars again, Star Trek fan, so not really sure. But the cheerleader bot would be excellent. It falls right in line with what we do. It amplifies the positive, it would be super helpful. Again, the issue would be transparency, right if we're using bots you know, the feeling is that people need to know about it. So that I don't know Kathy if that answered your question that kind of went all over the place
No, no, that's that's a great place to to land there. I think that we're kind of landing in this place of but there are a lot of a lot of questions out there and need to think about this Martin What about for from your perspective, what are the things that you would like to know more about? I know, you mentioned effectiveness too. How are we going to measure effectiveness? What else is on your mind?
Yeah, I'm just still digesting what what Alena said. I think the the techno skeptics definitely. Like it seems it makes sense to sort of ask like, how good is AI really at some of these things? And the question there is very open yet because like there's a lot of challenges to detecting, like dog whistle emojis, for example, or things like that. And, yeah, I think I'm starting to I'm starting to do some research on like, sort of fine grained hate speech detection and realizing like this idea of gloving it all into one concept to my kind of work in our internal definitions of like, oh, this is all toxic or something like that, but like really the issue is like, we need to go much deeper into like, what is the specific thing that this is like, embodying like is it like representing a very bad stereotype? Or is it sort of like you know, provisioning this particular kind of information and then doing that moderation at that level seems to be much more effective for AI systems. Then kind of asking you to like to take on the broad definition of should just be moderated or not. So that's kind of what I'm thinking but um, yeah, I think the the idea of having AI involved in this like sort of combating hate speech detection and all that stuff needs to be really carefully thought out and this we need Yeah, we need to be very careful about what we believe. We want these platforms to allow basically I was reading the chat today like this idea of like, AI helping people craft messages before they post send like my idea that I've always had for the worst social media in the world is like, you don't you get to write a message and then you have to like wait five hours, and then you have to go and rewrite that message again. And like, sort of, presumably, people will have calmed down if they're angry by that point. But that was just like, have the worst engagement ever and so no advertisers with everyone advertises. Um, yeah, but I think yeah, my questions for y'all specifically is kind of like how do we know if counter speaking is working? And yeah, how do we have once we have that then we can we can help a lot more. Yeah, with AI? Absolutely.
It seems like in my mind, I often think of but they're kind of questions in the what can the tools do? Like can they do this? And should they do this? Right? We have two larger sets of questions to be thinking about right now. But I would like to thank you both for that engaging discussion. And then turn it over to Dave Willner for some, some closing reflection on what he's just heard. And then we'll move to question and answer. So if you have questions that you've been thinking of during this conversation, if you could put them in the chat, we will start getting to those after we hear from Dave, thank you both.
Thanks both Elena and Martin. That was really, really great i on the sort of content moderation or how good are bots at this question? It? It seems to me the question is always compared to what? Right and in a lot of ways that ends up becoming a question of what scale are we trying to operate at? Right? If we're talking about individual conversations or small group activity by folks like Elena like, yeah, chatbots aren't going to compete. If we're talking about industrial scale moderation. It's absolutely true that a lot of our AI technology missed the nuance of emojis, but so does our current industrial scale moderation. That's part of why it doesn't work very well. And so which without even mentioning, as has been talked about in the chat, all of the suffering involved in the sort of running of those systems and so compared to what is always that sort of the sort of background that I think about, it was really interesting to hear in particular about the the specific experiences you had had Elena, trying to get into actually figuring out how to operationalize Counterspeech I'm also glad to point folks to some of the resources Justin provided earlier. If you're interested in sort of the current state of play about how well we've gotten the systems to operate. The short version is pretty well but not well enough yet. That said I think progress is happening pretty rapidly. And I personally actually share Elena's optimism about using these technologies for good as well as ill. If we can figure out how to use them. The problem is just figuring out how to use them for good is harder than figuring out how to use them for you. That's that's sort of what I took away from all of it.
Thank you. Great. So we have some questions that have started to come in and so I will read them as I get them and get some good responses. So the first question comes from Karen Middleton. With the increased regulation that's happening now. Eg digital Safety Act Online Safety Act. Have you looked at the possibilities for platforms to be compelled to develop and deploy AI tools to counter hate speech and misinformation? For example, all public content being reviewed before it's published?
Anyone have thoughts on that?
I'm glad to jump in if others don't. Yeah, um, so on mainstream platforms. Today, it is largely already the case that scanning is done overall content. That doesn't mean human review, but there it is generally the case that people are running various classifiers to try to weed out problematic content. Those have error rates, particularly at the very very high degree of scale that we're talking about. Prior human review of all content, I don't It seems hard to scale. Although I do think that as we get the sort of language moderation tools better dialed in. We may approach something that feels a lot more like that, in terms of level of nuance, and potentially even the level of explanation of the review that we're able to provide where we can today.
Next question, what are the main ways that Counterspeech can be amplified and promoted by algorithms? Ai? I know redirection is one of the more formal approaches, but what are more informally amplifying? Or what about more informally amplifying Counterspeech That Elena was talking about? Elena, do you want to take this one start?
I can attempt to take this one for sure. And also let me know if I've completely missed the mark on this answer. When we try to amplify we do, we put out posts asking for interactions we you know, Kathy, I'm not even sure if I'm gonna be able to answer this properly. But I guess informally, we appeal to our members. We spread it onto our social media that we do need help with this type of comment section. And then everybody's invited to boost it using replies and emojis and things like that. I don't know. We try to circumvent different algorithms we try to when we put up posts, we try to cross out or blackout certain words that maybe are trigger negative feedback from algorithms where maybe it's not boosted as much because it's using speech or language or words that aren't really as accepted. So we try to do that. We've tried putting links to different posts and things in the comments section instead of in the main posts, so that it gets seen by more people. We invite people to even just boost that particular post so it spreads to more members. We've really been struggling, I think, and trying to get a bigger reach and more engagement, and we aren't exactly sure how to get around that and we're also sometimes worried that by engaging with certain posts where there's a lot of hate speech and rhetoric that we're that might be a couple of days old that we're actually inviting more people and more backlash, and that edit amplifies the hate. So there's a lot of things that we try to take into consideration when we're doing it. And it's not always efficient, and we're not quite always sure how to do it effectively. And I have no idea if I answered that question appropriately.
Yeah, no, I think that that's a that's a good start. And when I think about this question, so I have kind of less information about how you would use AI to do this, but I know the research is pretty, pretty strong around the idea that Counterspeech kind of breeds more Counterspeech Right? It's a little bit mixed on is Counterspeech effective at changing people's minds or or especially if it's the person who had originally posted the hateful content, but there's pretty strong evidence that when there is Counterspeech, there is more Counterspeech After that first Counterspeech Right. So if we think about it in terms of just kind of discourse norms and the contagion effect of kind of how do people encourage more speech, like the kind of civil fact based speech that they are interjecting it becomes much easier to kind of be the second Oh, sorry, the second or third voice that enters that conversation, rather than the first one. So I don't know if they're kind of lessons from that that can be learned when we think about how to do this at scale, but just some thoughts there, but move on to the next question. So the next question is, does anyone have experience with prospective AI outside of using it for moderation on key websites, ie deploying it? Quote in the wild to see if people actually appreciate it in everyday interactions? I'm curious about methodologies at scale that can invite people to stop calm down and reflect before they post without outright censoring speech. Anyone have information about that, that they'd like to share? Know anything, Eric, someone else? Sorry.
We did a little work. It wasn't with AI because this was like 14 years ago, 13 years ago, but we did a little bit of work around this around specifically contact resolution or conflict. Resolution in the context of people reporting speech about themselves. They found speech or images about themselves, they found unflattering, but they didn't rise to the level of violating our rules. So it's not exactly one for one. And it was pretty effective. Again, though, maybe a less emotional and politicized set of subjects. It was pretty effective and people were okay with it. But the way to do it well was very culturally specific. So the interventions that worked well in the US did not work super well, in like Eastern Europe, which seems like a challenge if you're looking at sort of open world deployment where you don't know who your audience is.
Just have we tried to collect some examples of of papers that get it this question a bit in the syllabus document that I put there. I just put one example was plotted Glen's comment on Well, el Baldwin's post there, but but just to, I guess, somewhat echo Dave there, it just strikes me that it's gonna be so complicated, you know, in what contexts or work what does it where does it not Where, where are there cultural variables that play or political or other media, environmental variables, you know, this can be so many questions that have to get asked.
Yeah, thank you for bringing up that point. I know it's something that we maybe touched on a little bit throughout this conversation, but it's certainly something that came up in our interviews with Counterspeech acres we did include in our sample people from various parts of the world and there was real concern and and probably, you know, rightful skepticism and concern from our counter speakers coming out of Ethiopia and Cameroon, about what what kind of a deployment of tools like this would mean for them, and how, how they would translate across different cultural contexts and different language contexts and, and real concerns there. So I'm glad that you, you kind of have both brought that up. Another question? So Could one of the relevant factors in designing this tech indeed be that the attention economy has created the appetite for more and more extreme and sensationalist content to drive engagement? And this one of this is one of the cornerstones that we should be trying to seek out and there's some agreement in that with that question in the chat, but even have thoughts there
I'm glad to jump in if nobody else. A I don't remember where I ran into this, but it was a very sort of interesting take around the fragmentation of news from a small number of broadcast channels into a much larger number of much more specific cable channels. And we saw along with that sort of a change in the way people optimized their channels. So instead of being kind of bland, and boring and palatable to everybody, because there were only three channels and you were only competing with two other people for slightly more than 33% of the market, you are instead the sort of way to profit in cable news is getting like flat 5 million people to really really love the thing you are looking at. And so at least to me, that is points in a slightly different direction where I don't think the desire for the extreme content is new. Right? If you go back and reflect on yellow journalism or other similar things, it feels like that's been in us the whole time, but the economic models that make that profitable, might be not to do change, right, and can have influence on what the discourse looks like. And so I think yes, but I think it's less about how it's changed us and more about how our demands encounter the marketplace.
Yes, thank you. So I see that we are nearing the end of the hour after what's been a very, very engaging conversation, at least from from my perspective. So I'm going to pass it to Susan, for the the last word.
A very quick last word. Thank you. All so much for joining us. We hope to continue this conversation and definitely to continue this work. In in in careful ways, though, as as Martin and Justin and others pointed out, it certainly feels not only relevant but pretty urgent. a through line and all of our discussion over the over the last hour has been this question of of what Counterspeech And for that matter moderation are actually accomplishing? What effect are they having on people? It's, of course, terrifically important to study that. You need access to data for for such studies, but but there are many, many great possibilities for study that haven't been explored. And then in the meantime, I'd like to to draw attention to a very important feature of Kathy's work. She conducted the first ethnographic study of a large group of counter speakers. There are, as she noted, lots of people who volunteer their spare time to respond to people they passionately disagree with online this is a very much overlooked fact. And a gigantic mine of information itself on how people communicate and I wanted to point out one of her findings, or one one sort of collection of findings has to do with another thing that's often overlooked, which is the impact of Counterspeech on the counter speakers themselves. They continue doing it many of them for years and years and years you you may have noticed that Elena herself has been at it for at least five years. So have many other counter speakers like her so why do they keep doing this thing which is unpaid and often pretty dispiriting. Apparently, there are also some, some benefits including a sense of agency of engaging with other people in a society of trying to convince other people of things that really matter. Whether or not you know what impact you're having on other people, such an effort matters to, to, to the person who's doing apparently so so that's an effect that, that we know about. And that is important not to forget. And then And then I'd like to just once again, thank you for coming and to encourage you to stay in touch with us. We will we will keep going forward and perhaps convene other discussions like this on this topic and other work that we're doing at the dangerous beach project.
Thank you, everyone. And thank you for those that participated in the panel. We really appreciate your time by everyone. Thank you
