A New Techno-Communication Style (and Meta Media)

2:54AM Jul 27, 2020

Speakers:

Keywords:

climate change

claims

people

arguments

debate

questions

conversation

media

reasoning

argumentation

happening

global warming

imply

fact

communication

information

create

ideas

evidence

impact

Welcome back to hope 2020 everyone everywhere in the world. This is going to be the last temptation that we have this evening but afterwards. Don't turn away, we are going to have a performance, and a performance of original music by, hang on, get ready. It's going to be a blast. And I look forward to seeing it But first, Let me introduce our next speaker and presenter Jamie Joyce, the founder and executive director of the society library, a nonprofit organization that mines arguments claims and evidence for various forms of media to create a library database of a given society's ideas, social media and infocomm technologies have enabled communication capabilities to scale, however society has failed to get on the same page and is arguably more polarized than before. I hope you all enjoyed the presentation. And afterwards, QA.

Hello, my name is Jamie Joyce I'm the executive director of the society library. Society library is a nonprofit organization that extracts arguments claims and evidence from various sources of media in order to construct databases which has argumentation from all points of view on complex social and political issues. Essentially we analyze and extract arguments from conversations all over the web and all over the world in order to simulate what an informed inclusive conversation at scale about some complex issue could look like. Today I'm going to talk about our work analyzing American media for the past two years, and the new kinds of media I think we need to create in order to meet the needs of the public in the information age. So let's talk about media communication and technology, and I'd love to use the example topic of climate change, which we've been analyzing over the past two years. So climate change has been a persistently polarizing and partisan issue, both within the United States and concerning the United States on the international stage. It's also a topic of debate that some entities have claimed to have been exploited for disinformation campaigns by both domestic private interests and international groups. First let's just quickly talk about how it is domestically polarized and partisan in 2020 Pew Research circulated a survey asking what subjects should be prioritized by Congress and the President. And from this survey climate change was the subject with the largest gap in terms of priority between American Democrats and Republicans. Now Pew has been conducting similar surveys to this by annually since 2008. And even though there has been a large increase in the Express sentiment of democrats that climate change should be a priority, there's only been a modest increase in the same sentiment for Republicans. so here we can see that there is a persistent partisan gap in terms of how these parties view climate change or global warming as a priority. It seems as though they disagree about it greatly. Now obviously what people think about climate change is more nuanced than whether or not they feel it should be the highest priority for Congress and the President, however I want to talk about another interesting breakdown of American attitudes towards climate change. This right here is a framework and some data that's been pulled by an ongoing Yale program called climate change communication, they often collaborate with other entities such as the George Mason University Center for climate change communication. In order to conduct their surveys and produce this data. What they've done is break down American attitudes characteristics and behaviors related to climate change into six types. They call them the six Americas. These groups are referred to as the alarmed, the concerned. The cautious. The disengaged. The doubtful, and the dismissive. The group that is described as alarmed are certain that global warming is real, that it is human caused and that it is solvable. The concerned, quote, hosts similar beliefs and values to the alarmed but they hold these views, less strongly, they're less worried, less engaged, and less motivated to personal action and quote. The cautious and disengaged quote are two segments of the US public that think and care about global warming the least. The disengaged have the lowest trust in science and are most likely to believe that the Bible creation story literally happened, according to their data, quote, doubtful are evenly split among those who think global warming is happening, those who think it isn't and those who don't know, many within this group believe that global warming is happening. It is caused by natural changes in the environment, believe global warming won't harm people for many decades into the future if at all, and say that America is already doing enough to respond to the threat and put the dismissive believe that global warming is not real, and it.

Over time, the number of Americans broken up into these six groups change according to this survey that was distributed in November of 2019, the group of people considered alarm has nearly tripled in size since October 2014. However, what's happening with the American public and with American politics can be very different. Even though it seems like more and more Americans may be growing convinced and alarmed about climate change. At the same time on the international stage, the United States may not appear that way. Since the US alone is pulling out of the Paris Agreement, unless I'm mistaken officially come November of 2020 regardless of who is elected the next president the united states of america will be considered the only nation that has not signed on to this agreement. Since the decision was made by President Trump A few years ago, the remaining countries that were initially left out of the agreement have been signed on, I think it's fair to say that Americans are divided amongst themselves about climate change. And we're at odds with a lot of the world's about climate change and there are claims that this division was perhaps both caused by mis or disinformation campaigns and has also positioned us to be vulnerable to disinformation campaigns, although people who are alarmed about climate change and people who think climate change is a hoax. Talk about missing disinformation campaigns coming from the side opposite of their belief. There's also supposedly international actors who are playing both sides. These are some means from a report by the House Science space and technology Committee, which details how the Russian IRA, the internet research agency. Spend some time exploiting the partisan divide on climate change by sharing memes that appeal to conflicting sides of the climate change and renewable energy debates, if I'm not mistaken, the report claimed that this was in an effort to disrupt consensus on energy policy decisions. And of course, since polarization is a deathblow to democracy which operates by consensus. If you just want to pick up the efficiency of any democracy sowing discord and division may be a useful strategy. So all in all it seems as though climate change is an important subject for the American public and for American politics. And it may even be viewed as an American pain point, in and of itself, it is seen as a high impact subject. It's persistent and it's partisan and emotional as well as scientific and technical. However, oftentimes it's not all of those things all at once. Let's talk about climate change media and climate change communication. Americans are having all sorts of conversations about climate change, and they're having them through various forms of media. In a way it's almost like we're having a really disorganized asynchronous mesh societal communication was coming from all over the place. communications are being spilled out all over the public consciousness through documentaries about climate change. Television debates and interviews that are conducted about climate change, books that are published classes that are taught news pundits that rant on YouTube, and plenty of Facebook feuds perhaps some Twitter bots un summits, and massive worldwide school walkouts with messages being chanted in the streets and spelled out on signs.

It's like there's a collective conversation that's happening about climate change that people's points are being expressed all over the place, through various sorts of media, and it's not really coordinated nor constructive, in many cases, but what's amazing is that from our laptops we have access to a lot of this communication that's happening. We can look up some protest speeches we can watch some un meetings we can read the text of the green New Deal. But although this technology has enabled this messaging to be more accessible than ever. Like I said, it seems to be disorganized asynchronous communication. People are just expressing their arguments however they can. So it's all happening through different forms of media at different times to different audiences. It's not like a collective conversation was coordinated where all parties were willing to contribute their arguments and add fear to structured debate in the pursuit of truth collectively. So the nonprofit I work for the society library set out to simulate what a coordinated societal scale conversation between all parties could actually look like. We wanted to identify all the points of view and collect and articulate the reasoning and arguments from all sides. We call the project the gray American debate. And it's a kind of epistemological project, meaning we were modeling the logic and arguments being put forth through all this media that we were analyzing initially sheer spirit of inquiry after truth. I know there are literally hundreds of projects that are trying to facilitate the ability for humans to debate, build knowledge collectively and ask questions online. And from what I've seen a lot of them are most of them are doing this by hosting debate platforms or crowdsourcing information but I just want to be clear that that's not what we're doing. Instead of hosting or trying to generate conversations or debates between people on a platform. Our goal is to find and collect all the conversations and debates that are already happening, or have already happened between people and combine them to create a more comprehensive debate that's structured as a true collective inquiry. By deconstructing the logic of our collective public argumentation and documenting not only the arguments claims and evidence but also the means videos cultural assets. We could essentially create a structured database of the collective climate change communication, that's happening all across America. Now I'd actually like to get to talking a little bit about our processes and about our tech. So, to effectively combine all of this content, we first had to break down each piece of content. If we just combine the debates as they were occurring online, it would be wrought with incomplete reasoning and tons of logical fallacies. So if we wanted robust and formal argumentation from all points of view, we would have to develop a method for deconstructing the logic of narratives, so we could combine the arguments claims and evidence in a meaningful way. So step one collect content step two extract all that content. And then we would fill in a full bunch of steps in between modeling the logic of that content. Let's start with the process. When we first began this process we just started collecting any information on climate change. We collected documentaries podcasts books news articles scholarly articles social media posts pieces of legislation and website copy. We would then transcribe that content into text and parse it out sentence by sentence. So we prepared all of this material in a very standardized way. And then when we started actually extracting the claims and arguments, we began discovering how unbelievably and surprisingly complex dense and incomplete, a lot of our speeches.

When we analyze television or any unscripted audio, and sometimes a lot of social media too. It's almost like analyzing a completely different language. So much of what people say is nonsensical or incomplete. And it kind of amazes me that humans can infer so much meaningful content from really unstructured sentiments. However, we made it our job to pull out the logical reasoning and create self contained and complete claims and arguments from what people are saying, but we also have to take responsibility for when we the analysts have to infer meaning. So we develop standards for interpreting the implied drive and implicit claims of natural language text, which means we need to identify the extent to which our input shapes our understanding about what is being said. Sometimes that's easy, the written word is often much more complete in terms of reasoning because the act of writing and the ability of being able to edit before publishing is a system of refinement in thinking through and articulating a point but like I said sometimes it is not easy. So here's an example of how it can be really easy to pull claims from statements. As a statement we pulled from a television segment about the green New Deal. And these are the claims that we pulled from that statement. They're pretty simple claims that we can derive and make self contained and complete. They will then be categorized back checked and turned into questions to help direct further research. This also shows why it's important to break up language, because we would actually need to fact check each one of these claims and ultimately these claims may individually be relevant to other parts of the climate change debate. Well the whole statement may not be relevant everywhere. However, identifying or parsing out the claims of some statements can be really difficult. For example, we scraped a snippet from social media that said, global warming isn't happening because of Venus. And that just kind of blows your mind because you. I just can't imagine how that makes sense like, what, what is Venus doing, how, what, what has Venus caused me global warming other thing. I don't know this for us wasn't so much a claim, but a clue. Someone had seen or read or heard something perhaps they've heard some kind of argument about Venus or global warming or something. And they articulated what they thought they knew in this way. And since we want to collect all arguments related to climate change, which include arguments about global warming to include in this collective conversation we're simulating, we have to find out what this is referring to. Using the keywords global warming isn't happening in Venus, we can start doing a little research, and these are few possibilities that we found. It could be that this person is remembered or misheard an argument about how warming of Mars and Pluto is being used as evidence to suggest that Earth warming and warming across the solar system was actually due to solar activity. They may have just substituted Venus for Mars or Pluto, or perhaps they were referring to the idea that sulfur dioxide turns into sulfuric acid in Venus's atmosphere and supposedly reflects solar radiation and cools Venus a surface. This has supposedly inspired some geoengineering technique that was suggested for Earth's atmosphere to curb global warming. Maybe this is what they were referring to, maybe not. We can't really know, often on unscripted television and on social media people say things that are incomplete. All we can really do is ask them to clarify, ask questions about our own possible interpretations, do more research and understand that sometimes the game of telephone deeply distorts claims online, sometimes people just don't have the working memory or eloquence to articulate arguments just as they've originally heard them. Sometimes we're just not able to understand what someone is saying and sometimes people may be expressing ideas that they really don't understand so they don't know how logical OR thorough their ideas and explanations are. So all we can do is log the natural language snippet and identify what we may assess it could be referring to. So if anyone ever searches the database using that query, we can offer what we think they may have been looking for. As you can imagine, this may be labor intensive to try and make sense of all of the nonsensical stuff on the internet. But lucky for those society library following these quirky claims is fun, and trying to fill in the gaps of reasoning in these statements leads us to find all sorts of new nice interesting ideas that perhaps we may not have found otherwise. And so we've just done that process over and over and over again we've collected media we've transcribed it we've parsed it by sentence and extracted the claims and arguments line by line when we could. We did more research when we needed to and did that process again, again, again, and we just wanted to see what people were talking about related to climate change and how they were making their arguments.

And so over time we started noticing patterns of argumentation emerge. And those patterns built upon other patterns, and ultimately a structure of conversation actually started to emerge. We discovered that the topic of climate change for Americans was actually composed of at least 220 sub topics of debate. Each subtopic implied at least two positions and likely hundreds or thousands of arguments and pieces of evidence, each what's most interesting is that we also discovered that these 220 subtopics fit under six categories. However, these weren't the six American archetypes identified by the Yale group. These were six questions, which we now call the fundamental questions, under which the 220 subtopics of debate that those questions are, What is climate change is climate change happening. What causes climate change. What is the impact of climate change. What could or should we do about climate change. And why has the climate change debate persisted for so long. The first question, what is climate change is a question about sorting out terms. What is the relationship between climate change and global warming and global cooling, for example, it's a question that request clarification about the implications of the term climate change. Do we mean it is inherently catastrophic abrupt and imminent. Is it capital see climate change or is there a matter of fact way of explaining climate change in general is climate change to some just a completely nonsensical term because it's the name of some scam or hoax before building a conversation using the term climate change, we need to be clear about the immediate implications people bring to the table when they hear the term. So we want to get all definitions and connotations and articulate them in one place. Based on our understanding of how people have used the term. The second question is climate change happening. This is a question which implies a variety of positions. Yes No, impossible to know we don't know and it depends on the definition. So we can explore the reasons that are provided to support each position. This includes discussing observational phenomena historical records data from various sources and tools as well as criticisms about the reliability of those tools and records and questions about the reliability of institutions responsible for processing and presenting that data. The third question, what causes climate change. Although there are a number of causes offered this debate precisely encapsulate arguments to support which prominent causes are implying the present changes in the climate, and to what degree various causes are having an impact this debate goes into the details of cause and effect. It critically hinges on time, and the methods for how we isolate prominent causes are also explored. Fourth question, what is the impact of climate change. This question entails an extensive extensive account of claims being made about the impact of climate changing on various aspects of human civilization of the economy national security human health and well being, impact on various climate systems biological processes and impact on other life forms on Earth, fifth question is, what could or should we do about it. So much of this debate builds upon the reasoning of former questions implications about the domain of impact and severity of impact imply priorities about who should be taking what actions when and what goals need to be met and why this debate contains conversations about the feasibility impacts cost and convenience of over 100 solutions and actions. And it also contains appeals for inaction. The sixth question was the last to emerge for us. It asks, Why has the climate debate persisted for so long. It's a conversation about the existence and impact of disinformation campaigns polarization lack of trust and scientific literacy oversimplifying ideas. Logical Fallacies doubt as a product on the public mind and in political discourse. And of course these arguments come from all sides.

Like I said, these six questions, collectively contain 220 sub topics of debate.

And these questions, interestingly imply a hierarchical structure. The structure starts with a question, followed by positions layers of categories and sub categories of arguments and the actual arguments claims and evidence arguments without evidence, and then eventually references and original source materials. The bottom of many of these debates will likely be some philosophical discussions about science and physics based explanations on phenomena. Besides having an internal hierarchical structure. We also found that there may be a natural order to the questions themselves. When we order the questions in this particular way. It has interestingly kept a certain kind of structure to the debates and kept it from exploding in all directions. For example, in the question what is climate change, there is a definition that implies that climate change is a hoax, that premise will be relied on by other positions in different questions, but because the reasoning was established in your previous question, the following question can just build on what has already been established. So, relying on the premise in later questions doesn't mean we have to consistently and redundantly bring up the same points again and again, although in our software we can link claims from one debate to another. Now I just want to make sure this is clear. The society library set out to create a constructive coordinated conversation about climate change. We thought that in order to do that effectively, we should start standardizing and breaking down statements to create self contained claims, so that we could later combine them to create more robust argumentation and have the opportunity to deeply fact check these claims, find counterclaims all as a part of our process. When we started breaking down claims, that's when we started noticing these patterns emerge among this collection for climate change solutions for example we saw patterns about cost convenience, positive impact negative externalities feasibility of implementation and the priority of the solutions, finding these patterns did help us standardize protocols for the scope of information we should be looking for about other solutions. However, as these patterns emerged, and they point to certain questions that also helped create a certain order by determining containers for the relevance of claims. For example, let's take the second question. There is a pattern of argumentation where people make claims about how their frequency, intensity and severity of 26 different kinds of anomalous and extreme weather events is proof that the climate is changing. However, in the fourth question people also cite that the increased frequency, intensity and severity of the 26 kinds of anomalous and extreme weather events is also a consequence of the climate changing the reasoning that makes these same 26 kinds of anomalous and extreme weather events relevant to the questions being asked implies the container for relevance.

Although all of these subtopics relate to the anomalous and extreme weather events, the questions imply which of these subtopics is relevant at certain points in the conversation. If the impact of extreme weather events, was brought up by someone who is using extreme weather events as an example of science of climate change, it could derail the conversation from being pursued point by point, his impact was being discussed. Although the impact of extreme weather events is relevant to the climate change conversation as a whole, arguments about the impact of it may not be relevant to a discussion about whether or not climate change can be proved measured reasoned or observed to be happening through observations of these extreme weather events. So this technique that we developed to pull claims and arguments from snippets of natural language, line by line believe this these few useful insights. One is that when we break down what people are saying to its basic logical units, there may actually be an underlying structure that we can find which encompasses and accounts for the whole of claims and arguments, while also implying a certain order and arrangement to the conversation. Another insight that's gained from doing any work with us is really realizing how unbelievably dense and complex language really is and how amazing it is that human beings are able to fill in the blanks of meaning when they're communicating, even when people aren't fully articulating their ideas. We've learned just so much about logic and language and we're really excited to learn some more. For now, we have our 220 subtopic debate structure and most likely we will find more subtopics or patterns which will either diverged or combined as we go. We have paused on the climate change debate mapping so we can work on COVID subjects, however we want to continue fleshing out the arguments within each subtopic of debate about climate change, so we can fully articulate this American debate about climate change. This means chasing every conspiratorial rabbit hole, exploring every nonsensical claim to interrogate it for a possible truth and structuring this content in a database using our debate mapping software. It has incredible features which allows us to write different versions of claims based on reading level and how technical the languages. So people have different levels of familiarity with the subject can opt into different versions of the interface. It allows us to embed media graphs equations, it's wonderful for capturing information we're extracting and modeling it in the database so it can later be visualized in various ways. But why should we do this why would anyone want to do this. Well, when we wrote to our contact at one of the climate change communications groups to explain our work we did get a very helpful response from them, but it also included this side note, cook. One quick suggestion is the overall framing related to some of the questions. It's critical to distinguish what is a true versus foe debate. There are real debates to be had about solutions. There is no legitimate debate. However, over the basics that global warming is happening and human caused even framing it that way does real harm as it suggests there are two legitimate sides. I'm assuming you win sponsor debate over whether the earth is round or whether smoking is harmful to human health. Yet scientists are actually more certain that global warming is happening and due to human activities, and they are that smoking causes cancer in quote. For us, this person's claims about there being a true versus foe debate, or that framing the debate has a particular impact actually falls under the sixth question of why has this debate persisted for so long, which is a sort of meta debate about framing and fallacies in media disinformation science communication, etc. The statements made that scientists are actually more certain. They are that smoking causes cancer is actually broken up into two claims that fall under the question about whether climate change, although they specifically refer to global warming in this instance is happening, and the question of what causes it. Anyway, I believe I understand this person is concerned and I respect their point of view and the work that they do. But again, our questions emerge from the content we were examining, we didn't just prescribe or pose them. These questions exists because people have them, or people have conflicting answers to them. Many of you may be familiar with various debate platforms where moderators pose questions and collect answers from users. However, our debate project is about collectively modeling the debate that's already happening, looking for answers to questions that people already have and showing the counter arguments to claims that are already being made, not only online, but in books on television and in all forms of media we can manage to deconstruct developing our framework for analysis is just to make sure we are as exhaustively comprehensive as you possibly can be.

And I believe this is important because I believe it's a step in the right direction for how we need to evolve our media ecosystem. Social media has scaled our ability to communicate as a nation, but it has not really enabled a simultaneous comprehensive and inclusive conversation. The existing way we debate and these conversations is just inherently limited debates we have now are limited in participation in time and the amount of information that's present and exchanged. For example, let's say we're watching a debate on TV, it's just a few people, and most likely they're just taking turns making points, maybe prioritizing being more punchy than being precise, or actually engaging in the arguments their opponents are actually saying it's a broadcast communication. So it's a one way communication and the audience can't really ask questions or argue or engage documentaries, videos, podcasts debates on TV. I mean many of these types of media, even if it includes multiple people is designed to just be broadcasted and consumed by the public on platforms where the public can actually engage like on a Reddit thread or on social media, through my experience, even if people use the terms claim and phrases like demand evidence burden of proof are used, it doesn't appear to me that many people can deeply appreciate what it means to develop logical proofs and establish reasoning to justify conclusions of arguments. I found that debates online are often very disorderly and filled with lots of fallacious reasoning. The tools that are supposed to help us keep our public conversations in check, like fact checkers are also sometimes subject to logical fallacies themselves fact checkers may confirm or refute the fact that many of these platforms may neglect the context of the fact, in terms of depth or breadth of the point to which the fact relates, and therefore these entities, oftentimes may be effectively cherry picking and cherry picking is a logical fallacy. Now with new tools and techniques like the work we're doing modeling debate about climate change, we can take more of a whole system's approach to fact checking and change what it fundamentally means to have a conversation about something as a nation through media analysis claiming argument mining and then organizing the content by its emergent structure in a knowledge database that's modeled as a conversation. Starting from the fundamental questions about terms, all the way through to solutions and a meta discussion about the discussion itself. I believe we can begin to create a new way to explore collective ideas about the six Americas and about the many Americas on many different issues, and really start to work on collective nationwide conversation and develop new media and new visualizations that will help us see and understand the bigger picture while empowering us with tools that can help us walk through the details and down to the datum of every debate. I think the alternative is that we'll continue to see private interests and billionaires and foreign governments and NGOs on all sides just persist in creating all sorts of appeals and marketing ploys and even psyops techniques to convince or confuse the public mind through various forms of media. I'll be the first to say that I don't believe epistemological logic will change, American politics. I do believe that the world changes through storytelling and demonstrations and actions. However, I do believe the only intellectually honest way of trying to inspire and create change without steamrolling the public is to ground those stories in logic and evidence after truly inquiring into how what is known and decided is known and decided. And in response to that statement maybe there are just so many groups out there that are beating their heads against the wall saying that they have inquired and they have logically articulated it. So maybe then it is both very useful and very helpful for a nonpartisan entity like a library to dutifully collect and articulate those arguments and the arguments on all sides to make available for everyone, while also making available immediate reference library of the content from which all of that argumentation came. I think it's important to understand that different groups are at different stages of any given conversation. And by mapping out these conversations perhaps they don't need to percolate so slowly person by person, or so irresponsibly through massive campaigns that will enable this confusion and conflict to persist for decades in the digital and information age.

So thank you so much for listening to what we've learned about logic and language in the past two years. By exploring the climate change debate in the United States. If you would like to be among the first to see the map once it's more fully fleshed out, as well as see our other programs, about COVID-19 and other topics, then please go to the following program websites to learn more. The society library is powered by an amazing group of volunteers. So if you would like to become a trained analyst or you'd like to contribute to our machine learning or AI projects that assist in our analysis work, then please feel free to reach out.

The society library

is a nonprofit organization so we greatly appreciate any donations of time or treasure. And now I will take your questions and I asked that it be specifically about our work or what was described in this talk, because I cannot promise I will have an opinion on your favorite communication theory or logical paradigm. Thank you.

And we're back with Jamie Joyce.

Again, I'm gonna ask questions. Now, some of which came from the, from the element chat, again you can only ask questions of the presenters at hope 2020 by being an attendee. So, do you think that automated information filtering like Twitter's factchecking or Google's Knowledge Graph improves literacy or information scraping for folks.

Thank you. Um, so there's a couple of different things to that question. I think that perhaps Google's knowledge graph is a bit different than Twitter deploying factchecking on their platform. And as it relates to information literacy I think knowledge graphing may produce improvements to information literacy and that we are empowered with different tools in order to explore the full breadth of information and how it relates. However, while I think that fact checking is an honorable and noble solution that's applied to these large social media platforms in order to try to curb the issues of information disorder. I do not think that that improves information literacy. And I'm not sure if there is a solution to actually improve information literacy on these social media platforms. If there are you know little checkmarks or little symbols that appear next to certain news articles. I'm not sure that doesn't inspire a backfire effect with people who automatically agree with the premise of the headline. So I think that that is a. That is an observation that may be more rigorously tested through some additional research but I'm concerned that that may just cause it backfire and not necessarily improve information literacy.

Thinking of fact checking how did you fact check everything you scraped on this topic I mean it sounds like an immense amount of data.

It is an immense amount of data. So before we get to any of the fact checking what we actually do is we categorize claims. So we cluster them by their relation to topics. And so we reduce a lot of the redundancy of fact checking once we combine all of those together we want to keep every natural language snippet associated with a claim ID. So then we just end up fact checking the claim ID and not necessarily every single iteration that we've ever come across. So there's a little bit of pre processing before we get to fact checking which lessens the work but we keep on grinding but we will need like a whole building full of researchers to really do the work right and complete.

Now,

how do you handle disagreements on definitions because if we aren't definitions in common, it becomes difficult to argue, or if we're talking about accidental disagreements on definitions or deliberate differences. Yeah, so I think

that there are both accidental. You know disagreements meaning like perhaps misunderstandings about what someone means and how someone expresses a specific definition then of course there is, you know, in an intentional use of a term, but we're not the ones who make a call on what is the correct or incorrect definition, what we do is take these terms and scrape as many definitions that are provided. And then we do flag when a definition is not in agreement with the rest of a snippet. And so that's just a way of adding a particular badge to it saying this may be a misuse of a definition, but we're not the ones to say what definition is correct or incorrect. What we're trying to do is just understand where people are coming from and what they may mean because that's how they build upon their reasoning and argumentation which justifies arguments about more complex issues about climate change like arguing about solutions or things like that.

Now when you mentioned backfiring were you talking specifically about how there's a tendency for people to double down on beliefs in the presence of contrary information. Yes.

That is what I was referring to, people may dig in their heels, when someone tells them that they're wrong or they're incorrect or their sources terrible, I think, by and large, humans don't like to be told they're wrong or dumb or stupid. So, instead of, you know, being open and vulnerable to the idea that oh maybe I posted something that's actually not a great source or just make sense or it's an outright lie, just doubling down and maybe evading the conversation and attacking from a different angle is a useful debate technique. And you know, it keeps people from the embarrassment of like oh maybe I did something wrong.

Now a couple of questions more on the technical aspect of things. What are your thoughts on the difficulties of even modeling modeling and categorizing the discourse without introducing biases, or at the most, reducing biases. Right so

great technical question there's actually a lot there. So we have a three week logic, and argumentation training program so any of our analysts or volunteers who are working with us. They learn how to restructure and create self contained claims with as much information from the natural language snippet, but without adding more and as I kind of touched on in our presentation, whenever we do have to add more information or to make something makes sense, or we're actually not sure if we can derive that information in a way that is the complete sentiment of whomever wrote that natural language snippet. There's different grades in which we identified the extent to which we are certain long term, our goal is actually to create distributed content analysis software. So our entire analysis pipeline will be broken down into assembly line of micro tasks, and then given enough people enough resources. This software would actually distribute those tasks based on someone's proficiency working in a certain domain of analysis, and then that particular decision that's made we'll just be made redundantly over and over and over again, and then we'll we can actually determine if there's a statistical relevance of consensus. Our goal is, of course, it's not just, you know, whether or not something is. There's a degree of inter coder reliability because even one person who disagrees with the way something is analyzed. It's not just a binary like yes we agree that this is the appropriate way to extrapolate this or No we don't. If there's a disagreement inter coder reliability or there's a disagreement in the consensus, a part of the software that we're looking to design actually kind of turns it into almost a court case, why did you make these decisions so that we can gather more information, because it could very well be that someone is not in the majority, but they have identified and added a tag to information correctly. So it's really important there's extra steps to that process, not just is there agreement about how this particular micro task is executed, or not. So over time and with more resources and one of the reasons why it's really important for us to get these initial maps out is just have that proof of concept of, you know, conversation modeling at a societal scale is possible and it's useful and we can test how useful these knowledge bases actually are to people. And then, with more resources will actually build the software to be able to scale that up and be more rigorous and objective in our analysis process. I'm curious how

your sentences would affect your ability to parse arguments.

The ones that are just somehow glom together that almost sound right but once you dissect them, they are non logical.

Yeah, so I mean, our whole goal is to build the most robust argumentation from all sides, and we run into that a lot where you read a sentence, it seems like it makes sense but we actually have to establish logical proofs. So, and these can be pages and pages long logical proofs where you actually have to provide the reasoning for every single step of how every single concept in a claim relates to the next in order to build that reasoning. So while simple argumentation, you know, can be like a logical syllogism when you actually have to develop proofs. They can be extremely long, and so it's okay that someone's reasoning is not complete in a natural language snippet because by breaking things down to its like most base logical units and then organizing those by topic and category. We can start to see how all the claims present may actually fit into that complete proof, so that saves us a little bit of work but there's still a lot of work in actually making sound reasoning for any argument. It sounds

like you might have answered this other question from the chat, who gets to decide what the given truth is is there a truth committee what makes a proof. Great question.

So, we actually never decide what is true. We simply create the context of additional information that can give people other impressions like this is the most evidence base with most corroborated evidence or not. So, we at the society library we actually refer to truth as among the possibilities of that which remains once what is wrong, is proven. And so that's a, you know, that leads into like well how do you know something is wrong and that could be when there are misprints and mistakes when something is pulling from you know is referencing a scholarly article but they've actually extrapolated that conclusion of a scholarly article incorrectly. That's how we can say, and our process is actually we would then write to the, the authors of the scholarly article and get them in a survey saying this is the inappropriate extrapolation of the conclusion of our particular paper and then that's when we will have a little label saying this is you know incorrect or incomplete in some way. So we actually never determined what's true. I don't know if we can ever find the whole truth and nothing but the truth. But just giving people more information and the ability to get that bigger picture. I think serves our mission in giving people access to more information so that they can make more willful and informed decisions about what what it is that they want to do, especially politically

speaking of politically, um, one thing I've noticed in the differences between for example the American political system and a lot of other Western democracies. Is that so called minor parties have representation in government in either just having seats in the parliamentary body or being part of Coalition's and minorities. Again, if this is out of if, if you don't have an answer for it you can just say, okay, but, um, does the polarized political climate and the polarized media environment, impact the polarization online.

Um, Well, I do believe that people make decisions and their worldview is shaped in part through access to information. So if the majority of people are only consuming a majority of major media sources or participating in conversations dominated by majority ideas online, then I do believe, maybe some of the most major polarizations that we see are just split among like red or blue. When in the context of our work you know we're really digging for every single small little echo chamber and, and community that's arguing about this issue like our goal is like total comprehensive representation. So even though these ideas that were like looking to surface may not be the most popular ideas that are discussed on TV or you know that that is put forth in legislation, you know, our goal is complete and comprehensive representation and so we would have to scour all over the internet. In order to find those. And, yeah, I guess, if I am answering your question I do believe that polarization is in part due to how we consume information so the polarized media environment may have a big impact but of course I'm not certain. Okay.

And how does your system handle disingenuous argumentation people that just, are there to, you know, just cause trouble or to argue in bad faith.

Yeah, so it's pretty much impossible for us to really know the intention of the viewer. And we actually by and large anonymize arguments and argument is an argument unto itself doesn't matter the intention of why it was put forth. So when we actually derive those claims it just becomes a claim unto itself on occasions, it's actually very important for an argument or claim to be associated with who delivered it because that person's reputation in lieu of evidence or further reasoning for example, is what must be relied upon. So sometimes like that argumentation associated with identity is very important, but oftentimes it's not so when people put forth kind of, you know, even like nonsensical argumentation right like maybe they just want to. They want us to like go in a particular direction and waste our resources and time following something that will never ever find any resources for. So far we don't have any processes to make sure that doesn't happen But currently, because we're doing most of the scraping and we're just looking for conversations that have already happened. We're not yet really subject to people trying to send us information in bad faith that will be a, you know, an operational distraction. So for now we're safe we'll think more on how to defend against those types of, you know, DDoS intellectual attacks at a later time.

On a similar vein, just as a final question. How do you have any way of indexing sarcasm or detecting sarcasm in such argumentation or is that just again something that you can't handle per se.

Yeah, so we

actually we we do this in spreadsheets I didn't put our spreadsheets into the presentation because that's not glamorous. But we do we do make little notations about what the analyst believes maybe sarcasm what the analysts may believe you know his humor, or something like that so the most that we can do is that, this is our assessment of something, but our goal is not necessarily to have sarcastic arguments, be the ones that are represented if there is a sound and serious way of representing an argument. Then we'll find that so again by breaking it down by keywords and saying okay these these claims are essentially saying the same thing. This one's probably sarcastic sarcasm humor metaphors, those types of things may actually be very useful statements, because we're an educational entity, and someone hearing something in a humorous way or a sarcastic way may inspire understanding and comprehension in a way that straightforward communication may not happen so we collect them for now put them in a bucket we'll see if they're actually a useful tool for explaining information at a later time.

Thank you very much, we're just about out of time. One last comment. The style of your presentation received a lot of praise for the vaporwave theme. And thank you very much Jamie Joyce for joining us at hope 2020. Thank you and thank

you for your questions.