COVID-19: Combining crowdsourcing and AI to tackle the pandemic

5:05PM May 20, 2020

Speakers:

Ahmed Riad

Francois Grey

Keywords:

crowdsourcing

hackathon

ai

ideas

people

called

chess

platforms

question

problems

solutions

innovation

crowd

webinar

q&a

barbara

project

human

images

happening

Morning Good afternoon and good evening and welcome to the second episode of AI for Good innovation factory. Last week we had a startups pitching session, but this is the first webinar of the innovation factory. We hope that you found this your colleagues and your friends are all healthy and safe. My name is Ahmed Riyadh, from the IQ. I chose the International Telecommunication Union, and it is the United Nations agencies for the addition agency for ICT. We're also the organizers of the AI for Good Global Summit in partnership with the actually were guys that AI for Good.

Five factor applications of AI advanced them in sustainable development goals and have them to scale to a global impact. The actual aim of the innovation factory is to bring together a diverse and inclusive community to identify and scale the most innovative solutions and ideas using AI to achieve the SDGs. Now like most of the world, the AI for Good Summit has gone digital and we are moving forward with weekly online programming allows us to reach even more people throughout today's webinar is one element of the AI for Good innovation. And since it is a factory, we expect the Webinar to be very interactive with a lot of engagement from you. The attendees, especially at the webinar title is combining crowdsourcing and AI to tackle the pandemic. So we'll pretty much use crowd sourcing during this webinar to get some ideas from you on how to combine crowdsourcing the AI to tackle different them. There will be even a competition for the best idea which you can generate. As first exercise to test the level of engagement we have now. I invite you, the attendees everyone to tell us what country they're connecting from today, you can use the chat feature for that, not the q&a, the chat feature just right on the chat, where you're connecting from. And we actually are going to use the q&a later during this webinar for another exercise. So without further ado, I would like to introduce today's moderator, who will be will be further introducing the rest of the panelist is Professor Francois gray University of Geneva is director of Geneva Geneva initiative. And the coordinator of crowdflower, as the GS is also the co chair of the AI to preserve culture, heritage solution track, and they AI for Good Summit. Thanks very much, Francois, for being with us today. And please, the show's yours.

Thank you very much asked Matt. It's a great pleasure and honor to be part of the AI for Good seminars here and this webinar today. So I'm going to introduce shortly, several panelists, and we'll be discussing this theme of combining crowdsourcing and an AI to tackle the pandemic. Now, in the title here, I don't think I need to explain to anyone what AI means. And I think we're all aware of what the pandemic is all about. So one word in this title that does need a bit of explanation is crowd sourcing. So I thought I'd say a few words just to set the scene here. We are, you know, we're in the year 2020 of the word crowdsourcing is pretty widely known it means different things to different people. But not long ago, the word didn't exist. In fact, it came into general currency through an article in Wired magazine by a journalist called Jeff Hogg in 2006. So that's 14 years ago. And two years later, he wrote a book by the same name. So, Jeff Howe really spotted a trend. He sets the Zeitgeist of that period, which was this rise of participation, digital participation on a large scale in solving problems or achieving goals. And even at the time of his first article on this, he certainly weighed up the the benefits, but also some of the risks of this kind of crowdsourcing. So at the time, things that were new included Amazon Mechanical Turk, Nowadays a widely used and well known form of crowdsourcing to solve problems by inviting thousands and perhaps hundreds of thousands of people, forced micro tasking, so solving small tasks for small amounts of money. There are many of these platforms in the world, not just the Amazon Mechanical Turk. One of them here in Europe is very popular is clickworker, which started in Germany also about the same time as Amazon Mechanical Turk. So around 2005. And there are many of these solutions out there. And as we alluded to, in the introduction to the theme today, some of this clickworker activity has got a fairly negative image over the years is some exploitative views about what's happening. People are essentially being asked for very small amounts of money to do the dirty work of the internet. A film called the cleaners in 2018, I can strongly recommend a really shows this dark underbelly of crowdsourcing on the internet. But today we're here to talk about some positive and potentially high impact combinations of crowdsourcing and AI, bringing humans and machines together in in constructive synergies, and in particular to tackle COVID-19. So I'd like to introduce our first panelist indeed, I'd like her to introduce herself. This is Barbara Nietzsche from Politecnico di Milano. So, Barbara, tell us briefly about yourself.

Hello, Hello, everybody. I'm Rebecca Nietzsche from Politecnico di Milano in, in an engineering school I polytechnical and I'm a computer scientist, and particularly I work in Information Systems. My research is about in precise data and data quality information quality, and I focus on assessing the quality of the data and tried to pull it in several ways, including their processes for improving the data as it is being produced.

Great, thank you very much, Barbara. Now, you are a leading researcher in this field of AI, but you have a very interesting background. You know, no webinar on anything to do with AI would be complete without a mention of chess. And the challenge that AI has, has put forward around chess already. A couple of decades ago, you were a world chess champion. Can you tell us a little bit about how being a chess champion led you into the work you do today?

I was playing chess many, many years ago, I actually was an active player for 10 years, and I did the Italian championships, the Olympics. So no tournaments. So, I was meeting many chess players, but also meeting computers just at the time. That was the very early time of computers at chess. And I was fascinated by it. So in the end, one of the reasons for studying computer science was that chess was starting to be played by machines was still very badly at the time. But it was a very fascinating topic. Also, this has been a automatize in the game of chess has been a dream of many years, already in the 18th century. That was the original Mechanical Turk. Which was storing the courts in your much before the Amazon one, which was pretending to play chess automatically. Actually, this was done by a human hidden in a machine. But it's it has been a really ugly way of playing chess World Champion level since the beginning of this was very fascinating at that time.

So if I can add to that, so the Mechanical Turk, as you say, was a term from the 18th century this, this fake robots as it turned out a human pretending to be a machine and playing chess. And this gave us its word, which later on Amazon, coined as Amazon Mechanical Turk, apparently an idea that came right from Jeff Bezos. Now, that fake machine Amazon has called that artificial artificial intelligence. Now you were saying machines at the time you were a champion in Europe and the world, were not that strong. And then in the 90s, suddenly things took off. And of course, there was this dramatic point at which Deep Blue beat Kasparov in 97. But you've told me that you were a bit disappointed by this. This rise of AI just was not what you expected.

Yeah, there was a big change in recent times. So Deep Blue was still playing chess as a computer, which was instructed to play chess so so the rules were coded into machine. So there, of course, the machine was much faster than any human and in the end that didn't do mistakes, or at least was able to analyze computer games much faster. So is that the computational advantage. But what was happening recently in particular, this was happening in 2017. And the first paper was published about it on science. In 2018, that was completely new approach to compute a chess based on learning, machine learning. Basically the system which was called the alpha zero, and there are many versions of beta which are also open source, and is based on the idea that you can learn how to play chess, so just play a game against yourself as a computer and this type of learning, of course, we do computer power in just nine hours would reach a level which would be More than any other Computer Chess. So this was something which was unexpected because so for up to then the improvements were basically improvements based on the fact that new rules were being added. So that was an incremental improvement or systems which were anyway much better than any human player but still being stuck at that traditional way. This self learning mechanism has nothing to do basically with the knowledge of the game, but it is based on the experience so that you can accumulate to just play against yourself to and there is the way the systems are playing out, which is sort of reasonable. So the games can be interesting. And this is also another aspect of which is Making this AI approach to the making the play much more human like in a way of the games, of course, but much more human like than the previous computer chess players.

So so as as you're saying recently in recent years, AI is starting to play chess in a way that you find at least more compelling, more interesting, more innovative, if we just to round out this and you know, what is the connection with crowdsourcing, although everybody knows about this Kasparov Deep Blue match that was the sort of turning point of human versus machine in chess. There was two years later, actually, in 1999, this Kasparov versus the crowd, where a match was arranged, in which he played against 50,000 people roughly in 75 countries. 62 move game that took over four months. And he described it later as the greatest game in the history of chess. I'm reading this off of Wikipedia, by the way, which is another great example of crowdsourcing. And, you know, he says the sheer number of ideas, the complexity, the contribution is made to chess, make it the most important game ever played. Now, he might have been very positive because he won that game. You know, he didn't do as well as against the machines. In any case, there has been this sort of parallel evolution of crowdsourcing, any AI, both in chess and elsewhere. So, so Thank you, Barbara. Before I move on, I think azmuth just wanted to add something about chess in the context of this webinar series.

Yeah, thanks for Thanks very much, Barbara, for these great insights. super interesting. And actually, I thought it was worth mentioning that the next week, we will have an AI for Good Webinar that discusses the recent AI developments in chess and how it impacts you Played by humans. And we actually have Peter Benson who is who is the chess master and the coach of Magnus Carlsen, the well known world chess champion. We're also gonna have another speaker here who is Sebastien current. He's the CEO of chess 24. He, I think we'll be sharing the music, industry, the innovation, and even beyond, I think will be moderated as well, by some from the FBI kind of kicker from the economist. So I think it's super interesting for me for Barbara for any of our attendees who are interested in chess is going to be next week.

Okay, now that we've paid our respects to chess and connected AI and crowdsourcing to that theme, I'd like to move on to our next panelist, who is mighty Lopez, who's joining us from Barcelona. Mighty Would you kindly introduce yourself to the crowd?

Thank you, Martha. Yeah, I am teaching artificial intelligence in Barcelona at the University of Barcelona. I also coordinate a master and intern diversity master of artificial intelligence here and I do my most of my research in the artificial intelligence Research Institute, so many artificial intelligence around specifically in research and in the area of multi agent systems.

Maybe later.

Great, thank you for joining us my day. And you bring to this discussion, in particular and experience with new platforms that are being used for for digital direct democracy in Spain, one called console and another called SCD. Can you tell me something about how you became involved in the use of these crowd sourced democracy platforms?

Yeah, it was a kind of indirect approach. I was doing research on a project to study how could how we could better share and use responsible in a responsible way energy. And we had in this project some local initiatives. One was very interesting where we had an initiative for schools so that children could debate the better way to save energy in their school. So they agreed on the way they had to share energy. And of course, our research, which is related to give support to the wage to human debate, helping humans to reach agreement. We were in touch with these initiatives. And somehow people in Madrid from console, heard about that and invited me to your worship. This worship was for collectivity. intelligence in democracy. So there I learned about console, which as you mentioned, it's an open source tool for participatory democracy. And then when I got back to Barcelona, then I was able to get in touch with the people of the CDM, which is kind of similar to also for direct democracy.

Great, and we'll be talking quite a lot more about those platforms. But just one more thing at this point is, why do you think these participative democracy platforms started in Spain? I know, I know, there are others but these ones have become particularly popular with what do you think Spain's role is there?

So you know that in Spain, we had a crisis and economic crisis, that kid very strongly so it was kind of hard. And after having this crisis, people get somehow not happy about politicians, and how democracy works. going on. So there was this grassroots movement called 15. m, that was asking for real democracy. And first they went on demonstrations on their public places as squares. But then they just started to move on line. So they created the first platforms to gather proposals to improve democracy, and so on. And these are kind of the roots of more evolved systems as console or their CDN.

Thank you very much. So we'll come back a bit later to how this could benefit from AI. How is sort of crowd sourced democracy could actually be enhanced to artificial intelligence. At this point, though, I'm going to turn to our third panelist is he drew lasso from the European Commission. Isidro welcome and please feel free to introduce yourself

Thank you. Thank you very much for the invitation to be here with you. In Seattle. So my name is Andrew Leslie status. I'm from the UK commission in the deputy of the innovation ecosystems of Europe at the new European innovation Council in new dark paths. Many people call it before that they have been already now for 20 days in the commission, always in the area of independency startups. And before joining the commission, I was an enterpreneur. Myself, I set up two software companies, and from then a move to the big companies and then I ended up in their coverage.

And you've spent quite a while there, as you've mentioned, could you explain to me someone with your background in innovation in startups in industry, why you would end up in a place like the European Commission, which seems like a big bureaucracy from outside

But he still had the coalition has changed a lot in the last 20 years has changed a lot. 20 years ago, when I joined there was a lot of fun it was I looked over highly respected technical people who has been working in MIT, for example, in the area of research and innovation, but that's the idea that there was something very special type of contrast and our contemporary ages and a 90% of the departments in charge of research and innovation working for the agents. I had written myself several books, and I was a professor in several universities associated lecture. So join us as I mispronounce word nationalist been seconded to the European Commission as how does how he started in this very, very intellectually challenging bidelman and then well, the reality is, you know, very family oriented, my wife was here, copy dresses, we did okay in school, the kids etc. And then We decided to sue decided to stay here. And then of course, did as well. And that's how we ended up being 20 years. Well, initially, my contract worked for a maximum of four years as an expert so that's why I ended up being so long in the interbank commission.

I happy. Okay, you're happy. That's that's matters a lot these days. And you mentioned this term European DARPA to describe the European innovation Council, DARPA, just to remind the audience for those who might not have heard of it, was this this Defense Research, establishment in the US amongst other things, basically, created the internet or hire the scientists who went on to create the internet. So European DARPA, how how's that going to work?

So Well, they they look into it is just mini science because many times because it resonates very much Within this data community that does our customers, as a deep tip is that so the status we're working with for sample and that's the ones we want to support. We have a big budget at 10,000. A million euros a we the data with the debate. Question in the minds of the people of the startup community is that they created a Silicon Valley. They weren't the ones who were initially put in you mentioned internet, but many other projects. They started from that agency, that Defense Agency. So then that's a little bit who we want to do here. We want also to support the startups and scale ups in Europe to become as successful as it was coming from Silicon Valley. We don't want to Silicon Valley I always say that the last thing we want to do is to have our concentration like in Silicon Valley, we are fully against that. We want a more decentralized approach, and that's somehow working. Let's see now with the COVID crisis. is having a tremendous negative impact on the startups and the scallops, in terms of funding, etc. Let's see how that might be damaging this will be to the innovation ecosystems in Europe. But that's a bit a philosophy behind the look innovation comes.

Thanks. And the reason we've invited you onto this panel was the link with crowdsourcing. And in particular, the fact that you've been behind this massive crowdsourcing effort that was launched recently called you versus virus, a giant hackathon on the scale of the continent. Can you give us some figures about how big that event really was?

what it was a 21,000 participants 3500 teams working so to move tenuously in the IT infrastructure. It was it was a challenge that you can imagine. And we had the we got the world record in terms of solutions. Actually. Develop during the hackathon, to focus on 163 solutions. And this is the world record. The second one is initiative done by a hackathon organized also ready to innovators by Facebook, Microsoft, Google, and they had a 1600. So we paid them for, for more than 500. So and that was a day with all the power of the big companies or however, or, you know, working together was able to get more solutions. 2160 See, all the big tech companies together. So that was for us. For me, at least, I was not expecting this because a number of solutions. For me as the most important thing was that the number of participants or the number of teams, is the actual solutions that came out of that that showed that it was a very productive and now we are with a much effort.

Okay, and we'll come back to the we'll come back to this next stage of innovation because as you say, it's a large number of us ideas but how to make sure they go on and become. Some of them. become something that's implemented is more complicated. We'll come back to that, but allows me to introduce now, our fourth and last member of this panel sharada Mohanty, and four, four. We're doing an experiment here. Something you guys may remember, Mohanty, because he uses that name once he and I are in the same room two meters apart. Do you remember those sorts of meetings? So we're going to let Mohanty speak into my mic and hope that that works out. Okay. And clear for you.

hoping it works. All right. Hi, everyone. It's really good to be here.

And my name is I am a

researcher. I do research in AI from detecting or diseases of plant leaves to teaching how to make simulated skeletons work. And about a year

and a half ago,

I had the bright idea to switch my PhD and start a company around this project. is Dr. Dre my PhD. And that is something I have been focused on right now, which is called as a cloud, which is focused on building communities of AI experts and practitioners who can come together to collaboratively solve real world problems that matter.

Great. Thank you, Mohanty, and you are an adept hackathon participant you've been in I don't know how many hackathons I've seen you in many of them. Can you tell us from a user's perspective, what it's like to be in one of these events? I suspect many of the people watching this webinar have never tried a hackathon before. All right.

That's an interesting question. And yes, so hackathons are very interesting and more so when you are in technology and research. in context of research, your day job is your favorite hobby project. And at the same time, there's many other smaller hobby projects,

which are kind of left out.

So hackathons give you this option. Opportunity have two three or four days where you can go all in and and completely immersive way surrounded by highly energetic people who just inspire you. And you also have the responsibility to inspiring back. But then, you know, if you go in and you give it all in, then within about two days, you will have something really tangible of an idea that you have been toying with for quite some time.

Thank you. And next question for you Mohanty, as we try this new function of zoom where two people are in the same room. You are working with a very interesting crowdsourcing community. And this this it's a very specialized one on AI crowd. Can you tell us something more about what kind of crowd is like to work with some examples?

That's one common question we get right because people have completely different perceptions of what is crowdsourcing? And when we are saying, All right, we want to crowdsource the development of an AI most of the people Not really know how we can contribute. So this is usually a very specialized set of people with a very specialized set of skills. And it's also very important to kind of keep them engaged with enough good problems to begin with. And in setups like these, this is a background or let lay this small visualization we have of activity around a particular series of problems we deal in a particular competition. But these are a group of people who come together to solve these problems, and many of them are already experts, experts in a particular area of AI and they help others specifically work on that particular problem. And there's many others who are there to learn. They know a few particular aspects of AI some mathematics and programming, but they come into this community with this idea to learn and there are others who know a lot and they are there to mentor and help them learn while trying to solve it together. So that is interesting about this particular community is that it is also a learning space of sorts, where the community is not a fixed set of people with a fixed set of non evolving skills. But this is the community which is designed or wants to be a place where anyone can come in and learn in this in this community gets more and more specialized across many models.

And so that allows me to also introduce an example of how crowdsourcing and AI can work together an example that should hopefully stimulate people in the audience to have ideas themselves because as I mentioned at the beginning, Achmed wants us to compete here to come up with good ideas, and there will be a sort of prize a reward for the best idea in terms of actually implementing it. We'll come back to that. So the example I want to share is work that Mohanty and his team together with our team here at at citizen cyber lab at University of Geneva, did about 18 months ago, together with the UN library. So basically, a large data set Which is the archives of the League of Nations is being digitized scanned. This is three linear kilometers of documents of all forms, handwritten and typed and so on, on aging paper difficult to decode. And we got crowds involved in tagging the information in these documents and using this tag data to train on the AI cloud platform to train algorithms and find out which algorithm could do precisely this task the best. And so this sort of collaboration between a crowd which is necessary to get a certain amount of data actually tagged, recognized by humans for what it is, and then having machines and the people who develop the algorithms for the machines compete is is the sort of way that crowdsourcing any AI could really contribute, we think also to COVID and the COVID crisis. And so we're we're opening up the floor now. And I've met we'll say a little bit more for a sort of spontaneous crowdsource competition during this webinar.

So over to you, Matt.

Thanks for for this is this is great. I mean, thanks as well for all the panelists. I mean, it's been a great journey. And now we have this holistic overview of various use cases. I'm hearing from from the speakers on using combining crowd sourcing and AI. So I have to say that based on the responses we got, actually from the first ice breaking question about asking everyone where you're connecting from which country, we actually got more than a sponsor from more than I think 35 countries are here in this in this webinar, listening to us, so that's great. And I think it's a it's an opportunity to, like leverage this diverse brainpower that we have an us to launch this kind of competition and ask you guys, the attendees to come up with ways of combining crowdsourcing AI to tackle the pandemic. So before we start pitching your ideas to us, and he gave us some ideas on how to do that, there are two very important tools that we have to keep in mind. First, to submit your idea or pitch your idea, please use the q&a function at the feature a q&a on the chat. So

write the word idea

before you write your. So I think now it's a good time to start doing this, just use the QA, write the word idea and then write your idea of combining crowdsourcing the AI idea to tackle pandemic. And also have to say that the top idea the one that gets the most votes, and we're gonna let you know how to actually vote and this is going to come later in the webinar would be a crowd SG project, that we're going to have a dedicated team to work on this challenge in the upcoming open 17 Summer challenge. And many of you would know the opposite in summer change. And many of you would not know the opposite of a challenge I would like to invite for Swami to give us a little bit of an idea and extend little bit what is the open 17 summer challenge and and this concept of taking this project or the winning idea that we're going to receive from the audience day forward.

Yes, thank you for coming. It's so so just to explain so first of all, crowd for SDG has been mentioned a couple of times called for SDG is a new European project to combine crowdsourcing and artificial intelligence for the SDGs. It's being launched this month today is the public launch. Indeed, this event is part of that. And crowd for StG is supporting something called the open 17 summer challenge, which will happen in July, where we'll reach out to over one and a half million high school students and undergraduates and master students and invite the very top ideas that they have for credit. Switching an AI to sort of online summer school that will run for the whole month of July. So what we're proposing here is the best ideas that come out of this webinar. One of them will actually be developed to through this summer school. So that's why we're inviting you to crowdsource again, as Matt said, on the q&a function, put your question your ideas in their bracket idea in front of your idea, and we'll vote on that shortly. So, having introduced the panelists and having launched a crowdsourcing effort of our own here on the webinar, we're now going to dig more deeply into this issue of crowdsourcing and AI with each panelist before turning over the floor to you as a q&a session for roughly the last half hour or so. So I turned back to Barbara and Barbara, you've been combining AI and crowdsourcing for several years now, especially online social media in connection with emergencies. Can you give us a little more about that sort of obligation?

Yes, thank you for SWAT we, we've started to work on this in 2016 with the TMC project, which had the goal of supporting the Copernicus emergency management service in a specific activity they are doing which is producing that map so very rapidly after a natural event is struck. So, the idea there was to get some evidence about what was actually happening on the side the web the event that struck collect information from social media in particular we were focusing on with her but also on Netflix, YouTube. as examples where we could do this, and maybe we can see this in a slide with an example, what we wanted to show to operators mood in the maps, which were telling the first responders going to the place where her some problem was or whether it was the damage or people need them. So they're the two way we were building was to put the social media posts on a map, and to show them that to the very choose to show for instance, that they give a road that was flooded, or that giving collapses though there was a real problem and so on. To do this, you need a UI for several reasons. First of all, know where you get information from social media is You have to access those social media called information, but here we are looking for images which are giving evidence. So, not all the images that we are going to get are going to be really relevant to the event to take you places, we will probably get a lot of images and some of them will be irrelevant others will not. So, the goal here is to display to the ones really the maps. So only the ones that which are which are highly likely to be relevant. And also, we wanted to place them on a map and as you know, you might have place or some form of location site with or not. So what we wanted to add to the abilities to extract the location also from the information contained in the posters or from the text and other and the metadata available in the post. So, here we need the AI in different ways. So, we need to do the natural enter good understanding tokenizer the location Sir, we have to understand that where these locations are disambiguate them, we have to fit the images and this in this first project was done at trying to identify where the first is flawed, this was represented for instance in the sample You see, but this is something that was started there, but there was another part of component that which is mentioned here just briefly, which is the cause of assessment, we would give this information also to this picture the text to the counter to validate it to tell us whether it was relevant or not for this task, and also the validator the location, so the counter would Get the candidate locations possible locations have already emerged and for a crowdsourcing tool that is was correct or not. So the final result is a map with images that can be filter by relevance to the factor you can locate them, and that this is a basis for adding evidence just during the event just after the event occurs or not. After a long processing of this information, so for supporting the mass production or by the very laws, which are to enter within the Copernicus Emergency Management Service is this rapid map in their production? Set? Yes,

thanks. So I think this this is a really nice example of AI and crowdsource Producing data which is very close to being actionable. I mean, we're still at a research stage for some of this. You have recently joined in these online hackathons, including the massive you versus virus hackathon. And started to to take some of these approaches have synergies between crowdsourcing and AI into the COVID domain. Can you say a few words about what's what's happening there?

Yeah, if we go to the next slide, this was actually my second hackathon. We were doing this act on separation or so, for the crowd before as this G project and here this is the similar idea. We are showing images extracted from social media from Twitter. And we are asking questions which are related to the coffee and the particular to the behavior People related to COVID, whether they were wearing masks or whether they were keeping social distancing. So on. Here, the formula is a little bit more complex than before, because in this case, you're getting just general images. So the filter is not focused on a given special event like a flat, an earthquake, but it is a general image. So we need many more filters for the images to get the good results to, for images to be shown to the crowd. And here the crowd can help us in

building

the tent is to build some statistics about the behavior. So not only showing where these images are taken, but also showing percentages of different behavior. So in maybe thematic maps, so the result for the cow though, is something that it is used for providing evidence about what is happening in a given This area or a given company,

thank you so much for illustrating this. I hope this will have inspired people to think of other ways that crowdsourcing and AI might be combined. I see already several ideas in the q&a. So people have started putting ideas in there. So don't hesitate if you come up with an idea for this internal crowdsource competition of the webinar. I'm going to turn to mighty again now. So you were explaining earlier all about this siddim and these platforms for digital democracy. Now, you're working on ways to combine AI with these platforms? Please tell us how can I help a digital democracy platform?

Yes, before getting into the details, I would like to show a video to explain what this game is. I think it would be easy to get it. An idea

We'd have the video. Okay

I think we're missing some

mighty I think we lost the sound.

Yeah, but I am not playing the video.

Ah, okay.

Try one more time.

Or why don't we come back to that I think will will recognize sometimes technology doesn't want to do what we want it to do. You can say In a few words, I think you've already explained a little bit what this game does. What you want to emphasize and then let's look at how I can help.

Yeah, so I will share my screen to try to explain slightly. So, basically, this game, it's an online platform that allows many people from many places to get along and share some ideas and, and debate about ideas and proposals. So this image has been proven to be very useful to somehow strengthen communication between citizens and institutions, government or local governments, but also for organizations and individuals. You can explain. It's kind of nicely explained but basically in this, in this picture here, we can get the idea. People get connected, provide some ideas and all those provide opinions about these ideas. And what I was also meant to explain was that these days with a COVID-19 crisis, this idiom has been used by different organizations to gather some ideas and to conduct democratic debates. So, basically, there have been different alternatives and different initiatives. But I wanted to command here that, for example, in France by their government, the French government, they put it together many proposals for the COVID-19 crisis in Barcelona. This beam has has been posted in the city of Barcelona and battle analysis team. They had an issue initiative for taking care of people at home. So it's barthelona from home. And also there have been some hackathon COVID ideas from the Catalan government. So we have different examples where this has been years. And for you to get an idea, for example, this is a screenshot of one debate going on in a job that fair, and people was able to debate about the need for, for universal basic income or not. So it's not only that someone provides an idea and the rest vote, but we want to have a debate some reasons, in favor or against this proposal so we can all agree and reach agreements in an informed way. So at the end, it's a tool for collective decision making. So in this example, for example, as I was mentioned, Meaning, we have a proposal someone proposes this proposal. This one, as I mentioned, it's about basic income. And then many people can raise some commands or provide some reasons in favor or against. So for example, here we have someone providing a return in favor and some of us against. But it's not only that people can provide some arguments, they can also provide their opinions about the arguments of others. So, those arguments that have more support can be taken into account. But the real The thing is that, for example, in this case, we had 140 comments and arguments provided, but up to now, there is no waste of combining all these information. So any human that wants to get To reach a conclusion has to read everything. And our idea with artificial intelligence is to provide an automatic way of aggregating all these collective opinions so we can reach an agreement based on the individual opinions, but how to magically

Oh, that that's what basically I, I wanted to explain.

So that's that's the state of the art for the research now is to combine AI in a way that that extracts that. I think in the question time, we can come back to this issue, because of course, it would be great if AI can help these kind of debates converge. But there's also a certain sense of, you know, human democratic discussion being moderated by AI could seem a little bit scary to some a little bit like thought police. So we'll come back to that in the q&a. But thank you very much for that illustration. of how these two technologies are converging. Isidro, turning to you now crowdsource innovations with these hackathons. And now online hackathons, which seem to produce even more ideas and output. That's great. But can you tell us more about how you going on to ensure that some of these ideas actually end up having an impact somewhere in our society?

Yeah, we're, we're in the in the song, crowdsourcing exercise, I just said, we got ideas for solutions. And some even some mock ups was more evidently prototypes, or what is called an MVP, minimum viable products, but very, very basics. So that's part from from solving any problem. It's a good idea that most of the cases with these hackathons they end up in the in the cardboard of the person who was working If we came back, and then we didn't want this case because this case is very particular we have talked a lot about technology. We are talking about the economy or competitiveness, we are talking about saving lives, saving jobs for many people. We are talking about helping elderly people and all our grandparents in the center. So is it it's a very special situation. So we decided not to stop in the in the in the in the hackathon, but continue with something completely new and never been done in the world. That was something called match. So match iPhone comes from the world of matchmaking. So it must make in between and the ones who have the ideas. They Hackett's from the hackathon, the winning teams, and the ones who have the resources. So in this case, our corporate investors and public authorities, so we are putting them together into the next weekend this weekend now and they started on Friday, starting actually on on Thursday was a demo day in which we have selected For the demo day, we have selected the top. The 30 winners were more mature. And we have invited a bit that we have, they have been won more than 120. investor who will be the demo day. This is tomorrow. The day after tomorrow on Friday the 22. May, we'll have we'll start the big matching zone in which all the investors the cooperatives in total around 600 partners will meet and will much much make will do much making with the 120 best ideas from the hackathon. Our idea is that Monday the 25th of May, we'll get a list with three columns. The first column is name of the project. The second column is the name of the department's corporatist polygamists which investors who are supporting this particular project and the second column except liquidity cannot support. If it is a relation. If it is an investment if it is a Acceleration program is whatever it is income or cash contribution that they are doing. So we want to have this clear, edification, and the process doesn't end up here. The process is, has three phases. Phase One deviation, there was a hackathon, they must make it that is a microphone, and then is the realization that will happen in Europe innovation council called the platform, a secret platform, and in two months from now, so 25 of July, we will take a stop to see how many of the 120 winning ideas from the hackathons haften into actual solutions being used by hospitals where the universities by the MW houses so that's that's the rationale on our mind. And that's why didn't we didn't want to stop early in the ideation that does nice and is good for publicity and all these kind of things. We wanted to really to develop solutions that can help it can turn into reality to unhealthy people who are sad. So many analogies.

Thank you so much easy draw. I think this is really commendable. And a reminder, you know, that it's easy to make fun of big institutions. But occasionally you have the ability, the networks and so on to achieve things at scale that we simply can't have happening grant in a grassroots way to local hackathon. So what we'll see the jury will be out on results. But there's clearly a roadmap there that makes a lot of sense. And indeed, I think for the innovation factory that IQ is launching through this webinar series today. You know, there's a lot to learn there to share. So I hope there'll be interactions between IQ and E and EC on this. Finally, there's questions for you from the audience, but before we get to the q&a, I just like to turn back to our our fourth panelist Mohanty, and ask him about some COVID related challenges that they're about launch on their platform AI crowd. So over to you.

Hi everyone, we hope the AV is a bit better this time. But yeah, so it can be a community of

machine learning experts and enthusiasts. And over the last few weeks and months, there has been a lot of activity spread out all over the web, where people have been trying to help people in maker spaces, building devices, which can help people who are AI researchers building tools, which can help detect symptoms and chest x rays or from sounds out for detecting mask and a whole lot of unsolved problems. In our case, I think we've been a little bit careful in how we basically help synergize those efforts because it's very important that when we push out something, we do not also create the face, fake perception, any fake perception on mismanagement of expectations out of it. So in this case, we will be doing a series of these challenges are required. competitions around various problems on AI, where AI can help in this covert response. But all of these are also targeted in a way where these are mostly to help and empower a lot more people to be more informed on how these things work. For example, the first one that we are releasing is something that also came out of the community, where some of our really inspiring community members came with the idea that we want to run one sip or one competition, or where a lot of people can submit their models or their solutions, which can detect mask a simple problem like that. But how do we take it from a lot of spread out GitHub repositories to actually something that you can actually use something that is tangible. So in this particular competition that we are launching, which will be launched within a week right now it's still in static mode, we will. Oops, so here we will be taking this problem of mask detection to help anyone who's trying to learn how to detect masks. images and videos, and we injuries and you dataset. And then Apart from that, we also basically open the box and show how the whole thing is done. Again, you can make multiple adaptations to this particular problem to see how we think everything is working internally. But at least anyone who is willing to understand how these technologies work, they have at least a good starting point to get started. And that will be the goal of not only this competition and a series of other such competitions around which we are also internally doing this, these research to help engage a lot more people in these problems, and also help inform so that they have can make more informed decisions about how these tools can and should be used. And yeah, this was a game pretty much, you know, it's something that came out of the community where two of our really inspiring community members who have been winners across many competitions, they wanted to make this happen robot in Sharjah, and they put together both a data set these learning materials and the whole competition with the idea that online let's put it out there so that we can build a community around anyone who is trying to work on it. problems. We have to have a lot more over the next few weeks.

Great. So thank you so much Mohanty for that challenge on the AI crowd platform. I think this illustrates again, crowdsourcing in in all its variety. Before we move to the q&a session, I just want to thank Mohanty, personally. Mohanty was a young summer student with us some years ago, wanted to stick around in Geneva, but was you know, getting offers from MIT as happens with very bright people. And we were lucky enough to keep him here at our neighbor Institute, dp FL doing a PhD and he's done the right thing in stopping his PhD to start a startup. You know, that's a famous way to to succeed in this world we hear but we hope you'll come back and finish his PhD one day to moment you inspired many of the things we're doing around crowdsourcing and AI including this crowd for SDG initiative. So that is the end of the formal part of this discussion. And I'm handing over to admin to tell us a little bit about the many ideas that have been posted already and how we're going to vote on these now while we have our q&a.

Well, thank you very much for thanks for having you. Thanks. I mean, for the fans as well. I mean, I've heard a lot of service work and I think we can collaborate in this we had already innovation factory pitching session last week, we're gonna have more of that. So definitely, we're gonna keep the conversation going this direction to get more startups ideas that use AI to achieve the Sustainable Development Goals. For this particular webinar and effort that's happening here. We really got some really interesting ideas. And I think my colleagues is gonna run, launch a poll right now. Yes, as you can see it and all the attorneys and the participants, they can actually vote On the ideas, so please, you can upvote I think multiple ideas if you like, more than one, and you can please do so right now before we jump into our q&a.

And

yeah, and I think we were gonna calculate and see the voting while you first walk in. Go ahead with the with the q&a. I already saw some questions come to the panelists, and through the q&a or chat, but please use the q&a and ask questions a person. I think Francois can moderate the q&a segment now.

Wait, yes, thank you I've met so as the Save vote early vote often for your favorite picture and sorry if your pitch didn't get on, because it came up very recently. We apologize for that. We will track all these great ideas and you know, follow up if we can on some of them. And as mentioned, the outcome is For one of these ideas that we'll get a bunch of very bright summer students similar to Monte here a few years ago to work on that and who knows where that could lead. So come back to the q&a here. And if I take it from the top, I think we have. Missy, we have a question for Isidro easy to sorry, material. Yaga asks, were you able to tell how many of the solutions were similar or the same? And to what extent the source code is available for the solutions to be adopted by other innovators? So there's really two questions their ability to distinguish similar solutions and access to source code.

Whereas on the first one, that was not something we were looking for, so we didn't ask the jurors to look at that. They were evaluating every proposal on their own. And that's that's what we wanted. We wanted to identify The best proportions and does this have says we did was that was no looking was not comparing between themselves was evaluating the each one of the proposals on their own. That is normally how the commission evaluate any proportion. Of course, this is not a proposal for funding or the commission. This is a hackathon is different, but still we were following the same the same principles of transparency and evaluating every proposal on the table. So the answer to the first question is no, we didn't do that analysis. And the second question, everything that is about that wanted to be made available by the by the people who participated in the website in the US was radio dot orgy. So they were requested to have a video that was the minimum and a short description. But many of course that included more details because the monitor is a better son include some of them included. I don't think none of that included they open the shutter but it This is something that if you are interested, you can contact them via email in the platform, you can find that those data and they might provide it to you, or we do them by any means force them to follow any particular way of licensing because this is a data set, we were talking about more about ideas than about actual products or services or solutions.

Great. Thank you very much for that answer. Then I have a question from Rashid. And I think it's a question for the panel in total, so maybe going round the the, the virtual table here. The question is very simple. Can we create a group in telegram for collaborative innovation now, if we take away the particular platform? I think the question is, especially in these times of confinement, where we are exploring digital platforms like never before, what are the platforms or what should be the forms for collaborative innovation are the tools that we're using well adapted do they need to improve? So I'll start, perhaps with Mohanty, who is of course very interested in this being responsible for one of these platforms. But looking more broadly at innovation on digital platforms, what what is your advice? What do you think works? In

our case, with a cold, we have been trying to come up with a platform which solves

this one little window of series of problems in which how people can innovate together on a series of problems, but still constraint every single day, me and the team all of us learn something new, that these are some things that we should be exploring. It's still not a seamless experience, where you can have 10 different people collaborating effectively to build something which can actually be always converted into a product. So from a from platforms which help communicate better from two platforms. which helped correlate these ideas and gather and distill them into something so that you can stop the exploration phase and go into the phase where you take these ideas and start focusing how to build tools out of these. So separating out all these different phases of the whole innovation process, and having tools which cater to all these different aspects are kind of pretty critical. And right now, we do not have something that serves that purpose. So what we do is we take a buffet of tools and try to make do with something like this, we have a forum where people can talk, we have some humans whose job is to collect all these ideas, and then somehow distill and create separate problems on which some focused and dedicated teams can work. Then after that, we have another group of people who just try to take these problems and build tools out of this, but in practice, in a large community of people, where everyone is equally motivated to try and solve this problem, and everyone has a different role to play. This whole notion of self organization is that is something missing, and I hope

I had to go

into an answer to that and then really working very hard to find one, so that many people can come together and solve these problems together while they organize themselves.

Wait, wait. So there's a lot of work still to do. And it's good to hear that someone on the frontline thinks so. My take from your perspective, after all this siddim these sorts of platforms are supposed to be solving difficult social problems, how close are they to achieving what their mission is? And what's missing in your view?

Yes, I mentioned there are some challenges that need to be addressed. One thing is a quantity of information that they have to handle. We humans have a small capacity to handle interpret information. So we need ways of summarizing or reducing or, or having more visibility for more important issues and this has to be the owner. In a research level, we are working on that, but it's not there on the platforms already. But I am aware that they are working on on trying to link some of their platforms to other platforms. For example, this idiom is able to do the debate but then they are working to link it with Slack, for example, for doing a specific group work, or maybe to do more deep hop or repositories for code. So they are on the way but as far as I am aware, it's still not there. But they are they are aware.

Very, thank you very much. So I think I'll move on because there are some other questions. We have a webinar attendee from Fiji. I don't know what time it is in Fiji, but I don't think it's quite the nice, civilized time. It is here in Geneva. So thank you for joining And your question is, how to get governments involved in these kinds of initiatives? And that's a very interesting question. I mean, as we heard you versus viruses in a sense as an European wide initiative, there have been nationalist initiatives like vs virus. Isidro, do you see this spreading to national governments and Is this something that you know even in Fiji people could do

that well, this is a very good as it is a side product of of this exercise, I mean, objective was to develop solutions to fight about this tragedy, the tragedy, but as a side product, we have got a we have learned something you administration and neither hackathon is a useful policymaking to to reach out outside the Brussels bubble, the glasses micro cosmos and to be able to gather ideas for any CPC whatever they are the small village Narrow Island whatever they are they they have the same possibility to contribute to their ideas in the policymaking process. As any of the big companies or unions or the associations of SMEs who are over represented with a 10s or hundreds of lobbyists. People will love enough. This allows us to get outside the Ross's micro cosmos. And this the idea has been a very much well received by my colleagues. There are actually three services we're planning to do this hackathons are bullish activities, one is external service, they want to do something on the area of Kadima to the to get a solution so for remote education, or this are related to agriculture in Africa. And the third one is about sound financial digital finance FISMA so this is a under members the the same, the 27 member states because we have, we created a working group within ministry services of innovation. They're all very driven So do we be ready to get more hackathons coming, and not necessarily for COVID but for any other policy is of size across rotation that we normally do in brushes through the traditional way. And normally, even if we want to open it to the whole Europe, as you know, the days only for the usual suspects who are based in Brussels, so this is certainly a new, a new way of doing policymaking much more democratic more outside the big cities and the big conglomerates, reaching out to everybody. So definitely something that is going to stay here for long. And as I said not only for the coffee, but for many other issues or maybe also in Fiji and we see that happening why not so we seem so doesn't require the person who is going to contribute, to have money to be able to pay a ticket to go from the village to the capital for the wall. So for the conference was a completely new way of doing policymaking. I can imagine many people is going to lose power with this. And they might react. And I'm sure there will be a campaign saying that this is less effective for this. I think there will be studies showing how the hackathon also booth as the physical meetings with a lobbyists in Brussels or the capitals or across Europe, but still, if it is going to is going to stay, and it's going to play to start playing an important role that has never played before. So yeah, very good. unintentional consequences positive and intentional.

Indeed, this is this famous saying that, you know, in a crisis, there is opportunity, and that design is apparently one of the characters for crisis is also at the root of the word opportunity. And certainly, both in terms of crowdsourcing, I mean, we would never have imagined even trying to run a hackathon with 20,000 people had it not been for this covert crisis. So thank you easy draw. I'm going to take a break now to invite our host's to say what's happening in the pool? Because I think we have a result at this point. So I've met

thanks for I'm sorry, yes, we actually have some results as you can see it, I think it was also visible to the attendees and everyone have around 10 or 11 ideas that you can I can see now in the in the results. I think the top one got around like 44% of all the votes was interesting one says contain COVID-19 disinformation by quickly crowd sourcing from green health experts iversity score about damaging breaking news. That's, that's an interesting one. And also, another one that got 33% is about crowdsource information on the amount of people present in places like grocery shops, leeches, cheats, to be able to check ahead of time, how difficult social distancing will be, which is also interesting, but it also has another side of privacy as well. So it's, it's something that we need to discuss, and I think the two top ideas we got out of the 10. So one got for 1%. The other got 33%. And then we have other ideas with 29% 10%. I think it's up to you, of course one out to see if you're going to take one idea for your open 17. Or maybe the top two, I don't know, depends on on the arrangement you're having at your end.

Absolutely. Thank you I've met so we're certainly gonna look at that we have a track of who submitted them to at least if they have given our name, their name, so that we can thank you. I do think also that maybe a few people submitted just a few minutes after we started the poll. So thanks to them. We will look at your suggestions as well. Can you Chen and our allow our and Ilana heroes. So we will select one of these and we will of course run it as part of this summer school. And indeed there's been a question from Alexandra Mercurio, asking what kinds of summer school are we talking About here. So this allows me to say just a few words about what we're doing here. We're sitting in a place called the Geneva SDG solution space. And this is part of an initiative to to rethink education around the SDGs. And to get students really solving hands on problems from day one, whether it's in a summer school or two year master's program, or some kind of online coaching program, and the problems are coming. Not just bottom up, but from partners in the international organizations here in Geneva. Indeed, the organizations responsible for the different SDGs so we're talking about our neighbors here WTO who it EU Of course, and many others. So the summer school called Open 17 is based on an online coaching we've been doing for five years, called The 17th challenge, and it will take place in July, we will be selecting students using yet another crowdsourcing platform called good wall, which brings together one and a half million young people from around the world who want to solve sustainable development problems. So this will get underway, first of all in an idea phase starting today, and then in a online hands on activity to the month of July, which will be global. So I hope that answers the question from Alessandra and I see a very good point made by a man with a joke juvenility, saying some of the ideas have overlaps and could be fused into one true true crowdsourcing. And I like to turn to Barbara at this point. Maybe you have some thoughts here about, you know how AI could help us to sort out, you know, when we're talking about ideas like this, which ones are really different? Or how much difference there is between ideas? Is there anything that we could use from AI that would help us to narrow down from these large pools of ideas, the most relevant and perhaps fuse some together, like and then really suggest?

Yes, I think that

there are several issues here that you wanted to classify the topics I wanted to match text. But I think one of the things that was touched upon also by the other part and is, is that we have a lot of tools that potentially could help us in doing things. But the difficult thing to do is to decide that which is the set of tools we want to use, but also how to connect them so that you In the end, you can get to a result when you have been using different tools to perform different types of analysis. For instance, here from the ideas that could be the starting point or could be from the CDM, discussions and so on. And then find a way to do these things in the right order. Because you might do this in very with human intervention from time to time, you might do these in different orders and get different results so differently. And also, you might forget or not a result, depending on how you're going to use and combine the two. So So I think there are many things that that should be still started to get really performant and viable

approach to do these things.

Great. Thank you very much. Barbara IMC some comments in the chat. I think we've answered quite a few of the questions in the q&a at this point. But there is a recurring theme here. And it's a theme we touched on briefly and I don't think we could leave this meeting with a clear conscience without coming back to, as it were the dark underbelly of crowdsourcing in AI and ask the question, if we do combine these crowdsource solutions and AI solutions, what are the risks? What could go wrong? What could possibly go wrong? By having AI's say manage democratic debates? So I'm going to start with that particular one that's very sensitive and as mighty, as she starts working on this marriage of AI n crowdsourcing, what she will be and her colleagues will be concerned about in terms of what could go wrong.

Yeah, so The solutions that we are studying

the solutions that we are asserting are based really on the human input. So we want to keep all the opinions of humans we, we don't want to change those opinions. And what we are doing is to translate into mathematical formulas and algorithms, principles that can be understood by everyone and the process can be really transparent. Of course, then you can go to the source code and check if this is right or not, but the idea is to provide processes that can be understood by everyone. So for example, here I am sharing some of the principles that we are using to aggregate all the information from the opinions. So for example, we consider that when we have a proposal that is supported by strong arguments in favor then proposal should be somehow tended to be accepted. But if we have a strong arguments against then we should reject. So we should be able to come up with an explanation of what why a proposal has been selected and based on on the human opinions that were behind. So we want to respect the human opinions we want. We don't want to have a machine taking decisions for us, but helping us to aggregate and to take those decisions.

Very good. Thank you mighty. I think that also partially answers a question from Javier central area who asked how to ensure that the ones answering the crowdsourcing polls are actually not coming from AI with a misleading malicious call. So how would you check that the people contributing to this game are really people and not bots? I think there's some basic ways to do that. I don't know how, how deeply concerned about that

in this game. They have taken this very, very seriously because they are Relating and biting the decisions with a local government. So they have people registered. So it's a really like a certificate. It's it's a secure that the citizen and if it's in their local government then it has to be a citizen from these local plays and so on. So these issues have been tackled in the in this platforms.

Great. Thank you. Well, listen, I think we are winding up now this is very interesting discussion around crowdsourcing and AI for COVID. One of the questions I keep seeing here in the chat and in the q&a is, is there still a chance to submit one more idea? Is there any other way to participate in these events? I hope one of the messages from today is there going to be a lot of hackathons now? I think it's become clear that Online hackathons work, people are sorting out how to make them work better. But that's something that's going to stick. And I suspect it's going to be around even after confinement is relaxed most places. So I would say to everyone who's on this call and frustrated that they didn't have a chance to pitch an idea. Look out for hackathons where you can not only pitch your idea, but maybe get a whole team of people working on your idea. And I'm going to hand back who asked me to say a few words about how it you with this innovation factory is also working on this sort of crowdsourcing?

Thank you very for himself. This is exactly the thing I wanted to say actually, because I see a lot of questions asking about how submitted ideas still open. So So yes, as a part of the AI for Good innovation factory, we have an open call for idea, innovators, any idea that uses AI or machine learning to achieve the SDGs and also startup so if you have a startup so if any of the attendees now or listening to us have a startup that works using artificial intelligence to achieve one or more development goals, ideas, or startups, you can both apply to this open code application. We'll be open till the third end of June 30 of June. So applications are open and you can find it on our website. AI for Good that I do that. On the website, in the program tab, you can find previous webinars and session that we had recorded once so please go ahead and watch super interesting what about the AI to achieve that as it is, and also don't miss our next webinar next week in AI and just I just would like me at the end to really thank everyone. Thanks. Thanks again, Francois. Very much. Thanks, Barbara. Mighty sigil Monte everyone and of course Special thanks to the attendees who actually participated in this great crowdsourcing exercise. As somebody that ideas, I think we have the restrictions of the athlete as well. So we can always get in touch with them later for for the open 17 for the winning project for projects. And I also would like to remind everyone that we AI for Good Summit is like a always on all year. Every week, we have upcoming events from the innovation factory and from our social tracks and from our breakthroughs so please stay tuned for our Facebook, Twitter, LinkedIn pages, just the AI for Good, and I think nothing, nothing is less than just to thank everyone again, and have a great Wednesday. Thank you very much. Thank you