Can image-based AI meaningfully impact COVID-19 response in low resource settings?
3:20PM Jun 16, 2020
low resource settings
Live on custom live train service.
Hello to anyone viewing this. This is Bastiaan Quast from ideal the International Telecommunication Union, and we are quickly testing our live stream for our webinar taking place in 30 minutes.
So, please do tune in for the live webinars
and that we will conclude our tests. Thank you and speak to you in 30 minutes.
Okay, we're applying again.
Yes, I wasn't I wasn't trying my hardest there on the talk. That was the staccato
was good. You just very serious.
All right. Yeah, that would have been better if I shaved and I don't look so.
Great. I mean,
everyone Good morning. Good afternoon. How are you?
good? Yes, it's a big team. so thankful for your support and best in good to hear that you got home safely.
Yeah, I'm here. Oh, good.
okay, so So I think the person we need the most is sky. Do you know We'll be joining in time.
Yeah. Yeah. I think the moderator is always the person I'm most anxious about getting on the call, but everyone shouldn't be, you know, they know to be joining at 830. So we shouldn't be having them log on quite soon.
Okay, cool. Can I ask you one thing? Actually, can you check if I'm pronouncing the names correctly of the panel? Because, you know, I've only seen it in writing so. So I would can say, today, we have a very distinguished panel featuring Judy Sharia, Michael bagel held up on the coffee. And with the discussion facilitated by sky Gil.
Yeah, that's correct. Yeah,
no, yeah. Okay. pretty difficult element on the cars.
Yeah, that's good.
So we just did our live stream, so we'll be live streaming as well.
You're planning on a bathroom but maybe you close your window. As nice as the birds are.
We'll leave it open as long as possible
in passing and just confirming you were able to send those invites to all of the panelists and moderator yesterday as well.
Yeah, yesterday Sorry,
no, I just had this vision of you the just sending it to me as the words came out of my mouth, but
no, it's just that the links are okay. We have one panelist joining already. I see Hi. Karissa, Ashley. And
he shy team.
Okay, I see. Okay. I Yes, of course. I saw the email address. Yeah. And but yeah, no, I sent it to everyone who sent it to you last and like I had already done it, but the links are personalized and doesn't really matter. Sometimes if you use like someone else's link, you just like appear with the person's name. But then other times it blocks the correct person after that from joining And so yeah, avoid that situation. I send everyone an invite with link in the location box and then
So are you are you both in, in Boston or elsewhere?
I'm in Boston and Chris is actually in North Carolina.
Okay, I see.
So we're just putting a note together for our panelists to make sure they're planning on joining us
that's a good idea. Um, I do we do we have the Do you want me to do it? Are you gonna extract the email addresses for the
Okay, great. Thank you. This is
okay. Seems like version 5.1 is out from zoom but the movie
of the year
Just be back in a minute
while we wait for the panelists
Hi sky Good morning How are you?
I'm well how are you? Good and we didn't see any of the panelists join. So just wanted to make sure that you got the invitation In the invite, did that all come to you just to make sure we're not having any technical problems? We did get the invite. If you had a special facilitators link I have, so I didn't know and registered on the website. So if any of the panelists did that, they might want a quick email to just let them know that they should join this link rather than a webinar registration. But that's it. Yeah, it came through. He
forgot to mention Megan, Marco was online earlier. He joined really early, and then we told him that he could come back in at half past but he's not back yet. But he has the link. So we saw him so great.
Great, so Bastiaan this is Skye Gilbert, who's our moderator. I know you were eager to connect with her and bestie and perhaps you can introduce your colleagues as well.
Yeah. Hi. Let's get this nice guy. So this is lesson from He'll and with me here at genuine Ida colleagues and I've met as well, and part of the team that does the running of the webinar. So my role is normally to host and today also the emcee. So that means that I, you know, might be able to, you know, switch on people's cameras and stuff like that if they forget for audio, prompt them to unmute. And so yeah, likewise. And did you see my opening sort of remarks and closing remarks? It was attached to the invite, just so you could look at it. I mean,
I didn't see them, but I trust that whatever you're going to say is going to be great. I don't, that's totally fine.
Okay, okay. Well, let me just paste it in the chat here so that you have
So, I mean, feel free to have a look on if you have any changes. I mean, it's a little bit sort of hardwired at the moment already. But
if needed, I can, I can make some changes.
I'm going to request access.
Ah, I see. Okay, one second if you just change the
settings, and then you don't need to refresh access, or just
yeah, I think if you refresh the page, now it should work.
This is fantastic.
I love that you're doing this because these are the kinds of things that I often forget. So I'm grateful that you're here.
That are good afternoon. Hey.
Okay, so I think Yeah, so we thanks we we have quite a bit of experience in trying to see you know, what works with with attendees and people join late and you know, you need to just try to make it clear. Like one thing is q&a. One thing is q&a for the questions and one thing is chat for communicating with each other. And then you need to make sure you don't send it to panelists, which is the default for some reason. So there's a lot of stuff that's certainly a little bit. Okay, so we go live, normally five minutes or three minutes before the hour with looping slides. I don't know if you guys have seen any of our webinar recordings, but we start a little bit early. So the people who joined early they see these looping slides. And it shows the panel and it shows the topic and some other stuff. And then housekeeping housekeeping rules. And, you know, it's a bit more of a gentle in gentle intro than if you just see the zoom sort of spinning wheels waiting until the start. So we have maybe another 15 minutes.
It's okay with everyone to do a quick rehearsal.
And but we're, we were missing two panelists, but we can rehearse with who we have.
Yeah, I think okay, that would be great if
held up if that's okay with you.
Okay, great. So so we normally start with these looping slides. And that's at the time we asked everyone to keep their audio and their video of an actually we experienced this last week, even when you're muted. If there's a lot of noise, it's still going to pick up on it. It's not going to broadcast the noise, but it's going to show your your Max sort of Name box or a profile picture to the attendee. So, I mean, if possible, try to avoid any too much background noise, a little bit of noise won't do anything, but if there's a loss, it might happen. And and so the the sort of steps will be that I count down 10 to zero with the last three seconds silence. And then we broadcast at that point, we will already have our audio and video off, and the slides will be looping. That's three minutes before the hour probably. And then on the hour, we also start the live broadcast to Facebook and YouTube and LinkedIn, Twitter. So from that point, we should start and we give it usually about 10 seconds to start because that surface takes a little bit of time. And then what I do is I I will open my video and then my audio. So the ways to do that is there's keyboard shortcuts so that have V for video and alt eight for audio. And I want to I just want to make sure if when you're when you're speaking the best is if you always make sure that your video is open before you open your audio and that you close your audio before you close your video so that it never happens that your video is already off but you're still saying something or somehow it's picking up on something and it's just showing your you know the black box with your with your name rather than your video it's nicer if the video is always there. So I will I will turn on my video of the under my audio alt a and then I will do this opening remarks and then I will conclude with saying my head Hello sky How are you and then if sky if you could switch on your your video and then your audio so be in an old eight And then, so I can control what we see. But probably what we'll do, it's usually good to have a little bit of a transition. So we keep the gallery view on for, you know, a second or three, four. And then you can start with whatever you have. And I'll just nod and then I'll make myself disappear and you become speaker view only. And then once you start discussing with the panelists, and then go back to gallery view,
that is a term.
That sounds great.
So at the end of my, then just to say it again, the end of my intro would be Hello, Scott, how are you? Is that okay with you? Or do you want to
say, I'm great, I have my coffee because it's early for me. I'm looking forward to getting started and then I'll dive right in. Okay, great.
I see now that we have two more panelists. So shall we do this versus Okay, sorry. And then maybe just to finish the run through, then at the end of the tunnel. Sky, if you could tell me, at some point you you hand, the floor back to me. And I'll put some closing remarks. And then once we do the closing remarks, and it's done, we stopped the live broadcast. And we go back to the looping slide sort of means so that we stopped the streaming to YouTube and to Twitter and Facebook, all those things. But we keep the meeting open for participants because it feels a little bit abrupt if you just get kicked out at home. So what we do now is we have this looping slides again, and we just let people leave on their own accord. Okay. Okay.
Hello, guys, from
Hi, everyone. Thank you for joining, just saying hi to all of our panelists here today, and a big thank you to that. To you team for helping us organize all this and running the backend. We really appreciate the operational support and really looking forward to the content of the webinar today. And I think I see Judy down there and Marcus joined as well. Hi, guys.
I did a quick logistical question. I in terms of slides.
I don't have any I don't have anything right. So I just you guys are running with any slides you have for your five minute presentations.
I don't have any slides.
I could show. I don't have any slides.
I don't think they're necessary.
No slides for me. Okay.
Sorry, got my eye problem too. So fast. Nope.
All right, and Michael, will you share the slides off of your computer? Or those? Megan or Bastiaan? Will you be sharing the slides? How will that work?
Oh, are you sure with my computer into my computer? It's okay.
It's just just translates very, very small.
Did you want to just go ahead and practice screen sharing to make sure it works seamlessly from you?
Okay, it's a minute.
Every day so busy here.
Let's it's just true truth lights just a day okay.
And then repellants introductions Just so you know, I was going to commence with Judy, then move to Marco then tell them and my introductions in the interest of time I was going to keep very brief. I was gonna mostly do name, institution title and then give you five minutes to talk. Okay.
Sorry I was on a call when Bastien you were going through the logistics. Did you connect with Skye about filtering the q&a questions to her coming in from the audience?
A good point. Yeah. So we have, I don't know, it was named as you chat and q&a. And she will be trying to curate the list as much as possible. So questions that are answered and answered live will then sort of move to the side and it Also for the panelists, when you're not sort of speaking at the moment, sometimes questions appear in the q&a addressed to you personally. So you can answer those in writing, if you
Yeah, Scott, are you comfortable with that? Yeah, you can just open up the q&a tab and see the open ones if you'd like me to filter them as well, I can do that. But I'm not super well informed on the topic. So I don't know if I'll be filtering the right questions to you. But however you want to do is okay. No, I think I think it's just typically when I run when I've been in engaged in these editing, just seeing the raw q&a has been fine. The only exception would be if we had a huge volume or if if we had any spammers, that created a huge volume of useless questions. Is there spam? That's also what I'm there for. I'll just dismiss them and try and as people answer as well, I'll try to mark As answered live, or so that it goes into the answer category, it's open q&a. So I'll do my best as a non content specialist to filter it. But
and I'll be on the back end. If you do have any questions related to content like if this question is related or appropriate, you can always private message me and I can help you sift through that.
If there's nothing else, maybe we can practice quickly having all the audio and video off and looping the slides and then we can go after that we have to go live.
Okay, great. Thank you, and Keanu, would you be able to dispense please
Okay, so I think maybe just stepped away for a second. So we'll just skip this.
I'm straight to practice. So I'm going to switch on my video my audio and then say the first few sentences of my script and the last one and then Skye if you could then switch on your video and your audio and then we can talk a little bit.
Yeah, it's possible. Sure.
Okay, and and so when we're going live in a few minutes, what I'll do is I'll do the countdown 10 to zero with the last three seconds silent and then and then we start broadcasting and so done. It's important to keep the audio audio microphone off
10 98765 or
then we will go live now
and then three minutes later we would start broadcast.
Good morning, good afternoon. evening. Welcome to the joint webinar and with that I think it's now time for me to hand it over to our facilitator Scott Gill from fine Hello Scott How are you?
in morning good afternoon and good evening it's nice to hear your voice fasting and really excited to talk about image based AI in low resource settings for COVID-19
I so I started trying to start my video but I somehow need to do to activate it but I'll we'll figure it out.
You don't need to but I saw that it didn't work so I prompted you. But yeah, I think the best thing is just actually if you for me at least personally I do this alt v i wait until I see myself on the screen and then I do the alt a and then I start speaking Yeah, just if you know I might be on if you know you don't worry about me being on screen after I've said that for a couple of seconds with nothing happening. That's not a problem.
Okay, okay. So I'm clear is, Is it really necessary to do the alt? Or is it enough to click on the on the Start video?
It's definitely fine to do that. I just recommend that because it works well in my opinion, but no if you want to just do it, but
I can't find the Alt on the Mac.
Right? That's true, actually. Yeah. I don't know how it works. I'm sorry. Yeah.
Okay. So we're about four minutes to three.
Is there is there anything last minute that we need to do?
A question for anyone.
Okay, then. So
let me just to be completely clear, run through the beginning one more time, in one minute. Then we're going to switch over audio and video and I'm going to start The broadcast within zoom. So three minutes before the hour now already basically and then on the hour, I'm also going to start the live streaming to the live streaming services, Facebook, Twitter, all that that takes about 10 seconds to unfold so that all the surfaces are running. So and then 10 seconds after exactly three o'clock then I will open my video on my audio and I will do the opening remarks and then handed over to Scott. Is that okay with everyone? Okay, then I think if there's nothing else we can probably go live. Judy, if you wouldn't mind switching off your video, please. Thank you. Okay. Do you know
and there we go. Okay.
Last call for any things that we need to do before we broadcast.
Sounds a bit low right now actually.
That's better. Thank you.
So a little, still a little bit thing, but Okay, let's let's
Okay, so there we go 10 987654 PP
Good morning. Good afternoon. Good evening, and welcome to the joint webinar between Harvard's data science on AI Summit for health and AI for Good Webinar can image based meaningfully impact COVID-19 response in low resource settings. We hope that you your families and your friends are all safe and healthy. My name is Bastiaan Quast. From the it International Telecommunication you Union, the United Nations specialized agency for information and communication technologies, joining today from Geneva and have the privilege of introducing today's webinar. It was also the organizer of the AI for Good Global Summit, which are held annually in Geneva in partnership with X PRIZE 36 sister United Nations agencies, ACM and co convened with Switzerland the golden AI for Good Global Summit is to identify practical applications of artificial intelligence that can help achieve the Sustainable Development Goals. And like most of the world, the Summit has gone online. We are publishing weekly webinars, solution tracks and keynotes that will allow us to reach even more people throughout 2020. Today, we have a very distinguished panel featuring Judy Sharia Miko, bago elder on the cover, and with a discussion being facilitated by Skype, the panelists will present their remarks what were in particular counting on you, the attendees to participate and create an engaging discussion. For this, we'll be using the q&a functionality, which you can find just left of center at the bottom of your screen. You can also upvote questions that were posed by others but you would like the facilitator to ask. Additionally, there's the chat functionality which you can use to have discussions with other participants. And please make sure that if you do that, that you set the message recipient to everyone, and not just the panelists, you can select this just above the message box. Before we turn to our panel on gladdens, everyone to use the chat functionality and to let us know where you are connecting from which city or country. Please just type it into the chat and make sure you send it to everyone. That way we have an idea who's in the room. And in the meantime, I will just type Geneva Switzerland.
Okay, great. So we have people from Geneva, Jamaica, Denver, Kenya, Sao Paulo, Egypt, South Africa, many more. Well, I'm very much looking forward to engaging discussion with this diversity of perspectives. And I think with that it is now time for me to handed over to our facilitator Skye Gill from fight Hello sky How are you?
And morning and thank you Betsy and or good afternoon or good evening depending on where you are based. It is early for me so I have coffee. I am very excited to walk you through Today and to feature our three panelists, talking about how image based AI can meaningfully impact COVID-19 in the low resource contexts throughout the world. So the way this session is going to go is I will briefly introduce each of our three panelists. they're each going to speak for roughly five minutes about their work. And then we'll have a focused discussion for about 30 minutes talking about innovation, AI and COVID-19. In general, some specific challenges with image recognition in low resource settings. And then at the end for about 10 to 15 minutes, we'll open it to questions from you all. So be thinking about some good questions. And then at the end, I will hand it off to Bastiaan to close. So, introductions a little bit about me. My name is Skye Gilbert and I'm the Executive Director of Digital square at path. Our mission is to connect health Leaders with the resources necessary for digital transformation. And a lot of our work is trying to coordinate and align investment into digital health for low resource settings, supporting digital health software innovators in scaling their products, and then supporting country governments, hospital providers and implementing those health software solutions. So for us, I was very excited about this panel because as software rolls out and scales and you're having large volumes of data and systems, it creates an opportunity to use AI for health impact because of the volume of data being generated, so very, very eager to hear from our panelists. I'm going to go ahead and begin by introducing Dr. Judy, where did I get Oya? judy is an assistant professor at the Department of Radiology and imaging at Emory University School of Medicine. Judy, I'd love to hand this over to you to speak a little bit about
Thank you sky. My name is Judy, I, you know, my sort of my journey has been that I'm a physician from Kenya. I went to medical school in Eldoret in Kenya, so jumbo of my Kenyan friends on the on the chat. And and so I went to medical school in Canada and I went, I came to the US for a fellowship in informatics. And I finally found my passion, which is to think about the rooms that patients receive care whether it's the room is in their home, or their room is in hospital. And so my interest has always been to work on technologies, and see how they can save lives. And I'll start with a disclaimer, I think a lot of technology does not do good, in fact, has potential for harm. And so this also marks my interest in AI Summit. I'm an interventional radiologist by clinician by clinical training and was but I spent half of my time working on on how to make technology safe and how to use that to accelerate care. And so along the way, I am pretty much interested in figuring out when technology fails and studying that, and also advancing Open Open Science by supporting release of data sets and growing communities that can work with AI tools in mine and AI while they spend some time as a community maintainer for an open source project called liberal health, which is used to develop digital technologies for in an open source community for using emerging economies. And so if you have some time, you should check it out. But I'm happy to be on this panel and to look forward for an exciting now. Thank you.
Thanks, Judy. I was grateful hear from you. I I think that one of the points that you made that technology does not always do good is actually a really important one and something I hope we talk a little bit more about in the discussion. Our next panelist is Mr. Marco bago. Margo is the executive director of the chief innovation office at the University of Sao Paulo in Brazil. Welcome, Marco.
I would like to share some slides. But first of all, I'd like to thank the Novartis foundation for supporting our innovation and the intelligence program and the organization for to be invited. Just to mean it's just
it's like a year
yeah. I'd love to say hello from webinar colleagues Julian helden. And the to the audience. I just introduced the quickly the other speakers and then a six minute short presentation about our project is we using AI to fight against COVID-19 as I see the screen because is the largest hospital in Latin America, and then it's connected to University of San Paulo medical school. When I when I work with the radiology Institute and the Innovation Center they see it has been up to more than 75 years and has 3000 beds. And more than 60 research laboratories, any our our in our area, almost 19 nine 8090 million of medical images start very year and it have had the 13 years As of backswing three reports are a huge, huge beta data lake here, and the monitor page and nine 900 beds exclusively for COVID-19 patients here. And this is our broad plot. This starts when our AI laboratory assessing how other countries were we working with the AI, images enemy, we saw that there would be an opportunity to choose ovaries and to classify the likelihood, likelihood of Respiratory Syndrome by COVID-19. And now so to mark the area affected in the in the lung by the disease, but it's clear to me Never isn't true to analyze CT images was an opportunity to fight the dependence. But before we have to promise to solve and the first one is how to convince the doctors to use it is the technology and how to integrate to hospital because most of them are in distant region. bazoo is a huge country. And the technical team at these hospitals are already very busy because they defend them. But if this projects, this premises and we screened the market search for this solution to the right solution, but we don't don't don't find no one and the that met our needs that you need in the first time. So we have had to design a cloud architecture if the possibility of connecting many artists working together that's the idea as ever, so to arrive in this in this in this executor, the name is the project is how to be 19 and that has a cloud backs and that we connect with the banks or directly with the equipment's in the hospital. The hospital, the entering the program, and the packs received the images from the top they have these hosters, anonymize them, and send them to the algorithm platform that we have. Now they're two algorithm work together. And these, these AI, these AI algorithms, analyze the image and send again to the platform they they report and the image is marked the the area affected by the disease. These images returned to the hospitals and they enter directly inside this box. This box for the solve the first problem, we put a set of experienced doctors to talk to the other doctors in the hospice and discuss the case and the issue a second opinion if necessary. And these images that you entered in the in the system, we start down in our data lake here when we We get these images in we are creating data sets and drove our innovation program in August, we made an AI challenge to develop implement the algorithm developing Brazil on the platform, it's the new the new approach the second phase, and this phase was launched yesterday there is a site and then of our foundations is a strongly support those in this process and then your work could be there we are here in the fifth engine. And our proposal if this project is increasing medical productivity using the AI with with the aid of AI, and the standardized process of marking the compromise it alone by COVID-19. Because we know that this these areas change a lot depends on the doctor. But besides helping you to fight a pandemic, it's first as a huge opportunity to create an actor
connected these hospitals in for the scale use of AI in Brazil, and also test methodology and a structure to develop a newer version for other diseases and in the future. And that's the last works. Just finally some numbers. And this platform is start and then 54th fight for me for and the results. So far more down plan 20,000 platform access, we have 21 hospitals using the AI platform and in the hands of 16 inline the true true intern The next day, we have a read analyzing more than 5000 exams and every day 180 new exams entering the platform on average. And that's it I try to briefly convey the idea because the time and the End The End The project I hope I have achieved that. Thank you guys. I stopped in
thank you so much Margo. I think the the topic of AI sometimes can feel very abstract or very complex. And I think a lot of your visuals helped bring to life a lot of what, what professionals are working on day to day. So thank you for that. It'll be interesting, I think to talk a little bit more perhaps about the architectures as well because that's a that's a really important thing to get right. And the final panelist today is Dr. Eldad on the cabinet. That is the Chief Medical Officer of zebra medical vision. Welcome.
Thank you. Good Good. Good morning. Good afternoon. Good evening to everybody and I'm really thankful and privileged honor to be part of this, this panel. A little bit about my my background I I trained as a as a radiologist and and so Judy, as an interventional radiologist and I practice clinically as an interventional radiologist for about half the week. And then about six years ago, we started a company called zebra medical vision with the goal of being able really to distill radiology knowledge, clinical knowledge, as it's presented in images and reports and translate that into something which becomes transferable to to other to other stations to other to other radiology stations, and in a way to be able to listen to sort of the insight of thousands of physicians who have reviewed imaging, and when they're not around. And, and, and really, in the setting and in what we're seeing now, I think that's even more important. And, I mean, there's nothing more important than that. And in the title of this talk, the focus of the summit of this of this discussion is around the use of AI and low resource settings and I think in Some, in some regards in the best sense, AI can offer something that that distributes and resources and insight and across countries and across and across the two places where there are poor settings. And if we look at what we have here, we have very, very kind of resource intense and very dedicated people and generating tagged datasets with a lot of collaborative efforts across disciplines to generate software, which then might be very useful locally, but also, but but critical in places where there are a lot of people sick and there are very few physicians, and often very, very few radiologists to to make the findings in a time sensitive manner. And so that it fits really well with zebras vision and our clinical goals. And I'll just as a company, you know, we we've we've been able to To, in a sense, get a running start, because when we first saw the images of COVID 19, on CT, the radiologists in the group would look at it and say this, you know, this looks like we could characterize this will look similar to, there are some, there are some features, you know, crypto capture, like pneumonia, and you know, other features like ground glass opacities that we said we have those, we can identify those. And so we could actually begin to develop feature recognition software before we had really adequate amount of typical COVID-19 datasets. And then with those data sets, take it a little bit further. And then also do something which we like to do with zebra which is just to see as much of the picture as possible. Meaning that we're not just looking for COVID but it's relevant to look at how much emphysema the person has. And if they have a lot of coronary vascular disease, what their liver looks like. All this stuff is available on a CT image on a CT scan. And all of that is relevant in terms of persons reserve. So I think it's given us an opportunity to really remember why we're here. And, and it's it's a really encouraging and it's it's great to see the contributions and the effort that I see even you know, right here, and certainly globally, working to working to bring a good resolution to this. So again, thanks, everyone, and I'm looking forward to hearing questions and more of this discussion.
Thank you. Well done. Thank you, panelists. I think that I like the vision of using AI to improve the distribution of resources across different settings. I think the promise of that ticularly where you have shortages of skilled professionals could be could be very powerful. We're going to move into questions now. And so my first question is for each of the panelists to respond to, and it has to do with myth busting. So a favorite show of mine growing up was called Mythbusters. And in each episode, the hosts would take a common belief that most of us have, and then use science to determine whether that belief was true or false. And I just like to know from our panelists, if you could bust one myth about artificial intelligence, what would it be? And maybe we'll work backwards in order from the introductions and I'll come to you first of all,
I'm just one. Okay.
Look, I mean, I this is something that I'm so one myth is that it's intuitive.
It is it's intuitive.
And as much as, as we, as humans look at a picture and or look at anything, and we think we really, we know we're looking at, and we feed those pictures into it into into a software. It's there, there's it's always humbling to recognize how much how many assumptions you've got to look at and how much ironing of assumptions you need to do to be sure that what you're presenting is really what you want a machine to recognize. And so I think, again, the the the myth is that it's intuitive. And it's a matter of taking what looks to us to be a good data set without many questions and just feeding it into some sort of pipeline and ending up with
I definitely think the more I learn about and see How AI progresses the deeper virus stack for the incredible processing power of the human brain. I, let's go to you next Marco, what is one myth that you wish you could bust? in AI?
I see a question that I always hear is whether I will make a medical diagnosis and replace doctors if you get everybody everybody knows that they hear the same questions. And because I'm an engineer don't have any conflict with this question. This is from me. And I think he in this case, it's my opinion, okay. In this case, I would say that it's it's not at me that the AI system make the ignores along and I think that's the idea with the either the forecast and the probability, but it's nice that they actually will make decisions alone. I think it's not to happen. And I think it does. It can't say that the doctor will have the final say but with a powerful decision system support i think that's that's the idea of AI and they actually improve the law citizen and precision but these AI system make decisions in place of doctors i think is from it's not for now and I think it's probably is a niche now but there are a lot of Mythbusters in Mythbusters that no no but my opinion I think you know, it's an AI not not to not to replace the doctors in the in the final say, that's it.
God, how about your top myth for busting
so I'm in Morocco beat me to the punch here that was a gun with a I will replace radiologists and obviously I'm biased because I'm I am a radiologist. But I'll go to it. Ai is better than doctors. I think on almost every paper is reported AI beats doctors to pneumonia AI beat doctors to I'm pretty sure we'll have one for COVID if it's not there that I haven't seen yet. And so I think it's the question is when you read these webinars or when you hear these claims, you should ask better than which doctor and and so if you think about how we train AI, it is only a cross section of time most, I mean, less than 6% of papers published in 2018. had even prospective validation. So you pick data, you decide, gee, I'm gonna look at COVID I go to my PAC System PACs is the imaging archive database, and I pull up all the images and label them and decide whatever label that I want to use this and, you know, train this gory thing, the bigger the GPU resources you have them, the more grass, even bigger network you want to train and then you publish your paper maybe on archive or wherever, you know a peer reviewed journal and say that a bit doctors. And so when you read this this is very concerning because when I was training through radiology when I started my residency, the thing that I struggled with most as a first year was figuring out even the type of phases and how to organize my imaging for you can imagine if you have photographs, and you just want to organize them beautifully so that you can tell a story. That was what I I really struggled with. And then the next step when I was coming closer to completing my fellowship, what I cared most about was, could I measure these tumors and evaluate them in a quick way and effective way. And so then now as an interventional radiologists, when I look at images, I'm figuring out how am I going to save time so that I'm not using up so much video And keeping my kids shot. And so if you think about my evolution, ai doesn't grow like that, at least not today. In fact, we're funded by the NSF to sort of figure out how to train and human beings together so that AI can learn. And so when you say it's better than doctors, you have to specify better than who you better than the first year Judy resident, are you better than the interventional radiology,
And so, this is a myth, and we need better ground truth, and better benchmarks for comparison, before AI use is useful.
I think that's a that's a really important point, Judy and Marco around whether I will replace doctors and it sounds like there's a really big gap between the state of the evidence and some of the headlines that you see in in some of these papers. And I think I'd like to come one of the converse in conversations in the last few weeks. I was struck by how I elda you laid out what a good problem statement might look like for AI. And I'm wondering if that's something you can share, because I think that there's lots of energy around applying AI to many different kinds of problems. But I know that you have effective on some places where it might make more versus less sense.
And, sure, I mean, and to touch a bit on what what Judy and Marco were mentioning, and ai ai may be better in finding one thing on an image. It's it's just, I get what is what's what's what's really here, the thing we expect of radiologists and of doctors is to see everything and the challenges that we tend to focus on on a few things. So if there's a woman who's who's in the hospital because she has right lower quadrant pain and the radiologist finds the appendicitis That's good. But if no one really expects them to also diagnose osteoporosis and to also diagnose, and fatty liver or other or to really see every, every evidence of it enlarging aorta. And so when I say I say this for is that our concentration and what we focus on is limited and a good problem for AI to focus to to come into a good problem statement. Our goal
is to help us to notice what we overlook
systematically. And what I mean by that is that generally, radiologists were really good at finding acute things and single things that were less good at making a global assessment and assessing for chronic conditions on every scan. And I think one of the greatest contributions of AI will be in, in utilizing the kind of ubiquitous the tremendous amount of imaging that we obtained in order to conduct what we call opportunistic screens, the ability to To find chronic conditions then to assess them before they become acute. So that that's one, I think, useful way that AI will be manifested in medical imaging.
thinks about it. And you had a shout out in the chat on how you are sharing, I saw someone say that the icy brain knows. And I think that's very powerful. And I'm wondering if you if you have a good problem statement. And the next question is what you know, is you're compiling your data set, what does a good data set look like to help for AI and for addressing the problem? and Judy, I wonder if you could speak to that.
Thank you for this question. So we are actually working on a data set. This is not for from Kenya. But here in Atlanta. We are working on a data set for COVID we have Right now we have around 1800 chest x rays. And I haven't looked at the city's content, but, so looking through the data set one is it depends on your final output, right. So, if you want to first of all is determining the labeling pattern. So you can imagine we have the dictation of the reports. If you look at the most recommendations for the radiology societies initially, you know, people are not and it's true, you cannot look at at least as a radiologist I cannot look at as checks test X ray, and say that I this is coffee to say that these patterns can be seen in coffee, you know, and sort of just the style of reporting. And so, you can think about labeling for this data set. One could be just looking at their the radiology report, you can think that the second one which and we're doing all three is the second one is you are you just open the channel. x rays, you provide labels, I think for this x rays, the most notable labels are the ones done by check spot and the NIH data set, which are the 14 labels for for diseases, and then you add some about your probability uncertainity around copy. And then the fat where we are looking at this is looking with relation to the testing that's done with a, you know, we basically have the swabbing here for most of our patients. And so because we know that I have seen some of these x rays that are actually normal, and so you have to determine how you're going to look at your ground truth and how you're going to provide that. And then the second thing is if you're going to release this dataset to the public, you have to think about anonymization and anonymization. You know, this fix a lot of anonymization. And then if you want to release the clinical data set, you have to think about that too. And so, one one thing that we discovered going through imaging is that we have pixel level annotations, which is someone put like a date of the exam in the radiograph. And that's, you know, that looks trivial to the human eye. But for AI, that's completely possible, you cannot release that data set to the public. And if you end up, like putting a box around those labels, what we realize is that the AI algorithms on you know, this day, they have a feat and they start to learn, you know, every time I see this box, this is from Emory University. And I know that this is the rate of the rate of maybe coffee among this data set. And I can cheat because we know that these systems do this. And as has been documented by many authors, including one for detecting pneumonia, which just glands which are the ICU radiographs by just looking at the amount of wires and the label at the top. And so data preparation is very difficult. And so obviously, you could innovate internally by bringing that core thing To your enterprise, and running them internally, and then sharing up what the performance is. But if we are going to make useful AI, I think we have to spend a little more time on the data. But that's sort of the thought process, at least, from a practical perspective about the challenges that we are facing right now.
Thank you, JD. So you've got, you've got the complexity of searching for the right problem statement. Then once you have it, you have a number of challenges on getting the right volume of data on getting it secure, getting it anonymized getting it prepped for AI, and a lot of complexity that Judy outlined. And then when I think back to your slides, Marco, on top of all of that complexity you had on your slides, partnerships with Amazon Johnson and Johnson, Novartis, Siemens, Huawei, and so it looked very, very complicated and, and so my question for you Marco is why is there So much partnership and collaboration in the AI field. And well, if you didn't have all those partners, would it? Would it be simpler? Or? Or would it be impossible?
Skype is not easy to connect in all these these companies I think it's is a is a strongest process budget but in the first the first thing you need to connect it and clearly all the interest of which you are I think that is the first idea. But in end you'll have manufacturers to collaborate. I think that's idea you connect people and companies now. I'd like to, I think the two factor that the professor's gj and how that said it's very important to the motivation of the connection of companies and institution. I think that The first one, the data's I need a lot of data's and because of this you need the Amazon and Siemens and, and, and GE because they can't collect this data first. And I figured we have erosion as data organized. I think if the character is mentioned by by Professor, Professor judy is and connected this is very expensive, expensive and difficult. Therefore, calibrating that data exchanging is a good is a good possibility to try to minimize this problems if it is. And when you think in data is easier you explain for these companies what you need and what you can do together. And I think we have the other other condition for a business sister to be really effective is according to to hell. They think that that's the the real, the real. Changing when you AI, you must analyze several plumbers at the same time. That's the end. And it occurred to Elena and it's true AI using analyze one problem it in the same moment, because it is a second main algorithm and work together are necessary to to do these things. And for that it's easy work, we should partnership, then to develop and deploy everything along those effects thinking. And we I doubt burner sheepy would probably be his lower Mark expansively. And then for a few users, that the when you connect to more companies and more, more institutions, you can spread this the idea that the developer
AI seems to me to have a more adherence to a business business ecosystem, competing with another ecosystem because Do I think you can connect to the company? Then with companies competing with each other along? You know, I think the idea is the ecosystem working AI. I think it is. It happens. And I think it's sounds and works good. Then we we still on?
Thanks, Martha. I think I think that you know, to solve some of the challenges you highlighted, it sounds like there's some really key areas where where those partners bring values. So some of that complex navigating and brokering of partnerships is worth it. I wanted to turn to talk specifically about COVID-19. And ask each of you since COVID-19, became a global pandemic, what's changed about how you work And and perhaps Judy, we can start with you.
Thanks guy for this question. So, you know, I'm, I'm pretty much involved in direct patient care because I do procedures on patients. So really for me what changed was the walkthrough, bring in patients to do procedures for them. And but in the context of this talk, I'll mention something that I noticed that changed for the workshop that ideology is right. And so everyone is very interested on focusing on you know, the images but we forget that this the cameraman who goes to the field and takes these amazing pictures for you to walk on AI. And so we saw a lot of people move back home as a sort of to remove the doctors from the hospitals and most of those people were actually radiologists because as long as you had a good internet connection, and a home workstation that most people could actually work from home. So, we saw that and also to decongest, the working spaces so that people do not have to be too close together to to, to reduce the risk of person to person transmission. And so, but the person who sometimes is a little forgotten in this journey is the technologies they address on technologies and the CT technologists who are, who are in this patient's room. Taking pictures, we know that for just taking a CT scan during coffee, it was extremely and remains very difficult because for our institution, you had to bring the patient, the room has to have a terminal clean. And so you're talking of a turnaround time of around two hours just for for imaging. And so you have to ask yourself, do you really need imaging? Is this going to change your management? And so if we saw myself going into the patient's rooms to do the procedures in their rooms with ultrasound, because bringing them well first of all, you have to figure out what hardware you're going to bring to bring them in. Is it Who you know, are you going to make sure that people are clear, and then the room that you have to use will go down. And so I think I think this is a good reminder specifically for not just the technology itself, but to think about the impact and the people around. And if you if you can have a look at how most, quite a number of companies have always sold their product is saying will make you faster. But I don't think for AI for coffee is fast is what's needed. There. The rate of cameras and pictures that can be taken is already low, because patients don't want to come to the hospitals unless they need to. And they just the therapy, the throughput of the machines is gone. And so we need much better value return more than just seeing you faster. And so that's personally how my my life change in terms of work.
God I think that's a really important insight that even if AI were removes risk for certain populations or enables you know remote remote work for some that in other contexts and particularly around data collection. It can create more risk and and potentially some of the work can also increase cost and so it changes the ROI.
Marco How have things changed for you since COVID-19?
sky I'd say that's the biggest change was a speed up everything saddling, got faster sampling and the seizures approvals than they need is everything. And for example, what you did here in the last two months, probably under normal, normal condition would take at least six months I think that the velocity is so high and Because I think the other 10 but connected for Dan is St. Jude he said in a part of the speech is the use of electronic communication and collaboration tools and they're practically practically part of the everyday life now and in many cases I believe it probably they stay here to be they are here to stay this change Can I Can Can I imagine it I think this choo choo connection that the the speed and the technology I think it's enter a Latina in my my day my my schedule was full for at least two months and I probably in the next month or two. I think they that's the big change besides the other the other chains and then the social distancing guys like that.
Thank you laka I was asked about you
I can I, I have some resonance with with Marcos sentiment of experiencing
collaboration and, and, and less bureaucracy and and kind of Route and cooperation on the one hand. So the sense that you can get things done and people and you can get things done because it's an important time to get things done. On the other hand, a, you know, overall sense of, of bit of bewilderment. A lot of things are securities that we took for granted, we're just not there. And we didn't know. And, you know, we had we had a whole whole IR section of a different hospital that couldn't provide services because one of the One of the nurses was sick, and everyone for two weeks was gone. And so we split the team into two in case one of us got sick. I mean, these were some of the experiences that kind of, you know, revealed and the facade of security that we live under, we have to live under on the other hand, there was real action and progress and both in the hospital and also for zebra as a company.
And if I'm honest,
you know, if I'm honest, I think the biggest difference was that I was doing a lot of work, trying to do work, not doing work because my daughter wasn't in school. So it was just a lot of time. You know, like that. I think that's I've experienced a lot of people have of having to having to having to do it all for and prioritize. And I think it made us a lot of I think you've made most of us very efficient and focus on one or chief
Thank you. And my final question before we turn to the audience questions is specific for low resource settings. And I would just love for three of you to share in your mind, what are the what is the biggest opportunity you see right now to provide value in low resource settings and addressing the COVID-19 pandemic. And coming back to Judy's point that technology is not always good. What is the biggest risk you see?
And Marco, why don't we start with you?
Michael, can we start with you?
Sorry, okay. I have a problem with my My camera in the microphone here.
I think that the risk calls I'd say that it
is a is a is a
like a challenge and I think AI based systems
will be to reduce the cost of the health care and not increase it. I think it's it's a risk that you have because I think it is to stop the existing trade off in Brazil it's it's common in healthcare today you add in new technology and always this How is this technology this technology increasing costs in health care in the in the equipments and in the service? I think that's the biggest biggest risk that you need to to are they are the challenges and that you have in this moment in issue to use the AI the tech analogy that technology should reduce the cost. And I think and there are other other risk is according to Judean and how none of that data is now you need to to have the using data as any you can to connect to all the data that you need and the faster and because they are an issue to stay in compliance with the privacy laws and the regulatory I think that you need to think about it if you have
the best the best laws and regulatory
the regulation and that you have to to to improve the technology and improve the is faster and then
I think it's
I think that's the I think the other the other idea is to I think you can use it the moment it's an opportunity and a risk and opportunity if you don't, if you not use the moment you choose to introduce the the technolon the technology in the medical practice, I think that's a good idea of control. However, they need a new you have new needs, you have new problems, and it's an opportunity to use the technology to change the status quo in these moments. I think it's a risk that you need to to to fight down and and promote the change that we need. That I need to I can thinking in this monitor sky
QKD would you like to go next
So, you know, I noticed the webinar is about imaging, you know, imaging and specifically so when I hear imaging, I think more, you know, computer vision in terms of AI, and COVID. And so you essentially talking about radiology images. And I just think that chest x rays, I don't know that there will be much value in terms of specific thing that you can use this for diagnosis, maybe more things will come up on the spot, just seeing what I have walked on and seeing sort of testing the algorithms that are out there. Performance is really bad. And so I think maybe CT scans and maybe getting other markers or quantifying disease. The other interesting questions that you can ask from this, I think we can see some of those on the charts. Do you know people asking these patients more regularly or something like that. And so this power of AI to analyze large data set and also across multiple institutions, but also in areas where we are blinded to, as mentioned earlier, I think is a potential for good. I think that, you know, recently I actually asked my mother, what, who lives in the village in eastern Kenya and asked her what she thought about AI. I actually asked her this for a presentation I was doing. And you know, we've never talked about this and I asked, you know, about something similar, which is m pesa, which is a mobile transfer money transfer system widely used in Kenya. And as this changed your life, so what would be that changing? What will be the M pesa app for you? You know, and still, I think it made for an interesting discussion because I know that these technologies do not always get to the last mile. The people that need them. And so this just reminds us here, that when we design and build these technologies, we have to understand the context that they use. And so always assuming Africa as a country more than a continent, I specifically am speaking about Africa is a way to fail because we were so diverse and so different. And you have to understand that I see a gap in lack of regulation, specifically on sort of like, what's the gold standard, and what that means specifically for like, licensing, it's very difficult to just come up on the industry's this system in in the US system in there in versus just putting up something in Kenya and going ahead and using it, I think it's a little easier. And so you have to hold yourself to the standard that if you're deploying or developing these technologies, that you want your mother treated with them, and hopefully you love your mother and that you want to Do do them with the best standard, if not even a higher standard, especially in areas you have no, there are no dis protections. And so I see the, you know, I don't honestly see the rapid scale, you know, the the rapid acceleration, I agree and in terms of partnership has very, very big especially where there are many regulatory bodies. But going back in places in the emerging economies, we know that you can bypass these regulatory issues very easily. And that has potential to cause harm. And so it requires like this good citizenship to make sure that we are good stewards for our patients and for our families.
Thanks, Judy. There were some questions about regulatory in the q&a. So we might have a bit of at the end come back to regulation. I'd just love to hear your thoughts, opportunities and Add risks.
And I think I think I spoke a bit about the opportunities and before and the concept of basically distilling, you know, if you distilling the insights of physicians in AI, whether it's in imaging or elsewhere, that becomes a resource which can really be potentially distributed Well, in resource poor settings. But apart from that, really, I think that Judy and Marco has had a lot of insightful to say, I don't have much to add to that. And I really do agree very strongly with what Judy said, about regulation, the responsibility of anybody working in a place where there's a lack of regulation is lack of regulation because there's lack of resources. It's not because people don't care. And so you know, you it falls on your on your shoulders to make sure that what you're delivering, you know, is is valuable and is safe.
So, thanks for that.
So I think
our first question is that I wanted to come back to is is actually the one on there was a great question on regulation and, and just bear with me a moment because in the open question set, I'm just trying to find it. And let's see. Here we go. It's from TJ, Missouri. Thanks to all the panelists for participating in and it for creating this event. I'm interested in the effect of current and anticipated regulation across industries, borders, etc. On the purity or intentionality of a problem statement. I in this is asked questions. I think panelists if you want to respond to the question, just go ahead and activate your video. And then and then you can go ahead and start speaking and if there's two of you, then I'll call on one
can go ahead and tend to respond to to Misra. And so I mean, I don't think anyone of us knows how the regulation will change. But we can know what are the impetus for tension regulation. One I don't anticipate apart from Europe and the GDPR. I don't anticipate that we will have this sort of block regulation. So for example, like across the African country continent, and I think it's going to be very country specific, but we know things that accelerates regulation, and one, we see sort of like how the FDA is struggling with sort of regulating AI. One is, if you're regulated as a device, then you're saying, gee, I don't want I don't want you to keep changing fundamentally. And once you submit your test dataset, which can be just in in a number of 10s of thousands Then and we'll give you approval for these economic big changes around this. And so those, those type of regulations are very, you know, they can be very difficult as if you're on this panel under a technical user or develop of AI, you know that. I mean, like the GPT system for for terminologies has just come up almost every couple of months, there's an improvement. And so it's very difficult to figure out how you're gonna deal with software updates, or how you're going to deal with distribution or shifts, which is where data changes because you have a new system, or right now, every pneumonia we're seeing is covered by once everyone starts to get into hospital or we go to the fall, and we'll see new types of pneumonia, that the distribution of the incidence and the prevalence of disease changes in among your data set. How do you deal with that? So there's this post market surveillance that I think we're all trying to figure out how to deal with that and it's not necessarily regulated. And so I do think that things can that can accelerate regulation is harm. If something drastic happens, we will see faster regulation, but I'm not so sure that, you know, I think those are related question about the Ministry of Health that we are ahead of this or thinking about this and figuring out how to work on these types of regulations, especially in low and middle income countries.
Thanks, Jay. That was an articulate answer to a very tough question. I think the final question we have time for today as three boats it's from Antonio demin, Deena Celli and ties into some things that we've talked about, but I think I actually could be quite interesting because it's very specific. It's the COVID-19 is typically diagnosed with a test. How is it superior to you Image Analysis of lung scans instead, if the answer is that it's cheaper, more accessible to remote locations, what is the comparison of sensitivity specificity with a test versus diagnosing from lung scans? So I think this harkens back to is this a really a good problem statement? And if so, what makes it a good problem statement?
And I can take a shot at this. And so there's the the early papers from China that were interesting in that they looked at, at the CT scans of people who presented symptomatically, but were negative in terms of PCR, and found that there were a significant portion of people who had classic findings on CT, but were negative and then it had repeat exams over the course of a week and 90% of them converted to being positive. And the overall sensitivity on that if from that study was that it was that the sensitivity well, depending what you call the bolts are the gold standard was the was the PCR and But it was a PCR with symptoms. So if someone was asymptomatic, then it was three pcrs over the course of a week. And the gist there was that the imaging was superior in terms of sensitivity and specificity. It's a challenge to make that statistical analysis because of the bias. And when you actually perform imaging, but what I will say is that, you know, you're not going to get a CT, certainly a CT scan on everyone that you think may be sick. Some of the advantages are that it's, you know, you kind of have a sense right away. The other is that you there's other data apart from positive or negative in terms of the study. So the evidence of other comorbidities, certainly lung comorbidities, and cardiovascular, that's important information and may be important for triage, both in terms of the scope, the prevalence, the the magnitude of the actual disease covered and also the magnitude of underlying disease. And I'll leave it at that have more to say but I think that that's my brief answer is that there's this imaging is very rich data. It's not binary. It is immediate, but it's also expensive and not all that available. And, and that's, so those are the pluses, the advantages and disadvantages.
Thank you, dad. I'm mindful of time, although there are many other great questions. So I encourage you to connect with the panelists on LinkedIn or with each other. Because I think that there's a rich discussion that we've started here and and thank you all for participating and engaging and having such a rich dialogue. And many, many things Judy, Marco, and oh, dad for your wisdom and insights today. I'm going to go ahead and hand it over to Bastiaan for closing.
Yes, thank you very much guy. And thank you. Big thanks to our panelists and the participants as well from making this a very engaging discussion. And before we wrap up, I would like to highlight a few things that may be of interest to our participants. First of all, together with the World Health Organization, the I see us for the past two years been convening the focus group on artificial intelligence for health, which is an effort to create a benchmarking framework for the accuracy of AI in health diagnostic AIDS. And this also in particular interest to the restaurant the issues that were raised just now on the discussion, but especially is open to everyone. And we are looking for experts to contribute in a wide range of topic areas such as retinopathy, tuberculosis, and many others. We are pasting information on this in the chat as well as a link that you can use. This Thursday today after tomorrow, we have a webinar on AI driving digital divides and the future of African economies, followed by next week's webinar on Wednesday, the 24th, the global dialogue, dialogue on eSports and we're also posting this information in the chat so you can use those links to register for the webinars. You can also find information As well as much more background material on AI for Good at IMT. And with that we've reached the end of this webinar. I would like to once again thank everyone involved, our panel, our participants, our partners and sponsors and the CO convener Switzerland. Thank you very much and hope to see you on Thursday.