Loading...
this is, this is the standard about two years ago, in terms of using scenario based approach to really look at two things, two objectives. One is to look at what are the behavior, basic behavior, competency. Can these vehicles doing some of the basic things like car falling, Lane changing, navigate through roundabout and unprotected left turn and things like that. Second, what are the safety performance in comparison to human drivers, particularly in terms of accident per mile and things like that. Okay, so there's two objectives. However, this international standard doesn't really give you anything related with how we're going to do this. There's no procedure, there's no recommendation in terms of how we're going to do this, and so, so that's why we we've been working on this and cities Safety Assessment Program. We're trying to provide a framework for the industry to to adopt in terms of how we can help evaluate the safety performance. And we have two parts, one where I call driver license test. Well, that what I really means is that we want to make sure these vehicles has basic behavior competency before they can put on the road. We have seen from our own test track. By the way, I'm sitting it's a test track with test autonomous vehicle every day. And we have seen some of these vehicles on the road being deployed to our lace, and they failed, in some of our cases, basic behavior competencies. And so the second thing is that we like to see how these vehicle perform in comparison to human drivers in terms of accident per mile and and then things like that. And so there's two parts, in terms of our framework, how we evaluate the safety performance. And so both of
these two parts, in terms of our framework, really build upon two things. One is we want to build these naturalistic driving environment models so that we can realistically evaluate the safety performance of these autonomous vehicles. And that's not enough, because then even in simulation, you will need a really long time in simulation model to run to get to that point, so we have to somehow to accelerate. And so that's why we call it naturalistic and auto zero driving environment, so that we can accelerate the testing process for autonomous vehicles. And so that's essentially our framework. We're trying to build the model for Netflix driving environment. We're trying to accelerate this evaluation as well. Okay, so I can't show you in terms of our commercial testing, but we do testing for open source system, and all the wear is open source autonomous driving system is openly available online. And so we tested our latest work on all the way our universe. And so we do two parts. And first of the number one is looking at their basic behavior competency in terms of driver license test, we test 14 of the scenarios. And then if you
look at, in terms of, out of these 14 scenario, almost seven of these, they failed in some of the scenarios, okay? And so these are just basic behavior competency for these, for this particularly open source automated driving systems. And so that's the second column. In terms of pass and fail. You see number, your number of fails over there in terms of some basic behavior competency. Okay, we also so this is how we do the do the test. We test this in both the test track and also in simulation, as well, as you can see on your left hand side. Not sure whether I can speak. Let me see whether. Okay. The right side. It seems working. If you can, if you can click on the left hand side, this is how we do the real test in Michigan. You can see snow as well. And so autonomous vehicle crossing the intersection. At the time, there's a background vehicle doing the left turn. We want to see whether autonomous vehicle will be able to handle that. Obviously, you can't in the if you use a real background vehicle, you can't really do test these dangerous scenarios. So we that's why we on the right hand side, we test this in mixed reality. So the background vehicle is actually provided by through a simulation model. Okay, we also do the driver driving intelligence that we want to look at overall, in terms of accident rates and things like that. We look at in terms of how run the simulation model until they convert. And this, this is some of the situations. Let me see whether, whether you can click on any of these. Yeah, sure. Sure. Just this one, yeah. So you can see, in this type
of situation, the red car is the autonomous vehicle. Blue car copy in and and so the autonomous vehicle actually slowed down. That lead to a rare end accident, okay? And so these are the things we are testing within the simulation to see in terms of accident rates. And then there's multiple type of situations, and I don't think I have time to go through. Okay, all right, so that's my sort of my last slide, why we need a national framework for autonomous vehicle testing. I want to say three things. One is for autonomous vehicle, because we are evaluating not just the vehicle systems. We are evaluating the vehicle plus AI based drivers. Okay, so we want to move beyond from the functional safety, which we do for all the automotive industry, we do functional safety testing, we need to move beyond that into more behavioral testing from third party, from consumer perspective, how these vehicles perform in a naturalistic driving environment. The second thing I want to mention, we need to move beyond from the so called deterministic safety to more probabilistic safety. What we really mean by that is, if you look at these autonomous vehicle companies, in fact, if their safety performance is being validated, they are doing better than human drivers. We should help them to commercialize to scale. We should not penalize them because one single failed case, so that we that's what we were arguing. We should move beyond this deterministic case based reactive investigation into more probabilistic, overall proactive type of licensing and helping them to scale and to scale, to commercialize their technology. I think I have one second. That's my last. Thank