Service and Maintenance in ADAS & AD Vehicles
Welcome to the LeddarTech Lab podcast, where mobility innovation takes center stage! In this episode, we explore the unsung hero of advanced driver assistance systems (ADAS): service and calibration. Joining us are Thomas Patterson of GlassDoctor and Eadan Valent from LeddarTech. Together, they’ll unpack the intricacies of sensor calibration, the challenges of maintaining ADAS performance, and why precision is critical for safety and reliability. Whether it’s hardware, software, or cutting-edge AI, this conversation dives into the collaboration between tech and mechanics shaping the future of driving. Buckle up for a ride through the world of smarter, safer mobility!
Podcast Transcript
Michelle: Hello and welcome to the LeddarTech Lab podcast for all things mobility, ADAS, AD, and the tech and trends driving the automotive space. With more sophisticated ADAS applications becoming more prevalent in vehicles, ensuring their accurate performance through service and maintenance is critical. With us today, we have Thomas Patterson, Director of Product Development and Technical Training at GlassDoctor and Eadan Valent, Algorithm Group Manager at LeddarTech to talk more about this. Thank you both for being with me today. Welcome.
Eadan: Thank you, Michelle.
Michelle: Looking forward to getting into the conversation. Before we do, can I ask you to give us a brief bio if you can, starting with you, Thomas?
Thomas: All right. So my name is Thomas Patterson. I am with GlassDoctor. We’re a retail glass company. I’ve been in the glass business for 44 years, and I’ve been involved with ADAS calibration since its inception six or seven years back. And hopefully, I can add some valuable input.
Michelle: I’m sure you will. And Eadan?
Eadan: Sure. Yeah. So I’ve been working at LeddarTech for six and a half years now, currently managing the algorithms group. So we’re in charge of the AI. And I’ve been in this field for roughly eight and a bit years now. It’s made some big leaps and bounds in that time. And it’s an exciting industry to be a part of.
Michelle: So let’s start off here. When we talk about maintenance and servicing of advanced driving assistance systems, we’re referring to the hardware and the software of the system. The hardware comprising the sensors, while the software refers to the perception system. How does the perception software use sensors? What sensors are used and what role does calibration play?
Eadan: Yeah. So maybe I’ll even start one step before that. So what is the hardware and what is the software in an autonomous vehicle or an ADAS system? So imagine a human driver. We have our eyes, our ears, our senses. Those are the sensors, the digital twin of that. And then we have our brain trying to understand what’s going on around us and deciding how to drive. And in this case, we at LeddarTech, we work on perception, meaning we’re trying to understand the environment around us. Now, the sensors that we’re working with here are cameras, radars, lidars, IMU, GPS, CAN bus. So many different sensors. Maybe two of the most important ones that we usually work with are cameras and radars. So we can just kind of discuss those as an example.
So we do calibrate these sensors, and I’ll explain the two common types of calibrations that are being used. One of them is called the intrinsic calibration. This is commonly done on cameras. And what we do there is we basically learn the internal mechanism of the camera, its perception of the world. We want to get maybe some parameters that help us understand how the world is projected onto the sensor. And this is critical if you want to project any 3D point in the environment onto that image. But the other calibration that we need in order to do that is the extrinsic calibration. So this is not an internal parameter of the sensor, but an external one. And what do I mean by that? We want to know exactly where the camera is positioned in respect to the vehicle. So if we have several cameras, each one of them is going to have its position and angle in respect to the vehicle as part of that extrinsic calibration. And once we have both intrinsic and extrinsic calibrations, we can basically share information from different sensors and know how to project them on one another. So this is really critical for multimodal raw data fusion, the way we do here at LeddarTech. And that’s the engine of our perception stack, actually.
Michelle: Similar to the mechanical systems, ADAS applications also require regular servicing and maintenance. So Thomas, can you walk us through the different types of servicing and maintenance required of ADAS and AD systems? Why are they important? And typically, how is this routine maintenance conducted?
Thomas: So I am far downstream from Eadan and his group. So when you’re getting into layman’s terms in our world, I am basically verifying the existence of those components, their position in the world to make sure they’re correct. So we run what we call pre-scan system of the system to make sure that all the components are functioning correctly and where they belong before we do anything. At that point in time is when we will do what we need to do and then perform a calibration to reposition those and recenter those cameras. Where Eadan is doing intrinsic and extrinsic calibrations, we’re doing dynamic and static calibrations where we’re realigning those systems based on either a moving environment or we’re moving the vehicle through the environment, one of the two. And at the end, we’re just running a verification test to make sure they’re running functions correctly. We don’t maintain, perform maintenance on those particular sensors or cameras themselves. If we have an issue, we replace them as a component.
Michelle: What are the most common triggers for ADAS recalibration that repair shops should be aware of? How is maintenance of ADAS systems different from, say, a chassis repair?
Thomas: We basically go to the OEMs and the manufacturer and every OEM on earth puts out what we call a position statement to indicate those times when they want their system to be realigned. For instance, if the camera has been removed for any reason whatsoever, the optical camera from the windshield, it needs to be reinstalled and recalibrated. If the windshield’s been repaired or replaced, it should be recalibrated. If the vehicle’s lane keep assist or adaptive control system has been indicating a problem or flagged a code, we say, it needs to be calibrated. If it’s been an accident, if you’ve changed the size of the tires, if you’ve had a vehicle alignment done, there are a multitude of triggers. But bottom line is, if you think about it, if you’ve done anything to the suspension of the vehicle or if it’s thrown an error code, it should be recalibrated. Simple as that.
Michelle: Eadan, in the last question, I asked Thomas about triggers, but I want to ask you, how does the perception system recognize issues with sensors? What are some of the common types of issues?
Eadan: Yeah, so there’s many types of issues. Some of them are related to calibrations. So for example, a camera gets hit or a radar, you get in a fender bender and the radar shifts. And some of them are actually sensor issues that are regardless of the calibration quality. So for example, let’s say you have a camera and some mud gets on onto the lens. So this would be the type of issue that the perception stack would need to be aware of. For example, in the mud example, if you’re driving ahead and you want to know whether the traffic light is red or green, if the camera has mud over it, you don’t see anything, the radar is not able to pick up on its own. And that would be a dangerous scenario. So for each different type of issue, whether it be calibration, sensor malfunction, abstraction, like in this case, you would need to know how to manage it differently. And it’s a per-issue basis. And one of the biggest issues with large scale ADAS features is that if you have millions of cars on the road, things like that happen on a daily basis. So the way that we manage that is we actually train our system with all the different situations that we can imagine. We both tried to record real data with such issues. But we also find ourselves simulating a lot of the potential issues and training on simulated data of this type as well.
Michelle: So Eadan, LeddarTech has vehicles on the road demonstrating its ADAS and AD enabling AI sensor fusion and perception software. How does sensor calibration affect the perception system? And is it different for vehicle types, say passenger vehicle versus SUV, for example?
Eadan: Basically, the accuracy required from the calibration system is directly, it directly influences the accuracy we receive in the output of the system. So we output many things in our environmental model. But for example, if we just want to consider the example of object detection, so imagine you have a car nearby and you want to identify it within a five centimeter accuracy, for example. So this would mean that our calibration needs to be better than a five centimeter mistake or error. So when it comes to translational accuracy, we want to get to roughly one centimeter or lower for all the different sensors. But we also have the rotational accuracy that we’re looking for. And in that regard, we actually look at far away objects in this example. So imagine you have a car far away and you want to have an accuracy of 30 centimeters. So maybe slightly more is allowed in that case. If we translate that for 100 meters away to a rotational accuracy, it comes out to roughly one milliradian. So that would be roughly the accuracy needed on the rotational aspect. And for companies that are looking for higher accuracy or lower accuracy, according to the product, so it would be a similar calculation. Yeah. And regarding the issue of passenger vehicles versus SUVs and how calibration affects them. So it’s interesting because sensor placement, it has a very strong influence on the perception capabilities. So for example, we see that when we put a camera in the dashboard of a tall vehicle. So imagine like a Hummer that sits up high. Usually these vehicles can actually see above many of the vehicles around them and your long distance perception actually improves when you’re tall. And the opposite example for a low riding passenger vehicle. So you don’t see quote unquote quite as far, but you do see better for the nearby things. So very close next to you, a few feet away, that actually improves slightly when you’re lower. So this trade-off is an interesting side effect of sensor placement.
Michelle: Thomas, I want to get your opinion on this. A key challenge for OEMs and dealerships is that customers go to their local mechanic shop for non-complex issues. Could OEMs use ADAS and such technology to void customer warranties? Could ADAS be a way for OEMs to retain a higher percentage of after sales revenue?
Thomas: That’s a very good question. Yes. Yes. And yes. So they can use it to, if they marketed technology behind the vehicle, then customers will want to bring their vehicle back to the OEM just because they question anyone else who can do that. However, because of our Right to Repair Act and the Magnuson-Moss Act, they cannot hold the vehicle’s warranty hostage or anything like that to say it has to go back to the dealership. It doesn’t have to go back to the dealership. And that’s where we come in. So as a retail company, we’re trying to educate the consumer, but we’re also trying to educate the dealer and the OEM because the dealer doesn’t understand quite what calibration means. So Eadan’s group designs this wonderful system. And then a bunch of other people take in brackets and every other component you can think of to put that system into an environment and have it function. So it becomes critically important. The dealership and the OEMs could definitely use it to drive business back to them simply by, like I said, by marketing the delicacy of the calibration process and how accurate it must be. There’s not a large margin of error. However, as the glass company, and I pride myself on the fact that we’re probably one of the best glass companies out there, we are very cognizant of the importance of that calibration process and the fact that that car has to be setting back as close to OEM specs as it can. And if it’s not, we send them back to have the vehicle corrected so that its stance is right. Everything is correct the way it should be and the way it was designed. So we turn a lot of them away, believe it or not.
Michelle: And I want to touch on another pain point here. While OTA updates might be convenient, they also come with cybersecurity challenges. How do OEMs and consumers mitigate these threats?
Thomas: That’s an involved question. So there are a number of what we used to have security gateways built into the vehicle. You know, now we’re going at software gateways where a company called auto authorization that a lot of companies and OEMs are using now to protect that software from any outside threat. So in order for us to be able to repair that vehicle, we have to gain access to that firewall through that security gateway in order to access the software to perform the calibration. So the consumer themselves can no way, I don’t believe there’s any way. I’m sure someone somewhere, Eadan could probably crack it, but most normal people couldn’t crack into that through that firewall and access the operating system of that vehicle. So it’s very closely monitored who has access, when they were given access, and how long they had access to that operating system. And then once we’re done, it locks it back down again. So it’s pretty well guarded. So over the air updates shouldn’t be an issue because they’re going to be going through that gateway at the authorization point. So somebody’s monitoring every communication that goes through that operating system. So I don’t see that being a big issue, to be honest. I mean, it’s diagnostics over the internet is the new protocol language. So I think it’s going to be a pretty common place.
Michelle: So as we’re wrapping up here, any final thoughts from both of you?
Eadan: I think that the world is going in this direction of ADAS. Sensors are going to be a part of vehicles going forward and algorithms as well. And we’re still at the early stages. We’re learning how to work together, the hardware and the mechanical side of things and the algorithmic side. As the algorithms improve, they’ll slowly be able to take on more of these issues in an online manner, in an automatic manner. But we’re always going to need the boots on the ground for both the fixing and for the initial maintenance. So it’s interesting where it’s going to go. And I think it’s going to be exciting times ahead.
Thomas: I guess ADAS, if your collision center ADAS is critical and it’s critical that it’s done correctly. So if you’re going to perform calibration, educate yourselves, get the proper equipment, learn the proper software, have all the legal access to do so. And by all means, don’t just assume that because it did not throw a fault code, the system is functioning correctly. It must be recalibrated at the OEM’s request. So if the OEM says in their position statement, this event occurred, calibration needs performed, regardless of what your personal opinion is. Calibration is needed and required and necessary. Perform the calibration.
Michelle: Thomas Patterson, Director of Product Development and Technical Training at GlassDoctor and Eadan Valent, Algorithm Group Manager at LeddarTech. Thank you both for your time and being with me today. Great conversation, a lot of exciting things happening. So appreciate you being here today. Thank you. It was a pleasure. And I want to thank all of you for tuning in and listening to the LeddarTech Lab podcast. And of course, if you like what you heard today, be sure to subscribe so you can hear more engaging conversations like the one you heard today. Thanks again for listening. I’m your host, Michelle Dawn Mooney. We hope to connect with you on another podcast soon.
