You Can Fool Tesla’s Autopilot by Placing Small Stickers on the Ground, Study Finds
Researchers at Tencent Keen Security Lab were able to force a Tesla on Autopilot to change lanes using small red and white stickers.
Here's your periodic reminder to pay attention when using Tesla's Autopilot. Researchers at the Tencent Keen Security Lab found it unnervingly easy to force a Tesla Model S running Autopilot to change lanes, according to Digital Trends. We've seen Autopilot get confused by unclear road markings before, but now researchers say that the addition of as few as three color-coded stickers on the road can be enough to fool Autopilot into changing lanes, even into oncoming traffic.
There's a neat video outlining some of the research Tencent Keen Security Lab did into the ways Tesla's autopilot can be fooled. Much of their research is into Autopilot's use of neural networks to "see" what's on the road, notes IEEE Spectrum. Because these networks see things differently than humans, researchers were able to fool a Tesla Model S's systems into thinking it was raining and turning on the windshield wipers accordingly through an algorithm-generated image.
More worrisome is the lab's research into Autopilot's lane recognition systems, where the addition of three stickers in the middle of an intersection was all they needed to trick Autopilot into believing there was a lane, thus turning the car into oncoming traffic.
While the stickers shown in Tencent Keen Security Lab's video were white, the team was also able to achieve a similar result using small red stickers that wouldn't be as noticeable to humans. Digital Trends explains:
In one instance, small, red stickers were placed on the ground to make the vehicle believe it needed to change lanes. The attack could force a vehicle into oncoming traffic. The attack worked in daylight and didn’t require any sort of interference like snow or rain that might make things more difficult for the Autopilot system to process the road.
To Tesla's credit, the extent to which a lane marking has to be blurred for Autopilot to disregard it was fairly significant, IEEE Spectrum notes. However, the fact that Autopilot has been trained to recognize these blurred or broken lines as lane markers is exactly what makes it easier to trick in the other direction, with false lane markers that trick it into oncoming lanes of traffic.
The team was able to compromise the Tesla in other ways, bypassing several layers of security to take over the steering of the Model S using an app, which allowed them to steer the car using a game controller or smartphone. Digital Trends explained that the app was somewhat limited in functionality for cars that recently shifted from reverse to drive, but it was still able to fully override Autopilot and the steering wheel controls from a car that was in park or using cruise control.
That trick's a bit more labor-intensive than fooling the car with unclear lane markings, however, and involved breaking through several layers of security meant to keep such a hijack from happening. As anyone who's driven through a construction zone lately can attest, strange markings on the road are a relatively common sight, and a good reason to always pay attention while you're using Autopilot.
You can read Tencent Keen Security Lab's full paper which details their research into Tesla's Autopilot system here. The team of researchers presented their research to Tesla in hopes that Tesla will implement a way for the car to recognize reverse lanes of traffic.
In the meantime, though, it's still up to you to step in when Autopilot gets it wrong, which even Tesla noted in a statement back to the Tencent Keen Security Lab:
In this demonstration, the researchers adjusted the physical environment (e.g. placing tape on the road or altering lane lines) around the vehicle to make the car behave differently when Autopilot is in use. This is not a real-world concern given that a driver can easily override Autopilot at any time by using the steering wheel or brakes and should be prepared to do so at all times.
It's also worth noting, as IEEE Spectrum does, that Tencent Company (the sponsor of the Keen Security Lab) wants to develop its own autonomous driving systems that could be seen as a competitor to Autopilot, so they have some extra incentive to find flaws in what other companies have already made. Either way, it's a good reminder that semi-autonomous driving systems like Autopilot aren't truly self-driving yet.