We Made A Nurburgring For Autonomous Cars
And then we called it the Autono-Ring.
Benchmarks are a good thing. When a Porsche 918 laps the Nurburgring .4 seconds faster than the latest lawnmower out of Modena, we know who’s best, at least until next year. In the autonomous future, how are we to know which car is safest? It’s certainly not going to be through traditional crash testing. Notcrashing is the whole point. That’s why we need a Nurburgring for Autonomy.
I hereby declare: the Autono-Ring.
This Friday at 330PM EST, Mike Spinelli and I take the first stab at creating the Nurburgring for Autonomy on Episode 4 of this season’s /DRIVE on NBC Sports. Here’s the trailer.
Why An Autono-Ring?
If safety is the goal of Self-Driving Cars, what’s the benchmark? In a world of mixed human and semi/fully-autonomous cars, zero deaths is impossible. But we’ll never get to 100% autonomy—or safety—without a long period of development and real-world use of semi-autonomous cars. Between now and our fully autonomous future, we’re stuck with semi-autonomous technologies for which there are no meaningful benchmarks, let alone tests.
How have manufacturers tested cars until now? In and around where they’re likely to be sold, sometimes with additional hot and cold weather testing in places like the Sahara and the Arctic. That worked for human driving, but that’s not good enough for full or even semi-autonomous driving technologies. Lane markings, road signs and driver behavior vary all over the world, often even within a single country, which is why no simulation or test short of the real thing will ever suffice.
Garbage in, garbage out.
The necessity of an Autono-Ring was recently highlighted by my 10+ car ADAS showdown. ADAS—or Advanced Driver Assistance System—is the first step toward fully autonomous/Self-Driving Cars, and yet not all ADAS is created equal. ADAS, comprised of Automatic Emergency Braking (AEB), Active Cruise Control (ACC) and Lane Keeping Assistance (LKAS), is more than the sum of its parts. Acting in unison, every ADAS suite has its own behavior. All of the individual subsystems can function perfectly in a lab or closed testing facility, but put two or more in the real world, and strange (and occasionally unpleasant) things can happen.
So when the Drive's Executive Producer Mike Spinelli and I got an opportunity to evaluate the latest Tesla Model S and Mercedes-Benz E-Class on the same weekend this summer, we decided to create the first version of an Autono-Ring on the roads around San Francisco, replicating most of the conditions and scenarios a semi-autonomous car is likely to see. Streets, highways, traffic, dogs, jaywalkers, protesters. We even saw a hit-and-run.
Everything except weather, which was perfect.
Weather is the thing. In order for the Autono-Ring concept to be meaningful, you need more than one, with clearly defined sectors, tests and benchmarks, replicating all conditions, from rain to fog to snow. Our comparo was therefore as much an experiment in what an Autono-Ring testing regimen should be as it was a test of the two cars.
What did we learn? Any car with ADAS and up must be tested for behavior, situational awareness and interface as much as for outright safety performance. Just because an ADAS-equipped car will stop short of a pedestrian doesn’t make it “safe,” not if it’s lane keeping functionality is so poor that a driver would prefer to disable the entire suite. Just because LKAS works on the Autobahn doesn’t mean it’s going to work on I-95.
Our first Autono-Ring experiment is just the tip of the iceberg. The Autono-Ring concept needs to be honed, replicated and combined with traditional testing methods of the subsystems. Every market with sufficiently different conditions—i.e. Manhattan vs. Mumbai—will need its own version.
Until semi-autonomous driving technologies are tested against meaningful benchmarks, debate over things like Tesla Autopilot and the stillborn Comma One is pointless. People are going to be driving for a long, long time. And as long as there are humans in the loop, they deserve to know what they getting.
Let’s help them make better choices.
Tune in Friday at 3:30 pm EST on NBC Sports to see Autono-Ring 01, as well as our visit to Thunderhill for the worlds first Autonomous track day, and check out Ford's Level 4 Self-Driving prototype.
Alex Roy, Editor-at-Large for The Drive and author of The Driver, set the 2007 Transcontinental “Cannonball Run” Record in a BMW M5 in 31 hours & 4 minutes, and has set multiple "Cannonball" endurance driving records in Europe & the United States in the EV, 3-wheeler & Semi-Autonomous Classes. You may follow him on Facebook, Twitter and Instagram.
