The War For Autonomous Driving, Part III: US vs Germany vs Japan

A face-off between Audi, Cadillac, Chrysler, Ford, Honda, Jaguar, Lexus, Mazda, Mercedes, Volvo, & VW.

Autonomous Driving
Alex Roy/TheDrive.com

All the hype around self-driving cars has obscured the reality on the ground. The basic building blocks of Autonomous Driving are already on the market, and a closer look at the costly options rarely found on dealer lots offers a glimpse as to who is actually in the lead.

In Part 1 of The War For Autonomous Driving, I compared the latest semi-Autonomous systems from Mercedes-Benz and Tesla, as of July 2016. Despite expectations that the Mercedes Drivepilot system would match or surpass Tesla’s Autopilot, it was not the case. The 2017 E-Class offered no more than advanced driver assistance system (ADAS), while Tesla Autopilot 7 delivered true state-of-the-art Level 2 autonomy.

ADAS includes all the building blocks of semi-autonomous driving, systems like automatic emergency braking, radar cruise control, and lane-keeping assistance. Level 2 autonomy, meanwhile, is the government’s definition of a car that can drive itself for limited stretches, with a human monitoring the situation. In other words, you still need to keep your hands on the wheel.

In Part 2, I reviewed the E-Class ADAS in isolation, and came away impressed by Mercedes-Benz’s industry-leading hardware, but disappointed by its hard-coded software, whose lane-keeping system was no match for Tesla's, which incorporates Fleet Learning and wireless updates.

While Tesla has since released Autopilot 8 (via wireless update), the rest of the industry is releasing 2017 models, like the new E-Class, whose semi-autonomous/ADAS features are likely to remain unchanged throughout their three- to six-year product cycles. So on the one hand, you have Tesla, with its constantly-evolving software, and the rest of the industry, with individual models in various stages of obsolescence until the next new tech leap is unveiled. It makes harder an already logistically difficult job of head-to-head comparisons between systems.

This past week, thanks to a test day for journalists, I had the chance to get into a bunch of cars one after the other. It was a great opportunity to get an overview of the current crop of technologies, but required quick and efficient evaluation. I decided to focus on the feature most easily evaluated in a short span of time; a feature that, when it works, however briefly, shines a light on our autonomous future—but when it doesn’t, makes it seem like the old ways are still the best. That feature: lane-keeping assistance systems.

If the Mercedes-Benz versus Tesla comparo is any guide, not all LKAS is created equal. Let's see where we stand.

Alex Roy/TheDrive.com

The Battleground

Every year the International Motor Press Association (IMPA) hosts test days at which a wide variety of manufacturers bring their latest vehicles. This year the event was held at the Monticello Motor Club in upstate New York. While my colleague Lawrence Ulrich tested cars like the Dodge Viper GTC and Jaguar F-Type SVR on and off-track, I took the opposite tack. With over seventy cars to choose from—only ten of which the other journalists wanted to drive—my job was easy. Just walk up to anything on the lot with LKAS, and drive away.

I drove everything available within the time I had. Alas, Hyundai, Ford, and BMW didn’t have LKAS-equipped vehicles on site, and I didn’t have time to test the Toyota, Nissan or Infiniti products.

The Route

The Monticello Motor Club sits on Cantrell Road, an undulating, tree-lined, road with few markings. Head west, and in under a mile you hit Route 42. Turn left and you have 2.5 miles of ideal conditions suited to even the most primitive LKAS system, before you reach the Post Office at the intersection of Route 42 and French Clearing Road. Excluding Cantrell road, our round-trip test course covered five miles.

Alex Roy/TheDrive.com

The Conditions

Perfect.

The Test

To be fair, an LKAS test was really all I could do. No manufacturer but Tesla will claim anything but ADAS, and since one day isn’t enough time to adequately test even a single ADAS suite, LKAS it had to be. But this is 2017 stuff; Tesla Autopilot is out there at Level 2, and its Autosteer behavior is exceptional, so if manufacturers want to limit themselves to ADAS there's no excuse for releasing LKAS functionality that’s half-baked.

LKAS is easy to test. When your life depends on it, a feature either works or it doesn’t. As per manufacturer instructions, I kept one hand gently on the steering wheel at all times. I would head south on Route 42, at or below the speed limit, attempt to engage each car’s LKAS system, and try to keep it engaged for as long as possible. If the LKAS required cruise control, I activated that first. If the car made it the 2.5 miles to the Post office in relative safety (note that my safety standards are both low and flexible), I turned around and did the same thing on the way back.

If the car’s LKAS couldn’t engage in these perfect conditions, or wouldn’t remain engaged, then I would drive it back myself and rank it for safety against my Level 1 (i.e. non-autonomous) benchmark, my Cannonball-Record Setting 2000 Autobahn Polizei E39 BMW M5.

Why? Because I believe an attentive driver in a car with zero automation is safer than the average driver in a car with current-generation ADAS.

Scoring

That task of LKAS is to help keep a human driver, whose hand is on the wheel but whose attention is wandering, stay in their lane. It doesn't have to be perfect, but it has to work sufficiently that the driver trusts it—and it definitely can't work erratically to the point where it's dangerous to engage.

With that in mind, my scoring system is a simple yes-or-no question: Can you use it?

Let the battle begin.

2017 HONDA ACCORD

Alex Roy/TheDrive.com

According to George Hotz, a Honda Civic with a Touring package—almost $8,000 more than the base model—has the best LKAS system on the market other than Tesla Autopilot. Alas, no Civic was available, but a 2017 Honda Accord Hybrid was, and it had LKAS. Hotz said it wasn’t as good. I can’t speak to how it compares to the Civic’s, but I was impressed out of the gate.

The Accord made it most of the way to the post office without any drama, but the LKAS display is too small to see whether the LKAS lane markers are active (white) or inactive (hollow). Also, the lack of audible notifications is inexcusable.

There is a bizarre logic to the LKAS, which gives me pause. It works even if the Radar Cruise Control is disabled. Who wants to have the car steer while having to control the throttle? Not me.

Can You Use This? Yes(ish)you want to use this with cruise control, or not at all

Honda Accord or E39 BMW M5? Winner: Honda Accord

2017 VOLKSWAGEN PASSAT

Alex Roy/TheDrive.com

The VW system was unable to consistently engage, which was a disappointment, because the LKAS display was sufficiently large that I could understand when it was engaged—which wasn’t often enough to rely on it all. It did have audible warnings, though.

Can You Use This? No

Volkswagen Passat or E39 BMW M5? Winner: E39 BMW M5

2017 CHRYSLER PACIFICA

Alex Roy/TheDrive.com

A gorgeous minivan. Amazing. Wonderful. But the LKAS wouldn’t engage most of the time, and when it did, it could barely hold the lane. The large situational display is therefore useless. I loved the Pacifica. Its LKAS? Not so much.

Can You Use This? No

Chrysler Pacifica or E39 BMW M5? Winner: E39 BMW M5

2017 CD CT6

Alex Roy/TheDrive.com

Another gorgeous car, but it’s LKAS was inconsistent. Again, a tiny display that was all but useless. I liked the haptic lane-departure warning through the seat bolster, though.

Can You Use This? No

Cadillac CT6 or E39 BMW M5? Winner: E39 BMW M5

2017 MERCEDES-BENZ E-CLASS

Alex Roy/TheDrive.com

Testing this LKAS system in the Bay area, I hated it; likewise in and around New York City. But on Monticello’s perfect roads, I saw it in a different light. It was almost as good as the Honda, and made it all 2.5 miles to the post office with only two questionable moments. Its display is better than Honda's, but it has the same problems with lack of audible warnings for engagement and disengagement.

The lesson? LKAS performs very differently, even in conditions that would appear similar. What was it about the lane markings in the Bay Area and NYC that upset this Mercedes-Benz? There's a lot more work to be done here. In that vein:

Dear Mercedes-Benz,

Your R&D team needs to get out onto roads outside of Germany. Get this working on roads with a wider variety of markings, or duplicate Tesla’s Fleet Learning and wireless updates, and you’re onto something.

If only everyone lived with roads as good as Monticello’s. If only.

Can You Use This? Yes

Mercedes-Benz E-Class or E39 BMW M5? Winner: Mercedes-Benz E-Class

2017 VOLVO S90

Alex Roy/TheDrive.com

I set several driving records in a similarly gorgeous S90 across Europe and the United States this past summer, attempting to maximize Volvo’s PilotAssist along the way. At night and in light rain, it did better than expected—as long as road markings were clear. In other words, Swedish, French, and Spanish highways were ideal.

Here, the Volvo was slightly better than the Mercedes-Benz, and tied with the Honda for lane keeping. But the S90 had a tendency to hug the brighter of the two lane markings. Keep those hands on the wheel. Display and warnings are insufficient, but it made it to the post office with only one questionable moment, so it gets a "yes," in perfect conditions.

I wonder how it would do in the Bay Area or NYC environs, which is where the E-Class had real problems.

Can You Use This? Yes

Volvo S90 or E39 BMW M5? Winner: Volvo S90

2017 MAZDA CX-9

Alex Roy/TheDrive.com

I love Mazdas, especially the Miata, but an impressive ambassador of LKAS, this is not. Despite many attempts, it failed to engage, and the display was insufficient. Drive this one yourself, like all Mazdas are really meant to be driven, and you’ll love it.

Can You Use This? No

Mazda CX-9 or E39 BMW M5? Winner: E39 BMW M5

2017 LEXUS GS-F

Alex Roy/TheDrive.com

The system barely engaged and the display was tiny. But it's a terrific car, meant to be driven by a human. Turn off all the aids and enjoy it.

Can You Use This? No

Lexus GS-F or E39 BMW M5? Winner: Tie

2017 AUDI A6

Alex Roy/TheDrive.com

A nice display, but the LKAS just isn’t there. Come on, Audi. You’ve got to match Mercedes-Benz and Volvo. It’s late 2016.

Can You Use This? No

Audi A6 or E39 BMW M5? Winner: E39 BMW M5

2017 JAGUAR F-PACE

Alex Roy/TheDrive.com

Can you use this? No. It barely engaged. Small Display. I love all things currently Jaguar. The F-Pace is a wonderful car. The F-Type is amazing in all its forms. Jaguar is on a roll, but its LKAS hasn’t caught up.

Can You Use This? No

Jaguar F-Pace or E39 BMW M5? Winner: E39 BMW M5

Prediction: The Comma One Would Probably Beat Them All. Against Tesla? Forget it.

Manufacturers have a lot of work to do. In this limited test group, only Honda, Mercedes-Benz, and Volvo have LKAS that can be relied upon—and with the caveat that users really must understand the limitations.

Something tells me the Volvo and Mercedes-Benz LKAS might even have been updated versions of what I drove this summer, but my doubts highlight the problem with comparing different LKASs: Unless you compare systems head-to-head on the same road, there are too many variables for definitive rankings.

All of these companies are lucky Tesla wasn’t here. I recently lapped Manhattan in a Model S, using both Autopilot 7 and 8, and made it over 90 percent of the way with virtually no issues. I don't think anything at Monticello could do that.

There's no doubt that any automation leads to an atrophying of driving skills. And I think that atrophy is rapid. Unless a system’s software is wirelessly upgradeable, and is upgraded in a consistent and ongoing basis like Tesla’s (and, soon, Comma.ai's), I don't think it can possiblyimprove at a rate inverse to the decline of driving skill.

Until then, assume nothing, and keep your hands on the wheel. Don’t pay for LKAS options unless they’re part of an upgrade package that you have to have. Enjoy all the other safety features that an ADAS offers, and don’t become over-reliant on technology that isn’t quite there.

Something very interesting is coming, though, and that’s Comma.ai’s aftermarket upgrade for selected Honda and Acura models. George Hotz claims it replicates Tesla Autopilot 7, for a thousand bucks. I had a brief test last week in the Bay Area, on the same roads where Mercedes-Benz’s ADAS was beaten by Tesla Autopilot. Based on what I saw, the Comma One’s LKAS is not only superior to everything in the Monticello test, its display and warnings may match or surpass Tesla’s.

I’ll be road-testing a Comma One soon. Tesla's Enhanced Autopilot is coming sometime in the next few months. Who knows what Faraday Future, Atieva, and GM/Cruise have in the pipeline?

Meanwhile, the ADAS demonstrated at Monticello will remain static until the next model cycle, unless the legacy OEMs have something brewing.

Alex Roy is an Editor-at-Large for The Drive, author of The Driver, set the 2007 Transcontinental “Cannonball Run” Record in a BMW M5 in 31 hours & 4 minutes, and has set multiple "Cannonball" endurance driving records in Europe & the United States in the EV, 3-wheeler & Semi-Autonomous Classes. You may follow him on Facebook, Twitter and Instagram.

[UPDATE: Corrected to reflect the Pacifica display's effectiveness.]