In 2013, before it was spun off into Waymo, Google's self-driving car division was developing a semi-autonomous system that would allow for hands-free driving on highways.
Called "AutoPilot," the system required drivers to keep their eyes on the road, and Google told the employees who tested it that they would take it away if cameras placed in their cars showed that they didn't follow the company's instructions.
But some employees didn't follow those directions, and after one fell asleep while his car was traveling at around 55 mph, Google shut down the AutoPilot project, Waymo CEO John Krafcik said at the Frankfurt Auto Show on Thursday. (On Wednesday, Waymo posted a video on YouTube showing the ways drivers misused the system.)
"We had gained a priceless insight into a real conundrum," he said. "Our AutoPilot system was very advanced. It used multiple cameras, radar, and lidar, with a massive on-board computer. In fact, it was so advanced that human drivers became too comfortable, too quickly. And so we realized this was a big problem for driver-assist technology — the better you make it, the more likely humans would be to trust it too much."
Waymo's approach to autonomy is different than Tesla's and Cadillac's
Since then, Waymo has focused on developing technology that will allow vehicles to drive without any human supervision. Seen by many experts as the leader in the autonomous-vehicle industry, the company launched the first commercial autonomous ride-hailing service in the US, Waymo One, in parts of Arizona in 2018. The service is available only to a select group of users, and backup drivers sit behind the wheel during Waymo One rides.
Waymo's approach to autonomy contrasts with those pursued by Tesla and Cadillac, who have what Consumer Reports said in 2018 are the best of four semi-autonomous driver-assistance systems the publication tested.
Those systems, Tesla's Autopilot and Cadillac's Super Cruise, can in some situations control steering, braking, and acceleration with driver supervision. Tesla's Autopilot differs from Super Cruise and Google's AutoPilot because drivers cannot keep their hands off the steering wheel when using it. Super Cruise users can take their hands off the wheel, but must keep their eyes on the road.
High-profile crashes involving Autopilot have raised questions about the extent to which drivers are able to use it safely. Mary Cummings, a professor at Duke who studies the interaction between humans and autonomous driving systems, told Business Insider in February that systems like Autopilot, Super Cruise, Nissan's ProPilot Assist, and Volvo's Pilot Assist rely on unrealistic expectations about how well humans are able to pay attention to the road and quickly react if the semi-autonomous system needs them to take control.
"Any reliance on the human to intervene in any kind of safety-critical capacity is a red herring. They're never going to be able to do it reliably enough to make these systems, I think, safe for the road," she said.
Tesla and Cadillac did not immediately respond to Business Insider's requests for comment.
Are you a current or former Tesla employee? Do you have an opinion about what it's like to work there? Contact this reporter at [email protected].