How will driverless cars ‘talk’ to pedestrians? Waymo has a few ideas

[Collection]

Image: Waymo

For years, developers have been working on ways for driverless cars to communicate intent to other road users, either through audio recordings or visual cues. Today, Waymo says it wants to be one of the first companies to put some of these methods into practice.

The Alphabet-owned company’s driverless Jaguar I-Pace vehicles will use their roof domes, which are wrapped in LED displays, to communicate messages to other road users. For now, the company is going with just two messages: for pedestrians in front of the vehicle, shifting grey and white rectangles meant to communicate that the vehicle is yielding to them, and for drivers behind the vehicle, a yellow pedestrian symbol to let them know there’s a pedestrian crossing.

The symbols join other visual and audio cues that Waymo employs to “talk” to other road users. The company has used its roof dome to display the initials of the customer who is hailing the ride and to signal to cyclists that a passenger intends to open the door. Waymo also uses external audio alerts to communicate with emergency responders or to explain what the vehicle is going to do next, like rerouting.

Waymo calls these recordings and cues “tertiary communication.” Human drivers have gotten pretty good at communicating intent to the people around them: a little wave of the hand, a nod, maybe some eye contact. And, if all else fails, we can lean on our horns or roll down our windows and speak our minds (sometimes colorfully). Waymo vehicles have no hands or eyes with which to communicate, so new methods needed to be invented.

The most challenging thing was to figure out how to present them so they didn’t come off as telling other people what to do, said Orlee Smith, senior product manager at Waymo. After all, Waymo didn’t want to cast its driverless vehicles in the role of traffic cop, barking orders at pedestrians or trying to bully other drivers.

“We need to create a way for someone to say, ‘OK, I know what that Waymo is doing, that Waymo is yielding, it’s not going to drive right now,’” Smith said. “And as a driver behind this car, ‘This Waymo is stopped because it’s yielding to a pedestrian.’ So that was one of our big challenges.”

For now, the roof domes will just display the yielding-to-pedestrian symbols for road users in the front and rear of the vehicle. Waymo plans on rolling it out in San Francisco, followed by Phoenix and then Los Angeles. And since all Waymo vehicles are equipped with LED displays on their domes, the possibility of integrating future messages is all but certain.

For years, AV developers have been wrestling with the problem of how best to communicate with other road users

For years, AV developers have been wrestling with the problem of how best to communicate with other road users. Uber (when it still thought it would build its own self-driving cars) took the maximalist approach, slapping flashing directional lights on the side-view mirrors, doors, roof dome, and basically anywhere there was a bit of free real estate on the vehicle. Drive.ai, a now-defunct AV startup, was more subtle in its approach, relying on LED signs that use text and emoji-like pictures to communicate.

While it was still housed under Google’s X lab, Waymo filed patents for light-up “walk” or “don’t walk” signs on the car’s body, image displays, and audible signals similar to the ones used at busy crosswalks. Other types of notifications sounded more bizarre, like “a robotic hand to make gestures or robotic eyes on the vehicle that allow the pedestrian to recognize that the vehicle ‘sees’ the pedestrian.”

But despite being talked about for years, it’s taken until now for an AV operator to actually ship these features into the fleet. Meanwhile, Waymo and other driverless vehicles have been involved in a handful of crashes and other incidents, in San Francisco especially, that have angered emergency responders and irked residents. Better communication wouldn’t solve all these problems, but it could have been useful in some of them, such as when a driverless vehicle stalls in the middle of the road.

It’s taken until now for an AV operator to actually ship these features into the fleet

Orlee said that Waymo took years to research this subject because it wanted to make sure it got it right. The company started studying various communication methods back in 2019 and only began to arrive on shippable solutions this year.

But if Waymo starts using its own symbols, and then rival robotaxi companies like Cruise and Zoox and Motional arrive at completely different symbols, we could quickly arrive at a situation in which multiple self-driving cars are each displaying their own bespoke messages to pedestrians and other drivers. In other words, epistemological chaos.

Orlee said Waymo supports standardizing light patterns, sounds, and symbols across the industry so as to avoid this potentially confusing scenario. She said that Waymo is hoping to lead that conversation — but of course, other companies have their own ideas. For example, before it shuttered, Argo AI urged other companies to adopt its technical guidelines for safe interactions between robotaxis and bicyclists.

Ultimately, the onus will be on all of us — the pedestrians, the cyclists, and other drivers — to figure out what these eyeless, handless, mouthless, human-free cars are trying to tell us. And that could take a while, depending on how clear and universal the method of communication. “This is going to be a learned signal,” Smith said. “This is not something that when you see this for first time, you’re gonna say, ‘Oh, I know exactly what that means,’ right?”

And of course, if all else fails, Waymo’s cars can still honk their horns.