6 scenarios self-driving cars still can't handle

"Heavy snow and rain tend to confuse LiDAR sensors and also cameras," John Dolan, principle systems scientist at Carnegie Mellon's Robotics Institute, told Business Insider. "So you end up having some problems."

LiDAR refers to the light sensing radar that uses lasers to map the car's surroundings so it can "see" the world. When there's snow on the ground, the cars' LiDAR sensor and camera have a difficult time seeing the lane markers and other markers that help them drive safely. 

Ford made somewhat of a breakthrough in overcoming this problem when its driverless cars successfully drove in snow on a closed course.

Ford created high-resolution 3D maps that come with information not only about the road, but also what's above the road, like its topography and nearby signs and landmarks. This way, when the car can't see lane markings, it can use landmarks to pinpoint itself on the map.

Tesla CEO Elon Musk vented about this problem to a group of reporters in October, according to the Washington Post. At the time, he showed the lack of clear lane markings on Interstate 405 near Los Angeles International Airport.

When driverless cars can't distinguish the lanes, it makes it nearly impossible for them to drive or change lanes safely. Andrew Ng, chief scientists at Baidu, wrote in a Wired post that it will be necessary to make "modest changes to our infrastructure" for driverless cars to be successful on our streets.

There's a few reasons for this. The most obvious is that there are simply fewer things to worry about hitting when on a highway. Cities are a mess of pedestrians, cars, potholes, traffic cones — you get the point. All of those obstacles mean driverless cars have a lot to keep track of, and it can be easy to miss something.

But Dolan said it can also be difficult for the driverless car's GPS to locate properly in cities.

"If you’re trying to do urban driving and depending on GPS to a large extent, then when you get into areas where there are a lot of tall buildings it’s hard to receive the GPS signal and you’ll have drop outs," he said.

Sure, there are rules in driver's education about who gets the right of way at dreaded four-way intersections. But more times than not, we rely on waving to let someone know it's ok to go. Driverless cars don't have that luxury built in.

Dolan said the same issue goes for merging on a highway or changing lanes. There are many ways we convey intention that disappears when there's literally no driver in the front seat.

"We convey intentions in a way that result in natural interactions, rather than what you would call robotic interactions that would unnerve or frustrate a human being," he said.

In February, a Google car got into its first accident that could be considered the driverless car's fault. The car went to merge into the left lane when it saw a bus approaching. The car assumed the bus would stop or slow down and proceeded anyway, only to crash into the bus' side.

We can't say that if a human driver had been in that situation the accident would have been avoided. It's not like drivers are perfect at changing lanes. But there is something to be said about how a human driver could have signaled to the bus driver to see if it was ok to go.

Dolan noted that when human drivers try to merge onto roads with cars traveling at higher speeds, they tend to inch forward to make sure it's ok. Often, people will pull out in front of traffic under the assumption that cars will slow down for the merge, he added.

But a driverless car probably wouldn't take that risk because if it projected the velocity of the upcoming car, it would pull back to avoid a crash, he said.