[RCvetter]

The Navigant report is crap, though. Seems to me they cherry-picked categories, and if they had chosen different metrics the results would have been completely different.

 

[DrPhyzx]

The Navigant report is crap, though. Seems to me they cherry-picked categories, and if they had chosen different metrics the results would have been completely different.

Invective + "seems to me" opinion isn't a convincing argument against the content of the Navigant report.

The 2017 disengagement reports should be out soon, which is one of the best windows we have into Tesla's self driving program and where they are in experience and success relative to Waymo and GM, who are undoubtedly in the lead in terms of miles traveled and successful navigation of complex urban environments in fully autonomous mode. 2016 was the first year Tesla had any fully autonomous cars operating on CA roads - their only testing on public roads - and the data was sad for those hoping for L4 features in their cars any time soon.

At the moment, it looks like GM (and probably Waymo) are likely to have a fully autonomous taxi service on public roads in 2019. I say this as someone very skeptical of previous projections, and informed by friends in the industry that are working on this technology. I just don't think Tesla is anywhere close to that timeline.

 

[garsh]

At the moment, it looks like GM (and probably Waymo) are likely to have a fully autonomous taxi service on public roads in 2019.

I've seem plenty of press about Waymo, but I haven't heard anything in regards to GM making great inroads into self-driving (other than this Navigant report).

Do you have any other information available about GM's progress? You appear to believe that they're well along.

 

[DrPhyzx]

It's been all over the news this past year. You can google and find tons of information from all different angles. These will give you enough info to find more on your own:

Here are videos of some drives in gen-2 prototypes (modified Chevy Bolts) from about a year ago. It's a tough environment to test in, as you can see. I see the Waymo cars down here around Mountain View all the time, and they don't have to contend with anything like what is shown in these videos.
==================

NY Times article on their whole program and the gen-3 cars, designed specifically for L4/L5 automation with fully redundant systems (like a commercial airliner), of which they have made well over a hundred of now on the same line at Orion that produces the normal Bolt. These cars circulate SF 24/7/365 giving rides to GM and Cruise Automation employees for free. This article describes one such ride given to the author, and other publications have similar accounts of their rides on the demo day that you can find by searching:
==============
https://www.nytimes.com/2017/11/29/business/gm-driverless-cars.html

A news article about how they plan to begin testing Level 4 cars in NYC this year - the first in NYC. That's going to be an interesting challenge, no doubt.
==============================
https://www.theverge.com/2017/10/17/16488330/gm-cruise-nyc-self-driving-car-test-cuomo

Recent news article about the gen-4 design, that has *no steering wheel or pedals* in the car. Again, these have fully redundant systems, and are designed to be produced on the line at Orion.
=====================
https://www.theverge.com/2018/1/12/16880978/gm-autonomous-car-2019-detroit-auto-show-2018

Here is GM's self driving safety report submitted to USDOT, referred to in the previous article. (So far only GM and Waymo have submitted this report as requested by federal government.) It has some details about the program, the cars, safety systems, etc.
http://www.gm.com/content/dam/gm/en_us/english/selfdriving/gmsafetyreport.pdf

GM is being seriously aggressive about this. They'll ruffle some feathers and have some high-profile failures because of that, but they will also continue to make very rapid progress, which is all that matters.

I know some engineers at a small startup that is in this fray too, and while they have good software tech, they are seriously up a creek without a paddle on how to manufacture a car. Waymo is somewhere in between that and GM, where GM's strength is obviously auto manufacturing, but their acquisition of Cruise and Strobe and big investment in expanding those efforts are closing the gap on the self driving tech very quickly.

 

[DrPhyzx]

I just spotted this after digging up the other videos; a pretty entertaining presentation full of interesting detail on what goes into making these cars "see" - one of the biggest challenges.

Go to 31:30 for a good laugh if that's all you have time for, but there are a lot of examples of things that can go wrong in there.

 

[garsh]

Speeding up a video like this helps to hide all of the little mistakes the car makes. I noticed this with the video that Tesla released. Luckily, YouTube allows you to easily slow down a video. I watched Tesla's demonstration video at something closer to real-time speed, and wrote up a list of all of the anomalies I noticed. Maybe I'll find time to tackle this video too.

This is an older video that has been posted here before, in this thread. I actually watched this in real time and listed the anomolies.

Nice analysis, Melinda!
I decided to do one too.
0:13 After completing a left turn, it decides to stop because a bike blew through a stop sign coming the other way to make a right. It allowed the bike to pass it on the right.
0:16 Does not yet attempt rights on red.
0:26 Does a good job slowing down for a construction worker crossing the road.
0:30 Attempting left turn on yield green. Pulls out into intersection, then slows down to wait. Should wait for a clearing before pulling into intersection.
0:31 Worse, it starts to turn left before stopping. YOU SHOULD NEVER DO THIS! If you get rear-ended, you'll get pushed into oncoming traffic. Wait until you actually have a clearing before beginning to turn the steering wheel.
0:35 Driver hits a button on the steering wheel. Bad engineer! No cookie for you!
0:49 It seemed to slow down more than necessary to go around the hole in the middle of the road.
0:50 I disagree with Melinda here. The Bolt can probably sense that the oncoming car was decelerating rapidly to decide that it was safe to turn in front of it.
1:04 Why did it stop 12 feet before the intersection? I guess it didn't want to approach with a pedestrian crossing?
1:43 Why so slow on this stretch? Appears to have been confused by the bike lane separators.
2:06 Extra pause halfway through the intersection. Not clear why.
2:09 Here it pulls right up to an intersection where people are crossing. So why didn't it do this at 1:04?
2:30 Pulls up way too close to the parked truck before attempting a pass. Has to wait for other cars to pass both them and the truck.

In general, the car isn't very smooth on streets with on-street parking. It goes more to the right when there is an empty parking space, then closer to the center when there are cars. As a result, it seems to "swerve" a lot while driving down a road. They need to smooth out that algorithm some.

I've no doubt that GM is continuing to make big strides in this space, but so far it appears that their self-driving abilities aren't really much better than what Tesla has demonstrated.

So... I continue to believe that Waymo is 5-6 years ahead of everybody, including GM. But they need to partner with a car manufacturer soon to get this technology out to the public before everybody else catches up. Maybe the current partnership with Chrysler will turn into something bigger.

 

[George K]

Would you trust self driving car on a mountain road such as Rt 120 from Groveland CA to Yosemite and over mountains to Rt 395 ?
The road has many sharp switch backs, many with no guard rails, no shoulders ,no lane lines and 1000 ft cliffs for car to drop off of. This would be a real test, much better than coast to coast test on Interstates.

 

[SoFlaModel3]

My sister's boyfriend is a researcher at the Stanford lab focused on autonomous driving. I had a chance to check out the cars yesterday and its very interesting stuff that they're working on. It certainly does reaffirm my decision to pass on full self drive in my spec at this time. I don't think we're close...

 

[M3OC Rules]

Keep in mind Waymo already has cars driving on the streets without drivers. There is also a service running in the Villages in Florida.
So it kind of depends on what your definition is of the endpoint for determining how close we are.

 

[SoFlaModel3]

Keep in mind Waymo already has cars driving on the streets without drivers. There is also a service running in the Villages in Florida.
So it kind of depends on what your definition is of the endpoint for determining how close we are.

I'm highly skeptical of entering an address into my nav screen and safely ending up at the location.

 

[garsh]

I'm highly skeptical of entering an address into my nav screen and safely ending up at the location.

As long as you live in the Phoenix Metropolitan area, it's no problem.
https://waymo.com/apply/

Waymo still has to figure out how to drive in rain and snow, but they have the rest of it pretty well nailed down. They tend not to make all of the little anomalous mistakes that I notice GM and Tesla (and Uber) cars making.

 

[M3OC Rules]

I think its going to be incremental. People are getting too hung up on the ultimate level 5, 100x better than humans. Autonomous cars are happening now. It will slowly get better and increase in the situations and locations where its available.

Waymo says its starting services open to the public in 2018 and GM Cruise in 2019. I think Tesla will turn on level 4 of some sort in 2019. Who knows how many of the others will make it in the next few years but there will be a lot of pressure once 1 or 2 start rolling out services.

There are also dramatic improvements in the hardware processing and sensors and simulation tools given the 10's of billions in investment over the last couple years.

 

[DrPhyzx]

As long as you live in the Phoenix Metropolitan area, it's no problem.
https://waymo.com/apply/

Waymo still has to figure out how to drive in rain and snow, but they have the rest of it pretty well nailed down. They tend not to make all of the little anomalous mistakes that I notice GM and Tesla (and Uber) cars making.

Phoenix != San Francisco. Apples and Oranges. BTW, the Waymo cars still make mistakes and have disengagements in California and they mostly test in suburban areas that are much better controlled. True that Waymo has some cars where the Waymo employee doesn't sit in the driver's seat now operating in the US, but only in easy suburban environments.

I think we agree to disagree for now. Let's check back in a year.

 

[garsh]

Phoenix != San Francisco.

Waymo has been testing extensively in San Francisco and the Bay Area since 2009.
Cruise Automation was started in 2013. GM didn't acquire them until 2016.

they mostly test in suburban areas that are much better controlled.

Where are you getting this misinformation?
https://www.theverge.com/2018/1/12/16884064/waymo-self-driving-car-spotted-san-francisco

Waymo says it has tested its autonomous minivans in 24 cities across the US.​

Sure, we can agree to disagree on who's ahead in self-driving technology, but that kind of statement is simply incorrect.

 

[DrPhyzx]

I live here (Stanford), so I see where Waymo tests in the BA - very little in SF and nearly all in Mountain View (or elsewhere in the Bay Area). These are far and away different scenarios, but if you do not live here, I can understand that you will not appreciate that fact. Please read my statement (that you saw fit to quote) again. I did not say Waymo does not test in SF, but that they do most of their miles in much less demanding areas. (BTW, everything in the BA but SF is a suburb for this purpose). I'll stand by that statement, and it makes a big difference:

https://techcrunch.com/2017/10/03/g...03/gms-cruise-explains-why-self-driving-tests-in-dense-cities-gives-it-an-edge/

As to the videos: the first one was from Jan. 2017. At that time, GM/Cruise had logged <10K miles on public roads in CA according to their disengagement report. Waymo had logged over 630K in 2016 alone. GM's disengagement reports indeed showed that they were far behind but made rapid progress during 2016 with a very small fleet of Gen 2 cars (a few converted Bolts - not built on the assembly line). Article on 2016 reports that quotes these numbers:

https://driverless.wonderhowto.com/...reports-show-waymo-absolutely-crushing-competition-every-single-metric-0176110/

... consider the exponential chart of miles/disengagement for GM's paltry 10K miles. Then consider that GM has now built 180 cars (mostly the Gen 3) that are operating all the time in SF and Phoenix and that a good fraction of those cars are on the road at any given time, 24/7/365.

Let's come back to this in a few days when the 2017 disengagement reports are out. That's going to give a very clear picture of what has changed in the last year. I'm willing to bet that it's going to show that GM is the only one even close to Waymo right now, and Tesla is too busy with other things to be in the hunt.

Other notes:
- For the record, I never said GM was ahead of Waymo, just that they are in a comparable position relative to the others.
- Yes, as I said before, GM bought Cruise and has expanded and invested heavily. Cruise had no public track record and little public-road testing before the acquisition. As you can see from the 2016 disengagement reports, they were really just starting from scratch at the beginning of 2016 with real-world testing.

 

[sreams]

It's been all over the news this past year. You can google and find tons of information from all different angles. These will give you enough info to find more on your own:

Here are videos of some drives in gen-2 prototypes (modified Chevy Bolts) from about a year ago. It's a tough environment to test in, as you can see. I see the Waymo cars down here around Mountain View all the time, and they don't have to contend with anything like what is shown in these videos.
==================

I would have some questions about some of the things that happened on these drives. There were multiple times where the car ended up behind a double-parked vehicle that was unloading passengers or cargo, and the only way around that vehicle was to cross over the double yellow line. How did the car know to go around the vehicle (as opposed to staying behind a vehicle that was simply a part of stopped traffic), and how did it decide it was okay to cross the double yellow and drive partially in the oncoming lane (which is technically not legal)?

 

[garsh]

How did the car know to go around the vehicle (as opposed to staying behind a vehicle that was simply a part of stopped traffic), and how did it decide it was okay to cross the double yellow and drive partially in the oncoming lane (which is technically not legal)?

Well, how do *you* know? These cars are being programmed by people. Try to think of the clues you would use to determine if the car in front of you was parked or just stopped. They will program the car to take the same types of factors into consideration.

• Are the brake lights on? Stopped in traffic.
• Are the hazard lights on? Parked.
• Are there no lights on? Probably parked.
• Can you see somebody in the driver's seat? Probably stopped in traffic.
• Is there another vehicle just in front of that vehicle? Probably stopped in traffic.
• Have you been stuck behind it for a long time? Maybe parked.

I'm sure it gets pretty nuanced, and I'm sure my list is wrong.

 

[DrPhyzx]

Well, how do *you* know? These cars are being programmed by people. Try to think of the clues you would use to determine if the car in front of you was parked or just stopped. They will program the car to take the same types of factors into consideration.

• Are the brake lights on? Stopped in traffic.
• Are the hazard lights on? Parked.
• Are there no lights on? Probably parked.
• Can you see somebody in the driver's seat? Probably stopped in traffic.
• Is there another vehicle just in front of that vehicle? Probably stopped in traffic.
• Have you been stuck behind it for a long time? Maybe parked.

I'm sure it gets pretty nuanced, and I'm sure my list is wrong.

Absolutely, and this is why testing in the controlled chaos of someplace like SF or Manhattan really stresses the system. Not only do you encounter more of those scenarios, but they begin to coincide, which is where things get hairy: not just an emergency vehicle, or a double parked car, or a jaywalking pedestrian, but perhaps more than one of these things at a time, requiring some real decision making about how to do the safest thing (which may not always be the thing that violates the least rules) while continuing to make progress. We do these things all the time when we drive, and we forget how hard it is to learn.

One of the recent demo drives GM did for the press actually stumped a car when it got behind a taco truck pulled up in front of a construction site. You can imagine the computer has a hard time going around a double-parked food truck when there is a steady stream of hungry jaywalking construction workers approaching and milling around the truck for lunch.

https://jalopnik.com/gms-self-driving-bolt-gets-publicly-stumped-by-taco-tru-1820835350

How do you teach the car to be aggressive enough to get people in a situation like that to yield? Will people cede to an AV like they do to another driver out of courtesy?

 

[M3OC Rules]

I can't remember where I heard this (Maybe Nvidia at CES) but they were saying that its very difficult to generate the situations it needs to handle and that simulation is much more effective. Eventually you need to test on the road but it makes me wonder if things like disengagement reports are not a very good indicator. Plus disengagements are not all equal and could be biased by who is driving. You could have one problem that causes a 100 disengagements and then fix that problem. It doesn't erase the disengagements. If all that mattered were disengagements you would be incentivized to drive less or drive on easier streets. But that would be the opposite of what an engineer would want designing the system. And videos are definitely not good indicator unless you're going to watch thousands of hours which they don't provide. Its entertaining but not going to be used to validate safety.

I think the rubber hits the road when they open it to the public. Waymo will likely be first. Who knows who will end up being best. I'm rooting for all of the above. Competition will help prices and safety. Plus it will help get it out there faster. I can't wait to reduce the $500/month it takes to own my 10 year old car and not even own a car. In the meantime bring on the Model 3!

 

[DrPhyzx]

Indeed, the 2017 CA disengagement reports are in:
https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/disengagement_report_2017

First story I could find on this:
https://www.theverge.com/2018/1/31/16956902/california-dmv-self-driving-car-disengagement-2017

As expected, nobody but Waymo and GM logged more than a few thousand fully autonomous miles in CA in 2017.

In their report, GM states that nearly all of their miles are logged in SF:
"Last year we drove over 125,000 miles on San Francisco's complex city streets. All the attached data is from this urban driving. We drive in San Francisco because it allows us to improve more quickly. Cities like San Francisco contain significantly more people, cars, and cyclists that our self-driving vehicles must be aware of at any given time. That makes San Francisco one of the hardest places to test a self-driving vehicle, and creates a rich environment for testing our object detection, prediction, and response functions."

Yes, there is obviously testing in other states to consider, but these reports are one of the few windows into the otherwise secretive state-of-the-art in real-world testing of L4/L5 capabilities.