Uber self-driving car hits and kills pedestrian

[LetterRip]

There was a safety driver, so it might have been unavoidable (such as a person running out from between vehicles).  Uber has suspended their self-driving cars from all public testing for now.


https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html


https://www.wsj.com/articles/uber-suspends-driverless-car-program-after-pedestrian-is-struck-and-killed-1521480386

[JoshCrow]

Yeah, as an engineer I'm already cringing at the idea that people's poor understanding of statistics/risk and the media's desire for sensationalism will combine to create a totally baseless and irrational fear of self-driving vehicles.

I wish they added to this media story "and here's how many people were killed that same day by idiot drivers, and here's the fatality rate of human drivers vs. AI per vehicle...".

Anything that slows down our progress towards fostering a broad public acceptance of AI drivers is essentially contributing to the deadly status quo.

[D.W.]

Driver was present.
Pedestrian was not crossing at crosswalk.

Haven't heard other details yet.  But this sounds a lot more like the run of the mill, "accidents happen" incident, than a robot uprising requiring anything more than asking the question of "Can we do anything in terms of sensors to prevent this moving forward or did the pedestrian step out into traffic at a range from the car where it was unavoidable?"

[Seriati]

Yeah, as an engineer I'm already cringing at the idea that people's poor understanding of statistics/risk and the media's desire for sensationalism will combine to create a totally baseless and irrational fear of self-driving vehicles.

My "fear" isn't irrational.  I think self driving machines don't have adequate safety controls and security.  I think they are one step away from yet another breach of your rights when the government routinely inspects your travel logs, probably without even providing you notice that they have done so.

I'm concerned that the algorithms, even with machine learning, will never adaptive enough to predict the impact of all events that appear in a road context. 

I'm concerned about the impact of deliberate malicious manipulation of the code, terrorists have already shown us they are willing to use any device that they can control to harm people.  How hard is it to imagine conversion of driverless cars to the purposes of kidnapping, assassination, or just mass mayhem?

I wish they added to this media story "and here's how many people were killed that same day by idiot drivers, and here's the fatality rate of human drivers vs. AI per vehicle...".

Why? 

Anything that slows down our progress towards fostering a broad public acceptance of AI drivers is essentially contributing to the deadly status quo.

And to preserving your freedom and independence.

Driver was present.
Pedestrian was not crossing at crosswalk.

Haven't heard other details yet.  But this sounds a lot more like the run of the mill, "accidents happen" incident, than a robot uprising requiring anything more than asking the question of "Can we do anything in terms of sensors to prevent this moving forward or did the pedestrian step out into traffic at a range from the car where it was unavoidable?"

Nothing about a pedestrian crossing outside of the crosswalk screams to me that a driver should run them over.  In fact, I'm not sure why there's an argument that it wasn't the cars' fault.  I've certainly stopped for illegal crossers many times, including pedestrians, kids, deer, squirrels, balls, etc.  It's not even an unanticipated situation.

[JoshCrow]

My "fear" isn't irrational.  I think self driving machines don't have adequate safety controls and security.  I think they are one step away from yet another breach of your rights when the government routinely inspects your travel logs, probably without even providing you notice that they have done so.

I'm concerned that the algorithms, even with machine learning, will never adaptive enough to predict the impact of all events that appear in a road context.

What would you say if after years of use in <insert country here>, it was empirically shown that AI had safer judgement than humans 90% of the time? Would you be convinced that it surpasses human ability? What sort of performance would persuade you?

(btw, regarding surveillance, that particular cat has been out of the bag for some time in vehicles and has little bearing on who is driving)

 

I'm concerned about the impact of deliberate malicious manipulation of the code, terrorists have already shown us they are willing to use any device that they can control to harm people.  How hard is it to imagine conversion of driverless cars to the purposes of kidnapping, assassination, or just mass mayhem?

I can envision ways to abuse nearly any technology for mayhem - including cars with drivers. That's not an argument you would deploy against any technology you are already comfortable with by habit. Once AI is mainstream in cars, its occasional misuse will be notable and horrifying, but then we'll all get on with the 99.999% of the ordinary usage.

And to preserving your freedom and independence.

What about my freedom to do productive work (or even enjoy a book) while my car takes me somewhere? What about the independence of a blind or aged person who otherwise would be stuck calling an Uber?

[D.W.]

In fact, I'm not sure why there's an argument that it wasn't the cars' fault.

The question of fault becomes more interesting when property is "at fault". 

The only reason I pointed out the illegal / unexpected pedestrian crossing is that I don't know the speed the car was traveling at and the limitation of the sensors compared to a driver who MIGHT be better at anticipating the actions of the crosser than the car. 

I've stopped for deer, children, smart-phone zombies as well.  Working in Ann Arbor, you know that there are certain areas where you must treat everyone on the sidewalk as if they were suicidal and wanted to use your car as a means to end it all.  (freshman move in week) 

My point was it MAY not be the car's fault in that the average driver may not have been able to avoid the accident either.  JC's point is that the "average driver" kinda sucks and K.I.T. is still ahead of the curve even with this incident.

[Gaoics79]

There is simply not enough information yet to make a judgment. However, without a hint of hyperbole, (and a large amount of experience in this area) I would suggest that if this technology turns out to replicate human levels of reliability and safety, it would best to write off the whole enterprise, as the legal liability would be astronomical and unsustainable for the manufacturers

[Fenring]

There is simply not enough information yet to make a judgment. However, without a hint of hyperbole, (and a large amount of experience in this area) I would suggest that if this technology turns out to replicate human levels of reliability and safety, it would best to write off the whole enterprise, as the legal liability would be astronomical and unsustainable for the manufacturers

I would be surprised if the computers even as of their current level weren't already in the 90+th percentile of human safety standards. To me the question isn't whether the computers can do better than humans; it's clear they either can or will be able to soon. The question is whether there are obscure scenarios that occur where a human intelligence would adapt on the spot and deal with it whereas a computer would have difficulty assessing the correct action. These small 'holes' in programming are the main issue as I see it, if they exist.

But overall I agree entirely with JoshCrow on this matter. I see no good excuse to delay or obstruct the development of AI driving, and I will gleefully welcome the day when humans aren't even allowed to drive any more.

[Seriati]

What would you say if after years of use in <insert country here>, it was empirically shown that AI had safer judgement than humans 90% of the time? Would you be convinced that it surpasses human ability? What sort of performance would persuade you?

I'm not sure what you'd persuade me about.  I have no doubt that autodrive would be safer in a number of circumstances.  Turn it on for interstates.  Not sure I'd trust it on state highways as easily.  Parking lots?  Even less trust.

(btw, regarding surveillance, that particular cat has been out of the bag for some time in vehicles and has little bearing on who is driving)

Not sure that's the case, maybe when you use google maps, still that's a far cry from the kind of interconnection that would ultimately be required and recorded to route that many cars from point to point.  And it would only expand.  I have no doubt, that after a while, years of your specific locations would be obtainable. 

I can envision ways to abuse nearly any technology for mayhem - including cars with drivers. That's not an argument you would deploy against any technology you are already comfortable with by habit. Once AI is mainstream in cars, its occasional misuse will be notable and horrifying, but then we'll all get on with the 99.999% of the ordinary usage.

Well except you're asking us to entrust our lives directly to this technology.  Taking away our ability to stay safe, and accordingly the bar is higher. 

What self defense is available against a hacked car that kidnaps you?

What about my freedom to do productive work (or even enjoy a book) while my car takes me somewhere? What about the independence of a blind or aged person who otherwise would be stuck calling an Uber?

Lol, your argument is that I have to cede my freedom for your convenience?

I'm happy if you put them on the roads, with the caveat that harms caused invoke strict liability to the manufacturer and the owner. 

[LetterRip]

My "fear" isn't irrational.  I think self driving machines don't have adequate safety controls and security.  I think they are one step away from yet another breach of your rights when the government routinely inspects your travel logs, probably without even providing you notice that they have done so.

That does sound irrational.  I think the drive logs won't be anymore or less subject to search than existing logs of your phone location are.  So it is a wash as far as privacy.

I'm concerned that the algorithms, even with machine learning, will never adaptive enough to predict the impact of all events that appear in a road context.p

Since most humans are most of the time worse than the machines already I think this is an irrational fear.  Waymo cars have been driving in the same circumstances as humans for a significant period of time without incident.  I would be unsurprised if Uber cars are much poorer drivers - they don't have near as much R&D invested and have been functional for a far briefer time period.

  I'm concerned about the impact of deliberate malicious manipulation of the code, terrorists have already shown us they are willing to use any device that they can control to harm people.  How hard is it to imagine conversion of driverless cars to the purposes of kidnapping, assassination, or just mass mayhem?

Could do entirely remote control of a car with minor modifications today (hour of engineering/mechanical work with off the shelf parts, plus some testing time) and have far more control and successful usage.

I wish they added to this media story "and here's how many people were killed that same day by idiot drivers, and here's the fatality rate of human drivers vs. AI per vehicle...".

Why? 

Probably because AI controlled cars appear to be dramatically safer per vehicle mile than human driven cars.

Anything that slows down our progress towards fostering a broad public acceptance of AI drivers is essentially contributing to the deadly status quo.

And to preserving your freedom and independence.

I don't see how self driving wars will reduce freedom or independence.  Indeed they will likely greatly increase it (elderly, children, etc. - that might not be comfortable with being driven by a stranger who represents a safety risk).

Nothing about a pedestrian crossing outside of the crosswalk screams to me that a driver should run them over.  In fact, I'm not sure why there's an argument that it wasn't the cars' fault.  I've certainly stopped for illegal crossers many times, including pedestrians, kids, deer, squirrels, balls, etc.  It's not even an unanticipated situation.

Not enough information at this time to determine fault.  That said - there have almost certainly been tens of thousands of illegal crossers encountered by Waymo and Uber and other self driving cars without incident.  One of the main driving locations for Waymo and Uber test vehicles is right by ASU - with constant illegal crossings daily and I see their vehicles go by probably every minute or so throughout the day.  So just in the recent past there have probably been many thousands such illegal crossings safely navigated.

[LetterRip]

In fact, I'm not sure why there's an argument that it wasn't the cars' fault.

The question of fault becomes more interesting when property is "at fault". 

The only reason I pointed out the illegal / unexpected pedestrian crossing is that I don't know the speed the car was traveling at and the limitation of the sensors compared to a driver who MIGHT be better at anticipating the actions of the crosser than the car. 

I've stopped for deer, children, smart-phone zombies as well.  Working in Ann Arbor, you know that there are certain areas where you must treat everyone on the sidewalk as if they were suicidal and wanted to use your car as a means to end it all.  (freshman move in week) 

My point was it MAY not be the car's fault in that the average driver may not have been able to avoid the accident either.  JC's point is that the "average driver" kinda sucks and K.I.T. is still ahead of the curve even with this incident.

More information, the speed it was doing was 40.  The time of the accident was 10 pm.  The vehicle showed no signs of slowing.

]Uber’s self-driving car was traveling at a speed of 40 mph when it struck a 49-year-old woman in Arizona Sunday night, and did not show significant signs of slowing down, police said today.

https://www.theverge.com/2018/3/19/17140936/uber-self-driving-crash-death-homeless-arizona

[JoshCrow]

I'm not sure what you'd persuade me about.  I have no doubt that autodrive would be safer in a number of circumstances.  Turn it on for interstates.  Not sure I'd trust it on state highways as easily.  Parking lots?  Even less trust.

Well, trust is built. Eventually the record would speak for itself, wouldn't it? And if at that point you're just Grandpa Simpson yelling at a cloud that's going to be your fault.

Not sure that's the case, maybe when you use google maps, still that's a far cry from the kind of interconnection that would ultimately be required and recorded to route that many cars from point to point.  And it would only expand.  I have no doubt, that after a while, years of your specific locations would be obtainable. 

I've used OnStar for some time in my vehicle, mostly because I like the idea that if a criminal steals my car there's a chance they're foolish enough not to know how to disable it. I also leave my phone and GPS on to contribute to traffic data, which has saved me lots of time on the road heading to and from work. I'll give you that having no way to turn something off is a little different, but the data being collected tends to create convenience and to my knowledge has not much implicated innocent people.

Well except you're asking us to entrust our lives directly to this technology.  Taking away our ability to stay safe, and accordingly the bar is higher. 

Well, have you flown on an airplane before? You've entrusted your life to technology. 2017 was a banner year for aviation safety, which has gotten better with time.

What self defense is available against a hacked car that kidnaps you?

I'd imagine the brake pedal can do that.

Lol, your argument is that I have to cede my freedom for your convenience?

I'm just pointing out that "freedom" works both ways, not just "the way I want". Your freedom to drive vs. my freedom to not be driven over by a drunk. By the way, in that comparison it's YOUR freedom that looks a lot more like "convenience".

[Fenring]

What about my freedom to do productive work (or even enjoy a book) while my car takes me somewhere? What about the independence of a blind or aged person who otherwise would be stuck calling an Uber?

Lol, your argument is that I have to cede my freedom for your convenience?

Not sure how you can invoke "freedom" to the issue of driving a car, since (to use your phrasing) there is literally no law or even principal suggesting that people should be free to drive cars. Perhaps you mean "license"? It's true that banning human drivers (which to be fair is my ideal, but not necessarily the ideal of everyone advocating for AI drivers) would remove the license to drive, but for it to be a matter of freedom would imply that the government is preventing something that ought to be a natural human action, which driving a car certainly is not. On the contrary, being able to drive a car around is the convenience in the equation, which has thus far also been the most efficient mode of self-transport. We could even look at automobile deaths per year and suggest that it's a public safety service to eliminate human drivers, and that you would be the one willing to sacrifice lives for the sake of your convenience. I'm not directly saying that's the case, but the argument would theoretically hold water. And unlike the 2nd Amendment, there's no law linking a driver's license to any natural human right, so the "saving lives" argument would most certainly be admissible in the automobile situation as a reason not to allow human drivers. For my part the reason is more about systemic efficiency than saving lives: I personally believe that an AI driving grid would mostly eliminate traffic and prevent car accidents, both of which waste hours in the days of so many people that if you total the hours gained by removing needless traffic jams created by idiots it would probably be equivalent to "saving" countless lives per day (calculating hours saved per person per year, adding that all up together and totally how many total lives lived that would equal). It would also increase morale dramatically, I think, as well as greatly increasing incentives to get in the car and go somewhere - like to the store - without being worried about sitting in traffic and regretting the decision to go out. So I therefore additionally think that business would benefit from it, in possibly a significant way, and that's even putting aside the business of catering to the AI driving network itself.

So you save lives, probably gain jobs, reduce traffic, and make life more enjoyable. And this is supposed to be outweighed by giving people the "freedom" to screw up other drivers because they want the power of the steering wheel? And we all know everyone thinks it's the other guy who's the bad driver and causes all the slowdowns.

[yossarian22c]

While the Tempe Police Department has declined to determine fault yet, the city's police chief, Sylvia Moir, told The San Francisco Chronicle that from viewing videos of the collision taken by the car's onboard cameras, "it's very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway."

https://www.npr.org/sections/thetwo-way/2018/03/20/595115055/arizona-governor-helped-make-state-wild-west-for-driverless-cars

[DonaldD]

What self defense is available against a hacked car that kidnaps you?

I'd imagine the brake pedal can do that.

Or an on/off override.

Of course, it would be far easier just to stick a gun to somebody's head and tell them to hop into this comfortable trunk.

[msquared]

Serati

My guess is that eventually, the insurance to drive yourself will cost much more than if you have an AI driver. When the AI systems become more wide spread and the V2V portion really kicks in, you will see the number, and severity, of accidents drop to astoundingly low levels. And when the numbers show that most of the accidents (and the worst of them) are caused by manual drivers, I would not be surprised to see manual drive outlawed. Driving is not a right, so laws like that can be passed.

msquared

[DonaldD]

but for it to be a matter of freedom would imply that the government is preventing something that ought to be a natural human action

Let me Godwin this: in the same way as the natural right to self defense leads to the right to wield firearms, the natural right that is freedom of movement leads to the right to drive cars using a steering wheel.

[Fenring]

but for it to be a matter of freedom would imply that the government is preventing something that ought to be a natural human action

Let me Godwin this: in the same way as the natural right to self defense leads to the right to wield firearms, the natural right that is freedom of movement leads to the right to drive cars using a steering wheel.

Is that a real argument, or are you anticipating this argument and satirizing it in advance?

[D.W.]

Is that a real argument

The first time the headline reads, "Automated Car Kills Driver While Avoiding Pedestrian", it might be.

[LetterRip]

More updates,

the police chief has stated that she stepped off the center divider into the path of the car and based on the video footage they don't believe the AI/car/driver to be at fault.

“I suspect preliminarily it appears that the Uber would likely not be at fault in this accident," said Chief Sylvia Moir.

Herzberg was "pushing a bicycle laden with plastic shopping bags," according to the Chronicle's Carolyn Said, when she "abruptly walked from a center median into a lane of traffic."

After viewing video captured by the Uber vehicle, Moir concluded that “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway."

https://arstechnica.com/cars/2018/03/police-chief-uber-self-driving-car-likely-not-at-fault-in-fatal-crash/

[msquared]

This goes along with almost everything else I have heard about self driving cars in accidents. In over 90% of them, they have not been at fault (for example, they get rear ended when they stop for some pedestrian dashing out in front of them). Also, from what I have seen, when they are involved, the severity of the accident is less, since they were able to respond faster than a human.

Self driving cars will not eliminate accidents, but they will reduce there number and severity.

[DonaldD]

Let's not kid ourselves, though: there will be bugs/hardware failures/software failures.  So there will be accidents that would not have otherwise occurred if a human driver was at the wheel.  But those should become fewer and fewer (per capita) as the technology improves.  Those failures will likely never disappear completely.

[TheDrake]

Self-driving cars can see 360 degrees. They do not get tired. They don't drive drunk. They don't text. They don't get distracted by phone calls, kids, attractive pedestrians, or talk radio. They can see outside the visible spectrum, making them more aware of darkly clothed pedestrians or vehicles with their lights off.

If I were given a choice to walk blindly into the road, I'll prefer to risk the self-driving car rather than the average human.

But the general public has been given the idea that technology should be infallible rather than just better, and that's where things get strange. As far as fault in this particular case, I imagine a lot of engineers aren't going to get much sleep this month. Luckily, with a self-driving car, there is a host of information not available for the average accident. As LR points out, we have the clear video.

That said, it is inevitable that we will eventually have an incident where there is clear judgement of fault for the self-driving car's reaction that results in fatalities. This is especially true when self-driving technology migrates to more dangerous vehicles like buses, and when it gets used in adverse conditions like heavy fog or snow. Then a whole legal can of worms opens up. The insurance company blames the technology provider and wants compensation. Not that different from today, but it forms a basis for a class action by all the users of that technology that they were put negligently at risk.

[Seriati]

My "fear" isn't irrational.  I think self driving machines don't have adequate safety controls and security.  I think they are one step away from yet another breach of your rights when the government routinely inspects your travel logs, probably without even providing you notice that they have done so.

That does sound irrational.  I think the drive logs won't be anymore or less subject to search than existing logs of your phone location are.  So it is a wash as far as privacy.

First of all, barely a week goes by without a major story of a hack of a government agency (power grid by Russian hackers last week) or of a corporation, not to mention of a political campaign.  Not one piece of technology has so far been proof from malicious actors.

How exactly is it irrational to believe that computer controlled vehicles are subject to the same risk?  It actually sounds to me like complete irrationality to believe they wouldn't be.

As to data availability, google does track you, stores track, and locations and venues track you, or sometimes they just track that someone is there.  None of that information is as comprehensive as a travel log would be, you literally would be visible, everywhere you go at all times.  Those records will be subpeonable, in a divorce proceeding, when your employer is looking to fire you, in commercial disputes, and in criminal investigations.  We've already seen from the NSA's actions, that if the records exist the government will collect them, which means probable cause and your civil rights are going to be things of the past.

I'm concerned that the algorithms, even with machine learning, will never adaptive enough to predict the impact of all events that appear in a road context.p

Since most humans are most of the time worse than the machines already I think this is an irrational fear.  Waymo cars have been driving in the same circumstances as humans for a significant period of time without incident.  I would be unsurprised if Uber cars are much poorer drivers - they don't have near as much R&D invested and have been functional for a far briefer time period.

It's a fair point.  I agree cars may be safer self drivers than human driver.s

Could do entirely remote control of a car with minor modifications today (hour of engineering/mechanical work with off the shelf parts, plus some testing time) and have far more control and successful usage.

But you have to have access to the vehicle.  There is zero chance that self drive cars will not have connectivity wirelessly and there's no modifications needed to the physical systems.

I wish they added to this media story "and here's how many people were killed that same day by idiot drivers, and here's the fatality rate of human drivers vs. AI per vehicle...".

Why? 

Probably because AI controlled cars appear to be dramatically safer per vehicle mile than human driven cars.

That's of course why I asked the question.  The whole point of this thread is to declare that safety is the only relevant concern, which is exactly why you want that focus.  There are other factors than safety that matter.

I don't see how self driving wars will reduce freedom or independence.  Indeed they will likely greatly increase it (elderly, children, etc. - that might not be comfortable with being driven by a stranger who represents a safety risk).

If you can't see how, tracking your every movement reduces your freedom then I don't think you're trying very hard.  What if say, Trump decides to put a special focus on everyone who met with a certain political figure or group, all it would take is downloading the travel records (they already have the phone records) and they could specifically identify the lot of you.

We already have a situation where the police are almost completely dependent on technological aids to solve a crime, what happens when that gets worse and people figure out how to manipulate it?  Say, someone commits a murder and hacked your car to go the location at the time of the murder.  The first thing that's going to happen is the police are going to look at the local travel net reports for anonymous vehicles.   I can think of tons of mischief from being able to "put" someone someplace they weren't.  That's assuming that the records themself aren't easy to manipulate.  Pay a hacker, and you have "proof" of your wife's affair.

Or you know you could rely completely on the privacy settings of the tech company, like say trusting Facebook not to have delivered private information to Cambridge Analytics for use in targeting political messages.

[LetterRip]

How exactly is it irrational to believe that computer controlled vehicles are subject to the same risk?  It actually sounds to me like complete irrationality to believe they wouldn't be.

I find it irrational in that it provides little or no marginal risk.  The ship of 'constant tracking of my location' has sailed many years ago - and there are far more convenient methods for the government to get such data than via self driving cars.

Also the cars tracking data is less likely to be personally associated with you than your phone data.

Also almost every new car has GPS that constantly tracks and reports your location, the self-driving AI is unrelated to that concern.

As to data availability, google does track you, stores track, and locations and venues track you, or sometimes they just track that someone is there.  None of that information is as comprehensive as a travel log would be, you literally would be visible, everywhere you go at all times.

Your phone has a far more detailed log of your movements than your car would have.  Also self driving AI don't have to report their location or store their history (they probably will, but it isn't a design requirement).

Those records will be subpeonable, in a divorce proceeding, when your employer is looking to fire you, in commercial disputes, and in criminal investigations.  We've already seen from the NSA's actions, that if the records exist the government will collect them, which means probable cause and your civil rights are going to be things of the past.

If you've bought or driven a car in the past 5-10 years - all of that is already available.  OnStar has been around since 1996.

https://www.nytimes.com/2017/07/27/automobiles/wheels/car-data-tracking.html

But you have to have access to the vehicle.  There is zero chance that self drive cars will not have connectivity wirelessly and there's no modifications needed to the physical systems.

Ah - so you are thinking the 'remotely hack into the system'.  The authentication system that needs to be cracked to do so is going to be beyond most nation states.

If you can't see how, tracking your every movement reduces your freedom then I don't think you're trying very hard.

See above.  There is a good chance your car already tracks your every movement - no AI needed.  Your phone definitely does so already.  The self-driving AI doesn't make any of this worse.

  What if say, Trump decides to put a special focus on everyone who met with a certain political figure or group, all it would take is downloading the travel records (they already have the phone records) and they could specifically identify the lot of you.

See above - that scenario is already possible with the standard features of current cars and current phones.  The 'self driving AI' doesn't change anything.

We already have a situation where the police are almost completely dependent on technological aids to solve a crime, what happens when that gets worse and people figure out how to manipulate it?

People can already manipulate it.   If I have access to your phone I can falsify a complete travel history for you.  A nation state resources would probably be required to falsify the cell tower pings though - same issue for driving logs.

Say, someone commits a murder and hacked your car to go the location at the time of the murder.

That would be far harder than what could be currently done - steal your car, run a red light that gets a photo of your license plate.
 Or renting a car that looks like your own, and forging license plates, etc.  Framing using a self-driving car will actually be more difficult.  Especially if you have a phone that is tracking you at the same time.

The first thing that's going to happen is the police are going to look at the local travel net reports for anonymous vehicles.   I can think of tons of mischief from being able to "put" someone someplace they weren't.  That's assuming that the records themself aren't easy to manipulate.  Pay a hacker, and you have "proof" of your wife's affair.

Hackers taking control of a car makes for great TV - but the reality is that it will likely be nation state level R&D to do so and probably not even then.

Or you know you could rely completely on the privacy settings of the tech company, like say trusting Facebook not to have delivered private information to Cambridge Analytics for use in targeting political messages.

Actually you've misread what happened.  A third party had a facebook app.  That app harvested more data than is allowed by the facebook TOS.  The owner of the app then further violated the TOS to pass that data to Cambridge Analytics.

Regarding privacy settings - I suspect that facebooks privacy settings probably did prevent info being harvested for those who had it setup.  Most people don't have their privacy settings such that that would be prevented.

[TheDrake]

Tracking data? you mean like maps.google.com/timeline? Which has my location for the last 5+ YEARS? Including everywhere I went on foot, airplane, public transit, uber, and rental car? Far more comprehensive, with the exception of the times I ran out of charge. It's actually kind of fun. Jan 1, 2013, I clearly didn't leave the house.

Now, you can certainly turn this feature off, but I don't know why you would be any less able to turn such a feature off in a self-driving car (as long as it wasn't "in charge"). Meanwhile, having it on has helped me locate my lost phone, find the name of that cool restaurant that I went to last time I was in Boston, even dig up a friend's address.

Everyone should be able to have that option, and I don't see why you couldn't keep location data local and on an erase schedule of your choosing. Now, you can say you don't trust the companies involved, but to avoid this risk you'd have to avoid having a smartphone - since "bad google" could clearly track your data all the time if they were willing to do so, especially now that batteries are not removable.

[Fenring]

All major devices right now have been built with backdoors in place allowing the government to hack your phone anytime they want. This is true for devices and computers, and these backdoors are in the hardware, not software. Currently there is no way to keep the NSA (or hackers) out if they know the system and want to get in, so paranoia about being tracked at this point is moot. Regarding hacking the car for a hostile party to sabotage its operation is another matter. Assuming for the moment that the car is hooked into the 'grid' and is using GPS among other things, I would hope that the local AI would be shielded from outside interference and that the only interface between the 'real world' and the car would be its accessing GPS data and other things like from your phone. In other words, it should be a one-way information flow to the car, not from the car. I could see an issue where if someone literally sabotaged the GPS network the car would therefore not have the ability to know where it's going any more, and so this area would have to be secured in a major way, but that's a far more specific area of concern than "they'll hack my car". And, having seen the I, Robot film, I do agree with Seriati that I don't particularly care for the thought of the world's AI systems talking to my car and conspiring against me. So I think the security here would have to involve the car's internal processes being totally inaccessible from the outside, and only perhaps able to be altered by a licensed mechanic with hard mechanisms in place to protect it.

[LetterRip]

So I think the security here would have to involve the car's internal processes being totally inaccessible from the outside, and only perhaps able to be altered by a licensed mechanic with hard mechanisms in place to protect it.

There will probably have to be the ability for remote control by Waymo operators if there is something abnormal that the AI can't handle.  This might be able to go away eventually as the unique and abnormal problem situtations become fewer and further between.

[Seriati]

Lol, so the government can already see you at will, but they'd say never, redirect your car to their detention center?  They'll start by taking "terrorists."  I'm just surprised that several you acknowledge abusive data practices, and then waive your hands at the issue as if nothing can be done.  How about data privacy legislation?  How about mandatory wipes of records and barring storage?

[Seriati]

How exactly is it irrational to believe that computer controlled vehicles are subject to the same risk?  It actually sounds to me like complete irrationality to believe they wouldn't be.

I find it irrational in that it provides little or no marginal risk.  The ship of 'constant tracking of my location' has sailed many years ago - and there are far more convenient methods for the government to get such data than via self driving cars.

Also the cars tracking data is less likely to be personally associated with you than your phone data.

Also almost every new car has GPS that constantly tracks and reports your location, the self-driving AI is unrelated to that concern.

Position level data, at least so far, isn't constant and perfect.  Big data does make a lot of guesses.  Route information would be a step beyond what's there today.

Your phone has a far more detailed log of your movements than your car would have.  Also self driving AI don't have to report their location or store their history (they probably will, but it isn't a design requirement).

Self drive AI will never stay isolated.  It'll connect into local and national data services that track road conditions, accidents and current maps, it'll need constant communication for updates on conditions with immediate impact on the roads.  It'll connect, all most immediately, into traffic control webs that specify red lights and government routing decisions.

If you've bought or driven a car in the past 5-10 years - all of that is already available.  OnStar has been around since 1996.

Fair enough, but not everyone has OnStar nor is it criminal to disable it.  Whereas, the idea here would lead to prohibiting direct drivers over time. 

In any event though, I think we ought to be pushing back on use of personal data generally.

Ah - so you are thinking the 'remotely hack into the system'.  The authentication system that needs to be cracked to do so is going to be beyond most nation states.

And how will long will it stay there?  Honestly, you seem awfully confident given the history of the uncrackable becoming crackable with more advanced technology.

That's assuming that they don't include deliberate backdoors, which we know they will, you cited a few yourself, emergency overrides and I'm sure police controls.  Are you 100% confident that the passwords to a police override won't be a hack vector?  Perfect encryption is only as perfect as the weakest human link.

See above.  There is a good chance your car already tracks your every movement - no AI needed.  Your phone definitely does so already.  The self-driving AI doesn't make any of this worse.

I don't always carry my phone.  I frequently have location services off when I do (not that that does any good).

That would be far harder than what could be currently done - steal your car, run a red light that gets a photo of your license plate.
 Or renting a car that looks like your own, and forging license plates, etc.  Framing using a self-driving car will actually be more difficult.  Especially if you have a phone that is tracking you at the same time.

Neither of which actually works when your car is reporting its location as in your driveway.

Actually you've misread what happened.  A third party had a facebook app.  That app harvested more data than is allowed by the facebook TOS.  The owner of the app then further violated the TOS to pass that data to Cambridge Analytics.

FaceBook had pretty lax security with their App developers.  Of course it's also come out that they let the Obama campaign harvest information as well (with their knowledge).

By the way I didn't misread anything.  Nothing about what you said contradicts what I said.

Regarding privacy settings - I suspect that facebooks privacy settings probably did prevent info being harvested for those who had it setup.  Most people don't have their privacy settings such that that would be prevented.

Pretty sure that FB was already under a cease and desist based on failing to limit data sharing to the sharing allowed under a users privacy setting.  And pretty sure that they are under a new investigation for failing to meet the requirements of that order.

Web scrapping, is yet another practice where our government has failed to keep pace with technology.

If you guys really want to argue that you should have no privacy, please feel free.

[TheDrake]

It isn't that privacy is a non-issue, while I'm not worked up about it (my choice) - I do believe that people should have a choice that isn't restricted to dropping off the grid entirely. I just see so many larger threats that the addition of self-driving cars seems limited. For one thing, it will be a decade before they are even common in urban areas with sunny weather. It will be possible to avoid, unless DOT makes a rule that all cars must have self driving technology (turned on or off) - like a seatbelt rule.

IoT, migration away from anonymous financial transactions (cash), social media, cloud storage, search results, just about everything seems more dangerous to privacy in the immediate term.

I'll trade a lot of privacy away for the luxury of being able to watch TV and make wild claims on ornery during my daily commute. I have no real and immediate concerns of being framed for a felony, kidnapped, denied insurance, or being blackmailed - regardless of the technology being employed.

Could I become more worried? Sure. The worst case scenario is something like the movie Speed interbred with ransomware. If companies will pay to get their data unlocked, what will they pay to keep the CEOs wife from being rammed into a bridge abutment? Currently any kidnapping and ransom scenario involves physical risk and an elevated risk of being caught - plus the difficulties in acquiring physical access to the intended victim, who may likely be protected in various ways.

I wouldn't go so far with bold "unhackable" claims. The US Army managed to leave drone intelligence video unencrypted after all. By making it easier and safer to do such things, it might encourage more of it. I would guess that much like internet usage, the people who really care will have methods. Like folks that use a VPN service to avoid having their browsing traffic intercepted by ISPs. People who worry about email security use PGP. People who worry about text message security use What's App with end-to-end encryption. Most of us however will give two shifts about it, and I sure as hell don't want government to step in and make the transparent open stuff illegal, and I don't want them blocking deployment of cars that can continuously transmit their locations and drive more safely than the average person.

[LetterRip]

Lol, so the government can already see you at will, but they'd say never, redirect your car to their detention center?

No, they can't "see you at will" - they have to get a warrant.  As to 'redirect your car' - it would be kidnapping without a warrant, and they could only do so with the cooperation of Waymo/Uber/etc.

I'm just surprised that several you acknowledge abusive data practices, and then waive your hands at the issue as if nothing can be done.  How about data privacy legislation?  How about mandatory wipes of records and barring storage?

I think there should be legal protections, I just don't think it is different issue from what already exists.

[LetterRip]

Position level data, at least so far, isn't constant and perfect.  Big data does make a lot of guesses.  Route information would be a step beyond what's there today.

My phone is within the same location accuracy as the GPS of the car I'm in - usually 2 m or so.

Self drive AI will never stay isolated.  It'll connect into local and national data services that track road conditions, accidents and current maps, it'll need constant communication for updates on conditions with immediate impact on the roads.  It'll connect, all most immediately, into traffic control webs that specify red lights and government routing decisions.

I'm not sure what 'government routing decisions' would be relevant.  Again - my phone and GPS do all of this already - most cars do - that ship has sailed and is irrelevant to self driving cars.

Whereas, the idea here would lead to prohibiting direct drivers over time.

You won't be able to solve this by banning self driving cars.  The AI are completely irrelevant to the issue.  Within the time frame this will happen - there probably won't be anyone who doesn't have a cell phone that is doing this tracking 24/7.

In any event though, I think we ought to be pushing back on use of personal data generally.

Completely agree - that should be tackled via electing officials that enact privacy laws and buying privacy enacting products.  AI isn't really a factor either way.

And how will long will it stay there?  Honestly, you seem awfully confident given the history of the uncrackable becoming crackable with more advanced technology.

There have been some bugs in implementations but largely things have remained roughly as secure as predicted.  Most of the security things you have heard about - were utterly incompetents who didn't care at all about security - royally screwing up.

That's assuming that they don't include deliberate backdoors, which we know they will, you cited a few yourself, emergency overrides and I'm sure police controls.

It isn't a backdoor - it is integrated feature.  Leaving a backdoor would be dangerously stupid, and open the corporations up to liability.

Are you 100% confident that the passwords to a police override won't be a hack vector?  Perfect encryption is only as perfect as the weakest human link.

There isn't a police override, why would their be?  They contact Waymo/Uber with the proper authorization and then they do the override.  There will almost certainly never be a root password that works on all cars.  There will be a unique per card encryption key that can only be installed at the factory.

I don't always carry my phone.  I frequently have location services off when I do (not that that does any good).

You could walk, bike, etc. for the same privacy.  Any distance far enough that you would need to drive is also far enough that you will inevitably have your phone with you.

Neither of which actually works when your car is reporting its location as in your driveway.

I'm not clear what scenario of framing you have dreamed up.  Regardless, it will always be easier now, than hacking a self driving car.

By the way I didn't misread anything.  Nothing about what you said contradicts what I said.

What you had said was factually incorrect.  What I said was factually correct.

If you guys really want to argue that you should have no privacy, please feel free.

I'm arguing that even if you completely ban AI it will have close to zero impact on your privacy, since other existing technologies are far more effective in such 'invasion', and those technologies are far more integrated into our lives (and growing more so).  So the only way to 'fix' such privacy concerns is via legislative action.  Worrying about the specific technology itself is sort of silly in my view.

[D.W.]

As to 'redirect your car' - it would be kidnapping without a warrant, and they could only do so with the cooperation of Waymo/Uber/etc.

I fully expect that they will be networked in such a way that municipal law enforcement and first responders will have override capabilities. 

"We need to coordinate cars pulling off to the side to allow emergency vehicles to pass." 

OK, sounds legit. 

"Police need the ability to pull over automated cars.  The time it takes to report a vehicle, contact the manufacturer/service provider, issue the order and have it complied with could cost lives.  The patrol cars should have an override built in."

Hmm, ya, I can see that.

"You may as well just give us remote control override keys to the system.  It would be easier than coding these situational protocols.  It would also let us react to unanticipated situations."

I... I don't know...

[TheDeamon]

As to 'redirect your car' - it would be kidnapping without a warrant, and they could only do so with the cooperation of Waymo/Uber/etc.

I fully expect that they will be networked in such a way that municipal law enforcement and first responders will have override capabilities. 

"We need to coordinate cars pulling off to the side to allow emergency vehicles to pass." 

OK, sounds legit. 

"Police need the ability to pull over automated cars.  The time it takes to report a vehicle, contact the manufacturer/service provider, issue the order and have it complied with could cost lives.  The patrol cars should have an override built in."

Hmm, ya, I can see that.

"You may as well just give us remote control override keys to the system.  It would be easier than coding these situational protocols.  It would also let us react to unanticipated situations."

I... I don't know...

One thing to point out here: Most of what you pointed out are valid uses. But they're also "area effect" uses, rather than specific vehicles.

"We need all traffic to divert away from this roadway" is a bit different from "We want to redirect DW's car to this specific location."

Now that isn't to say that a clever LEO-type(or other) can't engineer a situation where they might be able to target such an area-effect broadcast to a specific vehicle. But at that point you're talking about odd hours of the day or mostly deserted roadways.

[LetterRip]

Here is a site that has video footage, both forward facing and of the driver.

https://jalopnik.com/video-shows-driver-in-fatal-autonomous-uber-crash-was-l-1823970417

It looks like the driver is texting or otherwise using her phone.  Her eyes are clearly not on the road for periods of time of 2-4 seconds.

The forward facing video there is maybe 1-2 seconds that you can see the biker before collision - she just appears out of a major shadow and takes two steps before the collision (video cuts out without showing the collision).  A driver of typical alertness probably wouldn't have seen her in time to have time to react sufficiently to avoid the collision.  A professional driver with amazing vision and reflexes might have been able to slam on the brakes, but definitely still would have hit her - though not necessarily fatally.

[TheDrake]

This should have been a case for infrared, lidar, and other sensors outperforming a human. It is nearly impossible to see this person. The engineers have work to do.

This is a scenario that plays itself out all too often long before autonomous cars - a pedestrian in dark clothing in a dark area who clearly didn't look down the road before crossing coupled with an inattentive driver and a street not designed with pedestrian traffic in mind (there could be an on-demand stoplight at that area).

A good reminder for all of us who ride bikes to make sure we've got functioning lights on our bicycles, I know I've been lax sometimes on that when batteries run out and I'm on neighborhood streets (not an eight lane throughfare). There's also a good point to be made for eight lane arteries to have fencing up forcing people to utilize managed crosswalks. All in addition to improving technology that can mitigate those factors.

[Mynnion]

In the long run this ladies death may lead to additional detection technologies that may save multiple lives in the long run.  As it is I don't believe that anyone, even a professional driver at peak alertness could have avoided hitting her.  That being said what is our expectation moving forward?  Do we let the cars back on the road now or do we wait until they have added additional safety features?

[LetterRip]

This should have been a case for infrared, lidar, and other sensors outperforming a human. It is nearly impossible to see this person. The engineers have work to do.

Yes I'm interested in the lidar/radar and infrared (if any?) sensor picture as well.  They should also use two camera sensors for increased dynamic range since the blown out blacks are useless.  For the lidar it really seems like it should have picked her up far earlier.

This is a scenario that plays itself out all too often long before autonomous cars - a pedestrian in dark clothing in a dark area who clearly didn't look down the road before crossing coupled with an inattentive driver and a street not designed with pedestrian traffic in mind (there could be an on-demand stoplight at that area).

Agreed, and given how frequently it occurs we should have seen this sort of thing a 100 times already if the sensors didn't typically catch this.  So it is suggestive either Ubers design is defective compared to Waymo and others, or they had an unlucky sensor or software fault at just the wrong time.

[Fenring]

I have to wonder how sympathetic I should be for a situation where a person is effectively committing suicide by car. That the literal intention of the person was probably "I want to cross the street" doesn't materially change the fact that not caring whether there are cars or not is effectively like saying "I don't care if I live or die". I'm not making a direct equivalence, but rather suggesting that I don't think overall effectiveness of self-driving AI should be based on the behavior of pedestrians who don't take the basic precautions necessary to live, such as not jumping in front of a car with no notice. This kind of scenario can surely be improved upon by the AI, and maybe with fancy sensors even this kind of scenario can have a happy ending, but I personally don't consider this type of fatality to be any strike against the system as-is, if I'm understanding it correctly.

Where I live the pedestrians often have a mindless disposition where they'll cross in numbers at random spots across 4-lane roads, under the groupthink assumption that because there are many of them at once nothing can happen to them. But then you also see individuals crossing while texting, or on the phone, or even doing nothing in particular but totally uninterested in whether there's traffic coming while they cross on a red. I've seen people walk directly into moving traffic on busy roads, and when the cars come screeching to a halt and the horns start honking in rage, they react in such a way as to suggest "what? what's your problem?" They are sometimes unable even to recognize that the will and reflexes of strangers saved their life. I've had to jam on the brakes when someone crossed on a red directly in front of me while I was about to enter an intersection, and upon stopping 2 feet from them, they don't even turn their head or react and just slowly continue sauntering to the next corner, unperturbed and locked inside their mental bubble.

Of people such as I'm describing here I have very mixed feelings. I believe that life is paramount, and that great steps should be taken to safeguard our fellow man, and yet a part of me also feels that someone who doesn't feel it's necessary to bother taking even the most basic steps to protect themselves from instant death, and who implicitly just allows everyone in the world except themselves to watch out for them and take care of them - almost willfully, as in "you all had better do your jobs and make way for me when I walk" - for such people I have difficulty weighing whether we're doing evolution a disservice by slamming on our brakes and risking a 10 car pileup in order to help these people breed. I'm basically stuck on the fence between complete bleeding heart and total fascist on this one, and not sure what to say other than I don't think these kinds of scenarios should be relevant when deciding how effective AI cars are.

[TheDeamon]

In the long run this ladies death may lead to additional detection technologies that may save multiple lives in the long run.  As it is I don't believe that anyone, even a professional driver at peak alertness could have avoided hitting her.  That being said what is our expectation moving forward?  Do we let the cars back on the road now or do we wait until they have added additional safety features?

One thing to note here:

This was the dashcam footage near as I can tell. This is the telemetry or other video feed that the car itself was using to operate.

Further, I know from personal experience with multiple dashcams and driving at night, that their nightvision is usually much worse than even human vision. This for me was most demonstrable driving down the highway where I could read roadside signs from some distance away, but on the dashcam you had to wait until you were almost ready to pass the sign before the camera could read it.

The Uber Driver was obviously distracted, and likewise his nightvision was likely poor because of it.

The person who died had more than their own fair share of blame to be had in this. I noticed no reflective devices on the bike wheels, and further, they were walking with their body between oncoming traffic and the bicycle. That the only "light-colored" article of clothing she was wearing were white shoes didn't help either.

Having the bicycle on the side of oncoming traffic might have helped a little. Having reflective (side) markers on the bicycle wheels would have likely helped significantly(at least on the human operator side). Further, wearing more reflective clothing in general would have likely changed the outcome as well.

Choosing a better lit place to cross would have been the penultimate safety step she could have, and should have taken. Apparently jaywalking just further compounded things.

An ancillary thing to me is how she seems to be oblivious to the oncoming car until it's too late. If I were in her position(which I wouldn't have been-but then I'm more safety conscious/aware than the average bear) I would have ditched the bike and booked it for the side of the street. But more likely I would have waited in the center median until the oncoming car passed before proceeding across, it doesn't seem there was much traffic on that street, waiting 10 to 25 seconds for that car to pass or turn away wouldn't kill me.

Obviously, not waiting did kill her because she expected the other person to see her and react appropriately. Which is exactly the wrong approach to make when dealing with motorists in general even in good lighting/weather conditions.

But then, not everyone gets to have the experience of driving down a dark frontage road in Dallas, Texas seeing what looks like a car stalled out in the left-hand land with hazards on(and traffic in the right preventing an immediate lane change)... Only to discover that there is another, second car about 80 feet behind it(closer to me) with its hazards on as well, only you couldn't see them because you have three black people in dark clothes standing behind that car positioned such that they block sight of the hazards(I guess they thought it would back-light them) waving white shirts around. (Because everyone knows "white is a reflective color")

I came within 30 feet of running both them and their car over with an 18 wheeler, and the only reason I didn't was I knew something wasn't right around there because of the other car further down the road.

[LetterRip]

Here is a video of the same area, the camera is set to a different gain.  The visibility is clearly much better, based on this video - I'd think the accident was fully avoidable by the driver and/or properly working equipment/sensors.

https://www.youtube.com/watch?v=CRW0q8i3u6E

[TheDrake]

Either way, I wouldn't call it "fully avoidable" but if this is truly representative, then I'd say "possibly avoidable or able to reduce harm".

It was definitely "fully avoidable" by the pedestrian however. I still agree that there is work to do and the engineers need to scrub telemetry and trace logs to determine what the car saw. Too bad we're unlikely to find out if there was a sensor failure (redundancy?), bad algorithm, confounding issue. Probably they built it on windows and it decided to patch itself. Hourglass, hourglass, Pedestrian!

It would also be interesting to know how many similar situations have occurred where the human driver did step in to save the day - not data that would need to be reported. When qualifying the "student AI driver" and giving it its license, what are the criteria? How many 17 year old inexperienced drivers would have handled this situation correctly? This is essentially like if a student driver struck somebody while the instructor sat there and never hit the brake. If I were to lay the most blame, it has to be with the "safety" driver, which kind of highlights how dangerous humans are.

[Seriati]

Here is a video of the same area, the camera is set to a different gain.  The visibility is clearly much better, based on this video - I'd think the accident was fully avoidable by the driver and/or properly working equipment/sensors.

https://www.youtube.com/watch?v=CRW0q8i3u6E

I find it hard to reconcile those two videos.  If this is accurate, I'd be willing to believe that an alert human driver would not have made this mistake.  I stipulate that human drivers make other mistakes that a machine would avoid.

I'm also concerned about diagnositics. While there are many conditions that can impact a human driver, tiredness, sickness, drunkness, the vast majority are detectable by a responsible person.  Tech issues are very difficult to deduce, particularly for a machine.  If a camera is inefficient, or has a arc error, how robust is the self diagnostic?

Heck, I'm aware that even with our direct description of a technical problem with our car (the car drops calls in areas where the phone is capable of maintaining the call), it's seemingly impossible for the computer  to log the error or the technicians to track down what's causing it. 

[TheDrake]

Clearly there are a lot of challenges. Is it foggy out, or is the camera obscured by smudges on the lens? What about radar calibrations, as circuits change over time (which is why police radar guns need to be adjusted for accuracy)?

The usual answers for this is to have redundancy, that's why space vehicles typically have three of everything. When one starts disagreeing, you know that's the bad one.

Just found this great article about the sensor arrays on autonomous vehicles. Wish I could read it all right now, but I have to get back to work. 

For the three-dimensional image to be realistic, usually the input signals from four to six cameras are required, and it is necessary to pay particular attention to the ‘image stitching’ to avoid loss of image information or generation of ghost images. Both 2-D and 3-D cameras require image sensors with very high dynamic range of more than 130 dB. This high dynamic range is absolutely necessary to deliver a clear image even with direct sunlight shining into the lens. The best available image sensors on the market have a dynamic range of 145 dB with a 24-bit deep interface to the ISP (image signal processor). This dynamic range is well above what common lens systems can offer.

So with between 4-6 cameras operating, I'm assuming there's enough overlap that if things badly disagree the car would quietly pull to a stop. For comparison on dynamic range, its a lot closer on this system than your average camera. Human night vision has about 160dB of dynamic range.

sensor deep dive

[TheDeamon]

Here is a video of the same area, the camera is set to a different gain.  The visibility is clearly much better, based on this video - I'd think the accident was fully avoidable by the driver and/or properly working equipment/sensors.

https://www.youtube.com/watch?v=CRW0q8i3u6E

Had to agree, the two videos were hard to reconcile, but the area entered around 34 seconds does appear to be right where the accident happened. I even went back and reviewed the police dashcam footage.

Basically, the Uber Dashcam was crap for night-drives. It's much more apparent in the second video, as it has a better gain setting, but there were TWO streetlights between crossing under that bridge and where the accident happened. In fact, from the looks of things, the Uber Car was no more than about 20 feet past the streetlight on the right when it hit her. And the light was on, as the Dash Cam shows as much, if you know where/when to look.

(It didn't help that the second video was pointed down at a sharper angle than the uber cam was, but that just meant more going back and forth in the recording to "place things" in the background)

"Her share" of the blame just dropped considerably. The driver should have seen her in more than sufficient time to stop or otherwise divert the car.

Even with the lousy dash camera, I still doubt she had side reflectors on the bike, she was wearing dark clothing, and she WAS jaywalking at night all the same, so she still has some blame to share in her death.

[TheDeamon]

For the three-dimensional image to be realistic, usually the input signals from four to six cameras are required, and it is necessary to pay particular attention to the ‘image stitching’ to avoid loss of image information or generation of ghost images. Both 2-D and 3-D cameras require image sensors with very high dynamic range of more than 130 dB. This high dynamic range is absolutely necessary to deliver a clear image even with direct sunlight shining into the lens. The best available image sensors on the market have a dynamic range of 145 dB with a 24-bit deep interface to the ISP (image signal processor). This dynamic range is well above what common lens systems can offer.

So with between 4-6 cameras operating, I'm assuming there's enough overlap that if things badly disagree the car would quietly pull to a stop. For comparison on dynamic range, its a lot closer on this system than your average camera. Human night vision has about 160dB of dynamic range.

sensor deep dive

And I had a typo in yesterdays post, that is NOT the telemetry that the car itself was using. The footage the police released was from the dashcam, which probably has zero input into the car's guidance systems.

I'd be interested in what their guidance cameras saw that night. Undoubtedly their techs(and the NTSB) are going over that with a very fine toothed comb now.

[LetterRip]

I find it hard to reconcile those two videos.  If this is accurate, I'd be willing to believe that an alert human driver would not have made this mistake.  I stipulate that human drivers make other mistakes that a machine would avoid.

The videos have different gain settings.  The Uber dashcam isn't what the Uber vehicle is using for driving/object recognition and apparently is tuned so it won't be 'blown out' by oncoming lights, which means it loses detail in shadows.  Unfortunately the new video isn't necessarily representative either (if you set the gain too high, the camera can pick up detail in shadows that might not be visible to humans).

I'm also concerned about diagnositics. While there are many conditions that can impact a human driver, tiredness, sickness, drunkness, the vast majority are detectable by a responsible person.  Tech issues are very difficult to deduce, particularly for a machine.  If a camera is inefficient, or has a arc error, how robust is the self diagnostic?

Depends on the design - well designed monitoring for sensors should be quite robust.  A failed sensor should be something that could be immediately detected by a reasonably designed algorithm.

[Fenring]

Even with the lousy dash camera, I still doubt she had side reflectors on the bike, she was wearing dark clothing, and she WAS jaywalking at night all the same, so she still has some blame to share in her death.

Maybe a lawyer could offer a more informed opinion about the level of legal culpability, but in my view if you do something illegal and dangerous you've just taken your life into your hands, and you'd better take care. I jaywalk too, and it's on me when I do that to watch my butt. That doesn't mean if you see a jaywalker you have carte blanche to steamroll them; you should still try to protect those unable to protect themselves for whatever reason. However in terms of 'fault' in a vaguely colloquial sense, I have trouble accepting that someone who jaywalks at night without looking or jumping out of the way when suddenly a car is there is anything but completely at fault in terms of 'this is on them'. It's true that in the arena of AI drivers we'd like the AI to be much better than human drivers and so the standard shouldn't be 'dullard driver'. But at the same time even if there had been a human dullard driver who just didn't pay proper attention or was tired or whatever else, I don't think the moral (not sure about legal) onus can be on them for failing to avoid people who throw their lives away. I've sort of diverged from the topic of AI drivers as I say this, because I fully believe that an AI driver should be able to far exceed the capabilities of a human driver and that 'bad night cam' isn't a good reason for an AI failing to perform.

[TheDeamon]

Depends on the design - well designed monitoring for sensors should be quite robust.  A failed sensor should be something that could be immediately detected by a reasonably designed algorithm.

Further, I'd expect the sensors to be "layered" at differing levels of gain for night driving in case an oncoming vehicle(or your own vehicle headlights) "white out" entire camera feeds or significant portions thereof. Or that they'd otherwise be employing additional methods that should be mostly immune to such things, such as FLIR(Foward Looking Infra-Red) or RADAR based options.