Things You Should Never Do With Data: Uber Makes 3 Classic Research Mistakes

Uber’s recent head-on collision with drunk driving data is a wonderful example of how not to use research. In January, Uber reported that cities using its ridesharing service had seen a reduction in drunk driving accidents, particularly among young people. But when ProPublica data reporter Ryann Grochowski Jones took a hard look at the numbers, she found the company’s claim to be lacking. See the Pro Publica post “Podcast: Uber’s Dodgy Drunk Driving Study.” Here is Uber’s 1/27/5 blog post on the research.

In today’s skeptical information-everywhere world, you probably won’t get away with drawing bad conclusions from incomplete data. There are simply too many data journalists out there waiting to pounce on flaws in your methodology.

The really sad thing is that this could have been a great story. Unfortunately, Uber made three classic mistakes that are all too frequent when publicists meet research:

1. They over-reached.
   To say that Uber prevented 1,800 crashes may be a great headline, but it’s a big red flag for anyone who knows the difference between correlation and causation.
2. They didn’t provide the methodology.
   Being open and honest about how you conduct research is critical if you want anyone to understand your procedure and trust you or your conclusions. One of the hallmarks of good research is that you share enough methodology so that your results are replicable. Openness and transparency does correlate with and cause trust to increase, but the inverse is also true. Obfuscation or withholding of information causes trust to decline.
3. They confused math with statistical validity.
   While their methodology eventually showed that their calculations were accurate, Uber’s research failed to show causality. The fact that drunk-driving accidents involving drivers under 30 declined just after Uber’s launch may be a fact, but there was nothing in the study that showed that drivers under 30 were in fact aware of or using Uber, (as opposed to calling a taxi). There may have been correlation, but no causality. ∞

 Thanks to Studio Blu for the image.

[/mepr-rule]