One of the most celebrated instances of data driving decisions at the front lines is CompStat: the effort led by then-New York City Police Chief William Bratton to put police resources where there was the most crime. He did that by instituting extensive analysis of crime statistics (CompStat is short for Computer Statistics) and holding weekly meetings at which his precinct captains and other field commanders had to review the latest data and what they were going to do because of it.
A recent report called some crime cutting attributed to CompStat in question by suggesting that those field commanders have gamed the system for years: they would, according to the report, downgrade crimes so that their stats would look better.
In today’s New York Times, Bratton launched a masterful counter-attack that serves as a model for anyone who has to defend their data.
First, he reminds us of CompStat’s original goals as well as its ancillary benefits and it’s successful use by other police forces:
“CompStat has been adopted with great success around the world. It is not merely an accountability device; it is a communications tool that keeps commanders aware of innovations in other precincts and a management development tool that enables leadership to identify their best mid-level officers.”
Then he offers a fresh take on the data that quickly – and objectively – refutes the claim in the critical report:
“First, categories of crime that are nearly impossible to downgrade, notably homicide and auto theft, have declined much more than the categories that might be more readily manipulated.”
Third, he points out how hard it would be for the alleged manipulation to have been sustained over a long period of time:
“Each successive commander would have to do the same for the statistics to stand — or manage the extraordinary feat of not only reducing crime further, but also accomplishing the reductions that his predecessor falsely claimed.”
Finally, he challenges the data of the critics:
“Moreover, not one respondent admitted to downgrading crime; if it were a widespread practice, one would expect that at least a handful of the 491 people surveyed would anonymously admit that they had engaged in it.”
Bratton’s refutation of his critics in thorough, well-constructed, and reliant on logical argument rather than emotional pleas or a spirited defense of himself. One would expect no less from the man who developed and evangelized the CompStat system. It is useful to deconstruct his op-ed and take from it lessons you may be able to use when next you have to defend you data. The full piece is worth a read.
