Aug 09, 2011

By Crawford Coates

Earlier this month, ShotSpotter released a study about its use and efficacy. The authors of this included Law Officer columnists Nick Selby and Dave Henderson. Read the complete study here. I had a chance to speak with Nick about this undertaking and some of its counterintuitive findings.

In layman’s terms, what is ShotSpotter?

ShotSpotter is a network of acoustical sensors that detect gun shots. It pinpoints things like location, time and number of shots, as well as information like were the shots moving—like in a drive-by. It turns out that this is not as easy as you'd think – imagine a city landscape and you'll see that there are a bunch of different and different kinds of reflective and refractive surfaces on buildings and things in the cityscape which will treat the sound differently. But over the past 12 years or so, ShotSpotter's gotten it down pretty well.

Nick, how did you and Dave get involved?

The company has a new CEO, Ralph Clark, and he and I had worked together in our previous lives – he was the CEO of an information information security company which was a client of mine at the analyst firm I used to run, so we knew each other and had stayed in loose touch over the years. When he took over as CEO he had his own plans for the company and the product, but he wanted to understand independently how their existing customers perceived and used the product. He knew what the company said it did, but what was it doing for the practitioner? How was it really being used in the field? How well did it work. He wanted to get it independently, and he remembered that this was the kind of thing we used to do, so we started talking about creating a study that would answer some of these kinds of questions.

If ShotSpotter’s paying you, how can we trust your conclusions in this report?

That was our immediate challenge -  how can we stay objective? Our co-author is a former psychology professor from the University of Pennsylvania who specializes in statistical analysis and data gathering, and we worked on a regime to ensure that we wouldn't bias the results with anything we did. We came up with a survey instrument that was consistent and clear, and then trained ourselves as interviewers, to deliver the questions the same way each time. For the study we interviewed five groups – detectives, analysts, patrol, dispatch and command staff – at seven agencies across the country, so each group had its own survey instrument, with questions specific to how that group of people used and experienced the product as part of their daily tasks. The other thing we do to stay honest and ensure transparency is that every conclusion we made we referenced and gave the complete interview transcripts  and, for peer reviewers, we even make available the audio files. So if someone has questions about our conclusions, they can easily go back and read what we asked and exactly what the respondent said, in context.  Some of our conclusions were non-obvious, but we can tell you how we reached them, and you can double-check our findings on your own.

So what do you think of the product?

ShotSpotter works, and it is highly accurate. It places empirically an event at a specific place and time. Patrol officers trust it. It empowers them to investigate specifically and quickly where the shot has happened.

Without ShotSpotter, when a guns shoots, maybe someone hears it. They might wonder if they heard a gunshot or whether it was a bottle rocket, and they take time to report it. The call is totally subjective, based on the listener's experience of the gunshot, and is maybe inaccurate. For example, someone might call 9-1-1 and say, “I think I heard a gunshot around the corner,” or “down the block.”

So let's look at the difference between ShotSpotter and 9-1-1. With a 9-1-1 report, the officer goes to the person who reported it, and if there’s no proof – a body, or a shell casing, or a smell of gunpowder, the officer basically has to say,  OK, but I don't see anything here – but he can’t say definitively that it was a false alarm. They just don’t know – so with 9-1-1, you have either a confirmation or an unconfirmed report with an unknown outcome. 

With ShotSpotter, the officer can go right to where the sound happened and determine if it was a shot or not. He can state definitively that, yes, this was a gunshot, or no, this was a false alarm. Meaning that an agency can determine a specific outcome to a report of gunfire. Now, ShotSpotter will trigger on a range of loud, concussive noises, like fireworks or backfires or steel plates on a highway, but dispatchers tell us that about 70% of the time, when ShotSpotter says it's a gunshot, it’s a gunshot.

Patrol trusts ShotSpotter more than 9-1-1. What's more, patrol isn’t bothered by the 30% false alarms, because even when it's a false alarm, it's something that can be investigated by them. So they trust the product because they’re confident it can identify exactly where the sound happened. One patrol officer said to us that, “If you go to the place on the map where ShotSpotter said the gunshot happened and you look down, you'll see shell casings.”

The flip side is that dispatchers don’t like it. It makes work for them, because the system is good at not missing shots – it's much worse to have a gunshot occur that the system doesn't catch than it is to have the system catch something that sounds like, but isn't a gunshot. But dispatchers find the product can be “noisy”, and when they're annoyed, dispatchers don't want to take the time to properly reclassify the event— for example, when it was a firecracker or steel plate on the highway, telling ShotSpotter that this was not a gunshot, but something else. If they don’t reclassify, then the system continues to get false alarms because it can't learn the difference.

This is actually an interesting technical point: when agencies properly use the system and reclassify false alerts, they can reduce the number of false positives without increasing the number of false negatives. In other words, you can catch more actual gunshots and reduce false calls.

What are some of the counterintuitive findings in this report?

ShotSpotter has changed the way detectives handle homicide investigations. In the past, they had to piece together a story at the scene by interviewing people, finding a common thread, corroborating witness stories and establishing a narrative. With ShotSpotter they arrive on the scene knowing empirically where the shots happened, when, and how many there were. So the very first questions they ask of potential witnesses are disqualifying and corroborative, as opposed to inquisitive and investigative.

Agencies can figure out a few things with ShotSpotter like know empirically the number of gun shots. They do this in Nassau County. They know exactly how many gunshots took place in a year. They can check this against 9-1-1 records to see if shots are being reported, and find the Delta between actual and reported gunfire incidents. This highlights issues of community and neighbourhood policing, because you can tell when people aren’t using 9-1-1 and do community outreach. In every community, the reaction to ShotSpotter’s use was very positive, and police agencies are able to leverage this positive sentiment to establish better overall community relations.

http://www.lawofficer.com/article/investigation/shotspotters-efficacy-study
Copyright © Law Officer

END OF ARTICLE

ShotSpotter contact
+1 650 960 9207
press@shotspotter.com