External review finds data inconsistencies in APD reporting on use of force

During a special called meeting of the City Council’s Public Safety Committee, Dr. Robin Engel, a Senior Research Scientist at The Ohio State University, presented findings from a recent re-examination of the Austin Police Department’s use of force. Engel said she found the department’s data analyses to be “rudimentary, inconsistent, and inaccurate,” among other concerns.

Engel kicked off the presentation with somber news. “Unfortunately, I did find a number of inconsistencies across the Austin Police Department (APD) in all aspects of use of force, including reporting, accountability and oversight mechanisms, and training,” she said.

She added that inconsistencies in these areas were common in police departments across the country, but APD was unique in its data integrity issues and its tendency towards reporting some numbers without context.

“The counting of the use of force is how you establish external credibility and legitimacy with your public,” she said.

Police Chief Lisa Davis implemented the review. She explained that, as a new chief, she wanted to bring people in to evaluate what was going on. She noted that she had also worked with Dr. Engel in her prior role with the Cincinnati Police Department.

Engel described her approach as “holistic”. She evaluated APD policies, training curriculum, internal reports, previous external reviews like the Kroll report, body camera footage and City Council resolutions dating as far back as 2019. She also conducted informal interviews with APD staff and leaders, met with Austin’s civilian-led Office of Police Oversight and rode along on some calls to Sixth Street, where there have been more documented incidents of use of force.

“Data doesn’t tell you everything you need to know,” Engel said. “You have to look at the dynamics on the ground.”

One of Engel’s primary concerns was APD’s system of counting incidents. In 2024, force was utilized towards 2,919 individuals. However, APD documented more than 6,000 incidents for the year. Each action implemented by an officer during the same interaction could be counted as a separate incident. Engel recommended that encounters be grouped by individuals or subjects instead. She also recommended APD change their terminology for these incidents from “response to resistance” to “use of force”.

APD documents these encounters using four categories. Level 1, which denotes the most severe use of force, comprises less than 1 percent of interactions. Most incidents (more than 56 percent) fall into Level 3. Engel believes APD is over-reporting at the Level 4 category (which typically comprises close to 40 percent of incidents). 

APD Assistant Chief Mike Chancellor noted that actions causing injuries to civilians are documented at Levels 1, 2 or 3. As the department defines it, a Level 4 incident could be feeling tension in someone’s arms as they are moved behind their back to be placed in handcuffs, for example. Similarly, if someone is on the ground, holding their feet would warrant a Level 4 report.

Based on Engel’s findings, the department has begun reorganizing the category and is considering eliminating and reorganizing some other Level 4 criteria.

According to the data, somewhere between 9 and 11 percent of recent APD arrests involved use of force, which is higher than national averages. Engel believes this could be in part due to the number of Level 4 reports. The four levels are also applied to situations that do not lead to arrest, such as diversion to a mental health facility.

Surprisingly, Engel said she found “very little ethnic and racial disparity” in use of force at all levels in her review so far.

Council Member Krista Laine said she wanted to see more information about how other vulnerable populations were treated during interactions with APD: “I hope there’s a way, as we move through the data, to see how mental health response, homeless response, that sort of thing, is impacting how much use of force is happening.”

Engel also noted that APD had a “significant data lag” in its reporting timeframe. She described past APD data analyses as “rudimentary, inconsistent, and inaccurate,” and attributed this to poor training.

Engel wants APD to implement a more comprehensive data audit. She is recommending that the department temporarily suspend public reporting of use of force data and re-issue historical reports from 2019-2022. A past-due report on APD’s use of force in 2023 will now be released at the same time as the report for 2024.

Per Engel’s recommendation, APD has implemented an internal task force that focuses on use of force. Engel has met with the group several times, and they will report their findings to Council. The department has also reorganized their Patrol Commanders so that the sectors all report to one Assistant Chief. This will help prevent variations in data or double reporting. For instance, the same incident may have previously been reported twice if officers in multiple sectors were involved.

Several committee members raised questions about APD’s training on use of force. The cadet academy currently implements a 16 hour training block called ICAT (Integrated Communications, Assessments, and Tactics) that focuses on de-escalation. Engel says she believes ICAT is effective.

“Prior to that, officers really thought ‘you’re going to teach me to hesitate, and you’re going to get me killed out there,’” she said.

She added that one of her concerns was addressing “training decay”  and determining how regularly officers need to conduct de-escalation trainings and how best to review the curriculum in-between trainings.

Chief Davis said she hoped Engel would continue working with the department for at least a year. “We want to get this right,” she said.

Engel said her main goal is for APD to develop a strategic plan to create a “culture of de-escalation”. She emphasized that the data was only valuable if it contributed to holistic cultural change.

“Many of the recommendations that have been made in the past…are not data driven,” she said. “When citizens come to the street and they demand reform, the reform that we put in place is often a knee jerk reaction.”

Engel described some trainings or policies implemented by police departments across the U.S. as “well-intentioned, but not well-researched,” noting that some of these changes were discovered years later not to have had the desired impact on policing culture.

“The guiding principle for all of this work…is: ‘What is the impact on our community?’” she said.

Photo made available through a Creative Commons license.

The Austin Monitor’s work is made possible by donations from the community. Though our reporting covers donors from time to time, we are careful to keep business and editorial efforts separate while maintaining transparency. A complete list of donors is available here, and our code of ethics is explained here.