Emails appear to validate methodology of controversial ethics report

A series of emails between a City of Austin consultant and city legal staff appears to validate the methodology of a recently uncovered report on how city employees viewed the ethical standards of their colleagues and higher-ups when the survey was taken in 2010. The emails were exchanged in 2010 and 2011.

 

In addition, the emails also raise questions about recent statements from Austin Human Resources Director Mark Washington – and, now a memo from City Manager Marc Ott – that city management failed to release the report because of concerns over the methodology of the study. In Fact Daily obtained the emails via public information request.

 

The emails further suggest that city staff worked with the firm that produced the study – a widely respected organization called the Ethics Resource Center – to  redraft it. It is not immediately clear from the emails what concerns the city had, and why the group hired to do the study, at a cost of $40,000, agreed to rewrite it.

 

However, in a memo to Council released late Monday afternoon, Ott detailed some of the management issues with the study when it was first unveiled in 2010. “Primary amongst those concerns was the disproportionality of the ERC’s national sample size of 285 public employees compared to 4,427 City of Austin employee respondents,” Ott wrote. “This didn’t feel like an ‘apples to apples’ comparison.

 

“There were also concerns about the controls in distribution, as some departments were reported to have a participation rate of greater than 100 percent,” the memo continues. “The primary concern expressed by many members of my executive team was that the results may not be representative of the entire workforce due to these survey control issues.”

 

When asked about the seeming contradiction between the 2010 emails and current staff position, Ott reiterated that he was under no obligation to release the report to Council. “It wasn’t a report I was obliged to send to Council,” he said. “The report was done for my benefit and the benefit of the executive team. You know, I was still relatively new back then and trying to get a handle on culture of the organization – and it served (that) purpose. It wasn’t commissioned for the Council. I’m not sure where that comes from.”

 

Ott further noted that the study is three years old. “It’s hard for me to comment on a three-year old report,” he said. Later adding, “To tell you the truth, I’ve moved on.”

 

Even re-written – as it apparently was – the report offers a scathing snapshot of city employees’ perceptions of city culture, including the ethics of management. Members of the Council’s Audit and Finance Committee ran into the study as part of a closer look at a smaller issue earlier this spring. Assistant City Attorney John Steiner – who was part of the team that commissioned the study – ruled that committee members were not properly posted to delve into the matter and initially barred them from further discussion of it.

 

Council members expressed no small amount of frustration at that ruling. At the time, Council Member Laura Morrison pronounced herself “flummoxed.”

 

As In Fact Daily first reported in early May, the document concludes that “two-thirds (65 percent) of employees perceive that the city has a strong ethical culture, compared to about 80 percent in the US Averages. Of the four component cultures, employee perceptions of senior leadership are the least favorable. This is typical in organizations; however, the disparity in perceptions about the senior leadership and the supervisor and coworker cultures is greater than that seen in the US Averages.”

 

The study also reveals that only 58 percent of those surveyed believed that their managers rewarded ethical behavior. It also suggests that 41 percent of those surveyed “do not believe that the city values them.”

 

When Council members raised questions about why the study was never released to them, Washington cited management concerns about methodology. However, the emails appear to run contrary to his statement.

 

The emails feature an exchange between Steiner, law department assistant Alicia Olmstead, and city business consultant Scott Truelove.

 

There, Truelove provides a fairly detailed examination of city concerns that some employees may have taken the questionnaire that informed the Ethics Resource Center’s work more than once. Truelove cites “an impressive study” produced by the Online Research Quality Council that concludes the “worst case duplication rate” – the highest rate of respondents filling out more than one study – is 16 percent.

 

“We’d be looking at less than a handful of percentage points in even this worst case scenario,” Truelove writes. “That is, it would not substantively affect the ERC’s conclusions. Perhaps on one or two items it might slightly shift the interpretation, but for the most part, the interpretation of the results would not change. And this is an empirically driven worst-case scenario. What’s much more likely is…That the results accurately reflect the opinions of the 10,000 employees at the COA.”

 

Truelove’s 2011 email also contained a prescient aside. Apparently informed or having inferred management’s approach to the study’s results, he urged them not to back away from the study.

 

“From my vantage point, though, there is little to gain and much to lose by attempting to discredit the results,” he wrote.

 

Referring to the city manager’s office, he said, “If what CMO really wants is to promote a strong ethical culture (and to minimize bad press), quibbling about potential data or interpretive issues will likely give the impression that CMO is more interested in looking good than being or doing good (it’s hard to look good when questioning such a large sample). Employees and the public (to say nothing of the press) may doubt CMOs motives if they take this tack.”

 

Council Member Bill Spelman, an academic statistician, told In Fact Daily that Truelove’s emails confirm, for him, the validity of the study. He noted both the overall quality of work performed by the Online Research Quality Council – and thus the worst-case 16 percent figure – and the arguments from Truelove. 

 

Nonetheless, Ott’s memo details management’s current position. “There were so many questions about the validity of the data and the benchmarking process, that we simply couldn’t define this as ‘solid data,’ it reads. “Therefore, we made lemonade out of lemons and used the recommendations to confirm our path and adjust it where appropriate.”  

 

For his part, Spelman suggested that Council members should request both drafts of the ERC report and request updates on what Ott and his team have done to address the issue.