Michigan's key COVID-19 statistic was wrong; blame Idaho

Craig Mauger
The Detroit News

Lansing — Amid fears of a second wave of COVID-19 hitting Michigan, the state's Chief Medical Executive Joneigh Khaldun stepped to the microphone during a July 28 press conference and noted a discouraging trend.

The percentage of tests yielding positive results in Michigan — a key statistic in tracking the virus — was rising and had reached 3.7% during that past week, Khaldun said. The rate, which spiked in the spring amid testing shortages, had been down to 2.2% in early June, according to state data.

A day after the press conference, Gov. Gretchen Whitmer cited rising cases statewide as she reimposed limits on bars and indoor gatherings in northern Lower Michigan and the Upper Peninsula.

While the 3.7% figure was accurate at the time of Khaldun's remarks, revisions made to data later now show the positivity rate was lower in July and hit 3.4% during the week Khaldun mentioned, much closer to the 3% benchmark that experts want to achieve to show the spread is limited.

One of the reasons for the adjustments was the state received testing results tied to a laboratory in Michigan without addresses. The lab was working with a health system in Idaho. Michigan later discovered results that flowed into the state's tracking system included people from Idaho whose tallies had to be removed.

A second reason was that officials discovered a lab's results weren't getting to the state. So officials had to go back in and add about 40,000 test results by the date a specimen was collected.

Chief Medical Executive and MDHHS Chief Deputy for Health Dr. Joneigh Khaldun

The differences and adjustments point to problems officials in Michigan and other states are experiencing in tracking and consistently reporting positivity rates, a metric they say is key in deciding when and how to reopen the economy and schools. Shifts in the data can push rates up or down by tenths of a percentage point, potentially playing into moves to limit public gatherings or lift restrictions.

It's a positive sign that Michigan is going back and updating data to ensure the numbers are correct, said Jennifer Nuzzo, an epidemiologist and professor at the Johns Hopkins Bloomberg School of Public Health. However, the lack of national standards for tracking and reporting testing is "problematic," she said.

"That makes it hard to compare positivity between states," said Nuzzo, noting states need to know how they compare with their neighbors.

While positivity rates are important and in the public spotlight, those numbers are "more stochastic" — meaning they involve random factors — than many people realize, said Sharon Kardia at the University of Michigan's School of Public Health.

"Those kinds of things eventually get resolved, but it takes time," said Kardia, who's working on the MI Start Map that officials are using to help track COVID-19 data.

While they acknowledged revisions and problems with numbers in other states, epidemiologists say they use a variety of metrics to track what's happening with the virus. 

"I am very interested in trend," said Sarah Lyon-Callo, Michigan's state epidemiologist.  "Is this number going up? Is it going down?"

Problems among other states

At some point in late July or early August, Michigan officials made significant revisions to statewide COVID-19 testing data, according to tracking by The Detroit News.

The changes were made because of problems with numbers reported by laboratories, including one lab not submitting its results to the state. In addition, a lab accidentally gave Michigan officials test results from a Pacific Northwest state.

The revisions usually shifted positive rates in July down two-tenths to three-tenths of a percentage point each week, something state officials have now acknowledged.

Michigan isn't the only place where positivity rate tracking has faced discrepancies. 

In Illinois, the state and its largest city were using different formulas for calculating the rate, according to WTTW-TV. The state compared positive tests to total tests — as Michigan has been doing — while Chicago compared positive tests to total individuals tested, which brought a higher rate of positives than the state method. The city eventually changed its process to match the state methodology.

In Oregon, the state Health Authority recently adopted a consistent process for calculating the rate after using different formulas for different public reports, according to Oregon Live on Tuesday.

In Washington, the state has stopped reporting daily tallies of negative test results because of duplicative negative tests, according to the Seattle Times, which found the lack of data provided "an incomplete picture of the virus’s spread in this state."

"Indeed, a number of other states have experienced challenges in tracking the numbers of overall tests conducted, as well as numbers of positives," said Stephen Hawes, professor and chairman of the Department of Epidemiology at the University of Washington.

The positivity rate is typically based on the total number of tests and the total positive tests in the last seven days, Hawes said.

'Focused on the directionality'

Over a three-week period from July 5-25, adjustments to Michigan's testing data usually decreased the weekly percentages of tests that were positive by about three-tenths of a percentage point.

The changes showed the positivity rate over that time was still increasing — meaning the trend was upward — but not as significantly as first indicated.

During the three-week period that later saw significant changes, Whitmer voiced concerns about the upward trajectory of cases and reopening Michigan's schools in the fall. On July 13, which falls near the middle of the three-week period, the governor issued an executive order that required businesses to ensure customers were wearing masks and imposed penalties for those who didn't comply.

In late July or early August, Michigan officials made significant revisions to statewide COVID-19 testing data, according to tracking by The Detroit News.

During a July 28 press conference, Michigan's top medical executive emphasized that 3.7% of tests brought positive results the week before.

"This is something that we definitely want to continue to monitor," Khaldun said of the positivity rate on July 28.

“When there’s increased testing and the percent of tests that are positive goes up, it indicates that there may be community spread going on," she said.

The original state data showed Michigan's weekly positivity rate going from 2.8% over June 28-July 4 to 3.4% over July 5-11 to 3.5% July 12-18 to 3.7% July 19-25. The revised data showed the rate going from 2.8% over June 28-July 4 to 3.2% over July 5-11 to 3.2% over July 12-18 to 3.4% over July 19-25.

Despite the adjustments made to July's testing data, the numbers still show a "slow gradual rise" in the positivity rate, said Lyon-Callo, the state's epidemiologist.

"We're also focused on the directionality," she said last week. "We're still seeing the gradual increase over the month of July."

Michigan's shifting COVID-19 picture

Last week, Michigan's weekly positivity rate declined to 3.2% from 3.4% the week before, according to current state data.

The decline came as Khaldun said during an Aug. 5 press conference that Michigan was seeing a "plateau" in new cases of the virus. Lyon-Callo said the numbers show Michigan is experiencing the "possibility" of a plateau after new cases began to rise in June. 

Most organizations that track what's happening with the virus in states are putting Michigan in lower-risk categories this month after noting potential spikes in July. But they're also citing different numbers for positivity rates.

COVID Act Now, the nonprofit organization that's been tracking states' performance combating the virus, says Michigan is among states seeing "slow disease growth," the second lowest-risk category.

Tracking from the organization COVID Act Now shows Michigan to have a positive test rate of 2.5% on Wednesday, Aug. 12, 2020.

The organization lists Michigan's seven-day positivity rate at 2.5%. It's unclear how COVID Act Now is determining that percentage.

The MI Start Map, which is produced by the University of Michigan and has been touted by Whitmer's administration, currently lists most of the state in the "medium-high risk" category. It puts the percentage of positive tests statewide at 3.2%, as The News analysis found.

The experts behind MI Start Map update the numbers based on changes to the data, said Kardia of UM's School of Public Health.

The numbers can be affected by random variables, said Kardia, noting a person's address might not be listed with his or her testing swab or an address is listed incorrectly.

The updates are the mark of a good health system, said Nuzzo of Johns Hopkins. Some states aren't consistently providing data on negative tests, she said.

The Johns Hopkins expert also cautioned states should use the positivity rates in conjunction with other metrics to analyze what's happening within their borders and the focus shouldn't be on the number itself but the trajectory.

"The number is not as important as the trend," Nuzzo said.