Studies versus Hype


If you read any magazine at your local grocery store or listen to any mainstream news channel, you’ll learn of a new study that has concluded something powerful, helpful, or frightening about your health. The article will sum up the findings in a sound bite, and you’ll feel reassured / empowered / worried to know that egg yolks are good (again) / harmless and / or the ultimate cause of your future but premature heart disease.

But what do you really know about that study? Do you know where it was published so that you can read it yourself? If you learned the research was funded by the egg industry, would you still believe it? What if the research were performed in a barn, based on the experience of 10 family members, rather than by professionals and based on 1,000 random participants nationwide.

Navigating the maze of “evidence” that exists about disinfection science is equally challenging. If we don’t know the right questions to ask about a “study” on new technology, we can’t be certain the claims of the technology are legitimate. Fortunately for us, a vetting system already exists within the academic world. Think of it as a staircase that must be climbed in order to take data and establish it as accepted scientific fact.

Step 1: Lab validation (sometimes called lab studies)

Any disinfection technology should be validated in an independent, 3rd party, CLIA-certified microbiology laboratory. This involves using the technology in the manner intended by the manufacturer to confirm that it will reduce or eliminate bacteria, bacterial spores, viruses, or any other targeted pathogens. Results are typically described as a % reduction in the pathogen or as a “log reduction.”

A good lab report describes how the results were achieved. For example, if the disinfectant is a chemical, the lab should describe the concentration of chemical they used and how long the bacteria were exposed to it. If the disinfectant is UV light, the lab should describe the distance between the light and the sample. The more test samples, the better – just testing 3 times isn’t good enough. Results have to be consistent across many, preferably >100 tests. At our independent lab, we’ve validated Xenex’s efficacy on over 2000 samples of the most common hospital pathogens, including C. diff and MRSA.

Step 2: Hospital environment testing

Testing in a lab is easy, clean, and controlled. Testing in a hospital explores the challenges of real-world situations. A good hospital environmental test should involve an actual room of a patient who had a known infection, so we can evaluate the disinfection technology on true contamination. Again, clear descriptions of methods, a number of samples, and results that are analyzed statistically are critical. A test for statistical significance, or a p-value, tells us that we can assume that the results can be attributed to the disinfection technology, and not due to chance. Results not analyzed for statistical significance are not scientifically reliable.

Step 3: Peer reviewed publication of environmental testing

The science world has a long history of requiring multiple, highly qualified, impartial scientists (peers) to review new findings before they are considered truly valid. Currently, the process is handled by journals that have teams of specialists to examine submitted studies and complete the “peer review” process. This process isn’t easy. Peer reviews can take over a year at times, and demand strict adherence to guidelines like no expression of opinions, only facts no use of adjectives and adverbs like “excellent” or “quickly” – only exact measurements the studies must be organized into strict formats any previous research utilized or consulted must be cited interpretation of test results must be thorough and repeatable statistical significance p-value of no more than 0.05, and preferably far less.

Surviving peer review and getting published is the process that takes your theory, testing, and results and establishes them as scientific fact. For example, our VRE study at MD Anderson established that Xenex was able to reduce hospital room contamination 20 times better than standard cleaning could, and that it could completely eliminate VRE from rooms of patients who had been in isolation for VRE. It was published in the Journal of Infection Control and Hospital Epidemiology.

Step 4: The Holy Grail of Disinfection Publication: Peer reviewed publication of HAI rate reductions

The top of the ladder of medical evidence is achieved when a drug, treatment, or disinfection technology can actually achieve a measurable, statistically significant (p-value <.05), useful impact for patients. We call these results HAI rate reductions and when we achieve them, we have to examine the proof through the same lens as in step 3, but even more:

HAI rates – not just numbers of infections but the frequency of infections per the total number of patients in a time period, and compare that to rates of earlier time periods.

Why is it crucial to see rates? Because this year a hospital might see fewer patients than last year, and a year with fewer patients will usually have fewer infections!

Other interventions – were there any other programs or tools being used that may actually be responsible for the improved patient outcomes, like hand-washing initiatives?

These results often take a long time to assess, as with Westchester Medical Center’s 22-month study recently published in the American Journal of Infection Prevention. While many disinfection technologies have achieved steps 1 and 2, and a few have even achieved publication, Xenex remains the only UV disinfection system with several published patient outcome studies in multiple peer reviewed journals.

Rachael Sparks is the Technical Director at Xenex Disinfection Services and was previously a transplant specialist working with hospitals throughout Texas.

Credit xenexadmin