Skip to content

In a week without major pandemic news, it seems that the medical and lay media have found time to speculate about how this pandemic might end. Of course this is premature, we hardly know what to expect next month, let alone the years to come. It does give me an excuse to make some observations on current goings-on.

Medical Pundits Aren't Reading the Literature Carefully

I was flabbergasted when I learned of major news media outlets like NPR and Wall Street Journal giving voice to poorly-informed opinions related to COVID-19 vaccination of young children. In one, a mother who also is an adult infectious diseases physician expounded on her plan to deliberately spread out the interval between vaccine doses for her child because she had concluded a longer interval between doses resulted in longer-lasting immunity. In explaining her rationale, she seemed to have discounted the fact that she was guessing on extrapolation of studies in adults in entirely different settings and vaccines, not to mention the fact that only a 3-week interval has been studied for the Pfizer vaccine in younger children. We simply don't know what spreading out intervals will do, although this is certainly something to be studied particularly if we need to incorporate regular COVID-19 vaccination within the regular pediatric well-child visit schema. I absolutely support this mother's right to make decisions about her child's healthcare, but is it necessary to promote this poorly substantiated thought to the general public?

Worse was another piece where the authors cherry-picked superficial data from some pretty dense discussions at FDA and ACIP to reach similarly unfounded views regarding risks and benefits of pediatric COVID-19 vaccines. It appeared they hadn't read the source documents but rather looked at some slide presentations from Pfizer to point out presumed holes in CDC recommendations. A deeper dive to look at detailed briefing documents from FDA as well as discussion of 6 different vaccine risk/benefit scenarios, all concluding benefits exceed risks of vaccination in the 5-11 year-old age group, seems to refute their editorial points. Anyone could correctly accuse me of cherry-picking my discussions for this blog, but this is always informed by careful analysis of the original source documents.

Bottom line? Reading an opinion written by pundits in a respected media source doesn't guarantee you are reading an evidence-based conclusion.

I Will Be Safe When Everyone Else is Safe

Earlier in the pandemic we talked about herd immunity and ending SARS-CoV-2 transmission. That happy ending doesn't seem likely now; talk to a white-tailed deer in Iowa about it. Just don't get too close.

Many of us in the US are guilty of not giving enough voice to the global situation. We are all very pleased with availability of vaccines including boosters in our country, even though our immunization rates pale beside what less-resourced countries have accomplished. Let me point you to 2 sources to give you a view of the "pan" in this pandemic.

First is a great data source from Our World in Data. This site, updated daily, gives both quick and detailed looks at progress (or lack thereof) for COVID-19 vaccination across the globe. Note from the first map the horrific gaps in coverage in some countries, as well as the relatively poor showing in the resource-rich US. I probably don't need to remind you of how isolated outbreaks can become global problems very quickly. Remember Ebola?

Second is an article appearing recently in the BMJ that looked at life expectancy and premature mortality from the pandemic in 37 upper-middle and high income countries utilizing corrections for population age spectrum and other factors not often considered in this type of report. The US was second-worst to Russia in changes in life expectancy for populations, far worse than countries at the other end of the analysis such as Iceland, Denmark, Norway, South Korea, Taiwan, and New Zealand. Although all of those countries have differing circumstances, we can learn much from study of their mitigation strategies.

Yesterday, October 30, was a day off for me. To celebrate, I attended the 7-hour online meeting of the Advisory Committee on Immunization Practices (ACIP) of the CDC. The topic was COVID-19 vaccines. This was the last regular meeting of the ACIP until February, although they will convene on an emergency basis before that time if/when a COVID-19 vaccine trial has enough data to merit discussion. Everyone fully expects that to happen within the next few to several weeks.

The day included presentations from 14 speakers representing FDA, CDC, and vaccine manufacturers and covered regulatory, ethical, scientific, and other topics. I found the modeling discussion most enlightening; it was an attempt to display various outcomes for infection rates and deaths based on how effective a potential vaccine might be, what groups are prioritized for vaccine administration in the early stages of vaccine deployment, and what the US epidemic curve is doing at the time immunization is begun. It was definitely not intended to be a predictive model but rather a general methodology to use whenever a vaccine is ready to be released for use. At that time, more specific data regarding vaccine efficacy and current epidemiology can be plugged into the model to help guide early deployment. More about the modeling later. Here are my take-home points for pediatric healthcare providers distilled from those 7 hours.

Worldwide we now have over 200 COVID-19 vaccines in various stages of study. Honing down to the US, we have 5 vaccines in either Phase I or Phase II testing in humans and 4 in Phase III. Let's focus on the Phase III products since 1 or more of those likely will have results to report in the next few weeks to months. Two of them, AZD1222 (AstraZeneca/Oxford, the UK vaccine you've probably heard a lot about) and Ad26CoV2S1 (Janssen) were paused for safety monitoring but now have resumed recruiting volunteers, though the AstraZeneca product is still on hold in the US. mRNA-BMT162 (Pfizer/BioNTech) is recruiting still but is far along, having enrolled around 42,000 subjects of which about 35,000 have received the second dose of the 2-dose series. Finally, mRNA-1273 from Moderna has completed enrollment of around 30,000 people of whom ~25,000 have received the second and final dose of that series. Most likely we will hear trial results on the Moderna product within a few weeks.

An FDA representative (full disclosure, happens to be a longtime colleague and friend of mine) provided an overview of how FDA rules will be applied in this situation. Again, you've probably seen a lot about this, with some back and forth on the application of product release under Emergency Use Authorization (EUA) which can only be applied in national emergency situations like we have now. Understand that in general EUA is a "lower bar" to clear than is full licensure, but the FDA has very clearly laid out their requirements in this situation. It's really a balance of ensuring safety but not delaying consideration for EUA for such a prolonged period of time that we find ourselves in a worse hole with cases and deaths.

Let's get to the modelling discussion. The group at CDC stratified the population into 5 age groups, 0-4 years of age, 5-17, 18-49, 50-64, and 65+. Note that this oldest category consists of 55 million people nationally (including yours truly!). They also stratified by low-risk or high-risk, the latter consisting of at least 1 of the following medical conditions: COPD, heart disease, diabetes, kidney disease, or obesity. Nationally 40% of adults, 100 million people total, fall into the high risk category. Another interesting tidbit is that ~40% of adults 18-64 years of age, or 80 million people, are classified as essential workers; 1/4 of those are healthcare workers. The modelers made various assumptions about vaccine efficacy in different age groups, etc, and the main focus of their presentation was in Phase 1 distribution of 200 million courses of vaccine. This phase has been divided into Phase 1A consisting of healthcare personnel (20 million courses) and Phase 1B for adults 65+, high-risk adults, and essential workers (180 million courses total). The modeling discussion was most interesting in trying to prioritize Phase 1B individuals - which of those 3 groups should go first, second, third. Various modeling assumptions and outcomes (i.e. what strategy prevents the most infections versus what prevents the most deaths) produced slightly different suggestions for vaccine prioritization.

What was most important, however, was that the timing of vaccine in relation to the epidemic surges going on was by far the biggest determinant of how effective a vaccine will be at any of these outcomes. This isn't surprising particularly, it's why we don't target annual flu vaccination to start in the middle of our annual epidemics. However, the modelling numbers were impressive and point to the main take-home message for all of us: it has never been more important than right now to use those SARS-CoV2 mitigation strategies. Failure to do so diminishes the benefit from a COVID-19 vaccine and makes it even more likely that we'll see more preventable deaths and more harm to our economy.

A few other interesting details from the session. Federal officials are working hard to reach out to everyone in our society to provide vaccine information, including providing information sheets in >20 languages. Also, vaccine recipients will be asked to use a smartphone app to provide real-time feedback for safety monitoring and illness after vaccination. CDC officials provided a brief review of the evidence to date regarding possibility of reinfection with SARS-CoV2. So far this is at most an uncommon event in the first 3 months following infection, but possibly could become more common if immunity wanes later after natural infection. Multiple individuals weighed in regarding vaccination during pregnancy. New data from CDC, so far unpublished, indicate that pregnant women are at higher risk for worse outcomes with COVID-19 (earlier published data were a bit more equivocal). We likely won't have a lot of data on vaccine safety in this population very soon, and it seems that pregnancy will be listed as a precaution but not a contraindication for vaccination.

Which brings us to our final group, children. We need to be very careful with the safety of any vaccine being administered to a healthy child, particularly for an infection that has a much lower complication rate than in adults. So far, we have no pediatric data at all about these vaccines. Certainly children will eventually be enrolled in vaccine trials, once we have sufficient longer term safety and efficacy information from the adult studies. We'll have to be a little patient here.

There is so much more I'd love to tell you about this session, but I've probably already used up some of that patience you need to save waiting for the pediatric vaccine trials. Soon more details from this meeting will be available at the ACIP website. Just know this: I am very reassured with the transparency surrounding vaccine development and distribution, and I am confident I'll see enough of the results from these trials that I'll be able to judge independently whether or not to recommend a vaccine for a specific group. Although you won't be providing COVID-19 vaccine for your pediatric patients anytime soon, you undoubtedly will hear a lot of vaccine questions from your patients and families. A primary care provider is probably the most important individual to help families with vaccine decisions, now more than ever. Whenever a vaccine becomes available for use in the US, of course I'll let you know what I think but know that ACIP/CDC will have toolkits available for you to consult and assess as well.

In the meantime, please ensure all your patients receive their seasonal flu vaccine and are practicing safe COVID-19 mitigation strategies.

2

It's always been hard to keep up with the medical literature, especially to figure out what original articles are of high enough quality to warrant a change in your clinical practice. It's not enough to just read the abstract, or to be reassured because the authors are from a reputable institution or the article is published in a reputable journal. I've been teaching Evidence Based Medicine (EBM) in various formats for over 20 years, including a full graduate school course for a while. I've learned a lot, both from reading but also from my students and colleagues, about how to sort through the jungle of words and diagrams in medical articles to pick up those rare pearls of good information.

EBM officially came into being in the early 1990s and, like most things, it has evolved. What hasn't changed much, however, are the forces that result in low quality evidence being published and advertised:

  • Pressure on researchers to "publish or perish." This not only involves job security and academic promotion but also a natural desire to make a name for oneself.
  • Pressure from academic institutions to ask their researchers to "hype" their studies in the hopes of increasing organizational rankings in national publications and also increase charitable donations.
  • Complicity from the lay press, anxious to describe in breathtaking fashion a new study, even if it has no direct relevance to clinical practice or improving lives of their viewers/readers.
  • Efforts from commercial organizations, such as pharmaceutical companies, test developers, and device manufacturers, to sell their products.
  • Predatory journals who will publish anything for a price. (One "gotcha" study showed how one of these journals published a report taken straight from the pages of a "Seinfeld" script - clearly totally bogus and obviously published in such a journal without any editorial review.)
  • Failure of the medical community as a whole to convey the inherent uncertainty in medical science - very few things are absolute "facts."

All of this just got worse in the pandemic era. Individual clinicians, researchers, and organizations seem bent on being the first to report the newest covid finding, and publishers and the lay press are anxious to help them. Unfortunately, things have moved too fast. Just recently, 3 major journals (New England Journal of Medicine, Lancet, and Annals of Internal Medicine) retracted publications due to, in my opinion, sloppy editing - plain rookie mistakes likely due to being in too much of a rush. (Actually as I'm writing this I heard about a potential new retraction with Proceedings of the National Academy of Sciences regarding mode of transmission of SARS-CoV-2). It is even harder now for those of us at the point of care to digest the onslaught of poor science looking for the truly helpful articles. However, there is still hope, and here are some quick guides to survival in the Pandemic Era of Medical Practice (PEMP, I just made up that acronym).

The image above is one I've used many times, most recently at a talk I gave at the AAP NCE meeting last fall. It is my version of the "evidence pyramid," a hierarchy of studies much misunderstood by the general medical public. Simply explained, results utilizing the study design types at the lower end of the pyramid are more likely to be shown to be wrong when subsequent studies, usually from a higher design type in the pyramid, are performed. Also, note that pure bench studies and animal studies aren't even part of the pyramid; those studies would not immediately impact clinical (human) medical practice. Also be aware that a poorly-designed randomized controlled trial (RCT) wouldn't be near the top of the pyramid; bad science can occur at all levels and trumps the pyramid ranking.

The vast majority of design types we are seeing related to COVID-19 are case series, i.e. just a report of what was tried and what happened, usually of a retrospective nature. It's not that these studies are bad, but compared to a randomized, placebo-controlled double blind trial of a new therapy, it just doesn't stand up. The gap between the lower and upper ends of the pyramid are magnified when we are dealing with a completely new disease like COVID-19.

(BTW, if you are wondering about GOBSAT, I wish I had invented the acronym but I didn't. It stands for Good Ol' Boys Sittin' Around a Table, another word for expert opinion. Again, if that is all we have to go on, I'm certainly interested in what experts think, but it's astonishing over the years how often GOBSAT opinions are reversed when better studies are performed.)

So, here's a quick and dirty approach of how I keep up with the flood of medical studies. First, I look at the abstract. If it sounds like something worth reading more, I then go immediately to the Methods section of the article. Yes, I know that section is the most painful of all, but that's where I figure out study design and whether the study may have critical flaws that would affect study results. Also, in spite of modern-day editing, even the best journals still allow conclusions to appear in the abstract that aren't supported by the study itself; usually they just represent the authors' conjectures but aren't labelled as such. If the Methods section doesn't pass muster, I don't read the rest of the article. If, however, the Methods look reasonably sound (remember, this is biology, we can't expect perfection in any study) I look through the results and discussion to see if this is something that would apply to my patients.

One more point that has just surfaced during PEMP. I'm starting to see increased alerts about manuscripts submitted to pre-publication web sites. Prior to the pandemic, these were sites where authors submitted data to be looked at by other scientists. They were not necessarily even submitted to a journal, just a way to increase transparency and actually a good thing. One key important fact is that the documents have not undergone any peer review at all. Unfortunately, now many authors are submitting results of case series and the like to these sites, and the lay press and even otherwise sound academicians are referring to these as "publications" when in fact they are nothing of the sort. As a reviewer for many medical journals and author of a few scientific articles, I can tell you that most articles submitted for publication undergo many, many significant changes before publication. I wouldn't advise clinicians to even look at these postings, they are useful only if someone is trying to design a research study on a similar topic. Some of the web sites include medrxiv.org and biorxiv.org. Again, nothing wrong with these sites other than how they are currently being misused by a few individuals.

So, I would advise you all not to be too discouraged by the confusion and flood of information. Listen to the lay press so you know what your patients and families are hearing, read the key articles, and be prepared to answer questions in your practice.