Primary care providers who have attended Children's National Hospital's Pediatric Health Network updates know that I have consistently advised against ordering SARS-CoV-2 antibody tests for individual patient use; the state of knowledge about accuracy and interpretation of such tests is not sufficient to give any advice to individual patients based on the result, it is purely a research and epidemiologic tool at this point. Now, we have a new meta-analysis that examines the current state of knowledge at a very detailed level.
The study is published in the Cochrane Database of Systematic Reviews, hands-down the highest quality source for systematic reviews. If you refer back to my recent post on the "evidence pyramid," you'll see that Systematic Reviews are at the top. That's not to say that any systematic review, including a Cochrane review, is the final word on the subject; in fact, sometimes the reviews are so strict that some useful publications are excluded because they don't meet the pre-determined trial design level of quality. Not infrequently, Cochrane reviews come to no conclusion due to lack of high-quality studies, but we know as clinicians we still must make management decisions in many settings where we lack high-quality studies to guide us.
This particular review is more than 300 pages, but rest assured I'm just going to give you the highlights here. Here are some key points to consider when digesting the findings:
- The review only covers publications through April 27, 2020. However, the authors do plan frequent updates, an advantage of an online journal like the Cochrane Database.
- They include "preprints" in their studies for analysis. These are the non-peer reviewed submissions to online sites like medrxiv that I cautioned against in my prior posting. However, in this case the Cochrane review is sort of like undergoing the typical journal peer review process, so I'll give them a pass on including these preprints. It is important to note, however, that preprints comprised about half of all the studies included in the meta-analysis.
- Overall, they identified 1430 studies to screen, using a detailed search strategy. On further analysis using their predetermined content and quality criteria, they distilled that down first to 266 studies to look at in detail. Of those, only 57 reports of 54 studies met final quality and content criteria to be included in the meta-analysis. This degree of whittling down the study numbers is not unusual in a very broad search needed for meta-analysis. Studies are excluded for a variety of reasons, including not only problems with study design but also with not providing enough detail to assess the study conclusion.
I include the diagram below from the report to illustrate how quality criteria are summarized from the 57 included reports. Here, "green is good." So, you can see that even among the studies that passed the criteria for inclusion, most of them had significant problems.
The authors had several key conclusions:
- Most of the studies included only hospitalized patients, which could lead to some bias by studying patients at the more severe end of the disease spectrum. We don't know if the results could apply to those with asymptomatic or mild disease.
- Antibody testing in general seemed to have lower reliability early after onset of symptoms, which is true for most infections. It takes time for antibody production to develop after infection. However, most studies did not follow patients for more than about a month after onset of symptoms, leaving unknown how long antibody can persist after infection.
- Overall the studies involved a multitude of different assays, each possibly different in terms of sensitivity and specificity, making any broad conclusions more difficult.
Like all Cochrane reviews, the authors included a "Plain Language Summary" of the results, and I think it's helpful to see their bottom line for implications: "...antibody tests could have a useful role in detecting if someone has COVID-19, but the timing of when the tests are used is important.... The tests are better at detecting COVID-19 in people two or more weeks after their symptoms started, but we do not know how well they work more than five weeks after symptoms started... Further research is needed into the use of antibody tests in people recovering from COVID-19 infection, and in people who have experienced mild symptoms or who never experienced symptoms."
I would add to this that the overwhelming majority of patients studied were adults; we don't know much about the pediatric population.
Caveat emptor.