Discussion Board

Expand all | Collapse all

Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

  • 1.  Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-06-2020 17:33
    For anyone on this listserv who is currently involved in testing patients for COVID-19, here is a new free article from JAMA evaluating when to test and how to interpret results. 

    Best,
    Helene 

      
       Website 
       Twitter 
     



  • 2.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-06-2020 18:55

    Thanks, Helene!  This is terrific.


    Hope all is well on your end of the world!


    Mike




    Michael A. Bruno, M.D., M.S., F.A.C.R.   
    Professor of Radiology & Medicine
    Vice Chair for Quality & Patient Safety
    Chief, Division of Emergency Radiology
    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737
     







  • 3.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-11-2020 16:06
      |   view attached

    Attached is what I have been sharing with our providers. This is a 'best-effort-good-faith' summation of my understanding, based on often imperfect literature (many papers do not provide details of the covid-19 test used, other than to say that it is an RT-PCR test, or a combined antibody assay) so it's hard to truly understand test sensitivity and specificity).

     

    Feel free to use, discard, send me suggestions for improvement or just share your thoughts/comments.

     

    Rana

     

    Rana Samuel, MD, FCAP

    Chief, Pathology and Laboratory Medicine Service (PALMS, 113)

    Medical Review Officer, Federal Drug Free Workplace Program (DFWP)

    VA Western New York Healthcare System (VAWNYHS)

    3495 Bailey Avenue, Buffalo, NY 14215

     

    Lead pathologist – VISN 2

    Regional Commissioner, Region 2, National Enforcement Office

     

    Ph:    716-862-8701

    Fax:  716-862-7824

    Rana.samuel@va.gov

     

     

     




    Attachment(s)



  • 4.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-11-2020 16:28

    The laboratory-based, IgG-only, EIA antibody assays have much better specificity, i.e. lower false-positive rate, than described in your document. The one I'm most familiar with, namely the one from Abbott, cites >99% specificity in their package insert (1 out of 186 pre- sera showed a false pos if I remember correctly). Unpublished data from this same assay in use at University of Washington shows similar results.

     

    Handheld rapid serology tests are almost certainly much less specific (and probably less sensitive as well), but there's really no good reason for anyone to be using those devices anyway.

     

    --Brian Jackson

     






  • 5.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-12-2020 10:01
    BMJ paper just published on-line explains how to interpret a COVID-19 RNA test and warns against believing every negative test result. Need to think like a bayesian and avoid base-rate neglect.
    jpg



    ------------------------------
    John Brush
    Sentara Healthcare
    ------------------------------



  • 6.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-12-2020 10:15
    So glad for this discussion and the words of wisdom you all share (Dr. Brush-I have a prized autographed copy of your book!). 

    Below is language we've developed for a recent manuscript. 

    Best,
    Steve 


    At the patient level, however, while SARS-CoV-2 can be ruled in by RT-PCR, it cannot be ruled out due to a sensitivity estimated at 70-80% (25–31). RT-PCR testing has generally been studied with inpatient populations and is less well understood in outpatient settings. Establishing a pre-test probability is complicated as COVID-19 symptoms are nonspecific (or even absent in asymptomatic testing). The decreased prevalence of COVID-19 in outpatient testing also lowers its negative predictive value. RT-PCR accuracy is further reduced by variation in the quality of specimen collection and its source (26). Sensitivity may be lower both early and late in the clinical course; viral dynamics indicate that the highest nasopharynx or sputum viral shedding is in the first week of illness, so there is a greater chance of a positive test at that time (32,33). If a patient is tested too early (i.e. on the first day of symptoms) or too late (i.e. when viral load is higher in the lower rather than upper respiratory tract), the RT-PCR could be falsely negative (26,34). Serology testing is not helpful for diagnosis as IgM does not become reliably positive until day 8 of symptoms (34).

    Many diagnoses of COVID-19 will be clinical given the limited effect a negative result would have on management (35). Patients may have not had, and may not need, a test for SARS-CoV-2. Instead, symptomatic patients should be advised to self-isolate per CDC recommendations (36). Should a RT-PCR test be performed, test results must be informed by clinical reasoning when making recommendations to patients (i.e. even with a negative test result, patients may still be infected and infectious). While this approach overcalls the number of patients with COVID-19, it is preferable to the risk of worsening contagion due to false reassurance.

    Image.jpeg








  • 7.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-13-2020 00:13
    Dr Brush,

    The BMJ authors did a calculation on how to further interpret results after TWO negative "independent" pcr tests;

    can you delineate further how you think that they defined "independence"??

    I assume its not simply because the tests were done at two separate times.... ???

    Isnt their conclusion problematic because we dont know the reasons why the first test was negative... ie theres a big difference between faulty specimen collection vs test done "too early" (or even too late...) in the disease course...

    I dont view the two tests as independent thus you cant combine the test results to calculate a "modified" NPV in my mind

    am i correct in my thought process?

    Thanks

    Tom









  • 8.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-13-2020 00:55
    The literature is mixed about whether repeated tests are independent or not.  (hemoccult, blood cultures, etc)   

    Independence has multiple definitions including the commonly used definition from the QUADAS-2 criteria, which state that independence means that the two tests in question are related neither in their ordering nor their interpretation.  If the decision to order two tests is made a priori, then it seems that independence (or at least a degree thereof) is a reasonable assumption.

    We must keep in mind that most of the loss of sensitivity of the PCR test for SARS-CoV-2 resides likely in the pre-analytic phase of the testing process.



    --
    Andrew P.J. Olson, MD
    Associate Professor of Medicine & Pediatrics
    Interim Director of Hospital Medicine, UMMC
    Director of Medical Education Research and Innovation
    Medical Education Outcomes Center
    Director, Becoming a Doctor Course
    University of Minnesota Medical School
    (o) 612-625-2290
    (p) 612-899-2999





  • 9.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-13-2020 01:16


    Thanks Andrew

    I've seen many learned people make the assertion/assumption that "poor" specimen collection is the etiology of false neg PCR tests results (including many lab directors who may be inherently biased to NOT "blame" the lab sequencing assay) but do we reallllllllllly know that that assertion is true??? (assuming that one is sampling the same body cavity and not comparing nasopharyngeal swabs vs rectal swabs...)

    Couldnt it be that SOME pts are not shedding much nasopharyngeal virus early in the course of the disease and that (at least for some pts) the test is a true negative early in the course? (similar to obtaining a false neg mammo or pap smear on a diminuitive carcinoma vs a later exophytic necrotic tumor)

    I sometimes wonder if we are giving the molecular folks too easy an "out" by "blaming" suboptimal collection technique on the front line teams (acknowledging that proper specimen collection is a legitimate concern)

    It seems to me that the only way to prove that two apriori ordered simultaneously obtained tests are "better" than one would be to do a prospective cohort with all consecutive pts sampled twice on purpose and determine the correlation between the first and second tests

    I'm not aware that that study has been done (to my knowledge all the studies that have done a second test (after a first negative nest) have obtained them at DIFFERENT points in time during the disease evolution which precludes knowing the accuracy of the first test

    we have a lot to learn here....

    Respectfully

    Tom






  • 10.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-13-2020 08:36
    I think it is important to remember that probability estimates about single patients are estimates with a lot of unavoidable imprecision. They can be used to calibrate one's intuition about a patient but should not be viewed as a precise calculation that can provide an absolute answer. After all, one still needs to determine a threshold for action and whether the probability estimate about a patient exceeds that threshold. Where you draw the line for that threshold is, itself, an additional subjective judgement. This unavoidable imprecision is recognized by thoughtful clinicians and is, I think, why they rely on intuition and don't routinely use calculations and formal decision analysis in daily practice.

    In our paper, we discussed the studies that help us determine the sensitivity and specificity of PCR testing for COVID-19. We discussed the lack of a true gold standard and variability among those studies about the sensitivity and specificity. We discussed how handy a likelihood ratio is, but remember, a likelihood ratio is an estimate which is calculated from imprecise numbers. Having said that, these numbers can still be used to semi-quantitatively calibrate one's Intuition about individual patients. 

    The main take-away from our paper is "don't believe a single negative test if you highly suspect COVID-19 in the first place." That would be a fallacy and could be a mistake that would result in carelessly exposing people to an infectious carrier of the virus. Repeated testing can build the case for negativity, but the calculations are based on an assumption of independence of testing (like independence of successive coin-tosses), and that raises questions about the definition of independence. 

    I would view this like getting multiple blood cultures to diagnose bacterial endocarditis. More samples increase the credibility of negative test results. It's not a perfect analogy, but it helps. Having a perfect definition of independence is as elusive as having a perfect measure of sensitivity here and might be another case of false precision. 

    There is abundant uncertainty about every aspect of COVID-19, which frustrates everyone. Thinking probabilistically about the test results does not eliminate the uncertainty, but helps us quantify the uncertainty to get our thinking in the right ball park.

    John Brush

    John E. Brush, Jr., M.D., FACC

    Professor of Medicine

    Eastern Virginia Medical School

    Sentara Cardiology Specialists

    Sent from my iPad





  • 11.  RE: Interpreting Diagnostic Tests for SARS-CoV-2 | JAMA | JAMA Network

    Posted 05-13-2020 14:51
    The important point is made in the BMJ article that a positive COVID-19 test with a likelihood ratio of 14 (greater than 10) is a very strong positive test result. It is due to this that a positive test is interpreted as indicating COVID-19 disease in any patient including asymptomatic patients in whom the prior probability of disease is less than 5 percent and thus the posterior probability is less than 50 percent. Therefore it is the overall performance of this test result due to its high likelihood ratio of 14 which makes it highly reliable in diagnosing COVID19 disease in any patient and not Bayesian analysis.
    A negative test with a likelihood ratio of 0.3 makes it practically worthless and it should not be taken into account in assessing whether COVID19 disease is absent  regardless of its prior probability. An assessment should be done purely on clinical grounds and appropriate action taken as is done at present.
    Till we have a better negative COVID19 test with likelihood ratio less than 0.1, I believe, a straightforward approach to interpreting  a COVID19 test would be; a positive test in any patient indicates presence of disease and a negative test should be ignored and assessment performed on clinical grounds only.

    Bimal Jain MD
    Northshore Medical Center
    Lynn MA 01904.