Discussion Board

Expand all | Collapse all

Decision support can improve diagnosis

  • 1.  Decision support can improve diagnosis

    Posted 10 days ago

    Many of our ideas on how to improve diagnosis will take years to achieve impact, but IMHO decision-support products to improve differential diagnosis are already here, and have the potential to reduce diagnostic errors TOMORROW if they would just be utilized. 

     

    I've enclosed a just-published study from Matt Sibbald et al who studied the Isabel system, and my editorial on this.  These resources need to be widely available to front-line clinicians, and used routinely in everyday practice.

     

       Mark

     

    Mark L Graber MD FACP

    Founder and President Emeritus, Society to Improve Diagnosis in Medicine

    Professor Emeritus, Stony Brook University, NY

    Cell:  919 667-8585

    Graber.Mark@Gmail.com

    Text  Description automatically generated with medium confidence

     



  • 2.  RE: Decision support can improve diagnosis

    Posted 10 days ago
    What a sad realization; "it will take years to achieve impact, but”. I hope some will visit my new web site https://drirawilliams.com and indicate some interest in what I have to offer - BUT - I have NO reason to be optimistic.

    Ira
    Dr. Ira Williams
    drirawilliams.com
    findtheblackbox.com
    healthcarewarriors.org




  • 3.  RE: Decision support can improve diagnosis

    Posted 10 days ago
    WONDERFUL amazing work!!!

    next we need Geoff and Sandra's teams (no insult to the lead and other authors!!) to replicate their expt in 1: dx scenarios where "actors/model pts" are used and not just in written scenarios (but still using fixed pt scenarios) and then 2: vary the scenarios from typical disease presentations to unusual, atypical presentations in both written and "live pt" settings

    we also need help from the EMR vendors both large and small to modify their software platforms to automatically incorporate these CDS tools into their software (BOTH at the front end AND back end!!!) in order to "push out" diff dx conclusions

    who will pay for that software implementation labor effort is an unanswered question......

    Thanks for a wonderful article and editorial ; truly inspiring stuff!!!


    Tom Westover MD









  • 4.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    The paper by Sibbald et al clearly demonstrates the value of an EDS like Isabel in generating a comprehensive differential diagnosis. This will help us most in thinking of a disease which has an atypical presentation or is rare, that is, one which has a low prior probability. Now when we test this disease and observe a highly informative test result, are we to combine it with the low prior probability in diagnosing this disease according to the prescribed Bayesian method or are we to diagnose it from the highly informative test result alone? I bring this point up because in a comparable situation in a diagnostic exercise in a real patient such as a CPC or a clinical problem- solving exercise, a highly informative test result alone seems to be employed for diagnosing a disease with a low prior probability with a high degree of accuracy.

    Bimal

    Bimal Jain MD
    MassGeneralBrigham/Salem Hospital
    Salem MA 01970





  • 5.  RE: Decision support can improve diagnosis

    Posted 10 days ago
    Where is treatment with either positive, intermediate, or negative results placed in the diagnostic process?

    Robert Bell, M.D.




  • 6.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    Good point, Bimal!  

    If the test has enough positive-predictive power, it will generally carry the day, despite a low pretest probability of disease.  But will we actually choose to use such a test in cases where the pretest probability of the disease in question is super-low?  Partly that depends on the barriers to use it, such as cost, scarcity, risks/invasiveness of the test, and other factors.  If the test is sensitive for the disease but not very specific, then we will undoubtedly get a lot of false positives.

    Something must guide our reasoning as to whether to employ the test in the first place.  I think that guidance should be based on experimental evidence.  But the kind of evidence we need is difficult to get in cases where we are dealing with very rare diseases, and it is hard to apply the evidence-based guidance in cases where we have a very unusual presentation of a less rare disease.  I do think that the dDx help, such as Isabel, can really help us in these sorts of situations. 

    The general approach ought to be that we first establish a diagnostic hypothesis based on the clinical presentation, history and risk factors, and then decide whether to run the most appropriate confirmatory test (if any exists) afterward.  In medicine today it seems to me that we are too often starting with "shotgun" testing, without first forming a diagnostic hypothesis, and there is a clear bias favoring dDx's for which tests are available over those for which the diagnosis is primarily based on history/physical, possibly with a therapeutic trial or conservative management, and then delayed follow-up where the treatment response (or non-response) is itself part of the diagnostic pathway.  

    At least that was what I learned in medical school, back when dinosaurs roamed the Earth.

    Mike


     


    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277





  • 7.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    I have a proposal for what should guide the decision to conduct specific diagnostic tests.

    How about how debilitated the patient is? If a patient is on her deathbed, and the highest probability diagnosis is not right, it seems you should conduct other tests, in some order that takes into account the evolving hypothesis, speed, probability and probably other things, until the correct diagnosis is found

    If the patient is so ill that she is unable to get out of bed and function, I would have almost as much urgency. 

    Less severe symptoms may warrant fewer tests, especially invasive and expensive ones. 

    Susannah Cahalan describes in her book Brain on Fire her experience being in a psychiatric institution until she was finally diagnosed with anti-NMDA-receptor encephalitis and fully recovered. I wonder how many others in psychiatric institutions have similar conditions? 

    It seems to me that patient symptoms and outcomes should be central to the diagnostic process. This applies not only to tests but to using resources like Isabel, consulting others, and problem solving. One example of where this didn't happen is the case in this Lisa Sanders column

    What do you think?
    Thanks,
    Megan

    --
    Megan Golden, CEO






  • 8.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    Hi Mike, thanks for your thoughtful comment. The reality in practice is that any given disease occurs in different patients with varying presentations and therefore with varying prior probabilities with our goal being to diagnose a disease accurately in every patient regardless of its prior probability. An EDS such as Isabel helps us in the first step of diagnosis in suspecting a disease with an atypical presentation and formulating it as a hypothesis. The question I raised is how do we verify this hypothesis as being correct by testing. In practice this seems to be done from observation of a highly informative test result (likelihood ratio (LR) greater than 10). For example, in a clinical problem solving exercise, that I have often discussed in my posts, acute MI is diagnosed accurately with a high degree of confidence from the highly informative test result, acute ST elevation EKG changes (LR 13) in a 40 year old woman with highly uncharacteristic chest pain (prior probability of acute MI of 7 percent). It is not diagnosed to be indeterminate from a calculated posterior probability of acute MI of 50 percent which would be done in the Bayesian method. The reason the discussing physician diagnoses from the test result alone is, because in his experience this test result diagnoses acute MI accurately in 8 to 9 out of 10 (86 percent) patients in a random sample of patients with varying prior probabilities in whom acute MI is suspected. He has no experience of observing a frequency of acute MI of 50 percent that corresponds to the Bayesian diagnosis of acute MI being indeterminate from a posterior probability of 50 percent. To have this experience, he would need to be living in a universe in which all patients in whom acute ST elevation EKG changes are observed, have the same prior probability of acute MI of 7 percent. It was in response to such a situation that Charles Sanders Peirce made the comment that Bayesian inference would be justified if "universes were as plenty as blackberries". What is needed to infer accurately in this one universe in which we encounter a disease, for example, in patients with varying prior probabilities, according to Peirce is a highly reliable procedure that leads to an accurate inference with a high frequency in patients that we actually encounter. The method that he developed for this purpose is the frequentist confidence method of statistical inference that was later refined by Neyman.
    It is unfortunate physicians have not looked at the frequentist methods (which include the confidence method) of statistical inference because they describe precisely how diseases are diagnosed in practice with a high degree of accuracy in patients with varying prior probabilities. The frequentist methods are not some esoteric methods, but a major group of methods, which are quite familiar to us from their use for assessing efficacy of a treatment by a clinical trial.
    It is not surprising the Bayesian method is not employed for diagnosis in practice, because this method, as I have pointed out repeatedly, was not prescribed due to its diagnostic accuracy, but due to its coherence, which does not concern us in diagnosis in practice.

    Bimal





  • 9.  RE: Decision support can improve diagnosis

    Posted 9 days ago
      |   view attached
    Hi Megan,

    I think my answer is that a test result is not the same as a diagnosis (correct or otherwise) except in rare cases.  For example, a bone fracture seen on an x-ray is a test result that pretty much equals a diagnosis.  But that is not typical.  Most of the time test results are only part of the picture, and they can often be misleading.  

    If someone is critically ill, seemingly on their deathbed, testing them for everything under the sun in a haphazard manner is more likely to give false positives and lead into a rabbit hole rather than reveal the correct answer.  In such cases, when the most likely or most-favored diagnosis has been excluded, the clinician needs to thoughtfully consider other diagnostic hypotheses, even though they may be, or at least seem to be, less likely.  That's where a clinical decision support system like Isabel can be so incredibly helpful.  It is designed to help the clinician re-think the possibilities when their initial diagnostic hypothesis has to be abandoned because it is not supported by new developments in the patient's condition.

    Consider a patient with undiagnosed sepsis, which is always a tough diagnosis to make prospectively, and which can be rapidly fatal.  If you are caring for a critically ill patient with sepsis, and you aren't sure what the problem is, it won't help at all to do an MRI of the brain and a CT scan for pulmonary embolism, troponins, d-Dimer, TSH, shoulder ultrasound, EEG, barium enema and a pregnancy test!  I'm being a little silly, but I think you see the point.  

    The doctor first needs a diagnostic hypothesis, and they also need to have enough open-mindedness to change or revise it when new data come along.  Once there is a solid diagnostic hypothesis, then the doctor can test that hypothesis... but only if there is a good test for that disease available.  In the example of sepsis, there really is NO good test for it.  A few inexpensive tests, like a CBC, can be very helpful, but no test fully establishes the diagnosis.  This is the more typical diagnostic scenario we find ourselves in, actually; a disease where there is no test that is truly diagnostic in and of itself, and no test that will give the doctor an answer they were not looking for.

    I've attached a nice paper that discusses the laboratory diagnosis of sepsis.  As you can see, it is not as simple as "order a test, get the answer."

    Mike



    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277



    Attachment(s)

    pdf
    Diagnosing sepsis.pdf   351 KB 1 version


  • 10.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    Thanks, Bimal.

    Nice example with the 7% MI pre-test probability... EKG testing doesn't take her to 50% (as predicted by a straightforward Bayes' calculation), but rather to a much higher likelihood when ST elevations are seen.  Bayesian reasoning seems to be the most useful when there aren't highly predictive tests to rely on.

    I'm thinking it might be a very nice seminar to host at a future DEM meeting to have you teach the frequentist vs. Bayesian approaches, using examples like this.  

    Mike
      


    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277





  • 11.  RE: Decision support can improve diagnosis

    Posted 9 days ago
    Mike, thanks for your kind comments. I will be happy to discuss Bayesian versus frequentist reasoning in diagnosis in any forum, perhaps virtual for the time being.
    Bimal