Discussion Board

Expand all | Collapse all

An assessment of suitability of the Bayesian method for diagnosis

  • 1.  An assessment of suitability of the Bayesian method for diagnosis

    Posted 07-30-2021 11:43
      |   view attached
    I have been writing for quite some time about the role of Bayesian reasoning in diagnosis without receiving any significant feedback from SIDM members. While all of us are interested in diagnosis, one of the reasons for this lack of feedback may be unfamiliarity with technical and historical aspects of Bayesian reasoning which I have come to appreciate after a lot of reading. It would help me immensely in carrying the discussion forward if SIDM members could express their thoughts in response to my writings, whether it is something they have read or something from their experience about diagnosis. In my view, it is not important whether members agree or disagree with me, but important to have a meaningful discussion from which we can all learn.
    In the attached essay, I have presented my thoughts on Bayesian reasoning in diagnosis as simply as I could in numbered items to make it easier to respond to specific points.

    Bimal
    Bimal Jain MD
    Mass General Brigham/Salem Hospital
    Salem MA 01970.
    The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

    Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 



  • 2.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 07-30-2021 12:13
    Hi Bimal,
    As I recently finished the volume, and it pertains directly to your points below, I thought that I would highly recommend that you take a look at David Cameron Morrell's "Diagnosis in General Practice: Art or Science" (John Fry Trust Fellowship, 1993). In particular, I believe you'll find chapters 4 and 5 of particular interest. As said monograph was written from a speech delivered in 1993, Morrell encapsulates a great deal of the concerns you have raised and refers to the prevailing literature up to the late 80s/early 90s.

    Since Morrell's lecture, further research, including that from SIDM members, has aimed to clarify aspects of the state of the art (e.g. here are two reviews to that end: Norman, Geoffrey. "Research in clinical reasoning: past history and current trends." Medical education 39, no. 4 (2005): 418-427.  AND Koufidis, Charilaos, Katri Manninen, Juha Nieminen, Martin Wohlin, and Charlotte Silén. "Unravelling the polyphony in clinical reasoning research in medical education." Journal of Evaluation in Clinical Practice 27, no. 2 (2021): 438-450.).
    Most importantly, I would argue that the nature of "the Bayesian brain" or Bayesian models of statistical reasoning for medical decision making (such as those proffered originally by Ledley and Lusted) and their relevance to medicine, both in terms of medical education and formal models of diagnosis/point of care clinical practice, have come to a consensus that diagnosis as practiced is not solely a technical/statistical algorithm (see Norman, Geoff, Meredith Young, and Lee Brooks. "Non‐analytical models of clinical reasoning: the role of experience." Medical education 41, no. 12 (2007): 1140-1145. AND Koufidis, Charilaos, Katri Manninen, Juha Nieminen, Martin Wohlin, and Charlotte Silén. "Representation, interaction and interpretation. Making sense of the context in clinical reasoning." Medical Education (2021). for example). Yet, I would note that Bayesian statistics has been the approach with which epidemiologic causality has been advanced most recently, which is arguably a component of the truth statement assessing the performance of diagnosis.

    Ultimately, whether we adopt a Bayesian or frequentist approach to counting and frame the task of diagnosis as one of classification or hypothetical-derived inference (not that this description is exhaustive), the problem as I see it is not a correct model, but rather a more appropriate mechanism to each the meta-cognitive aspects of diagnosis to learners such that they will be able to assess their performance. As said student's expertise grows (and as the practicing physician knows) they will begin to truck with mastery-based models of competence that eschew any semblance of formal statistical proofs for each diagnosis.

    Instead, in my mind research in diagnosis might benefit from a focus on engineering forensic methods (which may support either counting frame), or address areas of deficits seen in the processing of information by said frame, purposed to best support research into diagnosis in the future. That said, if you want to have a conversation about formal mathematical (rather than statistical) models of diagnosis, including information and the nature of nosology as it pertains to the task of diagnosis itself, I would highly recommend building upon and integrating the seminal work of both Marsden Blois (Information and the Nature of Medical Description) and Richard Stanley Melton (A Comparison of Clinical and Actuarial Methods of Prediction With an Assessment of the Relative Accuracy of Different Clinicians).

    Cheers,

    David
    --
    David Chartash B.E.Sc. (Western Ontario), M.H.Sc. (Toronto), Ph.D. (Indiana)
    Lecturer, Center for Medical Informatics, Yale University School of Medicine
    300 George Street, Suite 501 E-06, New Haven, Connecticut, United States of America, 06511

    Notice of Confidentiality
    The information transmitted is intended only for the person or entity to whom or which it is addressed and may contain confidential or privileged material.  Any review, retransmission, dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited.  If you receive this in error, please contact the sender immediately by return electronic transmission and then immediately delete this transmission including all attachments without copying, distributing, or disclosing content.

    Avis de confidentialité
    L'information transmise est strictement réservée à la personne ou à l'organisme auquel elle est adressée et peut être de nature confidentielle.  Toute lecture, retransmission, divulgation, ou autre de nature confidentielle.  Toute lecture, retransmission, divulgation, ou autre action prise sur la foi de cette information par des personnes ou organismes autres que son destinataire est interdite.  Si vous avez reçu cette information par erreur, veuillez contacter son expéditeur immédiatement par retour du courrier électronique puis supprimer cette information y compris toutes pièces jointes sans en avoir copié, divulgué, ou diffusé le contenu.





  • 3.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-01-2021 11:49
    Hi David,
    Thank you for your detailed, highly informative response with all the references. Some of the references I had read before while others I read after you pointed them out. In addition, I have ordered Morrell's book from Amazon which will be delivered after 3 to 4 weeks from a used books store.
    Recently, I have been reading history of science especially about how some well-known scientific theories were developed. Every theory that was proved to be correct was developed, as far as I could tell, by starting with a careful study and analysis of the 'phenomena', that is, of available empirical information about the subject of interest. For example, Newton started with a careful study of Kepler's observations represented by his three laws of planetary motion in developing his gravitational theory of planetary motion, Maxwell started with a study of Faraday's experimental researches on electricity and magnetism in developing his electromagnetic theory, Darwin started with study of artificial selection practices of horse and dog breeders in developing his theory of evolution by natural selection, Einstein started with a study of light signals in inertial systems at different speeds in developing his Special Theory of Relativity and Watson and Crick started with study of X-ray diffraction patterns of DNA in developing their theory of double helix structure of DNA.
    A theory that was developed without starting with a study of the 'phenomena' such as Descartes' vortex theory of planetary motion or Einstein's Unified Theory uniting gravitation and electromagnetism turned out to be incorrect.
    The great advantage of starting with study of phenomena in developing a theory is that such study often provides clues and hints of the correct theory. For example, it was by studying and analyzing Kepler's first and second laws that Newton developed the idea of a centrally located gravitational force of the sun keeping a planet in its elliptical orbit. Similarly, it was by studying Faraday's lines of force represented by arrangement of iron filings around a magnet that Maxwell developed the idea of an electromagnetic field.
    In the case of diagnosis, we have the 'phenomena' of physicians all over the world performing diagnosis on a daily basis with an overall high degree of accuracy of 85 to 90 percent. We have records of diagnosis in real patients by experienced physicians in published clinical-pathologic conferences (CPCs) and clinical problem-solving exercises and in a less structured way in hospital and office/clinic notes. It seems to me that none of the theories that have been proposed for diagnosis has been developed by starting with a study of the process of diagnosis in practice. A prime example of such a theory, in my view, is the Bayesian theory of diagnosis, which has been put forward purely on the basis of its coherence without developing it after study and analysis of diagnosis in practice. It is for this reason, I believe, this theory is incorrect and thus not employed for diagnosis in practice.
    I believe, we need to start with a careful study and analysis of the process of diagnosis in practice to develop a correct theory of diagnosis. I am trying to do so at present and if something comes out of this attempt, I shall post it on Discussion Board for review and comment.

    Bimal





  • 4.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-01-2021 13:32
    Hi Bimal,
    I think you've narrowed in on a fork in the road as it pertains to the study of diagnosis that results in two questions: are we studying signs, symptoms, disease presentation and the like such that we are studying humans during the clinical encounter and rendering a scientific theory that we call diagnosis OR are we studying the physician's act of diagnosis (and related signs/symptoms/diseases) such that we are rendering a scientific theory of the practice of diagnosis?

    What you've described, and certainly my understanding of how the history of science has framed the creation of theory from the Victorian era, has been that the investigation of fundamental measurements such that experimentation yields theory. This is emphasized in the successes of the 20st century from the basic natural sciences (namely physics, chemistry and biology), but has been limited in creating definitive theory in the social or human sciences.
    One can argue that this is the result of an incoherent standard system of measurement, or, as psychology has put it (per Cronbach, Lee J., and Paul E. Meehl. "Construct validity in psychological tests." Psychological bulletin 52, no. 4 (1955): 281.) an incoherent nomological network.
    Efforts to reconcile different scientific disciplines and theories have resulted what can be termed "meta-scientific" research, seeking to untangle the complexity of the relationships between disciplines. I would argue that with your examinations over the past year, you are edging closer to the spirit of meta-science best articulated by Mikhailov et al (Mikhailov, A. I., A. I. Chernyi, and R. S. Giliarevskii. "Scientific Communication and Informatics,(translated by RH Burger)." (1984).) in defining the field of informatics as a "science of science", albeit in a limited form. To that end, I would suggest that working to integrate the theoretical explanations of diagnosis is in order, not solely to create a better theory, but understand how their framings of measurement in clinical practice or diagnostic nosology matter.
    I think you will find Morrell's monograph, as well as the work more broadly by the generalist physicians of the 1960s (for example Crombie, D. L. "General practice today and tomorrow. X. Diagnostic methods." The Practitioner 191 (1963): 539-545. or Asher, Richard. "Clinical sense." British medical journal 1, no. 5178 (1960): 985.) or the musings of physicians prior to the turn of the epidemiological age (such as Collins, Sir William Job. "Physic and Metaphysic." St. Bartholomew's Hospital Journal, no. Supplement (April 1905): 1–8. or Herringham, W. P. "Observation and Imagination." St. Bartholomew's Hospital Journal, no. Supplement (February 1906): 1–4.) most useful.
    All of these physicians are attempting to interrogate the clinical act of diagnosis, primarily by examining the "core acts of doctoring" outside of the newer notion of the doctor-patient relationship (and indeed the eventual model proposed by the general practitioners of the patient-centered clinical method, emphasized by Stewart, Moira, et al. Patient-centered medicine: transforming the clinical method. Radcliffe Publishing, 2003.). While patient-centered medicine is both pragmatic and useful, especially in an age where primary care has been so changed by physician extenders... the larger question of the core act of diagnosis remains (especially that which is the physician's provenance, separate from the classification task often described by the physician extender).

    Whether such a theory of diagnosis means the inclusion of priors, or the counting of relative frequencies of symptoms, or the distinct identification of symptom-disease complexes (e.g. Irwin, Richard S., and J. Mark Madison. "Symptom research on chronic cough: a historical perspective." Annals of internal medicine 134, no. 9_Part_2 (2001): 809-814. or Kroenke, Kurt. "A practical and evidence-based approach to common symptoms: a narrative review." Annals of internal medicine 161, no. 8 (2014): 579-586.), empirically examining patients and physicians at scale would be my next step, with careful attention paid to the principles of observational social science.
    I look forward to any well reported examinations of a clinical practice, be it yours or others', that make good on such empirical observation.
    Cheers,

    David





  • 5.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-02-2021 09:46
    Hi David,
    Thanks for your response. I am trying to understand all the points raised by you.
    Regards
    Bimal





  • 6.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-02-2021 10:12

    Hi David, Hi Bimal,

     

    As always, this is a really fun discussion!  I have to agree that there aren't very many examples where the theory that proved correct was created de novo, without any experimental / experiential data first.  The best example is the Standard Model of particle physics, which led Higgs to predict the existence of a new heavy boson which had zero basis in experiment.  Only much later, after building the largest experimental apparatus ever constructed at CERN, was the existence of this particle verified.  So... it can happen.

     

    I think that there is a lot of emphasis placed on the final steps in establishing a diagnosis, i.e., the final diagnosis which glosses over some of the intermediate steps of high importance to the final product, i.e., the development of a differential diagnosis, advancement to a provisional diagnosis and later, a systematic evaluation of provisional diagnoses-which often takes form of a therapeutic trial.  Some of those intermediate steps are clearly Bayesian in practice as well as in theory.  The best example I can think of from my own experience is radiologists' interpretation of imaging studies, which often informs the final diagnosis substantially.  The reasoning behind those interpretations are largely Bayesian, due to the very high level of uncertainty involved.   

     

    So I still believe that Bayesian reasoning has a place in the diagnostic process-even if it is not the final step.

     

    All the best,

     

    image004.png@01D112FF.F77F98B0

    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  |  
    1571679014277

     

    *****E-Mail Confidentiality Notice*****
    This message (including any attachments) contains information intended for a specific individual(s) and purpose that may be privileged, confidential or otherwise protected from disclosure pursuant to applicable law.  Any inappropriate use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalty.  If you have received this transmission in error, please reply to the sender indicating this error and delete the transmission from your system immediately.

     






  • 7.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-02-2021 14:54

    The discussion reminds me of Arthur Elstein's work, where he was seeking to understand diagnosis by having them work through case scenarios, and explaining their reasoning.  This seems to be the kind of observational approach that Bimal is suggesting.  And Arthur's conclusion was that a great deal of diagnosis was indeed pattern recognition.

     

    It is also possible that more than one "answer" is correct.  Perhaps certain specialists (eg Radiologists, like Michael Bruno) might be more "Bayesian" in their approach than front-line clinicians seeing undifferentiated cases.  Or certain problems may be more amenable to a Bayesian approach  (eg pulmonary embolism) than undifferentiated problems seen in primary care.  For the diagnosis of pulmonary embolism, there is a ton of data on pre-test probabilities and on the power of the relevant diagnostic tests, with an abundance of tools to calculate the likelihood of PE in a Bayesian manner. 

     

    Mark

     

    Mark L Graber, MD FACP

    Founder and President Emeritus, Society to Improve Diagnosis in Medicine

    Professor Emeritus, Stony Brook University

    919 667-8585

    A picture containing logo  Description automatically generated

     

     






  • 8.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-02-2021 14:59
    Absolutely Mark!
    For those interested, I'd also recommend Janet Gale and Philip Marsden's "Medical Diagnosis: From Student to Clinician" who takes the Elstein study and extends it forward into the British clinical education environment for both neurology and endocrinology.
    I would also suggest that perhaps there is a difference between primary care and specialty care (similarly to the divide between the generalist and hospital-specialist, but more vocationally focusd) that might impact the clinical reasoning process.
    Cheers,

    David





  • 9.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-02-2021 16:21
    Three Questions.

    Will artificial intelligence for this topic help sort it all out? 

    When will what is discussed here about Bayesian diagnosis benefit patients?

    Would a survey now of our opinions on how long it will take for that benefit with patients to be seen and maybe help clarify something?

    Rob Bell





  • 10.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-03-2021 09:45
    Hi Mark,
    Thanks for your post. In the Bayesian approach, a disease is diagnosed from a posterior probability interpreted as total degree of belief that is generated by combining a prior probability and likelihood ratio (LR) for a test result. What I have been saying is that a disease is not diagnosed in practice in a Bayesian manner as we see clearly with the example of diagnosis of pulmonary embolism.
    Let us suppose the prior probability of pulmonary embolism is very high at 80 percent in a patient. We suspect pulmonary embolism and perform a chest CT angiogram which yields a positive result whose LR is known to be 20. The posterior probability that is generated is very high and we diagnose pulmonary embolism from it. 
    Let us now suppose the prior probability is very low at 5 percent in another patient in whom we suspect pulmonary embolism. We perform a D-Dimer test as recommended and its level is greater than 1. We then perform a chest CT angiogram which yields a positive result whose LR is 20. We combine the prior probability of 5 percent and LR of 20 to generate a posterior probability of 50 percent from which the Bayesian diagnosis would be of pulmonary embolism being indeterminate in this patient. But this is not what is done in practice; in practice we diagnose pulmonary embolism in this patient with near certainty from this test result alone in a non-Bayesian manner.
    In fact, pulmonary embolism is diagnosed with near certainty in any patient from a positive chest CT angiogram in any patient regardless of its prior probability with a high degree of accuracy all over the world. If we investigate why pulmonary embolism is diagnosed in this manner, we find it is the frequentist confidence method, that I have described elsewhere, is being employed instead of the Bayesian method for diagnosis.
    An even better and more topical example of non-employment of the Bayesian method for diagnosis is furnished by diagnosis of covid-19 disease. This disease is diagnosed with near certainty in any patient from a positive covid-19 PCR test with LR 14 with a high degree of accuracy in any patient regardless of its prior probability all over the world in a non-Bayesian manner.
    In my view, there is overwhelming evidence, if we look closely at diagnosis in practice, that any disease which has a test capable of generating a result with LR greater than 10 is not diagnosed in a Bayesian manner in practice. I believe, it is time now that instead of being stuck on the Bayesian method, we start investigating how a disease is actually diagnosed in practice. 
    Bimal





  • 11.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 08-03-2021 11:35

    Bimal –

     

    A quick note. "Diagnosis" is a decision, and so, posterior probabilities alone would not account for behavior; One must take into account thresholds as well, which are functions of the benefit to the patient of treating the disease and the cost (financial and safety) to the patient of not treating (assuming we take the patient's perspective in diagnosis).

     

    So, "diagnosing with near certainty" may simply mean that the posterior is so far away from threshold that it is doubtful that more information would change your mind. If my threshold for performing an LP is 1/1,000 (which residents have reported to me over 20 years' of asking), then a posterior of 10% is way over threshold, and I would have not doubt that the patient "needs" an LP. Whether I should be equally certain about the patient having meningitis---and therefore hospitalizing and treating, with a new set of costs---depends on a different threshold.

     

    That's the decision theory behind diagnosing. I don't know that we have a lot of empirical data on thresholds, but if one were to assemble a research program on diagnosing, that's the place of theory to start from. (Doesn't Benjamin Djulbegovic's work come into play here?), but, if

     

    Also, I think we have to be careful, though, in taking physicians' assessment of certainty as gold standard, since there is plenty of data on wrong diagnosis (with respect to autopsies) and on the lack of correlation between such certainty and accuracy.

     

    Harold

    ************************************************************************

    Harold Lehmann MD PhD

    Professor

    Section on Biomedical Informatics and Data Sciences

    Division of General Internal Medicine

    Department of Medicine

    Johns Hopkins School of Medicine

    lehmann@jhmi.edu

     






  • 12.  RE: An assessment of suitability of the Bayesian method for diagnosis

    Posted 07-30-2021 13:20
    Thanks Bimal,

    Importnat topic.

    For sometime I have been I have thought that SIDM could focus on BIG things they MAY be able change.

    It is my opinion that for various reasons many physicians do not like asking or even answering questions. There are many reasons for this. Does it needs studying?

    Just looking at the questions and answers on this list serve over a period time will help identify the problem.

    Do our patients eventually suffer with poor communication?

    Can we do anything?

    Robert Bell M.D.