Discussion Board

Expand all | Collapse all

Living with uncertainty

  • 1.  Living with uncertainty

    Posted 10-01-2021 10:51
      |   view attached
    Hi everyone!

    I wanted to introduce a new topic for discussion, namely the idea that we need to live with a high degree of uncertainty in medicine, and physicians and other providers especially need to bolster their ability to tolerate uncertainty as they are forming a diagnostic hypothesis.  

    There was an excellent article on this topic that was published in the New England Journal of Medicine in 2016.  Dr. David Chartash sent this article to me earlier this week, and Dr. Tim Mosher (a member of the SIDM Board of Directors) and I found it to be extremely insightful and useful, and we wanted to share it with the entire SIDM community.  The article is attached to this message.  Tim and I hope you can find time to read it and ponder its important message.

    In our current state, too many physicians (and patients, too!) conflate testing with diagnosis.  As a diagnostic hypothesis is formed for a patient, quite often there simply is no test available which could meaningfully reduce the uncertainty that we face.  This is an uncomfortable situation for many people--especially doctors--and so it results in quite a bit of over-testing being done in a futile attempt to reduce the psychological discomfort brought on by the unresolved uncertainty.  The over-testing actually harms patients in many ways: by generating false-positives, by increasing their health care costs, and from actual test-related risks when more invasive types of testing are used.

    To combat this problem, the various medical societies, such as the  American College of Radiology and the American Thoracic Society, among others, have developed guidelines, including Image Wisely and the ACR Appropriateness Criteria (ACR-AC).  

    Image Wisely home page. Image Wisely is a joint initiative of the American College of Radiology, Radiological Society of North America, American Society of Radiological Technologists and American Association of Physicists in Medicine.
    https://www.acr.org/Clinical-Resources/ACR-Appropriateness-Criteria
    The ACR Appropriateness Criteria ® (AC) are evidence-based guidelines to assist referring physicians and other providers in making the most appropriate imaging or treatment decision for a specific clinical condition. Employing these guidelines helps providers enhance quality of care and contribute to the most efficacious use of radiology.
    https://www.thoracic.org/statements/
    The American Thoracic Society improves global health by advancing research, patient care, and public health in pulmonary disease, critical illness, and sleep disorders.

    One of the things that these guidelines address is the limited scope of imaging and other tests to actually answer a particular diagnostic question.  CT scans and other tests can help to answer some questions, but not others, and certainly not all.  So when a physician chooses to order a diagnostic test of any kind which is outside of these guidelines, they do so knowing that the test is unlikely to help.  

    But, as the attached article discusses, they order the unhelpful tests anyway, and quite often, primarily (as my own research has also shown) because of their discomfort with the high level of uncertainty that they face.  Sadly, despite the investment of time, health care resources, and the patient's money, the degree of diagnostic uncertainty is rarely changed after a non-recommended / non-indicated test is completed.

    A useful strategy for improving diagnosis and stewardship of our patient's money and other heatlh care resources would be for all of us providers to think like scientists do: consider our provisional diagnoses as hypotheses, with varying degrees of uncertainty (often very high uncertainty) and only do the "experiments," such as lab tests, CT scans, etc.,  that are likely to actually reduce our uncertainty.  Once the uncertainty is as low as it can be, it makes no sense to do more testing.  Absolute certainty is an illusion, an impossibility.  We need to stop chasing after it.

    All the best,

    Mike




    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277

    Attachment(s)



  • 2.  RE: Living with uncertainty

    Posted 10-01-2021 11:03
    (1) The Ancient Greeks knew that a precondition for controlling something is that the thing have a name. Patients want a name for their illness.

    (2) We have a hard tine quantifying medical uncertainty at the individual patient level.


    Harry B. Burke, MD, PhD
    Professor of Medicine




  • 3.  RE: Living with uncertainty

    Posted 10-01-2021 19:16
    Yes to both 





  • 4.  RE: Living with uncertainty

    Posted 10-01-2021 11:18





    Would an accuracy scale, where it could be done, attached to all "tests" help in any way?

    Robert Bell
    ,





  • 5.  RE: Living with uncertainty

    Posted 10-01-2021 15:00

    Good Afternoon Robert:

     

    What you ask has been tried.  I was a member of the VA national workgroup on diabetes a decade ago and acted as the expert on the laboratory aspects of Hemoglobin A1c.  We determined that it was possible to use quality control data according to the Clinical Laboratory Standards Institute [CLSI] document EP23 to assess measurement uncertainty, to correlate results to blood glucose, and to use it to assess various long term risks.  We then developed a computer application to generate quantitative that could be reported with the HbA1c result with a link to a PDF that presented graphic information and textual information to help the clinician determine if any change in HBA1c was clinically significant or required another follow up value six months later.

     

    All well and good.  Unfortunately, there was tremendous resistance on the part of individual hospitals to implement this system and, in our hospital, which was used to Beta test the idea, our physicians ignored it.

     

    So, the idea is great.  The means of achieving this is available.  Gaining acceptance is an entirely different issue...

     

    Mark Gusack, M.D.

    President

    MANX Enterprises, Ltd.

    304 521-1980

    www.manxenterprises.com






  • 6.  RE: Living with uncertainty

    Posted 10-01-2021 15:28
    Hi Mike,
    Thank you for your post on uncertainty in medicine.
    First of all, I shall comment on the letter of John Keats about uncertainty with which the attached article starts. John Keats (1795-1821), the great English poet trained in medicine as a dresser at Guy's Hospital in London, but never practiced medicine for a single day. The quotation from his letter is part of a larger piece which reads, "What quality went to form a Man of Achievement, especially in literature and which Shakespeare possessed so enormously-I mean Negative Capability- that is, when a man is capable of being in uncertainties, mysteries, doubts, without any irritable reasoning after fact and reason". This piece defines his well-known notion of Negative Capability which is about the quality of living with uncertainty, which is possessed  by great literary figures. I do not think John Keats is talking about practicing physicians in this piece.
    As practicing physicians, our job is to reduce or eliminate uncertainty by diagnosing a disease accurately when a patient presents to us with symptoms so that it can be treated and prognosticated correctly. We do this by formulating a suspected disease as a diagnostic hypothesis which is evaluated by subjecting it to tests. We are successful to a great extent in diagnosing accurately, as the overall diagnostic accuracy in practice is 85 to 90 percent. I think the important point being made by you is that we should not order tests which are worthless when we are uncertain about  diagnosis in a patient. For this it is important for us to know about the informative content of a test result in terms of its likelihood ratio and its capability in terms of a frequency in diagnosing a disease accurately in patients with varying prior probabilities. For example, it is currently accepted that a test result with likelihood ratio greater than 10, such as acute ST elevation EKG changes, positive chest CT angiogram, positive venous ultrasound study, positive covid-19 PCR test is highly informative and diagnoses its respective disease with a frequency of 85 percent or greater. Therefore, these test results are widely used for diagnosing diseases in practice. I believe, it would be extremely helpful in preventing wasteful testing if the likelihood ratio and performance characteristics of various test results are made widely available. Where such information is not available, it should be studied by empirical studies.

    Bimal

    Bimal Jain MD
    Mass General Brigham/Salem Hospital
    Salem MA 01970














    The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at http://www.massgeneralbrigham.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

    Please note that this e-mail is not secure (encrypted).  If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately.  Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail. 






  • 7.  RE: Living with uncertainty

    Posted 10-01-2021 16:07

    Thanks, Bimal!  This approach, separating helpful tests from the unhelpful, is exactly what is needed.

     

    And when there is no helpful test, we must all learn to accept the inevitable uncertainty.

     

    image004.png@01D112FF.F77F98B0

    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Department of Radiology, H-066

    Penn State Milton S. Hershey Medical Center

    500 University Drive, Hershey PA 17033
    ( (717) 531-8703  |   6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    https://infonet.pennstatehershey.net/documents/396359/10678301/Medical+Center+Two+Color/4ea2250e-2e29-4b9f-8d2e-2911ed1af1ea?t=1456671057665

    *****E-Mail Confidentiality Notice*****
    This message (including any attachments) contains information intended for a specific individual(s) and purpose that may be privileged, confidential or otherwise protected from disclosure pursuant to applicable law.  Any inappropriate use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalty.  If you have received this transmission in error, please reply to the sender indicating this error and delete the transmission from your system immediately.

     






  • 8.  RE: Living with uncertainty

    Posted 10-01-2021 19:51
    From the trenches, a problem.
    I have to order the test before I know the result.
    So "helpful" vs "not helpful" cannot be used a priori.
    The input is a probability, not an outcome.
    I.e., if someone has signs/symptoms which might be ascribed to PE, d-dimer might help rule out (normal) to a degree of certainty but not absolutely.
    On the other hand, a (+) (>0.5) result might help... except when age-adjusted, or YEARS criteria, or risk-adjusted.
    So there's a lot of gray at the bedside. This is an area where SIDM could be of help. 
    What degree of certainty is "enough" and will this group put itself in the breech?
    Tom Benzoni





  • 9.  RE: Living with uncertainty

    Posted 10-02-2021 21:19
    Hi Tom,

    Thanks for your note "from the trenches."  The very point of evidence-based guidelines (EBG) are exactly to separate the "likely helpful" vs. "likely unhelpful," and thereby guide clinical decisions such as these.  These extensive published guidelines, such as the ACR Appropriateness Criteria and others, have already addressed the question as to what degree of likelihood is enough (although it is understood that they are not perfect).  By following them, waste is reduced, but not eliminated, and the odds of a missed diagnosis from omitting a test are very, very low.

    For the scenario you outline, i.e., for a patient with a clinically suspected PE, there are two well-established EBG systems that have been shown to be effective--the simplified Geneva criteria and the Modified Wells' score.  In the highly-cited Christopher study, the Wells' score was combined with the d-dimer, and an algorithm was developed to blend BOTH the EBG (Wells) and the d-dimer test, in order to determine which patients ought to be imaged.

    In our own center, despite an extensive multi-department educational outreach and a broad interdepartmental consensus agreement having been made between the Depts of Medicine, the Critical Care Committee, the Emergency Department and the Dept. of Radiology to follow this well-established algorithm to reduce over-utilization of CT scans for pulmonary embolism, most of our providers at the point-of-care were apparently unable to comply.  The result was an approximately 30% over-utilization of the imaging resource compared to what would have been done had the algorithm been followed.  That is to say, nearly a third of suspected PE patients were subjected to a scan when it was very unlikely to have been helpful.  This goes beyond waste, and into the territory of patient harm: financial harm, the harm related to false-positive tests, and the risk of iatrogenic injury, with no real benefit to counteract any of this.  In our experience, a very, very small number of these patients who were scanned outside of the guidelines turned out to have positive scans (again, the EBG criteria are not perfect).  

    When questioned about this behavioral choice, physicians and other providers most commonly point to the "gray at the bedside," that you mention.  The psychologically painful absence of certainty.  The discomfort they feel with the high degree of uncertainty that remains for those patients who have symptoms, but do not meet the criteria for the test.  For most of our providers, this was apparently a level of uncertainty they could not live with, despite the science, and despite the consensus agreement.  They ordered the scans.  

    Surprisingly, very few cited any concern that they might be sued if a PE was missed.  This did not seem to be an example of "defensive medicine."

    I've re-attached the Tolerating Uncertainty article to this message, along with another recent article on Value-Based Radiology that touches on the same issues from a different perspective.  I would encourage everyone to read these articles and think about their message, at least the article on tolerating uncertainty, and I would like to discuss the articles' content on this forum.

    What can SIDM do to help with this problem?  I think as an organization we ought to embrace the use of EBG's for guiding the use of diagnostic testing of all kinds, so as to avoid the risk of false-positive testing and misdiagnosis that can result, and also to prevent other avoidable patient harms, including the harm of patients having to pay for tests that are almost certainly not going to help them, and which expose them to other risks.  And we can use our resources to help doctors and other providers learn to live with the inevitable uncertainty of many diagnoses, despite our universal wish that it were simply not so.  

    We can also teach providers to respect and follow the evidence as a way to improve diagnosis, and not proceed on emotion or "hunch."  And we can help the public to understand that a test is not a diagnosis, and a diagnosis is not a test result.

    Respectfully,

    Mike



    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277







  • 10.  RE: Living with uncertainty

    Posted 10-04-2021 13:16
    Can someone please define “uncertainty.” How is the definition quantified?

    Furthermore, how do we apply the term to an individual rather than to a group, i.e., not the group's risk but the individual patient’s risk?

    Harry B. Burke, MD, PhD




  • 11.  RE: Living with uncertainty

    Posted 10-04-2021 13:35
    I had the great advantage of being in practice for many decades, with a panel of patients who had been with me for many years.
    It was not rare for me to say something like "I am not sure what is causing your symptoms, but based on my exam today, I am fairly sure it is not something that requires immediate treatment. Let's treat the symptoms and see if it does not get better on its own. If anything gets worse or changes, call me."  99% of the time, they were happy to follow this advice, and 95% of the time, whatever "it" was, went away.  When there was something not self-limited, in no instance did a delay of a week or so cause any harm.
    I saved my patients a lot of unnecessary tests.  If I was an ED doctor or urgent care doctor, I would probably not have been able to do this.
    Ed
    Edward P Hoffer MD, FACP, FACC





  • 12.  RE: Living with uncertainty

    Posted 10-05-2021 20:49
    Dr Bruno

    Thank you for your insightful comments as always...

    two thoughts come immediately to my mind
    1: value as defined by healthy policy wonks usually refers to an equation or balance between cost and health benefit for a population, NOT for any given individual patient.... one could theoretically "reliabily" diagnose every clinical problem IF one was willing to throw enough money at the problem; so the "value balance" is more about how much money we spend "in general" rather than for any individual pt

    2; time course plays a vital role in clinical decision making BUT this factor is not usually incorporated into decision rules; Eg, a pt presents with atypical chest pain "rule out PE" , many clinicians apply EBG rules to the first presentation BUTTTT those rules go out the window (and probably justifiably so) iffff the pt presents again (or repeatedly) with persistent and or progressive symptoms!!! Decision models rarely incorporate this time component piece into the original data set / modeling algorithm and thus clinicians instinctually understand this limitation

    Thank you

    Tom Westover MD








  • 13.  RE: Living with uncertainty

    Posted 10-06-2021 08:54
    Thanks, Tom!

    I think you raise an important issue--the existing EBGs are worked out for a single, presumably initial, patient encounter, and not for when a patient returns with persistent or escalating systems (with rare exception). The ones for Radiology utilization are generally based on a combination experimental / empirical evidence and expert consensus.  They are, as a result, very good and reliable, but they are not perfect by any means.  In their essence, they are an attempt to take population measures and apply them to individual patients for the purpose of rationing health care resources at the individual point-of-care.  While this idea may seem anathema to some of us, the word "ration" has the same root as the word "rational."  We are attempting to insert rational decision-making into the utilization process, because the resources we are protecting are limited.  

    I didn't bring it up before, but there is an obvious corollary: low-yield or superfluous use of resources consumes capacity needed by patients for whom the utilization is justified.  For example, if the MRI scanner is booked solid for weeks with patients for whom the EBG would not have supported the use of imaging, then patients with more urgent need will be turned away.  This is happening every day in our country, including at my own center where the backlog of MRI is huge, and patients wait 8 weeks or more for a slot.  They are also denied any ability to influence when they must come for their scan, which further erodes the concept of patient-centered care.  

    You are also 100% correct to say that value incorporates cost.  In an extremely low-yield situation, such as when many patients are referred for CT-PE scans despite a low Wells' score and/or a negative d-dimer, the costs pile up and benefits to individual patients are only rarely accrued.  The EBG in essence draw a line between "the cost is worth it" and "the cost is not worth it."  So they also have an economic rationing function.

    The situation you describe where the patient re-presents with the same or worsening complaints, the evidence is not so strong in most cases (low back pain being the main exception that I am aware of). This gets to the idea that diagnoses evolve with time as additional patient findings accrue, such as non-response to conservative management, or escalating symptoms.  In such cases, the clinician is more or less on their own as they will not get much help from the EBGs.

    I also very much appreciate the earlier comments on this thread about the importance of having a long-term relationship with patients; single-encounter specialties, such as Emergency Medicine, are at a distinct disadvantage.  The history they get is often incomplete and the opportunity for follow-up is essentially nil.  But these single encounter situations are precisely what the EBG were designed to address.

    A couple of people have made vague statements about "quantifying uncertainty" for individual patients.  This is not the point at all--the uncertainty is there whenever we can't definitively make a diagnosis.  What needs to be quantified (or estimated0 is the ability of a test to address the remaining uncertainty, and the EBG help us to do that.

    I am grateful for the mention that EBG in some situations can provide a "safe harbor" for clinicians by establishing a standard of care.  I think that this is a valid and important point. In medico-legal terms, the "standard of care" is based on a consensus of peers--what most similarly trained doctors would do in a similar situation.  Which means that for EBG to provide a safe-harbor/standard of care we all need to stick to them.

    All the best,

    Mike
      


    Michael A. Bruno, M.D., M.S., F.A.C.R.  
    Professor of Radiology & Medicine

    Vice Chair for Quality & Patient Safety

    Chief, Division of Emergency Radiology

    Penn State Milton S. Hershey Medical Center
    ( (717) 531-8703  |  6 (717) 531-5737

    * mbruno@pennstatehealth.psu.edu  

    1571679014277





  • 14.  RE: Living with uncertainty

    Posted 10-03-2021 12:50
    Mike - Thanks for sparking this discussion on uncertainty.  Your finding that this is what spawns unhelpful testing echo's the exact same conclusion Kassirer offered 30 years ago in his editorial on "Our Stubborn Quest for Certainty" : "...discomfort with uncertainty drives excessive testing."   Its not fear of malpractice.

    As you mentioned, trusting algorithms offers a potential solution to the problem, but many physicians are algorithm averse.  This is nothing new - ER docs famously disregarded the Goldman prediction rules for predicting MI, despite evidence that the algorithm outperformed the average clinician.  What's interesting is that this issue is  now a really hot topic in behavioral psych, reflecting the fact that the algorithms are getting better all the time, and on the human side we've grown very comfortable accepting algorithmic advice from Google searches, from online dating sites, recommendations on the '50 best movies on Netflix now', etc.  Whether clinicians will accept crowd-sourced advice on diagnosis is a related question, as that becomes available.  (See attached papers)

    From the perspective of improving diagnosis, it is very encouraging to see the algorithms improve, and all of the research being done to understand this balance between  algorithm appreciation and aversion.  Finding ways to increase our trust in the advice from algorithms seems like a productive path to follow given our goals to improve both testing and diagnosis.

    Mark

    Mark L Graber MD FACP
    Founder and President-Emeritus, SIDM

    ------------------------------
    Mark Graber
    ------------------------------



  • 15.  RE: Living with uncertainty

    Posted 10-04-2021 08:52
    I love this topic of dealing with uncertainty in medical practice (and in life). 

    The COVID pandemic offered several insights I'd not considered before.
    For example, regarding physician aversion to limiting testing to guidelines and protocols, we found opposite pressure in our large regional organization from the Hospitalists and ER docs, who requested that we build and officially sanction such limits. By doing so, we offered boundaries and some form of protection by (1) creating a standard of care and (2) backing up our docs against family pressure to have us order various drugs and other testing, and (3) similar protection for the pharmacy who only had limited supply or access to certain drugs, requiring rationing.   

    This delta COVID wave has especially opened up more aggressive pressure from patients and families to use inadequately tested therapies, as the affected (unvaccinated) population is already skewed against scientific thinking. 

    Perhaps framing guidelines and protocols in this light can sway providers toward more compliance moving forward.

    David Ryon, MD, CPPS