Thanks Dr Jain.... all Good points :)

the fact that most clinicians in the article were unable to accurately calculate either a PPV or NPV is not surprising... that simple observation combined with the wide spectrum of post test probabilities shown in the graphs (after both a pos test and a neg test) are a sobering summary of the state of dx accuracy in our country.... but these sobering concerns are as much about the nature of the test characteristics of the lab/imaging tests that we use to diagnose disease as much as the clinical reasoning skills of the clinicians

IFFFF the surveyors had instead asked "what is the PPV of a test in the setting of a disease prevalence of 10% , a sens of 90% and a spec of 95%?"

I would hazard a guess that the respondents would ALSO say "high" in that setting as well (and their intuition would be correct as the correct math answer would be 67%) so HOW you ask the question is as important as the knowledge you are trying to assess.... their usage of a very low prevalence of 1 in 1000 leads the intuition of the respondents astray....

To Dr Jains comments... >> I think we all get a little confused by the semantics of the dx process per se; both the frequentist method and the bayesian method use math and deductive reasoning to get from point A (pt presentation) to point B (a working dx / diff dx) and, in point of fact, further observation of the pt over time and their response (or lack there of) to any initial rx then leads to the "final" dx (which can of course be modified based on later observations)

a better way to describe the dx journey in my mind is using the scientific method as an analogous journey, you generate a hypothesis/prelim/working dx based on initial data and observations ... and then you accept or reject that hypothesis based on subsequent data and observations until you reach your final dx . This analogy emphasizes the dynamic nature of the dx journey and that we can NOT have dx error per se until our journey is complete

(of course one could argue that a certain amount of data should lead to a correct dx in a certain time frame and we can debate endlessly about how quick it should take to complete the dx journey ... To be perfectly blunt the AMOUNT of time that it takes to complete the dx journey is REALLY what this whole debate of dx error circles around in my opinion.... (and also what leads to pt harm and litigation)

I have two main comments/thoughts for Dr Jain and for the authors

1: one of the biggest issues with bayesian analysis in my opinion is that we really do NOT know what the "true" pretest prob is for any given pt until we take a thorough and complete history; eg lets take a young woman age 30 yrs who presents to ER with the pleuritic CP and mild dyspnea on exertion

her pretest prob for pulmonary embolism (high on the differential) is modified by her BMI, birth control usage, recent surgery, recent pregnancy, recent travel, recent lifestyle, exposure to ill family, job exposure, etc etc etc so the TRUE pretest probability can only be calculated AFTER a thorough evaluation of the pertinent positives and negatives in her history and THUS, her calculated pretest prob is mostly related to the adequacy and completeness of the history taking!!! (and I'm not even going to touch adequacy of the physical exam...)

Nowhere do the authors talk about this aspect of hx taking and its relevance to accurate calculation of pretest prob :(

In reality, there is NOT one pretest prob for this pt , there are a host of pretest prob that are higher or lower depending on the adequacy of assessment of the presence or absence of the relevant associated features as well as the adequacy of the symptom complex / description of symptoms over time!!!

2: the accuracy of any proposed dx depends on the test characteristics of the dx test used to establish said dx as Dr Jain astutely pointed out. A highly specific test such as CTA for PE or duplex US for DVT will generally lead to a high PPV no matter the pretest prob given its very high specifity (aka "pathognomic"); a less specific test such as portable CXR can never achieve the same clarity of dx.... so we need to be mindful of how we characterize dx accuracy depending of the test characteristics of the individual dx test that we are using to establish any given dx

I dont believe the authors took these factors into adequate consideration

I personally think it would be worthwhile for SIDM to focus on moving the concept of TIME into the dx error arena; ie focus on distinguishing between working/prelim/admitting dx versus final dx (of course this would make it harder to code diagnoses in EMRs but it would be worthwhile in my mind and lead to more accurate assessments of where when how and why dx errors occur

Thank you

Tom Westover MD