Over the past decade or so, interest in evidence-based practice (EBP) has steadily increased in many clinical fields—a movement that has emphasized the importance of providing empirical evidence to support various therapy interventions. To succeed in meeting the goals of EBP, clinicians must rely on more than intuition and clinical experience. In addition, they must be well versed in the methods of research and statistics to accurately evaluate and apply evidence that seems to support a particular intervention. As Guyatt et al. (2002) noted, the ability to critically appraise research literature and apply such findings is an essential skill for scientifically based treatment. Yet, my observations suggest that this concern is often neglected in graduate training programs as well as by many clinical practitioners and researchers. Furthermore, in this EBP era, all clinical professionals, not only clinical researchers but also clinical practitioners, are almost required to have the substantial knowledge of (1) how to measure the strength of clinical evidence accurately, and (2) how to interpret and report the findings. These are the essential components of EBP that will lead to improvement of one’s scientific literacy.
Scientific literacy is fundamental to the understanding of research methodology as well as the statistical assumptions and techniques used for the analysis and interpretation of data. In the absence of such understanding, it will be impossible for professionals to stay abreast of a rapidly flowing and ever-changing stream of information related to the study and treatment of speech, language, and hearing disorders. What is ultimately at stake is the credibility of the field to function as an independent discipline that presumably prides itself on contributing to a fund of knowledge leading to scientific advancements, not only in its own specialty areas but also for its contributions to the arena of the health science specialties at large. In the absence of such credibility, we will practice “unethically” by failing to provide the best possible services for the people we serve.
So, how does a clinician determine whether or not she is making an accurate, reliable, and credible EBP-oriented diagnosis and improves scientific literacy? One effective way is to learn how to evaluate the results of a diagnostic test accurately to find out the presence (or absence) of a particular disorder.
According to Hawkins (2005), EBP consists of the following four major steps: 1. Formulate a clear clinical question from a client’s problem; 2. Search the literature for relevant clinical articles; 3. Evaluate or clinically appraise the evidence for its validity and usefulness; 4. Implement useful findings into clinical practice. So, let us apply Hawkins’s principle to the diagnostic screening test process.
Despite the many applications of diagnostic test findings, the primary objective of any such test is to detect a particular disorder or disease when present. A good diagnostic test normally identifies people who have the particular disorder or disease of interest and excludes people who do not. To accurately measure the outcomes of a new test or a screening test, results obtained from it are generally compared with some other established test(s) viewed as the gold standard in yielding valid results. Even though such tests may not prove to be 100% accurate, they serve as the standard against which the merit of a new test can be judged. A logical question to ask is “If a test judged as the gold standard is doing a good job in accurately diagnosing a particular disorder or disease, why not use it in all cases?” The answer is that the gold standard for diagnosis can be time-consuming, expensive, and more difficult to perform. For this reason, a screening test is often used as an option during initial testing to decide who should be given a more definitive evaluation and who should not. Thus, an audiologist might give an audiometric screening test to decide when a more complete audiometric evaluation might be warranted. There are several major probabilities that constitute a screening test for determining the accuracy of the results. They are, namely, as follows:
- Prevalence of a disorder (denoted by D): P (D+) = Probability that the disorder (or disease) is present, whereas P (D−) = Probability that the disorder is absent.
- Test Results (denoted by T): P (T+) = Probability that the test is positive, whereas P (T−) = Probability that the test is negative.
- True Positive: P (D+ and T+) = Probability that the disorder is present and the test result is positive. People with the disorder are correctly identified as test positive.
- False Positive: P (D− and T+) = Probability that the disorder is absent but the test result shows positive. People without the disorder are falsely labeled as test positive.
- True Negative: P (D− and T−) = Probability that the disorder is absent and the test result is negative. People without the disorder are correctly identified as test negative.
- False Negative: P (D+ and T−) = Probability that the disorder is positive but the test result is negative. People with the disorder are falsely identified as test negative.
- Sensitivity of a test: It is defined as the probability that the test result is positive (T+) given that the disorder actually exists (D+). Symbolically, it is written as:
If a test has high sensitivity, it will have a low false-negative rate, that is, the probability that a subject who tests out as negative but who is actually positive, denoted by P (T− | D+). In such a case, the test result will seldom indicate that the disorder is not present when in fact it is present.
- Specificity of a test: It is defined as the probability that the test result is negative (T−) given that the disorder actually does not exist (D−). Symbolically, this is written as follows:
A test that has high specificity is one that has a low false-positive rate, denoted by P (T+ | D−), meaning that it will seldom predict the presence of a disorder that does not exist.Although test sensitivity and specificity are important preliminary steps in constructing a diagnostic screening test, these indices alone have limited application to actual diagnosis and clinical decision making. More specifically, although these values may be used to estimate the accuracy of a particular diagnostic test, it is the predictive values of a test that actually have practical/clinical values in detecting a disorder or disease. In the case of measures of sensitivity and specificity, in contrast with predictive values, the disorder or disease status is already known. However, as noted previously, what a clinician really wants to obtain is whether or not a disorder or disease actually exists based on the test result of a diagnostic screening test. Only the predictive values allow for forecasting actual clinical outcomes (EBP) based on test results. There are two major components of predictive values of a diagnostic screening test, namely, predictive value positive (PV+), and predictive value negative (PV−). In short, PV+ and PV− can be viewed as a calculus of evidence to further explore the accuracy of a screening test in a more precise manner.
- Predictive Value Positive (PV+): It refers to the probability that a disorder or disease exists when the test result is positive (T+). Symbolically, this is expressed as follows:
- Predictive Value Negative (PV−): It refers to the probability that a disorder or disease does not exist when the test result is negative. Symbolically, this is written as follows:
All probabilities defined above are summarized in Table 1 shown below.
TABLE 1: Probability Estimates of Test Results
Additionally, in communication disorders, clinical practitioners often look at the results of clinical trials they are investigating and are interested in the association (or relationship) between a treatment and an outcome. In some cases, they may find a strong association or, in another case, there may be no significant association. When clinical investigators try to show the degree of association between two events (control versus experimental, treatment A versus treatment B, etc.), they need to know how to measure the strength of association based on what they observed. To answer the question pertaining to measuring the strength of association, we often use such advanced measures as relative risk, absolute/relative risk reduction, and odds ratio.
In summary, the clinical professionals in the field of SLP and audiology have not quite caught up to the level that medical professionals have achieved in terms of EBP statistics education. At the time of my writing, EBP education has become well established as a component of both undergraduate, graduate, and postgraduate medical education. So, why not us? Now is the right time to check our scientific literacy skills and promote better understanding of EBP statistics to a much larger extent, so that all clinical researchers and practitioners in our field are able to interpret the results and make a diagnosis more accurately.
Guyatt, G. H., & Rennie, D. (2002). User’s guide to the medical literature: A manual for evidence-based clinical practice. Chicago, IL: AMA Press.
Hawkins, R. C. (2005). The evidence based medicine approach to diagnostic testing: Practicalities and limitations. The Clinical Biochemist Reviews, 26(2), 7–18.
Satake, E. (2014). Evidence-based statistics for clinical professionals: What really prevents us from moving forward. Keynote presentation at the annual research symposium of LSU-School of ALLIED Health, New Orleans, LA.
Satake, E. (2014). Statistical methods and reasoning for the clinical sciences: Evidence-based approach. San Diego, CA: Plural Publishing.
About the Author
Eiki Satake, PhD, is an associate professor of mathematics and statistics at Emerson College in Boston, Massachusetts. He has conducted several research seminars and short courses on evidence-based statistics at national and international academic conferences. His research interests include Bayesian statistical methods and probabilistic approaches to evidence-based practice. He has also written numerous scholastic articles and instructional textbooks on statistical methods and statistics education. His most recent textbook, Statistical Methods and Reasoning for the Clinical Sciences: Evidence-Based Practice, provides practitioners with the scientific literacy needed to understand statistical methods in order to increase the accuracy of their diagnoses.