Clinical Epidemiology & Evidence-Based Medicine Glossary:
Updated November 02, 2010
- Literature: The written repository of medical knowledge. It includes information sources such as: refereed scientific journals, practice oriented review journals, conference proceedings, trade journals, textbooks, product promotion materials, and internet e-mail communications, all which vary widely in strength of evidence.
- Primary Source: The information source from which evidence-based knowledge is derived. It has as a major component evidence derived directly from fully described (or referenced) formal observation, procedures or experiments performed with valid, scientifically accepted methods. In its strongest form, this material is usually (but not only) a paper in a refereed scientific publication.
- Scientific Refereed Journal: A journal that has a mission of publicizing and storing primary scientific evidence. By convention evidence published in such a journal is subjected to anonymous review by several experts (referees) in the field prior to publication and is published only once. The methods used to acquire the evidence must be described (or a primary reference cited) with sufficient detail that a person knowledgeable in the discipline can critically appraise the study and could replicate it if they so desired. Note that although the review process is the best we have for assuring evidence integrity, a significant proportion of such papers contain serious flaws in methods and interpretation, some which render the study invalid. The presence of these flaws is one of the primary reasons why literature assessment is a critical skill for clinicians. Repetition of the study by another research group, either in whole or in part, may support or refute a previous study. The critical reader looks for these additional studies.
- Scientific Proceedings: A collection of current research reports, usually presented as brief abstracts, from a scientific meeting. These are a much weaker form of a primary source than is a full scientific journal article because the selection of the abstracts, which are of varying quality, is based on a much more cursory review, the reports are usually incomplete, and much of the work is in-progress. As such, these represent a form of "pre-primary" source. Only half of the studies presented as abstracts ever appear in scientific journals, the balance for example not surviving the scientific publication review process or having findings that are reversed after collecting more data. The problem for the user is to decide which half.
- Integrative Source (Studies): A source reporting the results of meta-analysis, which is a statistical procedure to mathematically combine the results from a number of valid studies to arrive at a stronger conclusion. An exhaustive search for all of the studies relevant to the question at hand and a critical analysis of these studies to exclude those with serious design or procedural flaws is required. Integrative studies are based on objective quantitative analysis rather than the more subjective analysis of the conventional critical review. In human medicine, these reviews are being performed on the primary sources for selected areas of clinical care and are being compiled in Cochrane Collaboration databases to provide a stronger basis for the practice of evidence-based medicine.
- Secondary Source: An information source that does not have as a major component the description of formal observations or experiments but rather is synthesized from some combination of primary sources, experience, or authoritative belief (dogma). The primary literature used may have been selected in a biased or incomplete fashion and may have been used without comprehensive critical appraisal to establish the relative strength of evidence in each source. Examples of secondary sources are review articles, journals specializing in practitioner-oriented reviews, most practitioner-oriented conference proceedings, trade publications, most e-mail conversations, and authorities presenting information without supporting evidence in whatever format (lectures, CE meetings, e-mail forums).
- Tertiary Source: A compilation of information for application across a broad spectrum, typically represented by class notes and textbooks intended for use in core courses. The information is often presented in a dogmatic, authoritative fashion as a sequence of facts and interpretations of their meaning that the reader is to believe without reservation or evaluation. The strength of the underlying evidence is not indicated and any current controversy between researchers in the area is not addressed. The bibliography is usually predominately secondary literature and is usually intended to provide the interested reader with entry points to the underlying primary literature. Much of the evidence-based information contained in textbooks is filtered sufficiently that it is accepted by most all of the experts in the field, much of it is unlikely to change in the future, and most of the changes will be minor. However, depending on the field, textbooks contain a varying amount of dogma and interpretations of facts that will change with the progress of research in the area, sometimes significantly. Because clinical experience is often not examined critically, clinical textbooks tend to contain a larger proportion of dogma than do basic science textbooks. Class notes usually contain much more limited information than do textbooks and do not undergo the auditing process as part of publication as textbooks do.
- Derivative Service: A service that presents collections of abstracts, usually from a wide selection of primary literature, selected to meet the interests of a particular group of clinicians. Some, such as Medline, reproduce a copy of the abstract as written by the authors of the primary publication. These are often truncated if longer than some maximum limit, such as 250 words. Others, such as CAB Vet-CD, contain abstracts written by the derivative service rather than the authors of the original publication. Abstracts vary in quality and abstracters may interpret the evidence of the paper differently than intended by the original authors.
- Structured Abstract: An abstract format, adopted by many clinical journals, that generally contains the key components of study population and source, study question, study design, main findings, and clinical implication in a dense outline format. This format was developed because unstructured abstracts are of varying quality, often omitting key components, and many clinicians, particularly those using the derivative sources, read only the abstracts because of time constraints.
- Scientific Research: The systematic, controlled, empirical and critical investigation of hypothetical propositions about the presumed relations among natural phenomena (Kerlinger, 1964). The objective inquiry into natural phenomena using currently accepted investigation procedures, the immediate product of which is evidence, with the objective of discovering how that aspect of the physical world works. It is an empirical, conceptual system of learning about the physical world that organizes publicly observable facts and reasoning within a structure of theories and inferences. The methods of inquiry are constructed to minimize the effects of natural human biases in observation and interpretation. By convention, the evidence, the procedures used to acquire it, and subsequent interpretation is subjected to peer evaluation as a prelude to publication in the primary literature where it is publicly available for further scrutiny and use.
- Scientific Knowledge: The current set of peer-evaluated consensus models about how natural phenomena work, which often differ between groups of researchers at the research frontier. These models are established by evidence obtained from critical scientific inquiry that has been subjected to peer evaluation and replication. All scientific knowledge contains varying degrees of uncertainty and is constantly at varying risk of being modified or discarded as the result of evidence from further inquiry. Models are disproved by multiple findings of discrepant evidence, which is often the result of improvements in measurement technology. Discrepant findings are weakened by failure of legitimate independent replication and are strengthened by their success. Many repeated studies of the same design and execution that result in the same evidence do not significantly increase the likelihood that the model of the phenomena is correct. Similarly, a belief is not strengthened (converted to knowledge) by the weight of the number of people who hold it.
- Scientific Paradigm: The model shared by most but not all members of a scientific community, designed to describe and interpret observed or inferred phenomena, past or present, and aimed at building a testable body of knowledge open to rejection or confirmation. (Shermer, 1977)
- Scientific Progress: The cumulative growth of a system of knowledge over time, in which useful features are retained and nonuseful features are abandoned, based on the rejection or confirmation of testable knowledge. (Shermer, 1997) Important components of this progress are the evolution of the experimental and observation methods and the public presentation of findings in the refereed primary scientific literature for expansion, support (corroboration) or refutation (rejection) of previous findings and interpretations.
- Belief: The mental act or state of mind of an individual after they accept and internalize an external concept or idea, which then becomes part of further thought processes, often unrecognized, on related issues. Internalized deeply, belief becomes part of intuition. Belief can occur after deliberate, systematic, critical thinking or can occur with immediate, non-reasoned, uncritical acceptance. Once a belief is accepted that is in error, accepting a more correct belief becomes considerably more difficult than if no previous belief were held. The nature of human thinking is to weigh data that is consistent with the mistaken belief heavier and to ignore or discount discordant data, and to limit the search for additional data to that which has the potential of confirming rather than refuting a belief (e.g. selective necropsy to confirm a gross diagnosis). Prior belief biases subjective observation, such as occurs during the diagnostic process or during non-blinded measurement, because it subtly changes perception, particularly of vague or ambiguous characteristics. This bias occurs unbeknownst to the observer and despite their best intentions.
- Dogma: Those beliefs held as established or put forth as an authoritative or expert opinion, often contained in a secondary or tertiary source, but that have little or no supportive empirical evidence from primary sources. Medical dogma is usually derived from unevaluated biological hypotheses and uncritical observation or experience without recognition of the effects of chance, natural biological variation, and observer bias. An unknown but significant portion of medical practice falls into this category. Repetition across secondary and tertiary sources or the number of people, whatever their qualifications, that hold this belief does not change the status of such information. Most information conveyed in the instructional setting is given in the form of dogma without the associated information necessary to judge its credibility.
- Evidence: That which tends to support something or show that something is the case. Depending on how it was obtained, evidence varies greatly in strength. Note that a set of evidence can be correct but the underlying theory that the promoters allege the evidence supports can still be wrong.
- Empirical Evidence (Facts): Knowledge obtained by looking rather than reasoning or feeling. In the scientific sense, that knowledge comprised of the objective findings (but not broad interpretation) derived from analysis of objective data obtained from formal observational or experimental procedures that are potentially repeatable (verifiable) and that meet currently accepted standards of design, execution, and analysis. The strongest empirical evidence is obtained from rigorous methods incorporated into an experiment designed to have a clear, unequivocal supporting or refuting outcome. Empirical evidence is weakened by the opportunity for other explanations, due to weakness in methods, to account for the findings. As the opportunity for independent verification and for assessment of strength of evidence is a key component, the methods used to acquire the evidence must be described or referenced sufficiently that this verification and assessment can be done by independent investigators. As presented even in the refereed primary scientific journals, this evidence must be critically appraised by the reader because, depending on the methods used, it varies from strong and useful to weak, wrong, or irrelevant. Note that a set of evidence can be correct (e.g., the sun "rises" regularly) but the underlying theory that the promoters allege the evidence supports is wrong (e.g., the sun moves around the earth).
- Analogical Evidence: Evidence based on reasoning by analogy, which is concluding from comparing known similarities between two systems that a relationship shown to exist in one system but unknown in the other also exists in the other. For example, if drug X has been shown to be effective against disease Y in a species Z then perhaps the same relationship exists between similar drug or similar disease or similar species. Evidence based on analogical reasoning is common in medicine, as it is a necessary basis for action when empirical evidence is lacking. Detailed mechanisms of action for particular processes are often established in laboratory species (rodents) and extrapolated to other species in which direct investigation is impractical. However, analogical evidence is susceptible to unavoidable error because of the likelihood that different and unknown factors are operating in the two systems, which weaken the analogy. Because it is inherently a weaker form of evidence than is empirical evidence, it is likely the source of much unexamined dogma and is better used as a basis for generating hypotheses that are then empirically evaluated.
- Anecdotal Evidence (Case report): The description of the occurrence of single unique event, such as a miraculous medical recovery. Even if the occurrence of the event itself is without doubt, the reason that it occurred is often promoted as being due to an unusual therapy applied to the case and thus validating the theory that selection of the therapy was based upon. The probability of apparently unusual events is often considerably higher than we expect by intuition and other unrecognized factors (confounders) may have invalidated the initial prediction of demise, thus making the event not that unusual. As an anecdote is extremely weak evidence in support of a theory, an accumulation of similar anecdotes does not significantly increase support and at best may serve as a justification for a scientific experiment to empirically test the theory.
- Scientific Method: The scientific method is the conceptual process of organizing empirical facts and their inter-relationships in a structure of theories and inferences. It is the philosophical ideal of how scientists advance scientific knowledge by methodically and systematically applying procedures that reduce the likelihood of alternative explanations for their observations. The underlying principles are skepticism (an attitude of doubt toward and suspended judgment of statements, even when made by great authorities, prior to analyzing the underlying evidence and assumptions), determinism (the principle that all natural phenomena are caused previous events linked by fundamental physical laws that are the same everywhere in the universe) and empiricism (the practice of relying on observation and experiment for developing an understanding (theory) of natural phenomena).
- Hypothetico-deductive Model: The logical basis of the classical scientific method. It is comprised of the rigorous logical deduction of a precise prediction (causal hypothesis) based on the current scientific theory followed by the design and execution of a crucial study that will have one of two unequivocal outcomes to test this causal hypothesis, and finally the logical induction that the theory is supported or falsified based on the findings from this study. Falsification is usually regarded as logically more definitive than is support. Although it is a useful model of scientific logic, the "scientific method" is only an idealized concept of the actual process by which scientific knowledge advances.
- Deduction: Reasoning from the general theory to predict the specific circumstance (general to specific).
- Induction: Reasoning from the results of the specific circumstance to conclusions about the theory (specific to general).
- Scientific Model: A concept of the way a phenomenon in nature works, usually simplified from a theory to be useful for the purposes at hand. The usefulness of a model often depends on the context in which it is used. For example, the model of mass movement from Newtonian physics is adequate to predict what will happen when a car runs into a brick wall. The model of motion based quantum physics is more correct (in the sense that it covers more phenomena) but is not more useful and is more cumbersome in the scale of interest to the driver of car about to hit the wall. "All models are wrong; some are just more useful than others." (JP Box).
- Scientific Theory: The coherent, interrelated structure of scientific propositions and principles derived over time from scientific evidence that explains a class of observed phenomena or facts. Theories enable us to make sense of what we see and to make predictions. A scientific theory must have predictive power (predict phenomena that can be observed) and must be testable and falsifiable (the theory is false if the predicted phenomena are not observed in the appropriate experiment). A scientific theory must be consistent both internally and with broader, more fundamental theories related to other aspects of the phenomena. For example, a theory of disease treatment cannot be inconsistent with most of the more fundamental theories of chemistry. Although a theory that is consistent with a diversity of evidence generally is stronger (more certain) than one that is not, a theory cannot be proven as an absolute certainty (absolute truth) and is always at some risk of modification if not replacement. To replace a previously held theory, the new theory must explain those phenomena explained by the previous theory as well as those it did not adequately explain. Note that in non-scientific contexts the word theory is often used to mean a mere hypothesis or speculation, a much less reliable proposition than is a scientific theory.
- Pseudo-scientific Theory: Theories that are claimed to be scientific but are not. Hallmarks of pseudoscientific theories are: 1) Incompatible with broader, more fundamental scientific theories that are supported by overwhelming evidence, 2) Violation of the testability / falsifiability requirement; that is an experiment cannot be designed or an observation made that the results of which would refute the theory.
- Embedded Theory: A theory whose scientific propositions have been repeatedly tested and supported to the point that it becomes a widely accepted explanation for the phenomena involved.
- Scientific Proof: In the strict sense scientific proof only occurs in the disciplines of pure mathematics and logic, where it means the sequence of statements establishing the validity of a concluding statement in accordance with mathematical or logical principles. Scientific theories outside of these disciplines cannot be "proven". Scientific theories are supported by evidence, the amount of which can be such that it is unreasonable to withhold acceptance of the theory. In a non-scientific sense, some would consider such theories as "proven". Hallmarks of pseudoscience are claims about the presence or absence of "scientific proof".
- Scientific Truth: The simplest explanation that explains all currently known,
indisputable facts (Ockam's Razor). Note that scientific truth is not absolute truth in
the sense that scientific truth can change as more facts become known. In the scientific
process, the only absolutes are empirical data and the logical consistency
applied to it.
- Ockham's Razor (Occam, principle of parsimony, simplicity or economy): "pluralitas non est ponenda sine necessitate"; "Entities should not be multiplied unnecessarily" or "Multiplicity ought not be posited without necessity". The principle that if two theories explain a phenomenon equally, the simpler theory requiring fewer assumptions and explanatory principles is preferred and that generalizations should be based on observed facts and not on other generalizations. One hallmark of pseudoscience is the requirement for many assumptions and untestable explanatory principles to support the core theories, which proponents often change when an assumption or principle is critically refuted. Under this principle, the theory containing testable components is preferred over the theory containing an inherently untestable component. William of Ockham, 1285-1349.
- Scientific Hypothesis: A provisional or temporary conjecture about something’s
function or status that is based on limited or no preliminary evidence and that evidence
from further investigation will either support or refute. In this sense, a list of
differential diagnoses represents a set of hypotheses for which more evidence must be
obtained in the diagnostic process before a level of certainty is reached that is
sufficient for action. A logical problem is that although the facts derived from the test
of a hypothesis are correct, the broader underlying theory from which the hypothesis was
derived can still be wrong. Proponents of any hypothesis can defend it against contrary
evidence by ad hoc modifications of the underlying theory. Science often progresses by
researchers noting discrepant observations and pursuing their explanation with
investigations based on further hypotheses. Note that in non-scientific contexts what most
people call a "theory" is actually a hypothesis or speculation.
- Formal Hypothesis: A testable "if . . is related to . . then . ." statement of the following form. The tentative concept, the proposed relationship between two variables or factors, is contained in the "if" component and the predicted outcome of a test of this hypothesis, such the experimental manipulation of one of those variables or the observation of a natural experiment, is contained in the "then" component. Performing the test leads to a logical conclusion about the proposed relationship.
- Scientific Experiment: The execution of procedures or methods to test a hypothesis in a fashion that eliminates all reasonable basis for doubt as to the interpretation of the results as either supporting or refuting the hypothesis. In the strictest sense, the investigator controls all extraneous facts and manipulates only one or a few independent variables to determine the effect on a dependent variable. In observational experiments, the investigator cannot directly control manipulate variables but observes "experiments of nature". The description of methods used must be publicly available for rigorous scrutiny by the scientific community and must be sufficient to enable independent repetition of the experiment. The process of medical diagnosis and treatment of the individual patient mimics the hypothetico-deductive model of science in the sense that the selection of the treatment is based on a hypothesis about the nature of the illness, the application of the treatment is the execution of an experiment, and the patient’s response to the treatment is the experimental outcome. However, this process does not eliminate all reasonable basis for doubt about whether or not the outcome supports or refutes the hypothesis and thus the underlying theory upon which the hypothesis is based because many other factors (e.g., diagnostic error, biological variability, self-repair) could account for the outcome. Thus, by itself an individual case is more of an anecdote. Note that what most students experience in school laboratories are "replications" (demonstrations) rather than experiments as the outcome is known prior to the execution of the replication and if that result is not obtained, the replication was executed incorrectly. Experiments are based on a testable hypothesis and have several potential outcomes.
- Pseudoscience (Quackery): The promotion of false or unsubstantiated procedures and remedies, often with broad, non-specific claims covering a wide range of diseases, often through anecdotes, case histories and testimonials, often with complicated pseudomedical or pseudoscientific jargon of the author’s creation, sometimes by authors with impressive-looking degrees and titles (Dr., PhD, professor) intended to convey credibility. The clearest indication of pseudoscience is the lack of strong, diverse, public supporting evidence in the mainstream scientific literature. The results have not been independently verified through the use of objective scientific methods and have not been subjected to the scrutiny of the peer review process for publication in primary scientific journals. Often, any cited references are books or secondary sources, with few if any from the mainstream refereed scientific literature of recent publication. Often, the promoters claim that they or anyone friendly to their cause who attempts to verify their claims are persecuted by organized science or medicine because the institution does not want competition and their results are being suppressed because of their controversial nature or because of their alleged financial impact. The underlying hypotheses are often at serious odds with well-established fundamental theories of related disciplines as well as conventional medical practice. Promoters of pseudoscience often dismiss these conventional scientific theories as invalid on the grounds that these theories cannot be absolutely proven, which is true. Promoters of pseudoscience decry the lack of interest by those engaged in conventional science, fail to provide substantial public evidence through conventional publication mechanisms outside of their control that would attract interest from more mainstream scientists, do not understand the rigorous process for developing this evidence, and instead often resort to ad hominem attacks on their detractors. With radical claims, the burden of generating supporting evidence is on those making the extraordinary claims rather than the burden of denial being on more mainstream science. The more extreme the claim, the stronger the supporting evidence must be.
- Heretic: One who holds a markedly dissenting view compared to more widely held currently accepted beliefs.
- Endoheretic: In the scientific sense, an individual trained and working within a discipline but who holds hypotheses that are markedly inconsistent with major components of the theories in a discipline. Endoheretics follow the conventional process of publishing their evidence in the refereed scientific journals of their discipline. Examples are Galileo, Lister, an anesthesiologist who proposed the need for antisepsis before the germ theory of disease was known, Wegner, a geologist who proposed the theory of continental drift, and Prusner, who proposed that prions were responsible for transmissible encephalopathies. Issac Asimov estimated that about 1 in 50 of these hypotheses put forward by endoheretics are upheld and, when they are, a major paradigm shift occurs in that discipline. This shift often occurs slowly as a large amount of strong empirical evidence obtained through different means by different investigators is required to cause a marked shift in a discipline.
- Exoheretic: In the scientific sense, an individual who holds hypotheses that are markedly inconsistent with the theories of a discipline and often with more fundamental theories, often is not trained in that discipline, often works in isolation and does not submit their evidence to public scrutiny in the conventional scientific publishing process. They often hold a high opinion of themselves and a low opinion of scientists in the discipline. They often claim to be persecuted by government agencies and to be discriminated against by the conventional scientific establishment. They often claim that this occurs because their findings will completely revolutionize a discipline and have an adverse economic impact on its conventional members. Examples are Velikovsky, von Daniken, and many promoters of medical quackery. Pseudo-scientists usually fall into this category.