Which Is True of Meta-analysis and Systematic Review?
Systematic Review and Meta-analysis: When 1 Study Is Just non Enough
We live in the information age, and the practice of medicine is condign increasingly specialized. In the biomedical literature, the number of published studies has dramatically increased: There are now more than 15 1000000 citations in MEDLINE, with ten,000 to 20,000 new citations added each week (1). Multiple relevant studies ordinarily guide most clinical decisions. These studies often vary in their design; methodologic quality; population studied; and the intervention, test, or status considered. Because even highly cited trials may be challenged or refuted over fourth dimension (2), clinical decision-making requires ongoing reconciliation of studies that provide unlike answers to the same question. Both clinicians and researchers can also benefit from a summary of where uncertainty remains. Because it is oft impractical for readers to track down and review all of the primary studies (3), review articles are an of import source of summarized evidence on a particular topic (four).
Narrative Review, Systematic Review, and Meta-analysis
Review articles have traditionally taken the grade of a narrative review, whereby a content expert writes most a particular field, status, or treatment (5–7). Narrative reviews take many benefits, including a broad overview of relevant information tempered by years of practical noesis from an experienced author. Indeed, this article itself is in a narrative format, from authors who have published a number of meta-analyses in previous years.
In some circumstances, a reader wants to become very knowledgeable about specific details of a topic and wants some balls that the information presented is both comprehensive and unbiased. A narrative review typically uses an implicit process to compile prove to support the statements beingness fabricated. The reader often cannot tell which recommendations were based on the author's clinical experience, the breadth to which available literature was identified and compiled, and the reasons that some studies were given more emphasis than others. It is sometimes uncertain whether the author of a narrative review selectively cited reports that reinforced his or her preconceived ideas or promoted specific views of a topic. As well, a quantitative summary of the literature is often absent-minded in a narrative review.
A systematic review uses a process to place comprehensively all studies for a specific focused question (fatigued from inquiry and other sources), appraise the methods of the studies, summarize the results, nowadays key findings, identify reasons for different results across studies, and cite limitations of current knowledge (viii,9). In a systematic review, all decisions used to compile data are meant to be explicit, allowing the reader to gauge for him- or herself the quality of the review process and the potential for bias. In this fashion, systematic reviews tend to be more transparent than their narrative cousins, although they also can be biased if the pick or emphasis of certain primary studies is influenced past the preconceived notions of the authors or funding sources (10).
Depending on the nature of the data, the results of a systematic review can be summarized in text or graphic form. In graphic form, information technology is common for dissimilar trials to be depicted in a plot where the point guess and 95% conviction interval for each report are presented on an individual line (xi). When results are mathematically combined (a procedure sometimes referred to every bit pooling), this is referred to as meta-analysis. Graphically, the pooled result is often presented as a diamond at the bottom of the plot.
When performing a meta-analysis, a review team usually combines aggregate-level data reported in each principal written report (point and variance estimate of the summary measure). On occasion, a review team will obtain all of the individual patient data from each of the principal studies (12,thirteen). Although challenging to conduct (14), individual patient meta-analyses may have certain advantages over aggregate-level analyses. As highlighted in a review of angiotensin-converting enzyme (ACE) inhibitors for nondiabetic kidney disease, this includes the utilize of common definitions, coding and cutoff points betwixt studies, addressing questions not examined in the original publication, and a amend sense of the impact of private patient (versus report level) characteristics (12,15).
As first highlighted a decade ago (16), the number of systematic reviews in nephrology and other fields has increased dramatically with time, paralleling the rapid growth of biomedical literature during the past half century. Initiatives such as the Cochrane Collaboration accept further increased the profile and rigor of the systematic review process (details of the structured procedure of Cochrane systematic reviews are available through their Web site) (17,18). From 1990 to 2005, there were more than 400 systematic reviews and meta-analyses published in the bailiwick of nephrology (Effigy ane). Of these reviews, 40% pertained to chronic kidney disease or glomerulonephritis and 20, sixteen, 15, and 7% pertained to kidney transplantation, dialysis, acute kidney injury, and pediatric nephrology, respectively. As a publication blazon, notwithstanding, systematic reviews take non been without controversy: Some authors consider a meta-analysis the best possible apply of all bachelor data, whereas others question whether they add anything meaningful to scientific noesis (19). The strengths and weaknesses of this publication type are described next.
- Download figure
- Open up in new tab
- Download powerpoint
Effigy 1.
There have been more than than 400 systematic reviews and meta-analyses published in the subject field of nephrology since 1990, with the almanac number increasing with time. Frequencies were estimated from a MEDLINE and EMBASE search performed past an experienced renal librarian in December 2006. Citations were reviewed by a nephrologist for relevance. Duplicate publications from the same group of authors were counted only in one case.
Strengths of Systematic Review and Meta-analysis
Physicians make better clinical decisions when they understand the circumstances and preferences of their patients and combine their personal experience with clinical evidence underlying the available options (20). The public also expects that their physicians will integrate research findings into practice in a timely style (21). Thus, sound clinical or health policy decisions are facilitated by reviewing the available evidence (and its limitations), understanding reasons why some studies differ in their results (a finding sometimes referred to as heterogeneity among the primary studies), coming up with an assessment of the expected effect of an intervention or exposure (for questions of therapy or etiology), and and so integrating the new information with other relevant treatment, patient, and health care organization factors.
In this respect, reading a properly conducted systematic review is an efficient style to go familiar with the best bachelor research evidence for a focused clinical question. The review team may as well accept obtained information from the primary authors which was non available in the original reports. The presented summary allows the reader to have into account a whole range of relevant findings from research on a detail topic. The procedure can too plant whether the scientific findings are consistent and generalizable across populations, settings, and treatment variations and whether findings vary significantly by item subgroups. Again, the potential forcefulness of a systematic review lies in the transparency of each phase of the synthesis process, allowing the reader to focus on the merits of each determination fabricated in compiling the information, rather than a simple contrast of one study to another every bit sometimes occurs in other types of reviews.
For example, studies demonstrating a significant effect of treatment are more likely to exist published than studies with negative findings, are more than likely to exist published in English language, and more likely to be cited past others (22–27). A well-conducted systematic review attempts to reduce the possibility of bias in the method of identifying and selecting studies for review, by using a comprehensive search strategy and specifying inclusion criteria that ideally accept not been influenced by a priori noesis of the main studies.
Mathematically combining data from a series of well-conducted master studies may provide a more than precise estimate of the underlying "true effect" than any individual report (28). In other words, by combining the samples of the private studies, the size of the "overall sample" is increased, enhancing the statistical ability of the analysis and reducing the size of the confidence interval for the point estimate of the effect. Information technology is as well more efficient to communicate a pooled summary than to depict the results for each of the private studies. Sometimes, if the treatment issue in small-scale trials shows a nonsignificant trend toward efficacy, then pooling the results may found the benefits of therapy (16). For instance, x trials examined whether ACE inhibitors were more effective than other antihypertensive agents for the prevention of nondiabetic kidney failure (29). Many of the 95% conviction intervals for the estimate provided by each study overlapped with a finding of no effect; however, the overall pooled judge established a benefit of ACE inhibitors.
For these reasons, a meta-analysis of like, well-conducted, randomized, controlled trials has been considered one of the highest levels of bear witness (30–32). It is important to stress that the primary trials all take to be conducted with high methodologic rigor for the meta-analysis to be definitive. Alternatively, when the existing studies have of import scientific and methodologic limitations, including smaller sized samples (which is more often the case), the systematic review may identify where gaps be in the available literature. In this case, an exploratory meta-analysis can provide a plausible approximate of effect that can be tested in subsequent studies (33,34).
Limitations of Systematic Review and Meta-assay
This blazon of publication type has many potential limitations that should be appreciated by all readers. First, the summary provided in a systematic review and meta-analysis of the literature is only as reliable every bit the methods used to gauge the effect in each of the primary studies. In other words, conducting a meta-analysis does not overcome issues that were inherent in the pattern and execution of the primary studies. Information technology also does not correct biases as a issue of selective publication, whereby studies that report dramatic effects are more than probable to exist identified, summarized, and later on pooled in meta-analysis than studies that written report smaller issue sizes (an outcome referred to as publication bias). Because more than three quarters of meta-analyses did non report any empirical assessment of publication bias (35), the true frequency of this form of bias is unknown.
Controversies also ascend effectually the interpretation of summarized results, particularly when the results of discordant studies are pooled in meta-analysis (36). The review procedure inevitably identifies studies that are diverse in their pattern, methodologic quality, specific interventions used, and types of patients studied. There is often some subjectivity when deciding how similar studies must be earlier pooling is appropriate. Combining studies of poor quality with those that were more than rigorously conducted may non be useful and can lead to worse estimates of the underlying truth or a false sense of precision around the truth (36). A false sense of precision may likewise ascend when various subgroups of patients defined by characteristics such as their historic period or gender differ in their observed response. In such cases, reporting an amass pooled effect might exist misleading if there are important reasons to explain variable treatment furnishings across different types of patients (36–xl).
Finally, merely labeling a manuscript as a "systematic review" or "meta-analysis" does not guarantee that the review was conducted or reported with due rigor (41). To reduce the take chances of arriving at misleading conclusions, guidelines on the behave and reporting of systematic reviews were recently published (42,43); however, important methodologic flaws of systematic reviews published in peer-reviewed journals have been well described (44–54). For instance, of the 86 renal systematic reviews published in 2005, the majority (58%) had of import methodologic flaws (Mrkobrada M, Thiessen-Philbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). The most common flaws amidst these renal reviews were failure to assess the methodologic quality of included primary studies and failure to avoid bias in study inclusion (Mrkobrada 1000, Thiessen-Philbrook H, Haynes RB, Iansavichus AV, Rehman F, and Garg AX, submitted). In some cases, manufacture-supported reviews of drugs accept had fewer reservations about methodologic limitations of the included trials than rigorously conducted Cochrane reviews on the same topic (x); all the same, the hypothesis that less rigorous reviews more often report positive conclusions than good-quality reviews of the aforementioned topic has not been borne out in empirical assessment (48,53,55). Withal, like all good consumers, users of systematic reviews should carefully consider the quality of the product and attach to the dictum "caveat emptor": Permit the heir-apparent beware. The limitations described in this section may explicate differences in the results of meta-analyses as compared with subsequent big, randomized, controlled trials, which have occurred in approximately ane third of cases (56).
How to Appraise Critically a Systematic Review and Meta-analysis
Users of systematic reviews demand to assure themselves that the underlying methods used to get together relevant information were audio. Before considering the results or how the information could be appropriately applied in patient care (9), there are a few questions that the reader tin ask him- or herself when assessing the methodologic quality of a systematic review (Table 1).
Tabular array ane.
Questions to ask when assessing the quality of a systematic reviewa
Was the Review Conducted Co-ordinate to a Prespecified Protocol?
It is reassuring if a review was guided by a written protocol (prepared in advance) that describes the research question(s), hypotheses, review method, and program for how the data will be extracted and compiled. Such an approach minimizes the likelihood that the results or the expectations of the reviewing squad influenced study inclusion or synthesis. Although almost systematic reviews are conducted in a retrospective manner, reviews and meta-analyses tin can in theory be defined at the time several similar trials are beingness planned or under fashion. This allows a set of specific hypotheses, data collection procedures, and analytic strategies to be specified in advance earlier any of the results from the primary studies are known. Such a prospective attempt may provide more than reliable answers to medically relevant questions than the traditional retrospective approach (41).
Was the Question Focused?
Clinical questions often deal with problems of treatment, etiology, prognosis, and diagnosis. A well-formulated question usually specifies the patient'southward problem or diagnosis, the intervention or exposure of interest, whatsoever comparison group (if relevant), and the chief and secondary outcomes of involvement (57).
Were the "Right" Types of Studies Eligible for the Review?
Different study designs tin be used to answer different clinical questions. Randomized, controlled trials; observational studies; and cantankerous-sectional diagnostic studies may each be appropriate depending on the primary question posed in the review. When examining the eligible criteria for study inclusion, the reader should feel confident that a potential bias in the selection of studies was avoided. Specifically, the reader should inquire her- or himself whether the eligibility criteria for study inclusion were appropriate for the question asked. Whether the right types of studies were selected for the review besides depends on the depth and breadth of the underlying literature search.
For example, some review teams will consider only studies that were published in English. There is evidence that journals from sure countries publish a higher proportion of positive trials than others (58). Excluding non-English studies seemed to change the results of some reviews (59,60) but not others (61,62).
Some review teams utilize broad criteria for their inclusion of primary studies (e.yard., effects of agents that block the renin-angiotensin system on renal outcomes [63]), whereas other teams utilize more narrow inclusion criteria (east.g., restricting the analysis only to patients who take diabetes without evidence of nephropathy [64]). At that place is oftentimes no unmarried correct approach; however, the conclusions of any meta-assay that is highly sensitive to altering the entry criteria of included studies should be interpreted with some caution (25). For instance, ii different review teams considered whether synthetic dialysis membranes resulted in better clinical outcomes compared with cellulose-based membranes in patients with acute renal failure. In one meta-assay (65) but not the other (66), synthetic membranes reduced the gamble for death. The discordant results were due to the inclusion of a study that did non encounter eligibility for the second review (67).
Was the Method of Identifying All Relevant Information Comprehensive?
Identifying relevant studies for a given clinical question amidst the many potential sources of data is commonly a laborious process (68). Biomedical journals are the nearly common source of information, and bibliographic databases are often used to search for relevant articles. MEDLINE currently indexes approximately 4800 medical journals and contains 13 million citations (69). Similarly, EMBASE indexes approximately 5000 medical journals and contains more than than 11 one thousand thousand records. In that location are some key differences between EMBASE and MEDLINE, and the review team should take searched both databases (lxx–72). For example, EMBASE provides the all-time coverage of European research likewise as pharmaceutical inquiry including renal adverse events (73). Positive studies may exist more often published in journals that are indexed in MEDLINE, compared with nonindexed journals (25).
Depending on the question posed, other databases may also have been searched. For example, if a team is summarizing the effects of exercise grooming in patients who receive maintenance hemodialysis, then searching the Cumulative Index to Nursing and Centrolineal Health Literature (CINAHL) database would be advisable (74). Alternatively, the ECONOLIT database may exist useful for identifying information on the out-of-pocket expenses incurred by living kidney donors (75). As a supplementary method of identifying information, searching databases such as the Science Citation Alphabetize (which identifies all articles that cite a relevant commodity), equally well equally newer Net search engines such as Google Scholar and Elsevier's Scirus, tin can be useful for identifying articles that are not indexed well in traditional bibliographic databases (76). Searching bibliographies of retrieved articles can as well identify relevant articles that were missed.
Any bibliographic database was used, the review team should have used a search strategy that maximized the identification of relevant manufactures (77,78). Considering there is some subjectivity in screening databases, citations should be reviewed independently and in indistinguishable by two members of the reviewing team, with the full-text commodity retrieved for whatever commendation accounted relevant past any of the reviewers. There is also some subjectivity in assessing the eligibility of each full-text commodity, and the risk for incorrectly discarding relevant reports is reduced when two reviewers independently perform each assessment in a reliable manner (79).
Important sources of data other than journal articles should not be overlooked. Conference proceedings, abstracts, books, and manufacturers all can exist sources of potentially valuable information. Inquiries to experts, including those listed in trial registries, may have likewise proved useful (28).
A comprehensive search of bachelor literature reduces the possibility of publication bias, which occurs when studies with statistically meaning results are more likely to be published and cited (80,81). It is interesting that some recent reviews of acetylcysteine for the prevention of contrast nephropathy analyzed equally few as five studies, despite beingness submitted for publication virtually 1 year later publication of a review of 12 studies (82). Although there are many potential reasons for this, ane cannot exclude the possibility that some search strategies missed eligible trials. In improver to a comprehensive search method, which makes it unlikely that relevant studies were missed, information technology is oftentimes reassuring if the review team used graphic and statistical methods to ostend that there was fiddling chance that publication bias influenced the results (83).
Was the Data Brainchild from Each Study Appropriate?
In compiling relevant information, the review squad should have used a rigorous and reproducible method of abstracting all relevant data from the primary studies. Often two reviewers abstract key information from each chief study, including study and patient characteristics, setting, and details about the intervention, exposure, or diagnostic test as is appropriate. Linguistic communication translators may be needed. Teams who acquit their review with due rigor will indicate that they contacted the main authors from each of the primary studies to confirm the accuracy of bathetic data too as to provide additional relevant data that was non provided in the chief study. Some authors will go through the additional effort of blinding or masking the results from other study characteristics so that data abstraction is as objective as possible (84,85).
Ane element that should have been abstracted is the methodologic quality of each primary report (recognizing this is not always as straightforward as information technology may first seem) (86–91). The question to be posed by the reader is whether the reviewing team considered if each of the chief studies was designed, conducted, and analyzed in a way to minimize or avoid biases in the results (92). For randomized, controlled trials, lack of concealment of allotment, inadequate generation of the allocation sequence, and lack of double blinding can exaggerate estimates of the treatment effect (54,xc,93). The value of abstracting such data is that it may help to explain important differences in the results among the primary studies (90).
For instance, long-term risk estimates can go unreliable when participants are lost to study follow-up; those who participate in follow-up often systematically differ from nonparticipants. For this reason, prognosis studies are vulnerable to bias, unless the loss to follow-up is less than 20% (94). In a systematic review of 49 studies on the renal prognosis of diarrhea associated hemolytic uremic syndrome, on average, 21% of patients were lost to follow-up (range 0 to 59% beyond studies) (95). It was hypothesized that patients who were lost to follow-up would contribute to worse estimates of long-term prognosis because they are typically healthier than those who keep to be followed by their nephrologists. Indeed, studies with a higher proportion of patients lost to follow-upwards demonstrated a higher proportion of patients with long-term renal sequelae, explaining 28% of the between-study variability.
How Was the Information Synthesized and Summarized?
In cases in which the primary studies differ in the design, populations studied, interventions and comparisons used, or outcomes measured, it may have been appropriate for the review team simply to report the results descriptively using text and tables. When the chief studies are similar in these characteristics and the studies provide a like estimate of a truthful effect, then meta-assay may have been used to derive a more precise approximate of this outcome (96). In meta-analysis, data from the individual studies are not merely combined equally though they were from a single study; rather, greater weights are given to the results from studies that provide more information, because they are likely to be closer to true effect being estimated. Mathematically combining the results from the private studies tin be accomplished under the assumption of "fixed" effects or "random" effects model. Although a thorough description and merits of each arroyo is described elsewhere (97), it is fair to say that a random-effects model is more bourgeois than the stock-still-effects approach, and a finding that is statistically significant with the latter merely non the former should be viewed with skepticism.
Whenever individual studies are pooled in meta-analysis, it is important for the reader to determine whether it was reasonable to do and then. One way to assess the similarity of various studies is to inspect the graphic display of the results, looking for similarities in the direction of the estimated event. Fifty-fifty without because any combined meta-analytic upshot, a reader becomes much more confident when a similar outcome is beingness observed across many studies (i.e., the results have replicated across many studies). Some review teams may written report a statistical test to determine how dissimilar the studies are from one some other (as described previously, this is often termed heterogeneity of the study results [98]). This can help to prove or disprove that differences in the results that were observed between the primary studies is no different from what would be expected by chance. The most common statistical test to quantify heterogeneity is something called the Q statistic, which is similar in concept to a χ2 test. Although a nonsignificant result (by convention P > 0.ane) is oftentimes taken to bespeak that in that location are no substantial differences between the studies, information technology is of import to consider that this test is underpowered, peculiarly when the number of studies being pooled is small. A new statistic that is frequently beingness reported in meta-analysis these days is something called the Iii statistic. This statistic describes the percentage variability betwixt the studies that is present across what would be expected by chance. When interpreting an I2 statistic, values of 0 to thirty, 31 to 50, and >50% represent mild, moderate, and marked differences between the studies, respectively (99).
Whenever a review team identifies significant differences betwixt the master studies, they should try to explain possible reasons for these differences. This can be washed in an informal way by analyzing certain types of studies separately or by selectively combining studies to decide which are particularly different from the remaining studies. Alternatively, a statistical approach tin can exist taken to explore differences across studies, using a technique like to linear or logistic regression (which at the written report level is something called meta-regression) (100). Either style, a careful exploration of why study results differ tin yield important information about potential determinants of the effect being observed.
Conclusions
Similar all types of inquiry, systematic reviews and meta-analyses have both potential strengths and weaknesses. With the growth of renal clinical studies, an increasing number of these types of summary publications will certainly become available to nephrologists, researchers, administrators, and policy makers who seek to go on abreast of recent developments. To maximize their advantages, information technology is essential that time to come reviews be conducted and reported properly, with judicious estimation by the discriminating reader.
Acknowledgments
A.X.Chiliad. was supported past a Clinician Scientist Award from the Canadian Institutes of Wellness Research (CIHR). D.H. was supported by a CIHR Fellowship Award, the Chisholm Memorial Fellowship, and the Clinician-Scientist Training Plan of the Academy of Toronto. One thousand.T. was supported by a Population Health Investigator Honor from the Alberta Heritage Foundation for Medical Inquiry and a New Investigator Award from the CIHR.
We thank Drs. Chi Hsu and Harvey Feldman for help and advice. We thank Arthur Iansavichus, MLIS, who helped compile systematic reviews published in the discipline of nephrology.
Footnotes
-
Published online ahead of print. Publication date available at www.cjasn.org.
- Copyright © 2008 by the American Guild of Nephrology
References
- ↵
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
- ↵
-
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
-
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
-
-
-
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
mckinneysymee1942.blogspot.com
Source: https://cjasn.asnjournals.org/content/3/1/253
0 Response to "Which Is True of Meta-analysis and Systematic Review?"
Enregistrer un commentaire