Background: Randomised, controlled trials (RCT) and systematic reviews of RCT with meta-analysis are considered to be of highest methodological quality and therefore are given the highest level of evidence (Ia/b). Although, "low-quality" RCT may be downgraded to level of evidence IIb, the methodological quality of each individual RCT is not respected in detail in this classification of the level of evidence.
Materials and methods: Within a systematic Cochrane Review of RCT on short-term benefits of laparoscopic or conventional colorectal resections, the methodological quality of all included RCT was evaluated. All RCT were assessed by the Evans and Pollock questionnaire (E and P increasing quality from 0-100) and the Jadad score (increasing quality from 0-5).
Results: Publications from 28 RCT printed from 1996 to 2005 were included in the analysis. Methodological quality of RCT was only moderate [E & P 55 (32-84); Jadad 2 (1-5)]. There was a significant correlation between the E & P and the Jadad score (r = 0.788; p < 0.001). Methodological quality of RCT slightly increased with increasing number of patients included (r = 0.494; p = 0.009) and year of publication (r = 0.427; p = 0.03). Meta-analysis of all RCT yielded clinically relevant differences for overall and local morbidity when compared to meta-analysis of "high-quality" (E & P > 70) RCT only.
Conclusion: The methodological quality of reports of RCT comparing laparoscopic and open colorectal resection varies considerably. In a systematic review, methodological quality of RCT should be assessed because meta-analysis of "high-quality" RCT may yield different results than meta-analysis of all RCT.