Scientific validity
Research that is not conducted in conformity with valid scientific principles cannot produce relevant results. Indeed, such results may be misleading and contribute directly, and indirectly, to harmful clinical practices. Exposure of human participants to the risks and burdens of such research is unethical. The notion of social value presupposes scientific validity (1). Research ethics committees have been criticized for paying insufficient attention to the "methodological rigor in proposals presented for their attention."(2) In a published editorial entitled The scandal of poor medical research, Altman (3) also alludes to the inadequate review of the methodological aspects of research proposals by research ethics committees. Committees may not have members with the necessary expertise in the proposed research to make the necessary assessment concerning scientific validity. In addition, committees traditionally spend an inordinate amount of time parsing the informed consent document, and relatively little or no attention to the scientific aspects of the research proposal. Freedman and Shapiro (4) have commented on the tendency of research ethics committees to neglect design issues in clinical trials, thus overlooking many items (such as eligibility criteria and statistical considerations) pertinent to an assurance that the proposal meets the requirement of scientific validity and relevance. Unfortunately, in the case of multicenter trials, a local research ethics committee is not in a position to affect the design of the research protocol, and its consideration takes on the character of a take-it-or-leave-it proposition. In other cases, the erroneous assumption is often made that a particular proposal has undergone careful and credible scrutiny by a scientific merit, or like, committee. While such institutional committees exist, their ability to adequately assess the scientific validity of research proposals is unknown. Details concerning the nature and structure of such reviews, and the deliberations of such committees, are not generally made available to research ethics committees. In response to these current limitations, local research ethics committees should adopt a standard for the design and reporting of clinical trials that will ensure that proposals meet the requirement of scientific validity. The nature and scope of such a standard is elucidated in the following sections. Scientific validity and randomized controlled clinical trials The assessment of scientific validity with respect to randomized controlled clinical trials invokes notions of internal and external validity. Internal validity represents the extent to which systematic error (bias) is minimized in clinical trials, and external validity the extent to which the results of trials provide a correct basis for generalization to other clinical circumstances (5). This article by Juni et al. also explicates the nature of biases that may adversely affect internal validity: selection; performance; detection; and attrition biases. An assessment of generalizability requires attention to specific factors such as the clinical status of trial participants; details concerning the therapeutic components in the trial; the settings in which such components are administered; and the nature of the outcomes measured and duration of clinical follow-up. A systematic evaluation of these items is necessary to make a general judgment of the trial's quality, and hence its value. Fortunately, a pre-inception assessment of a trial's scientific validity is greatly facilitated by the CONSORT statement (6). The CONSORT Working Group has identified and elaborated on specific items that should be addressed by the designers of the trial to ensure internal, and to promote external, validity. The identification of such items is supported by empirical evidence of bias in the conduct and reporting of trials, fully referenced in the CONSORT's elaboration document (7). This section of the ASSERT checklist is thus comprised of specific items in the CONSORT checklist relevant to a determination of scientific validity. The associated principles and logic should be familiar to research ethics committee members. Trial designers and investigators responsible for the reporting of the trial's results should address these items in the Application for Research. By doing so, they will be able to correct flaws in the research design before the inception of research-related activities, and will be prepared to report the results of the trial in an appropriate manner. References References available online are hyperlinked 1. Freedman B. Scientific value and validity as ethical requirements for research: A proposed explication. IRB 1987; 9:7-10. 2. Silverman WA. Bad Science and the role of Institutional Review Boards. Arch Pediatr Adolesc Med 2000; 154:1183-1184 3. Altman DG. The scandal of poor medical research. BMJ 1994; 308:283-284. 4. Freedman B, Shapiro SH. Ethics and statistics in clinical research: towards a more comprehensive examination. Jrnl Statistical Planning and Inference 1994: 42:223-240. 5. Jüni P, Altman DG, Egger M. Systematic reviews in health care. Assessing the quality of controlled clinical trials. BMJ 2001;323: 42-46. 7. The CONSORT elaboration document.  |