PakMediNet Discussion Forum : Biostatistics : Meta-analysis - Formulating a Question
Meta-analysis - Formulating a Question
To conduct a meta-analysis, first, I would like to explain the importance of a good question, origins of good questions, how to choose a clinically relevant question, and formulation of a question in a way to get a clear answer.
IMPORTANCE OF A GOOD QUESTION:
A good meta-analysis is based on a well-formulated, clinically relevant, practically feasible, and answerable question. This question is the lifeline of the meta-analysis. A good question guides the meta-analysis at every step, from inclusion/exclusion criteria for studies, to the choice of statistical methods, to the presentation of the results, to writing a manuscript, to choice of the journal where you would like it to be published. On the other hand, a poor question will result in an incoherent effort and confused manuscript.
ORIGIN OF GOOD QUESTION
We find plenty of questions in our routine clinical practice. These questions may be asked by our patients (for example, will this cholesterol lowering drug increase my risk of developing cancer? or, can I lower my cholesterol by eating garlic?) or may appear in clinician’s mind in complex clinical cases. We may try to answer these questions with some guess-work, or with pseudo-confidence, or we may admit our lack of knowledge, or level of uncertainty in relation to that particular question. Once we admit uncertainty about the answer to a particular clinical problem, we should try to find its answer. Sometimes, answers are straightforward, while on other occasions, there is no clear answer providing a researcher an opportunity.
CHOOSING A CLINICALLY RELEVANT QUESTION
The number of questions with unclear answers is limitless, however, not all questions can be answered with a meta-analysis and the resources and time available to a researcher are quite limited. Thus, we have to choose the clinically most important question. The definition of the clinically most important may be quite variable depending on whether we choose clinician’s perspective, or patient’s perspective. Generally, a good question should have a major effect on patients as well as should improve clinical practice.
FORMULATING A QUESTION
Once we have decided that a question is worth asking, the next step is to formulate it adequately. A question should have four parts: 1) study population (for example, diabetics, etc.), 2) exposure (such as a risk factor, treatment, a diagnostic test etc.), 3) Control population, 4) Clinical outcomes (such as death, increased functional status, etc.).
SHORT-CUT
Updating a previous meta-analysis circumvents the problem of finding and formulating a question. One can update a previous meta-analysis if sufficient new data is available from studies completed after the publication of that met-analysis that can change its conclusions.
Posted by: rqayyumPosts: 199 :: 07-12-2005 :: | Reply to this Message
Re: Meta-analysis - Formulating a Question
Thank you "rqayyum" for a nice description on meta-analysis. It is indeed a difficult study to start with. One must have a significant experience before hand.
A similar explanation can be "Evidence Classes" given in American clinical guidelines. Like Class A Evidence means a evidence which is supported by studies and meta-analysis and without it, would be harmful.
Meta-analysis cannot be done in every case because of limitations in the number of studies and experiences.
Can a meta-analysis be done on an unpublished work / data?
Posted by: docosamaPosts: 333 :: 08-12-2005 :: | Reply to this Message
Re: Meta-analysis - Formulating a Question
Thank you Dr Osama for raising an interesting point. Generally, guidelines published by different organizations (such as American Heart Association etc.) rate the strength of evidence in support of their recommendations with alphabets "A", "B" (which can be further devided in B1 and B2), and "C". "A" rated evidence is well-proven facts supported by extensive clinical trial data, "B" rated evidence has some but not strong clinical evidence, and "C" rated evidence is generally expert consensus without any significant experimental data in support of it. Recommendations rated "B" are a good start to find a question for a meta-analysis, as there is some but not convincing data and a meta-analysis may be able to clear the dust.
Regarding including unpublished data in meta-analysis, allow me to quote directly from a BMJ article (by Smith and Egger):
"Publication bias, is a major threat to the validity of meta-analysis. Obtaining and including data from unpublished studies seems to be the obvious way of avoiding this problem. Including data from unpublished studies can itself introduce bias, however. Even after extensive consultation with the research community, unpublished studies may remain hidden. The unpublished studies that can be located may thus be an unrepresentative sample of unpublished studies. Whether bias is reduced or increased by including unpublished studies cannot formally be assessed as it is impossible to be certain that all unpublished studies have been located. A further problem relates to the willingness of investigators of located unpublished studies to provide data. This may depend on the findings of the study, more favourable results being provided more readily. This could again bias the findings of a meta-analysis."
"An analysis of 150 meta-analyses published between 1988 and 1991 showed that most meta-analysts had searched for unpublished material, although such data were located and included in only 31% of meta-analyses. A questionnaire assessing the attitudes towards inclusion of unpublished data was sent to the authors of these reports and to the editors of the journals that had published them: 78% of meta-analysts supported the use of unpublished material, compared with only 47% of journal editors. This lack of support by some editors is on the grounds that the data have not been peer reviewed. The refereeing process, however, has not always been a successful way of ensuring that published results are valid. On the other hand, meta-analyses of unpublished data from interested sources is clearly of concern. Such unchallengeable data have been produced in circumstances in which an obvious commercial interest exists."
"The most satisfactory approach to the inclusion of unpublished data in meta-analyses is to carry out an extensive search for such data and obtain them if possible. The analysis should then be performed with and without the unpublished data, as a form of sensitivity analysis. If the conclusions are altered through the inclusion or exclusion of such data, the results of either approach should be treated cautiously."
http://bmj.bmjjournals.com/archive/7126/7126ed8.htm
Posted by: rqayyumPosts: 199 :: 08-12-2005 :: | Reply to this Message