One of the primary tasks of being a New Investigator is getting to know about the research done by others.
Most often, the only source of information available to us is the study report, or published article, provided
by the authors. Very often, when reading these interesting studies, we end up thinking: “is this a good study”? However, due to space restrictions, authors are often unable to provide all the necessary information. As such, this question is difficult to answer. Luckily, the recently published CONSORT extension for PROs and the widely used Cochrane Risk of Bias tool can provide us with some guidelines. In this article, we want
to use these two excellent guidelines to highlight some of the issues we consider to be important in order to assess the quality of the design, conduct, analysis, and reporting of PRO research. Although these guidelines focus primarily on randomized controlled trails, we feel that the items may, to some extent, also be applicable to observational research. Below, you will find our item selection in question form.
Design:
Did the authors...
• ... provide a rationale for the
choice of the PRO instrument
used?
• ... provide or cite evidence of the
PRO instrument validity and reliability?
• ... provide information on the
generation of the randomized
sequence used?
• ... provide information on the
concealment of allocations prior
to assignment?
• ... report on the blinding of both
the participants and personnel,
as well as the outcome assessors?
• ... state the status of the PRO as
either a primary or secondary
outcome?
Conduct: Did the authors...
• ... provide the intended PRO data
collection schedule?
• ... report the mode of administration
of the PRO tool and the
methods of collecting data (e.g.
telephone, other)?
Analysis: Did the authors...
• ... conduct a power/sample size
calculation relevant to the PRO?
• ... explicitly state their statistical
approaches in dealing with missing
data?
• ... provide evidence of appropriate
statistical analysis and tests
of statistical significance?
• ... state how multiple comparisons
were addressed?
Reporting: Did the authors...
• ... report the extent of the data
missing?
• ... explain the reasons for missing
data?
• ... provide the magnitude of the
effect size for each statistically
significant PRO result?
• ... report the PRO outcomes in a
graphic format?
• ... describe study patients’ characteristics,
including baseline
PRO scores?
• ... explicitly discuss the limitations
of the PRO components of
the study?
• ... discuss generalizability issues
uniquely related to the PRO results?
• ... discuss the clinical significance
of the PRO findings?
We hope that the above mentioned items will prove helpful. Unfortunately, there is no consensus on the requirements that need to be met in order for research to be labeled ‘robust’. As such, quality assessments remain, like most aspects of research, a subjective and selective process – and is therefore best done by at least two independent reviewers. A suggestion from us would be to join or perhaps even start a journal club. This way, you can discuss the research of others with your peers, practice your skills, and have a lot of fun at the same time!
