Do the CONSORT guidelines need modifications for reporting Psycho-Social Interventions delivered through mobile Health? Recommendations from an eDelphi Study
There has been extensive research on the use of mobile Health (mHealth) technologies for a variety of mental and emotional health problems. A number of literature reviews of mHealth delivered interventions for a variety of disorders have been published. These interventions range from simple telephone reminders (Burgoyne, Acosta, & Yamamoto, 1983) to complex programs, like Virtual Reality Therapy (Park et al., 2011; Smith et al., 2015) or Avatar Therapy (Leff, Williams, Huckvale, Arbuthnot, & Leff, 2013).
In a recent literature review of eMedia delivered interventions for psychosis we have reported poor reporting of interventions in RCTs in this area (Naeem et al, under review). In order to build evidence base, and for replicability to be possible there is a need to develop strict protocols, that describe the processes concretely. Development and reporting of the interventions should be clear and precise (Naeem et al., 2015). The side effects should be carefully monitored (Naeem et al, under review) and the outcomes should be in line with the real world. Pioneering developers and the trialists should be provided with guidelines that they can follow in order to promote good science. This can then lead to reporting of the interventions that are safe and effective and can be prescribed with confidence. It is important that new trials in this area should comply with CONSORT guidance on design, conduct and reporting. However, CONSORT should be modified to encourage this. We are suggesting a Delphi study to determine expert consensus on reporting RCTs in which an intervention is delivered through digital Media.
Participants who are considered to be experts in this field will be invited to participate in this Delphi exercise. They will be identified from the published literature and a list will be compiled. Each participant will be invited to take part in the study via e-mail. We are anticipating approximately 50-60 participants will be invited. Invitations to participate will not be limited to a certain geographical, professional or gender background.
The project will be completed in three phases;
1- Engagement of stakeholders
2- eDelphi process
3- Wider consultation through social media
The project team has developed the project protocol, and are assembling the Advisory Group and recruiting various stakeholders (e.g., trialists, methodologists, practitioners, policy makers, funders, and services users). The project team will execute and monitor the Delphi process, consensus meeting, write-up of the resultant guideline documents, and the implementation strategy.
The advisory group will consist of the opinion leaders, who will advise at each project stage, including write up of the final draft and disseminate the final guideline documents. Members are leading experts in interventions across various disciplines (will be included in Acknowledgments section). They will help recruit stakeholders and will identify topics to at each stage.
The most widely used reporting guidelines have enlisted journal editors during development and acquired official journal endorsement upon completion. To begin this effort, editors of high-impact-factor journals in key disciplines will be approached.
We will develop a project website, and will regularly promote the website on social media, for example Facebook, Twitter, LinkedIn etc.
The eDelphi Process
The Delphi method is a structured method, originally developed as a systematic, interactive forecasting method which relies on a panel of experts. It is used commonly for forecasting and for policy making. The experts answer questionnaires in two or more rounds. After each round, a facilitator prepares an anonymous summary of the experts’ opinion from the previous round. This is then forwarded to the group for further consultation. The process continues until a consensus is reached. The experts are encouraged to revise their earlier answers in light of the replies of other members of their panel at each stage. During this filtration process, the range of the doubtful answers decreases and the group converges towards the "correct" answer. Predefined criteria are used to reach the final consensus (e.g. number of rounds, achievement of consensus, stability of results).
The Delphi process will consist of 3 stages, which are based on the stages outlined by
Langlands et al (Langlands, Jorm, Kelly, & Kitchener, 2008) and has been used extensively in literature. The approach to analysis will also be identical to one described by Langlands et al. (Langlands et al., 2008). The Delphi process will be entirely online in our project.
In ‘‘stage 1,’’ we will compile a list of relevant statements, that will be derived from the existing literature and CONSORT statements. The list will then be sent to participants who agree to participate in the study. They will be asked to add and respond to this list of statements. Amendments to the wording of the statements and new items will be added to the list of statements. In ‘‘stage 2,’’ the finalized list of items identified in stage 1 will be constructed and formatted to an online Delphi forecasting tool (http://armstrong.wharton.upenn.edu/delphi2/). Participants will be e-mailed a link to the online version and asked to complete the questionnaire by rating the importance of each item on a Likert rating scale of 1–5 (1, essential; 2, important; 3, do not know/depends; 4, unimportant; and 5, should not be included). The results from the questionnaire will be automatically entered into an anonymous database. Items to be included as a standard will be then extracted so that the questionnaire could be amended for use in stage 3. Survey responses will be recorded in an anonymous database and analyzed by obtaining group percentages. The following cut-off points will be used in relation to inclusion, re-rating, and exclusion criteria for the items.
1. If at least 80% or above panel members rate an item as essential or important, it will be included as a standard.
2. If 70%–79% of panel members rate an item essential or very important, we will ask all panel members to re-rate that item.
3. Any statements that do not meet the above 2 conditions will be excluded.
In ‘‘stage 3,’’ the participants from phase 2 will be sent a questionnaire of the same format as above, but incorporating only those items that 70%–79% of the panel members rate as essential or important in the previous stage. The above process will then be repeated to make a decision regarding these items, i.e., to exclude them or include them.
We will circulate the findings through the project website and through the social media, for example Facebook, Twitter and LinkedIn etc. for the final opinion from the interested parties.
We will prepare a list of participants. They will be invited through email to join the Delphi process. The AG will help identify an initial list of stakeholders who extensively publish, fund, or utilize social and psychological intervention research. We will use a ‘snowball recruitment’ approach via collaborators in relevant research and professional networks. The project website and the social media will be used to engage those who are not identified through snowball method. The project website will enable interested individuals to register their interest. They will be able to join a survey at the end of Delphi process, which will help them to register their satisfaction with each stem on a Likert scale. The summary will be available on project website. Intervention researchers, methodologists, and guideline developers will form a substantial number of the participants. Editors of high-impact-factor journals will be invited for their expertise and to ensure uptake upon completion .Funders of social and psychological intervention studies will be invited to provide expertise and to promote use of the guideline for assessing grant applications. Practitioners will help identity issues of relevance to practice. Policy makers will help identity items, and they will assist in the creation of a user-friendly document and standards. Representatives from consumer groups will advance the relevance of research reports to the ultimate recipients of services.
The entire process will be conducted online. The mode of the engagement will be emails, social media and website. The final product before submission will be circulated on social media.
We will get a final list of amended CONSORT guidelines at the end of the exercise. The results will be published in scientific journals and will be presented at National and International conferences.