Biostatistics, Health Research Methods and Critical Appraisal
Assignment
Exercise 1
A new study is proposed to compare a novel TNF- inhibitor and methotrexate in the treatment of Rheumatoid Arthritis in newly diagnosed, treatment naïve patients.
You are asked to draw up a protocol for this study making sure you cover the following topics.
1. The type of study you would design.
A double-blind randomized control trial.
Using EULAR guidelines1 and Previous RCTs (Biologics vs MTX – PREMIER2, ASPIRE3) to design study.
Currently EULAR guidelines recommends therapy with DMARDs as soon as diagnosis is made. Methotrexate is considered first-line therapy. Monitoring should be frequent in active disease (every 1-3 months) – if there is no improvement at most 3 months after the start of treatment or target has not been reached by 6 months, therapy should be adjusted1.
Study will compare 2 arms; 1 with Methotrexate only vs 1 with Novel TNF- inhibitor only.
Prior to study:
Comprehensive data and literature demonstrating efficacy of novel TNF- inhibitor against placebo.
Statistical analysis – Sample size to be determined for study power – needs to be large enough to ensure that there is a high probability for statistically significant difference.
Ethics approval
Inclusion:
Newly diagnosed Rheumatoid Arthritis based on 2010 ACR/EULAR Criteria (Score ≥ 6)4
Disease duration ≤ 3 years
Treatment-naïve – No prior treatment for Rheumatoid Arthritis
Age: between 18 – 75
Exclusion:
Previous treatment for Rheumatoid Arthritis
Contraindication to therapy – i.e. Tuberculosis, Hepatitis, Low VZV titres
Active infections
Malignancy
Deranged LFTs
Deranged Renal Profile
History of poor compliance with medication
Therefore, patients should be evaluated at 3 months (12 weeks) and at 6 months (24 weeks) as well as a 4 weekly review.
3 months – to ensure clinical improvement
6 months – to ensure clinical remission
1 year – follow up to review treatment effects.
As per the EULAR guidelines, the patients' therapy will be adjusted if there is no clinical improvement at 3 months or no target end-point (i.e. remission) after 6 months1.
2. The methods you would employ to ensure minimal bias
– Randomisation to ensure that both arms in the study are comparable population groups – also ensures that only variable is intervention
– Blinding to reduce selection bias – both investigators and patients do not know which treatment group patient has been allocated to.
– Double-dummy technique to conceal specific medication administered e.g. administering 2 products (drug and placebo) which look the same as the other treatment arm.
– Anonymizing treatment groups and data during statistical analysis e.g. outcome data can be coded.
– Intention to treat analysis – reduces reporting bias as it includes all trial participants regardless of whether they completed the study or not and reflects a more real-world scenario.
3. The type of data you would collect to assess the effectiveness of each treatment
– Clinical efficacy í ACR Score (Response), DAS-28 Score (Remission), Inflammatory markers (CRP & ESR)
– Functional Efficacy
– Structural Efficacy í Radiographic progression (Modified total Sharp Score, Erosion Score, Joint space narrowing score)
– Time to response
– Safety – adverse events, intolerances, infections, malignancy
– Costs of treatment
– Patient's perceptions (Subjective)
– Physician's global assessment of disease activity
– Patient's global assessment of disease activity
4. The methods you would use to analyse the collected data
– Statistical analysis –
– P-values of <0.05 are considered to be statistically significant
5. The possible pitfalls to be aware of and how you might manage them
– Serious adverse events
– Obvious differences in clinical response that may necessitate early termination of drug intervention. Study may need an interim analysis though this also increases the chance that a statistically insignificant result is obtained.
– Sub-therapeutic drug levels can impede results. Methotrexate usually has a step-up regimen up to 25mg/week over several weeks. Anti-TNF inhibitor trough levels can be measured.
6. The ethical considerations that might need to be taken into consideration.
– Informed consent – Patients need to be made aware of this trial, including current practice, benefits, possible side effects, also given an option to opt out from study.
– Capacity to consent
– Ethical approval prior to commencement of study to ensure that patient autonomy and safety as well as the patients' best interests are preserved.
– Intention to treat
References
1. Smolen JS, Landewé R, Bijlsma J, et al.EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2016 update. Annals of the Rheumatic Diseases Published Online First: 06 March 2017. doi: 10.1136/annrheumdis-2016-210715
2. Breedveld, F. C., Weisman, M. H., Kavanaugh, A. F., Cohen, S. B., Pavelka, K., Vollenhoven, R. v., Sharp, J., Perez, J. L. and Spencer-Green, G. T. (2006), The PREMIER study: A multicenter, randomized, double-blind clinical trial of combination therapy with adalimumab plus methotrexate versus methotrexate alone or adalimumab alone in patients with early, aggressive rheumatoid arthritis who had not had previous methotrexate treatment. Arthritis & Rheumatism, 54: 26-37. doi:10.1002/art.21519
3. St. Clair, E. W., van der Heijde, D. M. F. M., Smolen, J. S., Maini, R. N., Bathon, J. M., Emery, P., Keystone, E., Schiff, M., Kalden, J. R., Wang, B., DeWoody, K., Weiss, R., Baker, D. and Active-Controlled Study of Patients Receiving Infliximab for the Treatment of Rheumatoid Arthritis of Early Onset Study Group (2004), Combination of infliximab and methotrexate therapy for early rheumatoid arthritis: A randomized, controlled trial. Arthritis & Rheumatism, 50: 3432-3443. doi:10.1002/art.20568
4. Jonathan Kay, Katherine S. Upchurch; ACR/EULAR 2010 rheumatoid arthritis classification criteria, Rheumatology, Volume 51, Issue suppl_6, 1 December 2012, Pages vi5-vi9, https://doi.org/10.1093/rheumatology/kes279
Exercise 2
Define the principles of Good Clinical Practice (GCP)
Good Clinical Practice (GCP) is a standard that was defined by the ICH for clinical trials. The principles of GCP as per the ICH Guideline for Good Clinical Practice – 19961 is as follows:
1. Clinical trials should be conducted in accordance with the ethical principles that have their origin in the Declaration of Helsinki, and that are consistent with GCP and the applicable regulatory requirement(s).
2. Before a trial is initiated, foreseeable risks and inconveniences should be weighed against the anticipated benefit for the individual trial subject and society. A trial should be initiated and continued only if the anticipated benefits justify the risks.
3. The rights, safety, and well-being of the trial subjects are the most important considerations and should prevail over interests of science and society.
4. The available nonclinical and clinical information on an investigational product should be adequate to support the proposed clinical trial.
5. Clinical trials should be scientifically sound, and described in a clear, detailed protocol.
6. A trial should be conducted in compliance with the protocol that has received prior institutional review board (IRB)/independent ethics committee (IEC) approval/favourable opinion.
7. The medical care given to, and medical decisions made on behalf of, subjects should always be the responsibility of a qualified physician or, when appropriate, of a qualified dentist.
8. Each individual involved in conducting a trial should be qualified by education, training, and experience to perform his or her respective task(s).
9. Freely given informed consent should be obtained from every subject prior to clinical trial participation.
10. All clinical trial information should be recorded, handled, and stored in a way that allows its accurate reporting, interpretation and verification.
11. The confidentiality of records that could identify subjects should be protected, respecting the privacy and confidentiality rules in accordance with the applicable regulatory requirement(s).
12. Investigational products should be manufactured, handled, and stored in accordance with applicable good manufacturing practice (GMP). They should be used in accordance with the approved protocol.
13. Systems with procedures that assure the quality of every aspect of the trial should be implemented.
What is the role of the investigator in a GCP Inspection of a Clincial Trial?
In a GCP Inspection of a Clinical Trial, the investigator will need to demonstrate the following:
1. Appropriate qualification and training to conduct the study
2. Adequate resources in terms of time, recruiting of patients, staffing, and to ensure that all involved parties are adequately trained for the study (well-versed with protocol, the product and duties)
3. Trial Subjects have adequate medical care – an investigator, if a qualified physician, should be responsible for trial-related medical decisions, responsible for the management of any adverse events and in communication with the patient and primary physician when necessary regarding their medical care.
4. Communication with IRB/IEC – Approval of relevant documentation: Trial protocol, informed consent forms, procedures as well as provision of a current copy of the Investigator's Brochure
5. Compliance with Protocol – To follow, in concordance with IRB/IEC, the agreed upon trial protocol and to document and gain approval for any deviations from the protocol if needed.
6. Responsibility for the Investigational Product(s) – Accountability in terms of management of product (storage, inventory, use, return of unused items, expiration dates) as well as relevant documentation of the aforementioned details.
7. Randomisation Procedures and Unblinding – To show adherence to protocol and documentation for breaking/unblinding
8. Informed consent of trial subjects – To show compliance with regulatory requirements and adherence to GCP. Informed consent needs to ensure that the trial subjects are not only informed initially of the trial but are included in any updates in information, documentation of agreement and meets legal requirements to continue with study.
9. Accurate and Adequate records and reports – Covers all aspects of clinical trial including trial subject documentation, progress reports, safety reports (adverse events), suspension/termination of trial and final report(s).
An investigator is also expected to prepare for inspection (notified in advance ~ 4-6 weeks) by making this information readily available. In addition to this, by adopting a ethos of transparency during these proceedings, an investigator can ensure that all aspects of the clinical trial is well managed and that any deficiencies are identified and corrected promptly. This promotes the best outcome for all involved parties.
Reference:
1. ICH GCP Guidelines 1996. https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R1_Guideline.pdf [Accessed 13 Dec. 2017].
Exercise 3
Explain the following terms:
– Relative risk – This is the chance of an outcome on a particular treatment compared to the chance of an outcome to an alternative/no treatment. This can be calculated by dividing the absolute risk of events in the treatment groups by the absolute risk of events in the control group (ART/ARC). For example, if the risk of mortality from an MI is 10% with PCI and at baseline, is at 50% without intervention, the relative risk would be 20% (0.20).
– Odds Ratio – The odds ratio is similar to relative risk but looks at the odds of an event in the treatment group as a ratio to the odds of an event in the control group. As such using the above example, the odds of death in the PCI group would be 0.11 (1/9) and the odds of death in the non-intervention group would be 1.00 (1/1). Therefore, the odds ratio would be 0.11.
– 95% Confidence Interval for odds ratio or relative risk – This signifies the certainty of the range by which the results from a study fall under. For example, if there is 95% confidence interval that the odds ratio in the PCI group was between 0.18 to 0.22, it means that if the study were repeated, there is a 95% chance that the result will fall in that range. If the 95% Confidence Interval for the RR or OR includes '1', this implies that the result is not significant and that there is no difference between the treatment and control groups.
Which of these statistics would you use for a:
a) Case-control study – Odds Ratio: Case control studies are observational studies that aims to identify a causal factor while comparing subjects who have a disease with those who do not. The odds ratio therefore can be used in this setting to compare the odds of disease in the case and control groups.
b) Cohort study – Relative Risk: Cohort studies observe exposed and unexposed population groups and follows them to a predetermined outcome. Thus, a relative risk can be used to determine to compare the risk of an exposure to the eventual outcome with the risk without the exposure.
c) Randomised control trial – 95% Confidence Interval for OR: In a randomized control trial a 95% confidence interval should be incorporated to the odds ratio for disease/outcome measurement as it potentially implies a statistical significance to the results. This gives the findings more credibility and can be used therefore to help determine management of patient care.
References: BMJ stats website
Exercise 4
Cigarette Smoking and Lung Cancer
1. What makes the first study a case-control and the second a cohort?
Case-control studies are observational studies that aim to identify a causal factor while comparing subjects who have a disease with those who do not. The first case falls into this category as it identified 2 distinct groups; the cases (all patients with a new diagnosis of lung cancer) and the controls (patients hospitalized with other disorders). Both these groups were then reviewed based on their relation to the 'causal factor', in this case, their smoking habits.
Cohort studies observe exposed and unexposed population groups and follows them to a predetermined outcome. The second case conforms to this type of study. They initially identified patients based on their smoking habits and followed their outcomes (i.e. lung cancer and mortality) over the ensuing years.
2A) Why might hospitals have been chosen as the setting for the case-control study?
Case-control studies are useful in studying uncommon conditions. The hospital setting would have been ideal in this scenario as it would be able to identify patients with newly diagnosed lung cancer. This would have been much more difficult to trace through a general population. In addition, the hospital would have been an easier avenue to recruit patients for the study to obtain their information regarding their smoking habits.
2B) What are advantages of selecting controls from the same hospital as cases?
Selecting controls from the same hospital would help create 2 comparable groups in terms of patient demographic. It would potentially aid in reducing any confounding factors. Also the ease of recruitment for current inpatients would be advantageous.
2C) How representative of lung cancer cases are those hospitalized for lung cancer?
Hospitalisation for the diagnosis and complications (infections, haemoptysis) of lung cancer is fairly common and thus fairly representative. Most new cases of lung cancer are also diagnosed in-hospital from specific investigations (e.g. biochemistry, radiological, biopsies). However, it is important to note that there is a spectrum based on the course of the disease and that there is a subset of patients who are healthier (usually earlier in the disease-course) and are managed as outpatients and not hospitalized. This group would not be represented in the hospitalized cohort.
2D) Similarly, how representative are controls from hospital?
Controls from hospital are poor representatives as controls because they are usually admitted with other conditions and have other co-morbidities. In addition, there are also complications of smoking that require hospitalization that are not lung-cancer related such as exacerbations of COPD. This is a much more common cause of hospitalization than lung-cancer related admissions. These 'controls' are not as healthy as the general population and there is a likely higher proportion of smokers on this group.
2E) How might the representativeness affect the results?
A poor representation of the control population in this case can result in an underestimation of the risk of smoking in the incidence of lung cancer due to a higher proportion of smokers in the hospitalized control group. It can lead to inaccurate findings and correlations.
Table 1
From this table.
1. Proportion smoked (cases): 1350/1357 = 99.5%
Proportion smoked (controls): 1296/1357 = 95.5%
2. Odds of smoking among the cases: (1350/1357)/(7/1357) = 192.9:1
3. Odds of smoking among the controls: (1296/1357)/(61/1357) = 21.2:1
4. Odds ratio: 192.9/21.2 = 9.1
5. What do you infer about smoking and lung cancer?
This odds ratio implies that smokers are approximately 9 times as likely to develop lung cancer than non-smokers.
Assignment 1
Critical Appraisal
Summarise the study (max 300 words).
Critically appraise the above clinical trial, using a formal checklist.
Amethocaine versus EMLA for successful intravenous cannulation in a children's emergency department: a randomised controlled study
I General Considerations
1. Did the study examine a clearly defined question?
2. Was the study design appropriate for the question under examination?
II Validity
(check study design, possible bias and confounding)
3. Was the randomisation process correct?
4. Was the study blinded?
If "yes" was the blinding sufficient? If "no" why wasn't the study blinded?
5. Was a sufficient number of subjects recruited into the study?
6. Were all subjects followed up and evaluated in the same way?
7. Were all of the subjects recruited into the study accounted for?
III Results
Evaluate in terms of
o " Method of presentation of results used
o " Size of result
o " Precision (CI's and/or p value)
o " Relevance of result
IV Applicability
Are the results relevant for your patient population?
Are results in keeping with other available evidence on the subject? Should the results change your clinical practice?
Address the following questions
1. Were there any methodological problems?
2. In your opinion, do these have an impact on the validity of the findings of the trial?
3. In your opinion, are the results of this trial sufficient to change current practice in this area?